Allow AET to use PMT as loadable module
Subsystems (8)
Tony Luck
maintainerv14 cover Allow AET to use PMT as loadable module
Requiring INTEL_PMT_TELEMETRY=y to enable AET is a functional workaround to enable enumeration of Application Energy Telemetry (AET) events, but unacceptable to many users. It results in increased configuration complexity, increased kernel memory footprint and inability to patch problems by unloading a module and loading an updated version.
Add a registration function to the AET code that can be used by INTEL_PMT_TELEMETRY to provide the enumeration functions.
INTEL_PMT_TELEMETRY can be loaded/unloaded independently of resctrl file system mount/unmount. Perform enumeration on every mount and cleanup on every unmount.
Patch series based on TIP tree x86/cache branch current HEAD is:
2c10b3fa5bdf ("fs/resctrl: Ensure default group reports tasks on monitor-only systems")
Previous version (v13) here:
Changes since v13:
[PATCH v14 01/24] x86/cpufeatures: Add missing CQM feature dependency
Add AMD64 manual
s/Linux X86_FEATURE_CQM/X86_FEATURE_CQM/
Complete the trio of context, problem, solution paragraphs by adding the problem statement.
[PATCH v14 02/24] x86/resctrl: Check if monitoring features are supported
Move maintainer note after the "---"
Put the X86_FEATURE_* parenthetical comments back (without "Linux") s/monitoring features/resource monitoring features/g Add Reinette RB tag
Add problem paragraph between context and solution.
Add Chen Yu Tested-by tag
[PATCH v14 03/24] x86/resctrl: Centralize monitoring feature enumeration
Structure commit in context, problem, solution.
Add note that there is still one user of boot_cpu_data.
[PATCH v14 04/24] x86/resctrl: Apply Intel MBM quirk from rdt_get_l3_mon_config()
Replaced commit message with Reinette suggestion from v12 review. Two tweaks suggested by AI commit review:
Be more specific that "early" enumeration is simply before resctrl.
s/consumer/reader/g
[PATCH v14 05/24] x86/resctrl: Delete resctrl_cpu_detect()
Split first paragraph into two: context and problem. Don't list all the fields as a regex.
[PATCH v14 06/24] arm,x86,fs/resctrl: Replace architecture capability helpers with generic versions
New Subject: without {alloc,mon}
New commit message in context, problem, solution format
Did not apply Reinette RB tag because of commit message rewrite.
[PATCH v14 07/24] x86/resctrl: Update special case for Intel Haswell enumeration
Updated commit comment in context, problem, solution form.
[PATCH v14 08/24] x86/resctrl: Delete rdt_alloc_capable and rdt_mon_capable
New commit message in context, problem, solution format.
[PATCH v14 09/24] fs/resctrl: Remove redundant calls to resctrl_mon_capable()
New commit message confessing that this is fixing a performance problem created by an earlier patch this series.
[PATCH v14 10/24] x86/resctrl: Honor rdt={perf|energy} options to force enable AET events
New commit message. Dropped previous supplied Reinette RB tag.
[PATCH v14 11/24] fs/resctrl: Add interface to disable a monitor event
New commit message. Dropped previous applied Reinette RB tag.
[PATCH v14 12/24] arm,x86,fs/resctrl: Allocate maximum needed rmid_ptrs[]
New commit message.
[PATCH v14 13/24] arm,x86,fs/resctrl: Use right size for L3 monitor data structures
New commit message.
[PATCH v14 14/24] x86,fs/resctrl: Handle systems where AET is the only monitoring feature
New commit message.
[PATCH v14 15/24] x86/resctrl: Add PMT registration API for AET enumeration callbacks
New commit message.
[PATCH v14 16/24] platform/x86/intel/pmt: Register enumeration functions with resctrl
New commit message.
[PATCH v14 17/24] x86/resctrl: Use registered function pointers for AET enumeration
Updates to resolve locking problems reported by Sashiko in v13.
AET is now going to use a single mutex for both registration and in a later patch for device removal. So change the name from aet_register_lock to just aet_lock.
Reverse locking hierarchy between AET and pmt_telemetry. No longer OK to call into pmt_telemetry to enumerate features with the aet_lock held. Instead, make use of module pinning while holding aet_lock.
New commit message
[PATCH v14 18/24] arm,x86,fs/resctrl: Enumerate AET on every resctrl mount
New commit comment.
aet_register_lock name changed to aet_lock.
Babu: Add commit comment about event_group::force_off
[PATCH v14 19/24] x86/resctrl: Enforce system RMID limit on AET
New commit comment
[PATCH v14 20/24] x86/resctrl: Export interface to report telemetry unbind/remove
Dropped aet_mmio_lock. Now safe to use aet_lock.
New commit message
Add code comment noting that per-package may be over-reaction.
[PATCH v14 21/24] platform/x86/intel/pmt: Inform resctrl when MMIO maps are being removed
Added commit comment on lock order.
New commit comment.
[PATCH v14 22/24] x86/resctrl: Require 64-bit x86 for resctrl support
New commit message.
[PATCH v14 23/24] x86/resctrl: Simplify Kconfig options for resctrl
New commit message.
[PATCH v14 24/24] x86,fs/resctrl: Document telemetry mount timing caveat
Updated commit message. Include unmount/remount as solution if early mount did not include AET.
Tony Luck (24):
x86/cpufeatures: Add missing CQM feature dependency x86/resctrl: Check if monitoring features are supported x86/resctrl: Centralize monitoring feature enumeration x86/resctrl: Apply Intel MBM quirk from rdt_get_l3_mon_config() x86/resctrl: Delete resctrl_cpu_detect() arm,x86,fs/resctrl: Replace architecture capability helpers with
generic versions
x86/resctrl: Update special case for Intel Haswell enumeration x86/resctrl: Delete rdt_alloc_capable and rdt_mon_capable fs/resctrl: Remove redundant calls to resctrl_mon_capable() x86/resctrl: Honor rdt={perf|energy} options to force enable AET
events
fs/resctrl: Add interface to disable a monitor event arm,x86,fs/resctrl: Allocate maximum needed rmid_ptrs[] arm,x86,fs/resctrl: Use right size for L3 monitor data structures x86,fs/resctrl: Handle systems where AET is the only monitoring
feature
x86/resctrl: Add PMT registration API for AET enumeration callbacks platform/x86/intel/pmt: Register enumeration functions with resctrl x86/resctrl: Use registered function pointers for AET enumeration arm,x86,fs/resctrl: Enumerate AET on every resctrl mount x86/resctrl: Enforce system RMID limit on AET x86/resctrl: Export interface to report telemetry unbind/remove platform/x86/intel/pmt: Inform resctrl when MMIO maps are being
removed
x86/resctrl: Require 64-bit x86 for resctrl support x86/resctrl: Simplify Kconfig options for resctrl x86,fs/resctrl: Document telemetry mount timing caveat
Documentation/filesystems/resctrl.rst | 8 +- include/linux/arm_mpam.h | 3 - include/linux/intel_vsec.h | 14 ++ include/linux/resctrl.h | 84 ++++++++++- arch/x86/include/asm/processor.h | 4 - arch/x86/include/asm/resctrl.h | 24 +--- arch/x86/kernel/cpu/resctrl/internal.h | 33 ++--- arch/x86/kernel/cpu/amd.c | 3 - arch/x86/kernel/cpu/cpuid-deps.c | 1 + arch/x86/kernel/cpu/hygon.c | 3 - arch/x86/kernel/cpu/intel.c | 7 - arch/x86/kernel/cpu/resctrl/core.c | 150 +++++++++++--------- arch/x86/kernel/cpu/resctrl/intel_aet.c | 156 +++++++++++++++++++-- arch/x86/kernel/cpu/resctrl/monitor.c | 78 +++++++---- drivers/platform/x86/intel/pmt/telemetry.c | 45 +++++- drivers/resctrl/mpam_resctrl.c | 44 +++--- fs/resctrl/monitor.c | 111 +++++++++++---- fs/resctrl/pseudo_lock.c | 6 +- fs/resctrl/rdtgroup.c | 72 ++++++---- arch/x86/Kconfig | 15 +- arch/x86/kernel/cpu/resctrl/Makefile | 3 +- 21 files changed, 590 insertions(+), 274 deletions(-)
base-commit: 2c10b3fa5bdf61e344bc522e6c2e00d870ce7ab8
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 1/24 x86/cpufeatures: Add missing CQM feature dependency
The L3 monitoring features are enumerated from CPUID leaf 0xF. Both the Intel Software Developer's Manual, and the AMD64 Architecture Programmer's Manual, say that software must check that CPUID(0x7,0x0).EBX[12] (X86_FEATURE_CQM) is set before checking for monitoring features in CPUID leaf 0xF.
The CPU feature dependency table does not record this requirement.
Add the missing dependency of X86_FEATURE_CQM_LLC on X86_FEATURE_CQM.
---
v14:
Add AMD64 manual
s/Linux X86_FEATURE_CQM/X86_FEATURE_CQM/ Complete the trio of context, problem, solution paragraphs by adding the problem statement.
--- arch/x86/kernel/cpu/cpuid-deps.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kernel/cpu/cpuid-deps.c +1 -0
@@ -67,6 +67,7 @@ static const struct cpuid_dep cpuid_deps[] = { { X86_FEATURE_AVX512_4FMAPS, X86_FEATURE_AVX512F }, { X86_FEATURE_AVX512_VPOPCNTDQ, X86_FEATURE_AVX512F }, { X86_FEATURE_AVX512_VP2INTERSECT, X86_FEATURE_AVX512VL }, + { X86_FEATURE_CQM_LLC, X86_FEATURE_CQM }, { X86_FEATURE_CQM_OCCUP_LLC, X86_FEATURE_CQM_LLC }, { X86_FEATURE_CQM_MBM_TOTAL, X86_FEATURE_CQM_LLC }, { X86_FEATURE_CQM_MBM_LOCAL, X86_FEATURE_CQM_LLC },
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 2/24 x86/resctrl: Check if monitoring features are supported
In reply to Tony LuckBoth Intel and AMD manuals say that software must first check CPUID(0x7,0x0).EBX[12] (X86_FEATURE_CQM) to see if any resource monitoring features are supported before checking for resource specific features supported in subleaves. The L3 resource monitoring features are further gated by CPUID(0xF,0x0).EDX[1] (X86_FEATURE_CQM_LLC).
resctrl enables L3 monitoring based only on the individual monitoring features. It can thus enable L3 monitoring on a system that does not report support for resource monitoring.
Do not enable L3 monitoring unless both X86_FEATURE_CQM and X86_FEATURE_CQM_LLC are supported. This also applies to assignable counters, which rely on the RMID count and scale enumerated in CPUID leaf 0xF.
---
v14:
Maintainer note: No need to backport this fix. No issues have been seen from this failure to follow the exact enumeration process in the manual.
Move maintainer note after the "---"
Put the X86_FEATURE_* parenthetical comments back (without "Linux") s/monitoring features/resource monitoring features/g Add Reinette RB tag
Add problem paragraph between context and solution.
Add Chen Yu Tested-by tag
--- arch/x86/kernel/cpu/resctrl/core.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/x86/kernel/cpu/resctrl/core.c +8 -0
@@ -968,6 +968,14 @@ static __init bool get_rdt_mon_resources(void)struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;bool ret = false; +/* Any monitoring at all? */ +if (!cpu_feature_enabled (X86_FEATURE_CQM )) +return false; + +/* Any of the L3 monitoring features? */ +if (!cpu_feature_enabled (X86_FEATURE_CQM_LLC )) +return false; +if (rdt_cpu_has (X86_FEATURE_CQM_OCCUP_LLC )) {resctrl_enable_mon_event (QOS_L3_OCCUP_EVENT_ID , false,0 ,NULL ); ret = true;
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 3/24 x86/resctrl: Centralize monitoring feature enumeration
In reply to Tony LuckThe original "perf" implementation of Intel Cache QoS Monitoring (CQM) put parameters from CPUID into fields of the cpuinfo_x86 structure. The perf implementation was reverted and replaced by resctrl. The enumerated parameters remained in the boot_cpu_data structure.
resctrl is the only consumer of these values, yet it depends on generic x86 CPU setup to enumerate them.
Centralize all L3 monitoring enumeration within resctrl in preparation for removal of resctrl fields from struct cpuinfo_x86. The Intel MBM correction factor quirk remains the only reader of the cached values.
---
v14:
Structure commit in context, problem, solution. Add note that there is still one user of boot_cpu_data.
--- arch/x86/include/asm/resctrl.h | 9 ++++---- arch/x86/kernel/cpu/resctrl/monitor.c | 32 ++++++++++++++++++++++++--- 2 files changed, 33 insertions(+), 8 deletions(-)
arch/x86/include/asm/resctrl.h +4 -5
@@ -44,6 +44,7 @@ DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state);extern bool rdt_alloc_capable;extern bool rdt_mon_capable; +extern unsigned int rdt_l3_mon_scale;DECLARE_STATIC_KEY_FALSE (rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);
@@ -132,11 +133,9 @@ static inline void __resctrl_sched_in(struct task_struct *tsk)static inline unsigned int resctrl_arch_round_mon_val (unsigned int val) { -unsigned int scale = boot_cpu_data.x86_cache_occ_scale; - -/* h/w works in units of "boot_cpu_data.x86_cache_occ_scale" */ - val /= scale; -return val * scale; +/* Round down to nearest h/w monitoring unit */ + val /= rdt_l3_mon_scale; +return val * rdt_l3_mon_scale; } static inline void resctrl_arch_set_cpu_default_closid_rmid(int cpu, u32 closid,
arch/x86/kernel/cpu/resctrl/monitor.c +29 -3
@@ -32,6 +32,11 @@*/ bool rdt_mon_capable; +/* +* Scale factor to convert L3 monitor events to bytes. +*/ +unsigned int __ro_after_init rdt_l3_mon_scale; +#define CF (cf) ((unsigned long)(1048576 * (cf) + 0.5))static int snc_nodes_per_l3_cache =1 ;
@@ -418,16 +423,37 @@ static __init int snc_get_config(void)int __init rdt_get_l3_mon_config (struct rdt_resource *r) { -unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); +unsigned int mbm_offset;unsigned int threshold;u32 eax, ebx, ecx, edx; +u32 num_rmid; + +/* Resource monitoring leaf is 0xf. L3 monitoring details in subleaf 1 */ +cpuid_count (0xf ,1 , &eax, &ebx, &ecx, &edx); + mbm_offset = eax &GENMASK (7 ,0 ); + rdt_l3_mon_scale = ebx; + num_rmid = ecx +1 ; + +if (!mbm_offset) { +switch (boot_cpu_data.x86_vendor) { +case X86_VENDOR_AMD : + mbm_offset =MBM_CNTR_WIDTH_OFFSET_AMD ; +break ; +case X86_VENDOR_HYGON : + mbm_offset =MBM_CNTR_WIDTH_OFFSET_HYGON ; +break ; +default : +/* Leave mbm_offset as 0 */ +break ; + } + } snc_nodes_per_l3_cache =snc_get_config (); resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size *1024 ; - hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale / snc_nodes_per_l3_cache; - r->mon.num_rmid = (boot_cpu_data.x86_cache_max_rmid +1 ) / snc_nodes_per_l3_cache; + hw_res->mon_scale = rdt_l3_mon_scale / snc_nodes_per_l3_cache; + r->mon.num_rmid = num_rmid / snc_nodes_per_l3_cache; hw_res->mbm_width =MBM_CNTR_WIDTH_BASE ;if (mbm_offset >0 && mbm_offset <=MBM_CNTR_WIDTH_OFFSET_MAX )
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 4/24 x86/resctrl: Apply Intel MBM quirk from rdt_get_l3_mon_config()
In reply to Tony LuckThe Intel MBM correction factor quirk is applied during CPU model checks that precede resctrl feature enumeration, using the RMID count from cpuinfo_x86::x86_cache_max_rmid. It is the last reader of the resctrl fields in struct cpuinfo_x86 now that monitoring feature enumeration is contained within resctrl.
This lone reader prevents those fields from being removed, and applies MBM specific setup before resctrl has established that MBM is supported.
Apply the quirk during monitoring feature enumeration in preparation for removing the resctrl fields from struct cpuinfo_x86. Keep the decision on whether the quirk is needed with the other model specific tests to avoid scattering quirk handling.
No functional change intended.
---
v14:
Replaced commit message with Reinette suggestion from v12 review. Two tweaks suggested by AI commit review:
Be more specific that "early" enumeration is simply before resctrl.
s/consumer/reader/g
--- arch/x86/kernel/cpu/resctrl/internal.h | 4 +-- arch/x86/kernel/cpu/resctrl/core.c | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 40 ++++++++++++++++---------- 3 files changed, 28 insertions(+), 18 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -2
@@ -151,6 +151,8 @@ static inline struct rdt_hw_resource *resctrl_to_arch_res(struct rdt_resource *rextern struct rdt_hw_resource rdt_resources_all[]; +extern bool intel_rdt_mbm_need_quirk; +void arch_mon_domain_online (struct rdt_resource *r,struct rdt_l3_mon_domain *d);/* CPUID.(EAX=10H, ECX=ResID=1).EAX */
@@ -228,8 +230,6 @@ int rdt_get_l3_mon_config(struct rdt_resource *r);bool rdt_cpu_has (int flag); -void __init intel_rdt_mbm_apply_quirk (void ); -void rdt_domain_reconfigure_cdp (struct rdt_resource *r);void resctrl_arch_mbm_cntr_assign_set_one (struct rdt_resource *r);
arch/x86/kernel/cpu/resctrl/core.c +1 -1
@@ -1011,7 +1011,7 @@ static __init void __check_quirks_intel(void)set_rdt_options ("!l3cat" ); fallthrough;case INTEL_BROADWELL_X : -intel_rdt_mbm_apply_quirk (); + intel_rdt_mbm_need_quirk = true;break ; } }
arch/x86/kernel/cpu/resctrl/monitor.c +25 -15
@@ -37,6 +37,8 @@ bool rdt_mon_capable;*/ unsigned int __ro_after_init rdt_l3_mon_scale; +bool __initdata intel_rdt_mbm_need_quirk; +#define CF (cf) ((unsigned long)(1048576 * (cf) + 0.5))static int snc_nodes_per_l3_cache =1 ;
@@ -51,7 +53,7 @@ static int snc_nodes_per_l3_cache = 1;* 1. The threshold 0 is changed to rmid count - 1 so don't do correction * for the case. * 2. MBM total and local correction table indexed by core counter which is -* equal to (x86_cache_max_rmid + 1) / 8 - 1 and is from 0 up to 27. +* equal to num_rmid / 8 - 1 and is from 0 up to 27. * 3. The correction factor is normalized to 2^20 (1048576) so it's faster * to calculate corrected value by shifting: * corrected_value = (original_value * correction_factor) >> 20
@@ -421,6 +423,20 @@ static __init int snc_get_config(void)return ret; } +static void __init intel_rdt_mbm_apply_quirk (u32 num_rmid) +{ +unsigned int cf_index; + + cf_index = num_rmid /8 -1 ; +if (cf_index >=ARRAY_SIZE (mbm_cf_table)) { +pr_info ("No MBM correction factor available\n" ); +return ; + } + + mbm_cf_rmidthreshold = mbm_cf_table[cf_index].rmidthreshold; + mbm_cf = mbm_cf_table[cf_index].cf; +} +int __init rdt_get_l3_mon_config (struct rdt_resource *r) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r);
@@ -500,25 +516,19 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r) hw_res->mbm_cntr_assign_enabled = true; } +/* +* This quirk only applies to Intel Skylake and Broadwell +* CPUs. These don't support Sub-NUMA Cluster. Pass the +* raw number of RMIDs enumerated by CPUID(0xF,0x1).ECX. +*/ +if (intel_rdt_mbm_need_quirk) +intel_rdt_mbm_apply_quirk (num_rmid); + r->mon_capable = true;return 0 ; } -void __init intel_rdt_mbm_apply_quirk (void ) -{ -int cf_index; - - cf_index = (boot_cpu_data.x86_cache_max_rmid +1 ) /8 -1 ; -if (cf_index >=ARRAY_SIZE (mbm_cf_table)) { -pr_info ("No MBM correction factor available\n" ); -return ; - } - - mbm_cf_rmidthreshold = mbm_cf_table[cf_index].rmidthreshold; - mbm_cf = mbm_cf_table[cf_index].cf; -} -static void resctrl_abmc_set_one_amd (void *arg) {bool *enable = arg;
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 5/24 x86/resctrl: Delete resctrl_cpu_detect()
In reply to Tony Luckresctrl now enumerates its monitoring properties itself, leaving the resctrl fields in cpuinfo_x86 without any consumers. These resctrl specific fields are populated from the vendor CPU identification paths.
This unnecessarily placing resctrl details in generic x86 CPU setup.
Delete the unused fields together with the code that populates them, removing resctrl from early CPU identification.
---
v14:
Split first paragraph into two: context and problem. Don't list all the fields as a regex.
--- arch/x86/include/asm/processor.h | 4 --- arch/x86/include/asm/resctrl.h | 3 --- arch/x86/kernel/cpu/amd.c | 3 --- arch/x86/kernel/cpu/hygon.c | 3 --- arch/x86/kernel/cpu/intel.c | 7 ----- arch/x86/kernel/cpu/resctrl/core.c | 42 ------------------------------ 6 files changed, 62 deletions(-)
arch/x86/include/asm/processor.h +0 -4
@@ -185,10 +185,6 @@ struct cpuinfo_x86 {/* in KB - valid for CPUS which support this call: */ unsigned int x86_cache_size;int x86_cache_alignment;/* In bytes */ -/* Cache QoS architectural values, valid only on the BSP: */ -int x86_cache_max_rmid;/* max index */ -int x86_cache_occ_scale;/* scale to bytes */ -int x86_cache_mbm_width_offset;int x86_power;unsigned long loops_per_jiffy;/* protected processor identification number */
arch/x86/include/asm/resctrl.h +0 -3
@@ -193,12 +193,9 @@ static inline void resctrl_arch_mon_ctx_free(struct rdt_resource *r, enum resctrl_event_id evtid, void *ctx) { } -void resctrl_cpu_detect(struct cpuinfo_x86 *c); -#else static inline void resctrl_arch_sched_in (struct task_struct *tsk) {} -static inline void resctrl_cpu_detect (struct cpuinfo_x86 *c) {}#endif /* CONFIG_X86_CPU_RESCTRL */
arch/x86/kernel/cpu/amd.c +0 -3
@@ -23,7 +23,6 @@#include <asm/pci-direct.h> #include <asm/delay.h> #include <asm/debugreg.h> -#include <asm/resctrl.h> #include <asm/msr.h> #include <asm/sev.h>
@@ -475,8 +474,6 @@ static void bsp_init_amd(struct cpuinfo_x86 *c) } } -resctrl_cpu_detect (c); -/* Figure out Zen generations: */ switch (c->x86) {case 0x17 :
arch/x86/kernel/cpu/hygon.c +0 -3
@@ -17,7 +17,6 @@#include <asm/spec-ctrl.h> #include <asm/delay.h> #include <asm/msr.h> -#include <asm/resctrl.h> #include "cpu.h"
@@ -119,8 +118,6 @@ static void bsp_init_hygon(struct cpuinfo_x86 *c) x86_amd_ls_cfg_ssbd_mask =1ULL <<10 ; } } - -resctrl_cpu_detect (c); }static void early_init_hygon (struct cpuinfo_x86 *c)
arch/x86/kernel/cpu/intel.c +0 -7
@@ -22,7 +22,6 @@#include <asm/microcode.h> #include <asm/msr.h> #include <asm/numa.h> -#include <asm/resctrl.h> #include <asm/thermal.h> #include <asm/uaccess.h>
@@ -371,11 +370,6 @@ static void early_init_intel(struct cpuinfo_x86 *c)detect_tme_early (c); } -static void bsp_init_intel (struct cpuinfo_x86 *c) -{ -resctrl_cpu_detect (c); -} -#ifdef CONFIG_X86_32 /* * Early probe support logic for ppro memory erratum #50
@@ -804,7 +798,6 @@ static const struct cpu_dev intel_cpu_dev = {#endif .c_detect_tlb = intel_detect_tlb, .c_early_init = early_init_intel, - .c_bsp_init = bsp_init_intel, .c_init = init_intel, .c_x86_vendor = X86_VENDOR_INTEL, };
arch/x86/kernel/cpu/resctrl/core.c +0 -42
48 lines not on this page · raw message
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 6/24 arm,x86,fs/resctrl: Replace architecture capability helpers with generic versions
In reply to Tony Luckresctrl supports allocation or monitoring if any resource is capable of it. Each architecture answers this via resctrl_arch_alloc_capable() and resctrl_arch_mon_capable(). x86 maintains global flags for this in addition to the per-resource flags and must keep both in sync. MPAM derives the answer from its per-resource flags.
The per-resource flags are part of the resource description that the file system already has, so the architecture helpers duplicate information available to the file system.
Define resctrl_alloc_capable() and resctrl_mon_capable() in the file system based on the per-resource flags and remove the architecture helpers, in preparation for removing the x86 global flags. The result is unchanged for MPAM, which presents all monitors on the L3 resource.
---
v14:
New Subject: without {alloc,mon}
New commit message in context, problem, solution format Did not apply Reinette RB tag because of commit message rewrite.
--- include/linux/arm_mpam.h | 3 --- include/linux/resctrl.h | 30 ++++++++++++++++++++++++++++ arch/x86/include/asm/resctrl.h | 10 ---------- drivers/resctrl/mpam_resctrl.c | 26 ++---------------------- fs/resctrl/monitor.c | 10 +++++----- fs/resctrl/pseudo_lock.c | 6 +++--- fs/resctrl/rdtgroup.c | 36 +++++++++++++++++----------------- 7 files changed, 58 insertions(+), 63 deletions(-)
include/linux/arm_mpam.h +0 -3
@@ -50,9 +50,6 @@ static inline int mpam_ris_create(struct mpam_msc *msc, u8 ris_idx, }#endif -bool resctrl_arch_alloc_capable (void ); -bool resctrl_arch_mon_capable (void ); -void resctrl_arch_set_cpu_default_closid (int cpu,u32 closid);void resctrl_arch_set_closid_rmid (struct task_struct *tsk,u32 closid,u32 rmid);void resctrl_arch_set_cpu_default_closid_rmid (int cpu,u32 closid,u32 rmid);
include/linux/resctrl.h +30 -0
@@ -346,6 +346,36 @@ struct rdt_resource {*/ struct rdt_resource *resctrl_arch_get_resource (enum resctrl_res_level l); +/* +* resctrl_alloc_capable - Are any allocation resources enabled? +*/ +static inline bool resctrl_alloc_capable (void ) +{ +struct rdt_resource *r; + +for_each_rdt_resource (r) { +if (r->alloc_capable) +return true; + } + +return false; +} + +/* +* resctrl_mon_capable - Are any monitor resources enabled? +*/ +static inline bool resctrl_mon_capable (void ) +{ +struct rdt_resource *r; + +for_each_rdt_resource (r) { +if (r->mon_capable) +return true; + } + +return false; +} +/** * struct resctrl_schema - configuration abilities of a resource presented to * user-space
arch/x86/include/asm/resctrl.h +0 -10
@@ -50,11 +50,6 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); -static inline bool resctrl_arch_alloc_capable (void ) -{ -return rdt_alloc_capable; -} -static inline void resctrl_arch_enable_alloc (void ) {static_branch_enable_cpuslocked (&rdt_alloc_enable_key);
@@ -67,11 +62,6 @@ static inline void resctrl_arch_disable_alloc(void)static_branch_dec_cpuslocked (&rdt_enable_key); } -static inline bool resctrl_arch_mon_capable (void ) -{ -return rdt_mon_capable; -} -static inline void resctrl_arch_enable_mon (void ) {static_branch_enable_cpuslocked (&rdt_mon_enable_key);
drivers/resctrl/mpam_resctrl.c +2 -24
@@ -75,28 +75,6 @@ static DECLARE_WAIT_QUEUE_HEAD(wait_cacheinfo_ready);*/ static bool resctrl_enabled; -bool resctrl_arch_alloc_capable (void ) -{ -struct mpam_resctrl_res *res; -enum resctrl_res_level rid; - -for_each_mpam_resctrl_control (res, rid) { -if (res->resctrl_res.alloc_capable) -return true; - } - -return false; -} - -bool resctrl_arch_mon_capable (void ) -{ -struct mpam_resctrl_res *res = &mpam_resctrl_controls[RDT_RESOURCE_L3 ]; -struct rdt_resource *l3 = &res->resctrl_res; - -/* All monitors are presented as being on the L3 cache */ -return l3->mon_capable; -} -bool resctrl_arch_is_evt_configurable (enum resctrl_event_id evt) {return false;
@@ -1902,9 +1880,9 @@ int mpam_resctrl_setup(void)cpus_read_unlock (); -if (!resctrl_arch_alloc_capable () && !resctrl_arch_mon_capable ()) { +if (!resctrl_alloc_capable () && !resctrl_mon_capable ()) {pr_debug ("No alloc(%u) or monitor(%u) found - resctrl not supported\n" , -resctrl_arch_alloc_capable (),resctrl_arch_mon_capable ()); +resctrl_alloc_capable (),resctrl_mon_capable ());return -EOPNOTSUPP ; }
fs/resctrl/monitor.c +5 -5
@@ -338,7 +338,7 @@ void free_rmid(u32 closid, u32 rmid)* allows architectures that ignore the closid parameter to avoid an * unnecessary check. */ -if (!resctrl_arch_mon_capable () || +if (!resctrl_mon_capable () || idx ==resctrl_arch_rmid_idx_encode (RESCTRL_RESERVED_CLOSID ,RESCTRL_RESERVED_RMID ))return ;
@@ -888,7 +888,7 @@ void mbm_handle_overflow(struct work_struct *work)* If the filesystem has been unmounted this work no longer needs to * run. */ -if (!resctrl_mounted || !resctrl_arch_mon_capable ()) +if (!resctrl_mounted || !resctrl_mon_capable ()) goto out_unlock;/*
@@ -950,7 +950,7 @@ void mbm_setup_overflow_handler(struct rdt_l3_mon_domain *dom, unsigned long del* When a domain comes online there is no guarantee the filesystem is * mounted. If not, there is no need to catch counter overflow. */ -if (!resctrl_mounted || !resctrl_arch_mon_capable ()) +if (!resctrl_mounted || !resctrl_mon_capable ())return ; cpu =cpumask_any_housekeeping (&dom->hdr.cpu_mask, exclude_cpu); dom->mbm_work_cpu = cpu;
@@ -966,7 +966,7 @@ int setup_rmid_lru_list(void)u32 idx;int i; -if (!resctrl_arch_mon_capable ()) +if (!resctrl_mon_capable ())return 0 ;/*
@@ -1006,7 +1006,7 @@ int setup_rmid_lru_list(void)void free_rmid_lru_list (void ) { -if (!resctrl_arch_mon_capable ()) +if (!resctrl_mon_capable ())return ;mutex_lock (&rdtgroup_mutex);
fs/resctrl/pseudo_lock.c +3 -3
@@ -383,7 +383,7 @@ static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp)if (ret) goto err_cpus; -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) { ret =rdtgroup_kn_mode_restrict (rdtgrp,"mon_groups" );if (ret) goto err_cpus_list;
@@ -430,7 +430,7 @@ static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp)if (ret) goto err_cpus; -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) { ret =rdtgroup_kn_mode_restore (rdtgrp,"mon_groups" ,0777 );if (ret) goto err_cpus_list;
@@ -577,7 +577,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) {int ret; -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) { ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) {rdt_last_cmd_puts ("Out of RMIDs\n" );
fs/resctrl/rdtgroup.c +18 -18
@@ -691,7 +691,7 @@ static bool is_closid_match(struct task_struct *t, struct rdtgroup *r)static bool is_rmid_match (struct task_struct *t,struct rdtgroup *r) { -return (resctrl_arch_mon_capable () && (r->type ==RDTMON_GROUP ) && +return (resctrl_mon_capable () && (r->type ==RDTMON_GROUP ) &&resctrl_arch_match_rmid (t, r->mon.parent->closid, r->mon.rmid)); }
@@ -3157,9 +3157,9 @@ static void resctrl_unmount(void)resctrl_arch_reset_all_ctrls (r);resctrl_fs_teardown (); -if (resctrl_arch_alloc_capable ()) +if (resctrl_alloc_capable ())resctrl_arch_disable_alloc (); -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ())resctrl_arch_disable_mon (); resctrl_mounted = false;mutex_unlock (&rdtgroup_mutex);
@@ -3213,7 +3213,7 @@ static int rdt_get_tree(struct fs_context *fc)if (ret) goto out_schemata_free; -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ()) flags |=RFTYPE_MON ; ret =rdtgroup_add_files (rdtgroup_default.kn, flags);
@@ -3226,7 +3226,7 @@ static int rdt_get_tree(struct fs_context *fc)if (ret <0 ) goto out_closid_exit; -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) { ret =mongroup_create_dir (rdtgroup_default.kn, &rdtgroup_default,"mon_groups" , &kn_mongrp);
@@ -3246,12 +3246,12 @@ static int rdt_get_tree(struct fs_context *fc)if (ret) goto out_mondata; -if (resctrl_arch_alloc_capable ()) +if (resctrl_alloc_capable ())resctrl_arch_enable_alloc (); -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ())resctrl_arch_enable_mon (); -if (resctrl_arch_alloc_capable () ||resctrl_arch_mon_capable ()) +if (resctrl_alloc_capable () ||resctrl_mon_capable ()) resctrl_mounted = true;if (resctrl_is_mbm_enabled ()) {
@@ -3294,10 +3294,10 @@ static int rdt_get_tree(struct fs_context *fc)return ret; out_mondata: -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ())kernfs_remove (kn_mondata); out_mongrp: -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) {mon_put_kn_priv ();rdtgroup_unassign_cntrs (&rdtgroup_default);kernfs_remove (kn_mongrp);
@@ -3886,7 +3886,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) {int ret; -if (!resctrl_arch_mon_capable ()) +if (!resctrl_mon_capable ())return 0 ; ret =alloc_rmid (rdtgrp->closid);
@@ -3911,7 +3911,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)static void mkdir_rdt_prepare_rmid_free (struct rdtgroup *rgrp) { -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) {rdtgroup_unassign_cntrs (rgrp);free_rmid (rgrp->closid, rgrp->mon.rmid); }
@@ -4002,7 +4002,7 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,if (rtype ==RDTCTRL_GROUP ) { files =RFTYPE_BASE |RFTYPE_CTRL ; -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ()) files |=RFTYPE_MON ; }else { files =RFTYPE_BASE |RFTYPE_MON ;
@@ -4111,7 +4111,7 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn,list_add (&rdtgrp->rdtgroup_list, &rdt_all_groups); -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) {/* * Create an empty mon_groups directory to hold the subset * of tasks and cpus to monitor.
@@ -4152,11 +4152,11 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,* allocation is supported, add a control and monitoring * subdirectory */ -if (resctrl_arch_alloc_capable () && parent_kn == rdtgroup_default.kn) +if (resctrl_alloc_capable () && parent_kn == rdtgroup_default.kn)return rdtgroup_mkdir_ctrl_mon (parent_kn, name, mode);/* Else, attempt to add a monitoring subdirectory. */ -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ())return rdtgroup_mkdir_mon (parent_kn, name, mode);return -EPERM ;
@@ -4571,7 +4571,7 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *h* If resctrl is mounted, remove all the * per domain monitor data directories. */ -if (resctrl_mounted &&resctrl_arch_mon_capable ()) +if (resctrl_mounted &&resctrl_mon_capable ())rmdir_mondata_subdir_allrdtgrp (r, hdr);if (r->rid !=RDT_RESOURCE_L3 )
@@ -4708,7 +4708,7 @@ int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *hdr* by rdt_get_tree() calling mkdir_mondata_all(). * If resctrl is mounted, add per domain monitor data directories. */ -if (resctrl_mounted &&resctrl_arch_mon_capable ()) +if (resctrl_mounted &&resctrl_mon_capable ())mkdir_mondata_subdir_allrdtgrp (r, hdr); out_unlock:
Signature · 1 line
2.56.0
Re: [PATCH v14 06/24] arm,x86,fs/resctrl: Replace architecture capability helpers with generic versions
Hi Tony,
Quoted 10 lines · Tony Luck
resctrl supports allocation or monitoring if any resource is capable of it. Each architecture answers this via
resctrl_arch_alloc_capable()andresctrl_arch_mon_capable(). x86 maintains global flags for this in addition to the per-resource flags and must keep both in sync. MPAM derives the answer from its per-resource flags.The per-resource flags are part of the resource description that the file system already has, so the architecture helpers duplicate information available to the file system.
Quoted 9 lines
Define
resctrl_alloc_capable()andresctrl_mon_capable()in the file system based on the per-resource flags and remove the architecture helpers, in preparation for removing the x86 global flags. The result is unchanged for MPAM, which presents all monitors on the L3 resource.Suggested-by Reinette Chatre <[email protected]>Signed-off-by Tony Luck <[email protected]>---
Since this possible regress window is very small, and I did not find any issue in this change,
thanks,
Chenyu
Re: [PATCH v14 06/24] arm,x86,fs/resctrl: Replace architecture capability helpers with generic versions
In reply to Tony LuckHi Tony,
Quoted 10 lines · Tony Luck
resctrl supports allocation or monitoring if any resource is capable of it. Each architecture answers this via
resctrl_arch_alloc_capable()andresctrl_arch_mon_capable(). x86 maintains global flags for this in addition to the per-resource flags and must keep both in sync. MPAM derives the answer from its per-resource flags.The per-resource flags are part of the resource description that the file system already has, so the architecture helpers duplicate information available to the file system.
Quoted 7 lines
Define
resctrl_alloc_capable()andresctrl_mon_capable()in the file system based on the per-resource flags and remove the architecture helpers, in preparation for removing the x86 global flags. The result is unchanged for MPAM, which presents all monitors on the L3 resource.Suggested-by Reinette Chatre <[email protected]>Signed-off-by Tony Luck <[email protected]>
Looks good to me. For the MPAM parts:
Thanks,
Ben
Quoted 363 lines
---
v14:
New Subject: without {alloc,mon}
New commit message in context, problem, solution format Did not apply Reinette RB tag because of commit message rewrite.--- include/linux/arm_mpam.h | 3 --- include/linux/resctrl.h | 30 ++++++++++++++++++++++++++++ arch/x86/include/asm/resctrl.h | 10 ---------- drivers/resctrl/mpam_resctrl.c | 26 ++---------------------- fs/resctrl/monitor.c | 10 +++++----- fs/resctrl/pseudo_lock.c | 6 +++--- fs/resctrl/rdtgroup.c | 36 +++++++++++++++++----------------- 7 files changed, 58 insertions(+), 63 deletions(-)include/linux/arm_mpam.h +0
-3@@ -50,9 +50,6 @@ static inline int mpam_ris_create(struct mpam_msc *msc, u8 ris_idx, }#endif -bool resctrl_arch_alloc_capable (void ); -bool resctrl_arch_mon_capable (void ); -void resctrl_arch_set_cpu_default_closid (int cpu,u32 closid);void resctrl_arch_set_closid_rmid (struct task_struct *tsk,u32 closid,u32 rmid);void resctrl_arch_set_cpu_default_closid_rmid (int cpu,u32 closid,u32 rmid);include/linux/resctrl.h +30
-0@@ -346,6 +346,36 @@ struct rdt_resource {*/ struct rdt_resource *resctrl_arch_get_resource (enum resctrl_res_level l); +/* +* resctrl_alloc_capable - Are any allocation resources enabled? +*/ +static inline bool resctrl_alloc_capable (void ) +{ +struct rdt_resource *r; + +for_each_rdt_resource (r) { +if (r->alloc_capable) +return true; + } + +return false; +} + +/* +* resctrl_mon_capable - Are any monitor resources enabled? +*/ +static inline bool resctrl_mon_capable (void ) +{ +struct rdt_resource *r; + +for_each_rdt_resource (r) { +if (r->mon_capable) +return true; + } + +return false; +} +/** * struct resctrl_schema - configuration abilities of a resource presented to * user-space arch/x86/include/asm/resctrl.h +0
-10@@ -50,11 +50,6 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); -static inline bool resctrl_arch_alloc_capable (void ) -{ -return rdt_alloc_capable; -} -static inline void resctrl_arch_enable_alloc (void ) {static_branch_enable_cpuslocked (&rdt_alloc_enable_key);@@ -67,11 +62,6 @@ static inline void resctrl_arch_disable_alloc(void)static_branch_dec_cpuslocked (&rdt_enable_key); } -static inline bool resctrl_arch_mon_capable (void ) -{ -return rdt_mon_capable; -} -static inline void resctrl_arch_enable_mon (void ) {static_branch_enable_cpuslocked (&rdt_mon_enable_key);drivers/resctrl/mpam_resctrl.c +2
-24@@ -75,28 +75,6 @@ static DECLARE_WAIT_QUEUE_HEAD(wait_cacheinfo_ready);*/ static bool resctrl_enabled; -bool resctrl_arch_alloc_capable (void ) -{ -struct mpam_resctrl_res *res; -enum resctrl_res_level rid; - -for_each_mpam_resctrl_control (res, rid) { -if (res->resctrl_res.alloc_capable) -return true; - } - -return false; -} - -bool resctrl_arch_mon_capable (void ) -{ -struct mpam_resctrl_res *res = &mpam_resctrl_controls[RDT_RESOURCE_L3 ]; -struct rdt_resource *l3 = &res->resctrl_res; - -/* All monitors are presented as being on the L3 cache */ -return l3->mon_capable; -} -bool resctrl_arch_is_evt_configurable (enum resctrl_event_id evt) {return false;@@ -1902,9 +1880,9 @@ int mpam_resctrl_setup(void)cpus_read_unlock (); -if (!resctrl_arch_alloc_capable () && !resctrl_arch_mon_capable ()) { +if (!resctrl_alloc_capable () && !resctrl_mon_capable ()) {pr_debug ("No alloc(%u) or monitor(%u) found - resctrl not supported\n" , -resctrl_arch_alloc_capable (),resctrl_arch_mon_capable ()); +resctrl_alloc_capable (),resctrl_mon_capable ());return -EOPNOTSUPP ; }fs/resctrl/monitor.c +5
-5@@ -338,7 +338,7 @@ void free_rmid(u32 closid, u32 rmid)* allows architectures that ignore the closid parameter to avoid an * unnecessary check. */ -if (!resctrl_arch_mon_capable () || +if (!resctrl_mon_capable () || idx ==resctrl_arch_rmid_idx_encode (RESCTRL_RESERVED_CLOSID ,RESCTRL_RESERVED_RMID ))return ;@@ -888,7 +888,7 @@ void mbm_handle_overflow(struct work_struct *work)* If the filesystem has been unmounted this work no longer needs to * run. */ -if (!resctrl_mounted || !resctrl_arch_mon_capable ()) +if (!resctrl_mounted || !resctrl_mon_capable ()) goto out_unlock;/* @@ -950,7 +950,7 @@ void mbm_setup_overflow_handler(struct rdt_l3_mon_domain *dom, unsigned long del* When a domain comes online there is no guarantee the filesystem is * mounted. If not, there is no need to catch counter overflow. */ -if (!resctrl_mounted || !resctrl_arch_mon_capable ()) +if (!resctrl_mounted || !resctrl_mon_capable ())return ; cpu =cpumask_any_housekeeping (&dom->hdr.cpu_mask, exclude_cpu); dom->mbm_work_cpu = cpu;@@ -966,7 +966,7 @@ int setup_rmid_lru_list(void)u32 idx;int i; -if (!resctrl_arch_mon_capable ()) +if (!resctrl_mon_capable ())return 0 ;/* @@ -1006,7 +1006,7 @@ int setup_rmid_lru_list(void)void free_rmid_lru_list (void ) { -if (!resctrl_arch_mon_capable ()) +if (!resctrl_mon_capable ())return ;mutex_lock (&rdtgroup_mutex);fs/resctrl/pseudo_lock.c +3
-3@@ -383,7 +383,7 @@ static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp)if (ret) goto err_cpus; -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) { ret =rdtgroup_kn_mode_restrict (rdtgrp,"mon_groups" );if (ret) goto err_cpus_list;@@ -430,7 +430,7 @@ static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp)if (ret) goto err_cpus; -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) { ret =rdtgroup_kn_mode_restore (rdtgrp,"mon_groups" ,0777 );if (ret) goto err_cpus_list;@@ -577,7 +577,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) {int ret; -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) { ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) {rdt_last_cmd_puts ("Out of RMIDs\n" );fs/resctrl/rdtgroup.c +18
-18@@ -691,7 +691,7 @@ static bool is_closid_match(struct task_struct *t, struct rdtgroup *r)static bool is_rmid_match (struct task_struct *t,struct rdtgroup *r) { -return (resctrl_arch_mon_capable () && (r->type ==RDTMON_GROUP ) && +return (resctrl_mon_capable () && (r->type ==RDTMON_GROUP ) &&resctrl_arch_match_rmid (t, r->mon.parent->closid, r->mon.rmid)); }@@ -3157,9 +3157,9 @@ static void resctrl_unmount(void)resctrl_arch_reset_all_ctrls (r);resctrl_fs_teardown (); -if (resctrl_arch_alloc_capable ()) +if (resctrl_alloc_capable ())resctrl_arch_disable_alloc (); -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ())resctrl_arch_disable_mon (); resctrl_mounted = false;mutex_unlock (&rdtgroup_mutex);@@ -3213,7 +3213,7 @@ static int rdt_get_tree(struct fs_context *fc)if (ret) goto out_schemata_free; -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ()) flags |=RFTYPE_MON ; ret =rdtgroup_add_files (rdtgroup_default.kn, flags);@@ -3226,7 +3226,7 @@ static int rdt_get_tree(struct fs_context *fc)if (ret <0 ) goto out_closid_exit; -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) { ret =mongroup_create_dir (rdtgroup_default.kn, &rdtgroup_default,"mon_groups" , &kn_mongrp);@@ -3246,12 +3246,12 @@ static int rdt_get_tree(struct fs_context *fc)if (ret) goto out_mondata; -if (resctrl_arch_alloc_capable ()) +if (resctrl_alloc_capable ())resctrl_arch_enable_alloc (); -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ())resctrl_arch_enable_mon (); -if (resctrl_arch_alloc_capable () ||resctrl_arch_mon_capable ()) +if (resctrl_alloc_capable () ||resctrl_mon_capable ()) resctrl_mounted = true;if (resctrl_is_mbm_enabled ()) {@@ -3294,10 +3294,10 @@ static int rdt_get_tree(struct fs_context *fc)return ret; out_mondata: -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ())kernfs_remove (kn_mondata); out_mongrp: -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) {mon_put_kn_priv ();rdtgroup_unassign_cntrs (&rdtgroup_default);kernfs_remove (kn_mongrp);@@ -3886,7 +3886,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) {int ret; -if (!resctrl_arch_mon_capable ()) +if (!resctrl_mon_capable ())return 0 ; ret =alloc_rmid (rdtgrp->closid);@@ -3911,7 +3911,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)static void mkdir_rdt_prepare_rmid_free (struct rdtgroup *rgrp) { -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) {rdtgroup_unassign_cntrs (rgrp);free_rmid (rgrp->closid, rgrp->mon.rmid); }@@ -4002,7 +4002,7 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,if (rtype ==RDTCTRL_GROUP ) { files =RFTYPE_BASE |RFTYPE_CTRL ; -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ()) files |=RFTYPE_MON ; }else { files =RFTYPE_BASE |RFTYPE_MON ;@@ -4111,7 +4111,7 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn,list_add (&rdtgrp->rdtgroup_list, &rdt_all_groups); -if (resctrl_arch_mon_capable ()) { +if (resctrl_mon_capable ()) {/* * Create an empty mon_groups directory to hold the subset * of tasks and cpus to monitor. @@ -4152,11 +4152,11 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,* allocation is supported, add a control and monitoring * subdirectory */ -if (resctrl_arch_alloc_capable () && parent_kn == rdtgroup_default.kn) +if (resctrl_alloc_capable () && parent_kn == rdtgroup_default.kn)return rdtgroup_mkdir_ctrl_mon (parent_kn, name, mode);/* Else, attempt to add a monitoring subdirectory. */ -if (resctrl_arch_mon_capable ()) +if (resctrl_mon_capable ())return rdtgroup_mkdir_mon (parent_kn, name, mode);return -EPERM ;@@ -4571,7 +4571,7 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *h* If resctrl is mounted, remove all the * per domain monitor data directories. */ -if (resctrl_mounted &&resctrl_arch_mon_capable ()) +if (resctrl_mounted &&resctrl_mon_capable ())rmdir_mondata_subdir_allrdtgrp (r, hdr);if (r->rid !=RDT_RESOURCE_L3 )@@ -4708,7 +4708,7 @@ int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *hdr* by rdt_get_tree() calling mkdir_mondata_all(). * If resctrl is mounted, add per domain monitor data directories. */ -if (resctrl_mounted &&resctrl_arch_mon_capable ()) +if (resctrl_mounted &&resctrl_mon_capable ())mkdir_mondata_subdir_allrdtgrp (r, hdr); out_unlock:
Tony Luck
maintainerPATCH v14 7/24 x86/resctrl: Update special case for Intel Haswell enumeration
In reply to Tony LuckCertain Intel Haswell CPUs support L3 cache allocation but do not enumerate the feature. A quirk probes for support and provides hard-coded parameters. The quirk sets both the L3 resource and the global rdt_alloc_capable, but only the global is used to signal that normal enumeration can be skipped.
Setting the global in the quirk is redundant. The quirk runs before allocation enumeration and is the only code that can mark a resource as allocation capable that early, so finding any allocation capable resource at the start of enumeration already identifies the quirk case.
Stop setting rdt_alloc_capable in the quirk and skip allocation enumeration when resctrl_alloc_capable() reports a resource is already allocation capable, in preparation for removing the global.
No functional change intended.
---
v14:
Updated commit comment in context, problem, solution form.
--- arch/x86/kernel/cpu/resctrl/core.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +5 -3
@@ -177,8 +177,6 @@ static inline void cache_alloc_hsw_probe(void) r->cache.min_cbm_bits =2 ; r->cache.arch_has_sparse_bitmasks = false; r->alloc_capable = true; - - rdt_alloc_capable = true; }/*
@@ -930,7 +928,11 @@ static __init bool get_rdt_alloc_resources(void)struct rdt_resource *r;bool ret = false; -if (rdt_alloc_capable) +/* +* On Haswell systems cache_alloc_hsw_probe() already marked +* RDT_RESOURCE_L3 as alloc_capable. No further enumeration is needed. +*/ +if (resctrl_alloc_capable ())return true;if (!boot_cpu_has (X86_FEATURE_RDT_A ))
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 8/24 x86/resctrl: Delete rdt_alloc_capable and rdt_mon_capable
In reply to Tony LuckWhether the system supports allocation or monitoring is now determined from the per-resource capability flags via resctrl_alloc_capable() and resctrl_mon_capable(). The x86 globals rdt_alloc_capable and rdt_mon_capable are still written during enumeration and when telemetry monitoring is discovered at mount time, but nothing reads them.
Every place that writes a global also sets the matching resource flag, so the globals carry no information that the resource flags do not.
Delete rdt_alloc_capable and rdt_mon_capable.
No functional change intended.
---
v14:
New commit message in context, problem, solution format.
--- arch/x86/include/asm/resctrl.h | 2 -- arch/x86/kernel/cpu/resctrl/core.c | 13 +++---------- arch/x86/kernel/cpu/resctrl/monitor.c | 6 ------ 3 files changed, 3 insertions(+), 18 deletions(-)
arch/x86/include/asm/resctrl.h +0 -2
@@ -42,8 +42,6 @@ struct resctrl_pqr_state { DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state); -extern bool rdt_alloc_capable; -extern bool rdt_mon_capable;extern unsigned int rdt_l3_mon_scale;DECLARE_STATIC_KEY_FALSE (rdt_enable_key);
arch/x86/kernel/cpu/resctrl/core.c +3 -10
@@ -45,12 +45,6 @@ static DEFINE_MUTEX(domain_list_lock);*/ DEFINE_PER_CPU(struct resctrl_pqr_state, pqr_state); -/* -* Global boolean for rdt_alloc which is true if any -* resource allocation is enabled. -*/ -bool rdt_alloc_capable; -static void mba_wrmsr_intel (struct msr_param *m);static void cat_wrmsr (struct msr_param *m);static void mba_wrmsr_amd (struct msr_param *m);
@@ -781,7 +775,6 @@ void resctrl_arch_pre_mount(void)cpus_read_lock ();mutex_lock (&domain_list_lock); r->mon_capable = true; - rdt_mon_capable = true;for_each_online_cpu (cpu)domain_add_cpu_mon (cpu, r);mutex_unlock (&domain_list_lock);
@@ -1026,10 +1019,10 @@ static __init void check_quirks(void)static __init bool get_rdt_resources (void ) { - rdt_alloc_capable =get_rdt_alloc_resources (); - rdt_mon_capable =get_rdt_mon_resources (); +bool alloc =get_rdt_alloc_resources (); +bool mon =get_rdt_mon_resources (); -return (rdt_mon_capable || rdt_alloc_capable); +return alloc || mon; }static __init void rdt_init_res_defs_intel (void )
arch/x86/kernel/cpu/resctrl/monitor.c +0 -6
@@ -26,12 +26,6 @@#include "internal.h" -/* -* Global boolean for rdt_monitor which is true if any -* resource monitoring is enabled. -*/ -bool rdt_mon_capable; -/* * Scale factor to convert L3 monitor events to bytes. */
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 9/24 fs/resctrl: Remove redundant calls to resctrl_mon_capable()
In reply to Tony Luckresctrl_mon_capable() reports whether the system supports monitoring by scanning all resources. On x86 it used to read a single architecture-provided flag. Calls on paths that run repeatedly now pay for the scan.
Two such callers already know that monitoring is supported:
1) mbm_handle_overflow()
The MBM overflow handler runs once per second on each L3 monitoring domain. It is only set up when MBM events are enabled, which requires L3 monitoring support.
2) is_rmid_match()
Called for every task in the system when walking the task list to read a tasks file, remove a group, or unmount. It only matches MON groups, which can only be created on a system that supports monitoring.
Remove the call to resctrl_mon_capable() from both functions.
No functional change intended.
---
v14:
New commit message confessing that this is fixing a performance problem created by an earlier patch this series.
--- fs/resctrl/monitor.c | 2 +- fs/resctrl/rdtgroup.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
fs/resctrl/monitor.c +1 -1
@@ -888,7 +888,7 @@ void mbm_handle_overflow(struct work_struct *work)* If the filesystem has been unmounted this work no longer needs to * run. */ -if (!resctrl_mounted || !resctrl_mon_capable ()) +if (!resctrl_mounted) goto out_unlock;/*
fs/resctrl/rdtgroup.c +1 -1
@@ -691,7 +691,7 @@ static bool is_closid_match(struct task_struct *t, struct rdtgroup *r)static bool is_rmid_match (struct task_struct *t,struct rdtgroup *r) { -return (resctrl_mon_capable () && (r->type ==RDTMON_GROUP ) && +return (r->type ==RDTMON_GROUP &&resctrl_arch_match_rmid (t, r->mon.parent->closid, r->mon.rmid)); }
Signature · 1 line
2.56.0
Re: [PATCH v14 09/24] fs/resctrl: Remove redundant calls to resctrl_mon_capable()
Hi Tony,
Quoted 11 lines · Tony Luck
resctrl_mon_capable()reports whether the system supports monitoring by scanning all resources. On x86 it used to read a single architecture-provided flag. Calls on paths that run repeatedly now pay for the scan.Two such callers already know that monitoring is supported:
1)
mbm_handle_overflow()
The MBM overflow handler runs once per second on each L3 monitoring domain. It is only set up when MBM events are enabled, which requires L3 monitoring support.
There is also a resctrl_mon_capable() check in mbm_setup_overflow_handler(), do we need to remove it too? But it is only called once in mbm_setup_overflow_handler(), leave it as it is should be OK.
Quoted 11 lines
2)
is_rmid_match()
Called for every task in the system when walking the task list to read a tasks file, remove a group, or unmount. It only matches MON groups, which can only be created on a system that supports monitoring.Remove the call to
resctrl_mon_capable()from both functions.No functional change intended.
Signed-off-by Tony Luck <[email protected]>
thanks,
Chenyu
Luck, Tony
maintainerRe: [PATCH v14 09/24] fs/resctrl: Remove redundant calls to resctrl_mon_capable()
Hi Chenyu,
Quoted 15 lines · Chen Yu
Hi Tony,
Quoted 11 lines · Tony Luck
resctrl_mon_capable()reports whether the system supports monitoring by scanning all resources. On x86 it used to read a single architecture-provided flag. Calls on paths that run repeatedly now pay for the scan.Two such callers already know that monitoring is supported:
1)
mbm_handle_overflow()
The MBM overflow handler runs once per second on each L3 monitoring domain. It is only set up when MBM events are enabled, which requires L3 monitoring support.
Quoted 3 lines
There is also a
resctrl_mon_capable()check inmbm_setup_overflow_handler(), do we need to remove it too? But it is only called once inmbm_setup_overflow_handler(), leave it as it is should be OK.
Yes. I was only worried about calls in hot code paths. This call can happen more than once if CPU hotplug removes and adds a domain. But that's still not a hot code path.
Quoted 6 lines
Quoted 6 lines
2)
is_rmid_match()
Called for every task in the system when walking the task list to read a tasks file, remove a group, or unmount. It only matches MON groups, which can only be created on a system that supports monitoring.
Quoted 8 lines
Quoted 6 lines
Remove the call to
resctrl_mon_capable()from both functions.No functional change intended.
Signed-off-by Tony Luck <[email protected]>Reviewed-by Chen Yu <[email protected]>
Thanks for the review.
Quoted 3 lines
thanks,
Chenyu
-Tony
Tony Luck
maintainerPATCH v14 10/24 x86/resctrl: Honor rdt={perf|energy} options to force enable AET events
In reply to Tony Luckresctrl can disable a feature the platform supports, for example because of an erratum or, for Application Energy Telemetry (AET), because an event group has insufficient RMIDs. The "rdt=" kernel command line option lets the user force such a feature on anyway, and also lets the user disable supported features they do not need.
resctrl enumerates AET once, on first mount. Enumeration skips an event group that is marked disabled. An event group with insufficient RMIDs is marked disabled even when the user forced it on, but this happens after the test for forced off when enumeration begins for an event group.
This will be broken when AET is enumerated on every mount. The disable mark recorded for insufficient RMIDs on one mount causes the event group to be skipped on the next, ignoring the user's explicit rdt= request to enable it.
In preparation for enumerating AET on every mount, skip a disabled event group only if the user has not forced it on. As a result, when the command line both disables and enables the same event group, for example "rdt=!perf,perf:<guid>", enabling now takes precedence. This matches how rdt= handles the other resctrl features.
---
v14:
New commit message. Dropped previous supplied Reinette RB tag.
--- arch/x86/kernel/cpu/resctrl/intel_aet.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/intel_aet.c +1 -1
@@ -262,7 +262,7 @@ static bool enable_events(struct event_group *e, struct pmt_feature_group *p)struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_PERF_PKG ].r_resctrl;int skipped_events =0 ; -if (e->force_off) +if (e->force_off && !e->force_on)return false;if (!group_has_usable_regions (e, p))
Signature · 1 line
2.56.0
Re: [PATCH v14 10/24] x86/resctrl: Honor rdt={perf|energy} options to force enable AET events
Quoted 34 lines · Tony Luck
resctrl can disable a feature the platform supports, for example because of an erratum or, for Application Energy Telemetry (AET), because an event group has insufficient RMIDs. The "rdt=" kernel command line option lets the user force such a feature on anyway, and also lets the user disable supported features they do not need.
resctrl enumerates AET once, on first mount. Enumeration skips an event group that is marked disabled. An event group with insufficient RMIDs is marked disabled even when the user forced it on, but this happens after the test for forced off when enumeration begins for an event group.
This will be broken when AET is enumerated on every mount. The disable mark recorded for insufficient RMIDs on one mount causes the event group to be skipped on the next, ignoring the user's explicit rdt= request to enable it.
In preparation for enumerating AET on every mount, skip a disabled event group only if the user has not forced it on. As a result, when the command line both disables and enables the same event group, for example "rdt=!perf,perf:<guid>", enabling now takes precedence. This matches how rdt= handles the other resctrl features.
Signed-off-by Tony Luck <[email protected]>---
v14:
New commit message. Dropped previous supplied Reinette RB tag.--- arch/x86/kernel/cpu/resctrl/intel_aet.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kernel/cpu/resctrl/intel_aet.c +0
-0
arch/x86/kernel/cpu/resctrl/intel_aet.c static bool enable_events(struct event_group *e, struct pmt_feature_group *p) · line 267
Quoted 7 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +1
-1@@ -262,7 +262,7 @@ static bool enable_events(struct event_group *e, struct pmt_feature_group *p)struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_PERF_PKG ].r_resctrl;int skipped_events =0 ; -if (e->force_off) +if (e->force_off && !e->force_on)return false;
OK, it seems that the user always has the highest priority in deciding whether this event is enabled, which overrides the hardware/kernel decision. This will also be the design for region aware MBA's erdt:3waymba. Per my understanding,
thanks,
Chenyu
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +0
-0if (!group_has_usable_regions (e, p))Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 11/24 fs/resctrl: Add interface to disable a monitor event
In reply to Tony Luckresctrl assumes that every monitor event is enabled before any domain is created, since domains are created from CPU hotplug callbacks and their per-domain state depends on the events enabled at that time. Events, once enabled, stay enabled.
resctrl enumerates Application Energy Telemetry (AET) events from data provided by the pmt_telemetry driver, which currently must be built in. Allowing the driver to be a loadable module means AET events must be enumerated on every mount and disabled again on unmount, so the driver can be unloaded while resctrl is not mounted. There is no way to disable an event.
Add resctrl_disable_mon_event(). The architecture owns domain lifetime, so it is responsible for calling this only while resctrl is unmounted and for cleaning up any per-domain state. These requirements are not enforced in code, so document them in kerneldoc, alongside the matching requirements for enabling an event.
---
v14:
New commit message. Dropped previous applied Reinette RB tag.
--- include/linux/resctrl.h | 36 ++++++++++++++++++++++++++++++++++++ fs/resctrl/monitor.c | 15 +++++++++++++++ 2 files changed, 51 insertions(+)
include/linux/resctrl.h +36 -0
@@ -449,9 +449,45 @@ u32 resctrl_arch_get_num_closid(struct rdt_resource *r);u32 resctrl_arch_system_num_rmid_idx (void );int resctrl_arch_update_domains (struct rdt_resource *r,u32 closid); +/** +* resctrl_enable_mon_event() - Enable monitoring event +* @eventid: ID of the event. +* @any_cpu: True if event data can be read from any CPU. +* @binary_bits: Number of binary places of the fixed-point value expected to +* back a floating point event. Can only be set for floating point +* events. +* @arch_priv: Architecture private data associated with event. Passed back to +* architecture when reading the event via resctrl_arch_rmid_read(). +* +* The file system must not be mounted when enabling an event. +* +* Events that require per-domain (architectural and/or filesystem) state must +* be enabled before the domain structures are allocated. For example before +* CPU hotplug callbacks that allocate domain structures are registered. If the +* architecture discovers a resource after initialization it should enable +* events needing per-domain state before any domain structure allocation which +* should be coordinated with the CPU hotplug callbacks. +* +* Return: +* true if event was successfully enabled, false otherwise. +*/ bool resctrl_enable_mon_event (enum resctrl_event_id eventid,bool any_cpu,unsigned int binary_bits,void *arch_priv); +/** +* resctrl_disable_mon_event() - Disable monitoring event +* @eventid: ID of the event +* +* Never disable an event while the file system is mounted. Never +* disable an event that may be accessed while the file system is +* unmounted. +* +* Events that require per-domain (architectural and/or filesystem) state +* will require additional cleanup which should be coordinated with the CPU +* hotplug callbacks. +*/ +void resctrl_disable_mon_event (enum resctrl_event_id eventid); +bool resctrl_is_mon_event_enabled (enum resctrl_event_id eventid);bool resctrl_arch_is_evt_configurable (enum resctrl_event_id evt);
fs/resctrl/monitor.c +15 -0
@@ -1068,6 +1068,21 @@ bool resctrl_enable_mon_event(enum resctrl_event_id eventid, bool any_cpu,return true; } +void resctrl_disable_mon_event (enum resctrl_event_id eventid) +{ +if (WARN_ON_ONCE (eventid <QOS_FIRST_EVENT || eventid >=QOS_NUM_EVENTS )) +return ; +if (!mon_event_all[eventid].enabled) { +pr_warn ("Event %d already disabled\n" , eventid); +return ; + } + + mon_event_all[eventid].any_cpu = false; + mon_event_all[eventid].binary_bits =0 ; + mon_event_all[eventid].arch_priv =NULL ; + mon_event_all[eventid].enabled = false; +} +bool resctrl_is_mon_event_enabled (enum resctrl_event_id eventid) { return eventid >= QOS_FIRST_EVENT && eventid < QOS_NUM_EVENTS &&
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 12/24 arm,x86,fs/resctrl: Allocate maximum needed rmid_ptrs[]
In reply to Tony Luckresctrl keeps per-RMID state in rmid_ptrs[]. It is allocated on first mount, sized for the number of RMIDs available at that time, and reused by every subsequent mount. The number of RMIDs available is the smallest number supported across all monitoring resources.
Application Energy Telemetry (AET) requires the pmt_telemetry driver to be built in. Allowing it to be a module means AET may contribute to that smallest number on one mount and not on the next, so a later mount may need more entries than the first mount allocated. Reallocating per mount is not possible because the limbo handler continues to access rmid_ptrs[] after resctrl is unmounted.
Size rmid_ptrs[] for the maximum number of RMIDs the system can ever need, as reported by a new architecture hook. On x86 this is the number of RMIDs supported by L3 monitoring if present, since that bounds the number available on every mount, and otherwise the number of RMIDs that MSR_IA32_PQR_ASSOC.RMID can hold.
Rebuild the free list on every mount from the RMIDs available for that mount. Leave out RMIDs still dirty from an earlier mount, because the limbo handler returns them to the free list once they are clean.
---
v14:
New commit message.
--- include/linux/resctrl.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 27 +++++++++++++ drivers/resctrl/mpam_resctrl.c | 9 +++++ fs/resctrl/monitor.c | 64 ++++++++++++++++++++---------- 4 files changed, 79 insertions(+), 22 deletions(-)
include/linux/resctrl.h +1 -0
@@ -447,6 +447,7 @@ static inline u32 resctrl_get_default_ctrl(struct rdt_resource *r)/* The number of closid supported by this resource regardless of CDP */ u32 resctrl_arch_get_num_closid (struct rdt_resource *r);u32 resctrl_arch_system_num_rmid_idx (void ); +u32 resctrl_arch_system_max_rmid_idx (void );int resctrl_arch_update_domains (struct rdt_resource *r,u32 closid);/**
arch/x86/kernel/cpu/resctrl/core.c +27 -0
@@ -45,6 +45,9 @@ static DEFINE_MUTEX(domain_list_lock);*/ DEFINE_PER_CPU(struct resctrl_pqr_state, pqr_state); +/* Number of unique RMID values that can be written to MSR_IA32_PQR_ASSOC.RMID */ +static u32 pqr_assoc_num_rmid; +static void mba_wrmsr_intel (struct msr_param *m);static void cat_wrmsr (struct msr_param *m);static void mba_wrmsr_amd (struct msr_param *m);
@@ -124,6 +127,28 @@ u32 resctrl_arch_system_num_rmid_idx(void)return num_rmids ==U32_MAX ?0 : num_rmids; } +/** +* resctrl_arch_system_max_rmid_idx - Largest possible RMID index +* +* Return: Largest possible RMID index used for boot time allocations. +*/ +u32 resctrl_arch_system_max_rmid_idx (void ) +{ +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +u32 num_rmid = pqr_assoc_num_rmid; + +/* +* If the system is capable of L3 monitoring the maximum RMID value may +* be lower than the system maximum. Either because the L3 monitoring +* feature supports fewer RMIDs, or because SNC (Sub-NUMA Cluster) +* is enabled and divides RMIDs per cluster. +*/ +if (r->mon_capable) + num_rmid = r->mon.num_rmid; + +return num_rmid; +} +struct rdt_resource *resctrl_arch_get_resource (enum resctrl_res_level l) {if (l >=RDT_NUM_RESOURCES )
@@ -967,6 +992,8 @@ static __init bool get_rdt_mon_resources(void)if (!cpu_feature_enabled (X86_FEATURE_CQM ))return false; + pqr_assoc_num_rmid =cpuid_ebx (0xf ) +1 ; +/* Any of the L3 monitoring features? */ if (!cpu_feature_enabled (X86_FEATURE_CQM_LLC ))return false;
drivers/resctrl/mpam_resctrl.c +9 -0
@@ -257,6 +257,15 @@ u32 resctrl_arch_system_num_rmid_idx(void)return (mpam_pmg_max +1 ) * (mpam_partid_max +1 ); } +/* +* File system calls this for one-time allocation of structures. +* Return the largest possible value. +*/ +u32 resctrl_arch_system_max_rmid_idx (void ) +{ +return resctrl_arch_system_num_rmid_idx (); +} +u32 resctrl_arch_rmid_idx_encode (u32 closid,u32 rmid) {return closid * (mpam_pmg_max +1 ) + rmid;
fs/resctrl/monitor.c +42 -22
93 lines not on this page · raw message
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 13/24 arm,x86,fs/resctrl: Use right size for L3 monitor data structures
In reply to Tony LuckOn x86 the number of RMIDs available for use is the smallest number of RMIDs supported by any resource that supports monitoring.
Each L3 monitoring domain has structures with one entry per RMID index, allocated when the domain comes online and kept across mounts of the resctrl file system:
* the bitmap of RMIDs in limbo with L3 occupancy above the threshold,
* the per-event state used by the "mba_MBps" software controller to compute bandwidth,
* on x86, the per-event state that accumulates MBM counts across hardware counter overflow.
The resctrl file system sizes and walks the first two using the number of RMIDs available for use. x86 already sizes the third by the number of RMIDs supported by L3 monitoring.
The number of RMIDs available for use does not currently change between mounts because Application Energy Telemetry (AET) is enumerated only once. In preparation for building the pmt_telemetry driver as a module, AET will be enumerated on every mount, and its events will be available only when the driver is loaded at that time. The number of RMIDs available for use can then change between mounts while the L3 domain structures stay allocated. A domain that comes online while the number is small would be too small for a later mount. RMIDs that are still in limbo at unmount may also be beyond the number of RMIDs available for the next mount.
Add resctrl_arch_get_num_rmid_idx() for the architecture to report the number of RMID indexes a resource supports, which does not change between mounts. On x86 this is the number of RMIDs supported by L3 monitoring. MPAM keeps reporting the system-wide number. Use it to size and walk the L3 domain structures.
Keep checking every RMID supported by L3 for limbo. Do not return an RMID to the free list when it is beyond the number of RMIDs available for the current mount.
---
v14:
New commit message.
--- include/linux/resctrl.h | 7 +++++-- arch/x86/kernel/cpu/resctrl/core.c | 5 +++++ drivers/resctrl/mpam_resctrl.c | 5 +++++ fs/resctrl/monitor.c | 22 ++++++++++++++++++---- fs/resctrl/rdtgroup.c | 2 +- 5 files changed, 34 insertions(+), 7 deletions(-)
include/linux/resctrl.h +5 -2
@@ -183,10 +183,12 @@ struct mbm_cntr_cfg {* struct rdt_l3_mon_domain - group of CPUs sharing RDT_RESOURCE_L3 monitoring * @hdr: common header for different domain types * @ci_id: cache info id for this domain -* @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold +* @rmid_busy_llc: bitmap of which limbo RMIDs are above threshold. Sized for +* maximum supported RMIDs in L3 resource. * @mbm_states: Per-event pointer to the MBM event's saved state. * An MBM event's state is an array of struct mbm_state * indexed by RMID on x86 or combined CLOSID, RMID on Arm. +* Sized same as @rmid_busy_llc. * @mbm_over: worker to periodically read MBM h/w counters * @cqm_limbo: worker to periodically read CQM h/w counters * @mbm_work_cpu: worker CPU for MBM h/w counters
@@ -444,8 +446,9 @@ static inline u32 resctrl_get_default_ctrl(struct rdt_resource *r)return WARN_ON_ONCE (1 ); } -/* The number of closid supported by this resource regardless of CDP */ +/* The number of closid/rmid supported by this resource regardless of CDP */ u32 resctrl_arch_get_num_closid (struct rdt_resource *r); +u32 resctrl_arch_get_num_rmid_idx (struct rdt_resource *r);u32 resctrl_arch_system_num_rmid_idx (void );u32 resctrl_arch_system_max_rmid_idx (void );int resctrl_arch_update_domains (struct rdt_resource *r,u32 closid);
arch/x86/kernel/cpu/resctrl/core.c +5 -0
@@ -380,6 +380,11 @@ u32 resctrl_arch_get_num_closid(struct rdt_resource *r)return resctrl_to_arch_res (r)->num_closid; } +u32 resctrl_arch_get_num_rmid_idx (struct rdt_resource *r) +{ +return r->mon.num_rmid; +} +void rdt_ctrl_update (void *arg) {struct rdt_hw_resource *hw_res;
drivers/resctrl/mpam_resctrl.c +5 -0
@@ -252,6 +252,11 @@ u32 resctrl_arch_get_num_closid(struct rdt_resource *ignored)return mpam_partid_max +1 ; } +u32 resctrl_arch_get_num_rmid_idx (struct rdt_resource *ignored) +{ +return resctrl_arch_system_num_rmid_idx (); +} +u32 resctrl_arch_system_num_rmid_idx (void ) {return (mpam_pmg_max +1 ) * (mpam_partid_max +1 );
fs/resctrl/monitor.c +18 -4
@@ -120,10 +120,18 @@ static inline struct rmid_entry *__rmid_entry(u32 idx)static void limbo_release_entry (struct rmid_entry *entry) { +u32 cur_idx_limit =resctrl_arch_system_num_rmid_idx (); +lockdep_assert_held (&rdtgroup_mutex); rmid_limbo_count--; -list_add_tail (&entry->list, &rmid_free_lru); + +/* +* Limbo may be freeing an RMID from a previous mount where there +* were more RMIDs available. +*/ +if (resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid) < cur_idx_limit) +list_add_tail (&entry->list, &rmid_free_lru);if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) closid_num_dirty_rmid[entry->closid]--;
@@ -138,7 +146,7 @@ static void limbo_release_entry(struct rmid_entry *entry)void __check_limbo (struct rdt_l3_mon_domain *d,bool force_free) {struct rdt_resource *r =resctrl_arch_get_resource (RDT_RESOURCE_L3 ); -u32 idx_limit =resctrl_arch_system_num_rmid_idx (); +u32 idx_limit =resctrl_arch_get_num_rmid_idx (r);struct rmid_entry *entry;bool rmid_dirty = true;u32 idx, cur_idx =1 ;
@@ -159,6 +167,11 @@ void __check_limbo(struct rdt_l3_mon_domain *d, bool force_free)* are marked as busy for occupancy < threshold. If the occupancy * is less than the threshold decrement the busy counter of the * RMID and move it to the free list when the counter reaches 0. +* +* RMIDs will keep counts of allocated LLC entries after the resctrl +* file system is unmounted. So check all possible RMIDs since a +* previous mount cycle may have used more than are available in +* this mount cycle. */ for (;;) { idx =find_next_bit (d->rmid_busy_llc, idx_limit, cur_idx);
@@ -202,7 +215,8 @@ void __check_limbo(struct rdt_l3_mon_domain *d, bool force_free)bool has_busy_rmid (struct rdt_l3_mon_domain *d) { -u32 idx_limit =resctrl_arch_system_num_rmid_idx (); +struct rdt_resource *r =resctrl_arch_get_resource (RDT_RESOURCE_L3 ); +u32 idx_limit =resctrl_arch_get_num_rmid_idx (r);return find_first_bit (d->rmid_busy_llc, idx_limit) != idx_limit; }
@@ -1240,7 +1254,7 @@ static void mbm_cntr_free_all(struct rdt_resource *r, struct rdt_l3_mon_domain **/ static void resctrl_reset_rmid_all (struct rdt_resource *r,struct rdt_l3_mon_domain *d) { -u32 idx_limit =resctrl_arch_system_num_rmid_idx (); +u32 idx_limit =resctrl_arch_get_num_rmid_idx (r);enum resctrl_event_id evt;int idx;
fs/resctrl/rdtgroup.c +1 -1
@@ -4619,7 +4619,7 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *h*/ static int domain_setup_l3_mon_state (struct rdt_resource *r,struct rdt_l3_mon_domain *d) { -u32 idx_limit =resctrl_arch_system_num_rmid_idx (); +u32 idx_limit =resctrl_arch_get_num_rmid_idx (r);size_t tsize =sizeof (*d->mbm_states[0 ]);enum resctrl_event_id eventid;int idx;
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 14/24 x86,fs/resctrl: Handle systems where AET is the only monitoring feature
In reply to Tony LuckApplication Energy Telemetry (AET) is not enumerated via CPUID. Its presence is discovered on the first mount of the resctrl file system by querying the pmt_telemetry driver. resctrl is initialized at boot only if CPUID enumeration finds allocation or monitoring features.
On a system that supports monitoring (X86_FEATURE_CQM) but no L3 monitoring features, get_rdt_mon_resources() reports that no monitoring is supported. If the system also does not support allocation, resctrl is not initialized and AET can never be used.
Assume that AET may be supported whenever the system supports monitoring.
If AET is not supported either, the mount finds neither allocation nor monitoring support. rdt_get_tree() then lets the mount succeed with nothing to allocate or monitor, but does not mark resctrl as mounted. Fail the mount with an error message instead, before any of the file system is set up.
---
v14:
New commit message.
--- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/core.c | 22 +++++++++++++--------- arch/x86/kernel/cpu/resctrl/monitor.c | 4 +--- fs/resctrl/rdtgroup.c | 8 ++++++-- 4 files changed, 21 insertions(+), 15 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +1 -1
@@ -226,7 +226,7 @@ union l3_qos_abmc_cfg {void rdt_ctrl_update (void *arg); -int rdt_get_l3_mon_config (struct rdt_resource *r); +void rdt_get_l3_mon_config (struct rdt_resource *r);bool rdt_cpu_has (int flag);
arch/x86/kernel/cpu/resctrl/core.c +13 -9
@@ -991,7 +991,7 @@ static __init bool get_rdt_alloc_resources(void)static __init bool get_rdt_mon_resources (void ) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; -bool ret = false; +bool l3_mon_found = false;/* Any monitoring at all? */ if (!cpu_feature_enabled (X86_FEATURE_CQM ))
@@ -1001,27 +1001,31 @@ static __init bool get_rdt_mon_resources(void)/* Any of the L3 monitoring features? */ if (!cpu_feature_enabled (X86_FEATURE_CQM_LLC )) -return false; + goto out;if (rdt_cpu_has (X86_FEATURE_CQM_OCCUP_LLC )) {resctrl_enable_mon_event (QOS_L3_OCCUP_EVENT_ID , false,0 ,NULL ); - ret = true; + l3_mon_found = true; }if (rdt_cpu_has (X86_FEATURE_CQM_MBM_TOTAL )) {resctrl_enable_mon_event (QOS_L3_MBM_TOTAL_EVENT_ID , false,0 ,NULL ); - ret = true; + l3_mon_found = true; }if (rdt_cpu_has (X86_FEATURE_CQM_MBM_LOCAL )) {resctrl_enable_mon_event (QOS_L3_MBM_LOCAL_EVENT_ID , false,0 ,NULL ); - ret = true; + l3_mon_found = true; }if (rdt_cpu_has (X86_FEATURE_ABMC )) - ret = true; + l3_mon_found = true; -if (!ret) -return false; +if (l3_mon_found) +rdt_get_l3_mon_config (r); +else +pr_info ("CPUID reports L3 monitoring support, but no L3 monitoring features found\n" ); -return !rdt_get_l3_mon_config (r); +out: +/* Non-L3 features, e.g. AET, may be supported */ +return true; }static __init void __check_quirks_intel (void )
arch/x86/kernel/cpu/resctrl/monitor.c +1 -3
@@ -431,7 +431,7 @@ static void __init intel_rdt_mbm_apply_quirk(u32 num_rmid) mbm_cf = mbm_cf_table[cf_index].cf; } -int __init rdt_get_l3_mon_config (struct rdt_resource *r) +void __init rdt_get_l3_mon_config (struct rdt_resource *r) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r);unsigned int mbm_offset;
@@ -519,8 +519,6 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)intel_rdt_mbm_apply_quirk (num_rmid); r->mon_capable = true; - -return 0 ; }static void resctrl_abmc_set_one_amd (void *arg)
fs/resctrl/rdtgroup.c +6 -2
@@ -3193,6 +3193,11 @@ static int rdt_get_tree(struct fs_context *fc) goto out; } +if (!resctrl_alloc_capable () && !resctrl_mon_capable ()) { + ret =invalfc (fc,"No allocation or monitoring features are available or enabled" ); + goto out; + } + ret =setup_rmid_lru_list ();if (ret) goto out;
@@ -3251,8 +3256,7 @@ static int rdt_get_tree(struct fs_context *fc)if (resctrl_mon_capable ())resctrl_arch_enable_mon (); -if (resctrl_alloc_capable () ||resctrl_mon_capable ()) - resctrl_mounted = true; + resctrl_mounted = true;if (resctrl_is_mbm_enabled ()) { r =resctrl_arch_get_resource (RDT_RESOURCE_L3 );
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 14/24 x86,fs/resctrl: Handle systems where AET is the only monitoring feature
Application Energy Telemetry (AET) is not enumerated via CPUID. Its presence is discovered on the first mount of the resctrl file system by querying the pmt_telemetry driver. resctrl is initialized at boot only if CPUID enumeration finds allocation or monitoring features.
On a system that supports monitoring (X86_FEATURE_CQM) but no L3 monitoring features, get_rdt_mon_resources() reports that no monitoring is supported. If the system also does not support allocation, resctrl is not initialized and AET can never be used.
Assume that AET may be supported whenever the system supports monitoring, but check for non-zero supported RMIDs to deal sanely with misconfigured systems.
If AET is not supported either, the mount finds neither allocation nor monitoring support. rdt_get_tree() then lets the mount succeed with nothing to allocate or monitor, but does not mark resctrl as mounted. Fail the mount with an error message instead, before any of the file system is set up.
---
v14+
Add check that there are some RMIDs to address Sashiko report about booting with "clearcpuid=cqm".
arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/core.c | 22 +++++++++++++--------- arch/x86/kernel/cpu/resctrl/intel_aet.c | 3 +++ arch/x86/kernel/cpu/resctrl/monitor.c | 4 +--- fs/resctrl/rdtgroup.c | 8 ++++++-- 5 files changed, 24 insertions(+), 15 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +1 -1
@@ -226,7 +226,7 @@ union l3_qos_abmc_cfg {void rdt_ctrl_update (void *arg); -int rdt_get_l3_mon_config (struct rdt_resource *r); +void rdt_get_l3_mon_config (struct rdt_resource *r);bool rdt_cpu_has (int flag);
arch/x86/kernel/cpu/resctrl/core.c +13 -9
@@ -991,7 +991,7 @@ static __init bool get_rdt_alloc_resources(void)static __init bool get_rdt_mon_resources (void ) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; -bool ret = false; +bool l3_mon_found = false;/* Any monitoring at all? */ if (!cpu_feature_enabled (X86_FEATURE_CQM ))
@@ -1001,27 +1001,31 @@ static __init bool get_rdt_mon_resources(void)/* Any of the L3 monitoring features? */ if (!cpu_feature_enabled (X86_FEATURE_CQM_LLC )) -return false; + goto out;if (rdt_cpu_has (X86_FEATURE_CQM_OCCUP_LLC )) {resctrl_enable_mon_event (QOS_L3_OCCUP_EVENT_ID , false,0 ,NULL ); - ret = true; + l3_mon_found = true; }if (rdt_cpu_has (X86_FEATURE_CQM_MBM_TOTAL )) {resctrl_enable_mon_event (QOS_L3_MBM_TOTAL_EVENT_ID , false,0 ,NULL ); - ret = true; + l3_mon_found = true; }if (rdt_cpu_has (X86_FEATURE_CQM_MBM_LOCAL )) {resctrl_enable_mon_event (QOS_L3_MBM_LOCAL_EVENT_ID , false,0 ,NULL ); - ret = true; + l3_mon_found = true; }if (rdt_cpu_has (X86_FEATURE_ABMC )) - ret = true; + l3_mon_found = true; -if (!ret) -return false; +if (l3_mon_found) +rdt_get_l3_mon_config (r); +else +pr_info ("CPUID reports L3 monitoring support, but no L3 monitoring features found\n" ); -return !rdt_get_l3_mon_config (r); +out: +/* Non-L3 features, e.g. AET, may be supported */ +return true; }static __init void __check_quirks_intel (void )
arch/x86/kernel/cpu/resctrl/intel_aet.c +3 -0
@@ -343,6 +343,9 @@ bool intel_aet_get_events(void)struct event_group **peg;bool ret = false; +if (!resctrl_arch_system_max_rmid_idx ()) +return false; +for_each_event_group (peg) { pfid =lookup_pfid ((*peg)->pfname); p =intel_pmt_get_regions_by_feature (pfid);
arch/x86/kernel/cpu/resctrl/monitor.c +1 -3
@@ -431,7 +431,7 @@ static void __init intel_rdt_mbm_apply_quirk(u32 num_rmid) mbm_cf = mbm_cf_table[cf_index].cf; } -int __init rdt_get_l3_mon_config (struct rdt_resource *r) +void __init rdt_get_l3_mon_config (struct rdt_resource *r) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r);unsigned int mbm_offset;
@@ -519,8 +519,6 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)intel_rdt_mbm_apply_quirk (num_rmid); r->mon_capable = true; - -return 0 ; }static void resctrl_abmc_set_one_amd (void *arg)
fs/resctrl/rdtgroup.c +6 -2
@@ -3193,6 +3193,11 @@ static int rdt_get_tree(struct fs_context *fc) goto out; } +if (!resctrl_alloc_capable () && !resctrl_mon_capable ()) { + ret =invalfc (fc,"No allocation or monitoring features are available or enabled" ); + goto out; + } + ret =setup_rmid_lru_list ();if (ret) goto out;
@@ -3251,8 +3256,7 @@ static int rdt_get_tree(struct fs_context *fc)if (resctrl_mon_capable ())resctrl_arch_enable_mon (); -if (resctrl_alloc_capable () ||resctrl_mon_capable ()) - resctrl_mounted = true; + resctrl_mounted = true;if (resctrl_is_mbm_enabled ()) { r =resctrl_arch_get_resource (RDT_RESOURCE_L3 );
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 15/24 x86/resctrl: Add PMT registration API for AET enumeration callbacks
In reply to Tony Luckresctrl is always built in and calls into the pmt_telemetry driver to enumerate Application Energy Telemetry (AET) features. Built-in code cannot call directly into a module, so AET support requires CONFIG_INTEL_PMT_TELEMETRY=y.
In preparation for allowing pmt_telemetry to be built as a module, add and export functions with which pmt_telemetry registers its enumeration callbacks when it is loaded and unregisters them when it is unloaded. Record the registering module so that resctrl can hold a reference on it while it uses the callbacks.
---
v14:
New commit message.
--- include/linux/intel_vsec.h | 12 ++++++++++++ arch/x86/kernel/cpu/resctrl/intel_aet.c | 24 ++++++++++++++++++++++++ 2 files changed, 36 insertions(+)
include/linux/intel_vsec.h +12 -0
@@ -257,4 +257,16 @@ static inline void intel_pmt_put_feature_group(struct pmt_feature_group *feature_group) {}#endif +#ifdef CONFIG_X86_CPU_RESCTRL_INTEL_AET +void intel_aet_register_enumeration (struct module *module, +struct pmt_feature_group *(*get)(enum pmt_feature_id id), +void (*put)(struct pmt_feature_group *p)); +void intel_aet_unregister_enumeration (void ); +#else +static inline void intel_aet_register_enumeration (struct module *module, +struct pmt_feature_group *(*get)(enum pmt_feature_id id), +void (*put)(struct pmt_feature_group *p)) { } +static inline void intel_aet_unregister_enumeration (void ) { } +#endif /* CONFIG_X86_CPU_RESCTRL_INTEL_AET */ +#endif
arch/x86/kernel/cpu/resctrl/intel_aet.c +24 -0
@@ -17,12 +17,14 @@#include <linux/cpumask.h> #include <linux/err.h> #include <linux/errno.h> +#include <linux/export.h> #include <linux/gfp_types.h> #include <linux/init.h> #include <linux/intel_pmt_features.h> #include <linux/intel_vsec.h> #include <linux/io.h> #include <linux/minmax.h> +#include <linux/module.h> #include <linux/printk.h> #include <linux/rculist.h> #include <linux/rcupdate.h>
@@ -325,6 +327,10 @@ static enum pmt_feature_id lookup_pfid(const char *pfname)return FEATURE_INVALID ; } +static struct module *pmt_module; +static struct pmt_feature_group *(*get_feature)(enum pmt_feature_id id); +static void (*put_feature)(struct pmt_feature_group *p); +/* * Request a copy of struct pmt_feature_group for each event group. If there is * one, the returned structure has an array of telemetry_region structures,
@@ -359,6 +365,24 @@ bool intel_aet_get_events(void)return ret; } +void intel_aet_register_enumeration (struct module *module, +struct pmt_feature_group *(*get)(enum pmt_feature_id id), +void (*put)(struct pmt_feature_group *p)) +{ + get_feature = get; + put_feature = put; + pmt_module = module; +} +EXPORT_SYMBOL_NS_GPL (intel_aet_register_enumeration,"INTEL_PMT" ); + +void intel_aet_unregister_enumeration (void ) +{ + pmt_module =NULL ; + get_feature =NULL ; + put_feature =NULL ; +} +EXPORT_SYMBOL_NS_GPL (intel_aet_unregister_enumeration,"INTEL_PMT" ); + void __exit intel_aet_exit(void) {struct event_group **peg;
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 16/24 platform/x86/intel/pmt: Register enumeration functions with resctrl
In reply to Tony Luckresctrl enumerates Application Energy Telemetry (AET) features by calling pmt_telemetry functions directly, which requires pmt_telemetry to be built in. resctrl now provides functions with which pmt_telemetry can register those enumeration functions instead.
Register the enumeration functions with resctrl when pmt_telemetry is loaded, in preparation for resctrl calling them through the registration and for pmt_telemetry being built as a module.
Register only after the driver has probed all telemetry devices so that resctrl does not enumerate a partial set. Force synchronous probing to avoid issues if kernel command line or module parameter requests asynchronous operation.
Unregister the enumeration functions before tearing down the telemetry devices on unload, so that resctrl cannot start enumeration while they are being removed.
---
v14:
New commit message.
--- drivers/platform/x86/intel/pmt/telemetry.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/platform/x86/intel/pmt/telemetry.c +14 -1
@@ -427,16 +427,29 @@ static struct auxiliary_driver pmt_telem_aux_driver = { .id_table = pmt_telem_id_table, .remove = pmt_telem_remove, .probe = pmt_telem_probe, + .driver = { + .probe_type = PROBE_FORCE_SYNCHRONOUS, + }, };static int __init pmt_telem_init (void ) { -return auxiliary_driver_register (&pmt_telem_aux_driver); +int ret; + + ret =auxiliary_driver_register (&pmt_telem_aux_driver); +if (ret) +return ret; + +intel_aet_register_enumeration (THIS_MODULE , intel_pmt_get_regions_by_feature, + intel_pmt_put_feature_group); + +return 0 ; }module_init (pmt_telem_init); static void __exit pmt_telem_exit(void) { +intel_aet_unregister_enumeration ();auxiliary_driver_unregister (&pmt_telem_aux_driver);xa_destroy (&telem_array); }
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 17/24 x86/resctrl: Use registered function pointers for AET enumeration
In reply to Tony Luckresctrl enumerates Application Energy Telemetry (AET) features by calling pmt_telemetry functions directly. pmt_telemetry now also registers these functions with resctrl when it is loaded.
Calling them directly requires pmt_telemetry to be built in.
In preparation for pmt_telemetry being built as a loadable module, call the registered functions instead. Serialize registration and unregistration against enumeration. Take a reference on the pmt_telemetry module before enumerating, so that the functions cannot be unregistered while resctrl uses them.
AET support still requires pmt_telemetry to be built in, so it does not yet matter that the module reference is never dropped after a successful enumeration. It must be dropped on unmount before pmt_telemetry can be built as a module.
---
v14:
Updates to resolve locking problems reported by Sashiko in v13.
AET is now going to use a single mutex for both registration and in a later patch for device removal. So change the name from aet_register_lock to just aet_lock.
Reverse locking hierarchy between AET and pmt_telemetry. No longer OK to call into pmt_telemetry to enumerate features with the aet_lock held. Instead, make use of module pinning while holding aet_lock.
New commit message
--- arch/x86/kernel/cpu/resctrl/internal.h | 4 +- arch/x86/kernel/cpu/resctrl/core.c | 2 +- arch/x86/kernel/cpu/resctrl/intel_aet.c | 59 ++++++++++++++++++++++--- 3 files changed, 57 insertions(+), 8 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -2
@@ -234,15 +234,15 @@ void rdt_domain_reconfigure_cdp(struct rdt_resource *r);void resctrl_arch_mbm_cntr_assign_set_one (struct rdt_resource *r);#ifdef CONFIG_X86_CPU_RESCTRL_INTEL_AET -bool intel_aet_get_events (void ); void __exit intel_aet_exit(void); +bool intel_aet_pre_mount (void );int intel_aet_read_event (int domid,u32 rmid,void *arch_priv,u64 *val);void intel_aet_mon_domain_setup (int cpu,int id,struct rdt_resource *r,struct list_head *add_pos);bool intel_handle_aet_option (bool force_off,char *tok);#else -static inline bool intel_aet_get_events (void ) {return false; } static inline void __exit intel_aet_exit(void) { } +static inline bool intel_aet_pre_mount (void ) {return false; }static inline int intel_aet_read_event (int domid,u32 rmid,void *arch_priv,u64 *val) {return -EINVAL ;
arch/x86/kernel/cpu/resctrl/core.c +1 -1
@@ -795,7 +795,7 @@ void resctrl_arch_pre_mount(void)struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_PERF_PKG ].r_resctrl;int cpu; -if (!intel_aet_get_events ()) +if (!intel_aet_pre_mount ())return ;/*
arch/x86/kernel/cpu/resctrl/intel_aet.c +54 -5
118 lines not on this page · raw message
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 18/24 arm,x86,fs/resctrl: Enumerate AET on every resctrl mount
In reply to Tony LuckApplication Energy Telemetry (AET) is enumerated only on the first mount of the resctrl file system. A successful enumeration takes a reference on the pmt_telemetry module that is never dropped. This is harmless only because AET support requires pmt_telemetry to be built in.
In preparation for pmt_telemetry being built as a module, hold the module reference only while resctrl is mounted. resctrl must then enumerate AET on every mount and release everything obtained from pmt_telemetry on every unmount.
Call resctrl_arch_pre_mount() on every mount attempt. Add resctrl_arch_unmount() for the architecture to undo it after a failed mount or on unmount. Serialize both hooks with each other and with the check that resctrl is not already mounted, so that enumeration only happens while resctrl is not mounted. Use a new mutex because the hooks take cpus_read_lock(), which nests outside rdtgroup_mutex.
On x86, if AET was enumerated, unmount disables the AET events, returns the telemetry data to pmt_telemetry, drops the module reference and removes the AET monitoring domains. Cleanup at resctrl exit is no longer needed.
Do not re-enable an event group that was disabled during mount because the system supports too few RMIDs for it. The same check would disable it again on the next mount.
---
v14:
New commit comment.
aet_register_lock name changed to aet_lock.
Babu: Add commit comment about event_group::force_off
--- include/linux/resctrl.h | 10 +++++++-- arch/x86/kernel/cpu/resctrl/internal.h | 4 ++-- arch/x86/kernel/cpu/resctrl/core.c | 21 +++++++++++++++-- arch/x86/kernel/cpu/resctrl/intel_aet.c | 29 +++++++++++++++++++----- drivers/resctrl/mpam_resctrl.c | 4 ++++ fs/resctrl/rdtgroup.c | 30 ++++++++++++++++++++----- 6 files changed, 81 insertions(+), 17 deletions(-)
include/linux/resctrl.h +8 -2
@@ -611,11 +611,17 @@ void resctrl_online_cpu(unsigned int cpu);void resctrl_offline_cpu (unsigned int cpu);/* -* Architecture hook called at beginning of first file system mount attempt. -* No locks are held. +* Architecture hook called at beginning of each file system mount attempt. +* Serialized against other mount and unmount attempts. */ void resctrl_arch_pre_mount (void ); +/* +* Architecture hook called when mount fails, or on unmount. +* Serialized against other mount and unmount attempts. +*/ +void resctrl_arch_unmount (void ); +/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid * for this resource and domain.
arch/x86/kernel/cpu/resctrl/internal.h +2 -2
@@ -234,15 +234,15 @@ void rdt_domain_reconfigure_cdp(struct rdt_resource *r);void resctrl_arch_mbm_cntr_assign_set_one (struct rdt_resource *r);#ifdef CONFIG_X86_CPU_RESCTRL_INTEL_AET -void __exit intel_aet_exit(void);bool intel_aet_pre_mount (void ); +void intel_aet_unmount (void );int intel_aet_read_event (int domid,u32 rmid,void *arch_priv,u64 *val);void intel_aet_mon_domain_setup (int cpu,int id,struct rdt_resource *r,struct list_head *add_pos);bool intel_handle_aet_option (bool force_off,char *tok);#else -static inline void __exit intel_aet_exit(void) { }static inline bool intel_aet_pre_mount (void ) {return false; } +static inline void intel_aet_unmount (void ) { }static inline int intel_aet_read_event (int domid,u32 rmid,void *arch_priv,u64 *val) {return -EINVAL ;
arch/x86/kernel/cpu/resctrl/core.c +19 -2
@@ -811,6 +811,25 @@ void resctrl_arch_pre_mount(void)cpus_read_unlock (); } +void resctrl_arch_unmount (void ) +{ +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_PERF_PKG ].r_resctrl; +int cpu; + +if (!r->mon_capable) +return ; + +intel_aet_unmount (); + +cpus_read_lock (); +mutex_lock (&domain_list_lock); +for_each_online_cpu (cpu) +domain_remove_cpu_mon (cpu, r); + r->mon_capable = false; +mutex_unlock (&domain_list_lock); +cpus_read_unlock (); +} +enum {RDT_FLAG_CMT ,RDT_FLAG_MBM_TOTAL ,
@@ -1161,8 +1180,6 @@ late_initcall(resctrl_arch_late_init); static void __exit resctrl_arch_exit(void) { -intel_aet_exit (); -cpuhp_remove_state (rdt_online);resctrl_exit ();
arch/x86/kernel/cpu/resctrl/intel_aet.c +24 -5
@@ -331,7 +331,7 @@ static enum pmt_feature_id lookup_pfid(const char *pfname)/* * Serialises AET's view of pmt_telemetry: -* - pmt_module, get_feature, put_feature +* - pmt_module, get_feature, put_feature, pmt_in_use * - every event_group's ->pfg * * Lock ordering with pmt/telemetry.c's ep_lock is ep_lock -> aet_lock.
@@ -345,6 +345,11 @@ static struct module *pmt_module;static struct pmt_feature_group *(*get_feature)(enum pmt_feature_id id);static void (*put_feature)(struct pmt_feature_group *p); +/* +* Track whether pmt_telemetry enumeration succeeded during mount for use during unmount. +*/ +static bool pmt_in_use; +/* * Request a copy of struct pmt_feature_group for each event group. If there is * one, the returned structure has an array of telemetry_region structures,
@@ -429,19 +434,33 @@ bool intel_aet_pre_mount(void)return false; } + pmt_in_use = true; +return true; } -void __exit intel_aet_exit(void) +void intel_aet_unmount (void ) { +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_PERF_PKG ].r_resctrl;struct event_group **peg; +guard (mutex)(&aet_lock); +if (!pmt_in_use) +return ; +for_each_event_group (peg) { -if ((*peg)->pfg) { -put_feature ((*peg)->pfg); - (*peg)->pfg =NULL ; +struct event_group *e = *peg; + +if (e->pfg) { +for (int i =0 ; i < e->num_events; i++) +resctrl_disable_mon_event (e->evts[i].id); +put_feature (e->pfg); + e->pfg =NULL ; } } +module_put (pmt_module); + pmt_in_use = false; + r->mon.num_rmid =0 ; }#define DATA_VALID BIT_ULL(63)
drivers/resctrl/mpam_resctrl.c +4 -0
@@ -121,6 +121,10 @@ void resctrl_arch_pre_mount(void) { } +void resctrl_arch_unmount (void ) +{ +} +bool resctrl_arch_get_cdp_enabled (enum resctrl_res_level rid) {return mpam_resctrl_controls[rid].cdp_enabled;
fs/resctrl/rdtgroup.c +24 -6
@@ -30,6 +30,9 @@#include "internal.h" +/* Mutex protecting resctrl_mounted and mount/unmount operations */ +static DEFINE_MUTEX (resctrl_mount_lock); +/* Mutex to protect rdtgroup access. */ DEFINE_MUTEX (rdtgroup_mutex);
@@ -48,7 +51,10 @@ LIST_HEAD(resctrl_schema_all);*/ static LIST_HEAD (mon_data_kn_priv_list); -/* The filesystem can only be mounted once. */ +/* +* The filesystem can only be mounted once. Can only be updated +* while holding both resctrl_mount_lock and rdtgroup_mutex. +*/ bool resctrl_mounted;/* Kernel fs node for "info" directory under root */
@@ -3147,6 +3153,7 @@ static void resctrl_unmount(void) {struct rdt_resource *r; +mutex_lock (&resctrl_mount_lock);cpus_read_lock ();mutex_lock (&rdtgroup_mutex);
@@ -3164,6 +3171,8 @@ static void resctrl_unmount(void) resctrl_mounted = false;mutex_unlock (&rdtgroup_mutex);cpus_read_unlock (); +resctrl_arch_unmount (); +mutex_unlock (&resctrl_mount_lock); }static int rdt_get_tree (struct fs_context *fc)
@@ -3175,24 +3184,27 @@ static int rdt_get_tree(struct fs_context *fc)struct rdt_resource *r;int ret; -DO_ONCE_SLEEPABLE (resctrl_arch_pre_mount); +mutex_lock (&resctrl_mount_lock); -cpus_read_lock (); -mutex_lock (&rdtgroup_mutex);/* * resctrl file system can only be mounted once. */ if (resctrl_mounted) { ret = -EBUSY ; - goto out; + goto out_mount_unlock; }/* Avoid races from pending operations from a previous mount */ if (atomic_read (&rdtgroup_default.waitcount) !=0 ) { ret = -EBUSY ; - goto out; + goto out_mount_unlock; } +resctrl_arch_pre_mount (); + +cpus_read_lock (); +mutex_lock (&rdtgroup_mutex); +if (!resctrl_alloc_capable () && !resctrl_mon_capable ()) { ret =invalfc (fc,"No allocation or monitoring features are available or enabled" ); goto out;
@@ -3287,6 +3299,8 @@ static int rdt_get_tree(struct fs_context *fc)mutex_unlock (&rdtgroup_mutex);cpus_read_unlock (); +mutex_unlock (&resctrl_mount_lock); + ret =kernfs_get_tree (fc);/* * resctrl can only be mounted once, new superblock only expected
@@ -3318,6 +3332,10 @@ static int rdt_get_tree(struct fs_context *fc) out:mutex_unlock (&rdtgroup_mutex);cpus_read_unlock (); +resctrl_arch_unmount (); +out_mount_unlock: +mutex_unlock (&resctrl_mount_lock); +return ret; }
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 19/24 x86/resctrl: Enforce system RMID limit on AET
In reply to Tony LuckThe RMID a CPU uses for telemetry is the RMID written to MSR_IA32_PQR_ASSOC.RMID, so the number of RMIDs that register can hold (enumerated via CPUID) bounds every monitoring resource. Each AET (Application Energy Telemetry) event group separately describes how many RMIDs its counters support.
The number of RMIDs an event group supports may be higher than the number that can be written to MSR_IA32_PQR_ASSOC.RMID. The telemetry resource then advertises more RMIDs than can ever be used, both to user space in info/PERF_PKG_MON/num_rmids and, on systems where telemetry is the only monitoring resource, as the number of RMIDs available to the mount.
Cap the number of RMIDs of the telemetry resource at the number that can be written to MSR_IA32_PQR_ASSOC.RMID.
---
v14:
New commit comment
--- arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ arch/x86/kernel/cpu/resctrl/core.c | 2 +- arch/x86/kernel/cpu/resctrl/intel_aet.c | 6 ++++++ 3 files changed, 9 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
@@ -153,6 +153,8 @@ extern struct rdt_hw_resource rdt_resources_all[];extern bool intel_rdt_mbm_need_quirk; +extern u32 pqr_assoc_num_rmid; +void arch_mon_domain_online (struct rdt_resource *r,struct rdt_l3_mon_domain *d);/* CPUID.(EAX=10H, ECX=ResID=1).EAX */
arch/x86/kernel/cpu/resctrl/core.c +1 -1
@@ -46,7 +46,7 @@ static DEFINE_MUTEX(domain_list_lock); DEFINE_PER_CPU(struct resctrl_pqr_state, pqr_state);/* Number of unique RMID values that can be written to MSR_IA32_PQR_ASSOC.RMID */ -static u32 pqr_assoc_num_rmid; +u32 pqr_assoc_num_rmid;static void mba_wrmsr_intel (struct msr_param *m);static void cat_wrmsr (struct msr_param *m);
arch/x86/kernel/cpu/resctrl/intel_aet.c +6 -0
@@ -308,6 +308,12 @@ static bool enable_events(struct event_group *e, struct pmt_feature_group *p)else r->mon.num_rmid = e->num_rmid; +/* +* AET may claim to support more RMIDs than are permitted to be +* assigned to MSR_IA32_PQR_ASSOC.RMID. Enforce this limit. +*/ + r->mon.num_rmid =min (r->mon.num_rmid, pqr_assoc_num_rmid); +if (skipped_events)pr_info ("%s %s:0x%x monitoring detected (skipped %d events)\n" , r->name, e->pfname, e->guid, skipped_events);
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 20/24 x86/resctrl: Export interface to report telemetry unbind/remove
In reply to Tony Luckresctrl obtains the addresses of the telemetry MMIO regions from the pmt_telemetry driver at mount time and holds a reference on the module until unmount, so the module cannot be unloaded while resctrl may read the counters.
The module reference does not prevent a telemetry device from being unbound from the driver, or removed. Either tears down the MMIO mappings that hold the event counters, and a subsequent read of an event by resctrl faults on the stale address.
Provide an interface for pmt_telemetry to report that the mappings for a package are about to be torn down. Stop using the regions of that package and serialize event reads with the notification so that no read uses an address once it has been reported. Events of the affected package then read "Unavailable".
---
v14:
Dropped aet_mmio_lock. Now safe to use aet_lock. New commit message
Add code comment noting that per-package may be over-reaction.
--- include/linux/intel_vsec.h | 2 ++ arch/x86/kernel/cpu/resctrl/intel_aet.c | 30 +++++++++++++++++++++++++ 2 files changed, 32 insertions(+)
include/linux/intel_vsec.h +2 -0
@@ -262,11 +262,13 @@ void intel_aet_register_enumeration(struct module *module,struct pmt_feature_group *(*get)(enum pmt_feature_id id), void (*put)(struct pmt_feature_group *p));void intel_aet_unregister_enumeration (void ); +void intel_aet_invalidate (u8 package_id);#else static inline void intel_aet_register_enumeration (struct module *module,struct pmt_feature_group *(*get)(enum pmt_feature_id id),void (*put)(struct pmt_feature_group *p)) { }static inline void intel_aet_unregister_enumeration (void ) { } +static inline void intel_aet_invalidate (u8 package_id) { }#endif /* CONFIG_X86_CPU_RESCTRL_INTEL_AET */ #endif
arch/x86/kernel/cpu/resctrl/intel_aet.c +30 -0
@@ -339,6 +339,7 @@ static enum pmt_feature_id lookup_pfid(const char *pfname)* Serialises AET's view of pmt_telemetry: * - pmt_module, get_feature, put_feature, pmt_in_use * - every event_group's ->pfg +* - the telemetry_region::addr entries reachable through ->pfg * * Lock ordering with pmt/telemetry.c's ep_lock is ep_lock -> aet_lock. * AET therefore must not call get_feature() (which takes ep_lock) while
@@ -413,6 +414,33 @@ void intel_aet_unregister_enumeration(void) }EXPORT_SYMBOL_NS_GPL (intel_aet_unregister_enumeration,"INTEL_PMT" ); +/* +* pmt_telemetry driver calls while holding ep_lock. This is for unbind/remove +* operations that will invalidate the virtual addresses of MMIO registers +* provided by intel_pmt_get_regions_by_feature(). This simple interface may +* be overly broad if there are multiple telemetry devices per-package. An +* unbind/remove operation on one device will stop resctrl from reading +* event counters from all devices on the same package. +*/ +void intel_aet_invalidate (u8 package_id) +{ +struct event_group **peg; + +guard (mutex)(&aet_lock); + +for_each_event_group (peg) { +struct event_group *e = *peg; + +if (!e->pfg) +continue ; +for (int i =0 ; i < e->pfg->count; i++) { +if (e->pfg->regions[i].plat_info.package_id == package_id) + e->pfg->regions[i].addr =NULL ; + } + } +} +EXPORT_SYMBOL_NS_GPL (intel_aet_invalidate,"INTEL_PMT" ); +bool intel_aet_pre_mount (void ) {struct pmt_feature_group *(*get)(enum pmt_feature_id id);
@@ -488,6 +516,8 @@ int intel_aet_read_event(int domid, u32 rmid, void *arch_priv, u64 *val)void *pevt0;u32 idx; +guard (mutex)(&aet_lock); + pevt0 = pevt - pevt->idx; e = container_of(pevt0, struct event_group, evts); idx = rmid * e->num_events;
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 20/24 x86/resctrl: Export interface to report telemetry unbind/remove
resctrl obtains the addresses of the telemetry MMIO regions from the pmt_telemetry driver at mount time and holds a reference on the module until unmount, so the module cannot be unloaded while resctrl may read the counters.
The module reference does not prevent a telemetry device from being unbound from the driver, or removed. Either tears down the MMIO mappings that hold the event counters, and a subsequent read of an event by resctrl faults on the stale address.
Provide an interface for pmt_telemetry to report that the mappings for a package are about to be torn down. Stop using the regions of that package and serialize event reads with the notification so that no read uses an address once it has been reported. Events of the affected package then read "Unavailable".
Update AET enumeration to handle races between enumeration and device unbind/removal. Count invalidations in a generation counter protected by aet_lock. Sample it before calling get_feature() and, under aet_lock, check that it did not change before enabling events and publishing the copy. If it changed, drop the copy and retry. The new copy reflects the removal.
---
v14+
Handle races between enumeration and device unbind/remove by adding a count of invalidations that is updated & checked while holding aet_lock.
include/linux/intel_vsec.h | 2 + arch/x86/kernel/cpu/resctrl/intel_aet.c | 87 ++++++++++++++++++++++--- 2 files changed, 80 insertions(+), 9 deletions(-)
include/linux/intel_vsec.h +2 -0
@@ -262,11 +262,13 @@ void intel_aet_register_enumeration(struct module *module,struct pmt_feature_group *(*get)(enum pmt_feature_id id), void (*put)(struct pmt_feature_group *p));void intel_aet_unregister_enumeration (void ); +void intel_aet_invalidate (u8 package_id);#else static inline void intel_aet_register_enumeration (struct module *module,struct pmt_feature_group *(*get)(enum pmt_feature_id id),void (*put)(struct pmt_feature_group *p)) { }static inline void intel_aet_unregister_enumeration (void ) { } +static inline void intel_aet_invalidate (u8 package_id) { }#endif /* CONFIG_X86_CPU_RESCTRL_INTEL_AET */ #endif
arch/x86/kernel/cpu/resctrl/intel_aet.c +78 -9
126 lines not on this page · raw message
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 21/24 platform/x86/intel/pmt: Inform resctrl when MMIO maps are being removed
In reply to Tony Luckresctrl obtains the virtual addresses of the telemetry MMIO regions from intel_pmt_get_regions_by_feature() at mount time and reads event counters from them until unmount.
When a telemetry device is unbound from the pmt_telemetry driver or removed, the driver tears down those mappings. A subsequent read of a telemetry event by resctrl faults on the stale address.
Before destroying the devices, tell resctrl that the mappings of the device's package are about to be torn down and must not be used again. Telemetry regions are organised per package, so the package ID identifies the regions resctrl has to drop. This may be overly broad if there are multiple telemetry devices on a package.
Notify resctrl while holding ep_lock so that the notification is ordered against concurrent lookups of the regions. This is safe because ep_lock is always taken before aet_lock.
---
v14:
Added commit comment on lock order.
New commit comment.
--- drivers/platform/x86/intel/pmt/telemetry.c | 30 ++++++++++++++++++++++ 1 file changed, 30 insertions(+)
drivers/platform/x86/intel/pmt/telemetry.c +30 -0
@@ -366,12 +366,42 @@ pmt_telem_find_and_register_endpoint(struct device *dev, u32 guid, u16 pos) }EXPORT_SYMBOL_NS_GPL (pmt_telem_find_and_register_endpoint,"INTEL_PMT_TELEMETRY" ); +static int pmt_telem_get_package_id (struct pmt_telem_priv *priv) +{ +int i; + +for (i =0 ; i < priv->num_entries; i++) { +struct intel_pmt_entry *entry = &priv->entry[i]; +struct pci_dev *pdev =to_pci_dev (entry->ep->dev); +struct oobmsm_plat_info *plat_info; + + plat_info =intel_vsec_get_mapping (pdev); +if (!IS_ERR (plat_info)) { +/* All entries for a device have the same package ID */ +return plat_info->package_id; + } + } + +return -ENODEV ; +} +static void pmt_telem_remove (struct auxiliary_device *auxdev) {struct pmt_telem_priv *priv =auxiliary_get_drvdata (auxdev); +int package_id;int i;mutex_lock (&ep_lock); + package_id =pmt_telem_get_package_id (priv); +/* +* Tell resctrl/AET that virtual mappings for MMIO space in a CPU +* package are about to be torn down. +*/ +if (package_id <0 ) +dev_dbg (&auxdev->dev,"Unable to determine package ID for removed device\n" ); +else +intel_aet_invalidate (package_id); +for (i =0 ; i < priv->num_entries; i++) {struct intel_pmt_entry *entry = &priv->entry[i];
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 22/24 x86/resctrl: Require 64-bit x86 for resctrl support
In reply to Tony LuckIntel Application Energy Telemetry (AET) provides event counts in 64-bit MMIO registers. For accurate results these must be read with a single 64-bit load, which is only available to 64-bit kernels, so AET support depends on X86_64.
AET is about to become an unconditional part of x86 resctrl instead of being selected by the separate CONFIG_X86_CPU_RESCTRL_INTEL_AET option. resctrl can then no longer be built for 32-bit kernels.
Make resctrl depend on X86_64, in preparation for removing CONFIG_X86_CPU_RESCTRL_INTEL_AET. This drops resctrl support from 32-bit kernels, in line with Dave Hansen's proposal [1] to document reduced support for 32-bit x86. Keeping support for 32-bit kernels is not worth it. Modern systems that support resctrl almost always have more memory than can be addressed by a 32-bit Linux kernel.
---
v14:
New commit message.
--- arch/x86/Kconfig | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/Kconfig +2 -2
@@ -518,7 +518,7 @@ config X86_MPPARSE config X86_CPU_RESCTRL bool "x86 CPU resource control support" - depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD) + depends on X86_64 && (CPU_SUP_INTEL || CPU_SUP_AMD) depends on MISC_FILESYSTEMS select ARCH_HAS_CPU_RESCTRL select RESCTRL_FS
@@ -541,7 +541,7 @@ config X86_CPU_RESCTRL config X86_CPU_RESCTRL_INTEL_AET bool "Intel Application Energy Telemetry" - depends on X86_64 && X86_CPU_RESCTRL && CPU_SUP_INTEL && INTEL_PMT_TELEMETRY=y && INTEL_TPMI=y + depends on X86_CPU_RESCTRL && CPU_SUP_INTEL && INTEL_PMT_TELEMETRY=y && INTEL_TPMI=y help Enable per-RMID telemetry events in resctrl.
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 23/24 x86/resctrl: Simplify Kconfig options for resctrl
In reply to Tony LuckApplication Energy Telemetry (AET) support in resctrl is selected by CONFIG_X86_CPU_RESCTRL_INTEL_AET, which requires the pmt_telemetry and TPMI drivers to be built in. Kernels that build those drivers as modules, as distributions commonly do, cannot enable AET, and enabling it forces the drivers to be built in.
resctrl now obtains the telemetry enumeration functions from pmt_telemetry when that module loads and enumerates AET on every mount, so the built-in requirement is no longer needed. Keeping a separate option with relaxed dependencies would still add to the Kconfig complexity that Linus Torvalds objected to [1].
Drop CONFIG_X86_CPU_RESCTRL_INTEL_AET and always build AET support as part of x86 resctrl. AET remains inactive unless the pmt_telemetry driver registers with resctrl.
With AET no longer an optional Intel-specific component, rename its x86-internal functions from the intel_ prefix to the rdt_ prefix used by the other functions in internal.h. The functions exported to pmt_telemetry keep their names.
---
v14:
New commit message.
--- include/linux/intel_vsec.h | 4 ++-- arch/x86/kernel/cpu/resctrl/internal.h | 25 ++++++------------------- arch/x86/kernel/cpu/resctrl/core.c | 8 ++++---- arch/x86/kernel/cpu/resctrl/intel_aet.c | 14 +++++++------- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- arch/x86/Kconfig | 13 ------------- arch/x86/kernel/cpu/resctrl/Makefile | 3 +-- 7 files changed, 21 insertions(+), 48 deletions(-)
include/linux/intel_vsec.h +2 -2
@@ -257,7 +257,7 @@ static inline void intel_pmt_put_feature_group(struct pmt_feature_group *feature_group) {}#endif -#ifdef CONFIG_X86_CPU_RESCTRL_INTEL_AET +#ifdef CONFIG_X86_CPU_RESCTRL void intel_aet_register_enumeration (struct module *module,struct pmt_feature_group *(*get)(enum pmt_feature_id id),void (*put)(struct pmt_feature_group *p));
@@ -269,6 +269,6 @@ static inline void intel_aet_register_enumeration(struct module *module, void (*put)(struct pmt_feature_group *p)) { }static inline void intel_aet_unregister_enumeration (void ) { }static inline void intel_aet_invalidate (u8 package_id) { } -#endif /* CONFIG_X86_CPU_RESCTRL_INTEL_AET */ +#endif /* CONFIG_X86_CPU_RESCTRL */ #endif
arch/x86/kernel/cpu/resctrl/internal.h +6 -19
@@ -235,24 +235,11 @@ bool rdt_cpu_has(int flag);void rdt_domain_reconfigure_cdp (struct rdt_resource *r);void resctrl_arch_mbm_cntr_assign_set_one (struct rdt_resource *r); -#ifdef CONFIG_X86_CPU_RESCTRL_INTEL_AET -bool intel_aet_pre_mount (void ); -void intel_aet_unmount (void ); -int intel_aet_read_event (int domid,u32 rmid,void *arch_priv,u64 *val); -void intel_aet_mon_domain_setup (int cpu,int id,struct rdt_resource *r, -struct list_head *add_pos); -bool intel_handle_aet_option (bool force_off,char *tok); -#else -static inline bool intel_aet_pre_mount (void ) {return false; } -static inline void intel_aet_unmount (void ) { } -static inline int intel_aet_read_event (int domid,u32 rmid,void *arch_priv,u64 *val) -{ -return -EINVAL ; -} - -static inline void intel_aet_mon_domain_setup (int cpu,int id,struct rdt_resource *r, -struct list_head *add_pos) { } -static inline bool intel_handle_aet_option (bool force_off,char *tok) {return false; } -#endif +bool rdt_aet_pre_mount (void ); +void rdt_aet_unmount (void ); +int rdt_aet_read_event (int domid,u32 rmid,void *arch_priv,u64 *val); +void rdt_aet_mon_domain_setup (int cpu,int id,struct rdt_resource *r, +struct list_head *add_pos); +bool rdt_handle_aet_option (bool force_off,char *tok);#endif /* _ASM_X86_RESCTRL_INTERNAL_H */
arch/x86/kernel/cpu/resctrl/core.c +4 -4
@@ -616,7 +616,7 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r)break ;case RDT_RESOURCE_PERF_PKG :if (!hdr) -intel_aet_mon_domain_setup (cpu, id, r, add_pos); +rdt_aet_mon_domain_setup (cpu, id, r, add_pos);break ;default :pr_warn_once ("Unknown resource rid=%d\n" , r->rid);
@@ -795,7 +795,7 @@ void resctrl_arch_pre_mount(void)struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_PERF_PKG ].r_resctrl;int cpu; -if (!intel_aet_pre_mount ()) +if (!rdt_aet_pre_mount ())return ;/*
@@ -819,7 +819,7 @@ void resctrl_arch_unmount(void)if (!r->mon_capable)return ; -intel_aet_unmount (); +rdt_aet_unmount ();cpus_read_lock ();mutex_lock (&domain_list_lock);
@@ -885,7 +885,7 @@ static int __init set_rdt_options(char *str) force_off = *tok =='!' ;if (force_off) tok++; -if (intel_handle_aet_option (force_off, tok)) +if (rdt_handle_aet_option (force_off, tok))continue ;for (o = rdt_options; o < &rdt_options[NUM_RDT_OPTIONS ]; o++) {if (strcmp (tok, o->name) ==0 ) {
arch/x86/kernel/cpu/resctrl/intel_aet.c +7 -7
@@ -173,7 +173,7 @@ static struct event_group *known_event_groups[] = { _peg < &known_event_groups[ARRAY_SIZE (known_event_groups)]; \ _peg++) -bool intel_handle_aet_option (bool force_off,char *tok) +bool rdt_handle_aet_option (bool force_off,char *tok) {struct event_group **peg;bool ret = false;
@@ -228,7 +228,7 @@ static bool group_has_usable_regions(struct event_group *e, struct pmt_feature_gif (skip_telem_region (&p->regions[i], e)) {/* * Clear the address field of regions that did not pass the checks in -* skip_telem_region() so they will not be used by intel_aet_read_event(). +* skip_telem_region() so they will not be used by rdt_aet_read_event(). * This is safe to do because intel_pmt_get_regions_by_feature() allocates * a new pmt_feature_group structure to return to each caller and only makes * use of the pmt_feature_group::kref field when intel_pmt_put_feature_group()
@@ -441,7 +441,7 @@ void intel_aet_invalidate(u8 package_id) }EXPORT_SYMBOL_NS_GPL (intel_aet_invalidate,"INTEL_PMT" ); -bool intel_aet_pre_mount (void ) +bool rdt_aet_pre_mount (void ) {struct pmt_feature_group *(*get)(enum pmt_feature_id id);void (*put)(struct pmt_feature_group *p);
@@ -473,7 +473,7 @@ bool intel_aet_pre_mount(void)return true; } -void intel_aet_unmount (void ) +void rdt_aet_unmount (void ) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_PERF_PKG ].r_resctrl;struct event_group **peg;
@@ -506,7 +506,7 @@ void intel_aet_unmount(void)* the MMIO read indicates that data is not valid. Return success if at * least one aggregator has valid data. */ -int intel_aet_read_event (int domid,u32 rmid,void *arch_priv,u64 *val) +int rdt_aet_read_event (int domid,u32 rmid,void *arch_priv,u64 *val) {struct pmt_event *pevt = arch_priv;struct event_group *e;
@@ -546,8 +546,8 @@ int intel_aet_read_event(int domid, u32 rmid, void *arch_priv, u64 *val)return valid ?0 : -EINVAL ; } -void intel_aet_mon_domain_setup (int cpu,int id,struct rdt_resource *r, -struct list_head *add_pos) +void rdt_aet_mon_domain_setup (int cpu,int id,struct rdt_resource *r, +struct list_head *add_pos) {struct rdt_perf_pkg_mon_domain *d;int err;
arch/x86/kernel/cpu/resctrl/monitor.c +1 -1
@@ -255,7 +255,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,resctrl_arch_rmid_read_context_check ();if (r->rid ==RDT_RESOURCE_PERF_PKG ) -return intel_aet_read_event (hdr->id, rmid, arch_priv, val); +return rdt_aet_read_event (hdr->id, rmid, arch_priv, val);if (!domain_header_is_valid (hdr,RESCTRL_MON_DOMAIN ,RDT_RESOURCE_L3 ))return -EINVAL ;
arch/x86/Kconfig +0 -13
@@ -539,19 +539,6 @@ config X86_CPU_RESCTRL Say N if unsure. -config X86_CPU_RESCTRL_INTEL_AET - bool "Intel Application Energy Telemetry" - depends on X86_CPU_RESCTRL && CPU_SUP_INTEL && INTEL_PMT_TELEMETRY=y && INTEL_TPMI=y - help - Enable per-RMID telemetry events in resctrl. - - Intel feature that collects per-RMID execution data - about energy consumption, measure of frequency independent - activity and other performance metrics. Data is aggregated - per package. - - Say N if unsure. - config X86_FRED bool "Flexible Return and Event Delivery" depends on X86_64
arch/x86/kernel/cpu/resctrl/Makefile +1 -2
@@ -1,7 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_X86_CPU_RESCTRL) += core.o rdtgroup.o monitor.o -obj-$(CONFIG_X86_CPU_RESCTRL) += ctrlmondata.o -obj-$(CONFIG_X86_CPU_RESCTRL_INTEL_AET) += intel_aet.o +obj-$(CONFIG_X86_CPU_RESCTRL) += ctrlmondata.o intel_aet.o obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK) += pseudo_lock.o # To allow define_trace.h's recursive include:
Signature · 1 line
2.56.0
Tony Luck
maintainerPATCH v14 24/24 x86,fs/resctrl: Document telemetry mount timing caveat
In reply to Tony Luckresctrl enumerates telemetry events each time it is mounted, using functions that the pmt_telemetry driver registers when it loads. The driver can be built as a module, which is typically loaded by udev during boot.
An automatic mount of resctrl from /etc/fstab early in boot can run before pmt_telemetry has loaded. Telemetry events are then not enabled in that mount, with no indication why.
Document the race, how to avoid it by mounting resctrl after pmt_telemetry has loaded, and that unmounting and mounting resctrl again enables the telemetry events.
---
v14:
Updated commit message. Include unmount/remount as solution if early mount did not include AET.
--- Documentation/filesystems/resctrl.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
Documentation/filesystems/resctrl.rst +7 -1
@@ -629,7 +629,7 @@ When monitoring is enabled all MON groups will also contain: each instance of an L3 cache. Each directory contains files for the enabled L3 events (e.g. "llc_occupancy", "mbm_total_bytes", and "mbm_local_bytes"). - If telemetry monitoring is enabled, there will be a "mon_PERF_PKG_YY" + If telemetry monitoring is enabled [#]_, there will be a "mon_PERF_PKG_YY" directory for each physical processor package. Each directory contains files for the enabled telemetry events (e.g. "core_energy". "activity", "uops_retired", etc.)
@@ -668,6 +668,12 @@ When monitoring is enabled all MON groups will also contain: returned if the MBM event does not have an assigned counter in the CTRL_MON group nor in any of its associated MON groups. +.. [#] Telemetry features are enumerated asynchronously by the PMT driver, so + an automatic mount of resctrl from ``/etc/fstab`` at boot may not enable + them. To avoid this race, defer the mount until after the ``pmt_telemetry`` + module has loaded. Unmounting and remounting the resctrl file system will + fix the issue. + "mon_hw_id": Available only with debug option. The identifier used by hardware for the monitor group. On x86 this is the RMID.
Signature · 1 line
2.56.0
Luck, Tony
maintainerQuoted 6 lines · Tony Luck
Requiring INTEL_PMT_TELEMETRY=y to enable AET is a functional workaround to enable enumeration of Application Energy Telemetry (AET) events, but unacceptable to many users. It results in increased configuration complexity, increased kernel memory footprint and inability to patch problems by unloading a module and loading an updated version.
Quoted 6 lines
Add a registration function to the AET code that can be used by
INTEL_PMT_TELEMETRYto provide the enumeration functions.
INTEL_PMT_TELEMETRYcan be loaded/unloaded independently of resctrl file system mount/unmount. Perform enumeration on every mount and cleanup on every unmount.
Summary of Sashiko review: https://sashiko.dev/#/patchset/20261019205031.16977-1-tony.luck%40intel.com
Only issues in patches 12, 19, 20, 21
* Patch 12: [PATCH v14 12/24] arm,x86,fs/resctrl: Allocate maximum needed rmid_ptrs[]
Looks like a legitimate concern for a somewhat contrived problem. Users who use kernel command line "clearcpuid" option are always at risk of running into problems on the invalid h/w configuration they create.
If others think this is worth fixing, I can add additional checks to intel_aet_get_events().
* Patch 19: [PATCH v14 19/24] x86/resctrl: Enforce system RMID limit on AET
Same complaint as patch 12.
* Patch 20: [PATCH v14 20/24] x86/resctrl: Export interface to report telemetry unbind/remove
Looks like a legitimate report of a race between unbinding/removing devices in parallel with mounting resctrl. Should I care about this scenario?
* Patch 21: [PATCH v14 21/24] platform/x86/intel/pmt: Inform resctrl when MMIO maps are being removed
1) Sashiko (still) worried about non-PCI telemetry devices that don't exist.
2) Same device unbind/remove race with mount reported in patch 20.
-Tony