Skip to contentSkip to conversation

x86/resctrl: Refactor resctrl enumeration

RFC PATCH In review Tony Luck 3 participants 11 messages Aug 19 – Aug 21

Subsystems (3)
lore
Share & tools
lore

Tony Luck

author maintainer
#0 lore raw

cover x86/resctrl: Refactor resctrl enumeration

This series is posted as RFC to get some feedback before I incorporate it into the next version of my patches to fix Kconfig issues[1]. It was prompted by Reinette's comment[2]:

"It looks to me that moving monitoring enumeration done in resctrl_cpu_detect()
into resctrl proper where the RMID range checks can be centralized on features gated by rdt_cpu_has() would be simpler and accurate?"

Patch 1 adds a missing test that RDT monitoring is enabled in any form. This test has been missing from the very start of Linux implementation. I added a Fixes: tag abd it could be backported to stable and long term kernels, but there might not be much value in doing so.

The remaining patches just relocate the enumeration from an early call on the BSP, to the appropriate point in the normal "late_init" initialization of resctrl. No functional changes intended in these patches.

Tony Luck (4):
x86/resctrl: Check if monitoring features are enabled x86/resctrl: Enumerate monitor features in rdt_get_l3_mon_config() x86/resctrl: Apply Intel MBM quirk from rdt_get_l3_mon_config() x86/resctrl: Delete resctrl_cpu_detect()

 arch/x86/include/asm/processor.h       |  4 --
 arch/x86/include/asm/resctrl.h         | 12 ++---
 arch/x86/kernel/cpu/resctrl/internal.h |  4 +-
 arch/x86/kernel/cpu/amd.c              |  3 --
 arch/x86/kernel/cpu/hygon.c            |  3 --
 arch/x86/kernel/cpu/intel.c            |  7 ---
 arch/x86/kernel/cpu/resctrl/core.c     | 47 ++----------------
 arch/x86/kernel/cpu/resctrl/monitor.c  | 67 +++++++++++++++++++-------
 8 files changed, 59 insertions(+), 88 deletions(-)

base-commit: bd5f485f3f026225b86573e559af0b7254ef4184

Signature · 1 line
2.55.0
author maintainer
#1 lore raw

RFC PATCH 1/4 x86/resctrl: Check if monitoring features are enabled

Both Intel and AMD manuals say that software must first check CPUID(0x7,0x0).EBX[12] to see if any monitoring features are enabled before checking for specific features enabled in subleaves.

Add the check for X86_FEATURE_CQM.

Fixes cbc82b172638 ("x86: Add support for Intel Cache QoS Monitoring (CQM) detection")
---
 arch/x86/kernel/cpu/resctrl/core.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kernel/cpu/resctrl/core.c +3 -0
index 55214d6fdc49..2677b8a6c15b 100644
@@ -968,6 +968,9 @@ static __init bool get_rdt_mon_resources(void)
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
 	bool ret = false;
 
+	if (!cpu_feature_enabled(X86_FEATURE_CQM))
+		return false;
+
 	if (rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC)) {
 		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID, false, 0, NULL);
 		ret = true;
Signature · 1 line
2.55.0
maintainer
#2 lore raw

Re: [RFC PATCH 1/4] x86/resctrl: Check if monitoring features are enabled

Hi Tony,

Thank you for doing this.

Quoted 17 lines · Tony Luck

Both Intel and AMD manuals say that software must first check CPUID(0x7,0x0).EBX[12] to see if any monitoring features are enabled before checking for specific features enabled in subleaves.

Add the check for X86_FEATURE_CQM.

Fixes cbc82b172638 ("x86: Add support for Intel Cache QoS Monitoring (CQM) detection")
---
 arch/x86/kernel/cpu/resctrl/core.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kernel/cpu/resctrl/core.c +0 -0
index 55214d6fdc49..2677b8a6c15b 100644

arch/x86/kernel/cpu/resctrl/core.c static __init bool get_rdt_mon_resources(void) · line 972

Quoted 5 lines
arch/x86/kernel/cpu/resctrl/core.c +2 -0
@@ -968,6 +968,9 @@ static __init bool get_rdt_mon_resources(void)
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
 	bool ret = false;
 
+	if (!cpu_feature_enabled(X86_FEATURE_CQM))
+		return false;

Is this missing a check of X86_FEATURE_CQM_LLC also?

As I understand resctrl learns from X86_FEATURE_CQM whether the system supports resource monitoring in general. Contrary to what the feature name suggests, there is another step needed to determine which resource(s) support monitoring via CPUID(0xF,0x0).EDX where bit 1 indicates LLC monitoring that needs to be set before the LLC resource-specific monitoring properties can/should be determined.

Apart from the checks here I see that cpuid_deps[] accurately reflects the relationship between X86_FEATURE_CQM_LLC and the different LLC monitoring features checked for below. I do not see cpuid_deps[] capturing the relationship between X86_FEATURE_CQM and X86_FEATURE_CQM_LLC though. Could adding it complete the handling of relationships between these leaves?

Similarly I think cpuid_deps[] may be missing X86_FEATURE_ABMC's dependency on X86_FEATURE_CQM_LLC.

arch/x86/kernel/cpu/resctrl/core.c static __init bool get_rdt_mon_resources(void) · line 976

Quoted 4 lines
arch/x86/kernel/cpu/resctrl/core.c +1 -0
+
 	if (rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC)) {
 		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID, false, 0, NULL);
 		ret = true;

Reinette

author maintainer
#3 lore raw

Re: [RFC PATCH 1/4] x86/resctrl: Check if monitoring features are enabled

Quoted 30 lines · Reinette Chatre

Hi Tony,

Thank you for doing this.

Quoted 17 lines · Tony Luck

Both Intel and AMD manuals say that software must first check CPUID(0x7,0x0).EBX[12] to see if any monitoring features are enabled before checking for specific features enabled in subleaves.

Add the check for X86_FEATURE_CQM.

Fixes cbc82b172638 ("x86: Add support for Intel Cache QoS Monitoring (CQM) detection")
---
 arch/x86/kernel/cpu/resctrl/core.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kernel/cpu/resctrl/core.c +0 -0
index 55214d6fdc49..2677b8a6c15b 100644

arch/x86/kernel/cpu/resctrl/core.c static __init bool get_rdt_mon_resources(void) · line 972

Quoted 5 lines
arch/x86/kernel/cpu/resctrl/core.c +2 -0
@@ -968,6 +968,9 @@ static __init bool get_rdt_mon_resources(void)
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
 	bool ret = false;
 
+	if (!cpu_feature_enabled(X86_FEATURE_CQM))
+		return false;

Is this missing a check of X86_FEATURE_CQM_LLC also?

Quoted 5 lines

As I understand resctrl learns from X86_FEATURE_CQM whether the system supports resource monitoring in general. Contrary to what the feature name suggests, there is another step needed to determine which resource(s) support monitoring via CPUID(0xF,0x0).EDX where bit 1 indicates LLC monitoring that needs to be set before the LLC resource-specific monitoring properties can/should be determined.

Yes. Missing that check. I agree that the feature define names are bad.

To avoid continued confusion I should rename the #defines to match the bit names in the Intel SDM (but leaving the /proc/cpuinfo visible strings at "cqm" and "cqm_llc" as those are user ABI now).

X86_FEATURE_CQM -> X86_FEATURE_RDT_M
X86_FEATURE_CQM_LLC -> X86_FEATURE_L3_MON

Quoted 6 lines

Apart from the checks here I see that cpuid_deps[] accurately reflects the relationship between X86_FEATURE_CQM_LLC and the different LLC monitoring features checked for below. I do not see cpuid_deps[] capturing the relationship between X86_FEATURE_CQM and X86_FEATURE_CQM_LLC though. Could adding it complete the handling of relationships between these leaves?

Yes.

Quoted 3 lines

Similarly I think cpuid_deps[] may be missing X86_FEATURE_ABMC's dependency on X86_FEATURE_CQM_LLC.

Maybe no? X86_FEATURE_ABMC isn't enumerated in CPUID(0xF,*). It comes from the AMD CPUID(0x80000020,0)EBX{5}

Babu: The AMD architecture programmer's manual just says:

"Support for ABMC is identified by CPUID Fn8000_0020_EBX_x0[ABMC] (bit 5)
being set. If ABMC is supported, the feature’s attributes and capabilities are enumerated by CPUID Fn8000_0020_x5 as detailed in Appendix E of APM volume 3"

It isn't explicitly stated whether this depends on CPUID Fn0000_000F_x0 EDX{1}, which AMD names: "L3CacheMon - L3 monitoring capability"

Quoted 6 lines

arch/x86/kernel/cpu/resctrl/core.c static __init bool get_rdt_mon_resources(void) · line 976

Quoted 4 lines
arch/x86/kernel/cpu/resctrl/core.c +1 -0
+
 	if (rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC)) {
 		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID, false, 0, NULL);
 		ret = true;

Reinette

-Tony

maintainer
#4 lore raw

Re: [RFC PATCH 1/4] x86/resctrl: Check if monitoring features are enabled

Hi Tony,

Quoted 37 lines · Tony
Quoted 30 lines · Reinette Chatre

Hi Tony,

Thank you for doing this.

Quoted 17 lines · Tony Luck

Both Intel and AMD manuals say that software must first check CPUID(0x7,0x0).EBX[12] to see if any monitoring features are enabled before checking for specific features enabled in subleaves.

Add the check for X86_FEATURE_CQM.

Fixes cbc82b172638 ("x86: Add support for Intel Cache QoS Monitoring (CQM) detection")
---
 arch/x86/kernel/cpu/resctrl/core.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kernel/cpu/resctrl/core.c +0 -0
index 55214d6fdc49..2677b8a6c15b 100644

arch/x86/kernel/cpu/resctrl/core.c static __init bool get_rdt_mon_resources(void) · line 972

Quoted 5 lines
arch/x86/kernel/cpu/resctrl/core.c +2 -0
@@ -968,6 +968,9 @@ static __init bool get_rdt_mon_resources(void)
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
 	bool ret = false;
 
+	if (!cpu_feature_enabled(X86_FEATURE_CQM))
+		return false;

Is this missing a check of X86_FEATURE_CQM_LLC also?

Quoted 5 lines

As I understand resctrl learns from X86_FEATURE_CQM whether the system supports resource monitoring in general. Contrary to what the feature name suggests, there is another step needed to determine which resource(s) support monitoring via CPUID(0xF,0x0).EDX where bit 1 indicates LLC monitoring that needs to be set before the LLC resource-specific monitoring properties can/should be determined.

Quoted 5 lines

Yes. Missing that check. I agree that the feature define names are bad.

To avoid continued confusion I should rename the #defines to match the bit names in the Intel SDM (but leaving the /proc/cpuinfo visible strings at "cqm" and "cqm_llc" as those are user ABI now).

Naming is as always complicated and here I do not see why the name should be picked from Intel SDM instead of AMD's spec. Renaming may be secondary goal. It may even add to confusion to have feature name mismatch what is exposed to user space? A change like this could perhaps be punted to when/if monitoring of a new resource needs to be supported?

Quoted 19 lines

X86_FEATURE_CQM -> X86_FEATURE_RDT_M
X86_FEATURE_CQM_LLC -> X86_FEATURE_L3_MON

Quoted 6 lines

Apart from the checks here I see that cpuid_deps[] accurately reflects the relationship between X86_FEATURE_CQM_LLC and the different LLC monitoring features checked for below. I do not see cpuid_deps[] capturing the relationship between X86_FEATURE_CQM and X86_FEATURE_CQM_LLC though. Could adding it complete the handling of relationships between these leaves?

Yes.

Quoted 3 lines

Similarly I think cpuid_deps[] may be missing X86_FEATURE_ABMC's dependency on X86_FEATURE_CQM_LLC.

Maybe no? X86_FEATURE_ABMC isn't enumerated in CPUID(0xF,*). It comes from the AMD CPUID(0x80000020,0)EBX{5}

Quoted 8 lines

Babu: The AMD architecture programmer's manual just says:

"Support for ABMC is identified by CPUID Fn8000_0020_EBX_x0[ABMC] (bit 5)
being set. If ABMC is supported, the feature’s attributes and capabilities are enumerated by CPUID Fn8000_0020_x5 as detailed in Appendix E of APM volume 3"

It isn't explicitly stated whether this depends on CPUID Fn0000_000F_x0 EDX{1}, which AMD names: "L3CacheMon - L3 monitoring capability"

Even if the spec does not explicitly state this, the implementation requires this.

I believe patch 2/4 makes this clear since it demonstrates how resctrl obtains the general L3 monitoring properties from CPUID(0xF, 0x1) before moving to the ABMC feature specific properties.

How these general L3 monitoring properties are required by ABMC can also be seen in the ABMC counter reading code: resctrl_arch_cntr_read() calls get_corrected_val() that uses the general L3 monitoring properties hw_res->mbm_width and hw_res->mon_scale. ABMC feature properties also do not expose its own number of RMID supported, this is learned from general L3 monitoring properties.

One item of note here is just that ABMC (as I understand) does _not_ depend on any of the individual L3 monitoring features (X86_FEATURE_CQM_OCCUP_LLC, X86_FEATURE_CQM_MBM_TOTAL, and X86_FEATURE_CQM_MBM_LOCAL) since it defines its own transactions that can be counted.

Reinette

author maintainer
#5 lore raw

Re: [RFC PATCH 1/4] x86/resctrl: Check if monitoring features are enabled

Quoted 46 lines · Reinette Chatre

Hi Tony,

Quoted 37 lines · Tony
Quoted 30 lines · Reinette Chatre

Hi Tony,

Thank you for doing this.

Quoted 17 lines · Tony Luck

Both Intel and AMD manuals say that software must first check CPUID(0x7,0x0).EBX[12] to see if any monitoring features are enabled before checking for specific features enabled in subleaves.

Add the check for X86_FEATURE_CQM.

Fixes cbc82b172638 ("x86: Add support for Intel Cache QoS Monitoring (CQM) detection")
---
 arch/x86/kernel/cpu/resctrl/core.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kernel/cpu/resctrl/core.c +0 -0
index 55214d6fdc49..2677b8a6c15b 100644

arch/x86/kernel/cpu/resctrl/core.c static __init bool get_rdt_mon_resources(void) · line 972

Quoted 5 lines
arch/x86/kernel/cpu/resctrl/core.c +2 -0
@@ -968,6 +968,9 @@ static __init bool get_rdt_mon_resources(void)
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
 	bool ret = false;
 
+	if (!cpu_feature_enabled(X86_FEATURE_CQM))
+		return false;

Is this missing a check of X86_FEATURE_CQM_LLC also?

Quoted 5 lines

As I understand resctrl learns from X86_FEATURE_CQM whether the system supports resource monitoring in general. Contrary to what the feature name suggests, there is another step needed to determine which resource(s) support monitoring via CPUID(0xF,0x0).EDX where bit 1 indicates LLC monitoring that needs to be set before the LLC resource-specific monitoring properties can/should be determined.

Quoted 5 lines

Yes. Missing that check. I agree that the feature define names are bad.

To avoid continued confusion I should rename the #defines to match the bit names in the Intel SDM (but leaving the /proc/cpuinfo visible strings at "cqm" and "cqm_llc" as those are user ABI now).

Quoted 5 lines

Naming is as always complicated and here I do not see why the name should be picked from Intel SDM instead of AMD's spec. Renaming may be secondary goal. It may even add to confusion to have feature name mismatch what is exposed to user space? A change like this could perhaps be punted to when/if monitoring of a new resource needs to be supported?

Two reasons to pick the Intel SDM name:
1) Intel was here first
commit cbc82b172638 ("x86: Add support for Intel Cache QoS Monitoring (CQM) detection")
2) Symmetry with the feature bit for allocation:
#define X86_FEATURE_RDT_A ( 9*32+15) /* "rdt_a" Resource Director Technology Allocation */

Confusion between the string in /proc/cpuinfo and the X86_FEATURE define is already rampant. Just a few examples:

#define X86_FEATURE_XMM                 ( 0*32+25) /* "sse" */
#define X86_FEATURE_XSTORE              ( 5*32+ 2) /* "rng" RNG present (xstore) */
#define X86_FEATURE_SVML                (15*32+ 2) /* "svm_lock" SVM locking MSR */
#define X86_FEATURE_TSCRATEMSR          (15*32+ 4) /* "tsc_scale" TSC scaling support */

So anyone wanting to find the #define that goes with a feature name in /proc/cpuinfo would be advised to just grep for the string in <asm/cpufeatures.h>

Quoted 26 lines
Quoted 26 lines

X86_FEATURE_CQM -> X86_FEATURE_RDT_M
X86_FEATURE_CQM_LLC -> X86_FEATURE_L3_MON

Quoted 6 lines

Apart from the checks here I see that cpuid_deps[] accurately reflects the relationship between X86_FEATURE_CQM_LLC and the different LLC monitoring features checked for below. I do not see cpuid_deps[] capturing the relationship between X86_FEATURE_CQM and X86_FEATURE_CQM_LLC though. Could adding it complete the handling of relationships between these leaves?

Yes.

Quoted 3 lines

Similarly I think cpuid_deps[] may be missing X86_FEATURE_ABMC's dependency on X86_FEATURE_CQM_LLC.

Maybe no? X86_FEATURE_ABMC isn't enumerated in CPUID(0xF,*). It comes from the AMD CPUID(0x80000020,0)EBX{5}

Babu: The AMD architecture programmer's manual just says:

"Support for ABMC is identified by CPUID Fn8000_0020_EBX_x0[ABMC] (bit 5)
being set. If ABMC is supported, the feature’s attributes and capabilities are enumerated by CPUID Fn8000_0020_x5 as detailed in Appendix E of APM volume 3"

Quoted 8 lines
Quoted 2 lines

It isn't explicitly stated whether this depends on CPUID Fn0000_000F_x0 EDX{1}, which AMD names: "L3CacheMon - L3 monitoring capability"

Even if the spec does not explicitly state this, the implementation requires this.

I believe patch 2/4 makes this clear since it demonstrates how resctrl obtains the general L3 monitoring properties from CPUID(0xF, 0x1) before moving to the ABMC feature specific properties.

Agreed.

Quoted 11 lines

How these general L3 monitoring properties are required by ABMC can also be seen in the ABMC counter reading code: resctrl_arch_cntr_read() calls get_corrected_val() that uses the general L3 monitoring properties hw_res->mbm_width and hw_res->mon_scale. ABMC feature properties also do not expose its own number of RMID supported, this is learned from general L3 monitoring properties.

One item of note here is just that ABMC (as I understand) does _not_ depend on any of the individual L3 monitoring features (X86_FEATURE_CQM_OCCUP_LLC, X86_FEATURE_CQM_MBM_TOTAL, and X86_FEATURE_CQM_MBM_LOCAL) since it defines its own transactions that can be counted.

Reinette

-Tony

author maintainer
#6 lore raw

RFC PATCH 2/4 x86/resctrl: Enumerate monitor features in rdt_get_l3_mon_config()

In reply to Tony Luck

The original implementation of Intel Cache QoS Monitoring (CQM) planned to integrate with the "perf" and "cgroup" subsystems. With that plan it made sense for parameters from CPUID to be stored in fields of the cpuinfo_x86 structure. But that plan was abandoned and the resctrl file system user interface replaced it.

Enumerate the L3 monitoring features in rdt_get_l3_mon_config().

---
 arch/x86/include/asm/resctrl.h        |  9 ++++----
 arch/x86/kernel/cpu/resctrl/monitor.c | 32 ++++++++++++++++++++++++---
 2 files changed, 33 insertions(+), 8 deletions(-)
arch/x86/include/asm/resctrl.h +4 -5
index 8f6edcdcfd87..9bfa9b14002b 100644
@@ -44,6 +44,7 @@ DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state);
 
 extern bool rdt_alloc_capable;
 extern bool rdt_mon_capable;
+extern unsigned int __ro_after_init rdt_l3_mon_scale;
 
 DECLARE_STATIC_KEY_FALSE(rdt_enable_key);
 DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);
@@ -132,11 +133,9 @@ static inline void __resctrl_sched_in(struct task_struct *tsk)
 
 static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)
 {
-	unsigned int scale = boot_cpu_data.x86_cache_occ_scale;
-
-	/* h/w works in units of "boot_cpu_data.x86_cache_occ_scale" */
-	val /= scale;
-	return val * scale;
+	/* Round down to nearest h/w monitoring unit */
+	val /= rdt_l3_mon_scale;
+	return val * rdt_l3_mon_scale;
 }
 
 static inline void resctrl_arch_set_cpu_default_closid_rmid(int cpu, u32 closid,
arch/x86/kernel/cpu/resctrl/monitor.c +29 -3
index 3838e0a13d36..0c2aa20148a1 100644
@@ -32,6 +32,11 @@
  */
 bool rdt_mon_capable;
 
+/*
+ * Scale factor to convert L3 monitor events to bytes.
+ */
+unsigned int __ro_after_init rdt_l3_mon_scale;
+
 #define CF(cf)	((unsigned long)(1048576 * (cf) + 0.5))
 
 static int snc_nodes_per_l3_cache = 1;
@@ -418,16 +423,37 @@ static __init int snc_get_config(void)
 
 int __init rdt_get_l3_mon_config(struct rdt_resource *r)
 {
-	unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
+	unsigned int mbm_offset;
 	unsigned int threshold;
 	u32 eax, ebx, ecx, edx;
+	u32 num_rmid;
+
+	/* QoS sub-leaf, EAX=0Fh, ECX=1 */
+	cpuid_count(0xf, 1, &eax, &ebx, &ecx, &edx);
+	mbm_offset = eax & 0xff;
+	rdt_l3_mon_scale = ebx;
+	num_rmid = ecx + 1;
+
+	if (!mbm_offset) {
+		switch (boot_cpu_data.x86_vendor) {
+		case X86_VENDOR_AMD:
+			mbm_offset = MBM_CNTR_WIDTH_OFFSET_AMD;
+			break;
+		case X86_VENDOR_HYGON:
+			mbm_offset = MBM_CNTR_WIDTH_OFFSET_HYGON;
+			break;
+		default:
+			/* Leave mbm_offset as 0 */
+			break;
+		}
+	}
 
 	snc_nodes_per_l3_cache = snc_get_config();
 
 	resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size * 1024;
-	hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale / snc_nodes_per_l3_cache;
-	r->mon.num_rmid = (boot_cpu_data.x86_cache_max_rmid + 1) / snc_nodes_per_l3_cache;
+	hw_res->mon_scale = rdt_l3_mon_scale / snc_nodes_per_l3_cache;
+	r->mon.num_rmid = num_rmid / snc_nodes_per_l3_cache;
 	hw_res->mbm_width = MBM_CNTR_WIDTH_BASE;
 
 	if (mbm_offset > 0 && mbm_offset <= MBM_CNTR_WIDTH_OFFSET_MAX)
Signature · 1 line
2.55.0
author maintainer
#7 lore raw

RFC PATCH 3/4 x86/resctrl: Apply Intel MBM quirk from rdt_get_l3_mon_config()

In reply to Tony Luck

The Intel quirk to adjust Memory Bandwidth Monitoring (MBM) values on certain CPUs is applied early, before discovering if MBM is supported.

Move the call to rdt_get_l3_mon_config(), but keep the decision on whether it is needed in __check_quirks_intel() with all the other model specific tests.

---
 arch/x86/kernel/cpu/resctrl/internal.h |  4 +--
 arch/x86/kernel/cpu/resctrl/core.c     |  2 +-
 arch/x86/kernel/cpu/resctrl/monitor.c  | 35 +++++++++++++++-----------
 3 files changed, 23 insertions(+), 18 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -2
index e3cfa0c10e92..e46eb9a4c725 100644
@@ -151,6 +151,8 @@ static inline struct rdt_hw_resource *resctrl_to_arch_res(struct rdt_resource *r
 
 extern struct rdt_hw_resource rdt_resources_all[];
 
+extern bool __initdata intel_rdt_mbm_need_quirk;
+
 void arch_mon_domain_online(struct rdt_resource *r, struct rdt_l3_mon_domain *d);
 
 /* CPUID.(EAX=10H, ECX=ResID=1).EAX */
@@ -228,8 +230,6 @@ int rdt_get_l3_mon_config(struct rdt_resource *r);
 
 bool rdt_cpu_has(int flag);
 
-void __init intel_rdt_mbm_apply_quirk(void);
-
 void rdt_domain_reconfigure_cdp(struct rdt_resource *r);
 void resctrl_arch_mbm_cntr_assign_set_one(struct rdt_resource *r);
 
arch/x86/kernel/cpu/resctrl/core.c +1 -1
index 2677b8a6c15b..da94a12f9256 100644
@@ -1006,7 +1006,7 @@ static __init void __check_quirks_intel(void)
 			set_rdt_options("!l3cat");
 		fallthrough;
 	case INTEL_BROADWELL_X:
-		intel_rdt_mbm_apply_quirk();
+		intel_rdt_mbm_need_quirk = true;
 		break;
 	}
 }
arch/x86/kernel/cpu/resctrl/monitor.c +20 -15
index 0c2aa20148a1..0bce199a5f7b 100644
@@ -37,6 +37,8 @@ bool rdt_mon_capable;
  */
 unsigned int __ro_after_init rdt_l3_mon_scale;
 
+bool __initdata intel_rdt_mbm_need_quirk;
+
 #define CF(cf)	((unsigned long)(1048576 * (cf) + 0.5))
 
 static int snc_nodes_per_l3_cache = 1;
@@ -51,7 +53,7 @@ static int snc_nodes_per_l3_cache = 1;
  * 1. The threshold 0 is changed to rmid count - 1 so don't do correction
  *    for the case.
  * 2. MBM total and local correction table indexed by core counter which is
- *    equal to (x86_cache_max_rmid + 1) / 8 - 1 and is from 0 up to 27.
+ *    equal to r->mon.num_rmid / 8 - 1 and is from 0 up to 27.
  * 3. The correction factor is normalized to 2^20 (1048576) so it's faster
  *    to calculate corrected value by shifting:
  *    corrected_value = (original_value * correction_factor) >> 20
@@ -421,6 +423,20 @@ static __init int snc_get_config(void)
 	return ret;
 }
 
+static void __init intel_rdt_mbm_apply_quirk(u32 num_rmid)
+{
+	int cf_index;
+
+	cf_index = num_rmid / 8 - 1;
+	if (cf_index >= ARRAY_SIZE(mbm_cf_table)) {
+		pr_info("No MBM correction factor available\n");
+		return;
+	}
+
+	mbm_cf_rmidthreshold = mbm_cf_table[cf_index].rmidthreshold;
+	mbm_cf = mbm_cf_table[cf_index].cf;
+}
+
 int __init rdt_get_l3_mon_config(struct rdt_resource *r)
 {
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
@@ -500,25 +516,14 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
 		hw_res->mbm_cntr_assign_enabled = true;
 	}
 
+	if (intel_rdt_mbm_need_quirk)
+		intel_rdt_mbm_apply_quirk(r->mon.num_rmid);
+
 	r->mon_capable = true;
 
 	return 0;
 }
 
-void __init intel_rdt_mbm_apply_quirk(void)
-{
-	int cf_index;
-
-	cf_index = (boot_cpu_data.x86_cache_max_rmid + 1) / 8 - 1;
-	if (cf_index >= ARRAY_SIZE(mbm_cf_table)) {
-		pr_info("No MBM correction factor available\n");
-		return;
-	}
-
-	mbm_cf_rmidthreshold = mbm_cf_table[cf_index].rmidthreshold;
-	mbm_cf = mbm_cf_table[cf_index].cf;
-}
-
 static void resctrl_abmc_set_one_amd(void *arg)
 {
 	bool *enable = arg;
Signature · 1 line
2.55.0
author maintainer
#8 lore raw

RFC PATCH 4/4 x86/resctrl: Delete resctrl_cpu_detect()

In reply to Tony Luck

cpuinfo_x86::x86_cache_{max_rmid,occ_scale,mbm_width_offset} are no longer used.

Delete resctrl_cpu_detect() and the fields from struct cpuinfo_x86.

---
 arch/x86/include/asm/processor.h   |  4 ---
 arch/x86/include/asm/resctrl.h     |  3 ---
 arch/x86/kernel/cpu/amd.c          |  3 ---
 arch/x86/kernel/cpu/hygon.c        |  3 ---
 arch/x86/kernel/cpu/intel.c        |  7 -----
 arch/x86/kernel/cpu/resctrl/core.c | 42 ------------------------------
 6 files changed, 62 deletions(-)
arch/x86/include/asm/processor.h +0 -4
index ec9db0dfa0df..4deb88c61c94 100644
@@ -185,10 +185,6 @@ struct cpuinfo_x86 {
 	/* in KB - valid for CPUS which support this call: */
 	unsigned int		x86_cache_size;
 	int			x86_cache_alignment;	/* In bytes */
-	/* Cache QoS architectural values, valid only on the BSP: */
-	int			x86_cache_max_rmid;	/* max index */
-	int			x86_cache_occ_scale;	/* scale to bytes */
-	int			x86_cache_mbm_width_offset;
 	int			x86_power;
 	unsigned long		loops_per_jiffy;
 	/* protected processor identification number */
arch/x86/include/asm/resctrl.h +0 -3
index 9bfa9b14002b..ab7f6ccd149e 100644
@@ -193,12 +193,9 @@ static inline void resctrl_arch_mon_ctx_free(struct rdt_resource *r,
 					     enum resctrl_event_id evtid,
 					     void *ctx) { }
 
-void resctrl_cpu_detect(struct cpuinfo_x86 *c);
-
 #else
 
 static inline void resctrl_arch_sched_in(struct task_struct *tsk) {}
-static inline void resctrl_cpu_detect(struct cpuinfo_x86 *c) {}
 
 #endif /* CONFIG_X86_CPU_RESCTRL */
 
arch/x86/kernel/cpu/amd.c +0 -3
index 54e14ed276b5..dd82a11084f4 100644
@@ -23,7 +23,6 @@
 #include <asm/pci-direct.h>
 #include <asm/delay.h>
 #include <asm/debugreg.h>
-#include <asm/resctrl.h>
 #include <asm/msr.h>
 #include <asm/sev.h>
 
@@ -475,8 +474,6 @@ static void bsp_init_amd(struct cpuinfo_x86 *c)
 		}
 	}
 
-	resctrl_cpu_detect(c);
-
 	/* Figure out Zen generations: */
 	switch (c->x86) {
 	case 0x17:
arch/x86/kernel/cpu/hygon.c +0 -3
index ec51c2b9a257..0f226335dd9a 100644
@@ -17,7 +17,6 @@
 #include <asm/spec-ctrl.h>
 #include <asm/delay.h>
 #include <asm/msr.h>
-#include <asm/resctrl.h>
 
 #include "cpu.h"
 
@@ -119,8 +118,6 @@ static void bsp_init_hygon(struct cpuinfo_x86 *c)
 			x86_amd_ls_cfg_ssbd_mask = 1ULL << 10;
 		}
 	}
-
-	resctrl_cpu_detect(c);
 }
 
 static void early_init_hygon(struct cpuinfo_x86 *c)
arch/x86/kernel/cpu/intel.c +0 -7
index 4297ceb2cb24..2f09710e863c 100644
@@ -22,7 +22,6 @@
 #include <asm/microcode.h>
 #include <asm/msr.h>
 #include <asm/numa.h>
-#include <asm/resctrl.h>
 #include <asm/thermal.h>
 #include <asm/uaccess.h>
 
@@ -371,11 +370,6 @@ static void early_init_intel(struct cpuinfo_x86 *c)
 		detect_tme_early(c);
 }
 
-static void bsp_init_intel(struct cpuinfo_x86 *c)
-{
-	resctrl_cpu_detect(c);
-}
-
 #ifdef CONFIG_X86_32
 /*
  *	Early probe support logic for ppro memory erratum #50
@@ -804,7 +798,6 @@ static const struct cpu_dev intel_cpu_dev = {
 #endif
 	.c_detect_tlb	= intel_detect_tlb,
 	.c_early_init   = early_init_intel,
-	.c_bsp_init	= bsp_init_intel,
 	.c_init		= init_intel,
 	.c_x86_vendor	= X86_VENDOR_INTEL,
 };
arch/x86/kernel/cpu/resctrl/core.c +0 -42

48 lines not on this page · raw message

Signature · 1 line
2.55.0
maintainer Acked-by on v1 4/4
#9 lore raw

Re: [RFC PATCH 4/4] x86/resctrl: Delete resctrl_cpu_detect()

Quoted 6 lines · Tony Luck

cpuinfo_x86::x86_cache_{max_rmid,occ_scale,mbm_width_offset} are no longer used.

Delete resctrl_cpu_detect() and the fields from struct cpuinfo_x86.

Quoted 8 lines
---
 arch/x86/include/asm/processor.h   |  4 ---
 arch/x86/include/asm/resctrl.h     |  3 ---
 arch/x86/kernel/cpu/amd.c          |  3 ---
 arch/x86/kernel/cpu/hygon.c        |  3 ---
 arch/x86/kernel/cpu/intel.c        |  7 -----
 arch/x86/kernel/cpu/resctrl/core.c | 42 ------------------------------
 6 files changed, 62 deletions(-)

I obviously love that patch!

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#10 lore raw
In reply to Tony Luck
Quoted 3 lines

This series is posted as RFC to get some feedback before I incorporate it into the next version of my patches to fix Kconfig issues. It was prompted by Reinette's comment:

Sashiko review is done[1]. The only complaint is that I didn't address the pre-existing issue that a malicious/broken hypervisor might provide invalid return values from some CPUID invocations which could lead to divide-by-zero faults.

I don't intend to address this. Virtualization of RDT would require a great deal more support from the hypervisor that just getting this CPUID enumeration right.

-Tony

[1] https://sashiko.dev/#/patchset/20260819161323.11587-1-tony.luck%40intel.com