x86/resctrl: Work around MBM errata
Subsystems (3)
v3 cover x86/resctrl: Work around MBM errata
Intel Memory Bandwidth Monitoring (MBM) counters may report system memory bandwidth incorrectly on some Intel processors. The errata are reported in erratum SKX99 [1], erratum BDF102 [2] and RDT reference manual [3].
The errata are worked around using a correction factor table. Since the correction factor table is not publicly documented anywhere, it's documented in Documentation/x86/resctrl.rst along with the errata.
1. Erratum SKX99 in Intel Xeon Processor Scalable Family Specification Update:
http://web.archive.org/web/20200716124958/https://www.intel.com/content/www/us/en/processors/xeon/scalable/xeon-scalable-spec-update.html
2. Erratum BDF102 in Intel Xeon E5-2600 v4 Processor Product Family Specification Update:
http://web.archive.org/web/20191125200531/https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-e5-v4-spec-update.pdf
3. The errata in Intel Resource Director Technology (Intel RDT) on 2nd Generation Intel Xeon Scalable Processors Reference Manual:
https://software.intel.com/content/www/us/en/develop/articles/intel-resource-director-technology-rdt-reference-manual.html
Change Log:
v3:
- Remove unnecessary conf.py change in patch 1 (Randy).
v2:
- Document the errata and the correction table in resctrl.rst (Boris).
- Address various comments on patch 2 (Boris).
- Change the documentation URLs to stable archive.org (Tony).
Fenghua Yu (2):
Documentation: x86: Rename resctrl_ui.rst and add two errata to the
file
x86/resctrl: Correct MBM total and local values
Documentation/x86/index.rst | 2 +-
.../x86/{resctrl_ui.rst => resctrl.rst} | 82 +++++++++++++++++++
arch/x86/kernel/cpu/resctrl/core.c | 4 +
arch/x86/kernel/cpu/resctrl/internal.h | 1 +
arch/x86/kernel/cpu/resctrl/monitor.c | 82 ++++++++++++++++++-
5 files changed, 168 insertions(+), 3 deletions(-)
rename Documentation/x86/{resctrl_ui.rst => resctrl.rst} (92%)Signature · 1 line
2.28.0
PATCH v3 1/2 Documentation: x86: Rename resctrl_ui.rst and add two errata to the file
Intel Memory Bandwidth Monitoring (MBM) counters may report system memory bandwidth incorrectly on some Intel processors. The errata are reported in erratum SKX99 [1], erratum BDF102 [2] and RDT reference manual [3].
To work around the errata, MBM total and local readings are corrected using a correction factor table.
Since the correction factor table is not publicly documented anywhere, the table and the errata are documented in Documentation/x86/resctrl.rst for future reference. The resctrl.rst file is renamed from Documentation/x86/resctrl_ui.rst because the file won't contain user interface only anymore.
1. Erratum SKX99 in Intel Xeon Processor Scalable Family Specification Update:
http://web.archive.org/web/20200716124958/https://www.intel.com/content/www/us/en/processors/xeon/scalable/xeon-scalable-spec-update.html
2. Erratum BDF102 in Intel Xeon E5-2600 v4 Processor Product Family Specification Update:
http://web.archive.org/web/20191125200531/https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-e5-v4-spec-update.pdf
3. The errata in Intel Resource Director Technology (Intel RDT) on 2nd Generation Intel Xeon Scalable Processors Reference Manual:
https://software.intel.com/content/www/us/en/develop/articles/intel-resource-director-technology-rdt-reference-manual.html
---
Change Log:
v3:
- Remove unnecessary conf.py change in patch 1 (Randy).
v2:
- Document the correction factor table and errata in resctrl.rst (Boris).
- Change the documentation URLs to stable archive.org (Tony).
Documentation/x86/index.rst | 2 +-
.../x86/{resctrl_ui.rst => resctrl.rst} | 82 +++++++++++++++++++
2 files changed, 83 insertions(+), 1 deletion(-)
rename Documentation/x86/{resctrl_ui.rst => resctrl.rst} (92%)Documentation/x86/index.rst +1 -1
@@ -25,7 +25,7 @@ x86-specific Documentation pti mds microcode - resctrl_ui + resctrl tsx_async_abort usb-legacy-support i386/index
Documentation/x86/resctrl.rst renamed from Documentation/x86/resctrl_ui.rst +82 -0
85 lines not on this page · raw message
Signature · 1 line
2.28.0
[tip: x86/cache] Documentation/x86: Rename resctrl_ui.rst and add two errata to the file
The following commit has been merged into the x86/cache branch of tip:
The following commit has been merged into the x86/cache branch of tip:
Commit-ID: d1b22e36e3188f47dbf8aaef54d9e79df8e91f4c Gitweb: https://git.kernel.org/tip/d1b22e36e3188f47dbf8aaef54d9e79df8e91f4c Author: Fenghua Yu <[email protected]> AuthorDate: Wed, 14 Oct 2020 00:49:26 Committer: Borislav Petkov <[email protected]> CommitterDate: Tue, 27 Oct 2020 16:47:00 +01:00
Documentation/x86: Rename resctrl_ui.rst and add two errata to the file
Intel Memory Bandwidth Monitoring (MBM) counters may report system memory bandwidth incorrectly on some Intel processors. This is reported in documented in erratum SKX99, erratum BDF102 and in the RDT reference manual, see Documentation/x86/index.rst.
To work around the errata, MBM total and local readings are corrected using a correction factor table.
Since the correction factor table is not publicly documented anywhere, document the table and the errata in Documentation/x86/resctrl.rst for future reference.
[ bp: Move web links to the doc, massage. ]
--- Documentation/x86/index.rst | 2 +- Documentation/x86/resctrl.rst | 1304 +++++++++++++++++++++++++++++- Documentation/x86/resctrl_ui.rst | 1211 +--------------------------- 3 files changed, 1305 insertions(+), 1212 deletions(-) create mode 100644 Documentation/x86/resctrl.rst delete mode 100644 Documentation/x86/resctrl_ui.rst
Documentation/x86/index.rst +1 -1
@@ -27,7 +27,7 @@ x86-specific Documentation pti mds microcode - resctrl_ui + resctrl tsx_async_abort usb-legacy-support i386/index
Documentation/x86/resctrl.rst new file +1304 -0
1,304 lines not on this page · raw message
Documentation/x86/resctrl_ui.rst deleted +0 -1211
1,211 lines not on this page · raw message
PATCH v3 2/2 x86/resctrl: Correct MBM total and local values
In reply to Fenghua YuIntel Memory Bandwidth Monitoring (MBM) counters may report system memory bandwidth incorrectly on some Intel processors. The errata SKX99 for Skylake server, BDF102 for Broadwell server, and the correction factor table are documented in Documentation/x86/resctrl.rst.
Intel MBM counters track metrics according to the assigned Resource Monitor ID (RMID) for that logical core. The IA32_QM_CTR register (MSR 0xC8E), used to report these metrics, may report incorrect system bandwidth for certain RMID values.
Due to the errata, system memory bandwidth may not match what is reported.
To work around the errata, MBM total and local readings are corrected using the correction factor table. If rmid > rmid threshold, MBM total and local values should be multiplied by the correction factor.
---
Change Log:
v2:
- Add "const" for mbm_cf_table[] (Boris).
- Add verbs to corrected_mbm_counter() and intel_rdt_mbm_quirk() (Boris).
- Describe the errata in the commit message (Boris).
- Fix checkpatch.pl warning on "/* FALLTHROUGH */" (Boris).
arch/x86/kernel/cpu/resctrl/core.c | 4 ++ arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 82 +++++++++++++++++++++++++- 3 files changed, 85 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +4 -0
@@ -893,6 +893,10 @@ static __init void __check_quirks_intel(void)set_rdt_options ("!cmt,!mbmtotal,!mbmlocal,!l3cat" ); else set_rdt_options("!l3cat"); + fallthrough; +case INTEL_FAM6_BROADWELL_X : +intel_rdt_mbm_apply_quirk (); +break ; } }
arch/x86/kernel/cpu/resctrl/internal.h +1 -0
@@ -616,6 +616,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms);void mbm_handle_overflow (struct work_struct *work); +void __init intel_rdt_mbm_apply_quirk (void );bool is_mba_sc (struct rdt_resource *r);void setup_default_ctrlval (struct rdt_resource *r,u32 *dc,u32 *dm);u32 delay_bw_map (unsigned long bw,struct rdt_resource *r);
arch/x86/kernel/cpu/resctrl/monitor.c +80 -2
103 lines not on this page · raw message
Signature · 1 line
2.28.0
Borislav Petkov
maintainerRe: [PATCH v3 2/2] x86/resctrl: Correct MBM total and local values
arch/x86/kernel/cpu/resctrl/monitor.c unsigned int rdt_mon_features; · line 87
Quoted 4 lines · Fenghua Yu
arch/x86/kernel/cpu/resctrl/monitor.c +4
-0+static const struct mbm_correction_factor_table { +u32 rmidthreshold; +u64 cf; +} mbm_cf_table[] = {
That thing wants to be __initdata, AFAICT, since the only function touching it is __init.
Made it so.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH v3 2/2] x86/resctrl: Correct MBM total and local values
Hi, Boris,
Quoted 10 lines · Borislav Petkov
arch/x86/kernel/cpu/resctrl/monitor.c unsigned int rdt_mon_features; · line 87
Quoted 4 lines · Fenghua Yu
arch/x86/kernel/cpu/resctrl/monitor.c +4
-0+static const struct mbm_correction_factor_table { +u32 rmidthreshold; +u64 cf; +} mbm_cf_table[] = {That thing wants to be
__initdata, AFAICT, since the only function touching it is__init.Made it so.
You are right. The correction value is read from this table only once in __init. So this table should be __initdata.
Thank you very much for your review!
-Fenghua
[tip: x86/cache] x86/resctrl: Correct MBM total and local values
In reply to Fenghua YuThe following commit has been merged into the x86/cache branch of tip:
The following commit has been merged into the x86/cache branch of tip:
Commit-ID: 4868a61d498af3627c3b571aa48107a845001e51 Gitweb: https://git.kernel.org/tip/4868a61d498af3627c3b571aa48107a845001e51 Author: Fenghua Yu <[email protected]> AuthorDate: Wed, 14 Oct 2020 00:49:27 Committer: Borislav Petkov <[email protected]> CommitterDate: Tue, 27 Oct 2020 18:57:22 +01:00
x86/resctrl: Correct MBM total and local values
Intel Memory Bandwidth Monitoring (MBM) counters may report system memory bandwidth incorrectly on some Intel processors. The errata SKX99 for Skylake server, BDF102 for Broadwell server, and the correction factor table are documented in Documentation/x86/resctrl.rst.
Intel MBM counters track metrics according to the assigned Resource Monitor ID (RMID) for that logical core. The IA32_QM_CTR register (MSR 0xC8E) used to report these metrics, may report incorrect system bandwidth for certain RMID values.
Due to the errata, system memory bandwidth may not match what is reported.
To work around the errata, correct MBM total and local readings using a correction factor table. If rmid > rmid threshold, MBM total and local values should be multiplied by the correction factor.
[ bp: Mark mbm_cf_table[] __initdata. ]
--- arch/x86/kernel/cpu/resctrl/core.c | 4 +- arch/x86/kernel/cpu/resctrl/internal.h | 1 +- arch/x86/kernel/cpu/resctrl/monitor.c | 82 ++++++++++++++++++++++++- 3 files changed, 85 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +4 -0
@@ -893,6 +893,10 @@ static __init void __check_quirks_intel(void)set_rdt_options ("!cmt,!mbmtotal,!mbmlocal,!l3cat" ); else set_rdt_options("!l3cat"); + fallthrough; +case INTEL_FAM6_BROADWELL_X : +intel_rdt_mbm_apply_quirk (); +break ; } }
arch/x86/kernel/cpu/resctrl/internal.h +1 -0
@@ -616,6 +616,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms);void mbm_handle_overflow (struct work_struct *work); +void __init intel_rdt_mbm_apply_quirk (void );bool is_mba_sc (struct rdt_resource *r);void setup_default_ctrlval (struct rdt_resource *r,u32 *dc,u32 *dm);u32 delay_bw_map (unsigned long bw,struct rdt_resource *r);
arch/x86/kernel/cpu/resctrl/monitor.c +80 -2
103 lines not on this page · raw message