Skip to contentSkip to conversation

x86/resctrl: Workaround MBM errata

PATCH Superseded Fenghua Yu 2 participants 5 messages Oct 13, 2020

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Fenghua Yu

author
#0 lore raw

v2 cover x86/resctrl: Workaround MBM errata

Intel Memory Bandwidth Monitoring (MBM) counters may report system memory bandwidth incorrectly on some Intel processors. The errata are reported in erratum SKX99 [1], erratum BDF102 [2] and RDT reference manual [3].

The errata are worked around using a correction factor table. Since the correction factor table is not publicly documented anywhere, it's documented in Documentation/x86/resctrl.rst along with the errata.

1. Erratum SKX99 in Intel Xeon Processor Scalable Family Specification Update:
http://web.archive.org/web/20200716124958/https://www.intel.com/content/www/us/en/processors/xeon/scalable/xeon-scalable-spec-update.html
2. Erratum BDF102 in Intel Xeon E5-2600 v4 Processor Product Family Specification Update:
http://web.archive.org/web/20191125200531/https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-e5-v4-spec-update.pdf
3. The errata in Intel Resource Director Technology (Intel RDT) on 2nd Generation Intel Xeon Scalable Processors Reference Manual:
https://software.intel.com/content/www/us/en/develop/articles/intel-resource-director-technology-rdt-reference-manual.html

Change Log:
v2:
- Document the errata and the correction table in resctrl.rst (Boris).
- Address various comments on patch 2 (Boris).
- Change the documentation URLs to stable archive.org (Tony).

Fenghua Yu (2):
Documentation: x86: Rename resctrl_ui.rst and add two errata to the
file
x86/resctrl: Correct MBM total and local values

 Documentation/conf.py                         |  2 +-
 Documentation/x86/index.rst                   |  2 +-
 .../x86/{resctrl_ui.rst => resctrl.rst}       | 82 ++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/core.c            |  4 +
 arch/x86/kernel/cpu/resctrl/internal.h        |  1 +
 arch/x86/kernel/cpu/resctrl/monitor.c         | 83 ++++++++++++++++++-
 6 files changed, 170 insertions(+), 4 deletions(-)
 rename Documentation/x86/{resctrl_ui.rst => resctrl.rst} (92%)
Signature · 1 line
2.28.0
author
#1 lore raw

PATCH v2 1/2 Documentation: x86: Rename resctrl_ui.rst and add two errata to the file

Intel Memory Bandwidth Monitoring (MBM) counters may report system memory bandwidth incorrectly on some Intel processors. The errata are reported in erratum SKX99 [1], erratum BDF102 [2] and RDT reference manual [3].

To work around the errata, MBM total and local readings are corrected using a correction factor table.

Since the correction factor table is not publicly documented anywhere, the table and the errata are documented in Documentation/x86/resctrl.rst for future reference. The resctrl.rst file is renamed from Documentation/x86/resctrl_ui.rst because the file won't contain user interface only anymore.

1. Erratum SKX99 in Intel Xeon Processor Scalable Family Specification Update:
http://web.archive.org/web/20200716124958/https://www.intel.com/content/www/us/en/processors/xeon/scalable/xeon-scalable-spec-update.html
2. Erratum BDF102 in Intel Xeon E5-2600 v4 Processor Product Family Specification Update:
http://web.archive.org/web/20191125200531/https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-e5-v4-spec-update.pdf
3. The errata in Intel Resource Director Technology (Intel RDT) on 2nd Generation Intel Xeon Scalable Processors Reference Manual:
https://software.intel.com/content/www/us/en/develop/articles/intel-resource-director-technology-rdt-reference-manual.html

---
Change Log:
v2:
- Document the correction factor table and errata in resctrl.rst (Boris).
- Change the documentation URLs to stable archive.org (Tony).

 Documentation/conf.py                         |  2 +-
 Documentation/x86/index.rst                   |  2 +-
 .../x86/{resctrl_ui.rst => resctrl.rst}       | 82 +++++++++++++++++++
 3 files changed, 84 insertions(+), 2 deletions(-)
 rename Documentation/x86/{resctrl_ui.rst => resctrl.rst} (92%)
Documentation/conf.py +1 -1
index c503188880d9..b5b2be8eec22 100644
@@ -36,7 +36,7 @@ needs_sphinx = '1.3'
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain',
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include',
               'kfigure', 'sphinx.ext.ifconfig', 'automarkup',
               'maintainers_include', 'sphinx.ext.autosectionlabel' ]
 
Documentation/x86/index.rst +1 -1
index 265d9e9a093b..49d2fd9f0e5b 100644
@@ -25,7 +25,7 @@ x86-specific Documentation
    pti
    mds
    microcode
-   resctrl_ui
+   resctrl
    tsx_async_abort
    usb-legacy-support
    i386/index
Documentation/x86/resctrl.rst renamed from Documentation/x86/resctrl_ui.rst +82 -0

85 lines not on this page · raw message

Signature · 1 line
2.28.0
reviewer
#2 lore raw

Re: [PATCH v2 1/2] Documentation: x86: Rename resctrl_ui.rst and add two errata to the file

Quoted 43 lines · Fenghua Yu

Intel Memory Bandwidth Monitoring (MBM) counters may report system memory bandwidth incorrectly on some Intel processors. The errata are reported in erratum SKX99 [1], erratum BDF102 [2] and RDT reference manual [3].

To work around the errata, MBM total and local readings are corrected using a correction factor table.

Since the correction factor table is not publicly documented anywhere, the table and the errata are documented in Documentation/x86/resctrl.rst for future reference. The resctrl.rst file is renamed from Documentation/x86/resctrl_ui.rst because the file won't contain user interface only anymore.

1. Erratum SKX99 in Intel Xeon Processor Scalable Family Specification Update:
http://web.archive.org/web/20200716124958/https://www.intel.com/content/www/us/en/processors/xeon/scalable/xeon-scalable-spec-update.html
2. Erratum BDF102 in Intel Xeon E5-2600 v4 Processor Product Family Specification Update:
http://web.archive.org/web/20191125200531/https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-e5-v4-spec-update.pdf
3. The errata in Intel Resource Director Technology (Intel RDT) on 2nd Generation Intel Xeon Scalable Processors Reference Manual:
https://software.intel.com/content/www/us/en/develop/articles/intel-resource-director-technology-rdt-reference-manual.html

---
Change Log:
v2:
- Document the correction factor table and errata in resctrl.rst (Boris).
- Change the documentation URLs to stable archive.org (Tony).

 Documentation/conf.py                         |  2 +-
 Documentation/x86/index.rst                   |  2 +-
 .../x86/{resctrl_ui.rst => resctrl.rst}       | 82 +++++++++++++++++++
 3 files changed, 84 insertions(+), 2 deletions(-)
 rename Documentation/x86/{resctrl_ui.rst => resctrl.rst} (92%)
Documentation/conf.py +0 -0
index c503188880d9..b5b2be8eec22 100644

Documentation/conf.py needs_sphinx = '1.3' · line 41

Quoted 7 lines
Documentation/conf.py +1 -1
@@ -36,7 +36,7 @@ needs_sphinx = '1.3'
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain',
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include',
               'kfigure', 'sphinx.ext.ifconfig', 'automarkup',
               'maintainers_include', 'sphinx.ext.autosectionlabel' ]

Hi,
I don't see this change described in the patch description. Should it be here?

thanks.

Signature · 1 line
~Randy
author
#3 lore raw

Re: [PATCH v2 1/2] Documentation: x86: Rename resctrl_ui.rst and add two errata to the file

Hi, Randy,

Quoted 17 lines · Randy Dunlap

Documentation/conf.py needs_sphinx = '1.3' · line 41

Quoted 12 lines · Fenghua Yu
Documentation/conf.py +1 -1
index c503188880d9..b5b2be8eec22 100644
@@ -36,7 +36,7 @@ needs_sphinx = '1.3'
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain',
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include',
               'kfigure', 'sphinx.ext.ifconfig', 'automarkup',
               'maintainers_include', 'sphinx.ext.autosectionlabel' ]

Hi,
I don't see this change described in the patch description. Should it be here?

My bad. I forgot to remove this change. I will remove it in v3.

Thank you for your review!

-Fenghua

author
#4 lore raw

PATCH v2 2/2 x86/resctrl: Correct MBM total and local values

In reply to Fenghua Yu

Intel Memory Bandwidth Monitoring (MBM) counters may report system memory bandwidth incorrectly on some Intel processors. The errata SKX99 for Skylake server, BDF102 for Broadwell server, and the correction factor table are documented in Documentation/x86/resctrl.rst.

Intel MBM counters track metrics according to the assigned Resource Monitor ID (RMID) for that logical core. The IA32_QM_CTR register (MSR 0xC8E), used to report these metrics, may report incorrect system bandwidth for certain RMID values.

Due to the errata, system memory bandwidth may not match what is reported.

To work around the errata, MBM total and local readings are corrected using the correction factor table. If rmid > rmid threshold, MBM total and local values should be multiplied by the correction factor.

---
Change Log:
v2:
- Add "const" for mbm_cf_table[] (Boris).
- Add verbs to corrected_mbm_counter() and intel_rdt_mbm_quirk() (Boris).
- Describe the errata in the commit message (Boris).
- Fix checkpatch.pl warning on "/* FALLTHROUGH */" (Boris).

 arch/x86/kernel/cpu/resctrl/core.c     |  4 ++
 arch/x86/kernel/cpu/resctrl/internal.h |  1 +
 arch/x86/kernel/cpu/resctrl/monitor.c  | 83 +++++++++++++++++++++++++-
 3 files changed, 86 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +4 -0
index 9e1712e8aef7..b3bdc477599e 100644
@@ -893,6 +893,10 @@ static __init void __check_quirks_intel(void)
 			set_rdt_options("!cmt,!mbmtotal,!mbmlocal,!l3cat");
 		else
 			set_rdt_options("!l3cat");
+		fallthrough;
+	case INTEL_FAM6_BROADWELL_X:
+		intel_rdt_mbm_apply_quirk();
+		break;
 	}
 }
 
arch/x86/kernel/cpu/resctrl/internal.h +1 -0
index 80fa997fae60..35ececc0b48b 100644
@@ -616,6 +616,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
 void mbm_setup_overflow_handler(struct rdt_domain *dom,
 				unsigned long delay_ms);
 void mbm_handle_overflow(struct work_struct *work);
+void intel_rdt_mbm_apply_quirk(void);
 bool is_mba_sc(struct rdt_resource *r);
 void setup_default_ctrlval(struct rdt_resource *r, u32 *dc, u32 *dm);
 u32 delay_bw_map(unsigned long bw, struct rdt_resource *r);
arch/x86/kernel/cpu/resctrl/monitor.c +81 -2

110 lines not on this page · raw message

Signature · 1 line
2.28.0