x86/resctrl: Fix incorrect local bandwidth when mba_sc is enabled
Subsystems (3)
PATCH 1/1 x86/resctrl: Fix incorrect local bandwidth when mba_sc is enabled
MBA software controller (mba_sc) is a feedback loop which periodically reads MBM counters and tries to restrict the bandwidth below a user specified bandwidth. It tags along MBM counter overflow handler to do the updates with 1s interval in mbm_update() and update_mba_bw().
The purpose of mbm_update() is to periodically read the MBM counters to make sure that the hardware counter doesn't wrap around more than once between user samplings. mbm_update() calls __mon_event_count() for local bandwidth updating when mba_sc is not enabled, but calls mbm_bw_count() instead when mba_sc is enabled. __mon_event_count() will not be called for local bandwidth updating in MBM counter overflow handler, but it is still called when reading MBM local bandwidth counter file 'mbm_local_bytes', the call path is as below:
rdtgroup_mondata_show()
mon_event_read()
mon_event_count()
__mon_event_count()In __mon_event_count(), m->chunks is updated by delta chunks which is calculated from previous MSR value (m->prev_msr) and current MSR value. When mba_sc is enabled, m->chunks is also updated in mbm_update() by mistake by the delta chunks which is calculated from m->prev_bw_msr instead of m->prev_msr. But m->chunks is not used in update_mba_bw() in the mba_sc feedback loop.
When reading MBM local bandwidth counter file, m->chunks was changed unexpectedly by mbm_bw_count(). As a result, the incorrect local bandwidth counter which calculated from incorrect m->chunks is read out to the user.
Fix this by removing incorrect m->chunks updating in mbm_bw_count() in MBM counter overflow handler, and always calling __mon_event_count() in mbm_update() to make sure that the hardware local bandwidth counter doesn't wrap around.
Test steps:
# Run workload with aggressive memory bandwidth (e.g., 10 GB/s) git clone https://github.com/intel/intel-cmt-cat && cd intel-cmt-cat && make
./tools/membw/membw -c 0 -b 10000 --read
# Enable MBA software controller
mount -t resctrl resctrl -o mba_MBps /sys/fs/resctrl
# Create control group c1
mkdir /sys/fs/resctrl/c1
# Set MB throttle to 6 GB/s
echo "MB:0=6000;1=6000" > /sys/fs/resctrl/c1/schemata
# Write PID of the workload to tasks file
echo pidof membw > /sys/fs/resctrl/c1/tasks
# Read local bytes counters twice with 1s interval, the calculated
# local bandwidth is not as expected (approaching to 6 GB/s): local_1=`cat /sys/fs/resctrl/c1/mon_data/mon_L3_00/mbm_local_bytes` sleep 1 local_2=`cat /sys/fs/resctrl/c1/mon_data/mon_L3_00/mbm_local_bytes` echo "local b/w (bytes/s):" `expr $local_2 - $local_1`
Before fix:
local b/w (bytes/s): 11076796416
After fix:
local b/w (bytes/s): 5465014272
--- arch/x86/kernel/cpu/resctrl/monitor.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/resctrl/monitor.c +2 -4
@@ -279,7 +279,6 @@ static void mbm_bw_count(u32 rmid, struct rmid_read *rr)return ; chunks =mbm_overflow_count (m->prev_bw_msr, tval, rr->r->mbm_width); - m->chunks += chunks; cur_bw = (chunks * r->mon_scale) >>20 ;if (m->delta_comp)
@@ -450,15 +449,14 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) }if (is_mbm_local_enabled ()) { rr.evtid =QOS_L3_MBM_LOCAL_EVENT_ID ; +__mon_event_count (rmid, &rr);/* * Call the MBA software controller only for the * control groups and when user has enabled * the software controller explicitly. */ -if (!is_mba_sc (NULL )) -__mon_event_count (rmid, &rr); -else +if (is_mba_sc (NULL ))mbm_bw_count (rmid, &rr); } }
Signature · 1 line
1.8.3.1
The following commit has been merged into the x86/cache branch of tip:
The following commit has been merged into the x86/cache branch of tip:
Commit-ID: 2ba836dbe2467d31fffb439258c2f454c6f1a317 Gitweb: https://git.kernel.org/tip/2ba836dbe2467d31fffb439258c2f454c6f1a317 Author: Xiaochen Shen <[email protected]> AuthorDate: Fri, 04 Dec 2020 14:27:59 +08:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Wed, 09 Dec 2020 20:00:46 +01:00
x86/resctrl: Fix incorrect local bandwidth when mba_sc is enabled
The MBA software controller (mba_sc) is a feedback loop which periodically reads MBM counters and tries to restrict the bandwidth below a user specified bandwidth. It tags along the MBM counter overflow handler to do the updates with 1s interval in mbm_update() and update_mba_bw().
The purpose of mbm_update() is to periodically read the MBM counters to make sure that the hardware counter doesn't wrap around more than once between user samplings. mbm_update() calls __mon_event_count() for local bandwidth updating when mba_sc is not enabled, but calls mbm_bw_count() instead when mba_sc is enabled. __mon_event_count() will not be called for local bandwidth updating in MBM counter overflow handler, but it is still called when reading MBM local bandwidth counter file 'mbm_local_bytes', the call path is as below:
rdtgroup_mondata_show()
mon_event_read()
mon_event_count()
__mon_event_count()In __mon_event_count(), m->chunks is updated by delta chunks which is calculated from previous MSR value (m->prev_msr) and current MSR value. When mba_sc is enabled, m->chunks is also updated in mbm_update() by mistake by the delta chunks which is calculated from m->prev_bw_msr instead of m->prev_msr. But m->chunks is not used in update_mba_bw() in the mba_sc feedback loop.
When reading MBM local bandwidth counter file, m->chunks was changed unexpectedly by mbm_bw_count(). As a result, the incorrect local bandwidth counter which calculated using incorrect m->chunks is shown to the user.
Fix this by removing incorrect m->chunks updating in mbm_bw_count() in MBM counter overflow handler, and always calling __mon_event_count() in mbm_update() to make sure that the hardware local bandwidth counter doesn't wrap around.
Test steps:
# Run workload with aggressive memory bandwidth (e.g., 10 GB/s) git clone https://github.com/intel/intel-cmt-cat && cd intel-cmt-cat && make
./tools/membw/membw -c 0 -b 10000 --read
# Enable MBA software controller
mount -t resctrl resctrl -o mba_MBps /sys/fs/resctrl
# Create control group c1
mkdir /sys/fs/resctrl/c1
# Set MB throttle to 6 GB/s
echo "MB:0=6000;1=6000" > /sys/fs/resctrl/c1/schemata
# Write PID of the workload to tasks file
echo pidof membw > /sys/fs/resctrl/c1/tasks
# Read local bytes counters twice with 1s interval, the calculated
# local bandwidth is not as expected (approaching to 6 GB/s): local_1=`cat /sys/fs/resctrl/c1/mon_data/mon_L3_00/mbm_local_bytes` sleep 1 local_2=`cat /sys/fs/resctrl/c1/mon_data/mon_L3_00/mbm_local_bytes` echo "local b/w (bytes/s):" `expr $local_2 - $local_1`
Before fix:
local b/w (bytes/s): 11076796416
After fix:
local b/w (bytes/s): 5465014272
--- arch/x86/kernel/cpu/resctrl/monitor.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/resctrl/monitor.c +2 -4
@@ -320,7 +320,6 @@ static int __mon_event_count(u32 rmid, struct rmid_read *rr) } chunks =mbm_overflow_count (m->prev_msr, tval, rr->r->mbm_width); - m->chunks += chunks; m->prev_msr = tval; rr->val +=get_corrected_mbm_count (rmid, m->chunks);
@@ -514,15 +513,14 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) }if (is_mbm_local_enabled ()) { rr.evtid =QOS_L3_MBM_LOCAL_EVENT_ID ; +__mon_event_count (rmid, &rr);/* * Call the MBA software controller only for the * control groups and when user has enabled * the software controller explicitly. */ -if (!is_mba_sc (NULL )) -__mon_event_count (rmid, &rr); -else +if (is_mba_sc (NULL ))mbm_bw_count (rmid, &rr); } }
Borislav Petkov
maintainerarch/x86/kernel/cpu/resctrl/monitor.c static int __mon_event_count(u32 rmid, struct rmid_read *rr) · line 325
Quoted 12 lines · On Wed, Dec 09, 2020 at 07:06:58PM -0000, tip-bot2 for Xiaochen Shen wrote:
arch/x86/kernel/cpu/resctrl/monitor.c +0
-1@@ -320,7 +320,6 @@ static int __mon_event_count(u32 rmid, struct rmid_read *rr) } chunks =mbm_overflow_count (m->prev_msr, tval, rr->r->mbm_width); - m->chunks += chunks; m->prev_msr = tval; rr->val +=get_corrected_mbm_count (rmid, m->chunks);
Hmm, zapping this one. First, there's an unused variable warning:
https://lkml.kernel.org/r/[email protected]
and you should remove the chunks assignment too.
And then it didn't apply cleanly:
$ test-apply.sh /tmp/xiaochen.01 checking file arch/x86/kernel/cpu/resctrl/monitor.c Hunk #1 FAILED at 279. Hunk #2 succeeded at 514 (offset 64 lines). 1 out of 2 hunks FAILED
I wiggled it in but it ended up removing the wrong chunks inc line - not the one in mbm_bw_count() but in __mon_event_count() - which I just realized.
So please redo this patch against:
https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/log/?h=x86/cache
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteBorislav Petkov
maintainerQuoted 1 line · Borislav Petkov
and you should remove the chunks assignment too.
Yah, ignore that part - mbm_bw_count() does need chunks for cur_bw. Oh well.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteHi Boris,
Quoted 11 lines · Borislav Petkov
And then it didn't apply cleanly:
$ test-apply.sh /tmp/xiaochen.01 checking file arch/x86/kernel/cpu/resctrl/monitor.c Hunk #1 FAILED at 279. Hunk #2 succeeded at 514 (offset 64 lines). 1 out of 2 hunks FAILEDI wiggled it in but it ended up removing the wrong chunks inc line - not the one in
mbm_bw_count()but in__mon_event_count()- which I just realized.
Thank you for clarifying this issue. It is not a 0-DAY CI issue.
Quoted 6 lines
So please redo this patch against:
https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/log/?h=x86/cache
Thx.
I will send a patch against branch tip: x86/cache soon. Thank you.
Signature · 2 lines
Best regards, Xiaochen
Borislav Petkov
maintainerQuoted 1 line · Xiaochen Shen
Thank you for clarifying this issue. It is not a 0-DAY CI issue.
Which begs the question: this patch should be Cc: stable and should go in now, shouldn't it?
Because then the first submission applies cleanly ontop of tip:x86/urgent.
I mean, it is fixing only reporting but that reporting is kinda waaay off.
Hmmm?
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteHi Boris,
Quoted 5 lines · Borislav Petkov
Quoted 1 line · Xiaochen Shen
Thank you for clarifying this issue. It is not a 0-DAY CI issue.
Which begs the question: this patch should be Cc: stable and should go in now, shouldn't it?
Quoted 8 lines
Because then the first submission applies cleanly ontop of tip:x86/urgent.
I mean, it is fixing only reporting but that reporting is kinda waaay off.
Hmmm?
Yes. Thank you for pointing out this, we'd better add Cc: stable tag. But we have to backport the patch for -stable trees because the code base is changed by following upstream commits:
(1) For all stable trees: in commit abe8f12b4425 ("x86/resctrl: Remove unused struct mbm_state::chunks_bw"), removing unused struct mbm_state::chunks_bw changes the code base for this patch.
(2) For 4.14 and 4.19 stable trees: in commit fa7d949337cc ("x86/resctrl:
Rename and move rdt files to a separate directory"), the file arch/x86/kernel/cpu/intel_rdt_monitor.c has been renamed and moved to arch/x86/kernel/cpu/resctrl/monitor.c. This patch needs to be rebased.
I plan to do backporting for all -stable trees after this patch is merged into upstream.
Thank you.
Signature · 2 lines
Best regards, Xiaochen
Borislav Petkov
maintainerQuoted 2 lines · Xiaochen Shen
I plan to do backporting for all -stable trees after this patch is merged into upstream.
Ok, you can do that when Greg sends the mails that it cannot apply to the respective trees. Lemme queue this one to urgent now.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquettePATCH v2 1/1 x86/resctrl: Fix incorrect local bandwidth when mba_sc is enabled
In reply to Borislav PetkovMBA software controller (mba_sc) is a feedback loop which periodically reads MBM counters and tries to restrict the bandwidth below a user specified bandwidth. It tags along MBM counter overflow handler to do the updates with 1s interval in mbm_update() and update_mba_bw().
The purpose of mbm_update() is to periodically read the MBM counters to make sure that the hardware counter doesn't wrap around more than once between user samplings. mbm_update() calls __mon_event_count() for local bandwidth updating when mba_sc is not enabled, but calls mbm_bw_count() instead when mba_sc is enabled. __mon_event_count() will not be called for local bandwidth updating in MBM counter overflow handler, but it is still called when reading MBM local bandwidth counter file 'mbm_local_bytes', the call path is as below:
rdtgroup_mondata_show()
mon_event_read()
mon_event_count()
__mon_event_count()In __mon_event_count(), m->chunks is updated by delta chunks which is calculated from previous MSR value (m->prev_msr) and current MSR value. When mba_sc is enabled, m->chunks is also updated in mbm_update() by mistake by the delta chunks which is calculated from m->prev_bw_msr instead of m->prev_msr. But m->chunks is not used in update_mba_bw() in the mba_sc feedback loop.
When reading MBM local bandwidth counter file, m->chunks was changed unexpectedly by mbm_bw_count(). As a result, the incorrect local bandwidth counter which calculated from incorrect m->chunks is read out to the user.
Fix this by removing incorrect m->chunks updating in mbm_bw_count() in MBM counter overflow handler, and always calling __mon_event_count() in mbm_update() to make sure that the hardware local bandwidth counter doesn't wrap around.
Test steps:
# Run workload with aggressive memory bandwidth (e.g., 10 GB/s) git clone https://github.com/intel/intel-cmt-cat && cd intel-cmt-cat && make
./tools/membw/membw -c 0 -b 10000 --read
# Enable MBA software controller
mount -t resctrl resctrl -o mba_MBps /sys/fs/resctrl
# Create control group c1
mkdir /sys/fs/resctrl/c1
# Set MB throttle to 6 GB/s
echo "MB:0=6000;1=6000" > /sys/fs/resctrl/c1/schemata
# Write PID of the workload to tasks file
echo pidof membw > /sys/fs/resctrl/c1/tasks
# Read local bytes counters twice with 1s interval, the calculated
# local bandwidth is not as expected (approaching to 6 GB/s): local_1=`cat /sys/fs/resctrl/c1/mon_data/mon_L3_00/mbm_local_bytes` sleep 1 local_2=`cat /sys/fs/resctrl/c1/mon_data/mon_L3_00/mbm_local_bytes` echo "local b/w (bytes/s):" `expr $local_2 - $local_1`
Before fix:
local b/w (bytes/s): 11076796416
After fix:
local b/w (bytes/s): 5465014272
--- arch/x86/kernel/cpu/resctrl/monitor.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/resctrl/monitor.c +2 -4
@@ -343,7 +343,6 @@ static void mbm_bw_count(u32 rmid, struct rmid_read *rr)return ; chunks =mbm_overflow_count (m->prev_bw_msr, tval, rr->r->mbm_width); - m->chunks += chunks; cur_bw = (get_corrected_mbm_count (rmid, chunks) * r->mon_scale) >>20 ;if (m->delta_comp)
@@ -514,15 +513,14 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) }if (is_mbm_local_enabled ()) { rr.evtid =QOS_L3_MBM_LOCAL_EVENT_ID ; +__mon_event_count (rmid, &rr);/* * Call the MBA software controller only for the * control groups and when user has enabled * the software controller explicitly. */ -if (!is_mba_sc (NULL )) -__mon_event_count (rmid, &rr); -else +if (is_mba_sc (NULL ))mbm_bw_count (rmid, &rr); } }
Signature · 1 line
1.8.3.1
The following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 06c5fe9b12dde1b62821f302f177c972bb1c81f9 Gitweb: https://git.kernel.org/tip/06c5fe9b12dde1b62821f302f177c972bb1c81f9 Author: Xiaochen Shen <[email protected]> AuthorDate: Fri, 04 Dec 2020 14:27:59 +08:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Thu, 10 Dec 2020 17:52:37 +01:00
x86/resctrl: Fix incorrect local bandwidth when mba_sc is enabled
The MBA software controller (mba_sc) is a feedback loop which periodically reads MBM counters and tries to restrict the bandwidth below a user-specified value. It tags along the MBM counter overflow handler to do the updates with 1s interval in mbm_update() and update_mba_bw().
The purpose of mbm_update() is to periodically read the MBM counters to make sure that the hardware counter doesn't wrap around more than once between user samplings. mbm_update() calls __mon_event_count() for local bandwidth updating when mba_sc is not enabled, but calls mbm_bw_count() instead when mba_sc is enabled. __mon_event_count() will not be called for local bandwidth updating in MBM counter overflow handler, but it is still called when reading MBM local bandwidth counter file 'mbm_local_bytes', the call path is as below:
rdtgroup_mondata_show()
mon_event_read()
mon_event_count()
__mon_event_count()In __mon_event_count(), m->chunks is updated by delta chunks which is calculated from previous MSR value (m->prev_msr) and current MSR value. When mba_sc is enabled, m->chunks is also updated in mbm_update() by mistake by the delta chunks which is calculated from m->prev_bw_msr instead of m->prev_msr. But m->chunks is not used in update_mba_bw() in the mba_sc feedback loop.
When reading MBM local bandwidth counter file, m->chunks was changed unexpectedly by mbm_bw_count(). As a result, the incorrect local bandwidth counter which calculated from incorrect m->chunks is shown to the user.
Fix this by removing incorrect m->chunks updating in mbm_bw_count() in MBM counter overflow handler, and always calling __mon_event_count() in mbm_update() to make sure that the hardware local bandwidth counter doesn't wrap around.
Test steps:
# Run workload with aggressive memory bandwidth (e.g., 10 GB/s) git clone https://github.com/intel/intel-cmt-cat && cd intel-cmt-cat && make
./tools/membw/membw -c 0 -b 10000 --read
# Enable MBA software controller
mount -t resctrl resctrl -o mba_MBps /sys/fs/resctrl
# Create control group c1
mkdir /sys/fs/resctrl/c1
# Set MB throttle to 6 GB/s
echo "MB:0=6000;1=6000" > /sys/fs/resctrl/c1/schemata
# Write PID of the workload to tasks file
echo pidof membw > /sys/fs/resctrl/c1/tasks
# Read local bytes counters twice with 1s interval, the calculated
# local bandwidth is not as expected (approaching to 6 GB/s): local_1=`cat /sys/fs/resctrl/c1/mon_data/mon_L3_00/mbm_local_bytes` sleep 1 local_2=`cat /sys/fs/resctrl/c1/mon_data/mon_L3_00/mbm_local_bytes` echo "local b/w (bytes/s):" `expr $local_2 - $local_1`
Before fix:
local b/w (bytes/s): 11076796416
After fix:
local b/w (bytes/s): 5465014272
--- arch/x86/kernel/cpu/resctrl/monitor.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/resctrl/monitor.c +2 -4
@@ -279,7 +279,6 @@ static void mbm_bw_count(u32 rmid, struct rmid_read *rr)return ; chunks =mbm_overflow_count (m->prev_bw_msr, tval, rr->r->mbm_width); - m->chunks += chunks; cur_bw = (chunks * r->mon_scale) >>20 ;if (m->delta_comp)
@@ -450,15 +449,14 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) }if (is_mbm_local_enabled ()) { rr.evtid =QOS_L3_MBM_LOCAL_EVENT_ID ; +__mon_event_count (rmid, &rr);/* * Call the MBA software controller only for the * control groups and when user has enabled * the software controller explicitly. */ -if (!is_mba_sc (NULL )) -__mon_event_count (rmid, &rr); -else +if (is_mba_sc (NULL ))mbm_bw_count (rmid, &rr); } }