x86/resctrl: Don't try to free nonexistent RMIDs
Subsystems (2)
Dave Martin
reviewerPATCH 1/1 x86/resctrl: Don't try to free nonexistent RMIDs
Commit 6791e0ea3071 ("x86/resctrl: Access per-rmid structures by index") adds logic to map individual monitoring groups into a global index space used for tracking allocated RMIDs.
That patch keept the logic to ignore requests to free the default RMID in free_rmid(), and this works fine on x86.
With arm64 MPAM, there is a latent bug here: on platforms with no monitors exposed through resctrl, each control group still gets a different monitoring group ID as seen by the hardware, since the CLOSID always forms part of the monitoring group ID. This means that when removing a control group, the code may try to free this group's default monitoring group RMID for real. If there are no monitors however, the RMID tracking table rmid_ptrs[] would be a waste of memory and is never allocated, leading to a splat when a free_rmid() tries to dereference the table.
One option would be to treat RMID 0 as special for every CLOSID, but this would be ugly since we still want to do bookkeeping for these monitoring group IDs when there are monitors present in the hardware.
Instead, add a gating check of resctrl_arch_mon_capable() in free_rmid(), and just do nothing if the hardware doesn't have monitors.
This fix mirrors the gating checks already present in mkdir_rdt_prepare_rmid_alloc() and elsewhere.
No functional change on x86.
---
Based on v6.10-rc3.
Tested on x86 (But so far for the monitors-present case. Testing on Atom would be appreciated.)
Tested on arm64 for the no-monitors case.
--- arch/x86/kernel/cpu/resctrl/monitor.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/monitor.c +2 -1
@@ -519,7 +519,8 @@ void free_rmid(u32 closid, u32 rmid)* allows architectures that ignore the closid parameter to avoid an * unnecessary check. */ -if (idx ==resctrl_arch_rmid_idx_encode (RESCTRL_RESERVED_CLOSID , +if (!resctrl_arch_mon_capable () || + idx ==resctrl_arch_rmid_idx_encode (RESCTRL_RESERVED_CLOSID ,RESCTRL_RESERVED_RMID ))return ;
base-commit: 83a7eefedc9b56fe7bfeff13b6c7356688ffa670
Signature · 1 line
2.34.1
Reinette Chatre
maintainerHi Dave,
Quoted 5 lines · Dave Martin
Commit
6791e0ea3071("x86/resctrl: Access per-rmid structures by index") adds logic to map individual monitoring groups into a global index space used for tracking allocated RMIDs.That patch keept the logic to ignore requests to free the default
keept -> kept
nitpick: I actually do not know if "that patch" gets same hate as "this patch" so to avoid any potential feedback about this I'd like to suggest that this is rewritten without this term. Perhaps something like: "Requests to free the default RMID in free_rmid() are ignored, and this works fine on x86."
Quoted 26 lines
RMID in
free_rmid(), and this works fine on x86.With arm64 MPAM, there is a latent bug here: on platforms with no monitors exposed through resctrl, each control group still gets a different monitoring group ID as seen by the hardware, since the CLOSID always forms part of the monitoring group ID. This means that when removing a control group, the code may try to free this group's default monitoring group RMID for real. If there are no monitors however, the RMID tracking table rmid_ptrs[] would be a waste of memory and is never allocated, leading to a splat when a
free_rmid()tries to dereference the table.One option would be to treat RMID 0 as special for every CLOSID, but this would be ugly since we still want to do bookkeeping for these monitoring group IDs when there are monitors present in the hardware.
Instead, add a gating check of
resctrl_arch_mon_capable()infree_rmid(), and just do nothing if the hardware doesn't have monitors.This fix mirrors the gating checks already present in
mkdir_rdt_prepare_rmid_alloc()and elsewhere.No functional change on x86.
Quoted 8 lines
Fixes 6791e0ea3071 ("x86/resctrl: Access per-rmid structures by index")Signed-off-by Dave Martin <[email protected]>---
Based on v6.10-rc3.
Tested on x86 (But so far for the monitors-present case.
Tested by booting with "rdt=!cmt,!mbmtotal,!mbmlocal".
Quoted 13 lines
Testing on Atom would be appreciated.)
Tested on arm64 for the no-monitors case.
--- arch/x86/kernel/cpu/resctrl/monitor.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/x86/kernel/cpu/resctrl/monitor.c +0
-0
arch/x86/kernel/cpu/resctrl/monitor.c void free_rmid(u32 closid, u32 rmid) · line 526
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/monitor.c +2
-1@@ -519,7 +519,8 @@ void free_rmid(u32 closid, u32 rmid)* allows architectures that ignore the closid parameter to avoid an * unnecessary check. */ -if (idx ==resctrl_arch_rmid_idx_encode (RESCTRL_RESERVED_CLOSID , +if (!resctrl_arch_mon_capable () || + idx ==resctrl_arch_rmid_idx_encode (RESCTRL_RESERVED_CLOSID ,RESCTRL_RESERVED_RMID ))return ;
Thank you for catching this. This reveals how subtle resctrl is by calling free_rmid() in paths for convenience and then relying on an uninitialized variable for correct behavior.
This replaces this subtle assumption with an actual check.
With changelog comments addressed:
| Reviewed-by: Reinette Chatre <[email protected]>
Reinette
Dave Martin
reviewerHi Reinette,
Quoted 9 lines · Reinette Chatre
Hi Dave,
Quoted 5 lines · Dave Martin
Commit
6791e0ea3071("x86/resctrl: Access per-rmid structures by index") adds logic to map individual monitoring groups into a global index space used for tracking allocated RMIDs.That patch keept the logic to ignore requests to free the default
Quoted 10 lines
keept -> kept
nitpick: I actually do not know if "that patch" gets same hate as "this patch" so to avoid any potential feedback about this I'd like to suggest that this is rewritten without this term. Perhaps something like: "Requests to free the default RMID in
free_rmid()are ignored, and this works fine on x86."Quoted 2 lines
RMID in
free_rmid(), and this works fine on x86.
How about recasting the first paragraph into the past tense (since it relates a past commit), and rewording as "Requests to free the default RMID continued to be ignored in free_rmid(), and this works fine on x86."
(I agree that "this patch" would have been ambiguous. "That patch" was an attempt to be clearer, but felt a bit clumsy. Naming the commit again felt worse, though would have been clearer. I've noticed that people who do not have English as their first language tend to use "this" and "that" a little differently from native English speakers, so there is probably more scope for confusion here that I like to assume...)
Quoted 27 lines
Quoted 27 lines
With arm64 MPAM, there is a latent bug here: on platforms with no monitors exposed through resctrl, each control group still gets a different monitoring group ID as seen by the hardware, since the CLOSID always forms part of the monitoring group ID. This means that when removing a control group, the code may try to free this group's default monitoring group RMID for real. If there are no monitors however, the RMID tracking table rmid_ptrs[] would be a waste of memory and is never allocated, leading to a splat when a
free_rmid()tries to dereference the table.One option would be to treat RMID 0 as special for every CLOSID, but this would be ugly since we still want to do bookkeeping for these monitoring group IDs when there are monitors present in the hardware.
Instead, add a gating check of
resctrl_arch_mon_capable()infree_rmid(), and just do nothing if the hardware doesn't have monitors.This fix mirrors the gating checks already present in
mkdir_rdt_prepare_rmid_alloc()and elsewhere.No functional change on x86.
Fixes 6791e0ea3071 ("x86/resctrl: Access per-rmid structures by index")Signed-off-by Dave Martin <[email protected]>
Quoted 7 lines
Quoted 5 lines
---
Based on v6.10-rc3.
Tested on x86 (But so far for the monitors-present case.
Tested by booting with "rdt=!cmt,!mbmtotal,!mbmlocal".
Thanks (I take it that's your test, not a request to be more specific about mine?)
As it happens I tested with rdt=cmt,mbmtotal,mbmlocal,l3cat,l3cdp (though I made no effort to exercise these features other than running the selftests). I can note this in the commit if you prefer.
Quoted 27 lines
Quoted 13 lines
Testing on Atom would be appreciated.)
Tested on arm64 for the no-monitors case.
--- arch/x86/kernel/cpu/resctrl/monitor.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/x86/kernel/cpu/resctrl/monitor.c +0
-0arch/x86/kernel/cpu/resctrl/monitor.c void free_rmid(u32 closid, u32 rmid) · line 526
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/monitor.c +2
-1@@ -519,7 +519,8 @@ void free_rmid(u32 closid, u32 rmid)* allows architectures that ignore the closid parameter to avoid an * unnecessary check. */ -if (idx ==resctrl_arch_rmid_idx_encode (RESCTRL_RESERVED_CLOSID , +if (!resctrl_arch_mon_capable () || + idx ==resctrl_arch_rmid_idx_encode (RESCTRL_RESERVED_CLOSID ,RESCTRL_RESERVED_RMID ))return ;Thank you for catching this. This reveals how subtle resctrl is by calling
free_rmid()in paths for convenience and then relying on an uninitialized variable for correct behavior.
Quoted 6 lines
This replaces this subtle assumption with an actual check.
With changelog comments addressed:
| Reviewed-by: Reinette Chatre <[email protected]>Reinette
Thanks for that. I'll wait for your reply before respinning this.
Cheers
---Dave
Hi Dave,
Quoted 20 lines · Dave Martin
Hi Reinette,
Quoted 17 lines · Reinette Chatre
Hi Dave,
Quoted 5 lines · Dave Martin
Commit
6791e0ea3071("x86/resctrl: Access per-rmid structures by index") adds logic to map individual monitoring groups into a global index space used for tracking allocated RMIDs.That patch keept the logic to ignore requests to free the default
keept -> kept
nitpick: I actually do not know if "that patch" gets same hate as "this patch" so to avoid any potential feedback about this I'd like to suggest that this is rewritten without this term. Perhaps something like: "Requests to free the default RMID in
free_rmid()are ignored, and this works fine on x86."
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
RMID in
free_rmid(), and this works fine on x86.How about recasting the first paragraph into the past tense (since it relates a past commit), and rewording as "Requests to free the default RMID continued to be ignored in
free_rmid(), and this works fine on x86."
Please keep it in the present tense. I do not see this as relating to a "past commit" but instead it is an existing commit responsible for current behavior. Documentation/process/maintainer-tip.rst contains some example changelogs created by x86 maintainers that captures their requirements. The beginning "context" portion is always in present tense.
Quoted 38 lines
(I agree that "this patch" would have been ambiguous. "That patch" was an attempt to be clearer, but felt a bit clumsy. Naming the commit again felt worse, though would have been clearer. I've noticed that people who do not have English as their first language tend to use "this" and "that" a little differently from native English speakers, so there is probably more scope for confusion here that I like to assume...)
Quoted 29 lines
Quoted 29 lines
With arm64 MPAM, there is a latent bug here: on platforms with no monitors exposed through resctrl, each control group still gets a different monitoring group ID as seen by the hardware, since the CLOSID always forms part of the monitoring group ID. This means that when removing a control group, the code may try to free this group's default monitoring group RMID for real. If there are no monitors however, the RMID tracking table rmid_ptrs[] would be a waste of memory and is never allocated, leading to a splat when a
free_rmid()tries to dereference the table.One option would be to treat RMID 0 as special for every CLOSID, but this would be ugly since we still want to do bookkeeping for these monitoring group IDs when there are monitors present in the hardware.
Instead, add a gating check of
resctrl_arch_mon_capable()infree_rmid(), and just do nothing if the hardware doesn't have monitors.This fix mirrors the gating checks already present in
mkdir_rdt_prepare_rmid_alloc()and elsewhere.No functional change on x86.
Fixes 6791e0ea3071 ("x86/resctrl: Access per-rmid structures by index")Signed-off-by Dave Martin <[email protected]>---
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Based on v6.10-rc3.
Tested on x86 (But so far for the monitors-present case.
Tested by booting with "rdt=!cmt,!mbmtotal,!mbmlocal".
Thanks (I take it that's your test, not a request to be more specific about mine?)
Yes, I did test it with those parameters. You are also welcome to add:
Quoted 4 lines
As it happens I tested with rdt=cmt,mbmtotal,mbmlocal,l3cat,l3cdp (though I made no effort to exercise these features other than running the selftests). I can note this in the commit if you prefer.
hmmm ... those parameters should not be necessary unless the system has those features forced off by default because of errata. Doing functional testing on these systems via such enabling is fine though.
Reinette
Dave Martin
reviewerHi,
Quoted 31 lines · Reinette Chatre
Hi Dave,
Quoted 20 lines · Dave Martin
Hi Reinette,
Quoted 17 lines · Reinette Chatre
Hi Dave,
Quoted 5 lines · Dave Martin
Commit
6791e0ea3071("x86/resctrl: Access per-rmid structures by index") adds logic to map individual monitoring groups into a global index space used for tracking allocated RMIDs.That patch keept the logic to ignore requests to free the default
keept -> kept
nitpick: I actually do not know if "that patch" gets same hate as "this patch" so to avoid any potential feedback about this I'd like to suggest that this is rewritten without this term. Perhaps something like: "Requests to free the default RMID in
free_rmid()are ignored, and this works fine on x86."Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
RMID in
free_rmid(), and this works fine on x86.How about recasting the first paragraph into the past tense (since it relates a past commit), and rewording as "Requests to free the default RMID continued to be ignored in
free_rmid(), and this works fine on x86."
Quoted 6 lines
Please keep it in the present tense. I do not see this as relating to a "past commit" but instead it is an existing commit responsible for current behavior.
Documentation/process/maintainer-tip.rstcontains some example changelogs created by x86 maintainers that captures their requirements. The beginning "context" portion is always in present tense.
Fair enough. I'll reword this as per your suggestion.
[...]
Quoted 10 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Tested on x86 (But so far for the monitors-present case.
Tested by booting with "rdt=!cmt,!mbmtotal,!mbmlocal".
Thanks (I take it that's your test, not a request to be more specific about mine?)
Yes, I did test it with those parameters. You are also welcome to add:
Tested-by Reinette Chatre <[email protected]>
Ah, right. Thanks.
Quoted 10 lines
Quoted 3 lines
As it happens I tested with rdt=cmt,mbmtotal,mbmlocal,l3cat,l3cdp (though I made no effort to exercise these features other than running the selftests). I can note this in the commit if you prefer.
hmmm ... those parameters should not be necessary unless the system has those features forced off by default because of errata. Doing functional testing on these systems via such enabling is fine though.
Reinette
I got these kernel args from James. Apparently some of these features are defaulted to off by the firmware on the box I'm using, but we're not sure why. I haven't dug into it.
Cheers
---Dave
Reinette Chatre
maintainerHi Dave,
Quoted 12 lines · Dave Martin
Quoted 11 lines · Reinette Chatre
Quoted 3 lines · Dave Martin
As it happens I tested with rdt=cmt,mbmtotal,mbmlocal,l3cat,l3cdp (though I made no effort to exercise these features other than running the selftests). I can note this in the commit if you prefer.
hmmm ... those parameters should not be necessary unless the system has those features forced off by default because of errata. Doing functional testing on these systems via such enabling is fine though.
Quoted 5 lines
Quoted 1 line
Reinette
I got these kernel args from James. Apparently some of these features are defaulted to off by the firmware on the box I'm using, but we're not sure why. I haven't dug into it.
resctrl does not provide capability to override features that the hardware does not advertise as supporting. resctrl does sometimes force hardware advertised features off because of errata (for example, see [1]) and the kernel "rdt=" parameter can be used to override that. This same option also enables administrators to ignore some features on their platforms.
Reinette
Dave Martin
reviewerHi Reinette,
Quoted 27 lines · Reinette Chatre
Hi Dave,
Quoted 12 lines · Dave Martin
Quoted 11 lines · Reinette Chatre
Quoted 3 lines · Dave Martin
As it happens I tested with rdt=cmt,mbmtotal,mbmlocal,l3cat,l3cdp (though I made no effort to exercise these features other than running the selftests). I can note this in the commit if you prefer.
hmmm ... those parameters should not be necessary unless the system has those features forced off by default because of errata. Doing functional testing on these systems via such enabling is fine though.
Quoted 5 lines
Quoted 1 line
Reinette
I got these kernel args from James. Apparently some of these features are defaulted to off by the firmware on the box I'm using, but we're not sure why. I haven't dug into it.
resctrl does not provide capability to override features that the hardware does not advertise as supporting. resctrl does sometimes force hardware advertised features off because of errata (for example, see [1]) and the kernel "rdt=" parameter can be used to override that. This same option also enables administrators to ignore some features on their platforms.
Quoted 3 lines
Reinette
Possibly the box in question falls under this case. I'll need to check with James and/or experiment.
Either way, I don't think this impacts the tested-ness of this patch, but please shout if you have concerns.
Cheers
---Dave
Reinette Chatre
maintainerHi Dave,
Quoted 4 lines · Dave Martin
Either way, I don't think this impacts the tested-ness of this patch, but please shout if you have concerns.
I agree. The submission stated that it was tested on x86 "for the monitors-present case". When booting with "rdt=!cmt,!mbmtotal,!mbmlocal" (as I did in my test) resctrl is not able to discover monitoring support and it thus tested the "monitors _not_ present case".
Reinette
Dave Martin
reviewerHi,
Quoted 9 lines · Reinette Chatre
Hi Dave,
Quoted 4 lines · Dave Martin
Either way, I don't think this impacts the tested-ness of this patch, but please shout if you have concerns.
Quoted 7 lines
I agree. The submission stated that it was tested on x86 "for the monitors-present case". When booting with "rdt=!cmt,!mbmtotal,!mbmlocal" (as I did in my test) resctrl is not able to discover monitoring support and it thus tested the "monitors _not_ present case".
Reinette
Ah, right. I missed the significance of the options.
Thanks again!
---Dave