x86,fs/resctrl: Improve resctrl quality and consistency
Subsystems (2)
Reinette Chatre
maintainerv5 cover x86,fs/resctrl: Improve resctrl quality and consistency
Applies on top of v7.2-rc1 + v5 of "x86,fs/resctrl: Fix long-standing issues":
https://lore.kernel.org/lkml/[email protected]/
Changes since v4:
- v4: https://lore.kernel.org/lkml/[email protected]/
- Rebased on top of V5 "x86,fs/resctrl: Fix long-standing issues":
https://lore.kernel.org/lkml/[email protected]/
A significant change in that series is the addition of info_kn_lock()/info_kn_unlock() helpers to the info file handlers. With the addition of locking to the max_threshold_occ_write() in that series it seemed appropriate to re-order the last_cmd_status handling in max_threshold_occ_write() to be after the change to last_cmd_status custom to contain max_threshold_occ_write() changes to one patch.
- Add resctrl filename to new error messages to disambiguate the various "Invalid input" messages. (Tony)
- Add Tony's Reviewed-by tag to all patches.
Changes since v3:
- v3: https://lore.kernel.org/lkml/[email protected]/
- Rebased on v7.1-rc1.
- Drop "MAINTAINERS: Update resctrl entry" and "fs/resctrl: Add missing return value descriptions" that can be found in v7.1-rc1.
- Drop "fs/resctrl: Avoid "may be used uninitialized" warning" that just does unnecessary work to appease compiler warning that is disabled by default. (Boris)
- Use pr_warn() instead of WARN_ON() in "x86/resctrl: Protect against bad shift".
Changes since v2:
- v2: https://lore.kernel.org/lkml/[email protected]/
- Rebased on top of tip/master with pending resctrl selftest changes,
https://lore.kernel.org/lkml/[email protected]/,
applied on top to support testing with latest kernel that has pending resctrl changes merged.
- Drop "fs/resctrl: Use stricter checks on input to cpus/cpus_list file" that ended up being too strict and removed one use case. (Chris)
- max_threshold_occ_write() switch to guard(). (Chenyu)
- Add more detail to one return value description.
- Add tags.
- No opinions so far on needing to update last_cmd_status on success of all read commands. At this time last_cmd_status will continue to return previous failure message (if any) after a read command accessing static data succeeds.
Changes since v1:
- v1: https://lore.kernel.org/lkml/[email protected]/
- To simplify tracking, include patch to MAINTAINERS submitted separately:
https://lore.kernel.org/lkml/4274c478922c01f9ceebc805acf991f10a95519f.1771442788.git.reinette.chatre@intel.com/
- Follow recent upstream changes to add reference to tip tree handbook in MAINTAINERS entry.
- Use new pattern in resctrl to enable looping over all entries of an enum while making adding new enum entries more clear all while avoiding warnings when compiling with -Wswitch. (Ben)
- Add another last_cmd_status enhancement that appends "[truncated]" to output of info/last_cmd_status if the backing buffer overflowed.
- Please see patch changelogs for details of changes.
- Add tags.
Hi Everybody,
This is a collection of resctrl cleanups assembled together for convenience and simpler tracking. I'd be happy to split them up if it makes review and/or handling easier.
Summary of changes:
- Let resctrl pass stricter checks from various tools to provide a cleaner baseline with the goal to promote healthier contributions:
- ./tools/docs/kernel-doc -Wall -v <files>
- Build with W=12
- ./scripts/coccicheck
- Static checkers
- Use accurate and consistent type for all uses of resource ID.
- In the unlikely scenario that resctrl picked a wrong CPU to read an event from, pass the error through to user space instead of claiming to succeed and returning a (wrong) result.
- Since inception of last_cmd_status feature there have been mismatches between resctrl file operation failures and the contents of info/last_cmd_status. This pattern keeps propagating with each new resctrl feature. Establish a new baseline with a new pattern that ensures info/last_cmd_status contains an accurate failure description that matches the most recent resctrl file operation failure.
One potential open:
There remains an inconsistency between resctrl file operations that do/can _not_ fail and the contents of info/last_cmd_status. If a resctrl file operation fails and an informational error is printed to last_cmd_status then a subsequent reading of a resctrl file (specifically most of the files found in info/) may succeed while info/last_cmd_status may or may not return the error from previous failure.
Ensuring last_cmd_status is reset on every read carries the cost of taking rdtgroup_mutex on several more user space initiated paths and thus increase contention on rdtgroup_mutex. I opted to not make this change and instead focus this work on ensuring that last_cmd_status is accurate whenever there is a failure during any resctrl file operation. Please let me know if you have opinions in this regard.
Update v5: The addition of the info_kn_lock()/info_kn_unlock() helpers to info file handlers now means that rdtgroup_mutex is held every time an info file is opened. There is thus now less burden to reset last_cmd_status buffer on each info file access. Even so, since there has not been any opinions in this regard I opted to keep current behavior.
Any feedback is appreciated.
Regards,
Reinette
Reinette Chatre (10):
fs/resctrl: Use correct format specifier for printing error pointers x86/resctrl: Protect against bad shift
fs/resctrl: Change pattern used to track number of entries in enum fs/resctrl: Use accurate type for rdt_resource::rid fs/resctrl: Pass error reading event through to user space fs/resctrl: Use accurate and symmetric exit flows fs/resctrl: Change last_cmd_status custom during input parsing fs/resctrl: Add last_cmd_status support for writes to
max_threshold_occupancy
fs/resctrl: Communicate resource group deleted error via
last_cmd_status
fs/resctrl: Inform user space when status buffer overflowed
arch/x86/kernel/cpu/resctrl/core.c | 6 +- fs/resctrl/ctrlmondata.c | 71 ++++++++++------- fs/resctrl/monitor.c | 78 ++++++++++-------- fs/resctrl/rdtgroup.c | 123 +++++++++++++++++------------ include/linux/resctrl.h | 11 +-- 5 files changed, 173 insertions(+), 116 deletions(-)
Signature · 1 line
2.54.0
Reinette Chatre
maintainerPATCH v5 1/10 fs/resctrl: Use correct format specifier for printing error pointers
Use correct format specifier for error pointer as coccinelle suggests:
.../fs/resctrl/monitor.c:148:8-15: WARNING: Consider using %pe to print PTR_ERR()
.../fs/resctrl/monitor.c:760:9-16: WARNING: Consider using %pe to print PTR_ERR()---
Changes since v4:
- Add Tony's Reviewed-by tag.
--- fs/resctrl/monitor.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
fs/resctrl/monitor.c +4 -4
@@ -144,8 +144,8 @@ void __check_limbo(struct rdt_l3_mon_domain *d, bool force_free) arch_priv = mon_event_all[QOS_L3_OCCUP_EVENT_ID ].arch_priv; arch_mon_ctx =resctrl_arch_mon_ctx_alloc (r,QOS_L3_OCCUP_EVENT_ID );if (IS_ERR (arch_mon_ctx)) { -pr_warn_ratelimited ("Failed to allocate monitor context: %ld" , -PTR_ERR (arch_mon_ctx)); +pr_warn_ratelimited ("Failed to allocate monitor context: %pe" , + arch_mon_ctx);return ; }
@@ -771,8 +771,8 @@ static void mbm_update_one_event(struct rdt_resource *r, struct rdt_l3_mon_domai }else { rr.arch_mon_ctx =resctrl_arch_mon_ctx_alloc (rr.r, evtid);if (IS_ERR (rr.arch_mon_ctx)) { -pr_warn_ratelimited ("Failed to allocate monitor context: %ld" , -PTR_ERR (rr.arch_mon_ctx)); +pr_warn_ratelimited ("Failed to allocate monitor context: %pe" , + rr.arch_mon_ctx);return ; } }
Signature · 1 line
2.54.0
Reinette Chatre
maintainerPATCH v5 2/10 x86/resctrl: Protect against bad shift
In reply to Reinette ChatreThe size of the bandwidth specifier field is enumerated from AMD hardware. resctrl uses this field width to determine the maximum bandwidth supported that is stored in resctrl_membw::max_bw to which user space allocation requests are compared for validity.
resctrl_membw::max_bw is of type u32 while the register containing the bandwidth specifier field, L3QOS_BW_CONTROL_n, is 64 bits. While not an issue with current hardware it is theoretically possible that enumeration of maximum bandwidth may trigger invalid behavior if a future system can use a bandwidth specifier field larger than 32 bits. Whether this could ever represent a reasonable bandwidth value is unknown but addressing the issue will appease static checkers.
Ensure resctrl can accommodate the hardware's bandwidth specifier field width with an additional check. Switch to BIT() instead of open-coding the bitshift to avoid signed integer overflow if the number of bits is a valid 31.
---
Changes since v3:
- Use pr_warn() instead of WARN_ON().
sashiko: https://sashiko.dev/#/patchset/cover.1775576382.git.reinette.chatre%40intel.com
Changes since v4:
- Add Tony's Reviewed-by tag.
--- arch/x86/kernel/cpu/resctrl/core.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/core.c +5 -1
@@ -247,7 +247,11 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)cpuid_count (0x80000020 , subleaf, &eax, &ebx, &ecx, &edx); hw_res->num_closid = edx +1 ; - r->membw.max_bw =1 << eax; +if (BITS_PER_TYPE (r->membw.max_bw) <= eax) { +pr_warn ("Unable to support hardware's maximum bandwidth\n" ); +return false; + } + r->membw.max_bw =BIT (eax);/* AMD does not use delay */ r->membw.delay_linear = false;
Signature · 1 line
2.54.0
Borislav Petkov
maintainerRe: [PATCH v5 02/10] x86/resctrl: Protect against bad shift
Quoted 5 lines · Reinette Chatre
arch/x86/kernel/cpu/resctrl/core.c +0
-0
arch/x86/kernel/cpu/resctrl/core.c static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r) · line 254
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/core.c +5
-1@@ -247,7 +247,11 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)cpuid_count (0x80000020 , subleaf, &eax, &ebx, &ecx, &edx); hw_res->num_closid = edx +1 ; - r->membw.max_bw =1 << eax; +if (BITS_PER_TYPE (r->membw.max_bw) <= eax) { +pr_warn ("Unable to support hardware's maximum bandwidth\n" ); +return false; + } + r->membw.max_bw =BIT (eax);
Why?
Why go all the effort if the field can be 64 bits?
Why not make max_bw u64 and handle *all* possible values naturally?
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteReinette Chatre
maintainerRe: [PATCH v5 02/10] x86/resctrl: Protect against bad shift
Hi Boris,
Quoted 18 lines · Borislav Petkov
Quoted 5 lines · Reinette Chatre
arch/x86/kernel/cpu/resctrl/core.c +0
-0arch/x86/kernel/cpu/resctrl/core.c static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r) · line 254
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/core.c +5
-1@@ -247,7 +247,11 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)cpuid_count (0x80000020 , subleaf, &eax, &ebx, &ecx, &edx); hw_res->num_closid = edx +1 ; - r->membw.max_bw =1 << eax; +if (BITS_PER_TYPE (r->membw.max_bw) <= eax) { +pr_warn ("Unable to support hardware's maximum bandwidth\n" ); +return false; + } + r->membw.max_bw =BIT (eax);Why?
Quoted 4 lines
Why go all the effort if the field can be 64 bits?
Why not make
max_bwu64 and handle *all* possible values naturally?
resctrl does not support values larger than u32.
The value represented by max_bw provides the upper limit of control values that user space can configure via the schemata file, which in turn represents the control value programmed to hardware.
u32 is expected in several places:
- Parsing of user input in bw_validate() is done via kstrtou32() placing result in "u32 bw". It is this value that is compared against max_bw and would thus need to change as a consequence.
- Parsing of user input to schemata file via parse_bw() places parsed bandwidth in "u32 bw_val".
- Upon successful parsing the control value is staged by resctrl fs in resctrl_staged_config::new_ctrl that is u32.
- On receipt of programming request from resctrl fs, the x86 architectural specific code programs the control value to rdt_hw_ctrl_domain::ctrl_val[] that are u32.
resctrl, from fs to x86 architecture and now also MPAM, would thus require changes to support these AMD systems. If I get the math right this will be systems that support 1073741823.875GB/s and up?
If resctrl is required to support AMD hardware that can return up to 64bits here then that requires more changes than what I am able to take on at this time. From my side I prefer to stick to this resctrl safeguard for the scenario and leave support for this hardware to AMD.
Reinette
Borislav Petkov
maintainerRe: [PATCH v5 02/10] x86/resctrl: Protect against bad shift
Quoted 43 lines · Reinette Chatre
Hi Boris,
Quoted 18 lines · Borislav Petkov
Quoted 5 lines · Reinette Chatre
arch/x86/kernel/cpu/resctrl/core.c +0
-0arch/x86/kernel/cpu/resctrl/core.c static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r) · line 254
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/core.c +5
-1@@ -247,7 +247,11 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)cpuid_count (0x80000020 , subleaf, &eax, &ebx, &ecx, &edx); hw_res->num_closid = edx +1 ; - r->membw.max_bw =1 << eax; +if (BITS_PER_TYPE (r->membw.max_bw) <= eax) { +pr_warn ("Unable to support hardware's maximum bandwidth\n" ); +return false; + } + r->membw.max_bw =BIT (eax);Why?
Quoted 4 lines
Why go all the effort if the field can be 64 bits?
Why not make
max_bwu64 and handle *all* possible values naturally?resctrl does not support values larger than u32.
The value represented by
max_bwprovides the upper limit of control values that user space can configure via the schemata file, which in turn represents the control value programmed to hardware.u32 is expected in several places:
- Parsing of user input inbw_validate()is done viakstrtou32()placing result in "u32 bw". It is this value that is compared againstmax_bwand would thus need to change as a consequence.
- Parsing of user input to schemata file viaparse_bw()places parsed bandwidth in "u32bw_val".
- Upon successful parsing the control value is staged by resctrl fs inresctrl_staged_config::new_ctrlthat is u32.
- On receipt of programming request from resctrl fs, the x86 architectural specific code programs the control value to rdt_hw_ctrl_domain::ctrl_val[] that are u32.
Quoted 8 lines
resctrl, from fs to x86 architecture and now also MPAM, would thus require changes to support these AMD systems. If I get the math right this will be systems that support 1073741823.875GB/s and up?
If resctrl is required to support AMD hardware that can return up to 64bits here then that requires more changes than what I am able to take on at this time. From my side I prefer to stick to this resctrl safeguard for the scenario and leave support for this hardware to AMD.
So that bandwidth limit thing is read from those L3QOS_BW_CONTROL_n MSRs which have
BW_LEN-1:0 as the range which is the bandwidth limit, at BW_LEN there's the unlimited bandwidth limit
and that BW_LEN is read from CPUID.
And that is, kinda 12, right now.
And your worry is that some day that limit BW_LEN might go over 31?
Which your math shows how absurd this is.
So what are we actually fixing here?
Something some AI hallucinated that it might be possible at some point in the very distant, dark future?
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteReinette Chatre
maintainerRe: [PATCH v5 02/10] x86/resctrl: Protect against bad shift
Hi Boris,
Quoted 54 lines · Borislav Petkov
Quoted 41 lines · Reinette Chatre
Quoted 18 lines · Borislav Petkov
Quoted 5 lines · Reinette Chatre
arch/x86/kernel/cpu/resctrl/core.c +0
-0arch/x86/kernel/cpu/resctrl/core.c static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r) · line 254
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/core.c +5
-1@@ -247,7 +247,11 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)cpuid_count (0x80000020 , subleaf, &eax, &ebx, &ecx, &edx); hw_res->num_closid = edx +1 ; - r->membw.max_bw =1 << eax; +if (BITS_PER_TYPE (r->membw.max_bw) <= eax) { +pr_warn ("Unable to support hardware's maximum bandwidth\n" ); +return false; + } + r->membw.max_bw =BIT (eax);Why?
Quoted 4 lines
Why go all the effort if the field can be 64 bits?
Why not make
max_bwu64 and handle *all* possible values naturally?resctrl does not support values larger than u32.
The value represented by
max_bwprovides the upper limit of control values that user space can configure via the schemata file, which in turn represents the control value programmed to hardware.u32 is expected in several places:
- Parsing of user input inbw_validate()is done viakstrtou32()placing result in "u32 bw". It is this value that is compared againstmax_bwand would thus need to change as a consequence.
- Parsing of user input to schemata file viaparse_bw()places parsed bandwidth in "u32bw_val".
- Upon successful parsing the control value is staged by resctrl fs inresctrl_staged_config::new_ctrlthat is u32.
- On receipt of programming request from resctrl fs, the x86 architectural specific code programs the control value to rdt_hw_ctrl_domain::ctrl_val[] that are u32.Quoted 8 lines
resctrl, from fs to x86 architecture and now also MPAM, would thus require changes to support these AMD systems. If I get the math right this will be systems that support 1073741823.875GB/s and up?
If resctrl is required to support AMD hardware that can return up to 64bits here then that requires more changes than what I am able to take on at this time. From my side I prefer to stick to this resctrl safeguard for the scenario and leave support for this hardware to AMD.
So that bandwidth limit thing is read from those
L3QOS_BW_CONTROL_nMSRs which have
Quoted 8 lines
BW_LEN-1:0 as the range which is the bandwidth limit, at
BW_LENthere's the unlimited bandwidth limitand that
BW_LENis read from CPUID.And that is, kinda 12, right now.
And your worry is that some day that limit
BW_LENmight go over 31?
This is theoretical since the spec/hardware technically supports larger values than what resctrl supports. This could take the form of actual hardware or some model aimed at testing limits.
Quoted 7 lines
Which your math shows how absurd this is.
So what are we actually fixing here?
Something some AI hallucinated that it might be possible at some point in the very distant, dark future?
Not an AI hallucination but something encountered during my earlier attempts at getting resctrl to pass with fewer issues reported by the various deterministic tools, in this case a static checker, that I run as part of checking resctrl health.
I aim to highlight this caveat in the changelog with:
Whether this could ever represent a reasonable bandwidth value is unknown but addressing the issue will appease static checkers.
I see how such change could be considered absurd. I am ok to drop it from the series and just keep treating it's report locally as a false positive.
Reinette
Borislav Petkov
maintainerRe: [PATCH v5 02/10] x86/resctrl: Protect against bad shift
Quoted 2 lines · Reinette Chatre
I see how such change could be considered absurd. I am ok to drop it from the series and just keep treating it's report locally as a false positive.
Nah, but I'd like you, pls, to rewrite the commit message and explain more prominently
1. resctrl does not support values than u32 max_bw
2. what this is really fixing and where the potential issue can come from and what the situation is today.
I'd like to have more background in the commit message why we even arrived at this patch.
I wouldnt've even done it - we have bigger fish to fry - but if you think it is important for resctrl, sure.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteReinette Chatre
maintainerRe: [PATCH v5 02/10] x86/resctrl: Protect against bad shift
Hi Boris,
Quoted 9 lines · Borislav Petkov
Quoted 2 lines · Reinette Chatre
I see how such change could be considered absurd. I am ok to drop it from the series and just keep treating it's report locally as a false positive.
Nah, but I'd like you, pls, to rewrite the commit message and explain more prominently
1. resctrl does not support values than u32
max_bw
Quoted 8 lines
2. what this is really fixing and where the potential issue can come from and what the situation is today.
I'd like to have more background in the commit message why we even arrived at this patch.
I wouldnt've even done it - we have bigger fish to fry - but if you think it is important for resctrl, sure.
I am trying to help with bigger things in resctrl and find it easier to do so when working on a baseline that is without implicit assumptions.
I plan to use changelog below that aims to address the items you highlighted and also changed the math to make the bandwidth boundary where resctrl stop working clear:
x86/resctrl: Protect against bad shift
The size of the bandwidth field is enumerated from AMD hardware. resctrl uses this field width to determine the maximum bandwidth supported that is stored in resctrl_membw::max_bw. User space allocation requests ("control values") are compared against this maximum for validity before being programmed to hardware.
resctrl filesystem and resctrl x86 architecture code only support u32 control
values: resctrl_membw::max_bw is a u32, the control value provided by user space is parsed into u32 local variables, and after validity checks, the control value is staged into the u32 resctrl_staged_config::new_ctrl for architecture consumption. The resctrl x86 architecture code in turn caches the new control value into the u32 array rdt_hw_ctrl_domain::ctrl_val[].
The AMD bandwidth field to which control values are written can be up to 64 bits wide. While not an issue with current hardware (bandwidths that require more than a u32, more than 536870911.875 GB/s, seem unreasonable today), it is theoretically possible that enumeration of maximum bandwidth field width will return values that are according to specification but cannot be supported by resctrl.
Static checkers complain about this size mismatch.
Fix the static checker complaint by explicitly encoding the fact that resctrl is unable to support all values that the hardware specification allows. Switch to BIT() instead of open-coding the bitshift to avoid signed integer overflow if the number of bits is a valid 31.
Reinette
Borislav Petkov
maintainerRe: [PATCH v5 02/10] x86/resctrl: Protect against bad shift
Quoted 2 lines · Reinette Chatre
I am trying to help with bigger things in resctrl and find it easier to do so when working on a baseline that is without implicit assumptions.
Ofc.
Quoted 2 lines
I plan to use changelog below that aims to address the items you highlighted and also changed the math to make the bandwidth boundary where resctrl stop working clear:
LGTM.
You can send me an updated patch now and I can integrate it into the pile.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteReinette Chatre
maintainerRe: [PATCH v5 02/10] x86/resctrl: Protect against bad shift
Quoted 1 line · Borislav Petkov
You can send me an updated patch now and I can integrate it into the pile.
Thank you very much Boris. I'll do so shortly.
Reinette
Reinette Chatre
maintainerPATCH v5 2/10 x86/resctrl: Protect against bad shift
In reply to Borislav PetkovThe size of the bandwidth field is enumerated from AMD hardware. resctrl uses this field width to determine the maximum bandwidth supported that is stored in resctrl_membw::max_bw. User space allocation requests ("control values") are compared against this maximum for validity before being programmed to hardware.
resctrl filesystem and resctrl x86 architecture code only support u32 control values: resctrl_membw::max_bw is a u32, the control value provided by user space is parsed into u32 local variables, and after validity checks, the control value is staged into the u32 resctrl_staged_config::new_ctrl for architecture consumption. The resctrl x86 architecture code in turn caches the new control value into the u32 array rdt_hw_ctrl_domain::ctrl_val[].
The AMD bandwidth field to which control values are written can be up to 64 bits wide. While not an issue with current hardware (bandwidths that require more than a u32, more than 536870911.875 GB/s, seem unreasonable today), it is theoretically possible that enumeration of maximum bandwidth field width will return values that are according to specification but cannot be supported by resctrl.
Static checkers complain about this size mismatch.
Fix the static checker complaint by explicitly encoding the fact that resctrl is unable to support all values that the hardware specification allows. Switch to BIT() instead of open-coding the bitshift to avoid signed integer overflow if the number of bits is a valid 31.
---
Changes since v3:
- Use pr_warn() instead of WARN_ON().
sashiko: https://sashiko.dev/#/patchset/cover.1775576382.git.reinette.chatre%40intel.com
Changes since v4:
- Add Tony's Reviewed-by tag.
Changes since v5:
- Rewrite changelog to highlight that resctrl cannot support bandwidth needing more than u32 and what this patch really fixes. (Boris)
--- arch/x86/kernel/cpu/resctrl/core.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/core.c +5 -1
@@ -247,7 +247,11 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)cpuid_count (0x80000020 , subleaf, &eax, &ebx, &ecx, &edx); hw_res->num_closid = edx +1 ; - r->membw.max_bw =1 << eax; +if (BITS_PER_TYPE (r->membw.max_bw) <= eax) { +pr_warn ("Unable to support hardware's maximum bandwidth\n" ); +return false; + } + r->membw.max_bw =BIT (eax);/* AMD does not use delay */ r->membw.delay_linear = false;
Signature · 1 line
2.54.0
Reinette Chatre
maintainerPATCH v5 3/10 fs/resctrl: Change pattern used to track number of entries in enum
In reply to Reinette ChatreCDP_NUM_TYPES tracks the number of different configuration types that can be applied to a resource. CDP_NUM_TYPES is required to iterate over the different configurations but is not a member of enum resctrl_conf_type to avoid the warning generated with -Wswitch when CDP_NUM_TYPES lacks a case.
Add a new CDP_LAST enum entry used in CDP_NUM_TYPES definition to simplify adding a new enum entry. Do this to create a cleaner pattern for tracking the number of enum entries in resctrl in preparation for other enums needing to do so.
---
Changes since v2:
- Add Ben's RB tag.
Changes since v4:
- Add Tony's Reviewed-by tag.
--- include/linux/resctrl.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
include/linux/resctrl.h +2 -1
@@ -69,9 +69,10 @@ enum resctrl_conf_type {CDP_NONE ,CDP_CODE ,CDP_DATA , + CDP_LAST = CDP_DATA }; -#define CDP_NUM_TYPES (CDP_DATA + 1) +#define CDP_NUM_TYPES (CDP_LAST + 1)/* * struct pseudo_lock_region - pseudo-lock region information
Signature · 1 line
2.54.0
Reinette Chatre
maintainerPATCH v5 4/10 fs/resctrl: Use accurate type for rdt_resource::rid
In reply to Reinette ChatreEvery resctrl resource has a unique ID described by enum resctrl_res_level. enum resctrl_res_level is used in all resource ID initializations and all resource ID comparisons. All functions consuming the resource ID expects an enum resctrl_res_level. Of the four structures that contain a resource ID (struct mon_data, struct mon_evt, struct rdt_domain_hdr, and struct
rdt_resource) only struct rdt_resource does not use enum resctrl_res_level.
Switch the type of rdt_resource::rid to be enum resctrl_res_level to make it obvious what values are valid, match the type everywhere this member is used, and obtain benefits from tools that can flag any enum misuse.
Move define of RDT_NUM_RESOURCES outside the enum to enable tools to catch when a switch() on the resource ID does not handle all the resources and thus help flag which switch statements need an update when a new resource is added.
---
Changes since v1:
- Use RDT_RESOURCE_LAST = RDT_RESOURCE_PERF_PKG as last entry in enum.(Ben)
Changes since v2:
- Add Ben's RB tag.
Changes since v4:
- Add Tony's Reviewed-by tag.
--- include/linux/resctrl.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
include/linux/resctrl.h +4 -4
@@ -54,11 +54,11 @@ enum resctrl_res_level {RDT_RESOURCE_MBA ,RDT_RESOURCE_SMBA ,RDT_RESOURCE_PERF_PKG , - -/* Must be the last */ - RDT_NUM_RESOURCES, + RDT_RESOURCE_LAST = RDT_RESOURCE_PERF_PKG }; +#define RDT_NUM_RESOURCES (RDT_RESOURCE_LAST + 1) +/** * enum resctrl_conf_type - The type of configuration. * @CDP_NONE: No prioritisation, both code and data are controlled or monitored.
@@ -324,7 +324,7 @@ struct resctrl_mon {* @cdp_capable: Is the CDP feature available on this resource */ struct rdt_resource { -int rid; +enum resctrl_res_level rid;bool alloc_capable;bool mon_capable;enum resctrl_scope ctrl_scope;
Signature · 1 line
2.54.0
Reinette Chatre
maintainerPATCH v5 5/10 fs/resctrl: Pass error reading event through to user space
In reply to Reinette ChatreReading of event data is managed through populating a struct rmid_read with properties of event needing to be read. This data is dispatched to an appropriate CPU and upon completion any error can be found in rmid_read::err, or on success the event data will be in rmid_read::val.
rmid_read::err is not updated in the unlikely scenario that the reading of the event was dispatched to a wrong CPU. If this ever occurs due to a bug in resctrl the user space read will return "success" but the data reported will be invalid.
Ensure accurate error reporting so that if there may be an issue with how resctrl picks a CPU it could be learned with an error to user space instead of silent failure.
---
Changes since v1:
- Add Ben's RB tag.
Changes since V4:
- Add Tony's Reviewed-by tag.
--- fs/resctrl/monitor.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
fs/resctrl/monitor.c +6 -2
@@ -458,8 +458,10 @@ static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr) }/* Reading a single domain, must be on a CPU in that domain. */ -if (!cpumask_test_cpu (cpu, &d->hdr.cpu_mask)) +if (!cpumask_test_cpu (cpu, &d->hdr.cpu_mask)) { + rr->err = -EIO ;return -EINVAL ; + }if (rr->is_mbm_cntr) rr->err =resctrl_arch_cntr_read (rr->r, d, closid, rmid, cntr_id, rr->evt->evtid, &tval);
@@ -496,8 +498,10 @@ static int __l3_mon_event_count_sum(struct rdtgroup *rdtgrp, struct rmid_read *r }/* Summing domains that share a cache, must be on a CPU for that cache. */ -if (!cpumask_test_cpu (cpu, &rr->ci->shared_cpu_map)) +if (!cpumask_test_cpu (cpu, &rr->ci->shared_cpu_map)) { + rr->err = -EIO ;return -EINVAL ; + }/* * Legacy files must report the sum of an event across all
Signature · 1 line
2.54.0
Reinette Chatre
maintainerPATCH v5 6/10 fs/resctrl: Use accurate and symmetric exit flows
In reply to Reinette ChatreDuring schemata file write handling there is one error exit path labeled "out" that handles all cleanup and unlocking needed on exit. The staged configs cleared during an error exit may not have been used at the time of exit making the clearing of the staged configs unnecessary.
Access the exit code using two labels and only clear the staged configuration if it was in use at the time of exit. Doing so makes the code flow obvious and simplifies upcoming changes that improve the handling of failures during early input parsing.
---
Changes since v4:
- Add Tony's Reviewed-by tag.
--- fs/resctrl/ctrlmondata.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
fs/resctrl/ctrlmondata.c +7 -6
@@ -331,7 +331,7 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) { ret = -EINVAL ;rdt_last_cmd_puts ("Resource group is pseudo-locked\n" ); - goto out; + goto out_unlock; }rdt_staged_configs_clear ();
@@ -341,16 +341,16 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,if (!tok) {rdt_last_cmd_puts ("Missing ':'\n" ); ret = -EINVAL ; - goto out; + goto out_clear_staged; }if (tok[0 ] =='\0' ) {rdt_last_cmd_printf ("Missing '%s' value\n" , resname); ret = -EINVAL ; - goto out; + goto out_clear_staged; } ret =rdtgroup_parse_resource (resname, tok, rdtgrp);if (ret) - goto out; + goto out_clear_staged; }list_for_each_entry (s, &resctrl_schema_all, list) {
@@ -365,7 +365,7 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, ret =resctrl_arch_update_domains (r, rdtgrp->closid);if (ret) - goto out; + goto out_clear_staged; }if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) {
@@ -378,8 +378,9 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, ret =rdtgroup_pseudo_lock_create (rdtgrp); } -out: +out_clear_staged:rdt_staged_configs_clear (); +out_unlock:rdtgroup_kn_unlock (of->kn);return ret ?: nbytes; }
Signature · 1 line
2.54.0
Reinette Chatre
maintainerPATCH v5 7/10 fs/resctrl: Change last_cmd_status custom during input parsing
In reply to Reinette ChatreA pattern of usage of last_cmd_status was introduced during its enabling in commit
c377dcfbee80 ("x86/intel_rdt: Add diagnostics when writing the schemata file")
and since copied throughout resctrl to result in the following custom:
..._write()
{
/* Early parsing of input, exit on failure. */
/* Obtain rdtgroup_mutex */
rdt_last_cmd_clear(); /* Clear last_cmd_status buffer */
/*
* Act on user command, failures result in detail
* error message in last_cmd_status buffer via
* rdt_last_cmd_puts()/rdt_last_cmd_printf().
*/
/* Release rdtgroup_mutex */
}If resctrl exits with failure during early parsing of input there are two possible scenarios:
- The last_cmd_status buffer is empty and a user's read of info/last_cmd_status returns "ok".
- The last_cmd_status buffer contains details from an earlier ...write() failure and a user's read of info/last_cmd_status returns this outdated error description.
Writing to a resctrl file is considered a "resctrl command" and the resctrl documentation states the following about the last_cmd_status file:
"If the command failed, it will provide more information that can be
conveyed in the error returns from file operations."
Neither of the current scenarios is correct behavior.
Move early input parsing to be done with rdtgroup_mutex held after the last_cmd_status buffer is cleared. Let info/last_cmd_status be accurate when an error is encountered during parsing of user command.
---
Changes since v1:
- Add Ben's RB tag.
Changes since v4:
- Rebase on top of the "x86,fs/resctrl: Fix long-standing issues" series that added locking helpers to info file handlers.
- Add resctrl filename to new error messages to disambiguate the various "Invalid input" messages. (Tony)
- Add Tony's Reviewed-by tag.
--- fs/resctrl/ctrlmondata.c | 55 ++++++++++++++++---------- fs/resctrl/monitor.c | 60 ++++++++++++++++------------ fs/resctrl/rdtgroup.c | 84 +++++++++++++++++++++++----------------- 3 files changed, 120 insertions(+), 79 deletions(-)
fs/resctrl/ctrlmondata.c +35 -20
@@ -312,11 +312,6 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,char *tok, *resname;int ret =0 ; -/* Valid input requires a trailing newline */ -if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) -return -EINVAL ; - buf[nbytes -1 ] ='\0' ; - rdtgrp =rdtgroup_kn_lock_live (of->kn);if (!rdtgrp) {rdtgroup_kn_unlock (of->kn);
@@ -324,6 +319,15 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, }rdt_last_cmd_clear (); +/* Valid input requires a trailing newline */ +if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) { +rdt_last_cmd_puts ("schemata: Invalid input\n" ); + ret = -EINVAL ; + goto out_unlock; + } + + buf[nbytes -1 ] ='\0' ; +/* * No changes to pseudo-locked region allowed. It has to be removed * and re-created instead.
@@ -466,11 +470,6 @@ ssize_t rdtgroup_mba_mbps_event_write(struct kernfs_open_file *of,struct rdtgroup *rdtgrp;int ret =0 ; -/* Valid input requires a trailing newline */ -if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) -return -EINVAL ; - buf[nbytes -1 ] ='\0' ; - rdtgrp =rdtgroup_kn_lock_live (of->kn);if (!rdtgrp) {rdtgroup_kn_unlock (of->kn);
@@ -478,6 +477,15 @@ ssize_t rdtgroup_mba_mbps_event_write(struct kernfs_open_file *of, }rdt_last_cmd_clear (); +/* Valid input requires a trailing newline */ +if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) { +rdt_last_cmd_puts ("mba_MBps_event: Invalid input\n" ); + ret = -EINVAL ; + goto out_unlock; + } + + buf[nbytes -1 ] ='\0' ; +if (!strcmp (buf,"mbm_local_bytes" )) {if (resctrl_is_mon_event_enabled (QOS_L3_MBM_LOCAL_EVENT_ID )) rdtgrp->mba_mbps_event =QOS_L3_MBM_LOCAL_EVENT_ID ;
@@ -495,6 +503,7 @@ ssize_t rdtgroup_mba_mbps_event_write(struct kernfs_open_file *of,if (ret)rdt_last_cmd_printf ("Unsupported event id '%s'\n" , buf); +out_unlock:rdtgroup_kn_unlock (of->kn);return ret ?: nbytes;
@@ -858,16 +867,18 @@ ssize_t resctrl_io_alloc_write(struct kernfs_open_file *of, char *buf,bool enable;int ret; - ret =kstrtobool (buf, &enable); -if (ret) -return ret; -if (!info_kn_lock (of->kn))return -ENOENT ; r = s->res;rdt_last_cmd_clear (); + ret =kstrtobool (buf, &enable); +if (ret) { +rdt_last_cmd_puts ("io_alloc: Invalid input\n" ); + goto out_unlock; + } +if (!r->cache.io_alloc_capable) {rdt_last_cmd_printf ("io_alloc is not supported on %s\n" , s->name); ret = -ENODEV ;
@@ -1022,17 +1033,21 @@ ssize_t resctrl_io_alloc_cbm_write(struct kernfs_open_file *of, char *buf,u32 io_alloc_closid;int ret =0 ; -/* Valid input requires a trailing newline */ -if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) -return -EINVAL ; - - buf[nbytes -1 ] ='\0' ; -if (!info_kn_lock (of->kn))return -ENOENT ;rdt_last_cmd_clear (); r = s->res; + +/* Valid input requires a trailing newline */ +if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) { +rdt_last_cmd_puts ("io_alloc_cbm: Invalid input\n" ); + ret = -EINVAL ; + goto out_unlock; + } + + buf[nbytes -1 ] ='\0' ; +if (!r->cache.io_alloc_capable) {rdt_last_cmd_printf ("io_alloc is not supported on %s\n" , s->name); ret = -ENODEV ;
fs/resctrl/monitor.c +36 -24
112 lines not on this page · raw message
fs/resctrl/rdtgroup.c +49 -35
167 lines not on this page · raw message
Signature · 1 line
2.54.0
Reinette Chatre
maintainerPATCH v5 8/10 fs/resctrl: Add last_cmd_status support for writes to max_threshold_occupancy
In reply to Reinette Chatreinfo/last_cmd_status is intended to contain more information if a write to any resctrl file fails. Writes to max_threshold_occupancy did not receive last_cmd_status support during initial last_cmd_status enabling. Add it now.
---
Changes since v1:
- Add Ben's RB tag.
Changes since v2:
- Switch to guard(). (Chenyu)
- Drop Ben's RB tag due to significant code change.
Changes since v4:
- Rebased on top of the "x86,fs/resctrl: Fix long-standing issues" series that added locking helpers to info file handlers.
- Add Tony's Reviewed-by tag.
--- fs/resctrl/rdtgroup.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
fs/resctrl/rdtgroup.c +10 -4
@@ -1300,14 +1300,20 @@ static ssize_t max_threshold_occ_write(struct kernfs_open_file *of,unsigned int bytes;int ret; - ret =kstrtouint (buf,0 , &bytes); -if (ret) -return ret; -if (!info_kn_lock (of->kn))return -ENOENT ; +rdt_last_cmd_clear (); + + ret =kstrtouint (buf,0 , &bytes); +if (ret) { +rdt_last_cmd_puts ("max_threshold_occupancy: Invalid input\n" ); + goto out_unlock; + } +if (bytes > resctrl_rmid_realloc_limit) { +rdt_last_cmd_printf ("max_threshold_occupancy: Exceeds limit (before adjustment) of %u bytes\n" , + resctrl_rmid_realloc_limit); ret = -EINVAL ; goto out_unlock; }
Signature · 1 line
2.54.0
Reinette Chatre
maintainerPATCH v5 9/10 fs/resctrl: Communicate resource group deleted error via last_cmd_status
In reply to Reinette ChatreUser space expects last_cmd_status to contain additional information if any resctrl command fails.
A resctrl command may be blocked waiting on the rdtgroup_mutex waiting for another command to finish and find that once the mutex is available that the resource group has since been deleted. In this scenario the command will fail while last_cmd_status contains either "ok" if the last_cmd_status buffer is empty or an outdated error from a previous command failure if last_cmd_status buffer has content.
Include clearing of last_cmd_status buffer as part of rdtgroup_kn_lock_live() that is used to obtain access and needed locking to a resource group before attempting a command on the group. With the last_cmd_status buffer ready, provide an appropriate message to user space if the resource group has been deleted.
No last_cmd_status treatment is needed for the remaining failure of rdtgroup_kn_lock_live() encountering a non-existent resource group since that could only occur during an attempt to obtain a resource group lock on a file in info/ which is an invalid usage.
---
Changes since v1:
- Add snippet about why it is safe to dereference the kn to get to resource group name when resource group is deleted.
Changes since v4:
- Add Tony's Reviewed-by tag.
--- fs/resctrl/ctrlmondata.c | 3 --- fs/resctrl/monitor.c | 2 -- fs/resctrl/rdtgroup.c | 20 +++++++++++--------- 3 files changed, 11 insertions(+), 14 deletions(-)
fs/resctrl/ctrlmondata.c +0 -3
@@ -317,7 +317,6 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,rdtgroup_kn_unlock (of->kn);return -ENOENT ; } -rdt_last_cmd_clear ();/* Valid input requires a trailing newline */ if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) {
@@ -434,7 +433,6 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, } }else if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) {if (!rdtgrp->plr->d) { -rdt_last_cmd_clear ();rdt_last_cmd_puts ("Cache domain offline\n" ); ret = -ENODEV ; }else {
@@ -475,7 +473,6 @@ ssize_t rdtgroup_mba_mbps_event_write(struct kernfs_open_file *of,rdtgroup_kn_unlock (of->kn);return -ENOENT ; } -rdt_last_cmd_clear ();/* Valid input requires a trailing newline */ if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) {
fs/resctrl/monitor.c +0 -2
@@ -1696,7 +1696,6 @@ int mbm_L3_assignments_show(struct kernfs_open_file *of, struct seq_file *s, voi goto out_unlock; } -rdt_last_cmd_clear ();if (!resctrl_arch_mbm_cntr_assign_enabled (r)) {rdt_last_cmd_puts ("mbm_event counter assignment mode is not enabled\n" ); ret = -EINVAL ;
@@ -1836,7 +1835,6 @@ ssize_t mbm_L3_assignments_write(struct kernfs_open_file *of, char *buf,rdtgroup_kn_unlock (of->kn);return -ENOENT ; } -rdt_last_cmd_clear ();/* Valid input requires a trailing newline */ if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) {
fs/resctrl/rdtgroup.c +11 -9
@@ -359,7 +359,6 @@ static int rdtgroup_cpus_show(struct kernfs_open_file *of,if (rdtgrp) {if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) {if (!rdtgrp->plr->d) { -rdt_last_cmd_clear ();rdt_last_cmd_puts ("Cache domain offline\n" ); ret = -ENODEV ; }else {
@@ -522,8 +521,6 @@ static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of, goto out_unlock; } -rdt_last_cmd_clear (); -if (!buf) {rdt_last_cmd_printf ("%s: Invalid input\n" ,is_cpu_list (of) ?"cpus_list" :"cpus" );
@@ -788,7 +785,6 @@ static ssize_t rdtgroup_tasks_write(struct kernfs_open_file *of,rdtgroup_kn_unlock (of->kn);return -ENOENT ; } -rdt_last_cmd_clear ();if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED || rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) {
@@ -1530,7 +1526,6 @@ static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of,return -ENOENT ; } -rdt_last_cmd_clear ();/* Valid input requires a trailing newline */ if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) {rdt_last_cmd_puts ("mode: Invalid input\n" );
@@ -1667,7 +1662,6 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) {if (!rdtgrp->plr->d) { -rdt_last_cmd_clear ();rdt_last_cmd_puts ("Cache domain offline\n" ); ret = -ENODEV ; }else {
@@ -2728,10 +2722,20 @@ struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn)cpus_read_lock ();mutex_lock (&rdtgroup_mutex); +rdt_last_cmd_clear ();/* Was this group deleted while we waited? */ -if (rdtgrp->flags &RDT_DELETED ) +if (rdtgrp->flags &RDT_DELETED ) { +/* +* It is safe to dereference kn to obtain the resource group's +* name because one extra reference to kn is obtained +* during resource group creation that will be released by +* rdtgroup_remove() called by rdtgroup_kn_put(). +*/ +rdt_last_cmd_printf ("Resource group %s deleted. No commands possible.\n" , +rdt_kn_name (rdtgrp->kn));return NULL ; + }return rdtgrp; }
@@ -3945,8 +3949,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_unlock; } -rdt_last_cmd_clear (); -/* * Check that the parent directory for a monitor group is a "mon_groups" * directory.
Signature · 1 line
2.54.0
Reinette Chatre
maintainerPATCH v5 10/10 fs/resctrl: Inform user space when status buffer overflowed
In reply to Reinette Chatreresctrl fs commands are becoming more powerful with, for example, a user able to use syntax like '*' to make broad configuration changes. Such commands that span multiple domains may result in more than one message printed to the last_cmd_status buffer with more planned.
Display "[truncated]" to user space when displaying the last_cmd_status buffer to communicate if it overflowed. Upon encountering this user space is expected to combine details about the failure found in info/last_cmd_status with related resctrl files to learn the accurate system state after the command failure.
---
Changes since v2:
- Add Ben's RB tag.
Changes since V4:
- Rebased on "x86,fs/resctrl: Fix long-standing issues" series.
- Add Tony's Reviewed-by tag.
--- fs/resctrl/rdtgroup.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
fs/resctrl/rdtgroup.c +5 -2
@@ -979,10 +979,13 @@ static int rdt_last_cmd_status_show(struct kernfs_open_file *of,if (!info_kn_lock (of->kn))return -ENOENT ; len =seq_buf_used (&last_cmd_status); -if (len) +if (len) {seq_printf (seq,"%.*s" , len, last_cmd_status_buf); -else +if (seq_buf_has_overflowed (&last_cmd_status)) +seq_puts (seq,"[truncated]\n" ); + }else {seq_puts (seq,"ok\n" ); + }info_kn_unlock (of->kn);return 0 ; }
Signature · 1 line
2.54.0
Reinette Chatre
maintainerDear x86 maintainers,
Could you please consider this series for inclusion? It applies cleanly on top of x86/cache with HEAD at
2566b5cd6a27 ("fs/resctrl: Fix UAF from worker threads when domains are removed")
There is another resctrl series ("x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch") also ready for your consideration. There are no conflicts between it and this series, they can be considered in any order.
Thank you very much.
Reinette
[1] https://lore.kernel.org/lkml/[email protected]/
Quoted 133 lines · Reinette Chatre
Applies on top of v7.2-rc1 + v5 of "x86,fs/resctrl: Fix long-standing issues":
https://lore.kernel.org/lkml/[email protected]/Changes since v4:
- v4: https://lore.kernel.org/lkml/[email protected]/
- Rebased on top of V5 "x86,fs/resctrl: Fix long-standing issues":
https://lore.kernel.org/lkml/[email protected]/
A significant change in that series is the addition of info_kn_lock()/info_kn_unlock() helpers to the info file handlers. With the addition of locking to themax_threshold_occ_write()in that series it seemed appropriate to re-order thelast_cmd_statushandling inmax_threshold_occ_write()to be after the change tolast_cmd_statuscustom to containmax_threshold_occ_write()changes to one patch.
- Add resctrl filename to new error messages to disambiguate the various "Invalid input" messages. (Tony)
- Add Tony's Reviewed-by tag to all patches.Changes since v3:
- v3: https://lore.kernel.org/lkml/[email protected]/
- Rebased on v7.1-rc1.
- Drop "MAINTAINERS: Update resctrl entry" and "fs/resctrl: Add missing return value descriptions" that can be found in v7.1-rc1.
- Drop "fs/resctrl: Avoid "may be used uninitialized" warning" that just does unnecessary work to appease compiler warning that is disabled by default. (Boris)
- Usepr_warn()instead ofWARN_ON()in "x86/resctrl: Protect against bad shift".Changes since v2:
- v2: https://lore.kernel.org/lkml/[email protected]/
- Rebased on top of tip/master with pending resctrl selftest changes,
https://lore.kernel.org/lkml/[email protected]/,
applied on top to support testing with latest kernel that has pending resctrl changes merged.
- Drop "fs/resctrl: Use stricter checks on input to cpus/cpus_list file" that ended up being too strict and removed one use case. (Chris)
-max_threshold_occ_write()switch toguard(). (Chenyu)
- Add more detail to one return value description.
- Add tags.
- No opinions so far on needing to updatelast_cmd_statuson success of all read commands. At this timelast_cmd_statuswill continue to return previous failure message (if any) after a read command accessing static data succeeds.Changes since v1:
- v1: https://lore.kernel.org/lkml/[email protected]/
- To simplify tracking, include patch toMAINTAINERSsubmitted separately:
https://lore.kernel.org/lkml/4274c478922c01f9ceebc805acf991f10a95519f.1771442788.git.reinette.chatre@intel.com/
- Follow recent upstream changes to add reference to tip tree handbook inMAINTAINERSentry.
- Use new pattern in resctrl to enable looping over all entries of an enum while making adding new enum entries more clear all while avoiding warnings when compiling with -Wswitch. (Ben)
- Add anotherlast_cmd_statusenhancement that appends "[truncated]" to output of info/last_cmd_status if the backing buffer overflowed.
- Please see patch changelogs for details of changes.
- Add tags.Hi Everybody,
This is a collection of resctrl cleanups assembled together for convenience and simpler tracking. I'd be happy to split them up if it makes review and/or handling easier.
Summary of changes:
- Let resctrl pass stricter checks from various tools to provide a cleaner baseline with the goal to promote healthier contributions:
-./tools/docs/kernel-doc-Wall -v <files>
- Build with W=12
-./scripts/coccicheck
- Static checkers- Use accurate and consistent type for all uses of resource ID.
- In the unlikely scenario that resctrl picked a wrong CPU to read an event from, pass the error through to user space instead of claiming to succeed and returning a (wrong) result.
- Since inception of
last_cmd_statusfeature there have been mismatches between resctrl file operation failures and the contents of info/last_cmd_status. This pattern keeps propagating with each new resctrl feature. Establish a new baseline with a new pattern that ensures info/last_cmd_status contains an accurate failure description that matches the most recent resctrl file operation failure.One potential open:
There remains an inconsistency between resctrl file operations that do/can _not_ fail and the contents of info/last_cmd_status. If a resctrl file operation fails and an informational error is printed to
last_cmd_statusthen a subsequent reading of a resctrl file (specifically most of the files found in info/) may succeed while info/last_cmd_status may or may not return the error from previous failure.Ensuring
last_cmd_statusis reset on every read carries the cost of takingrdtgroup_mutexon several more user space initiated paths and thus increase contention onrdtgroup_mutex. I opted to not make this change and instead focus this work on ensuring thatlast_cmd_statusis accurate whenever there is a failure during any resctrl file operation. Please let me know if you have opinions in this regard.Update v5: The addition of the info_kn_lock()/info_kn_unlock() helpers to info file handlers now means that
rdtgroup_mutexis held every time an info file is opened. There is thus now less burden to resetlast_cmd_statusbuffer on each info file access. Even so, since there has not been any opinions in this regard I opted to keep current behavior.Any feedback is appreciated.
Regards,
Reinette
Reinette Chatre (10):
fs/resctrl: Use correct format specifier for printing error pointers x86/resctrl: Protect against bad shift
fs/resctrl: Change pattern used to track number of entries in enum fs/resctrl: Use accurate type forrdt_resource::ridfs/resctrl: Pass error reading event through to user space fs/resctrl: Use accurate and symmetric exit flows fs/resctrl: Changelast_cmd_statuscustom during input parsing fs/resctrl: Addlast_cmd_statussupport for writes to
max_threshold_occupancy
fs/resctrl: Communicate resource group deleted error via
last_cmd_status
fs/resctrl: Inform user space when status buffer overflowedarch/x86/kernel/cpu/resctrl/core.c | 6 +- fs/resctrl/ctrlmondata.c | 71 ++++++++++------- fs/resctrl/monitor.c | 78 ++++++++++-------- fs/resctrl/rdtgroup.c | 123 +++++++++++++++++------------ include/linux/resctrl.h | 11 +-- 5 files changed, 173 insertions(+), 116 deletions(-)
Borislav Petkov
maintainerQuoted 3 lines · Reinette Chatre
Dear x86 maintainers,
Could you please consider this series for inclusion?
/me looks.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRan few tests on the series. Looks good.
Quoted 133 lines · Reinette Chatre
Applies on top of v7.2-rc1 + v5 of "x86,fs/resctrl: Fix long-standing issues":
https://lore.kernel.org/lkml/[email protected]/Changes since v4:
- v4: https://lore.kernel.org/lkml/[email protected]/
- Rebased on top of V5 "x86,fs/resctrl: Fix long-standing issues":
https://lore.kernel.org/lkml/[email protected]/
A significant change in that series is the addition of info_kn_lock()/info_kn_unlock() helpers to the info file handlers. With the addition of locking to themax_threshold_occ_write()in that series it seemed appropriate to re-order thelast_cmd_statushandling inmax_threshold_occ_write()to be after the change tolast_cmd_statuscustom to containmax_threshold_occ_write()changes to one patch.
- Add resctrl filename to new error messages to disambiguate the various "Invalid input" messages. (Tony)
- Add Tony's Reviewed-by tag to all patches.Changes since v3:
- v3: https://lore.kernel.org/lkml/[email protected]/
- Rebased on v7.1-rc1.
- Drop "MAINTAINERS: Update resctrl entry" and "fs/resctrl: Add missing return value descriptions" that can be found in v7.1-rc1.
- Drop "fs/resctrl: Avoid "may be used uninitialized" warning" that just does unnecessary work to appease compiler warning that is disabled by default. (Boris)
- Usepr_warn()instead ofWARN_ON()in "x86/resctrl: Protect against bad shift".Changes since v2:
- v2: https://lore.kernel.org/lkml/[email protected]/
- Rebased on top of tip/master with pending resctrl selftest changes,
https://lore.kernel.org/lkml/[email protected]/,
applied on top to support testing with latest kernel that has pending resctrl changes merged.
- Drop "fs/resctrl: Use stricter checks on input to cpus/cpus_list file" that ended up being too strict and removed one use case. (Chris)
-max_threshold_occ_write()switch toguard(). (Chenyu)
- Add more detail to one return value description.
- Add tags.
- No opinions so far on needing to updatelast_cmd_statuson success of all read commands. At this timelast_cmd_statuswill continue to return previous failure message (if any) after a read command accessing static data succeeds.Changes since v1:
- v1: https://lore.kernel.org/lkml/[email protected]/
- To simplify tracking, include patch toMAINTAINERSsubmitted separately:https://lore.kernel.org/lkml/4274c478922c01f9ceebc805acf991f10a95519f.1771442788.git.reinette.chatre@intel.com/- Follow recent upstream changes to add reference to tip tree handbook in
MAINTAINERSentry.
- Use new pattern in resctrl to enable looping over all entries of an enum while making adding new enum entries more clear all while avoiding warnings when compiling with -Wswitch. (Ben)
- Add anotherlast_cmd_statusenhancement that appends "[truncated]" to output of info/last_cmd_status if the backing buffer overflowed.
- Please see patch changelogs for details of changes.
- Add tags.Hi Everybody,
This is a collection of resctrl cleanups assembled together for convenience and simpler tracking. I'd be happy to split them up if it makes review and/or handling easier.
Summary of changes:
- Let resctrl pass stricter checks from various tools to provide a cleaner baseline with the goal to promote healthier contributions:
-./tools/docs/kernel-doc-Wall -v <files>
- Build with W=12
-./scripts/coccicheck
- Static checkers- Use accurate and consistent type for all uses of resource ID.
- In the unlikely scenario that resctrl picked a wrong CPU to read an event from, pass the error through to user space instead of claiming to succeed and returning a (wrong) result.
- Since inception of
last_cmd_statusfeature there have been mismatches between resctrl file operation failures and the contents of info/last_cmd_status. This pattern keeps propagating with each new resctrl feature. Establish a new baseline with a new pattern that ensures info/last_cmd_status contains an accurate failure description that matches the most recent resctrl file operation failure.One potential open:
There remains an inconsistency between resctrl file operations that do/can _not_ fail and the contents of info/last_cmd_status. If a resctrl file operation fails and an informational error is printed to
last_cmd_statusthen a subsequent reading of a resctrl file (specifically most of the files found in info/) may succeed while info/last_cmd_status may or may not return the error from previous failure.Ensuring
last_cmd_statusis reset on every read carries the cost of takingrdtgroup_mutexon several more user space initiated paths and thus increase contention onrdtgroup_mutex. I opted to not make this change and instead focus this work on ensuring thatlast_cmd_statusis accurate whenever there is a failure during any resctrl file operation. Please let me know if you have opinions in this regard.Update v5: The addition of the info_kn_lock()/info_kn_unlock() helpers to info file handlers now means that
rdtgroup_mutexis held every time an info file is opened. There is thus now less burden to resetlast_cmd_statusbuffer on each info file access. Even so, since there has not been any opinions in this regard I opted to keep current behavior.Any feedback is appreciated.
Regards,
Reinette
Reinette Chatre (10):
fs/resctrl: Use correct format specifier for printing error pointers x86/resctrl: Protect against bad shift
fs/resctrl: Change pattern used to track number of entries in enum fs/resctrl: Use accurate type forrdt_resource::ridfs/resctrl: Pass error reading event through to user space fs/resctrl: Use accurate and symmetric exit flows fs/resctrl: Changelast_cmd_statuscustom during input parsing fs/resctrl: Addlast_cmd_statussupport for writes to
max_threshold_occupancy
fs/resctrl: Communicate resource group deleted error via
last_cmd_status
fs/resctrl: Inform user space when status buffer overflowedarch/x86/kernel/cpu/resctrl/core.c | 6 +- fs/resctrl/ctrlmondata.c | 71 ++++++++++------- fs/resctrl/monitor.c | 78 ++++++++++-------- fs/resctrl/rdtgroup.c | 123 +++++++++++++++++------------ include/linux/resctrl.h | 11 +-- 5 files changed, 173 insertions(+), 116 deletions(-)
Reinette Chatre
maintainerQuoted 3 lines · Babu
Ran few tests on the series. Looks good.
Tested-by Babu Moger <[email protected]>
Thank you very much Babu.
Reinette