x86/resctrl: monitored closid+rmid together, separate arch/fs locking
Subsystems (3)
James Morse
reviewerv7 cover x86/resctrl: monitored closid+rmid together, separate arch/fs locking
The only significant change in this series is to add cpus_read_lock() around the limbo and overflow handlers in the last patch. Other changes are a mix of comments and whitespace, or code marked __exit.
Changes are noted in each patch, some patches even say 'No changes since v6'! ~
This series does two things, it changes resctrl to call resctrl_arch_rmid_read() in a way that works for MPAM, and it separates the locking so that the arch code and filesystem code don't have to share a mutex. I tried to split this as two series, but these touch similar call sites, so it would create more work.
(What's MPAM? See the cover letter of the first series. [1])
On x86 the RMID is an independent number. MPAMs equivalent is PMG, but this isn't an independent number - it extends the PARTID (same as CLOSID) space with bits that aren't used to select the configuration. The monitors can then be told to match specific PMG values, allowing monitor-groups to be created.
But, MPAM expects the monitors to always monitor by PARTID. The Cache-storage-utilisation counters can only work this way.
(In the MPAM spec not setting the MATCH_PARTID bit is made CONSTRAINED UNPREDICTABLE - which is Arm's term to mean portable software can't rely on this)
It gets worse, as some SoCs may have very few PMG bits. I've seen the datasheet for one that has a single bit of PMG space.
To be usable, MPAM's counters always need the PARTID and the PMG. For resctrl, this means always making the CLOSID available when the RMID is used.
To ensure RMID are always unique, this series combines the CLOSID and RMID into an index, and manages RMID based on that. For x86, the index and RMID would always be the same.
Currently the architecture specific code in the cpuhp callbacks takes the rdtgroup_mutex. This means the filesystem code would have to export this lock, resulting in an ill-defined interface between the two, and the possibility of cross-architecture lock-ordering head aches.
The second part of this series adds a domain_list_lock to protect writes to the domain list, and protects the domain list with RCU - or cpus_read_lock().
Use of RCU is to allow lockless readers of the domain list. To get MPAMs monitors working, its very likely they'll need to be plumbed up to perf. An uncore PMU driver would need to be a lockless reader of the domain list.
This series is based on tip/master's commit 6b7b1e57a824, and can be retrieved from:
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/monitors_and_locking/v7
Bugs welcome,
Thanks,
James
[1] https://lore.kernel.org/lkml/[email protected]/
[v1] https://lore.kernel.org/all/[email protected]/
[v2] https://lore.kernel.org/lkml/[email protected]/
[v3] https://lore.kernel.org/r/[email protected]
[v4] https://lore.kernel.org/r/[email protected]
[v5] https://lore.kernel.org/lkml/[email protected]/
[v6] https://lore.kernel.org/all/[email protected]/
James Morse (24):
tick/nohz: Move tick_nohz_full_mask declaration outside the #ifdef x86/resctrl: kfree() rmid_ptrs from rdtgroup_exit() x86/resctrl: Create helper for RMID allocation and mondata dir
creation
x86/resctrl: Move rmid allocation out of mkdir_rdt_prepare() x86/resctrl: Track the closid with the rmid
x86/resctrl: Access per-rmid structures by index x86/resctrl: Allow RMID allocation to be scoped by CLOSID x86/resctrl: Track the number of dirty RMID a CLOSID has x86/resctrl: Use __set_bit()/__clear_bit() instead of open coding x86/resctrl: Allocate the cleanest CLOSID by searching
closid_num_dirty_rmid
x86/resctrl: Move CLOSID/RMID matching and setting to use helpers x86/resctrl: Add cpumask_any_housekeeping() for limbo/overflow x86/resctrl: Queue mon_event_read() instead of sending an IPI x86/resctrl: Allow resctrl_arch_rmid_read() to sleep x86/resctrl: Allow arch to allocate memory needed in
resctrl_arch_rmid_read()
x86/resctrl: Make resctrl_mounted checks explicit x86/resctrl: Move alloc/mon static keys into helpers x86/resctrl: Make rdt_enable_key the arch's decision to switch x86/resctrl: Add helpers for system wide mon/alloc capable x86/resctrl: Add CPU online callback for resctrl work x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but
cpu
x86/resctrl: Add CPU offline callback for resctrl work x86/resctrl: Move domain helper migration into resctrl_offline_cpu() x86/resctrl: Separate arch and fs resctrl locks
arch/x86/include/asm/resctrl.h | 90 +++++ arch/x86/kernel/cpu/resctrl/core.c | 94 ++--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 48 ++- arch/x86/kernel/cpu/resctrl/internal.h | 67 +++- arch/x86/kernel/cpu/resctrl/monitor.c | 453 +++++++++++++++++----- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 15 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 349 ++++++++++++----- include/linux/resctrl.h | 48 ++- include/linux/tick.h | 9 +- 9 files changed, 903 insertions(+), 270 deletions(-)
Signature · 1 line
2.39.2
James Morse
reviewerPATCH v7 1/24 tick/nohz: Move tick_nohz_full_mask declaration outside the #ifdef
tick_nohz_full_mask lists the CPUs that are nohz_full. This is only needed when CONFIG_NO_HZ_FULL is defined. tick_nohz_full_cpu() allows a specific CPU to be tested against the mask, and evaluates to false when CONFIG_NO_HZ_FULL is not defined.
The resctrl code needs to pick a CPU to run some work on, a new helper prefers housekeeping CPUs by examining the tick_nohz_full_mask. Hiding the declaration behind #ifdef CONFIG_NO_HZ_FULL forces all the users to be behind an ifdef too.
Move the tick_nohz_full_mask declaration, this lets callers drop the ifdef, and guard access to tick_nohz_full_mask with IS_ENABLED() or something like tick_nohz_full_cpu().
The definition does not need to be moved as any callers should be removed at compile time unless CONFIG_NO_HZ_FULL is defined.
---
No changes since v6
include/linux/tick.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
include/linux/tick.h +8 -1
@@ -164,9 +164,16 @@ static inline u64 get_cpu_idle_time_us(int cpu, u64 *unused) { return -1; }static inline u64 get_cpu_iowait_time_us (int cpu,u64 *unused) {return -1 ; }#endif /* !CONFIG_NO_HZ_COMMON */ +/* +* Mask of CPUs that are nohz_full. +* +* Users should be guarded by CONFIG_NO_HZ_FULL or a tick_nohz_full_cpu() +* check. +*/ +extern cpumask_var_t tick_nohz_full_mask; +#ifdef CONFIG_NO_HZ_FULL extern bool tick_nohz_full_running; -extern cpumask_var_t tick_nohz_full_mask;static inline bool tick_nohz_full_enabled (void ) {
Signature · 1 line
2.39.2
James Morse
reviewerPATCH v7 2/24 x86/resctrl: kfree() rmid_ptrs from rdtgroup_exit()
In reply to James Morsermid_ptrs[] is allocated from dom_data_init() but never free()d.
While the exit text ends up in the linker script's DISCARD section, the direction of travel is for resctrl to be/have loadable modules.
Add resctrl_exit_mon_l3_config() to cleanup any memory allocated by rdt_get_mon_l3_config().
There is no reason to backport this to a stable kernel.
---
Changes since v5:
* This patch is new
Changes since v6:
* Removed struct rdt_resource argument, added __exit markers to match the only caller.
* Adedd a whole stack of functions to maintain symmetry.
--- arch/x86/kernel/cpu/resctrl/core.c | 6 ++++++ arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 15 +++++++++++++++ 3 files changed, 22 insertions(+)
arch/x86/kernel/cpu/resctrl/core.c +6 -0
@@ -992,7 +992,13 @@ late_initcall(resctrl_late_init); static void __exit resctrl_exit(void) { +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +cpuhp_remove_state (rdt_online); + +if (r->mon_capable) +rdt_put_mon_l3_config (r); +rdtgroup_exit (); }
arch/x86/kernel/cpu/resctrl/internal.h +1 -0
@@ -546,6 +546,7 @@ void closid_free(int closid);int alloc_rmid (void );void free_rmid (u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r); +void __exit rdt_put_mon_l3_config(struct rdt_resource *r);bool __init rdt_cpu_has (int flag);void mon_event_count (void *info);int rdtgroup_mondata_show (struct seq_file *m,void *arg);
arch/x86/kernel/cpu/resctrl/monitor.c +15 -0
@@ -741,6 +741,16 @@ static int dom_data_init(struct rdt_resource *r)return 0 ; } +static void __exit dom_data_exit(struct rdt_resource *r) +{ +mutex_lock (&rdtgroup_mutex); + +kfree (rmid_ptrs); + rmid_ptrs =NULL ; + +mutex_unlock (&rdtgroup_mutex); +} +static struct mon_evt llc_occupancy_event = { .name ="llc_occupancy" , .evtid =QOS_L3_OCCUP_EVENT_ID ,
@@ -830,6 +840,11 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)return 0 ; } +void __exit rdt_put_mon_l3_config(struct rdt_resource *r) +{ +dom_data_exit (r); +} +void __init intel_rdt_mbm_apply_quirk (void ) {int cf_index;
Signature · 1 line
2.39.2
Reinette Chatre
maintainerRe: [PATCH v7 02/24] x86/resctrl: kfree() rmid_ptrs from rdtgroup_exit()
Hi James,
Subject refers to rdtgroup_exit() but the patch is actually changing resctrl_exit().
Quoted 7 lines · James Morse
rmid_ptrs[] is allocated from
dom_data_init()but never free()d.While the exit text ends up in the linker script's DISCARD section, the direction of travel is for resctrl to be/have loadable modules.
Add
resctrl_exit_mon_l3_config()to cleanup any memory allocated byrdt_get_mon_l3_config().
To match what patch actually does it looks like this should rather be:
"Add resctrl_exit_mon_l3_config()" -> "Add resctrl_put_mon_l3_config()"
Quoted 25 lines
There is no reason to backport this to a stable kernel.
Signed-off-by James Morse <[email protected]>---
Changes since v5:
* This patch is newChanges since v6:
* Removed structrdt_resourceargument, added__exitmarkers to match the only caller.
* Adedd a whole stack of functions to maintain symmetry.--- arch/x86/kernel/cpu/resctrl/core.c | 6 ++++++ arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 15 +++++++++++++++ 3 files changed, 22 insertions(+)arch/x86/kernel/cpu/resctrl/core.c +0
-0@@ -992,7 +992,13 @@ late_initcall(resctrl_late_init); static void __exit resctrl_exit(void)
arch/x86/kernel/cpu/resctrl/core.c late_initcall(resctrl_late_init); · line 1003
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +6
-0{ +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +cpuhp_remove_state (rdt_online); + +if (r->mon_capable) +rdt_put_mon_l3_config (r); +rdtgroup_exit (); }
I expect cleanup to do the inverse of init. I do not know what was the motivation for the rdtgroup_exit() to follow cpuhp_remove_state() but I was expecting this new cleanup to be done after rdtgroup_exit() to be inverse of init. This cleanup is inserted in middle of two existing cleanup - could you please elaborate how this location was chosen?
Reinette
James Morse
reviewerRe: [PATCH v7 02/24] x86/resctrl: kfree() rmid_ptrs from rdtgroup_exit()
Hi Reinette
Quoted 4 lines · Reinette Chatre
Hi James,
Subject refers to
rdtgroup_exit()but the patch is actually changingresctrl_exit().
I'll fix that,
Quoted 14 lines · James Morse
Quoted 7 lines
rmid_ptrs[] is allocated from
dom_data_init()but never free()d.While the exit text ends up in the linker script's DISCARD section, the direction of travel is for resctrl to be/have loadable modules.
Add
resctrl_exit_mon_l3_config()to cleanup any memory allocated byrdt_get_mon_l3_config().To match what patch actually does it looks like this should rather be:
"Addresctrl_exit_mon_l3_config()" -> "Addresctrl_put_mon_l3_config()"Quoted 2 lines
There is no reason to backport this to a stable kernel.
[...]
Quoted 6 lines
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/core.c +0
-0@@ -992,7 +992,13 @@ late_initcall(resctrl_late_init);
Quoted 14 lines
arch/x86/kernel/cpu/resctrl/core.c late_initcall(resctrl_late_init); · line 1003
Quoted 11 lines
arch/x86/kernel/cpu/resctrl/core.c +6
-0static void __exit resctrl_exit(void) { +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +cpuhp_remove_state (rdt_online); + +if (r->mon_capable) +rdt_put_mon_l3_config (r); +rdtgroup_exit (); }I expect cleanup to do the inverse of init. I do not know what was the motivation for the
rdtgroup_exit()to followcpuhp_remove_state()
This will invoke the hotplug callbacks, making it look to resctrl like all CPUs are offline. This means it is then impossible for rdtgroup_exit() to race with the hotplug notifiers. (if you could run this code...)
Quoted 4 lines
but I
was expecting this new cleanup to be done afterrdtgroup_exit()to be inverse of init. This cleanup is inserted in middle of two existing cleanup - could you please elaborate how this location was chosen?
rdtgroup_exit() does nothing with the resctrl structures, it removes sysfs and debugfs entries, and unregisters the filesystem.
Hypothetically, you can't observe any effect of the rmid_ptrs array being freed as all the CPUs are offline and the overflow/limbo threads should have been cancelled. Once cpuhp_remove_state() has been called, this really doesn't matter.
Thanks,
James
Reinette Chatre
maintainerRe: [PATCH v7 02/24] x86/resctrl: kfree() rmid_ptrs from rdtgroup_exit()
Hi James,
Quoted 2 lines · James Morse
Quoted 1 line · Reinette Chatre
On 10/25/2023 11:03 AM, James Morse wrote:
...
Quoted 21 lines
Quoted 6 lines
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/core.c +0
-0@@ -992,7 +992,13 @@ late_initcall(resctrl_late_init);Quoted 14 lines
arch/x86/kernel/cpu/resctrl/core.c late_initcall(resctrl_late_init); · line 1003
Quoted 11 lines
arch/x86/kernel/cpu/resctrl/core.c +6
-0static void __exit resctrl_exit(void) { +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +cpuhp_remove_state (rdt_online); + +if (r->mon_capable) +rdt_put_mon_l3_config (r); +rdtgroup_exit (); }I expect cleanup to do the inverse of init. I do not know what was the motivation for the
rdtgroup_exit()to followcpuhp_remove_state()
Quoted 4 lines
This will invoke the hotplug callbacks, making it look to resctrl like all CPUs are offline. This means it is then impossible for
rdtgroup_exit()to race with the hotplug notifiers. (if you could run this code...)
hmmm ... if there is a risk of such a race would the init code not also be vulnerable to that with the notifiers up before rdtgroup_init()? The races you mention are not obvious to me. I see the filesystem and hotplug code protected against races via the mutex and static keys. Could you please elaborate on the flows of concern?
I am not advocating for cpuhp_remove_state() to be called later. I understand that it simplifies the flows to consider.
Quoted 11 lines
Quoted 4 lines
but I
was expecting this new cleanup to be done afterrdtgroup_exit()to be inverse of init. This cleanup is inserted in middle of two existing cleanup - could you please elaborate how this location was chosen?
rdtgroup_exit()does nothing with the resctrl structures, it removes sysfs and debugfs entries, and unregisters the filesystem.Hypothetically, you can't observe any effect of the
rmid_ptrsarray being freed as all the CPUs are offline and the overflow/limbo threads should have been cancelled. Oncecpuhp_remove_state()has been called, this really doesn't matter.
Sounds like nothing prevents this code from following the custom of cleanup to be inverse of init (yet keep cpuhp_remove_state() first).
Reinette
James Morse
reviewerRe: [PATCH v7 02/24] x86/resctrl: kfree() rmid_ptrs from rdtgroup_exit()
Hi Reinette,
Quoted 29 lines · Reinette Chatre
Hi James,
Quoted 2 lines · James Morse
Quoted 1 line · Reinette Chatre
On 10/25/2023 11:03 AM, James Morse wrote:
...
Quoted 21 lines
Quoted 20 lines
Quoted 7 lines
arch/x86/kernel/cpu/resctrl/core.c +0
-0@@ -992,7 +992,13 @@ late_initcall(resctrl_late_init); static void __exit resctrl_exit(void)arch/x86/kernel/cpu/resctrl/core.c late_initcall(resctrl_late_init); · line 1003
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/core.c +6
-0{ +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +cpuhp_remove_state (rdt_online); + +if (r->mon_capable) +rdt_put_mon_l3_config (r); +rdtgroup_exit (); }I expect cleanup to do the inverse of init. I do not know what was the motivation for the
rdtgroup_exit()to followcpuhp_remove_state()
Quoted 6 lines
Quoted 3 lines
This will invoke the hotplug callbacks, making it look to resctrl like all CPUs are offline. This means it is then impossible for
rdtgroup_exit()to race with the hotplug notifiers. (if you could run this code...)hmmm ... if there is a risk of such a race would the init code not also be vulnerable to that with the notifiers up before
rdtgroup_init()?
Nope, because this array is allocated behind rdt_get_mon_l3_config(), which ultimately comes from get_rdt_resources() in resctrl_late_init() - which calls cpuhp_setup_state() after all this init work has been done.
(cpu hp always gives me a headache1)
Quoted 3 lines
The races you mention
are not obvious to me. I see the filesystem and hotplug code protected against races via the mutex and static keys. Could you please elaborate on the flows of concern?
Functions like __check_limbo() (calling __rmid_entry()) are called under the rdtgroup_mutex, but they don't consider that rmid_ptrs[] may be NULL.
But this could only happen if the limbo work ran after cpuhp_remove_state() - this can't happen because the hotplug callbacks cancel the limbo work, and won't reschedule it if the domain is going offline.
The only other path is via free_rmid(), I've not thought too much about this as resctrl_exit() can't actually be invoked - this code is discarded by the linker.
It could be run on MPAM, but only in response to an 'error interrupt' (which is optional)
- and all the MPAM error interrupts indicate a software bug.
I've only invoked this path once, and rdtgroup_exit()s unregister_filesystem() didn't remove all the files. I anticipate digging into this teardown code more once the bulk of the MPAM driver is upstream.
Quoted 11 lines
I am not advocating for
cpuhp_remove_state()to be called later. I understand that it simplifies the flows to consider.Quoted 8 lines
Quoted 4 lines
but I
was expecting this new cleanup to be done afterrdtgroup_exit()to be inverse of init. This cleanup is inserted in middle of two existing cleanup - could you please elaborate how this location was chosen?
rdtgroup_exit()does nothing with the resctrl structures, it removes sysfs and debugfs entries, and unregisters the filesystem.
Quoted 6 lines
Quoted 3 lines
Hypothetically, you can't observe any effect of the
rmid_ptrsarray being freed as all the CPUs are offline and the overflow/limbo threads should have been cancelled. Oncecpuhp_remove_state()has been called, this really doesn't matter.Sounds like nothing prevents this code from following the custom of cleanup to be inverse of init (yet keep
cpuhp_remove_state()first).
I'll put the the rdt_put_mon_l3_config() call after rdtgroup_exit()...
Thanks,
James
Reinette Chatre
maintainerRe: [PATCH v7 02/24] x86/resctrl: kfree() rmid_ptrs from rdtgroup_exit()
Hi James,
Quoted 36 lines · James Morse
Hi Reinette,
Quoted 32 lines · Reinette Chatre
Hi James,
Quoted 2 lines · James Morse
Quoted 1 line · Reinette Chatre
On 10/25/2023 11:03 AM, James Morse wrote:
...
Quoted 24 lines
Quoted 6 lines
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/core.c +0
-0@@ -992,7 +992,13 @@ late_initcall(resctrl_late_init);Quoted 14 lines
arch/x86/kernel/cpu/resctrl/core.c late_initcall(resctrl_late_init); · line 1003
Quoted 11 lines
arch/x86/kernel/cpu/resctrl/core.c +6
-0static void __exit resctrl_exit(void) { +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +cpuhp_remove_state (rdt_online); + +if (r->mon_capable) +rdt_put_mon_l3_config (r); +rdtgroup_exit (); }I expect cleanup to do the inverse of init. I do not know what was the motivation for the
rdtgroup_exit()to followcpuhp_remove_state()This will invoke the hotplug callbacks, making it look to resctrl like all CPUs are offline. This means it is then impossible for
rdtgroup_exit()to race with the hotplug notifiers. (if you could run this code...)
Quoted 8 lines
Quoted 2 lines
hmmm ... if there is a risk of such a race would the init code not also be vulnerable to that with the notifiers up before
rdtgroup_init()?Nope, because this array is allocated behind
rdt_get_mon_l3_config(), which ultimately comes fromget_rdt_resources()inresctrl_late_init()- which callscpuhp_setup_state()after all this init work has been done.(cpu hp always gives me a headache1)
Right. My comment was actually and specifically about rdtgroup_init() and attempting to understand your view of its races with the hotplug notifiers in response to your comment about its (the hotplug notifiers) races with rdtgroup_exit().
The current order of state initialization you mention and hotplug notifiers needing the state is sane and implies to expect an inverse order of teardown.
Quoted 12 lines
Quoted 3 lines
The races you mention
are not obvious to me. I see the filesystem and hotplug code protected against races via the mutex and static keys. Could you please elaborate on the flows of concern?Functions like
__check_limbo()(calling__rmid_entry()) are called under therdtgroup_mutex, but they don't consider that rmid_ptrs[] may be NULL.But this could only happen if the limbo work ran after
cpuhp_remove_state()- this can't happen because the hotplug callbacks cancel the limbo work, and won't reschedule it if the domain is going offline.
Quoted 5 lines
The only other path is via
free_rmid(), I've not thought too much about this asresctrl_exit()can't actually be invoked - this code is discarded by the linker.It could be run on MPAM, but only in response to an 'error interrupt' (which is optional)
- and all the MPAM error interrupts indicate a software bug.
This still just considers the resctrl state and hotplug notifiers.
I clearly am missing something. It is still not clear to me how this connects to your earlier comment about races with the rdtgroup_exit() code ... how the hotplug notifiers races with the filesystem register/unregister code.
Quoted 17 lines
I've only invoked this path once, and rdtgroup_exit()s
unregister_filesystem()didn't remove all the files. I anticipate digging into this teardown code more once the bulk of the MPAM driver is upstream.Quoted 11 lines
I am not advocating for
cpuhp_remove_state()to be called later. I understand that it simplifies the flows to consider.Quoted 8 lines
Quoted 4 lines
but I
was expecting this new cleanup to be done afterrdtgroup_exit()to be inverse of init. This cleanup is inserted in middle of two existing cleanup - could you please elaborate how this location was chosen?
rdtgroup_exit()does nothing with the resctrl structures, it removes sysfs and debugfs entries, and unregisters the filesystem.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Hypothetically, you can't observe any effect of the
rmid_ptrsarray being freed as all the CPUs are offline and the overflow/limbo threads should have been cancelled. Oncecpuhp_remove_state()has been called, this really doesn't matter.Sounds like nothing prevents this code from following the custom of cleanup to be inverse of init (yet keep
cpuhp_remove_state()first).I'll put the the
rdt_put_mon_l3_config()call afterrdtgroup_exit()...
thank you
Reinette
James Morse
reviewerRe: [PATCH v7 02/24] x86/resctrl: kfree() rmid_ptrs from rdtgroup_exit()
Hi Reinette,
Quoted 42 lines · Reinette Chatre
Quoted 10 lines · James Morse
Quoted 8 lines · Reinette Chatre
Quoted 7 lines · James Morse
Quoted 2 lines · Reinette Chatre
I expect cleanup to do the inverse of init. I do not know what was the motivation for the
rdtgroup_exit()to followcpuhp_remove_state()This will invoke the hotplug callbacks, making it look to resctrl like all CPUs are offline. This means it is then impossible for
rdtgroup_exit()to race with the hotplug notifiers. (if you could run this code...)Quoted 8 lines
Quoted 2 lines
hmmm ... if there is a risk of such a race would the init code not also be vulnerable to that with the notifiers up before
rdtgroup_init()?Nope, because this array is allocated behind
rdt_get_mon_l3_config(), which ultimately comes fromget_rdt_resources()inresctrl_late_init()- which callscpuhp_setup_state()after all this init work has been done.(cpu hp always gives me a headache1)
Right. My comment was actually and specifically about
rdtgroup_init()and attempting to understand your view of its races with the hotplug notifiers in response to your comment about its (the hotplug notifiers) races withrdtgroup_exit().The current order of state initialization you mention and hotplug notifiers needing the state is sane and implies to expect an inverse order of teardown.
Quoted 15 lines
Quoted 3 lines
The races you mention
are not obvious to me. I see the filesystem and hotplug code protected against races via the mutex and static keys. Could you please elaborate on the flows of concern?Functions like
__check_limbo()(calling__rmid_entry()) are called under therdtgroup_mutex, but they don't consider that rmid_ptrs[] may be NULL.But this could only happen if the limbo work ran after
cpuhp_remove_state()- this can't happen because the hotplug callbacks cancel the limbo work, and won't reschedule it if the domain is going offline.The only other path is via
free_rmid(), I've not thought too much about this asresctrl_exit()can't actually be invoked - this code is discarded by the linker.
Quoted 8 lines
Quoted 2 lines
It could be run on MPAM, but only in response to an 'error interrupt' (which is optional)
- and all the MPAM error interrupts indicate a software bug.This still just considers the resctrl state and hotplug notifiers.
I clearly am missing something. It is still not clear to me how this connects to your earlier comment about races with the
rdtgroup_exit()code ... how the hotplug notifiers races with the filesystem register/unregister code.
I don't think there is a specific problem there, this was mostly about unexpected surprises because cpuhp/limbo_handler/overflow_handler all run asynchronously. I may also have added confusion because the code added here moves into rdtgroup_exit() which is renamed resctrl_exit() as part of dragging all this out to /fs/. (This is also why I tried to initially add it in its final location)
Thanks,
James
Re: [PATCH v7 02/24] x86/resctrl: kfree() rmid_ptrs from rdtgroup_exit()
In reply to James MorseHi James,
Quoted 11 lines · James Morse
rmid_ptrs[] is allocated from
dom_data_init()but never free()d.While the exit text ends up in the linker script's DISCARD section, the direction of travel is for resctrl to be/have loadable modules.
Add
resctrl_exit_mon_l3_config()to cleanup any memory allocated byrdt_get_mon_l3_config().There is no reason to backport this to a stable kernel.
Signed-off-by James Morse <[email protected]>
Quoted 77 lines
---
Changes since v5:
* This patch is newChanges since v6:
* Removed structrdt_resourceargument, added__exitmarkers to match the only caller.
* Adedd a whole stack of functions to maintain symmetry.--- arch/x86/kernel/cpu/resctrl/core.c | 6 ++++++ arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 15 +++++++++++++++ 3 files changed, 22 insertions(+)arch/x86/kernel/cpu/resctrl/core.c +6
-0@@ -992,7 +992,13 @@ late_initcall(resctrl_late_init); static void __exit resctrl_exit(void) { +struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +cpuhp_remove_state (rdt_online); + +if (r->mon_capable) +rdt_put_mon_l3_config (r); +rdtgroup_exit (); }arch/x86/kernel/cpu/resctrl/internal.h +1
-0@@ -546,6 +546,7 @@ void closid_free(int closid);int alloc_rmid (void );void free_rmid (u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r); +void __exit rdt_put_mon_l3_config(struct rdt_resource *r);bool __init rdt_cpu_has (int flag);void mon_event_count (void *info);int rdtgroup_mondata_show (struct seq_file *m,void *arg);arch/x86/kernel/cpu/resctrl/monitor.c +15
-0@@ -741,6 +741,16 @@ static int dom_data_init(struct rdt_resource *r)return 0 ; } +static void __exit dom_data_exit(struct rdt_resource *r) +{ +mutex_lock (&rdtgroup_mutex); + +kfree (rmid_ptrs); + rmid_ptrs =NULL ; + +mutex_unlock (&rdtgroup_mutex); +} +static struct mon_evt llc_occupancy_event = { .name ="llc_occupancy" , .evtid =QOS_L3_OCCUP_EVENT_ID ,@@ -830,6 +840,11 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)return 0 ; } +void __exit rdt_put_mon_l3_config(struct rdt_resource *r) +{ +dom_data_exit (r); +} +void __init intel_rdt_mbm_apply_quirk (void ) {int cf_index;
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 02/24] x86/resctrl: kfree() rmid_ptrs from rdtgroup_exit()
Hi Babu,
Quoted 6 lines · Babu
Quoted 5 lines · James Morse
rmid_ptrs[] is allocated from
dom_data_init()but never free()d.While the exit text ends up in the linker script's DISCARD section, the direction of travel is for resctrl to be/have loadable modules.
Quoted 8 lines
Quoted 6 lines
Add
resctrl_exit_mon_l3_config()to cleanup any memory allocated byrdt_get_mon_l3_config().There is no reason to backport this to a stable kernel.
Signed-off-by James Morse <[email protected]>Reviewed-by Babu Moger <[email protected]>
Thanks!
James
James Morse
reviewerPATCH v7 3/24 x86/resctrl: Create helper for RMID allocation and mondata dir creation
In reply to James MorseWhen monitoring is supported, each monitor and control group is allocated an RMID. For control groups, rdtgroup_mkdir_ctrl_mon() later goes on to allocate the CLOSID.
MPAM's equivalent of RMID are not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
Move the RMID allocation and mondata dir creation to a helper, this makes a subsequent change easier to read.
---
Changes since v4:
* Fixed typo in commit message, moved some words around.
No changes since v6
--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 42 +++++++++++++++++--------- 1 file changed, 27 insertions(+), 15 deletions(-)
arch/x86/kernel/cpu/resctrl/rdtgroup.c +27 -15
@@ -3293,6 +3293,30 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)return ret; } +static int mkdir_rdt_prepare_rmid_alloc (struct rdtgroup *rdtgrp) +{ +int ret; + +if (!rdt_mon_capable) +return 0 ; + + ret =alloc_rmid (); +if (ret <0 ) { +rdt_last_cmd_puts ("Out of RMIDs\n" ); +return ret; + } + rdtgrp->mon.rmid = ret; + + ret =mkdir_mondata_all (rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn); +if (ret) { +rdt_last_cmd_puts ("kernfs subdir error\n" ); +free_rmid (rdtgrp->mon.rmid); +return ret; + } + +return 0 ; +} +static int mkdir_rdt_prepare (struct kernfs_node *parent_kn,const char *name,umode_t mode,enum rdt_group_type rtype,struct rdtgroup **r)
@@ -3365,20 +3389,10 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } -if (rdt_mon_capable) { - ret =alloc_rmid (); -if (ret <0 ) { -rdt_last_cmd_puts ("Out of RMIDs\n" ); - goto out_destroy; - } - rdtgrp->mon.rmid = ret; + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) + goto out_destroy; - ret =mkdir_mondata_all (kn, rdtgrp, &rdtgrp->mon.mon_data_kn); -if (ret) { -rdt_last_cmd_puts ("kernfs subdir error\n" ); - goto out_idfree; - } - }kernfs_activate (kn);/*
@@ -3386,8 +3400,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,*/ return 0 ; -out_idfree: -free_rmid (rdtgrp->mon.rmid); out_destroy:kernfs_put (rdtgrp->kn);kernfs_remove (rdtgrp->kn);
Signature · 1 line
2.39.2
Reinette Chatre
maintainerRe: [PATCH v7 03/24] x86/resctrl: Create helper for RMID allocation and mondata dir creation
Hi James,
Quoted 9 lines · James Morse
When monitoring is supported, each monitor and control group is allocated an RMID. For control groups,
rdtgroup_mkdir_ctrl_mon()later goes on to allocate the CLOSID.MPAM's equivalent of RMID are not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
Quoted 5 lines
The RMID allocation needs to move to be after the CLOSID has been allocated.
Move the RMID allocation and mondata dir creation to a helper, this makes a subsequent change easier to read.
Boris recently reminded us that text like "subsequent change" loses meaning once the commit forms part of the kernel. This term is used in a couple of places in this series. Please review all changelogs to remove this term to avoid another re-spin.
For reference:
https://lore.kernel.org/lkml/20231009171918.GPZSQ2Frs%2Fqp129wsP@fat_crate.local/
Quoted 7 lines
Tested-by Peter Newman <[email protected]>Tested-by Shaopeng Tan <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Ilpo Järvinen <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Please have tags follow "Ordering of commit tags" in Documentation/process/maintainer-tip.rst (author SOB appears before the Tested-by/Reviewed-by/Acked-by).
Reinette
Re: [PATCH v7 03/24] x86/resctrl: Create helper for RMID allocation and mondata dir creation
In reply to James MorseHi James,
Quoted 15 lines · James Morse
When monitoring is supported, each monitor and control group is allocated an RMID. For control groups,
rdtgroup_mkdir_ctrl_mon()later goes on to allocate the CLOSID.MPAM's equivalent of RMID are not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
Move the RMID allocation and mondata dir creation to a helper, this makes a subsequent change easier to read.
Quoted 6 lines
Tested-by Peter Newman <[email protected]>Tested-by Shaopeng Tan <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Ilpo Järvinen <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 77 lines
---
Changes since v4:
* Fixed typo in commit message, moved some words around.No changes since v6
--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 42 +++++++++++++++++--------- 1 file changed, 27 insertions(+), 15 deletions(-)arch/x86/kernel/cpu/resctrl/rdtgroup.c +27
-15@@ -3293,6 +3293,30 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)return ret; } +static int mkdir_rdt_prepare_rmid_alloc (struct rdtgroup *rdtgrp) +{ +int ret; + +if (!rdt_mon_capable) +return 0 ; + + ret =alloc_rmid (); +if (ret <0 ) { +rdt_last_cmd_puts ("Out of RMIDs\n" ); +return ret; + } + rdtgrp->mon.rmid = ret; + + ret =mkdir_mondata_all (rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn); +if (ret) { +rdt_last_cmd_puts ("kernfs subdir error\n" ); +free_rmid (rdtgrp->mon.rmid); +return ret; + } + +return 0 ; +} +static int mkdir_rdt_prepare (struct kernfs_node *parent_kn,const char *name,umode_t mode,enum rdt_group_type rtype,struct rdtgroup **r)@@ -3365,20 +3389,10 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } -if (rdt_mon_capable) { - ret =alloc_rmid (); -if (ret <0 ) { -rdt_last_cmd_puts ("Out of RMIDs\n" ); - goto out_destroy; - } - rdtgrp->mon.rmid = ret; + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) + goto out_destroy; - ret =mkdir_mondata_all (kn, rdtgrp, &rdtgrp->mon.mon_data_kn); -if (ret) { -rdt_last_cmd_puts ("kernfs subdir error\n" ); - goto out_idfree; - } - }kernfs_activate (kn);/* @@ -3386,8 +3400,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,*/ return 0 ; -out_idfree: -free_rmid (rdtgrp->mon.rmid); out_destroy:kernfs_put (rdtgrp->kn);kernfs_remove (rdtgrp->kn);
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 03/24] x86/resctrl: Create helper for RMID allocation and mondata dir creation
Hi Babu,
Quoted 16 lines · Babu
Quoted 15 lines · James Morse
When monitoring is supported, each monitor and control group is allocated an RMID. For control groups,
rdtgroup_mkdir_ctrl_mon()later goes on to allocate the CLOSID.MPAM's equivalent of RMID are not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
Move the RMID allocation and mondata dir creation to a helper, this makes a subsequent change easier to read.
Quoted 8 lines
Quoted 6 lines
Tested-by Peter Newman <[email protected]>Tested-by Shaopeng Tan <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Ilpo Järvinen <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>Reviewed-by Babu Moger <[email protected]>
Thanks!
James
James Morse
reviewerPATCH v7 4/24 x86/resctrl: Move rmid allocation out of mkdir_rdt_prepare()
In reply to James MorseRMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups, rdtgroup_mkdir_ctrl_mon() later goes on to allocate the CLOSID.
MPAM's equivalent of RMID is not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
Move the RMID allocation out of mkdir_rdt_prepare() to occur in its caller, after the mkdir_rdt_prepare() call. This allows the RMID allocator to know the CLOSID.
---
Changes since v2:
* Moved kernfs_activate() later to preserve atomicity of files being visible
Changes since v5:
* Renamed out_id_free as out_closid_free.
No changes since v6
--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 35 +++++++++++++++++++------- 1 file changed, 26 insertions(+), 9 deletions(-)
arch/x86/kernel/cpu/resctrl/rdtgroup.c +26 -9
@@ -3317,6 +3317,12 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)return 0 ; } +static void mkdir_rdt_prepare_rmid_free (struct rdtgroup *rgrp) +{ +if (rdt_mon_capable) +free_rmid (rgrp->mon.rmid); +} +static int mkdir_rdt_prepare (struct kernfs_node *parent_kn,const char *name,umode_t mode,enum rdt_group_type rtype,struct rdtgroup **r)
@@ -3389,12 +3395,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } - ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); -if (ret) - goto out_destroy; - -kernfs_activate (kn); -/* * The caller unlocks the parent_kn upon success. */
@@ -3413,7 +3413,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,static void mkdir_rdt_prepare_clean (struct rdtgroup *rgrp) {kernfs_remove (rgrp->kn); -free_rmid (rgrp->mon.rmid);rdtgroup_remove (rgrp); }
@@ -3435,12 +3434,21 @@ static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn, prgrp = rdtgrp->mon.parent; rdtgrp->closid = prgrp->closid; + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) { +mkdir_rdt_prepare_clean (rdtgrp); + goto out_unlock; + } + +kernfs_activate (rdtgrp->kn); +/* * Add the rdtgrp to the list of rdtgrps the parent * ctrl_mon group has to track. */ list_add_tail (&rdtgrp->mon.crdtgrp_list, &prgrp->mon.crdtgrp_list); +out_unlock:rdtgroup_kn_unlock (parent_kn);return ret; }
@@ -3471,9 +3479,16 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, ret =0 ; rdtgrp->closid = closid; + + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) + goto out_closid_free; + +kernfs_activate (rdtgrp->kn); + ret =rdtgroup_init_alloc (rdtgrp);if (ret <0 ) - goto out_id_free; + goto out_rmid_free;list_add (&rdtgrp->rdtgroup_list, &rdt_all_groups);
@@ -3493,7 +3508,9 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, out_del_list:list_del (&rdtgrp->rdtgroup_list); -out_id_free: +out_rmid_free: +mkdir_rdt_prepare_rmid_free (rdtgrp); +out_closid_free:closid_free (closid); out_common_fail:mkdir_rdt_prepare_clean (rdtgrp);
Signature · 1 line
2.39.2
Re: [PATCH v7 04/24] x86/resctrl: Move rmid allocation out of mkdir_rdt_prepare()
Hi James,
Quoted 38 lines · James Morse
RMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups,
rdtgroup_mkdir_ctrl_mon()later goes on to allocate the CLOSID.MPAM's equivalent of RMID is not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
Move the RMID allocation out of
mkdir_rdt_prepare()to occur in its caller, after themkdir_rdt_prepare()call. This allows the RMID allocator to know the CLOSID.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since v2:
* Movedkernfs_activate()later to preserve atomicity of files being visibleChanges since v5:
* Renamedout_id_freeasout_closid_free.No changes since v6
--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 35 +++++++++++++++++++------- 1 file changed, 26 insertions(+), 9 deletions(-)arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0
arch/x86/kernel/cpu/resctrl/rdtgroup.c static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) · line 3324
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +5
-0@@ -3317,6 +3317,12 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)return 0 ; } +static void mkdir_rdt_prepare_rmid_free (struct rdtgroup *rgrp) +{ +if (rdt_mon_capable) +free_rmid (rgrp->mon.rmid); +}
The check if (rdt_mon_capable) is extra here. Not required. Otherwise looks good.
James Morse
reviewerRe: [PATCH v7 04/24] x86/resctrl: Move rmid allocation out of mkdir_rdt_prepare()
Hi Babu,
Quoted 25 lines · Babu
Quoted 24 lines · James Morse
RMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups,
rdtgroup_mkdir_ctrl_mon()later goes on to allocate the CLOSID.MPAM's equivalent of RMID is not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
Move the RMID allocation out of
mkdir_rdt_prepare()to occur in its caller, after themkdir_rdt_prepare()call. This allows the RMID allocator to know the CLOSID.arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0@@ -3317,6 +3317,12 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)return 0 ; }
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) · line 3324
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +5
-0+static void mkdir_rdt_prepare_rmid_free (struct rdtgroup *rgrp) +{ +if (rdt_mon_capable) +free_rmid (rgrp->mon.rmid); +}The check if (
rdt_mon_capable) is extra here. Not required. Otherwise looks good.
It pairs with the behaviour of mkdir_rdt_prepare_rmid_alloc(), which returns early if (!rdt_mon_capable). This avoids double-adding rmid-0 to the limbo list.
Quoted 1 line
Reviewed-by Babu Moger <[email protected]>
Thanks! I assume you're happy with my above description of the rdt_mon_capable check is needed.
Thanks,
James
James Morse
reviewerPATCH v7 5/24 x86/resctrl: Track the closid with the rmid
In reply to James Morsex86's RMID are independent of the CLOSID. An RMID can be allocated, used and freed without considering the CLOSID.
MPAM's equivalent feature is PMG, which is not an independent number, it extends the CLOSID/PARTID space. For MPAM, only PMG-bits worth of 'RMID' can be allocated for a single CLOSID.
i.e. if there is 1 bit of PMG space, then each CLOSID can have two monitor groups.
To allow resctrl to disambiguate RMID values for different CLOSID, everything in resctrl that keeps an RMID value needs to know the CLOSID too. This will always be ignored on x86.
---
Is there a better term for 'the unique identifier for a monitor group'. Using RMID for that here may be confusing...
Changes since v1:
* Added comment in struct rmid_entry
Changes since v2:
* Moved X86_RESCTRL_BAD_CLOSID from a subsequent patch
Chances since v3:
* Renamed X86_RESCTRL_BAD_CLOSID to EMPTY
* Clarified a few comments and kernel-doc
Changes since v5:
* Use entry->closid from the iterator, instead of the parent control group.
* Move the reserved defines into this patch to reduce the churn.
* Added some kernel doc.
* Renamed some arch closid parameters as 'unused'.
Changes since v6:
* Changes to comments.
--- arch/x86/include/asm/resctrl.h | 7 +++ arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 74 ++++++++++++++--------- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 12 ++-- include/linux/resctrl.h | 16 ++++- 6 files changed, 77 insertions(+), 38 deletions(-)
arch/x86/include/asm/resctrl.h +7 -0
@@ -7,6 +7,13 @@#include <linux/sched.h> #include <linux/jump_label.h> +/* +* This value can never be a valid CLOSID, and is used when mapping a +* (closid, rmid) pair to an index and back. On x86 only the RMID is +* needed. The index is a software defined value. +*/ +#define X86_RESCTRL_EMPTY_CLOSID ((u32)~0) +/** * struct resctrl_pqr_state - State cache for the PQR MSR * @cur_rmid: The cached Resource Monitoring ID
arch/x86/kernel/cpu/resctrl/internal.h +1 -1
@@ -544,7 +544,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r);int closids_supported (void );void closid_free (int closid);int alloc_rmid (void ); -void free_rmid (u32 rmid); +void free_rmid (u32 closid,u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r); void __exit rdt_put_mon_l3_config(struct rdt_resource *r);bool __init rdt_cpu_has (int flag);
arch/x86/kernel/cpu/resctrl/monitor.c +47 -27
207 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +2 -2
@@ -752,7 +752,7 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)* anymore when this group would be used for pseudo-locking. This * is safe to call on platforms not capable of monitoring. */ -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); ret =0 ; goto out;
@@ -787,7 +787,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) ret =rdtgroup_locksetup_user_restore (rdtgrp);if (ret) { -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);return ret; }
arch/x86/kernel/cpu/resctrl/rdtgroup.c +6 -6
@@ -2842,7 +2842,7 @@ static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp) head = &rdtgrp->mon.crdtgrp_list;list_for_each_entry_safe (sentry, stmp, head, mon.crdtgrp_list) { -free_rmid (sentry->mon.rmid); +free_rmid (sentry->closid, sentry->mon.rmid);list_del (&sentry->mon.crdtgrp_list);if (atomic_read (&sentry->waitcount) !=0 )
@@ -2882,7 +2882,7 @@ static void rmdir_all_sub(void)cpumask_or (&rdtgroup_default.cpu_mask, &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask); -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);kernfs_remove (rdtgrp->kn);list_del (&rdtgrp->rdtgroup_list);
@@ -3310,7 +3310,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) ret =mkdir_mondata_all (rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn);if (ret) {rdt_last_cmd_puts ("kernfs subdir error\n" ); -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);return ret; }
@@ -3320,7 +3320,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)static void mkdir_rdt_prepare_rmid_free (struct rdtgroup *rgrp) {if (rdt_mon_capable) -free_rmid (rgrp->mon.rmid); +free_rmid (rgrp->closid, rgrp->mon.rmid); } static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,
@@ -3579,7 +3579,7 @@ static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)update_closid_rmid (tmpmask,NULL ); rdtgrp->flags =RDT_DELETED ; -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);/* * Remove the rdtgrp from the parent ctrl_mon group's list
@@ -3625,8 +3625,8 @@ static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)cpumask_or (tmpmask, tmpmask, &rdtgrp->cpu_mask);update_closid_rmid (tmpmask,NULL ); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);closid_free (rdtgrp->closid); -free_rmid (rdtgrp->mon.rmid);rdtgroup_ctrl_remove (rdtgrp);
include/linux/resctrl.h +14 -2
@@ -6,6 +6,10 @@#include <linux/list.h> #include <linux/pid.h> +/* CLOSID, RMID value used by the default control group */ +#define RESCTRL_RESERVED_CLOSID 0 +#define RESCTRL_RESERVED_RMID 0 +#ifdef CONFIG_PROC_CPU_RESCTRL int proc_resctrl_show(struct seq_file *m,
@@ -225,6 +229,9 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);* for this resource and domain. * @r: resource that the counter should be read from. * @d: domain that the counter should be read from. +* @closid: closid that matches the rmid. Depending on the architecture, the +* counter may match traffic of both @closid and @rmid, or @rmid +* only. * @rmid: rmid of the counter to read. * @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes.
@@ -235,20 +242,25 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);* 0 on success, or -EIO, -EINVAL etc on error. */ int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid,u64 *val); +u32 closid,u32 rmid,enum resctrl_event_id eventid, +u64 *val); +/** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid * and eventid. * @r: The domain's resource. * @d: The rmid's domain. +* @closid: closid that matches the rmid. Depending on the architecture, the +* counter may match traffic of both @closid and @rmid, or @rmid only. * @rmid: The rmid whose counter values should be reset. * @eventid: The eventid whose counter values should be reset. * * This can be called from any CPU. */ void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid); +u32 closid,u32 rmid, +enum resctrl_event_id eventid);/** * resctrl_arch_reset_rmid_all() - Reset all private state associated with
Signature · 1 line
2.39.2
Reinette Chatre
maintainerRe: [PATCH v7 05/24] x86/resctrl: Track the closid with the rmid
Hi James,
arch/x86/kernel/cpu/resctrl/monitor.c static int dom_data_init(struct rdt_resource *r) · line 756
Quoted 10 lines · James Morse
arch/x86/kernel/cpu/resctrl/monitor.c +2
-4@@ -732,10 +752,10 @@ static int dom_data_init(struct rdt_resource *r) }/* -* RMID 0 is special and is always allocated. It's used for all -* tasks that are not monitored. -*/ - entry =__rmid_entry (0 ); +* These are used for rdtgroup_default control group, which will be +* setup later in rdtgroup_init().
This sentence I provided was intended to just replace two sentences, not the entire paragraph you had. Note that I explicitly stated "change last two sentences". Without the original text the above "These" is not clear. If you just want to copy&paste here is the entire intended comment:
RESCTRL_RESERVED_CLOSID and RESCTRL_RESERVED_RMID are special and are always allocated. These are used for the rdtgroup_default control group, which will be setup later in rdtgroup_init().
Reinette
James Morse
reviewerRe: [PATCH v7 05/24] x86/resctrl: Track the closid with the rmid
Hi Reinette,
Quoted 17 lines · Reinette Chatre
arch/x86/kernel/cpu/resctrl/monitor.c static int dom_data_init(struct rdt_resource *r) · line 756
Quoted 10 lines · James Morse
arch/x86/kernel/cpu/resctrl/monitor.c +2
-4@@ -732,10 +752,10 @@ static int dom_data_init(struct rdt_resource *r) }/* -* RMID 0 is special and is always allocated. It's used for all -* tasks that are not monitored. -*/ - entry =__rmid_entry (0 ); +* These are used for rdtgroup_default control group, which will be +* setup later in rdtgroup_init(). This sentence I provided was intended to just replace two sentences, not the entire paragraph you had. Note that I explicitly stated "change last two sentences". Without the original text the above "These" is not clear. If you just want to copy&paste here is the entire intended comment:
Quoted 3 lines
RESCTRL_RESERVED_CLOSIDandRESCTRL_RESERVED_RMIDare special and are always allocated. These are used for therdtgroup_defaultcontrol group, which will be setup later inrdtgroup_init().
Thanks! I probably got muddled with whether this was against the existing code or not.
James
Moger, Babu
reviewerRe: [PATCH v7 05/24] x86/resctrl: Track the closid with the rmid
In reply to James MorseHi James,
Quoted 136 lines · James Morse
x86's RMID are independent of the CLOSID. An RMID can be allocated, used and freed without considering the CLOSID.
MPAM's equivalent feature is PMG, which is not an independent number, it extends the CLOSID/PARTID space. For MPAM, only PMG-bits worth of 'RMID' can be allocated for a single CLOSID.
i.e. if there is 1 bit of PMG space, then each CLOSID can have two monitor groups.To allow resctrl to disambiguate RMID values for different CLOSID, everything in resctrl that keeps an RMID value needs to know the CLOSID too. This will always be ignored on x86.
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Xin Hao <[email protected]>Signed-off-by James Morse <[email protected]>---
Is there a better term for 'the unique identifier for a monitor group'. Using RMID for that here may be confusing...Changes since v1:
* Added comment in structrmid_entryChanges since v2:
* MovedX86_RESCTRL_BAD_CLOSIDfrom a subsequent patchChances since v3:
* RenamedX86_RESCTRL_BAD_CLOSIDto EMPTY
* Clarified a few comments and kernel-docChanges since v5:
* Useentry->closidfrom the iterator, instead of the parent control group.
* Move the reserved defines into this patch to reduce the churn.
* Added some kernel doc.
* Renamed some arch closid parameters as 'unused'.Changes since v6:
* Changes to comments.--- arch/x86/include/asm/resctrl.h | 7 +++ arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 74 ++++++++++++++--------- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 12 ++-- include/linux/resctrl.h | 16 ++++- 6 files changed, 77 insertions(+), 38 deletions(-)arch/x86/include/asm/resctrl.h +7
-0@@ -7,6 +7,13 @@#include <linux/sched.h> #include <linux/jump_label.h> +/* +* This value can never be a valid CLOSID, and is used when mapping a +* (closid, rmid) pair to an index and back. On x86 only the RMID is +* needed. The index is a software defined value. +*/ +#define X86_RESCTRL_EMPTY_CLOSID ((u32)~0) +/** * struct resctrl_pqr_state - State cache for the PQR MSR * @cur_rmid: The cached Resource Monitoring ID arch/x86/kernel/cpu/resctrl/internal.h +1
-1@@ -544,7 +544,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r);int closids_supported (void );void closid_free (int closid);int alloc_rmid (void ); -void free_rmid (u32 rmid); +void free_rmid (u32 closid,u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r); void __exit rdt_put_mon_l3_config(struct rdt_resource *r);bool __init rdt_cpu_has (int flag);arch/x86/kernel/cpu/resctrl/monitor.c +18
-3@@ -24,7 +24,20 @@#include "internal.h" +/** +* struct rmid_entry - dirty tracking for all RMID. +* @closid: The CLOSID for this entry. +* @rmid: The RMID for this entry. +* @busy: The number of domains with cached data using this RMID. +* @list: Member of the rmid_free_lru list when busy == 0. +* +* Depending on the architecture the correct monitor is accessed using +* both @closid and @rmid, or @rmid only. +* +* Take the rdtgroup_mutex when accessing. +*/ struct rmid_entry { +u32 closid;u32 rmid;int busy;struct list_head list;@@ -136,7 +149,7 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val)return val; } -static inline struct rmid_entry *__rmid_entry (u32 rmid) +static inline struct rmid_entry *__rmid_entry (u32 closid,u32 rmid) {struct rmid_entry *entry;@@ -190,7 +203,8 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom, }void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid) +u32 unused,u32 rmid, +enum resctrl_event_id eventid) {struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);struct arch_mbm_state *am;@@ -230,7 +244,8 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width) }int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid,u64 *val) +u32 unused,u32 rmid,enum resctrl_event_id eventid, +u64 *val) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r);struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);
arch/x86/kernel/cpu/resctrl/monitor.c void __check_limbo(struct rdt_domain *d, bool force_free) · line 303
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/monitor.c +1
-1@@ -285,9 +300,9 @@ void __check_limbo(struct rdt_domain *d, bool force_free)if (nrmid >= r->num_rmid)break ; - entry =__rmid_entry (nrmid); + entry =__rmid_entry (X86_RESCTRL_EMPTY_CLOSID , nrmid);// temporary
What is temporary means here? Can you please elaborate(or remove)?
Thanks
Babu Moger
James Morse
reviewerRe: [PATCH v7 05/24] x86/resctrl: Track the closid with the rmid
Hi Babu,
Quoted 22 lines · Babu
Quoted 21 lines · James Morse
x86's RMID are independent of the CLOSID. An RMID can be allocated, used and freed without considering the CLOSID.
MPAM's equivalent feature is PMG, which is not an independent number, it extends the CLOSID/PARTID space. For MPAM, only PMG-bits worth of 'RMID' can be allocated for a single CLOSID.
i.e. if there is 1 bit of PMG space, then each CLOSID can have two monitor groups.To allow resctrl to disambiguate RMID values for different CLOSID, everything in resctrl that keeps an RMID value needs to know the CLOSID too. This will always be ignored on x86.
arch/x86/kernel/cpu/resctrl/monitor.c +0
-0@@ -285,9 +300,9 @@ void __check_limbo(struct rdt_domain *d, bool force_free)if (nrmid >= r->num_rmid)break ;
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/monitor.c void __check_limbo(struct rdt_domain *d, bool force_free) · line 303
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/monitor.c +1
-1- entry =__rmid_entry (nrmid); + entry =__rmid_entry (X86_RESCTRL_EMPTY_CLOSID , nrmid);// temporary What is temporary means here? Can you please elaborate(or remove)?
It means this line gets removed in a subsequent patch - don't spend too long rationalising the values used. This one gets removed in the next patch as __check_limbo() is changed to operate by index instead.
Thanks,
James
James Morse
reviewerPATCH v7 6/24 x86/resctrl: Access per-rmid structures by index
In reply to James Morsex86 systems identify traffic using the CLOSID and RMID. The CLOSID is used to lookup the control policy, the RMID is used for monitoring. For x86 these are independent numbers.
Arm's MPAM has equivalent features PARTID and PMG, where the PARTID is used to lookup the control policy. The PMG in contrast is a small number of bits that are used to subdivide PARTID when monitoring. The cache-occupancy monitors require the PARTID to be specified when monitoring.
This means MPAM's PMG field is not unique. There are multiple PMG-0, one per allocated CLOSID/PARTID. If PMG is treated as equivalent to RMID, it cannot be allocated as an independent number. Bitmaps like rmid_busy_llc need to be sized by the number of unique entries for this resource.
Treat the combined CLOSID and RMID as an index, and provide architecture helpers to pack and unpack an index. This makes the MPAM values unique. The domain's rmid_busy_llc and rmid_ptrs[] are then sized by index, as are domain mbm_local[] and mbm_total[].
x86 can ignore the CLOSID field when packing and unpacking an index, and report as many indexes as RMID.
---
Changes since v1:
* Added X86_BAD_CLOSID macro to make it clear what this value means
* Added second WARN_ON() for closid checking, and made both _ONCE()
Changes since v2:
* Added RESCTRL_RESERVED_CLOSID
* Removed a newline
* Repharsed some comments
* Renamed a variable 'ignore'd
* Moved X86_RESCTRL_BAD_CLOSID to a previous patch
Changes since v3:
* Changed a variable name
* Fixed various typos
Changes since v4:
* Removed resource parameter from has_busy_rmid()
* Rewrote commit message
Changes since v5:
* Used RESCTRL_RESERVED_RMID in clear_closid_rmid().
* Added comment against free_rmid()s index comparison tricks.
Changes since v6:
* Added a newline between some #includes.
* Juggled some plurals in a comment.
--- arch/x86/include/asm/resctrl.h | 17 +++++ arch/x86/kernel/cpu/resctrl/core.c | 5 +- arch/x86/kernel/cpu/resctrl/internal.h | 4 +- arch/x86/kernel/cpu/resctrl/monitor.c | 96 ++++++++++++++++++-------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 9 +-- 5 files changed, 94 insertions(+), 37 deletions(-)
arch/x86/include/asm/resctrl.h +17 -0
@@ -101,6 +101,23 @@ static inline void resctrl_sched_in(struct task_struct *tsk) __resctrl_sched_in(tsk); } +static inline u32 resctrl_arch_system_num_rmid_idx (void ) +{ +/* RMID are independent numbers for x86. num_rmid_idx == num_rmid */ +return boot_cpu_data.x86_cache_max_rmid +1 ; +} + +static inline void resctrl_arch_rmid_idx_decode (u32 idx,u32 *closid,u32 *rmid) +{ + *rmid = idx; + *closid =X86_RESCTRL_EMPTY_CLOSID ; +} + +static inline u32 resctrl_arch_rmid_idx_encode (u32 ignored,u32 rmid) +{ +return rmid; +} +void resctrl_cpu_detect (struct cpuinfo_x86 *c);#else
arch/x86/kernel/cpu/resctrl/core.c +3 -2
@@ -589,7 +589,7 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)mbm_setup_overflow_handler (d,0 ); }if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu && -has_busy_rmid (r, d)) { +has_busy_rmid (d)) {cancel_delayed_work (&d->cqm_limbo);cqm_setup_limbo_handler (d,0 ); }
@@ -604,7 +604,8 @@ static void clear_closid_rmid(int cpu) state->default_rmid =0 ; state->cur_closid =0 ; state->cur_rmid =0 ; -wrmsr (MSR_IA32_PQR_ASSOC ,0 ,0 ); +wrmsr (MSR_IA32_PQR_ASSOC ,RESCTRL_RESERVED_RMID , +RESCTRL_RESERVED_CLOSID ); }static int resctrl_online_cpu (unsigned int cpu)
arch/x86/kernel/cpu/resctrl/internal.h +3 -1
@@ -8,6 +8,8 @@#include <linux/fs_context.h> #include <linux/jump_label.h> +#include <asm/resctrl.h> +#define L3_QOS_CDP_ENABLE 0x01ULL#define L2_QOS_CDP_ENABLE 0x01ULL
@@ -560,7 +562,7 @@ void __init intel_rdt_mbm_apply_quirk(void);bool is_mba_sc (struct rdt_resource *r);void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms);void cqm_handle_limbo (struct work_struct *work); -bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d); +bool has_busy_rmid (struct rdt_domain *d);void __check_limbo (struct rdt_domain *d,bool force_free);void rdt_domain_reconfigure_cdp (struct rdt_resource *r);void __init thread_throttle_mode_init (void );
arch/x86/kernel/cpu/resctrl/monitor.c +66 -30
211 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/rdtgroup.c +5 -4
@@ -3894,7 +3894,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)if (is_mbm_enabled ())cancel_delayed_work (&d->mbm_over); -if (is_llc_occupancy_enabled () &&has_busy_rmid (r, d)) { +if (is_llc_occupancy_enabled () &&has_busy_rmid (d)) {/* * When a package is going down, forcefully * decrement rmid->ebusy. There is no way to know
@@ -3912,16 +3912,17 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)static int domain_setup_mon_state (struct rdt_resource *r,struct rdt_domain *d) { +u32 idx_limit =resctrl_arch_system_num_rmid_idx ();size_t tsize;if (is_llc_occupancy_enabled ()) { - d->rmid_busy_llc =bitmap_zalloc (r->num_rmid,GFP_KERNEL ); + d->rmid_busy_llc =bitmap_zalloc (idx_limit,GFP_KERNEL );if (!d->rmid_busy_llc)return -ENOMEM ; }if (is_mbm_total_enabled ()) { tsize =sizeof (*d->mbm_total); - d->mbm_total =kcalloc (r->num_rmid, tsize,GFP_KERNEL ); + d->mbm_total =kcalloc (idx_limit, tsize,GFP_KERNEL );if (!d->mbm_total) {bitmap_free (d->rmid_busy_llc);return -ENOMEM ;
@@ -3929,7 +3930,7 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) }if (is_mbm_local_enabled ()) { tsize =sizeof (*d->mbm_local); - d->mbm_local =kcalloc (r->num_rmid, tsize,GFP_KERNEL ); + d->mbm_local =kcalloc (idx_limit, tsize,GFP_KERNEL );if (!d->mbm_local) {bitmap_free (d->rmid_busy_llc);kfree (d->mbm_total);
Signature · 1 line
2.39.2
RE: [EXT] [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by index
Hi,
-----Original Message-----
From: James Morse <[email protected]>
Sent: Wednesday, October 25, 2023 11:33 PM
To: [email protected]; [email protected]
Cc: Fenghua Yu <[email protected]>; Reinette Chatre <[email protected]>; Thomas Gleixner <[email protected]>; Ingo Molnar <[email protected]>; Borislav Petkov <[email protected]>; H Peter Anvin <[email protected]>; Babu Moger <[email protected]>; James Morse <[email protected]>; [email protected]; D Scott Phillips OS <[email protected]>; [email protected]; Linu Cherian <[email protected]>; [email protected]; [email protected]; [email protected]; Jamie Iles <[email protected]>; Xin Hao <[email protected]>; [email protected]; [email protected]; Amit Singh Tomar <[email protected]>
Subject: [EXT] [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by index
External Email
----------------------------------------------------------------------
x86 systems identify traffic using the CLOSID and RMID. The CLOSID is used to lookup the control policy, the RMID is used for monitoring. For
x86 these are independent numbers.
Arm's MPAM has equivalent features PARTID and PMG, where the PARTID is used to lookup the control policy. The PMG in contrast is a small number of bits that are used to subdivide PARTID when monitoring. The cache-occupancy monitors require the PARTID to be specified when monitoring.
This means MPAM's PMG field is not unique. There are multiple PMG-0, one per allocated CLOSID/PARTID. If PMG is treated as equivalent to RMID, it cannot be allocated as an independent number. Bitmaps like rmid_busy_llc need to be sized by the number of unique entries for this resource.
Treat the combined CLOSID and RMID as an index, and provide architecture helpers to pack and unpack an index. This makes the MPAM values unique. The domain's rmid_busy_llc and rmid_ptrs[] are then sized by index, as are domain mbm_local[] and mbm_total[].
x86 can ignore the CLOSID field when packing and unpacking an index, and report as many indexes as RMID.
---
Changes since v1:
* Added X86_BAD_CLOSID macro to make it clear what this value means
* Added second WARN_ON() for closid checking, and made both _ONCE()
Changes since v2:
* Added RESCTRL_RESERVED_CLOSID
* Removed a newline
* Repharsed some comments
* Renamed a variable 'ignore'd
* Moved X86_RESCTRL_BAD_CLOSID to a previous patch
Changes since v3:
* Changed a variable name
* Fixed various typos
Changes since v4:
* Removed resource parameter from has_busy_rmid()
* Rewrote commit message
Changes since v5:
* Used RESCTRL_RESERVED_RMID in clear_closid_rmid().
* Added comment against free_rmid()s index comparison tricks.
Changes since v6:
* Added a newline between some #includes.
* Juggled some plurals in a comment.
--- arch/x86/include/asm/resctrl.h | 17 +++++ arch/x86/kernel/cpu/resctrl/core.c | 5 +- arch/x86/kernel/cpu/resctrl/internal.h | 4 +- arch/x86/kernel/cpu/resctrl/monitor.c | 96 ++++++++++++++++++-------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 9 +-- 5 files changed, 94 insertions(+), 37 deletions(-)
arch/x86/include/asm/resctrl.h +15 -0
@@ -101,6 +101,23 @@ static inline void resctrl_sched_in(struct task_struct *tsk) __resctrl_sched_in(tsk); } +static inline u32 resctrl_arch_system_num_rmid_idx (void ) +{ +/* RMID are independent numbers for x86. num_rmid_idx == num_rmid */ +return boot_cpu_data.x86_cache_max_rmid +1 ; } + +static inline void resctrl_arch_rmid_idx_decode (u32 idx,u32 *closid, +u32 *rmid) { + *rmid = idx; + *closid =X86_RESCTRL_EMPTY_CLOSID ; +} + +static inline u32 resctrl_arch_rmid_idx_encode (u32 ignored,u32 rmid) { +return rmid; +} +void resctrl_cpu_detect (struct cpuinfo_x86 *c);#else
arch/x86/kernel/cpu/resctrl/core.c +3 -2
@@ -589,7 +589,7 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)mbm_setup_overflow_handler (d,0 ); }if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu && -has_busy_rmid (r, d)) { +has_busy_rmid (d)) {cancel_delayed_work (&d->cqm_limbo);cqm_setup_limbo_handler (d,0 ); }
@@ -604,7 +604,8 @@ static void clear_closid_rmid(int cpu) state->default_rmid =0 ; state->cur_closid =0 ; state->cur_rmid =0 ; -wrmsr (MSR_IA32_PQR_ASSOC ,0 ,0 ); +wrmsr (MSR_IA32_PQR_ASSOC ,RESCTRL_RESERVED_RMID , +RESCTRL_RESERVED_CLOSID ); } static int resctrl_online_cpu(unsigned int cpu) diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index c836e3294e12..97ec24f91ac4 100644
arch/x86/kernel/cpu/resctrl/internal.h +3 -0
@@ -8,6 +8,8 @@#include <linux/fs_context.h> #include <linux/jump_label.h> +#include <asm/resctrl.h> +#define L3_QOS_CDP_ENABLE 0x01ULL#define L2_QOS_CDP_ENABLE 0x01ULL
@@ -560,7 +562,7 @@ void __init intel_rdt_mbm_apply_quirk(void); bool is_mba_sc(struct rdt_resource *r); void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms); void cqm_handle_limbo(struct work_struct *work); -bool has_busy_rmid(struct rdt_resource *r, struct rdt_domain *d); +bool has_busy_rmid (struct rdt_domain *d); void __check_limbo(struct rdt_domain *d, bool force_free); void rdt_domain_reconfigure_cdp(struct rdt_resource *r); void __init thread_throttle_mode_init(void); diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 2a0233cd0bc9..c02cf32cd17c 100644
arch/x86/kernel/cpu/resctrl/monitor.c +23 -3
@@ -149,12 +149,29 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val)return val; } -static inline struct rmid_entry *__rmid_entry (u32 closid,u32 rmid) +/* +* x86 and arm64 differ in their handling of monitoring. +* x86's RMID are independent numbers, there is only one source of +traffic +* with an RMID value of '1'. +* arm64's PMG extends the PARTID/CLOSID space, there are multiple +sources of +* traffic with a PMG value of '1', one for each CLOSID, meaning the +RMID +* value is no longer unique. +* To account for this, resctrl uses an index. On x86 this is just the +RMID, +* on arm64 it encodes the CLOSID and RMID. This gives a unique number. +* +* The domain's rmid_busy_llc and rmid_ptrs[] are sized by index. The +arch code +* must accept an attempt to read every index. +*/ +static inline struct rmid_entry *__rmid_entry (u32 idx) {struct rmid_entry *entry; +u32 closid, rmid; - entry = &rmid_ptrs[rmid]; -WARN_ON (entry->rmid != rmid); + entry = &rmid_ptrs[idx]; +resctrl_arch_rmid_idx_decode (idx, &closid, &rmid); +
+ WARN_ON_ONCE(entry->closid != closid);
+ WARN_ON_ONCE(entry->rmid != rmid);
return entry;
}
@@ -284,8 +301,9 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, void __check_limbo(struct rdt_domain *d, bool force_free) {
struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
+ u32 idx_limit = resctrl_arch_system_num_rmid_idx();
struct rmid_entry *entry;
- u32 crmid = 1, nrmid;
+ u32 idx, cur_idx = 1;
bool rmid_dirty;
u64 val = 0;
@@ -296,12 +314,11 @@ void __check_limbo(struct rdt_domain *d, bool force_free)
* RMID and move it to the free list when the counter reaches 0. */
for (;;) {
- nrmid = find_next_bit(d->rmid_busy_llc, r->num_rmid, crmid);
- if (nrmid >= r->num_rmid)
+ idx = find_next_bit(d->rmid_busy_llc, idx_limit, cur_idx);
+ if (idx >= idx_limit)
break;
- entry = __rmid_entry(X86_RESCTRL_EMPTY_CLOSID, nrmid);// temporary
-
+ entry = __rmid_entry(idx);
if (resctrl_arch_rmid_read(r, d, entry->closid, entry->rmid,
QOS_L3_OCCUP_EVENT_ID, &val)) {
rmid_dirty = true;
@@ -310,19 +327,21 @@ void __check_limbo(struct rdt_domain *d, bool force_free)
}
if (force_free || !rmid_dirty) {
- clear_bit(entry->rmid, d->rmid_busy_llc);
+ clear_bit(idx, d->rmid_busy_llc);
if (!--entry->busy) {
rmid_limbo_count--; list_add_tail(&entry->list, &rmid_free_lru);
}
}
- crmid = nrmid + 1;
+ cur_idx = idx + 1;
}
}
-bool has_busy_rmid(struct rdt_resource *r, struct rdt_domain *d) +bool has_busy_rmid(struct rdt_domain *d)
{
- return find_first_bit(d->rmid_busy_llc, r->num_rmid) != r->num_rmid;
+ u32 idx_limit = resctrl_arch_system_num_rmid_idx();
+
+ return find_first_bit(d->rmid_busy_llc, idx_limit) != idx_limit;
}
/*
@@ -352,6 +371,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)
struct rdt_domain *d;
int cpu, err;
u64 val = 0;
+ u32 idx;
+
+ idx = resctrl_arch_rmid_idx_encode(entry->closid, entry->rmid);
entry->busy = 0;
cpu = get_cpu();
@@ -369,9 +391,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)
* For the first limbo RMID in the domain,
* setup up the limbo worker.
*/
- if (!has_busy_rmid(r, d))
+ if (!has_busy_rmid(d))
cqm_setup_limbo_handler(d, CQM_LIMBOCHECK_INTERVAL);
- set_bit(entry->rmid, d->rmid_busy_llc);
+ set_bit(idx, d->rmid_busy_llc);
entry->busy++;
}
put_cpu();
@@ -384,14 +406,21 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)
void free_rmid(u32 closid, u32 rmid)
{
+ u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
struct rmid_entry *entry;
- if (!rmid)
- return;
-
lockdep_assert_held(&rdtgroup_mutex);
- entry = __rmid_entry(closid, rmid);
+ /*
+ * Do not allow the default rmid to be free'd. Comparing by index
+ * allows architectures that ignore the closid parameter to avoid an
+ * unnecessary check.
+ */
+ if (idx == resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID,
+ RESCTRL_RESERVED_RMID))
+ return;
+
+ entry = __rmid_entry(idx);
if (is_llc_occupancy_enabled())
add_rmid_to_limbo(entry);
@@ -402,11 +431,13 @@ void free_rmid(u32 closid, u32 rmid) static struct mbm_state *get_mbm_state(struct rdt_domain *d, u32 closid,
u32 rmid, enum resctrl_event_id evtid) {
+ u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
+
switch (evtid) {
case QOS_L3_MBM_TOTAL_EVENT_ID:
- return &d->mbm_total[rmid];
+ return &d->mbm_total[idx];
case QOS_L3_MBM_LOCAL_EVENT_ID:
- return &d->mbm_local[rmid];
+ return &d->mbm_local[idx];
default:
return NULL;
}
@@ -449,7 +480,8 @@ static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr)
*/
static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) {
- struct mbm_state *m = &rr->d->mbm_local[rmid];
+ u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid);
+ struct mbm_state *m = &rr->d->mbm_local[idx];
u64 cur_bw, bytes, cur_bytes;
cur_bytes = rr->val;
@@ -539,7 +571,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) {
u32 closid, rmid, cur_msr_val, new_msr_val;
struct mbm_state *pmbm_data, *cmbm_data;
- u32 cur_bw, delta_bw, user_bw;
+ u32 cur_bw, delta_bw, user_bw, idx;
struct rdt_resource *r_mba;
struct rdt_domain *dom_mba;
struct list_head *head;
@@ -552,7 +584,8 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm)
closid = rgrp->closid;
rmid = rgrp->mon.rmid;
- pmbm_data = &dom_mbm->mbm_local[rmid];
+ idx = resctrl_arch_rmid_idx_encode(closid, rmid);
+ pmbm_data = &dom_mbm->mbm_local[idx];
dom_mba = get_domain_from_cpu(smp_processor_id(), r_mba); if (!dom_mba) {
@@ -670,7 +703,7 @@ void cqm_handle_limbo(struct work_struct *work)
__check_limbo(d, false);
- if (has_busy_rmid(r, d))
+ if (has_busy_rmid(d))
schedule_delayed_work_on(cpu, &d->cqm_limbo, delay);
mutex_unlock(&rdtgroup_mutex);
@@ -735,19 +768,20 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)
static int dom_data_init(struct rdt_resource *r) {
+ u32 idx_limit = resctrl_arch_system_num_rmid_idx();
struct rmid_entry *entry = NULL;
- int i, nr_rmids;
+ u32 idx;
+ int i;
- nr_rmids = r->num_rmid;
- rmid_ptrs = kcalloc(nr_rmids, sizeof(struct rmid_entry), GFP_KERNEL);
+ rmid_ptrs = kcalloc(idx_limit, sizeof(struct rmid_entry), GFP_KERNEL);
[>>] Is there a chance, it could result in "ZERO_SIZE_PTR", and we should guard it against ZERO_OR_NULL_PTR in the following if condition?
It might be related, while testing the snapshot[1] (and subsequent snapshots has similar change) on x86 platform, Zahid is seeing Kernel panic:
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/fs/resctrl/monitor.c?h=mpam/snapshot/v6.2#n695
if (!rmid_ptrs)
return -ENOMEM;
- for (i = 0; i < nr_rmids; i++) {
+ for (i = 0; i < idx_limit; i++) {
entry = &rmid_ptrs[i];
INIT_LIST_HEAD(&entry->list);
- entry->rmid = i;
+ resctrl_arch_rmid_idx_decode(i, &entry->closid, &entry->rmid);
list_add_tail(&entry->list, &rmid_free_lru);
}
@@ -755,7 +789,9 @@ static int dom_data_init(struct rdt_resource *r)
* These are used for rdtgroup_default control group, which will be
* setup later in rdtgroup_init().
*/
- entry = __rmid_entry(RESCTRL_RESERVED_CLOSID, RESCTRL_RESERVED_RMID);
+ idx = resctrl_arch_rmid_idx_encode(RESCTRL_RESERVED_CLOSID,
+ RESCTRL_RESERVED_RMID);
+ entry = __rmid_entry(idx);
list_del(&entry->list);
return 0;
arch/x86/kernel/cpu/resctrl/rdtgroup.c +1 -1
@@ -3894,7 +3894,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)if (is_mbm_enabled ())cancel_delayed_work (&d->mbm_over); -if (is_llc_occupancy_enabled () &&has_busy_rmid (r, d)) { +if (is_llc_occupancy_enabled () &&has_busy_rmid (d)) {/* * When a package is going down, forcefully * decrement rmid->ebusy. There is no way to know @@ -3912,16 +3912,17 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)
static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) {
+ u32 idx_limit = resctrl_arch_system_num_rmid_idx();
size_t tsize;
if (is_llc_occupancy_enabled()) {
- d->rmid_busy_llc = bitmap_zalloc(r->num_rmid, GFP_KERNEL);
+ d->rmid_busy_llc = bitmap_zalloc(idx_limit, GFP_KERNEL);
if (!d->rmid_busy_llc)
return -ENOMEM;
}
if (is_mbm_total_enabled()) {
tsize = sizeof(*d->mbm_total);
- d->mbm_total = kcalloc(r->num_rmid, tsize, GFP_KERNEL);
+ d->mbm_total = kcalloc(idx_limit, tsize, GFP_KERNEL);
if (!d->mbm_total) {
bitmap_free(d->rmid_busy_llc);
return -ENOMEM;
@@ -3929,7 +3930,7 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d)
}
if (is_mbm_local_enabled()) {
tsize = sizeof(*d->mbm_local);
- d->mbm_local = kcalloc(r->num_rmid, tsize, GFP_KERNEL);
+ d->mbm_local = kcalloc(idx_limit, tsize, GFP_KERNEL);
if (!d->mbm_local) {
bitmap_free(d->rmid_busy_llc);
kfree(d->mbm_total);
--
2.39.2
James Morse
reviewerRe: [EXT] [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by index
Hi Amit,
Quoted 4 lines · Amit Singh Tomar
-----Original Message-----
From: James Morse <[email protected]>
Sent: Wednesday, October 25, 2023 11:33 PM
Subject: [EXT] [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by index
Looks like you are afflicted with outlook - let me know if I didn't find all the changes you made to the original message ...
[..]
Quoted 17 lines
arch/x86/kernel/cpu/resctrl/monitor.c +4
-3@@ -735,19 +768,20 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)static int dom_data_init (struct rdt_resource *r) { +u32 idx_limit =resctrl_arch_system_num_rmid_idx ();struct rmid_entry *entry =NULL ; -int i, nr_rmids; +u32 idx; +int i; - nr_rmids = r->num_rmid; - rmid_ptrs =kcalloc (nr_rmids,sizeof (struct rmid_entry ),GFP_KERNEL ); + rmid_ptrs =kcalloc (idx_limit,sizeof (struct rmid_entry ),GFP_KERNEL );
Quoted 3 lines
[>>] Is there a chance, it could result in "
ZERO_SIZE_PTR", and we should guard it againstZERO_OR_NULL_PTRin the following if condition?
It might be related, while testing the snapshot[1] (and subsequent snapshots has similar change) on x86 platform, Zahid is seeing Kernel panic:
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/fs/resctrl/monitor.c?h=mpam/snapshot/v6.2#n695
Interesting - I didn't think this could happen. Could you share the full splat?
This would imply idx_limit was zero, so boot_cpu_data.x86_cache_max_rmid would be -1. But wouldn't this happen before this patch? idx_limit has the same value as nr_rmids on x86, its only MPAM that needs a different value.
Thanks,
James
RE: [EXT] [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by index
Hi James,
-----Original Message-----
From: James Morse <[email protected]>
Sent: Monday, December 11, 2023 8:03 PM
To: Amit Singh Tomar <[email protected]>; [email protected]; [email protected]
Cc: Fenghua Yu <[email protected]>; Reinette Chatre <[email protected]>; Thomas Gleixner <[email protected]>; Ingo Molnar <[email protected]>; Borislav Petkov <[email protected]>; H Peter Anvin <[email protected]>; Babu Moger <[email protected]>; [email protected]; D Scott Phillips OS <[email protected]>; [email protected]; Linu Cherian <[email protected]>; [email protected]; [email protected]; [email protected]; Jamie Iles <[email protected]>; Xin Hao <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [EXT] [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by index
Hi Amit,
Quoted 5 lines · Amit Singh Tomar
-----Original Message-----
From: James Morse <[email protected]>
Sent: Wednesday, October 25, 2023 11:33 PM
Subject: [EXT] [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by index
Looks like you are afflicted with outlook - let me know if I didn't find all the changes you made to the original message ...
[..]
Quoted 23 lines
arch/x86/kernel/cpu/resctrl/monitor.c +5
-3@@ -735,19 +768,20 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)static int dom_data_init (struct rdt_resource *r) { +u32 idx_limit =resctrl_arch_system_num_rmid_idx ();struct rmid_entry *entry =NULL ; -int i, nr_rmids; +u32 idx; +int i; - nr_rmids = r->num_rmid; - rmid_ptrs =kcalloc (nr_rmids,sizeof (struct rmid_entry ),GFP_KERNEL ); + rmid_ptrs =kcalloc (idx_limit,sizeof (struct rmid_entry ), +GFP_KERNEL );[>>] Is there a chance, it could result in "
ZERO_SIZE_PTR", and we should guard it againstZERO_OR_NULL_PTRin the following if condition?
It might be related, while testing the snapshot[1] (and subsequent snapshots has similar change) on x86 platform, Zahid is seeing Kernel panic:
Quoted 6 lines
https://urldefense.proofpoint.com/v2/url?u=https-3A__git.kernel.org_pu
b_scm_linux_kernel_git_morse_linux.git_tree_fs_resctrl_monitor.c-3Fh-3 Dmpam_snapshot_v6.2-23n695&d=DwICaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=V_GK7jR uCHDErm6txmgDK1-MbUihtnSQ3gPgB-A-JKU&m=yHcjuc1ZrYfPWXGxTPifeglinf_gMfy AgnvZfOw-ZD2zRG8G61IfH8hignwaxlV6&s=X3Ie_NqTHtzN2ttkl3yiTYHzNpkWW2wPPI DJ7XTWW40&e=
Interesting - I didn't think this could happen. Could you share the full splat?
Unfortunately, I don't have access to the test set-up where this splat has been observed. However, I have requested Zahid (Cc) to provide the splat logs. Additionally, from what I've learned, this splat has been observed on an x86 machine that doesn't support monitor groups. Do you see this as problem?
This would imply idx_limit was zero, so boot_cpu_data.x86_cache_max_rmid would be -1.
But wouldn't this happen before this patch? idx_limit has the same value as nr_rmids on x86, its only MPAM that needs a different value.
Thanks,
-Amit
James Morse
reviewerRe: [EXT] [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by index
Hi Amit,
Quoted 37 lines · Amit Singh Tomar
-----Original Message-----
From: James Morse <[email protected]>
Sent: Monday, December 11, 2023 8:03 PM
To: Amit Singh Tomar <[email protected]>; [email protected]; [email protected]
Cc: Fenghua Yu <[email protected]>; Reinette Chatre <[email protected]>; Thomas Gleixner <[email protected]>; Ingo Molnar <[email protected]>; Borislav Petkov <[email protected]>; H Peter Anvin <[email protected]>; Babu Moger <[email protected]>; [email protected]; D Scott Phillips OS <[email protected]>; [email protected]; Linu Cherian <[email protected]>; [email protected]; [email protected]; [email protected]; Jamie Iles <[email protected]>; Xin Hao <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [EXT] [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by indexQuoted 29 lines · Amit Singh Tomar
-----Original Message-----
From: James Morse <[email protected]>
Sent: Wednesday, October 25, 2023 11:33 PM
Subject: [EXT] [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by indexarch/x86/kernel/cpu/resctrl/monitor.c +5
-3@@ -735,19 +768,20 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)static int dom_data_init (struct rdt_resource *r) { +u32 idx_limit =resctrl_arch_system_num_rmid_idx ();struct rmid_entry *entry =NULL ; -int i, nr_rmids; +u32 idx; +int i; - nr_rmids = r->num_rmid; - rmid_ptrs =kcalloc (nr_rmids,sizeof (struct rmid_entry ),GFP_KERNEL ); + rmid_ptrs =kcalloc (idx_limit,sizeof (struct rmid_entry ), +GFP_KERNEL );[>>] Is there a chance, it could result in "
ZERO_SIZE_PTR", and we should guard it againstZERO_OR_NULL_PTRin the following if condition?
It might be related, while testing the snapshot[1] (and subsequent snapshots has similar change) on x86 platform, Zahid is seeing Kernel panic:
Quoted 8 lines
Quoted 6 lines
https://urldefense.proofpoint.com/v2/url?u=https-3A__git.kernel.org_pu
b_scm_linux_kernel_git_morse_linux.git_tree_fs_resctrl_monitor.c-3Fh-3 Dmpam_snapshot_v6.2-23n695&d=DwICaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=V_GK7jR uCHDErm6txmgDK1-MbUihtnSQ3gPgB-A-JKU&m=yHcjuc1ZrYfPWXGxTPifeglinf_gMfy AgnvZfOw-ZD2zRG8G61IfH8hignwaxlV6&s=X3Ie_NqTHtzN2ttkl3yiTYHzNpkWW2wPPI DJ7XTWW40&e=Interesting - I didn't think this could happen. Could you share the full splat?
(this bit here is your reply?:)
Quoted 10 lines
Unfortunately, I don't have access to the test set-up where this splat has been observed. However, I have requested Zahid (Cc) to provide the splat logs. Additionally, from what I've learned, this splat has been observed on an x86 machine that doesn't support monitor groups. Do you see this as problem?
This would imply
idx_limitwas zero, so boot_cpu_data.x86_cache_max_rmid would be -1. But wouldn't this happen before this patch?idx_limithas the same value asnr_rmidson x86, its only MPAM that needs a different value.
Your 'doesn't support monitor groups' explains why boot_cpu_data.x86_cache_max_rmid is -1.
As you've said you're testing the whole tree - not this series, I suspect this is coming from "x86/resctrl: Move monitor init work to a resctrl init call", which moves initialisation of filesystem structures to filesystem code.
It looks like I missed that get_rdt_mon_resources() can bale out before calling rdt_get_mon_l3_config(), which I think would explain what you hint at here.
Adding this to the "x86/resctrl: Move monitor init work to a resctrl init call" should fix that. (It'll be in the next snapshot I push) ---------------------%<---------------------
arch/x86/kernel/cpu/resctrl/monitor.c +4 -2
@@ -1030,12 +1030,14 @@ int resctrl_mon_resource_init(void)struct rdt_resource *r =resctrl_arch_get_resource (RDT_RESOURCE_L3 );int ret; +if (!r->mon_capable) +return 0 ; + ret =dom_data_init (r);if (ret)return ret; -if (r->mon_capable) -l3_mon_evt_init (r); +l3_mon_evt_init (r);return 0 ; }
---------------------%<---------------------
Thanks,
James
Reinette Chatre
maintainerRe: [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by index
In reply to James MorseHi James,
On 10/25/2023 11:03 AM, James Morse wrote:
...
Quoted 14 lines
arch/x86/kernel/cpu/resctrl/core.c +1
-1@@ -589,7 +589,7 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)mbm_setup_overflow_handler (d,0 ); }if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu && -has_busy_rmid (r, d)) { +has_busy_rmid (d)) {cancel_delayed_work (&d->cqm_limbo);cqm_setup_limbo_handler (d,0 ); }
arch/x86/kernel/cpu/resctrl/core.c static void clear_closid_rmid(int cpu) · line 606
Quoted 3 lines
arch/x86/kernel/cpu/resctrl/core.c +0
-0@@ -604,7 +604,8 @@ static void clear_closid_rmid(int cpu) state->default_rmid =0 ; state->cur_closid =0 ; state->cur_rmid =0 ;
These assignments can also use the new RESERVED defines, no?
arch/x86/kernel/cpu/resctrl/core.c static void clear_closid_rmid(int cpu) · line 610
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/core.c +2
-1-wrmsr (MSR_IA32_PQR_ASSOC ,0 ,0 ); +wrmsr (MSR_IA32_PQR_ASSOC ,RESCTRL_RESERVED_RMID , +RESCTRL_RESERVED_CLOSID ); }
Looks like rdtgroup_setup_default() is also a candidate to replace hardcoded "0" for closid and rmid with the new defines.
...
arch/x86/kernel/cpu/resctrl/core.c void cqm_handle_limbo(struct work_struct *work) · line 708
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/core.c +1
-1@@ -670,7 +703,7 @@ void cqm_handle_limbo(struct work_struct *work)__check_limbo (d, false); -if (has_busy_rmid (r, d)) +if (has_busy_rmid (d))schedule_delayed_work_on (cpu, &d->cqm_limbo, delay);
This change results in a new compile warning:
arch/x86/kernel/cpu/resctrl/monitor.c: In function ‘cqm_handle_limbo’:arch/x86/kernel/cpu/resctrl/monitor.c:805:30: warning: variable ‘r’ set but not used [-Wunused-but-set-variable]
805 | struct rdt_resource *r;
Reinette
James Morse
reviewerRe: [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by index
Hi Reinette,
Quoted 11 lines · Reinette Chatre
arch/x86/kernel/cpu/resctrl/core.c static void clear_closid_rmid(int cpu) · line 606
Quoted 8 lines · James Morse
arch/x86/kernel/cpu/resctrl/core.c +0
-0@@ -604,7 +604,8 @@ static void clear_closid_rmid(int cpu) state->default_rmid =0 ; state->cur_closid =0 ; state->cur_rmid =0 ;These assignments can also use the new RESERVED defines, no?
Yup, done.
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/core.c static void clear_closid_rmid(int cpu) · line 610
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/core.c +2
-1-wrmsr (MSR_IA32_PQR_ASSOC ,0 ,0 ); +wrmsr (MSR_IA32_PQR_ASSOC ,RESCTRL_RESERVED_RMID , +RESCTRL_RESERVED_CLOSID ); }Looks like
rdtgroup_setup_default()is also a candidate to replace hardcoded "0" for closid and rmid with the new defines.
Sure, done.
Quoted 11 lines
arch/x86/kernel/cpu/resctrl/core.c void cqm_handle_limbo(struct work_struct *work) · line 708
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/core.c +1
-1@@ -670,7 +703,7 @@ void cqm_handle_limbo(struct work_struct *work)__check_limbo (d, false); -if (has_busy_rmid (r, d)) +if (has_busy_rmid (d))schedule_delayed_work_on (cpu, &d->cqm_limbo, delay);This change results in a new compile warning:
Quoted 3 lines
arch/x86/kernel/cpu/resctrl/monitor.c: In function ‘cqm_handle_limbo’:arch/x86/kernel/cpu/resctrl/monitor.c:805:30: warning: variable ‘r’ set but not used [-Wunused-but-set-variable]
805 | structrdt_resource*r;
Hmm, looks like the assignment foxes gcc-12 into thinking this is okay. I've removed that now.
Thanks,
James
Re: [PATCH v7 06/24] x86/resctrl: Access per-rmid structures by index
In reply to James MorseHi James,
Quoted 19 lines · James Morse
x86 systems identify traffic using the CLOSID and RMID. The CLOSID is used to lookup the control policy, the RMID is used for monitoring. For x86 these are independent numbers.
Arm's MPAM has equivalent features PARTID and PMG, where the PARTID is used to lookup the control policy. The PMG in contrast is a small number of bits that are used to subdivide PARTID when monitoring. The cache-occupancy monitors require the PARTID to be specified when monitoring.This means MPAM's PMG field is not unique. There are multiple PMG-0, one per allocated CLOSID/PARTID. If PMG is treated as equivalent to RMID, it cannot be allocated as an independent number. Bitmaps like
rmid_busy_llcneed to be sized by the number of unique entries for this resource.Treat the combined CLOSID and RMID as an index, and provide architecture helpers to pack and unpack an index. This makes the MPAM values unique. The domain's
rmid_busy_llcand rmid_ptrs[] are then sized by index, as are domain mbm_local[] and mbm_total[].
Quoted 7 lines
x86 can ignore the CLOSID field when packing and unpacking an index, and report as many indexes as RMID.
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 379 lines
---
Changes since v1:
* AddedX86_BAD_CLOSIDmacro to make it clear what this value means
* Added secondWARN_ON()for closid checking, and made both_ONCE()Changes since v2:
* AddedRESCTRL_RESERVED_CLOSID
* Removed a newline
* Repharsed some comments
* Renamed a variable 'ignore'd
* MovedX86_RESCTRL_BAD_CLOSIDto a previous patchChanges since v3:
* Changed a variable name
* Fixed various typosChanges since v4:
* Removed resource parameter fromhas_busy_rmid()
* Rewrote commit messageChanges since v5:
* UsedRESCTRL_RESERVED_RMIDinclear_closid_rmid().
* Added comment against free_rmid()s index comparison tricks.Changes since v6:
* Added a newline between some #includes.
* Juggled some plurals in a comment.--- arch/x86/include/asm/resctrl.h | 17 +++++ arch/x86/kernel/cpu/resctrl/core.c | 5 +- arch/x86/kernel/cpu/resctrl/internal.h | 4 +- arch/x86/kernel/cpu/resctrl/monitor.c | 96 ++++++++++++++++++-------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 9 +-- 5 files changed, 94 insertions(+), 37 deletions(-)arch/x86/include/asm/resctrl.h +17
-0@@ -101,6 +101,23 @@ static inline void resctrl_sched_in(struct task_struct *tsk) __resctrl_sched_in(tsk); } +static inline u32 resctrl_arch_system_num_rmid_idx (void ) +{ +/* RMID are independent numbers for x86. num_rmid_idx == num_rmid */ +return boot_cpu_data.x86_cache_max_rmid +1 ; +} + +static inline void resctrl_arch_rmid_idx_decode (u32 idx,u32 *closid,u32 *rmid) +{ + *rmid = idx; + *closid =X86_RESCTRL_EMPTY_CLOSID ; +} + +static inline u32 resctrl_arch_rmid_idx_encode (u32 ignored,u32 rmid) +{ +return rmid; +} +void resctrl_cpu_detect (struct cpuinfo_x86 *c);#else arch/x86/kernel/cpu/resctrl/core.c +3
-2@@ -589,7 +589,7 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)mbm_setup_overflow_handler (d,0 ); }if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu && -has_busy_rmid (r, d)) { +has_busy_rmid (d)) {cancel_delayed_work (&d->cqm_limbo);cqm_setup_limbo_handler (d,0 ); }@@ -604,7 +604,8 @@ static void clear_closid_rmid(int cpu) state->default_rmid =0 ; state->cur_closid =0 ; state->cur_rmid =0 ; -wrmsr (MSR_IA32_PQR_ASSOC ,0 ,0 ); +wrmsr (MSR_IA32_PQR_ASSOC ,RESCTRL_RESERVED_RMID , +RESCTRL_RESERVED_CLOSID ); }static int resctrl_online_cpu (unsigned int cpu)arch/x86/kernel/cpu/resctrl/internal.h +3
-1@@ -8,6 +8,8 @@#include <linux/fs_context.h> #include <linux/jump_label.h> +#include <asm/resctrl.h> +#define L3_QOS_CDP_ENABLE 0x01ULL#define L2_QOS_CDP_ENABLE 0x01ULL@@ -560,7 +562,7 @@ void __init intel_rdt_mbm_apply_quirk(void);bool is_mba_sc (struct rdt_resource *r);void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms);void cqm_handle_limbo (struct work_struct *work); -bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d); +bool has_busy_rmid (struct rdt_domain *d);void __check_limbo (struct rdt_domain *d,bool force_free);void rdt_domain_reconfigure_cdp (struct rdt_resource *r);void __init thread_throttle_mode_init (void );arch/x86/kernel/cpu/resctrl/monitor.c +66
-30@@ -149,12 +149,29 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val)return val; } -static inline struct rmid_entry *__rmid_entry (u32 closid,u32 rmid) +/* +* x86 and arm64 differ in their handling of monitoring. +* x86's RMID are independent numbers, there is only one source of traffic +* with an RMID value of '1'. +* arm64's PMG extends the PARTID/CLOSID space, there are multiple sources of +* traffic with a PMG value of '1', one for each CLOSID, meaning the RMID +* value is no longer unique. +* To account for this, resctrl uses an index. On x86 this is just the RMID, +* on arm64 it encodes the CLOSID and RMID. This gives a unique number. +* +* The domain's rmid_busy_llc and rmid_ptrs[] are sized by index. The arch code +* must accept an attempt to read every index. +*/ +static inline struct rmid_entry *__rmid_entry (u32 idx) {struct rmid_entry *entry; +u32 closid, rmid; - entry = &rmid_ptrs[rmid]; -WARN_ON (entry->rmid != rmid); + entry = &rmid_ptrs[idx]; +resctrl_arch_rmid_idx_decode (idx, &closid, &rmid); + +WARN_ON_ONCE (entry->closid != closid); +WARN_ON_ONCE (entry->rmid != rmid);return entry; }@@ -284,8 +301,9 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,void __check_limbo (struct rdt_domain *d,bool force_free) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +u32 idx_limit =resctrl_arch_system_num_rmid_idx ();struct rmid_entry *entry; -u32 crmid =1 , nrmid; +u32 idx, cur_idx =1 ;bool rmid_dirty;u64 val =0 ;@@ -296,12 +314,11 @@ void __check_limbo(struct rdt_domain *d, bool force_free)* RMID and move it to the free list when the counter reaches 0. */ for (;;) { - nrmid =find_next_bit (d->rmid_busy_llc, r->num_rmid, crmid); -if (nrmid >= r->num_rmid) + idx =find_next_bit (d->rmid_busy_llc, idx_limit, cur_idx); +if (idx >= idx_limit)break ; - entry =__rmid_entry (X86_RESCTRL_EMPTY_CLOSID , nrmid);// temporary - + entry =__rmid_entry (idx);if (resctrl_arch_rmid_read (r, d, entry->closid, entry->rmid,QOS_L3_OCCUP_EVENT_ID , &val)) { rmid_dirty = true;@@ -310,19 +327,21 @@ void __check_limbo(struct rdt_domain *d, bool force_free) }if (force_free || !rmid_dirty) { -clear_bit (entry->rmid, d->rmid_busy_llc); +clear_bit (idx, d->rmid_busy_llc);if (!--entry->busy) { rmid_limbo_count--;list_add_tail (&entry->list, &rmid_free_lru); } } - crmid = nrmid +1 ; + cur_idx = idx +1 ; } } -bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d) +bool has_busy_rmid (struct rdt_domain *d) { -return find_first_bit (d->rmid_busy_llc, r->num_rmid) != r->num_rmid; +u32 idx_limit =resctrl_arch_system_num_rmid_idx (); + +return find_first_bit (d->rmid_busy_llc, idx_limit) != idx_limit; }/* @@ -352,6 +371,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)struct rdt_domain *d;int cpu, err;u64 val =0 ; +u32 idx; + + idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); entry->busy =0 ; cpu =get_cpu ();@@ -369,9 +391,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)* For the first limbo RMID in the domain, * setup up the limbo worker. */ -if (!has_busy_rmid (r, d)) +if (!has_busy_rmid (d))cqm_setup_limbo_handler (d,CQM_LIMBOCHECK_INTERVAL ); -set_bit (entry->rmid, d->rmid_busy_llc); +set_bit (idx, d->rmid_busy_llc); entry->busy++; }put_cpu ();@@ -384,14 +406,21 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)void free_rmid (u32 closid,u32 rmid) { +u32 idx =resctrl_arch_rmid_idx_encode (closid, rmid);struct rmid_entry *entry; -if (!rmid) -return ; -lockdep_assert_held (&rdtgroup_mutex); - entry =__rmid_entry (closid, rmid); +/* +* Do not allow the default rmid to be free'd. Comparing by index +* allows architectures that ignore the closid parameter to avoid an +* unnecessary check. +*/ +if (idx ==resctrl_arch_rmid_idx_encode (RESCTRL_RESERVED_CLOSID , +RESCTRL_RESERVED_RMID )) +return ; + + entry =__rmid_entry (idx);if (is_llc_occupancy_enabled ())add_rmid_to_limbo (entry);@@ -402,11 +431,13 @@ void free_rmid(u32 closid, u32 rmid)static struct mbm_state *get_mbm_state (struct rdt_domain *d,u32 closid,u32 rmid,enum resctrl_event_id evtid) { +u32 idx =resctrl_arch_rmid_idx_encode (closid, rmid); +switch (evtid) {case QOS_L3_MBM_TOTAL_EVENT_ID : -return &d->mbm_total[rmid]; +return &d->mbm_total[idx];case QOS_L3_MBM_LOCAL_EVENT_ID : -return &d->mbm_local[rmid]; +return &d->mbm_local[idx];default :return NULL ; }@@ -449,7 +480,8 @@ static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr)*/ static void mbm_bw_count (u32 closid,u32 rmid,struct rmid_read *rr) { -struct mbm_state *m = &rr->d->mbm_local[rmid]; +u32 idx =resctrl_arch_rmid_idx_encode (closid, rmid); +struct mbm_state *m = &rr->d->mbm_local[idx];u64 cur_bw, bytes, cur_bytes; cur_bytes = rr->val;@@ -539,7 +571,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) {u32 closid, rmid, cur_msr_val, new_msr_val;struct mbm_state *pmbm_data, *cmbm_data; -u32 cur_bw, delta_bw, user_bw; +u32 cur_bw, delta_bw, user_bw, idx;struct rdt_resource *r_mba;struct rdt_domain *dom_mba;struct list_head *head;@@ -552,7 +584,8 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) closid = rgrp->closid; rmid = rgrp->mon.rmid; - pmbm_data = &dom_mbm->mbm_local[rmid]; + idx =resctrl_arch_rmid_idx_encode (closid, rmid); + pmbm_data = &dom_mbm->mbm_local[idx]; dom_mba =get_domain_from_cpu (smp_processor_id (), r_mba);if (!dom_mba) {@@ -670,7 +703,7 @@ void cqm_handle_limbo(struct work_struct *work)__check_limbo (d, false); -if (has_busy_rmid (r, d)) +if (has_busy_rmid (d))schedule_delayed_work_on (cpu, &d->cqm_limbo, delay);mutex_unlock (&rdtgroup_mutex);@@ -735,19 +768,20 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)static int dom_data_init (struct rdt_resource *r) { +u32 idx_limit =resctrl_arch_system_num_rmid_idx ();struct rmid_entry *entry =NULL ; -int i, nr_rmids; +u32 idx; +int i; - nr_rmids = r->num_rmid; - rmid_ptrs =kcalloc (nr_rmids,sizeof (struct rmid_entry ),GFP_KERNEL ); + rmid_ptrs =kcalloc (idx_limit,sizeof (struct rmid_entry ),GFP_KERNEL );if (!rmid_ptrs)return -ENOMEM ; -for (i =0 ; i < nr_rmids; i++) { +for (i =0 ; i < idx_limit; i++) { entry = &rmid_ptrs[i];INIT_LIST_HEAD (&entry->list); - entry->rmid = i; +resctrl_arch_rmid_idx_decode (i, &entry->closid, &entry->rmid);list_add_tail (&entry->list, &rmid_free_lru); }@@ -755,7 +789,9 @@ static int dom_data_init(struct rdt_resource *r)* These are used for rdtgroup_default control group, which will be * setup later in rdtgroup_init(). */ - entry =__rmid_entry (RESCTRL_RESERVED_CLOSID ,RESCTRL_RESERVED_RMID ); + idx =resctrl_arch_rmid_idx_encode (RESCTRL_RESERVED_CLOSID , +RESCTRL_RESERVED_RMID ); + entry =__rmid_entry (idx);list_del (&entry->list);return 0 ;arch/x86/kernel/cpu/resctrl/rdtgroup.c +5
-4@@ -3894,7 +3894,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)if (is_mbm_enabled ())cancel_delayed_work (&d->mbm_over); -if (is_llc_occupancy_enabled () &&has_busy_rmid (r, d)) { +if (is_llc_occupancy_enabled () &&has_busy_rmid (d)) {/* * When a package is going down, forcefully * decrement rmid->ebusy. There is no way to know @@ -3912,16 +3912,17 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)static int domain_setup_mon_state (struct rdt_resource *r,struct rdt_domain *d) { +u32 idx_limit =resctrl_arch_system_num_rmid_idx ();size_t tsize;if (is_llc_occupancy_enabled ()) { - d->rmid_busy_llc =bitmap_zalloc (r->num_rmid,GFP_KERNEL ); + d->rmid_busy_llc =bitmap_zalloc (idx_limit,GFP_KERNEL );if (!d->rmid_busy_llc)return -ENOMEM ; }if (is_mbm_total_enabled ()) { tsize =sizeof (*d->mbm_total); - d->mbm_total =kcalloc (r->num_rmid, tsize,GFP_KERNEL ); + d->mbm_total =kcalloc (idx_limit, tsize,GFP_KERNEL );if (!d->mbm_total) {bitmap_free (d->rmid_busy_llc);return -ENOMEM ;@@ -3929,7 +3930,7 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) }if (is_mbm_local_enabled ()) { tsize =sizeof (*d->mbm_local); - d->mbm_local =kcalloc (r->num_rmid, tsize,GFP_KERNEL ); + d->mbm_local =kcalloc (idx_limit, tsize,GFP_KERNEL );if (!d->mbm_local) {bitmap_free (d->rmid_busy_llc);kfree (d->mbm_total);
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerPATCH v7 7/24 x86/resctrl: Allow RMID allocation to be scoped by CLOSID
In reply to James MorseMPAMs RMID values are not unique unless the CLOSID is considered as well.
alloc_rmid() expects the RMID to be an independent number.
Pass the CLOSID in to alloc_rmid(). Use this to compare indexes when allocating. If the CLOSID is not relevant to the index, this ends up comparing the free RMID with itself, and the first free entry will be used. With MPAM the CLOSID is included in the index, so this becomes a walk of the free RMID entries, until one that matches the supplied CLOSID is found.
---
Changes since v2;
* Rephrased comment in resctrl_find_free_rmid() to describe this in terms of list_entry_first()
* Rephrased comment above alloc_rmid()
Changes since v3:
* Flipped conditions in alloc_rmid()
Changes since v4:
* Typo in comment
Changes since v5:
* Reworded two comments.
Changes since v6:
* Fixed a typo and some whitespace.
--- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 51 +++++++++++++++++------ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 2 +- 4 files changed, 41 insertions(+), 16 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +1 -1
@@ -545,7 +545,7 @@ void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int closids_supported (void );void closid_free (int closid); -int alloc_rmid (void ); +int alloc_rmid (u32 closid);void free_rmid (u32 closid,u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r); void __exit rdt_put_mon_l3_config(struct rdt_resource *r);
arch/x86/kernel/cpu/resctrl/monitor.c +38 -13
62 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +1 -1
@@ -777,7 +777,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)int ret;if (rdt_mon_capable) { - ret =alloc_rmid (); + ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) {rdt_last_cmd_puts ("Out of RMIDs\n" );return ret;
arch/x86/kernel/cpu/resctrl/rdtgroup.c +1 -1
@@ -3300,7 +3300,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)if (!rdt_mon_capable)return 0 ; - ret =alloc_rmid (); + ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) {rdt_last_cmd_puts ("Out of RMIDs\n" );return ret;
Signature · 1 line
2.39.2
Re: [PATCH v7 07/24] x86/resctrl: Allow RMID allocation to be scoped by CLOSID
Hi James,
Quoted 11 lines · James Morse
MPAMs RMID values are not unique unless the CLOSID is considered as well.
alloc_rmid()expects the RMID to be an independent number.Pass the CLOSID in to
alloc_rmid(). Use this to compare indexes when allocating. If the CLOSID is not relevant to the index, this ends up comparing the free RMID with itself, and the first free entry will be used. With MPAM the CLOSID is included in the index, so this becomes a walk of the free RMID entries, until one that matches the supplied CLOSID is found.
Quoted 5 lines
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>---
Reinette
Re: [PATCH v7 07/24] x86/resctrl: Allow RMID allocation to be scoped by CLOSID
In reply to James MorseHi James,
Quoted 11 lines · James Morse
MPAMs RMID values are not unique unless the CLOSID is considered as well.
alloc_rmid()expects the RMID to be an independent number.Pass the CLOSID in to
alloc_rmid(). Use this to compare indexes when allocating. If the CLOSID is not relevant to the index, this ends up comparing the free RMID with itself, and the first free entry will be used. With MPAM the CLOSID is included in the index, so this becomes a walk of the free RMID entries, until one that matches the supplied CLOSID is found.
Quoted 4 lines
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 130 lines
---
Changes since v2;
* Rephrased comment inresctrl_find_free_rmid()to describe this in terms oflist_entry_first()
* Rephrased comment abovealloc_rmid()Changes since v3:
* Flipped conditions inalloc_rmid()Changes since v4:
* Typo in commentChanges since v5:
* Reworded two comments.Changes since v6:
* Fixed a typo and some whitespace.--- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 51 +++++++++++++++++------ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 2 +- 4 files changed, 41 insertions(+), 16 deletions(-)arch/x86/kernel/cpu/resctrl/internal.h +1
-1@@ -545,7 +545,7 @@ void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int closids_supported (void );void closid_free (int closid); -int alloc_rmid (void ); +int alloc_rmid (u32 closid);void free_rmid (u32 closid,u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r); void __exit rdt_put_mon_l3_config(struct rdt_resource *r);arch/x86/kernel/cpu/resctrl/monitor.c +38
-13@@ -344,24 +344,49 @@ bool has_busy_rmid(struct rdt_domain *d)return find_first_bit (d->rmid_busy_llc, idx_limit) != idx_limit; } -/* -* As of now the RMIDs allocation is global. -* However we keep track of which packages the RMIDs -* are used to optimize the limbo list management. -*/ -int alloc_rmid (void ) +static struct rmid_entry *resctrl_find_free_rmid (u32 closid) { -struct rmid_entry *entry; - -lockdep_assert_held (&rdtgroup_mutex); +struct rmid_entry *itr; +u32 itr_idx, cmp_idx;if (list_empty (&rmid_free_lru)) -return rmid_limbo_count ? -EBUSY : -ENOSPC ; +return rmid_limbo_count ?ERR_PTR (-EBUSY ) :ERR_PTR (-ENOSPC ); + +list_for_each_entry (itr, &rmid_free_lru, list) { +/* +* Get the index of this free RMID, and the index it would need +* to be if it were used with this CLOSID. +* If the CLOSID is irrelevant on this architecture, the two +* index values are always the same on every entry and thus the +* very first entry will be returned. +*/ + itr_idx =resctrl_arch_rmid_idx_encode (itr->closid, itr->rmid); + cmp_idx =resctrl_arch_rmid_idx_encode (closid, itr->rmid); + +if (itr_idx == cmp_idx) +return itr; + } + +return ERR_PTR (-ENOSPC ); +} + +/* +* For MPAM the RMID value is not unique, and has to be considered with +* the CLOSID. The (CLOSID, RMID) pair is allocated on all domains, which +* allows all domains to be managed by a single free list. +* Each domain also has a rmid_busy_llc to reduce the work of the limbo handler. +*/ +int alloc_rmid (u32 closid) +{ +struct rmid_entry *entry; + +lockdep_assert_held (&rdtgroup_mutex); + + entry =resctrl_find_free_rmid (closid); +if (IS_ERR (entry)) +return PTR_ERR (entry); - entry =list_first_entry (&rmid_free_lru, - struct rmid_entry, list);list_del (&entry->list); -return entry->rmid; }arch/x86/kernel/cpu/resctrl/pseudo_lock.c +1
-1@@ -777,7 +777,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)int ret;if (rdt_mon_capable) { - ret =alloc_rmid (); + ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) {rdt_last_cmd_puts ("Out of RMIDs\n" );return ret;arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1@@ -3300,7 +3300,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)if (!rdt_mon_capable)return 0 ; - ret =alloc_rmid (); + ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) {rdt_last_cmd_puts ("Out of RMIDs\n" );return ret;
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerPATCH v7 8/24 x86/resctrl: Track the number of dirty RMID a CLOSID has
In reply to James MorseMPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Keep track of the number of RMID held in limbo each CLOSID has. This will allow a future helper to find the 'cleanest' CLOSID when allocating.
The array is only needed when CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is defined. This will never be the case on x86.
---
Changes since v4:
* Moved closid_num_dirty_rmid[] update under entry->busy check
* Take the mutex in dom_data_init() as the caller doesn't.
Changes since v5:
* Added braces after an else.
* Made closid_num_dirty_rmid an unsigned int.
* Moved mutex_lock() in dom_data_init() to cover the whole function.
Changes since v6:
* Made closid_num_dirty_rmid[] and associated tmp variables u32.
--- arch/x86/kernel/cpu/resctrl/monitor.c | 66 +++++++++++++++++++++++---- 1 file changed, 56 insertions(+), 10 deletions(-)
arch/x86/kernel/cpu/resctrl/monitor.c +56 -10
@@ -50,6 +50,13 @@ struct rmid_entry {*/ static LIST_HEAD (rmid_free_lru); +/* +* @closid_num_dirty_rmid The number of dirty RMID each CLOSID has. +* Only allocated when CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is defined. +* Indexed by CLOSID. Protected by rdtgroup_mutex. +*/ +static u32 *closid_num_dirty_rmid; +/* * @rmid_limbo_count - count of currently unused but (potentially) * dirty RMIDs.
@@ -292,6 +299,17 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,return 0 ; } +static void limbo_release_entry (struct rmid_entry *entry) +{ +lockdep_assert_held (&rdtgroup_mutex); + + rmid_limbo_count--; +list_add_tail (&entry->list, &rmid_free_lru); + +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) + closid_num_dirty_rmid[entry->closid]--; +} +/* * Check the RMIDs that are marked as busy for this domain. If the * reported LLC occupancy is below the threshold clear the busy bit and
@@ -328,10 +346,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free)if (force_free || !rmid_dirty) {clear_bit (idx, d->rmid_busy_llc); -if (!--entry->busy) { - rmid_limbo_count--; -list_add_tail (&entry->list, &rmid_free_lru); - } +if (!--entry->busy) +limbo_release_entry (entry); } cur_idx = idx +1 ; }
@@ -398,6 +414,8 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)u64 val =0 ;u32 idx; +lockdep_assert_held (&rdtgroup_mutex); + idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); entry->busy =0 ;
@@ -423,10 +441,13 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) }put_cpu (); -if (entry->busy) +if (entry->busy) { rmid_limbo_count++; -else +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) + closid_num_dirty_rmid[entry->closid]++; + }else {list_add_tail (&entry->list, &rmid_free_lru); + } }void free_rmid (u32 closid,u32 rmid)
@@ -794,13 +815,30 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)static int dom_data_init (struct rdt_resource *r) {u32 idx_limit =resctrl_arch_system_num_rmid_idx (); +u32 num_closid =resctrl_arch_get_num_closid (r);struct rmid_entry *entry =NULL ; +int err =0 , i;u32 idx; -int i; + +mutex_lock (&rdtgroup_mutex); +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) { +u32 *tmp; + + tmp =kcalloc (num_closid,sizeof (*tmp),GFP_KERNEL ); +if (!tmp) { + err = -ENOMEM ; + goto out_unlock; + } + + closid_num_dirty_rmid = tmp; + } rmid_ptrs =kcalloc (idx_limit,sizeof (struct rmid_entry ),GFP_KERNEL ); -if (!rmid_ptrs) -return -ENOMEM ; +if (!rmid_ptrs) { +kfree (closid_num_dirty_rmid); + err = -ENOMEM ; + goto out_unlock; + }for (i =0 ; i < idx_limit; i++) { entry = &rmid_ptrs[i];
@@ -819,13 +857,21 @@ static int dom_data_init(struct rdt_resource *r) entry =__rmid_entry (idx);list_del (&entry->list); -return 0 ; +out_unlock: +mutex_unlock (&rdtgroup_mutex); + +return err; } static void __exit dom_data_exit(struct rdt_resource *r) {mutex_lock (&rdtgroup_mutex); +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) { +kfree (closid_num_dirty_rmid); + closid_num_dirty_rmid =NULL ; + } +kfree (rmid_ptrs); rmid_ptrs =NULL ;
Signature · 1 line
2.39.2
Re: [PATCH v7 08/24] x86/resctrl: Track the number of dirty RMID a CLOSID has
Hi James,
Quoted 18 lines · James Morse
arch/x86/kernel/cpu/resctrl/monitor.c +11
-1@@ -794,13 +815,30 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)static int dom_data_init (struct rdt_resource *r) {u32 idx_limit =resctrl_arch_system_num_rmid_idx (); +u32 num_closid =resctrl_arch_get_num_closid (r);struct rmid_entry *entry =NULL ; +int err =0 , i;u32 idx; -int i; + +mutex_lock (&rdtgroup_mutex); +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) { +u32 *tmp; + + tmp =kcalloc (num_closid,sizeof (*tmp),GFP_KERNEL ); +if (!tmp) { + err = -ENOMEM ; + goto out_unlock;
arch/x86/kernel/cpu/resctrl/monitor.c void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) · line 838
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/monitor.c +6
-2+ } + + closid_num_dirty_rmid = tmp; + } rmid_ptrs =kcalloc (idx_limit,sizeof (struct rmid_entry ),GFP_KERNEL ); -if (!rmid_ptrs) -return -ENOMEM ; +if (!rmid_ptrs) { +kfree (closid_num_dirty_rmid);
Since this is a global variable and resctrl keeps running on this alloc failure I think it will be safer to add a:
closid_num_dirty_rmid = NULL
With that added you can add:
Reinette
James Morse
reviewerRe: [PATCH v7 08/24] x86/resctrl: Track the number of dirty RMID a CLOSID has
Hi Reinette,
Quoted 33 lines · Reinette Chatre
Quoted 18 lines · James Morse
arch/x86/kernel/cpu/resctrl/monitor.c +11
-1@@ -794,13 +815,30 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)static int dom_data_init (struct rdt_resource *r) {u32 idx_limit =resctrl_arch_system_num_rmid_idx (); +u32 num_closid =resctrl_arch_get_num_closid (r);struct rmid_entry *entry =NULL ; +int err =0 , i;u32 idx; -int i; + +mutex_lock (&rdtgroup_mutex); +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) { +u32 *tmp; + + tmp =kcalloc (num_closid,sizeof (*tmp),GFP_KERNEL ); +if (!tmp) { + err = -ENOMEM ; + goto out_unlock;arch/x86/kernel/cpu/resctrl/monitor.c void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) · line 838
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/monitor.c +6
-2+ } + + closid_num_dirty_rmid = tmp; + } rmid_ptrs =kcalloc (idx_limit,sizeof (struct rmid_entry ),GFP_KERNEL ); -if (!rmid_ptrs) -return -ENOMEM ; +if (!rmid_ptrs) { +kfree (closid_num_dirty_rmid);Since this is a global variable and resctrl keeps running on this alloc failure I think it will be safer to add a:
Quoted 1 line
closid_num_dirty_rmid= NULL
Yup, I had that for dom_data_exit(), but missed it here.
Quoted 3 lines
With that added you can add:
Reviewed-by Reinette Chatre <[email protected]>
Thanks!
James
Moger, Babu
reviewerRe: [PATCH v7 08/24] x86/resctrl: Track the number of dirty RMID a CLOSID has
In reply to James MorseHi James,
Quoted 118 lines · James Morse
MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Keep track of the number of RMID held in limbo each CLOSID has. This will allow a future helper to find the 'cleanest' CLOSID when allocating.
The array is only needed when
CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSIDis defined. This will never be the case on x86.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since v4:
* Moved closid_num_dirty_rmid[] update underentry->busycheck
* Take the mutex indom_data_init()as the caller doesn't.Changes since v5:
* Added braces after an else.
* Madeclosid_num_dirty_rmidan unsigned int.
* Movedmutex_lock()indom_data_init()to cover the whole function.Changes since v6:
* Made closid_num_dirty_rmid[] and associated tmp variables u32.--- arch/x86/kernel/cpu/resctrl/monitor.c | 66 +++++++++++++++++++++++---- 1 file changed, 56 insertions(+), 10 deletions(-)arch/x86/kernel/cpu/resctrl/monitor.c +32
-7@@ -50,6 +50,13 @@ struct rmid_entry {*/ static LIST_HEAD (rmid_free_lru); +/* +* @closid_num_dirty_rmid The number of dirty RMID each CLOSID has. +* Only allocated when CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is defined. +* Indexed by CLOSID. Protected by rdtgroup_mutex. +*/ +static u32 *closid_num_dirty_rmid; +/* * @rmid_limbo_count - count of currently unused but (potentially) * dirty RMIDs. @@ -292,6 +299,17 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,return 0 ; } +static void limbo_release_entry (struct rmid_entry *entry) +{ +lockdep_assert_held (&rdtgroup_mutex); + + rmid_limbo_count--; +list_add_tail (&entry->list, &rmid_free_lru); + +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) + closid_num_dirty_rmid[entry->closid]--; +} +/* * Check the RMIDs that are marked as busy for this domain. If the * reported LLC occupancy is below the threshold clear the busy bit and @@ -328,10 +346,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free)if (force_free || !rmid_dirty) {clear_bit (idx, d->rmid_busy_llc); -if (!--entry->busy) { - rmid_limbo_count--; -list_add_tail (&entry->list, &rmid_free_lru); - } +if (!--entry->busy) +limbo_release_entry (entry); } cur_idx = idx +1 ; }@@ -398,6 +414,8 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)u64 val =0 ;u32 idx; +lockdep_assert_held (&rdtgroup_mutex); + idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); entry->busy =0 ;@@ -423,10 +441,13 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) }put_cpu (); -if (entry->busy) +if (entry->busy) { rmid_limbo_count++; -else +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) + closid_num_dirty_rmid[entry->closid]++; + }else {list_add_tail (&entry->list, &rmid_free_lru); + } }void free_rmid (u32 closid,u32 rmid)@@ -794,13 +815,30 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)static int dom_data_init (struct rdt_resource *r) {u32 idx_limit =resctrl_arch_system_num_rmid_idx (); +u32 num_closid =resctrl_arch_get_num_closid (r);struct rmid_entry *entry =NULL ; +int err =0 , i;u32 idx; -int i; + +mutex_lock (&rdtgroup_mutex); +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) {
arch/x86/kernel/cpu/resctrl/monitor.c void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) · line 834
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/monitor.c +10
-0+u32 *tmp; + + tmp =kcalloc (num_closid,sizeof (*tmp),GFP_KERNEL ); +if (!tmp) { + err = -ENOMEM ; + goto out_unlock; + } + + closid_num_dirty_rmid = tmp; + }
Any reason tmp variable required here?
arch/x86/kernel/cpu/resctrl/monitor.c void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) · line 838
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/monitor.c +2
-2rmid_ptrs =kcalloc (idx_limit,sizeof (struct rmid_entry ),GFP_KERNEL ); -if (!rmid_ptrs) -return -ENOMEM ; +if (!rmid_ptrs) { +kfree (closid_num_dirty_rmid);
Should there be check here while feeing?
if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
Quoted 29 lines
arch/x86/kernel/cpu/resctrl/monitor.c +12
-1+ err = -ENOMEM ; + goto out_unlock; + }for (i =0 ; i < idx_limit; i++) { entry = &rmid_ptrs[i];@@ -819,13 +857,21 @@ static int dom_data_init(struct rdt_resource *r) entry =__rmid_entry (idx);list_del (&entry->list); -return 0 ; +out_unlock: +mutex_unlock (&rdtgroup_mutex); + +return err; } static void __exit dom_data_exit(struct rdt_resource *r) {mutex_lock (&rdtgroup_mutex); +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) { +kfree (closid_num_dirty_rmid); + closid_num_dirty_rmid =NULL ; + } +kfree (rmid_ptrs); rmid_ptrs =NULL ;
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 08/24] x86/resctrl: Track the number of dirty RMID a CLOSID has
Hi Babu,
Quoted 33 lines · Babu
Quoted 32 lines · James Morse
MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Keep track of the number of RMID held in limbo each CLOSID has. This will allow a future helper to find the 'cleanest' CLOSID when allocating.
The array is only needed when
CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSIDis defined. This will never be the case on x86.arch/x86/kernel/cpu/resctrl/monitor.c +9
-1@@ -794,13 +815,30 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)static int dom_data_init (struct rdt_resource *r) {u32 idx_limit =resctrl_arch_system_num_rmid_idx (); +u32 num_closid =resctrl_arch_get_num_closid (r);struct rmid_entry *entry =NULL ; +int err =0 , i;u32 idx; -int i; + +mutex_lock (&rdtgroup_mutex); +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) { +u32 *tmp; + + tmp =kcalloc (num_closid,sizeof (*tmp),GFP_KERNEL ); +if (!tmp) {
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/monitor.c void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) · line 834
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/monitor.c +6
-0+ err = -ENOMEM ; + goto out_unlock; + } + + closid_num_dirty_rmid = tmp; + }Any reason tmp variable required here?
Line length barking from checkpatch, the resulting newlines and indentation were hard to read, I figured this was more readable.
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/monitor.c void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) · line 838
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/monitor.c +2
-2rmid_ptrs =kcalloc (idx_limit,sizeof (struct rmid_entry ),GFP_KERNEL ); -if (!rmid_ptrs) -return -ENOMEM ; +if (!rmid_ptrs) { +kfree (closid_num_dirty_rmid);Should there be check here while feeing?
if (
IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
Not for the sake of kfree(), which is quite happy with NULL.
But it looks like it is needed for the compiler to realise that closid_num_dirty_rmid isn't used at all, and can be optimised out - which was my intention.
Thanks, I'll add that.
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/monitor.c void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms) · line 844
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/monitor.c +3
-0+ err = -ENOMEM ; + goto out_unlock; + }for (i =0 ; i < idx_limit; i++) { entry = &rmid_ptrs[i];
Thanks,
James
James Morse
reviewerPATCH v7 9/24 x86/resctrl: Use __set_bit()/__clear_bit() instead of open coding
In reply to James MorseThe resctrl CLOSID allocator uses a single 32bit word to track which CLOSID are free. The setting and clearing of bits is open coded.
A subsequent patch adds closid_allocated(), which adds more open coded bitmaps operations. These will eventually need changing to use the bitops helpers so that a CLOSID bitmap of the correct size can be allocated dynamically.
Convert the existing open coded bit manipulations of closid_free_map to use __set_bit() and friends. These don't need to be atomic as this list is protected by the mutex.
---
Changes since v6:
* Use the __ inatomic helpers and add lockdep_assert_held() annotations to document how this is safe.
* Fixed a resctrl_closid_is_free()/closid_allocated() rename in the commit message.
* Use RESCTRL_RESERVED_CLOSID to improve readability.
--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/resctrl/rdtgroup.c +11 -5
@@ -111,7 +111,7 @@ void rdt_staged_configs_clear(void)* - Our choices on how to configure each resource become progressively more * limited as the number of resources grows. */ -static int closid_free_map; +static unsigned long closid_free_map;static int closid_free_map_len;int closids_supported (void )
@@ -131,7 +131,7 @@ static void closid_init(void) closid_free_map =BIT_MASK (rdt_min_closid) -1 ;/* CLOSID 0 is always reserved for the default group */ - closid_free_map &= ~1 ; +__clear_bit (RESCTRL_RESERVED_CLOSID , &closid_free_map); closid_free_map_len = rdt_min_closid; }
@@ -139,17 +139,21 @@ static int closid_alloc(void) {u32 closid =ffs (closid_free_map); +lockdep_assert_held (&rdtgroup_mutex); +if (closid ==0 )return -ENOSPC ; closid--; - closid_free_map &= ~(1 << closid); +__clear_bit (closid, &closid_free_map);return closid; }void closid_free (int closid) { - closid_free_map |=1 << closid; +lockdep_assert_held (&rdtgroup_mutex); + +__set_bit (closid, &closid_free_map); }/**
@@ -161,7 +165,9 @@ void closid_free(int closid)*/ static bool closid_allocated (unsigned int closid) { -return (closid_free_map & (1 << closid)) ==0 ; +lockdep_assert_held (&rdtgroup_mutex); + +return !test_bit (closid, &closid_free_map); }/**
Signature · 1 line
2.39.2
Re: [PATCH v7 09/24] x86/resctrl: Use __set_bit()/__clear_bit() instead of open coding
Hi James,
Quoted 4 lines · James Morse
The resctrl CLOSID allocator uses a single 32bit word to track which CLOSID are free. The setting and clearing of bits is open coded.
A subsequent patch adds
closid_allocated(), which adds more open
(Note use of "A subsequent patch ")
Quoted 38 lines
coded bitmaps operations. These will eventually need changing to use the bitops helpers so that a CLOSID bitmap of the correct size can be allocated dynamically.
Convert the existing open coded bit manipulations of
closid_free_mapto use__set_bit()and friends. These don't need to be atomic as this list is protected by the mutex.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since v6:
* Use the __ inatomic helpers and addlockdep_assert_held()annotations to document how this is safe.
* Fixed a resctrl_closid_is_free()/closid_allocated() rename in the commit message.
* UseRESCTRL_RESERVED_CLOSIDto improve readability.--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1@@ -111,7 +111,7 @@ void rdt_staged_configs_clear(void)* - Our choices on how to configure each resource become progressively more * limited as the number of resources grows. */ -static int closid_free_map; +static unsigned long closid_free_map;static int closid_free_map_len;int closids_supported (void )
arch/x86/kernel/cpu/resctrl/rdtgroup.c static void closid_init(void) · line 133
Quoted 3 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0@@ -131,7 +131,7 @@ static void closid_init(void) closid_free_map =BIT_MASK (rdt_min_closid) -1 ;/* CLOSID 0 is always reserved for the default group */
Seems appropriate for the comment to be updated to the new define also.
With that addressed you can add:
Reinette
James Morse
reviewerRe: [PATCH v7 09/24] x86/resctrl: Use __set_bit()/__clear_bit() instead of open coding
Hi Reinette,
Quoted 7 lines · Reinette Chatre
Quoted 4 lines · James Morse
The resctrl CLOSID allocator uses a single 32bit word to track which CLOSID are free. The setting and clearing of bits is open coded.
A subsequent patch adds
closid_allocated(), which adds more open(Note use of "A subsequent patch ")
Yup, I've dropped this paragraph.
Quoted 15 lines
Quoted 15 lines
coded bitmaps operations. These will eventually need changing to use the bitops helpers so that a CLOSID bitmap of the correct size can be allocated dynamically.
Convert the existing open coded bit manipulations of
closid_free_mapto use__set_bit()and friends. These don't need to be atomic as this list is protected by the mutex.arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0@@ -131,7 +131,7 @@ static void closid_init(void) closid_free_map =BIT_MASK (rdt_min_closid) -1 ;
Quoted 3 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c static void closid_init(void) · line 133
Quoted 1 line
arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0/* CLOSID 0 is always reserved for the default group */ Seems appropriate for the comment to be updated to the new define also.
Sure,
Quoted 2 lines
With that addressed you can add:
Reviewed-by Reinette Chatre <[email protected]>
Thanks!
James
Re: [PATCH v7 09/24] x86/resctrl: Use __set_bit()/__clear_bit() instead of open coding
In reply to James MorseHi James,
Quoted 8 lines · James Morse
The resctrl CLOSID allocator uses a single 32bit word to track which CLOSID are free. The setting and clearing of bits is open coded.
A subsequent patch adds
closid_allocated(), which adds more open coded bitmaps operations. These will eventually need changing to use the bitops helpers so that a CLOSID bitmap of the correct size can be allocated dynamically.
Quoted 8 lines
Convert the existing open coded bit manipulations of
closid_free_mapto use__set_bit()and friends. These don't need to be atomic as this list is protected by the mutex.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 69 lines
---
Changes since v6:
* Use the __ inatomic helpers and addlockdep_assert_held()annotations to document how this is safe.
* Fixed a resctrl_closid_is_free()/closid_allocated() rename in the commit message.
* UseRESCTRL_RESERVED_CLOSIDto improve readability.--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)arch/x86/kernel/cpu/resctrl/rdtgroup.c +11
-5@@ -111,7 +111,7 @@ void rdt_staged_configs_clear(void)* - Our choices on how to configure each resource become progressively more * limited as the number of resources grows. */ -static int closid_free_map; +static unsigned long closid_free_map;static int closid_free_map_len;int closids_supported (void )@@ -131,7 +131,7 @@ static void closid_init(void) closid_free_map =BIT_MASK (rdt_min_closid) -1 ;/* CLOSID 0 is always reserved for the default group */ - closid_free_map &= ~1 ; +__clear_bit (RESCTRL_RESERVED_CLOSID , &closid_free_map); closid_free_map_len = rdt_min_closid; }@@ -139,17 +139,21 @@ static int closid_alloc(void) {u32 closid =ffs (closid_free_map); +lockdep_assert_held (&rdtgroup_mutex); +if (closid ==0 )return -ENOSPC ; closid--; - closid_free_map &= ~(1 << closid); +__clear_bit (closid, &closid_free_map);return closid; }void closid_free (int closid) { - closid_free_map |=1 << closid; +lockdep_assert_held (&rdtgroup_mutex); + +__set_bit (closid, &closid_free_map); }/** @@ -161,7 +165,9 @@ void closid_free(int closid)*/ static bool closid_allocated (unsigned int closid) { -return (closid_free_map & (1 << closid)) ==0 ; +lockdep_assert_held (&rdtgroup_mutex); + +return !test_bit (closid, &closid_free_map); }/**
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 09/24] x86/resctrl: Use __set_bit()/__clear_bit() instead of open coding
Hi Babu,
Quoted 9 lines · Babu
Quoted 8 lines · James Morse
The resctrl CLOSID allocator uses a single 32bit word to track which CLOSID are free. The setting and clearing of bits is open coded.
A subsequent patch adds
closid_allocated(), which adds more open coded bitmaps operations. These will eventually need changing to use the bitops helpers so that a CLOSID bitmap of the correct size can be allocated dynamically.
Quoted 5 lines
Quoted 3 lines
Convert the existing open coded bit manipulations of
closid_free_mapto use__set_bit()and friends. These don't need to be atomic as this list is protected by the mutex.Reviewed-by Babu Moger <[email protected]>
Thanks!
James
James Morse
reviewerPATCH v7 10/24 x86/resctrl: Allocate the cleanest CLOSID by searching closid_num_dirty_rmid
In reply to James MorseMPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Instead of allocating the first free CLOSID, on architectures where CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID is enabled, search closid_num_dirty_rmid[] to find the cleanest CLOSID.
The CLOSID found is returned to closid_alloc() for the free list to be updated.
---
Changes since v4:
* Dropped stale section from comment
Changes since v5:
* Renamed some variables.
No changes since v6
--- arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ arch/x86/kernel/cpu/resctrl/monitor.c | 45 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 ++++++++--- 3 files changed, 61 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
@@ -568,5 +568,7 @@ void rdt_domain_reconfigure_cdp(struct rdt_resource *r);void __init thread_throttle_mode_init (void );void __init mbm_config_rftype_init (const char *config);void rdt_staged_configs_clear (void ); +bool closid_allocated (unsigned int closid); +int resctrl_find_cleanest_closid (void );#endif /* _ASM_X86_RESCTRL_INTERNAL_H */
arch/x86/kernel/cpu/resctrl/monitor.c +45 -0
51 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/rdtgroup.c +14 -5
@@ -137,13 +137,22 @@ static void closid_init(void)static int closid_alloc (void ) { -u32 closid =ffs (closid_free_map); +int cleanest_closid; +u32 closid;lockdep_assert_held (&rdtgroup_mutex); -if (closid ==0 ) -return -ENOSPC ; - closid--; +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) { + cleanest_closid =resctrl_find_cleanest_closid (); +if (cleanest_closid <0 ) +return cleanest_closid; + closid = cleanest_closid; + }else { + closid =ffs (closid_free_map); +if (closid ==0 ) +return -ENOSPC ; + closid--; + }__clear_bit (closid, &closid_free_map);return closid;
@@ -163,7 +172,7 @@ void closid_free(int closid)* Return: true if @closid is currently associated with a resource group, * false if @closid is free */ -static bool closid_allocated (unsigned int closid) +bool closid_allocated (unsigned int closid) {lockdep_assert_held (&rdtgroup_mutex);
Signature · 1 line
2.39.2
Reinette Chatre
maintainerRe: [PATCH v7 10/24] x86/resctrl: Allocate the cleanest CLOSID by searching closid_num_dirty_rmid
Hi James,
Quoted 18 lines · James Morse
MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Instead of allocating the first free CLOSID, on architectures where
CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSIDis enabled, search closid_num_dirty_rmid[] to find the cleanest CLOSID.The CLOSID found is returned to
closid_alloc()for the free list to be updated.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 8 lines
---
Changes since v4:
* Dropped stale section from commentChanges since v5:
* Renamed some variables.No changes since v6
I use these patch changelogs to determine if I need to look at a patch for which I already provided a review tag. At first this patch appears to not deserve a second glance because I already provided a review tag and the above states "No changes since v6". Unfortunately this is false. I counted four changes. Now I cannot trust these "No changes since v6" and I need to dig out v6 to diff patches I already reviewed to determine if I need to look at them again. False patch changelogs make a patch series harder to review.
Reinette
James Morse
reviewerRe: [PATCH v7 10/24] x86/resctrl: Allocate the cleanest CLOSID by searching closid_num_dirty_rmid
Quoted 30 lines · Reinette Chatre
Hi James,
Quoted 18 lines · James Morse
MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Instead of allocating the first free CLOSID, on architectures where
CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSIDis enabled, search closid_num_dirty_rmid[] to find the cleanest CLOSID.The CLOSID found is returned to
closid_alloc()for the free list to be updated.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>Quoted 8 lines
---
Changes since v4:
* Dropped stale section from commentChanges since v5:
* Renamed some variables.No changes since v6
Quoted 8 lines
I use these patch changelogs to determine if I need to look at a patch for which I already provided a review tag. At first this patch appears to not deserve a second glance because I already provided a review tag and the above states "No changes since v6". Unfortunately this is false. I counted four changes. Now I cannot trust these "No changes since v6" and I need to dig out v6 to diff patches I already reviewed to determine if I need to look at them again. False patch changelogs make a patch series harder to review.
Sorry, looks like I applied the changes suggested by Babu, but didn't update this bit of text that doesn't get committed. I added this 'No changes' text to any patch that didn't have any entries.
You should be able to rely on people dropping Reviewed-by tags if there are substantial changes. This is the normal threshold for re-reviewing a patch.
James
Re: [PATCH v7 10/24] x86/resctrl: Allocate the cleanest CLOSID by searching closid_num_dirty_rmid
In reply to James MorseHi James,
Quoted 10 lines · James Morse
MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Instead of allocating the first free CLOSID, on architectures where
CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSIDis enabled, search closid_num_dirty_rmid[] to find the cleanest CLOSID.
Quoted 8 lines
The CLOSID found is returned to
closid_alloc()for the free list to be updated.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 122 lines
---
Changes since v4:
* Dropped stale section from commentChanges since v5:
* Renamed some variables.No changes since v6
--- arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ arch/x86/kernel/cpu/resctrl/monitor.c | 45 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 ++++++++--- 3 files changed, 61 insertions(+), 5 deletions(-)arch/x86/kernel/cpu/resctrl/internal.h +2
-0@@ -568,5 +568,7 @@ void rdt_domain_reconfigure_cdp(struct rdt_resource *r);void __init thread_throttle_mode_init (void );void __init mbm_config_rftype_init (const char *config);void rdt_staged_configs_clear (void ); +bool closid_allocated (unsigned int closid); +int resctrl_find_cleanest_closid (void );#endif /* _ASM_X86_RESCTRL_INTERNAL_H */ arch/x86/kernel/cpu/resctrl/monitor.c +45
-0@@ -386,6 +386,51 @@ static struct rmid_entry *resctrl_find_free_rmid(u32 closid)return ERR_PTR (-ENOSPC ); } +/** +* resctrl_find_cleanest_closid() - Find a CLOSID where all the associated +* RMID are clean, or the CLOSID that has +* the most clean RMID. +* +* MPAM's equivalent of RMID are per-CLOSID, meaning a freshly allocated CLOSID +* may not be able to allocate clean RMID. To avoid this the allocator will +* choose the CLOSID with the most clean RMID. +* +* When the CLOSID and RMID are independent numbers, the first free CLOSID will +* be returned. +*/ +int resctrl_find_cleanest_closid (void ) +{ +u32 cleanest_closid = ~0 ; +int i =0 ; + +lockdep_assert_held (&rdtgroup_mutex); + +if (!IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) +return -EIO ; + +for (i =0 ; i <closids_supported (); i++) { +int num_dirty; + +if (closid_allocated (i)) +continue ; + + num_dirty = closid_num_dirty_rmid[i]; +if (num_dirty ==0 ) +return i; + +if (cleanest_closid == ~0 ) + cleanest_closid = i; + +if (num_dirty < closid_num_dirty_rmid[cleanest_closid]) + cleanest_closid = i; + } + +if (cleanest_closid == ~0 ) +return -ENOSPC ; + +return cleanest_closid; +} +/* * For MPAM the RMID value is not unique, and has to be considered with * the CLOSID. The (CLOSID, RMID) pair is allocated on all domains, which arch/x86/kernel/cpu/resctrl/rdtgroup.c +14
-5@@ -137,13 +137,22 @@ static void closid_init(void)static int closid_alloc (void ) { -u32 closid =ffs (closid_free_map); +int cleanest_closid; +u32 closid;lockdep_assert_held (&rdtgroup_mutex); -if (closid ==0 ) -return -ENOSPC ; - closid--; +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) { + cleanest_closid =resctrl_find_cleanest_closid (); +if (cleanest_closid <0 ) +return cleanest_closid; + closid = cleanest_closid; + }else { + closid =ffs (closid_free_map); +if (closid ==0 ) +return -ENOSPC ; + closid--; + }__clear_bit (closid, &closid_free_map);return closid;@@ -163,7 +172,7 @@ void closid_free(int closid)* Return: true if @closid is currently associated with a resource group, * false if @closid is free */ -static bool closid_allocated (unsigned int closid) +bool closid_allocated (unsigned int closid) {lockdep_assert_held (&rdtgroup_mutex);
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 10/24] x86/resctrl: Allocate the cleanest CLOSID by searching closid_num_dirty_rmid
Hi Babu,
Quoted 7 lines · Babu
Quoted 6 lines · James Morse
MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Quoted 8 lines
Quoted 6 lines
Instead of allocating the first free CLOSID, on architectures where
CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSIDis enabled, search closid_num_dirty_rmid[] to find the cleanest CLOSID.The CLOSID found is returned to
closid_alloc()for the free list to be updated.Reviewed-by Babu Moger <[email protected]>
Thanks!
James
James Morse
reviewerPATCH v7 11/24 x86/resctrl: Move CLOSID/RMID matching and setting to use helpers
In reply to James MorseWhen switching tasks, the CLOSID and RMID that the new task should use are stored in struct task_struct. For x86 the CLOSID known by resctrl, the value in task_struct, and the value written to the CPU register are all the same thing.
MPAM's CPU interface has two different PARTID's one for data accesses the other for instruction fetch. Storing resctrl's CLOSID value in struct task_struct implies the arch code knows whether resctrl is using CDP.
Move the matching and setting of the struct task_struct properties to use helpers. This allows arm64 to store the hardware format of the register, instead of having to convert it each time.
__rdtgroup_move_task()s use of READ_ONCE()/WRITE_ONCE() ensures torn values aren't seen as another CPU may schedule the task being moved while the value is being changed. MPAM has an additional corner-case here as the PMG bits extend the PARTID space. If the scheduler sees a new-CLOSID but old-RMID, the task will dirty an RMID that the limbo code is not watching causing an inaccurate count. x86's RMID are independent values, so the limbo code will still be watching the old-RMID in this circumstance.
To avoid this, arm64 needs both the CLOSID/RMID WRITE_ONCE()d together. Both values must be provided together.
Because MPAM's RMID values are not unique, the CLOSID must be provided when matching the RMID.
---
Changes since v2:
* __rdtgroup_move_task() changed to set CLOSID from different CLOSID place depending on group type
No changes since v6
--- arch/x86/include/asm/resctrl.h | 18 ++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 62 ++++++++++++++++---------- 2 files changed, 56 insertions(+), 24 deletions(-)
arch/x86/include/asm/resctrl.h +18 -0
@@ -95,6 +95,24 @@ static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)return val * scale; } +static inline void resctrl_arch_set_closid_rmid (struct task_struct *tsk, +u32 closid,u32 rmid) +{ +WRITE_ONCE (tsk->closid, closid); +WRITE_ONCE (tsk->rmid, rmid); +} + +static inline bool resctrl_arch_match_closid (struct task_struct *tsk,u32 closid) +{ +return READ_ONCE (tsk->closid) == closid; +} + +static inline bool resctrl_arch_match_rmid (struct task_struct *tsk,u32 ignored, +u32 rmid) +{ +return READ_ONCE (tsk->rmid) == rmid; +} +static inline void resctrl_sched_in (struct task_struct *tsk) {if (static_branch_likely (&rdt_enable_key))
arch/x86/kernel/cpu/resctrl/rdtgroup.c +38 -24
114 lines not on this page · raw message
Signature · 1 line
2.39.2
Re: [PATCH v7 11/24] x86/resctrl: Move CLOSID/RMID matching and setting to use helpers
Hi James,
Quoted 25 lines · James Morse
When switching tasks, the CLOSID and RMID that the new task should use are stored in struct
task_struct. For x86 the CLOSID known by resctrl, the value intask_struct, and the value written to the CPU register are all the same thing.MPAM's CPU interface has two different PARTID's one for data accesses the other for instruction fetch. Storing resctrl's CLOSID value in struct
task_structimplies the arch code knows whether resctrl is using CDP.Move the matching and setting of the struct
task_structproperties to use helpers. This allows arm64 to store the hardware format of the register, instead of having to convert it each time.__rdtgroup_move_task()s use of READ_ONCE()/WRITE_ONCE() ensures torn values aren't seen as another CPU may schedule the task being moved while the value is being changed. MPAM has an additional corner-case here as the PMG bits extend the PARTID space. If the scheduler sees a new-CLOSID but old-RMID, the task will dirty an RMID that the limbo code is not watching causing an inaccurate count. x86's RMID are independent values, so the limbo code will still be watching the old-RMID in this circumstance.
To avoid this, arm64 needs both the CLOSID/RMID WRITE_ONCE()d together. Both values must be provided together.
Quoted 8 lines
Because MPAM's RMID values are not unique, the CLOSID must be provided when matching the RMID.
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 165 lines
---
Changes since v2:
*__rdtgroup_move_task()changed to set CLOSID from different CLOSID place depending on group typeNo changes since v6
--- arch/x86/include/asm/resctrl.h | 18 ++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 62 ++++++++++++++++---------- 2 files changed, 56 insertions(+), 24 deletions(-)arch/x86/include/asm/resctrl.h +18
-0@@ -95,6 +95,24 @@ static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)return val * scale; } +static inline void resctrl_arch_set_closid_rmid (struct task_struct *tsk, +u32 closid,u32 rmid) +{ +WRITE_ONCE (tsk->closid, closid); +WRITE_ONCE (tsk->rmid, rmid); +} + +static inline bool resctrl_arch_match_closid (struct task_struct *tsk,u32 closid) +{ +return READ_ONCE (tsk->closid) == closid; +} + +static inline bool resctrl_arch_match_rmid (struct task_struct *tsk,u32 ignored, +u32 rmid) +{ +return READ_ONCE (tsk->rmid) == rmid; +} +static inline void resctrl_sched_in (struct task_struct *tsk) {if (static_branch_likely (&rdt_enable_key))arch/x86/kernel/cpu/resctrl/rdtgroup.c +38
-24@@ -102,7 +102,7 @@ void rdt_staged_configs_clear(void)* * Using a global CLOSID across all resources has some advantages and * some drawbacks: -* + We can simply set "current->closid" to assign a task to a resource +* + We can simply set current's closid to assign a task to a resource * group. * + Context switch code can avoid extra memory references deciding which * CLOSID to load into the PQR_ASSOC MSR @@ -574,14 +574,26 @@ static void update_task_closid_rmid(struct task_struct *t)_update_task_closid_rmid (t); } +static bool task_in_rdtgroup (struct task_struct *tsk,struct rdtgroup *rdtgrp) +{ +u32 closid, rmid = rdtgrp->mon.rmid; + +if (rdtgrp->type ==RDTCTRL_GROUP ) + closid = rdtgrp->closid; +else if (rdtgrp->type ==RDTMON_GROUP ) + closid = rdtgrp->mon.parent->closid; +else +return false; + +return resctrl_arch_match_closid (tsk, closid) && +resctrl_arch_match_rmid (tsk, closid, rmid); +} +static int __rdtgroup_move_task (struct task_struct *tsk,struct rdtgroup *rdtgrp) {/* If the task is already in rdtgrp, no need to move the task. */ -if ((rdtgrp->type ==RDTCTRL_GROUP && tsk->closid == rdtgrp->closid && - tsk->rmid == rdtgrp->mon.rmid) || - (rdtgrp->type ==RDTMON_GROUP && tsk->rmid == rdtgrp->mon.rmid && - tsk->closid == rdtgrp->mon.parent->closid)) +if (task_in_rdtgroup (tsk, rdtgrp))return 0 ;/* @@ -592,19 +604,19 @@ static int __rdtgroup_move_task(struct task_struct *tsk,* For monitor groups, can move the tasks only from * their parent CTRL group. */ - -if (rdtgrp->type ==RDTCTRL_GROUP ) { -WRITE_ONCE (tsk->closid, rdtgrp->closid); -WRITE_ONCE (tsk->rmid, rdtgrp->mon.rmid); - }else if (rdtgrp->type ==RDTMON_GROUP ) { -if (rdtgrp->mon.parent->closid == tsk->closid) { -WRITE_ONCE (tsk->rmid, rdtgrp->mon.rmid); - }else { -rdt_last_cmd_puts ("Can't move task to different control group\n" ); -return -EINVAL ; - } +if (rdtgrp->type ==RDTMON_GROUP && + !resctrl_arch_match_closid (tsk, rdtgrp->mon.parent->closid)) { +rdt_last_cmd_puts ("Can't move task to different control group\n" ); +return -EINVAL ; } +if (rdtgrp->type ==RDTMON_GROUP ) +resctrl_arch_set_closid_rmid (tsk, rdtgrp->mon.parent->closid, + rdtgrp->mon.rmid); +else +resctrl_arch_set_closid_rmid (tsk, rdtgrp->closid, + rdtgrp->mon.rmid); +/* * Ensure the task's closid and rmid are written before determining if * the task is current that will decide if it will be interrupted. @@ -626,14 +638,15 @@ static int __rdtgroup_move_task(struct task_struct *tsk,static bool is_closid_match (struct task_struct *t,struct rdtgroup *r) { -return (rdt_alloc_capable && - (r->type ==RDTCTRL_GROUP ) && (t->closid == r->closid)); +return (rdt_alloc_capable && (r->type ==RDTCTRL_GROUP ) && +resctrl_arch_match_closid (t, r->closid)); }static bool is_rmid_match (struct task_struct *t,struct rdtgroup *r) { -return (rdt_mon_capable && - (r->type ==RDTMON_GROUP ) && (t->rmid == r->mon.rmid)); +return (rdt_mon_capable && (r->type ==RDTMON_GROUP ) && +resctrl_arch_match_rmid (t, r->mon.parent->closid, + r->mon.rmid)); }/** @@ -884,7 +897,7 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns, rdtg->mode != RDT_MODE_EXCLUSIVE)continue ; -if (rdtg->closid != tsk->closid) +if (!resctrl_arch_match_closid (tsk, rdtg->closid))continue ; seq_printf(s, "res:%s%s\n", (rdtg == &rdtgroup_default) ? "/" : "",@@ -892,7 +905,8 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns,seq_puts (s,"mon:" );list_for_each_entry (crg, &rdtg->mon.crdtgrp_list, mon.crdtgrp_list) { -if (tsk->rmid != crg->mon.rmid) +if (!resctrl_arch_match_rmid (tsk, crg->mon.parent->closid, + crg->mon.rmid))continue ;seq_printf (s,"%s" , crg->kn->name);break ;@@ -2825,8 +2839,8 @@ static void rdt_move_group_tasks(struct rdtgroup *from, struct rdtgroup *to,for_each_process_thread (p, t) {if (!from ||is_closid_match (t, from) ||is_rmid_match (t, from)) { -WRITE_ONCE (t->closid, to->closid); -WRITE_ONCE (t->rmid, to->mon.rmid); +resctrl_arch_set_closid_rmid (t, to->closid, + to->mon.rmid);/* * Order the closid/rmid stores above before the loads
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 11/24] x86/resctrl: Move CLOSID/RMID matching and setting to use helpers
Hi Babu,
Quoted 26 lines · Babu
Quoted 25 lines · James Morse
When switching tasks, the CLOSID and RMID that the new task should use are stored in struct
task_struct. For x86 the CLOSID known by resctrl, the value intask_struct, and the value written to the CPU register are all the same thing.MPAM's CPU interface has two different PARTID's one for data accesses the other for instruction fetch. Storing resctrl's CLOSID value in struct
task_structimplies the arch code knows whether resctrl is using CDP.Move the matching and setting of the struct
task_structproperties to use helpers. This allows arm64 to store the hardware format of the register, instead of having to convert it each time.__rdtgroup_move_task()s use of READ_ONCE()/WRITE_ONCE() ensures torn values aren't seen as another CPU may schedule the task being moved while the value is being changed. MPAM has an additional corner-case here as the PMG bits extend the PARTID space. If the scheduler sees a new-CLOSID but old-RMID, the task will dirty an RMID that the limbo code is not watching causing an inaccurate count. x86's RMID are independent values, so the limbo code will still be watching the old-RMID in this circumstance.
To avoid this, arm64 needs both the CLOSID/RMID WRITE_ONCE()d together. Both values must be provided together.
Quoted 4 lines
Quoted 2 lines
Because MPAM's RMID values are not unique, the CLOSID must be provided when matching the RMID.
Reviewed-by Babu Moger <[email protected]>
Thanks!
James
Re: [PATCH v7 11/24] x86/resctrl: Move CLOSID/RMID matching and setting to use helpers
In reply to James MorseQuoted 25 lines · James Morse
When switching tasks, the CLOSID and RMID that the new task should use are stored in struct
task_struct. For x86 the CLOSID known by resctrl, the value intask_struct, and the value written to the CPU register are all the same thing.MPAM's CPU interface has two different PARTID's one for data accesses the other for instruction fetch. Storing resctrl's CLOSID value in struct
task_structimplies the arch code knows whether resctrl is using CDP.Move the matching and setting of the struct
task_structproperties to use helpers. This allows arm64 to store the hardware format of the register, instead of having to convert it each time.__rdtgroup_move_task()s use of READ_ONCE()/WRITE_ONCE() ensures torn values aren't seen as another CPU may schedule the task being moved while the value is being changed. MPAM has an additional corner-case here as the PMG bits extend the PARTID space. If the scheduler sees a new-CLOSID but old-RMID, the task will dirty an RMID that the limbo code is not watching causing an inaccurate count. x86's RMID are independent values, so the limbo code will still be watching the old-RMID in this circumstance.
To avoid this, arm64 needs both the CLOSID/RMID WRITE_ONCE()d together. Both values must be provided together.
Quoted 8 lines
Because MPAM's RMID values are not unique, the CLOSID must be provided when matching the RMID.
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 165 lines
---
Changes since v2:
*__rdtgroup_move_task()changed to set CLOSID from different CLOSID place depending on group typeNo changes since v6
--- arch/x86/include/asm/resctrl.h | 18 ++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 62 ++++++++++++++++---------- 2 files changed, 56 insertions(+), 24 deletions(-)arch/x86/include/asm/resctrl.h +18
-0@@ -95,6 +95,24 @@ static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)return val * scale; } +static inline void resctrl_arch_set_closid_rmid (struct task_struct *tsk, +u32 closid,u32 rmid) +{ +WRITE_ONCE (tsk->closid, closid); +WRITE_ONCE (tsk->rmid, rmid); +} + +static inline bool resctrl_arch_match_closid (struct task_struct *tsk,u32 closid) +{ +return READ_ONCE (tsk->closid) == closid; +} + +static inline bool resctrl_arch_match_rmid (struct task_struct *tsk,u32 ignored, +u32 rmid) +{ +return READ_ONCE (tsk->rmid) == rmid; +} +static inline void resctrl_sched_in (struct task_struct *tsk) {if (static_branch_likely (&rdt_enable_key))arch/x86/kernel/cpu/resctrl/rdtgroup.c +38
-24@@ -102,7 +102,7 @@ void rdt_staged_configs_clear(void)* * Using a global CLOSID across all resources has some advantages and * some drawbacks: -* + We can simply set "current->closid" to assign a task to a resource +* + We can simply set current's closid to assign a task to a resource * group. * + Context switch code can avoid extra memory references deciding which * CLOSID to load into the PQR_ASSOC MSR @@ -574,14 +574,26 @@ static void update_task_closid_rmid(struct task_struct *t)_update_task_closid_rmid (t); } +static bool task_in_rdtgroup (struct task_struct *tsk,struct rdtgroup *rdtgrp) +{ +u32 closid, rmid = rdtgrp->mon.rmid; + +if (rdtgrp->type ==RDTCTRL_GROUP ) + closid = rdtgrp->closid; +else if (rdtgrp->type ==RDTMON_GROUP ) + closid = rdtgrp->mon.parent->closid; +else +return false; + +return resctrl_arch_match_closid (tsk, closid) && +resctrl_arch_match_rmid (tsk, closid, rmid); +} +static int __rdtgroup_move_task (struct task_struct *tsk,struct rdtgroup *rdtgrp) {/* If the task is already in rdtgrp, no need to move the task. */ -if ((rdtgrp->type ==RDTCTRL_GROUP && tsk->closid == rdtgrp->closid && - tsk->rmid == rdtgrp->mon.rmid) || - (rdtgrp->type ==RDTMON_GROUP && tsk->rmid == rdtgrp->mon.rmid && - tsk->closid == rdtgrp->mon.parent->closid)) +if (task_in_rdtgroup (tsk, rdtgrp))return 0 ;/* @@ -592,19 +604,19 @@ static int __rdtgroup_move_task(struct task_struct *tsk,* For monitor groups, can move the tasks only from * their parent CTRL group. */ - -if (rdtgrp->type ==RDTCTRL_GROUP ) { -WRITE_ONCE (tsk->closid, rdtgrp->closid); -WRITE_ONCE (tsk->rmid, rdtgrp->mon.rmid); - }else if (rdtgrp->type ==RDTMON_GROUP ) { -if (rdtgrp->mon.parent->closid == tsk->closid) { -WRITE_ONCE (tsk->rmid, rdtgrp->mon.rmid); - }else { -rdt_last_cmd_puts ("Can't move task to different control group\n" ); -return -EINVAL ; - } +if (rdtgrp->type ==RDTMON_GROUP && + !resctrl_arch_match_closid (tsk, rdtgrp->mon.parent->closid)) { +rdt_last_cmd_puts ("Can't move task to different control group\n" ); +return -EINVAL ; } +if (rdtgrp->type ==RDTMON_GROUP ) +resctrl_arch_set_closid_rmid (tsk, rdtgrp->mon.parent->closid, + rdtgrp->mon.rmid); +else +resctrl_arch_set_closid_rmid (tsk, rdtgrp->closid, + rdtgrp->mon.rmid); +/* * Ensure the task's closid and rmid are written before determining if * the task is current that will decide if it will be interrupted. @@ -626,14 +638,15 @@ static int __rdtgroup_move_task(struct task_struct *tsk,static bool is_closid_match (struct task_struct *t,struct rdtgroup *r) { -return (rdt_alloc_capable && - (r->type ==RDTCTRL_GROUP ) && (t->closid == r->closid)); +return (rdt_alloc_capable && (r->type ==RDTCTRL_GROUP ) && +resctrl_arch_match_closid (t, r->closid)); }static bool is_rmid_match (struct task_struct *t,struct rdtgroup *r) { -return (rdt_mon_capable && - (r->type ==RDTMON_GROUP ) && (t->rmid == r->mon.rmid)); +return (rdt_mon_capable && (r->type ==RDTMON_GROUP ) && +resctrl_arch_match_rmid (t, r->mon.parent->closid, + r->mon.rmid)); }/** @@ -884,7 +897,7 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns, rdtg->mode != RDT_MODE_EXCLUSIVE)continue ; -if (rdtg->closid != tsk->closid) +if (!resctrl_arch_match_closid (tsk, rdtg->closid))continue ; seq_printf(s, "res:%s%s\n", (rdtg == &rdtgroup_default) ? "/" : "",@@ -892,7 +905,8 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns,seq_puts (s,"mon:" );list_for_each_entry (crg, &rdtg->mon.crdtgrp_list, mon.crdtgrp_list) { -if (tsk->rmid != crg->mon.rmid) +if (!resctrl_arch_match_rmid (tsk, crg->mon.parent->closid, + crg->mon.rmid))continue ;seq_printf (s,"%s" , crg->kn->name);break ;@@ -2825,8 +2839,8 @@ static void rdt_move_group_tasks(struct rdtgroup *from, struct rdtgroup *to,for_each_process_thread (p, t) {if (!from ||is_closid_match (t, from) ||is_rmid_match (t, from)) { -WRITE_ONCE (t->closid, to->closid); -WRITE_ONCE (t->rmid, to->mon.rmid); +resctrl_arch_set_closid_rmid (t, to->closid, + to->mon.rmid);/* * Order the closid/rmid stores above before the loads
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerPATCH v7 12/24 x86/resctrl: Add cpumask_any_housekeeping() for limbo/overflow
In reply to James MorseThe limbo and overflow code picks a CPU to use from the domain's list of online CPUs. Work is then scheduled on these CPUs to maintain the limbo list and any counters that may overflow.
cpumask_any() may pick a CPU that is marked nohz_full, which will either penalise the work that CPU was dedicated to, or delay the processing of limbo list or counters that may overflow. Perhaps indefinitely. Delaying the overflow handling will skew the bandwidth values calculated by mba_sc, which expects to be called once a second.
Add cpumask_any_housekeeping() as a replacement for cpumask_any() that prefers housekeeping CPUs. This helper will still return a nohz_full CPU if that is the only option. The CPU to use is re-evaluated each time the limbo/overflow work runs. This ensures the work will move off a nohz_full CPU once a housekeeping CPU is available.
---
Changes since v3:
* typos fixed
Changes since v4:
* Made temporary variables unsigned
Changes since v5:
* Restructured cpumask_any_housekeeping() to avoid later churn.
Changes since v6:
* Update mbm_work_cpu/cqm_work_cpu when rescheduling.
--- arch/x86/kernel/cpu/resctrl/internal.h | 24 ++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/monitor.c | 20 +++++++++++++------- 2 files changed, 37 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +24 -0
@@ -7,6 +7,7 @@#include <linux/kernfs.h> #include <linux/fs_context.h> #include <linux/jump_label.h> +#include <linux/tick.h> #include <asm/resctrl.h>
@@ -56,6 +57,29 @@/* Max event bits supported */ #define MAX_EVT_CONFIG_BITS GENMASK(6, 0) +/** +* cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that +* aren't marked nohz_full +* @mask: The mask to pick a CPU from. +* +* Returns a CPU in @mask. If there are housekeeping CPUs that don't use +* nohz_full, these are preferred. +*/ +static inline unsigned int cpumask_any_housekeeping (const struct cpumask *mask) +{ +unsigned int cpu, hk_cpu; + + cpu =cpumask_any (mask); +if (!tick_nohz_full_cpu (cpu)) +return cpu; + + hk_cpu =cpumask_nth_andnot (0 , mask, tick_nohz_full_mask); +if (hk_cpu < nr_cpu_ids) + cpu = hk_cpu; + +return cpu; +} +struct rdt_fs_context {struct kernfs_fs_context kfc;bool enable_cdpl2;
arch/x86/kernel/cpu/resctrl/monitor.c +13 -7
@@ -783,7 +783,6 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,void cqm_handle_limbo (struct work_struct *work) {unsigned long delay =msecs_to_jiffies (CQM_LIMBOCHECK_INTERVAL ); -int cpu =smp_processor_id ();struct rdt_resource *r;struct rdt_domain *d;
@@ -794,8 +793,11 @@ void cqm_handle_limbo(struct work_struct *work)__check_limbo (d, false); -if (has_busy_rmid (d)) -schedule_delayed_work_on (cpu, &d->cqm_limbo, delay); +if (has_busy_rmid (d)) { + d->cqm_work_cpu =cpumask_any_housekeeping (&d->cpu_mask); +schedule_delayed_work_on (d->cqm_work_cpu, &d->cqm_limbo, + delay); + }mutex_unlock (&rdtgroup_mutex); }
@@ -805,7 +807,7 @@ void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms)unsigned long delay =msecs_to_jiffies (delay_ms);int cpu; - cpu =cpumask_any (&dom->cpu_mask); + cpu =cpumask_any_housekeeping (&dom->cpu_mask); dom->cqm_work_cpu = cpu;schedule_delayed_work_on (cpu, &dom->cqm_limbo, delay);
@@ -815,7 +817,6 @@ void mbm_handle_overflow(struct work_struct *work) {unsigned long delay =msecs_to_jiffies (MBM_OVERFLOW_INTERVAL );struct rdtgroup *prgrp, *crgrp; -int cpu =smp_processor_id ();struct list_head *head;struct rdt_resource *r;struct rdt_domain *d;
@@ -839,7 +840,12 @@ void mbm_handle_overflow(struct work_struct *work)update_mba_bw (prgrp, d); } -schedule_delayed_work_on (cpu, &d->mbm_over, delay); +/* +* Re-check for housekeeping CPUs. This allows the overflow handler to +* move off a nohz_full CPU quickly. +*/ + d->mbm_work_cpu =cpumask_any_housekeeping (&d->cpu_mask); +schedule_delayed_work_on (d->mbm_work_cpu, &d->mbm_over, delay); out_unlock:mutex_unlock (&rdtgroup_mutex);
@@ -852,7 +858,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)if (!static_branch_likely (&rdt_mon_enable_key))return ; - cpu =cpumask_any (&dom->cpu_mask); + cpu =cpumask_any_housekeeping (&dom->cpu_mask); dom->mbm_work_cpu = cpu;schedule_delayed_work_on (cpu, &dom->mbm_over, delay); }
Signature · 1 line
2.39.2
Re: [PATCH v7 12/24] x86/resctrl: Add cpumask_any_housekeeping() for limbo/overflow
Hi James,
Quoted 17 lines · James Morse
The limbo and overflow code picks a CPU to use from the domain's list of online CPUs. Work is then scheduled on these CPUs to maintain the limbo list and any counters that may overflow.
cpumask_any()may pick a CPU that is markednohz_full, which will either penalise the work that CPU was dedicated to, or delay the processing of limbo list or counters that may overflow. Perhaps indefinitely. Delaying the overflow handling will skew the bandwidth values calculated bymba_sc, which expects to be called once a second.Add
cpumask_any_housekeeping()as a replacement forcpumask_any()that prefers housekeeping CPUs. This helper will still return anohz_fullCPU if that is the only option. The CPU to use is re-evaluated each time the limbo/overflow work runs. This ensures the work will move off anohz_fullCPU once a housekeeping CPU is available.
Quoted 5 lines
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>---
Reinette
Re: [PATCH v7 12/24] x86/resctrl: Add cpumask_any_housekeeping() for limbo/overflow
In reply to James MorseQuoted 17 lines · James Morse
The limbo and overflow code picks a CPU to use from the domain's list of online CPUs. Work is then scheduled on these CPUs to maintain the limbo list and any counters that may overflow.
cpumask_any()may pick a CPU that is markednohz_full, which will either penalise the work that CPU was dedicated to, or delay the processing of limbo list or counters that may overflow. Perhaps indefinitely. Delaying the overflow handling will skew the bandwidth values calculated bymba_sc, which expects to be called once a second.Add
cpumask_any_housekeeping()as a replacement forcpumask_any()that prefers housekeeping CPUs. This helper will still return anohz_fullCPU if that is the only option. The CPU to use is re-evaluated each time the limbo/overflow work runs. This ensures the work will move off anohz_fullCPU once a housekeeping CPU is available.
Quoted 4 lines
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 125 lines
---
Changes since v3:
* typos fixedChanges since v4:
* Made temporary variables unsignedChanges since v5:
* Restructuredcpumask_any_housekeeping()to avoid later churn.Changes since v6:
* Update mbm_work_cpu/cqm_work_cpu when rescheduling.--- arch/x86/kernel/cpu/resctrl/internal.h | 24 ++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/monitor.c | 20 +++++++++++++------- 2 files changed, 37 insertions(+), 7 deletions(-)arch/x86/kernel/cpu/resctrl/internal.h +24
-0@@ -7,6 +7,7 @@#include <linux/kernfs.h> #include <linux/fs_context.h> #include <linux/jump_label.h> +#include <linux/tick.h> #include <asm/resctrl.h> @@ -56,6 +57,29 @@/* Max event bits supported */ #define MAX_EVT_CONFIG_BITS GENMASK(6, 0) +/** +* cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that +* aren't marked nohz_full +* @mask: The mask to pick a CPU from. +* +* Returns a CPU in @mask. If there are housekeeping CPUs that don't use +* nohz_full, these are preferred. +*/ +static inline unsigned int cpumask_any_housekeeping (const struct cpumask *mask) +{ +unsigned int cpu, hk_cpu; + + cpu =cpumask_any (mask); +if (!tick_nohz_full_cpu (cpu)) +return cpu; + + hk_cpu =cpumask_nth_andnot (0 , mask, tick_nohz_full_mask); +if (hk_cpu < nr_cpu_ids) + cpu = hk_cpu; + +return cpu; +} +struct rdt_fs_context {struct kernfs_fs_context kfc;bool enable_cdpl2;arch/x86/kernel/cpu/resctrl/monitor.c +13
-7@@ -783,7 +783,6 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,void cqm_handle_limbo (struct work_struct *work) {unsigned long delay =msecs_to_jiffies (CQM_LIMBOCHECK_INTERVAL ); -int cpu =smp_processor_id ();struct rdt_resource *r;struct rdt_domain *d;@@ -794,8 +793,11 @@ void cqm_handle_limbo(struct work_struct *work)__check_limbo (d, false); -if (has_busy_rmid (d)) -schedule_delayed_work_on (cpu, &d->cqm_limbo, delay); +if (has_busy_rmid (d)) { + d->cqm_work_cpu =cpumask_any_housekeeping (&d->cpu_mask); +schedule_delayed_work_on (d->cqm_work_cpu, &d->cqm_limbo, + delay); + }mutex_unlock (&rdtgroup_mutex); }@@ -805,7 +807,7 @@ void cqm_setup_limbo_handler(struct rdt_domain *dom, unsigned long delay_ms)unsigned long delay =msecs_to_jiffies (delay_ms);int cpu; - cpu =cpumask_any (&dom->cpu_mask); + cpu =cpumask_any_housekeeping (&dom->cpu_mask); dom->cqm_work_cpu = cpu;schedule_delayed_work_on (cpu, &dom->cqm_limbo, delay);@@ -815,7 +817,6 @@ void mbm_handle_overflow(struct work_struct *work) {unsigned long delay =msecs_to_jiffies (MBM_OVERFLOW_INTERVAL );struct rdtgroup *prgrp, *crgrp; -int cpu =smp_processor_id ();struct list_head *head;struct rdt_resource *r;struct rdt_domain *d;@@ -839,7 +840,12 @@ void mbm_handle_overflow(struct work_struct *work)update_mba_bw (prgrp, d); } -schedule_delayed_work_on (cpu, &d->mbm_over, delay); +/* +* Re-check for housekeeping CPUs. This allows the overflow handler to +* move off a nohz_full CPU quickly. +*/ + d->mbm_work_cpu =cpumask_any_housekeeping (&d->cpu_mask); +schedule_delayed_work_on (d->mbm_work_cpu, &d->mbm_over, delay); out_unlock:mutex_unlock (&rdtgroup_mutex);@@ -852,7 +858,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)if (!static_branch_likely (&rdt_mon_enable_key))return ; - cpu =cpumask_any (&dom->cpu_mask); + cpu =cpumask_any_housekeeping (&dom->cpu_mask); dom->mbm_work_cpu = cpu;schedule_delayed_work_on (cpu, &dom->mbm_over, delay); }
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerPATCH v7 13/24 x86/resctrl: Queue mon_event_read() instead of sending an IPI
In reply to James MorseIntel is blessed with an abundance of monitors, one per RMID, that can be read from any CPU in the domain. MPAMs monitors reside in the MMIO MSC, the number implemented is up to the manufacturer. This means when there are fewer monitors than needed, they need to be allocated and freed.
MPAM's CSU monitors are used to back the 'llc_occupancy' monitor file. The CSU counter is allowed to return 'not ready' for a small number of micro-seconds after programming. To allow one CSU hardware monitor to be used for multiple control or monitor groups, the CPU accessing the monitor needs to be able to block when configuring and reading the counter.
Worse, the domain may be broken up into slices, and the MMIO accesses for each slice may need performing from different CPUs.
These two details mean MPAMs monitor code needs to be able to sleep, and IPI another CPU in the domain to read from a resource that has been sliced.
mon_event_read() already invokes mon_event_count() via IPI, which means this isn't possible. On systems using nohz-full, some CPUs need to be interrupted to run kernel work as they otherwise stay in user-space running realtime workloads. Interrupting these CPUs should be avoided, and scheduling work on them may never complete.
Change mon_event_read() to pick a housekeeping CPU, (one that is not using nohz_full) and schedule mon_event_count() and wait. If all the CPUs in a domain are using nohz-full, then an IPI is used as the fallback.
This function is only used in response to a user-space filesystem request (not the timing sensitive overflow code).
This allows MPAM to hide the slice behaviour from resctrl, and to keep the monitor-allocation in monitor.c. When the IPI fallback is used on machines where MPAM needs to make an access on multiple CPUs, the counter read will always fail.
---
Changes since v2:
* Use cpumask_any_housekeeping() and fallback to an IPI if needed.
Changes since v3:
* Actually include the IPI fallback code.
Changes since v4:
* Tinkered with existing capitalisation.
Changes since v5:
* Added a newline.
Changes since v6:
* Moved lockdep annotations to a later patch.
--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 26 +++++++++++++++++++++-- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- 2 files changed, 25 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +24 -2
@@ -19,6 +19,8 @@#include <linux/kernfs.h> #include <linux/seq_file.h> #include <linux/slab.h> +#include <linux/tick.h> +#include "internal.h" /*
@@ -522,12 +524,21 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of,return ret; } +static int smp_mon_event_count (void *arg) +{ +mon_event_count (arg); + +return 0 ; +} +void mon_event_read (struct rmid_read *rr,struct rdt_resource *r,struct rdt_domain *d,struct rdtgroup *rdtgrp,int evtid,int first) { +int cpu; +/* -* setup the parameters to send to the IPI to read the data. +* Setup the parameters to pass to mon_event_count() to read the data. */ rr->rgrp = rdtgrp; rr->evtid = evtid;
@@ -536,7 +547,18 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, rr->val =0 ; rr->first = first; -smp_call_function_any (&d->cpu_mask, mon_event_count, rr,1 ); + cpu =cpumask_any_housekeeping (&d->cpu_mask); + +/* +* cpumask_any_housekeeping() prefers housekeeping CPUs, but +* are all the CPUs nohz_full? If yes, pick a CPU to IPI. +* MPAM's resctrl_arch_rmid_read() is unable to read the +* counters on some platforms if its called in irq context. +*/ +if (tick_nohz_full_cpu (cpu)) +smp_call_function_any (&d->cpu_mask, mon_event_count, rr,1 ); +else +smp_call_on_cpu (cpu, smp_mon_event_count, rr, false); }int rdtgroup_mondata_show (struct seq_file *m,void *arg)
arch/x86/kernel/cpu/resctrl/monitor.c +1 -1
@@ -588,7 +588,7 @@ static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) }/* -* This is called via IPI to read the CQM/MBM counters +* This is scheduled by mon_event_read() to read the CQM/MBM counters * on a domain. */ void mon_event_count (void *info)
Signature · 1 line
2.39.2
Re: [PATCH v7 13/24] x86/resctrl: Queue mon_event_read() instead of sending an IPI
Hi James,
Quoted 36 lines · James Morse
Intel is blessed with an abundance of monitors, one per RMID, that can be read from any CPU in the domain. MPAMs monitors reside in the MMIO MSC, the number implemented is up to the manufacturer. This means when there are fewer monitors than needed, they need to be allocated and freed.
MPAM's CSU monitors are used to back the '
llc_occupancy' monitor file. The CSU counter is allowed to return 'not ready' for a small number of micro-seconds after programming. To allow one CSU hardware monitor to be used for multiple control or monitor groups, the CPU accessing the monitor needs to be able to block when configuring and reading the counter.Worse, the domain may be broken up into slices, and the MMIO accesses for each slice may need performing from different CPUs.
These two details mean MPAMs monitor code needs to be able to sleep, and IPI another CPU in the domain to read from a resource that has been sliced.
mon_event_read()already invokesmon_event_count()via IPI, which means this isn't possible. On systems using nohz-full, some CPUs need to be interrupted to run kernel work as they otherwise stay in user-space running realtime workloads. Interrupting these CPUs should be avoided, and scheduling work on them may never complete.Change
mon_event_read()to pick a housekeeping CPU, (one that is not usingnohz_full) and schedulemon_event_count()and wait. If all the CPUs in a domain are using nohz-full, then an IPI is used as the fallback.This function is only used in response to a user-space filesystem request (not the timing sensitive overflow code).
This allows MPAM to hide the slice behaviour from resctrl, and to keep the monitor-allocation in
monitor.c. When the IPI fallback is used on machines where MPAM needs to make an access on multiple CPUs, the counter read will always fail.
Quoted 6 lines
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Peter Newman <[email protected]>Signed-off-by James Morse <[email protected]>---
Reinette
Moger, Babu
reviewerRe: [PATCH v7 13/24] x86/resctrl: Queue mon_event_read() instead of sending an IPI
In reply to James MorseHi James,
Quoted 75 lines · James Morse
Intel is blessed with an abundance of monitors, one per RMID, that can be read from any CPU in the domain. MPAMs monitors reside in the MMIO MSC, the number implemented is up to the manufacturer. This means when there are fewer monitors than needed, they need to be allocated and freed.
MPAM's CSU monitors are used to back the '
llc_occupancy' monitor file. The CSU counter is allowed to return 'not ready' for a small number of micro-seconds after programming. To allow one CSU hardware monitor to be used for multiple control or monitor groups, the CPU accessing the monitor needs to be able to block when configuring and reading the counter.Worse, the domain may be broken up into slices, and the MMIO accesses for each slice may need performing from different CPUs.
These two details mean MPAMs monitor code needs to be able to sleep, and IPI another CPU in the domain to read from a resource that has been sliced.
mon_event_read()already invokesmon_event_count()via IPI, which means this isn't possible. On systems using nohz-full, some CPUs need to be interrupted to run kernel work as they otherwise stay in user-space running realtime workloads. Interrupting these CPUs should be avoided, and scheduling work on them may never complete.Change
mon_event_read()to pick a housekeeping CPU, (one that is not usingnohz_full) and schedulemon_event_count()and wait. If all the CPUs in a domain are using nohz-full, then an IPI is used as the fallback.This function is only used in response to a user-space filesystem request (not the timing sensitive overflow code).
This allows MPAM to hide the slice behaviour from resctrl, and to keep the monitor-allocation in
monitor.c. When the IPI fallback is used on machines where MPAM needs to make an access on multiple CPUs, the counter read will always fail.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Peter Newman <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since v2:
* Usecpumask_any_housekeeping()and fallback to an IPI if needed.Changes since v3:
* Actually include the IPI fallback code.Changes since v4:
* Tinkered with existing capitalisation.Changes since v5:
* Added a newline.Changes since v6:
* Moved lockdep annotations to a later patch.--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 26 +++++++++++++++++++++-- arch/x86/kernel/cpu/resctrl/monitor.c | 2 +- 2 files changed, 25 insertions(+), 3 deletions(-)arch/x86/kernel/cpu/resctrl/ctrlmondata.c +2
-0@@ -19,6 +19,8 @@#include <linux/kernfs.h> #include <linux/seq_file.h> #include <linux/slab.h> +#include <linux/tick.h> +#include "internal.h" /*
arch/x86/kernel/cpu/resctrl/ctrlmondata.c int rdtgroup_schemata_show(struct kernfs_open_file *of, · line 532
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +6
-0@@ -522,12 +524,21 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of,return ret; } +static int smp_mon_event_count (void *arg) +{ +mon_event_count (arg); + +return 0 ; +}
Shouldn't this function defined as "void" similar to mon_event_count? Return code is not used anywhere.
Quoted 46 lines
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +16
-2+void mon_event_read (struct rmid_read *rr,struct rdt_resource *r,struct rdt_domain *d,struct rdtgroup *rdtgrp,int evtid,int first) { +int cpu; +/* -* setup the parameters to send to the IPI to read the data. +* Setup the parameters to pass to mon_event_count() to read the data. */ rr->rgrp = rdtgrp; rr->evtid = evtid;@@ -536,7 +547,18 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, rr->val =0 ; rr->first = first; -smp_call_function_any (&d->cpu_mask, mon_event_count, rr,1 ); + cpu =cpumask_any_housekeeping (&d->cpu_mask); + +/* +* cpumask_any_housekeeping() prefers housekeeping CPUs, but +* are all the CPUs nohz_full? If yes, pick a CPU to IPI. +* MPAM's resctrl_arch_rmid_read() is unable to read the +* counters on some platforms if its called in irq context. +*/ +if (tick_nohz_full_cpu (cpu)) +smp_call_function_any (&d->cpu_mask, mon_event_count, rr,1 ); +else +smp_call_on_cpu (cpu, smp_mon_event_count, rr, false); }int rdtgroup_mondata_show (struct seq_file *m,void *arg)arch/x86/kernel/cpu/resctrl/monitor.c +1
-1@@ -588,7 +588,7 @@ static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) }/* -* This is called via IPI to read the CQM/MBM counters +* This is scheduled by mon_event_read() to read the CQM/MBM counters * on a domain. */ void mon_event_count (void *info)
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 13/24] x86/resctrl: Queue mon_event_read() instead of sending an IPI
Hi Babu,
Quoted 46 lines · Babu
Quoted 45 lines · James Morse
Intel is blessed with an abundance of monitors, one per RMID, that can be read from any CPU in the domain. MPAMs monitors reside in the MMIO MSC, the number implemented is up to the manufacturer. This means when there are fewer monitors than needed, they need to be allocated and freed.
MPAM's CSU monitors are used to back the '
llc_occupancy' monitor file. The CSU counter is allowed to return 'not ready' for a small number of micro-seconds after programming. To allow one CSU hardware monitor to be used for multiple control or monitor groups, the CPU accessing the monitor needs to be able to block when configuring and reading the counter.Worse, the domain may be broken up into slices, and the MMIO accesses for each slice may need performing from different CPUs.
These two details mean MPAMs monitor code needs to be able to sleep, and IPI another CPU in the domain to read from a resource that has been sliced.
mon_event_read()already invokesmon_event_count()via IPI, which means this isn't possible. On systems using nohz-full, some CPUs need to be interrupted to run kernel work as they otherwise stay in user-space running realtime workloads. Interrupting these CPUs should be avoided, and scheduling work on them may never complete.Change
mon_event_read()to pick a housekeeping CPU, (one that is not usingnohz_full) and schedulemon_event_count()and wait. If all the CPUs in a domain are using nohz-full, then an IPI is used as the fallback.This function is only used in response to a user-space filesystem request (not the timing sensitive overflow code).
This allows MPAM to hide the slice behaviour from resctrl, and to keep the monitor-allocation in
monitor.c. When the IPI fallback is used on machines where MPAM needs to make an access on multiple CPUs, the counter read will always fail.arch/x86/kernel/cpu/resctrl/ctrlmondata.c +0
-0@@ -522,12 +524,21 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of,return ret; }
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/ctrlmondata.c int rdtgroup_schemata_show(struct kernfs_open_file *of, · line 532
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +6
-0+static int smp_mon_event_count (void *arg) +{ +mon_event_count (arg); + +return 0 ; +}Shouldn't this function defined as "void" similar to
mon_event_count? Return code is not used anywhere.
smp_call_on_cpu() requires it to return an int, even if the value is not used.
This wrapper only exists because smp_call_on_cpu() takes a different prototype to smp_call_function_any().
Thanks,
James
Quoted 19 lines
Quoted 19 lines
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +12
-1@@ -536,7 +547,18 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, rr->val =0 ; rr->first = first; -smp_call_function_any (&d->cpu_mask, mon_event_count, rr,1 ); + cpu =cpumask_any_housekeeping (&d->cpu_mask); + +/* +* cpumask_any_housekeeping() prefers housekeeping CPUs, but +* are all the CPUs nohz_full? If yes, pick a CPU to IPI. +* MPAM's resctrl_arch_rmid_read() is unable to read the +* counters on some platforms if its called in irq context. +*/ +if (tick_nohz_full_cpu (cpu)) +smp_call_function_any (&d->cpu_mask, mon_event_count, rr,1 ); +else +smp_call_on_cpu (cpu, smp_mon_event_count, rr, false); }
James Morse
reviewerPATCH v7 14/24 x86/resctrl: Allow resctrl_arch_rmid_read() to sleep
In reply to James MorseMPAM's cache occupancy counters can take a little while to settle once the monitor has been configured. The maximum settling time is described to the driver via a firmware table. The value could be large enough that it makes sense to sleep. To avoid exposing this to resctrl, it should be hidden behind MPAM's resctrl_arch_rmid_read().
resctrl_arch_rmid_read() may be called via IPI meaning it is unable to sleep. In this case resctrl_arch_rmid_read() should return an error if it needs to sleep. This will only affect MPAM platforms where the cache occupancy counter isn't available immediately, nohz_full is in use, and there are no housekeeping CPUs in the necessary domain.
There are three callers of resctrl_arch_rmid_read():__mon_event_count() and __check_limbo() are both called from a non-migrateable context. mon_event_read() invokes __mon_event_count() using smp_call_on_cpu(), which adds work to the target CPUs workqueue. rdtgroup_mutex() is held, meaning this cannot race with the resctrl cpuhp callback. __check_limbo() is invoked via schedule_delayed_work_on() also adds work to a per-cpu workqueue.
The remaining call is add_rmid_to_limbo() which is called in response to a user-space syscall that frees an RMID. This opportunistically reads the LLC occupancy counter on the current domain to see if the RMID is over the dirty threshold. This has to disable preemption to avoid reading the wrong domain's value. Disabling pre-emption here prevents resctrl_arch_rmid_read() from sleeping.
add_rmid_to_limbo() walks each domain, but only reads the counter on one domain. If the system has more than one domain, the RMID will always be added to the limbo list. If the RMIDs usage was not over the threshold, it will be removed from the list when __check_limbo() runs. Make this the default behaviour. Free RMIDs are always added to the limbo list for each domain.
The user visible effect of this is that a clean RMID is not available for re-allocation immediately after 'rmdir()' completes, this behaviour was never portable as it never happened on a machine with multiple domains.
Removing this path allows resctrl_arch_rmid_read() to sleep if its called with interrupts unmasked. Document this is the expected behaviour, and add a might_sleep() annotation to catch changes that won't work on arm64.
---
The previous version allowed resctrl_arch_rmid_read() to be called on the wrong CPUs, but now that this needs to take nohz_full and housekeeping into account, its too complex.
Changes since v3:
* Removed error handling for smp_call_function_any(), this can't race with the cpuhp callbacks as both hold rdtgroup_mutex.
* Switched to the alternative of removing the counter read, this simplifies things dramatically.
Changes since v4:
* Messed with capitalisation.
* Removed some dead code now that entry->busy will never be zero in add_rmid_to_limbo().
* Rephrased the comment above resctrl_arch_rmid_read_context_check().
Changes since v5:
* Really rephrased the comment above resctrl_arch_rmid_read_context_check().
No changes since v6
--- arch/x86/kernel/cpu/resctrl/monitor.c | 25 +++++-------------------- include/linux/resctrl.h | 23 ++++++++++++++++++++++- 2 files changed, 27 insertions(+), 21 deletions(-)
arch/x86/kernel/cpu/resctrl/monitor.c +5 -20
@@ -277,6 +277,8 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,u64 msr_val, chunks;int ret; +resctrl_arch_rmid_read_context_check (); +if (!cpumask_test_cpu (smp_processor_id (), &d->cpu_mask))return -EINVAL ;
@@ -455,8 +457,6 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rdt_domain *d; -int cpu, err; -u64 val =0 ;u32 idx;lockdep_assert_held (&rdtgroup_mutex);
@@ -464,17 +464,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); entry->busy =0 ; - cpu =get_cpu ();list_for_each_entry (d, &r->domains, list) { -if (cpumask_test_cpu (cpu, &d->cpu_mask)) { - err =resctrl_arch_rmid_read (r, d, entry->closid, - entry->rmid, -QOS_L3_OCCUP_EVENT_ID , - &val); -if (err || val <= resctrl_rmid_realloc_threshold) -continue ; - } -/* * For the first limbo RMID in the domain, * setup up the limbo worker.
@@ -484,15 +474,10 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)set_bit (idx, d->rmid_busy_llc); entry->busy++; } -put_cpu (); -if (entry->busy) { - rmid_limbo_count++; -if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) - closid_num_dirty_rmid[entry->closid]++; - }else { -list_add_tail (&entry->list, &rmid_free_lru); - } + rmid_limbo_count++; +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) + closid_num_dirty_rmid[entry->closid]++; }void free_rmid (u32 closid,u32 rmid)
include/linux/resctrl.h +22 -1
@@ -236,7 +236,12 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);* @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes. * -* Call from process context on a CPU that belongs to domain @d. +* Some architectures need to sleep when first programming some of the counters. +* (specifically: arm64's MPAM cache occupancy counters can return 'not ready' +* for a short period of time). Call from a non-migrateable process context on +* a CPU that belongs to domain @d. e.g. use smp_call_on_cpu() or +* schedule_work_on(). This function can be called with interrupts masked, +* e.g. using smp_call_function_any(), but may consistently return an error. * * Return: * 0 on success, or -EIO, -EINVAL etc on error.
@@ -245,6 +250,22 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, u32 closid, u32 rmid, enum resctrl_event_id eventid, u64 *val); +/** +* resctrl_arch_rmid_read_context_check() - warn about invalid contexts +* +* When built with CONFIG_DEBUG_ATOMIC_SLEEP generate a warning when +* resctrl_arch_rmid_read() is called with preemption disabled. +* +* The contract with resctrl_arch_rmid_read() is that if interrupts +* are unmasked, it can sleep. This allows NOHZ_FULL systems to use an +* IPI, (and fail if the call needed to sleep), while most of the time +* the work is scheduled, allowing the call to sleep. +*/ +static inline void resctrl_arch_rmid_read_context_check (void ) +{ +if (!irqs_disabled ()) +might_sleep (); +}/** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid
Signature · 1 line
2.39.2
Re: [PATCH v7 14/24] x86/resctrl: Allow resctrl_arch_rmid_read() to sleep
Hi James,
Quoted 64 lines · James Morse
MPAM's cache occupancy counters can take a little while to settle once the monitor has been configured. The maximum settling time is described to the driver via a firmware table. The value could be large enough that it makes sense to sleep. To avoid exposing this to resctrl, it should be hidden behind MPAM's
resctrl_arch_rmid_read().
resctrl_arch_rmid_read()may be called via IPI meaning it is unable to sleep. In this caseresctrl_arch_rmid_read()should return an error if it needs to sleep. This will only affect MPAM platforms where the cache occupancy counter isn't available immediately,nohz_fullis in use, and there are no housekeeping CPUs in the necessary domain.There are three callers of
resctrl_arch_rmid_read():__mon_event_count()and__check_limbo()are both called from a non-migrateable context.mon_event_read()invokes__mon_event_count()usingsmp_call_on_cpu(), which adds work to the target CPUs workqueue.rdtgroup_mutex()is held, meaning this cannot race with the resctrl cpuhp callback.__check_limbo()is invoked viaschedule_delayed_work_on()also adds work to a per-cpu workqueue.The remaining call is
add_rmid_to_limbo()which is called in response to a user-space syscall that frees an RMID. This opportunistically reads the LLC occupancy counter on the current domain to see if the RMID is over the dirty threshold. This has to disable preemption to avoid reading the wrong domain's value. Disabling pre-emption here preventsresctrl_arch_rmid_read()from sleeping.
add_rmid_to_limbo()walks each domain, but only reads the counter on one domain. If the system has more than one domain, the RMID will always be added to the limbo list. If the RMIDs usage was not over the threshold, it will be removed from the list when__check_limbo()runs. Make this the default behaviour. Free RMIDs are always added to the limbo list for each domain.The user visible effect of this is that a clean RMID is not available for re-allocation immediately after '
rmdir()' completes, this behaviour was never portable as it never happened on a machine with multiple domains.Removing this path allows
resctrl_arch_rmid_read()to sleep if its called with interrupts unmasked. Document this is the expected behaviour, and add amight_sleep()annotation to catch changes that won't work on arm64.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>---
The previous version allowedresctrl_arch_rmid_read()to be called on the wrong CPUs, but now that this needs to takenohz_fulland housekeeping into account, its too complex.Changes since v3:
* Removed error handling forsmp_call_function_any(), this can't race with the cpuhp callbacks as both holdrdtgroup_mutex.
* Switched to the alternative of removing the counter read, this simplifies things dramatically.Changes since v4:
* Messed with capitalisation.
* Removed some dead code now thatentry->busywill never be zero inadd_rmid_to_limbo().
* Rephrased the comment aboveresctrl_arch_rmid_read_context_check().
Quoted 4 lines
Changes since v5:
* Really rephrased the comment aboveresctrl_arch_rmid_read_context_check().No changes since v6
If I trusted this I would not have taken the time to review this patch.
Quoted 1 line
---
Reinette
James Morse
reviewerRe: [PATCH v7 14/24] x86/resctrl: Allow resctrl_arch_rmid_read() to sleep
Hi Reinette,
Quoted 43 lines · Reinette Chatre
Quoted 42 lines · James Morse
MPAM's cache occupancy counters can take a little while to settle once the monitor has been configured. The maximum settling time is described to the driver via a firmware table. The value could be large enough that it makes sense to sleep. To avoid exposing this to resctrl, it should be hidden behind MPAM's
resctrl_arch_rmid_read().
resctrl_arch_rmid_read()may be called via IPI meaning it is unable to sleep. In this caseresctrl_arch_rmid_read()should return an error if it needs to sleep. This will only affect MPAM platforms where the cache occupancy counter isn't available immediately,nohz_fullis in use, and there are no housekeeping CPUs in the necessary domain.There are three callers of
resctrl_arch_rmid_read():__mon_event_count()and__check_limbo()are both called from a non-migrateable context.mon_event_read()invokes__mon_event_count()usingsmp_call_on_cpu(), which adds work to the target CPUs workqueue.rdtgroup_mutex()is held, meaning this cannot race with the resctrl cpuhp callback.__check_limbo()is invoked viaschedule_delayed_work_on()also adds work to a per-cpu workqueue.The remaining call is
add_rmid_to_limbo()which is called in response to a user-space syscall that frees an RMID. This opportunistically reads the LLC occupancy counter on the current domain to see if the RMID is over the dirty threshold. This has to disable preemption to avoid reading the wrong domain's value. Disabling pre-emption here preventsresctrl_arch_rmid_read()from sleeping.
add_rmid_to_limbo()walks each domain, but only reads the counter on one domain. If the system has more than one domain, the RMID will always be added to the limbo list. If the RMIDs usage was not over the threshold, it will be removed from the list when__check_limbo()runs. Make this the default behaviour. Free RMIDs are always added to the limbo list for each domain.The user visible effect of this is that a clean RMID is not available for re-allocation immediately after '
rmdir()' completes, this behaviour was never portable as it never happened on a machine with multiple domains.Removing this path allows
resctrl_arch_rmid_read()to sleep if its called with interrupts unmasked. Document this is the expected behaviour, and add amight_sleep()annotation to catch changes that won't work on arm64.
[...]
Quoted 12 lines
Quoted 12 lines
Changes since v3:
* Removed error handling forsmp_call_function_any(), this can't race with the cpuhp callbacks as both holdrdtgroup_mutex.
* Switched to the alternative of removing the counter read, this simplifies things dramatically.Changes since v4:
* Messed with capitalisation.
* Removed some dead code now thatentry->busywill never be zero inadd_rmid_to_limbo().
* Rephrased the comment aboveresctrl_arch_rmid_read_context_check().
Quoted 6 lines
Quoted 4 lines
Changes since v5:
* Really rephrased the comment aboveresctrl_arch_rmid_read_context_check().No changes since v6
If I trusted this I would not have taken the time to review this patch.
I'm not sure what you want me to do from this comment ... but this 'no changes' annotation doesn't work for either of us, so I'll remove them.
Quoted 1 line
Reviewed-by Reinette Chatre <[email protected]>
Thanks,
James
Reinette Chatre
maintainerRe: [PATCH v7 14/24] x86/resctrl: Allow resctrl_arch_rmid_read() to sleep
Hi James,
Quoted 11 lines · James Morse
Quoted 6 lines · Reinette Chatre
Quoted 2 lines · James Morse
No changes since v6
If I trusted this I would not have taken the time to review this patch.
I'm not sure what you want me to do from this comment ... but this 'no changes' annotation doesn't work for either of us, so I'll remove them.
The "no changes" annotation is valuable when it is accurate. When the patch explicitly claims to have no changes since previous version when in fact there are changes then it complicates the review process.
Reinette
Moger, Babu
reviewerRe: [PATCH v7 14/24] x86/resctrl: Allow resctrl_arch_rmid_read() to sleep
In reply to James MorseHi James,
Quoted 20 lines · James Morse
MPAM's cache occupancy counters can take a little while to settle once the monitor has been configured. The maximum settling time is described to the driver via a firmware table. The value could be large enough that it makes sense to sleep. To avoid exposing this to resctrl, it should be hidden behind MPAM's
resctrl_arch_rmid_read().
resctrl_arch_rmid_read()may be called via IPI meaning it is unable to sleep. In this caseresctrl_arch_rmid_read()should return an error if it needs to sleep. This will only affect MPAM platforms where the cache occupancy counter isn't available immediately,nohz_fullis in use, and there are no housekeeping CPUs in the necessary domain.There are three callers of
resctrl_arch_rmid_read():__mon_event_count()and__check_limbo()are both called from a non-migrateable context.mon_event_read()invokes__mon_event_count()usingsmp_call_on_cpu(), which adds work to the target CPUs workqueue.rdtgroup_mutex()is held, meaning this cannot race with the resctrl cpuhp callback.__check_limbo()is invoked viaschedule_delayed_work_on()also adds work to a per-cpu workqueue.
Quoted 6 lines
The remaining call is
add_rmid_to_limbo()which is called in response to a user-space syscall that frees an RMID. This opportunistically reads the LLC occupancy counter on the current domain to see if the RMID is over the dirty threshold. This has to disable preemption to avoid reading the wrong domain's value. Disabling pre-emption here preventsresctrl_arch_rmid_read()from sleeping.
I dont know what did you mean by "This has to disable preemption to avoid reading the wrong domain's value."
Who is disabling the preemption here? Is that specific to ARM? Can you please make that clear? Or Am i missing something?
Thanks
Babu
Quoted 147 lines
add_rmid_to_limbo()walks each domain, but only reads the counter on one domain. If the system has more than one domain, the RMID will always be added to the limbo list. If the RMIDs usage was not over the threshold, it will be removed from the list when__check_limbo()runs. Make this the default behaviour. Free RMIDs are always added to the limbo list for each domain.The user visible effect of this is that a clean RMID is not available for re-allocation immediately after '
rmdir()' completes, this behaviour was never portable as it never happened on a machine with multiple domains.Removing this path allows
resctrl_arch_rmid_read()to sleep if its called with interrupts unmasked. Document this is the expected behaviour, and add amight_sleep()annotation to catch changes that won't work on arm64.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>---
The previous version allowedresctrl_arch_rmid_read()to be called on the wrong CPUs, but now that this needs to takenohz_fulland housekeeping into account, its too complex.Changes since v3:
* Removed error handling forsmp_call_function_any(), this can't race with the cpuhp callbacks as both holdrdtgroup_mutex.
* Switched to the alternative of removing the counter read, this simplifies things dramatically.Changes since v4:
* Messed with capitalisation.
* Removed some dead code now thatentry->busywill never be zero inadd_rmid_to_limbo().
* Rephrased the comment aboveresctrl_arch_rmid_read_context_check().Changes since v5:
* Really rephrased the comment aboveresctrl_arch_rmid_read_context_check().No changes since v6
--- arch/x86/kernel/cpu/resctrl/monitor.c | 25 +++++-------------------- include/linux/resctrl.h | 23 ++++++++++++++++++++++- 2 files changed, 27 insertions(+), 21 deletions(-)arch/x86/kernel/cpu/resctrl/monitor.c +5
-20@@ -277,6 +277,8 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,u64 msr_val, chunks;int ret; +resctrl_arch_rmid_read_context_check (); +if (!cpumask_test_cpu (smp_processor_id (), &d->cpu_mask))return -EINVAL ;@@ -455,8 +457,6 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rdt_domain *d; -int cpu, err; -u64 val =0 ;u32 idx;lockdep_assert_held (&rdtgroup_mutex);@@ -464,17 +464,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); entry->busy =0 ; - cpu =get_cpu ();list_for_each_entry (d, &r->domains, list) { -if (cpumask_test_cpu (cpu, &d->cpu_mask)) { - err =resctrl_arch_rmid_read (r, d, entry->closid, - entry->rmid, -QOS_L3_OCCUP_EVENT_ID , - &val); -if (err || val <= resctrl_rmid_realloc_threshold) -continue ; - } -/* * For the first limbo RMID in the domain, * setup up the limbo worker. @@ -484,15 +474,10 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)set_bit (idx, d->rmid_busy_llc); entry->busy++; } -put_cpu (); -if (entry->busy) { - rmid_limbo_count++; -if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) - closid_num_dirty_rmid[entry->closid]++; - }else { -list_add_tail (&entry->list, &rmid_free_lru); - } + rmid_limbo_count++; +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) + closid_num_dirty_rmid[entry->closid]++; }void free_rmid (u32 closid,u32 rmid)include/linux/resctrl.h +22
-1@@ -236,7 +236,12 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);* @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes. * -* Call from process context on a CPU that belongs to domain @d. +* Some architectures need to sleep when first programming some of the counters. +* (specifically: arm64's MPAM cache occupancy counters can return 'not ready' +* for a short period of time). Call from a non-migrateable process context on +* a CPU that belongs to domain @d. e.g. use smp_call_on_cpu() or +* schedule_work_on(). This function can be called with interrupts masked, +* e.g. using smp_call_function_any(), but may consistently return an error. * * Return: * 0 on success, or -EIO, -EINVAL etc on error. @@ -245,6 +250,22 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, u32 closid, u32 rmid, enum resctrl_event_id eventid, u64 *val); +/** +* resctrl_arch_rmid_read_context_check() - warn about invalid contexts +* +* When built with CONFIG_DEBUG_ATOMIC_SLEEP generate a warning when +* resctrl_arch_rmid_read() is called with preemption disabled. +* +* The contract with resctrl_arch_rmid_read() is that if interrupts +* are unmasked, it can sleep. This allows NOHZ_FULL systems to use an +* IPI, (and fail if the call needed to sleep), while most of the time +* the work is scheduled, allowing the call to sleep. +*/ +static inline void resctrl_arch_rmid_read_context_check (void ) +{ +if (!irqs_disabled ()) +might_sleep (); +}/** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 14/24] x86/resctrl: Allow resctrl_arch_rmid_read() to sleep
Hi Babu,
Quoted 28 lines · Babu
Quoted 20 lines · James Morse
MPAM's cache occupancy counters can take a little while to settle once the monitor has been configured. The maximum settling time is described to the driver via a firmware table. The value could be large enough that it makes sense to sleep. To avoid exposing this to resctrl, it should be hidden behind MPAM's
resctrl_arch_rmid_read().
resctrl_arch_rmid_read()may be called via IPI meaning it is unable to sleep. In this caseresctrl_arch_rmid_read()should return an error if it needs to sleep. This will only affect MPAM platforms where the cache occupancy counter isn't available immediately,nohz_fullis in use, and there are no housekeeping CPUs in the necessary domain.There are three callers of
resctrl_arch_rmid_read():__mon_event_count()and__check_limbo()are both called from a non-migrateable context.mon_event_read()invokes__mon_event_count()usingsmp_call_on_cpu(), which adds work to the target CPUs workqueue.rdtgroup_mutex()is held, meaning this cannot race with the resctrl cpuhp callback.__check_limbo()is invoked viaschedule_delayed_work_on()also adds work to a per-cpu workqueue.Quoted 6 lines
The remaining call is
add_rmid_to_limbo()which is called in response to a user-space syscall that frees an RMID. This opportunistically reads the LLC occupancy counter on the current domain to see if the RMID is over the dirty threshold. This has to disable preemption to avoid reading the wrong domain's value. Disabling pre-emption here preventsresctrl_arch_rmid_read()from sleeping.
Quoted 2 lines
I dont know what did you mean by "This has to disable preemption to avoid reading the wrong domain's value."
Pre-emption lets this thread be scheduled out, and potentially scheduled back in on a different CPU, possibly in a different domain. Any code with the concept of 'this domain' has to to ensure it can't be migrated. Disabling pre-emption is the most common way of doing that.
Disabling pre-emption also prevents the thread from sleeping, because it can't be scheduled out.
Quoted 2 lines
Who is disabling the preemption here? Is that specific to ARM? Can you please make that clear? Or Am i missing something?
add_rmid_to_limbo() is calling get_cpu(), which raises the pre-empt counter. If it only wanted the CPU number it could have just called smp_processor_id() - but that wouldn't be safe because the thread can be migrated, meaning the cpu number can change.
All this is to ensure that cpumask_test_cpu() and resctrl_arch_rmid_read() run on the same CPU.
Thanks,
James
Quoted 57 lines
Quoted 57 lines
add_rmid_to_limbo()walks each domain, but only reads the counter on one domain. If the system has more than one domain, the RMID will always be added to the limbo list. If the RMIDs usage was not over the threshold, it will be removed from the list when__check_limbo()runs. Make this the default behaviour. Free RMIDs are always added to the limbo list for each domain.The user visible effect of this is that a clean RMID is not available for re-allocation immediately after '
rmdir()' completes, this behaviour was never portable as it never happened on a machine with multiple domains.Removing this path allows
resctrl_arch_rmid_read()to sleep if its called with interrupts unmasked. Document this is the expected behaviour, and add amight_sleep()annotation to catch changes that won't work on arm64.arch/x86/kernel/cpu/resctrl/monitor.c +3
-18@@ -464,17 +464,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); entry->busy =0 ; - cpu =get_cpu ();list_for_each_entry (d, &r->domains, list) { -if (cpumask_test_cpu (cpu, &d->cpu_mask)) { - err =resctrl_arch_rmid_read (r, d, entry->closid, - entry->rmid, -QOS_L3_OCCUP_EVENT_ID , - &val); -if (err || val <= resctrl_rmid_realloc_threshold) -continue ; - } -/* * For the first limbo RMID in the domain, * setup up the limbo worker. @@ -484,15 +474,10 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)set_bit (idx, d->rmid_busy_llc); entry->busy++; } -put_cpu (); -if (entry->busy) { - rmid_limbo_count++; -if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) - closid_num_dirty_rmid[entry->closid]++; - }else { -list_add_tail (&entry->list, &rmid_free_lru); - } + rmid_limbo_count++; +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) + closid_num_dirty_rmid[entry->closid]++; }
James Morse
reviewerPATCH v7 15/24 x86/resctrl: Allow arch to allocate memory needed in resctrl_arch_rmid_read()
In reply to James MorseDepending on the number of monitors available, Arm's MPAM may need to allocate a monitor prior to reading the counter value. Allocating a contended resource may involve sleeping.
add_rmid_to_limbo() calls resctrl_arch_rmid_read() for multiple domains, the allocation should be valid for all domains.
__check_limbo() and mon_event_count() each make multiple calls to resctrl_arch_rmid_read(), to avoid extra work on contended systems, the allocation should be valid for multiple invocations of resctrl_arch_rmid_read().
Add arch hooks for this allocation, which need calling before resctrl_arch_rmid_read(). The allocated monitor is passed to resctrl_arch_rmid_read(), then freed again afterwards. The helper can be called on any CPU, and can sleep.
---
Changes since v3:
* Expanded comment.
* Removed stray header include.
* Reworded commit message.
* Made ctx a void * instead of an int.
Changes since v4:
* Used IS_ERR() in more places.
Changes since v5:
* Pass the error back from mon_event_read() as -EINVAL/Unavailable.
* Add some ratelimited warnings when failing to allocate a mon context
Changes since v6:
* Added a newline.
--- arch/x86/include/asm/resctrl.h | 11 +++++++ arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 7 +++++ arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 35 +++++++++++++++++++++-- include/linux/resctrl.h | 5 +++- 5 files changed, 55 insertions(+), 4 deletions(-)
arch/x86/include/asm/resctrl.h +11 -0
@@ -136,6 +136,17 @@ static inline u32 resctrl_arch_rmid_idx_encode(u32 ignored, u32 rmid) return rmid; } +/* x86 can always read an rmid, nothing needs allocating */ +struct rdt_resource ; +static inline void *resctrl_arch_mon_ctx_alloc (struct rdt_resource *r,int evtid) +{ +might_sleep (); +return NULL ; +}; + +static inline void resctrl_arch_mon_ctx_free (struct rdt_resource *r,int evtid, +void *ctx) { }; +void resctrl_cpu_detect (struct cpuinfo_x86 *c);#else
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +7 -0
@@ -546,6 +546,11 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, rr->d = d; rr->val =0 ; rr->first = first; + rr->arch_mon_ctx =resctrl_arch_mon_ctx_alloc (r, evtid); +if (IS_ERR (rr->arch_mon_ctx)) { + rr->err = -EINVAL ; +return ; + } cpu =cpumask_any_housekeeping (&d->cpu_mask);
@@ -559,6 +564,8 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,smp_call_function_any (&d->cpu_mask, mon_event_count, rr,1 ); elsesmp_call_on_cpu (cpu ,smp_mon_event_count ,rr ,false ); + +resctrl_arch_mon_ctx_free (r, evtid, rr->arch_mon_ctx); }int rdtgroup_mondata_show (struct seq_file *m,void *arg)
arch/x86/kernel/cpu/resctrl/internal.h +1 -0
@@ -138,6 +138,7 @@ struct rmid_read {bool first;int err;u64 val; +void *arch_mon_ctx; };extern bool rdt_alloc_capable;
arch/x86/kernel/cpu/resctrl/monitor.c +32 -3
@@ -269,7 +269,7 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d,u32 unused,u32 rmid,enum resctrl_event_id eventid, -u64 *val) +u64 *val,void *ignored) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r);struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);
@@ -324,9 +324,17 @@ void __check_limbo(struct rdt_domain *d, bool force_free)u32 idx_limit =resctrl_arch_system_num_rmid_idx ();struct rmid_entry *entry;u32 idx, cur_idx =1 ; +void *arch_mon_ctx;bool rmid_dirty;u64 val =0 ; + arch_mon_ctx =resctrl_arch_mon_ctx_alloc (r,QOS_L3_OCCUP_EVENT_ID ); +if (IS_ERR (arch_mon_ctx)) { +pr_warn_ratelimited ("Failed to allocate monitor context: %ld" , +PTR_ERR (arch_mon_ctx)); +return ; + } +/* * Skip RMID 0 and start from RMID 1 and check all the RMIDs that * are marked as busy for occupancy < threshold. If the occupancy
@@ -340,7 +348,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free) entry =__rmid_entry (idx);if (resctrl_arch_rmid_read (r, d, entry->closid, entry->rmid, -QOS_L3_OCCUP_EVENT_ID , &val)) { +QOS_L3_OCCUP_EVENT_ID , &val, + arch_mon_ctx)) { rmid_dirty = true; }else { rmid_dirty = (val >= resctrl_rmid_realloc_threshold);
@@ -353,6 +362,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } cur_idx = idx +1 ; } + +resctrl_arch_mon_ctx_free (r,QOS_L3_OCCUP_EVENT_ID , arch_mon_ctx); }bool has_busy_rmid (struct rdt_domain *d)
@@ -533,7 +544,7 @@ static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr) } rr->err =resctrl_arch_rmid_read (rr->r, rr->d, closid, rmid, rr->evtid, - &tval); + &tval, rr->arch_mon_ctx);if (rr->err)return rr->err;
@@ -744,11 +755,27 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,if (is_mbm_total_enabled ()) { rr.evtid =QOS_L3_MBM_TOTAL_EVENT_ID ; rr.val =0 ; + rr.arch_mon_ctx =resctrl_arch_mon_ctx_alloc (rr.r, rr.evtid); +if (IS_ERR (rr.arch_mon_ctx)) { +pr_warn_ratelimited ("Failed to allocate monitor context: %ld" , +PTR_ERR (rr.arch_mon_ctx)); +return ; + } +__mon_event_count (closid, rmid, &rr); + +resctrl_arch_mon_ctx_free (rr.r, rr.evtid, rr.arch_mon_ctx); }if (is_mbm_local_enabled ()) { rr.evtid =QOS_L3_MBM_LOCAL_EVENT_ID ; rr.val =0 ; + rr.arch_mon_ctx =resctrl_arch_mon_ctx_alloc (rr.r, rr.evtid); +if (IS_ERR (rr.arch_mon_ctx)) { +pr_warn_ratelimited ("Failed to allocate monitor context: %ld" , +PTR_ERR (rr.arch_mon_ctx)); +return ; + } +__mon_event_count (closid, rmid, &rr);/*
@@ -758,6 +785,8 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,*/ if (is_mba_sc (NULL ))mbm_bw_count (closid, rmid, &rr); + +resctrl_arch_mon_ctx_free (rr.r, rr.evtid, rr.arch_mon_ctx); } }
include/linux/resctrl.h +4 -1
@@ -235,6 +235,9 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);* @rmid: rmid of the counter to read. * @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes. +* @arch_mon_ctx: An architecture specific value from +* resctrl_arch_mon_ctx_alloc(), for MPAM this identifies +* the hardware monitor allocated for this read request. * * Some architectures need to sleep when first programming some of the counters. * (specifically: arm64's MPAM cache occupancy counters can return 'not ready'
@@ -248,7 +251,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);*/ int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d,u32 closid,u32 rmid,enum resctrl_event_id eventid, -u64 *val); +u64 *val,void *arch_mon_ctx);/** * resctrl_arch_rmid_read_context_check() - warn about invalid contexts
Signature · 1 line
2.39.2
Re: [PATCH v7 15/24] x86/resctrl: Allow arch to allocate memory needed in resctrl_arch_rmid_read()
Hi James,
Quoted 6 lines · James Morse
Depending on the number of monitors available, Arm's MPAM may need to allocate a monitor prior to reading the counter value. Allocating a contended resource may involve sleeping.
add_rmid_to_limbo()callsresctrl_arch_rmid_read()for multiple domains, the allocation should be valid for all domains.
This above sentence does not look correct after your previous patch [patch 14]. You removed resctrl_arch_rmid_read() from add_rmid_to_limbo().
Otherwise looks good.
Quoted 11 lines
__check_limbo()andmon_event_count()each make multiple calls toresctrl_arch_rmid_read(), to avoid extra work on contended systems, the allocation should be valid for multiple invocations ofresctrl_arch_rmid_read().Add arch hooks for this allocation, which need calling before
resctrl_arch_rmid_read(). The allocated monitor is passed toresctrl_arch_rmid_read(), then freed again afterwards. The helper can be called on any CPU, and can sleep.
Quoted 5 lines
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 202 lines
---
Changes since v3:
* Expanded comment.
* Removed stray header include.
* Reworded commit message.
* Made ctx a void * instead of an int.Changes since v4:
* UsedIS_ERR()in more places.Changes since v5:
* Pass the error back frommon_event_read()as -EINVAL/Unavailable.
* Add some ratelimited warnings when failing to allocate a mon contextChanges since v6:
* Added a newline.--- arch/x86/include/asm/resctrl.h | 11 +++++++ arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 7 +++++ arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 35 +++++++++++++++++++++-- include/linux/resctrl.h | 5 +++- 5 files changed, 55 insertions(+), 4 deletions(-)arch/x86/include/asm/resctrl.h +11
-0@@ -136,6 +136,17 @@ static inline u32 resctrl_arch_rmid_idx_encode(u32 ignored, u32 rmid) return rmid; } +/* x86 can always read an rmid, nothing needs allocating */ +struct rdt_resource ; +static inline void *resctrl_arch_mon_ctx_alloc (struct rdt_resource *r,int evtid) +{ +might_sleep (); +return NULL ; +}; + +static inline void resctrl_arch_mon_ctx_free (struct rdt_resource *r,int evtid, +void *ctx) { }; +void resctrl_cpu_detect (struct cpuinfo_x86 *c);#else arch/x86/kernel/cpu/resctrl/ctrlmondata.c +7
-0@@ -546,6 +546,11 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, rr->d = d; rr->val =0 ; rr->first = first; + rr->arch_mon_ctx =resctrl_arch_mon_ctx_alloc (r, evtid); +if (IS_ERR (rr->arch_mon_ctx)) { + rr->err = -EINVAL ; +return ; + } cpu =cpumask_any_housekeeping (&d->cpu_mask);@@ -559,6 +564,8 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,smp_call_function_any (&d->cpu_mask, mon_event_count, rr,1 ); elsesmp_call_on_cpu (cpu ,smp_mon_event_count ,rr ,false ); + +resctrl_arch_mon_ctx_free (r, evtid, rr->arch_mon_ctx); }int rdtgroup_mondata_show (struct seq_file *m,void *arg)arch/x86/kernel/cpu/resctrl/internal.h +1
-0@@ -138,6 +138,7 @@ struct rmid_read {bool first;int err;u64 val; +void *arch_mon_ctx; };extern bool rdt_alloc_capable;arch/x86/kernel/cpu/resctrl/monitor.c +32
-3@@ -269,7 +269,7 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d,u32 unused,u32 rmid,enum resctrl_event_id eventid, -u64 *val) +u64 *val,void *ignored) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r);struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);@@ -324,9 +324,17 @@ void __check_limbo(struct rdt_domain *d, bool force_free)u32 idx_limit =resctrl_arch_system_num_rmid_idx ();struct rmid_entry *entry;u32 idx, cur_idx =1 ; +void *arch_mon_ctx;bool rmid_dirty;u64 val =0 ; + arch_mon_ctx =resctrl_arch_mon_ctx_alloc (r,QOS_L3_OCCUP_EVENT_ID ); +if (IS_ERR (arch_mon_ctx)) { +pr_warn_ratelimited ("Failed to allocate monitor context: %ld" , +PTR_ERR (arch_mon_ctx)); +return ; + } +/* * Skip RMID 0 and start from RMID 1 and check all the RMIDs that * are marked as busy for occupancy < threshold. If the occupancy @@ -340,7 +348,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free) entry =__rmid_entry (idx);if (resctrl_arch_rmid_read (r, d, entry->closid, entry->rmid, -QOS_L3_OCCUP_EVENT_ID , &val)) { +QOS_L3_OCCUP_EVENT_ID , &val, + arch_mon_ctx)) { rmid_dirty = true; }else { rmid_dirty = (val >= resctrl_rmid_realloc_threshold);@@ -353,6 +362,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } cur_idx = idx +1 ; } + +resctrl_arch_mon_ctx_free (r,QOS_L3_OCCUP_EVENT_ID , arch_mon_ctx); }bool has_busy_rmid (struct rdt_domain *d)@@ -533,7 +544,7 @@ static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr) } rr->err =resctrl_arch_rmid_read (rr->r, rr->d, closid, rmid, rr->evtid, - &tval); + &tval, rr->arch_mon_ctx);if (rr->err)return rr->err;@@ -744,11 +755,27 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,if (is_mbm_total_enabled ()) { rr.evtid =QOS_L3_MBM_TOTAL_EVENT_ID ; rr.val =0 ; + rr.arch_mon_ctx =resctrl_arch_mon_ctx_alloc (rr.r, rr.evtid); +if (IS_ERR (rr.arch_mon_ctx)) { +pr_warn_ratelimited ("Failed to allocate monitor context: %ld" , +PTR_ERR (rr.arch_mon_ctx)); +return ; + } +__mon_event_count (closid, rmid, &rr); + +resctrl_arch_mon_ctx_free (rr.r, rr.evtid, rr.arch_mon_ctx); }if (is_mbm_local_enabled ()) { rr.evtid =QOS_L3_MBM_LOCAL_EVENT_ID ; rr.val =0 ; + rr.arch_mon_ctx =resctrl_arch_mon_ctx_alloc (rr.r, rr.evtid); +if (IS_ERR (rr.arch_mon_ctx)) { +pr_warn_ratelimited ("Failed to allocate monitor context: %ld" , +PTR_ERR (rr.arch_mon_ctx)); +return ; + } +__mon_event_count (closid, rmid, &rr);/* @@ -758,6 +785,8 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,*/ if (is_mba_sc (NULL ))mbm_bw_count (closid, rmid, &rr); + +resctrl_arch_mon_ctx_free (rr.r, rr.evtid, rr.arch_mon_ctx); } }include/linux/resctrl.h +4
-1@@ -235,6 +235,9 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);* @rmid: rmid of the counter to read. * @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes. +* @arch_mon_ctx: An architecture specific value from +* resctrl_arch_mon_ctx_alloc(), for MPAM this identifies +* the hardware monitor allocated for this read request. * * Some architectures need to sleep when first programming some of the counters. * (specifically: arm64's MPAM cache occupancy counters can return 'not ready' @@ -248,7 +251,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);*/ int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d,u32 closid,u32 rmid,enum resctrl_event_id eventid, -u64 *val); +u64 *val,void *arch_mon_ctx);/** * resctrl_arch_rmid_read_context_check() - warn about invalid contexts
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 15/24] x86/resctrl: Allow arch to allocate memory needed in resctrl_arch_rmid_read()
Hi Babu,
Quoted 10 lines · Babu
Quoted 6 lines · James Morse
Depending on the number of monitors available, Arm's MPAM may need to allocate a monitor prior to reading the counter value. Allocating a contended resource may involve sleeping.
add_rmid_to_limbo()callsresctrl_arch_rmid_read()for multiple domains, the allocation should be valid for all domains.This above sentence does not look correct after your previous patch [patch 14]. You removed
resctrl_arch_rmid_read()fromadd_rmid_to_limbo().
Ah, this was the justification for making the allocated resource global, when the patches were in a different order. It's also simpler to make them global. I've removed this paragraph, and replaced it with:
| The memory or hardware allocated is not specific to a domain.
(its fairly obvious from the interface, but I felt it needed calling out in the commit message)
Quoted 8 lines
Otherwise looks good.
Quoted 6 lines
__check_limbo()andmon_event_count()each make multiple calls toresctrl_arch_rmid_read(), to avoid extra work on contended systems, the allocation should be valid for multiple invocations ofresctrl_arch_rmid_read().
Quoted 6 lines
Quoted 4 lines
Add arch hooks for this allocation, which need calling before
resctrl_arch_rmid_read(). The allocated monitor is passed toresctrl_arch_rmid_read(), then freed again afterwards. The helper can be called on any CPU, and can sleep.Reviewed-by Babu Moger <[email protected]>
Thanks!
James
James Morse
reviewerPATCH v7 16/24 x86/resctrl: Make resctrl_mounted checks explicit
In reply to James MorseThe rdt_enable_key is switched when resctrl is mounted, and used to prevent a second mount of the filesystem. It also enables the architecture's context switch code.
This requires another architecture to have the same set of static-keys, as resctrl depends on them too. The existing users of these static-keys are implicitly also checking if the filesystem is mounted.
Make the resctrl_mounted checks explicit: resctrl can keep track of whether it has been mounted once. This doesn't need to be combined with whether the arch code is context switching the CLOSID.
rdt_mon_enable_key is never used just to test that resctrl is mounted, but does also have this implication. Add a resctrl_mounted to all uses of rdt_mon_enable_key. This will allow rdt_mon_enable_key to be swapped with a helper in a subsequent patch.
This will allow the static-key changing to be moved behind resctrl_arch_ calls.
---
Changes since v3:
* Removed a newline.
* Rephrased commit message
Changes since v4:
* Rephrased comment.
No changes since v6
--- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 12 ++++++++++-- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 23 +++++++++++++++++------ 3 files changed, 28 insertions(+), 8 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +1 -0
@@ -145,6 +145,7 @@ extern bool rdt_alloc_capable;extern bool rdt_mon_capable;extern unsigned int rdt_mon_features;extern struct list_head resctrl_schema_all; +extern bool resctrl_mounted;enum rdt_group_type {RDTCTRL_GROUP =0 ,
arch/x86/kernel/cpu/resctrl/monitor.c +10 -2
@@ -837,7 +837,11 @@ void mbm_handle_overflow(struct work_struct *work)mutex_lock (&rdtgroup_mutex); -if (!static_branch_likely (&rdt_mon_enable_key)) +/* +* If the filesystem has been unmounted this work no longer needs to +* run. +*/ +if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key)) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;
@@ -870,7 +874,11 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)unsigned long delay =msecs_to_jiffies (delay_ms);int cpu; -if (!static_branch_likely (&rdt_mon_enable_key)) +/* +* When a domain comes online there is no guarantee the filesystem is +* mounted. If not, there is no need to catch counter overflow. +*/ +if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key))return ; cpu =cpumask_any_housekeeping (&dom->cpu_mask); dom->mbm_work_cpu = cpu;
arch/x86/kernel/cpu/resctrl/rdtgroup.c +17 -6
@@ -42,6 +42,9 @@ LIST_HEAD(rdt_all_groups);/* list of entries for the schemata file */ LIST_HEAD (resctrl_schema_all); +/* The filesystem can only be mounted once. */ +bool resctrl_mounted; +/* Kernel fs node for "info" directory under root */ static struct kernfs_node *kn_info;
@@ -881,7 +884,7 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns,mutex_lock (&rdtgroup_mutex);/* Return empty if resctrl has not been mounted. */ -if (!static_branch_unlikely (&rdt_enable_key)) { +if (!resctrl_mounted) {seq_puts (s,"res:\nmon:\n" ); goto unlock; }
@@ -2613,7 +2616,7 @@ static int rdt_get_tree(struct fs_context *fc)/* * resctrl file system can only be mounted once. */ -if (static_branch_unlikely (&rdt_enable_key)) { +if (resctrl_mounted) { ret = -EBUSY ; goto out; }
@@ -2674,8 +2677,10 @@ static int rdt_get_tree(struct fs_context *fc)if (rdt_mon_capable)static_branch_enable_cpuslocked (&rdt_mon_enable_key); -if (rdt_alloc_capable || rdt_mon_capable) +if (rdt_alloc_capable || rdt_mon_capable) {static_branch_enable_cpuslocked (&rdt_enable_key); + resctrl_mounted = true; + }if (is_mbm_enabled ()) { r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;
@@ -2949,6 +2954,7 @@ static void rdt_kill_sb(struct super_block *sb)static_branch_disable_cpuslocked (&rdt_alloc_enable_key);static_branch_disable_cpuslocked (&rdt_mon_enable_key);static_branch_disable_cpuslocked (&rdt_enable_key); + resctrl_mounted = false;kernfs_kill_sb (sb);mutex_unlock (&rdtgroup_mutex);cpus_read_unlock ();
@@ -3918,7 +3924,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)* If resctrl is mounted, remove all the * per domain monitor data directories. */ -if (static_branch_unlikely (&rdt_mon_enable_key)) +if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key))rmdir_mondata_subdir_allrdtgrp (r, d->id);if (is_mbm_enabled ())
@@ -3995,8 +4001,13 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)if (is_llc_occupancy_enabled ())INIT_DELAYED_WORK (&d->cqm_limbo, cqm_handle_limbo); -/* If resctrl is mounted, add per domain monitor data directories. */ -if (static_branch_unlikely (&rdt_mon_enable_key)) +/* +* If the filesystem is not mounted then only the default resource group +* exists. Creation of its directories is deferred until mount time +* by rdt_get_tree() calling mkdir_mondata_all(). +* If resctrl is mounted, add per domain monitor data directories. +*/ +if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key))mkdir_mondata_subdir_allrdtgrp (r, d);return 0 ;
Signature · 1 line
2.39.2
Re: [PATCH v7 16/24] x86/resctrl: Make resctrl_mounted checks explicit
Quoted 17 lines · James Morse
The
rdt_enable_keyis switched when resctrl is mounted, and used to prevent a second mount of the filesystem. It also enables the architecture's context switch code.This requires another architecture to have the same set of static-keys, as resctrl depends on them too. The existing users of these static-keys are implicitly also checking if the filesystem is mounted.
Make the
resctrl_mountedchecks explicit: resctrl can keep track of whether it has been mounted once. This doesn't need to be combined with whether the arch code is context switching the CLOSID.
rdt_mon_enable_keyis never used just to test that resctrl is mounted, but does also have this implication. Add aresctrl_mountedto all uses ofrdt_mon_enable_key. This will allowrdt_mon_enable_keyto be swapped with a helper in a subsequent patch.
Quoted 8 lines
This will allow the static-key changing to be moved behind
resctrl_arch_calls.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 135 lines
---
Changes since v3:
* Removed a newline.
* Rephrased commit messageChanges since v4:
* Rephrased comment.No changes since v6
--- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 12 ++++++++++-- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 23 +++++++++++++++++------ 3 files changed, 28 insertions(+), 8 deletions(-)arch/x86/kernel/cpu/resctrl/internal.h +1
-0@@ -145,6 +145,7 @@ extern bool rdt_alloc_capable;extern bool rdt_mon_capable;extern unsigned int rdt_mon_features;extern struct list_head resctrl_schema_all; +extern bool resctrl_mounted;enum rdt_group_type {RDTCTRL_GROUP =0 ,arch/x86/kernel/cpu/resctrl/monitor.c +10
-2@@ -837,7 +837,11 @@ void mbm_handle_overflow(struct work_struct *work)mutex_lock (&rdtgroup_mutex); -if (!static_branch_likely (&rdt_mon_enable_key)) +/* +* If the filesystem has been unmounted this work no longer needs to +* run. +*/ +if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key)) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;@@ -870,7 +874,11 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)unsigned long delay =msecs_to_jiffies (delay_ms);int cpu; -if (!static_branch_likely (&rdt_mon_enable_key)) +/* +* When a domain comes online there is no guarantee the filesystem is +* mounted. If not, there is no need to catch counter overflow. +*/ +if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key))return ; cpu =cpumask_any_housekeeping (&dom->cpu_mask); dom->mbm_work_cpu = cpu;arch/x86/kernel/cpu/resctrl/rdtgroup.c +17
-6@@ -42,6 +42,9 @@ LIST_HEAD(rdt_all_groups);/* list of entries for the schemata file */ LIST_HEAD (resctrl_schema_all); +/* The filesystem can only be mounted once. */ +bool resctrl_mounted; +/* Kernel fs node for "info" directory under root */ static struct kernfs_node *kn_info;@@ -881,7 +884,7 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns,mutex_lock (&rdtgroup_mutex);/* Return empty if resctrl has not been mounted. */ -if (!static_branch_unlikely (&rdt_enable_key)) { +if (!resctrl_mounted) {seq_puts (s,"res:\nmon:\n" ); goto unlock; }@@ -2613,7 +2616,7 @@ static int rdt_get_tree(struct fs_context *fc)/* * resctrl file system can only be mounted once. */ -if (static_branch_unlikely (&rdt_enable_key)) { +if (resctrl_mounted) { ret = -EBUSY ; goto out; }@@ -2674,8 +2677,10 @@ static int rdt_get_tree(struct fs_context *fc)if (rdt_mon_capable)static_branch_enable_cpuslocked (&rdt_mon_enable_key); -if (rdt_alloc_capable || rdt_mon_capable) +if (rdt_alloc_capable || rdt_mon_capable) {static_branch_enable_cpuslocked (&rdt_enable_key); + resctrl_mounted = true; + }if (is_mbm_enabled ()) { r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;@@ -2949,6 +2954,7 @@ static void rdt_kill_sb(struct super_block *sb)static_branch_disable_cpuslocked (&rdt_alloc_enable_key);static_branch_disable_cpuslocked (&rdt_mon_enable_key);static_branch_disable_cpuslocked (&rdt_enable_key); + resctrl_mounted = false;kernfs_kill_sb (sb);mutex_unlock (&rdtgroup_mutex);cpus_read_unlock ();@@ -3918,7 +3924,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)* If resctrl is mounted, remove all the * per domain monitor data directories. */ -if (static_branch_unlikely (&rdt_mon_enable_key)) +if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key))rmdir_mondata_subdir_allrdtgrp (r, d->id);if (is_mbm_enabled ())@@ -3995,8 +4001,13 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)if (is_llc_occupancy_enabled ())INIT_DELAYED_WORK (&d->cqm_limbo, cqm_handle_limbo); -/* If resctrl is mounted, add per domain monitor data directories. */ -if (static_branch_unlikely (&rdt_mon_enable_key)) +/* +* If the filesystem is not mounted then only the default resource group +* exists. Creation of its directories is deferred until mount time +* by rdt_get_tree() calling mkdir_mondata_all(). +* If resctrl is mounted, add per domain monitor data directories. +*/ +if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key))mkdir_mondata_subdir_allrdtgrp (r, d);return 0 ;
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 16/24] x86/resctrl: Make resctrl_mounted checks explicit
Hi Babu,
Quoted 18 lines · Babu
Quoted 17 lines · James Morse
The
rdt_enable_keyis switched when resctrl is mounted, and used to prevent a second mount of the filesystem. It also enables the architecture's context switch code.This requires another architecture to have the same set of static-keys, as resctrl depends on them too. The existing users of these static-keys are implicitly also checking if the filesystem is mounted.
Make the
resctrl_mountedchecks explicit: resctrl can keep track of whether it has been mounted once. This doesn't need to be combined with whether the arch code is context switching the CLOSID.
rdt_mon_enable_keyis never used just to test that resctrl is mounted, but does also have this implication. Add aresctrl_mountedto all uses ofrdt_mon_enable_key. This will allowrdt_mon_enable_keyto be swapped with a helper in a subsequent patch.
Quoted 4 lines
Quoted 2 lines
This will allow the static-key changing to be moved behind
resctrl_arch_calls.Reviewed-by Babu Moger <[email protected]>
Thanks!
James
James Morse
reviewerPATCH v7 17/24 x86/resctrl: Move alloc/mon static keys into helpers
In reply to James Morseresctrl enables three static keys depending on the features it has enabled. Another architecture's context switch code may look different, any static keys that control it should be buried behind helpers.
Move the alloc/mon logic into arch-specific helpers as a preparatory step for making the rdt_enable_key's status something the arch code decides.
This means other architectures don't have to mirror the static keys.
---
No changes since v6
arch/x86/include/asm/resctrl.h | 20 ++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 5 ----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++---- 3 files changed, 24 insertions(+), 9 deletions(-)
arch/x86/include/asm/resctrl.h +20 -0
@@ -42,6 +42,26 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); +static inline void resctrl_arch_enable_alloc (void ) +{ +static_branch_enable_cpuslocked (&rdt_alloc_enable_key); +} + +static inline void resctrl_arch_disable_alloc (void ) +{ +static_branch_disable_cpuslocked (&rdt_alloc_enable_key); +} + +static inline void resctrl_arch_enable_mon (void ) +{ +static_branch_enable_cpuslocked (&rdt_mon_enable_key); +} + +static inline void resctrl_arch_disable_mon (void ) +{ +static_branch_disable_cpuslocked (&rdt_mon_enable_key); +} +/* * __resctrl_sched_in() - Writes the task's CLOSid/RMID to IA32_PQR_MSR *
arch/x86/kernel/cpu/resctrl/internal.h +0 -5
@@ -95,9 +95,6 @@ static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc) return container_of(kfc, struct rdt_fs_context, kfc); } -DECLARE_STATIC_KEY_FALSE (rdt_enable_key); -DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); -/** * struct mon_evt - Entry in the event list of a resource * @evtid: event id
@@ -454,8 +451,6 @@ extern struct mutex rdtgroup_mutex;extern struct rdt_hw_resource rdt_resources_all[];extern struct rdtgroup rdtgroup_default; -DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key); -extern struct dentry *debugfs_resctrl;enum resctrl_res_level {
arch/x86/kernel/cpu/resctrl/rdtgroup.c +4 -4
@@ -2673,9 +2673,9 @@ static int rdt_get_tree(struct fs_context *fc) goto out_psl;if (rdt_alloc_capable) -static_branch_enable_cpuslocked (&rdt_alloc_enable_key); +resctrl_arch_enable_alloc ();if (rdt_mon_capable) -static_branch_enable_cpuslocked (&rdt_mon_enable_key); +resctrl_arch_enable_mon ();if (rdt_alloc_capable || rdt_mon_capable) {static_branch_enable_cpuslocked (&rdt_enable_key);
@@ -2951,8 +2951,8 @@ static void rdt_kill_sb(struct super_block *sb) rdtgroup_default.mode =RDT_MODE_SHAREABLE ;schemata_list_destroy ();rdtgroup_destroy_root (); -static_branch_disable_cpuslocked (&rdt_alloc_enable_key); -static_branch_disable_cpuslocked (&rdt_mon_enable_key); +resctrl_arch_disable_alloc (); +resctrl_arch_disable_mon ();static_branch_disable_cpuslocked (&rdt_enable_key); resctrl_mounted = false;kernfs_kill_sb (sb);
Signature · 1 line
2.39.2
Re: [PATCH v7 17/24] x86/resctrl: Move alloc/mon static keys into helpers
Quoted 7 lines · James Morse
resctrl enables three static keys depending on the features it has enabled. Another architecture's context switch code may look different, any static keys that control it should be buried behind helpers.
Move the alloc/mon logic into arch-specific helpers as a preparatory step for making the rdt_enable_key's status something the arch code decides.
Quoted 7 lines
This means other architectures don't have to mirror the static keys.
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 89 lines
---
No changes since v6arch/x86/include/asm/resctrl.h | 20 ++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 5 ----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++---- 3 files changed, 24 insertions(+), 9 deletions(-)arch/x86/include/asm/resctrl.h +20
-0@@ -42,6 +42,26 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); +static inline void resctrl_arch_enable_alloc (void ) +{ +static_branch_enable_cpuslocked (&rdt_alloc_enable_key); +} + +static inline void resctrl_arch_disable_alloc (void ) +{ +static_branch_disable_cpuslocked (&rdt_alloc_enable_key); +} + +static inline void resctrl_arch_enable_mon (void ) +{ +static_branch_enable_cpuslocked (&rdt_mon_enable_key); +} + +static inline void resctrl_arch_disable_mon (void ) +{ +static_branch_disable_cpuslocked (&rdt_mon_enable_key); +} +/* * __resctrl_sched_in() - Writes the task's CLOSid/RMID to IA32_PQR_MSR * arch/x86/kernel/cpu/resctrl/internal.h +0
-5@@ -95,9 +95,6 @@ static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc) return container_of(kfc, struct rdt_fs_context, kfc); } -DECLARE_STATIC_KEY_FALSE (rdt_enable_key); -DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); -/** * struct mon_evt - Entry in the event list of a resource * @evtid: event id @@ -454,8 +451,6 @@ extern struct mutex rdtgroup_mutex;extern struct rdt_hw_resource rdt_resources_all[];extern struct rdtgroup rdtgroup_default; -DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key); -extern struct dentry *debugfs_resctrl;enum resctrl_res_level {arch/x86/kernel/cpu/resctrl/rdtgroup.c +4
-4@@ -2673,9 +2673,9 @@ static int rdt_get_tree(struct fs_context *fc) goto out_psl;if (rdt_alloc_capable) -static_branch_enable_cpuslocked (&rdt_alloc_enable_key); +resctrl_arch_enable_alloc ();if (rdt_mon_capable) -static_branch_enable_cpuslocked (&rdt_mon_enable_key); +resctrl_arch_enable_mon ();if (rdt_alloc_capable || rdt_mon_capable) {static_branch_enable_cpuslocked (&rdt_enable_key);@@ -2951,8 +2951,8 @@ static void rdt_kill_sb(struct super_block *sb) rdtgroup_default.mode =RDT_MODE_SHAREABLE ;schemata_list_destroy ();rdtgroup_destroy_root (); -static_branch_disable_cpuslocked (&rdt_alloc_enable_key); -static_branch_disable_cpuslocked (&rdt_mon_enable_key); +resctrl_arch_disable_alloc (); +resctrl_arch_disable_mon ();static_branch_disable_cpuslocked (&rdt_enable_key); resctrl_mounted = false;kernfs_kill_sb (sb);
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerPATCH v7 18/24 x86/resctrl: Make rdt_enable_key the arch's decision to switch
In reply to James Morserdt_enable_key is switched when resctrl is mounted. It was also previously used to prevent a second mount of the filesystem.
Any other architecture that wants to support resctrl has to provide identical static keys.
Now that there are helpers for enabling and disabling the alloc/mon keys, resctrl doesn't need to switch this extra key, it can be done by the arch code. Use the static-key increment and decrement helpers, and change resctrl to ensure the calls are balanced.
---
No changes since v6
arch/x86/include/asm/resctrl.h | 4 ++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 11 +++++------ 2 files changed, 9 insertions(+), 6 deletions(-)
arch/x86/include/asm/resctrl.h +4 -0
@@ -45,21 +45,25 @@ DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key);static inline void resctrl_arch_enable_alloc (void ) {static_branch_enable_cpuslocked (&rdt_alloc_enable_key); +static_branch_inc_cpuslocked (&rdt_enable_key); }static inline void resctrl_arch_disable_alloc (void ) {static_branch_disable_cpuslocked (&rdt_alloc_enable_key); +static_branch_dec_cpuslocked (&rdt_enable_key); }static inline void resctrl_arch_enable_mon (void ) {static_branch_enable_cpuslocked (&rdt_mon_enable_key); +static_branch_inc_cpuslocked (&rdt_enable_key); }static inline void resctrl_arch_disable_mon (void ) {static_branch_disable_cpuslocked (&rdt_mon_enable_key); +static_branch_dec_cpuslocked (&rdt_enable_key); }/*
arch/x86/kernel/cpu/resctrl/rdtgroup.c +5 -6
@@ -2677,10 +2677,8 @@ static int rdt_get_tree(struct fs_context *fc)if (rdt_mon_capable)resctrl_arch_enable_mon (); -if (rdt_alloc_capable || rdt_mon_capable) { -static_branch_enable_cpuslocked (&rdt_enable_key); +if (rdt_alloc_capable || rdt_mon_capable) resctrl_mounted = true; - }if (is_mbm_enabled ()) { r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;
@@ -2951,9 +2949,10 @@ static void rdt_kill_sb(struct super_block *sb) rdtgroup_default.mode =RDT_MODE_SHAREABLE ;schemata_list_destroy ();rdtgroup_destroy_root (); -resctrl_arch_disable_alloc (); -resctrl_arch_disable_mon (); -static_branch_disable_cpuslocked (&rdt_enable_key); +if (rdt_alloc_capable) +resctrl_arch_disable_alloc (); +if (rdt_mon_capable) +resctrl_arch_disable_mon (); resctrl_mounted = false;kernfs_kill_sb (sb);mutex_unlock (&rdtgroup_mutex);
Signature · 1 line
2.39.2
Re: [PATCH v7 18/24] x86/resctrl: Make rdt_enable_key the arch's decision to switch
Quoted 11 lines · James Morse
rdt_enable_keyis switched when resctrl is mounted. It was also previously used to prevent a second mount of the filesystem.Any other architecture that wants to support resctrl has to provide identical static keys.
Now that there are helpers for enabling and disabling the alloc/mon keys, resctrl doesn't need to switch this extra key, it can be done by the arch code. Use the static-key increment and decrement helpers, and change resctrl to ensure the calls are balanced.
Quoted 5 lines
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 67 lines
---
No changes since v6arch/x86/include/asm/resctrl.h | 4 ++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 11 +++++------ 2 files changed, 9 insertions(+), 6 deletions(-)arch/x86/include/asm/resctrl.h +4
-0@@ -45,21 +45,25 @@ DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key);static inline void resctrl_arch_enable_alloc (void ) {static_branch_enable_cpuslocked (&rdt_alloc_enable_key); +static_branch_inc_cpuslocked (&rdt_enable_key); }static inline void resctrl_arch_disable_alloc (void ) {static_branch_disable_cpuslocked (&rdt_alloc_enable_key); +static_branch_dec_cpuslocked (&rdt_enable_key); }static inline void resctrl_arch_enable_mon (void ) {static_branch_enable_cpuslocked (&rdt_mon_enable_key); +static_branch_inc_cpuslocked (&rdt_enable_key); }static inline void resctrl_arch_disable_mon (void ) {static_branch_disable_cpuslocked (&rdt_mon_enable_key); +static_branch_dec_cpuslocked (&rdt_enable_key); }/* arch/x86/kernel/cpu/resctrl/rdtgroup.c +5
-6@@ -2677,10 +2677,8 @@ static int rdt_get_tree(struct fs_context *fc)if (rdt_mon_capable)resctrl_arch_enable_mon (); -if (rdt_alloc_capable || rdt_mon_capable) { -static_branch_enable_cpuslocked (&rdt_enable_key); +if (rdt_alloc_capable || rdt_mon_capable) resctrl_mounted = true; - }if (is_mbm_enabled ()) { r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;@@ -2951,9 +2949,10 @@ static void rdt_kill_sb(struct super_block *sb) rdtgroup_default.mode =RDT_MODE_SHAREABLE ;schemata_list_destroy ();rdtgroup_destroy_root (); -resctrl_arch_disable_alloc (); -resctrl_arch_disable_mon (); -static_branch_disable_cpuslocked (&rdt_enable_key); +if (rdt_alloc_capable) +resctrl_arch_disable_alloc (); +if (rdt_mon_capable) +resctrl_arch_disable_mon (); resctrl_mounted = false;kernfs_kill_sb (sb);mutex_unlock (&rdtgroup_mutex);
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerPATCH v7 19/24 x86/resctrl: Add helpers for system wide mon/alloc capable
In reply to James Morseresctrl reads rdt_alloc_capable or rdt_mon_capable to determine whether any of the resources support the corresponding features. resctrl also uses the static-keys that affect the architecture's context-switch code to determine the same thing.
This forces another architecture to have the same static-keys.
As the static-key is enabled based on the capable flag, and none of the filesystem uses of these are in the scheduler path, move the capable flags behind helpers, and use these in the filesystem code instead of the static-key.
After this change, only the architecture code manages and uses the static-keys to ensure __resctrl_sched_in() does not need runtime checks.
This avoids multiple architectures having to define the same static-keys.
Cases where the static-key implicitly tested if the resctrl filesystem was mounted all have an explicit check added by a previous patch.
---
Changes since v1:
* Added missing conversion in mkdir_rdt_prepare_rmid_free()
Changes since v3:
* Expanded the commit message.
No changes since v6
--- arch/x86/include/asm/resctrl.h | 13 +++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 2 -- arch/x86/kernel/cpu/resctrl/monitor.c | 4 +-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 ++-- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 34 +++++++++++------------ 5 files changed, 35 insertions(+), 24 deletions(-)
arch/x86/include/asm/resctrl.h +13 -0
@@ -38,10 +38,18 @@ struct resctrl_pqr_state { DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state); +extern bool rdt_alloc_capable; +extern bool rdt_mon_capable; +DECLARE_STATIC_KEY_FALSE (rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); +static inline bool resctrl_arch_alloc_capable (void ) +{ +return rdt_alloc_capable; +} +static inline void resctrl_arch_enable_alloc (void ) {static_branch_enable_cpuslocked (&rdt_alloc_enable_key);
@@ -54,6 +62,11 @@ static inline void resctrl_arch_disable_alloc(void)static_branch_dec_cpuslocked (&rdt_enable_key); } +static inline bool resctrl_arch_mon_capable (void ) +{ +return rdt_mon_capable; +} +static inline void resctrl_arch_enable_mon (void ) {static_branch_enable_cpuslocked (&rdt_mon_enable_key);
arch/x86/kernel/cpu/resctrl/internal.h +0 -2
@@ -138,8 +138,6 @@ struct rmid_read {void *arch_mon_ctx; }; -extern bool rdt_alloc_capable; -extern bool rdt_mon_capable;extern unsigned int rdt_mon_features;extern struct list_head resctrl_schema_all;extern bool resctrl_mounted;
arch/x86/kernel/cpu/resctrl/monitor.c +2 -2
@@ -841,7 +841,7 @@ void mbm_handle_overflow(struct work_struct *work)* If the filesystem has been unmounted this work no longer needs to * run. */ -if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key)) +if (!resctrl_mounted || !resctrl_arch_mon_capable ()) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;
@@ -878,7 +878,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)* When a domain comes online there is no guarantee the filesystem is * mounted. If not, there is no need to catch counter overflow. */ -if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key)) +if (!resctrl_mounted || !resctrl_arch_mon_capable ())return ; cpu =cpumask_any_housekeeping (&dom->cpu_mask); dom->mbm_work_cpu = cpu;
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3 -3
@@ -581,7 +581,7 @@ static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp)if (ret) goto err_cpus; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =rdtgroup_kn_mode_restrict (rdtgrp,"mon_groups" );if (ret) goto err_cpus_list;
@@ -628,7 +628,7 @@ static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp)if (ret) goto err_cpus; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =rdtgroup_kn_mode_restore (rdtgrp,"mon_groups" ,0777 );if (ret) goto err_cpus_list;
@@ -776,7 +776,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) {int ret; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) {rdt_last_cmd_puts ("Out of RMIDs\n" );
arch/x86/kernel/cpu/resctrl/rdtgroup.c +17 -17
@@ -641,13 +641,13 @@ static int __rdtgroup_move_task(struct task_struct *tsk,static bool is_closid_match (struct task_struct *t,struct rdtgroup *r) { -return (rdt_alloc_capable && (r->type ==RDTCTRL_GROUP ) && +return (resctrl_arch_alloc_capable () && (r->type ==RDTCTRL_GROUP ) &&resctrl_arch_match_closid (t, r->closid)); }static bool is_rmid_match (struct task_struct *t,struct rdtgroup *r) { -return (rdt_mon_capable && (r->type ==RDTMON_GROUP ) && +return (resctrl_arch_mon_capable () && (r->type ==RDTMON_GROUP ) &&resctrl_arch_match_rmid (t, r->mon.parent->closid, r->mon.rmid)); }
@@ -2650,7 +2650,7 @@ static int rdt_get_tree(struct fs_context *fc)if (ret <0 ) goto out_schemata_free; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =mongroup_create_dir (rdtgroup_default.kn, &rdtgroup_default,"mon_groups" , &kn_mongrp);
@@ -2672,12 +2672,12 @@ static int rdt_get_tree(struct fs_context *fc)if (ret <0 ) goto out_psl; -if (rdt_alloc_capable) +if (resctrl_arch_alloc_capable ())resctrl_arch_enable_alloc (); -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())resctrl_arch_enable_mon (); -if (rdt_alloc_capable || rdt_mon_capable) +if (resctrl_arch_alloc_capable () ||resctrl_arch_mon_capable ()) resctrl_mounted = true;if (is_mbm_enabled ()) {
@@ -2691,10 +2691,10 @@ static int rdt_get_tree(struct fs_context *fc) out_psl:rdt_pseudo_lock_release (); out_mondata: -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())kernfs_remove (kn_mondata); out_mongrp: -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())kernfs_remove (kn_mongrp); out_info:kernfs_remove (kn_info);
@@ -2949,9 +2949,9 @@ static void rdt_kill_sb(struct super_block *sb) rdtgroup_default.mode =RDT_MODE_SHAREABLE ;schemata_list_destroy ();rdtgroup_destroy_root (); -if (rdt_alloc_capable) +if (resctrl_arch_alloc_capable ())resctrl_arch_disable_alloc (); -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())resctrl_arch_disable_mon (); resctrl_mounted = false;kernfs_kill_sb (sb);
@@ -3331,7 +3331,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) {int ret; -if (!rdt_mon_capable) +if (!resctrl_arch_mon_capable ())return 0 ; ret =alloc_rmid (rdtgrp->closid);
@@ -3353,7 +3353,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)static void mkdir_rdt_prepare_rmid_free (struct rdtgroup *rgrp) { -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())free_rmid (rgrp->closid, rgrp->mon.rmid); }
@@ -3526,7 +3526,7 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn,list_add (&rdtgrp->rdtgroup_list, &rdt_all_groups); -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) {/* * Create an empty mon_groups directory to hold the subset * of tasks and cpus to monitor.
@@ -3581,14 +3581,14 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,* allocation is supported, add a control and monitoring * subdirectory */ -if (rdt_alloc_capable && parent_kn == rdtgroup_default.kn) +if (resctrl_arch_alloc_capable () && parent_kn == rdtgroup_default.kn)return rdtgroup_mkdir_ctrl_mon (parent_kn, name, mode);/* * If RDT monitoring is supported and the parent directory is a valid * "mon_groups" directory, add a monitoring subdirectory. */ -if (rdt_mon_capable &&is_mon_groups (parent_kn, name)) +if (resctrl_arch_mon_capable () &&is_mon_groups (parent_kn, name))return rdtgroup_mkdir_mon (parent_kn, name, mode);return -EPERM ;
@@ -3923,7 +3923,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)* If resctrl is mounted, remove all the * per domain monitor data directories. */ -if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key)) +if (resctrl_mounted &&resctrl_arch_mon_capable ())rmdir_mondata_subdir_allrdtgrp (r, d->id);if (is_mbm_enabled ())
@@ -4006,7 +4006,7 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)* by rdt_get_tree() calling mkdir_mondata_all(). * If resctrl is mounted, add per domain monitor data directories. */ -if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key)) +if (resctrl_mounted &&resctrl_arch_mon_capable ())mkdir_mondata_subdir_allrdtgrp (r, d);return 0 ;
Signature · 1 line
2.39.2
Moger, Babu
reviewerRe: [PATCH v7 19/24] x86/resctrl: Add helpers for system wide mon/alloc capable
Hi James,
Thought you are replacing all rdt_mon_capable with resctrl_arch_mon_capable. But, you missed few.
1 rdtgroup.c rdt_get_tree 2640 if (rdt_mon_capable)
2 rdtgroup.c mkdir_rdt_prepare 3420 if (rdt_mon_capable)
Any reason these are not replaced?
thanks
Babu
Quoted 278 lines · James Morse
resctrl reads
rdt_alloc_capableorrdt_mon_capableto determine whether any of the resources support the corresponding features. resctrl also uses the static-keys that affect the architecture's context-switch code to determine the same thing.This forces another architecture to have the same static-keys.
As the static-key is enabled based on the capable flag, and none of the filesystem uses of these are in the scheduler path, move the capable flags behind helpers, and use these in the filesystem code instead of the static-key.
After this change, only the architecture code manages and uses the static-keys to ensure
__resctrl_sched_in()does not need runtime checks.This avoids multiple architectures having to define the same static-keys.
Cases where the static-key implicitly tested if the resctrl filesystem was mounted all have an explicit check added by a previous patch.
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since v1:
* Added missing conversion inmkdir_rdt_prepare_rmid_free()Changes since v3:
* Expanded the commit message.No changes since v6
--- arch/x86/include/asm/resctrl.h | 13 +++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 2 -- arch/x86/kernel/cpu/resctrl/monitor.c | 4 +-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 ++-- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 34 +++++++++++------------ 5 files changed, 35 insertions(+), 24 deletions(-)arch/x86/include/asm/resctrl.h +13
-0@@ -38,10 +38,18 @@ struct resctrl_pqr_state { DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state); +extern bool rdt_alloc_capable; +extern bool rdt_mon_capable; +DECLARE_STATIC_KEY_FALSE (rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); +static inline bool resctrl_arch_alloc_capable (void ) +{ +return rdt_alloc_capable; +} +static inline void resctrl_arch_enable_alloc (void ) {static_branch_enable_cpuslocked (&rdt_alloc_enable_key);@@ -54,6 +62,11 @@ static inline void resctrl_arch_disable_alloc(void)static_branch_dec_cpuslocked (&rdt_enable_key); } +static inline bool resctrl_arch_mon_capable (void ) +{ +return rdt_mon_capable; +} +static inline void resctrl_arch_enable_mon (void ) {static_branch_enable_cpuslocked (&rdt_mon_enable_key);arch/x86/kernel/cpu/resctrl/internal.h +0
-2@@ -138,8 +138,6 @@ struct rmid_read {void *arch_mon_ctx; }; -extern bool rdt_alloc_capable; -extern bool rdt_mon_capable;extern unsigned int rdt_mon_features;extern struct list_head resctrl_schema_all;extern bool resctrl_mounted;arch/x86/kernel/cpu/resctrl/monitor.c +2
-2@@ -841,7 +841,7 @@ void mbm_handle_overflow(struct work_struct *work)* If the filesystem has been unmounted this work no longer needs to * run. */ -if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key)) +if (!resctrl_mounted || !resctrl_arch_mon_capable ()) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;@@ -878,7 +878,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)* When a domain comes online there is no guarantee the filesystem is * mounted. If not, there is no need to catch counter overflow. */ -if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key)) +if (!resctrl_mounted || !resctrl_arch_mon_capable ())return ; cpu =cpumask_any_housekeeping (&dom->cpu_mask); dom->mbm_work_cpu = cpu;arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3
-3@@ -581,7 +581,7 @@ static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp)if (ret) goto err_cpus; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =rdtgroup_kn_mode_restrict (rdtgrp,"mon_groups" );if (ret) goto err_cpus_list;@@ -628,7 +628,7 @@ static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp)if (ret) goto err_cpus; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =rdtgroup_kn_mode_restore (rdtgrp,"mon_groups" ,0777 );if (ret) goto err_cpus_list;@@ -776,7 +776,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) {int ret; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) {rdt_last_cmd_puts ("Out of RMIDs\n" );arch/x86/kernel/cpu/resctrl/rdtgroup.c +17
-17@@ -641,13 +641,13 @@ static int __rdtgroup_move_task(struct task_struct *tsk,static bool is_closid_match (struct task_struct *t,struct rdtgroup *r) { -return (rdt_alloc_capable && (r->type ==RDTCTRL_GROUP ) && +return (resctrl_arch_alloc_capable () && (r->type ==RDTCTRL_GROUP ) &&resctrl_arch_match_closid (t, r->closid)); }static bool is_rmid_match (struct task_struct *t,struct rdtgroup *r) { -return (rdt_mon_capable && (r->type ==RDTMON_GROUP ) && +return (resctrl_arch_mon_capable () && (r->type ==RDTMON_GROUP ) &&resctrl_arch_match_rmid (t, r->mon.parent->closid, r->mon.rmid)); }@@ -2650,7 +2650,7 @@ static int rdt_get_tree(struct fs_context *fc)if (ret <0 ) goto out_schemata_free; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =mongroup_create_dir (rdtgroup_default.kn, &rdtgroup_default,"mon_groups" , &kn_mongrp);@@ -2672,12 +2672,12 @@ static int rdt_get_tree(struct fs_context *fc)if (ret <0 ) goto out_psl; -if (rdt_alloc_capable) +if (resctrl_arch_alloc_capable ())resctrl_arch_enable_alloc (); -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())resctrl_arch_enable_mon (); -if (rdt_alloc_capable || rdt_mon_capable) +if (resctrl_arch_alloc_capable () ||resctrl_arch_mon_capable ()) resctrl_mounted = true;if (is_mbm_enabled ()) {@@ -2691,10 +2691,10 @@ static int rdt_get_tree(struct fs_context *fc) out_psl:rdt_pseudo_lock_release (); out_mondata: -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())kernfs_remove (kn_mondata); out_mongrp: -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())kernfs_remove (kn_mongrp); out_info:kernfs_remove (kn_info);@@ -2949,9 +2949,9 @@ static void rdt_kill_sb(struct super_block *sb) rdtgroup_default.mode =RDT_MODE_SHAREABLE ;schemata_list_destroy ();rdtgroup_destroy_root (); -if (rdt_alloc_capable) +if (resctrl_arch_alloc_capable ())resctrl_arch_disable_alloc (); -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())resctrl_arch_disable_mon (); resctrl_mounted = false;kernfs_kill_sb (sb);@@ -3331,7 +3331,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) {int ret; -if (!rdt_mon_capable) +if (!resctrl_arch_mon_capable ())return 0 ; ret =alloc_rmid (rdtgrp->closid);@@ -3353,7 +3353,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)static void mkdir_rdt_prepare_rmid_free (struct rdtgroup *rgrp) { -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())free_rmid (rgrp->closid, rgrp->mon.rmid); }@@ -3526,7 +3526,7 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn,list_add (&rdtgrp->rdtgroup_list, &rdt_all_groups); -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) {/* * Create an empty mon_groups directory to hold the subset * of tasks and cpus to monitor. @@ -3581,14 +3581,14 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,* allocation is supported, add a control and monitoring * subdirectory */ -if (rdt_alloc_capable && parent_kn == rdtgroup_default.kn) +if (resctrl_arch_alloc_capable () && parent_kn == rdtgroup_default.kn)return rdtgroup_mkdir_ctrl_mon (parent_kn, name, mode);/* * If RDT monitoring is supported and the parent directory is a valid * "mon_groups" directory, add a monitoring subdirectory. */ -if (rdt_mon_capable &&is_mon_groups (parent_kn, name)) +if (resctrl_arch_mon_capable () &&is_mon_groups (parent_kn, name))return rdtgroup_mkdir_mon (parent_kn, name, mode);return -EPERM ;@@ -3923,7 +3923,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)* If resctrl is mounted, remove all the * per domain monitor data directories. */ -if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key)) +if (resctrl_mounted &&resctrl_arch_mon_capable ())rmdir_mondata_subdir_allrdtgrp (r, d->id);if (is_mbm_enabled ())@@ -4006,7 +4006,7 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)* by rdt_get_tree() calling mkdir_mondata_all(). * If resctrl is mounted, add per domain monitor data directories. */ -if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key)) +if (resctrl_mounted &&resctrl_arch_mon_capable ())mkdir_mondata_subdir_allrdtgrp (r, d);return 0 ;
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 19/24] x86/resctrl: Add helpers for system wide mon/alloc capable
Hi Babu,
Quoted 7 lines · Babu
Thought you are replacing all
rdt_mon_capablewithresctrl_arch_mon_capable. But, you missed few.1
rdtgroup.crdt_get_tree2640 if (rdt_mon_capable)
2rdtgroup.cmkdir_rdt_prepare3420 if (rdt_mon_capable)Any reason these are not replaced?
Nope,
These were added in tip, which I rebased on to in order to post this late, but didn't have the time to retest the ~100 other patches (mostly MPAM driver) that sit on top of this. x86 was quite happy without these. (There are too many moving parts!)
- thanks for catching these.
Thanks,
James
James Morse
reviewerPATCH v7 20/24 x86/resctrl: Add CPU online callback for resctrl work
In reply to James MorseThe resctrl architecture specific code may need to create a domain when a CPU comes online, it also needs to reset the CPUs PQR_ASSOC register. The resctrl filesystem code needs to update the rdtgroup_default CPU mask when CPUs are brought online.
Currently this is all done in one function, resctrl_online_cpu(). This will need to be split into architecture and filesystem parts before resctrl can be moved to /fs/.
Pull the rdtgroup_default update work out as a filesystem specific cpu_online helper. resctrl_online_cpu() is the obvious name for this, which means the version in core.c needs renaming.
resctrl_online_cpu() is called by the arch code once it has done the work to add the new CPU to any domains.
In future patches, resctrl_online_cpu() will take the rdtgroup_mutex itself.
---
Changes since v3:
* Renamed err to ret
Changes since v4:
* Changes in capitalisation.
Changes since v5:
* More changes in capitalisation.
* Made resctrl_online_cpu() return void.
No changes since v6
--- arch/x86/kernel/cpu/resctrl/core.c | 8 ++++---- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++++++ include/linux/resctrl.h | 1 + 3 files changed, 13 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +4 -4
@@ -608,16 +608,16 @@ static void clear_closid_rmid(int cpu) RESCTRL_RESERVED_CLOSID); } -static int resctrl_online_cpu (unsigned int cpu) +static int resctrl_arch_online_cpu (unsigned int cpu) {struct rdt_resource *r;mutex_lock (&rdtgroup_mutex);for_each_capable_rdt_resource (r)domain_add_cpu (cpu, r); -/* The cpu is set in default rdtgroup after online. */ -cpumask_set_cpu (cpu, &rdtgroup_default.cpu_mask);clear_closid_rmid (cpu); + +resctrl_online_cpu (cpu);mutex_unlock (&rdtgroup_mutex);return 0 ;
@@ -969,7 +969,7 @@ static int __init resctrl_late_init(void) state =cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"x86/resctrl/cat:online:" , - resctrl_online_cpu, resctrl_offline_cpu); + resctrl_arch_online_cpu, resctrl_offline_cpu);if (state <0 )return state;
arch/x86/kernel/cpu/resctrl/rdtgroup.c +8 -0
@@ -4012,6 +4012,14 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)return 0 ; } +void resctrl_online_cpu (unsigned int cpu) +{ +lockdep_assert_held (&rdtgroup_mutex); + +/* The CPU is set in default rdtgroup after online. */ +cpumask_set_cpu (cpu, &rdtgroup_default.cpu_mask); +} +/* * rdtgroup_init - rdtgroup initialization *
include/linux/resctrl.h +1 -0
@@ -223,6 +223,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d, u32 closid, enum resctrl_conf_type type);int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d);void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_online_cpu (unsigned int cpu);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
Signature · 1 line
2.39.2
Re: [PATCH v7 20/24] x86/resctrl: Add CPU online callback for resctrl work
Quoted 16 lines · James Morse
The resctrl architecture specific code may need to create a domain when a CPU comes online, it also needs to reset the CPUs
PQR_ASSOCregister. The resctrl filesystem code needs to update therdtgroup_defaultCPU mask when CPUs are brought online.Currently this is all done in one function,
resctrl_online_cpu(). This will need to be split into architecture and filesystem parts before resctrl can be moved to /fs/.Pull the
rdtgroup_defaultupdate work out as a filesystem specificcpu_onlinehelper.resctrl_online_cpu()is the obvious name for this, which means the version incore.cneeds renaming.
resctrl_online_cpu()is called by the arch code once it has done the work to add the new CPU to any domains.
Quoted 8 lines
In future patches,
resctrl_online_cpu()will take therdtgroup_mutexitself.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 82 lines
---
Changes since v3:
* Renamed err to retChanges since v4:
* Changes in capitalisation.Changes since v5:
* More changes in capitalisation.
* Maderesctrl_online_cpu()return void.No changes since v6
--- arch/x86/kernel/cpu/resctrl/core.c | 8 ++++---- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++++++ include/linux/resctrl.h | 1 + 3 files changed, 13 insertions(+), 4 deletions(-)arch/x86/kernel/cpu/resctrl/core.c +4
-4@@ -608,16 +608,16 @@ static void clear_closid_rmid(int cpu) RESCTRL_RESERVED_CLOSID); } -static int resctrl_online_cpu (unsigned int cpu) +static int resctrl_arch_online_cpu (unsigned int cpu) {struct rdt_resource *r;mutex_lock (&rdtgroup_mutex);for_each_capable_rdt_resource (r)domain_add_cpu (cpu, r); -/* The cpu is set in default rdtgroup after online. */ -cpumask_set_cpu (cpu, &rdtgroup_default.cpu_mask);clear_closid_rmid (cpu); + +resctrl_online_cpu (cpu);mutex_unlock (&rdtgroup_mutex);return 0 ;@@ -969,7 +969,7 @@ static int __init resctrl_late_init(void) state =cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"x86/resctrl/cat:online:" , - resctrl_online_cpu, resctrl_offline_cpu); + resctrl_arch_online_cpu, resctrl_offline_cpu);if (state <0 )return state;arch/x86/kernel/cpu/resctrl/rdtgroup.c +8
-0@@ -4012,6 +4012,14 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)return 0 ; } +void resctrl_online_cpu (unsigned int cpu) +{ +lockdep_assert_held (&rdtgroup_mutex); + +/* The CPU is set in default rdtgroup after online. */ +cpumask_set_cpu (cpu, &rdtgroup_default.cpu_mask); +} +/* * rdtgroup_init - rdtgroup initialization * include/linux/resctrl.h +1
-0@@ -223,6 +223,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d, u32 closid, enum resctrl_conf_type type);int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d);void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d); +void resctrl_online_cpu (unsigned int cpu);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 20/24] x86/resctrl: Add CPU online callback for resctrl work
Hi Babu,
Quoted 14 lines · Babu
Quoted 13 lines · James Morse
The resctrl architecture specific code may need to create a domain when a CPU comes online, it also needs to reset the CPUs
PQR_ASSOCregister. The resctrl filesystem code needs to update therdtgroup_defaultCPU mask when CPUs are brought online.Currently this is all done in one function,
resctrl_online_cpu(). This will need to be split into architecture and filesystem parts before resctrl can be moved to /fs/.Pull the
rdtgroup_defaultupdate work out as a filesystem specificcpu_onlinehelper.resctrl_online_cpu()is the obvious name for this, which means the version incore.cneeds renaming.
Quoted 7 lines
Quoted 5 lines
resctrl_online_cpu()is called by the arch code once it has done the work to add the new CPU to any domains.In future patches,
resctrl_online_cpu()will take therdtgroup_mutexitself.Reviewed-by Babu Moger <[email protected]>
Thanks!
James
James Morse
reviewerPATCH v7 21/24 x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but cpu
In reply to James MorseWhen a CPU is taken offline resctrl may need to move the overflow or limbo handlers to run on a different CPU.
Once the offline callbacks have been split, cqm_setup_limbo_handler() will be called while the CPU that is going offline is still present in the cpu_mask.
Pass the CPU to exclude to cqm_setup_limbo_handler() and mbm_setup_overflow_handler(). These functions can use a variant of cpumask_any_but() when selecting the CPU. -1 is used to indicate no CPUs need excluding.
A subsequent patch moves these calls to be before CPUs have been removed, so this exclude_cpus behaviour is temporary.
---
Changes since v2:
* Rephrased a comment to avoid a two letter bad-word. (we)
* Avoid assigning mbm_work_cpu if the domain is going to be free()d
* Added cpumask_any_housekeeping_but(), I dislike the name
Changes since v3:
* Marked an explanatory comment as temporary as the subsequent patch is no longer adjacent.
Changes since v4:
* Check against RESCTRL_PICK_ANY_CPU instead of -1.
* Leave cqm_work_cpu as nr_cpu_ids when no CPU is available.
* Made cpumask_any_housekeeping_but() more readable.
Changes since v5:
* Changes in captialisation, and a typo.
* Merged cpumask helpers.
Changes since v6:
* Added the missing dom parameter to some kernel doc.
* Re-added use of cpumask_any_but(),
* Expanded comment above cpumask_any_housekeeping(),
* Added some more comments for good measure.
* Added explicit IS_ENABLED() check as gcc-12 doesn't seem to work this out.
--- arch/x86/kernel/cpu/resctrl/core.c | 8 +++-- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +- arch/x86/kernel/cpu/resctrl/internal.h | 33 ++++++++++++++---- arch/x86/kernel/cpu/resctrl/monitor.c | 42 ++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 6 ++-- include/linux/resctrl.h | 2 ++ 6 files changed, 72 insertions(+), 21 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +6 -2
@@ -586,12 +586,16 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)if (r == &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl) {if (is_mbm_enabled () && cpu == d->mbm_work_cpu) {cancel_delayed_work (&d->mbm_over); -mbm_setup_overflow_handler (d,0 ); +/* +* temporary: exclude_cpu=-1 as this CPU has already +* been removed by cpumask_clear_cpu()d +*/ +mbm_setup_overflow_handler (d,0 ,RESCTRL_PICK_ANY_CPU ); }if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu &&has_busy_rmid (d)) {cancel_delayed_work (&d->cqm_limbo); -cqm_setup_limbo_handler (d,0 ); +cqm_setup_limbo_handler (d,0 ,RESCTRL_PICK_ANY_CPU ); } } }
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +1 -1
@@ -552,7 +552,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,return ; } - cpu =cpumask_any_housekeeping (&d->cpu_mask); + cpu =cpumask_any_housekeeping (&d->cpu_mask,RESCTRL_PICK_ANY_CPU );/* * cpumask_any_housekeeping() prefers housekeeping CPUs, but
arch/x86/kernel/cpu/resctrl/internal.h +26 -7
@@ -61,19 +61,36 @@* cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that * aren't marked nohz_full * @mask: The mask to pick a CPU from. +* @exclude_cpu:The CPU to avoid picking. * -* Returns a CPU in @mask. If there are housekeeping CPUs that don't use -* nohz_full, these are preferred. +* Returns a CPU from @mask, but not @exclude_cpu. If there are housekeeping +* CPUs that don't use nohz_full, these are preferred. Pass +* RESCTRL_PICK_ANY_CPU to avoid excluding any CPUs. +* +* When a CPU is excluded, returns >= nr_cpu_ids if no CPUs are available. */ -static inline unsigned int cpumask_any_housekeeping (const struct cpumask *mask) +static inline unsigned int +cpumask_any_housekeeping (const struct cpumask *mask,int exclude_cpu) {unsigned int cpu, hk_cpu; - cpu =cpumask_any (mask); -if (!tick_nohz_full_cpu (cpu)) +if (exclude_cpu ==RESCTRL_PICK_ANY_CPU ) + cpu =cpumask_any (mask); +else + cpu =cpumask_any_but (mask, exclude_cpu); + +if (!IS_ENABLED (CONFIG_NO_HZ_FULL )) +return cpu; + +/* If the CPU picked isn't marked nohz_full, we're done */ +if (cpu <= nr_cpu_ids && !tick_nohz_full_cpu (cpu))return cpu; +/* Try to find a CPU that isn't nohz_full to use in preference */ hk_cpu =cpumask_nth_andnot (0 , mask, tick_nohz_full_mask); +if (hk_cpu == exclude_cpu) + hk_cpu =cpumask_nth_andnot (1 , mask, tick_nohz_full_mask); +if (hk_cpu < nr_cpu_ids) cpu = hk_cpu;
@@ -575,11 +592,13 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_domain *d, struct rdtgroup *rdtgrp, int evtid, int first);void mbm_setup_overflow_handler (struct rdt_domain *dom, -unsigned long delay_ms); +unsigned long delay_ms, +int exclude_cpu);void mbm_handle_overflow (struct work_struct *work);void __init intel_rdt_mbm_apply_quirk (void );bool is_mba_sc (struct rdt_resource *r); -void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms); +void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms, +int exclude_cpu);void cqm_handle_limbo (struct work_struct *work);bool has_busy_rmid (struct rdt_domain *d);void __check_limbo (struct rdt_domain *d,bool force_free);
arch/x86/kernel/cpu/resctrl/monitor.c +33 -9
85 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/rdtgroup.c +4 -2
@@ -2683,7 +2683,8 @@ static int rdt_get_tree(struct fs_context *fc)if (is_mbm_enabled ()) { r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;list_for_each_entry (dom, &r->domains, list) -mbm_setup_overflow_handler (dom,MBM_OVERFLOW_INTERVAL ); +mbm_setup_overflow_handler (dom,MBM_OVERFLOW_INTERVAL , +RESCTRL_PICK_ANY_CPU ); } goto out;
@@ -3994,7 +3995,8 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)if (is_mbm_enabled ()) {INIT_DELAYED_WORK (&d->mbm_over, mbm_handle_overflow); -mbm_setup_overflow_handler (d,MBM_OVERFLOW_INTERVAL ); +mbm_setup_overflow_handler (d,MBM_OVERFLOW_INTERVAL , +RESCTRL_PICK_ANY_CPU ); }if (is_llc_occupancy_enabled ())
include/linux/resctrl.h +2 -0
@@ -10,6 +10,8 @@#define RESCTRL_RESERVED_CLOSID 0#define RESCTRL_RESERVED_RMID 0 +#define RESCTRL_PICK_ANY_CPU -1 +#ifdef CONFIG_PROC_CPU_RESCTRL int proc_resctrl_show(struct seq_file *m,
Signature · 1 line
2.39.2
Reinette Chatre
maintainerRe: [PATCH v7 21/24] x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but cpu
Hi James,
Quoted 7 lines · James Morse
When a CPU is taken offline resctrl may need to move the overflow or limbo handlers to run on a different CPU.
Once the offline callbacks have been split,
cqm_setup_limbo_handler()will be called while the CPU that is going offline is still present in thecpu_mask.
Quoted 7 lines
Pass the CPU to exclude to
cqm_setup_limbo_handler()andmbm_setup_overflow_handler(). These functions can use a variant ofcpumask_any_but()when selecting the CPU. -1 is used to indicate no CPUs need excluding.A subsequent patch moves these calls to be before CPUs have been removed, so this
exclude_cpusbehaviour is temporary.
Note "A subsequent patch". Please do go over your entire series. I may not have noticed all.
Quoted 98 lines
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>---
Changes since v2:
* Rephrased a comment to avoid a two letter bad-word. (we)
* Avoid assigningmbm_work_cpuif the domain is going to be free()d
* Addedcpumask_any_housekeeping_but(), I dislike the nameChanges since v3:
* Marked an explanatory comment as temporary as the subsequent patch is no longer adjacent.Changes since v4:
* Check againstRESCTRL_PICK_ANY_CPUinstead of -1.
* Leavecqm_work_cpuasnr_cpu_idswhen no CPU is available.
* Madecpumask_any_housekeeping_but()more readable.Changes since v5:
* Changes in captialisation, and a typo.
* Merged cpumask helpers.Changes since v6:
* Added the missing dom parameter to some kernel doc.
* Re-added use ofcpumask_any_but(),
* Expanded comment abovecpumask_any_housekeeping(),
* Added some more comments for good measure.
* Added explicitIS_ENABLED()check as gcc-12 doesn't seem to work this out.--- arch/x86/kernel/cpu/resctrl/core.c | 8 +++-- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +- arch/x86/kernel/cpu/resctrl/internal.h | 33 ++++++++++++++---- arch/x86/kernel/cpu/resctrl/monitor.c | 42 ++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 6 ++-- include/linux/resctrl.h | 2 ++ 6 files changed, 72 insertions(+), 21 deletions(-)arch/x86/kernel/cpu/resctrl/core.c +6
-2@@ -586,12 +586,16 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)if (r == &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl) {if (is_mbm_enabled () && cpu == d->mbm_work_cpu) {cancel_delayed_work (&d->mbm_over); -mbm_setup_overflow_handler (d,0 ); +/* +* temporary: exclude_cpu=-1 as this CPU has already +* been removed by cpumask_clear_cpu()d +*/ +mbm_setup_overflow_handler (d,0 ,RESCTRL_PICK_ANY_CPU ); }if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu &&has_busy_rmid (d)) {cancel_delayed_work (&d->cqm_limbo); -cqm_setup_limbo_handler (d,0 ); +cqm_setup_limbo_handler (d,0 ,RESCTRL_PICK_ANY_CPU ); } } }arch/x86/kernel/cpu/resctrl/ctrlmondata.c +1
-1@@ -552,7 +552,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,return ; } - cpu =cpumask_any_housekeeping (&d->cpu_mask); + cpu =cpumask_any_housekeeping (&d->cpu_mask,RESCTRL_PICK_ANY_CPU );/* * cpumask_any_housekeeping() prefers housekeeping CPUs, but arch/x86/kernel/cpu/resctrl/internal.h +8
-3@@ -61,19 +61,36 @@* cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that * aren't marked nohz_full * @mask: The mask to pick a CPU from. +* @exclude_cpu:The CPU to avoid picking. * -* Returns a CPU in @mask. If there are housekeeping CPUs that don't use -* nohz_full, these are preferred. +* Returns a CPU from @mask, but not @exclude_cpu. If there are housekeeping +* CPUs that don't use nohz_full, these are preferred. Pass +* RESCTRL_PICK_ANY_CPU to avoid excluding any CPUs. +* +* When a CPU is excluded, returns >= nr_cpu_ids if no CPUs are available. */ -static inline unsigned int cpumask_any_housekeeping (const struct cpumask *mask) +static inline unsigned int +cpumask_any_housekeeping (const struct cpumask *mask,int exclude_cpu) {unsigned int cpu, hk_cpu;
arch/x86/kernel/cpu/resctrl/internal.h line 83
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/internal.h +7
-2- cpu =cpumask_any (mask); -if (!tick_nohz_full_cpu (cpu)) +if (exclude_cpu ==RESCTRL_PICK_ANY_CPU ) + cpu =cpumask_any (mask); +else + cpu =cpumask_any_but (mask, exclude_cpu); + +if (!IS_ENABLED (CONFIG_NO_HZ_FULL )) +return cpu;
It is not clear to me how cpumask_any_but() failure is handled.
cpumask_any_but() returns ">= nr_cpu_ids if no cpus set" ...
arch/x86/kernel/cpu/resctrl/internal.h line 85
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/internal.h +2
-0+ +/* If the CPU picked isn't marked nohz_full, we're done */
Please don't impersonate code.
arch/x86/kernel/cpu/resctrl/internal.h line 87
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/internal.h +1
-0+if (cpu <= nr_cpu_ids && !tick_nohz_full_cpu (cpu))return cpu;
Is this intended to be "cpu < nr_cpu_ids"? But that would have code continue ... so maybe it needs explicit error check of cpumask_any_but() failure with an earlier exit?
arch/x86/kernel/cpu/resctrl/internal.h line 96
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/internal.h +4
-0+/* Try to find a CPU that isn't nohz_full to use in preference */ hk_cpu =cpumask_nth_andnot (0 , mask, tick_nohz_full_mask); +if (hk_cpu == exclude_cpu) + hk_cpu =cpumask_nth_andnot (1 , mask, tick_nohz_full_mask); +if (hk_cpu < nr_cpu_ids) cpu = hk_cpu;
Reinette
James Morse
reviewerRe: [PATCH v7 21/24] x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but cpu
Hi Reinette,
Quoted 13 lines · Reinette Chatre
Quoted 12 lines · James Morse
When a CPU is taken offline resctrl may need to move the overflow or limbo handlers to run on a different CPU.
Once the offline callbacks have been split,
cqm_setup_limbo_handler()will be called while the CPU that is going offline is still present in thecpu_mask.Pass the CPU to exclude to
cqm_setup_limbo_handler()andmbm_setup_overflow_handler(). These functions can use a variant ofcpumask_any_but()when selecting the CPU. -1 is used to indicate no CPUs need excluding.
Quoted 5 lines
Quoted 2 lines
A subsequent patch moves these calls to be before CPUs have been removed, so this
exclude_cpusbehaviour is temporary.Note "A subsequent patch". Please do go over your entire series. I may not have noticed all.
Yup, I've searched the git-log and removed those paragraphs from the x86 patches in my tree.
Quoted 36 lines
Quoted 23 lines
arch/x86/kernel/cpu/resctrl/internal.h +8
-3@@ -61,19 +61,36 @@* cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that * aren't marked nohz_full * @mask: The mask to pick a CPU from. +* @exclude_cpu:The CPU to avoid picking. * -* Returns a CPU in @mask. If there are housekeeping CPUs that don't use -* nohz_full, these are preferred. +* Returns a CPU from @mask, but not @exclude_cpu. If there are housekeeping +* CPUs that don't use nohz_full, these are preferred. Pass +* RESCTRL_PICK_ANY_CPU to avoid excluding any CPUs. +* +* When a CPU is excluded, returns >= nr_cpu_ids if no CPUs are available. */ -static inline unsigned int cpumask_any_housekeeping (const struct cpumask *mask) +static inline unsigned int +cpumask_any_housekeeping (const struct cpumask *mask,int exclude_cpu) {unsigned int cpu, hk_cpu;arch/x86/kernel/cpu/resctrl/internal.h line 83
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/internal.h +7
-2- cpu =cpumask_any (mask); -if (!tick_nohz_full_cpu (cpu)) +if (exclude_cpu ==RESCTRL_PICK_ANY_CPU ) + cpu =cpumask_any (mask); +else + cpu =cpumask_any_but (mask, exclude_cpu); + +if (!IS_ENABLED (CONFIG_NO_HZ_FULL )) +return cpu;It is not clear to me how
cpumask_any_but()failure is handled.
Quoted 1 line
cpumask_any_but()returns ">=nr_cpu_idsif no cpus set" ...
It wasn't a satisfiable request, there are no CPUs for this domain other than the one that was excluded. cpumask_any_but() also describes its errors as "returns >= nr_cpu_ids if no CPUs are available".
The places this can happen in resctrl are:cqm_setup_limbo_handler(), where it causes the schedule_delayed_work_on() call to be skipped. mbm_setup_overflow_handler(), which does similar.
These two cases are triggered from resctrl_offline_cpu() when the last CPU in a domain is going offline, and the domain is about to be free()d. This is how the limbo/overflow 'threads' stop.
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/internal.h line 85
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/internal.h +2
-0+ +/* If the CPU picked isn't marked nohz_full, we're done */ Please don't impersonate code.
arch/x86/kernel/cpu/resctrl/internal.h line 87
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/internal.h +1
-0+if (cpu <= nr_cpu_ids && !tick_nohz_full_cpu (cpu))return cpu;Is this intended to be "cpu <
nr_cpu_ids"?
Yes, fixed - thanks!
Quoted 3 lines
But that would have
code continue ... so maybe it needs explicit error check ofcpumask_any_but()failure with an earlier exit?
I'm not sure what the problem you refer to here is.
If 'cpu' is valid, and not marked nohz_full, nothing more needs doing.
If 'cpu' is invalid or a CPU marked nohz_full, then a second attempt is made to find a housekeeping CPU into 'hk_cpu'. If the second attempt is valid, it's used in preference.
An error is returned if the request couldn't be satisfied, i.e. an empty mask was passed, or the only CPU set in the mask was excluded.
There is a second attempt in this case for a housekeeping CPU - but that will fail too. As above, this only happens when CPUs are going offline, and this error is handled by the caller.
Thanks,
James
Reinette Chatre
maintainerRe: [PATCH v7 21/24] x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but cpu
Hi James,
Quoted 45 lines · James Morse
Quoted 38 lines · Reinette Chatre
Quoted 23 lines · James Morse
arch/x86/kernel/cpu/resctrl/internal.h +8
-3@@ -61,19 +61,36 @@* cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that * aren't marked nohz_full * @mask: The mask to pick a CPU from. +* @exclude_cpu:The CPU to avoid picking. * -* Returns a CPU in @mask. If there are housekeeping CPUs that don't use -* nohz_full, these are preferred. +* Returns a CPU from @mask, but not @exclude_cpu. If there are housekeeping +* CPUs that don't use nohz_full, these are preferred. Pass +* RESCTRL_PICK_ANY_CPU to avoid excluding any CPUs. +* +* When a CPU is excluded, returns >= nr_cpu_ids if no CPUs are available. */ -static inline unsigned int cpumask_any_housekeeping (const struct cpumask *mask) +static inline unsigned int +cpumask_any_housekeeping (const struct cpumask *mask,int exclude_cpu) {unsigned int cpu, hk_cpu;arch/x86/kernel/cpu/resctrl/internal.h line 83
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/internal.h +7
-2- cpu =cpumask_any (mask); -if (!tick_nohz_full_cpu (cpu)) +if (exclude_cpu ==RESCTRL_PICK_ANY_CPU ) + cpu =cpumask_any (mask); +else + cpu =cpumask_any_but (mask, exclude_cpu); + +if (!IS_ENABLED (CONFIG_NO_HZ_FULL )) +return cpu;It is not clear to me how
cpumask_any_but()failure is handled.Quoted 1 line
cpumask_any_but()returns ">=nr_cpu_idsif no cpus set" ...It wasn't a satisfiable request, there are no CPUs for this domain other than the one that was excluded.
cpumask_any_but()also describes its errors as "returns >=nr_cpu_idsif no CPUs are available".
Quoted 7 lines
The places this can happen in resctrl are:
cqm_setup_limbo_handler(), where it causes theschedule_delayed_work_on()call to be skipped.mbm_setup_overflow_handler(), which does similar.These two cases are triggered from
resctrl_offline_cpu()when the last CPU in a domain is going offline, and the domain is about to be free()d. This is how the limbo/overflow 'threads' stop.
Right ... yet this is a generic function, if there are any requirements on when/how it should be called then it needs to be specified in the function comments. I do not expect this to be the case for this function.
Quoted 13 lines
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/internal.h line 85
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/internal.h +2
-0+ +/* If the CPU picked isn't marked nohz_full, we're done */ Please don't impersonate code.
arch/x86/kernel/cpu/resctrl/internal.h line 87
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/internal.h +1
-0+if (cpu <= nr_cpu_ids && !tick_nohz_full_cpu (cpu))return cpu;Is this intended to be "cpu <
nr_cpu_ids"?Yes, fixed - thanks!
Quoted 8 lines
Quoted 3 lines
But that would have
code continue ... so maybe it needs explicit error check ofcpumask_any_but()failure with an earlier exit?I'm not sure what the problem you refer to here is.
If 'cpu' is valid, and not markednohz_full, nothing more needs doing.
If 'cpu' is invalid or a CPU markednohz_full, then a second attempt is made to find a housekeeping CPU into 'hk_cpu'. If the second attempt is valid, it's used in preference.
Considering that the second attempt can only be on the same or smaller set of CPUs, how could the second attempt ever succeed if the first attempt failed? I do not see why it is worth continuing.
Quoted 5 lines
An error is returned if the request couldn't be satisfied, i.e. an empty mask was passed, or the only CPU set in the mask was excluded.
There is a second attempt in this case for a housekeeping CPU - but that will fail too. As above, this only happens when CPUs are going offline, and this error is handled by the caller.
Reinette
James Morse
reviewerRe: [PATCH v7 21/24] x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but cpu
Hi Reinette,
Quoted 50 lines · Reinette Chatre
Quoted 49 lines · James Morse
Quoted 38 lines · Reinette Chatre
Quoted 23 lines · James Morse
arch/x86/kernel/cpu/resctrl/internal.h +8
-3@@ -61,19 +61,36 @@* cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that * aren't marked nohz_full * @mask: The mask to pick a CPU from. +* @exclude_cpu:The CPU to avoid picking. * -* Returns a CPU in @mask. If there are housekeeping CPUs that don't use -* nohz_full, these are preferred. +* Returns a CPU from @mask, but not @exclude_cpu. If there are housekeeping +* CPUs that don't use nohz_full, these are preferred. Pass +* RESCTRL_PICK_ANY_CPU to avoid excluding any CPUs. +* +* When a CPU is excluded, returns >= nr_cpu_ids if no CPUs are available. */ -static inline unsigned int cpumask_any_housekeeping (const struct cpumask *mask) +static inline unsigned int +cpumask_any_housekeeping (const struct cpumask *mask,int exclude_cpu) {unsigned int cpu, hk_cpu;arch/x86/kernel/cpu/resctrl/internal.h line 83
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/internal.h +7
-2- cpu =cpumask_any (mask); -if (!tick_nohz_full_cpu (cpu)) +if (exclude_cpu ==RESCTRL_PICK_ANY_CPU ) + cpu =cpumask_any (mask); +else + cpu =cpumask_any_but (mask, exclude_cpu); + +if (!IS_ENABLED (CONFIG_NO_HZ_FULL )) +return cpu;It is not clear to me how
cpumask_any_but()failure is handled.Quoted 1 line
cpumask_any_but()returns ">=nr_cpu_idsif no cpus set" ...It wasn't a satisfiable request, there are no CPUs for this domain other than the one that was excluded.
cpumask_any_but()also describes its errors as "returns >=nr_cpu_idsif no CPUs are available".The places this can happen in resctrl are:
cqm_setup_limbo_handler(), where it causes theschedule_delayed_work_on()call to be skipped.mbm_setup_overflow_handler(), which does similar.
Quoted 7 lines
Quoted 3 lines
These two cases are triggered from
resctrl_offline_cpu()when the last CPU in a domain is going offline, and the domain is about to be free()d. This is how the limbo/overflow 'threads' stop.Right ... yet this is a generic function, if there are any requirements on when/how it should be called then it needs to be specified in the function comments. I do not expect this to be the case for this function.
There are no special requirements, like all the other cpumask_foo() helpers, you can feed it an empty bitmap and it will return '>= nr_cpu_ids' as an error.
[...]
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
But that would have
code continue ... so maybe it needs explicit error check ofcpumask_any_but()failure with an earlier exit?
Quoted 8 lines
Quoted 4 lines
I'm not sure what the problem you refer to here is.
If 'cpu' is valid, and not markednohz_full, nothing more needs doing.
If 'cpu' is invalid or a CPU markednohz_full, then a second attempt is made to find a housekeeping CPU into 'hk_cpu'. If the second attempt is valid, it's used in preference.Considering that the second attempt can only be on the same or smaller set of CPUs, how could the second attempt ever succeed if the first attempt failed? I do not see why it is worth continuing.
Its harmless, its not on a performance sensitive path and it would take extra logic in the more common cases to detect this and return early.
Thanks,
James
Re: [PATCH v7 21/24] x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but cpu
In reply to James MorseQuoted 12 lines · James Morse
When a CPU is taken offline resctrl may need to move the overflow or limbo handlers to run on a different CPU.
Once the offline callbacks have been split,
cqm_setup_limbo_handler()will be called while the CPU that is going offline is still present in thecpu_mask.Pass the CPU to exclude to
cqm_setup_limbo_handler()andmbm_setup_overflow_handler(). These functions can use a variant ofcpumask_any_but()when selecting the CPU. -1 is used to indicate no CPUs need excluding.
Quoted 7 lines
A subsequent patch moves these calls to be before CPUs have been removed, so this
exclude_cpusbehaviour is temporary.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 263 lines
---
Changes since v2:
* Rephrased a comment to avoid a two letter bad-word. (we)
* Avoid assigningmbm_work_cpuif the domain is going to be free()d
* Addedcpumask_any_housekeeping_but(), I dislike the nameChanges since v3:
* Marked an explanatory comment as temporary as the subsequent patch is no longer adjacent.Changes since v4:
* Check againstRESCTRL_PICK_ANY_CPUinstead of -1.
* Leavecqm_work_cpuasnr_cpu_idswhen no CPU is available.
* Madecpumask_any_housekeeping_but()more readable.Changes since v5:
* Changes in captialisation, and a typo.
* Merged cpumask helpers.Changes since v6:
* Added the missing dom parameter to some kernel doc.
* Re-added use ofcpumask_any_but(),
* Expanded comment abovecpumask_any_housekeeping(),
* Added some more comments for good measure.
* Added explicitIS_ENABLED()check as gcc-12 doesn't seem to work this out.--- arch/x86/kernel/cpu/resctrl/core.c | 8 +++-- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 2 +- arch/x86/kernel/cpu/resctrl/internal.h | 33 ++++++++++++++---- arch/x86/kernel/cpu/resctrl/monitor.c | 42 ++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 6 ++-- include/linux/resctrl.h | 2 ++ 6 files changed, 72 insertions(+), 21 deletions(-)arch/x86/kernel/cpu/resctrl/core.c +6
-2@@ -586,12 +586,16 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)if (r == &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl) {if (is_mbm_enabled () && cpu == d->mbm_work_cpu) {cancel_delayed_work (&d->mbm_over); -mbm_setup_overflow_handler (d,0 ); +/* +* temporary: exclude_cpu=-1 as this CPU has already +* been removed by cpumask_clear_cpu()d +*/ +mbm_setup_overflow_handler (d,0 ,RESCTRL_PICK_ANY_CPU ); }if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu &&has_busy_rmid (d)) {cancel_delayed_work (&d->cqm_limbo); -cqm_setup_limbo_handler (d,0 ); +cqm_setup_limbo_handler (d,0 ,RESCTRL_PICK_ANY_CPU ); } } }arch/x86/kernel/cpu/resctrl/ctrlmondata.c +1
-1@@ -552,7 +552,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,return ; } - cpu =cpumask_any_housekeeping (&d->cpu_mask); + cpu =cpumask_any_housekeeping (&d->cpu_mask,RESCTRL_PICK_ANY_CPU );/* * cpumask_any_housekeeping() prefers housekeeping CPUs, but arch/x86/kernel/cpu/resctrl/internal.h +26
-7@@ -61,19 +61,36 @@* cpumask_any_housekeeping() - Choose any CPU in @mask, preferring those that * aren't marked nohz_full * @mask: The mask to pick a CPU from. +* @exclude_cpu:The CPU to avoid picking. * -* Returns a CPU in @mask. If there are housekeeping CPUs that don't use -* nohz_full, these are preferred. +* Returns a CPU from @mask, but not @exclude_cpu. If there are housekeeping +* CPUs that don't use nohz_full, these are preferred. Pass +* RESCTRL_PICK_ANY_CPU to avoid excluding any CPUs. +* +* When a CPU is excluded, returns >= nr_cpu_ids if no CPUs are available. */ -static inline unsigned int cpumask_any_housekeeping (const struct cpumask *mask) +static inline unsigned int +cpumask_any_housekeeping (const struct cpumask *mask,int exclude_cpu) {unsigned int cpu, hk_cpu; - cpu =cpumask_any (mask); -if (!tick_nohz_full_cpu (cpu)) +if (exclude_cpu ==RESCTRL_PICK_ANY_CPU ) + cpu =cpumask_any (mask); +else + cpu =cpumask_any_but (mask, exclude_cpu); + +if (!IS_ENABLED (CONFIG_NO_HZ_FULL )) +return cpu; + +/* If the CPU picked isn't marked nohz_full, we're done */ +if (cpu <= nr_cpu_ids && !tick_nohz_full_cpu (cpu))return cpu; +/* Try to find a CPU that isn't nohz_full to use in preference */ hk_cpu =cpumask_nth_andnot (0 , mask, tick_nohz_full_mask); +if (hk_cpu == exclude_cpu) + hk_cpu =cpumask_nth_andnot (1 , mask, tick_nohz_full_mask); +if (hk_cpu < nr_cpu_ids) cpu = hk_cpu;@@ -575,11 +592,13 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_domain *d, struct rdtgroup *rdtgrp, int evtid, int first);void mbm_setup_overflow_handler (struct rdt_domain *dom, -unsigned long delay_ms); +unsigned long delay_ms, +int exclude_cpu);void mbm_handle_overflow (struct work_struct *work);void __init intel_rdt_mbm_apply_quirk (void );bool is_mba_sc (struct rdt_resource *r); -void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms); +void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms, +int exclude_cpu);void cqm_handle_limbo (struct work_struct *work);bool has_busy_rmid (struct rdt_domain *d);void __check_limbo (struct rdt_domain *d,bool force_free);arch/x86/kernel/cpu/resctrl/monitor.c +33
-9@@ -481,7 +481,8 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)* setup up the limbo worker. */ if (!has_busy_rmid (d)) -cqm_setup_limbo_handler (d,CQM_LIMBOCHECK_INTERVAL ); +cqm_setup_limbo_handler (d,CQM_LIMBOCHECK_INTERVAL , +RESCTRL_PICK_ANY_CPU );set_bit (idx, d->rmid_busy_llc); entry->busy++; }@@ -808,7 +809,8 @@ void cqm_handle_limbo(struct work_struct *work)__check_limbo (d, false);if (has_busy_rmid (d)) { - d->cqm_work_cpu =cpumask_any_housekeeping (&d->cpu_mask); + d->cqm_work_cpu =cpumask_any_housekeeping (&d->cpu_mask, +RESCTRL_PICK_ANY_CPU );schedule_delayed_work_on (d->cqm_work_cpu, &d->cqm_limbo, delay); }@@ -816,15 +818,25 @@ void cqm_handle_limbo(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms) +/** +* cqm_setup_limbo_handler() - Schedule the limbo handler to run for this +* domain. +* @dom: The domain the limbo handler should run for. +* @delay_ms: How far in the future the handler should run. +* @exclude_cpu: Which CPU the handler should not run on, +* RESCTRL_PICK_ANY_CPU to pick any CPU. +*/ +void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms, +int exclude_cpu) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu; - cpu =cpumask_any_housekeeping (&dom->cpu_mask); + cpu =cpumask_any_housekeeping (&dom->cpu_mask, exclude_cpu); dom->cqm_work_cpu = cpu; -schedule_delayed_work_on (cpu, &dom->cqm_limbo, delay); +if (cpu < nr_cpu_ids) +schedule_delayed_work_on (cpu, &dom->cqm_limbo, delay); }void mbm_handle_overflow (struct work_struct *work)@@ -862,14 +874,24 @@ void mbm_handle_overflow(struct work_struct *work)* Re-check for housekeeping CPUs. This allows the overflow handler to * move off a nohz_full CPU quickly. */ - d->mbm_work_cpu =cpumask_any_housekeeping (&d->cpu_mask); + d->mbm_work_cpu =cpumask_any_housekeeping (&d->cpu_mask, +RESCTRL_PICK_ANY_CPU );schedule_delayed_work_on (d->mbm_work_cpu, &d->mbm_over, delay); out_unlock:mutex_unlock (&rdtgroup_mutex); } -void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms) +/** +* mbm_setup_overflow_handler() - Schedule the overflow handler to run for this +* domain. +* @dom: The domain the overflow handler should run for. +* @delay_ms: How far in the future the handler should run. +* @exclude_cpu: Which CPU the handler should not run on, +* RESCTRL_PICK_ANY_CPU to pick any CPU. +*/ +void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms, +int exclude_cpu) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;@@ -880,9 +902,11 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)*/ if (!resctrl_mounted || !resctrl_arch_mon_capable ())return ; - cpu =cpumask_any_housekeeping (&dom->cpu_mask); + cpu =cpumask_any_housekeeping (&dom->cpu_mask, exclude_cpu); dom->mbm_work_cpu = cpu; -schedule_delayed_work_on (cpu, &dom->mbm_over, delay); + +if (cpu < nr_cpu_ids) +schedule_delayed_work_on (cpu, &dom->mbm_over, delay); }static int dom_data_init (struct rdt_resource *r)arch/x86/kernel/cpu/resctrl/rdtgroup.c +4
-2@@ -2683,7 +2683,8 @@ static int rdt_get_tree(struct fs_context *fc)if (is_mbm_enabled ()) { r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;list_for_each_entry (dom, &r->domains, list) -mbm_setup_overflow_handler (dom,MBM_OVERFLOW_INTERVAL ); +mbm_setup_overflow_handler (dom,MBM_OVERFLOW_INTERVAL , +RESCTRL_PICK_ANY_CPU ); } goto out;@@ -3994,7 +3995,8 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)if (is_mbm_enabled ()) {INIT_DELAYED_WORK (&d->mbm_over, mbm_handle_overflow); -mbm_setup_overflow_handler (d,MBM_OVERFLOW_INTERVAL ); +mbm_setup_overflow_handler (d,MBM_OVERFLOW_INTERVAL , +RESCTRL_PICK_ANY_CPU ); }if (is_llc_occupancy_enabled ())include/linux/resctrl.h +2
-0@@ -10,6 +10,8 @@#define RESCTRL_RESERVED_CLOSID 0#define RESCTRL_RESERVED_RMID 0 +#define RESCTRL_PICK_ANY_CPU -1 +#ifdef CONFIG_PROC_CPU_RESCTRL int proc_resctrl_show(struct seq_file *m,
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 21/24] x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but cpu
Hi Babu,
Quoted 13 lines · Babu
Quoted 12 lines · James Morse
When a CPU is taken offline resctrl may need to move the overflow or limbo handlers to run on a different CPU.
Once the offline callbacks have been split,
cqm_setup_limbo_handler()will be called while the CPU that is going offline is still present in thecpu_mask.Pass the CPU to exclude to
cqm_setup_limbo_handler()andmbm_setup_overflow_handler(). These functions can use a variant ofcpumask_any_but()when selecting the CPU. -1 is used to indicate no CPUs need excluding.
Quoted 4 lines
Quoted 2 lines
A subsequent patch moves these calls to be before CPUs have been removed, so this
exclude_cpusbehaviour is temporary.Reviewed-by Babu Moger <[email protected]>
Thanks!
James
James Morse
reviewerPATCH v7 22/24 x86/resctrl: Add CPU offline callback for resctrl work
In reply to James MorseThe resctrl architecture specific code may need to free a domain when a CPU goes offline, it also needs to reset the CPUs PQR_ASSOC register. Amongst other things, the resctrl filesystem code needs to clear this CPU from the cpu_mask of any control and monitor groups.
Currently this is all done in core.c and called from resctrl_offline_cpu(), making the split between architecture and filesystem code unclear.
Move the filesystem work to remove the CPU from the control and monitor groups into a filesystem helper called resctrl_offline_cpu(), and rename the one in core.c resctrl_arch_offline_cpu().
---
No changes since v6
arch/x86/kernel/cpu/resctrl/core.c | 25 +++++-------------------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 24 ++++++++++++++++++++++++ include/linux/resctrl.h | 1 + 3 files changed, 30 insertions(+), 20 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +5 -20
@@ -627,31 +627,15 @@ static int resctrl_arch_online_cpu(unsigned int cpu)return 0 ; } -static void clear_childcpus (struct rdtgroup *r,unsigned int cpu) +static int resctrl_arch_offline_cpu (unsigned int cpu) { -struct rdtgroup *cr; - -list_for_each_entry (cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) { -if (cpumask_test_and_clear_cpu (cpu, &cr->cpu_mask)) { -break ; - } - } -} - -static int resctrl_offline_cpu (unsigned int cpu) -{ -struct rdtgroup *rdtgrp;struct rdt_resource *r;mutex_lock (&rdtgroup_mutex); +resctrl_offline_cpu (cpu); +for_each_capable_rdt_resource (r)domain_remove_cpu (cpu, r); -list_for_each_entry (rdtgrp, &rdt_all_groups, rdtgroup_list) { -if (cpumask_test_and_clear_cpu (cpu, &rdtgrp->cpu_mask)) { -clear_childcpus (rdtgrp, cpu); -break ; - } - }clear_closid_rmid (cpu);mutex_unlock (&rdtgroup_mutex);
@@ -973,7 +957,8 @@ static int __init resctrl_late_init(void) state =cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"x86/resctrl/cat:online:" , - resctrl_arch_online_cpu, resctrl_offline_cpu); + resctrl_arch_online_cpu, + resctrl_arch_offline_cpu);if (state <0 )return state;
arch/x86/kernel/cpu/resctrl/rdtgroup.c +24 -0
@@ -4022,6 +4022,30 @@ void resctrl_online_cpu(unsigned int cpu)cpumask_set_cpu (cpu, &rdtgroup_default.cpu_mask); } +static void clear_childcpus (struct rdtgroup *r,unsigned int cpu) +{ +struct rdtgroup *cr; + +list_for_each_entry (cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) { +if (cpumask_test_and_clear_cpu (cpu, &cr->cpu_mask)) +break ; + } +} + +void resctrl_offline_cpu (unsigned int cpu) +{ +struct rdtgroup *rdtgrp; + +lockdep_assert_held (&rdtgroup_mutex); + +list_for_each_entry (rdtgrp, &rdt_all_groups, rdtgroup_list) { +if (cpumask_test_and_clear_cpu (cpu, &rdtgrp->cpu_mask)) { +clear_childcpus (rdtgrp, cpu); +break ; + } + } +} +/* * rdtgroup_init - rdtgroup initialization *
include/linux/resctrl.h +1 -0
@@ -226,6 +226,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d);void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d);void resctrl_online_cpu (unsigned int cpu); +void resctrl_offline_cpu (unsigned int cpu);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
Signature · 1 line
2.39.2
Re: [PATCH v7 22/24] x86/resctrl: Add CPU offline callback for resctrl work
Quoted 13 lines · James Morse
The resctrl architecture specific code may need to free a domain when a CPU goes offline, it also needs to reset the CPUs
PQR_ASSOCregister. Amongst other things, the resctrl filesystem code needs to clear this CPU from thecpu_maskof any control and monitor groups.Currently this is all done in
core.cand called fromresctrl_offline_cpu(), making the split between architecture and filesystem code unclear.Move the filesystem work to remove the CPU from the control and monitor groups into a filesystem helper called
resctrl_offline_cpu(), and rename the one incore.cresctrl_arch_offline_cpu().
Quoted 5 lines
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 104 lines
---
No changes since v6arch/x86/kernel/cpu/resctrl/core.c | 25 +++++-------------------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 24 ++++++++++++++++++++++++ include/linux/resctrl.h | 1 + 3 files changed, 30 insertions(+), 20 deletions(-)arch/x86/kernel/cpu/resctrl/core.c +5
-20@@ -627,31 +627,15 @@ static int resctrl_arch_online_cpu(unsigned int cpu)return 0 ; } -static void clear_childcpus (struct rdtgroup *r,unsigned int cpu) +static int resctrl_arch_offline_cpu (unsigned int cpu) { -struct rdtgroup *cr; - -list_for_each_entry (cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) { -if (cpumask_test_and_clear_cpu (cpu, &cr->cpu_mask)) { -break ; - } - } -} - -static int resctrl_offline_cpu (unsigned int cpu) -{ -struct rdtgroup *rdtgrp;struct rdt_resource *r;mutex_lock (&rdtgroup_mutex); +resctrl_offline_cpu (cpu); +for_each_capable_rdt_resource (r)domain_remove_cpu (cpu, r); -list_for_each_entry (rdtgrp, &rdt_all_groups, rdtgroup_list) { -if (cpumask_test_and_clear_cpu (cpu, &rdtgrp->cpu_mask)) { -clear_childcpus (rdtgrp, cpu); -break ; - } - }clear_closid_rmid (cpu);mutex_unlock (&rdtgroup_mutex);@@ -973,7 +957,8 @@ static int __init resctrl_late_init(void) state =cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"x86/resctrl/cat:online:" , - resctrl_arch_online_cpu, resctrl_offline_cpu); + resctrl_arch_online_cpu, + resctrl_arch_offline_cpu);if (state <0 )return state;arch/x86/kernel/cpu/resctrl/rdtgroup.c +24
-0@@ -4022,6 +4022,30 @@ void resctrl_online_cpu(unsigned int cpu)cpumask_set_cpu (cpu, &rdtgroup_default.cpu_mask); } +static void clear_childcpus (struct rdtgroup *r,unsigned int cpu) +{ +struct rdtgroup *cr; + +list_for_each_entry (cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) { +if (cpumask_test_and_clear_cpu (cpu, &cr->cpu_mask)) +break ; + } +} + +void resctrl_offline_cpu (unsigned int cpu) +{ +struct rdtgroup *rdtgrp; + +lockdep_assert_held (&rdtgroup_mutex); + +list_for_each_entry (rdtgrp, &rdt_all_groups, rdtgroup_list) { +if (cpumask_test_and_clear_cpu (cpu, &rdtgrp->cpu_mask)) { +clear_childcpus (rdtgrp, cpu); +break ; + } + } +} +/* * rdtgroup_init - rdtgroup initialization * include/linux/resctrl.h +1
-0@@ -226,6 +226,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d);void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d);void resctrl_online_cpu (unsigned int cpu); +void resctrl_offline_cpu (unsigned int cpu);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 22/24] x86/resctrl: Add CPU offline callback for resctrl work
Hi Babu,
Quoted 10 lines · Babu
Quoted 9 lines · James Morse
The resctrl architecture specific code may need to free a domain when a CPU goes offline, it also needs to reset the CPUs
PQR_ASSOCregister. Amongst other things, the resctrl filesystem code needs to clear this CPU from thecpu_maskof any control and monitor groups.Currently this is all done in
core.cand called fromresctrl_offline_cpu(), making the split between architecture and filesystem code unclear.
Quoted 5 lines
Quoted 3 lines
Move the filesystem work to remove the CPU from the control and monitor groups into a filesystem helper called
resctrl_offline_cpu(), and rename the one incore.cresctrl_arch_offline_cpu().Reviewed-by Babu Moger <[email protected]>
Thanks!
James
James Morse
reviewerPATCH v7 23/24 x86/resctrl: Move domain helper migration into resctrl_offline_cpu()
In reply to James MorseWhen a CPU is taken offline the resctrl filesystem code needs to check if it was the CPU nominated to perform the periodic overflow and limbo work. If so, another CPU needs to be chosen to do this work.
This is currently done in core.c, mixed in with the code that removes the CPU from the domain's mask, and potentially free()s the domain.
Move the migration of the overflow and limbo helpers into the filesystem code, into resctrl_offline_cpu(). As resctrl_offline_cpu() runs before the architecture code has removed the CPU from the domain mask, the callers need to be told which CPU is being removed, to avoid picking it as the new CPU. This uses the exclude_cpu feature previously added.
---
Changes since v5:
* Changed fir tree order of variables.
* Added mon-capable check for cpu offline.
No changes since v6
--- arch/x86/kernel/cpu/resctrl/core.c | 16 ---------------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 18 ++++++++++++++++++ 2 files changed, 18 insertions(+), 16 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +0 -16
@@ -582,22 +582,6 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)return ; } - -if (r == &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl) { -if (is_mbm_enabled () && cpu == d->mbm_work_cpu) { -cancel_delayed_work (&d->mbm_over); -/* -* temporary: exclude_cpu=-1 as this CPU has already -* been removed by cpumask_clear_cpu()d -*/ -mbm_setup_overflow_handler (d,0 ,RESCTRL_PICK_ANY_CPU ); - } -if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu && -has_busy_rmid (d)) { -cancel_delayed_work (&d->cqm_limbo); -cqm_setup_limbo_handler (d,0 ,RESCTRL_PICK_ANY_CPU ); - } - } }static void clear_closid_rmid (int cpu)
arch/x86/kernel/cpu/resctrl/rdtgroup.c +18 -0
@@ -4034,7 +4034,9 @@ static void clear_childcpus(struct rdtgroup *r, unsigned int cpu)void resctrl_offline_cpu (unsigned int cpu) { +struct rdt_resource *l3 = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rdtgroup *rdtgrp; +struct rdt_domain *d;lockdep_assert_held (&rdtgroup_mutex);
@@ -4044,6 +4046,22 @@ void resctrl_offline_cpu(unsigned int cpu)break ; } } + +if (!l3->mon_capable) +return ; + + d =get_domain_from_cpu (cpu, l3); +if (d) { +if (is_mbm_enabled () && cpu == d->mbm_work_cpu) { +cancel_delayed_work (&d->mbm_over); +mbm_setup_overflow_handler (d,0 , cpu); + } +if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu && +has_busy_rmid (d)) { +cancel_delayed_work (&d->cqm_limbo); +cqm_setup_limbo_handler (d,0 , cpu); + } + } }/*
Signature · 1 line
2.39.2
Re: [PATCH v7 23/24] x86/resctrl: Move domain helper migration into resctrl_offline_cpu()
Quoted 14 lines · James Morse
When a CPU is taken offline the resctrl filesystem code needs to check if it was the CPU nominated to perform the periodic overflow and limbo work. If so, another CPU needs to be chosen to do this work.
This is currently done in
core.c, mixed in with the code that removes the CPU from the domain's mask, and potentially free()s the domain.Move the migration of the overflow and limbo helpers into the filesystem code, into
resctrl_offline_cpu(). Asresctrl_offline_cpu()runs before the architecture code has removed the CPU from the domain mask, the callers need to be told which CPU is being removed, to avoid picking it as the new CPU. This uses theexclude_cpufeature previously added.
Quoted 5 lines
Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Reviewed-by Reinette Chatre <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 75 lines
---
Changes since v5:
* Changed fir tree order of variables.
* Added mon-capable check for cpu offline.No changes since v6
--- arch/x86/kernel/cpu/resctrl/core.c | 16 ---------------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 18 ++++++++++++++++++ 2 files changed, 18 insertions(+), 16 deletions(-)arch/x86/kernel/cpu/resctrl/core.c +0
-16@@ -582,22 +582,6 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)return ; } - -if (r == &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl) { -if (is_mbm_enabled () && cpu == d->mbm_work_cpu) { -cancel_delayed_work (&d->mbm_over); -/* -* temporary: exclude_cpu=-1 as this CPU has already -* been removed by cpumask_clear_cpu()d -*/ -mbm_setup_overflow_handler (d,0 ,RESCTRL_PICK_ANY_CPU ); - } -if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu && -has_busy_rmid (d)) { -cancel_delayed_work (&d->cqm_limbo); -cqm_setup_limbo_handler (d,0 ,RESCTRL_PICK_ANY_CPU ); - } - } }static void clear_closid_rmid (int cpu)arch/x86/kernel/cpu/resctrl/rdtgroup.c +18
-0@@ -4034,7 +4034,9 @@ static void clear_childcpus(struct rdtgroup *r, unsigned int cpu)void resctrl_offline_cpu (unsigned int cpu) { +struct rdt_resource *l3 = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rdtgroup *rdtgrp; +struct rdt_domain *d;lockdep_assert_held (&rdtgroup_mutex);@@ -4044,6 +4046,22 @@ void resctrl_offline_cpu(unsigned int cpu)break ; } } + +if (!l3->mon_capable) +return ; + + d =get_domain_from_cpu (cpu, l3); +if (d) { +if (is_mbm_enabled () && cpu == d->mbm_work_cpu) { +cancel_delayed_work (&d->mbm_over); +mbm_setup_overflow_handler (d,0 , cpu); + } +if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu && +has_busy_rmid (d)) { +cancel_delayed_work (&d->cqm_limbo); +cqm_setup_limbo_handler (d,0 , cpu); + } + } }/*
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerPATCH v7 24/24 x86/resctrl: Separate arch and fs resctrl locks
In reply to James Morseresctrl has one mutex that is taken by the architecture specific code, and the filesystem parts. The two interact via cpuhp, where the architecture code updates the domain list. Filesystem handlers that walk the domains list should not run concurrently with the cpuhp callback modifying the list.
Exposing a lock from the filesystem code means the interface is not cleanly defined, and creates the possibility of cross-architecture lock ordering headaches. The interaction only exists so that certain filesystem paths are serialised against CPU hotplug. The CPU hotplug code already has a mechanism to do this using cpus_read_lock().
MPAM's monitors have an overflow interrupt, so it needs to be possible to walk the domains list in irq context. RCU is ideal for this, but some paths need to be able to sleep to allocate memory.
Because resctrl_{on,off}line_cpu() take the rdtgroup_mutex as part of a cpuhp callback, cpus_read_lock() must always be taken first. rdtgroup_schemata_write() already does this.
Most of the filesystem code's domain list walkers are currently protected by the rdtgroup_mutex taken in rdtgroup_kn_lock_live(). The exceptions are rdt_bit_usage_show() and the mon_config helpers which take the lock directly.
Make the domain list protected by RCU. An architecture-specific lock prevents concurrent writers. rdt_bit_usage_show() could walk the domain list using RCU, but to keep all the filesystem operations the same, this is changed to call cpus_read_lock(). The mon_config helpers send multiple IPIs, take the cpus_read_lock() in these cases.
The other filesystem list walkers need to be able to sleep. Add cpus_read_lock() to rdtgroup_kn_lock_live() so that the cpuhp callbacks can't be invoked when file system operations are occurring.
Add lockdep_assert_cpus_held() in the cases where the rdtgroup_kn_lock_live() call isn't obvious.
Resctrl's domain online/offline calls now need to take the rdtgroup_mutex themselves.
---
Changes since v2:
* Reworded a comment,
* Added a lockdep assertion
* Moved clear_closid_rmid() outside the locked region of cpu online/offline
Changes since v3:
* Added a header include
Changes since v5:
* Made rdt_bit_usage_show() take the cpus_read_lock() instead of using RCU.
Changes since v6:
* Added lockdep_is_cpus_held() to get_domain_from_cpu().
* Added cpus_read_lock() around overflow and limbo handlers.
--- arch/x86/kernel/cpu/resctrl/core.c | 44 +++++++++++---- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 15 ++++- arch/x86/kernel/cpu/resctrl/monitor.c | 8 +++ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 3 + arch/x86/kernel/cpu/resctrl/rdtgroup.c | 68 ++++++++++++++++++----- include/linux/resctrl.h | 2 +- 6 files changed, 112 insertions(+), 28 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +34 -10
@@ -16,6 +16,7 @@#define pr_fmt (fmt) "resctrl: " fmt +#include <linux/cpu.h> #include <linux/slab.h> #include <linux/err.h> #include <linux/cacheinfo.h>
@@ -25,8 +26,15 @@#include <asm/resctrl.h> #include "internal.h" -/* Mutex to protect rdtgroup access. */ -DEFINE_MUTEX (rdtgroup_mutex); +/* +* rdt_domain structures are kfree()d when their last CPU goes offline, +* and allocated when the first CPU in a new domain comes online. +* The rdt_resource's domain list is updated when this happens. Readers of +* the domain list must either take cpus_read_lock(), or rely on an RCU +* read-side critical section, to avoid observing concurrent modification. +* All writers take this mutex: +*/ +static DEFINE_MUTEX (domain_list_lock);/* * The cached resctrl_pqr_state is strictly per CPU and can never be
@@ -356,6 +364,15 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) {struct rdt_domain *d; +/* +* Walking r->domains, ensure it can't race with cpuhp. +* Because this is called via IPI by rdt_ctrl_update(), assertions +* about locks this thread holds will lead to false positives. Check +* someone is holding the CPUs lock. +*/ +if (IS_ENABLED (CONFIG_LOCKDEP )) +lockdep_is_cpus_held (); +list_for_each_entry (d, &r->domains, list) {/* Find the domain that contains this CPU */ if (cpumask_test_cpu (cpu, &d->cpu_mask))
@@ -512,6 +529,8 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)struct rdt_domain *d;int err; +lockdep_assert_held (&domain_list_lock); + d =rdt_find_domain (r, id, &add_pos);if (IS_ERR (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
@@ -545,11 +564,12 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; } -list_add_tail (&d->list, add_pos); +list_add_tail_rcu (&d->list, add_pos); err =resctrl_online_domain (r, d);if (err) { -list_del (&d->list); +list_del_rcu (&d->list); +synchronize_rcu ();domain_free (hw_dom); } }
@@ -560,6 +580,8 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)struct rdt_hw_domain *hw_dom;struct rdt_domain *d; +lockdep_assert_held (&domain_list_lock); + d =rdt_find_domain (r, id,NULL );if (IS_ERR_OR_NULL (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
@@ -570,7 +592,8 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)cpumask_clear_cpu (cpu, &d->cpu_mask);if (cpumask_empty (&d->cpu_mask)) {resctrl_offline_domain (r, d); -list_del (&d->list); +list_del_rcu (&d->list); +synchronize_rcu ();/* * rdt_domain "d" is going to be freed below, so clear
@@ -600,13 +623,13 @@ static int resctrl_arch_online_cpu(unsigned int cpu) {struct rdt_resource *r; -mutex_lock (&rdtgroup_mutex); +mutex_lock (&domain_list_lock);for_each_capable_rdt_resource (r)domain_add_cpu (cpu, r); -clear_closid_rmid (cpu); +mutex_unlock (&domain_list_lock); +clear_closid_rmid (cpu);resctrl_online_cpu (cpu); -mutex_unlock (&rdtgroup_mutex);return 0 ; }
@@ -615,13 +638,14 @@ static int resctrl_arch_offline_cpu(unsigned int cpu) {struct rdt_resource *r; -mutex_lock (&rdtgroup_mutex);resctrl_offline_cpu (cpu); +mutex_lock (&domain_list_lock);for_each_capable_rdt_resource (r)domain_remove_cpu (cpu, r); +mutex_unlock (&domain_list_lock); +clear_closid_rmid (cpu); -mutex_unlock (&rdtgroup_mutex);return 0 ; }
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +12 -3
@@ -212,6 +212,9 @@ static int parse_line(char *line, struct resctrl_schema *s,struct rdt_domain *d;unsigned long dom_id; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP && (r->rid ==RDT_RESOURCE_MBA || r->rid ==RDT_RESOURCE_SMBA )) {rdt_last_cmd_puts ("Cannot pseudo-lock MBA resource\n" );
@@ -316,6 +319,9 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)struct rdt_domain *d;u32 idx; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +if (!zalloc_cpumask_var (&cpu_mask,GFP_KERNEL ))return -ENOMEM ;
@@ -381,11 +387,9 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,return -EINVAL ; buf[nbytes -1 ] ='\0' ; -cpus_read_lock (); rdtgrp =rdtgroup_kn_lock_live (of->kn);if (!rdtgrp) {rdtgroup_kn_unlock (of->kn); -cpus_read_unlock ();return -ENOENT ; }rdt_last_cmd_clear ();
@@ -447,7 +451,6 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, out:rdt_staged_configs_clear ();rdtgroup_kn_unlock (of->kn); -cpus_read_unlock ();return ret ?: nbytes; }
@@ -467,6 +470,9 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clobool sep = false;u32 ctrl_val; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +seq_printf (s,"%*s:" , max_name_width, schema->name);list_for_each_entry (dom, &r->domains, list) {if (sep)
@@ -537,6 +543,9 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, {int cpu; +/* When picking a CPU from cpu_mask, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +/* * Setup the parameters to pass to mon_event_count() to read the data. */
arch/x86/kernel/cpu/resctrl/monitor.c +8 -0
@@ -15,6 +15,7 @@* Software Developer Manual June 2016, volume 3, section 17.17. */ +#include <linux/cpu.h> #include <linux/module.h> #include <linux/sizes.h> #include <linux/slab.h>
@@ -472,6 +473,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)lockdep_assert_held (&rdtgroup_mutex); +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); + idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); entry->busy =0 ;
@@ -801,6 +805,7 @@ void cqm_handle_limbo(struct work_struct *work)struct rdt_resource *r;struct rdt_domain *d; +cpus_read_lock ();mutex_lock (&rdtgroup_mutex); r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;
@@ -816,6 +821,7 @@ void cqm_handle_limbo(struct work_struct *work) }mutex_unlock (&rdtgroup_mutex); +cpus_read_unlock (); }/**
@@ -847,6 +853,7 @@ void mbm_handle_overflow(struct work_struct *work)struct rdt_resource *r;struct rdt_domain *d; +cpus_read_lock ();mutex_lock (&rdtgroup_mutex);/*
@@ -880,6 +887,7 @@ void mbm_handle_overflow(struct work_struct *work) out_unlock:mutex_unlock (&rdtgroup_mutex); +cpus_read_unlock (); }/**
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3 -0
@@ -844,6 +844,9 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)struct rdt_domain *d_i;bool ret = false; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +if (!zalloc_cpumask_var (&cpu_with_psl,GFP_KERNEL ))return true;
arch/x86/kernel/cpu/resctrl/rdtgroup.c +54 -14
225 lines not on this page · raw message
include/linux/resctrl.h +1 -1
@@ -159,7 +159,7 @@ struct resctrl_schema;* @cache_level: Which cache level defines scope of this resource * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. -* @domains: All domains for this resource +* @domains: RCU list of all domains for this resource * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent.
Signature · 1 line
2.39.2
Re: [PATCH v7 24/24] x86/resctrl: Separate arch and fs resctrl locks
Hi James,
Quoted 40 lines · James Morse
resctrl has one mutex that is taken by the architecture specific code, and the filesystem parts. The two interact via cpuhp, where the architecture code updates the domain list. Filesystem handlers that walk the domains list should not run concurrently with the cpuhp callback modifying the list.
Exposing a lock from the filesystem code means the interface is not cleanly defined, and creates the possibility of cross-architecture lock ordering headaches. The interaction only exists so that certain filesystem paths are serialised against CPU hotplug. The CPU hotplug code already has a mechanism to do this using
cpus_read_lock().MPAM's monitors have an overflow interrupt, so it needs to be possible to walk the domains list in irq context. RCU is ideal for this, but some paths need to be able to sleep to allocate memory.
Because resctrl_{on,off}line_cpu() take the
rdtgroup_mutexas part of a cpuhp callback,cpus_read_lock()must always be taken first.rdtgroup_schemata_write()already does this.Most of the filesystem code's domain list walkers are currently protected by the
rdtgroup_mutextaken inrdtgroup_kn_lock_live(). The exceptions arerdt_bit_usage_show()and themon_confighelpers which take the lock directly.Make the domain list protected by RCU. An architecture-specific lock prevents concurrent writers.
rdt_bit_usage_show()could walk the domain list using RCU, but to keep all the filesystem operations the same, this is changed to callcpus_read_lock(). Themon_confighelpers send multiple IPIs, take thecpus_read_lock()in these cases.The other filesystem list walkers need to be able to sleep. Add
cpus_read_lock()tordtgroup_kn_lock_live()so that the cpuhp callbacks can't be invoked when file system operations are occurring.Add
lockdep_assert_cpus_held()in the cases where therdtgroup_kn_lock_live()call isn't obvious.
Quoted 8 lines
Resctrl's domain online/offline calls now need to take the
rdtgroup_mutexthemselves.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>---
Reinette
James Morse
reviewerRe: [PATCH v7 24/24] x86/resctrl: Separate arch and fs resctrl locks
Hi Reinette,
Quoted 38 lines · Reinette Chatre
Quoted 37 lines · James Morse
resctrl has one mutex that is taken by the architecture specific code, and the filesystem parts. The two interact via cpuhp, where the architecture code updates the domain list. Filesystem handlers that walk the domains list should not run concurrently with the cpuhp callback modifying the list.
Exposing a lock from the filesystem code means the interface is not cleanly defined, and creates the possibility of cross-architecture lock ordering headaches. The interaction only exists so that certain filesystem paths are serialised against CPU hotplug. The CPU hotplug code already has a mechanism to do this using
cpus_read_lock().MPAM's monitors have an overflow interrupt, so it needs to be possible to walk the domains list in irq context. RCU is ideal for this, but some paths need to be able to sleep to allocate memory.
Because resctrl_{on,off}line_cpu() take the
rdtgroup_mutexas part of a cpuhp callback,cpus_read_lock()must always be taken first.rdtgroup_schemata_write()already does this.Most of the filesystem code's domain list walkers are currently protected by the
rdtgroup_mutextaken inrdtgroup_kn_lock_live(). The exceptions arerdt_bit_usage_show()and themon_confighelpers which take the lock directly.Make the domain list protected by RCU. An architecture-specific lock prevents concurrent writers.
rdt_bit_usage_show()could walk the domain list using RCU, but to keep all the filesystem operations the same, this is changed to callcpus_read_lock(). Themon_confighelpers send multiple IPIs, take thecpus_read_lock()in these cases.The other filesystem list walkers need to be able to sleep. Add
cpus_read_lock()tordtgroup_kn_lock_live()so that the cpuhp callbacks can't be invoked when file system operations are occurring.
Quoted 7 lines
Quoted 5 lines
Add
lockdep_assert_cpus_held()in the cases where therdtgroup_kn_lock_live()call isn't obvious.Resctrl's domain online/offline calls now need to take the
rdtgroup_mutexthemselves.Reviewed-by Reinette Chatre <[email protected]>
Thanks!
James
Re: [PATCH v7 24/24] x86/resctrl: Separate arch and fs resctrl locks
In reply to James MorseQuoted 40 lines · James Morse
resctrl has one mutex that is taken by the architecture specific code, and the filesystem parts. The two interact via cpuhp, where the architecture code updates the domain list. Filesystem handlers that walk the domains list should not run concurrently with the cpuhp callback modifying the list.
Exposing a lock from the filesystem code means the interface is not cleanly defined, and creates the possibility of cross-architecture lock ordering headaches. The interaction only exists so that certain filesystem paths are serialised against CPU hotplug. The CPU hotplug code already has a mechanism to do this using
cpus_read_lock().MPAM's monitors have an overflow interrupt, so it needs to be possible to walk the domains list in irq context. RCU is ideal for this, but some paths need to be able to sleep to allocate memory.
Because resctrl_{on,off}line_cpu() take the
rdtgroup_mutexas part of a cpuhp callback,cpus_read_lock()must always be taken first.rdtgroup_schemata_write()already does this.Most of the filesystem code's domain list walkers are currently protected by the
rdtgroup_mutextaken inrdtgroup_kn_lock_live(). The exceptions arerdt_bit_usage_show()and themon_confighelpers which take the lock directly.Make the domain list protected by RCU. An architecture-specific lock prevents concurrent writers.
rdt_bit_usage_show()could walk the domain list using RCU, but to keep all the filesystem operations the same, this is changed to callcpus_read_lock(). Themon_confighelpers send multiple IPIs, take thecpus_read_lock()in these cases.The other filesystem list walkers need to be able to sleep. Add
cpus_read_lock()tordtgroup_kn_lock_live()so that the cpuhp callbacks can't be invoked when file system operations are occurring.Add
lockdep_assert_cpus_held()in the cases where therdtgroup_kn_lock_live()call isn't obvious.
Quoted 7 lines
Resctrl's domain online/offline calls now need to take the
rdtgroup_mutexthemselves.Tested-by Shaopeng Tan <[email protected]>Tested-by Peter Newman <[email protected]>Reviewed-by Shaopeng Tan <[email protected]>Signed-off-by James Morse <[email protected]>
Quoted 546 lines
---
Changes since v2:
* Reworded a comment,
* Added a lockdep assertion
* Movedclear_closid_rmid()outside the locked region of cpu online/offlineChanges since v3:
* Added a header includeChanges since v5:
* Maderdt_bit_usage_show()take thecpus_read_lock()instead of using RCU.Changes since v6:
* Addedlockdep_is_cpus_held()toget_domain_from_cpu().
* Addedcpus_read_lock()around overflow and limbo handlers.--- arch/x86/kernel/cpu/resctrl/core.c | 44 +++++++++++---- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 15 ++++- arch/x86/kernel/cpu/resctrl/monitor.c | 8 +++ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 3 + arch/x86/kernel/cpu/resctrl/rdtgroup.c | 68 ++++++++++++++++++----- include/linux/resctrl.h | 2 +- 6 files changed, 112 insertions(+), 28 deletions(-)arch/x86/kernel/cpu/resctrl/core.c +34
-10@@ -16,6 +16,7 @@#define pr_fmt (fmt) "resctrl: " fmt +#include <linux/cpu.h> #include <linux/slab.h> #include <linux/err.h> #include <linux/cacheinfo.h> @@ -25,8 +26,15 @@#include <asm/resctrl.h> #include "internal.h" -/* Mutex to protect rdtgroup access. */ -DEFINE_MUTEX (rdtgroup_mutex); +/* +* rdt_domain structures are kfree()d when their last CPU goes offline, +* and allocated when the first CPU in a new domain comes online. +* The rdt_resource's domain list is updated when this happens. Readers of +* the domain list must either take cpus_read_lock(), or rely on an RCU +* read-side critical section, to avoid observing concurrent modification. +* All writers take this mutex: +*/ +static DEFINE_MUTEX (domain_list_lock);/* * The cached resctrl_pqr_state is strictly per CPU and can never be @@ -356,6 +364,15 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r) {struct rdt_domain *d; +/* +* Walking r->domains, ensure it can't race with cpuhp. +* Because this is called via IPI by rdt_ctrl_update(), assertions +* about locks this thread holds will lead to false positives. Check +* someone is holding the CPUs lock. +*/ +if (IS_ENABLED (CONFIG_LOCKDEP )) +lockdep_is_cpus_held (); +list_for_each_entry (d, &r->domains, list) {/* Find the domain that contains this CPU */ if (cpumask_test_cpu (cpu, &d->cpu_mask))@@ -512,6 +529,8 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)struct rdt_domain *d;int err; +lockdep_assert_held (&domain_list_lock); + d =rdt_find_domain (r, id, &add_pos);if (IS_ERR (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);@@ -545,11 +564,12 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; } -list_add_tail (&d->list, add_pos); +list_add_tail_rcu (&d->list, add_pos); err =resctrl_online_domain (r, d);if (err) { -list_del (&d->list); +list_del_rcu (&d->list); +synchronize_rcu ();domain_free (hw_dom); } }@@ -560,6 +580,8 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)struct rdt_hw_domain *hw_dom;struct rdt_domain *d; +lockdep_assert_held (&domain_list_lock); + d =rdt_find_domain (r, id,NULL );if (IS_ERR_OR_NULL (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);@@ -570,7 +592,8 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)cpumask_clear_cpu (cpu, &d->cpu_mask);if (cpumask_empty (&d->cpu_mask)) {resctrl_offline_domain (r, d); -list_del (&d->list); +list_del_rcu (&d->list); +synchronize_rcu ();/* * rdt_domain "d" is going to be freed below, so clear @@ -600,13 +623,13 @@ static int resctrl_arch_online_cpu(unsigned int cpu) {struct rdt_resource *r; -mutex_lock (&rdtgroup_mutex); +mutex_lock (&domain_list_lock);for_each_capable_rdt_resource (r)domain_add_cpu (cpu, r); -clear_closid_rmid (cpu); +mutex_unlock (&domain_list_lock); +clear_closid_rmid (cpu);resctrl_online_cpu (cpu); -mutex_unlock (&rdtgroup_mutex);return 0 ; }@@ -615,13 +638,14 @@ static int resctrl_arch_offline_cpu(unsigned int cpu) {struct rdt_resource *r; -mutex_lock (&rdtgroup_mutex);resctrl_offline_cpu (cpu); +mutex_lock (&domain_list_lock);for_each_capable_rdt_resource (r)domain_remove_cpu (cpu, r); +mutex_unlock (&domain_list_lock); +clear_closid_rmid (cpu); -mutex_unlock (&rdtgroup_mutex);return 0 ; }arch/x86/kernel/cpu/resctrl/ctrlmondata.c +12
-3@@ -212,6 +212,9 @@ static int parse_line(char *line, struct resctrl_schema *s,struct rdt_domain *d;unsigned long dom_id; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP && (r->rid ==RDT_RESOURCE_MBA || r->rid ==RDT_RESOURCE_SMBA )) {rdt_last_cmd_puts ("Cannot pseudo-lock MBA resource\n" );@@ -316,6 +319,9 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)struct rdt_domain *d;u32 idx; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +if (!zalloc_cpumask_var (&cpu_mask,GFP_KERNEL ))return -ENOMEM ;@@ -381,11 +387,9 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,return -EINVAL ; buf[nbytes -1 ] ='\0' ; -cpus_read_lock (); rdtgrp =rdtgroup_kn_lock_live (of->kn);if (!rdtgrp) {rdtgroup_kn_unlock (of->kn); -cpus_read_unlock ();return -ENOENT ; }rdt_last_cmd_clear ();@@ -447,7 +451,6 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, out:rdt_staged_configs_clear ();rdtgroup_kn_unlock (of->kn); -cpus_read_unlock ();return ret ?: nbytes; }@@ -467,6 +470,9 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clobool sep = false;u32 ctrl_val; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +seq_printf (s,"%*s:" , max_name_width, schema->name);list_for_each_entry (dom, &r->domains, list) {if (sep)@@ -537,6 +543,9 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, {int cpu; +/* When picking a CPU from cpu_mask, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +/* * Setup the parameters to pass to mon_event_count() to read the data. */ arch/x86/kernel/cpu/resctrl/monitor.c +8
-0@@ -15,6 +15,7 @@* Software Developer Manual June 2016, volume 3, section 17.17. */ +#include <linux/cpu.h> #include <linux/module.h> #include <linux/sizes.h> #include <linux/slab.h> @@ -472,6 +473,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)lockdep_assert_held (&rdtgroup_mutex); +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); + idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); entry->busy =0 ;@@ -801,6 +805,7 @@ void cqm_handle_limbo(struct work_struct *work)struct rdt_resource *r;struct rdt_domain *d; +cpus_read_lock ();mutex_lock (&rdtgroup_mutex); r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;@@ -816,6 +821,7 @@ void cqm_handle_limbo(struct work_struct *work) }mutex_unlock (&rdtgroup_mutex); +cpus_read_unlock (); }/** @@ -847,6 +853,7 @@ void mbm_handle_overflow(struct work_struct *work)struct rdt_resource *r;struct rdt_domain *d; +cpus_read_lock ();mutex_lock (&rdtgroup_mutex);/* @@ -880,6 +887,7 @@ void mbm_handle_overflow(struct work_struct *work) out_unlock:mutex_unlock (&rdtgroup_mutex); +cpus_read_unlock (); }/** arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3
-0@@ -844,6 +844,9 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)struct rdt_domain *d_i;bool ret = false; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +if (!zalloc_cpumask_var (&cpu_with_psl,GFP_KERNEL ))return true;arch/x86/kernel/cpu/resctrl/rdtgroup.c +54
-14@@ -35,6 +35,10 @@DEFINE_STATIC_KEY_FALSE (rdt_enable_key);DEFINE_STATIC_KEY_FALSE (rdt_mon_enable_key);DEFINE_STATIC_KEY_FALSE (rdt_alloc_enable_key); + +/* Mutex to protect rdtgroup access. */ +DEFINE_MUTEX (rdtgroup_mutex); +static struct kernfs_root *rdt_root;struct rdtgroup rdtgroup_default;LIST_HEAD (rdt_all_groups);@@ -1014,6 +1018,7 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of,bool sep = false;u32 ctrl_val; +cpus_read_lock ();mutex_lock (&rdtgroup_mutex); hw_shareable = r->cache.shareable_bits;list_for_each_entry (dom, &r->domains, list) {@@ -1074,6 +1079,7 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of, }seq_putc (seq,'\n' );mutex_unlock (&rdtgroup_mutex); +cpus_read_unlock ();return 0 ; }@@ -1329,6 +1335,9 @@ static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp)struct rdt_domain *d;u32 ctrl; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +list_for_each_entry (s, &resctrl_schema_all, list) { r = s->res;if (r->rid ==RDT_RESOURCE_MBA || r->rid ==RDT_RESOURCE_SMBA )@@ -1593,6 +1602,7 @@ static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtidstruct rdt_domain *dom;bool sep = false; +cpus_read_lock ();mutex_lock (&rdtgroup_mutex);list_for_each_entry (dom, &r->domains, list) {@@ -1609,6 +1619,7 @@ static int mbm_config_show(struct seq_file *s, struct rdt_resource *r, u32 evtidseq_puts (s,"\n" );mutex_unlock (&rdtgroup_mutex); +cpus_read_unlock ();return 0 ; }@@ -1700,6 +1711,9 @@ static int mon_config_write(struct rdt_resource *r, char *tok, u32 evtid)struct rdt_domain *d;int ret =0 ; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); + next:if (!tok || tok[0 ] =='\0' )return 0 ;@@ -1741,6 +1755,7 @@ static ssize_t mbm_total_bytes_config_write(struct kernfs_open_file *of,if (nbytes ==0 || buf[nbytes -1 ] !='\n' )return -EINVAL ; +cpus_read_lock ();mutex_lock (&rdtgroup_mutex);rdt_last_cmd_clear ();@@ -1750,6 +1765,7 @@ static ssize_t mbm_total_bytes_config_write(struct kernfs_open_file *of, ret =mon_config_write (r, buf,QOS_L3_MBM_TOTAL_EVENT_ID );mutex_unlock (&rdtgroup_mutex); +cpus_read_unlock ();return ret ?: nbytes; }@@ -1765,6 +1781,7 @@ static ssize_t mbm_local_bytes_config_write(struct kernfs_open_file *of,if (nbytes ==0 || buf[nbytes -1 ] !='\n' )return -EINVAL ; +cpus_read_lock ();mutex_lock (&rdtgroup_mutex);rdt_last_cmd_clear ();@@ -1774,6 +1791,7 @@ static ssize_t mbm_local_bytes_config_write(struct kernfs_open_file *of, ret =mon_config_write (r, buf,QOS_L3_MBM_LOCAL_EVENT_ID );mutex_unlock (&rdtgroup_mutex); +cpus_read_unlock ();return ret ?: nbytes; }@@ -2250,6 +2268,9 @@ static int set_cache_qos_cfg(int level, bool enable)struct rdt_domain *d;int cpu; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +if (level ==RDT_RESOURCE_L3 ) update = l3_qos_cfg_update;else if (level ==RDT_RESOURCE_L2 )@@ -2449,6 +2470,7 @@ struct rdtgroup *rdtgroup_kn_lock_live(struct kernfs_node *kn)rdtgroup_kn_get (rdtgrp, kn); +cpus_read_lock ();mutex_lock (&rdtgroup_mutex);/* Was this group deleted while we waited? */ @@ -2466,6 +2488,8 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn)return ;mutex_unlock (&rdtgroup_mutex); +cpus_read_unlock (); +rdtgroup_kn_put (rdtgrp, kn); }@@ -2798,6 +2822,9 @@ static int reset_all_ctrls(struct rdt_resource *r)struct rdt_domain *d;int i; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +if (!zalloc_cpumask_var (&cpu_mask,GFP_KERNEL ))return -ENOMEM ;@@ -3082,6 +3109,9 @@ static int mkdir_mondata_subdir_alldom(struct kernfs_node *parent_kn,struct rdt_domain *dom;int ret; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +list_for_each_entry (dom, &r->domains, list) { ret =mkdir_mondata_subdir (parent_kn, dom, r, prgrp);if (ret)@@ -3912,13 +3942,13 @@ static void domain_destroy_mon_state(struct rdt_domain *d)void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d) { -lockdep_assert_held (&rdtgroup_mutex); +mutex_lock (&rdtgroup_mutex);if (supports_mba_mbps () && r->rid ==RDT_RESOURCE_MBA )mba_sc_domain_destroy (r, d);if (!r->mon_capable) -return ; + goto out_unlock;/* * If resctrl is mounted, remove all the @@ -3943,6 +3973,9 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d) }domain_destroy_mon_state (d); + +out_unlock: +mutex_unlock (&rdtgroup_mutex); }static int domain_setup_mon_state (struct rdt_resource *r,struct rdt_domain *d)@@ -3978,20 +4011,22 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d)int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d) { -int err; +int err =0 ; -lockdep_assert_held (&rdtgroup_mutex); +mutex_lock (&rdtgroup_mutex); -if (supports_mba_mbps () && r->rid ==RDT_RESOURCE_MBA ) +if (supports_mba_mbps () && r->rid ==RDT_RESOURCE_MBA ) {/* RDT_RESOURCE_MBA is never mon_capable */ -return mba_sc_domain_allocate (r, d); + err =mba_sc_domain_allocate (r, d); + goto out_unlock; + }if (!r->mon_capable) -return 0 ; + goto out_unlock; err =domain_setup_mon_state (r, d);if (err) -return err; + goto out_unlock;if (is_mbm_enabled ()) {INIT_DELAYED_WORK (&d->mbm_over, mbm_handle_overflow);@@ -4011,15 +4046,18 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)if (resctrl_mounted &&resctrl_arch_mon_capable ())mkdir_mondata_subdir_allrdtgrp (r, d); -return 0 ; +out_unlock: +mutex_unlock (&rdtgroup_mutex); + +return err; }void resctrl_online_cpu (unsigned int cpu) { -lockdep_assert_held (&rdtgroup_mutex); - +mutex_lock (&rdtgroup_mutex);/* The CPU is set in default rdtgroup after online. */ cpumask_set_cpu (cpu, &rdtgroup_default.cpu_mask); +mutex_unlock (&rdtgroup_mutex); }static void clear_childcpus (struct rdtgroup *r,unsigned int cpu)@@ -4038,8 +4076,7 @@ void resctrl_offline_cpu(unsigned int cpu)struct rdtgroup *rdtgrp;struct rdt_domain *d; -lockdep_assert_held (&rdtgroup_mutex); - +mutex_lock (&rdtgroup_mutex);list_for_each_entry (rdtgrp, &rdt_all_groups, rdtgroup_list) {if (cpumask_test_and_clear_cpu (cpu, &rdtgrp->cpu_mask)) {clear_childcpus (rdtgrp, cpu);@@ -4048,7 +4085,7 @@ void resctrl_offline_cpu(unsigned int cpu) }if (!l3->mon_capable) -return ; + goto out_unlock; d =get_domain_from_cpu (cpu, l3);if (d) {@@ -4062,6 +4099,9 @@ void resctrl_offline_cpu(unsigned int cpu)cqm_setup_limbo_handler (d,0 , cpu); } } + +out_unlock: +mutex_unlock (&rdtgroup_mutex); }/* include/linux/resctrl.h +1
-1@@ -159,7 +159,7 @@ struct resctrl_schema;* @cache_level: Which cache level defines scope of this resource * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. -* @domains: All domains for this resource +* @domains: RCU list of all domains for this resource * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent.
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerRe: [PATCH v7 24/24] x86/resctrl: Separate arch and fs resctrl locks
Hi Babu,
Quoted 38 lines · Babu
Quoted 37 lines · James Morse
resctrl has one mutex that is taken by the architecture specific code, and the filesystem parts. The two interact via cpuhp, where the architecture code updates the domain list. Filesystem handlers that walk the domains list should not run concurrently with the cpuhp callback modifying the list.
Exposing a lock from the filesystem code means the interface is not cleanly defined, and creates the possibility of cross-architecture lock ordering headaches. The interaction only exists so that certain filesystem paths are serialised against CPU hotplug. The CPU hotplug code already has a mechanism to do this using
cpus_read_lock().MPAM's monitors have an overflow interrupt, so it needs to be possible to walk the domains list in irq context. RCU is ideal for this, but some paths need to be able to sleep to allocate memory.
Because resctrl_{on,off}line_cpu() take the
rdtgroup_mutexas part of a cpuhp callback,cpus_read_lock()must always be taken first.rdtgroup_schemata_write()already does this.Most of the filesystem code's domain list walkers are currently protected by the
rdtgroup_mutextaken inrdtgroup_kn_lock_live(). The exceptions arerdt_bit_usage_show()and themon_confighelpers which take the lock directly.Make the domain list protected by RCU. An architecture-specific lock prevents concurrent writers.
rdt_bit_usage_show()could walk the domain list using RCU, but to keep all the filesystem operations the same, this is changed to callcpus_read_lock(). Themon_confighelpers send multiple IPIs, take thecpus_read_lock()in these cases.The other filesystem list walkers need to be able to sleep. Add
cpus_read_lock()tordtgroup_kn_lock_live()so that the cpuhp callbacks can't be invoked when file system operations are occurring.
Quoted 7 lines
Quoted 5 lines
Add
lockdep_assert_cpus_held()in the cases where therdtgroup_kn_lock_live()call isn't obvious.Resctrl's domain online/offline calls now need to take the
rdtgroup_mutexthemselves.Reviewed-by Babu Moger <[email protected]>
Thanks!
James
Hi James,
I have tested the series on AMD systems. Done some basic sanity testing. Everything looks good.
Thanks
Babu
Quoted 108 lines · James Morse
The only significant change in this series is to add
cpus_read_lock()around the limbo and overflow handlers in the last patch. Other changes are a mix of comments and whitespace, or code marked__exit.Changes are noted in each patch, some patches even say 'No changes since v6'! ~
This series does two things, it changes resctrl to call
resctrl_arch_rmid_read()in a way that works for MPAM, and it separates the locking so that the arch code and filesystem code don't have to share a mutex. I tried to split this as two series, but these touch similar call sites, so it would create more work.(What's MPAM? See the cover letter of the first series. [1])
On x86 the RMID is an independent number. MPAMs equivalent is PMG, but this isn't an independent number - it extends the PARTID (same as CLOSID) space with bits that aren't used to select the configuration. The monitors can then be told to match specific PMG values, allowing monitor-groups to be created.
But, MPAM expects the monitors to always monitor by PARTID. The Cache-storage-utilisation counters can only work this way.
(In the MPAM spec not setting theMATCH_PARTIDbit is made CONSTRAINED UNPREDICTABLE - which is Arm's term to mean portable software can't rely on this)It gets worse, as some SoCs may have very few PMG bits. I've seen the datasheet for one that has a single bit of PMG space.
To be usable, MPAM's counters always need the PARTID and the PMG. For resctrl, this means always making the CLOSID available when the RMID is used.
To ensure RMID are always unique, this series combines the CLOSID and RMID into an index, and manages RMID based on that. For x86, the index and RMID would always be the same.
Currently the architecture specific code in the cpuhp callbacks takes the
rdtgroup_mutex. This means the filesystem code would have to export this lock, resulting in an ill-defined interface between the two, and the possibility of cross-architecture lock-ordering head aches.The second part of this series adds a
domain_list_lockto protect writes to the domain list, and protects the domain list with RCU - orcpus_read_lock().Use of RCU is to allow lockless readers of the domain list. To get MPAMs monitors working, its very likely they'll need to be plumbed up to perf. An uncore PMU driver would need to be a lockless reader of the domain list.
This series is based on tip/master's commit
6b7b1e57a824, and can be retrieved from:
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/monitors_and_locking/v7Bugs welcome,
Thanks,
James
[1] https://lore.kernel.org/lkml/[email protected]/
[v1] https://lore.kernel.org/all/[email protected]/
[v2] https://lore.kernel.org/lkml/[email protected]/
[v3] https://lore.kernel.org/r/[email protected]
[v4] https://lore.kernel.org/r/[email protected]
[v5] https://lore.kernel.org/lkml/[email protected]/
[v6] https://lore.kernel.org/all/[email protected]/James Morse (24):
tick/nohz: Movetick_nohz_full_maskdeclaration outside the #ifdef x86/resctrl:kfree()rmid_ptrsfromrdtgroup_exit()x86/resctrl: Create helper for RMID allocation and mondata dir
creation
x86/resctrl: Move rmid allocation out ofmkdir_rdt_prepare()x86/resctrl: Track the closid with the rmid
x86/resctrl: Access per-rmid structures by index x86/resctrl: Allow RMID allocation to be scoped by CLOSID x86/resctrl: Track the number of dirty RMID a CLOSID has x86/resctrl: Use __set_bit()/__clear_bit() instead of open coding x86/resctrl: Allocate the cleanest CLOSID by searching
closid_num_dirty_rmid
x86/resctrl: Move CLOSID/RMID matching and setting to use helpers x86/resctrl: Addcpumask_any_housekeeping()for limbo/overflow x86/resctrl: Queuemon_event_read()instead of sending an IPI x86/resctrl: Allowresctrl_arch_rmid_read()to sleep x86/resctrl: Allow arch to allocate memory needed in
resctrl_arch_rmid_read()
x86/resctrl: Makeresctrl_mountedchecks explicit x86/resctrl: Move alloc/mon static keys into helpers x86/resctrl: Makerdt_enable_keythe arch's decision to switch x86/resctrl: Add helpers for system wide mon/alloc capable x86/resctrl: Add CPU online callback for resctrl work x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but
cpu
x86/resctrl: Add CPU offline callback for resctrl work x86/resctrl: Move domain helper migration intoresctrl_offline_cpu()x86/resctrl: Separate arch and fs resctrl locksarch/x86/include/asm/resctrl.h | 90 +++++ arch/x86/kernel/cpu/resctrl/core.c | 94 ++--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 48 ++- arch/x86/kernel/cpu/resctrl/internal.h | 67 +++- arch/x86/kernel/cpu/resctrl/monitor.c | 453 +++++++++++++++++----- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 15 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 349 ++++++++++++----- include/linux/resctrl.h | 48 ++- include/linux/tick.h | 9 +- 9 files changed, 903 insertions(+), 270 deletions(-)
Signature · 2 lines
Thanks Babu Moger
James Morse
reviewerHi Babu,
Quoted 2 lines · Babu
I have tested the series on AMD systems. Done some basic sanity testing. Everything looks good.
I haven't found an AMD machine to test this on - thanks for doing this!
James
Hello James,
I tested this patch series(patch v7) on Intel(R) Xeon(R) Gold 6254 CPU with resctrl selftest. It is no problem. Thanks.
James Morse
reviewerHello!
Quoted 4 lines · Shaopeng Tan (Fujitsu)
I tested this patch series(patch v7) on Intel(R) Xeon(R) Gold 6254 CPU with resctrl selftest. It is no problem. Thanks.
Tested-by Shaopeng Tan <[email protected]>
Thanks for taking the time to re-test this version!
James