x86/resctrl: monitored closid+rmid together, separate arch/fs locking
Subsystems (2)
James Morse
reviewercover x86/resctrl: monitored closid+rmid together, separate arch/fs locking
Hello!
This series does two things, it changes resctrl to call resctrl_arch_rmid_read() in a way that works for MPAM, and it separates the locking so that the arch code and filesystem code don't have to share a mutex. I tried to split this as two series, but these touch similar call sites, so it would create more work.
(What's MPAM? See the cover letter of the first series. [1])
On x86 the RMID is an independent number. MPAMs equivalent is PMG, but this isn't an independent number - it extends the PARTID (same as CLOSID) space with bits that aren't used to select the configuration. The monitors can then be told to match specific PMG values, allowing monitor-groups to be created.
But, MPAM expects the monitors to always monitor by PARTID. The Cache-storage-utilisation counters can only work this way.
(In the MPAM spec not setting the MATCH_PARTID bit is made CONSTRAINED UNPREDICTABLE - which is Arm's term to mean portable software can't rely on this)
It gets worse, as some SoCs may have very few PMG bits. I've seen the datasheet for one that has a single bit of PMG space.
To be usable, MPAM's counters always need the PARTID and the PMG. For resctrl, this means always making the CLOSID available when the RMID is used.
To ensure RMID are always unique, this series combines the CLOSID and RMID into an index, and manages RMID based on that. For x86, the index and RMID would always be the same.
Currently the architecture specific code in the cpuhp callbacks takes the rdtgroup_mutex. This means the filesystem code would have to export this lock, resulting in an ill-defined interface between the two, and the possibility of cross-architecture lock-ordering head aches.
The second part of this series adds a domain_list_lock to protect writes to the domain list, and protects the domain list with RCU - or read_cpus_lock().
This series is based on v6.1-rc1, and can be retrieved from:
https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/monitors_and_locking/v1
Bugs welcome,
Thanks,
James
[1] https://lore.kernel.org/lkml/[email protected]/
James Morse (18):
x86/resctrl: Track the closid with the rmid
x86/resctrl: Access per-rmid structures by index x86/resctrl: Create helper for RMID allocation and mondata dir
creation
x86/resctrl: Move rmid allocation out of mkdir_rdt_prepare() x86/resctrl: Allow RMID allocation to be scoped by CLOSID x86/resctrl: Allow the allocator to check if a CLOSID can allocate
clean RMID
x86/resctrl: Move CLOSID/RMID matching and setting to use helpers x86/resctrl: Queue mon_event_read() instead of sending an IPI x86/resctrl: Allow resctrl_arch_rmid_read() to sleep x86/resctrl: Allow arch to allocate memory needed in
resctrl_arch_rmid_read()
x86/resctrl: Make resctrl_mounted checks explicit x86/resctrl: Move alloc/mon static keys into helpers x86/resctrl: Make rdt_enable_key the arch's decision to switch x86/resctrl: Add helpers for system wide mon/alloc capable x86/resctrl: Add cpu online callback for resctrl work x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but
cpu
x86/resctrl: Add cpu offline callback for resctrl work x86/resctrl: Separate arch and fs resctrl locks
arch/x86/include/asm/resctrl.h | 83 ++++++ arch/x86/kernel/cpu/resctrl/core.c | 71 ++--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 19 +- arch/x86/kernel/cpu/resctrl/internal.h | 24 +- arch/x86/kernel/cpu/resctrl/monitor.c | 333 ++++++++++++++++------ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 15 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 298 +++++++++++++------ include/linux/resctrl.h | 15 +- 8 files changed, 623 insertions(+), 235 deletions(-)
Signature · 1 line
2.30.2
James Morse
reviewerPATCH 1/18 x86/resctrl: Track the closid with the rmid
x86's RMID are independent of the CLOSID. An RMID can be allocated, used and freed without considering the CLOSID.
MPAM's equivalent feature is PMG, which is not an independent number, it extends the CLOSID/PARTID space. For MPAM, only PMG-bits worth of 'RMID' can be allocated for a single CLOSID.
i.e. if there is 1 bit of PMG space, then each CLOSID can have two monitor groups.
To allow rescrl to disambiguate RMID values for different CLOSID, everything in resctrl that keeps an RMID value needs to know the CLOSID too. This will always be ignored on x86.
---
Is there a better term for 'the unique identifier for a monitor group'. Using RMID for that here may be confusing...
--- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 54 +++++++++++++---------- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 12 ++--- include/linux/resctrl.h | 11 ++++- 5 files changed, 49 insertions(+), 34 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +1 -1
@@ -519,7 +519,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r);int closids_supported (void );void closid_free (int closid);int alloc_rmid (void ); -void free_rmid (u32 rmid); +void free_rmid (u32 closid,u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r);void mon_event_count (void *info);int rdtgroup_mondata_show (struct seq_file *m,void *arg);
arch/x86/kernel/cpu/resctrl/monitor.c +31 -23
171 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +2 -2
@@ -738,7 +738,7 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)* anymore when this group would be used for pseudo-locking. This * is safe to call on platforms not capable of monitoring. */ -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); ret =0 ; goto out;
@@ -773,7 +773,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) ret =rdtgroup_locksetup_user_restore (rdtgrp);if (ret) { -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);return ret; }
arch/x86/kernel/cpu/resctrl/rdtgroup.c +6 -6
@@ -2422,7 +2422,7 @@ static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp) head = &rdtgrp->mon.crdtgrp_list;list_for_each_entry_safe (sentry, stmp, head, mon.crdtgrp_list) { -free_rmid (sentry->mon.rmid); +free_rmid (sentry->closid, sentry->mon.rmid);list_del (&sentry->mon.crdtgrp_list);if (atomic_read (&sentry->waitcount) !=0 )
@@ -2462,7 +2462,7 @@ static void rmdir_all_sub(void)cpumask_or (&rdtgroup_default.cpu_mask, &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask); -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);kernfs_remove (rdtgrp->kn);list_del (&rdtgrp->rdtgroup_list);
@@ -2955,7 +2955,7 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,return 0 ; out_idfree: -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); out_destroy:kernfs_put (rdtgrp->kn);kernfs_remove (rdtgrp->kn);
@@ -2969,7 +2969,7 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,static void mkdir_rdt_prepare_clean (struct rdtgroup *rgrp) {kernfs_remove (rgrp->kn); -free_rmid (rgrp->mon.rmid); +free_rmid (rgrp->closid, rgrp->mon.rmid);rdtgroup_remove (rgrp); }
@@ -3118,7 +3118,7 @@ static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)update_closid_rmid (tmpmask,NULL ); rdtgrp->flags =RDT_DELETED ; -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);/* * Remove the rdtgrp from the parent ctrl_mon group's list
@@ -3164,8 +3164,8 @@ static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)cpumask_or (tmpmask, tmpmask, &rdtgrp->cpu_mask);update_closid_rmid (tmpmask,NULL ); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);closid_free (rdtgrp->closid); -free_rmid (rdtgrp->mon.rmid);rdtgroup_ctrl_remove (rdtgrp);
include/linux/resctrl.h +9 -2
@@ -225,6 +225,8 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);* for this resource and domain. * @r: resource that the counter should be read from. * @d: domain that the counter should be read from. +* @closid: closid that matches the rmid. The counter may +* match traffic of both closid and rmid, or rmid only. * @rmid: rmid of the counter to read. * @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes.
@@ -235,20 +237,25 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);* 0 on success, or -EIO, -EINVAL etc on error. */ int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid,u64 *val); +u32 closid,u32 rmid,enum resctrl_event_id eventid, +u64 *val); +/** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid * and eventid. * @r: The domain's resource. * @d: The rmid's domain. +* @closid: The closid that matches the rmid. Counters may match both +* closid and rmid, or rmid only. * @rmid: The rmid whose counter values should be reset. * @eventid: The eventid whose counter values should be reset. * * This can be called from any CPU. */ void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid); +u32 closid,u32 rmid, +enum resctrl_event_id eventid);extern unsigned int resctrl_rmid_realloc_threshold;extern unsigned int resctrl_rmid_realloc_limit;
Signature · 1 line
2.30.2
Re: [PATCH 01/18] x86/resctrl: Track the closid with the rmid
Hi James,
Quoted 10 lines · James Morse
x86's RMID are independent of the CLOSID. An RMID can be allocated, used and freed without considering the CLOSID.
MPAM's equivalent feature is PMG, which is not an independent number, it extends the CLOSID/PARTID space. For MPAM, only PMG-bits worth of 'RMID' can be allocated for a single CLOSID.
i.e. if there is 1 bit of PMG space, then each CLOSID can have two monitor groups.
Quoted 3 lines
To allow rescrl to disambiguate RMID values for different CLOSID, everything in resctrl that keeps an RMID value needs to know the CLOSID too. This will always be ignored on x86.
I recall that the allocation of RMIDs on MPAM was an open topic of discussion on the other thread. Can we finish that discussion first?
https://lore.kernel.org/lkml/[email protected]/
Thanks!
-Peter
Re: [PATCH 01/18] x86/resctrl: Track the closid with the rmid
In reply to James MorseQuoted 44 lines · James Morse
x86's RMID are independent of the CLOSID. An RMID can be allocated, used and freed without considering the CLOSID.
MPAM's equivalent feature is PMG, which is not an independent number, it extends the CLOSID/PARTID space. For MPAM, only PMG-bits worth of 'RMID' can be allocated for a single CLOSID.
i.e. if there is 1 bit of PMG space, then each CLOSID can have two monitor groups.To allow rescrl to disambiguate RMID values for different CLOSID, everything in resctrl that keeps an RMID value needs to know the CLOSID too. This will always be ignored on x86.
Signed-off-by James Morse <[email protected]>---
Is there a better term for 'the unique identifier for a monitor group'. Using RMID for that here may be confusing...--- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 54 +++++++++++++---------- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 12 ++--- include/linux/resctrl.h | 11 ++++- 5 files changed, 49 insertions(+), 34 deletions(-)arch/x86/kernel/cpu/resctrl/internal.h +1
-1@@ -519,7 +519,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r);int closids_supported (void );void closid_free (int closid);int alloc_rmid (void ); -void free_rmid (u32 rmid); +void free_rmid (u32 closid,u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r);void mon_event_count (void *info);int rdtgroup_mondata_show (struct seq_file *m,void *arg);arch/x86/kernel/cpu/resctrl/monitor.c +0
-0
arch/x86/kernel/cpu/resctrl/monitor.c line 28
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/monitor.c +1
-0@@ -25,6 +25,7 @@#include "internal.h" struct rmid_entry { +u32 closid;
We had better add some comments there for closid variable.
Quoted 295 lines
arch/x86/kernel/cpu/resctrl/monitor.c +30
-23u32 rmid;int busy;struct list_head list;@@ -136,7 +137,7 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val)return val; } -static inline struct rmid_entry *__rmid_entry (u32 rmid) +static inline struct rmid_entry *__rmid_entry (u32 closid,u32 rmid) {struct rmid_entry *entry;@@ -166,7 +167,8 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom, }void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid) +u32 closid,u32 rmid, +enum resctrl_event_id eventid) {struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);struct arch_mbm_state *am;@@ -185,7 +187,8 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width) }int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid,u64 *val) +u32 closid,u32 rmid,enum resctrl_event_id eventid, +u64 *val) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r);struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);@@ -251,9 +254,9 @@ void __check_limbo(struct rdt_domain *d, bool force_free)if (nrmid >= r->num_rmid)break ; - entry =__rmid_entry (nrmid); + entry =__rmid_entry (~0 , nrmid);// temporary -if (resctrl_arch_rmid_read (r, d, entry->rmid, +if (resctrl_arch_rmid_read (r, d, entry->closid, entry->rmid,QOS_L3_OCCUP_EVENT_ID , &val)) { rmid_dirty = true; }else {@@ -308,7 +311,8 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) cpu =get_cpu ();list_for_each_entry (d, &r->domains, list) {if (cpumask_test_cpu (cpu, &d->cpu_mask)) { - err =resctrl_arch_rmid_read (r, d, entry->rmid, + err =resctrl_arch_rmid_read (r, d, entry->closid, + entry->rmid,QOS_L3_OCCUP_EVENT_ID , &val);if (err || val <= resctrl_rmid_realloc_threshold)@@ -332,7 +336,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)list_add_tail (&entry->list, &rmid_free_lru); } -void free_rmid (u32 rmid) +void free_rmid (u32 closid,u32 rmid) {struct rmid_entry *entry;@@ -341,7 +345,7 @@ void free_rmid(u32 rmid)lockdep_assert_held (&rdtgroup_mutex); - entry =__rmid_entry (rmid); + entry =__rmid_entry (closid, rmid);if (is_llc_occupancy_enabled ())add_rmid_to_limbo (entry);@@ -349,15 +353,16 @@ void free_rmid(u32 rmid)list_add_tail (&entry->list, &rmid_free_lru); } -static int __mon_event_count (u32 rmid,struct rmid_read *rr) +static int __mon_event_count (u32 closid,u32 rmid,struct rmid_read *rr) {struct mbm_state *m;u64 tval =0 ;if (rr->first) -resctrl_arch_reset_rmid (rr->r, rr->d, rmid, rr->evtid); +resctrl_arch_reset_rmid (rr->r, rr->d, closid, rmid, rr->evtid); - rr->err =resctrl_arch_rmid_read (rr->r, rr->d, rmid, rr->evtid, &tval); + rr->err =resctrl_arch_rmid_read (rr->r, rr->d, closid, rmid, rr->evtid, + &tval);if (rr->err)return rr->err;@@ -400,7 +405,7 @@ static int __mon_event_count(u32 rmid, struct rmid_read *rr)* __mon_event_count() is compared with the chunks value from the previous * invocation. This must be called once per second to maintain values in MBps. */ -static void mbm_bw_count (u32 rmid,struct rmid_read *rr) +static void mbm_bw_count (u32 closid,u32 rmid,struct rmid_read *rr) {struct mbm_state *m = &rr->d->mbm_local[rmid];u64 cur_bw, bytes, cur_bytes;@@ -430,7 +435,7 @@ void mon_event_count(void *info) rdtgrp = rr->rgrp; - ret =__mon_event_count (rdtgrp->mon.rmid, rr); + ret =__mon_event_count (rdtgrp->closid, rdtgrp->mon.rmid, rr);/* * For Ctrl groups read data from child monitor groups and @@ -441,7 +446,8 @@ void mon_event_count(void *info)if (rdtgrp->type ==RDTCTRL_GROUP ) {list_for_each_entry (entry, head, mon.crdtgrp_list) { -if (__mon_event_count (entry->mon.rmid, rr) ==0 ) +if (__mon_event_count (rdtgrp->closid, entry->mon.rmid, + rr) ==0 ) ret =0 ; } }@@ -571,7 +577,8 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) } } -static void mbm_update (struct rdt_resource *r,struct rdt_domain *d,int rmid) +static void mbm_update (struct rdt_resource *r,struct rdt_domain *d, +u32 closid,u32 rmid) {struct rmid_read rr;@@ -586,12 +593,12 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid)if (is_mbm_total_enabled ()) { rr.evtid =QOS_L3_MBM_TOTAL_EVENT_ID ; rr.val =0 ; -__mon_event_count (rmid, &rr); +__mon_event_count (closid, rmid, &rr); }if (is_mbm_local_enabled ()) { rr.evtid =QOS_L3_MBM_LOCAL_EVENT_ID ; rr.val =0 ; -__mon_event_count (rmid, &rr); +__mon_event_count (closid, rmid, &rr);/* * Call the MBA software controller only for the @@ -599,7 +606,7 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid)* the software controller explicitly. */ if (is_mba_sc (NULL )) -mbm_bw_count (rmid, &rr); +mbm_bw_count (closid, rmid, &rr); } }@@ -656,11 +663,11 @@ void mbm_handle_overflow(struct work_struct *work) d = container_of(work, struct rdt_domain, mbm_over.work);list_for_each_entry (prgrp, &rdt_all_groups, rdtgroup_list) { -mbm_update (r, d, prgrp->mon.rmid); +mbm_update (r, d, prgrp->closid, prgrp->mon.rmid); head = &prgrp->mon.crdtgrp_list;list_for_each_entry (crgrp, head, mon.crdtgrp_list) -mbm_update (r, d, crgrp->mon.rmid); +mbm_update (r, d, crgrp->closid, crgrp->mon.rmid);if (is_mba_sc (NULL ))update_mba_bw (prgrp, d);@@ -703,10 +710,11 @@ static int dom_data_init(struct rdt_resource *r) }/* -* RMID 0 is special and is always allocated. It's used for all -* tasks that are not monitored. +* RMID 0 is special and is always allocated. It's used for the +* default_rdtgroup control group, which will be setup later. See +* rdtgroup_setup_root(). */ - entry =__rmid_entry (0 ); + entry =__rmid_entry (0 ,0 );list_del (&entry->list);return 0 ;arch/x86/kernel/cpu/resctrl/pseudo_lock.c +2
-2@@ -738,7 +738,7 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)* anymore when this group would be used for pseudo-locking. This * is safe to call on platforms not capable of monitoring. */ -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); ret =0 ; goto out;@@ -773,7 +773,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) ret =rdtgroup_locksetup_user_restore (rdtgrp);if (ret) { -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);return ret; }arch/x86/kernel/cpu/resctrl/rdtgroup.c +6
-6@@ -2422,7 +2422,7 @@ static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp) head = &rdtgrp->mon.crdtgrp_list;list_for_each_entry_safe (sentry, stmp, head, mon.crdtgrp_list) { -free_rmid (sentry->mon.rmid); +free_rmid (sentry->closid, sentry->mon.rmid);list_del (&sentry->mon.crdtgrp_list);if (atomic_read (&sentry->waitcount) !=0 )@@ -2462,7 +2462,7 @@ static void rmdir_all_sub(void)cpumask_or (&rdtgroup_default.cpu_mask, &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask); -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);kernfs_remove (rdtgrp->kn);list_del (&rdtgrp->rdtgroup_list);@@ -2955,7 +2955,7 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,return 0 ; out_idfree: -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); out_destroy:kernfs_put (rdtgrp->kn);kernfs_remove (rdtgrp->kn);@@ -2969,7 +2969,7 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,static void mkdir_rdt_prepare_clean (struct rdtgroup *rgrp) {kernfs_remove (rgrp->kn); -free_rmid (rgrp->mon.rmid); +free_rmid (rgrp->closid, rgrp->mon.rmid);rdtgroup_remove (rgrp); }@@ -3118,7 +3118,7 @@ static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)update_closid_rmid (tmpmask,NULL ); rdtgrp->flags =RDT_DELETED ; -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);/* * Remove the rdtgrp from the parent ctrl_mon group's list @@ -3164,8 +3164,8 @@ static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)cpumask_or (tmpmask, tmpmask, &rdtgrp->cpu_mask);update_closid_rmid (tmpmask,NULL ); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);closid_free (rdtgrp->closid); -free_rmid (rdtgrp->mon.rmid);rdtgroup_ctrl_remove (rdtgrp);include/linux/resctrl.h +7
-1@@ -225,6 +225,8 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);* for this resource and domain. * @r: resource that the counter should be read from. * @d: domain that the counter should be read from. +* @closid: closid that matches the rmid. The counter may +* match traffic of both closid and rmid, or rmid only. * @rmid: rmid of the counter to read. * @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes. @@ -235,20 +237,25 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);* 0 on success, or -EIO, -EINVAL etc on error. */ int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid,u64 *val); +u32 closid,u32 rmid,enum resctrl_event_id eventid, +u64 *val); +/** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid * and eventid. * @r: The domain's resource. * @d: The rmid's domain. +* @closid: The closid that matches the rmid. Counters may match both +* closid and rmid, or rmid only. * @rmid: The rmid whose counter values should be reset. * @eventid: The eventid whose counter values should be reset.
include/linux/resctrl.h void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); · line 261
Quoted 10 lines
include/linux/resctrl.h +2
-1* * This can be called from any CPU. */ void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid); +u32 closid,u32 rmid, +enum resctrl_event_id eventid);extern unsigned int resctrl_rmid_realloc_threshold;extern unsigned int resctrl_rmid_realloc_limit;
RE: [PATCH 01/18] x86/resctrl: Track the closid with the rmid
In reply to James MorseHi, James,
Quoted 5 lines
James Morse <[email protected]> writes:
x86's RMID are independent of the CLOSID. An RMID can be allocated, used and freed without considering the CLOSID.
Quoted 7 lines
MPAM's equivalent feature is PMG, which is not an independent number, it extends the CLOSID/PARTID space. For MPAM, only PMG-bits worth of 'RMID' can be allocated for a single CLOSID.
i.e. if there is 1 bit of PMG space, then each CLOSID can have two monitor groups.To allow rescrl to disambiguate RMID values for different CLOSID, everything in
s/rescrl/resctrl/
Quoted 33 lines
resctrl that keeps an RMID value needs to know the CLOSID too. This will always be ignored on x86.
Signed-off-by James Morse <[email protected]>---
Is there a better term for 'the unique identifier for a monitor group'. Using RMID for that here may be confusing...--- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 54 +++++++++++++---------- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 4 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 12 ++--- include/linux/resctrl.h | 11 ++++- 5 files changed, 49 insertions(+), 34 deletions(-)arch/x86/kernel/cpu/resctrl/internal.h +1
-0@@ -519,7 +519,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r); int closids_supported(void); void closid_free(int closid); int alloc_rmid(void); -void free_rmid(u32 rmid); +void free_rmid (u32 closid,u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r);void mon_event_count (void *info); int rdtgroup_mondata_show(struct seq_file *m, void *arg); diff --git
a/arch/x86/kernel/cpu/resctrl/monitor.cb/arch/x86/kernel/cpu/resctrl/monitor.c
indexefe0c30d3a12..f1f66c9942a5100644
arch/x86/kernel/cpu/resctrl/monitor.c line 28
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/monitor.c +1
-0@@ -25,6 +25,7 @@#include "internal.h" struct rmid_entry { +u32 closid;
Could you please add a comment for this closid field? It's expected to be form x86 RMID entry. So it's deserved a comment to explain a bit more on this field.
Quoted 316 lines
arch/x86/kernel/cpu/resctrl/monitor.c +10
-6u32 rmid;int busy;struct list_head list;@@ -136,7 +137,7 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val)return val; } -static inline struct rmid_entry *__rmid_entry (u32 rmid) +static inline struct rmid_entry *__rmid_entry (u32 closid,u32 rmid) {struct rmid_entry *entry;@@ -166,7 +167,8 @@ static struct arch_mbm_state *get_arch_mbm_state(struct rdt_hw_domain *hw_dom, } void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_domain *d, - u32 rmid, enum resctrl_event_id eventid) + u32 closid, u32 rmid, + enum resctrl_event_id eventid) { struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d); struct arch_mbm_state *am;@@ -185,7 +187,8 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width) }int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid,u64 *val) +u32 closid,u32 rmid,enum resctrl_event_id eventid, +u64 *val) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); struct rdt_hw_domain *hw_dom = resctrl_to_arch_dom(d); @@ -251,9 +254,9 @@ void __check_limbo(struct rdt_domain *d, bool force_free) if (nrmid >= r->num_rmid) break; - entry =__rmid_entry (nrmid); + entry =__rmid_entry (~0 , nrmid);// temporary -if (resctrl_arch_rmid_read (r, d, entry->rmid, +if (resctrl_arch_rmid_read (r, d, entry->closid, entry->rmid,QOS_L3_OCCUP_EVENT_ID , &val)) { rmid_dirty = true; }else {@@ -308,7 +311,8 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) cpu =get_cpu ();list_for_each_entry (d, &r->domains, list) {if (cpumask_test_cpu (cpu, &d->cpu_mask)) { - err =resctrl_arch_rmid_read (r, d, entry->rmid, + err =resctrl_arch_rmid_read (r, d, entry->closid, + entry->rmid,QOS_L3_OCCUP_EVENT_ID , &val); if (err || val <= resctrl_rmid_realloc_threshold) @@ -332,7 +336,7 @@ static void
add_rmid_to_limbo(struct rmid_entry *entry)arch/x86/kernel/cpu/resctrl/monitor.c +8
-6list_add_tail (&entry->list, &rmid_free_lru); } -void free_rmid (u32 rmid) +void free_rmid (u32 closid,u32 rmid) {struct rmid_entry *entry;@@ -341,7 +345,7 @@ void free_rmid(u32 rmid)lockdep_assert_held (&rdtgroup_mutex); - entry =__rmid_entry (rmid); + entry =__rmid_entry (closid, rmid);if (is_llc_occupancy_enabled ())add_rmid_to_limbo (entry);@@ -349,15 +353,16 @@ void free_rmid(u32 rmid)list_add_tail (&entry->list, &rmid_free_lru); } -static int __mon_event_count (u32 rmid,struct rmid_read *rr) +static int __mon_event_count (u32 closid,u32 rmid,struct rmid_read +*rr) {struct mbm_state *m;u64 tval =0 ;if (rr->first) -resctrl_arch_reset_rmid (rr->r, rr->d, rmid, rr->evtid); +resctrl_arch_reset_rmid (rr->r, rr->d, closid, rmid, rr->evtid); - rr->err =resctrl_arch_rmid_read (rr->r, rr->d, rmid, rr->evtid, &tval); + rr->err =resctrl_arch_rmid_read (rr->r, rr->d, closid, rmid, rr->evtid, + &tval);if (rr->err)return rr->err;@@ -400,7 +405,7 @@ static int __mon_event_count(u32 rmid, structrmid_read *rr) * __mon_event_count() is compared with the chunks value from the previous * invocation. This must be called once per second to maintain values in MBps. */ -static void mbm_bw_count (u32 rmid,struct rmid_read *rr) +static void mbm_bw_count (u32 closid,u32 rmid,struct rmid_read *rr) {struct mbm_state *m = &rr->d->mbm_local[rmid];arch/x86/kernel/cpu/resctrl/monitor.c +3
-2u64 cur_bw, bytes, cur_bytes;@@ -430,7 +435,7 @@ void mon_event_count(void *info) rdtgrp = rr->rgrp; - ret =__mon_event_count (rdtgrp->mon.rmid, rr); + ret =__mon_event_count (rdtgrp->closid, rdtgrp->mon.rmid, rr);/* * For Ctrl groups read data from child monitor groups and @@ -441,7 +446,8 @@ void mon_event_count(void *info) if (rdtgrp->type == RDTCTRL_GROUP) { list_for_each_entry(entry, head, mon.crdtgrp_list) { -if (__mon_event_count(entry->mon.rmid, rr) == 0) +if (__mon_event_count(rdtgrp->closid, entry- Quoted 1 line
mon.rmid,
arch/x86/kernel/cpu/resctrl/monitor.c +5
-3+ rr) == 0) ret =0 ; } }@@ -571,7 +577,8 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) } } -static void mbm_update (struct rdt_resource *r,struct rdt_domain *d,int rmid) +static void mbm_update (struct rdt_resource *r,struct rdt_domain *d, +u32 closid,u32 rmid) {struct rmid_read rr;@@ -586,12 +593,12 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid) if (is_mbm_total_enabled()) { rr.evtid =QOS_L3_MBM_TOTAL_EVENT_ID ; rr.val =0 ; -__mon_event_count (rmid, &rr); +__mon_event_count (closid, rmid, &rr); }if (is_mbm_local_enabled ()) { rr.evtid =QOS_L3_MBM_LOCAL_EVENT_ID ; rr.val =0 ; -__mon_event_count (rmid, &rr); +__mon_event_count (closid, rmid, &rr);/* * Call the MBA software controller only for the @@ -599,7
+606,7 @@ static voidmbm_update(struct rdt_resource *r, struct rdt_domain *d, int rmid)arch/x86/kernel/cpu/resctrl/monitor.c +3
-3* the software controller explicitly. */ if (is_mba_sc (NULL )) -mbm_bw_count (rmid, &rr); +mbm_bw_count (closid, rmid, &rr); } }@@ -656,11 +663,11 @@ void mbm_handle_overflow(struct work_struct *work) d = container_of(work, struct rdt_domain, mbm_over.work);list_for_each_entry (prgrp, &rdt_all_groups, rdtgroup_list) { -mbm_update (r, d, prgrp->mon.rmid); +mbm_update (r, d, prgrp->closid, prgrp->mon.rmid); head = &prgrp->mon.crdtgrp_list;list_for_each_entry (crgrp, head, mon.crdtgrp_list) -mbm_update (r, d, crgrp->mon.rmid); +mbm_update (r, d, crgrp->closid, crgrp->mon.rmid);if (is_mba_sc (NULL ))arch/x86/kernel/cpu/resctrl/monitor.c +4
-3update_mba_bw (prgrp, d);@@ -703,10 +710,11 @@ static int dom_data_init(struct rdt_resource *r) }/* -* RMID 0 is special and is always allocated. It's used for all -* tasks that are not monitored. +* RMID 0 is special and is always allocated. It's used for the +* default_rdtgroup control group, which will be setup later. See +* rdtgroup_setup_root(). */ - entry =__rmid_entry (0 ); + entry =__rmid_entry (0 ,0 );list_del (&entry->list);return 0 ;arch/x86/kernel/cpu/resctrl/pseudo_lock.c +2
-2@@ -738,7 +738,7 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)* anymore when this group would be used for pseudo-locking. This * is safe to call on platforms not capable of monitoring. */ -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); ret =0 ; goto out;@@ -773,7 +773,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) ret =rdtgroup_locksetup_user_restore (rdtgrp);if (ret) { -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);return ret; }arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1@@ -2422,7 +2422,7 @@ static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp) head = &rdtgrp->mon.crdtgrp_list;list_for_each_entry_safe (sentry, stmp, head, mon.crdtgrp_list) { -free_rmid (sentry->mon.rmid); +free_rmid (sentry->closid, sentry->mon.rmid);list_del (&sentry->mon.crdtgrp_list);if (
atomic_read(&sentry->waitcount)!= 0) @@ -2462,7 +2462,7
@@ static voidrmdir_all_sub(void)arch/x86/kernel/cpu/resctrl/rdtgroup.c +2
-2cpumask_or (&rdtgroup_default.cpu_mask, &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask); -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);kernfs_remove (rdtgrp->kn);list_del (&rdtgrp->rdtgroup_list);@@ -2955,7 +2955,7 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, return 0; out_idfree: -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); out_destroy:kernfs_put (rdtgrp->kn);arch/x86/kernel/cpu/resctrl/rdtgroup.c +2
-2kernfs_remove (rdtgrp->kn);@@ -2969,7 +2969,7 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, static void mkdir_rdt_prepare_clean(struct rdtgroup *rgrp) {kernfs_remove (rgrp->kn); -free_rmid (rgrp->mon.rmid); +free_rmid (rgrp->closid, rgrp->mon.rmid);rdtgroup_remove (rgrp); }@@ -3118,7 +3118,7 @@ static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)update_closid_rmid (tmpmask,NULL ); rdtgrp->flags =RDT_DELETED ; -free_rmid (rdtgrp->mon.rmid); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);/* * Remove the rdtgrp from the parent
ctrl_mongroup's list @@ -3164,8
+3164,8 @@ static intrdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask)arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1cpumask_or (tmpmask, tmpmask, &rdtgrp->cpu_mask);update_closid_rmid (tmpmask,NULL ); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);closid_free (rdtgrp->closid); -free_rmid (rdtgrp->mon.rmid);rdtgroup_ctrl_remove (rdtgrp);include/linux/resctrl.h index renamed from include/linux/resctrl.h +0
-0
0cf5b20c6ddf..641aea580a1f100644include/linux/resctrl.h +2
-0@@ -225,6 +225,8 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d); * for this resource and domain. * @r: resource that the counter should be read from. * @d: domain that the counter should be read from. + * @closid: closid that matches the rmid. The counter may + * match traffic of both closid and rmid, or rmid only. * @rmid: rmid of the counter to read. * @eventid: eventid to read, e.g. L3 occupancy.* @val: result of the counter read in bytes.
include/linux/resctrl.h +7
-2@@ -235,20 +237,25 @@ void resctrl_offline_domain(struct rdt_resource *r,struct rdt_domain *d); * 0 on success, or -EIO, -EINVAL etc on error. */ int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid,u64 *val); +u32 closid,u32 rmid,enum resctrl_event_id eventid, +u64 *val); +/** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid * and eventid. * @r: The domain's resource. * @d: The rmid's domain. +* @closid: The closid that matches the rmid. Counters may match both +* closid and rmid, or rmid only. * @rmid: The rmid whose counter values should be reset. * @eventid: The eventid whose counter values should be reset. * * This can be called from any CPU. */ void resctrl_arch_reset_rmid (struct rdt_resource *r,struct rdt_domain *d, -u32 rmid,enum resctrl_event_id eventid); +u32 closid,u32 rmid, +enum resctrl_event_id eventid); extern unsigned int resctrl_rmid_realloc_threshold; extern unsigned int
Quoted 3 lines
resctrl_rmid_realloc_limit;
--
2.30.2
Thanks.
-Fenghua
James Morse
reviewerRe: [PATCH 01/18] x86/resctrl: Track the closid with the rmid
Hi Fenghua,
Quoted 38 lines · Fenghua
Quoted 5 lines
James Morse <[email protected]> writes:
x86's RMID are independent of the CLOSID. An RMID can be allocated, used and freed without considering the CLOSID.
Quoted 7 lines
MPAM's equivalent feature is PMG, which is not an independent number, it extends the CLOSID/PARTID space. For MPAM, only PMG-bits worth of 'RMID' can be allocated for a single CLOSID.
i.e. if there is 1 bit of PMG space, then each CLOSID can have two monitor groups.To allow rescrl to disambiguate RMID values for different CLOSID, everything in
s/rescrl/resctrl/
Quoted 23 lines
resctrl that keeps an RMID value needs to know the CLOSID too. This will always be ignored on x86.
arch/x86/kernel/cpu/resctrl/internal.h +1
-0@@ -519,7 +519,7 @@ struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r); int closids_supported(void); void closid_free(int closid); int alloc_rmid(void); -void free_rmid(u32 rmid); +void free_rmid (u32 closid,u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r);void mon_event_count (void *info); int rdtgroup_mondata_show(struct seq_file *m, void *arg); diff --git
a/arch/x86/kernel/cpu/resctrl/monitor.cb/arch/x86/kernel/cpu/resctrl/monitor.c
indexefe0c30d3a12..f1f66c9942a5100644arch/x86/kernel/cpu/resctrl/monitor.c +0
-0@@ -25,6 +25,7 @@#include "internal.h"
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/monitor.c line 28
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/monitor.c +1
-0struct rmid_entry { +u32 closid;Could you please add a comment for this closid field? It's expected to be form x86 RMID entry. So it's deserved a comment to explain a bit more on this field.
Sure ... what does it need to convey?
I'm not sure what you mean by "expected to be form x86 RMID entry".
My medium verbosity version looks like this:
| /* | * Some architectures's resctrl_arch_rmid_read() needs the CLOSID value | * in order to access the correct monitor. This field provides the | * value to list walkers like __check_limbo(). On x86 this is ignored. | */
Does this cover it?
Thanks,
James
Quoted 3 lines
Quoted 3 lines
arch/x86/kernel/cpu/resctrl/monitor.c +0
-0u32 rmid;int busy;struct list_head list;
RE: [PATCH 01/18] x86/resctrl: Track the closid with the rmid
Hi, James,
Quoted 12 lines
Quoted 8 lines
Quoted 8 lines
a/arch/x86/kernel/cpu/resctrl/monitor.cb/arch/x86/kernel/cpu/resctrl/monitor.c
indexefe0c30d3a12..f1f66c9942a5100644arch/x86/kernel/cpu/resctrl/monitor.c +0
-0@@ -25,6 +25,7 @@#include "internal.h" Quoted 4 lines
arch/x86/kernel/cpu/resctrl/monitor.c line 28
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/monitor.c +1
-0struct rmid_entry { +u32 closid;Could you please add a comment for this closid field? It's expected to be form
Quoted 5 lines
x86 RMID entry.
Quoted 1 line
So it's deserved a comment to explain a bit more on this field.
Sure ... what does it need to convey?
I'm not sure what you mean by "expected to be form x86 RMID entry".
My typo, should be "not expected".
Quoted 10 lines
My medium verbosity version looks like this:
| /* | * Some architectures's resctrl_arch_rmid_read() needs the CLOSID value | * in order to access the correct monitor. This field provides the | * value to list walkers like __check_limbo(). On x86 this is ignored. | */Does this cover it?
Looks good to me.
Thanks.
-Fenghua
James Morse
reviewerPATCH 2/18 x86/resctrl: Access per-rmid structures by index
In reply to James MorseBecause of the differences between Intel RDT/AMD QoS and Arm's MPAM monitors, RMID values on arm64 are not unique unless the CLOSID is also included. Bitmaps like rmid_busy_llc need to be sized by the number of unique entries for this resource.
Add helpers to encode/decode the CLOSID and RMID to an index. The domain's busy_rmid_llc and the rmid_ptrs[] array are then sized by index. On x86, this is always just the RMID. This gives resctrl a unique value it can use to store monitor values, and allows MPAM to decode the closid when reading the hardware counters.
--- arch/x86/include/asm/resctrl.h | 17 ++++++ arch/x86/kernel/cpu/resctrl/internal.h | 2 + arch/x86/kernel/cpu/resctrl/monitor.c | 75 +++++++++++++++++--------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 7 +-- 4 files changed, 72 insertions(+), 29 deletions(-)
arch/x86/include/asm/resctrl.h +17 -0
@@ -96,6 +96,23 @@ static inline void resctrl_sched_in(void) __resctrl_sched_in(); } +static inline u32 resctrl_arch_system_num_rmid_idx (void ) +{ +/* RMID are independent numbers for x86. num_rmid_idx==num_rmid */ +return boot_cpu_data.x86_cache_max_rmid +1 ; +} + +static inline void resctrl_arch_rmid_idx_decode (u32 idx,u32 *closid,u32 *rmid) +{ + *rmid = idx; + *closid = ~0 ; +} + +static inline u32 resctrl_arch_rmid_idx_encode (u32 closid,u32 rmid) +{ +return rmid; +} +void resctrl_cpu_detect (struct cpuinfo_x86 *c);#else
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
@@ -8,6 +8,8 @@#include <linux/fs_context.h> #include <linux/jump_label.h> +#include <asm/resctrl.h> +#define MSR_IA32_L3_QOS_CFG 0xc81#define MSR_IA32_L2_QOS_CFG 0xc82#define MSR_IA32_L3_CBM_BASE 0xc90
arch/x86/kernel/cpu/resctrl/monitor.c +49 -26
189 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/rdtgroup.c +4 -3
@@ -3320,16 +3320,17 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)static int domain_setup_mon_state (struct rdt_resource *r,struct rdt_domain *d) { +u32 idx_limit =resctrl_arch_system_num_rmid_idx ();size_t tsize;if (is_llc_occupancy_enabled ()) { - d->rmid_busy_llc =bitmap_zalloc (r->num_rmid,GFP_KERNEL ); + d->rmid_busy_llc =bitmap_zalloc (idx_limit,GFP_KERNEL );if (!d->rmid_busy_llc)return -ENOMEM ; }if (is_mbm_total_enabled ()) { tsize =sizeof (*d->mbm_total); - d->mbm_total =kcalloc (r->num_rmid, tsize,GFP_KERNEL ); + d->mbm_total =kcalloc (idx_limit, tsize,GFP_KERNEL );if (!d->mbm_total) {bitmap_free (d->rmid_busy_llc);return -ENOMEM ;
@@ -3337,7 +3338,7 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) }if (is_mbm_local_enabled ()) { tsize =sizeof (*d->mbm_local); - d->mbm_local =kcalloc (r->num_rmid, tsize,GFP_KERNEL ); + d->mbm_local =kcalloc (idx_limit, tsize,GFP_KERNEL );if (!d->mbm_local) {bitmap_free (d->rmid_busy_llc);kfree (d->mbm_total);
Signature · 1 line
2.30.2
RE: [PATCH 02/18] x86/resctrl: Access per-rmid structures by index
Hi, James,
Quoted 28 lines
James Morse <[email protected]> writes:
Because of the differences between Intel RDT/AMD QoS and Arm's MPAM monitors, RMID values on arm64 are not unique unless the CLOSID is also included. Bitmaps likermid_busy_llcneed to be sized by the number of unique entries for this resource.Add helpers to encode/decode the CLOSID and RMID to an index. The domain's
busy_rmid_llcand the rmid_ptrs[] array are then sized by index. On x86, this is always just the RMID. This gives resctrl a unique value it can use to store monitor values, and allows MPAM to decode the closid when reading the hardware counters.Signed-off-by James Morse <[email protected]>--- arch/x86/include/asm/resctrl.h | 17 ++++++ arch/x86/kernel/cpu/resctrl/internal.h | 2 + arch/x86/kernel/cpu/resctrl/monitor.c | 75 +++++++++++++++++--------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 7 +-- 4 files changed, 72 insertions(+), 29 deletions(-)arch/x86/include/asm/resctrl.h +0
-0@@ -96,6 +96,23 @@ static inline void resctrl_sched_in(void)__resctrl_sched_in (); }
arch/x86/include/asm/resctrl.h static inline void resctrl_sched_in(void) · line 108
Quoted 10 lines
arch/x86/include/asm/resctrl.h +9
-0+static inline u32 resctrl_arch_system_num_rmid_idx (void ) +{ +/* RMID are independent numbers for x86. num_rmid_idx==num_rmid */ +return boot_cpu_data.x86_cache_max_rmid +1 ; } + +static inline void resctrl_arch_rmid_idx_decode (u32 idx,u32 *closid, +u32 *rmid) { + *rmid = idx; + *closid = ~0 ;
Should closid be 0 or ~0 on X86? Any special reason for closid to be ~0? Seems 0 is a natural value so that it's ignored on X86. And the value should be consistent on x86 and documented.
Quoted 47 lines
arch/x86/include/asm/resctrl.h +6
-0+} + +static inline u32 resctrl_arch_rmid_idx_encode (u32 closid,u32 rmid) { +return rmid; +} +void resctrl_cpu_detect (struct cpuinfo_x86 *c);#else arch/x86/kernel/cpu/resctrl/internal.h +2
-0@@ -8,6 +8,8 @@#include <linux/fs_context.h> #include <linux/jump_label.h> +#include <asm/resctrl.h> +#define MSR_IA32_L3_QOS_CFG 0xc81#define MSR_IA32_L2_QOS_CFG 0xc82#define MSR_IA32_L3_CBM_BASE 0xc90arch/x86/kernel/cpu/resctrl/monitor.c +13
-1@@ -137,11 +137,24 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val)return val; } -static inline struct rmid_entry *__rmid_entry (u32 closid,u32 rmid) +/* +* x86 and arm64 differ in their handling of monitoring. +* x86's RMID are an independent number, there is one RMID '1'. +* arm64's PMG extend the PARTID/CLOSID space, there is one RMID '1' +for each +* CLOSID. The RMID is no longer unique. +* To account for this, resctrl uses an index. On x86 this is just the +RMID, +* on arm64 it encodes the CLOSID and RMID. This gives a unique number. +* +* The domain's rmid_busy_llc and rmid_ptrs are sized by index. The +arch code +* must accept an attempt to read every index.
arch/x86/kernel/cpu/resctrl/monitor.c static inline u64 get_corrected_mbm_count(u32 rmid, · line 158
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/monitor.c +5
-1+*/ +static inline struct rmid_entry *__rmid_entry (u32 idx) {struct rmid_entry *entry; +u32 closid, rmid; - entry = &rmid_ptrs[rmid]; + entry = &rmid_ptrs[idx]; +resctrl_arch_rmid_idx_decode (idx, &closid, &rmid);WARN_ON (entry->rmid != rmid);
Will __rmid_entry() be moved to fs/?
Should add WARN_ON(entry->closid!=rmid) here?
Quoted 174 lines
arch/x86/kernel/cpu/resctrl/monitor.c +5
-5return entry;@@ -238,8 +251,9 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d, void __check_limbo(struct rdt_domain *d, bool force_free) { struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +u32 idx_limit =resctrl_arch_system_num_rmid_idx ();struct rmid_entry *entry; -u32 crmid =1 , nrmid; +u32 idx, cur_idx =1 ;bool rmid_dirty;u64 val =0 ;@@ -250,12 +264,11 @@ void __check_limbo(struct rdt_domain *d, boolforce_free) * RMID and move it to the free list when the counter reaches 0. */ for (;;) { - nrmid =find_next_bit (d->rmid_busy_llc, r->num_rmid, crmid); -if (nrmid >= r->num_rmid) + idx =find_next_bit (d->rmid_busy_llc, idx_limit, cur_idx); +if (idx >= idx_limit)break ; - entry =__rmid_entry (~0 , nrmid);// temporary - + entry =__rmid_entry (idx);if (resctrl_arch_rmid_read (r, d, entry->closid, entry->rmid,QOS_L3_OCCUP_EVENT_ID , &val)) {arch/x86/kernel/cpu/resctrl/monitor.c +9
-4rmid_dirty = true;@@ -264,19 +277,21 @@ void __check_limbo(struct rdt_domain *d, bool force_free) }if (force_free || !rmid_dirty) { -clear_bit (entry->rmid, d->rmid_busy_llc); +clear_bit (idx, d->rmid_busy_llc);if (!--entry->busy) { rmid_limbo_count--;list_add_tail (&entry->list, &rmid_free_lru); } } - crmid = nrmid +1 ; + cur_idx = idx +1 ; } }bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d) { -return find_first_bit (d->rmid_busy_llc, r->num_rmid) != r->num_rmid; +u32 idx_limit =resctrl_arch_system_num_rmid_idx (); + +return find_first_bit (d->rmid_busy_llc, idx_limit) != idx_limit; }/* @@ -306,6 +321,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)struct rdt_domain *d;int cpu, err;u64 val =0 ; +u32 idx; + + idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); entry->busy =0 ; cpu =get_cpu ();@@ -325,7 +343,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)*/ if (!has_busy_rmid (r, d))cqm_setup_limbo_handler (d,CQM_LIMBOCHECK_INTERVAL ); -set_bit (entry->rmid, d->rmid_busy_llc); +set_bit (idx, d->rmid_busy_llc); entry->busy++; }arch/x86/kernel/cpu/resctrl/monitor.c +6
-4put_cpu ();@@ -338,14 +356,16 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)void free_rmid (u32 closid,u32 rmid) { +u32 idx =resctrl_arch_rmid_idx_encode (closid, rmid);struct rmid_entry *entry; -if (!rmid) -return ; -lockdep_assert_held (&rdtgroup_mutex); - entry =__rmid_entry (closid, rmid); +/* do not allow the default rmid to be free'd */ +if (!idx) +return ; + + entry =__rmid_entry (idx);if (is_llc_occupancy_enabled ())arch/x86/kernel/cpu/resctrl/monitor.c +3
-2add_rmid_to_limbo (entry);@@ -355,6 +375,7 @@ void free_rmid(u32 closid, u32 rmid)static int __mon_event_count (u32 closid,u32 rmid,struct rmid_read *rr) { +u32 idx =resctrl_arch_rmid_idx_encode (closid, rmid);struct mbm_state *m;u64 tval =0 ;@@ -371,10 +392,10 @@ static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr) rr->val += tval;return 0 ;case QOS_L3_MBM_TOTAL_EVENT_ID : - m = &rr->d->mbm_total[rmid]; + m = &rr->d->mbm_total[idx];break ;case QOS_L3_MBM_LOCAL_EVENT_ID : - m = &rr->d->mbm_local[rmid]; + m = &rr->d->mbm_local[idx];break ;default :arch/x86/kernel/cpu/resctrl/monitor.c +5
-3/* @@ -407,7 +428,8 @@ static int __mon_event_count(u32 closid, u32 rmid,struct rmid_read *rr) */ static void mbm_bw_count (u32 closid,u32 rmid,struct rmid_read *rr) { -struct mbm_state *m = &rr->d->mbm_local[rmid]; +u32 idx =resctrl_arch_rmid_idx_encode (closid, rmid); +struct mbm_state *m = &rr->d->mbm_local[idx];u64 cur_bw, bytes, cur_bytes; cur_bytes = rr->val;@@ -497,7 +519,7 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) {u32 closid, rmid, cur_msr_val, new_msr_val;struct mbm_state *pmbm_data, *cmbm_data; -u32 cur_bw, delta_bw, user_bw; +u32 cur_bw, delta_bw, user_bw, idx;struct rdt_resource *r_mba;struct rdt_domain *dom_mba;struct list_head *head;@@ -510,7 +532,8 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_domain *dom_mbm) closid = rgrp->closid; rmid = rgrp->mon.rmid; - pmbm_data = &dom_mbm->mbm_local[rmid]; + idx =resctrl_arch_rmid_idx_encode (closid, rmid); + pmbm_data = &dom_mbm->mbm_local[idx]; dom_mba =get_domain_from_cpu (smp_processor_id (), r_mba);arch/x86/kernel/cpu/resctrl/monitor.c +5
-5if (!dom_mba) {@@ -693,19 +716,19 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)static int dom_data_init (struct rdt_resource *r) { +u32 nr_idx =resctrl_arch_system_num_rmid_idx ();struct rmid_entry *entry =NULL ; -int i, nr_rmids; +int i; - nr_rmids = r->num_rmid; - rmid_ptrs =kcalloc (nr_rmids,sizeof (struct rmid_entry ),GFP_KERNEL ); + rmid_ptrs =kcalloc (nr_idx,sizeof (struct rmid_entry ),GFP_KERNEL );if (!rmid_ptrs)return -ENOMEM ; -for (i =0 ; i < nr_rmids; i++) { +for (i =0 ; i < nr_idx; i++) { entry = &rmid_ptrs[i];INIT_LIST_HEAD (&entry->list); - entry->rmid = i; +resctrl_arch_rmid_idx_decode (i, &entry->closid, &entry->rmid);list_add_tail (&entry->list, &rmid_free_lru); }
arch/x86/kernel/cpu/resctrl/monitor.c static int dom_data_init(struct rdt_resource *r) · line 740
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/monitor.c +1
-1@@ -714,7 +737,7 @@ static int dom_data_init(struct rdt_resource *r)* default_rdtgroup control group, which will be setup later. See * rdtgroup_setup_root(). */ - entry =__rmid_entry (0 ,0 ); + entry =__rmid_entry (resctrl_arch_rmid_idx_encode (0 ,0 ));
Closid is 0 here on x86. We need to have a consistent closid value on x86. Maybe even define a macro for x86 closid value e.g. #define RMID_FIELD_CLOSID_IGNORED 0. So the macro value is used on X86 always.
Quoted 38 lines
arch/x86/kernel/cpu/resctrl/monitor.c +0
-0list_del (&entry->list);return 0 ;arch/x86/kernel/cpu/resctrl/rdtgroup.c +3
-2@@ -3320,16 +3320,17 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)static int domain_setup_mon_state (struct rdt_resource *r,struct rdt_domain *d) { +u32 idx_limit =resctrl_arch_system_num_rmid_idx ();size_t tsize;if (is_llc_occupancy_enabled ()) { - d->rmid_busy_llc =bitmap_zalloc (r->num_rmid,GFP_KERNEL ); + d->rmid_busy_llc =bitmap_zalloc (idx_limit,GFP_KERNEL );if (!d->rmid_busy_llc)return -ENOMEM ; }if (is_mbm_total_enabled ()) { tsize =sizeof (*d->mbm_total); - d->mbm_total =kcalloc (r->num_rmid, tsize,GFP_KERNEL ); + d->mbm_total =kcalloc (idx_limit, tsize,GFP_KERNEL );if (!d->mbm_total) {bitmap_free (d->rmid_busy_llc);arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1return -ENOMEM ;@@ -3337,7 +3338,7 @@ static int domain_setup_mon_state(struct rdt_resource *r, struct rdt_domain *d) } if (is_mbm_local_enabled()) { tsize =sizeof (*d->mbm_local); - d->mbm_local =kcalloc (r->num_rmid, tsize,GFP_KERNEL ); + d->mbm_local =kcalloc (idx_limit, tsize,GFP_KERNEL );if (!d->mbm_local) {bitmap_free (d->rmid_busy_llc);
Quoted 3 lines
kfree(d->mbm_total);
--
2.30.2
Thanks.
-Fenghua
James Morse
reviewerRe: [PATCH 02/18] x86/resctrl: Access per-rmid structures by index
Hi Fenghua,
Quoted 20 lines · Fenghua
Quoted 20 lines
James Morse <[email protected]> writes:
Because of the differences between Intel RDT/AMD QoS and Arm's MPAM monitors, RMID values on arm64 are not unique unless the CLOSID is also included. Bitmaps likermid_busy_llcneed to be sized by the number of unique entries for this resource.Add helpers to encode/decode the CLOSID and RMID to an index. The domain's
busy_rmid_llcand the rmid_ptrs[] array are then sized by index. On x86, this is always just the RMID. This gives resctrl a unique value it can use to store monitor values, and allows MPAM to decode the closid when reading the hardware counters.arch/x86/include/asm/resctrl.h +0
-0@@ -96,6 +96,23 @@ static inline void resctrl_sched_in(void)__resctrl_sched_in (); }
Quoted 12 lines
arch/x86/include/asm/resctrl.h static inline void resctrl_sched_in(void) · line 108
Quoted 10 lines
arch/x86/include/asm/resctrl.h +9
-0+static inline u32 resctrl_arch_system_num_rmid_idx (void ) +{ +/* RMID are independent numbers for x86. num_rmid_idx==num_rmid */ +return boot_cpu_data.x86_cache_max_rmid +1 ; } + +static inline void resctrl_arch_rmid_idx_decode (u32 idx,u32 *closid, +u32 *rmid) { + *rmid = idx; + *closid = ~0 ;Should closid be 0 or ~0 on X86? Any special reason for closid to be ~0?
I picked ~0 as its not a valid CLOSID, so anything that consumes it should choke quickly.
Quoted 2 lines
Seems 0 is a natural value so that it's ignored on X86. And the value should be consistent on x86 and documented.
Well 0 is a valid CLOSID value, you can't ignore it based on the value - that's done at compile time as the helpers don't use the value. Doing nothing here would leave an uninitialized value on the stack, which would then get passed to the next function (which should ignore it). I assume this is the sort of thing future compilers will complain about if they can't see that the next function doesn't use the value.
I'll give it a #define value to make it clear its a deliberate choice to initialise it with an out of range value.
Quoted 41 lines
Quoted 41 lines
arch/x86/include/asm/resctrl.h +6
-0+} + +static inline u32 resctrl_arch_rmid_idx_encode (u32 closid,u32 rmid) { +return rmid; +} +void resctrl_cpu_detect (struct cpuinfo_x86 *c);#else arch/x86/kernel/cpu/resctrl/monitor.c +16
-1@@ -137,11 +137,24 @@ static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val)return val; } -static inline struct rmid_entry *__rmid_entry (u32 closid,u32 rmid) +/* +* x86 and arm64 differ in their handling of monitoring. +* x86's RMID are an independent number, there is one RMID '1'. +* arm64's PMG extend the PARTID/CLOSID space, there is one RMID '1' +for each +* CLOSID. The RMID is no longer unique. +* To account for this, resctrl uses an index. On x86 this is just the +RMID, +* on arm64 it encodes the CLOSID and RMID. This gives a unique number. +* +* The domain's rmid_busy_llc and rmid_ptrs are sized by index. The +arch code +* must accept an attempt to read every index. +*/ +static inline struct rmid_entry *__rmid_entry (u32 idx) {struct rmid_entry *entry; +u32 closid, rmid;
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/monitor.c static inline u64 get_corrected_mbm_count(u32 rmid, unsigned long val) · line 158
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/monitor.c +2
-1- entry = &rmid_ptrs[rmid]; + entry = &rmid_ptrs[idx]; +resctrl_arch_rmid_idx_decode (idx, &closid, &rmid);WARN_ON (entry->rmid != rmid);Will
__rmid_entry()be moved tofs/?
It does as its not related to accessing the hardware. I hope to move as much as possible to avoid duplication.
Quoted 1 line
Should add
WARN_ON(entry->closid!=rmid)here?
Presumably WARN_ON(entry->closid != closid)?
That would force everything to be initalised to the official 'unitialised value', which is probably not a bad thing.
If I'm touching these, I 'll change it to WARN_ON_ONCE(), as if it breaks, chances are its going to trigger a few hundred times a second, which wouldn't help anyone trying to debug it.
Quoted 11 lines
arch/x86/kernel/cpu/resctrl/monitor.c static int dom_data_init(struct rdt_resource *r) · line 740
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/monitor.c +1
-1return entry;@@ -714,7 +737,7 @@ static int dom_data_init(struct rdt_resource *r)* default_rdtgroup control group, which will be setup later. See * rdtgroup_setup_root(). */ - entry =__rmid_entry (0 ,0 ); + entry =__rmid_entry (resctrl_arch_rmid_idx_encode (0 ,0 ));Closid is 0 here on x86.
It's supposed to be zero. This code is ensuring the monitors for the default control group will always be available, by removing it from rmid_free_lru, effectively allocating it.
It needs to be 0 for architectures where the CLOSID matters to resctrl_arch_rmid_idx_encode(), to ensure it allocates/reserves/leaks the correct hardware monitor.
Changing __rmid_entry() to take a struct rdtgroup might make the whole thing clearer, I didn't go that far as I thought it would be more churn.
Quoted 3 lines
We need to have a consistent closid value on x86.
Maybe even define a macro for x86 closid value e.g. #defineRMID_FIELD_CLOSID_IGNORED0. So the macro value is used on X86 always.
The value is ignored on encode by x86. You only need to provide a magic value for decode, and only because deterministic bugs and values you can grep for a good things!
Thanks,
James
James Morse
reviewerPATCH 3/18 x86/resctrl: Create helper for RMID allocation and mondata dir creation
In reply to James MorseRMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups, rdtgroup_mkdir_ctrl_mon() later goes on to allocate the CLOSID.
MPAM's equivalent of RMID are not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
To make a subsequent change that does this easier to read, move the RMID allocation and mondata dir creation to a helper.
--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 42 +++++++++++++++++--------- 1 file changed, 27 insertions(+), 15 deletions(-)
arch/x86/kernel/cpu/resctrl/rdtgroup.c +27 -15
@@ -2868,6 +2868,30 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)return 0 ; } +static int mkdir_rdt_prepare_rmid_alloc (struct rdtgroup *rdtgrp) +{ +int ret; + +if (!rdt_mon_capable) +return 0 ; + + ret =alloc_rmid (); +if (ret <0 ) { +rdt_last_cmd_puts ("Out of RMIDs\n" ); +return ret; + } + rdtgrp->mon.rmid = ret; + + ret =mkdir_mondata_all (rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn); +if (ret) { +rdt_last_cmd_puts ("kernfs subdir error\n" ); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); +return ret; + } + +return 0 ; +} +static int mkdir_rdt_prepare (struct kernfs_node *parent_kn,const char *name,umode_t mode,enum rdt_group_type rtype,struct rdtgroup **r)
@@ -2933,20 +2957,10 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } -if (rdt_mon_capable) { - ret =alloc_rmid (); -if (ret <0 ) { -rdt_last_cmd_puts ("Out of RMIDs\n" ); - goto out_destroy; - } - rdtgrp->mon.rmid = ret; + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) + goto out_destroy; - ret =mkdir_mondata_all (kn, rdtgrp, &rdtgrp->mon.mon_data_kn); -if (ret) { -rdt_last_cmd_puts ("kernfs subdir error\n" ); - goto out_idfree; - } - }kernfs_activate (kn);/*
@@ -2954,8 +2968,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,*/ return 0 ; -out_idfree: -free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); out_destroy:kernfs_put (rdtgrp->kn);kernfs_remove (rdtgrp->kn);
Signature · 1 line
2.30.2
RE: [PATCH 03/18] x86/resctrl: Create helper for RMID allocation and mondata dir creation
Hi, James,
Quoted 92 lines
James Morse <[email protected]> writes:
RMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups,
rdtgroup_mkdir_ctrl_mon()later goes on to allocate the CLOSID.MPAM's equivalent of RMID are not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
To make a subsequent change that does this easier to read, move the RMID allocation and mondata dir creation to a helper.
Signed-off-by James Morse <[email protected]>--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 42 +++++++++++++++++--------- 1 file changed, 27 insertions(+), 15 deletions(-)arch/x86/kernel/cpu/resctrl/rdtgroup.c +26
-13@@ -2868,6 +2868,30 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)return 0 ; } +static int mkdir_rdt_prepare_rmid_alloc (struct rdtgroup *rdtgrp) { +int ret; + +if (!rdt_mon_capable) +return 0 ; + + ret =alloc_rmid (); +if (ret <0 ) { +rdt_last_cmd_puts ("Out of RMIDs\n" ); +return ret; + } + rdtgrp->mon.rmid = ret; + + ret =mkdir_mondata_all (rdtgrp->kn, rdtgrp, &rdtgrp- >mon.mon_data_kn); +if (ret) { +rdt_last_cmd_puts ("kernfs subdir error\n" ); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); +return ret; + } + +return 0 ; +} +static int mkdir_rdt_prepare (struct kernfs_node *parent_kn,const char *name,umode_t mode,enum rdt_group_type rtype,struct rdtgroup **r)@@ -2933,20 +2957,10 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } -if (rdt_mon_capable) { - ret =alloc_rmid (); -if (ret <0 ) { -rdt_last_cmd_puts ("Out of RMIDs\n" ); - goto out_destroy; - } - rdtgrp->mon.rmid = ret; + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) + goto out_destroy; - ret =mkdir_mondata_all (kn, rdtgrp, &rdtgrp- >mon.mon_data_kn); -if (ret) { -rdt_last_cmd_puts ("kernfs subdir error\n" ); - goto out_idfree; - } - }kernfs_activate (kn);arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-2/* @@ -2954,8 +2968,6 @@ static int mkdir_rdt_prepare(struct kernfs_node*parent_kn, */ return 0 ; -out_idfree: -free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); out_destroy:kernfs_put (rdtgrp->kn);
Quoted 1 line
kernfs_remove(rdtgrp->kn);
Why not free allocated rmid? Rmid leak without freed.
Thanks.
-Fenghua
James Morse
reviewerRe: [PATCH 03/18] x86/resctrl: Create helper for RMID allocation and mondata dir creation
Hi Fenghua,
Quoted 39 lines · Fenghua
Quoted 39 lines
James Morse <[email protected]> writes:
RMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups,
rdtgroup_mkdir_ctrl_mon()later goes on to allocate the CLOSID.MPAM's equivalent of RMID are not an independent number, so can't be allocated until the CLOSID is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
To make a subsequent change that does this easier to read, move the RMID allocation and mondata dir creation to a helper.
arch/x86/kernel/cpu/resctrl/rdtgroup.c +14
-0@@ -2868,6 +2868,30 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)return 0 ; } +static int mkdir_rdt_prepare_rmid_alloc (struct rdtgroup *rdtgrp) { +int ret; + +if (!rdt_mon_capable) +return 0 ; + + ret =alloc_rmid (); +if (ret <0 ) { +rdt_last_cmd_puts ("Out of RMIDs\n" ); +return ret; + } + rdtgrp->mon.rmid = ret; + + ret = mkdir_mondata_all(rdtgrp->kn, rdtgrp, &rdtgrp-
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp) · line 2888
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +3
-0> mon.mon_data_kn); +if (ret) { +rdt_last_cmd_puts ("kernfs subdir error\n" ); +free_rmid (rdtgrp->closid, rdtgrp->mon.rmid);
^^^^^^^^^
Quoted 40 lines
Quoted 40 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +9
-13+return ret; + } + +return 0 ; +} +static int mkdir_rdt_prepare (struct kernfs_node *parent_kn,const char *name,umode_t mode,enum rdt_group_type rtype,struct rdtgroup **r)@@ -2933,20 +2957,10 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } -if (rdt_mon_capable) { - ret =alloc_rmid (); -if (ret <0 ) { -rdt_last_cmd_puts ("Out of RMIDs\n" ); - goto out_destroy; - } - rdtgrp->mon.rmid = ret; + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) + goto out_destroy; - ret =mkdir_mondata_all (kn, rdtgrp, &rdtgrp- > mon.mon_data_kn); -if (ret) { -rdt_last_cmd_puts ("kernfs subdir error\n" ); - goto out_idfree; - } - }kernfs_activate (kn);arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0/* @@ -2954,8 +2968,6 @@ static int mkdir_rdt_prepare(struct kernfs_node*parent_kn, */ return 0 ;
Quoted 7 lines
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-2-out_idfree: -free_rmid (rdtgrp->closid, rdtgrp->mon.rmid); out_destroy:kernfs_put (rdtgrp->kn);
kernfs_remove(rdtgrp->kn);Why not free allocated rmid? Rmid leak without freed.
It's just moved into the helper.
This free_rmid() is here because this function used to allocate the rmid and the mondata as separate steps. Now a helper does that in one go - and cleans up after itself.
If mkdir_rdt_prepare_rmid_alloc() fails, it didn't allocate an rmid .. if it secretly did, it free()s it itself, see the ^-lighted code above.
Thanks,
James
James Morse
reviewerPATCH 4/18 x86/resctrl: Move rmid allocation out of mkdir_rdt_prepare()
In reply to James MorseRMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups, rdtgroup_mkdir_ctrl_mon() later goes on to allocate the CLOSID.
MPAM's equivalent of RMID are not an independent number, so can't be allocated until the closid is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
Move the RMID allocation out of mkdir_rdt_prepare() to occur in its caller, after the mkdir_rdt_prepare() call. This allows the RMID allocator to know the CLOSID.
--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 29 +++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/resctrl/rdtgroup.c +22 -7
@@ -2892,6 +2892,12 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)return 0 ; } +static void mkdir_rdt_prepare_rmid_free (struct rdtgroup *rgrp) +{ +if (rdt_mon_capable) +free_rmid (rgrp->closid, rgrp->mon.rmid); +} +static int mkdir_rdt_prepare (struct kernfs_node *parent_kn,const char *name,umode_t mode,enum rdt_group_type rtype,struct rdtgroup **r)
@@ -2957,10 +2963,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } - ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); -if (ret) - goto out_destroy; -kernfs_activate (kn);/*
@@ -2981,7 +2983,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn,static void mkdir_rdt_prepare_clean (struct rdtgroup *rgrp) {kernfs_remove (rgrp->kn); -free_rmid (rgrp->closid, rgrp->mon.rmid);rdtgroup_remove (rgrp); }
@@ -3003,12 +3004,19 @@ static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn, prgrp = rdtgrp->mon.parent; rdtgrp->closid = prgrp->closid; + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) { +mkdir_rdt_prepare_clean (rdtgrp); + goto out_unlock; + } +/* * Add the rdtgrp to the list of rdtgrps the parent * ctrl_mon group has to track. */ list_add_tail (&rdtgrp->mon.crdtgrp_list, &prgrp->mon.crdtgrp_list); +out_unlock:rdtgroup_kn_unlock (parent_kn);return ret; }
@@ -3039,10 +3047,15 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, ret =0 ; rdtgrp->closid = closid; - ret =rdtgroup_init_alloc (rdtgrp); -if (ret <0 ) + + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) goto out_id_free; + ret =rdtgroup_init_alloc (rdtgrp); +if (ret <0 ) + goto out_rmid_free; +list_add (&rdtgrp->rdtgroup_list, &rdt_all_groups);if (rdt_mon_capable) {
@@ -3061,6 +3074,8 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, out_del_list:list_del (&rdtgrp->rdtgroup_list); +out_rmid_free: +mkdir_rdt_prepare_rmid_free (rdtgrp); out_id_free:closid_free (closid); out_common_fail:
Signature · 1 line
2.30.2
RE: [PATCH 04/18] x86/resctrl: Move rmid allocation out of mkdir_rdt_prepare()
Hi James,
Quoted 66 lines
RMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups,
rdtgroup_mkdir_ctrl_mon()later goes on to allocate the CLOSID.MPAM's equivalent of RMID are not an independent number, so can't be allocated until the closid is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
Move the RMID allocation out of
mkdir_rdt_prepare()to occur in its caller, after themkdir_rdt_prepare()call. This allows the RMID allocator to know the CLOSID.Signed-off-by James Morse <[email protected]>--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 29 +++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)arch/x86/kernel/cpu/resctrl/rdtgroup.c +4
-4@@ -2892,6 +2892,12 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)return 0 ; } +static void mkdir_rdt_prepare_rmid_free (struct rdtgroup *rgrp) { +if (rdt_mon_capable) +free_rmid (rgrp->closid, rgrp->mon.rmid); } +static int mkdir_rdt_prepare (struct kernfs_node *parent_kn,const char *name,umode_t mode,enum rdt_group_type rtype,struct rdtgroup **r)@@ -2957,10 +2963,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_destroy; } - ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); -if (ret) - goto out_destroy; -kernfs_activate (kn);arch/x86/kernel/cpu/resctrl/rdtgroup.c +4
-1/* @@ -2981,7 +2983,6 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, static void mkdir_rdt_prepare_clean(struct rdtgroup *rgrp) {kernfs_remove (rgrp->kn); -free_rmid (rgrp->closid, rgrp->mon.rmid);rdtgroup_remove (rgrp); }@@ -3003,12 +3004,19 @@ static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn, prgrp = rdtgrp->mon.parent; rdtgrp->closid = prgrp->closid; + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) { +mkdir_rdt_prepare_clean (rdtgrp); + goto out_unlock;
arch/x86/kernel/cpu/resctrl/rdtgroup.c static int rdtgroup_mkdir_mon(struct kernfs_node · line 3021
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +3
-0+ } +/* * Add the rdtgrp to the list of rdtgrps the parent * ctrl_mon group has to track. */ list_add_tail (&rdtgrp->mon.crdtgrp_list, &prgrp->mon.crdtgrp_list); +out_unlock:rdtgroup_kn_unlock (parent_kn);
Is there any case that a lock is not unlocked here?
Best regards,
Shaopeng Tan
Quoted 33 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0return ret;}
arch/x86/kernel/cpu/resctrl/rdtgroup.c +7
-2@@ -3039,10 +3047,15 @@ static int rdtgroup_mkdir_ctrl_mon(structkernfs_node *parent_kn, ret =0 ; rdtgrp->closid = closid; - ret =rdtgroup_init_alloc (rdtgrp); -if (ret <0 ) + + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) goto out_id_free; + ret =rdtgroup_init_alloc (rdtgrp); +if (ret <0 ) + goto out_rmid_free; +list_add (&rdtgrp->rdtgroup_list, &rdt_all_groups);arch/x86/kernel/cpu/resctrl/rdtgroup.c +2
-0if (rdt_mon_capable) {@@ -3061,6 +3074,8 @@ static int rdtgroup_mkdir_ctrl_mon(structkernfs_node *parent_kn, out_del_list: list_del(&rdtgrp->rdtgroup_list); +out_rmid_free: +mkdir_rdt_prepare_rmid_free (rdtgrp); out_id_free:closid_free (closid);
out_common_fail:
--
2.30.2
James Morse
reviewerRe: [PATCH 04/18] x86/resctrl: Move rmid allocation out of mkdir_rdt_prepare()
Hello!
Quoted 38 lines · Shaopeng Tan (Fujitsu)
Quoted 38 lines
RMID are allocated for each monitor or control group directory, because each of these needs its own RMID. For control groups,
rdtgroup_mkdir_ctrl_mon()later goes on to allocate the CLOSID.MPAM's equivalent of RMID are not an independent number, so can't be allocated until the closid is known. An RMID allocation for one CLOSID may fail, whereas another may succeed depending on how many monitor groups a control group has.
The RMID allocation needs to move to be after the CLOSID has been allocated.
Move the RMID allocation out of
mkdir_rdt_prepare()to occur in its caller, after themkdir_rdt_prepare()call. This allows the RMID allocator to know the CLOSID.arch/x86/kernel/cpu/resctrl/rdtgroup.c +6
-0@@ -3003,12 +3004,19 @@ static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn, prgrp = rdtgrp->mon.parent; rdtgrp->closid = prgrp->closid; + ret =mkdir_rdt_prepare_rmid_alloc (rdtgrp); +if (ret) { +mkdir_rdt_prepare_clean (rdtgrp); + goto out_unlock; + } +/* * Add the rdtgrp to the list of rdtgrps the parent * ctrl_mon group has to track. */ list_add_tail (&rdtgrp->mon.crdtgrp_list, &prgrp->mon.crdtgrp_list);
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c static int rdtgroup_mkdir_mon(struct kernfs_node *parent_kn, · line 3020
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-0+out_unlock:rdtgroup_kn_unlock (parent_kn);Is there any case that a lock is not unlocked here?
If you've found an example - please share!
The unlock is necessary because mkdir_rdt_prepare() calls rdtgroup_kn_lock_live(), but expects its caller to do the unlock.
This patch allows the rmid allocation to fail separately to mkdir_rdt_prepare(), so rdtgroup_mkdir_mon() needs to ensure the unlock() happens when it returns an error, this work was previously done by 'out_unlock' in mkdir_rdt_prepare().
I hope this helps,
Thanks,
James
James Morse
reviewerPATCH 5/18 x86/resctrl: Allow RMID allocation to be scoped by CLOSID
In reply to James MorseMPAMs RMID values are not unique unless the CLOSID is considered as well.
alloc_rmid() expects the RMID to be an independent number.
Pass the CLOSID in to alloc_rmid(). Use this to compare indexes when allocating. If the CLOSID is not relevant to the index, this ends up comparing the free RMID with itself, and the first free entry will be used. With MPAM the CLOSID is included in the index, so this becomes a walk of the free RMID entries, until one that matches the supplied CLOSID is found.
--- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 44 ++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 2 +- 4 files changed, 38 insertions(+), 12 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +1 -1
@@ -520,7 +520,7 @@ void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int closids_supported (void );void closid_free (int closid); -int alloc_rmid (void ); +int alloc_rmid (u32 closid);void free_rmid (u32 closid,u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r);void mon_event_count (void *info);
arch/x86/kernel/cpu/resctrl/monitor.c +35 -9
60 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +1 -1
@@ -763,7 +763,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp)int ret;if (rdt_mon_capable) { - ret =alloc_rmid (); + ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) {rdt_last_cmd_puts ("Out of RMIDs\n" );return ret;
arch/x86/kernel/cpu/resctrl/rdtgroup.c +1 -1
@@ -2875,7 +2875,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp)if (!rdt_mon_capable)return 0 ; - ret =alloc_rmid (); + ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) {rdt_last_cmd_puts ("Out of RMIDs\n" );return ret;
Signature · 1 line
2.30.2
James Morse
reviewerPATCH 6/18 x86/resctrl: Allow the allocator to check if a CLOSID can allocate clean RMID
In reply to James MorseMPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Add a helper to allow the CLOSID allocator to check if a CLOSID has dirty RMID values. This behaviour is enabled by a kconfig option selected by the architecture, which avoids a pointless search for x86.
--- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 31 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 18 +++++++++------ 3 files changed, 43 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +1 -0
@@ -519,6 +519,7 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp);void rdtgroup_pseudo_lock_remove (struct rdtgroup *rdtgrp);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int closids_supported (void ); +bool resctrl_closid_is_dirty (u32 closid);void closid_free (int closid);int alloc_rmid (u32 closid);void free_rmid (u32 closid,u32 rmid);
arch/x86/kernel/cpu/resctrl/monitor.c +31 -0
@@ -320,6 +320,37 @@ static struct rmid_entry *resctrl_find_free_rmid(u32 closid)return ERR_PTR (-ENOSPC ); } +/** +* resctrl_closid_is_dirty - Determine if clean RMID can be allocate for this +* CLOSID. +* @closid: The CLOSID that is being queried. +* +* MPAM's equivalent of RMID are per-CLOSID, meaning a freshly allocate CLOSID +* may not be able to allocate clean RMID. To avoid this the allocator will +* only return clean CLOSID. +*/ +bool resctrl_closid_is_dirty (u32 closid) +{ +struct rmid_entry *entry; +int i; + +lockdep_assert_held (&rdtgroup_mutex); + +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) +return false; + +for (i =0 ; i <resctrl_arch_system_num_rmid_idx (); i++) { + entry = &rmid_ptrs[i]; +if (entry->closid != closid) +continue ; + +if (entry->busy) +return true; + } + +return false; +} +/* * As of now the RMIDs allocation is the same in each domain. * However we keep track of which packages the RMIDs
arch/x86/kernel/cpu/resctrl/rdtgroup.c +11 -7
@@ -93,7 +93,7 @@ void rdt_last_cmd_printf(const char *fmt, ...)* - Our choices on how to configure each resource become progressively more * limited as the number of resources grows. */ -static int closid_free_map; +static unsigned long closid_free_map;static int closid_free_map_len;int closids_supported (void )
@@ -119,14 +119,18 @@ static void closid_init(void)static int closid_alloc (void ) { -u32 closid =ffs (closid_free_map); +u32 closid; -if (closid ==0 ) -return -ENOSPC ; - closid--; - closid_free_map &= ~(1 << closid); +for_each_set_bit (closid, &closid_free_map, closid_free_map_len) { +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID ) && +resctrl_closid_is_dirty (closid)) +continue ; -return closid; +clear_bit (closid, &closid_free_map); +return closid; + } + +return -ENOSPC ; }void closid_free (int closid)
Signature · 1 line
2.30.2
Re: [PATCH 06/18] x86/resctrl: Allow the allocator to check if a CLOSID can allocate clean RMID
Hi James,
Quoted 25 lines · James Morse
MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Add a helper to allow the CLOSID allocator to check if a CLOSID has dirty RMID values. This behaviour is enabled by a kconfig option selected by the architecture, which avoids a pointless search for x86.
Signed-off-by James Morse <[email protected]>--- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 31 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 18 +++++++++------ 3 files changed, 43 insertions(+), 7 deletions(-)arch/x86/kernel/cpu/resctrl/monitor.c +8
-0+/** +* resctrl_closid_is_dirty - Determine if clean RMID can be allocate for this +* CLOSID. +* @closid: The CLOSID that is being queried. +* +* MPAM's equivalent of RMID are per-CLOSID, meaning a freshly allocate CLOSID +* may not be able to allocate clean RMID. To avoid this the allocator will +* only return clean CLOSID.
arch/x86/kernel/cpu/resctrl/monitor.c static struct rmid_entry *resctrl_find_free_rmid(u32 closid) · line 340
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/monitor.c +10
-0+*/ +bool resctrl_closid_is_dirty (u32 closid) +{ +struct rmid_entry *entry; +int i; + +lockdep_assert_held (&rdtgroup_mutex); + +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) +return false;
Since dirty closid occurs when the kconfig option for MPAM is enabled, it seems that the condition of the if statement here should take the opposite value:
if (!IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
arch/x86/kernel/cpu/resctrl/monitor.c static struct rmid_entry *resctrl_find_free_rmid(u32 closid) · line 353
Quoted 13 lines
arch/x86/kernel/cpu/resctrl/monitor.c +13
-0+ +for (i =0 ; i <resctrl_arch_system_num_rmid_idx (); i++) { + entry = &rmid_ptrs[i]; +if (entry->closid != closid) +continue ; + +if (entry->busy) +return true; + } + +return false; +} +
Shawn
James Morse
reviewerRe: [PATCH 06/18] x86/resctrl: Allow the allocator to check if a CLOSID can allocate clean RMID
Hi Shawn,
Quoted 25 lines · Shawn Wang
Quoted 24 lines · James Morse
MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Add a helper to allow the CLOSID allocator to check if a CLOSID has dirty RMID values. This behaviour is enabled by a kconfig option selected by the architecture, which avoids a pointless search for x86.
arch/x86/kernel/cpu/resctrl/monitor.c +14
-0+/** +* resctrl_closid_is_dirty - Determine if clean RMID can be allocate for this +* CLOSID. +* @closid: The CLOSID that is being queried. +* +* MPAM's equivalent of RMID are per-CLOSID, meaning a freshly allocate CLOSID +* may not be able to allocate clean RMID. To avoid this the allocator will +* only return clean CLOSID. +*/ +bool resctrl_closid_is_dirty (u32 closid) +{ + struct rmid_entry *entry; + int i; +
Quoted 8 lines
Quoted 4 lines
+
lockdep_assert_held(&rdtgroup_mutex);
+
+ if (IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)) + return false;Since dirty closid occurs when the kconfig option for MPAM is enabled, it seems that the condition of the if statement here should take the opposite value:
if (!IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID))
Yup. Bother.
Thanks for spotting that! It was intended to avoid this work on x86 as its pointless, and the number of RMID could be large.
Thanks!
James
RE: [PATCH 06/18] x86/resctrl: Allow the allocator to check if a CLOSID can allocate clean RMID
In reply to James MorseHi James
Quoted 105 lines
MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Add a helper to allow the CLOSID allocator to check if a CLOSID has dirty RMID values. This behaviour is enabled by a kconfig option selected by the architecture, which avoids a pointless search for x86.
Signed-off-by James Morse <[email protected]>--- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 31 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 18 +++++++++------ 3 files changed, 43 insertions(+), 7 deletions(-)arch/x86/kernel/cpu/resctrl/internal.h +0
-0@@ -519,6 +519,7 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp); void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);struct
rdt_domain*get_domain_from_cpu(int cpu, structrdt_resource*r); intclosids_supported(void);arch/x86/kernel/cpu/resctrl/internal.h +1
-0+bool resctrl_closid_is_dirty (u32 closid);void closid_free (int closid);int alloc_rmid (u32 closid);void free_rmid (u32 closid,u32 rmid);arch/x86/kernel/cpu/resctrl/monitor.c +32
-0@@ -320,6 +320,37 @@ static struct rmid_entry *resctrl_find_free_rmid(u32 closid)return ERR_PTR (-ENOSPC ); } +/** +* resctrl_closid_is_dirty - Determine if clean RMID can be allocate for this +* CLOSID. +* @closid: The CLOSID that is being queried. +* +* MPAM's equivalent of RMID are per-CLOSID, meaning a freshly allocate +CLOSID +* may not be able to allocate clean RMID. To avoid this the allocator +will +* only return clean CLOSID. +*/ +bool resctrl_closid_is_dirty (u32 closid) { +struct rmid_entry *entry; +int i; + +lockdep_assert_held (&rdtgroup_mutex); + +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) +return false; + +for (i =0 ; i <resctrl_arch_system_num_rmid_idx (); i++) { + entry = &rmid_ptrs[i]; +if (entry->closid != closid) +continue ; + +if (entry->busy) +return true; + } + +return false; +} +/* * As of now the RMIDs allocation is the same in each domain. * However we keep track of which packages the RMIDs diff --git
a/arch/x86/kernel/cpu/resctrl/rdtgroup.cb/arch/x86/kernel/cpu/resctrl/rdtgroup.c
indexac88610a6946..59f33adcf6f8100644arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0@@ -93,7 +93,7 @@ void rdt_last_cmd_printf(const char *fmt, ...)* - Our choices on how to configure each resource become progressively more
* limited as the number of resources grows.
*/arch/x86/kernel/cpu/resctrl/rdtgroup.c +4
-6-static int closid_free_map; +static unsigned long closid_free_map;static int closid_free_map_len;int closids_supported (void )@@ -119,14 +119,18 @@ static void closid_init(void)static int closid_alloc (void ) { -u32 closid =ffs (closid_free_map); +u32 closid; -if (closid ==0 ) -return -ENOSPC ; - closid--; - closid_free_map &= ~(1 << closid); +for_each_set_bit (closid, &closid_free_map, closid_free_map_len) { + if(
IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)&&
arch/x86/kernel/cpu/resctrl/rdtgroup.c static void closid_init(void) · line 126
Quoted 1 line
arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-0+ resctrl_closid_is_dirty(closid))
IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID) is redundant here, since it is also at the beginning of function resctrl_closid_is_dirty(closid).
Best regards,
Shaopeng Tan
Quoted 13 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +6
-2+continue ; -return closid; +clear_bit (closid, &closid_free_map); +return closid; + } + +return -ENOSPC ; }void closid_free (int closid) --2.30.2
James Morse
reviewerRe: [PATCH 06/18] x86/resctrl: Allow the allocator to check if a CLOSID can allocate clean RMID
Hello,
Quoted 43 lines · Shaopeng Tan (Fujitsu)
Hi James
Quoted 41 lines
MPAM's PMG bits extend its PARTID space, meaning the same PMG value can be used for different control groups.
This means once a CLOSID is allocated, all its monitoring ids may still be dirty, and held in limbo.
Add a helper to allow the CLOSID allocator to check if a CLOSID has dirty RMID values. This behaviour is enabled by a kconfig option selected by the architecture, which avoids a pointless search for x86.
arch/x86/kernel/cpu/resctrl/monitor.c +8
-0@@ -320,6 +320,37 @@ static struct rmid_entry *resctrl_find_free_rmid(u32 +bool resctrl_closid_is_dirty (u32 closid) { +struct rmid_entry *entry; +int i; + +lockdep_assert_held (&rdtgroup_mutex); + +if (IS_ENABLED (CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID )) +return false;
a/arch/x86/kernel/cpu/resctrl/rdtgroup.cb/arch/x86/kernel/cpu/resctrl/rdtgroup.c
indexac88610a6946..59f33adcf6f8100644arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1@@ -119,14 +119,18 @@ static void closid_init(void)static int closid_alloc (void ) { -u32 closid =ffs (closid_free_map); +u32 closid;
Quoted 12 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c static void closid_init(void) · line 126
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +2
-4-if (closid ==0 ) -return -ENOSPC ; - closid--; - closid_free_map &= ~(1 << closid); +for_each_set_bit (closid, &closid_free_map, closid_free_map_len) { + if(
IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)&&arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-0+ resctrl_closid_is_dirty(closid))
IS_ENABLED(CONFIG_RESCTRL_RMID_DEPENDS_ON_CLOSID)is redundant here, since it is also at the beginning of functionresctrl_closid_is_dirty(closid).
This is true. I included it because resctrl_closid_is_dirty() is in a different compilation unit, so the compiler can't know it does nothing if that config option isn't enabled. This avoided a pointless call to a function that does nothing. But you're right it would be more readable without it, and creating a control group is hardly a performance critical path. I'll remove it.
Thanks,
James
James Morse
reviewerPATCH 7/18 x86/resctrl: Move CLOSID/RMID matching and setting to use helpers
In reply to James MorseWhen switching tasks, the CLOSID and RMID that the new task should use are stored in struct task_struct. For x86 the CLOSID known by resctrl, the value in task_struct, and the value written to the CPU register are all the same thing.
MPAM's CPU interface has two different PARTID's one for data accesses the other for instruction fetch. Storing resctrl's CLOSID value in struct task_struct implies the arch code knows whether resctrl is using CDP.
Move the matching and setting of the struct task_struct properties to use helpers. This allows arm64 to store the hardware format of the register, instead of having to convert it each time.
__rdtgroup_move_task()s use of READ_ONCE()/WRITE_ONCE() ensures torn values aren't seen as another CPU may schedule the task being moved while the value is being changed. MPAM has an additional corner-case here as the PMG bits extend the PARTID space. If the scheduler sees a new-CLOSID but old-RMID, the task will dirty an RMID that the limbo code is not watching causing an inaccurate count. x86's RMID are independent values, so the limbo code will still be watching the old-RMID in this circumstance.
To avoid this, arm64 needs both the CLOSID/RMID WRITE_ONCE()d together. Both values must be provided together.
Because MPAM's RMID values are not unique, the CLOSID must be provided when matching the RMID.
--- arch/x86/include/asm/resctrl.h | 18 ++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 57 +++++++++++++++----------- 2 files changed, 51 insertions(+), 24 deletions(-)
arch/x86/include/asm/resctrl.h +18 -0
@@ -90,6 +90,24 @@ static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)return val * scale; } +static inline void resctrl_arch_set_closid_rmid (struct task_struct *tsk, +u32 closid,u32 rmid) +{ +WRITE_ONCE (tsk->closid, closid); +WRITE_ONCE (tsk->rmid, rmid); +} + +static inline bool resctrl_arch_match_closid (struct task_struct *tsk,u32 closid) +{ +return READ_ONCE (tsk->closid) == closid; +} + +static inline bool resctrl_arch_match_rmid (struct task_struct *tsk,u32 ignored, +u32 rmid) +{ +return READ_ONCE (tsk->rmid) == rmid; +} +static inline void resctrl_sched_in (void ) {if (static_branch_likely (&rdt_enable_key))
arch/x86/kernel/cpu/resctrl/rdtgroup.c +33 -24
109 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [PATCH 07/18] x86/resctrl: Move CLOSID/RMID matching and setting to use helpers
Quoted 14 lines · James Morse
When switching tasks, the CLOSID and RMID that the new task should use are stored in struct
task_struct. For x86 the CLOSID known by resctrl, the value intask_struct, and the value written to the CPU register are all the same thing.MPAM's CPU interface has two different PARTID's one for data accesses the other for instruction fetch. Storing resctrl's CLOSID value in struct
task_structimplies the arch code knows whether resctrl is using CDP.Move the matching and setting of the struct
task_structproperties to use helpers. This allows arm64 to store the hardware format of the register, instead of having to convert it each time.
Quoted 11 lines
__rdtgroup_move_task()s use of READ_ONCE()/WRITE_ONCE() ensures torn values aren't seen as another CPU may schedule the task being moved while the value is being changed. MPAM has an additional corner-case here as the PMG bits extend the PARTID space. If the scheduler sees a new-CLOSID but old-RMID, the task will dirty an RMID that the limbo code is not watching causing an inaccurate count. x86's RMID are independent values, so the limbo code will still be watching the old-RMID in this circumstance.
To avoid this, arm64 needs both the CLOSID/RMID WRITE_ONCE()d together. Both values must be provided together.
I think I remember something about this... Is that about having them union'd and read/written as one? (just for my own curiosity)
Quoted 5 lines
Because MPAM's RMID values are not unique, the CLOSID must be provided when matching the RMID.
CC Valentin Schneider <[email protected]>Signed-off-by James Morse <[email protected]>
James Morse
reviewerRe: [PATCH 07/18] x86/resctrl: Move CLOSID/RMID matching and setting to use helpers
Hi Valentin,
Quoted 26 lines · Valentin Schneider
Quoted 14 lines · James Morse
When switching tasks, the CLOSID and RMID that the new task should use are stored in struct
task_struct. For x86 the CLOSID known by resctrl, the value intask_struct, and the value written to the CPU register are all the same thing.MPAM's CPU interface has two different PARTID's one for data accesses the other for instruction fetch. Storing resctrl's CLOSID value in struct
task_structimplies the arch code knows whether resctrl is using CDP.Move the matching and setting of the struct
task_structproperties to use helpers. This allows arm64 to store the hardware format of the register, instead of having to convert it each time.Quoted 10 lines
__rdtgroup_move_task()s use of READ_ONCE()/WRITE_ONCE() ensures torn values aren't seen as another CPU may schedule the task being moved while the value is being changed. MPAM has an additional corner-case here as the PMG bits extend the PARTID space. If the scheduler sees a new-CLOSID but old-RMID, the task will dirty an RMID that the limbo code is not watching causing an inaccurate count. x86's RMID are independent values, so the limbo code will still be watching the old-RMID in this circumstance.
To avoid this, arm64 needs both the CLOSID/RMID WRITE_ONCE()d together. Both values must be provided together.
Quoted 2 lines
I think I remember something about this... Is that about having them union'd and read/written as one? (just for my own curiosity)
Yup. Many moons ago you pointed out that storing partid/pmg separately lets the remote CPUs load torn values. arm64 not only needs the values as a pair, it needs to read/write them as a pair.
Quoted 4 lines
Quoted 4 lines
Because MPAM's RMID values are not unique, the CLOSID must be provided when matching the RMID.
CC Valentin Schneider <[email protected]>
(CC here also stands for celebrate-contribution!)
Quoted 3 lines
Quoted 1 line
Signed-off-by James Morse <[email protected]>Reviewed-by Valentin Schneider <[email protected]>
Thanks!
James
James Morse
reviewerPATCH 8/18 x86/resctrl: Queue mon_event_read() instead of sending an IPI
In reply to James Morsex86 is blessed with an abundance of monitors, one per RMID, that can be read from any CPU in the domain. MPAMs monitors reside in the MMIO MSC, the number implemented is up to the manufacturer. This means when there are fewer monitors than needed, they need to be allocated and freed.
Worse, the domain may be broken up into slices, and the MMIO accesses for each slice may need performing from different CPUs.
These two details mean MPAMs monitor code needs to be able to sleep, and IPI another CPU in the domain to read from a resource that has been sliced.
mon_event_read() already invokes mon_event_count() via IPI, which means this isn't possible.
Change mon_event_read() to schedule mon_event_count() on a remote CPU and wait, instead of sending an IPI. This function is only used in response to a user-space filesystem request (not the timing sensitive overflow code).
This allows MPAM to hide the slice behaviour from resctrl, and to keep the monitor-allocation in monitor.c.
--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 7 +++++-- arch/x86/kernel/cpu/resctrl/internal.h | 2 +- arch/x86/kernel/cpu/resctrl/monitor.c | 6 ++++-- 3 files changed, 10 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +5 -2
@@ -533,8 +533,11 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_domain *d, struct rdtgroup *rdtgrp, int evtid, int first) { +/* When picking a cpu from cpu_mask, ensure it can't race with cpuhp */ + lockdep_assert_held(&rdtgroup_mutex); +/* -* setup the parameters to send to the IPI to read the data. +* setup the parameters to pass to mon_event_count() to read the data. */ rr->rgrp = rdtgrp; rr->evtid = evtid;
@@ -543,7 +546,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, rr->val =0 ; rr->first = first; -smp_call_function_any (&d->cpu_mask, mon_event_count, rr,1 ); +smp_call_on_cpu (cpumask_any (&d->cpu_mask), mon_event_count, rr, false); }int rdtgroup_mondata_show (struct seq_file *m,void *arg)
arch/x86/kernel/cpu/resctrl/internal.h +1 -1
@@ -524,7 +524,7 @@ void closid_free(int closid);int alloc_rmid (u32 closid);void free_rmid (u32 closid,u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r); -void mon_event_count (void *info); +int mon_event_count (void *info);int rdtgroup_mondata_show (struct seq_file *m,void *arg); void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_domain *d, struct rdtgroup *rdtgrp,
arch/x86/kernel/cpu/resctrl/monitor.c +4 -2
@@ -502,10 +502,10 @@ static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr) }/* -* This is called via IPI to read the CQM/MBM counters +* This is scheduled by mon_event_read() to read the CQM/MBM counters * on a domain. */ -void mon_event_count (void *info) +int mon_event_count (void *info) {struct rdtgroup *rdtgrp, *entry;struct rmid_read *rr = info;
@@ -538,6 +538,8 @@ void mon_event_count(void *info)*/ if (ret ==0 ) rr->err =0 ; + +return 0 ; }/*
Signature · 1 line
2.30.2
James Morse
reviewerPATCH 9/18 x86/resctrl: Allow resctrl_arch_rmid_read() to sleep
In reply to James MorseMPAM's cache occupancy counters can take a little while to settle once the monitor has been configured. The maximum settling time is described to the driver via a firmware table. The value could be large enough that it makes sense to sleep.
To avoid exposing this to resctrl, it should be hidden behind MPAM's resctrl_arch_rmid_read(). But add_rmid_to_limbo() calls resctrl_arch_rmid_read() from a non-preemptible context.
add_rmid_to_limbo() is opportunistically reading the L3 occupancy counter on this domain to avoid adding the RMID to limbo if this domain's value has drifted below resctrl_rmid_realloc_threshold since the limbo handler last ran. Determining 'this domain' involves disabling preeption to prevent the thread being migrated to CPUs in a different domain between the check and resctrl_arch_rmid_read() call. The check is skipped for all remote domains.
Instead, call resctrl_arch_rmid_read() for each domain, and get it to read the arch specific counter via IPI if its called on a CPU outside the target domain. By covering remote domains, this change stops the limbo handler from being started unnecessarily.
This also allows resctrl_arch_rmid_read() to sleep.
---
The alternative is to remove the counter read from this path altogether, and assume user-space would never try to re-allocate the last RMID before the limbo handler runs next.
--- arch/x86/kernel/cpu/resctrl/monitor.c | 58 ++++++++++++++++++--------- 1 file changed, 38 insertions(+), 20 deletions(-)
arch/x86/kernel/cpu/resctrl/monitor.c +38 -20
@@ -199,17 +199,19 @@ static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)return chunks >> shift; } -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, -u32 closid,u32 rmid,enum resctrl_event_id eventid, -u64 *val) +struct __rmid_read_arg { -struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); -struct arch_mbm_state *am; -u64 msr_val, chunks; +u32 rmid; +enum resctrl_event_id eventid; -if (!cpumask_test_cpu (smp_processor_id (), &d->cpu_mask)) -return -EINVAL ; +u64 msr_val; +}; + +static void __rmid_read (void *arg) +{ +enum resctrl_event_id eventid = ((struct __rmid_read_arg *)arg)->eventid; +u32 rmid = ((struct __rmid_read_arg *)arg)->rmid; +u64 msr_val;/* * As per the SDM, when IA32_QM_EVTSEL.EvtID (bits 7:0) is configured
@@ -222,6 +224,28 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain *d,wrmsr (MSR_IA32_QM_EVTSEL , eventid, rmid);rdmsrl (MSR_IA32_QM_CTR , msr_val); + ((struct __rmid_read_arg *)arg)->msr_val = msr_val; +} + +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d, +u32 closid,u32 rmid,enum resctrl_event_id eventid, +u64 *val) +{ +struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); +struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d); +struct __rmid_read_arg arg; +struct arch_mbm_state *am; +u64 msr_val, chunks; +int err; + + arg.rmid = rmid; + arg.eventid = eventid; + + err =smp_call_function_any (&d->cpu_mask, __rmid_read, &arg, true); +if (err) +return err; + + msr_val = arg.msr_val;if (msr_val &RMID_VAL_ERROR )return -EIO ;if (msr_val &RMID_VAL_UNAVAIL )
@@ -376,23 +400,18 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rdt_domain *d; -int cpu, err;u64 val =0 ;u32 idx; +int err; idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); entry->busy =0 ; - cpu =get_cpu ();list_for_each_entry (d, &r->domains, list) { -if (cpumask_test_cpu (cpu, &d->cpu_mask)) { - err =resctrl_arch_rmid_read (r, d, entry->closid, - entry->rmid, -QOS_L3_OCCUP_EVENT_ID , - &val); -if (err || val <= resctrl_rmid_realloc_threshold) -continue ; - } + err =resctrl_arch_rmid_read (r, d, entry->closid, entry->rmid, +QOS_L3_OCCUP_EVENT_ID , &val); +if (err || val <= resctrl_rmid_realloc_threshold) +continue ;/* * For the first limbo RMID in the domain,
@@ -403,7 +422,6 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)set_bit (idx, d->rmid_busy_llc); entry->busy++; } -put_cpu ();if (entry->busy) rmid_limbo_count++;
Signature · 1 line
2.30.2
James Morse
reviewerPATCH 10/18 x86/resctrl: Allow arch to allocate memory needed in resctrl_arch_rmid_read()
In reply to James MorseDepending on the number of monitors available, Arm's MPAM may need to allocate a monitor prior to reading the counter value. Allocating a contended resource may involve sleeping.
All callers of resctrl_arch_rmid_read() read the counter on more than one domain. If the monitor is allocated globally, there is no need to allocate and free it for each call to resctrl_arch_rmid_read().
Add arch hooks for this allocation, which need calling before resctrl_arch_rmid_read(). The allocated monitor is passed to resctrl_arch_rmid_read(), then freed again afterwards. The helper can be called on any CPU, and can sleep.
--- arch/x86/include/asm/resctrl.h | 11 +++++++ arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 40 +++++++++++++++++++++++--- include/linux/resctrl.h | 4 +-- 4 files changed, 50 insertions(+), 6 deletions(-)
arch/x86/include/asm/resctrl.h +11 -0
@@ -131,6 +131,17 @@ static inline u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid) return rmid; } +/* x86 can always read an rmid, nothing needs allocating */ +struct rdt_resource ; +static inline int resctrl_arch_mon_ctx_alloc (struct rdt_resource *r,int evtid) +{ +might_sleep (); +return 0 ; +}; + +static inline void resctrl_arch_mon_ctx_free (struct rdt_resource *r,int evtid, +int ctx) { }; +void resctrl_cpu_detect (struct cpuinfo_x86 *c);#else
arch/x86/kernel/cpu/resctrl/internal.h +1 -0
@@ -98,6 +98,7 @@ struct rmid_read {bool first;int err;u64 val; +int arch_mon_ctx; };extern bool rdt_alloc_capable;
arch/x86/kernel/cpu/resctrl/monitor.c +36 -4
@@ -15,6 +15,7 @@* Software Developer Manual June 2016, volume 3, section 17.17. */ +#include <linux/cpu.h> #include <linux/module.h> #include <linux/sizes.h> #include <linux/slab.h>
@@ -229,7 +230,7 @@ static void __rmid_read(void *arg)int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d,u32 closid,u32 rmid,enum resctrl_event_id eventid, -u64 *val) +u64 *val,int ignored) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r);struct rdt_hw_domain *hw_dom =resctrl_to_arch_dom (d);
@@ -278,9 +279,14 @@ void __check_limbo(struct rdt_domain *d, bool force_free)u32 idx_limit =resctrl_arch_system_num_rmid_idx ();struct rmid_entry *entry;u32 idx, cur_idx =1 ; +int arch_mon_ctx;bool rmid_dirty;u64 val =0 ; + arch_mon_ctx =resctrl_arch_mon_ctx_alloc (r,QOS_L3_OCCUP_EVENT_ID ); +if (arch_mon_ctx <0 ) +return ; +/* * Skip RMID 0 and start from RMID 1 and check all the RMIDs that * are marked as busy for occupancy < threshold. If the occupancy
@@ -294,7 +300,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free) entry =__rmid_entry (idx);if (resctrl_arch_rmid_read (r, d, entry->closid, entry->rmid, -QOS_L3_OCCUP_EVENT_ID , &val)) { +QOS_L3_OCCUP_EVENT_ID , &val, + arch_mon_ctx)) { rmid_dirty = true; }else { rmid_dirty = (val >= resctrl_rmid_realloc_threshold);
@@ -309,6 +316,8 @@ void __check_limbo(struct rdt_domain *d, bool force_free) } cur_idx = idx +1 ; } + +resctrl_arch_mon_ctx_free (r,QOS_L3_OCCUP_EVENT_ID , arch_mon_ctx); }bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d)
@@ -400,16 +409,22 @@ static void add_rmid_to_limbo(struct rmid_entry *entry) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;struct rdt_domain *d; +int arch_mon_ctx;u64 val =0 ;u32 idx;int err; idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); + arch_mon_ctx =resctrl_arch_mon_ctx_alloc (r,QOS_L3_OCCUP_EVENT_ID ); +if (arch_mon_ctx <0 ) +return ; + entry->busy =0 ;list_for_each_entry (d, &r->domains, list) { err =resctrl_arch_rmid_read (r, d, entry->closid, entry->rmid, -QOS_L3_OCCUP_EVENT_ID , &val); +QOS_L3_OCCUP_EVENT_ID , &val, + arch_mon_ctx);if (err || val <= resctrl_rmid_realloc_threshold)continue ;
@@ -422,6 +437,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)set_bit (idx, d->rmid_busy_llc); entry->busy++; } +resctrl_arch_mon_ctx_free (r,QOS_L3_OCCUP_EVENT_ID , arch_mon_ctx);if (entry->busy) rmid_limbo_count++;
@@ -458,7 +474,7 @@ static int __mon_event_count(u32 closid, u32 rmid, struct rmid_read *rr)resctrl_arch_reset_rmid (rr->r, rr->d, closid, rmid, rr->evtid); rr->err =resctrl_arch_rmid_read (rr->r, rr->d, closid, rmid, rr->evtid, - &tval); + &tval, rr->arch_mon_ctx);if (rr->err)return rr->err;
@@ -531,6 +547,9 @@ int mon_event_count(void *info)int ret; rdtgrp = rr->rgrp; + rr->arch_mon_ctx =resctrl_arch_mon_ctx_alloc (rr->r, rr->evtid); +if (rr->arch_mon_ctx <0 ) +return rr->arch_mon_ctx; ret =__mon_event_count (rdtgrp->closid, rdtgrp->mon.rmid, rr);
@@ -557,6 +576,8 @@ int mon_event_count(void *info)if (ret ==0 ) rr->err =0 ; +resctrl_arch_mon_ctx_free (rr->r, rr->evtid, rr->arch_mon_ctx); +return 0 ; }
@@ -693,11 +714,21 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,if (is_mbm_total_enabled ()) { rr.evtid =QOS_L3_MBM_TOTAL_EVENT_ID ; rr.val =0 ; + rr.arch_mon_ctx =resctrl_arch_mon_ctx_alloc (rr.r, rr.evtid); +if (rr.arch_mon_ctx <0 ) +return ; +__mon_event_count (closid, rmid, &rr); + +resctrl_arch_mon_ctx_free (rr.r, rr.evtid, rr.arch_mon_ctx); }if (is_mbm_local_enabled ()) { rr.evtid =QOS_L3_MBM_LOCAL_EVENT_ID ; rr.val =0 ; + rr.arch_mon_ctx =resctrl_arch_mon_ctx_alloc (rr.r, rr.evtid); +if (rr.arch_mon_ctx <0 ) +return ; +__mon_event_count (closid, rmid, &rr);/*
@@ -707,6 +738,7 @@ static void mbm_update(struct rdt_resource *r, struct rdt_domain *d,*/ if (is_mba_sc (NULL ))mbm_bw_count (closid, rmid, &rr); +resctrl_arch_mon_ctx_free (rr.r, rr.evtid, rr.arch_mon_ctx); } }
include/linux/resctrl.h +2 -2
@@ -230,6 +230,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);* @rmid: rmid of the counter to read. * @eventid: eventid to read, e.g. L3 occupancy. * @val: result of the counter read in bytes. +* @arch_mon_ctx: An allocated context from resctrl_arch_mon_ctx_alloc(). * * Call from process context on a CPU that belongs to domain @d. *
@@ -238,8 +239,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d);*/ int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain *d,u32 closid,u32 rmid,enum resctrl_event_id eventid, -u64 *val); - +u64 *val,int arch_mon_ctx);/** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid
Signature · 1 line
2.30.2
James Morse
reviewerPATCH 11/18 x86/resctrl: Make resctrl_mounted checks explicit
In reply to James MorseThe rdt_enable_key is switched when resctrl is mounted, and used to prevent a second mount of the filesystem. It also enables the architecture's context switch code.
This requires another architecture to have the same set of static-keys, as resctrl depends on them too.
Make the resctrl_mounted checks explicit: resctrl can keep track of whether it has been mounted once. This doesn't need to be combined with whether the arch code is context switching the CLOSID.
Tests against the rdt_mon_enable_key become a test that resctrl is mounted and that monitoring is enabled.
This will allow the static-key changing to be moved behind resctrl_arch_ calls.
--- arch/x86/kernel/cpu/resctrl/internal.h | 1 + arch/x86/kernel/cpu/resctrl/monitor.c | 5 +++-- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 17 +++++++++++------ 3 files changed, 15 insertions(+), 8 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +1 -0
@@ -105,6 +105,7 @@ extern bool rdt_alloc_capable;extern bool rdt_mon_capable;extern unsigned int rdt_mon_features;extern struct list_head resctrl_schema_all; +extern bool resctrl_mounted;enum rdt_group_type {RDTCTRL_GROUP =0 ,
arch/x86/kernel/cpu/resctrl/monitor.c +3 -2
@@ -788,7 +788,7 @@ void mbm_handle_overflow(struct work_struct *work)mutex_lock (&rdtgroup_mutex); -if (!static_branch_likely (&rdt_mon_enable_key)) +if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key)) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;
@@ -816,8 +816,9 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)unsigned long delay =msecs_to_jiffies (delay_ms);int cpu; -if (!static_branch_likely (&rdt_mon_enable_key)) +if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key))return ; + cpu =cpumask_any (&dom->cpu_mask); dom->mbm_work_cpu = cpu;schedule_delayed_work_on (cpu, &dom->mbm_over, delay);
arch/x86/kernel/cpu/resctrl/rdtgroup.c +11 -6
@@ -42,6 +42,9 @@ LIST_HEAD(rdt_all_groups);/* list of entries for the schemata file */ LIST_HEAD (resctrl_schema_all); +/* the filesystem can only be mounted once */ +bool resctrl_mounted; +/* Kernel fs node for "info" directory under root */ static struct kernfs_node *kn_info;
@@ -795,7 +798,7 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns,mutex_lock (&rdtgroup_mutex);/* Return empty if resctrl has not been mounted. */ -if (!static_branch_unlikely (&rdt_enable_key)) { +if (!resctrl_mounted) {seq_puts (s,"res:\nmon:\n" ); goto unlock; }
@@ -2197,7 +2200,7 @@ static int rdt_get_tree(struct fs_context *fc)/* * resctrl file system can only be mounted once. */ -if (static_branch_unlikely (&rdt_enable_key)) { +if (resctrl_mounted) { ret = -EBUSY ; goto out; }
@@ -2245,8 +2248,10 @@ static int rdt_get_tree(struct fs_context *fc)if (rdt_mon_capable)static_branch_enable_cpuslocked (&rdt_mon_enable_key); -if (rdt_alloc_capable || rdt_mon_capable) +if (rdt_alloc_capable || rdt_mon_capable) {static_branch_enable_cpuslocked (&rdt_enable_key); + resctrl_mounted = true; + }if (is_mbm_enabled ()) { r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;
@@ -2513,6 +2518,7 @@ static void rdt_kill_sb(struct super_block *sb)static_branch_disable_cpuslocked (&rdt_alloc_enable_key);static_branch_disable_cpuslocked (&rdt_mon_enable_key);static_branch_disable_cpuslocked (&rdt_enable_key); + resctrl_mounted = false;kernfs_kill_sb (sb);mutex_unlock (&rdtgroup_mutex);cpus_read_unlock ();
@@ -3337,7 +3343,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)* If resctrl is mounted, remove all the * per domain monitor data directories. */ -if (static_branch_unlikely (&rdt_mon_enable_key)) +if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key))rmdir_mondata_subdir_allrdtgrp (r, d->id);if (is_mbm_enabled ())
@@ -3414,8 +3420,7 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)if (is_llc_occupancy_enabled ())INIT_DELAYED_WORK (&d->cqm_limbo, cqm_handle_limbo); -/* If resctrl is mounted, add per domain monitor data directories. */ -if (static_branch_unlikely (&rdt_mon_enable_key)) +if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key))mkdir_mondata_subdir_allrdtgrp (r, d);return 0 ;
Signature · 1 line
2.30.2
James Morse
reviewerPATCH 12/18 x86/resctrl: Move alloc/mon static keys into helpers
In reply to James Morseresctrl enables three static keys depending on the features it has enabled. Another architecture's context switch code may look different, any static keys that control it should be buried behind helpers.
Move the alloc/mon logic into arch-specific helpers as a preparatory step for making the rdt_enable_key's status something the arch code decides.
This means other architectures don't have to mirror the static keys.
--- arch/x86/include/asm/resctrl.h | 20 ++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 5 ----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 8 ++++---- 3 files changed, 24 insertions(+), 9 deletions(-)
arch/x86/include/asm/resctrl.h +20 -0
@@ -37,6 +37,26 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); +static inline void resctrl_arch_enable_alloc (void ) +{ +static_branch_enable_cpuslocked (&rdt_alloc_enable_key); +} + +static inline void resctrl_arch_disable_alloc (void ) +{ +static_branch_disable_cpuslocked (&rdt_alloc_enable_key); +} + +static inline void resctrl_arch_enable_mon (void ) +{ +static_branch_enable_cpuslocked (&rdt_mon_enable_key); +} + +static inline void resctrl_arch_disable_mon (void ) +{ +static_branch_disable_cpuslocked (&rdt_mon_enable_key); +} +/* * __resctrl_sched_in() - Writes the task's CLOSid/RMID to IA32_PQR_MSR *
arch/x86/kernel/cpu/resctrl/internal.h +0 -5
@@ -57,9 +57,6 @@ static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc) return container_of(kfc, struct rdt_fs_context, kfc); } -DECLARE_STATIC_KEY_FALSE (rdt_enable_key); -DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); -/** * struct mon_evt - Entry in the event list of a resource * @evtid: event id
@@ -415,8 +412,6 @@ extern struct mutex rdtgroup_mutex;extern struct rdt_hw_resource rdt_resources_all[];extern struct rdtgroup rdtgroup_default; -DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key); -extern struct dentry *debugfs_resctrl;enum resctrl_res_level {
arch/x86/kernel/cpu/resctrl/rdtgroup.c +4 -4
@@ -2244,9 +2244,9 @@ static int rdt_get_tree(struct fs_context *fc) goto out_psl;if (rdt_alloc_capable) -static_branch_enable_cpuslocked (&rdt_alloc_enable_key); +resctrl_arch_enable_alloc ();if (rdt_mon_capable) -static_branch_enable_cpuslocked (&rdt_mon_enable_key); +resctrl_arch_enable_mon ();if (rdt_alloc_capable || rdt_mon_capable) {static_branch_enable_cpuslocked (&rdt_enable_key);
@@ -2515,8 +2515,8 @@ static void rdt_kill_sb(struct super_block *sb)rdt_pseudo_lock_release (); rdtgroup_default.mode =RDT_MODE_SHAREABLE ;schemata_list_destroy (); -static_branch_disable_cpuslocked (&rdt_alloc_enable_key); -static_branch_disable_cpuslocked (&rdt_mon_enable_key); +resctrl_arch_disable_alloc (); +resctrl_arch_disable_mon ();static_branch_disable_cpuslocked (&rdt_enable_key); resctrl_mounted = false;kernfs_kill_sb (sb);
Signature · 1 line
2.30.2
James Morse
reviewerPATCH 13/18 x86/resctrl: Make rdt_enable_key the arch's decision to switch
In reply to James Morserdt_enable_key is switched when resctrl is mounted. It was also previously used to prevent a second mount of the filesystem.
Any other architecture that wants to support resctrl has to provide identical static keys.
Now that we have helpers for enablign and disabling the alloc/mon keys, resctrl doesn't need to switch this extra key, it can be done by the arch code. Use the static-key increment and decrement helpers, and change resctrl to ensure the calls are balanced.
--- arch/x86/include/asm/resctrl.h | 4 ++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 11 +++++------ 2 files changed, 9 insertions(+), 6 deletions(-)
arch/x86/include/asm/resctrl.h +4 -0
@@ -40,21 +40,25 @@ DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key);static inline void resctrl_arch_enable_alloc (void ) {static_branch_enable_cpuslocked (&rdt_alloc_enable_key); +static_branch_inc_cpuslocked (&rdt_enable_key); }static inline void resctrl_arch_disable_alloc (void ) {static_branch_disable_cpuslocked (&rdt_alloc_enable_key); +static_branch_dec_cpuslocked (&rdt_enable_key); }static inline void resctrl_arch_enable_mon (void ) {static_branch_enable_cpuslocked (&rdt_mon_enable_key); +static_branch_inc_cpuslocked (&rdt_enable_key); }static inline void resctrl_arch_disable_mon (void ) {static_branch_disable_cpuslocked (&rdt_mon_enable_key); +static_branch_dec_cpuslocked (&rdt_enable_key); }/*
arch/x86/kernel/cpu/resctrl/rdtgroup.c +5 -6
@@ -2248,10 +2248,8 @@ static int rdt_get_tree(struct fs_context *fc)if (rdt_mon_capable)resctrl_arch_enable_mon (); -if (rdt_alloc_capable || rdt_mon_capable) { -static_branch_enable_cpuslocked (&rdt_enable_key); +if (rdt_alloc_capable || rdt_mon_capable) resctrl_mounted = true; - }if (is_mbm_enabled ()) { r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;
@@ -2515,9 +2513,10 @@ static void rdt_kill_sb(struct super_block *sb)rdt_pseudo_lock_release (); rdtgroup_default.mode =RDT_MODE_SHAREABLE ;schemata_list_destroy (); -resctrl_arch_disable_alloc (); -resctrl_arch_disable_mon (); -static_branch_disable_cpuslocked (&rdt_enable_key); +if (rdt_alloc_capable) +resctrl_arch_disable_alloc (); +if (rdt_mon_capable) +resctrl_arch_disable_mon (); resctrl_mounted = false;kernfs_kill_sb (sb);mutex_unlock (&rdtgroup_mutex);
Signature · 1 line
2.30.2
James Morse
reviewerPATCH 14/18 x86/resctrl: Add helpers for system wide mon/alloc capable
In reply to James Morseresctrl reads rdt_alloc_capable or rdt_mon_capable to determine whether any of the resources support the corresponding features. resctrl also uses the static-keys that affect the architecture's context-switch code to determine the same thing.
This forces another architecture to have the same static-keys.
As the static-key is enabled based on the capable flag, and none of the filesystem uses of these are in the scheduler path, move the capable flags behind helpers, and use these in the filesystem code instead of the static-key.
After this change, only the architecture code manages and uses the static-keys to ensure __resctrl_sched_in() does not need runtime checks.
This avoids multiple architectures having to define the same static-keys.
--- arch/x86/include/asm/resctrl.h | 13 +++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 2 -- arch/x86/kernel/cpu/resctrl/monitor.c | 4 +-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 ++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 32 +++++++++++------------ 5 files changed, 34 insertions(+), 23 deletions(-)
arch/x86/include/asm/resctrl.h +13 -0
@@ -33,10 +33,18 @@ struct resctrl_pqr_state { DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state); +extern bool rdt_alloc_capable; +extern bool rdt_mon_capable; +DECLARE_STATIC_KEY_FALSE (rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); +static inline bool resctrl_arch_alloc_capable (void ) +{ +return rdt_alloc_capable; +} +static inline void resctrl_arch_enable_alloc (void ) {static_branch_enable_cpuslocked (&rdt_alloc_enable_key);
@@ -49,6 +57,11 @@ static inline void resctrl_arch_disable_alloc(void)static_branch_dec_cpuslocked (&rdt_enable_key); } +static inline bool resctrl_arch_mon_capable (void ) +{ +return rdt_mon_capable; +} +static inline void resctrl_arch_enable_mon (void ) {static_branch_enable_cpuslocked (&rdt_mon_enable_key);
arch/x86/kernel/cpu/resctrl/internal.h +0 -2
@@ -98,8 +98,6 @@ struct rmid_read {int arch_mon_ctx; }; -extern bool rdt_alloc_capable; -extern bool rdt_mon_capable;extern unsigned int rdt_mon_features;extern struct list_head resctrl_schema_all;extern bool resctrl_mounted;
arch/x86/kernel/cpu/resctrl/monitor.c +2 -2
@@ -788,7 +788,7 @@ void mbm_handle_overflow(struct work_struct *work)mutex_lock (&rdtgroup_mutex); -if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key)) +if (!resctrl_mounted || !resctrl_arch_mon_capable ()) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;
@@ -816,7 +816,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)unsigned long delay =msecs_to_jiffies (delay_ms);int cpu; -if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key)) +if (!resctrl_mounted || !resctrl_arch_mon_capable ())return ; cpu =cpumask_any (&dom->cpu_mask);
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3 -3
@@ -567,7 +567,7 @@ static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp)if (ret) goto err_cpus; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =rdtgroup_kn_mode_restrict (rdtgrp,"mon_groups" );if (ret) goto err_cpus_list;
@@ -614,7 +614,7 @@ static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp)if (ret) goto err_cpus; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =rdtgroup_kn_mode_restore (rdtgrp,"mon_groups" ,0777 );if (ret) goto err_cpus_list;
@@ -762,7 +762,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) {int ret; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) {rdt_last_cmd_puts ("Out of RMIDs\n" );
arch/x86/kernel/cpu/resctrl/rdtgroup.c +16 -16
@@ -610,13 +610,13 @@ static int __rdtgroup_move_task(struct task_struct *tsk,static bool is_closid_match (struct task_struct *t,struct rdtgroup *r) { -return (rdt_alloc_capable && (r->type ==RDTCTRL_GROUP ) && +return (resctrl_arch_alloc_capable () && (r->type ==RDTCTRL_GROUP ) &&resctrl_arch_match_closid (t, r->closid)); }static bool is_rmid_match (struct task_struct *t,struct rdtgroup *r) { -return (rdt_mon_capable && (r->type ==RDTMON_GROUP ) && +return (resctrl_arch_mon_capable () && (r->type ==RDTMON_GROUP ) &&resctrl_arch_match_rmid (t, r->mon.parent->closid, r->mon.rmid)); }
@@ -2221,7 +2221,7 @@ static int rdt_get_tree(struct fs_context *fc)if (ret <0 ) goto out_schemata_free; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =mongroup_create_dir (rdtgroup_default.kn, &rdtgroup_default,"mon_groups" , &kn_mongrp);
@@ -2243,12 +2243,12 @@ static int rdt_get_tree(struct fs_context *fc)if (ret <0 ) goto out_psl; -if (rdt_alloc_capable) +if (resctrl_arch_alloc_capable ())resctrl_arch_enable_alloc (); -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())resctrl_arch_enable_mon (); -if (rdt_alloc_capable || rdt_mon_capable) +if (resctrl_arch_alloc_capable () ||resctrl_arch_mon_capable ()) resctrl_mounted = true;if (is_mbm_enabled ()) {
@@ -2262,10 +2262,10 @@ static int rdt_get_tree(struct fs_context *fc) out_psl:rdt_pseudo_lock_release (); out_mondata: -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())kernfs_remove (kn_mondata); out_mongrp: -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())kernfs_remove (kn_mongrp); out_info:kernfs_remove (kn_info);
@@ -2513,9 +2513,9 @@ static void rdt_kill_sb(struct super_block *sb)rdt_pseudo_lock_release (); rdtgroup_default.mode =RDT_MODE_SHAREABLE ;schemata_list_destroy (); -if (rdt_alloc_capable) +if (resctrl_arch_alloc_capable ())resctrl_arch_disable_alloc (); -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())resctrl_arch_disable_mon (); resctrl_mounted = false;kernfs_kill_sb (sb);
@@ -2890,7 +2890,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) {int ret; -if (!rdt_mon_capable) +if (!resctrl_arch_mon_capable ())return 0 ; ret =alloc_rmid (rdtgrp->closid);
@@ -3076,7 +3076,7 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn,list_add (&rdtgrp->rdtgroup_list, &rdt_all_groups); -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) {/* * Create an empty mon_groups directory to hold the subset * of tasks and cpus to monitor.
@@ -3131,14 +3131,14 @@ static int rdtgroup_mkdir(struct kernfs_node *parent_kn, const char *name,* allocation is supported, add a control and monitoring * subdirectory */ -if (rdt_alloc_capable && parent_kn == rdtgroup_default.kn) +if (resctrl_arch_alloc_capable () && parent_kn == rdtgroup_default.kn)return rdtgroup_mkdir_ctrl_mon (parent_kn, name, mode);/* * If RDT monitoring is supported and the parent directory is a valid * "mon_groups" directory, add a monitoring subdirectory. */ -if (rdt_mon_capable &&is_mon_groups (parent_kn, name)) +if (resctrl_arch_mon_capable () &&is_mon_groups (parent_kn, name))return rdtgroup_mkdir_mon (parent_kn, name, mode);return -EPERM ;
@@ -3342,7 +3342,7 @@ void resctrl_offline_domain(struct rdt_resource *r, struct rdt_domain *d)* If resctrl is mounted, remove all the * per domain monitor data directories. */ -if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key)) +if (resctrl_mounted &&resctrl_arch_mon_capable ())rmdir_mondata_subdir_allrdtgrp (r, d->id);if (is_mbm_enabled ())
@@ -3419,7 +3419,7 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)if (is_llc_occupancy_enabled ())INIT_DELAYED_WORK (&d->cqm_limbo, cqm_handle_limbo); -if (resctrl_mounted &&static_branch_unlikely (&rdt_mon_enable_key)) +if (resctrl_mounted &&resctrl_arch_mon_capable ())mkdir_mondata_subdir_allrdtgrp (r, d);return 0 ;
Signature · 1 line
2.30.2
RE: [PATCH 14/18] x86/resctrl: Add helpers for system wide mon/alloc capable
Hi James,
Quoted 257 lines
resctrl reads
rdt_alloc_capableorrdt_mon_capableto determine whether any of the resources support the corresponding features.
resctrl also uses the static-keys that affect the architecture's context-switch code to determine the same thing.This forces another architecture to have the same static-keys.
As the static-key is enabled based on the capable flag, and none of the filesystem uses of these are in the scheduler path, move the capable flags behind helpers, and use these in the filesystem code instead of the static-key.
After this change, only the architecture code manages and uses the static-keys to ensure
__resctrl_sched_in()does not need runtime checks.This avoids multiple architectures having to define the same static-keys.
Signed-off-by James Morse <[email protected]>--- arch/x86/include/asm/resctrl.h | 13 +++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 2 -- arch/x86/kernel/cpu/resctrl/monitor.c | 4 +-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 6 ++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 32 +++++++++++------------ 5 files changed, 34 insertions(+), 23 deletions(-)arch/x86/include/asm/resctrl.h +11
-0@@ -33,10 +33,18 @@ struct resctrl_pqr_state { DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state); +extern bool rdt_alloc_capable; +extern bool rdt_mon_capable; +DECLARE_STATIC_KEY_FALSE (rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); +static inline bool resctrl_arch_alloc_capable (void ) { +return rdt_alloc_capable; +} +static inline void resctrl_arch_enable_alloc (void ) {static_branch_enable_cpuslocked (&rdt_alloc_enable_key);@@ -49,6 +57,11 @@ static inline void resctrl_arch_disable_alloc(void)static_branch_dec_cpuslocked (&rdt_enable_key); } +static inline bool resctrl_arch_mon_capable (void ) { +return rdt_mon_capable; +} +static inline void resctrl_arch_enable_mon (void ) {static_branch_enable_cpuslocked (&rdt_mon_enable_key);arch/x86/kernel/cpu/resctrl/internal.h +0
-3@@ -98,8 +98,6 @@ struct rmid_read {int arch_mon_ctx; }; -extern bool rdt_alloc_capable; -extern bool rdt_mon_capable;extern unsigned int rdt_mon_features;extern struct list_head resctrl_schema_all;extern bool resctrl_mounted; diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c
b/arch/x86/kernel/cpu/resctrl/monitor.c
index043fcb32435f..a9af7c56a04a100644arch/x86/kernel/cpu/resctrl/monitor.c +2
-2@@ -788,7 +788,7 @@ void mbm_handle_overflow(struct work_struct *work)mutex_lock (&rdtgroup_mutex); -if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key)) +if (!resctrl_mounted || !resctrl_arch_mon_capable ()) goto out_unlock; r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;@@ -816,7 +816,7 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)unsigned long delay =msecs_to_jiffies (delay_ms);int cpu; -if (!resctrl_mounted || !static_branch_likely (&rdt_mon_enable_key)) +if (!resctrl_mounted || !resctrl_arch_mon_capable ())return ;arch/x86/kernel/cpu/resctrl/monitor.c +0
-0cpu =cpumask_any (&dom->cpu_mask);arch/x86/kernel/cpu/resctrl/pseudo_lock.c +1
-1@@ -567,7 +567,7 @@ static int rdtgroup_locksetup_user_restrict(struct rdtgroup *rdtgrp) if (ret) goto err_cpus; -if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { ret =rdtgroup_kn_mode_restrict (rdtgrp,"mon_groups" );if (ret)arch/x86/kernel/cpu/resctrl/pseudo_lock.c +1
-1goto err_cpus_list;@@ -614,7 +614,7 @@ static int rdtgroup_locksetup_user_restore(struct rdtgroup *rdtgrp) if (ret) goto err_cpus; -if (rdt_mon_capable) { + if (resctrl_arch_mon_capable()) { ret =rdtgroup_kn_mode_restore (rdtgrp,"mon_groups" ,0777 );if (ret)arch/x86/kernel/cpu/resctrl/pseudo_lock.c +1
-1goto err_cpus_list;@@ -762,7 +762,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) {int ret; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =alloc_rmid (rdtgrp->closid);if (ret <0 ) { rdt_last_cmd_puts("Out of RMIDs\n"); diff --git
a/arch/x86/kernel/cpu/resctrl/rdtgroup.cb/arch/x86/kernel/cpu/resctrl/rdtgroup.c
indexf076e516f381..81ed458be70b100644arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1@@ -610,13 +610,13 @@ static int __rdtgroup_move_task(struct task_struct *tsk, static bool is_closid_match(struct task_struct *t, struct rdtgroup *r) { - return (rdt_alloc_capable && (r->type == RDTCTRL_GROUP) && + return (resctrl_arch_alloc_capable() && (r->type ==
RDTCTRL_GROUP) &&arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1resctrl_arch_match_closid(t, r->closid)); }static bool is_rmid_match (struct task_struct *t,struct rdtgroup *r) { - return (rdt_mon_capable && (r->type == RDTMON_GROUP) && + return (resctrl_arch_mon_capable() && (r->type ==
RDTMON_GROUP) &&arch/x86/kernel/cpu/resctrl/rdtgroup.c +10
-10resctrl_arch_match_rmid (t, r->mon.parent->closid, r->mon.rmid)); }@@ -2221,7 +2221,7 @@ static int rdt_get_tree(struct fs_context *fc)if (ret <0 ) goto out_schemata_free; -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) { ret =mongroup_create_dir (rdtgroup_default.kn, &rdtgroup_default,"mon_groups" , &kn_mongrp);@@ -2243,12 +2243,12 @@ static int rdt_get_tree(struct fs_context *fc)if (ret <0 ) goto out_psl; -if (rdt_alloc_capable) +if (resctrl_arch_alloc_capable ())resctrl_arch_enable_alloc (); -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())resctrl_arch_enable_mon (); -if (rdt_alloc_capable || rdt_mon_capable) +if (resctrl_arch_alloc_capable () ||resctrl_arch_mon_capable ()) resctrl_mounted = true;if (is_mbm_enabled ()) {@@ -2262,10 +2262,10 @@ static int rdt_get_tree(struct fs_context *fc) out_psl:rdt_pseudo_lock_release (); out_mondata: -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())kernfs_remove (kn_mondata); out_mongrp: -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())kernfs_remove (kn_mongrp); out_info:kernfs_remove (kn_info);@@ -2513,9 +2513,9 @@ static void rdt_kill_sb(struct super_block *sb)rdt_pseudo_lock_release (); rdtgroup_default.mode =RDT_MODE_SHAREABLE ;schemata_list_destroy (); -if (rdt_alloc_capable) +if (resctrl_arch_alloc_capable ())resctrl_arch_disable_alloc (); -if (rdt_mon_capable) +if (resctrl_arch_mon_capable ())resctrl_arch_disable_mon (); resctrl_mounted = false;kernfs_kill_sb (sb);@@ -2890,7 +2890,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) {int ret; -if (!rdt_mon_capable) +if (!resctrl_arch_mon_capable ())return 0 ; ret =alloc_rmid (rdtgrp->closid);@@ -3076,7 +3076,7 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn,list_add (&rdtgrp->rdtgroup_list, &rdt_all_groups); -if (rdt_mon_capable) { +if (resctrl_arch_mon_capable ()) {/* * Create an empty mon_groups directory to hold the subset arch/x86/kernel/cpu/resctrl/rdtgroup.c +2
-2* of tasks and cpus to monitor. @@ -3131,14 +3131,14 @@ static int rdtgroup_mkdir(struct kernfs_node*parent_kn, const char *name, * allocation is supported, add a control and monitoring * subdirectory */ -if (rdt_alloc_capable && parent_kn == rdtgroup_default.kn) +if (resctrl_arch_alloc_capable () && parent_kn == rdtgroup_default.kn)return rdtgroup_mkdir_ctrl_mon (parent_kn, name, mode);/* * If RDT monitoring is supported and the parent directory is a valid * "mon_groups" directory, add a monitoring subdirectory. */ -if (rdt_mon_capable &&is_mon_groups (parent_kn, name)) +if (resctrl_arch_mon_capable () &&is_mon_groups (parent_kn, name))arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1return rdtgroup_mkdir_mon (parent_kn, name, mode);return -EPERM ;@@ -3342,7 +3342,7 @@ void resctrl_offline_domain(struct rdt_resource *r,struct rdt_domain *d) * If resctrl is mounted, remove all the * per domain monitor data directories. */ -if (resctrl_mounted && static_branch_unlikely(&rdt_mon_enable_key)) +if (resctrl_mounted &&resctrl_arch_mon_capable ())rmdir_mondata_subdir_allrdtgrp (r, d->id);arch/x86/kernel/cpu/resctrl/rdtgroup.c +1
-1if (is_mbm_enabled ())@@ -3419,7 +3419,7 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)if (is_llc_occupancy_enabled ())INIT_DELAYED_WORK (&d->cqm_limbo, cqm_handle_limbo); -if (resctrl_mounted && static_branch_unlikely(&rdt_mon_enable_key)) + if (resctrl_mounted && resctrl_arch_mon_capable())
arch/x86/kernel/cpu/resctrl/rdtgroup.c int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d) · line 3425
Quoted 3 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c +0
-0mkdir_mondata_subdir_allrdtgrp (r, d);return 0 ;
arch/x86/kernel/cpu/resctrl/rdtgroup.c
static void mkdir_rdt_prepare_rmid_free(struct rdtgroup *rgrp)
{
if (rdt_mon_capable)
free_rmid(rgrp->closid, rgrp->mon.rmid); }
I think you forgot to fix rdt_mon_capable here. if (rdt_mon_capable) => if (resctrl_arch_mon_capable())
Best regards,
Shaopeng Tan
James Morse
reviewerRe: [PATCH 14/18] x86/resctrl: Add helpers for system wide mon/alloc capable
Hello!
Quoted 16 lines · Shaopeng Tan (Fujitsu)
Quoted 7 lines
resctrl reads
rdt_alloc_capableorrdt_mon_capableto determine whether any of the resources support the corresponding features.
resctrl also uses the static-keys that affect the architecture's context-switch code to determine the same thing.This forces another architecture to have the same static-keys.
Quoted 8 lines
As the static-key is enabled based on the capable flag, and none of the filesystem uses of these are in the scheduler path, move the capable flags behind helpers, and use these in the filesystem code instead of the static-key.
After this change, only the architecture code manages and uses the static-keys to ensure
__resctrl_sched_in()does not need runtime checks.This avoids multiple architectures having to define the same static-keys.
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/rdtgroup.c
static voidmkdir_rdt_prepare_rmid_free(struct rdtgroup *rgrp){ if (rdt_mon_capable) free_rmid(rgrp->closid, rgrp->mon.rmid);}
I think you forgot to fixrdt_mon_capablehere. if (rdt_mon_capable) => if (resctrl_arch_mon_capable())
Yes - looks like I did.
Thanks!
James
James Morse
reviewerPATCH 15/18 x86/resctrl: Add cpu online callback for resctrl work
In reply to James MorseThe resctrl architecture specific code may need to create a domain when a CPU comes online, it also needs to reset the CPUs PQR_ASSOC register. The resctrl filesystem code needs to update the rdtgroup_default cpu mask when cpus are brought online.
Currently this is all done in one function, resctrl_online_cpu(). This will need to be split into architecture and filesystem parts before resctrl can be moved to /fs/.
Pull the rdtgroup_default update work out as a filesystem specific cpu_online helper. resctrl_online_cpu() is the obvious name for this, which means the version in core.c needs renaming.
resctrl_online_cpu() is called by the arch code once it has done the work to add the new cpu to any domains.
In future patches, resctrl_online_cpu() will take the rdtgroup_mutex itself.
--- arch/x86/kernel/cpu/resctrl/core.c | 11 ++++++----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 10 ++++++++++ include/linux/resctrl.h | 1 + 3 files changed, 17 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +6 -5
@@ -584,19 +584,20 @@ static void clear_closid_rmid(int cpu)wrmsr (IA32_PQR_ASSOC ,0 ,0 ); } -static int resctrl_online_cpu (unsigned int cpu) +static int resctrl_arch_online_cpu (unsigned int cpu) {struct rdt_resource *r; +int err;mutex_lock (&rdtgroup_mutex);for_each_capable_rdt_resource (r)domain_add_cpu (cpu, r); -/* The cpu is set in default rdtgroup after online. */ -cpumask_set_cpu (cpu, &rdtgroup_default.cpu_mask);clear_closid_rmid (cpu); + + err =resctrl_online_cpu (cpu);mutex_unlock (&rdtgroup_mutex); -return 0 ; +return err; }static void clear_childcpus (struct rdtgroup *r,unsigned int cpu)
@@ -923,7 +924,7 @@ static int __init resctrl_late_init(void) state =cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"x86/resctrl/cat:online:" , - resctrl_online_cpu, resctrl_offline_cpu); + resctrl_arch_online_cpu, resctrl_offline_cpu);if (state <0 )return state;
arch/x86/kernel/cpu/resctrl/rdtgroup.c +10 -0
@@ -3425,6 +3425,16 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)return 0 ; } +int resctrl_online_cpu (unsigned int cpu) +{ +lockdep_assert_held (&rdtgroup_mutex); + +/* The cpu is set in default rdtgroup after online. */ +cpumask_set_cpu (cpu, &rdtgroup_default.cpu_mask); + +return 0 ; +} +/* * rdtgroup_init - rdtgroup initialization *
include/linux/resctrl.h +1 -0
@@ -219,6 +219,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d, u32 closid, enum resctrl_conf_type type);int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d);void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d); +int resctrl_online_cpu (unsigned int cpu);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
Signature · 1 line
2.30.2
James Morse
reviewerPATCH 16/18 x86/resctrl: Allow overflow/limbo handlers to be scheduled on any-but cpu
In reply to James MorseWhen a cpu is taken offline resctrl may need to move the overflow or limbo handlers to run on a different CPU.
Once the offline callbacks have been split, cqm_setup_limbo_handler() will be called while the CPU that is going offline is still present in the cpu_mask.
Pass the CPU to exclude to cqm_setup_limbo_handler() and mbm_setup_overflow_handler(). These functions can use cpumask_any_but() when selecting the CPU. -1 is used to indicate no CPUs need excluding.
---
Both cpumask_any() and cpumask_any_but() return a value >= nr_cpu_ids on error. schedule_delayed_work_on() doesn't appear to check. Add the error handling to be robust. It doesn't look like its possible to hit this.
--- arch/x86/kernel/cpu/resctrl/core.c | 6 ++-- arch/x86/kernel/cpu/resctrl/internal.h | 6 ++-- arch/x86/kernel/cpu/resctrl/monitor.c | 39 +++++++++++++++++++++----- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 4 +-- 4 files changed, 42 insertions(+), 13 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +4 -2
@@ -563,12 +563,14 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)if (r == &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl) {if (is_mbm_enabled () && cpu == d->mbm_work_cpu) {cancel_delayed_work (&d->mbm_over); -mbm_setup_overflow_handler (d,0 ); +/* exclude_cpu=-1 as we already cpumask_clear_cpu()d */ +mbm_setup_overflow_handler (d,0 ,-1 ); }if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu &&has_busy_rmid (r, d)) {cancel_delayed_work (&d->cqm_limbo); -cqm_setup_limbo_handler (d,0 ); +/* exclude_cpu=-1 as we already cpumask_clear_cpu()d */ +cqm_setup_limbo_handler (d,0 ,-1 ); } } }
arch/x86/kernel/cpu/resctrl/internal.h +4 -2
@@ -525,11 +525,13 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, struct rdt_domain *d, struct rdtgroup *rdtgrp, int evtid, int first);void mbm_setup_overflow_handler (struct rdt_domain *dom, -unsigned long delay_ms); +unsigned long delay_ms, +int exclude_cpu);void mbm_handle_overflow (struct work_struct *work);void __init intel_rdt_mbm_apply_quirk (void );bool is_mba_sc (struct rdt_resource *r); -void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms); +void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms, +int exclude_cpu);void cqm_handle_limbo (struct work_struct *work);bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d);void __check_limbo (struct rdt_domain *d,bool force_free);
arch/x86/kernel/cpu/resctrl/monitor.c +32 -7
@@ -433,7 +433,7 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)* setup up the limbo worker. */ if (!has_busy_rmid (r, d)) -cqm_setup_limbo_handler (d,CQM_LIMBOCHECK_INTERVAL ); +cqm_setup_limbo_handler (d,CQM_LIMBOCHECK_INTERVAL ,-1 );set_bit (idx, d->rmid_busy_llc); entry->busy++; }
@@ -766,15 +766,27 @@ void cqm_handle_limbo(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms) +/** +* cqm_setup_limbo_handler() - Schedule the limbo handler to run for this +* domain. +* @delay_ms: How far in the future the handler should run. +* @exclude_cpu: Which CPU the handler should not run on, -1 to pick any CPU. +*/ +void cqm_setup_limbo_handler (struct rdt_domain *dom,unsigned long delay_ms, +int exclude_cpu) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu; - cpu =cpumask_any (&dom->cpu_mask); +if (exclude_cpu ==-1 ) + cpu =cpumask_any (&dom->cpu_mask); +else + cpu =cpumask_any_but (&dom->cpu_mask, exclude_cpu); + dom->cqm_work_cpu = cpu; -schedule_delayed_work_on (cpu, &dom->cqm_limbo, delay); +if (cpu < nr_cpu_ids) +schedule_delayed_work_on (cpu, &dom->cqm_limbo, delay); }void mbm_handle_overflow (struct work_struct *work)
@@ -811,7 +823,14 @@ void mbm_handle_overflow(struct work_struct *work)mutex_unlock (&rdtgroup_mutex); } -void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms) +/** +* mbm_setup_overflow_handler() - Schedule the overflow handler to run for this +* domain. +* @delay_ms: How far in the future the handler should run. +* @exclude_cpu: Which CPU the handler should not run on, -1 to pick any CPU. +*/ +void mbm_setup_overflow_handler (struct rdt_domain *dom,unsigned long delay_ms, +int exclude_cpu) {unsigned long delay =msecs_to_jiffies (delay_ms);int cpu;
@@ -819,9 +838,15 @@ void mbm_setup_overflow_handler(struct rdt_domain *dom, unsigned long delay_ms)if (!resctrl_mounted || !resctrl_arch_mon_capable ())return ; - cpu =cpumask_any (&dom->cpu_mask); +if (exclude_cpu ==-1 ) + cpu =cpumask_any (&dom->cpu_mask); +else + cpu =cpumask_any_but (&dom->cpu_mask, exclude_cpu); + dom->mbm_work_cpu = cpu; -schedule_delayed_work_on (cpu, &dom->mbm_over, delay); + +if (cpu < nr_cpu_ids) +schedule_delayed_work_on (cpu, &dom->mbm_over, delay); }static int dom_data_init (struct rdt_resource *r)
arch/x86/kernel/cpu/resctrl/rdtgroup.c +2 -2
@@ -2254,7 +2254,7 @@ static int rdt_get_tree(struct fs_context *fc)if (is_mbm_enabled ()) { r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;list_for_each_entry (dom, &r->domains, list) -mbm_setup_overflow_handler (dom,MBM_OVERFLOW_INTERVAL ); +mbm_setup_overflow_handler (dom,MBM_OVERFLOW_INTERVAL ,-1 ); } goto out;
@@ -3413,7 +3413,7 @@ int resctrl_online_domain(struct rdt_resource *r, struct rdt_domain *d)if (is_mbm_enabled ()) {INIT_DELAYED_WORK (&d->mbm_over, mbm_handle_overflow); -mbm_setup_overflow_handler (d,MBM_OVERFLOW_INTERVAL ); +mbm_setup_overflow_handler (d,MBM_OVERFLOW_INTERVAL ,-1 ); }if (is_llc_occupancy_enabled ())
Signature · 1 line
2.30.2
James Morse
reviewerPATCH 17/18 x86/resctrl: Add cpu offline callback for resctrl work
In reply to James MorseThe resctrl architecture specific code may need to free a domain when a CPU goes offline, it also needs to reset the CPUs PQR_ASSOC register. The resctrl filesystem code needs to move the overflow and limbo work to run on a different CPU, and clear this CPU from the cpu_mask of control and monitor groups.
Currently this is all done in core.c and called from resctrl_offline_cpu(), making the split between architecture and filesystem code unclear.
Move the filesystem work into a filesystem helper called resctrl_offline_cpu(), and rename the one in core.c resctrl_arch_offline_cpu().
The rdtgroup_mutex is unlocked and locked again in the call in preparation for changing the locking rules for the architecture code.
resctrl_offline_cpu() is called before any of the resource/domains are updated, and makes use of the exclude_cpu feature we previously added.
--- arch/x86/kernel/cpu/resctrl/core.c | 39 ++++---------------------- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 39 ++++++++++++++++++++++++++ include/linux/resctrl.h | 1 + 3 files changed, 45 insertions(+), 34 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +5 -34
@@ -559,20 +559,6 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)return ; } - -if (r == &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl) { -if (is_mbm_enabled () && cpu == d->mbm_work_cpu) { -cancel_delayed_work (&d->mbm_over); -/* exclude_cpu=-1 as we already cpumask_clear_cpu()d */ -mbm_setup_overflow_handler (d,0 ,-1 ); - } -if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu && -has_busy_rmid (r, d)) { -cancel_delayed_work (&d->cqm_limbo); -/* exclude_cpu=-1 as we already cpumask_clear_cpu()d */ -cqm_setup_limbo_handler (d,0 ,-1 ); - } - } }static void clear_closid_rmid (int cpu)
@@ -602,31 +588,15 @@ static int resctrl_arch_online_cpu(unsigned int cpu)return err; } -static void clear_childcpus (struct rdtgroup *r,unsigned int cpu) +static int resctrl_arch_offline_cpu (unsigned int cpu) { -struct rdtgroup *cr; - -list_for_each_entry (cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) { -if (cpumask_test_and_clear_cpu (cpu, &cr->cpu_mask)) { -break ; - } - } -} - -static int resctrl_offline_cpu (unsigned int cpu) -{ -struct rdtgroup *rdtgrp;struct rdt_resource *r;mutex_lock (&rdtgroup_mutex); +resctrl_offline_cpu (cpu); +for_each_capable_rdt_resource (r)domain_remove_cpu (cpu, r); -list_for_each_entry (rdtgrp, &rdt_all_groups, rdtgroup_list) { -if (cpumask_test_and_clear_cpu (cpu, &rdtgrp->cpu_mask)) { -clear_childcpus (rdtgrp, cpu); -break ; - } - }clear_closid_rmid (cpu);mutex_unlock (&rdtgroup_mutex);
@@ -926,7 +896,8 @@ static int __init resctrl_late_init(void) state =cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"x86/resctrl/cat:online:" , - resctrl_arch_online_cpu, resctrl_offline_cpu); + resctrl_arch_online_cpu, + resctrl_arch_offline_cpu);if (state <0 )return state;
arch/x86/kernel/cpu/resctrl/rdtgroup.c +39 -0
@@ -3435,6 +3435,45 @@ int resctrl_online_cpu(unsigned int cpu)return 0 ; } +static void clear_childcpus (struct rdtgroup *r,unsigned int cpu) +{ +struct rdtgroup *cr; + +list_for_each_entry (cr, &r->mon.crdtgrp_list, mon.crdtgrp_list) { +if (cpumask_test_and_clear_cpu (cpu, &cr->cpu_mask)) +break ; + } +} + +void resctrl_offline_cpu (unsigned int cpu) +{ +struct rdt_domain *d; +struct rdtgroup *rdtgrp; +struct rdt_resource *l3 = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; + +lockdep_assert_held (&rdtgroup_mutex); + +list_for_each_entry (rdtgrp, &rdt_all_groups, rdtgroup_list) { +if (cpumask_test_and_clear_cpu (cpu, &rdtgrp->cpu_mask)) { +clear_childcpus (rdtgrp, cpu); +break ; + } + } + + d =get_domain_from_cpu (cpu, l3); +if (d) { +if (is_mbm_enabled () && cpu == d->mbm_work_cpu) { +cancel_delayed_work (&d->mbm_over); +mbm_setup_overflow_handler (d,0 , cpu); + } +if (is_llc_occupancy_enabled () && cpu == d->cqm_work_cpu && +has_busy_rmid (l3, d)) { +cancel_delayed_work (&d->cqm_limbo); +cqm_setup_limbo_handler (d,0 , cpu); + } + } +} +/* * rdtgroup_init - rdtgroup initialization *
include/linux/resctrl.h +1 -0
@@ -220,6 +220,7 @@ u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_domain *d,int resctrl_online_domain (struct rdt_resource *r,struct rdt_domain *d);void resctrl_offline_domain (struct rdt_resource *r,struct rdt_domain *d);int resctrl_online_cpu (unsigned int cpu); +void resctrl_offline_cpu (unsigned int cpu);/** * resctrl_arch_rmid_read() - Read the eventid counter corresponding to rmid
Signature · 1 line
2.30.2
James Morse
reviewerPATCH 18/18 x86/resctrl: Separate arch and fs resctrl locks
In reply to James Morseresctrl has one mutex that is taken by the architecture specific code, and the filesystem parts. The two interact via cpuhp, where the architecture code updates the domain list. Filesystem handlers that walk the domains list should not run concurrently with the cpuhp callback modifying the list.
Exposing a lock from the filesystem code means the interface is not cleanly defined, and creates the possibility of cross-architecture lock ordering headaches. The interaction only exists so that certain filesystem paths are serialised against cpu hotplug. The cpu hotplug code already has a mechanism to do this using cpus_read_lock().
MPAM's monitors have an overflow interrupt, so it needs to be possible to walk the domains list in irq context. RCU is ideal for this, but some paths need to be able to sleep to allocate memory.
Because resctrl_{on,off}line_cpu() take the rdtgroup_mutex as part of a cpuhp callback, cpus_read_lock() must always be taken first. rdtgroup_schemata_write() already does this.
All but one of the filesystem code's domain list walkers are currently protected by the rdtgroup_mutex taken in rdtgroup_kn_lock_live(). The exception is rdt_bit_usage_show() which takes the lock directly.
Make the domain list protected by RCU. An architecture-specific lock prevents concurrent writers. rdt_bit_usage_show() can walk the domain list under rcu_read_lock().
The other filesystem list walkers need to be able to sleep. Add cpus_read_lock() to rdtgroup_kn_lock_live() so that the cpuhp callbacks can't be invoked when file system operations are occurring.
Add lockdep_assert_cpus_held() in the cases where the rdtgroup_kn_lock_live() call isn't obvious.
Resctrl's domain online/offline calls now need to take the rdtgroup_mutex themselves.
--- arch/x86/kernel/cpu/resctrl/core.c | 33 ++++++++------ arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 14 ++++-- arch/x86/kernel/cpu/resctrl/monitor.c | 3 ++ arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 3 ++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 54 ++++++++++++++++++++--- include/linux/resctrl.h | 2 +- 6 files changed, 84 insertions(+), 25 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +20 -13
@@ -25,8 +25,14 @@#include <asm/resctrl.h> #include "internal.h" -/* Mutex to protect rdtgroup access. */ -DEFINE_MUTEX (rdtgroup_mutex); +/* +* rdt_domain structures are kfree()d when their last cpu goes offline, +* and allocated when the first cpu in a new domain comes online. +* The rdt_resource's domain list is updated when this happens. The domain +* list is protected by RCU, but callers can also take the cpus_read_lock() +* to prevent modification if they need to sleep. All writers take this mutex: +*/ +static DEFINE_MUTEX (domain_list_lock);/* * The cached resctrl_pqr_state is strictly per CPU and can never be
@@ -489,6 +495,8 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)struct rdt_domain *d;int err; +lockdep_assert_held (&domain_list_lock); + d =rdt_find_domain (r, id, &add_pos);if (IS_ERR (d)) {pr_warn ("Couldn't find cache id for CPU %d\n" , cpu);
@@ -522,11 +530,12 @@ static void domain_add_cpu(int cpu, struct rdt_resource *r)return ; } -list_add_tail (&d->list, add_pos); +list_add_tail_rcu (&d->list, add_pos); err =resctrl_online_domain (r, d);if (err) { -list_del (&d->list); +list_del_rcu (&d->list); +synchronize_rcu ();domain_free (hw_dom); } }
@@ -547,7 +556,8 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r)cpumask_clear_cpu (cpu, &d->cpu_mask);if (cpumask_empty (&d->cpu_mask)) {resctrl_offline_domain (r, d); -list_del (&d->list); +list_del_rcu (&d->list); +synchronize_rcu ();/* * rdt_domain "d" is going to be freed below, so clear
@@ -575,30 +585,27 @@ static void clear_closid_rmid(int cpu)static int resctrl_arch_online_cpu (unsigned int cpu) {struct rdt_resource *r; -int err; -mutex_lock (&rdtgroup_mutex); +mutex_lock (&domain_list_lock);for_each_capable_rdt_resource (r)domain_add_cpu (cpu, r);clear_closid_rmid (cpu); +mutex_unlock (&domain_list_lock); - err =resctrl_online_cpu (cpu); -mutex_unlock (&rdtgroup_mutex); - -return err; +return resctrl_online_cpu (cpu); }static int resctrl_arch_offline_cpu (unsigned int cpu) {struct rdt_resource *r; -mutex_lock (&rdtgroup_mutex);resctrl_offline_cpu (cpu); +mutex_lock (&domain_list_lock);for_each_capable_rdt_resource (r)domain_remove_cpu (cpu, r);clear_closid_rmid (cpu); -mutex_unlock (&rdtgroup_mutex); +mutex_unlock (&domain_list_lock);return 0 ; }
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +10 -4
@@ -209,6 +209,9 @@ static int parse_line(char *line, struct resctrl_schema *s,struct rdt_domain *d;unsigned long dom_id; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP && r->rid ==RDT_RESOURCE_MBA ) {rdt_last_cmd_puts ("Cannot pseudo-lock MBA resource\n" );
@@ -314,6 +317,9 @@ int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid)int cpu;u32 idx; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +if (!zalloc_cpumask_var (&cpu_mask,GFP_KERNEL ))return -ENOMEM ;
@@ -384,11 +390,9 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,return -EINVAL ; buf[nbytes -1 ] ='\0' ; -cpus_read_lock (); rdtgrp =rdtgroup_kn_lock_live (of->kn);if (!rdtgrp) {rdtgroup_kn_unlock (of->kn); -cpus_read_unlock ();return -ENOENT ; }rdt_last_cmd_clear ();
@@ -452,7 +456,6 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, out:rdtgroup_kn_unlock (of->kn); -cpus_read_unlock ();return ret ?: nbytes; }
@@ -472,6 +475,9 @@ static void show_doms(struct seq_file *s, struct resctrl_schema *schema, int clobool sep = false;u32 ctrl_val; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +seq_printf (s,"%*s:" , max_name_width, schema->name);list_for_each_entry (dom, &r->domains, list) {if (sep)
@@ -534,7 +540,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r, int evtid, int first) {/* When picking a cpu from cpu_mask, ensure it can't race with cpuhp */ -lockdep_assert_held (&rdtgroup_mutex); +lockdep_assert_cpus_held ();/* * setup the parameters to pass to mon_event_count() to read the data.
arch/x86/kernel/cpu/resctrl/monitor.c +3 -0
@@ -414,6 +414,9 @@ static void add_rmid_to_limbo(struct rmid_entry *entry)u32 idx;int err; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); + idx =resctrl_arch_rmid_idx_encode (entry->closid, entry->rmid); arch_mon_ctx =resctrl_arch_mon_ctx_alloc (r,QOS_L3_OCCUP_EVENT_ID );
arch/x86/kernel/cpu/resctrl/pseudo_lock.c +3 -0
@@ -830,6 +830,9 @@ bool rdtgroup_pseudo_locked_in_hierarchy(struct rdt_domain *d)struct rdt_domain *d_i;bool ret = false; +/* Walking r->domains, ensure it can't race with cpuhp */ +lockdep_assert_cpus_held (); +if (!zalloc_cpumask_var (&cpu_with_psl,GFP_KERNEL ))return true;
arch/x86/kernel/cpu/resctrl/rdtgroup.c +47 -7
150 lines not on this page · raw message
include/linux/resctrl.h +1 -1
@@ -153,7 +153,7 @@ struct resctrl_schema;* @cache_level: Which cache level defines scope of this resource * @cache: Cache allocation related data * @membw: If the component has bandwidth controls, their properties. -* @domains: All domains for this resource +* @domains: RCU list of all domains for this resource * @name: Name to use in "schemata" file. * @data_width: Character width of data when displaying * @default_ctrl: Specifies default cache cbm or memory B/W percent.
Signature · 1 line
2.30.2
Re: [PATCH 18/18] x86/resctrl: Separate arch and fs resctrl locks
Hi James,
Quoted 4 lines · James Morse
MPAM's monitors have an overflow interrupt, so it needs to be possible to walk the domains list in irq context. RCU is ideal for this, but some paths need to be able to sleep to allocate memory.
I'm curious about this requirement. There are already counters which can overflow on Intel, but we've been able to detect overflows soon enough by checking at a reasonable interval. Are we expecting MSCs to have counters that overflow so quickly that the overflows need to be handled directly in IRQ context vs being able to run a threaded handler before the second overflow?
It seems like MBM would be really intrusive if it could cause the system to process overflow IRQs at a high rate.
Also is the overflow interrupt handler in one of your MPAM preview branches? I was only able to find an error IRQ handler in mpam/snapshot/v6.0:
-Peter
James Morse
reviewerRe: [PATCH 18/18] x86/resctrl: Separate arch and fs resctrl locks
Hi Peter,
Quoted 6 lines · Peter Newman
Quoted 4 lines · James Morse
MPAM's monitors have an overflow interrupt, so it needs to be possible to walk the domains list in irq context. RCU is ideal for this, but some paths need to be able to sleep to allocate memory.
Quoted 6 lines
I'm curious about this requirement. There are already counters which can overflow on Intel, but we've been able to detect overflows soon enough by checking at a reasonable interval. Are we expecting MSCs to have counters that overflow so quickly that the overflows need to be handled directly in IRQ context vs being able to run a threaded handler before the second overflow?
Ultimately, I don't know. MPAM has three sizes of counter, 31, 44 and 63. I think its entirely possible someone builds a system with an inconvenient size for the way they use it - but this wasn't how I anticipated this getting used...
Quoted 2 lines
It seems like MBM would be really intrusive if it could cause the system to process overflow IRQs at a high rate.
... I agree ..
Quoted 5 lines
Also is the overflow interrupt handler in one of your MPAM preview branches? I was only able to find an error IRQ handler in mpam/snapshot/v6.0:
Because there probably won't be enough monitors to expose the free-running resctrl files, I anticipate that most use of the memory bandwidth counters will be via perf, which gives MPAM the start/stop calls it needs to allocate and free a monitor.
The PMU driver gets asked to read the counters in IRQ context, see __perf_event_read() called via smp_call_function_single(). (I'm sure there are others).
This is the first reason why the domain list needs to be protected by something like RCU.
Perf also has a sampling mode, where it sets the value to overflow after a specific number of events, and times how long that takes to occur. (I haven't completely got my head round it yet) In this mode, the MPAM driver would need to invoke the PMU driver to read the counters in IRQ context.
I haven't written the overflow interrupt code yet because I've got no access to a platform (virtual or otherwise) with working counters, so couldn't possibly test it. (See also, the 44 and 63 bit counter support!)
I was being lazy with the commit message and only describing the last case. Is the future-plot-arc important? It would result in a bit of a brain-dump/essay in the commit message.
Thanks,
James
Hi James,
I tested this patch series on Intel(R) Xeon(R) Gold 6254 CPU with resctrl selftest. It is no problem.
I am going to review this patch series.
James Morse
reviewerHello,
Quoted 6 lines · Shaopeng Tan (Fujitsu)
I tested this patch series on Intel(R) Xeon(R) Gold 6254 CPU with resctrl selftest. It is no problem.
I am going to review this patch series.
Tested-by Shaopeng Tan <[email protected]>
Thanks!
James