x86/resctrl telemetry monitoring
Subsystems (5)
Tony Luck
maintainercover x86/resctrl telemetry monitoring
Preamble
I'm trying here to provide a picture of the capabilities of telemetry monitoring with a possible resctrl implementation. I'm looking for high level comments on my general approach (especially as it may interact with monitoring features that other architectures may want to add in the future.
The first patch in the series just provides a fake copy of the enumeration interface that should show up in the OOBMSM driver in the near future. It allows building, and running, of this series on Intel (and perhaps AMD) systems that don't have h/w support.
Background
Telemetry features are being implemented in conjunction with the IA32_PQR_ASSOC.RMID value on each logical CPU. This is used to send counts for various events to a collector in a nearby OOMMSM device to be accumulated with counts for each <RMID, event> pair received from other CPUs. Cores send event counts when the RMID value changes, or after each 2ms elapsed time.
Each OOBMSM device may implement multiple event collectors with each servicing a subset of the logical CPUs on a package. In the initial hardware implementation, there are two categories of events:
1) Energy - Two counterscore_energy: This is an estimate of Joules consumed by each core. It is calculated based on the types of instructions executed, not from a power meter. This counter is useful to understand how much energy a workload is consuming.
activity: This measures "accumulated dynamic capacitance". Users who want to optimize energy consumption for a workload may use this rather than core_energy because it provides consistent results independent of any frequency or voltage changes that may occur during the runtime of the application (e.g. entry/exit from turbo mode).
2) Performance - Seven counters
These are similar events to those available via the Linux "perf" tool, but collected in a way with mush lower overhead (no need to collect data on every context switch).
stalls_llc_hit - Counts the total number of unhalted core clock cycles when the core is stalled due to a demand load miss which hit in the LLC
c1_res - Counts the total C1 residency across all cores. The underlying counter increments on 100MHz clock ticks
unhalted_core_cycles - Counts the total number of unhalted core clock cycles
stalls_llc_miss - Counts the total number of unhalted core clock cycles when the core is stalled due to a demand load miss which missed all the local caches
c6_res - Counts the total C6 residency. The underlying counter increments on crystal clock (25MHz) ticks
unhalted_ref_cycles - Counts the total number of unhalted reference clock (TSC) cycles
uops_retired - Counts the total number of uops retired
Enumeration
The only CPUID based enumeration for this feature is the legacy CPUID(eax=7,ecx=0).ebx{12} that indicates the presence of the IA32_PQR_ASSOC MSR and the RMID field within it.
The OOBMSM driver discovers which features are present via PCIe VSEC capabilities. Each feature is tagged with a unique identifier. These identifiers indicate which XML description file from
https://github.com/intel/Intel-PMT describes which event counters are available and their layout within the MMIO BAR space of the OOBMSM device.
Resctrl User Interface
Because there may be multiple OOBMSM collection agents per processor package, resctrl accumulates event counts from all agents on a package and presents a single value to users. This will provide a consistent user interface on future platforms that vary the number of collectors, or the mappings from logical CPUs to collectors.
Users will see the legacy monitoring files in the "L3" directories and the telemetry files in "PKG" directories (with each file providing the aggregated value from all OOBMSM collectors on that package).
$ tree /sys/fs/resctrl/mon_data/
/sys/fs/resctrl/mon_data/
├── mon_L3_00
│ ├── llc_occupancy
│ ├── mbm_local_bytes
│ └── mbm_total_bytes
├── mon_L3_01
│ ├── llc_occupancy
│ ├── mbm_local_bytes
│ └── mbm_total_bytes
├── mon_PKG_00
│ ├── activity
│ ├── c1_res
│ ├── c6_res
│ ├── core_energy
│ ├── stalls_llc_hit
│ ├── stalls_llc_miss
│ ├── unhalted_core_cycles
│ ├── unhalted_ref_cycles
│ └── uops_retired
└── mon_PKG_01
├── activity
├── c1_res
├── c6_res
├── core_energy
├── stalls_llc_hit
├── stalls_llc_miss
├── unhalted_core_cycles
├── unhalted_ref_cycles
└── uops_retiredResctrl Implementation
The OOBMSM driver exposes a function "intel_pmt_get_regions_by_feature()" that returns an array of structures describing the per-RMID groups it found from the VSEC enumeration. Linux looks at the unique identifiers for each group and enables resctrl for all groups with known unique identifiers.
The memory map for the counters for each <RMID, event> pair is described by the XML file. This is too unwieldy to use in the Linux kernel, so a simplified representation is built into the resctrl code. Note that the counters are in MMIO space instead of accessed using the IA32_QM_EVTSEL and IA32_QM_CTR MSRs. This means there is no need for cross-processor calls to read counters from a CPU in a specific domain. The counters can be read from any CPU.
High level description of code changes:
1) New scope RESCTRL_PACKAGE
2) New struct rdt_resource RDT_RESOURCE_INTEL_PMT
3) Refactor monitor code paths to split existing L3 paths from new ones. In some cases this ends up with:
switch (r->rid) {
case RDT_RESOURCE_L3:
helper for L3
break;
case RDT_RESOURCE_INTEL_PMT:
helper for PKG
break;
}4) New source code file "intel_pmt.c" for the code to enumerate, configure, and report event counts.
With only one platform providing this feature, it's tricky to tell exactly where it is going to go. I've made the event definitions platform specific (based on the unique ID from the VSEC enumeration). It seems possible/likely that the list of events may change from generation to generation.
I've picked names for events based on the descriptions in the XML file.
Tony Luck (17):
x86/rectrl: Fake OOBMSM interface
x86/resctrl: Move L3 initialization out of domain_add_cpu_mon() x86/resctrl: Refactor domain_remove_cpu_mon() ready for new domain
types
x86/resctrl: Split L3 specific code out of rdtgroup_mondata_show() x86/resctrl: Change generic monitor functions to use struct
rdt_domain_hdr
x86/resctrl: Add and initialize rdt_resource for package scope core
monitor
x86/resctrl: Prepare for resource specific event ids x86/resctrl: Add initialization hook for Intel PMT events x86/resctrl: Add Intel PMT domain specific code x86/resctrl: Add detailed descriptions for Clearwater Forest events x86/resctrl: Allocate per-package structures for known events x86/resctrl: Add list of known events to RDT_RESOURCE_INTEL_PMT x86/resctrl: Build lookup table for package events x86/resctrl: Add code in rdtgroup_mondata_show() for core monitor
events
x86/resctrl: Add status files to info/PKG_MON x86/resctrl: Enable package event monitoring x86/resctrl: Update Documentation for package events
Documentation/arch/x86/resctrl.rst | 25 +- include/linux/resctrl.h | 11 +- .../cpu/resctrl/fake_intel_pmt_features.h | 72 +++ arch/x86/kernel/cpu/resctrl/internal.h | 51 +- arch/x86/kernel/cpu/resctrl/core.c | 105 ++-- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 82 +-- .../cpu/resctrl/fake_intel_pmt_features.c | 64 +++ arch/x86/kernel/cpu/resctrl/intel_pmt.c | 486 ++++++++++++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 87 +++- arch/x86/Kconfig | 1 + arch/x86/kernel/cpu/resctrl/Makefile | 2 + drivers/platform/x86/intel/pmt/Kconfig | 3 + 12 files changed, 882 insertions(+), 107 deletions(-) create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_pmt_features.h create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_pmt_features.c create mode 100644 arch/x86/kernel/cpu/resctrl/intel_pmt.c
base-commit: 7eb172143d5508b4da468ed59ee857c6e5e01da6
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 1/17 x86/rectrl: Fake OOBMSM interface
Real version is coming soon ... this is here so the remaining parts will build (and run ... assuming a 2 socket system that supports RDT monitoring ... only missing part is that the event counters just report fixed values).
Real version of this would just add the INTEL_PMT_RESCTRL Kconfig option with dependency checks on
INTEL_VSEC=y && INTEL_PMT_TELEMETRY=y && INTEL_PMT_DISCOVERY=y
Just for RFC discussion.
--- .../cpu/resctrl/fake_intel_pmt_features.h | 72 +++++++++++++++++++ .../cpu/resctrl/fake_intel_pmt_features.c | 64 +++++++++++++++++ arch/x86/Kconfig | 1 + arch/x86/kernel/cpu/resctrl/Makefile | 1 + drivers/platform/x86/intel/pmt/Kconfig | 3 + 5 files changed, 141 insertions(+) create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_pmt_features.h create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_pmt_features.c
arch/x86/kernel/cpu/resctrl/fake_intel_pmt_features.h new file +72 -0
@@ -0,0 +1,72 @@ + +/* Bits stolen from OOBMSM VSEC discovery code */ + +enum pmt_feature_id { +FEATURE_INVALID =0x0 , +FEATURE_PER_CORE_PERF_TELEM =0x1 , +FEATURE_PER_CORE_ENV_TELEM =0x2 , +FEATURE_PER_RMID_PERF_TELEM =0x3 , +FEATURE_ACCEL_TELEM =0x4 , +FEATURE_UNCORE_TELEM =0x5 , +FEATURE_CRASH_LOG =0x6 , +FEATURE_PETE_LOG =0x7 , +FEATURE_TPMI_CTRL =0x8 , +FEATURE_RESERVED =0x9 , +FEATURE_TRACING =0xA , +FEATURE_PER_RMID_ENERGY_TELEM =0xB , +FEATURE_MAX =0xB , +}; + +/** +* struct oobmsm_plat_info - Platform information for a device instance +* @cdie_mask: Mask of all compute dies in the partition +* @package_id: CPU Package id +* @partition: Package partition id when multiple VSEC PCI devices per package +* @segment: PCI segment ID +* @bus_number: PCI bus number +* @device_number: PCI device number +* @function_number: PCI function number +* +* Structure to store platform data for a OOBMSM device instance. +*/ +struct oobmsm_plat_info { +u16 cdie_mask; +u8 package_id; +u8 partition; +u8 segment; +u8 bus_number; +u8 device_number; +u8 function_number; +}; + +enum oobmsm_supplier_type { +OOBMSM_SUP_PLAT_INFO , +OOBMSM_SUP_DISC_INFO , +OOBMSM_SUP_S3M_SIMICS , +OOBMSM_SUP_TYPE_MAX +}; + +struct oobmsm_mapping_supplier { +struct device *supplier_dev[OOBMSM_SUP_TYPE_MAX ]; +struct oobmsm_plat_info plat_info; +unsigned long features; +}; + +struct telemetry_region { +struct oobmsm_plat_info plat_info; +void __iomem *addr; +size_t size; +u32 guid; +u32 num_rmids; +}; + +struct pmt_feature_group { +enum pmt_feature_id id; +int count; +struct kref kref; +struct telemetry_region regions[]; +}; + +struct pmt_feature_group *intel_pmt_get_regions_by_feature (enum pmt_feature_id id); + +void intel_pmt_put_feature_group (struct pmt_feature_group *feature_group);
arch/x86/kernel/cpu/resctrl/fake_intel_pmt_features.c new file +64 -0
64 lines not on this page · raw message
arch/x86/Kconfig +1 -0
@@ -505,6 +505,7 @@ config X86_CPU_RESCTRL depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD) select KERNFS select PROC_CPU_RESCTRL if PROC_FS + select INTEL_PMT_RESCTRL help Enable x86 CPU resource control support.
arch/x86/kernel/cpu/resctrl/Makefile +1 -0
@@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_X86_CPU_RESCTRL) += core.o rdtgroup.o monitor.o obj-$(CONFIG_X86_CPU_RESCTRL) += ctrlmondata.o pseudo_lock.o +obj-$(CONFIG_INTEL_PMT_RESCTRL) += fake_intel_pmt_features.o CFLAGS_pseudo_lock.o = -I$(src)
drivers/platform/x86/intel/pmt/Kconfig +3 -0
@@ -38,3 +38,6 @@ config INTEL_PMT_CRASHLOG To compile this driver as a module, choose M here: the module will be called intel_pmt_crashlog. + +config INTEL_PMT_RESCTRL + bool
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 2/17 x86/resctrl: Move L3 initialization out of domain_add_cpu_mon()
In reply to Tony LuckTo prepare for additional types of monitoring domains, move all the L3 specific initialization into a helper function.
--- arch/x86/kernel/cpu/resctrl/core.c | 52 +++++++++++++++++------------- 1 file changed, 30 insertions(+), 22 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +30 -22
@@ -583,33 +583,12 @@ static void domain_add_cpu_ctrl(int cpu, struct rdt_resource *r) } } -static void domain_add_cpu_mon (int cpu,struct rdt_resource *r) +static void setup_l3_mon_domain (int cpu,int id,struct rdt_resource *r,struct list_head *add_pos) { -int id =get_domain_id_from_scope (cpu, r->mon_scope); -struct list_head *add_pos =NULL ;struct rdt_hw_mon_domain *hw_dom; -struct rdt_domain_hdr *hdr;struct rdt_mon_domain *d;int err; -lockdep_assert_held (&domain_list_lock); - -if (id <0 ) { -pr_warn_once ("Can't find monitor domain id for CPU:%d scope:%d for resource %s\n" , - cpu, r->mon_scope, r->name); -return ; - } - - hdr =rdt_find_domain (&r->mon_domains, id, &add_pos); -if (hdr) { -if (WARN_ON_ONCE (hdr->type !=RESCTRL_MON_DOMAIN )) -return ; - d = container_of(hdr, struct rdt_mon_domain, hdr); - -cpumask_set_cpu (cpu, &d->hdr.cpu_mask); -return ; - } - hw_dom =kzalloc_node (sizeof (*hw_dom),GFP_KERNEL ,cpu_to_node (cpu));if (!hw_dom)return ;
@@ -642,6 +621,35 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r) } } +static void domain_add_cpu_mon (int cpu,struct rdt_resource *r) +{ +int id =get_domain_id_from_scope (cpu, r->mon_scope); +struct list_head *add_pos =NULL ; +struct rdt_domain_hdr *hdr; + +lockdep_assert_held (&domain_list_lock); + +if (id <0 ) { +pr_warn_once ("Can't find monitor domain id for CPU:%d scope:%d for resource %s\n" , + cpu, r->mon_scope, r->name); +return ; + } + + hdr =rdt_find_domain (&r->mon_domains, id, &add_pos); +if (hdr) { +cpumask_set_cpu (cpu, &hdr->cpu_mask); +return ; + } + +switch (r->rid) { +case RDT_RESOURCE_L3 : +setup_l3_mon_domain (cpu, id, r, add_pos); +break ; +default : +WARN_ON_ONCE (1 ); + } +} +static void domain_add_cpu (int cpu,struct rdt_resource *r) {if (r->alloc_capable)
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 3/17 x86/resctrl: Refactor domain_remove_cpu_mon() ready for new domain types
In reply to Tony LuckDifferent types of domains require different actions when the last CPU in the domain is removed.
Refactor to make it easy to add new actions for new types of domains.
--- arch/x86/kernel/cpu/resctrl/core.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +8 -8
@@ -726,20 +726,20 @@ static void domain_remove_cpu_mon(int cpu, struct rdt_resource *r)return ; } -if (WARN_ON_ONCE (hdr->type !=RESCTRL_MON_DOMAIN )) +cpumask_clear_cpu (cpu, &hdr->cpu_mask); +if (!cpumask_empty (&hdr->cpu_mask))return ; - d = container_of(hdr, struct rdt_mon_domain, hdr); - hw_dom =resctrl_to_arch_mon_dom (d); +switch (r->rid) { +case RDT_RESOURCE_L3 : + d = container_of(hdr, struct rdt_mon_domain, hdr); + hw_dom =resctrl_to_arch_mon_dom (d); -cpumask_clear_cpu (cpu, &d->hdr.cpu_mask); -if (cpumask_empty (&d->hdr.cpu_mask)) {resctrl_offline_mon_domain (r, d); -list_del_rcu (&d->hdr.list); +list_del_rcu (&hdr->list);synchronize_rcu ();mon_domain_free (hw_dom); - -return ; +break ; } }
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 4/17 x86/resctrl: Split L3 specific code out of rdtgroup_mondata_show()
In reply to Tony LuckNew monitor resources require different operations to read counters.
Move the L3 specific code to a helper function.
--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 79 +++++++++++++---------- 1 file changed, 46 insertions(+), 33 deletions(-)
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +46 -33
@@ -627,31 +627,14 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,resctrl_arch_mon_ctx_free (r, evtid, rr->arch_mon_ctx); } -int rdtgroup_mondata_show (struct seq_file *m,void *arg) +static int rdtgroup_l3_data_show (struct seq_file *m,struct rdt_resource *r, +int domid,struct rdtgroup *rdtgrp,u32 evtid, +bool sum,struct rmid_read *rr) { -struct kernfs_open_file *of = m->private;struct rdt_domain_hdr *hdr; -struct rmid_read rr = {0 };struct rdt_mon_domain *d; -u32 resid, evtid, domid; -struct rdtgroup *rdtgrp; -struct rdt_resource *r; -union mon_data_bits md; -int ret =0 ; - - rdtgrp =rdtgroup_kn_lock_live (of->kn); -if (!rdtgrp) { - ret = -ENOENT ; - goto out; - } - md.priv = of->kn->priv; - resid = md.u.rid; - domid = md.u.domid; - evtid = md.u.evtid; - r = &rdt_resources_all[resid].r_resctrl; - -if (md.u.sum) { +if (sum) {/* * This file requires summing across all domains that share * the L3 cache id that was provided in the "domid" field of the
@@ -660,37 +643,67 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg)*/ list_for_each_entry (d, &r->mon_domains, hdr.list) {if (d->ci->id == domid) { - rr.ci = d->ci; -mon_event_read (&rr, r,NULL , rdtgrp, + rr->ci = d->ci; +mon_event_read (rr, r,NULL , rdtgrp, &d->ci->shared_cpu_map, evtid, false); goto checkresult; } } - ret = -ENOENT ; - goto out; +return -ENOENT ; }else {/* * This file provides data from a single domain. Search * the resource to find the domain with "domid". */ hdr =rdt_find_domain (&r->mon_domains, domid,NULL ); -if (!hdr ||WARN_ON_ONCE (hdr->type !=RESCTRL_MON_DOMAIN )) { - ret = -ENOENT ; - goto out; - } +if (!hdr ||WARN_ON_ONCE (hdr->type !=RESCTRL_MON_DOMAIN )) +return -ENOENT ; d = container_of(hdr, struct rdt_mon_domain, hdr); -mon_event_read (&rr, r, d, rdtgrp, &d->hdr.cpu_mask, evtid, false); +mon_event_read (rr, r, d, rdtgrp, &d->hdr.cpu_mask, evtid, false); } checkresult: -if (rr.err == -EIO ) +if (rr->err == -EIO )seq_puts (m,"Error\n" ); -else if (rr.err == -EINVAL ) +else if (rr->err == -EINVAL )seq_puts (m,"Unavailable\n" );else -seq_printf (m,"%llu\n" , rr.val); +seq_printf (m,"%llu\n" , rr->val); +return 0 ; +} + +int rdtgroup_mondata_show (struct seq_file *m,void *arg) +{ +struct kernfs_open_file *of = m->private; +struct rmid_read rr = {0 }; +u32 resid, evtid, domid; +struct rdtgroup *rdtgrp; +struct rdt_resource *r; +union mon_data_bits md; +int ret =0 ; + + rdtgrp =rdtgroup_kn_lock_live (of->kn); +if (!rdtgrp) { + ret = -ENOENT ; + goto out; + } + + md.priv = of->kn->priv; + resid = md.u.rid; + domid = md.u.domid; + evtid = md.u.evtid; + r = &rdt_resources_all[resid].r_resctrl; + +switch (r->rid) { +case RDT_RESOURCE_L3 : + ret =rdtgroup_l3_data_show (m, r, domid, rdtgrp, evtid, md.u.sum, &rr); +break ; +default : + ret = -EINVAL ; +break ; + } out:rdtgroup_kn_unlock (of->kn);return ret;
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 5/17 x86/resctrl: Change generic monitor functions to use struct rdt_domain_hdr
In reply to Tony LuckFunctions that don't need the internal details of the rdt_mon_domain can operate on just the rdt_domain_hdr.
--- include/linux/resctrl.h | 2 +- arch/x86/kernel/cpu/resctrl/core.c | 2 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 60 +++++++++++++++++--------- 3 files changed, 42 insertions(+), 22 deletions(-)
include/linux/resctrl.h +1 -1
@@ -261,7 +261,7 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,u32 resctrl_arch_get_config (struct rdt_resource *r,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type type);int resctrl_online_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d); -int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d); +int resctrl_online_mon_domain (struct rdt_resource *r,struct rdt_domain_hdr *hdr);void resctrl_offline_ctrl_domain (struct rdt_resource *r,struct rdt_ctrl_domain *d);void resctrl_offline_mon_domain (struct rdt_resource *r,struct rdt_mon_domain *d);void resctrl_online_cpu (unsigned int cpu);
arch/x86/kernel/cpu/resctrl/core.c +1 -1
@@ -613,7 +613,7 @@ static void setup_l3_mon_domain(int cpu, int id, struct rdt_resource *r, structlist_add_tail_rcu (&d->hdr.list, add_pos); - err =resctrl_online_mon_domain (r, d); + err =resctrl_online_mon_domain (r, &d->hdr);if (err) {list_del_rcu (&d->hdr.list);synchronize_rcu ();
arch/x86/kernel/cpu/resctrl/rdtgroup.c +40 -20
169 lines not on this page · raw message
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 6/17 x86/resctrl: Add and initialize rdt_resource for package scope core monitor
In reply to Tony LuckNew resource for monitoring core events reported at package level.
--- include/linux/resctrl.h | 1 + arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ arch/x86/kernel/cpu/resctrl/core.c | 12 ++++++++++++ 3 files changed, 15 insertions(+)
include/linux/resctrl.h +1 -0
@@ -180,6 +180,7 @@ enum resctrl_scope {RESCTRL_L2_CACHE =2 ,RESCTRL_L3_CACHE =3 ,RESCTRL_L3_NODE , + RESCTRL_PACKAGE, };/**
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
@@ -317,6 +317,7 @@ struct rdtgroup {#define RFTYPE_RES_CACHE BIT(8)#define RFTYPE_RES_MB BIT(9)#define RFTYPE_DEBUG BIT(10) +#define RFTYPE_RES_PKG BIT(11)#define RFTYPE_CTRL_INFO (RFTYPE_INFO | RFTYPE_CTRL)#define RFTYPE_MON_INFO (RFTYPE_INFO | RFTYPE_MON)#define RFTYPE_TOP_INFO (RFTYPE_INFO | RFTYPE_TOP)
@@ -517,6 +518,7 @@ enum resctrl_res_level {RDT_RESOURCE_L2 ,RDT_RESOURCE_MBA ,RDT_RESOURCE_SMBA , +RDT_RESOURCE_INTEL_PMT ,/* Must be the last */ RDT_NUM_RESOURCES,
arch/x86/kernel/cpu/resctrl/core.c +12 -0
@@ -117,6 +117,16 @@ struct rdt_hw_resource rdt_resources_all[] = { .fflags = RFTYPE_RES_MB, }, }, + [RDT_RESOURCE_INTEL_PMT ] = + { + .r_resctrl = { + .rid =RDT_RESOURCE_INTEL_PMT , + .name ="PKG" , + .mon_scope =RESCTRL_PACKAGE , + .mon_domains =mon_domain_init (RDT_RESOURCE_INTEL_PMT ), + .fflags =RFTYPE_RES_PKG , + }, + }, };u32 resctrl_arch_system_num_rmid_idx (void )
@@ -521,6 +531,8 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)return get_cpu_cacheinfo_id (cpu, scope);case RESCTRL_L3_NODE :return cpu_to_node (cpu); +case RESCTRL_PACKAGE : +return topology_physical_package_id (cpu);default :break ; }
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 7/17 x86/resctrl: Prepare for resource specific event ids
In reply to Tony LuckEvent ids ought to be specific to each resource. Keep enum resctrl_event_id for event ids in the RDT_RESOURCE_L3. But change the type in generic areas to unsigned int so that new resources can have their own event ids.
--- arch/x86/kernel/cpu/resctrl/internal.h | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +7 -7
@@ -114,13 +114,13 @@ static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc)/** * struct mon_evt - Entry in the event list of a resource -* @evtid: event id +* @evtid: per resource event id * @name: name of the event * @configurable: true if the event is configurable * @list: entry in &rdt_resource->evt_list */ struct mon_evt { -enum resctrl_event_id evtid; +unsigned int evtid;char *name;bool configurable;struct list_head list;
@@ -131,7 +131,7 @@ struct mon_evt {* @priv: Used to store monitoring event data in @u * as kernfs private data. * @u.rid: Resource id associated with the event file. -* @u.evtid: Event id associated with the event file. +* @u.evtid: Per resource event id associated with the event file. * @u.sum: Set when event must be summed across multiple * domains. * @u.domid: When @u.sum is zero this is the domain to which
@@ -143,10 +143,10 @@ struct mon_evt {union mon_data_bits {void *priv;struct { -unsigned int rid :10 ; -enum resctrl_event_id evtid :7 ; -unsigned int sum :1 ; -unsigned int domid :14 ; +unsigned int rid :10 ; +unsigned int evtid :7 ; +unsigned int sum :1 ; +unsigned int domid :14 ; } u; };
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 8/17 x86/resctrl: Add initialization hook for Intel PMT events
In reply to Tony LuckCall the OOBMSM discovery code to find out if there are any event groups that match unique identifiers understood by resctrl.
Note that initiialzation must happen in two phases because the OOBMSM VSEC discovery process is not complete at resctrl "lateinit()" initialization time. So there is an initial hook that assumes that Intel PMT will exist, called early so that package scoped domain groups are initialized.
At first mount the remainder of initialization is done. If there are no Intel PMT events, the package domain lists are removed.
Events for specific systems to be added by a separate patch.
--- include/linux/resctrl.h | 8 ++ arch/x86/kernel/cpu/resctrl/internal.h | 7 + arch/x86/kernel/cpu/resctrl/core.c | 9 +- arch/x86/kernel/cpu/resctrl/intel_pmt.c | 169 ++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 3 + arch/x86/kernel/cpu/resctrl/Makefile | 1 + 6 files changed, 194 insertions(+), 3 deletions(-) create mode 100644 arch/x86/kernel/cpu/resctrl/intel_pmt.c
include/linux/resctrl.h +8 -0
@@ -118,6 +118,14 @@ struct rdt_mon_domain {int cqm_work_cpu; }; +/** +* struct rdt_core_mon_domain - CPUs sharing an Intel-PMT-scoped resctrl monitor resource +* @hdr: common header for different domain types +*/ +struct rdt_core_mon_domain { +struct rdt_domain_hdr hdr; +}; +/** * struct resctrl_cache - Cache allocation related data * @cbm_len: Length of the cache bit mask
arch/x86/kernel/cpu/resctrl/internal.h +7 -0
@@ -637,6 +637,13 @@ void closid_free(int closid);int alloc_rmid (u32 closid);void free_rmid (u32 closid,u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r); +#ifdef CONFIG_INTEL_PMT_RESCTRL +int rdt_get_intel_pmt_mon_config (void ); +void rdt_get_intel_pmt_mount (void ); +#else +static inline int rdt_get_intel_pmt_mon_config (void ) {return 0 ; } +static inline void rdt_get_intel_pmt_mount (void ) { } +#endif void __exit rdt_put_mon_l3_config(void);bool __init rdt_cpu_has (int flag);void mon_event_count (void *info);
arch/x86/kernel/cpu/resctrl/core.c +6 -3
@@ -972,6 +972,7 @@ static __init bool get_rdt_alloc_resources(void)static __init bool get_rdt_mon_resources (void ) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl; +int ret1 = -EINVAL , ret2;if (rdt_cpu_has (X86_FEATURE_CQM_OCCUP_LLC )) rdt_mon_features |= (1 <<QOS_L3_OCCUP_EVENT_ID );
@@ -980,15 +981,17 @@ static __init bool get_rdt_mon_resources(void)if (rdt_cpu_has (X86_FEATURE_CQM_MBM_LOCAL )) rdt_mon_features |= (1 <<QOS_L3_MBM_LOCAL_EVENT_ID ); -if (!rdt_mon_features) -return false; +if (rdt_mon_features) + ret1 =rdt_get_mon_l3_config (r); + + ret2 =rdt_get_intel_pmt_mon_config ();if (is_mbm_local_enabled ()) mba_mbps_default_event =QOS_L3_MBM_LOCAL_EVENT_ID ;else if (is_mbm_total_enabled ()) mba_mbps_default_event =QOS_L3_MBM_TOTAL_EVENT_ID ; -return !rdt_get_mon_l3_config (r); +return ret1 ==0 || ret2; }static __init void __check_quirks_intel (void )
arch/x86/kernel/cpu/resctrl/intel_pmt.c new file +169 -0
169 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/rdtgroup.c +3 -0
@@ -2645,6 +2645,9 @@ static int rdt_get_tree(struct fs_context *fc)cpus_read_lock ();mutex_lock (&rdtgroup_mutex); + +rdt_get_intel_pmt_mount (); +/* * resctrl file system can only be mounted once. */
arch/x86/kernel/cpu/resctrl/Makefile +1 -0
@@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_X86_CPU_RESCTRL) += core.o rdtgroup.o monitor.o obj-$(CONFIG_X86_CPU_RESCTRL) += ctrlmondata.o pseudo_lock.o +obj-$(CONFIG_INTEL_PMT_RESCTRL) += intel_pmt.o obj-$(CONFIG_INTEL_PMT_RESCTRL) += fake_intel_pmt_features.o CFLAGS_pseudo_lock.o = -I$(src)
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 9/17 x86/resctrl: Add Intel PMT domain specific code
In reply to Tony LuckDomain specific initialization and tear down. Very simple as there are no domain specific elements that need to be allocated or initiailazed. Just the domain itself.
This does need the mkdir_mondata_subdir_allrdtgrp() and rmdir_mondata_subdir_allrdtgrp() functions. So change them from static to global so they can be called from other source files.
--- arch/x86/kernel/cpu/resctrl/internal.h | 7 +++++++ arch/x86/kernel/cpu/resctrl/core.c | 12 ++++++++++++ arch/x86/kernel/cpu/resctrl/intel_pmt.c | 16 ++++++++++++++++ arch/x86/kernel/cpu/resctrl/rdtgroup.c | 6 ++---- 4 files changed, 37 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +7 -0
@@ -640,9 +640,12 @@ int rdt_get_mon_l3_config(struct rdt_resource *r);#ifdef CONFIG_INTEL_PMT_RESCTRL int rdt_get_intel_pmt_mon_config (void );void rdt_get_intel_pmt_mount (void ); +void setup_intel_pmt_mon_domain (int cpu,int id,struct rdt_resource *r,struct list_head *add_pos);#else static inline int rdt_get_intel_pmt_mon_config (void ) {return 0 ; }static inline void rdt_get_intel_pmt_mount (void ) { } +static inline void setup_intel_pmt_mon_domain (int cpu,int id,struct rdt_resource *r, +struct list_head *add_pos) { }#endif void __exit rdt_put_mon_l3_config(void);bool __init rdt_cpu_has (int flag);
@@ -667,4 +670,8 @@ void resctrl_file_fflags_init(const char *config, unsigned long fflags);void rdt_staged_configs_clear (void );bool closid_allocated (unsigned int closid);int resctrl_find_cleanest_closid (void ); +void mkdir_mondata_subdir_allrdtgrp (struct rdt_resource *r, +struct rdt_domain_hdr *hdr); +void rmdir_mondata_subdir_allrdtgrp (struct rdt_resource *r, +struct rdt_domain_hdr *hdr);#endif /* _ASM_X86_RESCTRL_INTERNAL_H */
arch/x86/kernel/cpu/resctrl/core.c +12 -0
@@ -657,6 +657,9 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r)case RDT_RESOURCE_L3 :setup_l3_mon_domain (cpu, id, r, add_pos);break ; +case RDT_RESOURCE_INTEL_PMT : +setup_intel_pmt_mon_domain (cpu, id, r, add_pos); +break ;default :WARN_ON_ONCE (1 ); }
@@ -752,6 +755,15 @@ static void domain_remove_cpu_mon(int cpu, struct rdt_resource *r)synchronize_rcu ();mon_domain_free (hw_dom);break ; +case RDT_RESOURCE_INTEL_PMT : +mutex_lock (&rdtgroup_mutex); +if (resctrl_mounted &&resctrl_arch_mon_capable ()) +rmdir_mondata_subdir_allrdtgrp (r, hdr); +mutex_unlock (&rdtgroup_mutex); +list_del_rcu (&hdr->list); +synchronize_rcu (); +kfree (container_of (hdr ,struct rdt_core_mon_domain ,hdr )); +break ; } }
arch/x86/kernel/cpu/resctrl/intel_pmt.c +16 -0
@@ -167,3 +167,19 @@ void rdt_get_intel_pmt_mount(void) r->mon_capable = false; } } + +void setup_intel_pmt_mon_domain (int cpu,int id,struct rdt_resource *r,struct list_head *add_pos) +{ +struct rdt_core_mon_domain *hw_dom; + + hw_dom =kzalloc_node (sizeof (*hw_dom),GFP_KERNEL ,cpu_to_node (cpu)); +if (!hw_dom) +return ; + + hw_dom->hdr.id = id; + hw_dom->hdr.type =RESCTRL_MON_DOMAIN ; +cpumask_set_cpu (cpu, &hw_dom->hdr.cpu_mask); +list_add_tail_rcu (&hw_dom->hdr.list, add_pos); +if (resctrl_mounted &&resctrl_arch_mon_capable ()) +mkdir_mondata_subdir_allrdtgrp (r, &hw_dom->hdr); +}
arch/x86/kernel/cpu/resctrl/rdtgroup.c +2 -4
@@ -3041,8 +3041,7 @@ static void mon_rmdir_one_subdir(struct kernfs_node *pkn, char *name, char *subn* Remove files and directories containing "sum" of domain data * when last domain being summed is removed. */ -static void rmdir_mondata_subdir_allrdtgrp (struct rdt_resource *r, -struct rdt_domain_hdr *hdr) +void rmdir_mondata_subdir_allrdtgrp (struct rdt_resource *r,struct rdt_domain_hdr *hdr) {struct rdtgroup *prgrp, *crgrp;char subname[32 ];
@@ -3170,8 +3169,7 @@ static int mkdir_mondata_subdir(struct kernfs_node *parent_kn,* Add all subdirectories of mon_data for "ctrl_mon" groups * and "monitor" groups with given domain id. */ -static void mkdir_mondata_subdir_allrdtgrp (struct rdt_resource *r, -struct rdt_domain_hdr *hdr) +void mkdir_mondata_subdir_allrdtgrp (struct rdt_resource *r,struct rdt_domain_hdr *hdr) {struct kernfs_node *parent_kn;struct rdtgroup *prgrp, *crgrp;
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 10/17 x86/resctrl: Add detailed descriptions for Clearwater Forest events
In reply to Tony LuckThere are two event groups one for energy reporting and another for "perf" events.
See the XML description files in https://github.com/intel/Intel-PMT
in the xml/CWF/OOBMSM/{RMID-ENERGY,RMID-PERF}/ for the detailed descriptions that were used to derive these descriptions.
--- arch/x86/kernel/cpu/resctrl/intel_pmt.c | 43 +++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
arch/x86/kernel/cpu/resctrl/intel_pmt.c +43 -0
@@ -70,7 +70,50 @@ struct telem_entry {struct pmt_event evts[]; }; +#define CWF_NUM_RMIDS 576 +#define CWF_ENERGY_GUID 0x26696143 +#define CWF_PERF_GUID 0x26557651 +#define CWF_ENERGY_STRIDE 0x10 +#define CWF_PERF_STRIDE 0x38 + +static struct telem_entry cwf_energy = { + .name ="energy" , + .guid =CWF_ENERGY_GUID , + .num_rmids =CWF_NUM_RMIDS , + .stride =CWF_ENERGY_STRIDE , + .overflow_counter_off =CWF_NUM_RMIDS *CWF_ENERGY_STRIDE , + .last_overflow_tstamp_off =CWF_NUM_RMIDS *CWF_ENERGY_STRIDE +8 , + .last_update_tstamp_off =CWF_NUM_RMIDS *CWF_ENERGY_STRIDE +16 , + .evts = { +EVT (PMT_EVENT_ENERGY ,"core_energy" ,0x0 ,EVT_U46_18 ), +EVT (PMT_EVENT_ACTIVITY ,"activity" ,0x8 ,EVT_U46_18 ), + { } + } +}; + +static struct telem_entry cwf_perf = { + .name ="perf" , + .guid =CWF_PERF_GUID , + .num_rmids =CWF_NUM_RMIDS , + .stride =CWF_PERF_STRIDE , + .overflow_counter_off =CWF_NUM_RMIDS *CWF_PERF_STRIDE , + .last_overflow_tstamp_off =CWF_NUM_RMIDS *CWF_PERF_STRIDE +8 , + .last_update_tstamp_off =CWF_NUM_RMIDS *CWF_PERF_STRIDE +16 , + .evts = { +EVT (PMT_EVENT_STALLS_LLC_HIT ,"stalls_llc_hit" ,0x0 ,EVT_U64 ), +EVT (PMT_EVENT_C1_RES ,"c1_res" ,0x8 ,EVT_U64 ), +EVT (PMT_EVENT_UNHALTED_CORE_CYCLES ,"unhalted_core_cycles" ,0x10 ,EVT_U64 ), +EVT (PMT_EVENT_STALLS_LLC_MISS ,"stalls_llc_miss" ,0x18 ,EVT_U64 ), +EVT (PMT_EVENT_AUTO_C6_RES ,"c6_res" ,0x20 ,EVT_U64 ), +EVT (PMT_EVENT_UNHALTED_REF_CYCLES ,"unhalted_ref_cycles" ,0x28 ,EVT_U64 ), +EVT (PMT_EVENT_UOPS_RETIRED ,"uops_retired" ,0x30 ,EVT_U64 ), + { } + } +}; +static struct telem_entry *telem_entry[] = { + &cwf_energy, + &cwf_perf,NULL };
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 11/17 x86/resctrl: Allocate per-package structures for known events
In reply to Tony LuckUse the per-package counts of known events to allocate arrays to make a copy of just the known events.
Add hook into resctrl_exit() to cleanup.
--- arch/x86/kernel/cpu/resctrl/internal.h | 2 + arch/x86/kernel/cpu/resctrl/core.c | 2 + arch/x86/kernel/cpu/resctrl/intel_pmt.c | 59 ++++++++++++++++++++++++- 3 files changed, 62 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
@@ -641,11 +641,13 @@ int rdt_get_mon_l3_config(struct rdt_resource *r);int rdt_get_intel_pmt_mon_config (void );void rdt_get_intel_pmt_mount (void );void setup_intel_pmt_mon_domain (int cpu,int id,struct rdt_resource *r,struct list_head *add_pos); +void rdt_intel_pmt_exit (void );#else static inline int rdt_get_intel_pmt_mon_config (void ) {return 0 ; }static inline void rdt_get_intel_pmt_mount (void ) { }static inline void setup_intel_pmt_mon_domain (int cpu,int id,struct rdt_resource *r,struct list_head *add_pos) { } +static inline void rdt_intel_pmt_exit (void ) { };#endif void __exit rdt_put_mon_l3_config(void);bool __init rdt_cpu_has (int flag);
arch/x86/kernel/cpu/resctrl/core.c +2 -0
@@ -1170,6 +1170,8 @@ static void __exit resctrl_exit(void)cpuhp_remove_state (rdt_online); +rdt_intel_pmt_exit (); +rdtgroup_exit ();if (r->mon_capable)
arch/x86/kernel/cpu/resctrl/intel_pmt.c +58 -1
89 lines not on this page · raw message
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 12/17 x86/resctrl: Add list of known events to RDT_RESOURCE_INTEL_PMT
In reply to Tony LuckCore code uses this list to populate "mon_data" directories.
--- arch/x86/kernel/cpu/resctrl/intel_pmt.c | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/intel_pmt.c +23 -1
@@ -13,6 +13,7 @@#include <linux/cpu.h> #include <linux/cleanup.h> +#include <linux/minmax.h> #include <linux/slab.h> #include "fake_intel_pmt_features.h" #include <linux/intel_vsec.h>
@@ -254,14 +255,35 @@ void rdt_get_intel_pmt_mount(void) {struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_INTEL_PMT ].r_resctrl;struct rdt_core_mon_domain *d, *tmp; +struct telem_entry **tentry;static int do_one_time; +struct mon_evt *evt; +bool ret = false;if (do_one_time)return ; do_one_time =1 ; -if (!get_events ()) { +if (!get_events ()) + goto done; + +for (tentry = telem_entry; *tentry; tentry++) { +if (!(*tentry)->active) +continue ; +for (int i =0 ; (*tentry)->evts[i].evt.name; i++) { + evt = &(*tentry)->evts[i].evt; +list_add_tail (&evt->list, &r->evt_list); + ret = true; + } +if (!r->num_rmid) + r->num_rmid = (*tentry)->num_rmids; +else + r->num_rmid =min (r->num_rmid, (*tentry)->num_rmids); + } +done: +if (!ret) { + r->mon_capable = false;list_for_each_entry_safe (d, tmp, &r->mon_domains, hdr.list)kfree (d); r->mon_capable = false;
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 13/17 x86/resctrl: Build lookup table for package events
In reply to Tony LuckThe resctrl filesystem saves the evt_type in the private data pointer of the kernfs_node of each file in the mon_data directories.
To print the values for each file the show() function will need to map from this evtid to:
num_rmids - to make sure data for this file exists guid - to pick the pmt_event(s) for each package stride/offset - to compute MMIO offset for this RMID/event
Build a lookup table for each event to save searching through lists and add macros for ease of use.
--- arch/x86/kernel/cpu/resctrl/intel_pmt.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
arch/x86/kernel/cpu/resctrl/intel_pmt.c +15 -0
@@ -38,6 +38,18 @@ enum pmt_event_id {PMT_NUM_EVENTS }; +static struct evtinfo { +struct telem_entry *telem_entry; +struct pmt_event *pmt_event; +} evtinfo[PMT_NUM_EVENTS ]; + +#define EVT_NUM_RMIDS (evtid) (evtinfo[evtid].telem_entry->num_rmids) +#define EVT_STRIDE (evtid) (evtinfo[evtid].telem_entry->stride) +#define EVT_GUID (evtid) (evtinfo[evtid].telem_entry->guid) + +#define EVT_OFFSET (evtid) (evtinfo[evtid].pmt_event->evt_offset) +#define EVT_TYPE (evtid) (evtinfo[evtid].pmt_event->evt_type) +enum evt_type {EVT_U64 ,EVT_U46_18 ,
@@ -274,6 +286,9 @@ void rdt_get_intel_pmt_mount(void)for (int i =0 ; (*tentry)->evts[i].evt.name; i++) { evt = &(*tentry)->evts[i].evt;list_add_tail (&evt->list, &r->evt_list); + + evtinfo[evt->evtid].telem_entry = *tentry; + evtinfo[evt->evtid].pmt_event = &(*tentry)->evts[i]; ret = true; }if (!r->num_rmid)
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 14/17 x86/resctrl: Add code in rdtgroup_mondata_show() for core monitor events
In reply to Tony LuckThere may be multiple devices tracking each package, so scan all of them and add up counters.
Output format depends on the data type. Either a 63 bit integer, or a fixed point decimal.
--- arch/x86/kernel/cpu/resctrl/internal.h | 6 ++ arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 3 + arch/x86/kernel/cpu/resctrl/intel_pmt.c | 88 +++++++++++++++++++++++ 3 files changed, 97 insertions(+)
arch/x86/kernel/cpu/resctrl/internal.h +6 -0
@@ -639,11 +639,17 @@ void free_rmid(u32 closid, u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r);#ifdef CONFIG_INTEL_PMT_RESCTRL int rdt_get_intel_pmt_mon_config (void ); +int rdtgroup_intel_pmt_data_show (struct seq_file *m,struct rdt_resource *r, +int domid,struct rdtgroup *rdtgrp,u32 evtid, +struct rmid_read *rr);void rdt_get_intel_pmt_mount (void );void setup_intel_pmt_mon_domain (int cpu,int id,struct rdt_resource *r,struct list_head *add_pos);void rdt_intel_pmt_exit (void );#else static inline int rdt_get_intel_pmt_mon_config (void ) {return 0 ; } +static inline int rdtgroup_intel_pmt_data_show (struct seq_file *m,struct rdt_resource *r, +int domid,struct rdtgroup *rdtgrp,u32 evtid, +struct rmid_read *rr) {return 0 ; }static inline void rdt_get_intel_pmt_mount (void ) { }static inline void setup_intel_pmt_mon_domain (int cpu,int id,struct rdt_resource *r,struct list_head *add_pos) { }
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +3 -0
@@ -700,6 +700,9 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg)case RDT_RESOURCE_L3 : ret =rdtgroup_l3_data_show (m, r, domid, rdtgrp, evtid, md.u.sum, &rr);break ; +case RDT_RESOURCE_INTEL_PMT : + ret =rdtgroup_intel_pmt_data_show (m, r, domid, rdtgrp, evtid, &rr); +break ;default : ret = -EINVAL ;break ;
arch/x86/kernel/cpu/resctrl/intel_pmt.c +88 -0
91 lines not on this page · raw message
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 15/17 x86/resctrl: Add status files to info/PKG_MON
In reply to Tony LuckEach of the package event groups includes status about the number of missed events and timestamps (from a 25 MHz clock) on last time an update was missed and last time an update was processed.
Add a three info files to report per-aggregator values of these status values.
--- arch/x86/kernel/cpu/resctrl/internal.h | 13 ++++ arch/x86/kernel/cpu/resctrl/intel_pmt.c | 92 ++++++++++++++++++++++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 22 +++++- 3 files changed, 117 insertions(+), 10 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +13 -0
@@ -321,6 +321,7 @@ struct rdtgroup {#define RFTYPE_CTRL_INFO (RFTYPE_INFO | RFTYPE_CTRL)#define RFTYPE_MON_INFO (RFTYPE_INFO | RFTYPE_MON)#define RFTYPE_TOP_INFO (RFTYPE_INFO | RFTYPE_TOP) +#define RFTYPE_PKG_INFO (RFTYPE_INFO | RFTYPE_RES_PKG)#define RFTYPE_CTRL_BASE (RFTYPE_BASE | RFTYPE_CTRL)#define RFTYPE_MON_BASE (RFTYPE_BASE | RFTYPE_MON)
@@ -642,6 +643,12 @@ int rdt_get_intel_pmt_mon_config(void);int rdtgroup_intel_pmt_data_show (struct seq_file *m,struct rdt_resource *r,int domid,struct rdtgroup *rdtgrp,u32 evtid,struct rmid_read *rr); +int rdtgroup_last_update_show (struct kernfs_open_file *of, +struct seq_file *seq,void *v); +int rdtgroup_overflows_show (struct kernfs_open_file *of, +struct seq_file *seq,void *v); +int rdtgroup_overflow_timestamp_show (struct kernfs_open_file *of, +struct seq_file *seq,void *v);void rdt_get_intel_pmt_mount (void );void setup_intel_pmt_mon_domain (int cpu,int id,struct rdt_resource *r,struct list_head *add_pos);void rdt_intel_pmt_exit (void );
@@ -650,6 +657,12 @@ static inline int rdt_get_intel_pmt_mon_config(void) { return 0; }static inline int rdtgroup_intel_pmt_data_show (struct seq_file *m,struct rdt_resource *r,int domid,struct rdtgroup *rdtgrp,u32 evtid,struct rmid_read *rr) {return 0 ; } +static inline int rdtgroup_last_update_show (struct kernfs_open_file *of, +struct seq_file *seq,void *v) {return 0 ; }; +static inline int rdtgroup_overflows_show (struct kernfs_open_file *of, +struct seq_file *seq,void *v) {return 0 ; }; +static inline int rdtgroup_overflow_timestamp_show (struct kernfs_open_file *of, +struct seq_file *seq,void *v) {return 0 ; };static inline void rdt_get_intel_pmt_mount (void ) { }static inline void setup_intel_pmt_mon_domain (int cpu,int id,struct rdt_resource *r,struct list_head *add_pos) { }
arch/x86/kernel/cpu/resctrl/intel_pmt.c +83 -9
125 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/rdtgroup.c +21 -1
@@ -1986,7 +1986,27 @@ static struct rftype res_common_files[] = { .seq_show = rdtgroup_closid_show, .fflags = RFTYPE_CTRL_BASE | RFTYPE_DEBUG, }, - + { + .name = "last_update", + .mode = 0444, + .kf_ops = &rdtgroup_kf_single_ops, + .seq_show = rdtgroup_last_update_show, + .fflags = RFTYPE_PKG_INFO, + }, + { + .name = "overflows", + .mode = 0444, + .kf_ops = &rdtgroup_kf_single_ops, + .seq_show = rdtgroup_overflows_show, + .fflags = RFTYPE_PKG_INFO, + }, + { + .name = "overflow_timestamp", + .mode = 0444, + .kf_ops = &rdtgroup_kf_single_ops, + .seq_show = rdtgroup_overflow_timestamp_show, + .fflags = RFTYPE_PKG_INFO, + }, }; static int rdtgroup_add_files(struct kernfs_node *kn, unsigned long fflags)
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 16/17 x86/resctrl: Enable package event monitoring
In reply to Tony LuckAll the code to support this is in place. Unconditionally enable the package resource during early initialization so that domain lists are built.
Enumeration of the package events happens at first mount, if there are no package scoped events, then this resource is disabled.
TODO: Cleanup the domains.
--- arch/x86/kernel/cpu/resctrl/intel_pmt.c | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/kernel/cpu/resctrl/intel_pmt.c +2 -0
@@ -244,6 +244,8 @@ int rdt_get_intel_pmt_mon_config(void)INIT_LIST_HEAD (&r->evt_list); + r->mon_capable = true; +return 1 ; }
Signature · 1 line
2.48.1
Tony Luck
maintainerRFC PATCH 17/17 x86/resctrl: Update Documentation for package events
In reply to Tony LuckEach "mon_data" directory is now divided between L3 events and package events.
--- Documentation/arch/x86/resctrl.rst | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-)
Documentation/arch/x86/resctrl.rst +16 -9
@@ -366,15 +366,22 @@ When control is enabled all CTRL_MON groups will also contain: When monitoring is enabled all MON groups will also contain: "mon_data": - This contains a set of files organized by L3 domain and by - RDT event. E.g. on a system with two L3 domains there will - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these - directories have one file per event (e.g. "llc_occupancy", - "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these - files provide a read out of the current value of the event for - all tasks in the group. In CTRL_MON groups these files provide - the sum for all tasks in the CTRL_MON group and all tasks in - MON groups. Please see example section for more details on usage. + This contains a set of directories, one for each instance + of an L3 cache, or of a processor package. The L3 cache + directories are named "mon_L3_00", "mon_L3_01" etc. The + package directories "mon_PKG_00", "mon_PKG_01" etc. + + Within each directory there is one file per event. In + the L3 directories: "llc_occupancy", "mbm_total_bytes", + and "mbm_local_bytes". In the PKG directories: "core_energy", + "activity", etc. + + In a MON group these files provide a read out of the current + value of the event for all tasks in the group. In CTRL_MON groups + these files provide the sum for all tasks in the CTRL_MON group + and all tasks in MON groups. Please see example section for more + details on usage. + On systems with Sub-NUMA Cluster (SNC) enabled there are extra directories for each node (located within the "mon_L3_XX" directory for the L3 cache they occupy). These are named "mon_sub_L3_YY"
Signature · 1 line
2.48.1