Skip to contentSkip to conversation

x86/resctrl telemetry monitoring

PATCH Superseded Tony Luck 2 participants 37 messages Mar 21, 2025 – Apr 1, 2025

Subsystems (5)

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Tony Luck

author maintainer
#0 lore raw

v2 cover x86/resctrl telemetry monitoring

First version posted as RFC here:

This series is based on James Morse's "fs/resctrl/" snapshot.

With Boris applying 30 patches from the monster series to tip x86/cache we are now close to the finish line of the BIG MOVE. So I moved this series to be on top of:

git://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git mpam/move_to_fs/v7

My main goal in doing so is to shine a light on the FS / ARCH boundary as new things that aren't an easy match for existing things in order to figure out which new interfaces are needed. Also I expect the remainder of the big move to complete before this series is ready, so I might as well get it in shape to apply post-move.

A couple of items I noted:

1) These counters are 63-bits, so wraparound isn't an issue. But space to save wider copies of counts is built into the filesystem layer with space allocated in the domains, and periodic polling.

2) I have alloc/free of my domains in the filesystem layer. But this only works because I don't need any arch specific bits (see above).

3) Some of my counters report fixed-point fractional values. So we need a way to communicate a "type" from arch code back up to rdtgroup_mondata_show(). My solution in this series doesn't feel very elegant.

Other changes since the RFC:

1) Names changed. This feature is officially:
"Intel(R) Application Energy Telemetry"

2) Many comments added to code

3) James suggested resolving the "these counters can be read from any CPU" by providing "cpu_online_mask" and relying on smp_call*() functions to just pick the current CPU. So I did that.

Remainder of this cover letter pasted from the V1/RFC
===
The first patch in the series just provides a fake copy of the enumeration interface that should show up in the OOBMSM driver in the near future. It allows building, and running, of this series on Intel (and perhaps AMD) systems that don't have h/w support.

Background

Telemetry features are being implemented in conjunction with the IA32_PQR_ASSOC.RMID value on each logical CPU. This is used to send counts for various events to a collector in a nearby OOMMSM device to be accumulated with counts for each <RMID, event> pair received from other CPUs. Cores send event counts when the RMID value changes, or after each 2ms elapsed time.

Each OOBMSM device may implement multiple event collectors with each servicing a subset of the logical CPUs on a package. In the initial hardware implementation, there are two categories of events:

1) Energy - Two counters
core_energy: This is an estimate of Joules consumed by each core. It is calculated based on the types of instructions executed, not from a power meter. This counter is useful to understand how much energy a workload is consuming.

activity: This measures "accumulated dynamic capacitance". Users who want to optimize energy consumption for a workload may use this rather than core_energy because it provides consistent results independent of any frequency or voltage changes that may occur during the runtime of the application (e.g. entry/exit from turbo mode).

2) Performance - Seven counters
These are similar events to those available via the Linux "perf" tool, but collected in a way with mush lower overhead (no need to collect data on every context switch).

stalls_llc_hit - Counts the total number of unhalted core clock cycles when the core is stalled due to a demand load miss which hit in the LLC

c1_res - Counts the total C1 residency across all cores. The underlying counter increments on 100MHz clock ticks

unhalted_core_cycles - Counts the total number of unhalted core clock cycles

stalls_llc_miss - Counts the total number of unhalted core clock cycles when the core is stalled due to a demand load miss which missed all the local caches

c6_res - Counts the total C6 residency. The underlying counter increments on crystal clock (25MHz) ticks

unhalted_ref_cycles - Counts the total number of unhalted reference clock (TSC) cycles

uops_retired - Counts the total number of uops retired

Enumeration

The only CPUID based enumeration for this feature is the legacy CPUID(eax=7,ecx=0).ebx{12} that indicates the presence of the IA32_PQR_ASSOC MSR and the RMID field within it.

The OOBMSM driver discovers which features are present via PCIe VSEC capabilities. Each feature is tagged with a unique identifier. These identifiers indicate which XML description file from
https://github.com/intel/Intel-PMT describes which event counters are available and their layout within the MMIO BAR space of the OOBMSM device.

Resctrl User Interface

Because there may be multiple OOBMSM collection agents per processor package, resctrl accumulates event counts from all agents on a package and presents a single value to users. This will provide a consistent user interface on future platforms that vary the number of collectors, or the mappings from logical CPUs to collectors.

Users will see the legacy monitoring files in the "L3" directories and the telemetry files in "PKG" directories (with each file providing the aggregated value from all OOBMSM collectors on that package).

$ tree /sys/fs/resctrl/mon_data/
/sys/fs/resctrl/mon_data/
├── mon_L3_00
│   ├── llc_occupancy
│   ├── mbm_local_bytes
│   └── mbm_total_bytes
├── mon_L3_01
│   ├── llc_occupancy
│   ├── mbm_local_bytes
│   └── mbm_total_bytes
├── mon_PKG_00
│   ├── activity
│   ├── c1_res
│   ├── c6_res
│   ├── core_energy
│   ├── stalls_llc_hit
│   ├── stalls_llc_miss
│   ├── unhalted_core_cycles
│   ├── unhalted_ref_cycles
│   └── uops_retired
└── mon_PKG_01
    ├── activity
    ├── c1_res
    ├── c6_res
    ├── core_energy
    ├── stalls_llc_hit
    ├── stalls_llc_miss
    ├── unhalted_core_cycles
    ├── unhalted_ref_cycles
    └── uops_retired

Resctrl Implementation

The OOBMSM driver exposes a function "intel_pmt_get_regions_by_feature()" that returns an array of structures describing the per-RMID groups it found from the VSEC enumeration. Linux looks at the unique identifiers for each group and enables resctrl for all groups with known unique identifiers.

The memory map for the counters for each <RMID, event> pair is described by the XML file. This is too unwieldy to use in the Linux kernel, so a simplified representation is built into the resctrl code. Note that the counters are in MMIO space instead of accessed using the IA32_QM_EVTSEL and IA32_QM_CTR MSRs. This means there is no need for cross-processor calls to read counters from a CPU in a specific domain. The counters can be read from any CPU.

High level description of code changes:

1) New scope RESCTRL_PACKAGE
2) New struct rdt_resource RDT_RESOURCE_INTEL_PMT
3) Refactor monitor code paths to split existing L3 paths from new ones. In some cases this ends up with:

        switch (r->rid) {
        case RDT_RESOURCE_L3:
                helper for L3
                break;
        case RDT_RESOURCE_INTEL_PMT:
                helper for PKG
                break;
        }

4) New source code file "intel_pmt.c" for the code to enumerate, configure, and report event counts.

With only one platform providing this feature, it's tricky to tell exactly where it is going to go. I've made the event definitions platform specific (based on the unique ID from the VSEC enumeration). It seems possible/likely that the list of events may change from generation to generation.

I've picked names for events based on the descriptions in the XML file.

Tony Luck (16):
x86/rectrl: Fake OOBMSM interface
x86/resctrl: Move L3 initialization out of domain_add_cpu_mon() x86/resctrl: Refactor domain_remove_cpu_mon() ready for new domain
types
x86/resctrl: Change generic monitor functions to use struct
rdt_domain_hdr
x86/resctrl: Add and initialize rdt_resource for package scope core
monitor
x86/resctrl: Prepare for resource specific event ids x86/resctrl: Add initialization hook for Intel PMT events x86/resctrl: Add Intel PMT domain specific code x86/resctrl: Add detailed descriptions for Clearwater Forest events x86/resctrl: Allocate per-package structures for known events x86/resctrl: Link known events onto RDT_RESOURCE_INTEL_AET.evt_list x86/resctrl: Build lookup table for package events x86/resctrl: Add code to display core telemetry events x86/resctrl: Add status files to info/PKG_MON x86/resctrl: Enable package event monitoring x86/resctrl: Update Documentation for package events

 Documentation/filesystems/resctrl.rst         |  25 +-
 include/linux/resctrl.h                       |  32 +-
 include/linux/resctrl_types.h                 |  15 +
 .../cpu/resctrl/fake_intel_aet_features.h     |  73 +++
 arch/x86/kernel/cpu/resctrl/internal.h        |   8 +
 fs/resctrl/internal.h                         |  28 +-
 arch/x86/kernel/cpu/resctrl/core.c            | 123 +++--
 .../cpu/resctrl/fake_intel_aet_features.c     |  65 +++
 arch/x86/kernel/cpu/resctrl/intel_aet.c       | 488 ++++++++++++++++++
 fs/resctrl/ctrlmondata.c                      |  23 +-
 fs/resctrl/monitor.c                          |  23 +-
 fs/resctrl/rdtgroup.c                         |  94 +++-
 arch/x86/Kconfig                              |   1 +
 arch/x86/kernel/cpu/resctrl/Makefile          |   2 +
 drivers/platform/x86/intel/pmt/Kconfig        |   3 +
 15 files changed, 915 insertions(+), 88 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h
 create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c
 create mode 100644 arch/x86/kernel/cpu/resctrl/intel_aet.c
Signature · 1 line
2.48.1
author maintainer
#1 lore raw

PATCH v2 1/16 x86/rectrl: Fake OOBMSM interface

Real version is coming soon ... this is here so the remaining parts will build (and run ... assuming a 2 socket system that supports RDT monitoring ... only missing part is that the event counters just report fixed values).

Real version of this would just add the INTEL_AET_RESCTRL Kconfig option with dependency checks on
INTEL_VSEC=y && INTEL_AET_TELEMETRY=y && INTEL_AET_DISCOVERY=y

Just for RFC discussion.

---
 .../cpu/resctrl/fake_intel_aet_features.h     | 73 +++++++++++++++++++
 .../cpu/resctrl/fake_intel_aet_features.c     | 65 +++++++++++++++++
 arch/x86/Kconfig                              |  1 +
 arch/x86/kernel/cpu/resctrl/Makefile          |  1 +
 drivers/platform/x86/intel/pmt/Kconfig        |  3 +
 5 files changed, 143 insertions(+)
 create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h
 create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h new file +73 -0
new file mode 100644
index 000000000000..c835c4108abc
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/* Bits stolen from OOBMSM VSEC discovery code */
+
+enum pmt_feature_id {
+	FEATURE_INVALID			= 0x0,
+	FEATURE_PER_CORE_PERF_TELEM	= 0x1,
+	FEATURE_PER_CORE_ENV_TELEM	= 0x2,
+	FEATURE_PER_RMID_PERF_TELEM	= 0x3,
+	FEATURE_ACCEL_TELEM		= 0x4,
+	FEATURE_UNCORE_TELEM		= 0x5,
+	FEATURE_CRASH_LOG		= 0x6,
+	FEATURE_PETE_LOG		= 0x7,
+	FEATURE_TPMI_CTRL		= 0x8,
+	FEATURE_RESERVED		= 0x9,
+	FEATURE_TRACING			= 0xA,
+	FEATURE_PER_RMID_ENERGY_TELEM	= 0xB,
+	FEATURE_MAX			= 0xB,
+};
+
+/**
+ * struct oobmsm_plat_info - Platform information for a device instance
+ * @cdie_mask:       Mask of all compute dies in the partition
+ * @package_id:      CPU Package id
+ * @partition:       Package partition id when multiple VSEC PCI devices per package
+ * @segment:         PCI segment ID
+ * @bus_number:      PCI bus number
+ * @device_number:   PCI device number
+ * @function_number: PCI function number
+ *
+ * Structure to store platform data for a OOBMSM device instance.
+ */
+struct oobmsm_plat_info {
+	u16 cdie_mask;
+	u8 package_id;
+	u8 partition;
+	u8 segment;
+	u8 bus_number;
+	u8 device_number;
+	u8 function_number;
+};
+
+enum oobmsm_supplier_type {
+	OOBMSM_SUP_PLAT_INFO,
+	OOBMSM_SUP_DISC_INFO,
+	OOBMSM_SUP_S3M_SIMICS,
+	OOBMSM_SUP_TYPE_MAX
+};
+
+struct oobmsm_mapping_supplier {
+	struct device *supplier_dev[OOBMSM_SUP_TYPE_MAX];
+	struct oobmsm_plat_info plat_info;
+	unsigned long features;
+};
+
+struct telemetry_region {
+	struct oobmsm_plat_info	plat_info;
+	void __iomem		*addr;
+	size_t			size;
+	u32			guid;
+	u32			num_rmids;
+};
+
+struct pmt_feature_group {
+	enum pmt_feature_id	id;
+	int			count;
+	struct kref		kref;
+	struct telemetry_region	regions[];
+};
+
+struct pmt_feature_group *intel_pmt_get_regions_by_feature(enum pmt_feature_id id);
+
+void intel_pmt_put_feature_group(struct pmt_feature_group *feature_group);
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c new file +65 -0

65 lines not on this page · raw message

arch/x86/Kconfig +1 -0
index ea29d22a621f..6112cb6cad05 100644
@@ -504,6 +504,7 @@ config X86_CPU_RESCTRL
 	bool "x86 CPU resource control support"
 	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
 	depends on MISC_FILESYSTEMS
+	select INTEL_AET_RESCTRL	if X86_64
 	select ARCH_HAS_CPU_RESCTRL
 	select RESCTRL_FS
 	select RESCTRL_FS_PSEUDO_LOCK
arch/x86/kernel/cpu/resctrl/Makefile +1 -0
index 909be78ec6da..2c3b09f95127 100644
@@ -2,6 +2,7 @@
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= core.o rdtgroup.o monitor.o
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= ctrlmondata.o
 obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= pseudo_lock.o
+obj-$(CONFIG_INTEL_AET_RESCTRL)		+= fake_intel_aet_features.o
 
 # To allow define_trace.h's recursive include:
 CFLAGS_pseudo_lock.o = -I$(src)
drivers/platform/x86/intel/pmt/Kconfig +3 -0
index e916fc966221..6d3b1f64efe9 100644
@@ -38,3 +38,6 @@ config INTEL_PMT_CRASHLOG
 
 	  To compile this driver as a module, choose M here: the module
 	  will be called intel_pmt_crashlog.
+
+config INTEL_AET_RESCTRL
+	bool
Signature · 1 line
2.48.1
maintainer
#2 lore raw

Re: [PATCH v2 01/16] x86/rectrl: Fake OOBMSM interface

Hi Tony,

(nit in shortlog: rectrl -> resctrl)

Quoted 10 lines · Tony Luck

Real version is coming soon ... this is here so the remaining parts will build (and run ... assuming a 2 socket system that supports RDT monitoring ... only missing part is that the event counters just report fixed values).

Real version of this would just add the INTEL_AET_RESCTRL Kconfig option with dependency checks on
INTEL_VSEC=y && INTEL_AET_TELEMETRY=y && INTEL_AET_DISCOVERY=y

Just for RFC discussion.

Would appreciate a bit more detail about what OOBMSM provides to be able to understand this series.

Even though changelog mentions "event counters" I am not able to recognize any unique events provided by this interface. Instead it just seems to provide a memory region that is entirely up to resctrl to interpret based on the "unique identifier" hinted to in cover letter. I could not find any description that connects the "unique identifier" to the "guid" used in following patches. I think it will be helpful to upfront connect the high level "unique identifier" with the "guid" that the patches use to make this obvious.

Closest information to something that can be used by resctrl is "num_rmids". Could you please add information on how "num_rmids" relates to the memory region that is only specified via an addr and size?

Quoted 70 lines
---
 .../cpu/resctrl/fake_intel_aet_features.h     | 73 +++++++++++++++++++
 .../cpu/resctrl/fake_intel_aet_features.c     | 65 +++++++++++++++++
 arch/x86/Kconfig                              |  1 +
 arch/x86/kernel/cpu/resctrl/Makefile          |  1 +
 drivers/platform/x86/intel/pmt/Kconfig        |  3 +
 5 files changed, 143 insertions(+)
 create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h
 create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h new file +52 -0
new file mode 100644
index 000000000000..c835c4108abc
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/* Bits stolen from OOBMSM VSEC discovery code */
+
+enum pmt_feature_id {
+	FEATURE_INVALID			= 0x0,
+	FEATURE_PER_CORE_PERF_TELEM	= 0x1,
+	FEATURE_PER_CORE_ENV_TELEM	= 0x2,
+	FEATURE_PER_RMID_PERF_TELEM	= 0x3,
+	FEATURE_ACCEL_TELEM		= 0x4,
+	FEATURE_UNCORE_TELEM		= 0x5,
+	FEATURE_CRASH_LOG		= 0x6,
+	FEATURE_PETE_LOG		= 0x7,
+	FEATURE_TPMI_CTRL		= 0x8,
+	FEATURE_RESERVED		= 0x9,
+	FEATURE_TRACING			= 0xA,
+	FEATURE_PER_RMID_ENERGY_TELEM	= 0xB,
+	FEATURE_MAX			= 0xB,
+};
+
+/**
+ * struct oobmsm_plat_info - Platform information for a device instance
+ * @cdie_mask:       Mask of all compute dies in the partition
+ * @package_id:      CPU Package id
+ * @partition:       Package partition id when multiple VSEC PCI devices per package
+ * @segment:         PCI segment ID
+ * @bus_number:      PCI bus number
+ * @device_number:   PCI device number
+ * @function_number: PCI function number
+ *
+ * Structure to store platform data for a OOBMSM device instance.
+ */
+struct oobmsm_plat_info {
+	u16 cdie_mask;
+	u8 package_id;
+	u8 partition;
+	u8 segment;
+	u8 bus_number;
+	u8 device_number;
+	u8 function_number;
+};
+
+enum oobmsm_supplier_type {
+	OOBMSM_SUP_PLAT_INFO,
+	OOBMSM_SUP_DISC_INFO,
+	OOBMSM_SUP_S3M_SIMICS,
+	OOBMSM_SUP_TYPE_MAX
+};
+
+struct oobmsm_mapping_supplier {
+	struct device *supplier_dev[OOBMSM_SUP_TYPE_MAX];
+	struct oobmsm_plat_info plat_info;

arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h line 62

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h new file +10 -0
+	unsigned long features;
+};
+
+struct telemetry_region {
+	struct oobmsm_plat_info	plat_info;
+	void __iomem		*addr;
+	size_t			size;
+	u32			guid;
+	u32			num_rmids;
+};

Could there be some description of what a "telemetry_region" is and how the members should be interpreted by resctrl?

Quoted 54 lines
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h new file +11 -0
+
+struct pmt_feature_group {
+	enum pmt_feature_id	id;
+	int			count;
+	struct kref		kref;
+	struct telemetry_region	regions[];
+};
+
+struct pmt_feature_group *intel_pmt_get_regions_by_feature(enum pmt_feature_id id);
+
+void intel_pmt_put_feature_group(struct pmt_feature_group *feature_group);
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c new file +37 -0
new file mode 100644
index 000000000000..b537068d99fb
@@ -0,0 +1,65 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <linux/cleanup.h>
+#include <linux/minmax.h>
+#include <linux/slab.h>
+#include "fake_intel_aet_features.h"
+#include <linux/intel_vsec.h>
+#include <linux/resctrl.h>
+
+#include "internal.h"
+
+#define ENERGY_QWORDS	((576 * 2) + 3)
+#define PERF_QWORDS	((576 * 7) + 3)
+
+static long pg[4 * ENERGY_QWORDS + 2 * PERF_QWORDS];
+
+static int __init fill(void)
+{
+	u64 val = 0;
+
+	for (int i = 0; i < sizeof(pg); i += sizeof(val)) {
+		pg[i / sizeof(val)] = BIT_ULL(63) + val;
+		val++;
+	}
+	return 0;
+}
+device_initcall(fill);
+
+#define PKG_REGION(_entry, _guid, _addr, _pkg)	\
+	[_entry] = { .guid = _guid, .addr = (void __iomem *)_addr, .plat_info = { .package_id = _pkg }}
+
+static struct pmt_feature_group fake_energy = {
+	.count = 4,
+	.regions = {
+		PKG_REGION(0, 0x26696143, &pg[0 * ENERGY_QWORDS], 0),
+		PKG_REGION(1, 0x26696143, &pg[1 * ENERGY_QWORDS], 0),
+		PKG_REGION(2, 0x26696143, &pg[2 * ENERGY_QWORDS], 1),
+		PKG_REGION(3, 0x26696143, &pg[3 * ENERGY_QWORDS], 1)

arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c line 47

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c new file +10 -0
+	}
+};
+
+static struct pmt_feature_group fake_perf = {
+	.count = 2,
+	.regions = {
+		PKG_REGION(0, 0x26557651, &pg[4 * ENERGY_QWORDS + 0 * PERF_QWORDS], 0),
+		PKG_REGION(1, 0x26557651, &pg[4 * ENERGY_QWORDS + 1 * PERF_QWORDS], 1)
+	}
+};

Could there be some guidance on how to interpret the hardcoded data? For example, one group contains two regions and the other four. Was this just done for testing to ensure implementation supports multiple regions per package or ...? Is it expected that multiple feature groups could have different number of regions? I also think initializing the id would be helpful to understand the example better.

Quoted 23 lines
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c new file +18 -0
+
+struct pmt_feature_group *
+intel_pmt_get_regions_by_feature(enum pmt_feature_id id)
+{
+	switch (id) {
+	case FEATURE_PER_RMID_ENERGY_TELEM:
+		return &fake_energy;
+	case FEATURE_PER_RMID_PERF_TELEM:
+		return &fake_perf;
+	default:
+		return ERR_PTR(-ENOENT);
+	}
+	return ERR_PTR(-ENOENT);
+}
+
+void intel_pmt_put_feature_group(struct pmt_feature_group *feature_group)
+{
+}
arch/x86/Kconfig +0 -0
index ea29d22a621f..6112cb6cad05 100644

arch/x86/Kconfig config X86_CPU_RESCTRL · line 507

Quoted 4 lines
arch/x86/Kconfig +1 -0
@@ -504,6 +504,7 @@ config X86_CPU_RESCTRL
 	bool "x86 CPU resource control support"
 	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
 	depends on MISC_FILESYSTEMS
+	select INTEL_AET_RESCTRL	if X86_64

I expect something like this will stay (i.e. not part of the "Fake" code). In this case, should this perhaps only be selected on Intel (CPU_SUP_INTEL)? (nit: the tab is unexpected)

Quoted 20 lines
arch/x86/Kconfig +0 -0
 	select ARCH_HAS_CPU_RESCTRL
 	select RESCTRL_FS
 	select RESCTRL_FS_PSEUDO_LOCK
arch/x86/kernel/cpu/resctrl/Makefile +1 -0
index 909be78ec6da..2c3b09f95127 100644
@@ -2,6 +2,7 @@
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= core.o rdtgroup.o monitor.o
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= ctrlmondata.o
 obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= pseudo_lock.o
+obj-$(CONFIG_INTEL_AET_RESCTRL)		+= fake_intel_aet_features.o
 
 # To allow define_trace.h's recursive include:
 CFLAGS_pseudo_lock.o = -I$(src)
drivers/platform/x86/intel/pmt/Kconfig +0 -0
index e916fc966221..6d3b1f64efe9 100644

drivers/platform/x86/intel/pmt/Kconfig config INTEL_PMT_CRASHLOG · line 43

Quoted 6 lines
drivers/platform/x86/intel/pmt/Kconfig +3 -0
@@ -38,3 +38,6 @@ config INTEL_PMT_CRASHLOG
 
 	  To compile this driver as a module, choose M here: the module
 	  will be called intel_pmt_crashlog.
+
+config INTEL_AET_RESCTRL
+	bool

I expect that this will also stay ... if so, could this be expanded to have needed dependency and also be accompanied by some documentation. Something like "Architecture selects this when ...." This will make it clear that this is not something a user will select during kernel build while also explaining what it is used for.

Reinette

author maintainer
#3 lore raw

Re: [PATCH v2 01/16] x86/rectrl: Fake OOBMSM interface

Quoted 1 line · Reinette Chatre

Hi Tony,

Thanks for the review.

Quoted 8 lines

(nit in shortlog: rectrl -> resctrl)

Quoted 5 lines · Tony Luck

Real version is coming soon ... this is here so the remaining parts will build (and run ... assuming a 2 socket system that supports RDT monitoring ... only missing part is that the event counters just report fixed values).

Quoted 8 lines
Quoted 5 lines

Real version of this would just add the INTEL_AET_RESCTRL Kconfig option with dependency checks on
INTEL_VSEC=y && INTEL_AET_TELEMETRY=y && INTEL_AET_DISCOVERY=y

Just for RFC discussion.

Would appreciate a bit more detail about what OOBMSM provides to be able to understand this series.

Short answer is just what you see in this structure:

struct telemetry_region {
struct oobmsm_plat_info plat_info;

   void __iomem            *addr;
   size_t                  size;
   u32                     guid;
   u32                     num_rmids;

};

I've passed on your suggestion for some comments on the definition to David Box, since this structure will be part of his patches to enable the discovery of OOBMSM features.

The trail for breadcrumbs from this to event counters is:
1) Lookup the "guid" in XML files at https://github.com/intel/Intel-PMT
2) Each of those lists each event counter in the MMIO region referred to by the <addr,size> fields in excruciating detail. E.g. for the first register (at offset 0 from "addr"):

<TELI:description>Accumulated core energy usage across all cores running RMID 0</TELI:description>

          <TELI:SampleType>Snapshot</TELI:SampleType>
          <TransFormInputs xmlns="http://schemas.intel.com/telemetry/base/common">
                  <TransFormInput varName="parameter_0">
                          <sampleGroupIDREF>Container_0</sampleGroupIDREF>
                          <sampleIDREF>RMID_0_CORE_ENERGY</sampleIDREF>
                  </TransFormInput>
          </TransFormInputs>
          <TELI:transformREF>U63.45.18</TELI:transformREF>

</TELI:T_AggregatorSample>
<TELI:T_AggregatorSample sampleName="RMID_0_CORE_ENERGY_VALID" sampleGroup="RMID_0_CORE_ENERGY" datatypeIDREF="tcounter_valid" sampleID="1">
<TELI:description>If set, RMID_0_CORE_ENERGY counter reading is valid</TELI: description> <TELI:SampleType>Snapshot</TELI:SampleType>
<TransFormInputs xmlns="http://schemas.intel.com/telemetry/base/common">
<TransFormInput varName="parameter_0">
<sampleGroupIDREF>Container_0</sampleGroupIDREF> <sampleIDREF>RMID_0_CORE_ENERGY_VALID</sampleIDREF>
</TransFormInput>
</TransFormInputs>
<TELI:transformREF>passthru</TELI:transformREF>
</TELI:T_AggregatorSample>

This verbosity repeats for each of the events for RMID 0, then for RMID 1, 2, ...

The important bits for Linux are the name, the type, and the valid bit.

I will add more comments to the Linux structures when they are added in one of the later patches in this series.

Quoted 89 lines

Even though changelog mentions "event counters" I am not able to recognize any unique events provided by this interface. Instead it just seems to provide a memory region that is entirely up to resctrl to interpret based on the "unique identifier" hinted to in cover letter. I could not find any description that connects the "unique identifier" to the "guid" used in following patches. I think it will be helpful to upfront connect the high level "unique identifier" with the "guid" that the patches use to make this obvious.

Closest information to something that can be used by resctrl is "num_rmids". Could you please add information on how "num_rmids" relates to the memory region that is only specified via an addr and size?

Quoted 75 lines
---
 .../cpu/resctrl/fake_intel_aet_features.h     | 73 +++++++++++++++++++
 .../cpu/resctrl/fake_intel_aet_features.c     | 65 +++++++++++++++++
 arch/x86/Kconfig                              |  1 +
 arch/x86/kernel/cpu/resctrl/Makefile          |  1 +
 drivers/platform/x86/intel/pmt/Kconfig        |  3 +
 5 files changed, 143 insertions(+)
 create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h
 create mode 100644 arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h new file +57 -0
new file mode 100644
index 000000000000..c835c4108abc
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/* Bits stolen from OOBMSM VSEC discovery code */
+
+enum pmt_feature_id {
+	FEATURE_INVALID			= 0x0,
+	FEATURE_PER_CORE_PERF_TELEM	= 0x1,
+	FEATURE_PER_CORE_ENV_TELEM	= 0x2,
+	FEATURE_PER_RMID_PERF_TELEM	= 0x3,
+	FEATURE_ACCEL_TELEM		= 0x4,
+	FEATURE_UNCORE_TELEM		= 0x5,
+	FEATURE_CRASH_LOG		= 0x6,
+	FEATURE_PETE_LOG		= 0x7,
+	FEATURE_TPMI_CTRL		= 0x8,
+	FEATURE_RESERVED		= 0x9,
+	FEATURE_TRACING			= 0xA,
+	FEATURE_PER_RMID_ENERGY_TELEM	= 0xB,
+	FEATURE_MAX			= 0xB,
+};
+
+/**
+ * struct oobmsm_plat_info - Platform information for a device instance
+ * @cdie_mask:       Mask of all compute dies in the partition
+ * @package_id:      CPU Package id
+ * @partition:       Package partition id when multiple VSEC PCI devices per package
+ * @segment:         PCI segment ID
+ * @bus_number:      PCI bus number
+ * @device_number:   PCI device number
+ * @function_number: PCI function number
+ *
+ * Structure to store platform data for a OOBMSM device instance.
+ */
+struct oobmsm_plat_info {
+	u16 cdie_mask;
+	u8 package_id;
+	u8 partition;
+	u8 segment;
+	u8 bus_number;
+	u8 device_number;
+	u8 function_number;
+};
+
+enum oobmsm_supplier_type {
+	OOBMSM_SUP_PLAT_INFO,
+	OOBMSM_SUP_DISC_INFO,
+	OOBMSM_SUP_S3M_SIMICS,
+	OOBMSM_SUP_TYPE_MAX
+};
+
+struct oobmsm_mapping_supplier {
+	struct device *supplier_dev[OOBMSM_SUP_TYPE_MAX];
+	struct oobmsm_plat_info plat_info;
+	unsigned long features;
+};
+
+struct telemetry_region {
+	struct oobmsm_plat_info	plat_info;
Quoted 8 lines

arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h line 62

Quoted 5 lines
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h new file +5 -0
+	void __iomem		*addr;
+	size_t			size;
+	u32			guid;
+	u32			num_rmids;
+};

Could there be some description of what a "telemetry_region" is and how the members should be interpreted by resctrl?

As mentioned above. I passed this request to David Box.

Quoted 62 lines
Quoted 62 lines
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.h new file +11 -0
+
+struct pmt_feature_group {
+	enum pmt_feature_id	id;
+	int			count;
+	struct kref		kref;
+	struct telemetry_region	regions[];
+};
+
+struct pmt_feature_group *intel_pmt_get_regions_by_feature(enum pmt_feature_id id);
+
+void intel_pmt_put_feature_group(struct pmt_feature_group *feature_group);
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c new file +45 -0
new file mode 100644
index 000000000000..b537068d99fb
@@ -0,0 +1,65 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <linux/cleanup.h>
+#include <linux/minmax.h>
+#include <linux/slab.h>
+#include "fake_intel_aet_features.h"
+#include <linux/intel_vsec.h>
+#include <linux/resctrl.h>
+
+#include "internal.h"
+
+#define ENERGY_QWORDS	((576 * 2) + 3)
+#define PERF_QWORDS	((576 * 7) + 3)
+
+static long pg[4 * ENERGY_QWORDS + 2 * PERF_QWORDS];
+
+static int __init fill(void)
+{
+	u64 val = 0;
+
+	for (int i = 0; i < sizeof(pg); i += sizeof(val)) {
+		pg[i / sizeof(val)] = BIT_ULL(63) + val;
+		val++;
+	}
+	return 0;
+}
+device_initcall(fill);
+
+#define PKG_REGION(_entry, _guid, _addr, _pkg)	\
+	[_entry] = { .guid = _guid, .addr = (void __iomem *)_addr, .plat_info = { .package_id = _pkg }}
+
+static struct pmt_feature_group fake_energy = {
+	.count = 4,
+	.regions = {
+		PKG_REGION(0, 0x26696143, &pg[0 * ENERGY_QWORDS], 0),
+		PKG_REGION(1, 0x26696143, &pg[1 * ENERGY_QWORDS], 0),
+		PKG_REGION(2, 0x26696143, &pg[2 * ENERGY_QWORDS], 1),
+		PKG_REGION(3, 0x26696143, &pg[3 * ENERGY_QWORDS], 1)
+	}
+};
+
+static struct pmt_feature_group fake_perf = {
+	.count = 2,
+	.regions = {
+		PKG_REGION(0, 0x26557651, &pg[4 * ENERGY_QWORDS + 0 * PERF_QWORDS], 0),
+		PKG_REGION(1, 0x26557651, &pg[4 * ENERGY_QWORDS + 1 * PERF_QWORDS], 1)
Quoted 8 lines

arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c line 47

Quoted 2 lines
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c new file +2 -0
+	}
+};

Could there be some guidance on how to interpret the hardcoded data? For example, one group contains two regions and the other four. Was this just done for testing to ensure implementation supports multiple regions per package or ...? Is it expected that multiple feature groups could have different number of regions? I also think initializing the id would be helpful to understand the example better.

You are correct. The varying number of regions was simply to exercise the code that needs to aggregate values from multiple regions. The first implementation of this has homogeneous aggregators all working for the same events from the same CPUs. But I don't see that as a requirement. I could envision a system with different aggregators for different events, perhaps servicing different groups of CPUs on the same package.

Quoted 23 lines
Quoted 23 lines
arch/x86/kernel/cpu/resctrl/fake_intel_aet_features.c new file +18 -0
+
+struct pmt_feature_group *
+intel_pmt_get_regions_by_feature(enum pmt_feature_id id)
+{
+	switch (id) {
+	case FEATURE_PER_RMID_ENERGY_TELEM:
+		return &fake_energy;
+	case FEATURE_PER_RMID_PERF_TELEM:
+		return &fake_perf;
+	default:
+		return ERR_PTR(-ENOENT);
+	}
+	return ERR_PTR(-ENOENT);
+}
+
+void intel_pmt_put_feature_group(struct pmt_feature_group *feature_group)
+{
+}
arch/x86/Kconfig +0 -0
index ea29d22a621f..6112cb6cad05 100644
@@ -504,6 +504,7 @@ config X86_CPU_RESCTRL
Quoted 8 lines

arch/x86/Kconfig config X86_CPU_RESCTRL · line 507

Quoted 4 lines
arch/x86/Kconfig +1 -0
 	bool "x86 CPU resource control support"
 	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
 	depends on MISC_FILESYSTEMS
+	select INTEL_AET_RESCTRL	if X86_64

I expect something like this will stay (i.e. not part of the "Fake" code). In this case, should this perhaps only be selected on Intel (CPU_SUP_INTEL)? (nit: the tab is unexpected)

Yes, this part isn't fake, and is should depend on CPU_SUP_INTEL.

Quoted 21 lines
Quoted 21 lines
arch/x86/Kconfig +0 -0
 	select ARCH_HAS_CPU_RESCTRL
 	select RESCTRL_FS
 	select RESCTRL_FS_PSEUDO_LOCK
arch/x86/kernel/cpu/resctrl/Makefile +1 -0
index 909be78ec6da..2c3b09f95127 100644
@@ -2,6 +2,7 @@
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= core.o rdtgroup.o monitor.o
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= ctrlmondata.o
 obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= pseudo_lock.o
+obj-$(CONFIG_INTEL_AET_RESCTRL)		+= fake_intel_aet_features.o
 
 # To allow define_trace.h's recursive include:
 CFLAGS_pseudo_lock.o = -I$(src)
drivers/platform/x86/intel/pmt/Kconfig +0 -0
index e916fc966221..6d3b1f64efe9 100644
@@ -38,3 +38,6 @@ config INTEL_PMT_CRASHLOG
 
Quoted 11 lines

drivers/platform/x86/intel/pmt/Kconfig config INTEL_PMT_CRASHLOG · line 43

Quoted 5 lines
drivers/platform/x86/intel/pmt/Kconfig +3 -0
 	  To compile this driver as a module, choose M here: the module
 	  will be called intel_pmt_crashlog.
+
+config INTEL_AET_RESCTRL
+	bool

I expect that this will also stay ... if so, could this be expanded to have needed dependency and also be accompanied by some documentation. Something like "Architecture selects this when ...." This will make it clear that this is not something a user will select during kernel build while also explaining what it is used for.

This part will also stay. I will add a comment as you suggest.

Quoted 1 line

Reinette

-Tony

author maintainer
#4 lore raw

PATCH v2 2/16 x86/resctrl: Move L3 initialization out of domain_add_cpu_mon()

In reply to Tony Luck

To prepare for additional types of monitoring domains, move all the L3 specific initialization into a helper function.

---
 arch/x86/kernel/cpu/resctrl/core.c | 52 +++++++++++++++++-------------
 1 file changed, 30 insertions(+), 22 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +30 -22
index 6ed0d4f5d6a3..4205aeb4f979 100644
@@ -492,33 +492,12 @@ static void domain_add_cpu_ctrl(int cpu, struct rdt_resource *r)
 	}
 }
 
-static void domain_add_cpu_mon(int cpu, struct rdt_resource *r)
+static void setup_l3_mon_domain(int cpu, int id, struct rdt_resource *r, struct list_head *add_pos)
 {
-	int id = get_domain_id_from_scope(cpu, r->mon_scope);
-	struct list_head *add_pos = NULL;
 	struct rdt_hw_mon_domain *hw_dom;
-	struct rdt_domain_hdr *hdr;
 	struct rdt_mon_domain *d;
 	int err;
 
-	lockdep_assert_held(&domain_list_lock);
-
-	if (id < 0) {
-		pr_warn_once("Can't find monitor domain id for CPU:%d scope:%d for resource %s\n",
-			     cpu, r->mon_scope, r->name);
-		return;
-	}
-
-	hdr = resctrl_find_domain(&r->mon_domains, id, &add_pos);
-	if (hdr) {
-		if (WARN_ON_ONCE(hdr->type != RESCTRL_MON_DOMAIN))
-			return;
-		d = container_of(hdr, struct rdt_mon_domain, hdr);
-
-		cpumask_set_cpu(cpu, &d->hdr.cpu_mask);
-		return;
-	}
-
 	hw_dom = kzalloc_node(sizeof(*hw_dom), GFP_KERNEL, cpu_to_node(cpu));
 	if (!hw_dom)
 		return;
@@ -551,6 +530,35 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r)
 	}
 }
 
+static void domain_add_cpu_mon(int cpu, struct rdt_resource *r)
+{
+	int id = get_domain_id_from_scope(cpu, r->mon_scope);
+	struct list_head *add_pos = NULL;
+	struct rdt_domain_hdr *hdr;
+
+	lockdep_assert_held(&domain_list_lock);
+
+	if (id < 0) {
+		pr_warn_once("Can't find monitor domain id for CPU:%d scope:%d for resource %s\n",
+			     cpu, r->mon_scope, r->name);
+		return;
+	}
+
+	hdr = resctrl_find_domain(&r->mon_domains, id, &add_pos);
+	if (hdr) {
+		cpumask_set_cpu(cpu, &hdr->cpu_mask);
+		return;
+	}
+
+	switch (r->rid) {
+	case RDT_RESOURCE_L3:
+		setup_l3_mon_domain(cpu, id, r, add_pos);
+		break;
+	default:
+		WARN_ON_ONCE(1);
+	}
+}
+
 static void domain_add_cpu(int cpu, struct rdt_resource *r)
 {
 	if (r->alloc_capable)
Signature · 1 line
2.48.1
author maintainer
#5 lore raw

PATCH v2 3/16 x86/resctrl: Refactor domain_remove_cpu_mon() ready for new domain types

In reply to Tony Luck

Different types of domains require different actions when the last CPU in the domain is removed.

Refactor to make it easy to add new actions for new types of domains.

---
 arch/x86/kernel/cpu/resctrl/core.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +8 -7
index 4205aeb4f979..3c343d0d18fb 100644
@@ -635,20 +635,21 @@ static void domain_remove_cpu_mon(int cpu, struct rdt_resource *r)
 		return;
 	}
 
-	if (WARN_ON_ONCE(hdr->type != RESCTRL_MON_DOMAIN))
+	cpumask_clear_cpu(cpu, &hdr->cpu_mask);
+	if (!cpumask_empty(&hdr->cpu_mask))
 		return;
 
 	d = container_of(hdr, struct rdt_mon_domain, hdr);
-	hw_dom = resctrl_to_arch_mon_dom(d);
 
-	cpumask_clear_cpu(cpu, &d->hdr.cpu_mask);
-	if (cpumask_empty(&d->hdr.cpu_mask)) {
+	switch (r->rid) {
+	case RDT_RESOURCE_L3:
+		hw_dom = resctrl_to_arch_mon_dom(d);
+
 		resctrl_offline_mon_domain(r, d);
-		list_del_rcu(&d->hdr.list);
+		list_del_rcu(&hdr->list);
 		synchronize_rcu();
 		mon_domain_free(hw_dom);
-
-		return;
+		break;
 	}
 }
 
Signature · 1 line
2.48.1
author maintainer
#6 lore raw

PATCH v2 4/16 x86/resctrl: Change generic monitor functions to use struct rdt_domain_hdr

In reply to Tony Luck

Functions that don't need the internal details of the rdt_mon_domain can operate on just the rdt_domain_hdr.

---
 include/linux/resctrl.h            |  2 +-
 arch/x86/kernel/cpu/resctrl/core.c |  2 +-
 fs/resctrl/rdtgroup.c              | 57 +++++++++++++++++++-----------
 3 files changed, 39 insertions(+), 22 deletions(-)
include/linux/resctrl.h +1 -1
index 5c7c8bf2c47f..25c3ee78de3d 100644
@@ -413,7 +413,7 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d,
 u32 resctrl_arch_get_config(struct rdt_resource *r, struct rdt_ctrl_domain *d,
 			    u32 closid, enum resctrl_conf_type type);
 int resctrl_online_ctrl_domain(struct rdt_resource *r, struct rdt_ctrl_domain *d);
-int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_mon_domain *d);
+int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *hdr);
 void resctrl_offline_ctrl_domain(struct rdt_resource *r, struct rdt_ctrl_domain *d);
 void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_mon_domain *d);
 void resctrl_online_cpu(unsigned int cpu);
arch/x86/kernel/cpu/resctrl/core.c +1 -1
index 3c343d0d18fb..c316268b4442 100644
@@ -522,7 +522,7 @@ static void setup_l3_mon_domain(int cpu, int id, struct rdt_resource *r, struct
 
 	list_add_tail_rcu(&d->hdr.list, add_pos);
 
-	err = resctrl_online_mon_domain(r, d);
+	err = resctrl_online_mon_domain(r, &d->hdr);
 	if (err) {
 		list_del_rcu(&d->hdr.list);
 		synchronize_rcu();
fs/resctrl/rdtgroup.c +37 -20

170 lines not on this page · raw message

Signature · 1 line
2.48.1
maintainer
#7 lore raw

Re: [PATCH v2 04/16] x86/resctrl: Change generic monitor functions to use struct rdt_domain_hdr

Hi Tony,

Quoted 2 lines · Tony Luck

Functions that don't need the internal details of the rdt_mon_domain can operate on just the rdt_domain_hdr.

This does not seem accurate. The functions are modified to take rdt_domain_hdr as parameter but then the functions are modified to extract rdt_mon_domain based on rdt_domain_hdr .... and proceeds to operate on internals of rdt_mon_domain in a way that contradicts the changelog.

Considering what comes later this seems risky to me to rely on the code flow to interpret which structure rdt_domain_hdr forms part of. I think that it will be safer if rdt_domain_hdr gets an identifier that reflects which structure it forms part of so that the accessors could be made explicit and have error checking.

Reinette

author maintainer
#8 lore raw

Re: [PATCH v2 04/16] x86/resctrl: Change generic monitor functions to use struct rdt_domain_hdr

Quoted 11 lines · Reinette Chatre

Hi Tony,

Quoted 2 lines · Tony Luck

Functions that don't need the internal details of the rdt_mon_domain can operate on just the rdt_domain_hdr.

This does not seem accurate. The functions are modified to take rdt_domain_hdr as parameter but then the functions are modified to extract rdt_mon_domain based on rdt_domain_hdr .... and proceeds to operate on internals of rdt_mon_domain in a way that contradicts the changelog.

Quoted 5 lines

Considering what comes later this seems risky to me to rely on the code flow to interpret which structure rdt_domain_hdr forms part of. I think that it will be safer if rdt_domain_hdr gets an identifier that reflects which structure it forms part of so that the accessors could be made explicit and have error checking.

This needs some more thought and cleanup. I ended up with a mix of places that really just wanted the header. E.g. adding second and subsequent CPUs to a domain, or deleting any but the last CPU from a domain, only need to update the cpu_mask. But other places end up clumsily using container_of() to get to the surrounding mon_domain. With different types of mon_domain a type field would help sanity checking.

Quoted 2 lines

Reinette

-Tony

author maintainer
#9 lore raw

PATCH v2 5/16 x86/resctrl: Add and initialize rdt_resource for package scope core monitor

In reply to Tony Luck

New resource for monitoring core events reported at package level.

---
 include/linux/resctrl.h            |  1 +
 include/linux/resctrl_types.h      |  1 +
 fs/resctrl/internal.h              |  2 ++
 arch/x86/kernel/cpu/resctrl/core.c | 11 +++++++++++
 fs/resctrl/rdtgroup.c              |  2 ++
 5 files changed, 17 insertions(+)
include/linux/resctrl.h +1 -0
index 25c3ee78de3d..6c895ec220fe 100644
@@ -233,6 +233,7 @@ enum resctrl_scope {
 	RESCTRL_L2_CACHE = 2,
 	RESCTRL_L3_CACHE = 3,
 	RESCTRL_L3_NODE,
+	RESCTRL_PACKAGE,
 };
 
 /**
include/linux/resctrl_types.h +1 -0
index a7faf2cd5406..8f967e03af5a 100644
@@ -39,6 +39,7 @@ enum resctrl_res_level {
 	RDT_RESOURCE_L2,
 	RDT_RESOURCE_MBA,
 	RDT_RESOURCE_SMBA,
+	RDT_RESOURCE_INTEL_AET,
 
 	/* Must be the last */
 	RDT_NUM_RESOURCES,
fs/resctrl/internal.h +2 -0
index ec3863d18f68..3a100007301d 100644
@@ -240,6 +240,8 @@ struct rdtgroup {
 
 #define RFTYPE_DEBUG			BIT(10)
 
+#define RFTYPE_RES_PKG			BIT(11)
+
 #define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
 
 #define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
arch/x86/kernel/cpu/resctrl/core.c +11 -0
index c316268b4442..c8cc3104f56c 100644
@@ -99,6 +99,15 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCES] = {
 			.schema_fmt		= RESCTRL_SCHEMA_RANGE,
 		},
 	},
+	[RDT_RESOURCE_INTEL_AET] =
+	{
+		.r_resctrl = {
+			.rid			= RDT_RESOURCE_INTEL_AET,
+			.name			= "PKG",
+			.mon_scope		= RESCTRL_PACKAGE,
+			.mon_domains		= mon_domain_init(RDT_RESOURCE_INTEL_AET),
+		},
+	},
 };
 
 u32 resctrl_arch_system_num_rmid_idx(void)
@@ -430,6 +439,8 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)
 		return get_cpu_cacheinfo_id(cpu, scope);
 	case RESCTRL_L3_NODE:
 		return cpu_to_node(cpu);
+	case RESCTRL_PACKAGE:
+		return topology_physical_package_id(cpu);
 	default:
 		break;
 	}
fs/resctrl/rdtgroup.c +2 -0
index dbfb7d3bc3bc..a90291f57330 100644
@@ -2179,6 +2179,8 @@ static unsigned long fflags_from_resource(struct rdt_resource *r)
 	case RDT_RESOURCE_MBA:
 	case RDT_RESOURCE_SMBA:
 		return RFTYPE_RES_MB;
+	case RDT_RESOURCE_INTEL_AET:
+		return RFTYPE_RES_PKG;
 	}
 
 	return WARN_ON_ONCE(1);
Signature · 1 line
2.48.1
maintainer
#10 lore raw

Re: [PATCH v2 05/16] x86/resctrl: Add and initialize rdt_resource for package scope core monitor

Hi Tony,

Quoted 1 line · Tony Luck

New resource for monitoring core events reported at package level.

Could you please add proper (per Documentation/process/maintainer-tip.rst) changelogs to all patches in the series?

Quoted 27 lines
---
 include/linux/resctrl.h            |  1 +
 include/linux/resctrl_types.h      |  1 +
 fs/resctrl/internal.h              |  2 ++
 arch/x86/kernel/cpu/resctrl/core.c | 11 +++++++++++
 fs/resctrl/rdtgroup.c              |  2 ++
 5 files changed, 17 insertions(+)
include/linux/resctrl.h +1 -0
index 25c3ee78de3d..6c895ec220fe 100644
@@ -233,6 +233,7 @@ enum resctrl_scope {
 	RESCTRL_L2_CACHE = 2,
 	RESCTRL_L3_CACHE = 3,
 	RESCTRL_L3_NODE,
+	RESCTRL_PACKAGE,
 };
 
 /**
include/linux/resctrl_types.h +0 -0
index a7faf2cd5406..8f967e03af5a 100644

include/linux/resctrl_types.h enum resctrl_res_level { · line 42

Quoted 4 lines
include/linux/resctrl_types.h +1 -0
@@ -39,6 +39,7 @@ enum resctrl_res_level {
 	RDT_RESOURCE_L2,
 	RDT_RESOURCE_MBA,
 	RDT_RESOURCE_SMBA,
+	RDT_RESOURCE_INTEL_AET,

This is fs code so architecture specific code needs to be avoided. Any other architecture that may need to report similar data would be forced to use this resource name.

fs/resctrl/internal.h struct rdtgroup { · line 243

Quoted 12 lines
include/linux/resctrl_types.h +0 -0
 	/* Must be the last */
 	RDT_NUM_RESOURCES,
fs/resctrl/internal.h +1 -0
index ec3863d18f68..3a100007301d 100644
@@ -240,6 +240,8 @@ struct rdtgroup {
 
 #define RFTYPE_DEBUG			BIT(10)
 
+#define RFTYPE_RES_PKG			BIT(11)

"PKG" is not the resource but instead the scope, no?

Quoted 9 lines
fs/resctrl/internal.h +1 -0
+
 #define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
 
 #define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
arch/x86/kernel/cpu/resctrl/core.c +0 -0
index c316268b4442..c8cc3104f56c 100644

arch/x86/kernel/cpu/resctrl/core.c struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCES] = { · line 106

Quoted 8 lines
arch/x86/kernel/cpu/resctrl/core.c +5 -0
@@ -99,6 +99,15 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCES] = {
 			.schema_fmt		= RESCTRL_SCHEMA_RANGE,
 		},
 	},
+	[RDT_RESOURCE_INTEL_AET] =
+	{
+		.r_resctrl = {
+			.rid			= RDT_RESOURCE_INTEL_AET,
+			.name			= "PKG",

The resource name should not be the scope. It should be a name that reflects the resource being monitored. In this case it is "core"/"cpu"? I understand that this may create confusion since the data is provided at package scope so perhaps the "resource" can be "perf" and then all the events can include "core" in their name? If the intention is to guide user space in how to interpret the domain IDs then we could consider something like "perf_pkg" or even "phys_pkg_perf" that prepares resctrl for future perf events that may need reporting at a different scope?

Quoted 21 lines
arch/x86/kernel/cpu/resctrl/core.c +6 -0
+			.mon_scope		= RESCTRL_PACKAGE,
+			.mon_domains		= mon_domain_init(RDT_RESOURCE_INTEL_AET),
+		},
+	},
 };
 
 u32 resctrl_arch_system_num_rmid_idx(void)
@@ -430,6 +439,8 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)
 		return get_cpu_cacheinfo_id(cpu, scope);
 	case RESCTRL_L3_NODE:
 		return cpu_to_node(cpu);
+	case RESCTRL_PACKAGE:
+		return topology_physical_package_id(cpu);
 	default:
 		break;
 	}
fs/resctrl/rdtgroup.c +0 -0
index dbfb7d3bc3bc..a90291f57330 100644

fs/resctrl/rdtgroup.c static unsigned long fflags_from_resource(struct rdt_resource *r) · line 2186

Quoted 8 lines
fs/resctrl/rdtgroup.c +2 -0
@@ -2179,6 +2179,8 @@ static unsigned long fflags_from_resource(struct rdt_resource *r)
 	case RDT_RESOURCE_MBA:
 	case RDT_RESOURCE_SMBA:
 		return RFTYPE_RES_MB;
+	case RDT_RESOURCE_INTEL_AET:
+		return RFTYPE_RES_PKG;
 	}
 
 	return WARN_ON_ONCE(1);

Reinette

author maintainer
#11 lore raw

Re: [PATCH v2 05/16] x86/resctrl: Add and initialize rdt_resource for package scope core monitor

Quoted 7 lines · Reinette Chatre

Hi Tony,

Quoted 1 line · Tony Luck

New resource for monitoring core events reported at package level.

Could you please add proper (per Documentation/process/maintainer-tip.rst) changelogs to all patches in the series?

Will do.

Quoted 21 lines
Quoted 21 lines
---
 include/linux/resctrl.h            |  1 +
 include/linux/resctrl_types.h      |  1 +
 fs/resctrl/internal.h              |  2 ++
 arch/x86/kernel/cpu/resctrl/core.c | 11 +++++++++++
 fs/resctrl/rdtgroup.c              |  2 ++
 5 files changed, 17 insertions(+)
include/linux/resctrl.h +1 -0
index 25c3ee78de3d..6c895ec220fe 100644
@@ -233,6 +233,7 @@ enum resctrl_scope {
 	RESCTRL_L2_CACHE = 2,
 	RESCTRL_L3_CACHE = 3,
 	RESCTRL_L3_NODE,
+	RESCTRL_PACKAGE,
 };
 
Quoted 14 lines

include/linux/resctrl_types.h enum resctrl_res_level { · line 42

Quoted 10 lines
include/linux/resctrl.h +0 -0
 /**
include/linux/resctrl_types.h +1 -0
index a7faf2cd5406..8f967e03af5a 100644
@@ -39,6 +39,7 @@ enum resctrl_res_level {
 	RDT_RESOURCE_L2,
 	RDT_RESOURCE_MBA,
 	RDT_RESOURCE_SMBA,
+	RDT_RESOURCE_INTEL_AET,

This is fs code so architecture specific code needs to be avoided. Any other architecture that may need to report similar data would be forced to use this resource name.

Maybe RDT_RESOURCE_PERF?

Quoted 9 lines
Quoted 9 lines
include/linux/resctrl_types.h +0 -0
 	/* Must be the last */
 	RDT_NUM_RESOURCES,
fs/resctrl/internal.h +0 -0
index ec3863d18f68..3a100007301d 100644
@@ -240,6 +240,8 @@ struct rdtgroup {
 
Quoted 5 lines

fs/resctrl/internal.h struct rdtgroup { · line 243

Quoted 3 lines
fs/resctrl/internal.h +1 -0
 #define RFTYPE_DEBUG			BIT(10)
 
+#define RFTYPE_RES_PKG			BIT(11)

"PKG" is not the resource but instead the scope, no?

Correct. I missed this one when doing some renaming. It should get a "perf" name (to match what we pick for the RDT_RESOURCE)

Quoted 18 lines
Quoted 9 lines
fs/resctrl/internal.h +1 -0
+
 #define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
 
 #define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
arch/x86/kernel/cpu/resctrl/core.c +0 -0
index c316268b4442..c8cc3104f56c 100644

arch/x86/kernel/cpu/resctrl/core.c struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCES] = { · line 106

Quoted 8 lines
arch/x86/kernel/cpu/resctrl/core.c +5 -0
@@ -99,6 +99,15 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCES] = {
 			.schema_fmt		= RESCTRL_SCHEMA_RANGE,
 		},
 	},
+	[RDT_RESOURCE_INTEL_AET] =
+	{
+		.r_resctrl = {
+			.rid			= RDT_RESOURCE_INTEL_AET,
+			.name			= "PKG",
Quoted 7 lines

The resource name should not be the scope. It should be a name that reflects the resource being monitored. In this case it is "core"/"cpu"? I understand that this may create confusion since the data is provided at package scope so perhaps the "resource" can be "perf" and then all the events can include "core" in their name? If the intention is to guide user space in how to interpret the domain IDs then we could consider something like "perf_pkg" or even "phys_pkg_perf" that prepares resctrl for future perf events that may need reporting at a different scope?

"perf_pkg" looks like a good option. Should it be "PERF_PKG". It appears to the user in the names of the "mon_data/mon_%s_%.2d" directories. Also in info/%s_MON.

Resctrl uses "L3" in captitals in those places.

Quoted 32 lines
Quoted 21 lines
arch/x86/kernel/cpu/resctrl/core.c +6 -0
+			.mon_scope		= RESCTRL_PACKAGE,
+			.mon_domains		= mon_domain_init(RDT_RESOURCE_INTEL_AET),
+		},
+	},
 };
 
 u32 resctrl_arch_system_num_rmid_idx(void)
@@ -430,6 +439,8 @@ static int get_domain_id_from_scope(int cpu, enum resctrl_scope scope)
 		return get_cpu_cacheinfo_id(cpu, scope);
 	case RESCTRL_L3_NODE:
 		return cpu_to_node(cpu);
+	case RESCTRL_PACKAGE:
+		return topology_physical_package_id(cpu);
 	default:
 		break;
 	}
fs/resctrl/rdtgroup.c +0 -0
index dbfb7d3bc3bc..a90291f57330 100644

fs/resctrl/rdtgroup.c static unsigned long fflags_from_resource(struct rdt_resource *r) · line 2186

Quoted 8 lines
fs/resctrl/rdtgroup.c +2 -0
@@ -2179,6 +2179,8 @@ static unsigned long fflags_from_resource(struct rdt_resource *r)
 	case RDT_RESOURCE_MBA:
 	case RDT_RESOURCE_SMBA:
 		return RFTYPE_RES_MB;
+	case RDT_RESOURCE_INTEL_AET:
+		return RFTYPE_RES_PKG;
 	}
 
 	return WARN_ON_ONCE(1);

Reinette

maintainer
#12 lore raw

Re: [PATCH v2 05/16] x86/resctrl: Add and initialize rdt_resource for package scope core monitor

Hi Tony,

Quoted 47 lines · Tony
Quoted 7 lines · Reinette Chatre

Hi Tony,

Quoted 1 line · Tony Luck

New resource for monitoring core events reported at package level.

Could you please add proper (per Documentation/process/maintainer-tip.rst) changelogs to all patches in the series?

Will do.

Quoted 21 lines
Quoted 21 lines
---
 include/linux/resctrl.h            |  1 +
 include/linux/resctrl_types.h      |  1 +
 fs/resctrl/internal.h              |  2 ++
 arch/x86/kernel/cpu/resctrl/core.c | 11 +++++++++++
 fs/resctrl/rdtgroup.c              |  2 ++
 5 files changed, 17 insertions(+)
include/linux/resctrl.h +1 -0
index 25c3ee78de3d..6c895ec220fe 100644
@@ -233,6 +233,7 @@ enum resctrl_scope {
 	RESCTRL_L2_CACHE = 2,
 	RESCTRL_L3_CACHE = 3,
 	RESCTRL_L3_NODE,
+	RESCTRL_PACKAGE,
 };
 
Quoted 14 lines

include/linux/resctrl_types.h enum resctrl_res_level { · line 42

Quoted 10 lines
include/linux/resctrl.h +0 -0
 /**
include/linux/resctrl_types.h +1 -0
index a7faf2cd5406..8f967e03af5a 100644
@@ -39,6 +39,7 @@ enum resctrl_res_level {
 	RDT_RESOURCE_L2,
 	RDT_RESOURCE_MBA,
 	RDT_RESOURCE_SMBA,
+	RDT_RESOURCE_INTEL_AET,

This is fs code so architecture specific code needs to be avoided. Any other architecture that may need to report similar data would be forced to use this resource name.

Quoted 2 lines

Maybe RDT_RESOURCE_PERF?

Would be ideal if it matches the name of the resource (below).

Quoted 44 lines
Quoted 9 lines
Quoted 9 lines
include/linux/resctrl_types.h +0 -0
 	/* Must be the last */
 	RDT_NUM_RESOURCES,
fs/resctrl/internal.h +0 -0
index ec3863d18f68..3a100007301d 100644
@@ -240,6 +240,8 @@ struct rdtgroup {
 
Quoted 5 lines

fs/resctrl/internal.h struct rdtgroup { · line 243

Quoted 3 lines
fs/resctrl/internal.h +1 -0
 #define RFTYPE_DEBUG			BIT(10)
 
+#define RFTYPE_RES_PKG			BIT(11)

"PKG" is not the resource but instead the scope, no?

Correct. I missed this one when doing some renaming. It should get a "perf" name (to match what we pick for the RDT_RESOURCE)

Quoted 18 lines
Quoted 9 lines
fs/resctrl/internal.h +1 -0
+
 #define RFTYPE_CTRL_INFO		(RFTYPE_INFO | RFTYPE_CTRL)
 
 #define RFTYPE_MON_INFO			(RFTYPE_INFO | RFTYPE_MON)
arch/x86/kernel/cpu/resctrl/core.c +0 -0
index c316268b4442..c8cc3104f56c 100644

arch/x86/kernel/cpu/resctrl/core.c struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCES] = { · line 106

Quoted 8 lines
arch/x86/kernel/cpu/resctrl/core.c +5 -0
@@ -99,6 +99,15 @@ struct rdt_hw_resource rdt_resources_all[RDT_NUM_RESOURCES] = {
 			.schema_fmt		= RESCTRL_SCHEMA_RANGE,
 		},
 	},
+	[RDT_RESOURCE_INTEL_AET] =
+	{
+		.r_resctrl = {
+			.rid			= RDT_RESOURCE_INTEL_AET,
+			.name			= "PKG",
Quoted 7 lines

The resource name should not be the scope. It should be a name that reflects the resource being monitored. In this case it is "core"/"cpu"? I understand that this may create confusion since the data is provided at package scope so perhaps the "resource" can be "perf" and then all the events can include "core" in their name? If the intention is to guide user space in how to interpret the domain IDs then we could consider something like "perf_pkg" or even "phys_pkg_perf" that prepares resctrl for future perf events that may need reporting at a different scope?

Quoted 5 lines

"perf_pkg" looks like a good option. Should it be "PERF_PKG". It appears to the user in the names of the "mon_data/mon_%s_%.2d" directories. Also in info/%s_MON.

Resctrl uses "L3" in captitals in those places.

I do not know where the capitalization custom originated but PERF_PKG does seem like a good start.

Reinette

author maintainer
#13 lore raw

PATCH v2 6/16 x86/resctrl: Prepare for resource specific event ids

In reply to Tony Luck

Event ids ought to be specific to each resource. Keep enum resctrl_event_id for event ids in the RDT_RESOURCE_L3. But change the type in generic areas to unsigned int so that new resources can have their own event ids.

---
 fs/resctrl/internal.h | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
fs/resctrl/internal.h +5 -5
index 3a100007301d..422f36573db7 100644
@@ -69,13 +69,13 @@ static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc)
 
 /**
  * struct mon_evt - Entry in the event list of a resource
- * @evtid:		event id
+ * @evtid:		per resource event id
  * @name:		name of the event
  * @configurable:	true if the event is configurable
  * @list:		entry in &rdt_resource->evt_list
  */
 struct mon_evt {
-	enum resctrl_event_id	evtid;
+	unsigned int		evtid;
 	char			*name;
 	bool			configurable;
 	struct list_head	list;
@@ -84,7 +84,7 @@ struct mon_evt {
 /**
  * struct mon_data - Monitoring details for each event file.
  * @rid:             Resource id associated with the event file.
- * @evtid:           Event id associated with the event file.
+ * @evtid:           Per resource event id associated with the event file.
  * @sum:             Set when event must be summed across multiple
  *                   domains.
  * @domid:           When @sum is zero this is the domain to which
@@ -97,7 +97,7 @@ struct mon_evt {
  */
 struct mon_data {
 	unsigned int rid;
-	enum resctrl_event_id evtid;
+	unsigned int evtid;
 	unsigned int sum;
 	unsigned int domid;
 };
@@ -124,7 +124,7 @@ struct rmid_read {
 	struct rdtgroup		*rgrp;
 	struct rdt_resource	*r;
 	struct rdt_mon_domain	*d;
-	enum resctrl_event_id	evtid;
+	unsigned int		evtid;
 	bool			first;
 	struct cacheinfo	*ci;
 	int			err;
Signature · 1 line
2.48.1
author maintainer
#14 lore raw

PATCH v2 7/16 x86/resctrl: Add initialization hook for Intel PMT events

In reply to Tony Luck

Call the OOBMSM discovery code to find out if there are any event groups that match unique identifiers understood by resctrl.

Note that initiialzation must happen in two phases because the OOBMSM VSEC discovery process is not complete at resctrl "lateinit()" initialization time. So there is an initial hook that assumes that Intel PMT will exist, called early so that package scoped domain groups are initialized.

At first mount the remainder of initialization is done. If there are no Intel PMT events, the package domain lists are removed.

Move definition of struct mon_evt to <linux/resctrl_types.h>

Events for specific systems to be added by a separate patch.

---
 include/linux/resctrl.h                 |  14 ++
 include/linux/resctrl_types.h           |  14 ++
 arch/x86/kernel/cpu/resctrl/internal.h  |   6 +
 fs/resctrl/internal.h                   |  14 --
 arch/x86/kernel/cpu/resctrl/core.c      |   9 +-
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 211 ++++++++++++++++++++++++
 fs/resctrl/rdtgroup.c                   |   3 +
 arch/x86/kernel/cpu/resctrl/Makefile    |   1 +
 8 files changed, 255 insertions(+), 17 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/resctrl/intel_aet.c
include/linux/resctrl.h +14 -0
index 6c895ec220fe..999e0802a26e 100644
@@ -170,6 +170,14 @@ struct rdt_mon_domain {
 	int				cqm_work_cpu;
 };
 
+/**
+ * struct rdt_core_mon_domain - CPUs sharing an Intel-PMT-scoped resctrl monitor resource
+ * @hdr:		common header for different domain types
+ */
+struct rdt_core_mon_domain {
+	struct rdt_domain_hdr	hdr;
+};
+
 /**
  * struct resctrl_cache - Cache allocation related data
  * @cbm_len:		Length of the cache bit mask
@@ -522,4 +530,10 @@ extern unsigned int resctrl_rmid_realloc_limit;
 int resctrl_init(void);
 void resctrl_exit(void);
 
+#ifdef CONFIG_INTEL_AET_RESCTRL
+void rdt_get_intel_aet_mount(void);
+#else
+static inline void rdt_get_intel_aet_mount(void) { }
+#endif
+
 #endif /* _RESCTRL_H */
include/linux/resctrl_types.h +14 -0
index 8f967e03af5a..d56fcd082d2c 100644
@@ -57,4 +57,18 @@ enum resctrl_event_id {
 
 #define QOS_NUM_EVENTS		(QOS_L3_MBM_LOCAL_EVENT_ID + 1)
 
+/**
+ * struct mon_evt - Entry in the event list of a resource
+ * @evtid:		per resource event id
+ * @name:		name of the event
+ * @configurable:	true if the event is configurable
+ * @list:		entry in &rdt_resource->evt_list
+ */
+struct mon_evt {
+	unsigned int		evtid;
+	char			*name;
+	bool			configurable;
+	struct list_head	list;
+};
+
 #endif /* __LINUX_RESCTRL_TYPES_H */
arch/x86/kernel/cpu/resctrl/internal.h +6 -0
index 521db28efb3f..ada402c7678b 100644
@@ -168,6 +168,12 @@ void rdt_ctrl_update(void *arg);
 
 int rdt_get_mon_l3_config(struct rdt_resource *r);
 
+#ifdef CONFIG_INTEL_AET_RESCTRL
+int rdt_get_intel_aet_mon_config(void);
+#else
+static inline int rdt_get_intel_aet_mon_config(void) { return 0; }
+#endif
+
 bool rdt_cpu_has(int flag);
 
 void __init intel_rdt_mbm_apply_quirk(void);
fs/resctrl/internal.h +0 -14
index 422f36573db7..f5a698b49e97 100644
@@ -67,20 +67,6 @@ static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc)
 	return container_of(kfc, struct rdt_fs_context, kfc);
 }
 
-/**
- * struct mon_evt - Entry in the event list of a resource
- * @evtid:		per resource event id
- * @name:		name of the event
- * @configurable:	true if the event is configurable
- * @list:		entry in &rdt_resource->evt_list
- */
-struct mon_evt {
-	unsigned int		evtid;
-	char			*name;
-	bool			configurable;
-	struct list_head	list;
-};
-
 /**
  * struct mon_data - Monitoring details for each event file.
  * @rid:             Resource id associated with the event file.
arch/x86/kernel/cpu/resctrl/core.c +6 -3
index c8cc3104f56c..1ab0f5eec244 100644
@@ -882,6 +882,7 @@ static __init bool get_rdt_alloc_resources(void)
 static __init bool get_rdt_mon_resources(void)
 {
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
+	int ret1 = -EINVAL, ret2;
 
 	if (rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC))
 		rdt_mon_features |= (1 << QOS_L3_OCCUP_EVENT_ID);
@@ -890,10 +891,12 @@ static __init bool get_rdt_mon_resources(void)
 	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL))
 		rdt_mon_features |= (1 << QOS_L3_MBM_LOCAL_EVENT_ID);
 
-	if (!rdt_mon_features)
-		return false;
+	if (rdt_mon_features)
+		ret1 = rdt_get_mon_l3_config(r);
+
+	ret2 = rdt_get_intel_aet_mon_config();
 
-	return !rdt_get_mon_l3_config(r);
+	return ret1 == 0 || ret2;
 }
 
 static __init void __check_quirks_intel(void)
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +211 -0

211 lines not on this page · raw message

fs/resctrl/rdtgroup.c +3 -0
index a90291f57330..4833dfa08ce3 100644
@@ -2572,6 +2572,9 @@ static int rdt_get_tree(struct fs_context *fc)
 
 	cpus_read_lock();
 	mutex_lock(&rdtgroup_mutex);
+
+	rdt_get_intel_aet_mount();
+
 	/*
 	 * resctrl file system can only be mounted once.
 	 */
arch/x86/kernel/cpu/resctrl/Makefile +1 -0
index 2c3b09f95127..a47e1c214087 100644
@@ -2,6 +2,7 @@
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= core.o rdtgroup.o monitor.o
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= ctrlmondata.o
 obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= pseudo_lock.o
+obj-$(CONFIG_INTEL_AET_RESCTRL)		+= intel_aet.o
 obj-$(CONFIG_INTEL_AET_RESCTRL)		+= fake_intel_aet_features.o
 
 # To allow define_trace.h's recursive include:
Signature · 1 line
2.48.1
maintainer
#15 lore raw

Re: [PATCH v2 07/16] x86/resctrl: Add initialization hook for Intel PMT events

Hi Tony,

Quoted 4 lines · Tony Luck

Call the OOBMSM discovery code to find out if there are any event groups that match unique identifiers understood by resctrl.

Note that initiialzation must happen in two phases because the

"initiialzation" -> "initialization"

Quoted 9 lines

OOBMSM VSEC discovery process is not complete at resctrl "lateinit()" initialization time. So there is an initial hook that assumes that Intel PMT will exist, called early so that package scoped domain groups are initialized.

At first mount the remainder of initialization is done. If there are no Intel PMT events, the package domain lists are removed.

Move definition of struct mon_evt to <linux/resctrl_types.h>

Why not include/resctrl.h?

Quoted 21 lines

Events for specific systems to be added by a separate patch.

---
 include/linux/resctrl.h                 |  14 ++
 include/linux/resctrl_types.h           |  14 ++
 arch/x86/kernel/cpu/resctrl/internal.h  |   6 +
 fs/resctrl/internal.h                   |  14 --
 arch/x86/kernel/cpu/resctrl/core.c      |   9 +-
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 211 ++++++++++++++++++++++++
 fs/resctrl/rdtgroup.c                   |   3 +
 arch/x86/kernel/cpu/resctrl/Makefile    |   1 +
 8 files changed, 255 insertions(+), 17 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/resctrl/intel_aet.c
include/linux/resctrl.h +0 -0
index 6c895ec220fe..999e0802a26e 100644

include/linux/resctrl.h struct rdt_mon_domain { · line 174

Quoted 5 lines
include/linux/resctrl.h +2 -0
@@ -170,6 +170,14 @@ struct rdt_mon_domain {
 	int				cqm_work_cpu;
 };
 
+/**
+ * struct rdt_core_mon_domain - CPUs sharing an Intel-PMT-scoped resctrl monitor resource

Please no arch specific references in fs code. I think it will help to explain and consume this work if patches are split into fs and arch patches.

Quoted 10 lines
include/linux/resctrl.h +6 -0
+ * @hdr:		common header for different domain types
+ */
+struct rdt_core_mon_domain {
+	struct rdt_domain_hdr	hdr;
+};
+
 /**
  * struct resctrl_cache - Cache allocation related data
  * @cbm_len:		Length of the cache bit mask

include/linux/resctrl.h extern unsigned int resctrl_rmid_realloc_limit; · line 534

Quoted 5 lines
include/linux/resctrl.h +2 -0
@@ -522,4 +530,10 @@ extern unsigned int resctrl_rmid_realloc_limit;
 int resctrl_init(void);
 void resctrl_exit(void);
 
+#ifdef CONFIG_INTEL_AET_RESCTRL
+void rdt_get_intel_aet_mount(void);

Please no arch specific helpers in fs code. This could instead be a generic "resctrl_arch_*" helper that resctrl fs calls at beginning of fs mount for architectures to do what is needed.

Quoted 133 lines
include/linux/resctrl.h +4 -0
+#else
+static inline void rdt_get_intel_aet_mount(void) { }
+#endif
+
 #endif /* _RESCTRL_H */
include/linux/resctrl_types.h +14 -0
index 8f967e03af5a..d56fcd082d2c 100644
@@ -57,4 +57,18 @@ enum resctrl_event_id {
 
 #define QOS_NUM_EVENTS		(QOS_L3_MBM_LOCAL_EVENT_ID + 1)
 
+/**
+ * struct mon_evt - Entry in the event list of a resource
+ * @evtid:		per resource event id
+ * @name:		name of the event
+ * @configurable:	true if the event is configurable
+ * @list:		entry in &rdt_resource->evt_list
+ */
+struct mon_evt {
+	unsigned int		evtid;
+	char			*name;
+	bool			configurable;
+	struct list_head	list;
+};
+
 #endif /* __LINUX_RESCTRL_TYPES_H */
arch/x86/kernel/cpu/resctrl/internal.h +6 -0
index 521db28efb3f..ada402c7678b 100644
@@ -168,6 +168,12 @@ void rdt_ctrl_update(void *arg);
 
 int rdt_get_mon_l3_config(struct rdt_resource *r);
 
+#ifdef CONFIG_INTEL_AET_RESCTRL
+int rdt_get_intel_aet_mon_config(void);
+#else
+static inline int rdt_get_intel_aet_mon_config(void) { return 0; }
+#endif
+
 bool rdt_cpu_has(int flag);
 
 void __init intel_rdt_mbm_apply_quirk(void);
fs/resctrl/internal.h +0 -14
index 422f36573db7..f5a698b49e97 100644
@@ -67,20 +67,6 @@ static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc)
 	return container_of(kfc, struct rdt_fs_context, kfc);
 }
 
-/**
- * struct mon_evt - Entry in the event list of a resource
- * @evtid:		per resource event id
- * @name:		name of the event
- * @configurable:	true if the event is configurable
- * @list:		entry in &rdt_resource->evt_list
- */
-struct mon_evt {
-	unsigned int		evtid;
-	char			*name;
-	bool			configurable;
-	struct list_head	list;
-};
-
 /**
  * struct mon_data - Monitoring details for each event file.
  * @rid:             Resource id associated with the event file.
arch/x86/kernel/cpu/resctrl/core.c +6 -3
index c8cc3104f56c..1ab0f5eec244 100644
@@ -882,6 +882,7 @@ static __init bool get_rdt_alloc_resources(void)
 static __init bool get_rdt_mon_resources(void)
 {
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
+	int ret1 = -EINVAL, ret2;
 
 	if (rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC))
 		rdt_mon_features |= (1 << QOS_L3_OCCUP_EVENT_ID);
@@ -890,10 +891,12 @@ static __init bool get_rdt_mon_resources(void)
 	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL))
 		rdt_mon_features |= (1 << QOS_L3_MBM_LOCAL_EVENT_ID);
 
-	if (!rdt_mon_features)
-		return false;
+	if (rdt_mon_features)
+		ret1 = rdt_get_mon_l3_config(r);
+
+	ret2 = rdt_get_intel_aet_mon_config();
 
-	return !rdt_get_mon_l3_config(r);
+	return ret1 == 0 || ret2;
 }
 
 static __init void __check_quirks_intel(void)
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +29 -0
new file mode 100644
index 000000000000..9a8ccb62b4ab
@@ -0,0 +1,211 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Resource Director Technology(RDT)
+ * - Intel Application Energy Telemetry
+ *
+ * Copyright (C) 2025 Intel Corporation
+ *
+ * Author:
+ *    Tony Luck <[email protected]>
+ */
+
+#define pr_fmt(fmt)   "resctrl: " fmt
+
+#include <linux/cpu.h>
+#include <linux/cleanup.h>
+#include <linux/slab.h>
+#include "fake_intel_aet_features.h"
+#include <linux/intel_vsec.h>
+#include <asm/resctrl.h>
+
+#include "internal.h"
+
+static struct pmt_feature_group *feat_energy;
+static struct pmt_feature_group *feat_perf;
+
+/* All telemetry events from Intel CPUs */
+enum pmt_event_id {
+	PMT_EVENT_ENERGY,
+	PMT_EVENT_ACTIVITY,

arch/x86/kernel/cpu/resctrl/intel_aet.c line 39

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +10 -0
+	PMT_EVENT_STALLS_LLC_HIT,
+	PMT_EVENT_C1_RES,
+	PMT_EVENT_UNHALTED_CORE_CYCLES,
+	PMT_EVENT_STALLS_LLC_MISS,
+	PMT_EVENT_AUTO_C6_RES,
+	PMT_EVENT_UNHALTED_REF_CYCLES,
+	PMT_EVENT_UOPS_RETIRED,
+
+	PMT_NUM_EVENTS
+};

I expect the above to become part of resctrl fs. Actually, I think all properties of the new events, the id, name and how to display it, should be part of resctrl fs.

We do not want other architectures to create their own display names for the same events. I expect that this will require more plumbing between arch and fs code to communicate which events are supported, similar to what exists for the L3 events (for example, resctrl_arch_is_mbm_total_enabled()).

This may result in struct rdt_core_mon_domain to no longer be empty but instead for resctrl to use it to manage state of which events are enabled or not. Theoretically all could be managed by the architecture but I think that could result in inconsistent error codes to user space.

Quoted 25 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +25 -0
+
+/**
+ * enum evt_type - Type for values for each event.
+ * @EVT_U64:	Integer up to 64 bits
+ * @EVT_U46_18:	Fixed point binary, 46 bits for integer, 18 binary place bits
+ */
+enum evt_type {
+	EVT_U64,
+	EVT_U46_18,
+};
+
+#define EVT(id, evtname, offset, t)		\
+	{					\
+		.evt = {			\
+			.evtid = id,		\
+			.name = evtname		\
+		},				\
+		.evt_offset = offset,		\
+		.evt_type = t			\
+	}
+
+/**
+ * struct pmt_event - Telemetry event.
+ * @evt:	Resctrl event
+ * @evt_offset:	MMIO offset of counter

arch/x86/kernel/cpu/resctrl/intel_aet.c line 74

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +10 -0
+ * @evt_type:	Type
+ */
+struct pmt_event {
+	struct mon_evt	evt;
+	int		evt_offset;
+	enum evt_type	evt_type;
+};
+
+/**
+ * struct telem_entry - Summarized form from XML telemetry description

It is not clear to me how useful it is to document that this is "Summarized form from XML telemetry description". Either more detail should be added to help reader understand what XML is being talked about or the description should be a summary of what this data structure represents.

arch/x86/kernel/cpu/resctrl/intel_aet.c line 79

Quoted 5 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +5 -0
+ * @name:			Name for this group of events
+ * @guid:			Unique ID for this group
+ * @num_rmids:			Number of RMIDS supported
+ * @stride:			Number of bytes of counters for each RMID
+ * @overflow_counter_off:	Offset od overflow count

od -> of

arch/x86/kernel/cpu/resctrl/intel_aet.c line 82

Quoted 3 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +3 -0
+ * @last_overflow_tstamp_off:	Offset of overflow timestamp
+ * @last_update_tstamp_off:	Offset of last update timestamp
+ * @active:			Marks this group as active on current CPU

Could you please elaborate what "on current CPU" implies with the events being "per package"?

arch/x86/kernel/cpu/resctrl/intel_aet.c line 83

Quoted 1 line
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +1 -0
+ * @evts:			Telemetry events in this group

Since this is an array, could this comment also describe how the number of entries are determined?

Quoted 16 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +16 -0
+ */
+struct telem_entry {
+	char	*name;
+	int	guid;
+	int	num_rmids;
+	int	stride;
+	int	overflow_counter_off;
+	int	last_overflow_tstamp_off;
+	int	last_update_tstamp_off;
+	bool	active;
+	struct pmt_event evts[];
+};
+
+/* All known telemetry event groups */
+static struct telem_entry *telem_entry[] = {
+	NULL

arch/x86/kernel/cpu/resctrl/intel_aet.c line 109

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +10 -0
+};
+
+/* Per-package event groups active on this machine */
+static struct pkg_info {
+	int			count;
+	struct telemetry_region	*regions;
+} *pkg_info;
+
+/*
+ * Scan a feature group looking for guids recognized by this code

"this code" can be dropped

Quoted 9 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +9 -0
+ * and update the per-package counts of known groups.
+ */
+static bool count_events(struct pkg_info *pkg, int max_pkgs, struct pmt_feature_group *p)
+{
+	struct telem_entry **tentry;
+	bool found = false;
+
+	for (int i = 0; i < p->count; i++) {
+		struct telemetry_region *tr = &p->regions[i];

arch/x86/kernel/cpu/resctrl/intel_aet.c line 128

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +10 -0
+
+		for (tentry = telem_entry; *tentry; tentry++) {
+			if (tr->guid == (*tentry)->guid) {
+				if (tr->plat_info.package_id > max_pkgs) {
+					pr_warn_once("Bad package %d\n", tr->plat_info.package_id);
+					continue;
+				}
+				found = true;
+				(*tentry)->active = true;
+				pkg[tr->plat_info.package_id].count++;

There seems to be some duplicate information between the structures. For example, telem_entry::num_rmids and the num_rmids from the pmt_feature_group. Are these expected to be identical? Since the num_rmids from telem_entry are hardcoded and the ones from pmt_feature_group discovered then this sounds like opportunity to add some sanity checking.
Similarly, could there be a check here to ensure that the size of the memory region provided matches the expected size based on all the hardcoded properties?

Quoted 14 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +14 -0
+				break;
+			}
+		}
+	}
+
+	return found;
+}
+
+DEFINE_FREE(intel_pmt_put_feature_group, struct pmt_feature_group *,	\
+	if (!IS_ERR_OR_NULL(_T))					\
+		intel_pmt_put_feature_group(_T))
+
+/*
+ * Ask OOBMSM discovery driver for all the RMID based telemetry groups

arch/x86/kernel/cpu/resctrl/intel_aet.c line 152

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +10 -0
+ * that it supports.
+ */
+static bool get_events(void)
+{
+	struct pmt_feature_group *p1 __free(intel_pmt_put_feature_group) = NULL;
+	struct pmt_feature_group *p2 __free(intel_pmt_put_feature_group) = NULL;
+	int num_pkgs = topology_max_packages();
+	struct pkg_info *pkg __free(kfree) = NULL;
+
+	pkg = kmalloc_array(num_pkgs, sizeof(*pkg_info), GFP_KERNEL | __GFP_ZERO);

kcalloc() ?

arch/x86/kernel/cpu/resctrl/intel_aet.c line 164

Quoted 12 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +12 -0
+	if (!pkg)
+		return false;
+
+	p1 = intel_pmt_get_regions_by_feature(FEATURE_PER_RMID_ENERGY_TELEM);
+	p2 = intel_pmt_get_regions_by_feature(FEATURE_PER_RMID_PERF_TELEM);
+
+	if (IS_ERR_VALUE(p1) && IS_ERR_VALUE(p1))
+		return false;
+
+	if (!IS_ERR_VALUE(p1))
+		if (!count_events(pkg, num_pkgs, p1))
+			intel_pmt_put_feature_group(no_free_ptr(p1));

This seems to defeat the purpose of the cleanup handler.

Quoted 66 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +47 -0
+	if (!IS_ERR_VALUE(p2))
+		if (!count_events(pkg, num_pkgs, p2))
+			intel_pmt_put_feature_group(no_free_ptr(p2));
+
+	if (!IS_ERR_OR_NULL(p1))
+		feat_energy = no_free_ptr(p1);
+	if (!IS_ERR_OR_NULL(p2))
+		feat_perf = no_free_ptr(p2);
+	pkg_info = no_free_ptr(pkg);
+
+	return true;
+}
+
+/*
+ * Early initialization. Cannot do much here because OOBMSM has not
+ * completed enumeration of telemetry event groups.
+ */
+int rdt_get_intel_aet_mon_config(void)
+{
+	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_INTEL_AET].r_resctrl;
+
+	INIT_LIST_HEAD(&r->evt_list);
+
+	return 1;
+}
+
+/*
+ * Late (first mount) initialization. Safe to ask OOBMSM which telemetry
+ * event groups are supported.
+ */
+void rdt_get_intel_aet_mount(void)
+{
+	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_INTEL_AET].r_resctrl;
+	struct rdt_core_mon_domain *d, *tmp;
+	static int do_one_time;
+
+	if (do_one_time)
+		return;
+
+	do_one_time = 1;
+
+	if (!get_events()) {
+		list_for_each_entry_safe(d, tmp, &r->mon_domains, hdr.list)
+			kfree(d);
+		r->mon_capable = false;
+	}
+}
fs/resctrl/rdtgroup.c +3 -0
index a90291f57330..4833dfa08ce3 100644
@@ -2572,6 +2572,9 @@ static int rdt_get_tree(struct fs_context *fc)
 
 	cpus_read_lock();
 	mutex_lock(&rdtgroup_mutex);
+
+	rdt_get_intel_aet_mount();
+
 	/*
 	 * resctrl file system can only be mounted once.
 	 */
arch/x86/kernel/cpu/resctrl/Makefile +0 -0
index 2c3b09f95127..a47e1c214087 100644

arch/x86/kernel/cpu/resctrl/Makefile line 8

Quoted 7 lines
arch/x86/kernel/cpu/resctrl/Makefile +1 -0
@@ -2,6 +2,7 @@
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= core.o rdtgroup.o monitor.o
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= ctrlmondata.o
 obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= pseudo_lock.o
+obj-$(CONFIG_INTEL_AET_RESCTRL)		+= intel_aet.o
 obj-$(CONFIG_INTEL_AET_RESCTRL)		+= fake_intel_aet_features.o
 
 # To allow define_trace.h's recursive include:

Reinette

author maintainer
#16 lore raw

Re: [PATCH v2 07/16] x86/resctrl: Add initialization hook for Intel PMT events

Quoted 9 lines · Reinette Chatre

Hi Tony,

Quoted 4 lines · Tony Luck

Call the OOBMSM discovery code to find out if there are any event groups that match unique identifiers understood by resctrl.

Note that initiialzation must happen in two phases because the

"initiialzation" -> "initialization"

Wil fix.

Quoted 11 lines
Quoted 9 lines

OOBMSM VSEC discovery process is not complete at resctrl "lateinit()" initialization time. So there is an initial hook that assumes that Intel PMT will exist, called early so that package scoped domain groups are initialized.

At first mount the remainder of initialization is done. If there are no Intel PMT events, the package domain lists are removed.

Move definition of struct mon_evt to <linux/resctrl_types.h>

Why not include/resctrl.h?

I put in in resctrl_types.h because it is a type definition? I'm not sure of the criteria James used to decide what goes in resctrl_types.h and what goes in resctrl.h

Quoted 24 lines
Quoted 24 lines

Events for specific systems to be added by a separate patch.

---
 include/linux/resctrl.h                 |  14 ++
 include/linux/resctrl_types.h           |  14 ++
 arch/x86/kernel/cpu/resctrl/internal.h  |   6 +
 fs/resctrl/internal.h                   |  14 --
 arch/x86/kernel/cpu/resctrl/core.c      |   9 +-
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 211 ++++++++++++++++++++++++
 fs/resctrl/rdtgroup.c                   |   3 +
 arch/x86/kernel/cpu/resctrl/Makefile    |   1 +
 8 files changed, 255 insertions(+), 17 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/resctrl/intel_aet.c
include/linux/resctrl.h +0 -0
index 6c895ec220fe..999e0802a26e 100644
@@ -170,6 +170,14 @@ struct rdt_mon_domain {
 	int				cqm_work_cpu;
 };
 
Quoted 5 lines

include/linux/resctrl.h struct rdt_mon_domain { · line 174

Quoted 2 lines
include/linux/resctrl.h +2 -0
+/**
+ * struct rdt_core_mon_domain - CPUs sharing an Intel-PMT-scoped resctrl monitor resource

Please no arch specific references in fs code. I think it will help to explain and consume this work if patches are split into fs and arch patches.

Domain add/remove needs more thought on the hooks between FS and ARCH layers. Splitting the patches into "new infrastructure for FS" and "new ARCH implememtations" will help me get something cleaner.

Quoted 13 lines
Quoted 13 lines
include/linux/resctrl.h +6 -0
+ * @hdr:		common header for different domain types
+ */
+struct rdt_core_mon_domain {
+	struct rdt_domain_hdr	hdr;
+};
+
 /**
  * struct resctrl_cache - Cache allocation related data
  * @cbm_len:		Length of the cache bit mask
@@ -522,4 +530,10 @@ extern unsigned int resctrl_rmid_realloc_limit;
 int resctrl_init(void);
 void resctrl_exit(void);
 
Quoted 6 lines

include/linux/resctrl.h extern unsigned int resctrl_rmid_realloc_limit; · line 534

Quoted 2 lines
include/linux/resctrl.h +2 -0
+#ifdef CONFIG_INTEL_AET_RESCTRL
+void rdt_get_intel_aet_mount(void);

Please no arch specific helpers in fs code. This could instead be a generic "resctrl_arch_*" helper that resctrl fs calls at beginning of fs mount for architectures to do what is needed.

Agreed.

Quoted 140 lines
Quoted 140 lines
include/linux/resctrl.h +4 -0
+#else
+static inline void rdt_get_intel_aet_mount(void) { }
+#endif
+
 #endif /* _RESCTRL_H */
include/linux/resctrl_types.h +14 -0
index 8f967e03af5a..d56fcd082d2c 100644
@@ -57,4 +57,18 @@ enum resctrl_event_id {
 
 #define QOS_NUM_EVENTS		(QOS_L3_MBM_LOCAL_EVENT_ID + 1)
 
+/**
+ * struct mon_evt - Entry in the event list of a resource
+ * @evtid:		per resource event id
+ * @name:		name of the event
+ * @configurable:	true if the event is configurable
+ * @list:		entry in &rdt_resource->evt_list
+ */
+struct mon_evt {
+	unsigned int		evtid;
+	char			*name;
+	bool			configurable;
+	struct list_head	list;
+};
+
 #endif /* __LINUX_RESCTRL_TYPES_H */
arch/x86/kernel/cpu/resctrl/internal.h +6 -0
index 521db28efb3f..ada402c7678b 100644
@@ -168,6 +168,12 @@ void rdt_ctrl_update(void *arg);
 
 int rdt_get_mon_l3_config(struct rdt_resource *r);
 
+#ifdef CONFIG_INTEL_AET_RESCTRL
+int rdt_get_intel_aet_mon_config(void);
+#else
+static inline int rdt_get_intel_aet_mon_config(void) { return 0; }
+#endif
+
 bool rdt_cpu_has(int flag);
 
 void __init intel_rdt_mbm_apply_quirk(void);
fs/resctrl/internal.h +0 -14
index 422f36573db7..f5a698b49e97 100644
@@ -67,20 +67,6 @@ static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc)
 	return container_of(kfc, struct rdt_fs_context, kfc);
 }
 
-/**
- * struct mon_evt - Entry in the event list of a resource
- * @evtid:		per resource event id
- * @name:		name of the event
- * @configurable:	true if the event is configurable
- * @list:		entry in &rdt_resource->evt_list
- */
-struct mon_evt {
-	unsigned int		evtid;
-	char			*name;
-	bool			configurable;
-	struct list_head	list;
-};
-
 /**
  * struct mon_data - Monitoring details for each event file.
  * @rid:             Resource id associated with the event file.
arch/x86/kernel/cpu/resctrl/core.c +6 -3
index c8cc3104f56c..1ab0f5eec244 100644
@@ -882,6 +882,7 @@ static __init bool get_rdt_alloc_resources(void)
 static __init bool get_rdt_mon_resources(void)
 {
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
+	int ret1 = -EINVAL, ret2;
 
 	if (rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC))
 		rdt_mon_features |= (1 << QOS_L3_OCCUP_EVENT_ID);
@@ -890,10 +891,12 @@ static __init bool get_rdt_mon_resources(void)
 	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL))
 		rdt_mon_features |= (1 << QOS_L3_MBM_LOCAL_EVENT_ID);
 
-	if (!rdt_mon_features)
-		return false;
+	if (rdt_mon_features)
+		ret1 = rdt_get_mon_l3_config(r);
+
+	ret2 = rdt_get_intel_aet_mon_config();
 
-	return !rdt_get_mon_l3_config(r);
+	return ret1 == 0 || ret2;
 }
 
 static __init void __check_quirks_intel(void)
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +35 -0
new file mode 100644
index 000000000000..9a8ccb62b4ab
@@ -0,0 +1,211 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Resource Director Technology(RDT)
+ * - Intel Application Energy Telemetry
+ *
+ * Copyright (C) 2025 Intel Corporation
+ *
+ * Author:
+ *    Tony Luck <[email protected]>
+ */
+
+#define pr_fmt(fmt)   "resctrl: " fmt
+
+#include <linux/cpu.h>
+#include <linux/cleanup.h>
+#include <linux/slab.h>
+#include "fake_intel_aet_features.h"
+#include <linux/intel_vsec.h>
+#include <asm/resctrl.h>
+
+#include "internal.h"
+
+static struct pmt_feature_group *feat_energy;
+static struct pmt_feature_group *feat_perf;
+
+/* All telemetry events from Intel CPUs */
+enum pmt_event_id {
+	PMT_EVENT_ENERGY,
+	PMT_EVENT_ACTIVITY,
+	PMT_EVENT_STALLS_LLC_HIT,
+	PMT_EVENT_C1_RES,
+	PMT_EVENT_UNHALTED_CORE_CYCLES,
+	PMT_EVENT_STALLS_LLC_MISS,
+	PMT_EVENT_AUTO_C6_RES,
+	PMT_EVENT_UNHALTED_REF_CYCLES,
Quoted 8 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c line 39

Quoted 4 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +4 -0
+	PMT_EVENT_UOPS_RETIRED,
+
+	PMT_NUM_EVENTS
+};

I expect the above to become part of resctrl fs. Actually, I think all properties of the new events, the id, name and how to display it, should be part of resctrl fs.

I'm not convinced about the amount of re-use that there will be for these events. I took a quick look at the current plan for a processor that follows Clearwater Forest and I see 22 events, only 3 of them match events in the above list.

Quoted 4 lines

We do not want other architectures to create their own display names for the same events. I expect that this will require more plumbing between arch and fs code to communicate which events are supported, similar to what exists for the L3 events (for example, resctrl_arch_is_mbm_total_enabled()).

Supported monitor events are indicated by setting bits in "rdt_mon_features" currently "unsigned int" but could become "long" or a longer bitmap if we really want the FS layer to keep track of every possible event for every architecture.

Quoted 5 lines

This may result in struct rdt_core_mon_domain to no longer be empty but instead for resctrl to use it to manage state of which events are enabled or not. Theoretically all could be managed by the architecture but I think that could result in inconsistent error codes to user space.

rdt_core_mon_domain seems like the wrong level (unless we expect to have different enabled events in different domains). rdt_resource seems plausible ... or just expand "rdt_mon_features".

Quoted 32 lines
Quoted 32 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +32 -0
+
+/**
+ * enum evt_type - Type for values for each event.
+ * @EVT_U64:	Integer up to 64 bits
+ * @EVT_U46_18:	Fixed point binary, 46 bits for integer, 18 binary place bits
+ */
+enum evt_type {
+	EVT_U64,
+	EVT_U46_18,
+};
+
+#define EVT(id, evtname, offset, t)		\
+	{					\
+		.evt = {			\
+			.evtid = id,		\
+			.name = evtname		\
+		},				\
+		.evt_offset = offset,		\
+		.evt_type = t			\
+	}
+
+/**
+ * struct pmt_event - Telemetry event.
+ * @evt:	Resctrl event
+ * @evt_offset:	MMIO offset of counter
+ * @evt_type:	Type
+ */
+struct pmt_event {
+	struct mon_evt	evt;
+	int		evt_offset;
+	enum evt_type	evt_type;
+};
Quoted 8 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c line 74

Quoted 3 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +3 -0
+
+/**
+ * struct telem_entry - Summarized form from XML telemetry description

It is not clear to me how useful it is to document that this is "Summarized form from XML telemetry description". Either more detail should be added to help reader understand what XML is being talked about or the description should be a summary of what this data structure represents.

More detail here is the right direction.

Quoted 6 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c line 79

Quoted 5 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +5 -0
+ * @name:			Name for this group of events
+ * @guid:			Unique ID for this group
+ * @num_rmids:			Number of RMIDS supported
+ * @stride:			Number of bytes of counters for each RMID
+ * @overflow_counter_off:	Offset od overflow count
Quoted 8 lines

od -> of

arch/x86/kernel/cpu/resctrl/intel_aet.c line 82

Quoted 3 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +3 -0
+ * @last_overflow_tstamp_off:	Offset of overflow timestamp
+ * @last_update_tstamp_off:	Offset of last update timestamp
+ * @active:			Marks this group as active on current CPU

Could you please elaborate what "on current CPU" implies with the events being "per package"?

I should have said "on current system" rather than using the overworked "CPU" word.

Quoted 4 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c line 83

Quoted 1 line
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +1 -0
+ * @evts:			Telemetry events in this group

Since this is an array, could this comment also describe how the number of entries are determined?

Will add comment that it is terminated with a null entry.

Quoted 21 lines
Quoted 21 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +20 -0
+ */
+struct telem_entry {
+	char	*name;
+	int	guid;
+	int	num_rmids;
+	int	stride;
+	int	overflow_counter_off;
+	int	last_overflow_tstamp_off;
+	int	last_update_tstamp_off;
+	bool	active;
+	struct pmt_event evts[];
+};
+
+/* All known telemetry event groups */
+static struct telem_entry *telem_entry[] = {
+	NULL
+};
+
+/* Per-package event groups active on this machine */
+static struct pkg_info {
Quoted 8 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c line 109

Quoted 6 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +6 -0
+	int			count;
+	struct telemetry_region	*regions;
+} *pkg_info;
+
+/*
+ * Scan a feature group looking for guids recognized by this code

"this code" can be dropped

Agreed.

Quoted 20 lines
Quoted 9 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +9 -0
+ * and update the per-package counts of known groups.
+ */
+static bool count_events(struct pkg_info *pkg, int max_pkgs, struct pmt_feature_group *p)
+{
+	struct telem_entry **tentry;
+	bool found = false;
+
+	for (int i = 0; i < p->count; i++) {
+		struct telemetry_region *tr = &p->regions[i];

arch/x86/kernel/cpu/resctrl/intel_aet.c line 128

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +10 -0
+
+		for (tentry = telem_entry; *tentry; tentry++) {
+			if (tr->guid == (*tentry)->guid) {
+				if (tr->plat_info.package_id > max_pkgs) {
+					pr_warn_once("Bad package %d\n", tr->plat_info.package_id);
+					continue;
+				}
+				found = true;
+				(*tentry)->active = true;
+				pkg[tr->plat_info.package_id].count++;
Quoted 7 lines

There seems to be some duplicate information between the structures. For example, telem_entry::num_rmids and the num_rmids from the pmt_feature_group. Are these expected to be identical? Since the num_rmids from telem_entry are hardcoded and the ones from pmt_feature_group discovered then this sounds like opportunity to add some sanity checking.
Similarly, could there be a check here to ensure that the size of the memory region provided matches the expected size based on all the hardcoded properties?

I should also sanity check against CPUID(0xF,0x0).EBX

Quoted 19 lines
Quoted 19 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +18 -0
+				break;
+			}
+		}
+	}
+
+	return found;
+}
+
+DEFINE_FREE(intel_pmt_put_feature_group, struct pmt_feature_group *,	\
+	if (!IS_ERR_OR_NULL(_T))					\
+		intel_pmt_put_feature_group(_T))
+
+/*
+ * Ask OOBMSM discovery driver for all the RMID based telemetry groups
+ * that it supports.
+ */
+static bool get_events(void)
+{
Quoted 8 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c line 152

Quoted 6 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +6 -0
+	struct pmt_feature_group *p1 __free(intel_pmt_put_feature_group) = NULL;
+	struct pmt_feature_group *p2 __free(intel_pmt_put_feature_group) = NULL;
+	int num_pkgs = topology_max_packages();
+	struct pkg_info *pkg __free(kfree) = NULL;
+
+	pkg = kmalloc_array(num_pkgs, sizeof(*pkg_info), GFP_KERNEL | __GFP_ZERO);

kcalloc() ?

Yes.

Quoted 6 lines
Quoted 6 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +6 -0
+	if (!pkg)
+		return false;
+
+	p1 = intel_pmt_get_regions_by_feature(FEATURE_PER_RMID_ENERGY_TELEM);
+	p2 = intel_pmt_get_regions_by_feature(FEATURE_PER_RMID_PERF_TELEM);
+
Quoted 8 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c line 164

Quoted 6 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +6 -0
+	if (IS_ERR_VALUE(p1) && IS_ERR_VALUE(p1))
+		return false;
+
+	if (!IS_ERR_VALUE(p1))
+		if (!count_events(pkg, num_pkgs, p1))
+			intel_pmt_put_feature_group(no_free_ptr(p1));

This seems to defeat the purpose of the cleanup handler.

Maybe this isn't needed. I'll have to dig through the various cases of return values, and whether the .guid matches a known value.

Quoted 72 lines
Quoted 72 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +47 -0
+	if (!IS_ERR_VALUE(p2))
+		if (!count_events(pkg, num_pkgs, p2))
+			intel_pmt_put_feature_group(no_free_ptr(p2));
+
+	if (!IS_ERR_OR_NULL(p1))
+		feat_energy = no_free_ptr(p1);
+	if (!IS_ERR_OR_NULL(p2))
+		feat_perf = no_free_ptr(p2);
+	pkg_info = no_free_ptr(pkg);
+
+	return true;
+}
+
+/*
+ * Early initialization. Cannot do much here because OOBMSM has not
+ * completed enumeration of telemetry event groups.
+ */
+int rdt_get_intel_aet_mon_config(void)
+{
+	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_INTEL_AET].r_resctrl;
+
+	INIT_LIST_HEAD(&r->evt_list);
+
+	return 1;
+}
+
+/*
+ * Late (first mount) initialization. Safe to ask OOBMSM which telemetry
+ * event groups are supported.
+ */
+void rdt_get_intel_aet_mount(void)
+{
+	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_INTEL_AET].r_resctrl;
+	struct rdt_core_mon_domain *d, *tmp;
+	static int do_one_time;
+
+	if (do_one_time)
+		return;
+
+	do_one_time = 1;
+
+	if (!get_events()) {
+		list_for_each_entry_safe(d, tmp, &r->mon_domains, hdr.list)
+			kfree(d);
+		r->mon_capable = false;
+	}
+}
fs/resctrl/rdtgroup.c +3 -0
index a90291f57330..4833dfa08ce3 100644
@@ -2572,6 +2572,9 @@ static int rdt_get_tree(struct fs_context *fc)
 
 	cpus_read_lock();
 	mutex_lock(&rdtgroup_mutex);
+
+	rdt_get_intel_aet_mount();
+
 	/*
 	 * resctrl file system can only be mounted once.
 	 */
arch/x86/kernel/cpu/resctrl/Makefile +1 -0
index 2c3b09f95127..a47e1c214087 100644
@@ -2,6 +2,7 @@
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= core.o rdtgroup.o monitor.o
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= ctrlmondata.o
 obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= pseudo_lock.o
+obj-$(CONFIG_INTEL_AET_RESCTRL)		+= intel_aet.o
 obj-$(CONFIG_INTEL_AET_RESCTRL)		+= fake_intel_aet_features.o
 
Quoted 3 lines

arch/x86/kernel/cpu/resctrl/Makefile line 8

Quoted 1 line
arch/x86/kernel/cpu/resctrl/Makefile +0 -0
 # To allow define_trace.h's recursive include:

Reinette

-Tony

maintainer
#17 lore raw

Re: [PATCH v2 07/16] x86/resctrl: Add initialization hook for Intel PMT events

Hi Tony,

Quoted 21 lines · Tony
Quoted 9 lines · Reinette Chatre

Hi Tony,

Quoted 4 lines · Tony Luck

Call the OOBMSM discovery code to find out if there are any event groups that match unique identifiers understood by resctrl.

Note that initiialzation must happen in two phases because the

"initiialzation" -> "initialization"

Wil fix.

Quoted 8 lines
Quoted 8 lines

OOBMSM VSEC discovery process is not complete at resctrl "lateinit()" initialization time. So there is an initial hook that assumes that Intel PMT will exist, called early so that package scoped domain groups are initialized.

At first mount the remainder of initialization is done. If there are no Intel PMT events, the package domain lists are removed.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

Move definition of struct mon_evt to <linux/resctrl_types.h>

Why not include/resctrl.h?

I put in in resctrl_types.h because it is a type definition? I'm not sure of the criteria James used to decide what goes in resctrl_types.h and what goes in resctrl.h

Changelog of
commit f16adbaf9272 ("x86/resctrl: Move resctrl types to a separate header") explains it well.
Essentially I expect a new entry in resctrl_types.h to have a user in arch/x86/include/asm/resctrl.h and since cover letter shows no changes to this file the addition is unexpected to me.

Quoted 51 lines
Quoted 42 lines
Quoted 42 lines

static __init void __check_quirks_intel(void)

arch/x86/kernel/cpu/resctrl/intel_aet.c new file +35 -0
new file mode 100644
index 000000000000..9a8ccb62b4ab
@@ -0,0 +1,211 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Resource Director Technology(RDT)
+ * - Intel Application Energy Telemetry
+ *
+ * Copyright (C) 2025 Intel Corporation
+ *
+ * Author:
+ *    Tony Luck <[email protected]>
+ */
+
+#define pr_fmt(fmt)   "resctrl: " fmt
+
+#include <linux/cpu.h>
+#include <linux/cleanup.h>
+#include <linux/slab.h>
+#include "fake_intel_aet_features.h"
+#include <linux/intel_vsec.h>
+#include <asm/resctrl.h>
+
+#include "internal.h"
+
+static struct pmt_feature_group *feat_energy;
+static struct pmt_feature_group *feat_perf;
+
+/* All telemetry events from Intel CPUs */
+enum pmt_event_id {
+	PMT_EVENT_ENERGY,
+	PMT_EVENT_ACTIVITY,
+	PMT_EVENT_STALLS_LLC_HIT,
+	PMT_EVENT_C1_RES,
+	PMT_EVENT_UNHALTED_CORE_CYCLES,
+	PMT_EVENT_STALLS_LLC_MISS,
+	PMT_EVENT_AUTO_C6_RES,
+	PMT_EVENT_UNHALTED_REF_CYCLES,
Quoted 8 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c line 39

Quoted 4 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c new file +4 -0
+	PMT_EVENT_UOPS_RETIRED,
+
+	PMT_NUM_EVENTS
+};

I expect the above to become part of resctrl fs. Actually, I think all properties of the new events, the id, name and how to display it, should be part of resctrl fs.

Quoted 5 lines

I'm not convinced about the amount of re-use that there will be for these events. I took a quick look at the current plan for a processor that follows Clearwater Forest and I see 22 events, only 3 of them match events in the above list.

This is not about re-use but instead these events becoming part of resctrl fs ABI.

Quoted 9 lines
Quoted 4 lines

We do not want other architectures to create their own display names for the same events. I expect that this will require more plumbing between arch and fs code to communicate which events are supported, similar to what exists for the L3 events (for example, resctrl_arch_is_mbm_total_enabled()).

Supported monitor events are indicated by setting bits in "rdt_mon_features" currently "unsigned int" but could become "long" or a longer bitmap if we really want the FS layer to keep track of every possible event for every architecture.

That sounds fine. rdt_mon_features is x86 arch code and that can change as needed. Previous discussions ended up with rdt_mon_features staying with x86 arch code and accessed from resctrl fs with helpers.

Quoted 9 lines
Quoted 5 lines

This may result in struct rdt_core_mon_domain to no longer be empty but instead for resctrl to use it to manage state of which events are enabled or not. Theoretically all could be managed by the architecture but I think that could result in inconsistent error codes to user space.

rdt_core_mon_domain seems like the wrong level (unless we expect to have different enabled events in different domains). rdt_resource seems plausible ... or just expand "rdt_mon_features".

Understood on the domain level. This work introduced quite a few new data structures that I have not yet digested. I cannot tell at this time if new resctrl fs helpers will have easier time with unique perf related data structure or if x86's internal rdt_mon_features need changes in addition to all these data structures.

Reinette

author maintainer
#18 lore raw

PATCH v2 8/16 x86/resctrl: Add Intel PMT domain specific code

In reply to Tony Luck

Domain specific initialization and tear down. Very simple as there are no domain specific elements that need to be allocated or initiailazed. Just the domain itself.

---
 arch/x86/kernel/cpu/resctrl/core.c | 32 ++++++++++++++++++++++++++++++
 fs/resctrl/rdtgroup.c              | 11 ++++++++--
 2 files changed, 41 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +32 -0
index 1ab0f5eec244..2adf40d8de32 100644
@@ -541,6 +541,29 @@ static void setup_l3_mon_domain(int cpu, int id, struct rdt_resource *r, struct
 	}
 }
 
+static void setup_intel_aet_mon_domain(int cpu, int id, struct rdt_resource *r,
+				       struct list_head *add_pos)
+{
+	struct rdt_core_mon_domain *d;
+	int err;
+
+	d = kzalloc_node(sizeof(*d), GFP_KERNEL, cpu_to_node(cpu));
+	if (!d)
+		return;
+
+	d->hdr.id = id;
+	d->hdr.type = RESCTRL_MON_DOMAIN;
+	cpumask_set_cpu(cpu, &d->hdr.cpu_mask);
+	list_add_tail_rcu(&d->hdr.list, add_pos);
+
+	err = resctrl_online_mon_domain(r, &d->hdr);
+	if (err) {
+		list_del_rcu(&d->hdr.list);
+		synchronize_rcu();
+		kfree(d);
+	}
+}
+
 static void domain_add_cpu_mon(int cpu, struct rdt_resource *r)
 {
 	int id = get_domain_id_from_scope(cpu, r->mon_scope);
@@ -565,6 +588,9 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r)
 	case RDT_RESOURCE_L3:
 		setup_l3_mon_domain(cpu, id, r, add_pos);
 		break;
+	case RDT_RESOURCE_INTEL_AET:
+		setup_intel_aet_mon_domain(cpu, id, r, add_pos);
+		break;
 	default:
 		WARN_ON_ONCE(1);
 	}
@@ -661,6 +687,12 @@ static void domain_remove_cpu_mon(int cpu, struct rdt_resource *r)
 		synchronize_rcu();
 		mon_domain_free(hw_dom);
 		break;
+	case RDT_RESOURCE_INTEL_AET:
+		resctrl_offline_mon_domain(r, d);
+		list_del_rcu(&hdr->list);
+		synchronize_rcu();
+		kfree(container_of(hdr, struct rdt_core_mon_domain, hdr));
+		break;
 	}
 }
 
fs/resctrl/rdtgroup.c +9 -2
index 4833dfa08ce3..d3919642aa9b 100644
@@ -4077,6 +4077,9 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_mon_domain *d
 	if (resctrl_mounted && resctrl_arch_mon_capable())
 		rmdir_mondata_subdir_allrdtgrp(r, &d->hdr);
 
+	if (r->rid == RDT_RESOURCE_INTEL_AET)
+		goto done;
+
 	if (resctrl_is_mbm_enabled())
 		cancel_delayed_work(&d->mbm_over);
 	if (resctrl_arch_is_llc_occupancy_enabled() && has_busy_rmid(d)) {
@@ -4093,7 +4096,7 @@ void resctrl_offline_mon_domain(struct rdt_resource *r, struct rdt_mon_domain *d
 	}
 
 	domain_destroy_mon_state(d);
-
+done:
 	mutex_unlock(&rdtgroup_mutex);
 }
 
@@ -4160,11 +4163,14 @@ int resctrl_online_ctrl_domain(struct rdt_resource *r, struct rdt_ctrl_domain *d
 int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *hdr)
 {
 	struct rdt_mon_domain *d;
-	int err;
+	int err = 0;
 
 	d = container_of(hdr, struct rdt_mon_domain, hdr);
 	mutex_lock(&rdtgroup_mutex);
 
+	if (r->rid == RDT_RESOURCE_INTEL_AET)
+		goto do_mkdir;
+
 	err = domain_setup_mon_state(r, d);
 	if (err)
 		goto out_unlock;
@@ -4178,6 +4184,7 @@ int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *hdr
 	if (resctrl_arch_is_llc_occupancy_enabled())
 		INIT_DELAYED_WORK(&d->cqm_limbo, cqm_handle_limbo);
 
+do_mkdir:
 	/*
 	 * If the filesystem is not mounted then only the default resource group
 	 * exists. Creation of its directories is deferred until mount time
Signature · 1 line
2.48.1
author maintainer
#19 lore raw

PATCH v2 9/16 x86/resctrl: Add detailed descriptions for Clearwater Forest events

In reply to Tony Luck

There are two event groups one for energy reporting and another for "perf" events.

See the XML description files in https://github.com/intel/Intel-PMT
in the xml/CWF/OOBMSM/{RMID-ENERGY,RMID-PERF}/ for the detailed descriptions that were used to derive these descriptions.

---
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 54 +++++++++++++++++++++++++
 1 file changed, 54 insertions(+)
arch/x86/kernel/cpu/resctrl/intel_aet.c +54 -0
index 9a8ccb62b4ab..67862e81b9e0 100644
@@ -94,8 +94,62 @@ struct telem_entry {
 	struct pmt_event evts[];
 };
 
+/*
+ * Known events from Intel Clearwater Forest CPU.
+ */
+#define CWF_NUM_RMIDS	576
+#define CWF_ENERGY_GUID	0x26696143
+#define CWF_PERF_GUID	0x26557651
+#define	CWF_ENERGY_STRIDE	0x10
+#define CWF_PERF_STRIDE	0x38
+
+/*
+ * https://github.com/intel/Intel-PMT
+ * xml/CWF/OOBMSM/RMID-ENERGY *.xml
+ */
+static struct telem_entry cwf_energy = {
+	.name				= "energy",
+	.guid				= CWF_ENERGY_GUID,
+	.num_rmids			= CWF_NUM_RMIDS,
+	.stride				= CWF_ENERGY_STRIDE,
+	.overflow_counter_off		= CWF_NUM_RMIDS * CWF_ENERGY_STRIDE,
+	.last_overflow_tstamp_off	= CWF_NUM_RMIDS * CWF_ENERGY_STRIDE + 8,
+	.last_update_tstamp_off		= CWF_NUM_RMIDS * CWF_ENERGY_STRIDE + 16,
+	.evts				= {
+		EVT(PMT_EVENT_ENERGY, "core_energy", 0x0, EVT_U46_18),
+		EVT(PMT_EVENT_ACTIVITY, "activity", 0x8, EVT_U46_18),
+		{ }
+	}
+};
+
+/*
+ * https://github.com/intel/Intel-PMT
+ * xml/CWF/OOBMSM/RMID-PERF *.xml
+ */
+static struct telem_entry cwf_perf = {
+	.name				= "perf",
+	.guid				= CWF_PERF_GUID,
+	.num_rmids			= CWF_NUM_RMIDS,
+	.stride				= CWF_PERF_STRIDE,
+	.overflow_counter_off		= CWF_NUM_RMIDS * CWF_PERF_STRIDE,
+	.last_overflow_tstamp_off	= CWF_NUM_RMIDS * CWF_PERF_STRIDE + 8,
+	.last_update_tstamp_off		= CWF_NUM_RMIDS * CWF_PERF_STRIDE + 16,
+	.evts				= {
+		EVT(PMT_EVENT_STALLS_LLC_HIT, "stalls_llc_hit", 0x0, EVT_U64),
+		EVT(PMT_EVENT_C1_RES, "c1_res", 0x8, EVT_U64),
+		EVT(PMT_EVENT_UNHALTED_CORE_CYCLES, "unhalted_core_cycles", 0x10, EVT_U64),
+		EVT(PMT_EVENT_STALLS_LLC_MISS, "stalls_llc_miss", 0x18, EVT_U64),
+		EVT(PMT_EVENT_AUTO_C6_RES, "c6_res", 0x20, EVT_U64),
+		EVT(PMT_EVENT_UNHALTED_REF_CYCLES, "unhalted_ref_cycles", 0x28, EVT_U64),
+		EVT(PMT_EVENT_UOPS_RETIRED, "uops_retired", 0x30, EVT_U64),
+		{ }
+	}
+};
+
 /* All known telemetry event groups */
 static struct telem_entry *telem_entry[] = {
+	&cwf_energy,
+	&cwf_perf,
 	NULL
 };
 
Signature · 1 line
2.48.1
maintainer
#20 lore raw

Re: [PATCH v2 09/16] x86/resctrl: Add detailed descriptions for Clearwater Forest events

Hi Tony,

Quoted 6 lines · Tony Luck

There are two event groups one for energy reporting and another for "perf" events.

See the XML description files in https://github.com/intel/Intel-PMT
in the xml/CWF/OOBMSM/{RMID-ENERGY,RMID-PERF}/ for the detailed descriptions that were used to derive these descriptions.

It is unexpected to me that this is made model specific while the implementation is built around a guid. What will happen when a new system using the same event layout arrives? Will the url above be duplicated for this new system's acronym and contain duplicate data? How will the resctrl support change? If I understand correctly resctrl will not need to be changed but instead the "CWF" events will just automatically be used for this new hypothetical system? This makes me think that this should not be so CWF specific.

Reinette

author maintainer
#21 lore raw

Re: [PATCH v2 09/16] x86/resctrl: Add detailed descriptions for Clearwater Forest events

Quoted 10 lines · Reinette Chatre

Hi Tony,

Quoted 6 lines · Tony Luck

There are two event groups one for energy reporting and another for "perf" events.

See the XML description files in https://github.com/intel/Intel-PMT
in the xml/CWF/OOBMSM/{RMID-ENERGY,RMID-PERF}/ for the detailed descriptions that were used to derive these descriptions.

Quoted 8 lines

It is unexpected to me that this is made model specific while the implementation is built around a guid. What will happen when a new system using the same event layout arrives? Will the url above be duplicated for this new system's acronym and contain duplicate data? How will the resctrl support change? If I understand correctly resctrl will not need to be changed but instead the "CWF" events will just automatically be used for this new hypothetical system? This makes me think that this should not be so CWF specific.

I was told that we might expect to see new guid values to describe different event register layouts for the same CPU model. The event aggregators are all firmware driven. So a BIOS update could make changes.

So I've left open the option to add additional structure defintions for Clearwater Forest with some future firmware update.

If a future processor uses the exact same layout with the same guid value, then no Linux update would be needed. We'd just have the slight oddity that a structure named "cwf_*" would match and be used.

Next system to implement these telemetry events has a very different list of supported events.

Quoted 1 line

Reinette

-Tony

maintainer
#22 lore raw

Re: [PATCH v2 09/16] x86/resctrl: Add detailed descriptions for Clearwater Forest events

Hi Tony,

Quoted 20 lines · Tony
Quoted 10 lines · Reinette Chatre

Hi Tony,

Quoted 6 lines · Tony Luck

There are two event groups one for energy reporting and another for "perf" events.

See the XML description files in https://github.com/intel/Intel-PMT
in the xml/CWF/OOBMSM/{RMID-ENERGY,RMID-PERF}/ for the detailed descriptions that were used to derive these descriptions.

Quoted 8 lines

It is unexpected to me that this is made model specific while the implementation is built around a guid. What will happen when a new system using the same event layout arrives? Will the url above be duplicated for this new system's acronym and contain duplicate data? How will the resctrl support change? If I understand correctly resctrl will not need to be changed but instead the "CWF" events will just automatically be used for this new hypothetical system? This makes me think that this should not be so CWF specific.

Quoted 3 lines

I was told that we might expect to see new guid values to describe different event register layouts for the same CPU model. The event aggregators are all firmware driven. So a BIOS update could make changes.

Sounds like this supports my point. Naming these data structures based on the CPU would create naming challenges when this scenario arrives, no?

Quoted 3 lines

So I've left open the option to add additional structure defintions for Clearwater Forest with some future firmware update.

Why not use the guid as part of the naming to make clear that CPU model does not dictate the layout?

Quoted 8 lines

If a future processor uses the exact same layout with the same guid value, then no Linux update would be needed. We'd just have the slight oddity that a structure named "cwf_*" would match and be used.

Next system to implement these telemetry events has a very different list of supported events.

Reinette

author maintainer
#23 lore raw

PATCH v2 10/16 x86/resctrl: Allocate per-package structures for known events

In reply to Tony Luck

Use the per-package counts of known events to allocate arrays to make a copy of just the known events.

Add hook into resctrl_exit() to cleanup.

---
 arch/x86/kernel/cpu/resctrl/internal.h  |  2 +
 arch/x86/kernel/cpu/resctrl/core.c      |  2 +
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 60 ++++++++++++++++++++++++-
 3 files changed, 63 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
index ada402c7678b..2503a24e4177 100644
@@ -170,8 +170,10 @@ int rdt_get_mon_l3_config(struct rdt_resource *r);
 
 #ifdef CONFIG_INTEL_AET_RESCTRL
 int rdt_get_intel_aet_mon_config(void);
+void rdt_intel_aet_exit(void);
 #else
 static inline int rdt_get_intel_aet_mon_config(void) { return 0; }
+static inline void rdt_intel_aet_exit(void) { };
 #endif
 
 bool rdt_cpu_has(int flag);
arch/x86/kernel/cpu/resctrl/core.c +2 -0
index 2adf40d8de32..d011c095aafa 100644
@@ -1095,6 +1095,8 @@ static void __exit resctrl_arch_exit(void)
 {
 	cpuhp_remove_state(rdt_online);
 
+	rdt_intel_aet_exit();
+
 	resctrl_exit();
 }
 
arch/x86/kernel/cpu/resctrl/intel_aet.c +59 -1

90 lines not on this page · raw message

Signature · 1 line
2.48.1
maintainer
#24 lore raw

Re: [PATCH v2 10/16] x86/resctrl: Allocate per-package structures for known events

hi Tony,

Quoted 4 lines · Tony Luck

Use the per-package counts of known events to allocate arrays to make a copy of just the known events.

Add hook into resctrl_exit() to cleanup.

(above is not done in patch)

Quoted 41 lines
---
 arch/x86/kernel/cpu/resctrl/internal.h  |  2 +
 arch/x86/kernel/cpu/resctrl/core.c      |  2 +
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 60 ++++++++++++++++++++++++-
 3 files changed, 63 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
index ada402c7678b..2503a24e4177 100644
@@ -170,8 +170,10 @@ int rdt_get_mon_l3_config(struct rdt_resource *r);
 
 #ifdef CONFIG_INTEL_AET_RESCTRL
 int rdt_get_intel_aet_mon_config(void);
+void rdt_intel_aet_exit(void);
 #else
 static inline int rdt_get_intel_aet_mon_config(void) { return 0; }
+static inline void rdt_intel_aet_exit(void) { };
 #endif
 
 bool rdt_cpu_has(int flag);
arch/x86/kernel/cpu/resctrl/core.c +2 -0
index 2adf40d8de32..d011c095aafa 100644
@@ -1095,6 +1095,8 @@ static void __exit resctrl_arch_exit(void)
 {
 	cpuhp_remove_state(rdt_online);
 
+	rdt_intel_aet_exit();
+
 	resctrl_exit();
 }
 
arch/x86/kernel/cpu/resctrl/intel_aet.c +0 -0
index 67862e81b9e0..e2d8eab997fc 100644

arch/x86/kernel/cpu/resctrl/intel_aet.c static bool count_events(struct pkg_info *pkg, int max_pkgs, struct pmt_feature_ · line 191

Quoted 4 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +1 -0
@@ -188,6 +188,26 @@ static bool count_events(struct pkg_info *pkg, int max_pkgs, struct pmt_feature_
 	return found;
 }
 
+static int setup(struct pkg_info *pkg, int pkgnum, struct pmt_feature_group *p, int slot)

Could you please add a comment to this function to explain what it does?

Quoted 49 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +35 -0
+{
+	struct telem_entry **tentry;
+
+	for (int i = 0; i < p->count; i++) {
+		for (tentry = telem_entry; *tentry; tentry++) {
+			if (!(*tentry)->active)
+				continue;
+			if (pkgnum != p->regions[i].plat_info.package_id)
+				continue;
+			if (p->regions[i].guid != (*tentry)->guid)
+				continue;
+
+			pkg[pkgnum].regions[slot++] =  p->regions[i];
+		}
+	}
+
+	return slot;
+}
+
 DEFINE_FREE(intel_pmt_put_feature_group, struct pmt_feature_group *,	\
 	if (!IS_ERR_OR_NULL(_T))					\
 		intel_pmt_put_feature_group(_T))
@@ -202,6 +222,8 @@ static bool get_events(void)
 	struct pmt_feature_group *p2 __free(intel_pmt_put_feature_group) = NULL;
 	int num_pkgs = topology_max_packages();
 	struct pkg_info *pkg __free(kfree) = NULL;
+	bool found_known_features = false;
+	int i, slot;
 
 	pkg = kmalloc_array(num_pkgs, sizeof(*pkg_info), GFP_KERNEL | __GFP_ZERO);
 	if (!pkg)
@@ -220,13 +242,32 @@ static bool get_events(void)
 		if (!count_events(pkg, num_pkgs, p2))
 			intel_pmt_put_feature_group(no_free_ptr(p2));
 
+	for (i = 0; i < num_pkgs; i++) {
+		if (!pkg[i].count)
+			continue;
+		found_known_features = true;
+		pkg[i].regions = kmalloc_array(pkg[i].count, sizeof(*pkg[i].regions), GFP_KERNEL);
+		if (!pkg[i].regions)
+			goto fail;
+
+		slot = 0;
+		if (!IS_ERR_VALUE(p1))
+			slot = setup(pkg, i, p1, slot);
+		if (!IS_ERR_VALUE(p2))
+			slot = setup(pkg, i, p2, slot);
+	}

arch/x86/kernel/cpu/resctrl/intel_aet.c static bool get_events(void) · line 267

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +3 -1
+
 	if (!IS_ERR_OR_NULL(p1))
 		feat_energy = no_free_ptr(p1);
 	if (!IS_ERR_OR_NULL(p2))
 		feat_perf = no_free_ptr(p2);
 	pkg_info = no_free_ptr(pkg);
 
-	return true;
+	return found_known_features;
+fail:

include/linux/cleanup.h has this to say about mixing goto and free helpers:
"convert all resources that need a "goto" cleanup to scope-based cleanup, or convert none of them"

Quoted 20 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +13 -0
+	while (--i > 0)
+		kfree(pkg[i].regions);
+	return false;
 }
 
 /*
@@ -242,6 +283,23 @@ int rdt_get_intel_aet_mon_config(void)
 	return 1;
 }
 
+/* Clean up when resctrl shuts down completely */
+void rdt_intel_aet_exit(void)
+{
+	int num_pkgs = topology_max_packages();
+
+	if (pkg_info) {
+		for (int i = 0; i < num_pkgs; i++)
+			kfree(pkg_info[i].regions);
+		kfree(pkg_info);
+	}

arch/x86/kernel/cpu/resctrl/intel_aet.c int rdt_get_intel_aet_mon_config(void) · line 305

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +7 -0
+
+	if (feat_energy)
+		intel_pmt_put_feature_group(feat_energy);
+	if (feat_perf)
+		intel_pmt_put_feature_group(feat_perf);
+}
+
 /*
  * Late (first mount) initialization. Safe to ask OOBMSM which telemetry
  * event groups are supported.

Reinette

author maintainer
#25 lore raw

Re: [PATCH v2 10/16] x86/resctrl: Allocate per-package structures for known events

Quoted 9 lines · Reinette Chatre

hi Tony,

Quoted 4 lines · Tony Luck

Use the per-package counts of known events to allocate arrays to make a copy of just the known events.

Add hook into resctrl_exit() to cleanup.

(above is not done in patch)

I think I shuffled it to a different patch. I will track it down and move this comment to the right place.

Quoted 44 lines
Quoted 44 lines
---
 arch/x86/kernel/cpu/resctrl/internal.h  |  2 +
 arch/x86/kernel/cpu/resctrl/core.c      |  2 +
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 60 ++++++++++++++++++++++++-
 3 files changed, 63 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
index ada402c7678b..2503a24e4177 100644
@@ -170,8 +170,10 @@ int rdt_get_mon_l3_config(struct rdt_resource *r);
 
 #ifdef CONFIG_INTEL_AET_RESCTRL
 int rdt_get_intel_aet_mon_config(void);
+void rdt_intel_aet_exit(void);
 #else
 static inline int rdt_get_intel_aet_mon_config(void) { return 0; }
+static inline void rdt_intel_aet_exit(void) { };
 #endif
 
 bool rdt_cpu_has(int flag);
arch/x86/kernel/cpu/resctrl/core.c +2 -0
index 2adf40d8de32..d011c095aafa 100644
@@ -1095,6 +1095,8 @@ static void __exit resctrl_arch_exit(void)
 {
 	cpuhp_remove_state(rdt_online);
 
+	rdt_intel_aet_exit();
+
 	resctrl_exit();
 }
 
arch/x86/kernel/cpu/resctrl/intel_aet.c +0 -0
index 67862e81b9e0..e2d8eab997fc 100644
@@ -188,6 +188,26 @@ static bool count_events(struct pkg_info *pkg, int max_pkgs, struct pmt_feature_
 	return found;
 }
 
Quoted 3 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c static bool count_events(struct pkg_info *pkg, int max_pkgs, struct pmt_feature_ · line 191

Quoted 1 line
arch/x86/kernel/cpu/resctrl/intel_aet.c +1 -0
+static int setup(struct pkg_info *pkg, int pkgnum, struct pmt_feature_group *p, int slot)

Could you please add a comment to this function to explain what it does?

Indeed yes, it really needs some description.

Quoted 57 lines
Quoted 57 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +36 -0
+{
+	struct telem_entry **tentry;
+
+	for (int i = 0; i < p->count; i++) {
+		for (tentry = telem_entry; *tentry; tentry++) {
+			if (!(*tentry)->active)
+				continue;
+			if (pkgnum != p->regions[i].plat_info.package_id)
+				continue;
+			if (p->regions[i].guid != (*tentry)->guid)
+				continue;
+
+			pkg[pkgnum].regions[slot++] =  p->regions[i];
+		}
+	}
+
+	return slot;
+}
+
 DEFINE_FREE(intel_pmt_put_feature_group, struct pmt_feature_group *,	\
 	if (!IS_ERR_OR_NULL(_T))					\
 		intel_pmt_put_feature_group(_T))
@@ -202,6 +222,8 @@ static bool get_events(void)
 	struct pmt_feature_group *p2 __free(intel_pmt_put_feature_group) = NULL;
 	int num_pkgs = topology_max_packages();
 	struct pkg_info *pkg __free(kfree) = NULL;
+	bool found_known_features = false;
+	int i, slot;
 
 	pkg = kmalloc_array(num_pkgs, sizeof(*pkg_info), GFP_KERNEL | __GFP_ZERO);
 	if (!pkg)
@@ -220,13 +242,32 @@ static bool get_events(void)
 		if (!count_events(pkg, num_pkgs, p2))
 			intel_pmt_put_feature_group(no_free_ptr(p2));
 
+	for (i = 0; i < num_pkgs; i++) {
+		if (!pkg[i].count)
+			continue;
+		found_known_features = true;
+		pkg[i].regions = kmalloc_array(pkg[i].count, sizeof(*pkg[i].regions), GFP_KERNEL);
+		if (!pkg[i].regions)
+			goto fail;
+
+		slot = 0;
+		if (!IS_ERR_VALUE(p1))
+			slot = setup(pkg, i, p1, slot);
+		if (!IS_ERR_VALUE(p2))
+			slot = setup(pkg, i, p2, slot);
+	}
+
 	if (!IS_ERR_OR_NULL(p1))
 		feat_energy = no_free_ptr(p1);
 	if (!IS_ERR_OR_NULL(p2))
 		feat_perf = no_free_ptr(p2);
 	pkg_info = no_free_ptr(pkg);
 
Quoted 7 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c static bool get_events(void) · line 267

Quoted 3 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +2 -1
-	return true;
+	return found_known_features;
+fail:

include/linux/cleanup.h has this to say about mixing goto and free helpers:
"convert all resources that need a "goto" cleanup to scope-based cleanup, or convert none of them"

Seems an awkward restriction for this case. "pkg" is a pointer to a dynamic array, and each of the elements of the array might have been initialized by another allocation. "pkg" is under control of the __free() cleanup function.

Maybe I could define a custom cleanup (syntax of multi-statement action to be figured out):

DEFINE_FREE(pkg_free, struct pkg_info *,

	if (_T)
		for (int i = 0; i < num_pkgs; i++)
			kfree(_T[i].regions);
	kfree(_T)

)

	struct pkg_info *pkg __free(pkg_free) = NULL;
Quoted 33 lines
Quoted 21 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +13 -0
+	while (--i > 0)
+		kfree(pkg[i].regions);
+	return false;
 }
 
 /*
@@ -242,6 +283,23 @@ int rdt_get_intel_aet_mon_config(void)
 	return 1;
 }
 
+/* Clean up when resctrl shuts down completely */
+void rdt_intel_aet_exit(void)
+{
+	int num_pkgs = topology_max_packages();
+
+	if (pkg_info) {
+		for (int i = 0; i < num_pkgs; i++)
+			kfree(pkg_info[i].regions);
+		kfree(pkg_info);
+	}

arch/x86/kernel/cpu/resctrl/intel_aet.c int rdt_get_intel_aet_mon_config(void) · line 305

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +7 -0
+
+	if (feat_energy)
+		intel_pmt_put_feature_group(feat_energy);
+	if (feat_perf)
+		intel_pmt_put_feature_group(feat_perf);
+}
+
 /*
  * Late (first mount) initialization. Safe to ask OOBMSM which telemetry
  * event groups are supported.

Reinette

author maintainer
#26 lore raw

Re: [PATCH v2 10/16] x86/resctrl: Allocate per-package structures for known events

Quoted 13 lines · Tony
Quoted 3 lines · Reinette Chatre

include/linux/cleanup.h has this to say about mixing goto and free helpers:
"convert all resources that need a "goto" cleanup to scope-based cleanup, or convert none of them"

Seems an awkward restriction for this case. "pkg" is a pointer to a dynamic array, and each of the elements of the array might have been initialized by another allocation. "pkg" is under control of the __free() cleanup function.

Maybe I could define a custom cleanup (syntax of multi-statement action to be figured out):

Quoted 8 lines

DEFINE_FREE(pkg_free, struct pkg_info *,

	if (_T)
		for (int i = 0; i < num_pkgs; i++)
			kfree(_T[i].regions);
	kfree(_T)

)

	struct pkg_info *pkg __free(pkg_free) = NULL;

Tried this out and it works (without any odd syntax for the multi-line action (though I did miss that num_pkgs was a local variable, thankfully simply initialized from topology_max_packages().

DEFINE_FREE(free_pkg_info, struct pkg_info *,				\
	if (_T)								\
		for (int i = 0; i < topology_max_packages(); i++)	\
			kfree(_T[i].regions);				\
	kfree(_T))

...

	struct pkg_info *pkg __free(free_pkg_info) = NULL;

-Tony

author maintainer
#27 lore raw

PATCH v2 11/16 x86/resctrl: Link known events onto RDT_RESOURCE_INTEL_AET.evt_list

In reply to Tony Luck

Core code uses this list to populate "mon_data" directories.

---
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/intel_aet.c +22 -1
index e2d8eab997fc..9ac912742ef1 100644
@@ -13,6 +13,7 @@
 
 #include <linux/cpu.h>
 #include <linux/cleanup.h>
+#include <linux/minmax.h>
 #include <linux/slab.h>
 #include "fake_intel_aet_features.h"
 #include <linux/intel_vsec.h>
@@ -308,14 +309,34 @@ void rdt_get_intel_aet_mount(void)
 {
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_INTEL_AET].r_resctrl;
 	struct rdt_core_mon_domain *d, *tmp;
+	struct telem_entry **tentry;
 	static int do_one_time;
+	struct mon_evt *evt;
+	bool ret = false;
 
 	if (do_one_time)
 		return;
 
 	do_one_time = 1;
 
-	if (!get_events()) {
+	if (!get_events())
+		goto done;
+
+	for (tentry = telem_entry; *tentry; tentry++) {
+		if (!(*tentry)->active)
+			continue;
+		for (int i = 0; (*tentry)->evts[i].evt.name; i++) {
+			evt = &(*tentry)->evts[i].evt;
+			list_add_tail(&evt->list, &r->evt_list);
+			ret = true;
+		}
+		if (!r->num_rmid)
+			r->num_rmid = (*tentry)->num_rmids;
+		else
+			r->num_rmid = min(r->num_rmid, (*tentry)->num_rmids);
+	}
+done:
+	if (!ret) {
 		list_for_each_entry_safe(d, tmp, &r->mon_domains, hdr.list)
 			kfree(d);
 		r->mon_capable = false;
Signature · 1 line
2.48.1
maintainer
#28 lore raw

Re: [PATCH v2 11/16] x86/resctrl: Link known events onto RDT_RESOURCE_INTEL_AET.evt_list

Hi Tony,

Quoted 35 lines · Tony Luck

Core code uses this list to populate "mon_data" directories.

---
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/intel_aet.c +5 -1
index e2d8eab997fc..9ac912742ef1 100644
@@ -13,6 +13,7 @@
 
 #include <linux/cpu.h>
 #include <linux/cleanup.h>
+#include <linux/minmax.h>
 #include <linux/slab.h>
 #include "fake_intel_aet_features.h"
 #include <linux/intel_vsec.h>
@@ -308,14 +309,34 @@ void rdt_get_intel_aet_mount(void)
 {
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_INTEL_AET].r_resctrl;
 	struct rdt_core_mon_domain *d, *tmp;
+	struct telem_entry **tentry;
 	static int do_one_time;
+	struct mon_evt *evt;
+	bool ret = false;
 
 	if (do_one_time)
 		return;
 
 	do_one_time = 1;
 
-	if (!get_events()) {
+	if (!get_events())

arch/x86/kernel/cpu/resctrl/intel_aet.c void rdt_get_intel_aet_mount(void) · line 332

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +10 -0
+		goto done;
+
+	for (tentry = telem_entry; *tentry; tentry++) {
+		if (!(*tentry)->active)
+			continue;
+		for (int i = 0; (*tentry)->evts[i].evt.name; i++) {
+			evt = &(*tentry)->evts[i].evt;
+			list_add_tail(&evt->list, &r->evt_list);
+			ret = true;
+		}

Architecture code should not be doing this. I expect this will be something
similar to l3_mon_evt_init() done by fs code after the architecture had
opportunity to configure which events are supported.
I wonder if resctrl_mon_resource_init() could be moved to rdt_get_tree()
to be called after the new (yet to be named) "resctrl_arch_for_arch_to_do_needed_on_resctrl_mount()" resctrl_mon_resource_init() could be enhanced to do any needed resctrl fs initialization for this new feature. This will include being able to
learn the accurate counts of rmid supported by the system to be able to
create monitor groups that can be supported by all monitoring resources?

arch/x86/kernel/cpu/resctrl/intel_aet.c void rdt_get_intel_aet_mount(void) · line 342

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +7 -0
+		if (!r->num_rmid)
+			r->num_rmid = (*tentry)->num_rmids;
+		else
+			r->num_rmid = min(r->num_rmid, (*tentry)->num_rmids);
+	}
+done:
+	if (!ret) {
 		list_for_each_entry_safe(d, tmp, &r->mon_domains, hdr.list)
 			kfree(d);
 		r->mon_capable = false;

Reinette

author maintainer
#29 lore raw

Re: [PATCH v2 11/16] x86/resctrl: Link known events onto RDT_RESOURCE_INTEL_AET.evt_list

Quoted 36 lines · Reinette Chatre

Hi Tony,

Quoted 33 lines · Tony Luck

Core code uses this list to populate "mon_data" directories.

---
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/intel_aet.c +4 -0
index e2d8eab997fc..9ac912742ef1 100644
@@ -13,6 +13,7 @@
 
 #include <linux/cpu.h>
 #include <linux/cleanup.h>
+#include <linux/minmax.h>
 #include <linux/slab.h>
 #include "fake_intel_aet_features.h"
 #include <linux/intel_vsec.h>
@@ -308,14 +309,34 @@ void rdt_get_intel_aet_mount(void)
 {
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_INTEL_AET].r_resctrl;
 	struct rdt_core_mon_domain *d, *tmp;
+	struct telem_entry **tentry;
 	static int do_one_time;
+	struct mon_evt *evt;
+	bool ret = false;
 
 	if (do_one_time)
 		return;
 
 	do_one_time = 1;
 
Quoted 16 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c void rdt_get_intel_aet_mount(void) · line 332

Quoted 12 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +11 -1
-	if (!get_events()) {
+	if (!get_events())
+		goto done;
+
+	for (tentry = telem_entry; *tentry; tentry++) {
+		if (!(*tentry)->active)
+			continue;
+		for (int i = 0; (*tentry)->evts[i].evt.name; i++) {
+			evt = &(*tentry)->evts[i].evt;
+			list_add_tail(&evt->list, &r->evt_list);
+			ret = true;
+		}

Architecture code should not be doing this. I expect this will be something similar to l3_mon_evt_init() done by fs code after the architecture had opportunity to configure which events are supported.

If we do put all possible events into the architecture code, then it can do something like:

	for_each_set_bit(evt, &rdt_mon_features, NUM_EVENTS)
		list_add(...);

My speculative patch:

	https://lore.kernel.org/all/[email protected]/

did this for the existing events, so could build on it.

Quoted 7 lines

I wonder if resctrl_mon_resource_init() could be moved to rdt_get_tree()
to be called after the new (yet to be named) "resctrl_arch_for_arch_to_do_needed_on_resctrl_mount()" resctrl_mon_resource_init() could be enhanced to do any needed resctrl fs initialization for this new feature. This will include being able to
learn the accurate counts of rmid supported by the system to be able to
create monitor groups that can be supported by all monitoring resources?

Quoted 12 lines

arch/x86/kernel/cpu/resctrl/intel_aet.c void rdt_get_intel_aet_mount(void) · line 342

Quoted 10 lines
arch/x86/kernel/cpu/resctrl/intel_aet.c +7 -0
+		if (!r->num_rmid)
+			r->num_rmid = (*tentry)->num_rmids;
+		else
+			r->num_rmid = min(r->num_rmid, (*tentry)->num_rmids);
+	}
+done:
+	if (!ret) {
 		list_for_each_entry_safe(d, tmp, &r->mon_domains, hdr.list)
 			kfree(d);
 		r->mon_capable = false;

Reinette

-Tony

author maintainer
#30 lore raw

PATCH v2 12/16 x86/resctrl: Build lookup table for package events

In reply to Tony Luck

The resctrl filesystem saves the evt_type in the private data pointer of the kernfs_node of each file in the mon_data directories.

To print the values for each file the show() function will need to map from this evtid to:

num_rmids - to make sure data for this file exists guid - to pick the pmt_event(s) for each package stride/offset - to compute MMIO offset for this RMID/event

Build a lookup table for each event to save searching through lists and add macros for ease of use.

---
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
arch/x86/kernel/cpu/resctrl/intel_aet.c +15 -0
index 9ac912742ef1..bab8e4de26b3 100644
@@ -39,6 +39,18 @@ enum pmt_event_id {
 	PMT_NUM_EVENTS
 };
 
+static struct evtinfo {
+	struct telem_entry	*telem_entry;
+	struct pmt_event	*pmt_event;
+} evtinfo[PMT_NUM_EVENTS];
+
+#define EVT_NUM_RMIDS(evtid)	(evtinfo[evtid].telem_entry->num_rmids)
+#define EVT_STRIDE(evtid)	(evtinfo[evtid].telem_entry->stride)
+#define EVT_GUID(evtid)		(evtinfo[evtid].telem_entry->guid)
+
+#define EVT_OFFSET(evtid)	(evtinfo[evtid].pmt_event->evt_offset)
+#define EVT_TYPE(evtid)		(evtinfo[evtid].pmt_event->evt_type)
+
 /**
  * enum evt_type - Type for values for each event.
  * @EVT_U64:	Integer up to 64 bits
@@ -328,6 +340,9 @@ void rdt_get_intel_aet_mount(void)
 		for (int i = 0; (*tentry)->evts[i].evt.name; i++) {
 			evt = &(*tentry)->evts[i].evt;
 			list_add_tail(&evt->list, &r->evt_list);
+
+			evtinfo[evt->evtid].telem_entry = *tentry;
+			evtinfo[evt->evtid].pmt_event = &(*tentry)->evts[i];
 			ret = true;
 		}
 		if (!r->num_rmid)
Signature · 1 line
2.48.1
author maintainer
#31 lore raw

PATCH v2 13/16 x86/resctrl: Add code to display core telemetry events

In reply to Tony Luck

These can be read from any CPU. Rely on the smp_call*() functions picking the current CPU when given a free choice from cpu_online_mask.

There may be multiple devices tracking each package, so scan all of them and add up counters.

Output format depends on the data type. Either a 63 bit integer, or a fixed point decimal.

---
 include/linux/resctrl.h                 |  3 ++
 fs/resctrl/internal.h                   |  4 +-
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 53 +++++++++++++++++++++++++
 fs/resctrl/ctrlmondata.c                | 23 ++++++++++-
 fs/resctrl/monitor.c                    | 23 +++++++++--
 5 files changed, 100 insertions(+), 6 deletions(-)
include/linux/resctrl.h +3 -0
index 999e0802a26e..e900764393f4 100644
@@ -532,8 +532,11 @@ void resctrl_exit(void);
 
 #ifdef CONFIG_INTEL_AET_RESCTRL
 void rdt_get_intel_aet_mount(void);
+bool intel_aet_read_event(int domid, int rmid, int evtid, u64 *val, bool *fptype);
 #else
 static inline void rdt_get_intel_aet_mount(void) { }
+static inline bool intel_aet_read_event(int domid, int rmid, int evtid, u64 *val,
+					bool *fptype) { return false; }
 #endif
 
 #endif /* _RESCTRL_H */
fs/resctrl/internal.h +3 -1
index f5a698b49e97..4d65a781034e 100644
@@ -98,6 +98,7 @@ struct mon_data {
  *	   domains in @r sharing L3 @ci.id
  * @evtid: Which monitor event to read.
  * @first: Initialize MBM counter when true.
+ * @fptype:If true indicates @val is in 46.18 fixed point format
  * @ci:    Cacheinfo for L3. Only set when @d is NULL. Used when summing domains.
  * @err:   Error encountered when reading counter.
  * @val:   Returned value of event counter. If @rgrp is a parent resource group,
@@ -112,6 +113,7 @@ struct rmid_read {
 	struct rdt_mon_domain	*d;
 	unsigned int		evtid;
 	bool			first;
+	bool			fptype;
 	struct cacheinfo	*ci;
 	int			err;
 	u64			val;
@@ -343,7 +345,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg);
 
 void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
 		    struct rdt_mon_domain *d, struct rdtgroup *rdtgrp,
-		    cpumask_t *cpumask, int evtid, int first);
+		    const cpumask_t *cpumask, int evtid, int first);
 
 int resctrl_mon_resource_init(void);
 
arch/x86/kernel/cpu/resctrl/intel_aet.c +53 -0

56 lines not on this page · raw message

fs/resctrl/ctrlmondata.c +21 -2
index d56b78450a99..5612f5f64574 100644
@@ -548,7 +548,7 @@ struct rdt_domain_hdr *resctrl_find_domain(struct list_head *h, int id,
 
 void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
 		    struct rdt_mon_domain *d, struct rdtgroup *rdtgrp,
-		    cpumask_t *cpumask, int evtid, int first)
+		    const cpumask_t *cpumask, int evtid, int first)
 {
 	int cpu;
 
@@ -585,6 +585,21 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
 	resctrl_arch_mon_ctx_free(r, evtid, rr->arch_mon_ctx);
 }
 
+#define NUM_FRAC_BITS	18
+#define FRAC_MASK	GENMASK(NUM_FRAC_BITS - 1, 0)
+
+static void show_fp_value(struct seq_file *m, u64 val)
+{
+	u64 frac;
+
+	frac = val & FRAC_MASK;
+	frac = frac * 1000000;
+	frac += 1ul << (NUM_FRAC_BITS - 1);
+	frac >>= NUM_FRAC_BITS;
+
+	seq_printf(m, "%llu.%06llu\n", val >> NUM_FRAC_BITS, frac);
+}
+
 int rdtgroup_mondata_show(struct seq_file *m, void *arg)
 {
 	struct kernfs_open_file *of = m->private;
@@ -594,6 +609,7 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg)
 	u32 resid, evtid, domid;
 	struct rdtgroup *rdtgrp;
 	struct rdt_resource *r;
+	const cpumask_t *mask;
 	struct mon_data *md;
 	int ret = 0;
 
@@ -642,7 +658,8 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg)
 			goto out;
 		}
 		d = container_of(hdr, struct rdt_mon_domain, hdr);
-		mon_event_read(&rr, r, d, rdtgrp, &d->hdr.cpu_mask, evtid, false);
+		mask = (resid == RDT_RESOURCE_L3) ? &d->hdr.cpu_mask : cpu_online_mask;
+		mon_event_read(&rr, r, d, rdtgrp, mask, evtid, false);
 	}
 
 checkresult:
@@ -651,6 +668,8 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg)
 		seq_puts(m, "Error\n");
 	else if (rr.err == -EINVAL)
 		seq_puts(m, "Unavailable\n");
+	else if (rr.fptype)
+		show_fp_value(m, rr.val);
 	else
 		seq_printf(m, "%llu\n", rr.val);
 
fs/resctrl/monitor.c +20 -3
index 3fe21dcf0fde..f4049ae5344c 100644
@@ -447,6 +447,24 @@ static void mbm_bw_count(u32 closid, u32 rmid, struct rmid_read *rr)
 	m->prev_bw = cur_bw;
 }
 
+static int mon_event_count_one_group(int closid, int rmid, struct rmid_read *rr)
+{
+	bool ret;
+
+	switch (rr->r->rid) {
+	case RDT_RESOURCE_L3:
+		rr->fptype = false;
+		return __mon_event_count(closid, rmid, rr);
+	case RDT_RESOURCE_INTEL_AET:
+		ret = intel_aet_read_event(rr->d->hdr.id, rmid, rr->evtid, &rr->val, &rr->fptype);
+		if (!ret)
+			rr->err = -EINVAL;
+		return ret ? 0 : -EINVAL;
+	}
+
+	return -EINVAL;
+}
+
 /*
  * This is scheduled by mon_event_read() to read the CQM/MBM counters
  * on a domain.
@@ -460,7 +478,7 @@ void mon_event_count(void *info)
 
 	rdtgrp = rr->rgrp;
 
-	ret = __mon_event_count(rdtgrp->closid, rdtgrp->mon.rmid, rr);
+	ret = mon_event_count_one_group(rdtgrp->closid, rdtgrp->mon.rmid, rr);
 
 	/*
 	 * For Ctrl groups read data from child monitor groups and
@@ -471,8 +489,7 @@ void mon_event_count(void *info)
 
 	if (rdtgrp->type == RDTCTRL_GROUP) {
 		list_for_each_entry(entry, head, mon.crdtgrp_list) {
-			if (__mon_event_count(entry->closid, entry->mon.rmid,
-					      rr) == 0)
+			if (mon_event_count_one_group(entry->closid, entry->mon.rmid, rr) == 0)
 				ret = 0;
 		}
 	}
Signature · 1 line
2.48.1
maintainer
#32 lore raw

Re: [PATCH v2 13/16] x86/resctrl: Add code to display core telemetry events

Hi Tony,

(nit: "Add code to" can be dropped from shortlog)

Quoted 9 lines · Tony Luck

These can be read from any CPU. Rely on the smp_call*() functions picking the current CPU when given a free choice from cpu_online_mask.

There may be multiple devices tracking each package, so scan all of them and add up counters.

Output format depends on the data type. Either a 63 bit integer, or a fixed point decimal.

At this point the architecture and fs code is very intertwined. I hope that some of the items I mentioned in earlier patches will help to support a clear separation that will make the code that follows from here on easier to split between arch and fs.
For example, I think this may end up with the new event enums defined in include/linux/resctrl_types.h to support new architectural helpers that take the enum as argument that the fs code can use to request the event value from the architecture.

Reinette

author maintainer
#33 lore raw

Re: [PATCH v2 13/16] x86/resctrl: Add code to display core telemetry events

Quoted 15 lines · Reinette Chatre

Hi Tony,

(nit: "Add code to" can be dropped from shortlog)

Quoted 9 lines · Tony Luck

These can be read from any CPU. Rely on the smp_call*() functions picking the current CPU when given a free choice from cpu_online_mask.

There may be multiple devices tracking each package, so scan all of them and add up counters.

Output format depends on the data type. Either a 63 bit integer, or a fixed point decimal.

Quoted 8 lines

At this point the architecture and fs code is very intertwined. I hope that some of the items I mentioned in earlier patches will help to support a clear separation that will make the code that follows from here on easier to split between arch and fs.
For example, I think this may end up with the new event enums defined in include/linux/resctrl_types.h to support new architectural helpers that take the enum as argument that the fs code can use to request the event value from the architecture.

I have a solution for the separtion for this. Each mon_evt structure gets two new fields.

The first is "bool any_cpu;" if this is set to true the event can be read from any CPU (and we can use James suggestion to pick from "online_cpu_mask" instead of "d->hdr.cpu_mask" and let the optimizations in smp_call*() avoid the IPI.

The second is a "enum format" field that specifies how to display the value returned to FS code from architecture. Existing events all print as a decimal number. I need to add the binary fixed point with 18 binary places to be printed as a floating point number.

Note that this solution needs to copy these fields from the mon_evt structure to the mon_data_bits union (this is easier if James' pending patch to convert to a mon_data structure removes the restriction that all fields fit into 32-bits).

Quoted 2 lines

Reinette

-Tony

author maintainer
#34 lore raw

PATCH v2 14/16 x86/resctrl: Add status files to info/PKG_MON

In reply to Tony Luck

Each of the package event groups includes status about the number of missed events and timestamps (from a 25 MHz clock) on last time an update was missed and last time an update was processed.

Add a three info files to report per-aggregator values of these status values.

---
 include/linux/resctrl.h                 | 12 ++++
 fs/resctrl/internal.h                   |  2 +
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 90 ++++++++++++++++++++++---
 fs/resctrl/rdtgroup.c                   | 21 ++++++
 4 files changed, 117 insertions(+), 8 deletions(-)
include/linux/resctrl.h +12 -0
index e900764393f4..9f6a0b26eec3 100644
@@ -533,10 +533,22 @@ void resctrl_exit(void);
 #ifdef CONFIG_INTEL_AET_RESCTRL
 void rdt_get_intel_aet_mount(void);
 bool intel_aet_read_event(int domid, int rmid, int evtid, u64 *val, bool *fptype);
+int rdtgroup_last_update_show(struct kernfs_open_file *of,
+			      struct seq_file *seq, void *v);
+int rdtgroup_overflows_show(struct kernfs_open_file *of,
+			    struct seq_file *seq, void *v);
+int rdtgroup_overflow_timestamp_show(struct kernfs_open_file *of,
+				     struct seq_file *seq, void *v);
 #else
 static inline void rdt_get_intel_aet_mount(void) { }
 static inline bool intel_aet_read_event(int domid, int rmid, int evtid, u64 *val,
 					bool *fptype) { return false; }
+static inline int rdtgroup_last_update_show(struct kernfs_open_file *of,
+					    struct seq_file *seq, void *v) { return 0; };
+static inline int rdtgroup_overflows_show(struct kernfs_open_file *of,
+					  struct seq_file *seq, void *v) { return 0; };
+static inline int rdtgroup_overflow_timestamp_show(struct kernfs_open_file *of,
+						   struct seq_file *seq, void *v) { return 0; };
 #endif
 
 #endif /* _RESCTRL_H */
fs/resctrl/internal.h +2 -0
index 4d65a781034e..53590b1fa8c3 100644
@@ -236,6 +236,8 @@ struct rdtgroup {
 
 #define RFTYPE_TOP_INFO			(RFTYPE_INFO | RFTYPE_TOP)
 
+#define RFTYPE_PKG_INFO			(RFTYPE_INFO | RFTYPE_RES_PKG)
+
 #define RFTYPE_CTRL_BASE		(RFTYPE_BASE | RFTYPE_CTRL)
 
 #define RFTYPE_MON_BASE			(RFTYPE_BASE | RFTYPE_MON)
arch/x86/kernel/cpu/resctrl/intel_aet.c +82 -8

120 lines not on this page · raw message

fs/resctrl/rdtgroup.c +21 -0
index d3919642aa9b..9021c60fd05b 100644
@@ -1971,6 +1971,27 @@ static struct rftype res_common_files[] = {
 		.seq_show	= rdtgroup_closid_show,
 		.fflags		= RFTYPE_CTRL_BASE | RFTYPE_DEBUG,
 	},
+	{
+		.name		= "last_update",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_last_update_show,
+		.fflags		= RFTYPE_PKG_INFO,
+	},
+	{
+		.name		= "overflows",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_overflows_show,
+		.fflags		= RFTYPE_PKG_INFO,
+	},
+	{
+		.name		= "overflow_timestamp",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_overflow_timestamp_show,
+		.fflags		= RFTYPE_PKG_INFO,
+	},
 };
 
 static int rdtgroup_add_files(struct kernfs_node *kn, unsigned long fflags)
Signature · 1 line
2.48.1
author maintainer
#35 lore raw

PATCH v2 15/16 x86/resctrl: Enable package event monitoring

In reply to Tony Luck

All the code to support this is in place. Unconditionally enable the package resource during early initialization so that domain lists are built.

Enumeration of the package events happens at first mount, if there are no package scoped events, then this resource is disabled.

---
 arch/x86/kernel/cpu/resctrl/intel_aet.c | 2 ++
 1 file changed, 2 insertions(+)
arch/x86/kernel/cpu/resctrl/intel_aet.c +2 -0
index 28692c6ec425..593e838fe5b6 100644
@@ -293,6 +293,8 @@ int rdt_get_intel_aet_mon_config(void)
 
 	INIT_LIST_HEAD(&r->evt_list);
 
+	r->mon_capable = true;
+
 	return 1;
 }
 
Signature · 1 line
2.48.1
author maintainer
#36 lore raw

PATCH v2 16/16 x86/resctrl: Update Documentation for package events

In reply to Tony Luck

Each "mon_data" directory is now divided between L3 events and package events.

---
 Documentation/filesystems/resctrl.rst | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)
Documentation/filesystems/resctrl.rst +16 -9
index 6768fc1fad16..0e915b7c55aa 100644
@@ -366,15 +366,22 @@ When control is enabled all CTRL_MON groups will also contain:
 When monitoring is enabled all MON groups will also contain:
 
 "mon_data":
-	This contains a set of files organized by L3 domain and by
-	RDT event. E.g. on a system with two L3 domains there will
-	be subdirectories "mon_L3_00" and "mon_L3_01".	Each of these
-	directories have one file per event (e.g. "llc_occupancy",
-	"mbm_total_bytes", and "mbm_local_bytes"). In a MON group these
-	files provide a read out of the current value of the event for
-	all tasks in the group. In CTRL_MON groups these files provide
-	the sum for all tasks in the CTRL_MON group and all tasks in
-	MON groups. Please see example section for more details on usage.
+	This contains a set of directories, one for each instance
+	of an L3 cache, or of a processor package. The L3 cache
+	directories are named "mon_L3_00", "mon_L3_01" etc. The
+	package directories "mon_PKG_00", "mon_PKG_01" etc.
+
+	Within each directory there is one file per event. In
+	the L3 directories: "llc_occupancy", "mbm_total_bytes",
+	and "mbm_local_bytes".	In the PKG directories: "core_energy",
+	"activity", etc.
+
+	In a MON group these files provide a read out of the current
+	value of the event for all tasks in the group. In CTRL_MON groups
+	these files provide the sum for all tasks in the CTRL_MON group
+	and all tasks in MON groups. Please see example section for more
+	details on usage.
+
 	On systems with Sub-NUMA Cluster (SNC) enabled there are extra
 	directories for each node (located within the "mon_L3_XX" directory
 	for the L3 cache they occupy). These are named "mon_sub_L3_YY"
Signature · 1 line
2.48.1