perf: qcom: Add L3 cache PMU driver
PATCH v5 1/1 perf: qcom: Add L3 cache PMU driver
This adds a new dynamic PMU to the Perf Events framework to program and control the L3 cache PMUs in some Qualcomm Technologies SOCs.
The driver supports a distributed cache architecture where the overall cache for a socket is comprised of multiple slices each with its own PMU. Access to each individual PMU is provided even though all CPUs share all the slices. User space needs to aggregate to individual counts to provide a global picture.
The driver exports formatting and event information to sysfs so it can be used by the perf user space tools with the syntaxes:
perf stat -a -e l3cache_0_0/read-miss/
perf stat -a -e l3cache_0_0/event=0x21/
--- Documentation/perf/qcom_l3_pmu.txt | 25 ++ drivers/perf/Kconfig | 10 + drivers/perf/Makefile | 1 + drivers/perf/qcom_l3_pmu.c | 824 +++++++++++++++++++++++++++++++++++++ include/linux/cpuhotplug.h | 1 + 5 files changed, 861 insertions(+) create mode 100644 Documentation/perf/qcom_l3_pmu.txt create mode 100644 drivers/perf/qcom_l3_pmu.c
Documentation/perf/qcom_l3_pmu.txt new file +25 -0
@@ -0,0 +1,25 @@ +Qualcomm Datacenter Technologies L3 Cache Performance Monitoring Unit (PMU) +=========================================================================== + +This driver supports the L3 cache PMUs found in Qualcomm Datacenter Technologies +Centriq SoCs. The L3 cache on these SOCs is composed of multiple slices, shared +by all cores within a socket. Each slice is exposed as a separate uncore perf +PMU with device name l3cache_<socket>_<instance>. User space is responsible +for aggregating across slices. + +The driver provides a description of its available events and configuration +options in sysfs, see /sys/devices/l3cache*. Given that these are uncore PMUs +the driver also exposes a "cpumask" sysfs attribute which contains a mask +consisting of one CPU per socket which will be used to handle all the PMU +events on that socket. + +The hardware implements 32bit event counters and has a flat 8bit event space +exposed via the "event" format attribute. In addition to the 32bit physical +counters the driver supports virtual 64bit hardware counters by using hardware +counter chaining. This feature is exposed via the "lc" (long counter) format +flag. E.g.: + + perf stat -e l3cache_0_0/read-miss,lc/ + +Given that these are uncore PMUs the driver does not support sampling, therefore +"perf record" will not work. Per-task perf sessions are not supported.
drivers/perf/Kconfig +10 -0
@@ -21,6 +21,16 @@ config QCOM_L2_PMU Adds the L2 cache PMU into the perf events subsystem for monitoring L2 cache events. +config QCOM_L3_PMU + bool "Qualcomm Technologies L3-cache PMU" + depends on ARCH_QCOM && ARM64 && PERF_EVENTS && ACPI + select QCOM_IRQ_COMBINER + help + Provides support for the L3 cache performance monitor unit (PMU) + in Qualcomm Technologies processors. + Adds the L3 cache PMU into the perf events subsystem for + monitoring L3 cache events. + config XGENE_PMU depends on PERF_EVENTS && ARCH_XGENE bool "APM X-Gene SoC PMU"
drivers/perf/Makefile +1 -0
@@ -1,3 +1,4 @@ obj-$(CONFIG_ARM_PMU) += arm_pmu.o obj-$(CONFIG_QCOM_L2_PMU) += qcom_l2_pmu.o +obj-$(CONFIG_QCOM_L3_PMU) += qcom_l3_pmu.o obj-$(CONFIG_XGENE_PMU) += xgene_pmu.o
drivers/perf/qcom_l3_pmu.c new file +824 -0
824 lines not on this page · raw message
include/linux/cpuhotplug.h +1 -0
@@ -137,6 +137,7 @@ enum cpuhp_state {CPUHP_AP_PERF_ARM_CCN_ONLINE ,CPUHP_AP_PERF_ARM_L2X0_ONLINE ,CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE , +CPUHP_AP_PERF_ARM_QCOM_L3_ONLINE ,CPUHP_AP_WORKQUEUE_ONLINE ,CPUHP_AP_RCUTREE_ONLINE , CPUHP_AP_ONLINE_DYN,
Signature · 2 lines
Qualcomm Datacenter Technologies, Inc. on behalf of the Qualcomm Technologies, Inc. Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
Mark Rutland
maintainerHi Agustin,
Quoted 9 lines · Agustin Vega-Frias
This adds a new dynamic PMU to the Perf Events framework to program and control the L3 cache PMUs in some Qualcomm Technologies SOCs.
The driver supports a distributed cache architecture where the overall cache for a socket is comprised of multiple slices each with its own PMU. Access to each individual PMU is provided even though all CPUs share all the slices. User space needs to aggregate to individual counts to provide a global picture.
Quoted 6 lines
The driver exports formatting and event information to sysfs so it can be used by the perf user space tools with the syntaxes:
perf stat -a -e l3cache_0_0/read-miss/ perf stat -a -e l3cache_0_0/event=0x21/Signed-off-by Agustin Vega-Frias <[email protected]>
Thanks for respinning this; it looks good to me.
Quoted 9 lines
drivers/perf/qcom_l3_pmu.c new file +8
-0+static int qcom_l3_cache__event_init (struct perf_event *event) +{ +struct l3cache_pmu *l3pmu =to_l3cache_pmu (event->pmu); +struct hw_perf_event *hwc = &event->hw; +struct perf_event *sibling; +/* +* We must NOT create groups containing mixed PMUs, although software +* events are acceptable
drivers/perf/qcom_l3_pmu.c line 493
Quoted 10 lines
drivers/perf/qcom_l3_pmu.c new file +10
-0+*/ +if (event->group_leader->pmu != event->pmu && + !is_software_event (event->group_leader)) +return -EINVAL ; + +list_for_each_entry (sibling, &event->group_leader->sibling_list, + group_entry) +if (sibling->pmu != event->pmu && + !is_software_event (sibling)) +return -EINVAL ;
Sorry for spotting this so late, but I just noticed that we don't validate the group isn't too large to ever fit in the HW, which is important for avoiding pointless work in perf_rotate_context() when the mux hrtimer callback occurs.
We fail to do that in other drivers, too, so that's something we should clean up more generally.
Here, I'd like to factor the group validation logic out into a helper:
#define event_num_counters(e) (event_uses_long_counter(e) ? 2 : 1)
static bool qcom_l3_cache__validate_event_group(struct perf_event *event)
{
struct perf_event *leader = event->group_leader;
struct perf_event *sibling;
int counters = 0;
/*
* We must NOT create groups containing mixed PMUs, although
* software.
*/
if (leader->pmu != event->pmu && !is_software_event(leader))
return false;
counters = event_num_counters(event);
counters += event_num_counters(leader);
list_for_each_entry(sibling, &leader->sibling_list, group_entry) {
if (is_software_event(sibling))
continue;
if (sibling->pmu != event->pmu)
return false;
counters += event_num_counters(sibling);
}
/*
* If the group requires more counters than the HW has, it
* cannot ever be scheduled.
*/
return counters <= L3_NUM_COUNTERS;
}... where in qcom_l3_cache__event_init() we'd do:
if (!qcom_l3_cache__validate_event_group(event)) return -EINVAL;
If you're happy with that, I can make that change when applying.
Thanks,
Mark.
Hey Mark,
Quoted 89 lines · Mark Rutland
Hi Agustin,
Quoted 12 lines · Agustin Vega-Frias
This adds a new dynamic PMU to the Perf Events framework to program and control the L3 cache PMUs in some Qualcomm Technologies SOCs.
The driver supports a distributed cache architecture where the overall cache for a socket is comprised of multiple slices each with its own PMU.
Access to each individual PMU is provided even though all CPUs share all
the slices. User space needs to aggregate to individual counts to provide
a global picture.Quoted 6 lines
The driver exports formatting and event information to sysfs so it can be used by the perf user space tools with the syntaxes:
perf stat -a -e l3cache_0_0/read-miss/ perf stat -a -e l3cache_0_0/event=0x21/Signed-off-by Agustin Vega-Frias <[email protected]>Thanks for respinning this; it looks good to me.
Quoted 10 lines
drivers/perf/qcom_l3_pmu.c new file +7
-0+static int qcom_l3_cache__event_init (struct perf_event *event) +{ +struct l3cache_pmu *l3pmu =to_l3cache_pmu (event->pmu); +struct hw_perf_event *hwc = &event->hw; +struct perf_event *sibling; +/* +* We must NOT create groups containing mixed PMUs, although software
drivers/perf/qcom_l3_pmu.c new file +1
-0+* events are acceptable drivers/perf/qcom_l3_pmu.c line 493
Quoted 10 lines
drivers/perf/qcom_l3_pmu.c new file +10
-0+*/ +if (event->group_leader->pmu != event->pmu && + !is_software_event (event->group_leader)) +return -EINVAL ; + +list_for_each_entry (sibling, &event->group_leader->sibling_list, + group_entry) +if (sibling->pmu != event->pmu && + !is_software_event (sibling)) +return -EINVAL ;Sorry for spotting this so late, but I just noticed that we don't validate the group isn't too large to ever fit in the HW, which is important for avoiding pointless work in
perf_rotate_context()when the mux hrtimer callback occurs.We fail to do that in other drivers, too, so that's something we should clean up more generally.
Here, I'd like to factor the group validation logic out into a helper:
#define
event_num_counters(e)(event_uses_long_counter(e)? 2 : 1)static bool
qcom_l3_cache__validate_event_group(struct perf_event *event){ struct perf_event *leader = event->group_leader; struct perf_event *sibling; int counters = 0; /* * We must NOT create groups containing mixed PMUs, although * software. */ if (leader->pmu != event->pmu && !is_software_event(leader)) return false; counters = event_num_counters(event); counters += event_num_counters(leader); list_for_each_entry(sibling, &leader->sibling_list, group_entry) { if (is_software_event(sibling)) continue; if (sibling->pmu != event->pmu) return false; counters += event_num_counters(sibling); } /* * If the group requires more counters than the HW has, it * cannot ever be scheduled. */ return counters <= L3_NUM_COUNTERS; }
Quoted 6 lines
... where in
qcom_l3_cache__event_init()we'd do:if (!qcom_l3_cache__validate_event_group(event)) return -EINVAL;If you're happy with that, I can make that change when applying.
Sounds good. I'll also apply this change internally and test it out. I'll send it as V6 if I get it ready today.
Thanks,
Agustin
Signature · 4 lines
Qualcomm Datacenter Technologies, Inc. on behalf of the Qualcomm Technologies, Inc. Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.