Skip to contentSkip to conversation

perf: add qcom l2 cache perf events driver

PATCH In review Neil Leeder 2 participants 2 messages Feb 7, 2017

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH v10 1/1 perf: add qcom l2 cache perf events driver

Adds perf events support for L2 cache PMU.

The L2 cache PMU driver is named 'l2cache_0' and can be used with perf events to profile L2 events such as cache hits and misses on Qualcomm Technologies processors.

---
v10:
Remove unnecessary cross-call for reset Remove filter match and move check in get_event_idx Check event pointer is valid in interrupt handler

v9:
Add support for maxcpus < all cpus:
Move some of probe functionality to hotplug online callback Call hotplug callbacks from probe
Make percpu cpu->cluster map dynamic and associated with PMU instead of global Use *sysreg_s() for portability reasons
Add documentation file
Rebase on 4.10-rc6
Misc cleanups

v8:
Various style changes for function names & code restructuring Replace dev_warn with ratelimited debug prints
Move hotplug registration before PMU registration Reload counters with a fixed value
Add column-exclusion check for events in same group Rebase on 4.10-rc3

v7:
Move to drivers/perf
Rebased on 4.9-rc1
Ran perf fuzzer against driver for 4 hours with no crashes

v6: restore accidentally dropped Kconfig dependencies

v5:
Fold the header and l2-accessors into .c file Use multi-instance framework for hotplug Change terminology from slice to cluster for clarity Remove unnecessary rmw sequence for enable registers Use prev_count in hwc rather than in slice Enforce all events in same group on same CPU Add comments, rename variables for clarity

v4:
Replace notifier with hotplug statemachine
Allocate PMU struct dynamically

v3:
Remove exports from l2-accessors
Change l2-accessors Kconfig to make it not user-selectable Reorder and remove unnecessary includes

v2:
Add the l2-accessors patch to this patchset, previously posted separately. Remove sampling and per-task functionality for this uncore PMU. Use cpumask to replace code which filtered events to one cpu per slice. Replace manual event filtering with filter_match callback.
Use a separate used_mask for event groups.
Add hotplug notifier for CPU and irq migration.
Remove extraneous synchronisation instructions.
Other miscellaneous cleanup.

 Documentation/perf/qcom_l2_pmu.txt |   38 ++
 drivers/perf/Kconfig               |    9 +
 drivers/perf/Makefile              |    1 +
 drivers/perf/qcom_l2_pmu.c         | 1012 ++++++++++++++++++++++++++++++++++++
 include/linux/cpuhotplug.h         |    1 +
 5 files changed, 1061 insertions(+)
 create mode 100644 Documentation/perf/qcom_l2_pmu.txt
 create mode 100644 drivers/perf/qcom_l2_pmu.c
Documentation/perf/qcom_l2_pmu.txt new file +38 -0
new file mode 100644
index 0000000..b25b976
@@ -0,0 +1,38 @@
+Qualcomm Technologies Level-2 Cache Performance Monitoring Unit (PMU)
+=====================================================================
+
+This driver supports the L2 cache clusters found in Qualcomm Technologies
+Centriq SoCs. There are multiple physical L2 cache clusters, each with their
+own PMU. Each cluster has one or more CPUs associated with it.
+
+There is one logical L2 PMU exposed, which aggregates the results from
+the physical PMUs.
+
+The driver provides a description of its available events and configuration
+options in sysfs, see /sys/devices/l2cache_0.
+
+The "format" directory describes the format of the events.
+
+Events can be envisioned as a 2-dimensional array. Each column represents
+a group of events. There are 8 groups. Only one entry from each
+group can be in use at a time. If multiple events from the same group
+are specified, the conflicting events cannot be counted at the same time.
+
+Events are specified as 0xCCG, where CC is 2 hex digits specifying
+the code (array row) and G specifies the group (column) 0-7.
+
+In addition there is a cycle counter event specified by the value 0xFE
+which is outside the above scheme.
+
+The driver provides a "cpumask" sysfs attribute which contains a mask
+consisting of one CPU per cluster which will be used to handle all the PMU
+events on that cluster.
+
+Examples for use with perf:
+
+  perf stat -e l2cache_0/config=0x001/,l2cache_0/config=0x042/ -a sleep 1
+
+  perf stat -e l2cache_0/config=0xfe/ -C 2 sleep 1
+
+The driver does not support sampling, therefore "perf record" will
+not work. Per-task perf sessions are not supported.
drivers/perf/Kconfig +9 -0
index 4d5c5f9..9365190 100644
@@ -12,6 +12,15 @@ config ARM_PMU
 	  Say y if you want to use CPU performance monitors on ARM-based
 	  systems.
 
+config QCOM_L2_PMU
+	bool "Qualcomm Technologies L2-cache PMU"
+	depends on ARCH_QCOM && ARM64 && PERF_EVENTS && ACPI
+	  help
+	  Provides support for the L2 cache performance monitor unit (PMU)
+	  in Qualcomm Technologies processors.
+	  Adds the L2 cache PMU into the perf events subsystem for
+	  monitoring L2 cache events.
+
 config XGENE_PMU
         depends on PERF_EVENTS && ARCH_XGENE
         bool "APM X-Gene SoC PMU"
drivers/perf/Makefile +1 -0
index b116e98..ef24833 100644
@@ -1,2 +1,3 @@
 obj-$(CONFIG_ARM_PMU) += arm_pmu.o
+obj-$(CONFIG_QCOM_L2_PMU)	+= qcom_l2_pmu.o
 obj-$(CONFIG_XGENE_PMU) += xgene_pmu.o
drivers/perf/qcom_l2_pmu.c new file +1012 -0

1,012 lines not on this page · raw message

include/linux/cpuhotplug.h +1 -0
index d936a00..b7d100e 100644
@@ -140,6 +140,7 @@ enum cpuhp_state {
 	CPUHP_AP_PERF_ARM_CCI_ONLINE,
 	CPUHP_AP_PERF_ARM_CCN_ONLINE,
 	CPUHP_AP_PERF_ARM_L2X0_ONLINE,
+	CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE,
 	CPUHP_AP_WORKQUEUE_ONLINE,
 	CPUHP_AP_RCUTREE_ONLINE,
 	CPUHP_AP_ONLINE_DYN,
Signature · 3 lines
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.
maintainer Reviewed-by on v10 1/1
#1 lore raw

Hi Neil,

Quoted 7 lines · Neil Leeder

Adds perf events support for L2 cache PMU.

The L2 cache PMU driver is named 'l2cache_0' and can be used with perf events to profile L2 events such as cache hits and misses on Qualcomm Technologies processors.

Thanks for respinning this. This looks good to me now:

Will and I should be able to pick this up shortly.

There's one minor thing I'd like to clean up below, but we can sort that out when applying -- there's no need to respin.

Quoted 18 lines
drivers/perf/qcom_l2_pmu.c new file +18 -0
+static struct cluster_pmu *l2_cache_associate_cpu_with_cluster(
+	struct l2cache_pmu *l2cache_pmu, int cpu)
+{
+	u64 mpidr;
+	int cpu_cluster_id;
+	struct cluster_pmu *cluster;
+
+	/*
+	 * This assumes that the cluster_id is in MPIDR[aff1] for
+	 * single-threaded cores, and MPIDR[aff2] for multi-threaded
+	 * cores. This logic will have to be updated if this changes.
+	 */
+	mpidr = read_cpuid_mpidr();
+	if (mpidr & MPIDR_MT_BITMASK)
+		cpu_cluster_id = MPIDR_AFFINITY_LEVEL(mpidr, 2);
+	else
+		cpu_cluster_id = MPIDR_AFFINITY_LEVEL(mpidr, 1);
+

drivers/perf/qcom_l2_pmu.c line 767

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+	list_for_each_entry(cluster, &l2cache_pmu->clusters, next) {
+		if (cluster->cluster_id == cpu_cluster_id) {
+			dev_info(&l2cache_pmu->pdev->dev,
+				 "CPU%d associated with cluster %d\n", cpu,
+				 cluster->cluster_id);
+			cpumask_set_cpu(cpu, &cluster->cluster_cpus);
+			*per_cpu_ptr(l2cache_pmu->pmu_cluster, cpu) = cluster;
+			return cluster;
+		}
+	}

To minimise nesting, I'd like to fix this up as:

	list_for_each_entry(cluster, &l2cache_pmu->clusters, next) {
		if (cluster->cluster_id != cpu_cluster_id)
			continue;

		dev_info(&l2cache_pmu->pdev->dev,
			 "CPU%d associated with cluster %d\n", cpu,
			 cluster->cluster_id);
		cpumask_set_cpu(cpu, &cluster->cluster_cpus);
		*per_cpu_ptr(l2cache_pmu->pmu_cluster, cpu) = cluster;
		return cluster;
	}

Regardless, this is fine by me.

Thanks,
Mark.