Skip to contentSkip to conversation

soc: qcom: add l2 cache perf events driver

PATCH Superseded Neil Leeder 3 participants 7 messages Oct 28, 2016 – Nov 16, 2016

Subsystems (3)

v7 of 10 Superseded Latest is v10 →

lore
Share & tools
lore
#0 lore raw

PATCH v7 1/1 soc: qcom: add l2 cache perf events driver

Adds perf events support for L2 cache PMU.

The L2 cache PMU driver is named 'l2cache_0' and can be used with perf events to profile L2 events such as cache hits and misses on Qualcomm Technologies processors.

---
v7:
Move to drivers/perf
Rebased on 4.9-rc1
Successfully ran perf fuzzer against driver

v6: restore accidentally dropped Kconfig dependencies

v5:
Fold the header and l2-accessors into .c file Use multi-instance framework for hotplug Change terminology from slice to cluster for clarity Remove unnecessary rmw sequence for enable registers Use prev_count in hwc rather than in slice Enforce all events in same group on same CPU Add comments, rename variables for clarity

v4:
Replace notifier with hotplug statemachine
Allocate PMU struct dynamically

v3:
Remove exports from l2-accessors
Change l2-accessors Kconfig to make it not user-selectable Reorder and remove unnecessary includes

v2:
Add the l2-accessors patch to this patchset, previously posted separately. Remove sampling and per-task functionality for this uncore PMU. Use cpumask to replace code which filtered events to one cpu per slice. Replace manual event filtering with filter_match callback.
Use a separate used_mask for event groups.
Add hotplug notifier for CPU and irq migration.
Remove extraneous synchronisation instructions.
Other miscellaneous cleanup.

 drivers/perf/Kconfig       |   9 +
 drivers/perf/Makefile      |   1 +
 drivers/perf/qcom_l2_pmu.c | 947 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/cpuhotplug.h |   1 +
 4 files changed, 958 insertions(+)
 create mode 100644 drivers/perf/qcom_l2_pmu.c
drivers/perf/Kconfig +9 -0
index 4d5c5f9..9365190 100644
@@ -12,6 +12,15 @@ config ARM_PMU
 	  Say y if you want to use CPU performance monitors on ARM-based
 	  systems.
 
+config QCOM_L2_PMU
+	bool "Qualcomm Technologies L2-cache PMU"
+	depends on ARCH_QCOM && ARM64 && PERF_EVENTS && ACPI
+	  help
+	  Provides support for the L2 cache performance monitor unit (PMU)
+	  in Qualcomm Technologies processors.
+	  Adds the L2 cache PMU into the perf events subsystem for
+	  monitoring L2 cache events.
+
 config XGENE_PMU
         depends on PERF_EVENTS && ARCH_XGENE
         bool "APM X-Gene SoC PMU"
drivers/perf/Makefile +1 -0
index b116e98..ef24833 100644
@@ -1,2 +1,3 @@
 obj-$(CONFIG_ARM_PMU) += arm_pmu.o
+obj-$(CONFIG_QCOM_L2_PMU)	+= qcom_l2_pmu.o
 obj-$(CONFIG_XGENE_PMU) += xgene_pmu.o
drivers/perf/qcom_l2_pmu.c new file +947 -0

947 lines not on this page · raw message

include/linux/cpuhotplug.h +1 -0
index 9b207a8..fbe33d2 100644
@@ -117,6 +117,7 @@ enum cpuhp_state {
 	CPUHP_AP_PERF_ARM_CCI_ONLINE,
 	CPUHP_AP_PERF_ARM_CCN_ONLINE,
 	CPUHP_AP_PERF_ARM_L2X0_ONLINE,
+	CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE,
 	CPUHP_AP_WORKQUEUE_ONLINE,
 	CPUHP_AP_RCUTREE_ONLINE,
 	CPUHP_AP_NOTIFY_ONLINE,
Signature · 3 lines
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.
maintainer
#1 lore raw

Hi Neil,

Apologies for the delay in replying to this.

This is looking good. I have a few specific comments and a couple of general concerns below.

Will, please see the bit below about cluster/socket aggregation (grep for your name).

drivers/perf/qcom_l2_pmu.c line 15

Quoted 4 lines · Neil Leeder
drivers/perf/qcom_l2_pmu.c new file +4 -0
+#include <linux/acpi.h>
+#include <linux/interrupt.h>
+#include <linux/perf_event.h>
+#include <linux/platform_device.h>

While scanning the driver. I noticed a number of other headers that are required for things the driver explicitly references. Please add the following so that we're not relying on (fragile) transitive includes:

#include <linux/bitops.h>
#include <linux/bug.h>
#include <linux/cpuhotplug.h>
#include <linux/cpumask.h>
#include <linux/device.h>
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/percpu.h>
#include <linux/smp.h>
#include <linux/spinlock.h>
#include <linux/sysfs.h>
#include <linux/types.h>

#include <asm/barrier.h>
#include <asm/local64.h>
#include <asm/sysreg.h>

[...]

Quoted 12 lines
drivers/perf/qcom_l2_pmu.c new file +12 -0
+/*
+ * Aggregate PMU. Implements the core pmu functions and manages
+ * the hardware PMUs.
+ */
+struct l2cache_pmu {
+	struct hlist_node node;
+	u32 num_pmus;
+	struct pmu pmu;
+	int num_counters;
+	cpumask_t cpumask;
+	struct platform_device *pdev;
+};

drivers/perf/qcom_l2_pmu.c line 141

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+
+/*
+ * The cache is made up of one or more clusters, each cluster has its own PMU.
+ * Each cluster is associated with one or more CPUs.
+ * This structure represents one of the hardware PMUs.
+ *
+ * Events can be envisioned as a 2-dimensional array. Each column represents
+ * a group of events. There are 8 groups. Only one entry from each
+ * group can be in use at a time. When an event is assigned a counter
+ * by *_event_add(), the counter index is assigned to group_to_counter[group].

If I've followed correctly, this means each group has a dedicated counter?

I take it groups are not symmetric (i.e. each column has different events)? Or does each column contain the same events?

Is there any overlap?

Quoted 9 lines

+ * This allows *filter_match() to detect and reject conflicting events in
+ * the same group.
+ * Events are specified as 0xCCG, where CC is 2 hex digits specifying
+ * the code (array row) and G specifies the group (column).
+ *
+ * In addition there is a cycle counter event specified by L2CYCLE_CTR_RAW_CODE
+ * which is outside the above scheme.
+ */
+struct hml2_pmu {

It's not clear to me what "hml2" is, and now we seem to use "cluster" and "hml2" interchangeably in comments and code. Can you please define what "hml2" is in a comment, mentioning that we refer to this (or its container?) as a cluster?

Can we also please s/hml2/cluster/ in the code, for consistency?

drivers/perf/qcom_l2_pmu.c line 160

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+	struct perf_event *events[MAX_L2_CTRS];
+	struct l2cache_pmu *l2cache_pmu;
+	DECLARE_BITMAP(used_counters, MAX_L2_CTRS);
+	DECLARE_BITMAP(used_groups, L2_EVT_GROUP_MAX + 1);
+	int group_to_counter[L2_EVT_GROUP_MAX + 1];
+	int irq;
+	/* The CPU that is used for collecting events on this cluster */
+	int on_cpu;
+	/* All the CPUs associated with this cluster */
+	cpumask_t cluster_cpus;

I'm still uncertain about aggregating all cluster PMUs into a larger PMU, given the cluster PMUs are logically independent (at least in terms of the programming model).

However, from what I understand the x86 uncore PMU drivers aggregate symmetric instances of uncore PMUs (and also aggregate across packages to the same logical PMU).

Whatever we do, it would be nice for the uncore drivers to align on a common behaviour (and I think we're currently going the oppposite route with Cavium's uncore PMU). Will, thoughts?

Quoted 2 lines
drivers/perf/qcom_l2_pmu.c new file +2 -0
+	spinlock_t pmu_lock;
+};

[...]

drivers/perf/qcom_l2_pmu.c line 287

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+static void hml2_pmu__set_resr(struct hml2_pmu *cluster,
+			       u32 event_group, u32 event_cc)
+{
+	u64 field;
+	u64 resr_val;
+	u32 shift;
+	unsigned long flags;
+
+	shift = L2PMRESR_GROUP_BITS * event_group;
+	field = ((u64)(event_cc & L2PMRESR_GROUP_MASK) << shift) | L2PMRESR_EN;

The L2PMRESR_EN isn't part of the shifted field...

drivers/perf/qcom_l2_pmu.c line 293

Quoted 6 lines
drivers/perf/qcom_l2_pmu.c new file +6 -0
+
+	spin_lock_irqsave(&cluster->pmu_lock, flags);
+
+	resr_val = get_l2_indirect_reg(L2PMRESR);
+	resr_val &= ~(L2PMRESR_GROUP_MASK << shift);
+	resr_val |= field;

... so can we please or that in on a separate line here, e.g.

	resr_val |= L2PMRESR_EN;

That said, do we need to set this bit each time we program an event?

Quoted 4 lines
drivers/perf/qcom_l2_pmu.c new file +4 -0
+	set_l2_indirect_reg(L2PMRESR, resr_val);
+
+	spin_unlock_irqrestore(&cluster->pmu_lock, flags);
+}

[...]

Quoted 19 lines
drivers/perf/qcom_l2_pmu.c new file +19 -0
+static void l2_cache__event_update_from_cluster(struct perf_event *event,
+						struct hml2_pmu *cluster)
+{
+	struct hw_perf_event *hwc = &event->hw;
+	u64 delta64, prev, now;
+	u32 delta;
+	u32 idx = hwc->idx;
+
+	do {
+		prev = local64_read(&hwc->prev_count);
+		now = hml2_pmu__counter_get_value(idx);
+	} while (local64_cmpxchg(&hwc->prev_count, prev, now) != prev);
+
+	if (idx == l2_cycle_ctr_idx) {
+		/*
+		 * The cycle counter is 64-bit so needs separate handling
+		 * of 64-bit delta.
+		 */
+		delta64 = now - prev;

drivers/perf/qcom_l2_pmu.c line 360

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+		local64_add(delta64, &event->count);
+	} else {
+		/*
+		 * 32-bit counters need the unsigned 32-bit math to handle
+		 * overflow and now < prev
+		 */
+		delta = now - prev;
+		local64_add(delta, &event->count);
+	}
+}

I believe you can get rid of the delta/delta64 split with something like the below (with a 64-bit delta):

	/*
	 * The cycle counter is 64-bit, but all other counters are
	 * 32-bit, and we must handle 32-bit overflow explicitly.
	 */
	delta = now - prev;
	if (idx != l2_cycle_ctr_idx)
		delta &= 0xffffffff;
	
	local64_add(delta, &event->count);

drivers/perf/qcom_l2_pmu.c line 365

Quoted 5 lines
drivers/perf/qcom_l2_pmu.c new file +5 -0
+
+static void l2_cache__cluster_set_period(struct hml2_pmu *cluster,
+				       struct hw_perf_event *hwc)
+{
+	u64 base = L2_MAX_PERIOD - (L2_CNT_PERIOD - 1);

Is L2_CNT_PERIOD a hw-derived value? Or just something that happened to be small enough to avoid overflow in testing?

Quoted 14 lines
drivers/perf/qcom_l2_pmu.c new file +14 -0
+	u32 idx = hwc->idx;
+	u64 prev = local64_read(&hwc->prev_count);
+	u64 value;
+
+	/*
+	 * Limit the maximum period to prevent the counter value
+	 * from overtaking the one we are about to program.
+	 * Use a starting value which is high enough that after
+	 * an overflow, interrupt latency will not cause the count
+	 * to reach the base value. If the previous value
+	 * is below the base, increase it to be above the base
+	 * and update prev_count accordingly. Otherwise if
+	 * the previous value is already above the base
+	 * nothing needs to be done to prev_count.

drivers/perf/qcom_l2_pmu.c line 389

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+	 */
+	if (prev < base) {
+		value = base + prev;
+		local64_set(&hwc->prev_count, value);
+	} else {
+		value = prev;
+	}
+
+	hml2_pmu__counter_set_value(idx, value);
+}

Given that we don't do sampling, and are only using the interrupt to prevent losing events in the case of overflow, I think we can use a constant value to handle all cases.

Additionally, I think we need to handle the cycle counter separately, since it has a different width. Something like the below (perhaps with different constants):

static void l2_cache__cluster_set_period(struct hml2_pmu *cluster,
struct hw_perf_event *hwc)

{
	u64 new;
	
	/*
	 * We limit the max period to half of the max counter value so
	 * that even in the case of extreme interrupt latency the
	 * counter will (hopefully) not wrap past its initial value.
	 */
	if (hwc->idx == l2_cycle_ctr_idx)
		new = BIT_ULL(63);
	else
		new = BIT_ULL(31); 
	
	local64_set(&hwc->prev_count, new);

	hml2_pmu__counter_set_value(idx, new);
}
Quoted 13 lines
drivers/perf/qcom_l2_pmu.c new file +13 -0
+static int l2_cache__get_event_idx(struct hml2_pmu *cluster,
+				   struct perf_event *event)
+{
+	struct hw_perf_event *hwc = &event->hw;
+	int idx;
+
+	if (hwc->config_base == L2CYCLE_CTR_RAW_CODE) {
+		if (test_and_set_bit(l2_cycle_ctr_idx, cluster->used_counters))
+			return -EAGAIN;
+
+		return l2_cycle_ctr_idx;
+	}
+

drivers/perf/qcom_l2_pmu.c line 414

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+	for (idx = 0; idx < cluster->l2cache_pmu->num_counters - 1; idx++) {
+		if (!test_and_set_bit(idx, cluster->used_counters)) {
+			set_bit(L2_EVT_GROUP(hwc->config_base),
+				cluster->used_groups);
+			return idx;
+		}
+	}
+	/* The counters are all in use. */
+	return -EAGAIN;
+}

Are we racing with another bitmap manipulation here? Or can we split the find and set of the bit, i.e.

	int num_ctrs = cluster->l2cache_pmu->num_counters;

	int idx = find_first_zero_bit(cluster->used_counters, num_ctrs);
	if (idx == num_ctrs)
		return -EAGAIN;
	set_bit(idx, cluster->used_counters)

Otherwise, I think the outer loop can use for_each_clear_bit().

[...]

Quoted 6 lines
drivers/perf/qcom_l2_pmu.c new file +6 -0
+static int l2_cache__event_init(struct perf_event *event)
+{
+	struct hw_perf_event *hwc = &event->hw;
+	struct hml2_pmu *cluster;
+	struct perf_event *sibling;
+	struct l2cache_pmu *l2cache_pmu;

drivers/perf/qcom_l2_pmu.c line 492

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+
+	if (event->attr.type != event->pmu->type)
+		return -ENOENT;
+
+	l2cache_pmu = to_l2cache_pmu(event->pmu);
+
+	if (hwc->sample_period) {
+		dev_warn(&l2cache_pmu->pdev->dev, "Sampling not supported\n");
+		return -EOPNOTSUPP;
+	}

Please remove these dev_warns, or at lease relegate them to ratelimited debug messages rather than warning messages. Users can easily trigger these by accident and/or when probing capabilities in the usual fashion, and there's no need for them.

The CCN driver is an unfortunate bad example in this respect.

[...]

Quoted 16 lines
drivers/perf/qcom_l2_pmu.c new file +16 -0
+	/* Don't allow groups with mixed PMUs, except for s/w events */
+	if (event->group_leader->pmu != event->pmu &&
+	    !is_software_event(event->group_leader)) {
+		dev_warn(&l2cache_pmu->pdev->dev,
+			 "Can't create mixed PMU group\n");
+		return -EINVAL;
+	}
+
+	list_for_each_entry(sibling, &event->group_leader->sibling_list,
+			    group_entry)
+		if (sibling->pmu != event->pmu &&
+		    !is_software_event(sibling)) {
+			dev_warn(&l2cache_pmu->pdev->dev,
+				 "Can't create mixed PMU group\n");
+			return -EINVAL;
+		}

drivers/perf/qcom_l2_pmu.c line 539

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+
+	/* Ensure all events in a group are on the same cpu */
+	cluster = get_hml2_pmu(event->cpu);
+	if ((event->group_leader != event) &&
+	    (cluster->on_cpu != event->group_leader->cpu)) {
+		dev_warn(&l2cache_pmu->pdev->dev,
+			 "Can't create group on CPUs %d and %d",
+			 event->cpu, event->group_leader->cpu);
+		return -EINVAL;
+	}

It's probably worth also checking that the events are co-schedulable (e.g. they don't conflict column-wise).

[...]

Quoted 5 lines
drivers/perf/qcom_l2_pmu.c new file +5 -0
+static void l2_cache__event_start(struct perf_event *event, int flags)
+{
+	struct hml2_pmu *cluster;
+	struct hw_perf_event *hwc = &event->hw;
+	int idx = hwc->idx;

drivers/perf/qcom_l2_pmu.c line 567

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+	u32 config;
+	u32 event_cc, event_group;
+
+	hwc->state = 0;
+
+	cluster = get_hml2_pmu(event->cpu);
+	l2_cache__cluster_set_period(cluster, hwc);
+
+	if (hwc->config_base == L2CYCLE_CTR_RAW_CODE) {
+		hml2_pmu__set_evccntcr(0x0);

Nit: s/0x0/0/ -- there's no other hex usage to be consistent with, and a single digit is clearer.

drivers/perf/qcom_l2_pmu.c line 573

Quoted 6 lines
drivers/perf/qcom_l2_pmu.c new file +6 -0
+	} else {
+		config = hwc->config_base;
+		event_cc    = L2_EVT_CODE(config);
+		event_group = L2_EVT_GROUP(config);
+
+		hml2_pmu__set_evcntcr(idx, 0x0);

Nit: s/0x0/0/ -- as above.

Quoted 6 lines
drivers/perf/qcom_l2_pmu.c new file +6 -0
+		hml2_pmu__set_evtyper(idx, event_group);
+		hml2_pmu__set_resr(cluster, event_group, event_cc);
+		hml2_pmu__set_evfilter_sys_mode(idx);
+	}
+
+	hml2_pmu__counter_enable_interrupt(idx);

drivers/perf/qcom_l2_pmu.c line 589

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+	hml2_pmu__counter_enable(idx);
+}
+
+static void l2_cache__event_stop(struct perf_event *event, int flags)
+{
+	struct hml2_pmu *cluster;
+	struct hw_perf_event *hwc = &event->hw;
+	int idx = hwc->idx;
+
+	if (!(hwc->state & PERF_HES_STOPPED)) {

It's better to have an early return, e.g.

	/* If already stopped, we have nothing to do */
	if (hwc->state & PERF_HES_STOPPED)
		return;

	...
	remaining logic here, not in a block
	...
Quoted 14 lines
drivers/perf/qcom_l2_pmu.c new file +14 -0
+		cluster = get_hml2_pmu(event->cpu);
+		hml2_pmu__counter_disable_interrupt(idx);
+		hml2_pmu__counter_disable(idx);
+
+		if (flags & PERF_EF_UPDATE)
+			l2_cache__event_update_from_cluster(event, cluster);
+		hwc->state |= PERF_HES_STOPPED | PERF_HES_UPTODATE;
+	}
+}
+
+static int l2_cache__event_add(struct perf_event *event, int flags)
+{
+	struct hw_perf_event *hwc = &event->hw;
+	int idx;

drivers/perf/qcom_l2_pmu.c line 613

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+	int err = 0;
+	struct hml2_pmu *cluster;
+
+	cluster = get_hml2_pmu(event->cpu);
+
+	idx = l2_cache__get_event_idx(cluster, event);
+	if (idx < 0) {
+		err = idx;
+		return err;
+	}

You can return idx directly and avoid the block here.

drivers/perf/qcom_l2_pmu.c line 619

Quoted 5 lines
drivers/perf/qcom_l2_pmu.c new file +5 -0
+	hwc->idx = idx;
+	hwc->state = PERF_HES_STOPPED | PERF_HES_UPTODATE;
+	cluster->events[idx] = event;
+	cluster->group_to_counter[L2_EVT_GROUP(hwc->config_base)] = idx;
+	local64_set(&hwc->prev_count, 0ULL);

Nit: s/0ULL/0/

[...]

drivers/perf/qcom_l2_pmu.c line 661

Quoted 13 lines
drivers/perf/qcom_l2_pmu.c new file +13 -0
+static int l2_cache_filter_match(struct perf_event *event)
+{
+	struct hw_perf_event *hwc = &event->hw;
+	struct hml2_pmu *cluster = get_hml2_pmu(event->cpu);
+	unsigned int group = L2_EVT_GROUP(hwc->config_base);
+
+	/* check for column exclusion: group already in use by another event */
+	if (test_bit(group, cluster->used_groups) &&
+	    cluster->events[cluster->group_to_counter[group]] != event)
+		return 0;
+
+	return 1;
+}

I'm still not keen on this. I'm still hoping to kill off filter_match, as for many reasons it is not great, and I know that x86 have similar scheduling requirements, and somehow manage without this.

It would be worth adding justification for this in a comment block above, such as why you need this, and why this won't result in sub-optimal behaviour.

[...]

Quoted 5 lines
drivers/perf/qcom_l2_pmu.c new file +5 -0
+	if (acpi_bus_get_device(ACPI_HANDLE(dev), &device))
+		return -ENODEV;
+
+	if (kstrtol(device->pnp.unique_id, 10, &fw_cluster_id) < 0) {
+		dev_err(&pdev->dev, "unable to read ACPI uid\n");

drivers/perf/qcom_l2_pmu.c line 812

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +9 -0
+		return -ENODEV;
+	}
 
+	cluster->l2cache_pmu = l2cache_pmu;
+	for_each_present_cpu(cpu) {
+		if (topology_physical_package_id(cpu) == fw_cluster_id) {
+			cpumask_set_cpu(cpu, &cluster->cluster_cpus);
+			per_cpu(pmu_cluster, cpu) = cluster;
+		}
+	}

I'm still uneasy about this.

The topology_* API doesn't have a well-defined mapping to the MPIDR.Aff* levels, which itself also don't have a well-defined mapping to your hardware's clusters (and therefore fw_cluster_id).

Thus, I'm rather worried that this is going to get broken easily, either by changes in the kernel, or in future HW revisions where the mapping of clusters to MPIDR.Aff* fields changes.

[...]

drivers/perf/qcom_l2_pmu.c line 832

Quoted 2 lines
drivers/perf/qcom_l2_pmu.c new file +2 -0
+	err = devm_request_irq(&pdev->dev, irq, l2_cache__handle_irq,
+			       IRQF_NOBALANCING, "l2-cache-pmu", cluster);

I believe this also needs IRQF_NO_THREAD.

[...]

drivers/perf/qcom_l2_pmu.c line 843

Quoted 1 line
drivers/perf/qcom_l2_pmu.c new file +1 -0
+	cluster->pmu_lock = __SPIN_LOCK_UNLOCKED(cluster->pmu_lock);

Please use:

	spin_lock_init(&cluster->pmu_lock);

[...]

drivers/perf/qcom_l2_pmu.c line 865

Quoted 4 lines
drivers/perf/qcom_l2_pmu.c new file +4 -0
+	platform_set_drvdata(pdev, l2cache_pmu);
+	l2cache_pmu->pmu = (struct pmu) {
+		/* suffix is instance id for future use with multiple sockets */
+		.name		= "l2cache_0",

Sorry to go back and forth on this, but as above with the aggregation comments, we might not need/want the suffix after all, or we might want per-cluster PMUs.

drivers/perf/qcom_l2_pmu.c line 877

Quoted 12 lines
drivers/perf/qcom_l2_pmu.c new file +12 -0
+		.task_ctx_nr    = perf_invalid_context,
+		.pmu_enable	= l2_cache__pmu_enable,
+		.pmu_disable	= l2_cache__pmu_disable,
+		.event_init	= l2_cache__event_init,
+		.add		= l2_cache__event_add,
+		.del		= l2_cache__event_del,
+		.start		= l2_cache__event_start,
+		.stop		= l2_cache__event_stop,
+		.read		= l2_cache__event_read,
+		.attr_groups	= l2_cache_pmu_attr_grps,
+		.filter_match   = l2_cache_filter_match,
+	};

As a general note, can we please s/__/_/ for function names? The double underscore gains us nothing.

[...]

drivers/perf/qcom_l2_pmu.c line 883

Quoted 2 lines
drivers/perf/qcom_l2_pmu.c new file +2 -0
+	l2_counter_present_mask = GENMASK(l2cache_pmu->num_counters - 2, 0) |
+		L2PM_CC_ENABLE;

This looks confusing, and it's the only use of L2PM_CC_ENABLE.

Perhaps use BIT(L2CYCLE_CTR_BIT) instead, and get rid of L2PM_CC_ENABLE?

[...]

Quoted 4 lines
drivers/perf/qcom_l2_pmu.c new file +4 -0
+	err = perf_pmu_register(&l2cache_pmu->pmu, l2cache_pmu->pmu.name, -1);
+	if (err < 0) {
+		dev_err(&pdev->dev, "Error %d registering L2 cache PMU\n", err);
+		return err;

drivers/perf/qcom_l2_pmu.c line 911

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+	}
+
+	dev_info(&pdev->dev, "Registered L2 cache PMU using %d HW PMUs\n",
+		 l2cache_pmu->num_pmus);
+
+	err = cpuhp_state_add_instance_nocalls(CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE,
+					       &l2cache_pmu->node);
+
+	return err;
+}

Can we race with hotplug? I believe the hotplug callback registration should come before the perf_pmu_register()

drivers/perf/qcom_l2_pmu.c line 922

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+static int l2_cache_pmu_remove(struct platform_device *pdev)
+{
+	struct l2cache_pmu *l2cache_pmu =
+		to_l2cache_pmu(platform_get_drvdata(pdev));
+
+	cpuhp_state_remove_instance_nocalls(CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE,
+					       &l2cache_pmu->node);
+	perf_pmu_unregister(&l2cache_pmu->pmu);
+	return 0;
+}

Mirroring the above, I think we should unregister the PMU before removing the hotplug callbacks.

Thanks,
Mark.

#2 lore raw
Quoted 16 lines · Mark Rutland

drivers/perf/qcom_l2_pmu.c line 160

Quoted 10 lines · Neil Leeder
drivers/perf/qcom_l2_pmu.c new file +10 -0
+	struct perf_event *events[MAX_L2_CTRS];
+	struct l2cache_pmu *l2cache_pmu;
+	DECLARE_BITMAP(used_counters, MAX_L2_CTRS);
+	DECLARE_BITMAP(used_groups, L2_EVT_GROUP_MAX + 1);
+	int group_to_counter[L2_EVT_GROUP_MAX + 1];
+	int irq;
+	/* The CPU that is used for collecting events on this cluster */
+	int on_cpu;
+	/* All the CPUs associated with this cluster */
+	cpumask_t cluster_cpus;

I'm still uncertain about aggregating all cluster PMUs into a larger PMU, given the cluster PMUs are logically independent (at least in terms of the programming model).

Quoted 7 lines

However, from what I understand the x86 uncore PMU drivers aggregate symmetric instances of uncore PMUs (and also aggregate across packages to the same logical PMU).

Whatever we do, it would be nice for the uncore drivers to align on a common behaviour (and I think we're currently going the oppposite route with Cavium's uncore PMU). Will, thoughts?

I'm not a big fan of aggregating this stuff. Ultimately, the user in the driving seat of perf is going to need some knowledge about the toplogy of the system in order to perform sensible profiling using an uncore PMU. If the kernel tries to present a single, unified PMU then we paint ourselves into a corner when the hardware isn't as symmetric as we want it to be (big/little on the CPU side is the extreme example of this). If we want to be consistent, then exposing each uncore unit as a separate PMU is the way to go. That doesn't mean we can't aggregate the components of a distributed PMU (e.g. the CCN or the SMMU), but we don't want to aggregate at the programming interface/IP block level.

We could consider exposing some topology information in sysfs if that's seen as an issue with the non-aggregated case.

Will

#3 lore raw

Hi Will,

Quoted 36 lines · Will Deacon
Quoted 16 lines · Mark Rutland

drivers/perf/qcom_l2_pmu.c line 160

Quoted 10 lines · Neil Leeder
drivers/perf/qcom_l2_pmu.c new file +10 -0
+	struct perf_event *events[MAX_L2_CTRS];
+	struct l2cache_pmu *l2cache_pmu;
+	DECLARE_BITMAP(used_counters, MAX_L2_CTRS);
+	DECLARE_BITMAP(used_groups, L2_EVT_GROUP_MAX + 1);
+	int group_to_counter[L2_EVT_GROUP_MAX + 1];
+	int irq;
+	/* The CPU that is used for collecting events on this cluster */
+	int on_cpu;
+	/* All the CPUs associated with this cluster */
+	cpumask_t cluster_cpus;

I'm still uncertain about aggregating all cluster PMUs into a larger PMU, given the cluster PMUs are logically independent (at least in terms of the programming model).

Quoted 7 lines

However, from what I understand the x86 uncore PMU drivers aggregate symmetric instances of uncore PMUs (and also aggregate across packages to the same logical PMU).

Whatever we do, it would be nice for the uncore drivers to align on a common behaviour (and I think we're currently going the oppposite route with Cavium's uncore PMU). Will, thoughts?

I'm not a big fan of aggregating this stuff. Ultimately, the user in the driving seat of perf is going to need some knowledge about the toplogy of the system in order to perform sensible profiling using an uncore PMU. If the kernel tries to present a single, unified PMU then we paint ourselves into a corner when the hardware isn't as symmetric as we want it to be (big/little on the CPU side is the extreme example of this). If we want to be consistent, then exposing each uncore unit as a separate PMU is the way to go. That doesn't mean we can't aggregate the components of a distributed PMU (e.g. the CCN or the SMMU), but we don't want to aggregate at the programming interface/IP block level.

Quoted 4 lines

We could consider exposing some topology information in sysfs if that's seen as an issue with the non-aggregated case.

Will

So is there a use-case for individual uncore PMUs when they can't be used in task mode or per-cpu?

The main (only?) use will be in system mode, in which case surely it makes sense to provide a single aggregated count?

With individual PMUs exposed there will be potentially dozens of nodes for userspace to collect from which would make perf command-line usage unwieldy at best.

Neil

Signature · 4 lines
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm 
Technologies Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.
maintainer
#4 lore raw
Quoted 5 lines · Neil

So is there a use-case for individual uncore PMUs when they can't be used in task mode or per-cpu?

The main (only?) use will be in system mode, in which case surely it makes sense to provide a single aggregated count?

If you are aware of the system topology, the numbers may be more useful than the summed count. If you aren't, it's still possible to sum them in userspace.

Having them summed by the kernel means that the kernel is implying it supports group semantics that it cannot, since it cannot start/stop all counters in a group atomically if they're split across several units.

Quoted 3 lines

With individual PMUs exposed there will be potentially dozens of nodes for userspace to collect from which would make perf command-line usage unwieldy at best.

FWIW, for uncore/system PMUs, even on x86 there are a number of independent units.

Some PMUs (those which are symmetric across the topology) get hidden behind the same struct PMU, but instances are still isolated (and their values not summed).

Thanks,
Mark.

#5 lore raw
In reply to Mark Rutland

Hi Mark,
Thanks for the review. I'll handle all the syntactic comments, so I won't reply to them all individually here.

For the aggregation, I'll reply separately to Will's post to keep all those comments together.

Quoted 14 lines · Mark Rutland

drivers/perf/qcom_l2_pmu.c line 141

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+
+/*
+ * The cache is made up of one or more clusters, each cluster has its own PMU.
+ * Each cluster is associated with one or more CPUs.
+ * This structure represents one of the hardware PMUs.
+ *
+ * Events can be envisioned as a 2-dimensional array. Each column represents
+ * a group of events. There are 8 groups. Only one entry from each
+ * group can be in use at a time. When an event is assigned a counter
+ * by *_event_add(), the counter index is assigned to group_to_counter[group].

If I've followed correctly, this means each group has a dedicated counter?

Quoted 4 lines

I take it groups are not symmetric (i.e. each column has different events)? Or does each column contain the same events?

Is there any overlap?

Each group will have at most one counter, but it's not dedicated. There's no requirement that an implementation have as many counters as there are groups, so an event can be assigned to any available counter.

Every entry in the 2-dimensional array is unique, so no duplicates. Once you have used an event, you cannot use any other event from its column. As an aside, hardware designers put in some effort to ensure that events which may need to be collected at the same time are located in different columns.

Quoted 2 lines
Quoted 2 lines
drivers/perf/qcom_l2_pmu.c new file +2 -0
+static int l2_cache__event_init(struct perf_event *event)
+{

[...]

Quoted 21 lines
Quoted 21 lines
drivers/perf/qcom_l2_pmu.c new file +21 -0
+	/* Don't allow groups with mixed PMUs, except for s/w events */
+	if (event->group_leader->pmu != event->pmu &&
+	    !is_software_event(event->group_leader)) {
+		dev_warn(&l2cache_pmu->pdev->dev,
+			 "Can't create mixed PMU group\n");
+		return -EINVAL;
+	}
+
+	list_for_each_entry(sibling, &event->group_leader->sibling_list,
+			    group_entry)
+		if (sibling->pmu != event->pmu &&
+		    !is_software_event(sibling)) {
+			dev_warn(&l2cache_pmu->pdev->dev,
+				 "Can't create mixed PMU group\n");
+			return -EINVAL;
+		}
+
+	/* Ensure all events in a group are on the same cpu */
+	cluster = get_hml2_pmu(event->cpu);
+	if ((event->group_leader != event) &&
+	    (cluster->on_cpu != event->group_leader->cpu)) {
Quoted 8 lines

drivers/perf/qcom_l2_pmu.c line 539

Quoted 5 lines
drivers/perf/qcom_l2_pmu.c new file +5 -0
+		dev_warn(&l2cache_pmu->pdev->dev,
+			 "Can't create group on CPUs %d and %d",
+			 event->cpu, event->group_leader->cpu);
+		return -EINVAL;
+	}

It's probably worth also checking that the events are co-schedulable (e.g. they don't conflict column-wise).

That's what filter_match() is doing - stopping column-conflicting events from even getting to init(). In init() we don't have a record of which other events are being co-scheduled. We could keep a list of groups used by other events to compare against, but because there's no matching term() function there's no obvious way of removing them from the list.

In my very first patchset I had the check inside event_add() because entries could be removed in event_del(). That was where you suggested that I use filter_match().

Quoted 19 lines
Quoted 6 lines
drivers/perf/qcom_l2_pmu.c new file +5 -0
+	if (acpi_bus_get_device(ACPI_HANDLE(dev), &device))
+		return -ENODEV;
+
+	if (kstrtol(device->pnp.unique_id, 10, &fw_cluster_id) < 0) {
+		dev_err(&pdev->dev, "unable to read ACPI uid\n");

drivers/perf/qcom_l2_pmu.c line 812

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +9 -0
+		return -ENODEV;
+	}
 
+	cluster->l2cache_pmu = l2cache_pmu;
+	for_each_present_cpu(cpu) {
+		if (topology_physical_package_id(cpu) == fw_cluster_id) {
+			cpumask_set_cpu(cpu, &cluster->cluster_cpus);
+			per_cpu(pmu_cluster, cpu) = cluster;
+		}
+	}

I'm still uneasy about this.

Quoted 7 lines

The topology_* API doesn't have a well-defined mapping to the MPIDR.Aff* levels, which itself also don't have a well-defined mapping to your hardware's clusters (and therefore fw_cluster_id).

Thus, I'm rather worried that this is going to get broken easily, either by changes in the kernel, or in future HW revisions where the mapping of clusters to MPIDR.Aff* fields changes.

I'm not sure how else to get a mapping of CPU to cluster which doesn't eventually end with MPIDR.

This is the definition of topology_physical_package_id() from asm/topology.h:

#define topology_physical_package_id(cpu)
(cpu_topology[cpu].cluster_id)

It seems to be a pretty solid connection between cpu and cluster. I don't think this is an abuse of this function. Unless there is some other way of getting this mapping I'd suggest using this, and if some future chip should change MPIDR usage it can be addressed it then.

Quoted 4 lines

Thanks,
Mark.

Thanks,

Neil

Signature · 4 lines
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm 
Technologies Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.
maintainer
#6 lore raw
Quoted 20 lines · Neil
Quoted 14 lines · Mark Rutland

drivers/perf/qcom_l2_pmu.c line 141

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +10 -0
+
+/*
+ * The cache is made up of one or more clusters, each cluster has its own PMU.
+ * Each cluster is associated with one or more CPUs.
+ * This structure represents one of the hardware PMUs.
+ *
+ * Events can be envisioned as a 2-dimensional array. Each column represents
+ * a group of events. There are 8 groups. Only one entry from each
+ * group can be in use at a time. When an event is assigned a counter
+ * by *_event_add(), the counter index is assigned to group_to_counter[group].

If I've followed correctly, this means each group has a dedicated counter?

Quoted 4 lines

I take it groups are not symmetric (i.e. each column has different events)? Or does each column contain the same events?

Is there any overlap?

Quoted 8 lines

Each group will have at most one counter, but it's not dedicated. There's no requirement that an implementation have as many counters as there are groups, so an event can be assigned to any available counter.

Every entry in the 2-dimensional array is unique, so no duplicates. Once you have used an event, you cannot use any other event from its column.

Ok; thanks for clarifying that!

Quoted 32 lines
Quoted 23 lines
Quoted 23 lines
drivers/perf/qcom_l2_pmu.c new file +22 -0
+static int l2_cache__event_init(struct perf_event *event)
 
+	/* Don't allow groups with mixed PMUs, except for s/w events */
+	if (event->group_leader->pmu != event->pmu &&
+	    !is_software_event(event->group_leader)) {
+		dev_warn(&l2cache_pmu->pdev->dev,
+			 "Can't create mixed PMU group\n");
+		return -EINVAL;
+	}
+
+	list_for_each_entry(sibling, &event->group_leader->sibling_list,
+			    group_entry)
+		if (sibling->pmu != event->pmu &&
+		    !is_software_event(sibling)) {
+			dev_warn(&l2cache_pmu->pdev->dev,
+				 "Can't create mixed PMU group\n");
+			return -EINVAL;
+		}
+
+	/* Ensure all events in a group are on the same cpu */
+	cluster = get_hml2_pmu(event->cpu);
+	if ((event->group_leader != event) &&
+	    (cluster->on_cpu != event->group_leader->cpu)) {
Quoted 8 lines

drivers/perf/qcom_l2_pmu.c line 539

Quoted 5 lines
drivers/perf/qcom_l2_pmu.c new file +5 -0
+		dev_warn(&l2cache_pmu->pdev->dev,
+			 "Can't create group on CPUs %d and %d",
+			 event->cpu, event->group_leader->cpu);
+		return -EINVAL;
+	}

It's probably worth also checking that the events are co-schedulable (e.g. they don't conflict column-wise).

Quoted 6 lines

That's what filter_match() is doing - stopping column-conflicting events from even getting to init(). In init() we don't have a record of which other events are being co-scheduled. We could keep a list of groups used by other events to compare against, but because there's no matching term() function there's no obvious way of removing them from the list.

I mean within the group, in addition to the filter_match() logic.

When you event_init() an event, you can determine whether there is any column conflict within the group the new events is being placed in, and whether you have sufficient counters to ever be able to schedule that group. If not, the group should be rejected.

Other PMUs have similar checks; see l2x0_pmu_group_is_valid() in arch/arm/mm/cache-l2x0-pmu.c, and valiate_group() in drivers/perf/arm_pmu.c.

I don't believe that filer_match() can catch that, as it's called on each event in a group individually prior to add() time, and thus there's no visibility of the group as a whole.

Regardless, we can and should catch that case far earlier.

[...]

Quoted 22 lines
Quoted 22 lines
Quoted 5 lines
drivers/perf/qcom_l2_pmu.c new file +5 -0
+	if (acpi_bus_get_device(ACPI_HANDLE(dev), &device))
+		return -ENODEV;
+
+	if (kstrtol(device->pnp.unique_id, 10, &fw_cluster_id) < 0) {
+		dev_err(&pdev->dev, "unable to read ACPI uid\n");

drivers/perf/qcom_l2_pmu.c line 812

Quoted 10 lines
drivers/perf/qcom_l2_pmu.c new file +9 -0
+		return -ENODEV;
+	}
 
+	cluster->l2cache_pmu = l2cache_pmu;
+	for_each_present_cpu(cpu) {
+		if (topology_physical_package_id(cpu) == fw_cluster_id) {
+			cpumask_set_cpu(cpu, &cluster->cluster_cpus);
+			per_cpu(pmu_cluster, cpu) = cluster;
+		}
+	}

I'm still uneasy about this.

The topology_* API doesn't have a well-defined mapping to the MPIDR.Aff* levels, which itself also don't have a well-defined mapping to your hardware's clusters (and therefore fw_cluster_id).

Quoted 6 lines
Quoted 3 lines

Thus, I'm rather worried that this is going to get broken easily, either by changes in the kernel, or in future HW revisions where the mapping of clusters to MPIDR.Aff* fields changes.

I'm not sure how else to get a mapping of CPU to cluster which doesn't eventually end with MPIDR.

This is unfortunate. :(

It would have been much nicer if the FW also provided the MPIDR.Aff<n> level to match up to, as that would be unambiguous.

Quoted 7 lines

This is the definition of topology_physical_package_id() from asm/topology.h:

#define topology_physical_package_id(cpu)
(cpu_topology[cpu].cluster_id)

It seems to be a pretty solid connection between cpu and cluster.

As I mentioned above, there's no well-defined mapping from MPIDR.Aff* to the topology API levels. The "cluster_id" here is a guess, and one that might change in future based on other heuristics.

Quoted 3 lines

I don't think this is an abuse of this function. Unless there is some other way of getting this mapping I'd suggest using this, and if some future chip should change MPIDR usage it can be addressed it then.

I don't think it's an abuse, as such, but I don't think that it is reliable.

That said, I don't see that we can do any better, as you say.

It's probably worth adding a comment block regarding our expectations, i.e. that cluster_id means Aff1 for CPUs without multi-threading, Aff2 otherwise, and that we hope future systems don't choose another MPIDR.Aff* mapping scheme.

Thanks,
Mark.