Skip to contentSkip to conversation

arm64: perf: Add support for ARMv8.5-PMU 64-bit counters

PATCH In review Mark Rutland 3 participants 8 messages Mar 2, 2020 – Mar 18, 2020

Subsystems (4)
lore
Share & tools
lore

Mark Rutland

author maintainer
#0 lore raw

v6 cover arm64: perf: Add support for ARMv8.5-PMU 64-bit counters

This is a respin of Andrew Murray's series to enable support for 64-bit counters as introduced in ARMv8.5.

I've given this a spin on (ARMv8.2) hardware, to test that there are no regressions, but I have not had the chance to test in an ARMv8.5 model (which I beleive Andrew had previously tested).

Since v5 [1]:
* Don't treat perfmon ID fields as signed
* Fix up ID field names
* Explicitly compare ARMV8.5 PMU value

[1] https://lkml.kernel.org/r/[email protected]

Thanks,
Mark.

Andrew Murray (3):
arm64: cpufeature: Extract capped perfmon fields
KVM: arm64: limit PMU version to PMUv3 for ARMv8.1
arm64: perf: Add support for ARMv8.5-PMU 64-bit counters

 arch/arm64/include/asm/cpufeature.h | 23 ++++++++++
 arch/arm64/include/asm/perf_event.h |  3 +-
 arch/arm64/include/asm/sysreg.h     | 10 +++++
 arch/arm64/kernel/perf_event.c      | 86 +++++++++++++++++++++++++++++--------
 arch/arm64/kvm/sys_regs.c           | 10 +++++
 include/linux/perf/arm_pmu.h        |  1 +
 6 files changed, 115 insertions(+), 18 deletions(-)
Signature · 6 lines
2.11.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#1 lore raw

PATCH v6 1/3 arm64: cpufeature: Extract capped perfmon fields

From: Andrew Murray <[email protected]>

When emulating ID registers there is often a need to cap the version bits of a feature such that the guest will not use features that the host is not aware of. For example, when KVM mediates access to the PMU by emulating register accesses.

Let's add a helper that extracts a performance monitors ID field and caps the version to a given value.

Fields that identify the version of the Performance Monitors Extension do not follow the standard ID scheme, and instead follow the scheme described in ARM DDI 0487E.a page D13-2825 "Alternative ID scheme used for the Performance Monitors Extension version". The value 0xF means an IMPLEMENTATION DEFINED PMU is present, and values 0x0-OxE can be treated the same as an unsigned field with 0x0 meaning no PMU is present.

[Mark: rework to handle perfmon fields]
---
 arch/arm64/include/asm/cpufeature.h | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
arch/arm64/include/asm/cpufeature.h +23 -0
index 92ef9539874a..186f4e19207e 100644
@@ -447,6 +447,29 @@ cpuid_feature_extract_unsigned_field(u64 features, int field)
 	return cpuid_feature_extract_unsigned_field_width(features, field, 4);
 }
 
+/*
+ * Fields that identify the version of the Performance Monitors Extension do
+ * not follow the standard ID scheme. See ARM DDI 0487E.a page D13-2825,
+ * "Alternative ID scheme used for the Performance Monitors Extension version".
+ */
+static inline u64 __attribute_const__
+cpuid_feature_cap_perfmon_field(u64 features, int field, u64 cap)
+{
+	u64 val = cpuid_feature_extract_unsigned_field(features, field);
+	u64 mask = GENMASK_ULL(field + 3, field);
+
+	/* Treat IMPLEMENTATION DEFINED functionality as unimplemented */
+	if (val == 0xf)
+		val = 0;
+
+	if (val > cap) {
+		features &= ~mask;
+		features |= (cap << field) & mask;
+	}
+
+	return features;
+}
+
 static inline u64 arm64_ftr_mask(const struct arm64_ftr_bits *ftrp)
 {
 	return (u64)GENMASK(ftrp->shift + ftrp->width - 1, ftrp->shift);
Signature · 6 lines
2.11.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#2 lore raw

PATCH v6 2/3 KVM: arm64: limit PMU version to PMUv3 for ARMv8.1

In reply to Mark Rutland

From: Andrew Murray <[email protected]>

We currently expose the PMU version of the host to the guest via emulation of the DFR0_EL1 and AA64DFR0_EL1 debug feature registers. However many of the features offered beyond PMUv3 for 8.1 are not supported in KVM. Examples of this include support for the PMMIR registers (added in PMUv3 for ARMv8.4) and 64-bit event counters added in (PMUv3 for ARMv8.5).

Let's trap the Debug Feature Registers in order to limit PMUVer/PerfMon in the Debug Feature Registers to PMUv3 for ARMv8.1 to avoid unexpected behaviour.

Both ID_AA64DFR0.PMUVer and ID_DFR0.PerfMon follow the "Alternative ID scheme used for the Performance Monitors Extension version" where 0xF means an IMPLEMENTATION DEFINED PMU is implemented, and values 0x0-0xE are treated as with an unsigned field (with 0x0 meaning no PMU is present). As we don't expect to expose an IMPLEMENTATION DEFINED PMU, and our cap is below 0xF, we can treat these fields as unsigned when applying the cap.

[Mark: make field names consistent, use perfmon cap]
---
 arch/arm64/include/asm/sysreg.h |  6 ++++++
 arch/arm64/kvm/sys_regs.c       | 10 ++++++++++
 2 files changed, 16 insertions(+)
arch/arm64/include/asm/sysreg.h +6 -0
index b91570ff9db1..d8f1eed070f0 100644
@@ -702,6 +702,12 @@
 #define ID_AA64DFR0_TRACEVER_SHIFT	4
 #define ID_AA64DFR0_DEBUGVER_SHIFT	0
 
+#define ID_AA64DFR0_PMUVER_8_1		0x4
+
+#define ID_DFR0_PERFMON_SHIFT		24
+
+#define ID_DFR0_PERFMON_8_1		0x4
+
 #define ID_ISAR5_RDM_SHIFT		24
 #define ID_ISAR5_CRC32_SHIFT		16
 #define ID_ISAR5_SHA2_SHIFT		12
arch/arm64/kvm/sys_regs.c +10 -0
index 3e909b117f0c..b0a3e8976b90 100644
@@ -1085,6 +1085,16 @@ static u64 read_id_reg(const struct kvm_vcpu *vcpu,
 			 (0xfUL << ID_AA64ISAR1_API_SHIFT) |
 			 (0xfUL << ID_AA64ISAR1_GPA_SHIFT) |
 			 (0xfUL << ID_AA64ISAR1_GPI_SHIFT));
+	} else if (id == SYS_ID_AA64DFR0_EL1) {
+		/* Limit guests to PMUv3 for ARMv8.1 */
+		val = cpuid_feature_cap_perfmon_field(val,
+						ID_AA64DFR0_PMUVER_SHIFT,
+						ID_AA64DFR0_PMUVER_8_1);
+	} else if (id == SYS_ID_DFR0_EL1) {
+		/* Limit guests to PMUv3 for ARMv8.1 */
+		val = cpuid_feature_cap_perfmon_field(val,
+						ID_DFR0_PERFMON_SHIFT,
+						ID_DFR0_PERFMON_8_1);
 	}
 
 	return val;
Signature · 6 lines
2.11.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#3 lore raw

PATCH v6 3/3 arm64: perf: Add support for ARMv8.5-PMU 64-bit counters

In reply to Mark Rutland

From: Andrew Murray <[email protected]>

At present ARMv8 event counters are limited to 32-bits, though by using the CHAIN event it's possible to combine adjacent counters to achieve 64-bits. The perf config1:0 bit can be set to use such a configuration.

With the introduction of ARMv8.5-PMU support, all event counters can now be used as 64-bit counters.

Let's enable 64-bit event counters where support exists. Unless the user sets config1:0 we will adjust the counter value such that it overflows upon 32-bit overflow. This follows the same behaviour as the cycle counter which has always been (and remains) 64-bits.

[Mark: fix ID field names, compare with 8.5 value]
---
 arch/arm64/include/asm/perf_event.h |  3 +-
 arch/arm64/include/asm/sysreg.h     |  4 ++
 arch/arm64/kernel/perf_event.c      | 86 +++++++++++++++++++++++++++++--------
 include/linux/perf/arm_pmu.h        |  1 +
 4 files changed, 76 insertions(+), 18 deletions(-)
arch/arm64/include/asm/perf_event.h +2 -1
index 2bdbc79bbd01..e7765b62c712 100644
@@ -176,9 +176,10 @@
 #define ARMV8_PMU_PMCR_X	(1 << 4) /* Export to ETM */
 #define ARMV8_PMU_PMCR_DP	(1 << 5) /* Disable CCNT if non-invasive debug*/
 #define ARMV8_PMU_PMCR_LC	(1 << 6) /* Overflow on 64 bit cycle counter */
+#define ARMV8_PMU_PMCR_LP	(1 << 7) /* Long event counter enable */
 #define	ARMV8_PMU_PMCR_N_SHIFT	11	 /* Number of counters supported */
 #define	ARMV8_PMU_PMCR_N_MASK	0x1f
-#define	ARMV8_PMU_PMCR_MASK	0x7f	 /* Mask for writable bits */
+#define	ARMV8_PMU_PMCR_MASK	0xff	 /* Mask for writable bits */
 
 /*
  * PMOVSR: counters overflow flag status reg
arch/arm64/include/asm/sysreg.h +4 -0
index d8f1eed070f0..9b66c5b5b36f 100644
@@ -702,7 +702,11 @@
 #define ID_AA64DFR0_TRACEVER_SHIFT	4
 #define ID_AA64DFR0_DEBUGVER_SHIFT	0
 
+#define ID_AA64DFR0_PMUVER_8_0		0x1
 #define ID_AA64DFR0_PMUVER_8_1		0x4
+#define ID_AA64DFR0_PMUVER_8_4		0x5
+#define ID_AA64DFR0_PMUVER_8_5		0x6
+#define ID_AA64DFR0_PMUVER_IMP_DEF	0xf
 
 #define ID_DFR0_PERFMON_SHIFT		24
 
arch/arm64/kernel/perf_event.c +69 -17

155 lines not on this page · raw message

include/linux/perf/arm_pmu.h +1 -0
index 71f525a35ac2..5b616dde9a4c 100644
@@ -80,6 +80,7 @@ struct arm_pmu {
 	struct pmu	pmu;
 	cpumask_t	supported_cpus;
 	char		*name;
+	int		pmuver;
 	irqreturn_t	(*handle_irq)(struct arm_pmu *pmu);
 	void		(*enable)(struct perf_event *event);
 	void		(*disable)(struct perf_event *event);
Signature · 6 lines
2.11.0

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#4 lore raw
In reply to Mark Rutland
Quoted 6 lines · Mark Rutland

This is a respin of Andrew Murray's series to enable support for 64-bit counters as introduced in ARMv8.5.

I've given this a spin on (ARMv8.2) hardware, to test that there are no regressions, but I have not had the chance to test in an ARMv8.5 model (which I beleive Andrew had previously tested).

Bad news; this is broken. :(

While perf-stat works as expected, perf-record doesn't get samples for any of the programmable counters.

In ARMv8.4 mode I can do:

| / # perf record -a -c 1 -e armv8_pmuv3/inst_retired/ true
| [ perf record: Woken up 1 times to write data ]
| [ perf record: Captured and wrote 0.023 MB perf.data (367 samples) ]
| / # perf record -a -c 1 -e armv8_pmuv3/inst_retired,long/ true
| [ perf record: Woken up 1 times to write data ]
| [ perf record: Captured and wrote 0.022 MB perf.data (353 samples) ]

... so regular 32-bit and chained events work correctly.

But in ARMv8.5 mode I get no samples in either case:

| / # perf record -a -c 1 -e armv8_pmuv3/inst_retired/ true
| [ perf record: Woken up 1 times to write data ]
| [ perf record: Captured and wrote 0.008 MB perf.data ]
| / # perf report | grep samples
| Error:
| The perf.data file has no samples!
| / # perf record -a -c 1 -e armv8_pmuv3/inst_retired,long/ true
| [ perf record: Woken up 1 times to write data ]
| [ perf record: Captured and wrote 0.008 MB perf.data ]
| / # perf report | grep samples
| Error:
| The perf.data file has no samples!

I'll have to trace the driver to see what's going on. I suspect we've missed some bias handling, but it's possible that this is a model bug.

Thanks,
Mark.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

#5 lore raw
Quoted 39 lines · Mark Rutland
Quoted 6 lines · Mark Rutland

This is a respin of Andrew Murray's series to enable support for 64-bit counters as introduced in ARMv8.5.

I've given this a spin on (ARMv8.2) hardware, to test that there are no regressions, but I have not had the chance to test in an ARMv8.5 model (which I beleive Andrew had previously tested).

Bad news; this is broken. :(

While perf-stat works as expected, perf-record doesn't get samples for any of the programmable counters.

In ARMv8.4 mode I can do:

| / # perf record -a -c 1 -e armv8_pmuv3/inst_retired/ true
| [ perf record: Woken up 1 times to write data ]
| [ perf record: Captured and wrote 0.023 MB perf.data (367 samples) ]
| / # perf record -a -c 1 -e armv8_pmuv3/inst_retired,long/ true
| [ perf record: Woken up 1 times to write data ]
| [ perf record: Captured and wrote 0.022 MB perf.data (353 samples) ]

... so regular 32-bit and chained events work correctly.

But in ARMv8.5 mode I get no samples in either case:

| / # perf record -a -c 1 -e armv8_pmuv3/inst_retired/ true
| [ perf record: Woken up 1 times to write data ]
| [ perf record: Captured and wrote 0.008 MB perf.data ]
| / # perf report | grep samples
| Error:
| The perf.data file has no samples!
| / # perf record -a -c 1 -e armv8_pmuv3/inst_retired,long/ true
| [ perf record: Woken up 1 times to write data ]
| [ perf record: Captured and wrote 0.008 MB perf.data ]
| / # perf report | grep samples
| Error:
| The perf.data file has no samples!
Quoted 2 lines

I'll have to trace the driver to see what's going on. I suspect we've missed some bias handling, but it's possible that this is a model bug.

For the record, further evidence has indeed pointed to there being a bug in the model's implementation of ARMv8.5-PMU. It's been raised with the models team, so we'll have wait and see what they say...

Robin.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

maintainer
#6 lore raw
In reply to Mark Rutland
Quoted 11 lines · Mark Rutland

This is a respin of Andrew Murray's series to enable support for 64-bit counters as introduced in ARMv8.5.

I've given this a spin on (ARMv8.2) hardware, to test that there are no regressions, but I have not had the chance to test in an ARMv8.5 model (which I beleive Andrew had previously tested).

Since v5 [1]:
* Don't treat perfmon ID fields as signed
* Fix up ID field names
* Explicitly compare ARMV8.5 PMU value

I'm betting on your issue being a model bug, so I've queued this on top of Robin's enable/disable rework. Please take a look at for-next/perf [1] in case I screwed it up.

Thanks,

Will

[1] https://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git/log/?h=for-next/perf
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author maintainer
#7 lore raw
Quoted 8 lines · Will Deacon
Quoted 7 lines · Mark Rutland

This is a respin of Andrew Murray's series to enable support for 64-bit counters as introduced in ARMv8.5.

I've given this a spin on (ARMv8.2) hardware, to test that there are no regressions, but I have not had the chance to test in an ARMv8.5 model (which I beleive Andrew had previously tested).

Quoted 8 lines
Quoted 4 lines

Since v5 [1]:
* Don't treat perfmon ID fields as signed
* Fix up ID field names
* Explicitly compare ARMV8.5 PMU value

I'm betting on your issue being a model bug, so I've queued this on top of Robin's enable/disable rework. Please take a look at for-next/perf [1] in case I screwed it up.

From a cursory review, that all looks good to me.

I'll poke if the issue turns out to be anything beyond a model bug. but I agree it seems that's all it is.

Thanks,
Mark.

Quoted 6 lines

Thanks,

Will

[1] https://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git/log/?h=for-next/perf

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm