Skip to contentSkip to conversation

Perf tool: Enable Arm arch timer counter and arm-spe's timestamp

PATCH Superseded Leo Yan 2 participants 6 messages Jul 15, 2020 – Jul 16, 2020

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Leo Yan

author
#0 lore raw

cover Perf tool: Enable Arm arch timer counter and arm-spe's timestamp

This patch set is to enable Arm arch timer counter and Arm SPE is the first customer to use arch timer counter for its timestamp.

The patch 01 is to retrieve arch timer's parameters from mmaped page; patch 02 provides APIs for the conversion between arch timer's counter and time; patch 03 is to enable timestamp based on the new introduced APIs for timestamp related calculation.

Note, this patch set is depend on the kernel changes for "arm64: perf:
Proper cap_user_time* support" [1].

This patch set has been rebased on mainline kernel with the latest commit 11ba468877bb ("Linux 5.8-rc5") and also applied on top of patch set [1]. After enabling the timestamp, the samples can show the timestamp, and the timestamp is consistent with the kernel's timestamp by checking the kernel log:

  $ perf script -F,+time
    dd  6799 [034] 25496.733475:          1              l1d-access:      ffff87f37b88 _dl_start+0x288 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733475:          1              tlb-access:      ffff87f37b88 _dl_start+0x288 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733479:          1              l1d-access:      ffff87f37c74 _dl_start+0x374 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733479:          1              tlb-access:      ffff87f37c74 _dl_start+0x374 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733485:          1              l1d-access:      ffff87f49af4 __GI___tunables_init+0x3c (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733485:          1              tlb-access:      ffff87f49af4 __GI___tunables_init+0x3c (/usr/lib/aarch64-linux-gnu/ld-2.28.so)

[1] https://patchwork.kernel.org/project/linux-arm-kernel/list/?series=318401

Leo Yan (3):
perf tools: Support Arm arch timer counter
perf arm_arch_timer: Convert between counter and timestamp perf arm-spe: Enable timestamp with arch timer counter

 tools/perf/arch/arm64/util/Build        |  1 +
 tools/perf/arch/arm64/util/arch_timer.c | 50 +++++++++++++++++++++++++
 tools/perf/arch/arm64/util/arm-spe.c    | 17 +++++++++
 tools/perf/util/Build                   |  1 +
 tools/perf/util/arm-spe.c               | 16 +++++++-
 tools/perf/util/arm-spe.h               |  5 +++
 tools/perf/util/arm_arch_timer.c        | 28 ++++++++++++++
 tools/perf/util/arm_arch_timer.h        | 23 ++++++++++++
 8 files changed, 139 insertions(+), 2 deletions(-)
 create mode 100644 tools/perf/arch/arm64/util/arch_timer.c
 create mode 100644 tools/perf/util/arm_arch_timer.c
 create mode 100644 tools/perf/util/arm_arch_timer.h
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH 1/3 perf tools: Support Arm arch timer counter

The Arm arch timer can be used to calculate timestamp, the basic idea is the arch timer's counter value can be recorded in the hardware tracing data, e.g. the arch timer's counter value can be used for Arm CoreSight (not now but might be implemented later) and Arm SPE. So we need a way to convert the arch timer's counter to the system time, the conversion is dependent on some related parameters, e.g. 'time_shift', 'time_mult', 'time_offset', etc; furthermore, to handle the counter wrapping issue, perf tool needs to know 'time_cycles' and 'time_mask' for correction.

This patch is to support Arm arch timer by reading out the relevant parameters from the head of first mmaped page. And these parameters will be stored into the structure 'perf_arch_timer_conversion' for later calculation timestamp.

---
 tools/perf/arch/arm64/util/Build        |  1 +
 tools/perf/arch/arm64/util/arch_timer.c | 50 +++++++++++++++++++++++++
 tools/perf/util/arm_arch_timer.h        | 20 ++++++++++
 3 files changed, 71 insertions(+)
 create mode 100644 tools/perf/arch/arm64/util/arch_timer.c
 create mode 100644 tools/perf/util/arm_arch_timer.h
tools/perf/arch/arm64/util/Build +1 -0
index 5c13438c7bd4..77f4d7b30932 100644
@@ -1,3 +1,4 @@
+perf-y += arch_timer.o
 perf-y += header.o
 perf-y += machine.o
 perf-y += perf_regs.o
tools/perf/arch/arm64/util/arch_timer.c new file +50 -0

50 lines not on this page · raw message

tools/perf/util/arm_arch_timer.h new file +20 -0
new file mode 100644
index 000000000000..a3263cc4e5cf
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __PERF_ARM_ARCH_TIMER_H
+#define __PERF_ARM_ARCH_TIMER_H
+
+#include <linux/types.h>
+
+struct perf_arch_timer_conversion {
+	u16 time_shift;
+	u32 time_mult;
+	u64 time_zero;
+	u64 time_cycles;
+	u64 time_mask;
+};
+
+struct perf_event_mmap_page;
+
+int perf_read_arch_timer_conversion(const struct perf_event_mmap_page *pc,
+				    struct perf_arch_timer_conversion *tc);
+
+#endif // __PERF_ARM_ARCH_TIMER_H
Signature · 1 line
2.17.1
author
#2 lore raw

PATCH 2/3 perf arm_arch_timer: Convert between counter and timestamp

In reply to Leo Yan

This patch introduces two new APIs, one is to calculate from converting counter to timestamp and provides a reverse flow to convert timestamp to counter.

---
 tools/perf/util/Build            |  1 +
 tools/perf/util/arm_arch_timer.c | 28 ++++++++++++++++++++++++++++
 tools/perf/util/arm_arch_timer.h |  3 +++
 3 files changed, 32 insertions(+)
 create mode 100644 tools/perf/util/arm_arch_timer.c
tools/perf/util/Build +1 -0
index 8d18380ecd10..ba504549844f 100644
@@ -101,6 +101,7 @@ perf-y += call-path.o
 perf-y += rwsem.o
 perf-y += thread-stack.o
 perf-y += spark.o
+perf-y += arm_arch_timer.o
 perf-$(CONFIG_AUXTRACE) += auxtrace.o
 perf-$(CONFIG_AUXTRACE) += intel-pt-decoder/
 perf-$(CONFIG_AUXTRACE) += intel-pt.o
tools/perf/util/arm_arch_timer.c new file +28 -0
new file mode 100644
index 000000000000..c37ffa4d9710
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/compiler.h>
+#include <linux/types.h>
+
+#include "arm_arch_timer.h"
+
+u64 perf_time_to_arch_timer_cyc(u64 ns, struct perf_arch_timer_conversion *tc)
+{
+	u64 t, quot, rem;
+
+	t = ns - tc->time_zero;
+	quot = t / tc->time_mult;
+	rem  = t % tc->time_mult;
+	return (quot << tc->time_shift) +
+	       (rem << tc->time_shift) / tc->time_mult;
+}
+
+u64 arch_timer_cyc_to_perf_time(u64 cyc, struct perf_arch_timer_conversion *tc)
+{
+	u64 quot, rem;
+
+	cyc = tc->time_cycles + ((cyc - tc->time_cycles) & tc->time_mask);
+
+	quot = cyc >> tc->time_shift;
+	rem  = cyc & (((u64)1 << tc->time_shift) - 1);
+	return tc->time_zero + quot * tc->time_mult +
+	       ((rem * tc->time_mult) >> tc->time_shift);
+}
tools/perf/util/arm_arch_timer.h +3 -0
index a3263cc4e5cf..7d9271f544f2 100644
@@ -17,4 +17,7 @@ struct perf_event_mmap_page;
 int perf_read_arch_timer_conversion(const struct perf_event_mmap_page *pc,
 				    struct perf_arch_timer_conversion *tc);
 
+u64 arch_timer_cyc_to_perf_time(u64 cyc, struct perf_arch_timer_conversion *tc);
+u64 perf_time_to_arch_timer_cyc(u64 ns, struct perf_arch_timer_conversion *tc);
+
 #endif // __PERF_ARM_ARCH_TIMER_H
Signature · 1 line
2.17.1
author
#3 lore raw

PATCH 3/3 perf arm-spe: Enable timestamp with arch timer counter

In reply to Leo Yan

Since the arch timer's counter is used for SPE tracing data, and now it's ready to use arch timer for sample's timestamp in Perf tool, this patch is to enable timestamp by convert the arch timer counter value to the time.

After enabling timestamp for Arm SPE with this patch, we can see the timestamp with 'time' field:

  $ perf script -F,+time
    dd  6799 [034] 25496.733475:          1              l1d-access:      ffff87f37b88 _dl_start+0x288 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733475:          1              tlb-access:      ffff87f37b88 _dl_start+0x288 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733479:          1              l1d-access:      ffff87f37c74 _dl_start+0x374 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733479:          1              tlb-access:      ffff87f37c74 _dl_start+0x374 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733485:          1              l1d-access:      ffff87f49af4 __GI___tunables_init+0x3c (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733485:          1              tlb-access:      ffff87f49af4 __GI___tunables_init+0x3c (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
---
 tools/perf/arch/arm64/util/arm-spe.c | 17 +++++++++++++++++
 tools/perf/util/arm-spe.c            | 16 ++++++++++++++--
 tools/perf/util/arm-spe.h            |  5 +++++
 3 files changed, 36 insertions(+), 2 deletions(-)
tools/perf/arch/arm64/util/arm-spe.c +17 -0
index e3593063b3d1..a7cbd5fdb30e 100644
@@ -11,6 +11,7 @@
 #include <linux/zalloc.h>
 #include <time.h>
 
+#include "../../../util/arm_arch_timer.h"
 #include "../../../util/cpumap.h"
 #include "../../../util/event.h"
 #include "../../../util/evsel.h"
@@ -22,6 +23,7 @@
 #include "../../../util/auxtrace.h"
 #include "../../../util/record.h"
 #include "../../../util/arm-spe.h"
+#include "../../../util/mmap.h"
 
 #define KiB(x) ((x) * 1024)
 #define MiB(x) ((x) * 1024 * 1024)
@@ -47,6 +49,9 @@ static int arm_spe_info_fill(struct auxtrace_record *itr,
 	struct arm_spe_recording *sper =
 			container_of(itr, struct arm_spe_recording, itr);
 	struct perf_pmu *arm_spe_pmu = sper->arm_spe_pmu;
+	struct perf_event_mmap_page *pc;
+	struct perf_arch_timer_conversion tc = { 0 };
+	int err;
 
 	if (priv_size != ARM_SPE_AUXTRACE_PRIV_SIZE)
 		return -EINVAL;
@@ -54,8 +59,20 @@ static int arm_spe_info_fill(struct auxtrace_record *itr,
 	if (!session->evlist->core.nr_mmaps)
 		return -EINVAL;
 
+	pc = session->evlist->mmap[0].core.base;
+	if (pc) {
+		err = perf_read_arch_timer_conversion(pc, &tc);
+		if (err)
+			return err;
+	}
+
 	auxtrace_info->type = PERF_AUXTRACE_ARM_SPE;
 	auxtrace_info->priv[ARM_SPE_PMU_TYPE] = arm_spe_pmu->type;
+	auxtrace_info->priv[ARM_SPE_TIME_SHIFT] = tc.time_shift;
+	auxtrace_info->priv[ARM_SPE_TIME_MULT] = tc.time_mult;
+	auxtrace_info->priv[ARM_SPE_TIME_ZERO] = tc.time_zero;
+	auxtrace_info->priv[ARM_SPE_TIME_CYCLES] = tc.time_cycles;
+	auxtrace_info->priv[ARM_SPE_TIME_MASK] = tc.time_mask;
 
 	return 0;
 }
tools/perf/util/arm-spe.c +14 -2
index 3882a5360ada..07232664c927 100644
@@ -16,6 +16,7 @@
 #include <stdlib.h>
 #include <unistd.h>
 
+#include "arm_arch_timer.h"
 #include "auxtrace.h"
 #include "color.h"
 #include "debug.h"
@@ -44,6 +45,7 @@ struct arm_spe {
 	struct perf_session		*session;
 	struct machine			*machine;
 	u32				pmu_type;
+	struct perf_arch_timer_conversion tc;
 
 	u8				timeless_decoding;
 	u8				data_queued;
@@ -229,7 +231,8 @@ static void arm_spe_prep_sample(struct arm_spe *spe,
 	struct arm_spe_record *record = &speq->decoder->record;
 
 	if (!spe->timeless_decoding)
-		sample->time = speq->timestamp;
+		sample->time = arch_timer_cyc_to_perf_time(speq->timestamp,
+							   &spe->tc);
 
 	sample->ip = record->from_ip;
 	sample->cpumode = arm_spe_cpumode(spe, sample->ip);
@@ -350,6 +353,7 @@ static int arm_spe_sample(struct arm_spe_queue *speq)
 static int arm_spe_run_decoder(struct arm_spe_queue *speq, u64 *timestamp)
 {
 	struct arm_spe *spe = speq->spe;
+	struct arm_spe_record *record;
 	int ret;
 
 	if (!spe->kernel_start)
@@ -369,6 +373,9 @@ static int arm_spe_run_decoder(struct arm_spe_queue *speq, u64 *timestamp)
 		if (ret < 0)
 			continue;
 
+		record = &speq->decoder->record;
+		speq->timestamp = record->timestamp;
+
 		ret = arm_spe_sample(speq);
 		if (ret)
 			return ret;
@@ -585,7 +592,7 @@ static int arm_spe_process_event(struct perf_session *session,
 	}
 
 	if (sample->time && (sample->time != (u64) -1))
-		timestamp = sample->time;
+		timestamp = perf_time_to_arch_timer_cyc(sample->time, &spe->tc);
 	else
 		timestamp = 0;
 
@@ -934,6 +941,11 @@ int arm_spe_process_auxtrace_info(union perf_event *event,
 	spe->machine = &session->machines.host; /* No kvm support */
 	spe->auxtrace_type = auxtrace_info->type;
 	spe->pmu_type = auxtrace_info->priv[ARM_SPE_PMU_TYPE];
+	spe->tc.time_shift = auxtrace_info->priv[ARM_SPE_TIME_SHIFT];
+	spe->tc.time_mult = auxtrace_info->priv[ARM_SPE_TIME_MULT];
+	spe->tc.time_zero = auxtrace_info->priv[ARM_SPE_TIME_ZERO];
+	spe->tc.time_cycles = auxtrace_info->priv[ARM_SPE_TIME_CYCLES];
+	spe->tc.time_mask = auxtrace_info->priv[ARM_SPE_TIME_MASK];
 
 	spe->timeless_decoding = arm_spe__is_timeless_decoding(spe);
 	spe->auxtrace.process_event = arm_spe_process_event;
tools/perf/util/arm-spe.h +5 -0
index 98d3235781c3..8baa32ad179d 100644
@@ -12,6 +12,11 @@
 enum {
 	ARM_SPE_PMU_TYPE,
 	ARM_SPE_PER_CPU_MMAPS,
+	ARM_SPE_TIME_SHIFT,
+	ARM_SPE_TIME_MULT,
+	ARM_SPE_TIME_ZERO,
+	ARM_SPE_TIME_CYCLES,
+	ARM_SPE_TIME_MASK,
 	ARM_SPE_AUXTRACE_PRIV_MAX,
 };
 
Signature · 1 line
2.17.1
reviewer
#4 lore raw
In reply to Leo Yan
Quoted 26 lines · Leo Yan

This patch set is to enable Arm arch timer counter and Arm SPE is the first customer to use arch timer counter for its timestamp.

The patch 01 is to retrieve arch timer's parameters from mmaped page; patch 02 provides APIs for the conversion between arch timer's counter and time; patch 03 is to enable timestamp based on the new introduced APIs for timestamp related calculation.

Note, this patch set is depend on the kernel changes for "arm64: perf:
Proper cap_user_time* support" [1].

This patch set has been rebased on mainline kernel with the latest commit 11ba468877bb ("Linux 5.8-rc5") and also applied on top of patch set [1]. After enabling the timestamp, the samples can show the timestamp, and the timestamp is consistent with the kernel's timestamp by checking the kernel log:

  $ perf script -F,+time
    dd  6799 [034] 25496.733475:          1              l1d-access:      ffff87f37b88 _dl_start+0x288 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733475:          1              tlb-access:      ffff87f37b88 _dl_start+0x288 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733479:          1              l1d-access:      ffff87f37c74 _dl_start+0x374 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733479:          1              tlb-access:      ffff87f37c74 _dl_start+0x374 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733485:          1              l1d-access:      ffff87f49af4 __GI___tunables_init+0x3c (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733485:          1              tlb-access:      ffff87f49af4 __GI___tunables_init+0x3c (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
Quoted 1 line

[1] https://patchwork.kernel.org/project/linux-arm-kernel/list/?series=318401

This looks good to me! Would it be possible to add a test on the functionality? It's possible to add a C test in tools/perf/arch/arm64/tests or something related to you perf script command line in tools/perf/tests/shell.

Thanks,
Ian

Quoted 21 lines

Leo Yan (3):
perf tools: Support Arm arch timer counter
perf arm_arch_timer: Convert between counter and timestamp perf arm-spe: Enable timestamp with arch timer counter

 tools/perf/arch/arm64/util/Build        |  1 +
 tools/perf/arch/arm64/util/arch_timer.c | 50 +++++++++++++++++++++++++
 tools/perf/arch/arm64/util/arm-spe.c    | 17 +++++++++
 tools/perf/util/Build                   |  1 +
 tools/perf/util/arm-spe.c               | 16 +++++++-
 tools/perf/util/arm-spe.h               |  5 +++
 tools/perf/util/arm_arch_timer.c        | 28 ++++++++++++++
 tools/perf/util/arm_arch_timer.h        | 23 ++++++++++++
 8 files changed, 139 insertions(+), 2 deletions(-)
 create mode 100644 tools/perf/arch/arm64/util/arch_timer.c
 create mode 100644 tools/perf/util/arm_arch_timer.c
 create mode 100644 tools/perf/util/arm_arch_timer.h

--
2.17.1

author
#5 lore raw

Hi Ian,

Quoted 27 lines · Ian Rogers
Quoted 26 lines · Leo Yan

This patch set is to enable Arm arch timer counter and Arm SPE is the first customer to use arch timer counter for its timestamp.

The patch 01 is to retrieve arch timer's parameters from mmaped page; patch 02 provides APIs for the conversion between arch timer's counter and time; patch 03 is to enable timestamp based on the new introduced APIs for timestamp related calculation.

Note, this patch set is depend on the kernel changes for "arm64: perf:
Proper cap_user_time* support" [1].

This patch set has been rebased on mainline kernel with the latest commit 11ba468877bb ("Linux 5.8-rc5") and also applied on top of patch set [1]. After enabling the timestamp, the samples can show the timestamp, and the timestamp is consistent with the kernel's timestamp by checking the kernel log:

  $ perf script -F,+time
    dd  6799 [034] 25496.733475:          1              l1d-access:      ffff87f37b88 _dl_start+0x288 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733475:          1              tlb-access:      ffff87f37b88 _dl_start+0x288 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733479:          1              l1d-access:      ffff87f37c74 _dl_start+0x374 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733479:          1              tlb-access:      ffff87f37c74 _dl_start+0x374 (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733485:          1              l1d-access:      ffff87f49af4 __GI___tunables_init+0x3c (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
    dd  6799 [034] 25496.733485:          1              tlb-access:      ffff87f49af4 __GI___tunables_init+0x3c (/usr/lib/aarch64-linux-gnu/ld-2.28.so)
Quoted 6 lines
Quoted 1 line

[1] https://patchwork.kernel.org/project/linux-arm-kernel/list/?series=318401

This looks good to me! Would it be possible to add a test on the functionality? It's possible to add a C test in tools/perf/arch/arm64/tests or something related to you perf script command line in tools/perf/tests/shell.

Yeah, will look into testing and it's good add two testing cases, one is for arch timer testing in C, another is for SPE testing with shell script.

Thanks for suggestions,
Leo