perf tools: Fix bug when recording SPE and non SPE events
PATCH 1/1 perf tools: Fix bug when recording SPE and non SPE events
This patch fixes an issue when non Arm SPE events are specified after an Arm SPE event. In that case, perf will exit with an error code and not produce a record file. This is because a loop index is used to store the location of the relevant Arm SPE PMU, but if non SPE PMUs follow, that index will be overwritten. Fix this issue by saving the PMU into a variable instead of using the index, and also add an error message.
Before the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? 237
After the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $?
...
0--- tools/perf/arch/arm/util/auxtrace.c | 10 +++++----- tools/perf/arch/arm64/util/arm-spe.c | 1 + 2 files changed, 6 insertions(+), 5 deletions(-)
tools/perf/arch/arm/util/auxtrace.c +5 -5
@@ -54,9 +54,9 @@ struct auxtrace_record *auxtrace_record__init(struct evlist *evlist, int *err) {struct perf_pmu *cs_etm_pmu; +struct perf_pmu *arm_spe_pmu =NULL ;struct evsel *evsel;bool found_etm = false; -bool found_spe = false;static struct perf_pmu **arm_spe_pmus =NULL ;static int nr_spes =0 ;int i =0 ;
@@ -79,13 +79,13 @@ struct auxtrace_recordfor (i =0 ; i < nr_spes; i++) {if (evsel->core.attr.type == arm_spe_pmus[i]->type) { - found_spe = true; + arm_spe_pmu = arm_spe_pmus[i];break ; } } } -if (found_etm && found_spe) { +if (found_etm && arm_spe_pmu) {pr_err ("Concurrent ARM Coresight ETM and SPE operation not currently supported\n" ); *err = -EOPNOTSUPP ;return NULL ;
@@ -95,8 +95,8 @@ struct auxtrace_recordreturn cs_etm_record_init (err);#if defined(__aarch64__) -if (found_spe) -return arm_spe_recording_init (err, arm_spe_pmus[i]); +if (arm_spe_pmu) +return arm_spe_recording_init (err, arm_spe_pmu);#endif /*
tools/perf/arch/arm64/util/arm-spe.c +1 -0
@@ -178,6 +178,7 @@ struct auxtrace_record *arm_spe_recording_init(int *err,struct arm_spe_recording *sper;if (!arm_spe_pmu) { +pr_err ("Attempted to initialise null SPE PMU\n" ); *err = -ENODEV ;return NULL ; }
Signature · 1 line
2.24.0
Hi James,
Quoted 7 lines · James Clark
This patch fixes an issue when non Arm SPE events are specified after an Arm SPE event. In that case, perf will exit with an error code and not produce a record file. This is because a loop index is used to store the location of the relevant Arm SPE PMU, but if non SPE PMUs follow, that index will be overwritten. Fix this issue by saving the PMU into a variable instead of using the index, and also add an error message.
Quoted 8 lines
Before the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? 237After the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? ... 0
Just bring up a question related with PMU event registration. Let's see the DT binding in arch/arm64/boot/dts/arm/fvp-base-revc.dts:
spe-pmu {
compatible = "arm,statistical-profiling-extension-v1";
interrupts = <GIC_PPI 5 IRQ_TYPE_LEVEL_HIGH>;
};Now SPE registers PMU event for every CPU; seem to me, though SPE is an IP binding per CPU, it should register into perf framework with single pmu event, ARM's PMU/ETM/IntelPT all use this way to regsiter PMU event; this can allow perf tool logic to be more neat.
After the driver changes to use single PMU registration, the perf tool code can be changed to use simple way to find perf_pmu and this data structure can be not bound to a specific CPU. Finally, this bug can be smoothly dismissed.
Thanks,
Leo
Quoted 73 lines
Signed-off-by James Clark <[email protected]>Cc Mathieu Poirier <[email protected]>Cc Suzuki K Poulose <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Ingo Molnar <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Cc Mark Rutland <[email protected]>Cc Alexander Shishkin <[email protected]>Cc Jiri Olsa <[email protected]>Cc Namhyung Kim <[email protected]>Cc Igor Lubashev <[email protected]>--- tools/perf/arch/arm/util/auxtrace.c | 10 +++++----- tools/perf/arch/arm64/util/arm-spe.c | 1 + 2 files changed, 6 insertions(+), 5 deletions(-)tools/perf/arch/arm/util/auxtrace.c +5
-5@@ -54,9 +54,9 @@ struct auxtrace_record *auxtrace_record__init(struct evlist *evlist, int *err) {struct perf_pmu *cs_etm_pmu; +struct perf_pmu *arm_spe_pmu =NULL ;struct evsel *evsel;bool found_etm = false; -bool found_spe = false;static struct perf_pmu **arm_spe_pmus =NULL ;static int nr_spes =0 ;int i =0 ;@@ -79,13 +79,13 @@ struct auxtrace_recordfor (i =0 ; i < nr_spes; i++) {if (evsel->core.attr.type == arm_spe_pmus[i]->type) { - found_spe = true; + arm_spe_pmu = arm_spe_pmus[i];break ; } } } -if (found_etm && found_spe) { +if (found_etm && arm_spe_pmu) {pr_err ("Concurrent ARM Coresight ETM and SPE operation not currently supported\n" ); *err = -EOPNOTSUPP ;return NULL ;@@ -95,8 +95,8 @@ struct auxtrace_recordreturn cs_etm_record_init (err);#if defined(__aarch64__) -if (found_spe) -return arm_spe_recording_init (err, arm_spe_pmus[i]); +if (arm_spe_pmu) +return arm_spe_recording_init (err, arm_spe_pmu);#endif /* tools/perf/arch/arm64/util/arm-spe.c +1
-0@@ -178,6 +178,7 @@ struct auxtrace_record *arm_spe_recording_init(int *err,struct arm_spe_recording *sper;if (!arm_spe_pmu) { +pr_err ("Attempted to initialise null SPE PMU\n" ); *err = -ENODEV ;return NULL ; }Signature · 1 line
2.24.0
Hi Leo,
Do you mean that you would never expect there to be more than one SPE file like /sys/bus/event_source/devices/arm_spe_0?
If that is the case then do you know why there is still a number appended to the file?
Thanks
James
Quoted 114 lines · Leo Yan
Hi James,
Quoted 7 lines · James Clark
This patch fixes an issue when non Arm SPE events are specified after an Arm SPE event. In that case, perf will exit with an error code and not produce a record file. This is because a loop index is used to store the location of the relevant Arm SPE PMU, but if non SPE PMUs follow, that index will be overwritten. Fix this issue by saving the PMU into a variable instead of using the index, and also add an error message.
Quoted 8 lines
Before the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? 237After the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? ... 0Just bring up a question related with PMU event registration. Let's see the DT binding in
arch/arm64/boot/dts/arm/fvp-base-revc.dts:spe-pmu { compatible = "arm,statistical-profiling-extension-v1"; interrupts = <GIC_PPI 5 IRQ_TYPE_LEVEL_HIGH>; };Now SPE registers PMU event for every CPU; seem to me, though SPE is an IP binding per CPU, it should register into perf framework with single pmu event, ARM's PMU/ETM/IntelPT all use this way to regsiter PMU event; this can allow perf tool logic to be more neat.
After the driver changes to use single PMU registration, the perf tool code can be changed to use simple way to find
perf_pmuand this data structure can be not bound to a specific CPU. Finally, this bug can be smoothly dismissed.Thanks,
LeoQuoted 73 lines
Signed-off-by James Clark <[email protected]>Cc Mathieu Poirier <[email protected]>Cc Suzuki K Poulose <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Ingo Molnar <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Cc Mark Rutland <[email protected]>Cc Alexander Shishkin <[email protected]>Cc Jiri Olsa <[email protected]>Cc Namhyung Kim <[email protected]>Cc Igor Lubashev <[email protected]>--- tools/perf/arch/arm/util/auxtrace.c | 10 +++++----- tools/perf/arch/arm64/util/arm-spe.c | 1 + 2 files changed, 6 insertions(+), 5 deletions(-)tools/perf/arch/arm/util/auxtrace.c +5
-5@@ -54,9 +54,9 @@ struct auxtrace_record *auxtrace_record__init(struct evlist *evlist, int *err) {struct perf_pmu *cs_etm_pmu; +struct perf_pmu *arm_spe_pmu =NULL ;struct evsel *evsel;bool found_etm = false; -bool found_spe = false;static struct perf_pmu **arm_spe_pmus =NULL ;static int nr_spes =0 ;int i =0 ;@@ -79,13 +79,13 @@ struct auxtrace_recordfor (i =0 ; i < nr_spes; i++) {if (evsel->core.attr.type == arm_spe_pmus[i]->type) { - found_spe = true; + arm_spe_pmu = arm_spe_pmus[i];break ; } } } -if (found_etm && found_spe) { +if (found_etm && arm_spe_pmu) {pr_err ("Concurrent ARM Coresight ETM and SPE operation not currently supported\n" ); *err = -EOPNOTSUPP ;return NULL ;@@ -95,8 +95,8 @@ struct auxtrace_recordreturn cs_etm_record_init (err);#if defined(__aarch64__) -if (found_spe) -return arm_spe_recording_init (err, arm_spe_pmus[i]); +if (arm_spe_pmu) +return arm_spe_recording_init (err, arm_spe_pmu);#endif /* tools/perf/arch/arm64/util/arm-spe.c +1
-0@@ -178,6 +178,7 @@ struct auxtrace_record *arm_spe_recording_init(int *err,struct arm_spe_recording *sper;if (!arm_spe_pmu) { +pr_err ("Attempted to initialise null SPE PMU\n" ); *err = -ENODEV ;return NULL ; }Signature · 1 line
2.24.0
Hi James,
Quoted 3 lines · James Clark
Hi Leo,
Do you mean that you would never expect there to be more than one SPE file like
/sys/bus/event_source/devices/arm_spe_0?
Yeah.
To be more accurate, I'd suggest to be only one SPE file if CPUs have the same SPE version. If there have multiple SPE files under '/sys/bus/event_source/devices/', this means the CPUs have different SPE versions
Let's see an example for SMP platform with 4xCA53 CPUs:/sys/bus/event_source/devices/armv8_cortex_a53
For big.LITTLE system, we can see below nodes:/sys/bus/event_source/devices/armv8_cortex_a53/sys/bus/event_source/devices/armv8_cortex_a72
If SPE has the same IP for all CPUs, this would be simple to only create on file under /sys/bus/event_source/devices/.
Quoted 1 line
If that is the case then do you know why there is still a number appended to the file?
This is caused by the code [1]. But I don't know what's the reason for adding index to PMU event name.
Thanks,
Leo Yan
Quoted 115 lines · Leo Yan
Quoted 114 lines
Hi James,
Quoted 7 lines · James Clark
This patch fixes an issue when non Arm SPE events are specified after an Arm SPE event. In that case, perf will exit with an error code and not produce a record file. This is because a loop index is used to store the location of the relevant Arm SPE PMU, but if non SPE PMUs follow, that index will be overwritten. Fix this issue by saving the PMU into a variable instead of using the index, and also add an error message.
Quoted 8 lines
Before the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? 237After the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? ... 0Just bring up a question related with PMU event registration. Let's see the DT binding in
arch/arm64/boot/dts/arm/fvp-base-revc.dts:spe-pmu { compatible = "arm,statistical-profiling-extension-v1"; interrupts = <GIC_PPI 5 IRQ_TYPE_LEVEL_HIGH>; };Now SPE registers PMU event for every CPU; seem to me, though SPE is an IP binding per CPU, it should register into perf framework with single pmu event, ARM's PMU/ETM/IntelPT all use this way to regsiter PMU event; this can allow perf tool logic to be more neat.
After the driver changes to use single PMU registration, the perf tool code can be changed to use simple way to find
perf_pmuand this data structure can be not bound to a specific CPU. Finally, this bug can be smoothly dismissed.Thanks,
LeoQuoted 73 lines
Signed-off-by James Clark <[email protected]>Cc Mathieu Poirier <[email protected]>Cc Suzuki K Poulose <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Ingo Molnar <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Cc Mark Rutland <[email protected]>Cc Alexander Shishkin <[email protected]>Cc Jiri Olsa <[email protected]>Cc Namhyung Kim <[email protected]>Cc Igor Lubashev <[email protected]>--- tools/perf/arch/arm/util/auxtrace.c | 10 +++++----- tools/perf/arch/arm64/util/arm-spe.c | 1 + 2 files changed, 6 insertions(+), 5 deletions(-)tools/perf/arch/arm/util/auxtrace.c +5
-5@@ -54,9 +54,9 @@ struct auxtrace_record *auxtrace_record__init(struct evlist *evlist, int *err) {struct perf_pmu *cs_etm_pmu; +struct perf_pmu *arm_spe_pmu =NULL ;struct evsel *evsel;bool found_etm = false; -bool found_spe = false;static struct perf_pmu **arm_spe_pmus =NULL ;static int nr_spes =0 ;int i =0 ;@@ -79,13 +79,13 @@ struct auxtrace_recordfor (i =0 ; i < nr_spes; i++) {if (evsel->core.attr.type == arm_spe_pmus[i]->type) { - found_spe = true; + arm_spe_pmu = arm_spe_pmus[i];break ; } } } -if (found_etm && found_spe) { +if (found_etm && arm_spe_pmu) {pr_err ("Concurrent ARM Coresight ETM and SPE operation not currently supported\n" ); *err = -EOPNOTSUPP ;return NULL ;@@ -95,8 +95,8 @@ struct auxtrace_recordreturn cs_etm_record_init (err);#if defined(__aarch64__) -if (found_spe) -return arm_spe_recording_init (err, arm_spe_pmus[i]); +if (arm_spe_pmu) +return arm_spe_recording_init (err, arm_spe_pmu);#endif /* tools/perf/arch/arm64/util/arm-spe.c +1
-0@@ -178,6 +178,7 @@ struct auxtrace_record *arm_spe_recording_init(int *err,struct arm_spe_recording *sper;if (!arm_spe_pmu) { +pr_err ("Attempted to initialise null SPE PMU\n" ); *err = -ENODEV ;return NULL ; }Signature · 1 line
2.24.0
Suzuki Kuruppassery Poulose
maintainerHi Leo,
Quoted 22 lines · Leo Yan
Hi James,
Quoted 7 lines · James Clark
This patch fixes an issue when non Arm SPE events are specified after an Arm SPE event. In that case, perf will exit with an error code and not produce a record file. This is because a loop index is used to store the location of the relevant Arm SPE PMU, but if non SPE PMUs follow, that index will be overwritten. Fix this issue by saving the PMU into a variable instead of using the index, and also add an error message.
Quoted 8 lines
Before the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? 237After the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? ... 0Just bring up a question related with PMU event registration. Let's see the DT binding in
arch/arm64/boot/dts/arm/fvp-base-revc.dts:
Quoted 7 lines
spe-pmu { compatible = "arm,statistical-profiling-extension-v1"; interrupts = <GIC_PPI 5 IRQ_TYPE_LEVEL_HIGH>; };Now SPE registers PMU event for every CPU; seem to me, though SPE is an
Do you mean "SPE PMU" here ? SPE is different from ETM, where the trace data is micro-architecture dependent. And thus you cannot mix the trace on different CPUs with different micro-archs.
As such I don't see any issue with this patch.
Suzuki
Hi Suzuki,
Quoted 31 lines · Suzuki Kuruppassery Poulose
Hi Leo,
Quoted 28 lines · Leo Yan
Hi James,
Quoted 7 lines · James Clark
This patch fixes an issue when non Arm SPE events are specified after an Arm SPE event. In that case, perf will exit with an error code and not produce a record file. This is because a loop index is used to store the location of the relevant Arm SPE PMU, but if non SPE PMUs follow, that index will be overwritten. Fix this issue by saving the PMU into a variable instead of using the index, and also add an error message.
Quoted 8 lines
Before the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? 237After the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? ... 0Just bring up a question related with PMU event registration. Let's see the DT binding in
arch/arm64/boot/dts/arm/fvp-base-revc.dts:spe-pmu { compatible = "arm,statistical-profiling-extension-v1"; interrupts = <GIC_PPI 5 IRQ_TYPE_LEVEL_HIGH>; };
Quoted 5 lines
Quoted 1 line
Now SPE registers PMU event for every CPU; seem to me, though SPE is an
Do you mean "SPE PMU" here ? SPE is different from ETM, where the trace data is micro-architecture dependent. And thus you cannot mix the trace on different CPUs with different micro-archs.
Understood that SPE is micro-architecture dependent.
Usually, we should register PMU event once for the same SPE version and CPUs can create multiple instances. My concern at here is the PMU event is regsitered for multiple times for the same SPE version. Please correct me if I misunderstand.
Quoted 1 line
As such I don't see any issue with this patch.
Regard this patch, it does fix the issue if based on current PMU event registration; so it's okay for me to merge it if now it's not necessary to change PMU event registration.
Thanks,
Leo Yan
On Mon, Jan 13, 2020 at 10:17:51PM +0800, Leo Yan wrote:
[...]
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines · James Clark
Quoted 7 lines
This patch fixes an issue when non Arm SPE events are specified after an Arm SPE event. In that case, perf will exit with an error code and not produce a record file. This is because a loop index is used to store the location of the relevant Arm SPE PMU, but if non SPE PMUs follow, that index will be overwritten. Fix this issue by saving the PMU into a variable instead of using the index, and also add an error message.
Quoted 8 lines
Before the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? 237After the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? ... 0Just bring up a question related with PMU event registration. Let's see the DT binding in
arch/arm64/boot/dts/arm/fvp-base-revc.dts:spe-pmu { compatible = "arm,statistical-profiling-extension-v1"; interrupts = <GIC_PPI 5 IRQ_TYPE_LEVEL_HIGH>; };
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Now SPE registers PMU event for every CPU; seem to me, though SPE is an
Do you mean "SPE PMU" here ? SPE is different from ETM, where the trace data is micro-architecture dependent. And thus you cannot mix the trace on different CPUs with different micro-archs.
Understood that SPE is micro-architecture dependent.
Maybe SPE is more general than we think :)
Since SPE is defined in ARMv8 architecture reference manual (ARM DDI 0487D.a); should SPE trace data format is unified and defined in Chapter D9 "Statistical Profiling Extension Sample Record Specification"?
Thanks,
Leo
Hi Leo,
Quoted 4 lines
Since SPE is defined in ARMv8 architecture reference manual (ARM DDI 0487D.a); should SPE trace data format is unified and defined in Chapter D9 "Statistical Profiling Extension Sample Record Specification"?
I'm not sure what you mean exactly, but the trace data format is described in section D10.
Thanks
James
Quoted 16 lines · James Clark
This patch fixes an issue when non Arm SPE events are specified after an Arm SPE event. In that case, perf will exit with an error code and not produce a record file. This is because a loop index is used to store the location of the relevant Arm SPE PMU, but if non SPE PMUs follow, that index will be overwritten. Fix this issue by saving the PMU into a variable instead of using the index, and also add an error message.
Before the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? 237After the fix:
./perf record -e arm_spe/ts_enable=1/ -e branch-misses ls; echo $? ... 0
Quoted 11 lines
Signed-off-by James Clark <[email protected]>Cc Mathieu Poirier <[email protected]>Cc Suzuki K Poulose <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Ingo Molnar <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Cc Mark Rutland <[email protected]>Cc Alexander Shishkin <[email protected]>Cc Jiri Olsa <[email protected]>Cc Namhyung Kim <[email protected]>Cc Igor Lubashev <[email protected]>