Skip to contentSkip to conversation

TPEBS counting mode support

RFC PATCH Superseded [email protected] 3 participants 27 messages May 15, 2024 – May 21, 2024 perf events

v8 of 18 Superseded Latest is v18 →

lore
Share & tools
lore
#0 lore raw

v8 cover TPEBS counting mode support

From: Weilin Wang <[email protected]>

Changes in v8:
- In this revision, the code is updated to base on Ian's patch on R modifier parser https://lore.kernel.org/lkml/[email protected]/
After this change, there is no special code required for R modifier in metricgroup.c and metricgroup.h files.

Caveat of this change:
Ideally, we will need to add special handling to skip counting events with R
modifier in evsel. Currently, this is not implemented so the event with :R will be both counted and sampled. Usually, in a metric formula that uses retire_latency, it would already require to count the event. As a result, we will endup count the same event twice. This should be able to be handled properly when we finalize our design on evsel R modifier support.

- Move TPEBS specific code out from main perf stat code to separate files in util/intel-tpebs.c and util/intel-tpebs.h. [Namhyung]
- Use --control:fifo to ack perf stat from forked perf record instead of sleep(2) [Namhyung]
- Add introductions about TPEBS and R modifier in Documents. [Namhyung]

Changes in v7:
- Update code and comments for better code quality [Namhyung]
- Add a separate commit for perf data [Namhyung]
- Update retire latency print function to improve alignment [Namhyung]

Changes in v6:
- Update code and add comments for better code quality [Namhyung]
- Remove the added fd var and directly pass the opened fd to data.file.fd [Namhyung]
- Add kill() to stop perf record when perf stat exists early [Namhyung]
- Add command opt check to ensure only start perf record when -a/-C given [Namhyung]
- Squash commits [Namhyung]

Changes in v5:
- Update code and add comments for better code quality [Ian]

Changes in v4:
- Remove uncessary debug print and update code and comments for better readability and quality [Namhyung]
- Update mtl metric json file with consistent TmaL1 and TopdownL1 metricgroup

Changes in v3:
- Remove ':' when event name has '@' [Ian]
- Use 'R' as the modifier instead of "retire_latency" [Ian]

Changes in v2:
- Add MTL metric file
- Add more descriptions and example to the patch [Arnaldo]

Here is an example of running perf stat to collect a metric that uses retire_latency value of event MEM_INST_RETIRED.STLB_HIT_STORES on a MTL system.

In this simple example, there is no MEM_INST_RETIRED.STLB_HIT_STORES sample. Therefore, the MEM_INST_RETIRED.STLB_HIT_STORES:p count and retire_latency value are all 0.

./perf stat -M tma_dtlb_store -a -- sleep 1

[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.000 MB - ]

Performance counter stats for 'system wide':

       181,047,168      cpu_core/TOPDOWN.SLOTS/          #      0.6 %  tma_dtlb_store
         3,195,608      cpu_core/topdown-retiring/
        40,156,649      cpu_core/topdown-mem-bound/
         3,550,925      cpu_core/topdown-bad-spec/
       117,571,818      cpu_core/topdown-fe-bound/
        57,118,087      cpu_core/topdown-be-bound/
            69,179      cpu_core/EXE_ACTIVITY.BOUND_ON_STORES/
             4,582      cpu_core/MEM_INST_RETIRED.STLB_HIT_STORES/
        30,183,104      cpu_core/CPU_CLK_UNHALTED.DISTRIBUTED/
        30,556,790      cpu_core/CPU_CLK_UNHALTED.THREAD/
           168,486      cpu_core/DTLB_STORE_MISSES.WALK_ACTIVE/

0.00 MEM_INST_RETIRED.STLB_HIT_STORES:p 0 0

1.003105924 seconds time elapsed

v1:
TPEBS is one of the features provided by the next generation of Intel PMU. Please refer to Section 8.4.1 of "Intel® Architecture Instruction Set Extensions Programming Reference" [1] for more details about this feature.

This set of patches supports TPEBS in counting mode. The code works in the following way: it forks a perf record process from perf stat when retire_latency of one or more events are used in a metric formula. Perf stat would send a SIGTERM signal to perf record before it needs the retire latency value for metric calculation. Perf stat will then process sample data to extract the retire latency data for metric calculations. Currently, the code uses the arithmetic average of retire latency values.

[1] https://www.intel.com/content/www/us/en/content-details/812218/intel-architecture-instruction-set-extensions-programming-reference.html?wapkw=future%20features

Weilin Wang (7):
perf Document: Add TPEBS to Documents
perf data: Allow to use given fd in data->file.fd
perf stat: Fork and launch perf record when perf stat needs to get
retire latency value for a metric.
perf stat: Add retire latency values into the expr_parse_ctx to
prepare for final metric calculation
perf stat: Add retire latency print functions to print out at the very
end of print out
perf vendor events intel: Add MTL metric json files
perf stat: Skip read retire_lat counters and plugin retire_lat data
from sampled data

 tools/perf/Documentation/perf-list.txt        |    1 +
 tools/perf/Documentation/topdown.txt          |   18 +
 tools/perf/arch/x86/util/evlist.c             |    6 +
 tools/perf/builtin-stat.c                     |   19 +
 .../arch/x86/meteorlake/metricgroups.json     |  127 +
 .../arch/x86/meteorlake/mtl-metrics.json      | 2551 +++++++++++++++++
 tools/perf/util/Build                         |    1 +
 tools/perf/util/data.c                        |    7 +-
 tools/perf/util/evsel.h                       |    5 +
 tools/perf/util/intel-tpebs.c                 |  285 ++
 tools/perf/util/intel-tpebs.h                 |   29 +
 tools/perf/util/stat-display.c                |   74 +
 tools/perf/util/stat-shadow.c                 |   23 +
 tools/perf/util/stat.h                        |    3 +
 14 files changed, 3148 insertions(+), 1 deletion(-)
 create mode 100644 tools/perf/pmu-events/arch/x86/meteorlake/metricgroups.json
 create mode 100644 tools/perf/pmu-events/arch/x86/meteorlake/mtl-metrics.json
 create mode 100644 tools/perf/util/intel-tpebs.c
 create mode 100644 tools/perf/util/intel-tpebs.h

--
2.43.0

#1 lore raw

RFC PATCH v8 1/7 perf Document: Add TPEBS to Documents

From: Weilin Wang <[email protected]>

TPEBS is a new feature in next generation of Intel PMU. It will be used in new TMA releases. Adding related introduction to documents while adding new code to support it in perf stat.

---
 tools/perf/Documentation/perf-list.txt |  1 +
 tools/perf/Documentation/topdown.txt   | 18 ++++++++++++++++++
 2 files changed, 19 insertions(+)
tools/perf/Documentation/perf-list.txt +1 -0
index 6bf2468f59d3..dea005410ec0 100644
@@ -72,6 +72,7 @@ counted. The following modifiers exist:
  W - group is weak and will fallback to non-group if not schedulable,
  e - group or event are exclusive and do not share the PMU
  b - use BPF aggregration (see perf stat --bpf-counters)
+ R - retire latency value of the event
 
 The 'p' modifier can be used for specifying how precise the instruction
 address should be. The 'p' modifier can be specified multiple times:
tools/perf/Documentation/topdown.txt +18 -0
index ae0aee86844f..e6c4424e8bf2 100644
@@ -325,6 +325,24 @@ other four level 2 metrics by subtracting corresponding metrics as below.
     Fetch_Bandwidth = Frontend_Bound - Fetch_Latency
     Core_Bound = Backend_Bound - Memory_Bound
 
+TPEBS in TopDown
+================
+
+TPEBS is one of the features provided by the next generation of Intel PMU. The
+TPEBS feature adds a 16 bit retire latency field in the Basic Info group of the
+PEBS record. It records the Core cycles since the retirement of the previous
+instruction to the retirement of current instruction. Please refer to Section
+8.4.1 of "Intel® Architecture Instruction Set Extensions Programming Reference"
+for more details about this feature.
+
+In the most recent release of TMA, the metrics begin to use event retire_latency
+values in some of the metrics’ formulas on processors that support TPEBS feature.
+For previous generations that do not support TPEBS, the values are static and
+predefined per processor family by the hardware architects. Due to the diversity
+of workloads in execution environments, retire latency values measured at real
+time are more accurate. Therefore, new TMA metrics that use TPEBS will provide
+more accurate performance analysis results.
+
 
 [1] https://software.intel.com/en-us/top-down-microarchitecture-analysis-method-win
 [2] https://sites.google.com/site/analysismethods/yasin-pubs
Signature · 1 line
2.43.0
reviewer
#2 lore raw

Re: [RFC PATCH v8 1/7] perf Document: Add TPEBS to Documents

Quoted 30 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

TPEBS is a new feature in next generation of Intel PMU. It will be used in new TMA releases. Adding related introduction to documents while adding new code to support it in perf stat.

---
 tools/perf/Documentation/perf-list.txt |  1 +
 tools/perf/Documentation/topdown.txt   | 18 ++++++++++++++++++
 2 files changed, 19 insertions(+)
tools/perf/Documentation/perf-list.txt +1 -0
index 6bf2468f59d3..dea005410ec0 100644
@@ -72,6 +72,7 @@ counted. The following modifiers exist:
  W - group is weak and will fallback to non-group if not schedulable,
  e - group or event are exclusive and do not share the PMU
  b - use BPF aggregration (see perf stat --bpf-counters)
+ R - retire latency value of the event
 
 The 'p' modifier can be used for specifying how precise the instruction
 address should be. The 'p' modifier can be specified multiple times:
tools/perf/Documentation/topdown.txt +0 -0
index ae0aee86844f..e6c4424e8bf2 100644

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting corresponding metrics as below. · line 331

Quoted 7 lines
tools/perf/Documentation/topdown.txt +4 -0
@@ -325,6 +325,24 @@ other four level 2 metrics by subtracting corresponding metrics as below.
     Fetch_Bandwidth = Frontend_Bound - Fetch_Latency
     Core_Bound = Backend_Bound - Memory_Bound
 
+TPEBS in TopDown
+================
+
+TPEBS is one of the features provided by the next generation of Intel PMU. The

As this documentation will live a while "next generation" could become ambiguous. I think it would be better to mention core ultra or some other term to more specifically describe which PMUs have TPEBS.

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting corresponding metrics as below. · line 336

Quoted 5 lines
tools/perf/Documentation/topdown.txt +5 -0
+TPEBS feature adds a 16 bit retire latency field in the Basic Info group of the
+PEBS record. It records the Core cycles since the retirement of the previous
+instruction to the retirement of current instruction. Please refer to Section
+8.4.1 of "Intel® Architecture Instruction Set Extensions Programming Reference"
+for more details about this feature.

Perhaps capture that this is placed in the perf event sample in the weights section as TPEBS isn't exposed except within the kernel PMU driver.

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting corresponding metrics as below. · line 344

Quoted 8 lines
tools/perf/Documentation/topdown.txt +8 -0
+
+In the most recent release of TMA, the metrics begin to use event retire_latency
+values in some of the metrics’ formulas on processors that support TPEBS feature.
+For previous generations that do not support TPEBS, the values are static and
+predefined per processor family by the hardware architects. Due to the diversity
+of workloads in execution environments, retire latency values measured at real
+time are more accurate. Therefore, new TMA metrics that use TPEBS will provide
+more accurate performance analysis results.

Do you want to capture what the value will be when there hasn't been a sample? This corner case could be considered broken in the new approach.

Thanks,
Ian

Quoted 7 lines
tools/perf/Documentation/topdown.txt +1 -1
+
 
 [1] https://software.intel.com/en-us/top-down-microarchitecture-analysis-method-win
 [2] https://sites.google.com/site/analysismethods/yasin-pubs
--

2.43.0

#3 lore raw

RE: [RFC PATCH v8 1/7] perf Document: Add TPEBS to Documents

Quoted 60 lines

-----Original Message-----
From: Ian Rogers <[email protected]>
Sent: Thursday, May 16, 2024 9:11 AM
To: Wang, Weilin <[email protected]>
Cc: Namhyung Kim <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 1/7] perf Document: Add TPEBS to Documents

Quoted 5 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

TPEBS is a new feature in next generation of Intel PMU. It will be used in new TMA releases. Adding related introduction to documents while adding new

code to

Review

Quoted 9 lines

support it in perf stat.

---
 tools/perf/Documentation/perf-list.txt |  1 +
 tools/perf/Documentation/topdown.txt   | 18 ++++++++++++++++++
 2 files changed, 19 insertions(+)
(quoted diff) +0 -0

b/tools/perf/Documentation/perf-list.txt

Review

Quoted 12 lines

index 6bf2468f59d3..dea005410ec0 100644

tools/perf/Documentation/perf-list.txt +1 -0
@@ -72,6 +72,7 @@ counted. The following modifiers exist:
  W - group is weak and will fallback to non-group if not schedulable,
  e - group or event are exclusive and do not share the PMU
  b - use BPF aggregration (see perf stat --bpf-counters)
+ R - retire latency value of the event
 
 The 'p' modifier can be used for specifying how precise the instruction
 address should be. The 'p' modifier can be specified multiple times:
(quoted diff) +0 -0

b/tools/perf/Documentation/topdown.txt

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting

Quoted 4 lines

index ae0aee86844f..e6c4424e8bf2 100644

tools/perf/Documentation/topdown.txt +0 -0
@@ -325,6 +325,24 @@ other four level 2 metrics by subtracting

corresponding metrics as below.

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting corresponding metrics as below.

Quoted 7 lines
tools/perf/Documentation/topdown.txt +4 -0
     Fetch_Bandwidth = Frontend_Bound - Fetch_Latency
     Core_Bound = Backend_Bound - Memory_Bound
 
+TPEBS in TopDown
+================
+
+TPEBS is one of the features provided by the next generation of Intel PMU.

The

Quoted 3 lines

As this documentation will live a while "next generation" could become ambiguous. I think it would be better to mention core ultra or some other term to more specifically describe which PMUs have TPEBS.

Hi Ian,

Yes, you are right, I will update it.

Quoted 22 lines

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting corresponding metrics as below.

Quoted 5 lines
tools/perf/Documentation/topdown.txt +4 -0
+TPEBS feature adds a 16 bit retire latency field in the Basic Info group of the
+PEBS record. It records the Core cycles since the retirement of the previous
+instruction to the retirement of current instruction. Please refer to Section
+8.4.1 of "Intel® Architecture Instruction Set Extensions Programming

Reference"

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting corresponding metrics as below. · line 336

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+for more details about this feature.

Perhaps capture that this is placed in the perf event sample in the weights section as TPEBS isn't exposed except within the kernel PMU driver.

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 338

Quoted 2 lines
tools/perf/Documentation/topdown.txt +2 -0
+
+In the most recent release of TMA, the metrics begin to use event

retire_latency

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 339

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+values in some of the metrics’ formulas on processors that support TPEBS

feature.

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 340

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+For previous generations that do not support TPEBS, the values are static

and

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 341

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+predefined per processor family by the hardware architects. Due to the

diversity

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 342

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+of workloads in execution environments, retire latency values measured at
Quoted 8 lines

real

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 343

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+time are more accurate. Therefore, new TMA metrics that use TPEBS will

provide

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting corresponding metrics as below. · line 344

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+more accurate performance analysis results.

Do you want to capture what the value will be when there hasn't been a sample? This corner case could be considered broken in the new approach.

When there is no sample, we should expect it to be 0 or use default value. I will add this information here. I don’t think this is broken from the approach's perspective. We do need to add code to return this value when there is no sample.

Thanks,
Weilin

Quoted 11 lines

Thanks,
Ian

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting corresponding metrics as below.

Quoted 3 lines
tools/perf/Documentation/topdown.txt +1 -0
+
 
 [1] https://software.intel.com/en-us/top-down-microarchitecture-analysis-

method-win

Quoted 4 lines
tools/perf/Documentation/topdown.txt +0 -1
 [2] https://sites.google.com/site/analysismethods/yasin-pubs
--

2.43.0

maintainer
#4 lore raw

Re: [RFC PATCH v8 1/7] perf Document: Add TPEBS to Documents

Hello,

Quoted 61 lines · Weilin
Quoted 61 lines

-----Original Message-----
From: Ian Rogers <[email protected]>
Sent: Thursday, May 16, 2024 9:11 AM
To: Wang, Weilin <[email protected]>
Cc: Namhyung Kim <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 1/7] perf Document: Add TPEBS to Documents

Quoted 5 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

TPEBS is a new feature in next generation of Intel PMU. It will be used in new TMA releases. Adding related introduction to documents while adding new

code to

Review

Quoted 9 lines

support it in perf stat.

---
 tools/perf/Documentation/perf-list.txt |  1 +
 tools/perf/Documentation/topdown.txt   | 18 ++++++++++++++++++
 2 files changed, 19 insertions(+)
(quoted diff) +0 -0

b/tools/perf/Documentation/perf-list.txt

Review

Quoted 12 lines

index 6bf2468f59d3..dea005410ec0 100644

tools/perf/Documentation/perf-list.txt +1 -0
@@ -72,6 +72,7 @@ counted. The following modifiers exist:
  W - group is weak and will fallback to non-group if not schedulable,
  e - group or event are exclusive and do not share the PMU
  b - use BPF aggregration (see perf stat --bpf-counters)
+ R - retire latency value of the event
 
 The 'p' modifier can be used for specifying how precise the instruction
 address should be. The 'p' modifier can be specified multiple times:
(quoted diff) +0 -0

b/tools/perf/Documentation/topdown.txt

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting

Quoted 4 lines

index ae0aee86844f..e6c4424e8bf2 100644

tools/perf/Documentation/topdown.txt +0 -0
@@ -325,6 +325,24 @@ other four level 2 metrics by subtracting

corresponding metrics as below.

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting corresponding metrics as below.

Quoted 7 lines
tools/perf/Documentation/topdown.txt +4 -0
     Fetch_Bandwidth = Frontend_Bound - Fetch_Latency
     Core_Bound = Backend_Bound - Memory_Bound
 
+TPEBS in TopDown
+================
+
+TPEBS is one of the features provided by the next generation of Intel PMU.

The

Quoted 7 lines
Quoted 3 lines

As this documentation will live a while "next generation" could become ambiguous. I think it would be better to mention core ultra or some other term to more specifically describe which PMUs have TPEBS.

Hi Ian,

Yes, you are right, I will update it.

Also I'd be nice if you can tell what 'T' means in TPEBS. :)

Quoted 27 lines
Quoted 27 lines

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting corresponding metrics as below.

Quoted 5 lines
tools/perf/Documentation/topdown.txt +4 -0
+TPEBS feature adds a 16 bit retire latency field in the Basic Info group of the
+PEBS record. It records the Core cycles since the retirement of the previous
+instruction to the retirement of current instruction. Please refer to Section
+8.4.1 of "Intel® Architecture Instruction Set Extensions Programming

Reference"

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting corresponding metrics as below. · line 336

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+for more details about this feature.

Perhaps capture that this is placed in the perf event sample in the weights section as TPEBS isn't exposed except within the kernel PMU driver.

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 338

Quoted 2 lines
tools/perf/Documentation/topdown.txt +2 -0
+
+In the most recent release of TMA, the metrics begin to use event

retire_latency

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 339

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+values in some of the metrics’ formulas on processors that support TPEBS

feature.

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 340

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+For previous generations that do not support TPEBS, the values are static

and

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 341

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+predefined per processor family by the hardware architects. Due to the

diversity

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 342

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+of workloads in execution environments, retire latency values measured at

real

tools/perf/Documentation/topdown.txt other four level 2 metrics by subtracting · line 343

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+time are more accurate. Therefore, new TMA metrics that use TPEBS will

provide

Quoted 1 line
tools/perf/Documentation/topdown.txt +1 -0
+more accurate performance analysis results.
Quoted 8 lines
Quoted 3 lines

Do you want to capture what the value will be when there hasn't been a sample? This corner case could be considered broken in the new approach.

When there is no sample, we should expect it to be 0 or use default value. I will add this information here. I don’t think this is broken from the approach's perspective. We do need to add code to return this value when there is no sample.

More importantly, I think the documentation should say that the retire_latency is in PEBS which means it needs samples in the precise events. So it would run perf record in the background for events with retire_latency even if users just want to see the value of counters or metric in perf stat.

Thanks,
Namhyung

#5 lore raw

RFC PATCH v8 2/7 perf data: Allow to use given fd in data->file.fd

In reply to [email protected]

From: Weilin Wang <[email protected]>

When in PIPE mode, allow to use fd dynamically opened and asigned to data->file.fd instead of STDIN_FILENO or STDOUT_FILENO.

---
 tools/perf/util/data.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tools/perf/util/data.c +6 -1
index 08c4bfbd817f..98661ede2a73 100644
@@ -204,7 +204,12 @@ static bool check_pipe(struct perf_data *data)
 				data->file.fd = fd;
 				data->use_stdio = false;
 			}
-		} else {
+
+		/*
+		 * When is_pipe and data->file.fd is given, use given fd
+		 * instead of STDIN_FILENO or STDOUT_FILENO
+		 */
+		} else if (data->file.fd <= 0) {
 			data->file.fd = fd;
 		}
 	}
Signature · 1 line
2.43.0
reviewer Reviewed-by on v8 2/7
#6 lore raw

Re: [RFC PATCH v8 2/7] perf data: Allow to use given fd in data->file.fd

Quoted 7 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

When in PIPE mode, allow to use fd dynamically opened and asigned to data->file.fd instead of STDIN_FILENO or STDOUT_FILENO.

Thanks,
Ian

Quoted 25 lines
---
 tools/perf/util/data.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tools/perf/util/data.c +6 -2
index 08c4bfbd817f..98661ede2a73 100644
@@ -204,7 +204,12 @@ static bool check_pipe(struct perf_data *data)
                                data->file.fd = fd;
                                data->use_stdio = false;
                        }
-               } else {
+
+               /*
+                * When is_pipe and data->file.fd is given, use given fd
+                * instead of STDIN_FILENO or STDOUT_FILENO
+                */
+               } else if (data->file.fd <= 0) {
                        data->file.fd = fd;
                }
        }
--

2.43.0

#7 lore raw

RFC PATCH v8 3/7 perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

In reply to [email protected]

From: Weilin Wang <[email protected]>

When retire_latency value is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect required retire_latency values in parallel while perf stat is collecting counting values.

At the point of time that perf stat stops counting, it would send sigterm signal to perf record process and receiving sampling data back from perf record from a pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.

Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.

---
 tools/perf/builtin-stat.c     |  19 +++
 tools/perf/util/Build         |   1 +
 tools/perf/util/intel-tpebs.c | 285 ++++++++++++++++++++++++++++++++++
 tools/perf/util/intel-tpebs.h |  29 ++++
 tools/perf/util/stat.h        |   3 +
 5 files changed, 337 insertions(+)
 create mode 100644 tools/perf/util/intel-tpebs.c
 create mode 100644 tools/perf/util/intel-tpebs.h
tools/perf/builtin-stat.c +19 -0
index 428e9721b908..85927cf45adb 100644
@@ -70,6 +70,7 @@
 #include "util/bpf_counter.h"
 #include "util/iostat.h"
 #include "util/util.h"
+#include "util/intel-tpebs.h"
 #include "asm/bug.h"
 
 #include <linux/time64.h>
@@ -94,6 +95,7 @@
 #include <perf/evlist.h>
 #include <internal/threadmap.h>
 
+
 #define DEFAULT_SEPARATOR	" "
 #define FREEZE_ON_SMI_PATH	"devices/cpu/freeze_on_smi"
 
@@ -162,6 +164,8 @@ static struct perf_stat_config stat_config = {
 	.ctl_fd			= -1,
 	.ctl_fd_ack		= -1,
 	.iostat_run		= false,
+	.tpebs_results		= LIST_HEAD_INIT(stat_config.tpebs_results),
+	.tpebs_pid              = -1,
 };
 
 static void evlist__check_cpu_maps(struct evlist *evlist)
@@ -653,6 +657,8 @@ static enum counter_recovery stat_handle_error(struct evsel *counter)
 
 	if (child_pid != -1)
 		kill(child_pid, SIGTERM);
+	if (stat_config.tpebs_pid != -1)
+		stop_tpebs();
 	return COUNTER_FATAL;
 }
 
@@ -673,6 +679,10 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx)
 	int err;
 	bool second_pass = false;
 
+	err = start_tpebs(&stat_config, evsel_list);
+	if (err < 0)
+		return err;
+
 	if (forks) {
 		if (evlist__prepare_workload(evsel_list, &target, argv, is_pipe, workload_exec_failed_signal) < 0) {
 			perror("failed to prepare workload");
@@ -878,6 +888,10 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx)
 
 	t1 = rdclock();
 
+	err = stop_tpebs();
+	if (err < 0)
+		return err;
+
 	if (stat_config.walltime_run_table)
 		stat_config.walltime_run[run_idx] = t1 - t0;
 
@@ -985,6 +999,9 @@ static void sig_atexit(void)
 	if (child_pid != -1)
 		kill(child_pid, SIGTERM);
 
+	if (stat_config.tpebs_pid != -1)
+		kill(stat_config.tpebs_pid, SIGTERM);
+
 	sigprocmask(SIG_SETMASK, &oset, NULL);
 
 	if (signr == -1)
@@ -2918,5 +2935,7 @@ int cmd_stat(int argc, const char **argv)
 	metricgroup__rblist_exit(&stat_config.metric_events);
 	evlist__close_control(stat_config.ctl_fd, stat_config.ctl_fd_ack, &stat_config.ctl_fd_close);
 
+	tpebs_data__delete();
+
 	return status;
 }
tools/perf/util/Build +1 -0
index 292170a99ab6..c9f1d0bb6bf8 100644
@@ -153,6 +153,7 @@ perf-y += clockid.o
 perf-y += list_sort.o
 perf-y += mutex.o
 perf-y += sharded_mutex.o
+perf-y += intel-tpebs.o
 
 perf-$(CONFIG_LIBBPF) += bpf_map.o
 perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.o
tools/perf/util/intel-tpebs.c new file +285 -0

285 lines not on this page · raw message

tools/perf/util/intel-tpebs.h new file +29 -0
new file mode 100644
index 000000000000..e8e2bb2f479b
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * intel_pt.h: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.
+ */
+#include "stat.h"
+
+#ifndef INCLUDE__PERF_INTEL_TPEBS_H__
+#define INCLUDE__PERF_INTEL_TPEBS_H__
+
+struct tpebs_retire_lat {
+	struct list_head nd;
+	/* Event name */
+	const char *name;
+	/* Event name with the TPEBS modifier R */
+	const char *tpebs_name;
+	/* Count of retire_latency values found in sample data */
+	size_t count;
+	/* Sum of all the retire_latency values in sample data */
+	int sum;
+	/* Average of retire_latency, val = sum / count */
+	double val;
+};
+
+int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist *evsel_list);
+int stop_tpebs(void);
+void tpebs_data__delete(void);
+
+#endif
tools/perf/util/stat.h +3 -0
index fd7a187551bd..c6c2aa43030f 100644
@@ -110,6 +110,9 @@ struct perf_stat_config {
 	struct cpu_aggr_map	*cpus_aggr_map;
 	u64			*walltime_run;
 	struct rblist		 metric_events;
+	size_t			 tpebs_event_size;
+	struct list_head	 tpebs_results;
+	pid_t			 tpebs_pid;
 	int			 ctl_fd;
 	int			 ctl_fd_ack;
 	bool			 ctl_fd_close;
Signature · 1 line
2.43.0
reviewer
#8 lore raw

Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 13 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

When retire_latency value is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect required retire_latency values in parallel while perf stat is collecting counting values.

At the point of time that perf stat stops counting, it would send sigterm signal to perf record process and receiving sampling data back from perf record from a pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.

Quoted 5 lines

Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.

So I don't think this is the correct way to add this. My reviewed-by was based on the fact that I was going to refactor this after landing. It didn't land and I already sent out:
"Retirement latency perf stat support"
https://lore.kernel.org/lkml/[email protected]/
v3 to just land the tool portion:
https://lore.kernel.org/lkml/[email protected]/
So my reviewed-by no longer stands. The changes I've sent out haven't been reviewed. Given you're trying to land this, can we work on reviewing those changes? The v3 was specifically done just so that we can have the cleaner basis for adding tpebs to the evsel.

Let me point to specific issues below:

Quoted 31 lines
---
 tools/perf/builtin-stat.c     |  19 +++
 tools/perf/util/Build         |   1 +
 tools/perf/util/intel-tpebs.c | 285 ++++++++++++++++++++++++++++++++++
 tools/perf/util/intel-tpebs.h |  29 ++++
 tools/perf/util/stat.h        |   3 +
 5 files changed, 337 insertions(+)
 create mode 100644 tools/perf/util/intel-tpebs.c
 create mode 100644 tools/perf/util/intel-tpebs.h
tools/perf/builtin-stat.c +2 -0
index 428e9721b908..85927cf45adb 100644
@@ -70,6 +70,7 @@
 #include "util/bpf_counter.h"
 #include "util/iostat.h"
 #include "util/util.h"
+#include "util/intel-tpebs.h"
 #include "asm/bug.h"
 
 #include <linux/time64.h>
@@ -94,6 +95,7 @@
 #include <perf/evlist.h>
 #include <internal/threadmap.h>
 
+
 #define DEFAULT_SEPARATOR      " "
 #define FREEZE_ON_SMI_PATH     "devices/cpu/freeze_on_smi"
 

tools/perf/builtin-stat.c static struct perf_stat_config stat_config = { · line 168

Quoted 5 lines
tools/perf/builtin-stat.c +2 -0
@@ -162,6 +164,8 @@ static struct perf_stat_config stat_config = {
        .ctl_fd                 = -1,
        .ctl_fd_ack             = -1,
        .iostat_run             = false,
+       .tpebs_results          = LIST_HEAD_INIT(stat_config.tpebs_results),
+       .tpebs_pid              = -1,

Here we're adding state to the stat_config for the sake of tpebs events. Were the state in the evsel, as in my changes, we'd not need to carry new global state for tpebs.

tools/perf/builtin-stat.c static enum counter_recovery stat_handle_error(struct evsel *counter) · line 661

Quoted 9 lines
tools/perf/builtin-stat.c +2 -0
 };
 
 static void evlist__check_cpu_maps(struct evlist *evlist)
@@ -653,6 +657,8 @@ static enum counter_recovery stat_handle_error(struct evsel *counter)
 
        if (child_pid != -1)
                kill(child_pid, SIGTERM);
+       if (stat_config.tpebs_pid != -1)
+               stop_tpebs();

This logic is builtin-stat but what about other commands that could be using TPEBS for counters?

Quoted 95 lines
tools/perf/builtin-stat.c +13 -0
        return COUNTER_FATAL;
 }
 
@@ -673,6 +679,10 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx)
        int err;
        bool second_pass = false;
 
+       err = start_tpebs(&stat_config, evsel_list);
+       if (err < 0)
+               return err;
+
        if (forks) {
                if (evlist__prepare_workload(evsel_list, &target, argv, is_pipe, workload_exec_failed_signal) < 0) {
                        perror("failed to prepare workload");
@@ -878,6 +888,10 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx)
 
        t1 = rdclock();
 
+       err = stop_tpebs();
+       if (err < 0)
+               return err;
+
        if (stat_config.walltime_run_table)
                stat_config.walltime_run[run_idx] = t1 - t0;
 
@@ -985,6 +999,9 @@ static void sig_atexit(void)
        if (child_pid != -1)
                kill(child_pid, SIGTERM);
 
+       if (stat_config.tpebs_pid != -1)
+               kill(stat_config.tpebs_pid, SIGTERM);
+
        sigprocmask(SIG_SETMASK, &oset, NULL);
 
        if (signr == -1)
@@ -2918,5 +2935,7 @@ int cmd_stat(int argc, const char **argv)
        metricgroup__rblist_exit(&stat_config.metric_events);
        evlist__close_control(stat_config.ctl_fd, stat_config.ctl_fd_ack, &stat_config.ctl_fd_close);
 
+       tpebs_data__delete();
+
        return status;
 }
tools/perf/util/Build +1 -0
index 292170a99ab6..c9f1d0bb6bf8 100644
@@ -153,6 +153,7 @@ perf-y += clockid.o
 perf-y += list_sort.o
 perf-y += mutex.o
 perf-y += sharded_mutex.o
+perf-y += intel-tpebs.o
 
 perf-$(CONFIG_LIBBPF) += bpf_map.o
 perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.o
tools/perf/util/intel-tpebs.c new file +34 -0
new file mode 100644
index 000000000000..4b7a98794fae
@@ -0,0 +1,285 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * intel_pt.c: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.
+ */
+
+
+#include <sys/param.h>
+#include <subcmd/run-command.h>
+#include <thread.h>
+#include "intel-tpebs.h"
+#include <linux/list.h>
+#include <linux/zalloc.h>
+#include <linux/err.h>
+#include "sample.h"
+#include "debug.h"
+#include "evlist.h"
+#include "evsel.h"
+#include "session.h"
+#include "tool.h"
+#include "metricgroup.h"
+#include <sys/stat.h>
+#include <sys/file.h>
+
+
+
+#define PERF_DATA              "-"
+#define CONTROL                        "/tmp/control"
+#define ACK                    "/tmp/ack"
+pthread_t reader_thread;
+struct child_process *cmd;
+struct perf_stat_config *stat_config;
+
+static int get_perf_record_args(const char **record_argv)

tools/perf/util/intel-tpebs.c line 44

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+{
+       int i = 0;
+       struct tpebs_retire_lat *e;
+
+       pr_debug("Prepare perf record for retire_latency\n");
+
+       record_argv[i++] = "perf";
+       record_argv[i++] = "record";
+       record_argv[i++] = "-W";
+       record_argv[i++] = "--synth=no";

There should be more things to disable, like BPF events, we don't need the dummy, etc.

Quoted 134 lines
tools/perf/util/intel-tpebs.c new file +134 -0
+       record_argv[i++] = "--control=fifo:/tmp/control,/tmp/ack";
+
+       if (stat_config->user_requested_cpu_list) {
+               record_argv[i++] = "-C";
+               record_argv[i++] = stat_config->user_requested_cpu_list;
+       }
+
+       if (stat_config->system_wide)
+               record_argv[i++] = "-a";
+
+       if (!stat_config->system_wide && !stat_config->user_requested_cpu_list) {
+               pr_err("Require -a or -C option to run sampling.\n");
+               return -ECANCELED;
+       }
+
+       list_for_each_entry(e, &stat_config->tpebs_results, nd) {
+               record_argv[i++] = "-e";
+               record_argv[i++] = e->name;
+       }
+
+       record_argv[i++] = "-o";
+       record_argv[i++] = PERF_DATA;
+
+       return 0;
+}
+
+static int prepare_run_command(const char **argv)
+{
+       cmd = zalloc(sizeof(struct child_process));
+       if (!cmd)
+               return -ENOMEM;
+       cmd->argv = argv;
+       cmd->out = -1;
+       return 0;
+}
+
+static int prepare_perf_record(int tpebs_event_size)
+{
+       const char **record_argv;
+       int ret;
+
+       /*Create control and ack fd for --control*/
+       if (mkfifo(CONTROL, 0600)) {
+               pr_err("Failed to create control fifo");
+               return -1;
+       }
+       if (mkfifo(ACK, 0600)) {
+               pr_err("Failed to create control fifo");
+               return -1;
+       }
+
+       record_argv = calloc(10 + 2 * tpebs_event_size, sizeof(char *));
+       if (!record_argv)
+               return -ENOMEM;
+
+
+       ret = get_perf_record_args(record_argv);
+       if (ret)
+               goto out;
+
+       ret = prepare_run_command(record_argv);
+       if (ret)
+               goto out;
+       ret = start_command(cmd);
+out:
+       free(record_argv);
+       return ret;
+}
+struct sample_data_reader {
+       struct perf_tool        tool;
+       struct perf_session     *session;
+};
+
+static void tpebs_retire_lat__delete(struct tpebs_retire_lat *r)
+{
+       zfree(&r->name);
+       zfree(&r->tpebs_name);
+       free(r);
+}
+
+void tpebs_data__delete(void)
+{
+       struct tpebs_retire_lat *r, *rtmp;
+
+       list_for_each_entry_safe(r, rtmp, &stat_config->tpebs_results, nd) {
+               list_del_init(&r->nd);
+               tpebs_retire_lat__delete(r);
+       }
+       free(cmd);
+}
+
+static int process_sample_event(struct perf_tool *tool __maybe_unused,
+                               union perf_event *event __maybe_unused,
+                               struct perf_sample *sample,
+                               struct evsel *evsel,
+                               struct machine *machine __maybe_unused)
+{
+       int ret = 0;
+       const char *evname;
+       struct tpebs_retire_lat *t;
+
+       evname = evsel__name(evsel);
+
+       /*
+        * Need to handle per core results? We are assuming average retire
+        * latency value will be used. Save the number of samples and the sum of
+        * retire latency value for each event.
+        */
+       list_for_each_entry(t, &stat_config->tpebs_results, nd) {
+               if (!strcmp(evname, t->name)) {
+                       t->count += 1;
+                       t->sum += sample->retire_lat;
+                       t->val = (double) t->sum / t->count;
+                       break;
+               }
+       }
+
+       return ret;
+}
+
+static int process_feature_event(struct perf_session *session,
+                                union perf_event *event)
+{
+       if (event->feat.feat_id < HEADER_LAST_FEATURE)
+               return perf_event__process_feature(session, event);
+       return 0;
+}
+
+static void *__sample_reader(void *arg)
+{
+       struct child_process *child = arg;
+       struct perf_session *session;
+       struct perf_data data = {
+               .mode = PERF_DATA_MODE_READ,

tools/perf/util/intel-tpebs.c line 188

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+               .path = PERF_DATA,
+               .file.fd = child->out,
+       };
+       struct sample_data_reader reader = {
+               .tool = {
+               .sample          = process_sample_event,
+               .feature         = process_feature_event,
+               .attr            = perf_event__process_attr,
+               },
+       };

I prefer the reading approach here over what I did here:
https://lore.kernel.org/lkml/CAM9d7cgzTsfk3C+dTN80f5FhB1rmfturjuUUwvSTeUvny5eWKw@mail.gmail.com/
I'd done that as Namhyung had commented on using perf report:
https://lore.kernel.org/lkml/CAM9d7cgdQQn5GYB7t++xuoMdeqPXiEkkcop69+rD06RAnu9-EQ@mail.gmail.com/
It was also less work. I don't see why we can't move the reading logic into a function like evsel__read_retire_latency that's in my change. I also think that in the first instance all reading logic should be implemented to return 0 and we only do the forking, etc. when a command line flag is passed.

Quoted 116 lines
tools/perf/util/intel-tpebs.c new file +97 -0
+
+       session = perf_session__new(&data, &reader.tool);
+       if (IS_ERR(session))
+               return NULL;
+       reader.session = session;
+       perf_session__process_events(session);
+       perf_session__delete(session);
+
+       return NULL;
+}
+
+
+int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist *evsel_list)
+{
+       int ret = 0;
+       struct evsel *evsel;
+       int control = -1, ack = -1;
+
+       stat_config = perf_stat_config;
+       /*
+        * Prepare perf record for sampling event retire_latency before fork and
+        * prepare workload
+        */
+       evlist__for_each_entry(evsel_list, evsel) {
+               if (evsel->retire_lat) {
+                       struct tpebs_retire_lat *new = malloc(sizeof(struct tpebs_retire_lat));
+                       int i;
+                       char *name;
+
+                       pr_debug("perf stat retire latency %s required\n", evsel->name);
+                       if (!new)
+                               return -1;
+                       for (i = strlen(evsel->name) - 1; i > 0; i--) {
+                               if (evsel->name[i] == 'R')
+                                       break;
+                       }
+                       if (i <= 0 || evsel->name[i] != 'R')
+                               return -1;
+
+                       name = strdup(evsel->name);
+                       if (!name)
+                               return -ENOMEM;
+                       name[i] = 'p';
+                       new->name = strdup(name);
+                       free(name);
+                       new->tpebs_name = strdup(evsel->name);
+                       if (!new->tpebs_name)
+                               return -ENOMEM;
+                       new->count = 0;
+                       new->sum = 0;
+                       list_add_tail(&new->nd, &stat_config->tpebs_results);
+                       stat_config->tpebs_event_size += 1;
+               }
+       }
+
+       if (stat_config->tpebs_event_size > 0) {
+               ret = prepare_perf_record(stat_config->tpebs_event_size);
+               if (ret)
+                       return ret;
+               if (pthread_create(&reader_thread, NULL, __sample_reader, cmd)) {
+                       kill(cmd->pid, SIGTERM);
+                       close(cmd->out);
+                       pr_err("Could not create thread to process sample data.\n");
+                       return -1;
+               }
+               /* Wait for perf record initialization a little bit.*/
+               control = open(CONTROL, O_RDONLY, O_NONBLOCK);
+               if (!control)
+                       return -1;
+               close(control);
+               ack = open(ACK, O_RDONLY, O_NONBLOCK);
+               if (!ack)
+                       return -1;
+               close(ack);
+               pr_debug("Received ack from perf record\n");
+       }
+
+       return ret;
+}
+
+
+int stop_tpebs(void)
+{
+       int ret = 0;
+
+       if (stat_config->tpebs_event_size > 0) {
+               kill(cmd->pid, SIGTERM);
+               pthread_join(reader_thread, NULL);
+               close(cmd->out);
+               ret = finish_command(cmd);
+               if (ret == -ERR_RUN_COMMAND_WAITPID_SIGNAL)
+                       ret = 0;
+               remove(CONTROL);
+               remove(ACK);
+       }
+       return ret;
+}
tools/perf/util/intel-tpebs.h new file +13 -0
new file mode 100644
index 000000000000..e8e2bb2f479b
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * intel_pt.h: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.
+ */
+#include "stat.h"
+
+#ifndef INCLUDE__PERF_INTEL_TPEBS_H__
+#define INCLUDE__PERF_INTEL_TPEBS_H__
+
+struct tpebs_retire_lat {
+       struct list_head nd;
+       /* Event name */

tools/perf/util/intel-tpebs.h line 23

Quoted 10 lines
tools/perf/util/intel-tpebs.h new file +10 -0
+       const char *name;
+       /* Event name with the TPEBS modifier R */
+       const char *tpebs_name;
+       /* Count of retire_latency values found in sample data */
+       size_t count;
+       /* Sum of all the retire_latency values in sample data */
+       int sum;
+       /* Average of retire_latency, val = sum / count */
+       double val;
+};

An evsel has a pretty much all of this and so we're duplicating in particular the counting logic which then needs later work to integrate and is why I'd prefer we went the evsel route.

Thanks,
Ian

Quoted 23 lines
tools/perf/util/intel-tpebs.h new file +6 -0
+
+int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist *evsel_list);
+int stop_tpebs(void);
+void tpebs_data__delete(void);
+
+#endif
tools/perf/util/stat.h +3 -1
index fd7a187551bd..c6c2aa43030f 100644
@@ -110,6 +110,9 @@ struct perf_stat_config {
        struct cpu_aggr_map     *cpus_aggr_map;
        u64                     *walltime_run;
        struct rblist            metric_events;
+       size_t                   tpebs_event_size;
+       struct list_head         tpebs_results;
+       pid_t                    tpebs_pid;
        int                      ctl_fd;
        int                      ctl_fd_ack;
        bool                     ctl_fd_close;
--

2.43.0

#9 lore raw

RE: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 39 lines

-----Original Message-----
From: Ian Rogers <[email protected]>
Sent: Thursday, May 16, 2024 9:43 AM
To: Wang, Weilin <[email protected]>
Cc: Namhyung Kim <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 9 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

When retire_latency value is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect required retire_latency values in parallel while perf stat is collecting counting values.

At the point of time that perf stat stops counting, it would send sigterm

signal

Quoted 1 line

to perf record process and receiving sampling data back from perf record

from a

Quoted 8 lines

pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.

Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.

Quoted 14 lines

So I don't think this is the correct way to add this. My reviewed-by was based on the fact that I was going to refactor this after landing. It didn't land and I already sent out:
"Retirement latency perf stat support"
https://lore.kernel.org/lkml/20240428053616.1125891-1-
[email protected]/
v3 to just land the tool portion:
https://lore.kernel.org/lkml/20240503232849.17752-1-
[email protected]/
So my reviewed-by no longer stands. The changes I've sent out haven't been reviewed. Given you're trying to land this, can we work on reviewing those changes? The v3 was specifically done just so that we can have the cleaner basis for adding tpebs to the evsel.

Sorry about carrying the reviewed-by in wrong places. I will remove them.

Quoted 33 lines

Let me point to specific issues below:

Quoted 4 lines
---
 tools/perf/builtin-stat.c     |  19 +++
 tools/perf/util/Build         |   1 +
 tools/perf/util/intel-tpebs.c | 285

++++++++++++++++++++++++++++++++++

Quoted 26 lines
 tools/perf/util/intel-tpebs.h |  29 ++++
 tools/perf/util/stat.h        |   3 +
 5 files changed, 337 insertions(+)
 create mode 100644 tools/perf/util/intel-tpebs.c
 create mode 100644 tools/perf/util/intel-tpebs.h
tools/perf/builtin-stat.c +2 -0
index 428e9721b908..85927cf45adb 100644
@@ -70,6 +70,7 @@
 #include "util/bpf_counter.h"
 #include "util/iostat.h"
 #include "util/util.h"
+#include "util/intel-tpebs.h"
 #include "asm/bug.h"
 
 #include <linux/time64.h>
@@ -94,6 +95,7 @@
 #include <perf/evlist.h>
 #include <internal/threadmap.h>
 
+
 #define DEFAULT_SEPARATOR      " "
 #define FREEZE_ON_SMI_PATH     "devices/cpu/freeze_on_smi"
 
Quoted 10 lines

tools/perf/builtin-stat.c static struct perf_stat_config stat_config = { · line 168

Quoted 6 lines
tools/perf/builtin-stat.c +2 -0
@@ -162,6 +164,8 @@ static struct perf_stat_config stat_config = {
        .ctl_fd                 = -1,
        .ctl_fd_ack             = -1,
        .iostat_run             = false,
+       .tpebs_results          = LIST_HEAD_INIT(stat_config.tpebs_results),
+       .tpebs_pid              = -1,

Here we're adding state to the stat_config for the sake of tpebs events. Were the state in the evsel, as in my changes, we'd not need to carry new global state for tpebs.

Yes, I should remove the .tpebs_results out. How about .tpebs_pid? It is used in builtin-stat later.

Quoted 7 lines

tools/perf/builtin-stat.c static enum counter_recovery

Quoted 5 lines
tools/perf/builtin-stat.c +0 -0
 };
 
 static void evlist__check_cpu_maps(struct evlist *evlist)
@@ -653,6 +657,8 @@ static enum counter_recovery

stat_handle_error(struct evsel *counter)

Quoted 1 line
Quoted 7 lines

tools/perf/builtin-stat.c static enum counter_recovery stat_handle_error(struct evsel *counter) · line 661

Quoted 4 lines
        if (child_pid != -1)
                kill(child_pid, SIGTERM);
tools/perf/builtin-stat.c +2 -0
+       if (stat_config.tpebs_pid != -1)
+               stop_tpebs();

This logic is builtin-stat but what about other commands that could be using TPEBS for counters?

Could you please let me know the commands that could use it?
I was thinking we could do this logic in evsel once the per core per event fork get updated. I think that would work for all the commands.

Quoted 105 lines

tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char

Quoted 5 lines
        return COUNTER_FATAL;

}

tools/perf/builtin-stat.c +0 -0
@@ -673,6 +679,10 @@ static int __run_perf_stat(int argc, const char

**argv, int run_idx)

tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char **argv, int run_idx)

Quoted 9 lines
tools/perf/builtin-stat.c +4 -0
        int err;
        bool second_pass = false;
 
+       err = start_tpebs(&stat_config, evsel_list);
+       if (err < 0)
+               return err;
+
        if (forks) {
                if (evlist__prepare_workload(evsel_list, &target, argv, is_pipe,

workload_exec_failed_signal) < 0) {

tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char

Quoted 2 lines
tools/perf/builtin-stat.c +0 -0
                        perror("failed to prepare workload");
@@ -878,6 +888,10 @@ static int __run_perf_stat(int argc, const char

**argv, int run_idx)

Quoted 21 lines
tools/perf/builtin-stat.c +7 -0
        t1 = rdclock();
 
+       err = stop_tpebs();
+       if (err < 0)
+               return err;
+
        if (stat_config.walltime_run_table)
                stat_config.walltime_run[run_idx] = t1 - t0;
 
@@ -985,6 +999,9 @@ static void sig_atexit(void)
        if (child_pid != -1)
                kill(child_pid, SIGTERM);
 
+       if (stat_config.tpebs_pid != -1)
+               kill(stat_config.tpebs_pid, SIGTERM);
+
        sigprocmask(SIG_SETMASK, &oset, NULL);
 
        if (signr == -1)

tools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv) · line 2936

Quoted 2 lines
tools/perf/builtin-stat.c +0 -0
@@ -2918,5 +2935,7 @@ int cmd_stat(int argc, const char **argv)
        metricgroup__rblist_exit(&stat_config.metric_events);
        evlist__close_control(stat_config.ctl_fd, stat_config.ctl_fd_ack,

&stat_config.ctl_fd_close);

Quoted 62 lines
tools/perf/builtin-stat.c +2 -0
+       tpebs_data__delete();
+
        return status;
 }
tools/perf/util/Build +1 -0
index 292170a99ab6..c9f1d0bb6bf8 100644
@@ -153,6 +153,7 @@ perf-y += clockid.o
 perf-y += list_sort.o
 perf-y += mutex.o
 perf-y += sharded_mutex.o
+perf-y += intel-tpebs.o
 
 perf-$(CONFIG_LIBBPF) += bpf_map.o
 perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.o
tools/perf/util/intel-tpebs.c new file +39 -0
new file mode 100644
index 000000000000..4b7a98794fae
@@ -0,0 +1,285 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * intel_pt.c: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.
+ */
+
+
+#include <sys/param.h>
+#include <subcmd/run-command.h>
+#include <thread.h>
+#include "intel-tpebs.h"
+#include <linux/list.h>
+#include <linux/zalloc.h>
+#include <linux/err.h>
+#include "sample.h"
+#include "debug.h"
+#include "evlist.h"
+#include "evsel.h"
+#include "session.h"
+#include "tool.h"
+#include "metricgroup.h"
+#include <sys/stat.h>
+#include <sys/file.h>
+
+
+
+#define PERF_DATA              "-"
+#define CONTROL                        "/tmp/control"
+#define ACK                    "/tmp/ack"
+pthread_t reader_thread;
+struct child_process *cmd;
+struct perf_stat_config *stat_config;
+
+static int get_perf_record_args(const char **record_argv)
+{
+       int i = 0;
+       struct tpebs_retire_lat *e;
+
+       pr_debug("Prepare perf record for retire_latency\n");
Quoted 8 lines

tools/perf/util/intel-tpebs.c line 44

Quoted 5 lines
tools/perf/util/intel-tpebs.c new file +5 -0
+
+       record_argv[i++] = "perf";
+       record_argv[i++] = "record";
+       record_argv[i++] = "-W";
+       record_argv[i++] = "--synth=no";

There should be more things to disable, like BPF events, we don't need the dummy, etc.

Ok, I will add that.

Quoted 151 lines
Quoted 98 lines
tools/perf/util/intel-tpebs.c new file +11 -0
+       record_argv[i++] = "--control=fifo:/tmp/control,/tmp/ack";
+
+       if (stat_config->user_requested_cpu_list) {
+               record_argv[i++] = "-C";
+               record_argv[i++] = stat_config->user_requested_cpu_list;
+       }
+
+       if (stat_config->system_wide)
+               record_argv[i++] = "-a";
+
+       if (!stat_config->system_wide && !stat_config-

user_requested_cpu_list) {

tools/perf/util/intel-tpebs.c new file +85 -0
+               pr_err("Require -a or -C option to run sampling.\n");
+               return -ECANCELED;
+       }
+
+       list_for_each_entry(e, &stat_config->tpebs_results, nd) {
+               record_argv[i++] = "-e";
+               record_argv[i++] = e->name;
+       }
+
+       record_argv[i++] = "-o";
+       record_argv[i++] = PERF_DATA;
+
+       return 0;
+}
+
+static int prepare_run_command(const char **argv)
+{
+       cmd = zalloc(sizeof(struct child_process));
+       if (!cmd)
+               return -ENOMEM;
+       cmd->argv = argv;
+       cmd->out = -1;
+       return 0;
+}
+
+static int prepare_perf_record(int tpebs_event_size)
+{
+       const char **record_argv;
+       int ret;
+
+       /*Create control and ack fd for --control*/
+       if (mkfifo(CONTROL, 0600)) {
+               pr_err("Failed to create control fifo");
+               return -1;
+       }
+       if (mkfifo(ACK, 0600)) {
+               pr_err("Failed to create control fifo");
+               return -1;
+       }
+
+       record_argv = calloc(10 + 2 * tpebs_event_size, sizeof(char *));
+       if (!record_argv)
+               return -ENOMEM;
+
+
+       ret = get_perf_record_args(record_argv);
+       if (ret)
+               goto out;
+
+       ret = prepare_run_command(record_argv);
+       if (ret)
+               goto out;
+       ret = start_command(cmd);
+out:
+       free(record_argv);
+       return ret;
+}
+struct sample_data_reader {
+       struct perf_tool        tool;
+       struct perf_session     *session;
+};
+
+static void tpebs_retire_lat__delete(struct tpebs_retire_lat *r)
+{
+       zfree(&r->name);
+       zfree(&r->tpebs_name);
+       free(r);
+}
+
+void tpebs_data__delete(void)
+{
+       struct tpebs_retire_lat *r, *rtmp;
+
+       list_for_each_entry_safe(r, rtmp, &stat_config->tpebs_results, nd) {
+               list_del_init(&r->nd);
+               tpebs_retire_lat__delete(r);
+       }
+       free(cmd);
+}
+
+static int process_sample_event(struct perf_tool *tool __maybe_unused,
+                               union perf_event *event __maybe_unused,
+                               struct perf_sample *sample,
+                               struct evsel *evsel,
+                               struct machine *machine __maybe_unused)

tools/perf/util/intel-tpebs.c

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+{
+       int ret = 0;
+       const char *evname;
+       struct tpebs_retire_lat *t;
+
+       evname = evsel__name(evsel);
+
+       /*
+        * Need to handle per core results? We are assuming average retire
+        * latency value will be used. Save the number of samples and the sum

of

Quoted 28 lines
tools/perf/util/intel-tpebs.c new file +28 -0
+        * retire latency value for each event.
+        */
+       list_for_each_entry(t, &stat_config->tpebs_results, nd) {
+               if (!strcmp(evname, t->name)) {
+                       t->count += 1;
+                       t->sum += sample->retire_lat;
+                       t->val = (double) t->sum / t->count;
+                       break;
+               }
+       }
+
+       return ret;
+}
+
+static int process_feature_event(struct perf_session *session,
+                                union perf_event *event)
+{
+       if (event->feat.feat_id < HEADER_LAST_FEATURE)
+               return perf_event__process_feature(session, event);
+       return 0;
+}
+
+static void *__sample_reader(void *arg)
+{
+       struct child_process *child = arg;
+       struct perf_session *session;
+       struct perf_data data = {
+               .mode = PERF_DATA_MODE_READ,

tools/perf/util/intel-tpebs.c line 188

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+               .path = PERF_DATA,
+               .file.fd = child->out,
+       };
+       struct sample_data_reader reader = {
+               .tool = {
+               .sample          = process_sample_event,
+               .feature         = process_feature_event,
+               .attr            = perf_event__process_attr,
+               },
+       };

I prefer the reading approach here over what I did here:
https://lore.kernel.org/lkml/CAM9d7cgzTsfk3C+dTN80f5FhB1rmfturjuUUwv
[email protected]/

Quoted 8 lines

I'd done that as Namhyung had commented on using perf report:
https://lore.kernel.org/lkml/CAM9d7cgdQQn5GYB7t++xuoMdeqPXiEkkcop69
[email protected]/
It was also less work. I don't see why we can't move the reading logic into a function like evsel__read_retire_latency that's in my change. I also think that in the first instance all reading logic should be implemented to return 0 and we only do the forking, etc. when a command line flag is passed.

Yes, a command line flag to turn it on is a good idea. Andi also suggested this. I want to add this option in next version.

I tried to refactor the code so that you could easily reuse it later. You could either put this code directly into evsel__read_retire_latency or call this function, whichever way you feel better.

Quoted 126 lines
Quoted 4 lines
tools/perf/util/intel-tpebs.c new file +3 -0
+
+       session = perf_session__new(&data, &reader.tool);
+       if (IS_ERR(session))

tools/perf/util/intel-tpebs.c

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+               return NULL;
+       reader.session = session;
+       perf_session__process_events(session);
+       perf_session__delete(session);
+
+       return NULL;
+}
+
+
+int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist

*evsel_list)

tools/perf/util/intel-tpebs.c

Quoted 13 lines
tools/perf/util/intel-tpebs.c new file +13 -0
+{
+       int ret = 0;
+       struct evsel *evsel;
+       int control = -1, ack = -1;
+
+       stat_config = perf_stat_config;
+       /*
+        * Prepare perf record for sampling event retire_latency before fork and
+        * prepare workload
+        */
+       evlist__for_each_entry(evsel_list, evsel) {
+               if (evsel->retire_lat) {
+                       struct tpebs_retire_lat *new = malloc(sizeof(struct

tpebs_retire_lat));

Quoted 97 lines
tools/perf/util/intel-tpebs.c new file +4 -0
+                       int i;
+                       char *name;
+
+                       pr_debug("perf stat retire latency %s required\n", evsel-

name);

tools/perf/util/intel-tpebs.c new file +67 -0
+                       if (!new)
+                               return -1;
+                       for (i = strlen(evsel->name) - 1; i > 0; i--) {
+                               if (evsel->name[i] == 'R')
+                                       break;
+                       }
+                       if (i <= 0 || evsel->name[i] != 'R')
+                               return -1;
+
+                       name = strdup(evsel->name);
+                       if (!name)
+                               return -ENOMEM;
+                       name[i] = 'p';
+                       new->name = strdup(name);
+                       free(name);
+                       new->tpebs_name = strdup(evsel->name);
+                       if (!new->tpebs_name)
+                               return -ENOMEM;
+                       new->count = 0;
+                       new->sum = 0;
+                       list_add_tail(&new->nd, &stat_config->tpebs_results);
+                       stat_config->tpebs_event_size += 1;
+               }
+       }
+
+       if (stat_config->tpebs_event_size > 0) {
+               ret = prepare_perf_record(stat_config->tpebs_event_size);
+               if (ret)
+                       return ret;
+               if (pthread_create(&reader_thread, NULL, __sample_reader, cmd)) {
+                       kill(cmd->pid, SIGTERM);
+                       close(cmd->out);
+                       pr_err("Could not create thread to process sample data.\n");
+                       return -1;
+               }
+               /* Wait for perf record initialization a little bit.*/
+               control = open(CONTROL, O_RDONLY, O_NONBLOCK);
+               if (!control)
+                       return -1;
+               close(control);
+               ack = open(ACK, O_RDONLY, O_NONBLOCK);
+               if (!ack)
+                       return -1;
+               close(ack);
+               pr_debug("Received ack from perf record\n");
+       }
+
+       return ret;
+}
+
+
+int stop_tpebs(void)
+{
+       int ret = 0;
+
+       if (stat_config->tpebs_event_size > 0) {
+               kill(cmd->pid, SIGTERM);
+               pthread_join(reader_thread, NULL);
+               close(cmd->out);
+               ret = finish_command(cmd);
+               if (ret == -ERR_RUN_COMMAND_WAITPID_SIGNAL)
+                       ret = 0;
+               remove(CONTROL);
+               remove(ACK);
+       }
+       return ret;
+}
tools/perf/util/intel-tpebs.h new file +19 -0
new file mode 100644
index 000000000000..e8e2bb2f479b
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * intel_pt.h: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.
+ */
+#include "stat.h"
+
+#ifndef INCLUDE__PERF_INTEL_TPEBS_H__
+#define INCLUDE__PERF_INTEL_TPEBS_H__
+
+struct tpebs_retire_lat {
+       struct list_head nd;
+       /* Event name */
+       const char *name;
+       /* Event name with the TPEBS modifier R */
+       const char *tpebs_name;
+       /* Count of retire_latency values found in sample data */
+       size_t count;
+       /* Sum of all the retire_latency values in sample data */
Quoted 8 lines

tools/perf/util/intel-tpebs.h line 23

Quoted 4 lines
tools/perf/util/intel-tpebs.h new file +4 -0
+       int sum;
+       /* Average of retire_latency, val = sum / count */
+       double val;
+};

An evsel has a pretty much all of this and so we're duplicating in particular the counting logic which then needs later work to integrate and is why I'd prefer we went the evsel route.

I wanted to reuse evsel as much as I could. But as you know, I had some issue to use it and had to remove the code. I'm going to explore more and see if I could get it work with evsel better.

Comparing with last version, do you think this change is toward the direct that you Want? I believe we could get this code to use evsel more.

Thanks,
Weilin

Quoted 28 lines

Thanks,
Ian

tools/perf/util/intel-tpebs.h line 25

Quoted 2 lines
tools/perf/util/intel-tpebs.h new file +2 -0
+
+int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist

*evsel_list);

Quoted 21 lines
tools/perf/util/intel-tpebs.h new file +4 -0
+int stop_tpebs(void);
+void tpebs_data__delete(void);
+
+#endif
tools/perf/util/stat.h +3 -1
index fd7a187551bd..c6c2aa43030f 100644
@@ -110,6 +110,9 @@ struct perf_stat_config {
        struct cpu_aggr_map     *cpus_aggr_map;
        u64                     *walltime_run;
        struct rblist            metric_events;
+       size_t                   tpebs_event_size;
+       struct list_head         tpebs_results;
+       pid_t                    tpebs_pid;
        int                      ctl_fd;
        int                      ctl_fd_ack;
        bool                     ctl_fd_close;
--

2.43.0

reviewer
#10 lore raw

Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 114 lines · Weilin
Quoted 38 lines

-----Original Message-----
From: Ian Rogers <[email protected]>
Sent: Thursday, May 16, 2024 9:43 AM
To: Wang, Weilin <[email protected]>
Cc: Namhyung Kim <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 9 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

When retire_latency value is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect required retire_latency values in parallel while perf stat is collecting counting values.

At the point of time that perf stat stops counting, it would send sigterm

signal

Quoted 1 line

to perf record process and receiving sampling data back from perf record

from a

Quoted 8 lines

pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.

Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.

Quoted 14 lines

So I don't think this is the correct way to add this. My reviewed-by was based on the fact that I was going to refactor this after landing. It didn't land and I already sent out:
"Retirement latency perf stat support"
https://lore.kernel.org/lkml/20240428053616.1125891-1-
[email protected]/
v3 to just land the tool portion:
https://lore.kernel.org/lkml/20240503232849.17752-1-
[email protected]/
So my reviewed-by no longer stands. The changes I've sent out haven't been reviewed. Given you're trying to land this, can we work on reviewing those changes? The v3 was specifically done just so that we can have the cleaner basis for adding tpebs to the evsel.

Sorry about carrying the reviewed-by in wrong places. I will remove them.

Quoted 33 lines

Let me point to specific issues below:

Quoted 4 lines
---
 tools/perf/builtin-stat.c     |  19 +++
 tools/perf/util/Build         |   1 +
 tools/perf/util/intel-tpebs.c | 285

++++++++++++++++++++++++++++++++++

Quoted 26 lines
 tools/perf/util/intel-tpebs.h |  29 ++++
 tools/perf/util/stat.h        |   3 +
 5 files changed, 337 insertions(+)
 create mode 100644 tools/perf/util/intel-tpebs.c
 create mode 100644 tools/perf/util/intel-tpebs.h
tools/perf/builtin-stat.c +2 -0
index 428e9721b908..85927cf45adb 100644
@@ -70,6 +70,7 @@
 #include "util/bpf_counter.h"
 #include "util/iostat.h"
 #include "util/util.h"
+#include "util/intel-tpebs.h"
 #include "asm/bug.h"
 
 #include <linux/time64.h>
@@ -94,6 +95,7 @@
 #include <perf/evlist.h>
 #include <internal/threadmap.h>
 
+
 #define DEFAULT_SEPARATOR      " "
 #define FREEZE_ON_SMI_PATH     "devices/cpu/freeze_on_smi"
 
Quoted 10 lines

tools/perf/builtin-stat.c static struct perf_stat_config stat_config = { · line 168

Quoted 6 lines
tools/perf/builtin-stat.c +2 -0
@@ -162,6 +164,8 @@ static struct perf_stat_config stat_config = {
        .ctl_fd                 = -1,
        .ctl_fd_ack             = -1,
        .iostat_run             = false,
+       .tpebs_results          = LIST_HEAD_INIT(stat_config.tpebs_results),
+       .tpebs_pid              = -1,

Here we're adding state to the stat_config for the sake of tpebs events. Were the state in the evsel, as in my changes, we'd not need to carry new global state for tpebs.

Yes, I should remove the .tpebs_results out. How about .tpebs_pid? It is used in builtin-stat later.

Quoted 12 lines

tools/perf/builtin-stat.c static enum counter_recovery

Quoted 5 lines
tools/perf/builtin-stat.c +0 -0
 };
 
 static void evlist__check_cpu_maps(struct evlist *evlist)
@@ -653,6 +657,8 @@ static enum counter_recovery

stat_handle_error(struct evsel *counter)

Quoted 5 lines
tools/perf/builtin-stat.c +2 -0
        if (child_pid != -1)
                kill(child_pid, SIGTERM);
+       if (stat_config.tpebs_pid != -1)
+               stop_tpebs();
Quoted 6 lines
Quoted 2 lines

This logic is builtin-stat but what about other commands that could be using TPEBS for counters?

Could you please let me know the commands that could use it?
I was thinking we could do this logic in evsel once the per core per event fork get updated. I think that would work for all the commands.

Using this code refactored into evsels is what I'd like to see, so I think we're agreed. I'm not thinking of a main command reading counters. The evlist and evsel are exposed through python so potentially any script could use the values, which is why it'd be best to abstract them behind an evsel.

Quoted 114 lines
Quoted 105 lines

tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char

Quoted 5 lines
tools/perf/builtin-stat.c +0 -0
        return COUNTER_FATAL;
 }
 
@@ -673,6 +679,10 @@ static int __run_perf_stat(int argc, const char

**argv, int run_idx)

tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char **argv, int run_idx)

Quoted 9 lines
tools/perf/builtin-stat.c +4 -0
        int err;
        bool second_pass = false;
 
+       err = start_tpebs(&stat_config, evsel_list);
+       if (err < 0)
+               return err;
+
        if (forks) {
                if (evlist__prepare_workload(evsel_list, &target, argv, is_pipe,

workload_exec_failed_signal) < 0) {

tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char

Quoted 2 lines
tools/perf/builtin-stat.c +0 -0
                        perror("failed to prepare workload");
@@ -878,6 +888,10 @@ static int __run_perf_stat(int argc, const char

**argv, int run_idx)

Quoted 21 lines
tools/perf/builtin-stat.c +7 -0
        t1 = rdclock();
 
+       err = stop_tpebs();
+       if (err < 0)
+               return err;
+
        if (stat_config.walltime_run_table)
                stat_config.walltime_run[run_idx] = t1 - t0;
 
@@ -985,6 +999,9 @@ static void sig_atexit(void)
        if (child_pid != -1)
                kill(child_pid, SIGTERM);
 
+       if (stat_config.tpebs_pid != -1)
+               kill(stat_config.tpebs_pid, SIGTERM);
+
        sigprocmask(SIG_SETMASK, &oset, NULL);
 
        if (signr == -1)

tools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv) · line 2936

Quoted 2 lines
tools/perf/builtin-stat.c +0 -0
@@ -2918,5 +2935,7 @@ int cmd_stat(int argc, const char **argv)
        metricgroup__rblist_exit(&stat_config.metric_events);
        evlist__close_control(stat_config.ctl_fd, stat_config.ctl_fd_ack,

&stat_config.ctl_fd_close);

Quoted 62 lines
tools/perf/builtin-stat.c +2 -0
+       tpebs_data__delete();
+
        return status;
 }
tools/perf/util/Build +1 -0
index 292170a99ab6..c9f1d0bb6bf8 100644
@@ -153,6 +153,7 @@ perf-y += clockid.o
 perf-y += list_sort.o
 perf-y += mutex.o
 perf-y += sharded_mutex.o
+perf-y += intel-tpebs.o
 
 perf-$(CONFIG_LIBBPF) += bpf_map.o
 perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.o
tools/perf/util/intel-tpebs.c new file +39 -0
new file mode 100644
index 000000000000..4b7a98794fae
@@ -0,0 +1,285 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * intel_pt.c: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.
+ */
+
+
+#include <sys/param.h>
+#include <subcmd/run-command.h>
+#include <thread.h>
+#include "intel-tpebs.h"
+#include <linux/list.h>
+#include <linux/zalloc.h>
+#include <linux/err.h>
+#include "sample.h"
+#include "debug.h"
+#include "evlist.h"
+#include "evsel.h"
+#include "session.h"
+#include "tool.h"
+#include "metricgroup.h"
+#include <sys/stat.h>
+#include <sys/file.h>
+
+
+
+#define PERF_DATA              "-"
+#define CONTROL                        "/tmp/control"
+#define ACK                    "/tmp/ack"
+pthread_t reader_thread;
+struct child_process *cmd;
+struct perf_stat_config *stat_config;
+
+static int get_perf_record_args(const char **record_argv)
+{
+       int i = 0;
+       struct tpebs_retire_lat *e;
+
+       pr_debug("Prepare perf record for retire_latency\n");
Quoted 8 lines

tools/perf/util/intel-tpebs.c line 44

Quoted 5 lines
tools/perf/util/intel-tpebs.c new file +5 -0
+
+       record_argv[i++] = "perf";
+       record_argv[i++] = "record";
+       record_argv[i++] = "-W";
+       record_argv[i++] = "--synth=no";

There should be more things to disable, like BPF events, we don't need the dummy, etc.

Quoted 1 line

Ok, I will add that.

Well this is my comment that we shouldn't be beefing up "perf record" for retirement latencies, it'd be better to avoid the fork altogether. Adding "--no-bpf-event" is simple.

Quoted 160 lines
Quoted 151 lines
Quoted 98 lines
tools/perf/util/intel-tpebs.c new file +11 -0
+       record_argv[i++] = "--control=fifo:/tmp/control,/tmp/ack";
+
+       if (stat_config->user_requested_cpu_list) {
+               record_argv[i++] = "-C";
+               record_argv[i++] = stat_config->user_requested_cpu_list;
+       }
+
+       if (stat_config->system_wide)
+               record_argv[i++] = "-a";
+
+       if (!stat_config->system_wide && !stat_config-

user_requested_cpu_list) {

tools/perf/util/intel-tpebs.c new file +85 -0
+               pr_err("Require -a or -C option to run sampling.\n");
+               return -ECANCELED;
+       }
+
+       list_for_each_entry(e, &stat_config->tpebs_results, nd) {
+               record_argv[i++] = "-e";
+               record_argv[i++] = e->name;
+       }
+
+       record_argv[i++] = "-o";
+       record_argv[i++] = PERF_DATA;
+
+       return 0;
+}
+
+static int prepare_run_command(const char **argv)
+{
+       cmd = zalloc(sizeof(struct child_process));
+       if (!cmd)
+               return -ENOMEM;
+       cmd->argv = argv;
+       cmd->out = -1;
+       return 0;
+}
+
+static int prepare_perf_record(int tpebs_event_size)
+{
+       const char **record_argv;
+       int ret;
+
+       /*Create control and ack fd for --control*/
+       if (mkfifo(CONTROL, 0600)) {
+               pr_err("Failed to create control fifo");
+               return -1;
+       }
+       if (mkfifo(ACK, 0600)) {
+               pr_err("Failed to create control fifo");
+               return -1;
+       }
+
+       record_argv = calloc(10 + 2 * tpebs_event_size, sizeof(char *));
+       if (!record_argv)
+               return -ENOMEM;
+
+
+       ret = get_perf_record_args(record_argv);
+       if (ret)
+               goto out;
+
+       ret = prepare_run_command(record_argv);
+       if (ret)
+               goto out;
+       ret = start_command(cmd);
+out:
+       free(record_argv);
+       return ret;
+}
+struct sample_data_reader {
+       struct perf_tool        tool;
+       struct perf_session     *session;
+};
+
+static void tpebs_retire_lat__delete(struct tpebs_retire_lat *r)
+{
+       zfree(&r->name);
+       zfree(&r->tpebs_name);
+       free(r);
+}
+
+void tpebs_data__delete(void)
+{
+       struct tpebs_retire_lat *r, *rtmp;
+
+       list_for_each_entry_safe(r, rtmp, &stat_config->tpebs_results, nd) {
+               list_del_init(&r->nd);
+               tpebs_retire_lat__delete(r);
+       }
+       free(cmd);
+}
+
+static int process_sample_event(struct perf_tool *tool __maybe_unused,
+                               union perf_event *event __maybe_unused,
+                               struct perf_sample *sample,
+                               struct evsel *evsel,
+                               struct machine *machine __maybe_unused)

tools/perf/util/intel-tpebs.c

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+{
+       int ret = 0;
+       const char *evname;
+       struct tpebs_retire_lat *t;
+
+       evname = evsel__name(evsel);
+
+       /*
+        * Need to handle per core results? We are assuming average retire
+        * latency value will be used. Save the number of samples and the sum

of

Quoted 28 lines
tools/perf/util/intel-tpebs.c new file +28 -0
+        * retire latency value for each event.
+        */
+       list_for_each_entry(t, &stat_config->tpebs_results, nd) {
+               if (!strcmp(evname, t->name)) {
+                       t->count += 1;
+                       t->sum += sample->retire_lat;
+                       t->val = (double) t->sum / t->count;
+                       break;
+               }
+       }
+
+       return ret;
+}
+
+static int process_feature_event(struct perf_session *session,
+                                union perf_event *event)
+{
+       if (event->feat.feat_id < HEADER_LAST_FEATURE)
+               return perf_event__process_feature(session, event);
+       return 0;
+}
+
+static void *__sample_reader(void *arg)
+{
+       struct child_process *child = arg;
+       struct perf_session *session;
+       struct perf_data data = {
+               .mode = PERF_DATA_MODE_READ,

tools/perf/util/intel-tpebs.c line 188

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+               .path = PERF_DATA,
+               .file.fd = child->out,
+       };
+       struct sample_data_reader reader = {
+               .tool = {
+               .sample          = process_sample_event,
+               .feature         = process_feature_event,
+               .attr            = perf_event__process_attr,
+               },
+       };

I prefer the reading approach here over what I did here:
https://lore.kernel.org/lkml/CAM9d7cgzTsfk3C+dTN80f5FhB1rmfturjuUUwv
[email protected]/

Quoted 8 lines

I'd done that as Namhyung had commented on using perf report:
https://lore.kernel.org/lkml/CAM9d7cgdQQn5GYB7t++xuoMdeqPXiEkkcop69
[email protected]/
It was also less work. I don't see why we can't move the reading logic into a function like evsel__read_retire_latency that's in my change. I also think that in the first instance all reading logic should be implemented to return 0 and we only do the forking, etc. when a command line flag is passed.

Quoted 6 lines

Yes, a command line flag to turn it on is a good idea. Andi also suggested this. I want to add this option in next version.

I tried to refactor the code so that you could easily reuse it later. You could either put this code directly into evsel__read_retire_latency or call this function, whichever way you feel better.

Could you add that into the series rather than leaving it as work for me?

Quoted 135 lines
Quoted 126 lines
Quoted 4 lines
tools/perf/util/intel-tpebs.c new file +3 -0
+
+       session = perf_session__new(&data, &reader.tool);
+       if (IS_ERR(session))

tools/perf/util/intel-tpebs.c

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+               return NULL;
+       reader.session = session;
+       perf_session__process_events(session);
+       perf_session__delete(session);
+
+       return NULL;
+}
+
+
+int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist

*evsel_list)

tools/perf/util/intel-tpebs.c

Quoted 13 lines
tools/perf/util/intel-tpebs.c new file +13 -0
+{
+       int ret = 0;
+       struct evsel *evsel;
+       int control = -1, ack = -1;
+
+       stat_config = perf_stat_config;
+       /*
+        * Prepare perf record for sampling event retire_latency before fork and
+        * prepare workload
+        */
+       evlist__for_each_entry(evsel_list, evsel) {
+               if (evsel->retire_lat) {
+                       struct tpebs_retire_lat *new = malloc(sizeof(struct

tpebs_retire_lat));

Quoted 97 lines
tools/perf/util/intel-tpebs.c new file +4 -0
+                       int i;
+                       char *name;
+
+                       pr_debug("perf stat retire latency %s required\n", evsel-

name);

tools/perf/util/intel-tpebs.c new file +67 -0
+                       if (!new)
+                               return -1;
+                       for (i = strlen(evsel->name) - 1; i > 0; i--) {
+                               if (evsel->name[i] == 'R')
+                                       break;
+                       }
+                       if (i <= 0 || evsel->name[i] != 'R')
+                               return -1;
+
+                       name = strdup(evsel->name);
+                       if (!name)
+                               return -ENOMEM;
+                       name[i] = 'p';
+                       new->name = strdup(name);
+                       free(name);
+                       new->tpebs_name = strdup(evsel->name);
+                       if (!new->tpebs_name)
+                               return -ENOMEM;
+                       new->count = 0;
+                       new->sum = 0;
+                       list_add_tail(&new->nd, &stat_config->tpebs_results);
+                       stat_config->tpebs_event_size += 1;
+               }
+       }
+
+       if (stat_config->tpebs_event_size > 0) {
+               ret = prepare_perf_record(stat_config->tpebs_event_size);
+               if (ret)
+                       return ret;
+               if (pthread_create(&reader_thread, NULL, __sample_reader, cmd)) {
+                       kill(cmd->pid, SIGTERM);
+                       close(cmd->out);
+                       pr_err("Could not create thread to process sample data.\n");
+                       return -1;
+               }
+               /* Wait for perf record initialization a little bit.*/
+               control = open(CONTROL, O_RDONLY, O_NONBLOCK);
+               if (!control)
+                       return -1;
+               close(control);
+               ack = open(ACK, O_RDONLY, O_NONBLOCK);
+               if (!ack)
+                       return -1;
+               close(ack);
+               pr_debug("Received ack from perf record\n");
+       }
+
+       return ret;
+}
+
+
+int stop_tpebs(void)
+{
+       int ret = 0;
+
+       if (stat_config->tpebs_event_size > 0) {
+               kill(cmd->pid, SIGTERM);
+               pthread_join(reader_thread, NULL);
+               close(cmd->out);
+               ret = finish_command(cmd);
+               if (ret == -ERR_RUN_COMMAND_WAITPID_SIGNAL)
+                       ret = 0;
+               remove(CONTROL);
+               remove(ACK);
+       }
+       return ret;
+}
tools/perf/util/intel-tpebs.h new file +19 -0
new file mode 100644
index 000000000000..e8e2bb2f479b
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * intel_pt.h: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.
+ */
+#include "stat.h"
+
+#ifndef INCLUDE__PERF_INTEL_TPEBS_H__
+#define INCLUDE__PERF_INTEL_TPEBS_H__
+
+struct tpebs_retire_lat {
+       struct list_head nd;
+       /* Event name */
+       const char *name;
+       /* Event name with the TPEBS modifier R */
+       const char *tpebs_name;
+       /* Count of retire_latency values found in sample data */
+       size_t count;
+       /* Sum of all the retire_latency values in sample data */
Quoted 8 lines

tools/perf/util/intel-tpebs.h line 23

Quoted 4 lines
tools/perf/util/intel-tpebs.h new file +4 -0
+       int sum;
+       /* Average of retire_latency, val = sum / count */
+       double val;
+};

An evsel has a pretty much all of this and so we're duplicating in particular the counting logic which then needs later work to integrate and is why I'd prefer we went the evsel route.

Quoted 3 lines

I wanted to reuse evsel as much as I could. But as you know, I had some issue to use it and had to remove the code. I'm going to explore more and see if I could get it work with evsel better.

Ok, it was never clear to me what the issue was.

Quoted 2 lines

Comparing with last version, do you think this change is toward the direct that you Want? I believe we could get this code to use evsel more.

I think with the evsels and the record/reading logic here things will be in a good place. There will be more work to do to have the retirement latency respect options like the cgroups, perhaps the BPF filters are an option there.
https://lore.kernel.org/lkml/[email protected]/

Thanks,
Ian

Quoted 31 lines

Thanks,
Weilin

Quoted 28 lines

Thanks,
Ian

tools/perf/util/intel-tpebs.h line 25

Quoted 2 lines
tools/perf/util/intel-tpebs.h new file +2 -0
+
+int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist

*evsel_list);

Quoted 21 lines
tools/perf/util/intel-tpebs.h new file +4 -0
+int stop_tpebs(void);
+void tpebs_data__delete(void);
+
+#endif
tools/perf/util/stat.h +3 -1
index fd7a187551bd..c6c2aa43030f 100644
@@ -110,6 +110,9 @@ struct perf_stat_config {
        struct cpu_aggr_map     *cpus_aggr_map;
        u64                     *walltime_run;
        struct rblist            metric_events;
+       size_t                   tpebs_event_size;
+       struct list_head         tpebs_results;
+       pid_t                    tpebs_pid;
        int                      ctl_fd;
        int                      ctl_fd_ack;
        bool                     ctl_fd_close;
--

2.43.0

#11 lore raw

RE: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 439 lines

-----Original Message-----
From: Ian Rogers <[email protected]>
Sent: Thursday, May 16, 2024 11:07 AM
To: Wang, Weilin <[email protected]>
Cc: Namhyung Kim <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 11 lines · Weilin
Quoted 11 lines

-----Original Message-----
From: Ian Rogers <[email protected]>
Sent: Thursday, May 16, 2024 9:43 AM
To: Wang, Weilin <[email protected]>
Cc: Namhyung Kim <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor,

Perry

Quoted 1 line
Quoted 1 line

<[email protected]>; Alt, Samantha <[email protected]>;

Biggers,

Quoted 2 lines
Quoted 2 lines

Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record

when

Quoted 7 lines
Quoted 7 lines

perf stat needs to get retire latency value for a metric.

Quoted 4 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

When retire_latency value is used in a metric formula, perf stat would

fork a

Quoted 9 lines
Quoted 9 lines
Quoted 5 lines

perf record process with "-e" and "-W" options. Perf record will collect required retire_latency values in parallel while perf stat is collecting counting values.

At the point of time that perf stat stops counting, it would send sigterm

signal

Quoted 1 line

to perf record process and receiving sampling data back from perf record

from a

Quoted 1 line

pipe. Perf stat will then process the received data to get retire latency

data

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

and calculate metric result.

Another thread is required to synchronize between perf stat and perf

record

Quoted 81 lines
Quoted 5 lines
Quoted 4 lines

when we pass data through pipe.

Quoted 14 lines

So I don't think this is the correct way to add this. My reviewed-by was based on the fact that I was going to refactor this after landing. It didn't land and I already sent out:
"Retirement latency perf stat support"
https://lore.kernel.org/lkml/20240428053616.1125891-1-
[email protected]/
v3 to just land the tool portion:
https://lore.kernel.org/lkml/20240503232849.17752-1-
[email protected]/
So my reviewed-by no longer stands. The changes I've sent out haven't been reviewed. Given you're trying to land this, can we work on reviewing those changes? The v3 was specifically done just so that we can have the cleaner basis for adding tpebs to the evsel.

Sorry about carrying the reviewed-by in wrong places. I will remove them.

Quoted 33 lines

Let me point to specific issues below:

Quoted 4 lines
---
 tools/perf/builtin-stat.c     |  19 +++
 tools/perf/util/Build         |   1 +
 tools/perf/util/intel-tpebs.c | 285

++++++++++++++++++++++++++++++++++

Quoted 26 lines
 tools/perf/util/intel-tpebs.h |  29 ++++
 tools/perf/util/stat.h        |   3 +
 5 files changed, 337 insertions(+)
 create mode 100644 tools/perf/util/intel-tpebs.c
 create mode 100644 tools/perf/util/intel-tpebs.h
tools/perf/builtin-stat.c +2 -0
index 428e9721b908..85927cf45adb 100644
@@ -70,6 +70,7 @@
 #include "util/bpf_counter.h"
 #include "util/iostat.h"
 #include "util/util.h"
+#include "util/intel-tpebs.h"
 #include "asm/bug.h"
 
 #include <linux/time64.h>
@@ -94,6 +95,7 @@
 #include <perf/evlist.h>
 #include <internal/threadmap.h>
 
+
 #define DEFAULT_SEPARATOR      " "
 #define FREEZE_ON_SMI_PATH     "devices/cpu/freeze_on_smi"
 
Quoted 10 lines

tools/perf/builtin-stat.c static struct perf_stat_config stat_config = { · line 168

Quoted 6 lines
tools/perf/builtin-stat.c +2 -0
@@ -162,6 +164,8 @@ static struct perf_stat_config stat_config = {
        .ctl_fd                 = -1,
        .ctl_fd_ack             = -1,
        .iostat_run             = false,
+       .tpebs_results          = LIST_HEAD_INIT(stat_config.tpebs_results),
+       .tpebs_pid              = -1,

Here we're adding state to the stat_config for the sake of tpebs events. Were the state in the evsel, as in my changes, we'd not need to carry new global state for tpebs.

Yes, I should remove the .tpebs_results out. How about .tpebs_pid? It is used in builtin-stat later.

Quoted 12 lines

tools/perf/builtin-stat.c static enum counter_recovery

Quoted 5 lines
tools/perf/builtin-stat.c +0 -0
 };
 
 static void evlist__check_cpu_maps(struct evlist *evlist)
@@ -653,6 +657,8 @@ static enum counter_recovery

stat_handle_error(struct evsel *counter)

Quoted 5 lines
tools/perf/builtin-stat.c +2 -0
        if (child_pid != -1)
                kill(child_pid, SIGTERM);
+       if (stat_config.tpebs_pid != -1)
+               stop_tpebs();
Quoted 6 lines
Quoted 2 lines

This logic is builtin-stat but what about other commands that could be using TPEBS for counters?

Could you please let me know the commands that could use it?
I was thinking we could do this logic in evsel once the per core per event fork get updated. I think that would work for all the commands.

Using this code refactored into evsels is what I'd like to see, so I think we're agreed. I'm not thinking of a main command reading counters. The evlist and evsel are exposed through python so potentially any script could use the values, which is why it'd be best to abstract them behind an evsel.

Quoted 114 lines
Quoted 105 lines

tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char

Quoted 5 lines
tools/perf/builtin-stat.c +0 -0
        return COUNTER_FATAL;
 }
 
@@ -673,6 +679,10 @@ static int __run_perf_stat(int argc, const char

**argv, int run_idx)

tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char **argv, int run_idx)

Quoted 9 lines
tools/perf/builtin-stat.c +4 -0
        int err;
        bool second_pass = false;
 
+       err = start_tpebs(&stat_config, evsel_list);
+       if (err < 0)
+               return err;
+
        if (forks) {
                if (evlist__prepare_workload(evsel_list, &target, argv, is_pipe,

workload_exec_failed_signal) < 0) {

tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char

Quoted 2 lines
tools/perf/builtin-stat.c +0 -0
                        perror("failed to prepare workload");
@@ -878,6 +888,10 @@ static int __run_perf_stat(int argc, const char

**argv, int run_idx)

Quoted 21 lines
tools/perf/builtin-stat.c +7 -0
        t1 = rdclock();
 
+       err = stop_tpebs();
+       if (err < 0)
+               return err;
+
        if (stat_config.walltime_run_table)
                stat_config.walltime_run[run_idx] = t1 - t0;
 
@@ -985,6 +999,9 @@ static void sig_atexit(void)
        if (child_pid != -1)
                kill(child_pid, SIGTERM);
 
+       if (stat_config.tpebs_pid != -1)
+               kill(stat_config.tpebs_pid, SIGTERM);
+
        sigprocmask(SIG_SETMASK, &oset, NULL);
 
        if (signr == -1)

tools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv) · line 2936

Quoted 2 lines
tools/perf/builtin-stat.c +0 -0
@@ -2918,5 +2935,7 @@ int cmd_stat(int argc, const char **argv)
        metricgroup__rblist_exit(&stat_config.metric_events);
        evlist__close_control(stat_config.ctl_fd, stat_config.ctl_fd_ack,

&stat_config.ctl_fd_close);

Quoted 62 lines
tools/perf/builtin-stat.c +2 -0
+       tpebs_data__delete();
+
        return status;
 }
tools/perf/util/Build +1 -0
index 292170a99ab6..c9f1d0bb6bf8 100644
@@ -153,6 +153,7 @@ perf-y += clockid.o
 perf-y += list_sort.o
 perf-y += mutex.o
 perf-y += sharded_mutex.o
+perf-y += intel-tpebs.o
 
 perf-$(CONFIG_LIBBPF) += bpf_map.o
 perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.o
tools/perf/util/intel-tpebs.c new file +39 -0
new file mode 100644
index 000000000000..4b7a98794fae
@@ -0,0 +1,285 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * intel_pt.c: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.
+ */
+
+
+#include <sys/param.h>
+#include <subcmd/run-command.h>
+#include <thread.h>
+#include "intel-tpebs.h"
+#include <linux/list.h>
+#include <linux/zalloc.h>
+#include <linux/err.h>
+#include "sample.h"
+#include "debug.h"
+#include "evlist.h"
+#include "evsel.h"
+#include "session.h"
+#include "tool.h"
+#include "metricgroup.h"
+#include <sys/stat.h>
+#include <sys/file.h>
+
+
+
+#define PERF_DATA              "-"
+#define CONTROL                        "/tmp/control"
+#define ACK                    "/tmp/ack"
+pthread_t reader_thread;
+struct child_process *cmd;
+struct perf_stat_config *stat_config;
+
+static int get_perf_record_args(const char **record_argv)
+{
+       int i = 0;
+       struct tpebs_retire_lat *e;
+
+       pr_debug("Prepare perf record for retire_latency\n");
Quoted 8 lines

tools/perf/util/intel-tpebs.c line 44

Quoted 5 lines
tools/perf/util/intel-tpebs.c new file +5 -0
+
+       record_argv[i++] = "perf";
+       record_argv[i++] = "record";
+       record_argv[i++] = "-W";
+       record_argv[i++] = "--synth=no";

There should be more things to disable, like BPF events, we don't need the dummy, etc.

Quoted 1 line

Ok, I will add that.

Well this is my comment that we shouldn't be beefing up "perf record" for retirement latencies, it'd be better to avoid the fork altogether. Adding "--no-bpf-event" is simple.

Quoted 86 lines
Quoted 86 lines
Quoted 86 lines
tools/perf/util/intel-tpebs.c new file +11 -0
+       record_argv[i++] = "--control=fifo:/tmp/control,/tmp/ack";
+
+       if (stat_config->user_requested_cpu_list) {
+               record_argv[i++] = "-C";
+               record_argv[i++] = stat_config->user_requested_cpu_list;
+       }
+
+       if (stat_config->system_wide)
+               record_argv[i++] = "-a";
+
+       if (!stat_config->system_wide && !stat_config-

user_requested_cpu_list) {

tools/perf/util/intel-tpebs.c new file +73 -0
+               pr_err("Require -a or -C option to run sampling.\n");
+               return -ECANCELED;
+       }
+
+       list_for_each_entry(e, &stat_config->tpebs_results, nd) {
+               record_argv[i++] = "-e";
+               record_argv[i++] = e->name;
+       }
+
+       record_argv[i++] = "-o";
+       record_argv[i++] = PERF_DATA;
+
+       return 0;
+}
+
+static int prepare_run_command(const char **argv)
+{
+       cmd = zalloc(sizeof(struct child_process));
+       if (!cmd)
+               return -ENOMEM;
+       cmd->argv = argv;
+       cmd->out = -1;
+       return 0;
+}
+
+static int prepare_perf_record(int tpebs_event_size)
+{
+       const char **record_argv;
+       int ret;
+
+       /*Create control and ack fd for --control*/
+       if (mkfifo(CONTROL, 0600)) {
+               pr_err("Failed to create control fifo");
+               return -1;
+       }
+       if (mkfifo(ACK, 0600)) {
+               pr_err("Failed to create control fifo");
+               return -1;
+       }
+
+       record_argv = calloc(10 + 2 * tpebs_event_size, sizeof(char *));
+       if (!record_argv)
+               return -ENOMEM;
+
+
+       ret = get_perf_record_args(record_argv);
+       if (ret)
+               goto out;
+
+       ret = prepare_run_command(record_argv);
+       if (ret)
+               goto out;
+       ret = start_command(cmd);
+out:
+       free(record_argv);
+       return ret;
+}
+struct sample_data_reader {
+       struct perf_tool        tool;
+       struct perf_session     *session;
+};
+
+static void tpebs_retire_lat__delete(struct tpebs_retire_lat *r)
+{
+       zfree(&r->name);
+       zfree(&r->tpebs_name);
+       free(r);
+}
+
+void tpebs_data__delete(void)
+{
+       struct tpebs_retire_lat *r, *rtmp;
+
Quoted 8 lines
Quoted 8 lines

tools/perf/util/intel-tpebs.c

Quoted 8 lines
tools/perf/util/intel-tpebs.c new file +8 -0
+       list_for_each_entry_safe(r, rtmp, &stat_config->tpebs_results, nd) {
+               list_del_init(&r->nd);
+               tpebs_retire_lat__delete(r);
+       }
+       free(cmd);
+}
+
+static int process_sample_event(struct perf_tool *tool

__maybe_unused,

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
tools/perf/util/intel-tpebs.c new file +6 -0
+                               union perf_event *event __maybe_unused,
+                               struct perf_sample *sample,
+                               struct evsel *evsel,
+                               struct machine *machine __maybe_unused)
+{
+       int ret = 0;
Quoted 8 lines
Quoted 8 lines

tools/perf/util/intel-tpebs.c

Quoted 8 lines
tools/perf/util/intel-tpebs.c new file +8 -0
+       const char *evname;
+       struct tpebs_retire_lat *t;
+
+       evname = evsel__name(evsel);
+
+       /*
+        * Need to handle per core results? We are assuming average retire
+        * latency value will be used. Save the number of samples and the

sum

Quoted 34 lines
Quoted 34 lines

of

Quoted 33 lines
tools/perf/util/intel-tpebs.c new file +33 -0
+        * retire latency value for each event.
+        */
+       list_for_each_entry(t, &stat_config->tpebs_results, nd) {
+               if (!strcmp(evname, t->name)) {
+                       t->count += 1;
+                       t->sum += sample->retire_lat;
+                       t->val = (double) t->sum / t->count;
+                       break;
+               }
+       }
+
+       return ret;
+}
+
+static int process_feature_event(struct perf_session *session,
+                                union perf_event *event)
+{
+       if (event->feat.feat_id < HEADER_LAST_FEATURE)
+               return perf_event__process_feature(session, event);
+       return 0;
+}
+
+static void *__sample_reader(void *arg)
+{
+       struct child_process *child = arg;
+       struct perf_session *session;
+       struct perf_data data = {
+               .mode = PERF_DATA_MODE_READ,
+               .path = PERF_DATA,
+               .file.fd = child->out,
+       };
+       struct sample_data_reader reader = {
+               .tool = {
Quoted 8 lines
Quoted 8 lines

tools/perf/util/intel-tpebs.c line 188

Quoted 5 lines
tools/perf/util/intel-tpebs.c new file +5 -0
+               .sample          = process_sample_event,
+               .feature         = process_feature_event,
+               .attr            = perf_event__process_attr,
+               },
+       };

I prefer the reading approach here over what I did here:

https://lore.kernel.org/lkml/CAM9d7cgzTsfk3C+dTN80f5FhB1rmfturjuUUwv

Quoted 3 lines
Quoted 3 lines

[email protected]/
I'd done that as Namhyung had commented on using perf report:

https://lore.kernel.org/lkml/CAM9d7cgdQQn5GYB7t++xuoMdeqPXiEkkcop69

Quoted 10 lines
Quoted 6 lines

[email protected]/
It was also less work. I don't see why we can't move the reading logic into a function like evsel__read_retire_latency that's in my change. I also think that in the first instance all reading logic should be implemented to return 0 and we only do the forking, etc. when a command line flag is passed.

Yes, a command line flag to turn it on is a good idea. Andi also suggested this. I want to add this option in next version.

Quoted 6 lines
Quoted 1 line

I tried to refactor the code so that you could easily reuse it later. You could

either

Quoted 2 lines

put this code directly into evsel__read_retire_latency or call this function, whichever way you feel better.

Could you add that into the series rather than leaving it as work for me?

Sorry about miscommunicating. I didn’t mean to left the work for you. I was just trying to say if you don't agree with the code at any point of time, you could move it around or do whatever way you'd like.

Quoted 138 lines
Quoted 16 lines
Quoted 16 lines
Quoted 5 lines
tools/perf/util/intel-tpebs.c new file +3 -0
+
+       session = perf_session__new(&data, &reader.tool);
+       if (IS_ERR(session))

tools/perf/util/intel-tpebs.c

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+               return NULL;
+       reader.session = session;
+       perf_session__process_events(session);
+       perf_session__delete(session);
+
+       return NULL;
+}
+
+
+int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist

*evsel_list)

Quoted 8 lines
Quoted 8 lines

tools/perf/util/intel-tpebs.c

Quoted 8 lines
tools/perf/util/intel-tpebs.c new file +8 -0
+{
+       int ret = 0;
+       struct evsel *evsel;
+       int control = -1, ack = -1;
+
+       stat_config = perf_stat_config;
+       /*
+        * Prepare perf record for sampling event retire_latency before fork

and

Quoted 33 lines
Quoted 33 lines

tools/perf/util/intel-tpebs.c

Quoted 5 lines
tools/perf/util/intel-tpebs.c new file +5 -0
+        * prepare workload
+        */
+       evlist__for_each_entry(evsel_list, evsel) {
+               if (evsel->retire_lat) {
+                       struct tpebs_retire_lat *new = malloc(sizeof(struct

tpebs_retire_lat));

Quoted 27 lines
tools/perf/util/intel-tpebs.c new file +4 -0
+                       int i;
+                       char *name;
+
+                       pr_debug("perf stat retire latency %s required\n", evsel-

name);

tools/perf/util/intel-tpebs.c new file +22 -0
+                       if (!new)
+                               return -1;
+                       for (i = strlen(evsel->name) - 1; i > 0; i--) {
+                               if (evsel->name[i] == 'R')
+                                       break;
+                       }
+                       if (i <= 0 || evsel->name[i] != 'R')
+                               return -1;
+
+                       name = strdup(evsel->name);
+                       if (!name)
+                               return -ENOMEM;
+                       name[i] = 'p';
+                       new->name = strdup(name);
+                       free(name);
+                       new->tpebs_name = strdup(evsel->name);
+                       if (!new->tpebs_name)
+                               return -ENOMEM;
+                       new->count = 0;
+                       new->sum = 0;
+                       list_add_tail(&new->nd, &stat_config->tpebs_results);
+                       stat_config->tpebs_event_size += 1;
Quoted 8 lines
Quoted 8 lines

tools/perf/util/intel-tpebs.c

Quoted 8 lines
tools/perf/util/intel-tpebs.c new file +8 -0
+               }
+       }
+
+       if (stat_config->tpebs_event_size > 0) {
+               ret = prepare_perf_record(stat_config->tpebs_event_size);
+               if (ret)
+                       return ret;
+               if (pthread_create(&reader_thread, NULL, __sample_reader,

cmd)) {

Quoted 71 lines
Quoted 70 lines
Quoted 56 lines
tools/perf/util/intel-tpebs.c new file +37 -0
+                       kill(cmd->pid, SIGTERM);
+                       close(cmd->out);
+                       pr_err("Could not create thread to process sample data.\n");
+                       return -1;
+               }
+               /* Wait for perf record initialization a little bit.*/
+               control = open(CONTROL, O_RDONLY, O_NONBLOCK);
+               if (!control)
+                       return -1;
+               close(control);
+               ack = open(ACK, O_RDONLY, O_NONBLOCK);
+               if (!ack)
+                       return -1;
+               close(ack);
+               pr_debug("Received ack from perf record\n");
+       }
+
+       return ret;
+}
+
+
+int stop_tpebs(void)
+{
+       int ret = 0;
+
+       if (stat_config->tpebs_event_size > 0) {
+               kill(cmd->pid, SIGTERM);
+               pthread_join(reader_thread, NULL);
+               close(cmd->out);
+               ret = finish_command(cmd);
+               if (ret == -ERR_RUN_COMMAND_WAITPID_SIGNAL)
+                       ret = 0;
+               remove(CONTROL);
+               remove(ACK);
+       }
+       return ret;
+}
tools/perf/util/intel-tpebs.h new file +13 -0
new file mode 100644
index 000000000000..e8e2bb2f479b
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * intel_pt.h: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.
+ */
+#include "stat.h"
+
+#ifndef INCLUDE__PERF_INTEL_TPEBS_H__
+#define INCLUDE__PERF_INTEL_TPEBS_H__
+
+struct tpebs_retire_lat {
+       struct list_head nd;
+       /* Event name */

tools/perf/util/intel-tpebs.h line 23

Quoted 10 lines
tools/perf/util/intel-tpebs.h new file +10 -0
+       const char *name;
+       /* Event name with the TPEBS modifier R */
+       const char *tpebs_name;
+       /* Count of retire_latency values found in sample data */
+       size_t count;
+       /* Sum of all the retire_latency values in sample data */
+       int sum;
+       /* Average of retire_latency, val = sum / count */
+       double val;
+};

An evsel has a pretty much all of this and so we're duplicating in particular the counting logic which then needs later work to integrate and is why I'd prefer we went the evsel route.

Quoted 7 lines
Quoted 1 line

I wanted to reuse evsel as much as I could. But as you know, I had some

issue

Quoted 1 line

to use it and had to remove the code. I'm going to explore more and see if I

could

Quoted 1 line

get it work with evsel better.

Ok, it was never clear to me what the issue was.

Yes, I was swamped by some work and haven't been able to work on this specific issue and send you the code. I will do this as quick as possible.

Thanks,
Weilin

Quoted 46 lines
Quoted 2 lines

Comparing with last version, do you think this change is toward the direct

that you

Quoted 1 line

Want? I believe we could get this code to use evsel more.

I think with the evsels and the record/reading logic here things will be in a good place. There will be more work to do to have the retirement latency respect options like the cgroups, perhaps the BPF filters are an option there.
https://lore.kernel.org/lkml/20240516041948.3546553-1-
[email protected]/

Thanks,
Ian

Quoted 31 lines

Thanks,
Weilin

Quoted 28 lines

Thanks,
Ian

tools/perf/util/intel-tpebs.h line 25

Quoted 2 lines
tools/perf/util/intel-tpebs.h new file +2 -0
+
+int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist

*evsel_list);

Quoted 21 lines
tools/perf/util/intel-tpebs.h new file +4 -0
+int stop_tpebs(void);
+void tpebs_data__delete(void);
+
+#endif
tools/perf/util/stat.h +3 -1
index fd7a187551bd..c6c2aa43030f 100644
@@ -110,6 +110,9 @@ struct perf_stat_config {
        struct cpu_aggr_map     *cpus_aggr_map;
        u64                     *walltime_run;
        struct rblist            metric_events;
+       size_t                   tpebs_event_size;
+       struct list_head         tpebs_results;
+       pid_t                    tpebs_pid;
        int                      ctl_fd;
        int                      ctl_fd_ack;
        bool                     ctl_fd_close;
--

2.43.0

maintainer
#12 lore raw

Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

In reply to [email protected]
Quoted 19 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

When retire_latency value is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect required retire_latency values in parallel while perf stat is collecting counting values.

At the point of time that perf stat stops counting, it would send sigterm signal to perf record process and receiving sampling data back from perf record from a pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.

Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.

---

[SNIP]

tools/perf/util/intel-tpebs.c line 4

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +4 -0
new file mode 100644
index 000000000000..4b7a98794fae
@@ -0,0 +1,285 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * intel_pt.c: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.

This needs some updates. :)

Quoted 18 lines
tools/perf/util/intel-tpebs.c new file +18 -0
+ */
+
+
+#include <sys/param.h>
+#include <subcmd/run-command.h>
+#include <thread.h>
+#include "intel-tpebs.h"
+#include <linux/list.h>
+#include <linux/zalloc.h>
+#include <linux/err.h>
+#include "sample.h"
+#include "debug.h"
+#include "evlist.h"
+#include "evsel.h"
+#include "session.h"
+#include "tool.h"
+#include "metricgroup.h"
+#include <sys/stat.h>

tools/perf/util/intel-tpebs.c line 32

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+#include <sys/file.h>
+
+
+
+#define PERF_DATA              "-"
+#define CONTROL                        "/tmp/control"
+#define ACK                    "/tmp/ack"
+pthread_t reader_thread;
+struct child_process *cmd;
+struct perf_stat_config *stat_config;

static ?

tools/perf/util/intel-tpebs.c line 44

Quoted 12 lines
tools/perf/util/intel-tpebs.c new file +12 -0
+
+static int get_perf_record_args(const char **record_argv)
+{
+       int i = 0;
+       struct tpebs_retire_lat *e;
+
+       pr_debug("Prepare perf record for retire_latency\n");
+
+       record_argv[i++] = "perf";
+       record_argv[i++] = "record";
+       record_argv[i++] = "-W";
+       record_argv[i++] = "--synth=no";

Unfortunately this still synthesizes MMAP records for the kernel and modules. As we don't care about them and just want to minimize the overhead at the beginning, we can add "--tail-synthesize" too.

tools/perf/util/intel-tpebs.c line 45

Quoted 1 line
tools/perf/util/intel-tpebs.c new file +1 -0
+       record_argv[i++] = "--control=fifo:/tmp/control,/tmp/ack";

This hard-coded path won't work well when more than one users want to run the perf command at the same time.

Thanks,
Namhyung

Quoted 26 lines
tools/perf/util/intel-tpebs.c new file +25 -0
+
+       if (stat_config->user_requested_cpu_list) {
+               record_argv[i++] = "-C";
+               record_argv[i++] = stat_config->user_requested_cpu_list;
+       }
+
+       if (stat_config->system_wide)
+               record_argv[i++] = "-a";
+
+       if (!stat_config->system_wide && !stat_config->user_requested_cpu_list) {
+               pr_err("Require -a or -C option to run sampling.\n");
+               return -ECANCELED;
+       }
+
+       list_for_each_entry(e, &stat_config->tpebs_results, nd) {
+               record_argv[i++] = "-e";
+               record_argv[i++] = e->name;
+       }
+
+       record_argv[i++] = "-o";
+       record_argv[i++] = PERF_DATA;
+
+       return 0;
+}
+
#13 lore raw

RE: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 104 lines

-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 17, 2024 2:43 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 9 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

When retire_latency value is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect required retire_latency values in parallel while perf stat is collecting counting values.

At the point of time that perf stat stops counting, it would send sigterm

signal

Quoted 1 line

to perf record process and receiving sampling data back from perf record

from a

Quoted 9 lines

pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.

Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.

---

[SNIP]

tools/perf/util/intel-tpebs.c line 4

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +4 -0
new file mode 100644
index 000000000000..4b7a98794fae
@@ -0,0 +1,285 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * intel_pt.c: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.

This needs some updates. :)

Quoted 18 lines
tools/perf/util/intel-tpebs.c new file +18 -0
+ */
+
+
+#include <sys/param.h>
+#include <subcmd/run-command.h>
+#include <thread.h>
+#include "intel-tpebs.h"
+#include <linux/list.h>
+#include <linux/zalloc.h>
+#include <linux/err.h>
+#include "sample.h"
+#include "debug.h"
+#include "evlist.h"
+#include "evsel.h"
+#include "session.h"
+#include "tool.h"
+#include "metricgroup.h"
+#include <sys/stat.h>

tools/perf/util/intel-tpebs.c line 32

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+#include <sys/file.h>
+
+
+
+#define PERF_DATA              "-"
+#define CONTROL                        "/tmp/control"
+#define ACK                    "/tmp/ack"
+pthread_t reader_thread;
+struct child_process *cmd;
+struct perf_stat_config *stat_config;

static ?

tools/perf/util/intel-tpebs.c line 44

Quoted 12 lines
tools/perf/util/intel-tpebs.c new file +12 -0
+
+static int get_perf_record_args(const char **record_argv)
+{
+       int i = 0;
+       struct tpebs_retire_lat *e;
+
+       pr_debug("Prepare perf record for retire_latency\n");
+
+       record_argv[i++] = "perf";
+       record_argv[i++] = "record";
+       record_argv[i++] = "-W";
+       record_argv[i++] = "--synth=no";

Unfortunately this still synthesizes MMAP records for the kernel and modules. As we don't care about them and just want to minimize the overhead at the beginning, we can add "--tail-synthesize" too.

Quoted 5 lines

tools/perf/util/intel-tpebs.c line 45

Quoted 1 line
tools/perf/util/intel-tpebs.c new file +1 -0
+       record_argv[i++] = "--control=fifo:/tmp/control,/tmp/ack";

This hard-coded path won't work well when more than one users want to run the perf command at the same time.

Hi Namhyung,

Yes, this is not a good implementation. I'm going to update this named pipe and all the related code.

Thanks,
Weilin

Quoted 30 lines

Thanks,
Namhyung

Quoted 27 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+
+       if (stat_config->user_requested_cpu_list) {
+               record_argv[i++] = "-C";
+               record_argv[i++] = stat_config->user_requested_cpu_list;
+       }
+
+       if (stat_config->system_wide)
+               record_argv[i++] = "-a";
+
+       if (!stat_config->system_wide && !stat_config-

user_requested_cpu_list) {

tools/perf/util/intel-tpebs.c new file +15 -0
+               pr_err("Require -a or -C option to run sampling.\n");
+               return -ECANCELED;
+       }
+
+       list_for_each_entry(e, &stat_config->tpebs_results, nd) {
+               record_argv[i++] = "-e";
+               record_argv[i++] = e->name;
+       }
+
+       record_argv[i++] = "-o";
+       record_argv[i++] = PERF_DATA;
+
+       return 0;
+}
+
#14 lore raw

RE: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

In reply to Namhyung Kim
Quoted 94 lines

-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 17, 2024 2:43 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 9 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

When retire_latency value is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect required retire_latency values in parallel while perf stat is collecting counting values.

At the point of time that perf stat stops counting, it would send sigterm

signal

Quoted 1 line

to perf record process and receiving sampling data back from perf record

from a

Quoted 9 lines

pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.

Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.

---

[SNIP]

tools/perf/util/intel-tpebs.c line 4

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +4 -0
new file mode 100644
index 000000000000..4b7a98794fae
@@ -0,0 +1,285 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * intel_pt.c: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.

This needs some updates. :)

Quoted 18 lines
tools/perf/util/intel-tpebs.c new file +18 -0
+ */
+
+
+#include <sys/param.h>
+#include <subcmd/run-command.h>
+#include <thread.h>
+#include "intel-tpebs.h"
+#include <linux/list.h>
+#include <linux/zalloc.h>
+#include <linux/err.h>
+#include "sample.h"
+#include "debug.h"
+#include "evlist.h"
+#include "evsel.h"
+#include "session.h"
+#include "tool.h"
+#include "metricgroup.h"
+#include <sys/stat.h>

tools/perf/util/intel-tpebs.c line 32

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+#include <sys/file.h>
+
+
+
+#define PERF_DATA              "-"
+#define CONTROL                        "/tmp/control"
+#define ACK                    "/tmp/ack"
+pthread_t reader_thread;
+struct child_process *cmd;
+struct perf_stat_config *stat_config;

static ?

Quoted 9 lines
tools/perf/util/intel-tpebs.c new file +9 -0
+
+static int get_perf_record_args(const char **record_argv)
+{
+       int i = 0;
+       struct tpebs_retire_lat *e;
+
+       pr_debug("Prepare perf record for retire_latency\n");
+
+       record_argv[i++] = "perf";
Quoted 8 lines

tools/perf/util/intel-tpebs.c line 44

Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3 -0
+       record_argv[i++] = "record";
+       record_argv[i++] = "-W";
+       record_argv[i++] = "--synth=no";

Unfortunately this still synthesizes MMAP records for the kernel and modules. As we don't care about them and just want to minimize the overhead at the beginning, we can add "--tail-synthesize" too.

Hi Namhyung,

I'm trying out the "--tail-synthesize" option but failed to get it work correctly. Could you please take a look at this command and let me know what's the problem?

"perf record -e cycles:p --synth=no --tail-synthesize -W -a -o - sleep 1 | perf script -F retire_lat -i -".

I got an error "0x40 [0x40]: failed to process type: 9" when run this command.

If I run the command in two steps without pipe, they work fine.

Thanks,
Weilin

Quoted 37 lines

tools/perf/util/intel-tpebs.c line 45

Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +1 -0
+       record_argv[i++] = "--control=fifo:/tmp/control,/tmp/ack";

This hard-coded path won't work well when more than one users want to run the perf command at the same time.

Thanks,
Namhyung

Quoted 27 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+
+       if (stat_config->user_requested_cpu_list) {
+               record_argv[i++] = "-C";
+               record_argv[i++] = stat_config->user_requested_cpu_list;
+       }
+
+       if (stat_config->system_wide)
+               record_argv[i++] = "-a";
+
+       if (!stat_config->system_wide && !stat_config-

user_requested_cpu_list) {

tools/perf/util/intel-tpebs.c new file +15 -0
+               pr_err("Require -a or -C option to run sampling.\n");
+               return -ECANCELED;
+       }
+
+       list_for_each_entry(e, &stat_config->tpebs_results, nd) {
+               record_argv[i++] = "-e";
+               record_argv[i++] = e->name;
+       }
+
+       record_argv[i++] = "-o";
+       record_argv[i++] = PERF_DATA;
+
+       return 0;
+}
+
maintainer
#15 lore raw

Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Hello,

Quoted 110 lines · Weilin
Quoted 95 lines

-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 17, 2024 2:43 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 9 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

When retire_latency value is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect required retire_latency values in parallel while perf stat is collecting counting values.

At the point of time that perf stat stops counting, it would send sigterm

signal

Quoted 1 line

to perf record process and receiving sampling data back from perf record

from a

Quoted 9 lines

pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.

Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.

---

[SNIP]

tools/perf/util/intel-tpebs.c line 4

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +4 -0
new file mode 100644
index 000000000000..4b7a98794fae
@@ -0,0 +1,285 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * intel_pt.c: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.

This needs some updates. :)

Quoted 18 lines
tools/perf/util/intel-tpebs.c new file +18 -0
+ */
+
+
+#include <sys/param.h>
+#include <subcmd/run-command.h>
+#include <thread.h>
+#include "intel-tpebs.h"
+#include <linux/list.h>
+#include <linux/zalloc.h>
+#include <linux/err.h>
+#include "sample.h"
+#include "debug.h"
+#include "evlist.h"
+#include "evsel.h"
+#include "session.h"
+#include "tool.h"
+#include "metricgroup.h"
+#include <sys/stat.h>

tools/perf/util/intel-tpebs.c line 32

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+#include <sys/file.h>
+
+
+
+#define PERF_DATA              "-"
+#define CONTROL                        "/tmp/control"
+#define ACK                    "/tmp/ack"
+pthread_t reader_thread;
+struct child_process *cmd;
+struct perf_stat_config *stat_config;

static ?

Quoted 9 lines
tools/perf/util/intel-tpebs.c new file +9 -0
+
+static int get_perf_record_args(const char **record_argv)
+{
+       int i = 0;
+       struct tpebs_retire_lat *e;
+
+       pr_debug("Prepare perf record for retire_latency\n");
+
+       record_argv[i++] = "perf";
Quoted 8 lines

tools/perf/util/intel-tpebs.c line 44

Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3 -0
+       record_argv[i++] = "record";
+       record_argv[i++] = "-W";
+       record_argv[i++] = "--synth=no";

Unfortunately this still synthesizes MMAP records for the kernel and modules. As we don't care about them and just want to minimize the overhead at the beginning, we can add "--tail-synthesize" too.

Hi Namhyung,

I'm trying out the "--tail-synthesize" option but failed to get it work correctly. Could you please take a look at this command and let me know what's the problem?

Quoted 5 lines

"perf record -e cycles:p --synth=no --tail-synthesize -W -a -o - sleep 1 | perf script -F retire_lat -i -".

I got an error "0x40 [0x40]: failed to process type: 9" when run this command.

If I run the command in two steps without pipe, they work fine.

Hmm... it may not work with pipes. Let's go without the tail-synthesize then. Sorry for the noise.

Thanks,
Namhyung

#16 lore raw

RE: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Quoted 140 lines

-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Monday, May 20, 2024 10:43 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.

Hello,

Quoted 13 lines · Weilin
Quoted 13 lines

-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 17, 2024 2:43 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor,

Perry

Quoted 1 line
Quoted 1 line

<[email protected]>; Alt, Samantha <[email protected]>;

Biggers,

Quoted 2 lines
Quoted 2 lines

Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 3/7] perf stat: Fork and launch perf record

when

Quoted 7 lines
Quoted 7 lines

perf stat needs to get retire latency value for a metric.

Quoted 4 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

When retire_latency value is used in a metric formula, perf stat would

fork a

Quoted 9 lines
Quoted 9 lines
Quoted 5 lines

perf record process with "-e" and "-W" options. Perf record will collect required retire_latency values in parallel while perf stat is collecting counting values.

At the point of time that perf stat stops counting, it would send sigterm

signal

Quoted 1 line

to perf record process and receiving sampling data back from perf record

from a

Quoted 1 line

pipe. Perf stat will then process the received data to get retire latency

data

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

and calculate metric result.

Another thread is required to synchronize between perf stat and perf

record

Quoted 69 lines
Quoted 60 lines
Quoted 5 lines

when we pass data through pipe.

---

[SNIP]

tools/perf/util/intel-tpebs.c line 4

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +4 -0
new file mode 100644
index 000000000000..4b7a98794fae
@@ -0,0 +1,285 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * intel_pt.c: Intel Processor Trace support
+ * Copyright (c) 2013-2015, Intel Corporation.

This needs some updates. :)

Quoted 18 lines
tools/perf/util/intel-tpebs.c new file +18 -0
+ */
+
+
+#include <sys/param.h>
+#include <subcmd/run-command.h>
+#include <thread.h>
+#include "intel-tpebs.h"
+#include <linux/list.h>
+#include <linux/zalloc.h>
+#include <linux/err.h>
+#include "sample.h"
+#include "debug.h"
+#include "evlist.h"
+#include "evsel.h"
+#include "session.h"
+#include "tool.h"
+#include "metricgroup.h"
+#include <sys/stat.h>

tools/perf/util/intel-tpebs.c line 32

Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10 -0
+#include <sys/file.h>
+
+
+
+#define PERF_DATA              "-"
+#define CONTROL                        "/tmp/control"
+#define ACK                    "/tmp/ack"
+pthread_t reader_thread;
+struct child_process *cmd;
+struct perf_stat_config *stat_config;

static ?

Quoted 9 lines
tools/perf/util/intel-tpebs.c new file +9 -0
+
+static int get_perf_record_args(const char **record_argv)
+{
+       int i = 0;
+       struct tpebs_retire_lat *e;
+
+       pr_debug("Prepare perf record for retire_latency\n");
+
+       record_argv[i++] = "perf";
Quoted 8 lines

tools/perf/util/intel-tpebs.c line 44

Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3 -0
+       record_argv[i++] = "record";
+       record_argv[i++] = "-W";
+       record_argv[i++] = "--synth=no";

Unfortunately this still synthesizes MMAP records for the kernel and modules. As we don't care about them and just want to minimize the overhead at the beginning, we can add "--tail-synthesize" too.

Quoted 7 lines

Hi Namhyung,

I'm trying out the "--tail-synthesize" option but failed to get it work correctly. Could you please take a look at this command and let me know what's the problem?

"perf record -e cycles:p --synth=no --tail-synthesize -W -a -o - sleep 1 | perf

script -F retire_lat -i -".

Quoted 1 line
Quoted 7 lines
Quoted 1 line

I got an error "0x40 [0x40]: failed to process type: 9" when run this

command.

Quoted 2 lines

If I run the command in two steps without pipe, they work fine.

Hmm... it may not work with pipes. Let's go without the tail-synthesize then. Sorry for the noise.

No problem.

Thanks,
Weilin

Quoted 3 lines

Thanks,
Namhyung

#17 lore raw

RFC PATCH v8 4/7 perf stat: Add retire latency values into the expr_parse_ctx to prepare for final metric calculation

In reply to [email protected]

From: Weilin Wang <[email protected]>

Retire latency values of events are used in metric formulas. This update adds code to process data from perf record for required retire latency values.

---
 tools/perf/util/stat-shadow.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
tools/perf/util/stat-shadow.c +23 -0
index 3466aa952442..b97d23bfeac1 100644
@@ -15,6 +15,7 @@
 #include <linux/zalloc.h>
 #include "iostat.h"
 #include "util/hashmap.h"
+#include "util/intel-tpebs.h"
 
 struct stats walltime_nsecs_stats;
 struct rusage_stats ru_stats;
@@ -355,6 +356,20 @@ static void print_nsecs(struct perf_stat_config *config,
 		print_metric(config, ctxp, NULL, NULL, "CPUs utilized", 0);
 }
 
+static int prepare_retire_lat(struct expr_parse_ctx *pctx,
+			     struct list_head *retire_lats)
+{
+	int ret = 0;
+	struct tpebs_retire_lat *t;
+
+	list_for_each_entry(t, retire_lats, nd) {
+		ret = expr__add_id_val(pctx, strdup(t->tpebs_name), t->val);
+		if (ret < 0)
+			return ret;
+	}
+	return ret;
+}
+
 static int prepare_metric(const struct metric_expr *mexp,
 			  const struct evsel *evsel,
 			  struct expr_parse_ctx *pctx,
@@ -369,6 +384,9 @@ static int prepare_metric(const struct metric_expr *mexp,
 		double val;
 		int source_count = 0;
 
+		if (metric_events[i]->retire_lat)
+			continue;
+
 		if (evsel__is_tool(metric_events[i])) {
 			struct stats *stats;
 			double scale;
@@ -486,6 +504,11 @@ static void generic_metric(struct perf_stat_config *config,
 		pctx->sctx.user_requested_cpu_list = strdup(config->user_requested_cpu_list);
 	pctx->sctx.runtime = runtime;
 	pctx->sctx.system_wide = config->system_wide;
+	i = prepare_retire_lat(pctx, &config->tpebs_results);
+	if (i < 0) {
+		expr__ctx_free(pctx);
+		return;
+	}
 	i = prepare_metric(mexp, evsel, pctx, aggr_idx);
 	if (i < 0) {
 		expr__ctx_free(pctx);
Signature · 1 line
2.43.0
reviewer
#18 lore raw

Re: [RFC PATCH v8 4/7] perf stat: Add retire latency values into the expr_parse_ctx to prepare for final metric calculation

Quoted 8 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

Retire latency values of events are used in metric formulas. This update adds code to process data from perf record for required retire latency values.

In the evsel approach:
https://lore.kernel.org/lkml/[email protected]/
we don't need to special case tpebs events like this. I'd prefer we went that route. My reviewed-by no longer stands.

Thanks,
Ian

Quoted 62 lines
---
 tools/perf/util/stat-shadow.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
tools/perf/util/stat-shadow.c +23 -1
index 3466aa952442..b97d23bfeac1 100644
@@ -15,6 +15,7 @@
 #include <linux/zalloc.h>
 #include "iostat.h"
 #include "util/hashmap.h"
+#include "util/intel-tpebs.h"
 
 struct stats walltime_nsecs_stats;
 struct rusage_stats ru_stats;
@@ -355,6 +356,20 @@ static void print_nsecs(struct perf_stat_config *config,
                print_metric(config, ctxp, NULL, NULL, "CPUs utilized", 0);
 }
 
+static int prepare_retire_lat(struct expr_parse_ctx *pctx,
+                            struct list_head *retire_lats)
+{
+       int ret = 0;
+       struct tpebs_retire_lat *t;
+
+       list_for_each_entry(t, retire_lats, nd) {
+               ret = expr__add_id_val(pctx, strdup(t->tpebs_name), t->val);
+               if (ret < 0)
+                       return ret;
+       }
+       return ret;
+}
+
 static int prepare_metric(const struct metric_expr *mexp,
                          const struct evsel *evsel,
                          struct expr_parse_ctx *pctx,
@@ -369,6 +384,9 @@ static int prepare_metric(const struct metric_expr *mexp,
                double val;
                int source_count = 0;
 
+               if (metric_events[i]->retire_lat)
+                       continue;
+
                if (evsel__is_tool(metric_events[i])) {
                        struct stats *stats;
                        double scale;
@@ -486,6 +504,11 @@ static void generic_metric(struct perf_stat_config *config,
                pctx->sctx.user_requested_cpu_list = strdup(config->user_requested_cpu_list);
        pctx->sctx.runtime = runtime;
        pctx->sctx.system_wide = config->system_wide;
+       i = prepare_retire_lat(pctx, &config->tpebs_results);
+       if (i < 0) {
+               expr__ctx_free(pctx);
+               return;
+       }
        i = prepare_metric(mexp, evsel, pctx, aggr_idx);
        if (i < 0) {
                expr__ctx_free(pctx);
--

2.43.0

#19 lore raw

RFC PATCH v8 5/7 perf stat: Add retire latency print functions to print out at the very end of print out

In reply to [email protected]

From: Weilin Wang <[email protected]>

Add print out functions so that users could read retire latency values.

Example output:

Performance counter stats for 'system wide':

            25,717      MEM_INST_RETIRED.SPLIT_STORES    #      2.2 %  tma_split_stores
        28,365,080      CPU_CLK_UNHALTED.THREAD
             24.00      MEM_INST_RETIRED.SPLIT_STORES:R  #       96        4
       2.054365083 seconds time elapsed

This output of retire latency data is in format:
<val> <event-name:R> # <sum> <count>.

---
 tools/perf/util/stat-display.c | 69 ++++++++++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)
tools/perf/util/stat-display.c +69 -0
index bfc1d705f437..b9c3978cc99c 100644
@@ -21,6 +21,7 @@
 #include "iostat.h"
 #include "pmu.h"
 #include "pmus.h"
+#include "intel-tpebs.h"
 
 #define CNTR_NOT_SUPPORTED	"<not supported>"
 #define CNTR_NOT_COUNTED	"<not counted>"
@@ -34,6 +35,7 @@
 #define COMM_LEN     16
 #define PID_LEN       7
 #define CPUS_LEN      4
+#define RETIRE_LEN    8
 
 static int aggr_header_lens[] = {
 	[AGGR_CORE] 	= 18,
@@ -426,6 +428,71 @@ static void print_metric_std(struct perf_stat_config *config,
 	fprintf(out, " %-*s", METRIC_LEN - n - 1, unit);
 }
 
+static void print_retire_lat_std(struct perf_stat_config *config,
+				 struct outstate *os)
+{
+	FILE *out = os->fh;
+	bool newline = os->newline;
+	struct tpebs_retire_lat *t;
+	struct list_head *retire_lats = &config->tpebs_results;
+
+	list_for_each_entry(t, retire_lats, nd) {
+		if (newline)
+			do_new_line_std(config, os);
+		fprintf(out, "%'*.2f ", COUNTS_LEN, t->val);
+		/* For print alignment */
+		fprintf(out, "%-*s ", config->unit_width, "");
+		fprintf(out, "%-*s", EVNAME_LEN, t->tpebs_name);
+		fprintf(out, " # ");
+		fprintf(out, "%*d %*ld\n", RETIRE_LEN, t->sum,
+			 RETIRE_LEN, t->count);
+	}
+}
+
+static void print_retire_lat_csv(struct perf_stat_config *config,
+				 struct outstate *os)
+{
+	FILE *out = os->fh;
+	struct tpebs_retire_lat *t;
+	struct list_head *retire_lats = &config->tpebs_results;
+	const char *sep = config->csv_sep;
+
+	list_for_each_entry(t, retire_lats, nd) {
+		fprintf(out, "%f%s%s%s%s%ld%s%d\n", t->val, sep, sep, t->tpebs_name, sep,
+			t->count, sep, t->sum);
+	}
+}
+
+static void print_retire_lat_json(struct perf_stat_config *config,
+				  struct outstate *os)
+{
+	FILE *out = os->fh;
+	struct tpebs_retire_lat *t;
+	struct list_head *retire_lats = &config->tpebs_results;
+
+	fprintf(out, "{");
+	list_for_each_entry(t, retire_lats, nd) {
+		fprintf(out, "\"retire_latency-value\" : \"%f\", ", t->val);
+		fprintf(out, "\"name\" : \"%s\"", t->tpebs_name);
+		fprintf(out, "\"sample-counts\" : \"%ld\"", t->count);
+		fprintf(out, "\"retire_latency-sum\" : \"%d\"", t->sum);
+	}
+	fprintf(out, "}");
+}
+
+static void print_retire_lat(struct perf_stat_config *config,
+			     struct outstate *os)
+{
+	if (!&config->tpebs_results)
+		return;
+	if (config->json_output)
+		print_retire_lat_json(config, os);
+	else if (config->csv_output)
+		print_retire_lat_csv(config, os);
+	else
+		print_retire_lat_std(config, os);
+}
+
 static void new_line_csv(struct perf_stat_config *config, void *ctx)
 {
 	struct outstate *os = ctx;
@@ -1609,6 +1676,8 @@ void evlist__print_counters(struct evlist *evlist, struct perf_stat_config *conf
 		break;
 	}
 
+	print_retire_lat(config, &os);
+
 	print_footer(config);
 
 	fflush(config->output);
Signature · 1 line
2.43.0
reviewer
#20 lore raw

Re: [RFC PATCH v8 5/7] perf stat: Add retire latency print functions to print out at the very end of print out

Quoted 13 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

Add print out functions so that users could read retire latency values.

Example output:

Performance counter stats for 'system wide':

            25,717      MEM_INST_RETIRED.SPLIT_STORES    #      2.2 %  tma_split_stores
        28,365,080      CPU_CLK_UNHALTED.THREAD
             24.00      MEM_INST_RETIRED.SPLIT_STORES:R  #       96        4
Quoted 7 lines
       2.054365083 seconds time elapsed

This output of retire latency data is in format:
<val> <event-name:R> # <sum> <count>.

My usual complaint that I hate the stat-display spaghetti code. We keep putting more spaghetti on the plate and this change does this too. In the evsel approach:
https://lore.kernel.org/lkml/[email protected]/
retirement latency events just update the counts for the event and so we don't need to special case tpebs events like this. I'd prefer we went that route. My reviewed-by no longer stands.

Thanks,
Ian

Quoted 108 lines
---
 tools/perf/util/stat-display.c | 69 ++++++++++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)
tools/perf/util/stat-display.c +69 -1
index bfc1d705f437..b9c3978cc99c 100644
@@ -21,6 +21,7 @@
 #include "iostat.h"
 #include "pmu.h"
 #include "pmus.h"
+#include "intel-tpebs.h"
 
 #define CNTR_NOT_SUPPORTED     "<not supported>"
 #define CNTR_NOT_COUNTED       "<not counted>"
@@ -34,6 +35,7 @@
 #define COMM_LEN     16
 #define PID_LEN       7
 #define CPUS_LEN      4
+#define RETIRE_LEN    8
 
 static int aggr_header_lens[] = {
        [AGGR_CORE]     = 18,
@@ -426,6 +428,71 @@ static void print_metric_std(struct perf_stat_config *config,
        fprintf(out, " %-*s", METRIC_LEN - n - 1, unit);
 }
 
+static void print_retire_lat_std(struct perf_stat_config *config,
+                                struct outstate *os)
+{
+       FILE *out = os->fh;
+       bool newline = os->newline;
+       struct tpebs_retire_lat *t;
+       struct list_head *retire_lats = &config->tpebs_results;
+
+       list_for_each_entry(t, retire_lats, nd) {
+               if (newline)
+                       do_new_line_std(config, os);
+               fprintf(out, "%'*.2f ", COUNTS_LEN, t->val);
+               /* For print alignment */
+               fprintf(out, "%-*s ", config->unit_width, "");
+               fprintf(out, "%-*s", EVNAME_LEN, t->tpebs_name);
+               fprintf(out, " # ");
+               fprintf(out, "%*d %*ld\n", RETIRE_LEN, t->sum,
+                        RETIRE_LEN, t->count);
+       }
+}
+
+static void print_retire_lat_csv(struct perf_stat_config *config,
+                                struct outstate *os)
+{
+       FILE *out = os->fh;
+       struct tpebs_retire_lat *t;
+       struct list_head *retire_lats = &config->tpebs_results;
+       const char *sep = config->csv_sep;
+
+       list_for_each_entry(t, retire_lats, nd) {
+               fprintf(out, "%f%s%s%s%s%ld%s%d\n", t->val, sep, sep, t->tpebs_name, sep,
+                       t->count, sep, t->sum);
+       }
+}
+
+static void print_retire_lat_json(struct perf_stat_config *config,
+                                 struct outstate *os)
+{
+       FILE *out = os->fh;
+       struct tpebs_retire_lat *t;
+       struct list_head *retire_lats = &config->tpebs_results;
+
+       fprintf(out, "{");
+       list_for_each_entry(t, retire_lats, nd) {
+               fprintf(out, "\"retire_latency-value\" : \"%f\", ", t->val);
+               fprintf(out, "\"name\" : \"%s\"", t->tpebs_name);
+               fprintf(out, "\"sample-counts\" : \"%ld\"", t->count);
+               fprintf(out, "\"retire_latency-sum\" : \"%d\"", t->sum);
+       }
+       fprintf(out, "}");
+}
+
+static void print_retire_lat(struct perf_stat_config *config,
+                            struct outstate *os)
+{
+       if (!&config->tpebs_results)
+               return;
+       if (config->json_output)
+               print_retire_lat_json(config, os);
+       else if (config->csv_output)
+               print_retire_lat_csv(config, os);
+       else
+               print_retire_lat_std(config, os);
+}
+
 static void new_line_csv(struct perf_stat_config *config, void *ctx)
 {
        struct outstate *os = ctx;
@@ -1609,6 +1676,8 @@ void evlist__print_counters(struct evlist *evlist, struct perf_stat_config *conf
                break;
        }
 
+       print_retire_lat(config, &os);
+
        print_footer(config);
 
        fflush(config->output);
--

2.43.0

#21 lore raw

RE: [RFC PATCH v8 5/7] perf stat: Add retire latency print functions to print out at the very end of print out

Quoted 40 lines

-----Original Message-----
From: Ian Rogers <[email protected]>
Sent: Thursday, May 16, 2024 9:47 AM
To: Wang, Weilin <[email protected]>
Cc: Namhyung Kim <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 5/7] perf stat: Add retire latency print functions to print out at the very end of print out

Quoted 10 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

Add print out functions so that users could read retire latency values.

Example output:

Performance counter stats for 'system wide':

            25,717      MEM_INST_RETIRED.SPLIT_STORES    #      2.2 %

tma_split_stores

Quoted 10 lines
        28,365,080      CPU_CLK_UNHALTED.THREAD
             24.00      MEM_INST_RETIRED.SPLIT_STORES:R  #       96        4

       2.054365083 seconds time elapsed

This output of retire latency data is in format:
<val> <event-name:R> # <sum> <count>.

Quoted 9 lines

My usual complaint that I hate the stat-display spaghetti code. We keep putting more spaghetti on the plate and this change does this too. In the evsel approach:
https://lore.kernel.org/lkml/20240428053616.1125891-1-
[email protected]/
retirement latency events just update the counts for the event and so we don't need to special case tpebs events like this. I'd prefer we went that route. My reviewed-by no longer stands.

Based on the current stat-display code and the original TPEBS counting code, I believe the code in this commit and last commit was the best way to do the metric calculation and printout.

But I totally agree with you that if we could get evsel with retire latency work, this code is not necessary.

I was thinking to plugin the retire latency value into evsel so that I could delete this and last commit. Do you think that would be a solution?

Thanks,
Weilin

Quoted 115 lines

Thanks,
Ian

Quoted 2 lines
---
 tools/perf/util/stat-display.c | 69

++++++++++++++++++++++++++++++++++

tools/perf/util/stat-display.c static void print_metric_std(struct

Quoted 23 lines
 1 file changed, 69 insertions(+)
tools/perf/util/stat-display.c +2 -0
index bfc1d705f437..b9c3978cc99c 100644
@@ -21,6 +21,7 @@
 #include "iostat.h"
 #include "pmu.h"
 #include "pmus.h"
+#include "intel-tpebs.h"
 
 #define CNTR_NOT_SUPPORTED     "<not supported>"
 #define CNTR_NOT_COUNTED       "<not counted>"
@@ -34,6 +35,7 @@
 #define COMM_LEN     16
 #define PID_LEN       7
 #define CPUS_LEN      4
+#define RETIRE_LEN    8
 
 static int aggr_header_lens[] = {
        [AGGR_CORE]     = 18,
@@ -426,6 +428,71 @@ static void print_metric_std(struct

perf_stat_config *config,

tools/perf/util/stat-display.c void evlist__print_counters(struct evlist *evlist,

Quoted 73 lines
tools/perf/util/stat-display.c +65 -0
        fprintf(out, " %-*s", METRIC_LEN - n - 1, unit);
 }
 
+static void print_retire_lat_std(struct perf_stat_config *config,
+                                struct outstate *os)
+{
+       FILE *out = os->fh;
+       bool newline = os->newline;
+       struct tpebs_retire_lat *t;
+       struct list_head *retire_lats = &config->tpebs_results;
+
+       list_for_each_entry(t, retire_lats, nd) {
+               if (newline)
+                       do_new_line_std(config, os);
+               fprintf(out, "%'*.2f ", COUNTS_LEN, t->val);
+               /* For print alignment */
+               fprintf(out, "%-*s ", config->unit_width, "");
+               fprintf(out, "%-*s", EVNAME_LEN, t->tpebs_name);
+               fprintf(out, " # ");
+               fprintf(out, "%*d %*ld\n", RETIRE_LEN, t->sum,
+                        RETIRE_LEN, t->count);
+       }
+}
+
+static void print_retire_lat_csv(struct perf_stat_config *config,
+                                struct outstate *os)
+{
+       FILE *out = os->fh;
+       struct tpebs_retire_lat *t;
+       struct list_head *retire_lats = &config->tpebs_results;
+       const char *sep = config->csv_sep;
+
+       list_for_each_entry(t, retire_lats, nd) {
+               fprintf(out, "%f%s%s%s%s%ld%s%d\n", t->val, sep, sep, t-
 tpebs_name, sep,
+                       t->count, sep, t->sum);
+       }
+}
+
+static void print_retire_lat_json(struct perf_stat_config *config,
+                                 struct outstate *os)
+{
+       FILE *out = os->fh;
+       struct tpebs_retire_lat *t;
+       struct list_head *retire_lats = &config->tpebs_results;
+
+       fprintf(out, "{");
+       list_for_each_entry(t, retire_lats, nd) {
+               fprintf(out, "\"retire_latency-value\" : \"%f\", ", t->val);
+               fprintf(out, "\"name\" : \"%s\"", t->tpebs_name);
+               fprintf(out, "\"sample-counts\" : \"%ld\"", t->count);
+               fprintf(out, "\"retire_latency-sum\" : \"%d\"", t->sum);
+       }
+       fprintf(out, "}");
+}
+
+static void print_retire_lat(struct perf_stat_config *config,
+                            struct outstate *os)
+{
+       if (!&config->tpebs_results)
+               return;
+       if (config->json_output)
+               print_retire_lat_json(config, os);
+       else if (config->csv_output)
+               print_retire_lat_csv(config, os);
+       else
+               print_retire_lat_std(config, os);
+}
+
 static void new_line_csv(struct perf_stat_config *config, void *ctx)
 {
        struct outstate *os = ctx;
tools/perf/util/stat-display.c +0 -0
@@ -1609,6 +1676,8 @@ void evlist__print_counters(struct evlist *evlist,

struct perf_stat_config *conf

Quoted 11 lines
tools/perf/util/stat-display.c +2 -1
                break;
        }
 
+       print_retire_lat(config, &os);
+
        print_footer(config);
 
        fflush(config->output);
--

2.43.0

reviewer
#22 lore raw

Re: [RFC PATCH v8 5/7] perf stat: Add retire latency print functions to print out at the very end of print out

Quoted 54 lines · Weilin
Quoted 41 lines

-----Original Message-----
From: Ian Rogers <[email protected]>
Sent: Thursday, May 16, 2024 9:47 AM
To: Wang, Weilin <[email protected]>
Cc: Namhyung Kim <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 5/7] perf stat: Add retire latency print functions to print out at the very end of print out

Quoted 10 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

Add print out functions so that users could read retire latency values.

Example output:

Performance counter stats for 'system wide':

            25,717      MEM_INST_RETIRED.SPLIT_STORES    #      2.2 %

tma_split_stores

Quoted 10 lines
        28,365,080      CPU_CLK_UNHALTED.THREAD
             24.00      MEM_INST_RETIRED.SPLIT_STORES:R  #       96        4

       2.054365083 seconds time elapsed

This output of retire latency data is in format:
<val> <event-name:R> # <sum> <count>.

Quoted 9 lines

My usual complaint that I hate the stat-display spaghetti code. We keep putting more spaghetti on the plate and this change does this too. In the evsel approach:
https://lore.kernel.org/lkml/20240428053616.1125891-1-
[email protected]/
retirement latency events just update the counts for the event and so we don't need to special case tpebs events like this. I'd prefer we went that route. My reviewed-by no longer stands.

Based on the current stat-display code and the original TPEBS counting code, I believe the code in this commit and last commit was the best way to do the metric calculation and printout.

Quoted 5 lines

But I totally agree with you that if we could get evsel with retire latency work, this code is not necessary.

I was thinking to plugin the retire latency value into evsel so that I could delete this and last commit. Do you think that would be a solution?

Yep, I think that'd be good.

Thanks,
Ian

Quoted 118 lines

Thanks,
Weilin

Quoted 115 lines

Thanks,
Ian

Quoted 2 lines
---
 tools/perf/util/stat-display.c | 69

++++++++++++++++++++++++++++++++++

tools/perf/util/stat-display.c static void print_metric_std(struct

Quoted 23 lines
 1 file changed, 69 insertions(+)
tools/perf/util/stat-display.c +2 -0
index bfc1d705f437..b9c3978cc99c 100644
@@ -21,6 +21,7 @@
 #include "iostat.h"
 #include "pmu.h"
 #include "pmus.h"
+#include "intel-tpebs.h"
 
 #define CNTR_NOT_SUPPORTED     "<not supported>"
 #define CNTR_NOT_COUNTED       "<not counted>"
@@ -34,6 +35,7 @@
 #define COMM_LEN     16
 #define PID_LEN       7
 #define CPUS_LEN      4
+#define RETIRE_LEN    8
 
 static int aggr_header_lens[] = {
        [AGGR_CORE]     = 18,
@@ -426,6 +428,71 @@ static void print_metric_std(struct

perf_stat_config *config,

tools/perf/util/stat-display.c void evlist__print_counters(struct evlist *evlist,

Quoted 73 lines
tools/perf/util/stat-display.c +65 -0
        fprintf(out, " %-*s", METRIC_LEN - n - 1, unit);
 }
 
+static void print_retire_lat_std(struct perf_stat_config *config,
+                                struct outstate *os)
+{
+       FILE *out = os->fh;
+       bool newline = os->newline;
+       struct tpebs_retire_lat *t;
+       struct list_head *retire_lats = &config->tpebs_results;
+
+       list_for_each_entry(t, retire_lats, nd) {
+               if (newline)
+                       do_new_line_std(config, os);
+               fprintf(out, "%'*.2f ", COUNTS_LEN, t->val);
+               /* For print alignment */
+               fprintf(out, "%-*s ", config->unit_width, "");
+               fprintf(out, "%-*s", EVNAME_LEN, t->tpebs_name);
+               fprintf(out, " # ");
+               fprintf(out, "%*d %*ld\n", RETIRE_LEN, t->sum,
+                        RETIRE_LEN, t->count);
+       }
+}
+
+static void print_retire_lat_csv(struct perf_stat_config *config,
+                                struct outstate *os)
+{
+       FILE *out = os->fh;
+       struct tpebs_retire_lat *t;
+       struct list_head *retire_lats = &config->tpebs_results;
+       const char *sep = config->csv_sep;
+
+       list_for_each_entry(t, retire_lats, nd) {
+               fprintf(out, "%f%s%s%s%s%ld%s%d\n", t->val, sep, sep, t-
 tpebs_name, sep,
+                       t->count, sep, t->sum);
+       }
+}
+
+static void print_retire_lat_json(struct perf_stat_config *config,
+                                 struct outstate *os)
+{
+       FILE *out = os->fh;
+       struct tpebs_retire_lat *t;
+       struct list_head *retire_lats = &config->tpebs_results;
+
+       fprintf(out, "{");
+       list_for_each_entry(t, retire_lats, nd) {
+               fprintf(out, "\"retire_latency-value\" : \"%f\", ", t->val);
+               fprintf(out, "\"name\" : \"%s\"", t->tpebs_name);
+               fprintf(out, "\"sample-counts\" : \"%ld\"", t->count);
+               fprintf(out, "\"retire_latency-sum\" : \"%d\"", t->sum);
+       }
+       fprintf(out, "}");
+}
+
+static void print_retire_lat(struct perf_stat_config *config,
+                            struct outstate *os)
+{
+       if (!&config->tpebs_results)
+               return;
+       if (config->json_output)
+               print_retire_lat_json(config, os);
+       else if (config->csv_output)
+               print_retire_lat_csv(config, os);
+       else
+               print_retire_lat_std(config, os);
+}
+
 static void new_line_csv(struct perf_stat_config *config, void *ctx)
 {
        struct outstate *os = ctx;
tools/perf/util/stat-display.c +0 -0
@@ -1609,6 +1676,8 @@ void evlist__print_counters(struct evlist *evlist,

struct perf_stat_config *conf

Quoted 11 lines
tools/perf/util/stat-display.c +2 -1
                break;
        }
 
+       print_retire_lat(config, &os);
+
        print_footer(config);
 
        fflush(config->output);
--

2.43.0

#23 lore raw

RFC PATCH v8 6/7 perf vendor events intel: Add MTL metric json files

In reply to [email protected]

From: Weilin Wang <[email protected]>

Add MTL metric json file at TMA4.7 [1]. Some of the metrics' formulas use TPEBS retire_latency in MTL.

[1] https://lore.kernel.org/all/[email protected]/

---
 .../arch/x86/meteorlake/metricgroups.json     |  127 +
 .../arch/x86/meteorlake/mtl-metrics.json      | 2551 +++++++++++++++++
 2 files changed, 2678 insertions(+)
 create mode 100644 tools/perf/pmu-events/arch/x86/meteorlake/metricgroups.json
 create mode 100644 tools/perf/pmu-events/arch/x86/meteorlake/mtl-metrics.json
tools/perf/pmu-events/arch/x86/meteorlake/metricgroups.json new file +127 -0
new file mode 100644
index 000000000000..7a03835f262c
@@ -0,0 +1,127 @@
+{
+    "Backend": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Bad": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "BadSpec": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "BigFootprint": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "BrMispredicts": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Branches": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "C0Wait": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "CacheHits": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "CodeGen": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Compute": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Cor": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "DSB": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "DSBmiss": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "DataSharing": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Fed": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "FetchBW": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "FetchLat": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Flops": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "FpScalar": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "FpVector": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Frontend": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "HPC": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "IcMiss": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "InsType": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "IntVector": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "L2Evicts": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "LSD": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "MachineClears": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Machine_Clears": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Mem": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "MemOffcore": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "MemoryBW": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "MemoryBound": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "MemoryLat": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "MemoryTLB": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Memory_BW": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Memory_Lat": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "MicroSeq": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "OS": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Offcore": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "PGO": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Pipeline": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "PortsUtil": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Power": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Prefetches": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Ret": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Retire": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "SMT": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Server": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Snoop": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "SoC": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "Summary": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "TmaL1": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "TmaL2": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "TmaL3mem": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet",
+    "TopdownL1": "Metrics for top-down breakdown at level 1",
+    "TopdownL2": "Metrics for top-down breakdown at level 2",
+    "TopdownL3": "Metrics for top-down breakdown at level 3",
+    "TopdownL4": "Metrics for top-down breakdown at level 4",
+    "TopdownL5": "Metrics for top-down breakdown at level 5",
+    "TopdownL6": "Metrics for top-down breakdown at level 6",
+    "tma_L1_group": "Metrics for top-down breakdown at level 1",
+    "tma_L2_group": "Metrics for top-down breakdown at level 2",
+    "tma_L3_group": "Metrics for top-down breakdown at level 3",
+    "tma_L4_group": "Metrics for top-down breakdown at level 4",
+    "tma_L5_group": "Metrics for top-down breakdown at level 5",
+    "tma_L6_group": "Metrics for top-down breakdown at level 6",
+    "tma_alu_op_utilization_group": "Metrics contributing to tma_alu_op_utilization category",
+    "tma_assists_group": "Metrics contributing to tma_assists category",
+    "tma_backend_bound_aux_group": "Metrics contributing to tma_backend_bound_aux category",
+    "tma_backend_bound_group": "Metrics contributing to tma_backend_bound category",
+    "tma_bad_speculation_group": "Metrics contributing to tma_bad_speculation category",
+    "tma_base_group": "Metrics contributing to tma_base category",
+    "tma_branch_mispredicts_group": "Metrics contributing to tma_branch_mispredicts category",
+    "tma_branch_resteers_group": "Metrics contributing to tma_branch_resteers category",
+    "tma_core_bound_group": "Metrics contributing to tma_core_bound category",
+    "tma_dram_bound_group": "Metrics contributing to tma_dram_bound category",
+    "tma_dtlb_load_group": "Metrics contributing to tma_dtlb_load category",
+    "tma_dtlb_store_group": "Metrics contributing to tma_dtlb_store category",
+    "tma_fetch_bandwidth_group": "Metrics contributing to tma_fetch_bandwidth category",
+    "tma_fetch_latency_group": "Metrics contributing to tma_fetch_latency category",
+    "tma_fp_arith_group": "Metrics contributing to tma_fp_arith category",
+    "tma_fp_vector_group": "Metrics contributing to tma_fp_vector category",
+    "tma_frontend_bound_group": "Metrics contributing to tma_frontend_bound category",
+    "tma_heavy_operations_group": "Metrics contributing to tma_heavy_operations category",
+    "tma_int_operations_group": "Metrics contributing to tma_int_operations category",
+    "tma_issue2P": "Metrics related by the issue $issue2P",
+    "tma_issueBM": "Metrics related by the issue $issueBM",
+    "tma_issueBW": "Metrics related by the issue $issueBW",
+    "tma_issueComp": "Metrics related by the issue $issueComp",
+    "tma_issueD0": "Metrics related by the issue $issueD0",
+    "tma_issueFB": "Metrics related by the issue $issueFB",
+    "tma_issueFL": "Metrics related by the issue $issueFL",
+    "tma_issueL1": "Metrics related by the issue $issueL1",
+    "tma_issueLat": "Metrics related by the issue $issueLat",
+    "tma_issueMC": "Metrics related by the issue $issueMC",
+    "tma_issueMS": "Metrics related by the issue $issueMS",
+    "tma_issueMV": "Metrics related by the issue $issueMV",
+    "tma_issueRFO": "Metrics related by the issue $issueRFO",
+    "tma_issueSL": "Metrics related by the issue $issueSL",
+    "tma_issueSO": "Metrics related by the issue $issueSO",
+    "tma_issueSmSt": "Metrics related by the issue $issueSmSt",
+    "tma_issueSpSt": "Metrics related by the issue $issueSpSt",
+    "tma_issueSyncxn": "Metrics related by the issue $issueSyncxn",
+    "tma_issueTLB": "Metrics related by the issue $issueTLB",
+    "tma_l1_bound_group": "Metrics contributing to tma_l1_bound category",
+    "tma_l3_bound_group": "Metrics contributing to tma_l3_bound category",
+    "tma_light_operations_group": "Metrics contributing to tma_light_operations category",
+    "tma_load_op_utilization_group": "Metrics contributing to tma_load_op_utilization category",
+    "tma_machine_clears_group": "Metrics contributing to tma_machine_clears category",
+    "tma_mem_latency_group": "Metrics contributing to tma_mem_latency category",
+    "tma_mem_scheduler_group": "Metrics contributing to tma_mem_scheduler category",
+    "tma_memory_bound_group": "Metrics contributing to tma_memory_bound category",
+    "tma_microcode_sequencer_group": "Metrics contributing to tma_microcode_sequencer category",
+    "tma_mite_group": "Metrics contributing to tma_mite category",
+    "tma_nuke_group": "Metrics contributing to tma_nuke category",
+    "tma_other_light_ops_group": "Metrics contributing to tma_other_light_ops category",
+    "tma_ports_utilization_group": "Metrics contributing to tma_ports_utilization category",
+    "tma_ports_utilized_0_group": "Metrics contributing to tma_ports_utilized_0 category",
+    "tma_ports_utilized_3m_group": "Metrics contributing to tma_ports_utilized_3m category",
+    "tma_resource_bound_group": "Metrics contributing to tma_resource_bound category",
+    "tma_retiring_group": "Metrics contributing to tma_retiring category",
+    "tma_serializing_operation_group": "Metrics contributing to tma_serializing_operation category",
+    "tma_store_bound_group": "Metrics contributing to tma_store_bound category",
+    "tma_store_op_utilization_group": "Metrics contributing to tma_store_op_utilization category"
+}
tools/perf/pmu-events/arch/x86/meteorlake/mtl-metrics.json new file +2551 -0

2,551 lines not on this page · raw message

Signature · 1 line
2.43.0
reviewer
#24 lore raw

Re: [RFC PATCH v8 6/7] perf vendor events intel: Add MTL metric json files

Quoted 10 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

Add MTL metric json file at TMA4.7 [1]. Some of the metrics' formulas use TPEBS retire_latency in MTL.

[1] https://lore.kernel.org/all/[email protected]/

This change works either with the approach in this series or with the evsel approach so I don't mind my reviewed-by standing. I'd prefer we could have an evsel read counter implementation that returns 0 so that we can run without retirement latency gathering.

TMA 4.7 is broken in that the tma_lock_latency metric uses a retirement latency event but not within a max function so having the read counter return 0 would break the metric:

+ {
+ "BriefDescription": "This metric represents fraction of
cycles the CPU spent handling cache misses due to lock operations",
+ "MetricExpr": "MEM_INST_RETIRED.LOCK_LOADS *
MEM_INST_RETIRED.LOCK_LOADS:R / tma_info_thread_clks",
+ "MetricGroup":
"Offcore;TopdownL4;tma_L4_group;tma_issueRFO;tma_l1_bound_group",
+ "MetricName": "tma_lock_latency",
+ "MetricThreshold": "tma_lock_latency > 0.2 & (tma_l1_bound >
0.1 & (tma_memory_bound > 0.2 & tma_backend_bound > 0.2))",
+ "PublicDescription": "This metric represents fraction of
cycles the CPU spent handling cache misses due to lock operations. Due to the microarchitecture handling of locks; they are classified as L1_Bound regardless of what memory source satisfied them. Sample with:
MEM_INST_RETIRED.LOCK_LOADS_PS. Related metrics: tma_store_latency",
+ "ScaleUnit": "100%",
+ "Unit": "cpu_core"
+ },

Other metrics then use that metric specifically
tma_info_bottleneck_memory_data_tlbs and
tma_info_bottleneck_cache_memory_bandwidth.

I couldn't see in the TMA 4.8 release the updated MTL metrics:
https://github.com/intel/perfmon/pull/181/commits/d54c847b2f863c98a917bdd31a0680f4d50ff75c
but my belief is that this issue hasn't been addressed.

Thanks,
Ian

#25 lore raw

RE: [RFC PATCH v8 6/7] perf vendor events intel: Add MTL metric json files

Quoted 59 lines

-----Original Message-----
From: Ian Rogers <[email protected]>
Sent: Thursday, May 16, 2024 9:57 AM
To: Wang, Weilin <[email protected]>
Cc: Namhyung Kim <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v8 6/7] perf vendor events intel: Add MTL metric json files

Quoted 4 lines · On Tue, May 14, 2024 at 10:44 PM <[email protected]> wrote:

From: Weilin Wang <[email protected]>

Add MTL metric json file at TMA4.7 [1]. Some of the metrics' formulas use

TPEBS

Quoted 3 lines

retire_latency in MTL.

[1] https://lore.kernel.org/all/20240214011820.644458-1-

[email protected]/

Quoted 3 lines

This change works either with the approach in this series or with the evsel approach so I don't mind my reviewed-by standing. I'd prefer we could have an evsel read counter implementation that returns 0 so that we can run without retirement latency gathering.

TMA 4.7 is broken in that the tma_lock_latency metric uses a retirement latency event but not within a max function so having the read counter return 0 would break the metric:

(quoted diff) +2 -0
+    {
+        "BriefDescription": "This metric represents fraction of

cycles the CPU spent handling cache misses due to lock operations",
+ "MetricExpr": "MEM_INST_RETIRED.LOCK_LOADS *
MEM_INST_RETIRED.LOCK_LOADS:R / tma_info_thread_clks",
+ "MetricGroup":
"Offcore;TopdownL4;tma_L4_group;tma_issueRFO;tma_l1_bound_group",

(quoted diff) +2 -0
+        "MetricName": "tma_lock_latency",
+        "MetricThreshold": "tma_lock_latency > 0.2 & (tma_l1_bound >

0.1 & (tma_memory_bound > 0.2 & tma_backend_bound > 0.2))",
+ "PublicDescription": "This metric represents fraction of
cycles the CPU spent handling cache misses due to lock operations. Due to the microarchitecture handling of locks; they are classified as L1_Bound regardless of what memory source satisfied them. Sample with:
MEM_INST_RETIRED.LOCK_LOADS_PS. Related metrics: tma_store_latency",

tools/perf/pmu-events/arch/x86/meteorlake/mtl-metrics.json new file +3 -0
+        "ScaleUnit": "100%",
+        "Unit": "cpu_core"
+    },
Quoted 8 lines

Other metrics then use that metric specifically
tma_info_bottleneck_memory_data_tlbs and
tma_info_bottleneck_cache_memory_bandwidth.

I couldn't see in the TMA 4.8 release the updated MTL metrics:
https://github.com/intel/perfmon/pull/181/commits/d54c847b2f863c98a9
17bdd31a0680f4d50ff75c
but my belief is that this issue hasn't been addressed.

I did not include TMA4.8 here because our release of TMA 4.8 is not finalized by the time I sent this patch set. I will add TMA 4.8 and latest E-Core TMA next time.

Thanks,
Weilin

Quoted 3 lines

Thanks,
Ian

#26 lore raw

RFC PATCH v8 7/7 perf stat: Skip read retire_lat counters and plugin retire_lat data from sampled data

In reply to [email protected]

From: Weilin Wang <[email protected]>

In current :R parsing implementation, the parser would recognize events with retire_latency modifier and insert them into the evlist like a normal event. Ideally, we need to avoid counting these events.

In this commit, at the stage of metric calculation, the code uses sampled retire latency value instead of counted event value.

---
 tools/perf/arch/x86/util/evlist.c | 6 ++++++
 tools/perf/util/evsel.h           | 5 +++++
 tools/perf/util/stat-display.c    | 5 +++++
 tools/perf/util/stat-shadow.c     | 2 +-
 4 files changed, 17 insertions(+), 1 deletion(-)
tools/perf/arch/x86/util/evlist.c +6 -0
index b1ce0c52d88d..cebdd483149e 100644
@@ -89,6 +89,12 @@ int arch_evlist__cmp(const struct evsel *lhs, const struct evsel *rhs)
 			return 1;
 	}
 
+	/* Retire latency event should not be group leader*/
+	if (lhs->retire_lat && !rhs->retire_lat)
+		return 1;
+	if (!lhs->retire_lat && rhs->retire_lat)
+		return -1;
+
 	/* Default ordering by insertion index. */
 	return lhs->core.idx - rhs->core.idx;
 }
tools/perf/util/evsel.h +5 -0
index bd8e84954e34..aaf572317e92 100644
@@ -303,6 +303,11 @@ static inline bool evsel__is_tool(const struct evsel *evsel)
 	return evsel->tool_event != PERF_TOOL_NONE;
 }
 
+static inline bool evsel__is_retire_lat(const struct evsel *evsel)
+{
+	return evsel->retire_lat;
+}
+
 const char *evsel__group_name(struct evsel *evsel);
 int evsel__group_desc(struct evsel *evsel, char *buf, size_t size);
 
tools/perf/util/stat-display.c +5 -0
index b9c3978cc99c..499fe0ddca93 100644
@@ -1013,6 +1013,9 @@ static bool should_skip_zero_counter(struct perf_stat_config *config,
 	/* Tool events have the software PMU but are only gathered on 1. */
 	if (evsel__is_tool(counter))
 		return true;
+	/* Retire latency events are read from sampling data. */
+	if (evsel__is_retire_lat(counter))
+		return true;
 
 	/*
 	 * Skip value 0 when it's an uncore event and the given aggr id
@@ -1653,6 +1656,8 @@ void evlist__print_counters(struct evlist *evlist, struct perf_stat_config *conf
 		} else {
 			print_metric_begin(config, evlist, &os, /*aggr_idx=*/0);
 			evlist__for_each_entry(evlist, counter) {
+				if (evsel__is_retire_lat(counter))
+					continue;
 				print_counter(config, counter, &os);
 			}
 			print_metric_end(config, &os);
tools/perf/util/stat-shadow.c +1 -1
index b97d23bfeac1..45e0b21a4724 100644
@@ -384,7 +384,7 @@ static int prepare_metric(const struct metric_expr *mexp,
 		double val;
 		int source_count = 0;
 
-		if (metric_events[i]->retire_lat)
+		if (evsel__is_retire_lat(metric_events[i]))
 			continue;
 
 		if (evsel__is_tool(metric_events[i])) {
Signature · 1 line
2.43.0