TPEBS counting mode support
v9 cover TPEBS counting mode support
From: Weilin Wang <[email protected]>
Changes in v9:
- Update the retire_latency result print and metric calculation method. Plugin the value to evsel so that no special code is required.
- Update --control:fifo to use pipe instead of named pipe.
- Add test for TPEBS counting mode.
- Update Document with more details.
Changes in v8:
- In this revision, the code is updated to base on Ian's patch on R modifier parser https://lore.kernel.org/lkml/[email protected]/
After this change, there is no special code required for R modifier in metricgroup.c and metricgroup.h files.
Caveat of this change:
Ideally, we will need to add special handling to skip counting events with R
modifier in evsel. Currently, this is not implemented so the event with :R will be both counted and sampled. Usually, in a metric formula that uses retire_latency, it would already require to count the event. As a result, we will endup count the same event twice. This should be able to be handled properly when we finalize our design on evsel R modifier support.
- Move TPEBS specific code out from main perf stat code to separate files in util/intel-tpebs.c and util/intel-tpebs.h. [Namhyung]
- Use --control:fifo to ack perf stat from forked perf record instead of sleep(2) [Namhyung]
- Add introductions about TPEBS and R modifier in Documents. [Namhyung]
Changes in v7:
- Update code and comments for better code quality [Namhyung]
- Add a separate commit for perf data [Namhyung]
- Update retire latency print function to improve alignment [Namhyung]
Changes in v6:
- Update code and add comments for better code quality [Namhyung]
- Remove the added fd var and directly pass the opened fd to data.file.fd [Namhyung]
- Add kill() to stop perf record when perf stat exists early [Namhyung]
- Add command opt check to ensure only start perf record when -a/-C given [Namhyung]
- Squash commits [Namhyung]
Changes in v5:
- Update code and add comments for better code quality [Ian]
Changes in v4:
- Remove uncessary debug print and update code and comments for better readability and quality [Namhyung]
- Update mtl metric json file with consistent TmaL1 and TopdownL1 metricgroup
Changes in v3:
- Remove ':' when event name has '@' [Ian]
- Use 'R' as the modifier instead of "retire_latency" [Ian]
Changes in v2:
- Add MTL metric file
- Add more descriptions and example to the patch [Arnaldo]
Here is an example of running perf stat to collect a metric that uses retire_latency value of event MEM_INST_RETIRED.STLB_HIT_STORES on a MTL system.
In this simple example, there is no MEM_INST_RETIRED.STLB_HIT_STORES sample. Therefore, the MEM_INST_RETIRED.STLB_HIT_STORES:p count and retire_latency value are all 0.
./perf stat -M tma_dtlb_store -a -- sleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.000 MB - ]
Performance counter stats for 'system wide':
181,047,168 cpu_core/TOPDOWN.SLOTS/ # 0.6 % tma_dtlb_store
3,195,608 cpu_core/topdown-retiring/
40,156,649 cpu_core/topdown-mem-bound/
3,550,925 cpu_core/topdown-bad-spec/
117,571,818 cpu_core/topdown-fe-bound/
57,118,087 cpu_core/topdown-be-bound/
69,179 cpu_core/EXE_ACTIVITY.BOUND_ON_STORES/
4,582 cpu_core/MEM_INST_RETIRED.STLB_HIT_STORES/
30,183,104 cpu_core/CPU_CLK_UNHALTED.DISTRIBUTED/
30,556,790 cpu_core/CPU_CLK_UNHALTED.THREAD/
168,486 cpu_core/DTLB_STORE_MISSES.WALK_ACTIVE/0.00 MEM_INST_RETIRED.STLB_HIT_STORES:p 0 0
1.003105924 seconds time elapsed
v1:
TPEBS is one of the features provided by the next generation of Intel PMU. Please refer to Section 8.4.1 of "Intel® Architecture Instruction Set Extensions Programming Reference" [1] for more details about this feature.
This set of patches supports TPEBS in counting mode. The code works in the following way: it forks a perf record process from perf stat when retire_latency of one or more events are used in a metric formula. Perf stat would send a SIGTERM signal to perf record before it needs the retire latency value for metric calculation. Perf stat will then process sample data to extract the retire latency data for metric calculations. Currently, the code uses the arithmetic average of retire latency values.
Weilin Wang (7):
perf vendor events intel: Add MTL metric json files perf data: Allow to use given fd in data->file.fd
perf stat: Fork and launch perf record when perf stat needs to get
retire latency value for a metric.
perf stat: Plugin retire_lat value from sampled data to evsel perf stat: Add command line option for enabling tpebs recording perf Document: Add TPEBS to Documents
perf test: Add test for Intel TPEBS counting mode
tools/perf/Documentation/perf-list.txt | 1 + tools/perf/Documentation/topdown.txt | 30 + tools/perf/arch/x86/util/evlist.c | 6 + tools/perf/builtin-stat.c | 24 + .../arch/x86/meteorlake/metricgroups.json | 127 + .../arch/x86/meteorlake/mtl-metrics.json | 2551 +++++++++++++++++ .../perf/tests/shell/test_stat_intel_tpebs.sh | 27 + tools/perf/util/Build | 1 + tools/perf/util/data.c | 7 +- tools/perf/util/evsel.c | 53 + tools/perf/util/evsel.h | 5 + tools/perf/util/intel-tpebs.c | 301 ++ tools/perf/util/intel-tpebs.h | 30 + tools/perf/util/stat.h | 1 + 14 files changed, 3163 insertions(+), 1 deletion(-) create mode 100644 tools/perf/pmu-events/arch/x86/meteorlake/metricgroups.json create mode 100644 tools/perf/pmu-events/arch/x86/meteorlake/mtl-metrics.json create mode 100755 tools/perf/tests/shell/test_stat_intel_tpebs.sh create mode 100644 tools/perf/util/intel-tpebs.c create mode 100644 tools/perf/util/intel-tpebs.h
--
2.43.0
RFC PATCH v9 1/7 perf vendor events intel: Add MTL metric json files
From: Weilin Wang <[email protected]>
Add MTL metric json file at TMA4.7 [1]. Some of the metrics' formulas use TPEBS retire_latency in MTL.
[1] https://lore.kernel.org/all/[email protected]/
--- .../arch/x86/meteorlake/metricgroups.json | 127 + .../arch/x86/meteorlake/mtl-metrics.json | 2551 +++++++++++++++++ 2 files changed, 2678 insertions(+) create mode 100644 tools/perf/pmu-events/arch/x86/meteorlake/metricgroups.json create mode 100644 tools/perf/pmu-events/arch/x86/meteorlake/mtl-metrics.json
tools/perf/pmu-events/arch/x86/meteorlake/metricgroups.json new file +127 -0
@@ -0,0 +1,127 @@ +{ + "Backend": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Bad": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "BadSpec": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "BigFootprint": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "BrMispredicts": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Branches": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "C0Wait": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "CacheHits": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "CodeGen": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Compute": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Cor": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "DSB": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "DSBmiss": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "DataSharing": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Fed": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "FetchBW": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "FetchLat": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Flops": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "FpScalar": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "FpVector": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Frontend": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "HPC": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "IcMiss": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "InsType": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "IntVector": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "L2Evicts": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "LSD": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "MachineClears": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Machine_Clears": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Mem": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "MemOffcore": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "MemoryBW": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "MemoryBound": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "MemoryLat": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "MemoryTLB": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Memory_BW": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Memory_Lat": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "MicroSeq": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "OS": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Offcore": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "PGO": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Pipeline": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "PortsUtil": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Power": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Prefetches": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Ret": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Retire": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "SMT": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Server": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Snoop": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "SoC": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "Summary": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "TmaL1": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "TmaL2": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "TmaL3mem": "Grouping from Top-down Microarchitecture Analysis Metrics spreadsheet", + "TopdownL1": "Metrics for top-down breakdown at level 1", + "TopdownL2": "Metrics for top-down breakdown at level 2", + "TopdownL3": "Metrics for top-down breakdown at level 3", + "TopdownL4": "Metrics for top-down breakdown at level 4", + "TopdownL5": "Metrics for top-down breakdown at level 5", + "TopdownL6": "Metrics for top-down breakdown at level 6", + "tma_L1_group": "Metrics for top-down breakdown at level 1", + "tma_L2_group": "Metrics for top-down breakdown at level 2", + "tma_L3_group": "Metrics for top-down breakdown at level 3", + "tma_L4_group": "Metrics for top-down breakdown at level 4", + "tma_L5_group": "Metrics for top-down breakdown at level 5", + "tma_L6_group": "Metrics for top-down breakdown at level 6", + "tma_alu_op_utilization_group": "Metrics contributing to tma_alu_op_utilization category", + "tma_assists_group": "Metrics contributing to tma_assists category", + "tma_backend_bound_aux_group": "Metrics contributing to tma_backend_bound_aux category", + "tma_backend_bound_group": "Metrics contributing to tma_backend_bound category", + "tma_bad_speculation_group": "Metrics contributing to tma_bad_speculation category", + "tma_base_group": "Metrics contributing to tma_base category", + "tma_branch_mispredicts_group": "Metrics contributing to tma_branch_mispredicts category", + "tma_branch_resteers_group": "Metrics contributing to tma_branch_resteers category", + "tma_core_bound_group": "Metrics contributing to tma_core_bound category", + "tma_dram_bound_group": "Metrics contributing to tma_dram_bound category", + "tma_dtlb_load_group": "Metrics contributing to tma_dtlb_load category", + "tma_dtlb_store_group": "Metrics contributing to tma_dtlb_store category", + "tma_fetch_bandwidth_group": "Metrics contributing to tma_fetch_bandwidth category", + "tma_fetch_latency_group": "Metrics contributing to tma_fetch_latency category", + "tma_fp_arith_group": "Metrics contributing to tma_fp_arith category", + "tma_fp_vector_group": "Metrics contributing to tma_fp_vector category", + "tma_frontend_bound_group": "Metrics contributing to tma_frontend_bound category", + "tma_heavy_operations_group": "Metrics contributing to tma_heavy_operations category", + "tma_int_operations_group": "Metrics contributing to tma_int_operations category", + "tma_issue2P": "Metrics related by the issue $issue2P", + "tma_issueBM": "Metrics related by the issue $issueBM", + "tma_issueBW": "Metrics related by the issue $issueBW", + "tma_issueComp": "Metrics related by the issue $issueComp", + "tma_issueD0": "Metrics related by the issue $issueD0", + "tma_issueFB": "Metrics related by the issue $issueFB", + "tma_issueFL": "Metrics related by the issue $issueFL", + "tma_issueL1": "Metrics related by the issue $issueL1", + "tma_issueLat": "Metrics related by the issue $issueLat", + "tma_issueMC": "Metrics related by the issue $issueMC", + "tma_issueMS": "Metrics related by the issue $issueMS", + "tma_issueMV": "Metrics related by the issue $issueMV", + "tma_issueRFO": "Metrics related by the issue $issueRFO", + "tma_issueSL": "Metrics related by the issue $issueSL", + "tma_issueSO": "Metrics related by the issue $issueSO", + "tma_issueSmSt": "Metrics related by the issue $issueSmSt", + "tma_issueSpSt": "Metrics related by the issue $issueSpSt", + "tma_issueSyncxn": "Metrics related by the issue $issueSyncxn", + "tma_issueTLB": "Metrics related by the issue $issueTLB", + "tma_l1_bound_group": "Metrics contributing to tma_l1_bound category", + "tma_l3_bound_group": "Metrics contributing to tma_l3_bound category", + "tma_light_operations_group": "Metrics contributing to tma_light_operations category", + "tma_load_op_utilization_group": "Metrics contributing to tma_load_op_utilization category", + "tma_machine_clears_group": "Metrics contributing to tma_machine_clears category", + "tma_mem_latency_group": "Metrics contributing to tma_mem_latency category", + "tma_mem_scheduler_group": "Metrics contributing to tma_mem_scheduler category", + "tma_memory_bound_group": "Metrics contributing to tma_memory_bound category", + "tma_microcode_sequencer_group": "Metrics contributing to tma_microcode_sequencer category", + "tma_mite_group": "Metrics contributing to tma_mite category", + "tma_nuke_group": "Metrics contributing to tma_nuke category", + "tma_other_light_ops_group": "Metrics contributing to tma_other_light_ops category", + "tma_ports_utilization_group": "Metrics contributing to tma_ports_utilization category", + "tma_ports_utilized_0_group": "Metrics contributing to tma_ports_utilized_0 category", + "tma_ports_utilized_3m_group": "Metrics contributing to tma_ports_utilized_3m category", + "tma_resource_bound_group": "Metrics contributing to tma_resource_bound category", + "tma_retiring_group": "Metrics contributing to tma_retiring category", + "tma_serializing_operation_group": "Metrics contributing to tma_serializing_operation category", + "tma_store_bound_group": "Metrics contributing to tma_store_bound category", + "tma_store_op_utilization_group": "Metrics contributing to tma_store_op_utilization category" +}
tools/perf/pmu-events/arch/x86/meteorlake/mtl-metrics.json new file +2551 -0
2,551 lines not on this page · raw message
Signature · 1 line
2.43.0
RFC PATCH v9 2/7 perf data: Allow to use given fd in data->file.fd
In reply to [email protected]From: Weilin Wang <[email protected]>
When in PIPE mode, allow to use fd dynamically opened and asigned to data->file.fd instead of STDIN_FILENO or STDOUT_FILENO.
--- tools/perf/util/data.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
tools/perf/util/data.c +6 -1
@@ -204,7 +204,12 @@ static bool check_pipe(struct perf_data *data) data->file.fd = fd; data->use_stdio = false; } - }else { + +/* +* When is_pipe and data->file.fd is given, use given fd +* instead of STDIN_FILENO or STDOUT_FILENO +*/ + }else if (data->file.fd <=0 ) { data->file.fd = fd; } }
--
2.43.0
RFC PATCH v9 3/7 perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.
In reply to [email protected]From: Weilin Wang <[email protected]>
When retire_latency value is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect required retire_latency values in parallel while perf stat is collecting counting values.
At the point of time that perf stat stops counting, it would send sigterm signal to perf record process and receiving sampling data back from perf record from a pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.
Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.
--- tools/perf/builtin-stat.c | 17 ++ tools/perf/util/Build | 1 + tools/perf/util/intel-tpebs.c | 301 ++++++++++++++++++++++++++++++++++ tools/perf/util/intel-tpebs.h | 30 ++++ tools/perf/util/stat.h | 1 + 5 files changed, 350 insertions(+) create mode 100644 tools/perf/util/intel-tpebs.c create mode 100644 tools/perf/util/intel-tpebs.h
tools/perf/builtin-stat.c +17 -0
@@ -70,6 +70,7 @@#include "util/bpf_counter.h" #include "util/iostat.h" #include "util/util.h" +#include "util/intel-tpebs.h" #include "asm/bug.h" #include <linux/time64.h>
@@ -162,6 +163,7 @@ static struct perf_stat_config stat_config = { .ctl_fd = -1, .ctl_fd_ack = -1, .iostat_run = false, + .tpebs_pid = -1, };static void evlist__check_cpu_maps (struct evlist *evlist)
@@ -653,6 +655,8 @@ static enum counter_recovery stat_handle_error(struct evsel *counter)if (child_pid !=-1 )kill (child_pid,SIGTERM ); +if (stat_config.tpebs_pid !=-1 ) +stop_tpebs ();return COUNTER_FATAL ; }
@@ -673,6 +677,10 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx)int err;bool second_pass = false; + err =start_tpebs (&stat_config, evsel_list); +if (err <0 ) +return err; +if (forks) {if (evlist__prepare_workload (evsel_list, &target, argv, is_pipe, workload_exec_failed_signal) <0 ) {perror ("failed to prepare workload" );
@@ -878,6 +886,10 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx) t1 =rdclock (); + err =stop_tpebs (); +if (err <0 ) +return err; +if (stat_config.walltime_run_table) stat_config.walltime_run[run_idx] = t1 - t0;
@@ -985,6 +997,9 @@ static void sig_atexit(void)if (child_pid !=-1 )kill (child_pid,SIGTERM ); +if (stat_config.tpebs_pid !=-1 ) +kill (stat_config.tpebs_pid,SIGTERM ); +sigprocmask (SIG_SETMASK , &oset,NULL );if (signr ==-1 )
@@ -2918,5 +2933,7 @@ int cmd_stat(int argc, const char **argv)metricgroup__rblist_exit (&stat_config.metric_events);evlist__close_control (stat_config.ctl_fd, stat_config.ctl_fd_ack, &stat_config.ctl_fd_close); +tpebs_data__delete (); +return status; }
tools/perf/util/Build +1 -0
@@ -153,6 +153,7 @@ perf-y += clockid.o perf-y += list_sort.o perf-y += mutex.o perf-y += sharded_mutex.o +perf-y += intel-tpebs.o perf-$(CONFIG_LIBBPF) += bpf_map.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.o
tools/perf/util/intel-tpebs.c new file +301 -0
301 lines not on this page · raw message
tools/perf/util/intel-tpebs.h new file +30 -0
@@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* intel_tpebs.h: Intel TEPBS support +*/ +#include "stat.h" + +#ifndef INCLUDE__PERF_INTEL_TPEBS_H__ +#define INCLUDE__PERF_INTEL_TPEBS_H__ + +struct tpebs_retire_lat { +struct list_head nd; +/* Event name */ +const char *name; +/* Event name with the TPEBS modifier R */ +const char *tpebs_name; +/* Count of retire_latency values found in sample data */ +size_t count; +/* Sum of all the retire_latency values in sample data */ +int sum; +/* Average of retire_latency, val = sum / count */ +double val; +}; + +extern struct list_head tpebs_results; + +int start_tpebs (struct perf_stat_config *perf_stat_config,struct evlist *evsel_list); +int stop_tpebs (void ); +void tpebs_data__delete (void ); + +#endif
tools/perf/util/stat.h +1 -0
@@ -110,6 +110,7 @@ struct perf_stat_config {struct cpu_aggr_map *cpus_aggr_map;u64 *walltime_run;struct rblist metric_events; +pid_t tpebs_pid;int ctl_fd;int ctl_fd_ack;bool ctl_fd_close;
Signature · 1 line
2.43.0
Namhyung Kim
maintainerRe: [RFC PATCH v9 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.
Hello,
Quoted 40 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
When
retire_latencyvalue is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect requiredretire_latencyvalues in parallel while perf stat is collecting counting values.At the point of time that perf stat stops counting, it would send sigterm signal to perf record process and receiving sampling data back from perf record from a pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.
Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.
Signed-off-by Weilin Wang <[email protected]>--- tools/perf/builtin-stat.c | 17 ++ tools/perf/util/Build | 1 + tools/perf/util/intel-tpebs.c | 301 ++++++++++++++++++++++++++++++++++ tools/perf/util/intel-tpebs.h | 30 ++++ tools/perf/util/stat.h | 1 + 5 files changed, 350 insertions(+) create mode 100644 tools/perf/util/intel-tpebs.c create mode 100644 tools/perf/util/intel-tpebs.htools/perf/builtin-stat.c +1
-0@@ -70,6 +70,7 @@#include "util/bpf_counter.h" #include "util/iostat.h" #include "util/util.h" +#include "util/intel-tpebs.h" #include "asm/bug.h" #include <linux/time64.h>
tools/perf/builtin-stat.c static struct perf_stat_config stat_config = { · line 166
Quoted 4 lines
tools/perf/builtin-stat.c +1
-0@@ -162,6 +163,7 @@ static struct perf_stat_config stat_config = { .ctl_fd = -1, .ctl_fd_ack = -1, .iostat_run = false, + .tpebs_pid = -1,
Where is this set?
Quoted 57 lines
tools/perf/builtin-stat.c +15
-0};static void evlist__check_cpu_maps (struct evlist *evlist)@@ -653,6 +655,8 @@ static enum counter_recovery stat_handle_error(struct evsel *counter)if (child_pid !=-1 )kill (child_pid,SIGTERM ); +if (stat_config.tpebs_pid !=-1 ) +stop_tpebs ();return COUNTER_FATAL ; }@@ -673,6 +677,10 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx)int err;bool second_pass = false; + err =start_tpebs (&stat_config, evsel_list); +if (err <0 ) +return err; +if (forks) {if (evlist__prepare_workload (evsel_list, &target, argv, is_pipe, workload_exec_failed_signal) <0 ) {perror ("failed to prepare workload" );@@ -878,6 +886,10 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx) t1 =rdclock (); + err =stop_tpebs (); +if (err <0 ) +return err; +if (stat_config.walltime_run_table) stat_config.walltime_run[run_idx] = t1 - t0;@@ -985,6 +997,9 @@ static void sig_atexit(void)if (child_pid !=-1 )kill (child_pid,SIGTERM ); +if (stat_config.tpebs_pid !=-1 ) +kill (stat_config.tpebs_pid,SIGTERM ); +sigprocmask (SIG_SETMASK , &oset,NULL );if (signr ==-1 )@@ -2918,5 +2933,7 @@ int cmd_stat(int argc, const char **argv)metricgroup__rblist_exit (&stat_config.metric_events);evlist__close_control (stat_config.ctl_fd, stat_config.ctl_fd_ack, &stat_config.ctl_fd_close); +tpebs_data__delete (); +return status; }tools/perf/util/Build +0
-0
tools/perf/util/Build perf-y += clockid.o · line 156
Quoted 4 lines
tools/perf/util/Build +1
-0@@ -153,6 +153,7 @@ perf-y += clockid.o perf-y += list_sort.o perf-y += mutex.o perf-y += sharded_mutex.o +perf-y += intel-tpebs.o
Can we make this Intel (or x86) only?
Quoted 26 lines
tools/perf/util/Build +0
-0perf-$(CONFIG_LIBBPF) += bpf_map.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.otools/perf/util/intel-tpebs.c new file +17
-0@@ -0,0 +1,301 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* intel_tpebs.c: Intel TPEBS support +*/ + + +#include <sys/param.h> +#include <subcmd/run-command.h> +#include <thread.h> +#include "intel-tpebs.h" +#include <linux/list.h> +#include <linux/zalloc.h> +#include <linux/err.h> +#include "sample.h" +#include "debug.h" +#include "evlist.h" +#include "evsel.h"
tools/perf/util/intel-tpebs.c line 27
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+#include "session.h" +#include "tool.h" +#include "metricgroup.h" +#include <sys/stat.h> +#include <sys/file.h> +#include <poll.h> + +#define PERF_DATA "-" + +struct list_head tpebs_results =LIST_HEAD_INIT (tpebs_results);
static LIST_HEAD(tpebs_results);
tools/perf/util/intel-tpebs.c line 29
Quoted 2 lines
tools/perf/util/intel-tpebs.c new file +2
-0+static pthread_t reader_thread; +static struct child_process *cmd;
Maybe better to have the 'tpebs_' prefix.
tools/perf/util/intel-tpebs.c line 30
Quoted 1 line
tools/perf/util/intel-tpebs.c new file +1
-0+static struct perf_stat_config *stat_config;
Is this really needed? ...
tools/perf/util/intel-tpebs.c line 33
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+static size_t tpebs_event_size; + +static int get_perf_record_args (const char **record_argv,char buf[])
... I think you can just pass the stat_config argument from start_tpebs().
Quoted 60 lines
tools/perf/util/intel-tpebs.c new file +60
-0+{ +struct tpebs_retire_lat *e; +int i =0 ; + +pr_debug ("Prepare perf record for retire_latency\n" ); + + record_argv[i++] ="perf" ; + record_argv[i++] ="record" ; + record_argv[i++] ="-W" ; + record_argv[i++] ="--synth=no" ; + record_argv[i++] = buf; + +if (stat_config->user_requested_cpu_list) { + record_argv[i++] ="-C" ; + record_argv[i++] = stat_config->user_requested_cpu_list; + } + +if (stat_config->system_wide) + record_argv[i++] ="-a" ; + +if (!stat_config->system_wide && !stat_config->user_requested_cpu_list) { +pr_err ("Require -a or -C option to run sampling.\n" ); +return -ECANCELED ; + } + +list_for_each_entry (e, &tpebs_results, nd) { + record_argv[i++] ="-e" ; + record_argv[i++] = e->name; + } + + record_argv[i++] ="-o" ; + record_argv[i++] =PERF_DATA ; + +return 0 ; +} + +static int prepare_run_command (const char **argv) +{ + cmd =zalloc (sizeof (struct child_process )); +if (!cmd) +return -ENOMEM ; + cmd->argv = argv; + cmd->out =-1 ; +return 0 ; +} + +static int prepare_perf_record (int control_fd[],int ack_fd[]) +{ +const char **record_argv; +int ret; +char buf[32 ]; + +scnprintf (buf,sizeof (buf),"--control=fd:%d,%d" , control_fd[0 ], ack_fd[1 ]); + + record_argv =calloc (12 +2 * tpebs_event_size,sizeof (char *)); +if (!record_argv) +return -ENOMEM ; + + ret =get_perf_record_args (record_argv, buf); +if (ret)
tools/perf/util/intel-tpebs.c line 103
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ goto out; + + ret =prepare_run_command (record_argv); +if (ret) + goto out; + ret =start_command (cmd); +out: +free (record_argv); +return ret; +}
Please add a blank line.
tools/perf/util/intel-tpebs.c line 106
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+struct sample_data_reader { +struct perf_tool tool; +struct perf_session *session;
You don't need this, right? Why not pass the 'tool' directly?
Quoted 24 lines
tools/perf/util/intel-tpebs.c new file +24
-0+}; + +static void tpebs_retire_lat__delete (struct tpebs_retire_lat *r) +{ +zfree (&r->name); +zfree (&r->tpebs_name); +free (r); +} + +void tpebs_data__delete (void ) +{ +struct tpebs_retire_lat *r, *rtmp; + +list_for_each_entry_safe (r, rtmp, &tpebs_results, nd) { +list_del_init (&r->nd); +tpebs_retire_lat__delete (r); + } +free (cmd); +} + +static int process_sample_event(struct perf_tool *tool __maybe_unused, + union perf_event *event __maybe_unused, + struct perf_sample *sample, + struct evsel *evsel,
tools/perf/util/intel-tpebs.c line 140
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ struct machine *machine __maybe_unused) +{ +int ret =0 ; +const char *evname; +struct tpebs_retire_lat *t; + + evname =evsel__name (evsel); + +/* +* Need to handle per core results? We are assuming average retire
I don't know. It depends on the use case where you want different per-core retire-latency for the (per-core) metric.
Quoted 27 lines
tools/perf/util/intel-tpebs.c new file +27
-0+* latency value will be used. Save the number of samples and the sum of +* retire latency value for each event. +*/ +list_for_each_entry (t, &tpebs_results, nd) { +if (!strcmp (evname, t->name)) { + t->count +=1 ; + t->sum += sample->retire_lat; + t->val = (double ) t->sum / t->count; +break ; + } + } + +return ret; +} + +static int process_feature_event (struct perf_session *session, +union perf_event *event) +{ +if (event->feat.feat_id <HEADER_LAST_FEATURE ) +return perf_event__process_feature (session, event); +return 0 ; +} + +static void *__sample_reader (void *arg) +{ +struct child_process *child = arg; +struct perf_session *session;
tools/perf/util/intel-tpebs.c line 177
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+struct perf_data data = { + .mode =PERF_DATA_MODE_READ , + .path =PERF_DATA , + .file.fd = child->out, + }; +struct sample_data_reader reader = { + .tool = { + .sample = process_sample_event, + .feature = process_feature_event, + .attr = perf_event__process_attr,
Strange indentations.
Quoted 16 lines
tools/perf/util/intel-tpebs.c new file +16
-0+ }, + }; + + session =perf_session__new (&data, &reader.tool); +if (IS_ERR (session)) +return NULL ; + reader.session = session; +perf_session__process_events (session); +perf_session__delete (session); + +return NULL ; +} + + +int start_tpebs (struct perf_stat_config *perf_stat_config,struct evlist *evsel_list) +{
tools/perf/util/intel-tpebs.c line 203
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+int ret =0 ; +struct evsel *evsel; + + stat_config = perf_stat_config; +/* +* Prepare perf record for sampling event retire_latency before fork and +* prepare workload +*/ +evlist__for_each_entry (evsel_list, evsel) { +if (evsel->retire_lat) {
To reduce the indentation, consider early continue like
if (!evsel->retire_lat)
continue;tools/perf/util/intel-tpebs.c line 204
Quoted 1 line
tools/perf/util/intel-tpebs.c new file +1
-0+struct tpebs_retire_lat *new =malloc (sizeof (struct tpebs_retire_lat ));
I prefer sizeof(*new) instead but it doesn't matter. But you might use zalloc() and forget about the 0 initialization later.
tools/perf/util/intel-tpebs.c line 213
Quoted 9 lines
tools/perf/util/intel-tpebs.c new file +9
-0+int i; +char *name; + +pr_debug ("perf stat retire latency %s required\n" , evsel->name); +if (!new) +return -1 ; +for (i =strlen (evsel->name) -1 ; i >0 ; i--) { +if (evsel->name[i] =='R' ) +break ;
I remember Ian checked '/' and ':' too. Otherwise it can detect R in the event name and treat it as a modifier.
tools/perf/util/intel-tpebs.c line 216
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+ } +if (i <=0 || evsel->name[i] !='R' ) +return -1 ;
You need to free the memory here and other places.
tools/perf/util/intel-tpebs.c line 223
Quoted 7 lines
tools/perf/util/intel-tpebs.c new file +7
-0+ + name =strdup (evsel->name); +if (!name) +return -ENOMEM ; + name[i] ='p' ; + new->name =strdup (name); +free (name);
Seems like an unnecessary allocation. Why not use 'name' directly?
tools/perf/util/intel-tpebs.c line 226
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+ new->tpebs_name =strdup (evsel->name); +if (!new->tpebs_name) +return -ENOMEM ;
Maybe orig_name? But I'm not sure if it's really needed.. Can we just teach perf record to understand 'R' and to act like 'p'?
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ new->count =0 ; + new->sum =0 ; +list_add_tail (&new->nd, &tpebs_results); + tpebs_event_size +=1 ; + } + } + +if (tpebs_event_size >0 ) { +struct pollfd pollfd = { .events =POLLIN , }; +int control_fd[2 ], ack_fd[2 ], len;
tools/perf/util/intel-tpebs.c line 246
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+char ack_buf[8 ]; + +/*Create control and ack fd for --control*/ +if (pipe (control_fd) <0 ) { +pr_err ("Failed to create control fifo" ); +return -1 ; + } +if (pipe (ack_fd) <0 ) { +pr_err ("Failed to create control fifo" ); +return -1 ;
Need to close the pipes here and other places.
Quoted 12 lines
tools/perf/util/intel-tpebs.c new file +12
-0+ } + + ret =prepare_perf_record (control_fd, ack_fd); +if (ret) +return ret; +if (pthread_create (&reader_thread,NULL , __sample_reader, cmd)) { +kill (cmd->pid,SIGTERM ); +close (cmd->out); +pr_err ("Could not create thread to process sample data.\n" ); +return -1 ; + } +/* Wait for perf record initialization.*/
tools/perf/util/intel-tpebs.c line 268
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ len =strlen ("enable" ); + ret =write (control_fd[1 ],"enable" , len); +if (ret != len) { +pr_err ("perf record control write control message failed\n" ); + goto out; + } + + pollfd.fd = ack_fd[0 ]; +if (!poll (&pollfd,1 ,2000 )) + goto out;
Do we need this? Why not just read?
Quoted 57 lines
tools/perf/util/intel-tpebs.c new file +33
-0+ + ret =read (ack_fd[0 ], ack_buf,sizeof (ack_buf)); +if (ret >0 ) + ret =strcmp (ack_buf,"ack\n" ); +else { +pr_err ("perf record control ack failed\n" ); + goto out; + } +pr_debug ("Received ack from perf record\n" ); +out: +close (control_fd[0 ]); +close (control_fd[1 ]); +close (ack_fd[0 ]); +close (ack_fd[1 ]); + } +return ret; +} + + +int stop_tpebs (void ) +{ +int ret =0 ; + +if (tpebs_event_size >0 ) { +kill (cmd->pid,SIGTERM ); +pthread_join (reader_thread,NULL ); +close (cmd->out); + ret =finish_command (cmd); +if (ret == -ERR_RUN_COMMAND_WAITPID_SIGNAL ) + ret =0 ; + } +return ret; +}tools/perf/util/intel-tpebs.h new file +18
-0@@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* intel_tpebs.h: Intel TEPBS support +*/ +#include "stat.h" + +#ifndef INCLUDE__PERF_INTEL_TPEBS_H__ +#define INCLUDE__PERF_INTEL_TPEBS_H__ + +struct tpebs_retire_lat { + struct list_head nd; + /* Event name */ + const char *name; + /* Event name with the TPEBS modifier R */ + const char *tpebs_name; + /* Count of retire_latency values found in sample data */ + size_t count; + /* Sum of all the retire_latency values in sample data */
tools/perf/util/intel-tpebs.h line 28
Quoted 10 lines
tools/perf/util/intel-tpebs.h new file +10
-0+int sum; +/* Average of retire_latency, val = sum / count */ +double val; +}; + +extern struct list_head tpebs_results; + +int start_tpebs (struct perf_stat_config *perf_stat_config,struct evlist *evsel_list); +int stop_tpebs (void ); +void tpebs_data__delete (void );
I think it's better to have the same prefix.
tpebs_start()tpebs_stop()tpebs_delete()
Thanks,
Namhyung
Quoted 17 lines
tools/perf/util/intel-tpebs.h new file +2
-0+ +#endif tools/perf/util/stat.h +1
-1@@ -110,6 +110,7 @@ struct perf_stat_config {struct cpu_aggr_map *cpus_aggr_map;u64 *walltime_run;struct rblist metric_events; +pid_t tpebs_pid;int ctl_fd;int ctl_fd_ack;bool ctl_fd_close; --2.43.0
RE: [RFC PATCH v9 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.
Quoted 350 lines
-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 24, 2024 4:02 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v9 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.Hello,
Quoted 9 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
When
retire_latencyvalue is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect requiredretire_latencyvalues in parallel while perf stat is collecting counting values.At the point of time that perf stat stops counting, it would send sigterm
signal
Quoted 1 line
to perf record process and receiving sampling data back from perf record
from a
Quoted 11 lines
pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.
Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.
Signed-off-by Weilin Wang <[email protected]>--- tools/perf/builtin-stat.c | 17 ++ tools/perf/util/Build | 1 + tools/perf/util/intel-tpebs.c | 301++++++++++++++++++++++++++++++++++
Quoted 19 lines
tools/perf/util/intel-tpebs.h | 30 ++++ tools/perf/util/stat.h | 1 + 5 files changed, 350 insertions(+) create mode 100644 tools/perf/util/intel-tpebs.c create mode 100644 tools/perf/util/intel-tpebs.htools/perf/builtin-stat.c +1
-0@@ -70,6 +70,7 @@#include "util/bpf_counter.h" #include "util/iostat.h" #include "util/util.h" +#include "util/intel-tpebs.h" #include "asm/bug.h" #include <linux/time64.h> tools/perf/builtin-stat.c static struct perf_stat_config stat_config = { · line 166
Quoted 4 lines
tools/perf/builtin-stat.c +1
-0@@ -162,6 +163,7 @@ static struct perf_stat_config stat_config = { .ctl_fd = -1, .ctl_fd_ack = -1, .iostat_run = false, + .tpebs_pid = -1,Where is this set?
tools/perf/builtin-stat.c static enum counter_recovery
Quoted 4 lines
tools/perf/builtin-stat.c +0
-0};static void evlist__check_cpu_maps (struct evlist *evlist)@@ -653,6 +655,8 @@ static enum counter_recovery
stat_handle_error(struct evsel *counter)tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char
Quoted 9 lines
tools/perf/builtin-stat.c +2
-0if (child_pid !=-1 )kill (child_pid,SIGTERM ); +if (stat_config.tpebs_pid !=-1 ) +stop_tpebs ();return COUNTER_FATAL ; }@@ -673,6 +677,10 @@ static int __run_perf_stat(int argc, const char**argv, int
run_idx)tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char **argv, int run_idx)
Quoted 9 lines
tools/perf/builtin-stat.c +4
-0int err;bool second_pass = false; + err =start_tpebs (&stat_config, evsel_list); +if (err <0 ) +return err; +if (forks) { if (evlist__prepare_workload(evsel_list, &target, argv, is_pipe,
workload_exec_failed_signal) < 0) {tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char
Quoted 2 lines
tools/perf/builtin-stat.c +0
-0perror ("failed to prepare workload" );@@ -878,6 +886,10 @@ static int __run_perf_stat(int argc, const char**argv, int
run_idx)Quoted 21 lines
tools/perf/builtin-stat.c +7
-0t1 =rdclock (); + err =stop_tpebs (); +if (err <0 ) +return err; +if (stat_config.walltime_run_table) stat_config.walltime_run[run_idx] = t1 - t0;@@ -985,6 +997,9 @@ static void sig_atexit(void)if (child_pid !=-1 )kill (child_pid,SIGTERM ); +if (stat_config.tpebs_pid !=-1 ) +kill (stat_config.tpebs_pid,SIGTERM ); +sigprocmask (SIG_SETMASK , &oset,NULL );if (signr ==-1 )tools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv) · line 2934
Quoted 2 lines
tools/perf/builtin-stat.c +0
-0@@ -2918,5 +2933,7 @@ int cmd_stat(int argc, const char **argv)metricgroup__rblist_exit (&stat_config.metric_events); evlist__close_control(stat_config.ctl_fd, stat_config.ctl_fd_ack,&stat_config.ctl_fd_close);
Quoted 10 lines
tools/perf/builtin-stat.c +2
-0+tpebs_data__delete (); +return status; }tools/perf/util/Build +0
-0tools/perf/util/Build perf-y += clockid.o · line 156
Quoted 4 lines
tools/perf/util/Build +1
-0@@ -153,6 +153,7 @@ perf-y += clockid.o perf-y += list_sort.o perf-y += mutex.o perf-y += sharded_mutex.o +perf-y += intel-tpebs.oCan we make this Intel (or x86) only?
Quoted 26 lines
tools/perf/util/Build +0
-0perf-$(CONFIG_LIBBPF) += bpf_map.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.otools/perf/util/intel-tpebs.c new file +17
-0@@ -0,0 +1,301 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* intel_tpebs.c: Intel TPEBS support +*/ + + +#include <sys/param.h> +#include <subcmd/run-command.h> +#include <thread.h> +#include "intel-tpebs.h" +#include <linux/list.h> +#include <linux/zalloc.h> +#include <linux/err.h> +#include "sample.h" +#include "debug.h" +#include "evlist.h" +#include "evsel.h" tools/perf/util/intel-tpebs.c line 27
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+#include "session.h" +#include "tool.h" +#include "metricgroup.h" +#include <sys/stat.h> +#include <sys/file.h> +#include <poll.h> + +#define PERF_DATA "-" + +struct list_head tpebs_results =LIST_HEAD_INIT (tpebs_results);static
LIST_HEAD(tpebs_results);tools/perf/util/intel-tpebs.c line 29
Quoted 2 lines
tools/perf/util/intel-tpebs.c new file +2
-0+static pthread_t reader_thread; +static struct child_process *cmd;Maybe better to have the '
tpebs_' prefix.tools/perf/util/intel-tpebs.c line 30
Quoted 1 line
tools/perf/util/intel-tpebs.c new file +1
-0+static struct perf_stat_config *stat_config;Is this really needed? ...
tools/perf/util/intel-tpebs.c line 33
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+static size_t tpebs_event_size; + +static int get_perf_record_args (const char **record_argv,char buf[])... I think you can just pass the
stat_configargument fromstart_tpebs().Quoted 44 lines
tools/perf/util/intel-tpebs.c new file +21
-0+{ + struct tpebs_retire_lat *e; + int i = 0; + + pr_debug("Prepare perf record for retire_latency\n"); + + record_argv[i++] = "perf"; + record_argv[i++] = "record"; + record_argv[i++] = "-W"; + record_argv[i++] = "--synth=no"; + record_argv[i++] = buf; + + if (stat_config->user_requested_cpu_list) { + record_argv[i++] = "-C"; + record_argv[i++] = stat_config->user_requested_cpu_list; + } + + if (stat_config->system_wide) + record_argv[i++] = "-a"; + + if (!stat_config->system_wide && !stat_config-
user_requested_cpu_list) {tools/perf/util/intel-tpebs.c new file +22
-0+pr_err ("Require -a or -C option to run sampling.\n" ); +return -ECANCELED ; + } + +list_for_each_entry (e, &tpebs_results, nd) { + record_argv[i++] ="-e" ; + record_argv[i++] = e->name; + } + + record_argv[i++] ="-o" ; + record_argv[i++] =PERF_DATA ; + +return 0 ; +} + +static int prepare_run_command (const char **argv) +{ + cmd =zalloc (sizeof (struct child_process )); +if (!cmd) +return -ENOMEM ; + cmd->argv = argv; + cmd->out =-1 ;tools/perf/util/intel-tpebs.c
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+return 0 ; +} + +static int prepare_perf_record (int control_fd[],int ack_fd[]) +{ +const char **record_argv; +int ret; +char buf[32 ]; + + scnprintf(buf, sizeof(buf), "--control=fd:%d,%d", control_fd[0],ack_fd[1]);
Quoted 7 lines
tools/perf/util/intel-tpebs.c new file +7
-0+ + record_argv =calloc (12 +2 * tpebs_event_size,sizeof (char *)); +if (!record_argv) +return -ENOMEM ; + + ret =get_perf_record_args (record_argv, buf); +if (ret)tools/perf/util/intel-tpebs.c line 103
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ goto out; + + ret =prepare_run_command (record_argv); +if (ret) + goto out; + ret =start_command (cmd); +out: +free (record_argv); +return ret; +}Please add a blank line.
tools/perf/util/intel-tpebs.c line 106
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+struct sample_data_reader { +struct perf_tool tool; +struct perf_session *session;You don't need this, right? Why not pass the 'tool' directly?
Quoted 24 lines
tools/perf/util/intel-tpebs.c new file +24
-0+}; + +static void tpebs_retire_lat__delete (struct tpebs_retire_lat *r) +{ +zfree (&r->name); +zfree (&r->tpebs_name); +free (r); +} + +void tpebs_data__delete (void ) +{ +struct tpebs_retire_lat *r, *rtmp; + +list_for_each_entry_safe (r, rtmp, &tpebs_results, nd) { +list_del_init (&r->nd); +tpebs_retire_lat__delete (r); + } +free (cmd); +} + +static int process_sample_event(struct perf_tool *tool __maybe_unused, + union perf_event *event __maybe_unused, + struct perf_sample *sample, + struct evsel *evsel,tools/perf/util/intel-tpebs.c line 140
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ struct machine *machine __maybe_unused) +{ +int ret =0 ; +const char *evname; +struct tpebs_retire_lat *t; + + evname =evsel__name (evsel); + +/* +* Need to handle per core results? We are assuming average retire I don't know. It depends on the use case where you want different per-core retire-latency for the (per-core) metric.
Quoted 1 line
+ * latency value will be used. Save the number of samples and the sum
of
Quoted 30 lines
tools/perf/util/intel-tpebs.c new file +30
-0+* retire latency value for each event. +*/ +list_for_each_entry (t, &tpebs_results, nd) { +if (!strcmp (evname, t->name)) { + t->count +=1 ; + t->sum += sample->retire_lat; + t->val = (double ) t->sum / t->count; +break ; + } + } + +return ret; +} + +static int process_feature_event (struct perf_session *session, +union perf_event *event) +{ +if (event->feat.feat_id <HEADER_LAST_FEATURE ) +return perf_event__process_feature (session, event); +return 0 ; +} + +static void *__sample_reader (void *arg) +{ +struct child_process *child = arg; +struct perf_session *session; +struct perf_data data = { + .mode =PERF_DATA_MODE_READ , + .path =PERF_DATA , + .file.fd = child->out,
Quoted 8 lines
tools/perf/util/intel-tpebs.c line 177
Quoted 6 lines
tools/perf/util/intel-tpebs.c new file +6
-0+ }; +struct sample_data_reader reader = { + .tool = { + .sample = process_sample_event, + .feature = process_feature_event, + .attr = perf_event__process_attr,Strange indentations.
Hi Namhyung,
The indentations here look correct on the webpage and in my environment. I'm not sure why they are different here.
Quoted 96 lines
Quoted 7 lines
tools/perf/util/intel-tpebs.c new file +5
-0+ }, + }; + + session =perf_session__new (&data, &reader.tool); +if (IS_ERR (session))tools/perf/util/intel-tpebs.c
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+return NULL ; + reader.session = session; +perf_session__process_events (session); +perf_session__delete (session); + +return NULL ; +} + + +int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist
*evsel_list)tools/perf/util/intel-tpebs.c line 203
Quoted 11 lines
tools/perf/util/intel-tpebs.c new file +11
-0+{ +int ret =0 ; +struct evsel *evsel; + + stat_config = perf_stat_config; +/* +* Prepare perf record for sampling event retire_latency before fork and +* prepare workload +*/ +evlist__for_each_entry (evsel_list, evsel) { +if (evsel->retire_lat) {To reduce the indentation, consider early continue like
if (!evsel->retire_lat) continue;Quoted 1 line
+ struct
tpebs_retire_lat*new = malloc(sizeof(struct
tpebs_retire_lat));I prefer
sizeof(*new)instead but it doesn't matter. But you might usezalloc()and forget about the 0 initialization later.tools/perf/util/intel-tpebs.c line 213
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +4
-0+int i; +char *name; + + pr_debug("perf stat retire latency %s required\n", evsel-name);
tools/perf/util/intel-tpebs.c new file +5
-0+if (!new) +return -1 ; +for (i =strlen (evsel->name) -1 ; i >0 ; i--) { +if (evsel->name[i] =='R' ) +break ;I remember Ian checked '/' and ':' too. Otherwise it can detect R in the event name and treat it as a modifier.
tools/perf/util/intel-tpebs.c line 216
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+ } +if (i <=0 || evsel->name[i] !='R' ) +return -1 ;You need to free the memory here and other places.
tools/perf/util/intel-tpebs.c line 223
Quoted 7 lines
tools/perf/util/intel-tpebs.c new file +7
-0+ + name =strdup (evsel->name); +if (!name) +return -ENOMEM ; + name[i] ='p' ; + new->name =strdup (name); +free (name);Seems like an unnecessary allocation. Why not use 'name' directly?
tools/perf/util/intel-tpebs.c line 226
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+ new->tpebs_name =strdup (evsel->name); +if (!new->tpebs_name) +return -ENOMEM ;Maybe
orig_name? But I'm not sure if it's really needed.. Can we just teach perf record to understand 'R' and to act like 'p'?Quoted 14 lines
tools/perf/util/intel-tpebs.c new file +14
-0+ new->count =0 ; + new->sum =0 ; +list_add_tail (&new->nd, &tpebs_results); + tpebs_event_size +=1 ; + } + } + +if (tpebs_event_size >0 ) { +struct pollfd pollfd = { .events =POLLIN , }; +int control_fd[2 ], ack_fd[2 ], len; +char ack_buf[8 ]; + +/*Create control and ack fd for --control*/ +if (pipe (control_fd) <0 ) {
Quoted 8 lines
tools/perf/util/intel-tpebs.c line 246
Quoted 6 lines
tools/perf/util/intel-tpebs.c new file +6
-0+pr_err ("Failed to create control fifo" ); +return -1 ; + } +if (pipe (ack_fd) <0 ) { +pr_err ("Failed to create control fifo" ); +return -1 ;Need to close the pipes here and other places.
The pipes are closed at the end of the function.
Quoted 17 lines
Quoted 17 lines
tools/perf/util/intel-tpebs.c new file +16
-0+ } + + ret =prepare_perf_record (control_fd, ack_fd); +if (ret) +return ret; +if (pthread_create (&reader_thread,NULL , __sample_reader, cmd)) { +kill (cmd->pid,SIGTERM ); +close (cmd->out); +pr_err ("Could not create thread to process sample data.\n" ); +return -1 ; + } +/* Wait for perf record initialization.*/ + len =strlen ("enable" ); + ret =write (control_fd[1 ],"enable" , len); +if (ret != len) { +pr_err ("perf record control write control message failed\n" );
Quoted 8 lines
tools/perf/util/intel-tpebs.c line 268
Quoted 6 lines
tools/perf/util/intel-tpebs.c new file +6
-0+ goto out; + } + + pollfd.fd = ack_fd[0 ]; +if (!poll (&pollfd,1 ,2000 )) + goto out;Do we need this? Why not just read?
I could remove this part if we don't need it.
Thanks,
Weilin
Quoted 96 lines
Quoted 56 lines
tools/perf/util/intel-tpebs.c new file +33
-0+ + ret =read (ack_fd[0 ], ack_buf,sizeof (ack_buf)); +if (ret >0 ) + ret =strcmp (ack_buf,"ack\n" ); +else { +pr_err ("perf record control ack failed\n" ); + goto out; + } +pr_debug ("Received ack from perf record\n" ); +out: +close (control_fd[0 ]); +close (control_fd[1 ]); +close (ack_fd[0 ]); +close (ack_fd[1 ]); + } +return ret; +} + + +int stop_tpebs (void ) +{ +int ret =0 ; + +if (tpebs_event_size >0 ) { +kill (cmd->pid,SIGTERM ); +pthread_join (reader_thread,NULL ); +close (cmd->out); + ret =finish_command (cmd); +if (ret == -ERR_RUN_COMMAND_WAITPID_SIGNAL ) + ret =0 ; + } +return ret; +}tools/perf/util/intel-tpebs.h new file +16
-0@@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* intel_tpebs.h: Intel TEPBS support +*/ +#include "stat.h" + +#ifndef INCLUDE__PERF_INTEL_TPEBS_H__ +#define INCLUDE__PERF_INTEL_TPEBS_H__ + +struct tpebs_retire_lat { + struct list_head nd; + /* Event name */ + const char *name; + /* Event name with the TPEBS modifier R */ + const char *tpebs_name; + /* Count of retire_latency values found in sample data */tools/perf/util/intel-tpebs.h line 26
Quoted 10 lines
tools/perf/util/intel-tpebs.h new file +10
-0+size_t count; +/* Sum of all the retire_latency values in sample data */ +int sum; +/* Average of retire_latency, val = sum / count */ +double val; +}; + +extern struct list_head tpebs_results; + +int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist
*evsel_list);tools/perf/util/intel-tpebs.h line 28
Quoted 2 lines
tools/perf/util/intel-tpebs.h new file +2
-0+int stop_tpebs (void ); +void tpebs_data__delete (void );I think it's better to have the same prefix.
tpebs_start()tpebs_stop()tpebs_delete()Thanks,
NamhyungQuoted 17 lines
tools/perf/util/intel-tpebs.h new file +2
-0+ +#endif tools/perf/util/stat.h +1
-1@@ -110,6 +110,7 @@ struct perf_stat_config {struct cpu_aggr_map *cpus_aggr_map;u64 *walltime_run;struct rblist metric_events; +pid_t tpebs_pid;int ctl_fd;int ctl_fd_ack;bool ctl_fd_close; --2.43.0
Namhyung Kim
maintainerRe: [RFC PATCH v9 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.
Quoted 360 lines · Weilin
Quoted 351 lines
-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 24, 2024 4:02 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v9 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.Hello,
Quoted 9 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
When
retire_latencyvalue is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect requiredretire_latencyvalues in parallel while perf stat is collecting counting values.At the point of time that perf stat stops counting, it would send sigterm
signal
Quoted 1 line
to perf record process and receiving sampling data back from perf record
from a
Quoted 11 lines
pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.
Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.
Signed-off-by Weilin Wang <[email protected]>--- tools/perf/builtin-stat.c | 17 ++ tools/perf/util/Build | 1 + tools/perf/util/intel-tpebs.c | 301++++++++++++++++++++++++++++++++++
Quoted 19 lines
tools/perf/util/intel-tpebs.h | 30 ++++ tools/perf/util/stat.h | 1 + 5 files changed, 350 insertions(+) create mode 100644 tools/perf/util/intel-tpebs.c create mode 100644 tools/perf/util/intel-tpebs.htools/perf/builtin-stat.c +1
-0@@ -70,6 +70,7 @@#include "util/bpf_counter.h" #include "util/iostat.h" #include "util/util.h" +#include "util/intel-tpebs.h" #include "asm/bug.h" #include <linux/time64.h> tools/perf/builtin-stat.c static struct perf_stat_config stat_config = { · line 166
Quoted 4 lines
tools/perf/builtin-stat.c +1
-0@@ -162,6 +163,7 @@ static struct perf_stat_config stat_config = { .ctl_fd = -1, .ctl_fd_ack = -1, .iostat_run = false, + .tpebs_pid = -1,Where is this set?
tools/perf/builtin-stat.c static enum counter_recovery
Quoted 4 lines
tools/perf/builtin-stat.c +0
-0};static void evlist__check_cpu_maps (struct evlist *evlist)@@ -653,6 +655,8 @@ static enum counter_recovery
stat_handle_error(struct evsel *counter)tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char
Quoted 9 lines
tools/perf/builtin-stat.c +2
-0if (child_pid !=-1 )kill (child_pid,SIGTERM ); +if (stat_config.tpebs_pid !=-1 ) +stop_tpebs ();return COUNTER_FATAL ; }@@ -673,6 +677,10 @@ static int __run_perf_stat(int argc, const char**argv, int
run_idx)tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char **argv, int run_idx)
Quoted 9 lines
tools/perf/builtin-stat.c +4
-0int err;bool second_pass = false; + err =start_tpebs (&stat_config, evsel_list); +if (err <0 ) +return err; +if (forks) { if (evlist__prepare_workload(evsel_list, &target, argv, is_pipe,
workload_exec_failed_signal) < 0) {tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char
Quoted 2 lines
tools/perf/builtin-stat.c +0
-0perror ("failed to prepare workload" );@@ -878,6 +886,10 @@ static int __run_perf_stat(int argc, const char**argv, int
run_idx)Quoted 21 lines
tools/perf/builtin-stat.c +7
-0t1 =rdclock (); + err =stop_tpebs (); +if (err <0 ) +return err; +if (stat_config.walltime_run_table) stat_config.walltime_run[run_idx] = t1 - t0;@@ -985,6 +997,9 @@ static void sig_atexit(void)if (child_pid !=-1 )kill (child_pid,SIGTERM ); +if (stat_config.tpebs_pid !=-1 ) +kill (stat_config.tpebs_pid,SIGTERM ); +sigprocmask (SIG_SETMASK , &oset,NULL );if (signr ==-1 )tools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv) · line 2934
Quoted 2 lines
tools/perf/builtin-stat.c +0
-0@@ -2918,5 +2933,7 @@ int cmd_stat(int argc, const char **argv)metricgroup__rblist_exit (&stat_config.metric_events); evlist__close_control(stat_config.ctl_fd, stat_config.ctl_fd_ack,&stat_config.ctl_fd_close);
Quoted 10 lines
tools/perf/builtin-stat.c +2
-0+tpebs_data__delete (); +return status; }tools/perf/util/Build +0
-0tools/perf/util/Build perf-y += clockid.o · line 156
Quoted 4 lines
tools/perf/util/Build +1
-0@@ -153,6 +153,7 @@ perf-y += clockid.o perf-y += list_sort.o perf-y += mutex.o perf-y += sharded_mutex.o +perf-y += intel-tpebs.oCan we make this Intel (or x86) only?
Quoted 26 lines
tools/perf/util/Build +0
-0perf-$(CONFIG_LIBBPF) += bpf_map.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.otools/perf/util/intel-tpebs.c new file +17
-0@@ -0,0 +1,301 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* intel_tpebs.c: Intel TPEBS support +*/ + + +#include <sys/param.h> +#include <subcmd/run-command.h> +#include <thread.h> +#include "intel-tpebs.h" +#include <linux/list.h> +#include <linux/zalloc.h> +#include <linux/err.h> +#include "sample.h" +#include "debug.h" +#include "evlist.h" +#include "evsel.h" tools/perf/util/intel-tpebs.c line 27
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+#include "session.h" +#include "tool.h" +#include "metricgroup.h" +#include <sys/stat.h> +#include <sys/file.h> +#include <poll.h> + +#define PERF_DATA "-" + +struct list_head tpebs_results =LIST_HEAD_INIT (tpebs_results);static
LIST_HEAD(tpebs_results);tools/perf/util/intel-tpebs.c line 29
Quoted 2 lines
tools/perf/util/intel-tpebs.c new file +2
-0+static pthread_t reader_thread; +static struct child_process *cmd;Maybe better to have the '
tpebs_' prefix.tools/perf/util/intel-tpebs.c line 30
Quoted 1 line
tools/perf/util/intel-tpebs.c new file +1
-0+static struct perf_stat_config *stat_config;Is this really needed? ...
tools/perf/util/intel-tpebs.c line 33
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+static size_t tpebs_event_size; + +static int get_perf_record_args (const char **record_argv,char buf[])... I think you can just pass the
stat_configargument fromstart_tpebs().Quoted 44 lines
tools/perf/util/intel-tpebs.c new file +21
-0+{ + struct tpebs_retire_lat *e; + int i = 0; + + pr_debug("Prepare perf record for retire_latency\n"); + + record_argv[i++] = "perf"; + record_argv[i++] = "record"; + record_argv[i++] = "-W"; + record_argv[i++] = "--synth=no"; + record_argv[i++] = buf; + + if (stat_config->user_requested_cpu_list) { + record_argv[i++] = "-C"; + record_argv[i++] = stat_config->user_requested_cpu_list; + } + + if (stat_config->system_wide) + record_argv[i++] = "-a"; + + if (!stat_config->system_wide && !stat_config-
user_requested_cpu_list) {tools/perf/util/intel-tpebs.c new file +22
-0+pr_err ("Require -a or -C option to run sampling.\n" ); +return -ECANCELED ; + } + +list_for_each_entry (e, &tpebs_results, nd) { + record_argv[i++] ="-e" ; + record_argv[i++] = e->name; + } + + record_argv[i++] ="-o" ; + record_argv[i++] =PERF_DATA ; + +return 0 ; +} + +static int prepare_run_command (const char **argv) +{ + cmd =zalloc (sizeof (struct child_process )); +if (!cmd) +return -ENOMEM ; + cmd->argv = argv; + cmd->out =-1 ;tools/perf/util/intel-tpebs.c
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+return 0 ; +} + +static int prepare_perf_record (int control_fd[],int ack_fd[]) +{ +const char **record_argv; +int ret; +char buf[32 ]; + + scnprintf(buf, sizeof(buf), "--control=fd:%d,%d", control_fd[0],ack_fd[1]);
Quoted 7 lines
tools/perf/util/intel-tpebs.c new file +7
-0+ + record_argv =calloc (12 +2 * tpebs_event_size,sizeof (char *)); +if (!record_argv) +return -ENOMEM ; + + ret =get_perf_record_args (record_argv, buf); +if (ret)tools/perf/util/intel-tpebs.c line 103
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ goto out; + + ret =prepare_run_command (record_argv); +if (ret) + goto out; + ret =start_command (cmd); +out: +free (record_argv); +return ret; +}Please add a blank line.
tools/perf/util/intel-tpebs.c line 106
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+struct sample_data_reader { +struct perf_tool tool; +struct perf_session *session;You don't need this, right? Why not pass the 'tool' directly?
Quoted 24 lines
tools/perf/util/intel-tpebs.c new file +24
-0+}; + +static void tpebs_retire_lat__delete (struct tpebs_retire_lat *r) +{ +zfree (&r->name); +zfree (&r->tpebs_name); +free (r); +} + +void tpebs_data__delete (void ) +{ +struct tpebs_retire_lat *r, *rtmp; + +list_for_each_entry_safe (r, rtmp, &tpebs_results, nd) { +list_del_init (&r->nd); +tpebs_retire_lat__delete (r); + } +free (cmd); +} + +static int process_sample_event(struct perf_tool *tool __maybe_unused, + union perf_event *event __maybe_unused, + struct perf_sample *sample, + struct evsel *evsel,tools/perf/util/intel-tpebs.c line 140
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ struct machine *machine __maybe_unused) +{ +int ret =0 ; +const char *evname; +struct tpebs_retire_lat *t; + + evname =evsel__name (evsel); + +/* +* Need to handle per core results? We are assuming average retire I don't know. It depends on the use case where you want different per-core retire-latency for the (per-core) metric.
Quoted 1 line
+ * latency value will be used. Save the number of samples and the sum
of
Quoted 30 lines
tools/perf/util/intel-tpebs.c new file +30
-0+* retire latency value for each event. +*/ +list_for_each_entry (t, &tpebs_results, nd) { +if (!strcmp (evname, t->name)) { + t->count +=1 ; + t->sum += sample->retire_lat; + t->val = (double ) t->sum / t->count; +break ; + } + } + +return ret; +} + +static int process_feature_event (struct perf_session *session, +union perf_event *event) +{ +if (event->feat.feat_id <HEADER_LAST_FEATURE ) +return perf_event__process_feature (session, event); +return 0 ; +} + +static void *__sample_reader (void *arg) +{ +struct child_process *child = arg; +struct perf_session *session; +struct perf_data data = { + .mode =PERF_DATA_MODE_READ , + .path =PERF_DATA , + .file.fd = child->out,Quoted 8 lines
tools/perf/util/intel-tpebs.c line 177
Quoted 6 lines
tools/perf/util/intel-tpebs.c new file +6
-0+ }; +struct sample_data_reader reader = { + .tool = { + .sample = process_sample_event, + .feature = process_feature_event, + .attr = perf_event__process_attr,Strange indentations.
Quoted 4 lines
Hi Namhyung,
The indentations here look correct on the webpage and in my environment. I'm not sure why they are different here.
Sounds like a tab vs space issue.
Quoted 106 lines
Quoted 97 lines
Quoted 8 lines
tools/perf/util/intel-tpebs.c new file +5
-0+ }, + }; + + session =perf_session__new (&data, &reader.tool); +if (IS_ERR (session))tools/perf/util/intel-tpebs.c
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+return NULL ; + reader.session = session; +perf_session__process_events (session); +perf_session__delete (session); + +return NULL ; +} + + +int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist
*evsel_list)tools/perf/util/intel-tpebs.c line 203
Quoted 11 lines
tools/perf/util/intel-tpebs.c new file +11
-0+{ +int ret =0 ; +struct evsel *evsel; + + stat_config = perf_stat_config; +/* +* Prepare perf record for sampling event retire_latency before fork and +* prepare workload +*/ +evlist__for_each_entry (evsel_list, evsel) { +if (evsel->retire_lat) {To reduce the indentation, consider early continue like
if (!evsel->retire_lat) continue;Quoted 1 line
+ struct
tpebs_retire_lat*new = malloc(sizeof(struct
tpebs_retire_lat));I prefer
sizeof(*new)instead but it doesn't matter. But you might usezalloc()and forget about the 0 initialization later.tools/perf/util/intel-tpebs.c line 213
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +4
-0+int i; +char *name; + + pr_debug("perf stat retire latency %s required\n", evsel-name);
tools/perf/util/intel-tpebs.c new file +5
-0+if (!new) +return -1 ; +for (i =strlen (evsel->name) -1 ; i >0 ; i--) { +if (evsel->name[i] =='R' ) +break ;I remember Ian checked '/' and ':' too. Otherwise it can detect R in the event name and treat it as a modifier.
tools/perf/util/intel-tpebs.c line 216
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+ } +if (i <=0 || evsel->name[i] !='R' ) +return -1 ;You need to free the memory here and other places.
tools/perf/util/intel-tpebs.c line 223
Quoted 7 lines
tools/perf/util/intel-tpebs.c new file +7
-0+ + name =strdup (evsel->name); +if (!name) +return -ENOMEM ; + name[i] ='p' ; + new->name =strdup (name); +free (name);Seems like an unnecessary allocation. Why not use 'name' directly?
tools/perf/util/intel-tpebs.c line 226
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+ new->tpebs_name =strdup (evsel->name); +if (!new->tpebs_name) +return -ENOMEM ;Maybe
orig_name? But I'm not sure if it's really needed.. Can we just teach perf record to understand 'R' and to act like 'p'?Quoted 14 lines
tools/perf/util/intel-tpebs.c new file +14
-0+ new->count =0 ; + new->sum =0 ; +list_add_tail (&new->nd, &tpebs_results); + tpebs_event_size +=1 ; + } + } + +if (tpebs_event_size >0 ) { +struct pollfd pollfd = { .events =POLLIN , }; +int control_fd[2 ], ack_fd[2 ], len; +char ack_buf[8 ]; + +/*Create control and ack fd for --control*/ +if (pipe (control_fd) <0 ) {Quoted 8 lines
tools/perf/util/intel-tpebs.c line 246
Quoted 6 lines
tools/perf/util/intel-tpebs.c new file +6
-0+pr_err ("Failed to create control fifo" ); +return -1 ; + } +if (pipe (ack_fd) <0 ) { +pr_err ("Failed to create control fifo" ); +return -1 ;Need to close the pipes here and other places.
Quoted 1 line
The pipes are closed at the end of the function.
Yep, but you didn't go there. :)
Thanks,
Namhyung
Quoted 127 lines
Quoted 18 lines
Quoted 18 lines
tools/perf/util/intel-tpebs.c new file +16
-0+ } + + ret =prepare_perf_record (control_fd, ack_fd); +if (ret) +return ret; +if (pthread_create (&reader_thread,NULL , __sample_reader, cmd)) { +kill (cmd->pid,SIGTERM ); +close (cmd->out); +pr_err ("Could not create thread to process sample data.\n" ); +return -1 ; + } +/* Wait for perf record initialization.*/ + len =strlen ("enable" ); + ret =write (control_fd[1 ],"enable" , len); +if (ret != len) { +pr_err ("perf record control write control message failed\n" );Quoted 8 lines
tools/perf/util/intel-tpebs.c line 268
Quoted 6 lines
tools/perf/util/intel-tpebs.c new file +6
-0+ goto out; + } + + pollfd.fd = ack_fd[0 ]; +if (!poll (&pollfd,1 ,2000 )) + goto out;Do we need this? Why not just read?
I could remove this part if we don't need it.
Thanks,
WeilinQuoted 96 lines
Quoted 56 lines
tools/perf/util/intel-tpebs.c new file +33
-0+ + ret =read (ack_fd[0 ], ack_buf,sizeof (ack_buf)); +if (ret >0 ) + ret =strcmp (ack_buf,"ack\n" ); +else { +pr_err ("perf record control ack failed\n" ); + goto out; + } +pr_debug ("Received ack from perf record\n" ); +out: +close (control_fd[0 ]); +close (control_fd[1 ]); +close (ack_fd[0 ]); +close (ack_fd[1 ]); + } +return ret; +} + + +int stop_tpebs (void ) +{ +int ret =0 ; + +if (tpebs_event_size >0 ) { +kill (cmd->pid,SIGTERM ); +pthread_join (reader_thread,NULL ); +close (cmd->out); + ret =finish_command (cmd); +if (ret == -ERR_RUN_COMMAND_WAITPID_SIGNAL ) + ret =0 ; + } +return ret; +}tools/perf/util/intel-tpebs.h new file +16
-0@@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* intel_tpebs.h: Intel TEPBS support +*/ +#include "stat.h" + +#ifndef INCLUDE__PERF_INTEL_TPEBS_H__ +#define INCLUDE__PERF_INTEL_TPEBS_H__ + +struct tpebs_retire_lat { + struct list_head nd; + /* Event name */ + const char *name; + /* Event name with the TPEBS modifier R */ + const char *tpebs_name; + /* Count of retire_latency values found in sample data */tools/perf/util/intel-tpebs.h line 26
Quoted 10 lines
tools/perf/util/intel-tpebs.h new file +10
-0+size_t count; +/* Sum of all the retire_latency values in sample data */ +int sum; +/* Average of retire_latency, val = sum / count */ +double val; +}; + +extern struct list_head tpebs_results; + +int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist
*evsel_list);tools/perf/util/intel-tpebs.h line 28
Quoted 2 lines
tools/perf/util/intel-tpebs.h new file +2
-0+int stop_tpebs (void ); +void tpebs_data__delete (void );I think it's better to have the same prefix.
tpebs_start()tpebs_stop()tpebs_delete()Thanks,
NamhyungQuoted 17 lines
tools/perf/util/intel-tpebs.h new file +2
-0+ +#endif tools/perf/util/stat.h +1
-1@@ -110,6 +110,7 @@ struct perf_stat_config {struct cpu_aggr_map *cpus_aggr_map;u64 *walltime_run;struct rblist metric_events; +pid_t tpebs_pid;int ctl_fd;int ctl_fd_ack;bool ctl_fd_close; --2.43.0
RE: [RFC PATCH v9 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.
In reply to Namhyung KimQuoted 411 lines
-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 24, 2024 4:02 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v9 3/7] perf stat: Fork and launch perf record when perf stat needs to get retire latency value for a metric.Hello,
Quoted 9 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
When
retire_latencyvalue is used in a metric formula, perf stat would fork a perf record process with "-e" and "-W" options. Perf record will collect requiredretire_latencyvalues in parallel while perf stat is collecting counting values.At the point of time that perf stat stops counting, it would send sigterm
signal
Quoted 1 line
to perf record process and receiving sampling data back from perf record
from a
Quoted 11 lines
pipe. Perf stat will then process the received data to get retire latency data and calculate metric result.
Another thread is required to synchronize between perf stat and perf record when we pass data through pipe.
Signed-off-by Weilin Wang <[email protected]>--- tools/perf/builtin-stat.c | 17 ++ tools/perf/util/Build | 1 + tools/perf/util/intel-tpebs.c | 301++++++++++++++++++++++++++++++++++
Quoted 19 lines
tools/perf/util/intel-tpebs.h | 30 ++++ tools/perf/util/stat.h | 1 + 5 files changed, 350 insertions(+) create mode 100644 tools/perf/util/intel-tpebs.c create mode 100644 tools/perf/util/intel-tpebs.htools/perf/builtin-stat.c +1
-0@@ -70,6 +70,7 @@#include "util/bpf_counter.h" #include "util/iostat.h" #include "util/util.h" +#include "util/intel-tpebs.h" #include "asm/bug.h" #include <linux/time64.h> tools/perf/builtin-stat.c static struct perf_stat_config stat_config = { · line 166
Quoted 4 lines
tools/perf/builtin-stat.c +1
-0@@ -162,6 +163,7 @@ static struct perf_stat_config stat_config = { .ctl_fd = -1, .ctl_fd_ack = -1, .iostat_run = false, + .tpebs_pid = -1,Where is this set?
tools/perf/builtin-stat.c static enum counter_recovery
Quoted 4 lines
tools/perf/builtin-stat.c +0
-0};static void evlist__check_cpu_maps (struct evlist *evlist)@@ -653,6 +655,8 @@ static enum counter_recovery
stat_handle_error(struct evsel *counter)tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char
Quoted 9 lines
tools/perf/builtin-stat.c +2
-0if (child_pid !=-1 )kill (child_pid,SIGTERM ); +if (stat_config.tpebs_pid !=-1 ) +stop_tpebs ();return COUNTER_FATAL ; }@@ -673,6 +677,10 @@ static int __run_perf_stat(int argc, const char**argv, int
run_idx)tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char **argv, int run_idx)
Quoted 9 lines
tools/perf/builtin-stat.c +4
-0int err;bool second_pass = false; + err =start_tpebs (&stat_config, evsel_list); +if (err <0 ) +return err; +if (forks) { if (evlist__prepare_workload(evsel_list, &target, argv, is_pipe,
workload_exec_failed_signal) < 0) {tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char
Quoted 2 lines
tools/perf/builtin-stat.c +0
-0perror ("failed to prepare workload" );@@ -878,6 +886,10 @@ static int __run_perf_stat(int argc, const char**argv, int
run_idx)Quoted 21 lines
tools/perf/builtin-stat.c +7
-0t1 =rdclock (); + err =stop_tpebs (); +if (err <0 ) +return err; +if (stat_config.walltime_run_table) stat_config.walltime_run[run_idx] = t1 - t0;@@ -985,6 +997,9 @@ static void sig_atexit(void)if (child_pid !=-1 )kill (child_pid,SIGTERM ); +if (stat_config.tpebs_pid !=-1 ) +kill (stat_config.tpebs_pid,SIGTERM ); +sigprocmask (SIG_SETMASK , &oset,NULL );if (signr ==-1 )tools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv) · line 2934
Quoted 2 lines
tools/perf/builtin-stat.c +0
-0@@ -2918,5 +2933,7 @@ int cmd_stat(int argc, const char **argv)metricgroup__rblist_exit (&stat_config.metric_events); evlist__close_control(stat_config.ctl_fd, stat_config.ctl_fd_ack,&stat_config.ctl_fd_close);
Quoted 10 lines
tools/perf/builtin-stat.c +2
-0+tpebs_data__delete (); +return status; }tools/perf/util/Build +0
-0tools/perf/util/Build perf-y += clockid.o · line 156
Quoted 4 lines
tools/perf/util/Build +1
-0@@ -153,6 +153,7 @@ perf-y += clockid.o perf-y += list_sort.o perf-y += mutex.o perf-y += sharded_mutex.o +perf-y += intel-tpebs.oCan we make this Intel (or x86) only?
Quoted 26 lines
tools/perf/util/Build +0
-0perf-$(CONFIG_LIBBPF) += bpf_map.o perf-$(CONFIG_PERF_BPF_SKEL) += bpf_counter.otools/perf/util/intel-tpebs.c new file +17
-0@@ -0,0 +1,301 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* intel_tpebs.c: Intel TPEBS support +*/ + + +#include <sys/param.h> +#include <subcmd/run-command.h> +#include <thread.h> +#include "intel-tpebs.h" +#include <linux/list.h> +#include <linux/zalloc.h> +#include <linux/err.h> +#include "sample.h" +#include "debug.h" +#include "evlist.h" +#include "evsel.h" tools/perf/util/intel-tpebs.c line 27
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+#include "session.h" +#include "tool.h" +#include "metricgroup.h" +#include <sys/stat.h> +#include <sys/file.h> +#include <poll.h> + +#define PERF_DATA "-" + +struct list_head tpebs_results =LIST_HEAD_INIT (tpebs_results);static
LIST_HEAD(tpebs_results);tools/perf/util/intel-tpebs.c line 29
Quoted 2 lines
tools/perf/util/intel-tpebs.c new file +2
-0+static pthread_t reader_thread; +static struct child_process *cmd;Maybe better to have the '
tpebs_' prefix.tools/perf/util/intel-tpebs.c line 30
Quoted 1 line
tools/perf/util/intel-tpebs.c new file +1
-0+static struct perf_stat_config *stat_config;Is this really needed? ...
tools/perf/util/intel-tpebs.c line 33
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+static size_t tpebs_event_size; + +static int get_perf_record_args (const char **record_argv,char buf[])... I think you can just pass the
stat_configargument fromstart_tpebs().Quoted 44 lines
tools/perf/util/intel-tpebs.c new file +21
-0+{ + struct tpebs_retire_lat *e; + int i = 0; + + pr_debug("Prepare perf record for retire_latency\n"); + + record_argv[i++] = "perf"; + record_argv[i++] = "record"; + record_argv[i++] = "-W"; + record_argv[i++] = "--synth=no"; + record_argv[i++] = buf; + + if (stat_config->user_requested_cpu_list) { + record_argv[i++] = "-C"; + record_argv[i++] = stat_config->user_requested_cpu_list; + } + + if (stat_config->system_wide) + record_argv[i++] = "-a"; + + if (!stat_config->system_wide && !stat_config-
user_requested_cpu_list) {tools/perf/util/intel-tpebs.c new file +22
-0+pr_err ("Require -a or -C option to run sampling.\n" ); +return -ECANCELED ; + } + +list_for_each_entry (e, &tpebs_results, nd) { + record_argv[i++] ="-e" ; + record_argv[i++] = e->name; + } + + record_argv[i++] ="-o" ; + record_argv[i++] =PERF_DATA ; + +return 0 ; +} + +static int prepare_run_command (const char **argv) +{ + cmd =zalloc (sizeof (struct child_process )); +if (!cmd) +return -ENOMEM ; + cmd->argv = argv; + cmd->out =-1 ;tools/perf/util/intel-tpebs.c
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+return 0 ; +} + +static int prepare_perf_record (int control_fd[],int ack_fd[]) +{ +const char **record_argv; +int ret; +char buf[32 ]; + + scnprintf(buf, sizeof(buf), "--control=fd:%d,%d", control_fd[0],ack_fd[1]);
Quoted 7 lines
tools/perf/util/intel-tpebs.c new file +7
-0+ + record_argv =calloc (12 +2 * tpebs_event_size,sizeof (char *)); +if (!record_argv) +return -ENOMEM ; + + ret =get_perf_record_args (record_argv, buf); +if (ret)tools/perf/util/intel-tpebs.c line 103
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ goto out; + + ret =prepare_run_command (record_argv); +if (ret) + goto out; + ret =start_command (cmd); +out: +free (record_argv); +return ret; +}Please add a blank line.
tools/perf/util/intel-tpebs.c line 106
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+struct sample_data_reader { +struct perf_tool tool; +struct perf_session *session;You don't need this, right? Why not pass the 'tool' directly?
Quoted 24 lines
tools/perf/util/intel-tpebs.c new file +24
-0+}; + +static void tpebs_retire_lat__delete (struct tpebs_retire_lat *r) +{ +zfree (&r->name); +zfree (&r->tpebs_name); +free (r); +} + +void tpebs_data__delete (void ) +{ +struct tpebs_retire_lat *r, *rtmp; + +list_for_each_entry_safe (r, rtmp, &tpebs_results, nd) { +list_del_init (&r->nd); +tpebs_retire_lat__delete (r); + } +free (cmd); +} + +static int process_sample_event(struct perf_tool *tool __maybe_unused, + union perf_event *event __maybe_unused, + struct perf_sample *sample, + struct evsel *evsel,tools/perf/util/intel-tpebs.c line 140
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ struct machine *machine __maybe_unused) +{ +int ret =0 ; +const char *evname; +struct tpebs_retire_lat *t; + + evname =evsel__name (evsel); + +/* +* Need to handle per core results? We are assuming average retire I don't know. It depends on the use case where you want different per-core retire-latency for the (per-core) metric.
Quoted 1 line
+ * latency value will be used. Save the number of samples and the sum
of
Quoted 26 lines
tools/perf/util/intel-tpebs.c new file +26
-0+* retire latency value for each event. +*/ +list_for_each_entry (t, &tpebs_results, nd) { +if (!strcmp (evname, t->name)) { + t->count +=1 ; + t->sum += sample->retire_lat; + t->val = (double ) t->sum / t->count; +break ; + } + } + +return ret; +} + +static int process_feature_event (struct perf_session *session, +union perf_event *event) +{ +if (event->feat.feat_id <HEADER_LAST_FEATURE ) +return perf_event__process_feature (session, event); +return 0 ; +} + +static void *__sample_reader (void *arg) +{ +struct child_process *child = arg; +struct perf_session *session;tools/perf/util/intel-tpebs.c line 177
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+struct perf_data data = { + .mode =PERF_DATA_MODE_READ , + .path =PERF_DATA , + .file.fd = child->out, + }; +struct sample_data_reader reader = { + .tool = { + .sample = process_sample_event, + .feature = process_feature_event, + .attr = perf_event__process_attr,Strange indentations.
Quoted 5 lines
tools/perf/util/intel-tpebs.c new file +5
-0+ }, + }; + + session =perf_session__new (&data, &reader.tool); +if (IS_ERR (session))tools/perf/util/intel-tpebs.c
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+return NULL ; + reader.session = session; +perf_session__process_events (session); +perf_session__delete (session); + +return NULL ; +} + + +int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist
*evsel_list)tools/perf/util/intel-tpebs.c line 203
Quoted 11 lines
tools/perf/util/intel-tpebs.c new file +11
-0+{ +int ret =0 ; +struct evsel *evsel; + + stat_config = perf_stat_config; +/* +* Prepare perf record for sampling event retire_latency before fork and +* prepare workload +*/ +evlist__for_each_entry (evsel_list, evsel) { +if (evsel->retire_lat) {To reduce the indentation, consider early continue like
if (!evsel->retire_lat) continue;Quoted 1 line
+ struct
tpebs_retire_lat*new = malloc(sizeof(struct
tpebs_retire_lat));I prefer
sizeof(*new)instead but it doesn't matter. But you might usezalloc()and forget about the 0 initialization later.tools/perf/util/intel-tpebs.c line 213
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +4
-0+int i; +char *name; + + pr_debug("perf stat retire latency %s required\n", evsel-name);
tools/perf/util/intel-tpebs.c new file +5
-0+if (!new) +return -1 ; +for (i =strlen (evsel->name) -1 ; i >0 ; i--) { +if (evsel->name[i] =='R' ) +break ;
Quoted 2 lines
I remember Ian checked '/' and ':' too. Otherwise it can detect R in the event name and treat it as a modifier.
Sorry, I forgot this one in last email. This patch set depends on Ian's 'R' parser code patch. When we reach here, the evsel->name should be in the format like event_name:R or cpu@event_name@R. This code is to replace 'R' with 'p' for perf record. Therefore, I think we just need to find the position of the last 'R'.
Thanks,
Weilin
Quoted 170 lines
tools/perf/util/intel-tpebs.c line 216
Quoted 4 lines
tools/perf/util/intel-tpebs.c new file +3
-0+ } +if (i <=0 || evsel->name[i] !='R' ) +return -1 ;You need to free the memory here and other places.
tools/perf/util/intel-tpebs.c line 223
Quoted 7 lines
tools/perf/util/intel-tpebs.c new file +7
-0+ + name =strdup (evsel->name); +if (!name) +return -ENOMEM ; + name[i] ='p' ; + new->name =strdup (name); +free (name);Seems like an unnecessary allocation. Why not use 'name' directly?
tools/perf/util/intel-tpebs.c line 226
Quoted 3 lines
tools/perf/util/intel-tpebs.c new file +3
-0+ new->tpebs_name =strdup (evsel->name); +if (!new->tpebs_name) +return -ENOMEM ;Maybe
orig_name? But I'm not sure if it's really needed.. Can we just teach perf record to understand 'R' and to act like 'p'?Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ new->count =0 ; + new->sum =0 ; +list_add_tail (&new->nd, &tpebs_results); + tpebs_event_size +=1 ; + } + } + +if (tpebs_event_size >0 ) { +struct pollfd pollfd = { .events =POLLIN , }; +int control_fd[2 ], ack_fd[2 ], len;tools/perf/util/intel-tpebs.c line 246
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+char ack_buf[8 ]; + +/*Create control and ack fd for --control*/ +if (pipe (control_fd) <0 ) { +pr_err ("Failed to create control fifo" ); +return -1 ; + } +if (pipe (ack_fd) <0 ) { +pr_err ("Failed to create control fifo" ); +return -1 ;Need to close the pipes here and other places.
Quoted 12 lines
tools/perf/util/intel-tpebs.c new file +12
-0+ } + + ret =prepare_perf_record (control_fd, ack_fd); +if (ret) +return ret; +if (pthread_create (&reader_thread,NULL , __sample_reader, cmd)) { +kill (cmd->pid,SIGTERM ); +close (cmd->out); +pr_err ("Could not create thread to process sample data.\n" ); +return -1 ; + } +/* Wait for perf record initialization.*/ tools/perf/util/intel-tpebs.c line 268
Quoted 10 lines
tools/perf/util/intel-tpebs.c new file +10
-0+ len =strlen ("enable" ); + ret =write (control_fd[1 ],"enable" , len); +if (ret != len) { +pr_err ("perf record control write control message failed\n" ); + goto out; + } + + pollfd.fd = ack_fd[0 ]; +if (!poll (&pollfd,1 ,2000 )) + goto out;Do we need this? Why not just read?
Quoted 55 lines
tools/perf/util/intel-tpebs.c new file +33
-0+ + ret =read (ack_fd[0 ], ack_buf,sizeof (ack_buf)); +if (ret >0 ) + ret =strcmp (ack_buf,"ack\n" ); +else { +pr_err ("perf record control ack failed\n" ); + goto out; + } +pr_debug ("Received ack from perf record\n" ); +out: +close (control_fd[0 ]); +close (control_fd[1 ]); +close (ack_fd[0 ]); +close (ack_fd[1 ]); + } +return ret; +} + + +int stop_tpebs (void ) +{ +int ret =0 ; + +if (tpebs_event_size >0 ) { +kill (cmd->pid,SIGTERM ); +pthread_join (reader_thread,NULL ); +close (cmd->out); + ret =finish_command (cmd); +if (ret == -ERR_RUN_COMMAND_WAITPID_SIGNAL ) + ret =0 ; + } +return ret; +}tools/perf/util/intel-tpebs.h new file +16
-0@@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* intel_tpebs.h: Intel TEPBS support +*/ +#include "stat.h" + +#ifndef INCLUDE__PERF_INTEL_TPEBS_H__ +#define INCLUDE__PERF_INTEL_TPEBS_H__ + +struct tpebs_retire_lat { + struct list_head nd; + /* Event name */ + const char *name; + /* Event name with the TPEBS modifier R */ + const char *tpebs_name; + /* Count of retire_latency values found in sample data */tools/perf/util/intel-tpebs.h line 26
Quoted 10 lines
tools/perf/util/intel-tpebs.h new file +10
-0+size_t count; +/* Sum of all the retire_latency values in sample data */ +int sum; +/* Average of retire_latency, val = sum / count */ +double val; +}; + +extern struct list_head tpebs_results; + +int start_tpebs(struct perf_stat_config *perf_stat_config, struct evlist
*evsel_list);tools/perf/util/intel-tpebs.h line 28
Quoted 2 lines
tools/perf/util/intel-tpebs.h new file +2
-0+int stop_tpebs (void ); +void tpebs_data__delete (void );I think it's better to have the same prefix.
tpebs_start()tpebs_stop()tpebs_delete()Thanks,
NamhyungQuoted 17 lines
tools/perf/util/intel-tpebs.h new file +2
-0+ +#endif tools/perf/util/stat.h +1
-1@@ -110,6 +110,7 @@ struct perf_stat_config {struct cpu_aggr_map *cpus_aggr_map;u64 *walltime_run;struct rblist metric_events; +pid_t tpebs_pid;int ctl_fd;int ctl_fd_ack;bool ctl_fd_close; --2.43.0
RFC PATCH v9 4/7 perf stat: Plugin retire_lat value from sampled data to evsel
In reply to [email protected]From: Weilin Wang <[email protected]>
In current :R parsing implementation, the parser would recognize events with retire_latency modifier and insert them into the evlist like a normal event. Ideally, we need to avoid counting these events.
In this commit, at the time when a retire_latency evsel is read, set the retire latency value processed from the sampled data to count value. This sampled retire latency value will be used for metric calculation and final event count print out.
--- tools/perf/arch/x86/util/evlist.c | 6 +++++ tools/perf/util/evsel.c | 44 +++++++++++++++++++++++++++++++ tools/perf/util/evsel.h | 5 ++++ 3 files changed, 55 insertions(+)
tools/perf/arch/x86/util/evlist.c +6 -0
@@ -89,6 +89,12 @@ int arch_evlist__cmp(const struct evsel *lhs, const struct evsel *rhs)return 1 ; } +/* Retire latency event should not be group leader*/ +if (lhs->retire_lat && !rhs->retire_lat) +return 1 ; +if (!lhs->retire_lat && rhs->retire_lat) +return -1 ; +/* Default ordering by insertion index. */ return lhs->core.idx - rhs->core.idx; }
tools/perf/util/evsel.c +44 -0
68 lines not on this page · raw message
tools/perf/util/evsel.h +5 -0
@@ -303,6 +303,11 @@ static inline bool evsel__is_tool(const struct evsel *evsel)return evsel->tool_event !=PERF_TOOL_NONE ; } +static inline bool evsel__is_retire_lat (const struct evsel *evsel) +{ +return evsel->retire_lat; +} +const char *evsel__group_name (struct evsel *evsel);int evsel__group_desc (struct evsel *evsel,char *buf,size_t size);
Signature · 1 line
2.43.0
Namhyung Kim
maintainerRe: [RFC PATCH v9 4/7] perf stat: Plugin retire_lat value from sampled data to evsel
Quoted 11 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
In current :R parsing implementation, the parser would recognize events with
retire_latencymodifier and insert them into the evlist like a normal event. Ideally, we need to avoid counting these events.In this commit, at the time when a
retire_latencyevsel is read, set the retire latency value processed from the sampled data to count value. This sampled retire latency value will be used for metric calculation and final event count print out.
I'm confused. Do you mean you don't count the event with 'R' modifier (w/ perf stat) and just print the (average) retire latency (from perf record)?
Quoted 13 lines
Signed-off-by Weilin Wang <[email protected]>--- tools/perf/arch/x86/util/evlist.c | 6 +++++ tools/perf/util/evsel.c | 44 +++++++++++++++++++++++++++++++ tools/perf/util/evsel.h | 5 ++++ 3 files changed, 55 insertions(+)tools/perf/arch/x86/util/evlist.c +0
-0
tools/perf/arch/x86/util/evlist.c int arch_evlist__cmp(const struct evsel *lhs, const struct evsel *rhs) · line 92
Quoted 4 lines
tools/perf/arch/x86/util/evlist.c +1
-0@@ -89,6 +89,12 @@ int arch_evlist__cmp(const struct evsel *lhs, const struct evsel *rhs)return 1 ; } +/* Retire latency event should not be group leader*/
Hmm.. why?
Quoted 45 lines
tools/perf/arch/x86/util/evlist.c +5
-0+if (lhs->retire_lat && !rhs->retire_lat) +return 1 ; +if (!lhs->retire_lat && rhs->retire_lat) +return -1 ; +/* Default ordering by insertion index. */ return lhs->core.idx - rhs->core.idx; }tools/perf/util/evsel.c +22
-0@@ -58,6 +58,7 @@#include <internal/xyarray.h> #include <internal/lib.h> #include <internal/threadmap.h> +#include "util/intel-tpebs.h" #include <linux/ctype.h> @@ -1523,6 +1524,40 @@ static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread)return perf_evsel__read (&evsel->core, cpu_map_idx, thread, count); } +static int evsel__set_retire_lat (struct evsel *evsel,int cpu_map_idx,int thread) +{ +struct perf_counts_values *count; +struct tpebs_retire_lat *t; +bool found = false; +__u64 val; + + count =perf_counts (evsel->counts, cpu_map_idx, thread); + +list_for_each_entry (t, &tpebs_results, nd) { +if (!strcmp (t->tpebs_name, evsel->name)) { + found = true; +break ; + } + } + +if (!found) +return -1 ; + +/* +* Only set retire_latency value to the first CPU and thread.
tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread) · line 1557
Quoted 10 lines
tools/perf/util/evsel.c +10
-0+*/ +if (cpu_map_idx ==0 && thread ==0 ) + val = t->val; +else + val =0 ; + + count->val = val; +/* Set ena and run to non-zero */ + count->ena = count->run =1 ; + count->lost =0 ;
So here it seems you discard the actual count of the events and replace it with the retire latency. That means you don't need to open the event in perf stat, and probably just have a placeholder, right?
Btw, I think it's better to move this logic to intel-tpebs.c file and rename to tpebs_set_retire_lat().
tools/perf/util/evsel.c static void evsel__set_count(struct evsel *counter, int cpu_map_idx, int thread, · line 1568
Quoted 11 lines
tools/perf/util/evsel.c +4
-0+return 0 ; +} +static void evsel__set_count (struct evsel *counter,int cpu_map_idx,int thread,u64 val,u64 ena,u64 run,u64 lost) {@@ -1530,6 +1565,12 @@ static void evsel__set_count(struct evsel *counter, int cpu_map_idx, int thread, count =perf_counts (counter->counts, cpu_map_idx, thread); +if (counter->retire_lat) {
if (evsel__is_retire_lat(counter)) ?
Quoted 9 lines
tools/perf/util/evsel.c +5
-0+evsel__set_retire_lat (counter, cpu_map_idx, thread); +perf_counts__set_loaded (counter->counts, cpu_map_idx, thread, true); +return ; + } + count->val = val; count->ena = ena; count->run = run;
tools/perf/util/evsel.c int evsel__read_counter(struct evsel *evsel, int cpu_map_idx, int thread) · line 1824
Quoted 6 lines
tools/perf/util/evsel.c +3
-0@@ -1778,6 +1819,9 @@ int evsel__read_counter(struct evsel *evsel, int cpu_map_idx, int thread)if (evsel__is_tool (evsel))return evsel__read_tool (evsel, cpu_map_idx, thread); +if (evsel__is_retire_lat (evsel)) +return evsel__set_retire_lat (evsel, cpu_map_idx, thread); +
I'm not sure if it works well with group event. Probably that's why you wanted to prevent group leaders. But I guess you can just check this after the PERF_FORMAT_GROUP, no?
Thanks,
Namhyung
Quoted 22 lines
tools/perf/util/evsel.c +0
-0if (evsel->core.attr.read_format &PERF_FORMAT_GROUP )return evsel__read_group (evsel, cpu_map_idx, thread);tools/perf/util/evsel.h +5
-1@@ -303,6 +303,11 @@ static inline bool evsel__is_tool(const struct evsel *evsel)return evsel->tool_event !=PERF_TOOL_NONE ; } +static inline bool evsel__is_retire_lat (const struct evsel *evsel) +{ +return evsel->retire_lat; +} +const char *evsel__group_name (struct evsel *evsel);int evsel__group_desc (struct evsel *evsel,char *buf,size_t size); --2.43.0
RE: [RFC PATCH v9 4/7] perf stat: Plugin retire_lat value from sampled data to evsel
Quoted 26 lines
-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 24, 2024 4:17 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v9 4/7] perf stat: Pluginretire_latvalue from sampled data to evselQuoted 4 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
In current :R parsing implementation, the parser would recognize events
with
Quoted 3 lines
retire_latencymodifier and insert them into the evlist like a normal event. Ideally, we need to avoid counting these events.
Quoted 7 lines
Quoted 4 lines
In this commit, at the time when a
retire_latencyevsel is read, set the retire latency value processed from the sampled data to count value. This sampled retire latency value will be used for metric calculation and final event count print out.I'm confused. Do you mean you don't count the event with 'R' modifier (w/ perf stat) and just print the (average) retire latency (from perf record)?
In metric formulas, event without 'R' modifier is included as a normal event already. So we don't need to count the event that with 'R' modifier. They only need to be sampled.
Quoted 16 lines
Review
Quoted 10 lines
Signed-off-by Weilin Wang <[email protected]>--- tools/perf/arch/x86/util/evlist.c | 6 +++++ tools/perf/util/evsel.c | 44 +++++++++++++++++++++++++++++++ tools/perf/util/evsel.h | 5 ++++ 3 files changed, 55 insertions(+)(quoted diff) +0
-0
b/tools/perf/arch/x86/util/evlist.ctools/perf/arch/x86/util/evlist.c int arch_evlist__cmp(const struct evsel *lhs, const
Quoted 4 lines
index
b1ce0c52d88d..cebdd483149e100644tools/perf/arch/x86/util/evlist.c +0
-0@@ -89,6 +89,12 @@ int arch_evlist__cmp(const struct evsel *lhs, conststruct evsel *rhs)
Quoted 6 lines
tools/perf/arch/x86/util/evlist.c int arch_evlist__cmp(const struct evsel *lhs, const struct evsel *rhs) · line 92
Quoted 4 lines
tools/perf/arch/x86/util/evlist.c +1
-0return 1 ; } +/* Retire latency event should not be group leader*/ Hmm.. why?
Because we don't want to count them. Make them the group leader would not work.
Quoted 64 lines
tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel,
Quoted 22 lines
tools/perf/arch/x86/util/evlist.c +5
-0+if (lhs->retire_lat && !rhs->retire_lat) +return 1 ; +if (!lhs->retire_lat && rhs->retire_lat) +return -1 ; +/* Default ordering by insertion index. */ return lhs->core.idx - rhs->core.idx; }tools/perf/util/evsel.c +1
-0@@ -58,6 +58,7 @@#include <internal/xyarray.h> #include <internal/lib.h> #include <internal/threadmap.h> +#include "util/intel-tpebs.h" #include <linux/ctype.h> @@ -1523,6 +1524,40 @@ static int evsel__read_one(struct evsel *evsel,int
cpu_map_idx, int thread)tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread)
Quoted 4 lines
tools/perf/util/evsel.c +1
-0return perf_evsel__read (&evsel->core, cpu_map_idx, thread, count); } +static int evsel__set_retire_lat(struct evsel *evsel, int cpu_map_idx, intthread)
Quoted 20 lines
tools/perf/util/evsel.c +20
-0+{ +struct perf_counts_values *count; +struct tpebs_retire_lat *t; +bool found = false; +__u64 val; + + count =perf_counts (evsel->counts, cpu_map_idx, thread); + +list_for_each_entry (t, &tpebs_results, nd) { +if (!strcmp (t->tpebs_name, evsel->name)) { + found = true; +break ; + } + } + +if (!found) +return -1 ; + +/* +* Only set retire_latency value to the first CPU and thread. tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread) · line 1557
Quoted 10 lines
tools/perf/util/evsel.c +10
-0+*/ +if (cpu_map_idx ==0 && thread ==0 ) + val = t->val; +else + val =0 ; + + count->val = val; +/* Set ena and run to non-zero */ + count->ena = count->run =1 ; + count->lost =0 ;So here it seems you discard the actual count of the events and replace it with the retire latency. That means you don't need to open the event in perf stat, and probably just have a placeholder, right?
Quoted 2 lines
Btw, I think it's better to move this logic to
intel-tpebs.cfile and rename totpebs_set_retire_lat().
Ian wants this to be here and also suggested me to rename this function to evsel__read_retire_lat(). I'm ok with either way.
Thanks,
Weilin
Quoted 68 lines
tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread)
Quoted 6 lines
tools/perf/util/evsel.c +3
-0+return 0 ; +} + static void evsel__set_count(struct evsel *counter, int cpu_map_idx, intthread,
tools/perf/util/evsel.c static void evsel__set_count(struct evsel
Quoted 3 lines
tools/perf/util/evsel.c +0
-0u64 val, u64 ena, u64 run, u64 lost) {@@ -1530,6 +1565,12 @@ static void evsel__set_count(struct evsel*counter, int
cpu_map_idx, int thread,tools/perf/util/evsel.c static void evsel__set_count(struct evsel *counter, int cpu_map_idx, int thread, · line 1568
Quoted 4 lines
tools/perf/util/evsel.c +1
-0count =perf_counts (counter->counts, cpu_map_idx, thread); +if (counter->retire_lat) {if (
evsel__is_retire_lat(counter)) ?tools/perf/util/evsel.c static void evsel__set_count(struct evsel · line 1569
Quoted 2 lines
tools/perf/util/evsel.c +2
-0+evsel__set_retire_lat (counter, cpu_map_idx, thread); + perf_counts__set_loaded(counter->counts, cpu_map_idx, thread,true);
tools/perf/util/evsel.c int evsel__read_counter(struct evsel *evsel, int
Quoted 7 lines
tools/perf/util/evsel.c +3
-0+return ; + } + count->val = val; count->ena = ena; count->run = run;@@ -1778,6 +1819,9 @@ int evsel__read_counter(struct evsel *evsel, int
cpu_map_idx, int thread)tools/perf/util/evsel.c int evsel__read_counter(struct evsel *evsel, int cpu_map_idx, int thread) · line 1824
Quoted 6 lines
tools/perf/util/evsel.c +3
-0if (evsel__is_tool (evsel))return evsel__read_tool (evsel, cpu_map_idx, thread); +if (evsel__is_retire_lat (evsel)) +return evsel__set_retire_lat (evsel, cpu_map_idx, thread); +I'm not sure if it works well with group event. Probably that's why you wanted to prevent group leaders. But I guess you can just check this after the
PERF_FORMAT_GROUP, no?Thanks,
Namhyungtools/perf/util/evsel.h static inline bool evsel__is_tool(const struct evsel
Quoted 8 lines
tools/perf/util/evsel.c +0
-0if (evsel->core.attr.read_format &PERF_FORMAT_GROUP )return evsel__read_group (evsel, cpu_map_idx, thread);tools/perf/util/evsel.h +0
-0@@ -303,6 +303,11 @@ static inline bool evsel__is_tool(const struct evsel*evsel)
Quoted 14 lines
tools/perf/util/evsel.h +5
-1return evsel->tool_event !=PERF_TOOL_NONE ; } +static inline bool evsel__is_retire_lat (const struct evsel *evsel) +{ +return evsel->retire_lat; +} +const char *evsel__group_name (struct evsel *evsel);int evsel__group_desc (struct evsel *evsel,char *buf,size_t size); --2.43.0
Namhyung Kim
maintainerRe: [RFC PATCH v9 4/7] perf stat: Plugin retire_lat value from sampled data to evsel
Quoted 32 lines · Weilin
Quoted 32 lines
-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 24, 2024 4:17 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v9 4/7] perf stat: Pluginretire_latvalue from sampled data to evselQuoted 4 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
In current :R parsing implementation, the parser would recognize events
with
Quoted 7 lines
retire_latencymodifier and insert them into the evlist like a normal event. Ideally, we need to avoid counting these events.In this commit, at the time when a
retire_latencyevsel is read, set the retire latency value processed from the sampled data to count value. This sampled retire latency value will be used for metric calculation and final event count print out.
Quoted 6 lines
Quoted 2 lines
I'm confused. Do you mean you don't count the event with 'R' modifier (w/ perf stat) and just print the (average) retire latency (from perf record)?
In metric formulas, event without 'R' modifier is included as a normal event already. So we don't need to count the event that with 'R' modifier. They only need to be sampled.
Oh, you have the event in the metric expression twice. I thought of one. Then IIUC the metric looks something like this.
myevent1 + (myevent2 * myevent1:R)
I think you'll have 2 myevent1 in perf stat and 1 in perf record, right? But the second one in perf stat is never used and the value is updated from perf record.
Then we can simply remove the event from the evlist (or replace it with a dummy) to reduce the overheads (of open and read).
Quoted 17 lines
Quoted 17 lines
Review
Quoted 11 lines
Signed-off-by Weilin Wang <[email protected]>--- tools/perf/arch/x86/util/evlist.c | 6 +++++ tools/perf/util/evsel.c | 44 +++++++++++++++++++++++++++++++ tools/perf/util/evsel.h | 5 ++++ 3 files changed, 55 insertions(+)(quoted diff) +0
-0
b/tools/perf/arch/x86/util/evlist.ctools/perf/arch/x86/util/evlist.c int arch_evlist__cmp(const struct evsel *lhs, const
Quoted 4 lines
index
b1ce0c52d88d..cebdd483149e100644tools/perf/arch/x86/util/evlist.c +0
-0@@ -89,6 +89,12 @@ int arch_evlist__cmp(const struct evsel *lhs, conststruct evsel *rhs)
Quoted 7 lines
Quoted 6 lines
tools/perf/arch/x86/util/evlist.c int arch_evlist__cmp(const struct evsel *lhs, const struct evsel *rhs) · line 92
Quoted 4 lines
tools/perf/arch/x86/util/evlist.c +1
-0return 1 ; } +/* Retire latency event should not be group leader*/ Hmm.. why?
Because we don't want to count them. Make them the group leader would not work.
I don't understand. You'll read the event regardless of being a leader or not.
Quoted 65 lines
Quoted 65 lines
tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel,
Quoted 23 lines
tools/perf/arch/x86/util/evlist.c +5
-0+if (lhs->retire_lat && !rhs->retire_lat) +return 1 ; +if (!lhs->retire_lat && rhs->retire_lat) +return -1 ; +/* Default ordering by insertion index. */ return lhs->core.idx - rhs->core.idx; }tools/perf/util/evsel.c +1
-0@@ -58,6 +58,7 @@#include <internal/xyarray.h> #include <internal/lib.h> #include <internal/threadmap.h> +#include "util/intel-tpebs.h" #include <linux/ctype.h> @@ -1523,6 +1524,40 @@ static int evsel__read_one(struct evsel *evsel,int
cpu_map_idx, int thread)tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread)
Quoted 4 lines
tools/perf/util/evsel.c +1
-0return perf_evsel__read (&evsel->core, cpu_map_idx, thread, count); } +static int evsel__set_retire_lat(struct evsel *evsel, int cpu_map_idx, intthread)
Quoted 20 lines
tools/perf/util/evsel.c +20
-0+{ +struct perf_counts_values *count; +struct tpebs_retire_lat *t; +bool found = false; +__u64 val; + + count =perf_counts (evsel->counts, cpu_map_idx, thread); + +list_for_each_entry (t, &tpebs_results, nd) { +if (!strcmp (t->tpebs_name, evsel->name)) { + found = true; +break ; + } + } + +if (!found) +return -1 ; + +/* +* Only set retire_latency value to the first CPU and thread. tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread) · line 1557
Quoted 10 lines
tools/perf/util/evsel.c +10
-0+*/ +if (cpu_map_idx ==0 && thread ==0 ) + val = t->val; +else + val =0 ; + + count->val = val; +/* Set ena and run to non-zero */ + count->ena = count->run =1 ; + count->lost =0 ;So here it seems you discard the actual count of the events and replace it with the retire latency. That means you don't need to open the event in perf stat, and probably just have a placeholder, right?
Quoted 5 lines
Quoted 2 lines
Btw, I think it's better to move this logic to
intel-tpebs.cfile and rename totpebs_set_retire_lat().Ian wants this to be here and also suggested me to rename this function to
evsel__read_retire_lat(). I'm ok with either way.
I think it's better to have the tpebs logic together.
Thanks,
Namhyung
Quoted 69 lines
Quoted 69 lines
tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread)
Quoted 7 lines
tools/perf/util/evsel.c +3
-0+return 0 ; +} + static void evsel__set_count(struct evsel *counter, int cpu_map_idx, intthread,
tools/perf/util/evsel.c static void evsel__set_count(struct evsel
Quoted 3 lines
tools/perf/util/evsel.c +0
-0u64 val, u64 ena, u64 run, u64 lost) {@@ -1530,6 +1565,12 @@ static void evsel__set_count(struct evsel*counter, int
cpu_map_idx, int thread,tools/perf/util/evsel.c static void evsel__set_count(struct evsel *counter, int cpu_map_idx, int thread, · line 1568
Quoted 4 lines
tools/perf/util/evsel.c +1
-0count =perf_counts (counter->counts, cpu_map_idx, thread); +if (counter->retire_lat) {if (
evsel__is_retire_lat(counter)) ?tools/perf/util/evsel.c static void evsel__set_count(struct evsel · line 1569
Quoted 2 lines
tools/perf/util/evsel.c +2
-0+evsel__set_retire_lat (counter, cpu_map_idx, thread); + perf_counts__set_loaded(counter->counts, cpu_map_idx, thread,true);
tools/perf/util/evsel.c int evsel__read_counter(struct evsel *evsel, int
Quoted 7 lines
tools/perf/util/evsel.c +3
-0+return ; + } + count->val = val; count->ena = ena; count->run = run;@@ -1778,6 +1819,9 @@ int evsel__read_counter(struct evsel *evsel, int
cpu_map_idx, int thread)tools/perf/util/evsel.c int evsel__read_counter(struct evsel *evsel, int cpu_map_idx, int thread) · line 1824
Quoted 6 lines
tools/perf/util/evsel.c +3
-0if (evsel__is_tool (evsel))return evsel__read_tool (evsel, cpu_map_idx, thread); +if (evsel__is_retire_lat (evsel)) +return evsel__set_retire_lat (evsel, cpu_map_idx, thread); +I'm not sure if it works well with group event. Probably that's why you wanted to prevent group leaders. But I guess you can just check this after the
PERF_FORMAT_GROUP, no?Thanks,
Namhyungtools/perf/util/evsel.h static inline bool evsel__is_tool(const struct evsel
Quoted 8 lines
tools/perf/util/evsel.c +0
-0if (evsel->core.attr.read_format &PERF_FORMAT_GROUP )return evsel__read_group (evsel, cpu_map_idx, thread);tools/perf/util/evsel.h +0
-0@@ -303,6 +303,11 @@ static inline bool evsel__is_tool(const struct evsel*evsel)
Quoted 14 lines
tools/perf/util/evsel.h +5
-1return evsel->tool_event !=PERF_TOOL_NONE ; } +static inline bool evsel__is_retire_lat (const struct evsel *evsel) +{ +return evsel->retire_lat; +} +const char *evsel__group_name (struct evsel *evsel);int evsel__group_desc (struct evsel *evsel,char *buf,size_t size); --2.43.0
RE: [RFC PATCH v9 4/7] perf stat: Plugin retire_lat value from sampled data to evsel
Quoted 69 lines
-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Sunday, May 26, 2024 11:02 AM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v9 4/7] perf stat: Pluginretire_latvalue from sampled data to evselQuoted 13 lines · Weilin
Quoted 13 lines
-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 24, 2024 4:17 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor,Perry
Quoted 1 line
Quoted 1 line
<[email protected]>; Alt, Samantha <[email protected]>;
Biggers,
Quoted 2 lines
Quoted 2 lines
Caleb <[email protected]>
Subject: Re: [RFC PATCH v9 4/7] perf stat: Pluginretire_latvalue fromsampled
Quoted 4 lines
Quoted 4 lines
data to evsel
Quoted 1 line · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
From: Weilin Wang <[email protected]>
In current :R parsing implementation, the parser would recognize events
with
Quoted 4 lines
retire_latencymodifier and insert them into the evlist like a normal event. Ideally, we need to avoid counting these events.In this commit, at the time when a
retire_latencyevsel is read, set theretire
Quoted 1 line
Quoted 1 line
Quoted 1 line
latency value processed from the sampled data to count value. This
sampled
Quoted 1 line
Quoted 1 line
Quoted 1 line
retire latency value will be used for metric calculation and final event
count
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
print out.
I'm confused. Do you mean you don't count the event with 'R' modifier (w/ perf stat) and just print the (average) retire latency (from perf record)?
In metric formulas, event without 'R' modifier is included as a normal event
already.
Quoted 1 line
So we don't need to count the event that with 'R' modifier. They only need to
be
Quoted 1 line
sampled.
Oh, you have the event in the metric expression twice. I thought of one. Then IIUC the metric looks something like this.
Quoted 8 lines
myevent1 + (myevent2 * myevent1:R)
I think you'll have 2 myevent1 in perf stat and 1 in perf record, right? But the second one in perf stat is never used and the value is updated from perf record.
Then we can simply remove the event from the evlist (or replace it with a dummy) to reduce the overheads (of open and read).
Yes, you are right. Ideally, we want to not do the extra counting on the :R event. At the same time, I think Ian also wants to reuse code in evsel for the :R events, so that we don't need special code to handle retire_latency value printout and metric calculation. Therefore, I think we need to keep retire_latency events in evlist and set processed retire_latency values to the evsel.
If we replace it with a dummy for open and read and then set retire_latency value to it later, will it still be used in metric calculation and printout?
Thanks,
Weilin
Quoted 180 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Signed-off-by Weilin Wang <[email protected]>--- tools/perf/arch/x86/util/evlist.c | 6 +++++ tools/perf/util/evsel.c | 44+++++++++++++++++++++++++++++++
Quoted 10 lines
Quoted 10 lines
Review
Quoted 4 lines
tools/perf/util/evsel.h | 5 ++++ 3 files changed, 55 insertions(+)(quoted diff) +0
-0
b/tools/perf/arch/x86/util/evlist.ctools/perf/arch/x86/util/evlist.c int arch_evlist__cmp(const struct evsel *lhs, const
Quoted 4 lines
index
b1ce0c52d88d..cebdd483149e100644tools/perf/arch/x86/util/evlist.c +0
-0@@ -89,6 +89,12 @@ int arch_evlist__cmp(const struct evsel *lhs, conststruct evsel *rhs)
Quoted 7 lines
Quoted 6 lines
tools/perf/arch/x86/util/evlist.c int arch_evlist__cmp(const struct evsel *lhs, const struct evsel *rhs) · line 92
Quoted 4 lines
tools/perf/arch/x86/util/evlist.c +1
-0return 1 ; } +/* Retire latency event should not be group leader*/ Hmm.. why?
Because we don't want to count them. Make them the group leader would
not work.
I don't understand. You'll read the event regardless of being a leader or not.
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
tools/perf/arch/x86/util/evlist.c +5
-0+if (lhs->retire_lat && !rhs->retire_lat) +return 1 ; +if (!lhs->retire_lat && rhs->retire_lat) +return -1 ; +/* Default ordering by insertion index. */ return lhs->core.idx - rhs->core.idx; }tools/perf/util/evsel.c +1
-0@@ -58,6 +58,7 @@#include <internal/xyarray.h> #include <internal/lib.h> #include <internal/threadmap.h> +#include "util/intel-tpebs.h" Quoted 3 lines
Quoted 3 lines
tools/perf/util/evsel.c static int evsel__read_one(struct evsel
Quoted 3 lines
tools/perf/util/evsel.c +0
-0#include <linux/ctype.h> @@ -1523,6 +1524,40 @@ static int evsel__read_one(struct evsel*evsel,
Quoted 42 lines
Quoted 42 lines
int
cpu_map_idx, int thread)tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread)
Quoted 4 lines
tools/perf/util/evsel.c +1
-0return perf_evsel__read (&evsel->core, cpu_map_idx, thread, count); } +static int evsel__set_retire_lat(struct evsel *evsel, int cpu_map_idx, intthread)
Quoted 20 lines
tools/perf/util/evsel.c +20
-0+{ +struct perf_counts_values *count; +struct tpebs_retire_lat *t; +bool found = false; +__u64 val; + + count =perf_counts (evsel->counts, cpu_map_idx, thread); + +list_for_each_entry (t, &tpebs_results, nd) { +if (!strcmp (t->tpebs_name, evsel->name)) { + found = true; +break ; + } + } + +if (!found) +return -1 ; + +/* +* Only set retire_latency value to the first CPU and thread. tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread) · line 1557
Quoted 10 lines
tools/perf/util/evsel.c +10
-0+*/ +if (cpu_map_idx ==0 && thread ==0 ) + val = t->val; +else + val =0 ; + + count->val = val; +/* Set ena and run to non-zero */ + count->ena = count->run =1 ; + count->lost =0 ;So here it seems you discard the actual count of the events and replace it with the retire latency. That means you don't need to open the event in perf stat, and probably just have a placeholder, right?
Quoted 5 lines
Quoted 2 lines
Btw, I think it's better to move this logic to
intel-tpebs.cfile and rename totpebs_set_retire_lat().Ian wants this to be here and also suggested me to rename this function to
evsel__read_retire_lat(). I'm ok with either way.I think it's better to have the tpebs logic together.
Thanks,
NamhyungQuoted 15 lines
Quoted 15 lines
tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread)
Quoted 7 lines
tools/perf/util/evsel.c +3
-0+return 0 ; +} + static void evsel__set_count(struct evsel *counter, int cpu_map_idx, intthread,
tools/perf/util/evsel.c static void evsel__set_count(struct evsel
Quoted 3 lines
tools/perf/util/evsel.c +0
-0u64 val, u64 ena, u64 run, u64 lost) {@@ -1530,6 +1565,12 @@ static void evsel__set_count(struct evsel*counter, int
cpu_map_idx, int thread,Quoted 3 lines
tools/perf/util/evsel.c +0
-0count =perf_counts (counter->counts, cpu_map_idx, thread);Quoted 7 lines
Quoted 7 lines
tools/perf/util/evsel.c static void evsel__set_count(struct evsel *counter, int cpu_map_idx, int thread, · line 1568
Quoted 1 line
tools/perf/util/evsel.c +1
-0+if (counter->retire_lat) {if (
evsel__is_retire_lat(counter)) ?Quoted 2 lines
tools/perf/util/evsel.c +2
-0+evsel__set_retire_lat (counter, cpu_map_idx, thread); + perf_counts__set_loaded(counter->counts, cpu_map_idx,thread,
Quoted 8 lines
Quoted 8 lines
true);
tools/perf/util/evsel.c int evsel__read_counter(struct evsel *evsel,
Quoted 7 lines
tools/perf/util/evsel.c +3
-0+return ; + } + count->val = val; count->ena = ena; count->run = run;@@ -1778,6 +1819,9 @@ int evsel__read_counter(struct evsel *evsel,int
Quoted 16 lines
Quoted 16 lines
cpu_map_idx, int thread)tools/perf/util/evsel.c int evsel__read_counter(struct evsel *evsel, int cpu_map_idx, int thread) · line 1824
Quoted 6 lines
tools/perf/util/evsel.c +3
-0if (evsel__is_tool (evsel))return evsel__read_tool (evsel, cpu_map_idx, thread); +if (evsel__is_retire_lat (evsel)) +return evsel__set_retire_lat (evsel, cpu_map_idx, thread); +I'm not sure if it works well with group event. Probably that's why you wanted to prevent group leaders. But I guess you can just check this after the
PERF_FORMAT_GROUP, no?Thanks,
NamhyungQuoted 8 lines
Quoted 8 lines
tools/perf/util/evsel.h static inline bool evsel__is_tool(const struct
Quoted 8 lines
tools/perf/util/evsel.c +0
-0if (evsel->core.attr.read_format &PERF_FORMAT_GROUP )return evsel__read_group (evsel, cpu_map_idx, thread);tools/perf/util/evsel.h +0
-0@@ -303,6 +303,11 @@ static inline bool evsel__is_tool(const structevsel
Quoted 15 lines
Quoted 15 lines
*evsel)
Quoted 14 lines
tools/perf/util/evsel.h +5
-1return evsel->tool_event !=PERF_TOOL_NONE ; } +static inline bool evsel__is_retire_lat (const struct evsel *evsel) +{ +return evsel->retire_lat; +} +const char *evsel__group_name (struct evsel *evsel);int evsel__group_desc (struct evsel *evsel,char *buf,size_t size); --2.43.0
Ian Rogers
reviewerRe: [RFC PATCH v9 4/7] perf stat: Plugin retire_lat value from sampled data to evsel
In reply to Namhyung KimQuoted 146 lines · Namhyung Kim
Quoted 32 lines · Weilin
Quoted 32 lines
-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 24, 2024 4:17 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v9 4/7] perf stat: Pluginretire_latvalue from sampled data to evselQuoted 4 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
In current :R parsing implementation, the parser would recognize events
with
Quoted 7 lines
retire_latencymodifier and insert them into the evlist like a normal event. Ideally, we need to avoid counting these events.In this commit, at the time when a
retire_latencyevsel is read, set the retire latency value processed from the sampled data to count value. This sampled retire latency value will be used for metric calculation and final event count print out.Quoted 6 lines
Quoted 2 lines
I'm confused. Do you mean you don't count the event with 'R' modifier (w/ perf stat) and just print the (average) retire latency (from perf record)?
In metric formulas, event without 'R' modifier is included as a normal event already. So we don't need to count the event that with 'R' modifier. They only need to be sampled.
Oh, you have the event in the metric expression twice. I thought of one. Then IIUC the metric looks something like this.
myevent1 + (myevent2 * myevent1:R)
I think you'll have 2 myevent1 in perf stat and 1 in perf record, right? But the second one in perf stat is never used and the value is updated from perf record.
Then we can simply remove the event from the evlist (or replace it with a dummy) to reduce the overheads (of open and read).
Quoted 17 lines
Quoted 17 lines
Review
Quoted 11 lines
Signed-off-by Weilin Wang <[email protected]>--- tools/perf/arch/x86/util/evlist.c | 6 +++++ tools/perf/util/evsel.c | 44 +++++++++++++++++++++++++++++++ tools/perf/util/evsel.h | 5 ++++ 3 files changed, 55 insertions(+)(quoted diff) +0
-0
b/tools/perf/arch/x86/util/evlist.ctools/perf/arch/x86/util/evlist.c int arch_evlist__cmp(const struct evsel *lhs, const
Quoted 4 lines
index
b1ce0c52d88d..cebdd483149e100644tools/perf/arch/x86/util/evlist.c +0
-0@@ -89,6 +89,12 @@ int arch_evlist__cmp(const struct evsel *lhs, conststruct evsel *rhs)
Quoted 7 lines
Quoted 6 lines
tools/perf/arch/x86/util/evlist.c int arch_evlist__cmp(const struct evsel *lhs, const struct evsel *rhs) · line 92
Quoted 4 lines
tools/perf/arch/x86/util/evlist.c +1
-0return 1 ; } +/* Retire latency event should not be group leader*/ Hmm.. why?
Because we don't want to count them. Make them the group leader would not work.
I don't understand. You'll read the event regardless of being a leader or not.
Quoted 65 lines
Quoted 65 lines
tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel,
Quoted 23 lines
tools/perf/arch/x86/util/evlist.c +5
-0+if (lhs->retire_lat && !rhs->retire_lat) +return 1 ; +if (!lhs->retire_lat && rhs->retire_lat) +return -1 ; +/* Default ordering by insertion index. */ return lhs->core.idx - rhs->core.idx; }tools/perf/util/evsel.c +1
-0@@ -58,6 +58,7 @@#include <internal/xyarray.h> #include <internal/lib.h> #include <internal/threadmap.h> +#include "util/intel-tpebs.h" #include <linux/ctype.h> @@ -1523,6 +1524,40 @@ static int evsel__read_one(struct evsel *evsel,int
cpu_map_idx, int thread)tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread)
Quoted 4 lines
tools/perf/util/evsel.c +1
-0return perf_evsel__read (&evsel->core, cpu_map_idx, thread, count); } +static int evsel__set_retire_lat(struct evsel *evsel, int cpu_map_idx, intthread)
Quoted 20 lines
tools/perf/util/evsel.c +20
-0+{ +struct perf_counts_values *count; +struct tpebs_retire_lat *t; +bool found = false; +__u64 val; + + count =perf_counts (evsel->counts, cpu_map_idx, thread); + +list_for_each_entry (t, &tpebs_results, nd) { +if (!strcmp (t->tpebs_name, evsel->name)) { + found = true; +break ; + } + } + +if (!found) +return -1 ; + +/* +* Only set retire_latency value to the first CPU and thread. tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread) · line 1557
Quoted 10 lines
tools/perf/util/evsel.c +10
-0+*/ +if (cpu_map_idx ==0 && thread ==0 ) + val = t->val; +else + val =0 ; + + count->val = val; +/* Set ena and run to non-zero */ + count->ena = count->run =1 ; + count->lost =0 ;So here it seems you discard the actual count of the events and replace it with the retire latency. That means you don't need to open the event in perf stat, and probably just have a placeholder, right?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Btw, I think it's better to move this logic to
intel-tpebs.cfile and rename totpebs_set_retire_lat().Ian wants this to be here and also suggested me to rename this function to
evsel__read_retire_lat(). I'm ok with either way.I think it's better to have the tpebs logic together.
I think the tpebs functions can be in a tpebs file. I'd rather have the retirement latency events (from the metric) be evsels for a few reasons:
1) I'd rather everything in a metric be evsels, so things like "num_cpus_online" should really be a tool event rather than a special literal kind of thing. I'd like to reduce special cases over time, in part as it should help with portability. For example, if only Intel x86 can parse :R then someone trying to parse an Intel x86 metric on ARM may get parser errors.
2) When we change from forking perf record to directly opening a sampling ring buffer then it makes sense that we use/update the evsel/evlist logic.
Thanks,
Ian
Quoted 72 lines
Thanks,
NamhyungQuoted 69 lines
Quoted 69 lines
tools/perf/util/evsel.c static int evsel__read_one(struct evsel *evsel, int cpu_map_idx, int thread)
Quoted 7 lines
tools/perf/util/evsel.c +3
-0+return 0 ; +} + static void evsel__set_count(struct evsel *counter, int cpu_map_idx, intthread,
tools/perf/util/evsel.c static void evsel__set_count(struct evsel
Quoted 3 lines
tools/perf/util/evsel.c +0
-0u64 val, u64 ena, u64 run, u64 lost) {@@ -1530,6 +1565,12 @@ static void evsel__set_count(struct evsel*counter, int
cpu_map_idx, int thread,tools/perf/util/evsel.c static void evsel__set_count(struct evsel *counter, int cpu_map_idx, int thread, · line 1568
Quoted 4 lines
tools/perf/util/evsel.c +1
-0count =perf_counts (counter->counts, cpu_map_idx, thread); +if (counter->retire_lat) {if (
evsel__is_retire_lat(counter)) ?tools/perf/util/evsel.c static void evsel__set_count(struct evsel · line 1569
Quoted 2 lines
tools/perf/util/evsel.c +2
-0+evsel__set_retire_lat (counter, cpu_map_idx, thread); + perf_counts__set_loaded(counter->counts, cpu_map_idx, thread,true);
tools/perf/util/evsel.c int evsel__read_counter(struct evsel *evsel, int
Quoted 7 lines
tools/perf/util/evsel.c +3
-0+return ; + } + count->val = val; count->ena = ena; count->run = run;@@ -1778,6 +1819,9 @@ int evsel__read_counter(struct evsel *evsel, int
cpu_map_idx, int thread)tools/perf/util/evsel.c int evsel__read_counter(struct evsel *evsel, int cpu_map_idx, int thread) · line 1824
Quoted 6 lines
tools/perf/util/evsel.c +3
-0if (evsel__is_tool (evsel))return evsel__read_tool (evsel, cpu_map_idx, thread); +if (evsel__is_retire_lat (evsel)) +return evsel__set_retire_lat (evsel, cpu_map_idx, thread); +I'm not sure if it works well with group event. Probably that's why you wanted to prevent group leaders. But I guess you can just check this after the
PERF_FORMAT_GROUP, no?Thanks,
Namhyungtools/perf/util/evsel.h static inline bool evsel__is_tool(const struct evsel
Quoted 8 lines
tools/perf/util/evsel.c +0
-0if (evsel->core.attr.read_format &PERF_FORMAT_GROUP )return evsel__read_group (evsel, cpu_map_idx, thread);tools/perf/util/evsel.h +0
-0@@ -303,6 +303,11 @@ static inline bool evsel__is_tool(const struct evsel*evsel)
Quoted 14 lines
tools/perf/util/evsel.h +5
-1return evsel->tool_event !=PERF_TOOL_NONE ; } +static inline bool evsel__is_retire_lat (const struct evsel *evsel) +{ +return evsel->retire_lat; +} +const char *evsel__group_name (struct evsel *evsel);int evsel__group_desc (struct evsel *evsel,char *buf,size_t size); --2.43.0
RFC PATCH v9 5/7 perf stat: Add command line option for enabling tpebs recording
In reply to [email protected]From: Weilin Wang <[email protected]>
With this command line option, tpebs recording is turned off in perf stat on default. It will only be turned on when this option is given in perf stat command.
--- tools/perf/builtin-stat.c | 19 +++++++++++++------ tools/perf/util/evsel.c | 19 ++++++++++++++----- 2 files changed, 27 insertions(+), 11 deletions(-)
tools/perf/builtin-stat.c +13 -6
@@ -117,6 +117,7 @@ static volatile sig_atomic_t child_pid = -1;static int detailed_run =0 ;static bool transaction_run;static bool topdown_run = false; +static bool tpebs_recording = false;static bool smi_cost = false;static bool smi_reset = false;static int big_num_opt =-1 ;
@@ -677,9 +678,11 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx)int err;bool second_pass = false; - err =start_tpebs (&stat_config, evsel_list); -if (err <0 ) -return err; +if (tpebs_recording) { + err =start_tpebs (&stat_config, evsel_list); +if (err <0 ) +return err; + }if (forks) {if (evlist__prepare_workload (evsel_list, &target, argv, is_pipe, workload_exec_failed_signal) <0 ) {
@@ -886,9 +889,11 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx) t1 =rdclock (); - err =stop_tpebs (); -if (err <0 ) -return err; +if (tpebs_recording) { + err =stop_tpebs (); +if (err <0 ) +return err; + }if (stat_config.walltime_run_table) stat_config.walltime_run[run_idx] = t1 - t0;
@@ -1246,6 +1251,8 @@ static struct option stat_options[] = { "disable adding events for the metric threshold calculation"),OPT_BOOLEAN (0 ,"topdown" , &topdown_run,"measure top-down statistics" ), +OPT_BOOLEAN (0 ,"enable-tpebs-recording" , &tpebs_recording, +"enable recording for tpebs when retire_latency required" ),OPT_UINTEGER (0 ,"td-level" , &stat_config.topdown_level,"Set the metrics level for the top-down statistics (0: max level)" ), OPT_BOOLEAN(0, "smi-cost", &smi_cost,
tools/perf/util/evsel.c +14 -5
@@ -1540,21 +1540,30 @@ static int evsel__set_retire_lat(struct evsel *evsel, int cpu_map_idx, int threa } } -if (!found) -return -1 ; +/* Set ena and run to non-zero */ + count->ena = count->run =1 ; + count->lost =0 ; + +if (!found) { +/* +* Set default value or 0 when retire_latency for this event is +* not found from sampling data (enable_tpebs_recording not set +* or 0 sample recorded). +*/ + val =0 ; +return 0 ; + }/* * Only set retire_latency value to the first CPU and thread. */ if (cpu_map_idx ==0 && thread ==0 ) +/* Lost precision when casting from double to __u64. Any improvement? */ val = t->val;else val =0 ; count->val = val; -/* Set ena and run to non-zero */ - count->ena = count->run =1 ; - count->lost =0 ;return 0 ; }
Signature · 1 line
2.43.0
Namhyung Kim
maintainerRe: [RFC PATCH v9 5/7] perf stat: Add command line option for enabling tpebs recording
Quoted 56 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
With this command line option, tpebs recording is turned off in perf stat on default. It will only be turned on when this option is given in perf stat command.
Signed-off-by Weilin Wang <[email protected]>--- tools/perf/builtin-stat.c | 19 +++++++++++++------ tools/perf/util/evsel.c | 19 ++++++++++++++----- 2 files changed, 27 insertions(+), 11 deletions(-)tools/perf/builtin-stat.c +11
-6@@ -117,6 +117,7 @@ static volatile sig_atomic_t child_pid = -1;static int detailed_run =0 ;static bool transaction_run;static bool topdown_run = false; +static bool tpebs_recording = false;static bool smi_cost = false;static bool smi_reset = false;static int big_num_opt =-1 ;@@ -677,9 +678,11 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx)int err;bool second_pass = false; - err =start_tpebs (&stat_config, evsel_list); -if (err <0 ) -return err; +if (tpebs_recording) { + err =start_tpebs (&stat_config, evsel_list); +if (err <0 ) +return err; + }if (forks) {if (evlist__prepare_workload (evsel_list, &target, argv, is_pipe, workload_exec_failed_signal) <0 ) {@@ -886,9 +889,11 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx) t1 =rdclock (); - err =stop_tpebs (); -if (err <0 ) -return err; +if (tpebs_recording) { + err =stop_tpebs (); +if (err <0 ) +return err; + }if (stat_config.walltime_run_table) stat_config.walltime_run[run_idx] = t1 - t0;
tools/perf/builtin-stat.c static struct option stat_options[] = { · line 1254
Quoted 4 lines
tools/perf/builtin-stat.c +1
-0@@ -1246,6 +1251,8 @@ static struct option stat_options[] = { "disable adding events for the metric threshold calculation"),OPT_BOOLEAN (0 ,"topdown" , &topdown_run,"measure top-down statistics" ), + OPT_BOOLEAN(0, "enable-tpebs-recording", &tpebs_recording,
Just --tpebs or --tpebs-record? I just prefer short names. :)
Quoted 23 lines
tools/perf/builtin-stat.c +1
-0+ "enable recording for tpebs when retire_latency required"),OPT_UINTEGER (0 ,"td-level" , &stat_config.topdown_level,"Set the metrics level for the top-down statistics (0: max level)" ), OPT_BOOLEAN(0, "smi-cost", &smi_cost,tools/perf/util/evsel.c +9
-2@@ -1540,21 +1540,30 @@ static int evsel__set_retire_lat(struct evsel *evsel, int cpu_map_idx, int threa } } -if (!found) -return -1 ; +/* Set ena and run to non-zero */ + count->ena = count->run =1 ; + count->lost =0 ; + +if (!found) { +/* +* Set default value or 0 when retire_latency for this event is +* not found from sampling data (enable_tpebs_recording not set +* or 0 sample recorded).
tools/perf/util/evsel.c static int evsel__set_retire_lat(struct evsel *evsel, int cpu_map_idx, int threa · line 1561
Quoted 10 lines
tools/perf/util/evsel.c +5
-0+*/ + val =0 ; +return 0 ; + }/* * Only set retire_latency value to the first CPU and thread. */ if (cpu_map_idx ==0 && thread ==0 ) +/* Lost precision when casting from double to __u64. Any improvement? */
Maybe you can save val * 1000 and then later
convert back to double and divide by 1000?
Thanks,
Namhyung
Quoted 14 lines
tools/perf/util/evsel.c +0
-4val = t->val; else val =0 ; count->val = val; -/* Set ena and run to non-zero */ - count->ena = count->run =1 ; - count->lost =0 ;return 0 ; } --2.43.0
Namhyung Kim
maintainerRe: [RFC PATCH v9 5/7] perf stat: Add command line option for enabling tpebs recording
Quoted 8 lines · Namhyung Kim
Quoted 6 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
With this command line option, tpebs recording is turned off in perf stat on default. It will only be turned on when this option is given in perf stat command.
It'd be nice to include an example command line and output.
Thanks,
Namhyung
Quoted 2 lines
Quoted 2 lines
Signed-off-by Weilin Wang <[email protected]>
RE: [RFC PATCH v9 5/7] perf stat: Add command line option for enabling tpebs recording
In reply to Namhyung KimQuoted 120 lines
-----Original Message-----
From: Namhyung Kim <[email protected]>
Sent: Friday, May 24, 2024 4:21 PM
To: Wang, Weilin <[email protected]>
Cc: Ian Rogers <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v9 5/7] perf stat: Add command line option for enabling tpebs recordingtools/perf/builtin-stat.c static volatile sig_atomic_t child_pid =
Quoted 18 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
With this command line option, tpebs recording is turned off in perf stat on default. It will only be turned on when this option is given in perf stat command.
Signed-off-by Weilin Wang <[email protected]>--- tools/perf/builtin-stat.c | 19 +++++++++++++------ tools/perf/util/evsel.c | 19 ++++++++++++++----- 2 files changed, 27 insertions(+), 11 deletions(-)tools/perf/builtin-stat.c +0
-0@@ -117,6 +117,7 @@ static volatile sig_atomic_t child_pid =-1;
tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char
Quoted 8 lines
tools/perf/builtin-stat.c +1
-0static int detailed_run =0 ;static bool transaction_run;static bool topdown_run = false; +static bool tpebs_recording = false;static bool smi_cost = false;static bool smi_reset = false;static int big_num_opt =-1 ;@@ -677,9 +678,11 @@ static int __run_perf_stat(int argc, const char**argv, int
run_idx)Quoted 4 lines
tools/perf/builtin-stat.c +0
-1int err;bool second_pass = false; - err =start_tpebs (&stat_config, evsel_list);tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char **argv, int run_idx)
Quoted 10 lines
tools/perf/builtin-stat.c +5
-2-if (err <0 ) -return err; +if (tpebs_recording) { + err =start_tpebs (&stat_config, evsel_list); +if (err <0 ) +return err; + }if (forks) { if (evlist__prepare_workload(evsel_list, &target, argv, is_pipe,
workload_exec_failed_signal) < 0) {tools/perf/builtin-stat.c static int __run_perf_stat(int argc, const char
Quoted 1 line
tools/perf/builtin-stat.c +0
-0@@ -886,9 +889,11 @@ static int __run_perf_stat(int argc, const char**argv, int
run_idx)Quoted 15 lines
tools/perf/builtin-stat.c +5
-3t1 =rdclock (); - err =stop_tpebs (); -if (err <0 ) -return err; +if (tpebs_recording) { + err =stop_tpebs (); +if (err <0 ) +return err; + }if (stat_config.walltime_run_table) stat_config.walltime_run[run_idx] = t1 - t0;tools/perf/builtin-stat.c static struct option stat_options[] = { · line 1254
Quoted 4 lines
tools/perf/builtin-stat.c +1
-0@@ -1246,6 +1251,8 @@ static struct option stat_options[] = { "disable adding events for the metric threshold calculation"),OPT_BOOLEAN (0 ,"topdown" , &topdown_run,"measure top-down statistics" ), + OPT_BOOLEAN(0, "enable-tpebs-recording", &tpebs_recording,Just --tpebs or --tpebs-record? I just prefer short names. :)
tools/perf/util/evsel.c static int evsel__set_retire_lat(struct evsel
Quoted 9 lines
tools/perf/builtin-stat.c +1
-0+ "enable recording for tpebs when retire_latency required"),OPT_UINTEGER (0 ,"td-level" , &stat_config.topdown_level,"Set the metrics level for the top-down statistics (0: max level)" ), OPT_BOOLEAN(0, "smi-cost", &smi_cost,tools/perf/util/evsel.c +0
-0@@ -1540,21 +1540,30 @@ static int evsel__set_retire_lat(struct evsel*evsel, int
cpu_map_idx, int threaQuoted 14 lines
tools/perf/util/evsel.c +9
-2} } -if (!found) -return -1 ; +/* Set ena and run to non-zero */ + count->ena = count->run =1 ; + count->lost =0 ; + +if (!found) { +/* +* Set default value or 0 when retire_latency for this event is +* not found from sampling data (enable_tpebs_recording not set +* or 0 sample recorded). tools/perf/util/evsel.c static int evsel__set_retire_lat(struct evsel *evsel, int cpu_map_idx, int threa
Quoted 10 lines
tools/perf/util/evsel.c +5
-0+*/ + val =0 ; +return 0 ; + }/* * Only set retire_latency value to the first CPU and thread. */ if (cpu_map_idx ==0 && thread ==0 ) +/* Lost precision when casting from double to __u64. Any
Quoted 4 lines
improvement? */
Maybe you can save val * 1000 and then later
convert back to double and divide by 1000?
I also thought about this method. But this will require special handling somewhere, which looks like is not we want. Maybe we could leave this not here and handle this later if we found the precision is important?
Thanks,
Weilin
Quoted 19 lines
Thanks,
NamhyungQuoted 14 lines
tools/perf/util/evsel.c +0
-4val = t->val; else val =0 ; count->val = val; -/* Set ena and run to non-zero */ - count->ena = count->run =1 ; - count->lost =0 ;return 0 ; } --2.43.0
RFC PATCH v9 6/7 perf Document: Add TPEBS to Documents
In reply to [email protected]From: Weilin Wang <[email protected]>
TPEBS is a new feature Intel PMU from Granite Rapids microarchitecture. It will be used in new TMA releases. Adding related introduction to documents while adding new code to support it in perf stat.
--- tools/perf/Documentation/perf-list.txt | 1 + tools/perf/Documentation/topdown.txt | 30 ++++++++++++++++++++++++++ 2 files changed, 31 insertions(+)
tools/perf/Documentation/perf-list.txt +1 -0
@@ -72,6 +72,7 @@ counted. The following modifiers exist: W - group is weak and will fallback to non-group if not schedulable, e - group or event are exclusive and do not share the PMU b - use BPF aggregration (see perf stat --bpf-counters) + R - retire latency value of the event The 'p' modifier can be used for specifying how precise the instruction address should be. The 'p' modifier can be specified multiple times:
tools/perf/Documentation/topdown.txt +30 -0
@@ -325,6 +325,36 @@ other four level 2 metrics by subtracting corresponding metrics as below. Fetch_Bandwidth = Frontend_Bound - Fetch_Latency Core_Bound = Backend_Bound - Memory_Bound +TPEBS in TopDown +================ + +TPEBS (Timed PEBS) is one of the new Intel PMU features provided since Granite +Rapids microarchitecture. The TPEBS feature adds a 16 bit retire_latency field +in the Basic Info group of the PEBS record. It records the Core cycles since the +retirement of the previous instruction to the retirement of current instruction. +Please refer to Section 8.4.1 of "Intel® Architecture Instruction Set Extensions +Programming Reference" for more details about this feature. Because this feature +extends PEBS record, sampling with weight option is required to get the +retire_latency value. + + perf record -e event_name -W ... + +In the most recent release of TMA, the metrics begin to use event retire_latency +values in some of the metrics’ formulas on processors that support TPEBS feature. +For previous generations that do not support TPEBS, the values are static and +predefined per processor family by the hardware architects. Due to the diversity +of workloads in execution environments, retire_latency values measured at real +time are more accurate. Therefore, new TMA metrics that use TPEBS will provide +more accurate performance analysis results. + +To support TPEBS in TMA metrics, a new modifier :R on event is added. Perf would +capture retire_latency value of required events(event with :R in metric formula) +with perf record. The retire_latency value would be used in metric calculation. +Currently, this feature is supported through perf stat + + perf stat -M metric_name --enable-tpebs-recording ... + + [1] https://software.intel.com/en-us/top-down-microarchitecture-analysis-method-win [2] https://sites.google.com/site/analysismethods/yasin-pubs
Signature · 1 line
2.43.0
RFC PATCH v9 7/7 perf test: Add test for Intel TPEBS counting mode
In reply to [email protected]From: Weilin Wang <[email protected]>
Intel TPEBS sampling mode is supported through perf record. The counting mode code uses perf record to capture retire_latency value and use it in metric calculation. This test checks the counting mode code.
--- .../perf/tests/shell/test_stat_intel_tpebs.sh | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100755 tools/perf/tests/shell/test_stat_intel_tpebs.sh
tools/perf/tests/shell/test_stat_intel_tpebs.sh new file +27 -0
@@ -0,0 +1,27 @@ +#!/bin/sh +# test Intel TPEBS counting mode +# SPDX-License-Identifier: GPL-2.0 + +set e +err=0 + +# Use this event for testing because it should exist in all platforms +e=cache-misses:R + +# Without this cmd option, default value or zero is returned +echo "Testing without --enable-tpebs-recording" +result=$(perf stat -e "$e" true 2>&1) +[[ "$result" =~ "$e" ]] && err=0 || err=1 + +if [[ "$err" == 1 ]] +then + exit "$err" +fi + +# In platforms that do not support TPEBS, it should execute without error. +echo "Testing with --enable-tpebs-recording" +result=$(perf stat -e "$e" --enable-tpebs-recording -a sleep 0.01 2>&1) +[[ "$result" =~ "perf record" && "$result" =~ "$e" ]] && err=0 || err=1 + + +exit "$err"
Signature · 1 line
2.43.0
Namhyung Kim
maintainerRe: [RFC PATCH v9 7/7] perf test: Add test for Intel TPEBS counting mode
Quoted 28 lines · On Tue, May 21, 2024 at 10:40 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
Intel TPEBS sampling mode is supported through perf record. The counting mode code uses perf record to capture
retire_latencyvalue and use it in metric calculation. This test checks the counting mode code.Signed-off-by Weilin Wang <[email protected]>--- .../perf/tests/shell/test_stat_intel_tpebs.sh | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100755 tools/perf/tests/shell/test_stat_intel_tpebs.shtools/perf/tests/shell/test_stat_intel_tpebs.sh new file +9
-0@@ -0,0 +1,27 @@ +#!/bin/sh +# test Intel TPEBS counting mode +# SPDX-License-Identifier: GPL-2.0 + +set e +err=0 + +# Use this event for testing because it should exist in all platforms +e=cache-misses:R
tools/perf/tests/shell/test_stat_intel_tpebs.sh line 19
Quoted 10 lines
tools/perf/tests/shell/test_stat_intel_tpebs.sh new file +10
-0+ +# Without this cmd option, default value or zero is returned +echo "Testing without --enable-tpebs-recording" +result=$(perf stat -e "$e" true 2>&1) +[[ "$result" =~ "$e" ]] && err=0 || err=1 + +if [[ "$err" == 1 ]] +then + exit "$err" +fi
Probably you can just do
[[ "$result" =~ "$e" ]] || exit 1
tools/perf/tests/shell/test_stat_intel_tpebs.sh line 24
Quoted 5 lines
tools/perf/tests/shell/test_stat_intel_tpebs.sh new file +5
-0+ +# In platforms that do not support TPEBS, it should execute without error. +echo "Testing with --enable-tpebs-recording" +result=$(perf stat -e "$e" --enable-tpebs-recording -a sleep 0.01 2>&1) +[[ "$result" =~ "perf record" && "$result" =~ "$e" ]] && err=0 || err=1
Ditto.
Thanks,
Namhyung
Quoted 6 lines
tools/perf/tests/shell/test_stat_intel_tpebs.sh new file +3
-1+ + +exit "$err" --2.43.0
Ian Rogers
reviewerQuoted 10 lines · On Tue, May 21, 2024 at 10:39 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
Changes in v9:
- Update theretire_latencyresult print and metric calculation method. Plugin the value to evsel so that no special code is required.
- Update --control:fifo to use pipe instead of named pipe.
- Add test for TPEBS counting mode.
- Update Document with more details.
Quoted 3 lines
Changes in v8:
- In this revision, the code is updated to base on Ian's patch on R modifier parser https://lore.kernel.org/lkml/[email protected]/
I pulled the first patch of these 3, to clean up the tool events into its own patch and hopefully addressed Namhyung's feedback:
https://lore.kernel.org/lkml/[email protected]/
It is looking for review.
I think in general it'd be useful when posting the series to include the patches it is dependent on, that way we can also make changes to those patches (for example the missed perf-list documentation of the 'R' modifier).
Thanks,
Ian
Quoted 124 lines
After this change, there is no special code required for R modifier in
metricgroup.candmetricgroup.hfiles.Caveat of this change:
Ideally, we will need to add special handling to skip counting events with R
modifier in evsel. Currently, this is not implemented so the event with :R will be both counted and sampled. Usually, in a metric formula that usesretire_latency, it would already require to count the event. As a result, we will endup count the same event twice. This should be able to be handled properly when we finalize our design on evsel R modifier support.- Move TPEBS specific code out from main perf stat code to separate files in
util/intel-tpebs.candutil/intel-tpebs.h. [Namhyung]
- Use --control:fifo to ack perf stat from forked perf record instead of sleep(2) [Namhyung]
- Add introductions about TPEBS and R modifier in Documents. [Namhyung]Changes in v7:
- Update code and comments for better code quality [Namhyung]
- Add a separate commit for perf data [Namhyung]
- Update retire latency print function to improve alignment [Namhyung]Changes in v6:
- Update code and add comments for better code quality [Namhyung]
- Remove the added fd var and directly pass the opened fd to data.file.fd [Namhyung]
- Addkill()to stop perf record when perf stat exists early [Namhyung]
- Add command opt check to ensure only start perf record when -a/-C given [Namhyung]
- Squash commits [Namhyung]Changes in v5:
- Update code and add comments for better code quality [Ian]Changes in v4:
- Remove uncessary debug print and update code and comments for better readability and quality [Namhyung]
- Update mtl metric json file with consistent TmaL1 and TopdownL1 metricgroupChanges in v3:
- Remove ':' when event name has '@' [Ian]
- Use 'R' as the modifier instead of "retire_latency" [Ian]Changes in v2:
- Add MTL metric file
- Add more descriptions and example to the patch [Arnaldo]Here is an example of running perf stat to collect a metric that uses
retire_latencyvalue of event MEM_INST_RETIRED.STLB_HIT_STORES on a MTL system.In this simple example, there is no MEM_INST_RETIRED.STLB_HIT_STORES sample. Therefore, the MEM_INST_RETIRED.STLB_HIT_STORES:p count and
retire_latencyvalue are all 0../perf stat -M
tma_dtlb_store-a -- sleep 1[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.000 MB - ]Performance counter stats for 'system wide':
181,047,168 cpu_core/TOPDOWN.SLOTS/ # 0.6 % tma_dtlb_store 3,195,608 cpu_core/topdown-retiring/ 40,156,649 cpu_core/topdown-mem-bound/ 3,550,925 cpu_core/topdown-bad-spec/ 117,571,818 cpu_core/topdown-fe-bound/ 57,118,087 cpu_core/topdown-be-bound/ 69,179 cpu_core/EXE_ACTIVITY.BOUND_ON_STORES/ 4,582 cpu_core/MEM_INST_RETIRED.STLB_HIT_STORES/ 30,183,104 cpu_core/CPU_CLK_UNHALTED.DISTRIBUTED/ 30,556,790 cpu_core/CPU_CLK_UNHALTED.THREAD/ 168,486 cpu_core/DTLB_STORE_MISSES.WALK_ACTIVE/0.00 MEM_INST_RETIRED.STLB_HIT_STORES:p 0 0
1.003105924 seconds time elapsed
v1:
TPEBS is one of the features provided by the next generation of Intel PMU. Please refer to Section 8.4.1 of "Intel® Architecture Instruction Set Extensions Programming Reference" [1] for more details about this feature.This set of patches supports TPEBS in counting mode. The code works in the following way: it forks a perf record process from perf stat when
retire_latencyof one or more events are used in a metric formula. Perf stat would send a SIGTERM signal to perf record before it needs the retire latency value for metric calculation. Perf stat will then process sample data to extract the retire latency data for metric calculations. Currently, the code uses the arithmetic average of retire latency values.Weilin Wang (7):
perf vendor events intel: Add MTL metric json files perf data: Allow to use given fd indata->file.fd
perf stat: Fork and launch perf record when perf stat needs to get
retire latency value for a metric.
perf stat: Pluginretire_latvalue from sampled data to evsel perf stat: Add command line option for enabling tpebs recording perf Document: Add TPEBS to Documents
perf test: Add test for Intel TPEBS counting modetools/perf/Documentation/perf-list.txt | 1 + tools/perf/Documentation/topdown.txt | 30 + tools/perf/arch/x86/util/evlist.c | 6 + tools/perf/builtin-stat.c | 24 + .../arch/x86/meteorlake/metricgroups.json | 127 + .../arch/x86/meteorlake/mtl-metrics.json | 2551 +++++++++++++++++ .../perf/tests/shell/test_stat_intel_tpebs.sh | 27 + tools/perf/util/Build | 1 + tools/perf/util/data.c | 7 +- tools/perf/util/evsel.c | 53 + tools/perf/util/evsel.h | 5 + tools/perf/util/intel-tpebs.c | 301 ++ tools/perf/util/intel-tpebs.h | 30 + tools/perf/util/stat.h | 1 + 14 files changed, 3163 insertions(+), 1 deletion(-) create mode 100644 tools/perf/pmu-events/arch/x86/meteorlake/metricgroups.json create mode 100644 tools/perf/pmu-events/arch/x86/meteorlake/mtl-metrics.json create mode 100755 tools/perf/tests/shell/test_stat_intel_tpebs.sh create mode 100644 tools/perf/util/intel-tpebs.c create mode 100644 tools/perf/util/intel-tpebs.h--
2.43.0
Quoted 38 lines
-----Original Message-----
From: Ian Rogers <[email protected]>
Sent: Wednesday, May 22, 2024 9:04 PM
To: Wang, Weilin <[email protected]>
Cc: Namhyung Kim <[email protected]>; Arnaldo Carvalho de Melo <[email protected]>; Peter Zijlstra <[email protected]>; Ingo Molnar <[email protected]>; Alexander Shishkin <[email protected]>; Jiri Olsa <[email protected]>; Hunter, Adrian <[email protected]>; Kan Liang <[email protected]>; [email protected]; [email protected]; Taylor, Perry <[email protected]>; Alt, Samantha <[email protected]>; Biggers, Caleb <[email protected]>
Subject: Re: [RFC PATCH v9 0/7] TPEBS counting mode supportQuoted 10 lines · On Tue, May 21, 2024 at 10:39 AM <[email protected]> wrote:
From: Weilin Wang <[email protected]>
Changes in v9:
- Update theretire_latencyresult print and metric calculation method. Plugin the value to evsel so that no special code is required.
- Update --control:fifo to use pipe instead of named pipe.
- Add test for TPEBS counting mode.
- Update Document with more details.Quoted 3 lines
Changes in v8:
- In this revision, the code is updated to base on Ian's patch on R modifier parser https://lore.kernel.org/lkml/20240428053616.1125891-3-I pulled the first patch of these 3, to clean up the tool events into its own patch and hopefully addressed Namhyung's feedback:
https://lore.kernel.org/lkml/20240503232849.17752-1-
[email protected]/
It is looking for review.
Quoted 4 lines
I think in general it'd be useful when posting the series to include the patches it is dependent on, that way we can also make changes to those patches (for example the missed perf-list documentation of the 'R' modifier).
Hi Ian,
I will include the 2nd patch from the series in the TPEBS patch set next time.
Thanks,
Weilin
Quoted 147 lines
Thanks,
IanQuoted 7 lines
After this change, there is no special code required for R modifier in
metricgroup.candmetricgroup.hfiles.Caveat of this change:
Ideally, we will need to add special handling to skip counting events with R
modifier in evsel. Currently, this is not implemented so the event with :R will be both counted and sampled. Usually, in a metric formula that uses
retire_latency,Quoted 1 line
it would already require to count the event. As a result, we will endup count
the
Quoted 1 line
same event twice. This should be able to be handled properly when we
finalize our
Quoted 5 lines
design on evsel R modifier support.
- Move TPEBS specific code out from main perf stat code to separate files in
util/intel-tpebs.candutil/intel-tpebs.h. [Namhyung]
- Use --control:fifo to ack perf stat from forked perf record instead ofsleep(2) [Namhyung]
Quoted 11 lines
- Add introductions about TPEBS and R modifier in Documents. [Namhyung]
Changes in v7:
- Update code and comments for better code quality [Namhyung]
- Add a separate commit for perf data [Namhyung]
- Update retire latency print function to improve alignment [Namhyung]Changes in v6:
- Update code and add comments for better code quality [Namhyung]
- Remove the added fd var and directly pass the opened fd to data.file.fd[Namhyung]
Quoted 2 lines
- Add
kill()to stop perf record when perf stat exists early [Namhyung]
- Add command opt check to ensure only start perf record when -a/-C given[Namhyung]
Quoted 9 lines
- Squash commits [Namhyung]
Changes in v5:
- Update code and add comments for better code quality [Ian]Changes in v4:
- Remove uncessary debug print and update code and comments for better readability and quality [Namhyung]
- Update mtl metric json file with consistent TmaL1 and TopdownL1metricgroup
Quoted 11 lines
Changes in v3:
- Remove ':' when event name has '@' [Ian]
- Use 'R' as the modifier instead of "retire_latency" [Ian]Changes in v2:
- Add MTL metric file
- Add more descriptions and example to the patch [Arnaldo]Here is an example of running perf stat to collect a metric that uses
retire_latencyvalue of event MEM_INST_RETIRED.STLB_HIT_STORES on aMTL system.
Quoted 2 lines
In this simple example, there is no MEM_INST_RETIRED.STLB_HIT_STORES
sample.
Quoted 1 line
Therefore, the MEM_INST_RETIRED.STLB_HIT_STORES:p count and
retire_latencyvalueQuoted 10 lines
are all 0.
./perf stat -M
tma_dtlb_store-a -- sleep 1[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.000 MB - ]Performance counter stats for 'system wide':
181,047,168 cpu_core/TOPDOWN.SLOTS/ # 0.6 %
tma_dtlb_storeQuoted 10 lines
3,195,608 cpu_core/topdown-retiring/ 40,156,649 cpu_core/topdown-mem-bound/ 3,550,925 cpu_core/topdown-bad-spec/ 117,571,818 cpu_core/topdown-fe-bound/ 57,118,087 cpu_core/topdown-be-bound/ 69,179 cpu_core/EXE_ACTIVITY.BOUND_ON_STORES/ 4,582 cpu_core/MEM_INST_RETIRED.STLB_HIT_STORES/ 30,183,104 cpu_core/CPU_CLK_UNHALTED.DISTRIBUTED/ 30,556,790 cpu_core/CPU_CLK_UNHALTED.THREAD/ 168,486 cpu_core/DTLB_STORE_MISSES.WALK_ACTIVE/Quoted 7 lines
0.00 MEM_INST_RETIRED.STLB_HIT_STORES:p 0 0
1.003105924 seconds time elapsed
v1:
TPEBS is one of the features provided by the next generation of Intel PMU. Please refer to Section 8.4.1 of "Intel® Architecture Instruction SetExtensions
Quoted 4 lines
Programming Reference" [1] for more details about this feature.
This set of patches supports TPEBS in counting mode. The code works in the following way: it forks a perf record process from perf stat when
retire_latencyQuoted 7 lines
of one or more events are used in a metric formula. Perf stat would send a SIGTERM signal to perf record before it needs the retire latency value for metric calculation. Perf stat will then process sample data to extract the retire latency data for metric calculations. Currently, the code uses the arithmetic average of retire latency values.
details/812218/intel-architecture-instruction-set-extensions-programming- reference.html?wapkw=future%20features
Quoted 12 lines
Weilin Wang (7):
perf vendor events intel: Add MTL metric json files perf data: Allow to use given fd indata->file.fd
perf stat: Fork and launch perf record when perf stat needs to get
retire latency value for a metric.
perf stat: Pluginretire_latvalue from sampled data to evsel perf stat: Add command line option for enabling tpebs recording perf Document: Add TPEBS to Documents
perf test: Add test for Intel TPEBS counting modeQuoted 16 lines
tools/perf/Documentation/perf-list.txt | 1 + tools/perf/Documentation/topdown.txt | 30 + tools/perf/arch/x86/util/evlist.c | 6 + tools/perf/builtin-stat.c | 24 + .../arch/x86/meteorlake/metricgroups.json | 127 + .../arch/x86/meteorlake/mtl-metrics.json | 2551 +++++++++++++++++ .../perf/tests/shell/test_stat_intel_tpebs.sh | 27 + tools/perf/util/Build | 1 + tools/perf/util/data.c | 7 +- tools/perf/util/evsel.c | 53 + tools/perf/util/evsel.h | 5 + tools/perf/util/intel-tpebs.c | 301 ++ tools/perf/util/intel-tpebs.h | 30 + tools/perf/util/stat.h | 1 + 14 files changed, 3163 insertions(+), 1 deletion(-) create mode 100644 tools/perf/pmu-
events/arch/x86/meteorlake/metricgroups.jsonQuoted 1 line
create mode 100644 tools/perf/pmu-events/arch/x86/meteorlake/mtl-metrics.json
Quoted 7 lines
create mode 100755 tools/perf/tests/shell/test_stat_intel_tpebs.sh create mode 100644 tools/perf/util/intel-tpebs.c create mode 100644 tools/perf/util/intel-tpebs.h--
2.43.0