perf/core: Add ability for an event to "pause" or "resume" AUX area tracing
Adrian Hunter
reviewerv16 cover perf/core: Add ability for an event to "pause" or "resume" AUX area tracing
Hi
Note for V15:
Same as V14 but without kernel patches because they have been applied, and updated "missing_features" patch for the new way of detecting missing features.
Hardware traces, such as instruction traces, can produce a vast amount of trace data, so being able to reduce tracing to more specific circumstances can be useful.
The ability to pause or resume tracing when another event happens, can do that.
These patches add such a facilty and show how it would work for Intel Processor Trace.
Maintainers of other AUX area tracing implementations are requested to consider if this is something they might employ and then whether or not the ABI would work for them. Note, thank you to James Clark (ARM) for evaluating the API for Coresight. Suzuki K Poulose (ARM) also responded positively to the RFC.
Changes to perf tools are now (since V4) fleshed out.
Please note, Intel® Architecture Instruction Set Extensions and Future Features Programming Reference March 2024 319433-052, currently:
https://cdrdv2.intel.com/v1/dl/getContent/671368
introduces hardware pause / resume for Intel PT in a feature named Intel PT Trigger Tracing.
For that more fields in perf_event_attr will be necessary. The main differences are:
- it can be applied not just to overflows, but optionally to every event
- a packet is emitted into the trace, optionally with IP information
- no PMI
- works with PMC and DR (breakpoint) events only
Here are the proposed additions to perf_event_attr, please comment:
tools/include/uapi/linux/perf_event.h +29 -4
@@ -369,6 +369,22 @@ enum perf_event_read_format { PERF_FORMAT_MAX = 1U << 5, /* non-ABI */ }; +enum { +PERF_AUX_ACTION_START_PAUSED =1U <<0 , +PERF_AUX_ACTION_PAUSE =1U <<1 , +PERF_AUX_ACTION_RESUME =1U <<2 , +PERF_AUX_ACTION_EMIT =1U <<3 , +PERF_AUX_ACTION_NR =0x1f <<4 , +PERF_AUX_ACTION_NO_IP =1U <<9 , +PERF_AUX_ACTION_PAUSE_ON_EVT =1U <<10 , +PERF_AUX_ACTION_RESUME_ON_EVT =1U <<11 , +PERF_AUX_ACTION_EMIT_ON_EVT =1U <<12 , +PERF_AUX_ACTION_NR_ON_EVT =0x1f <<13 , +PERF_AUX_ACTION_NO_IP_ON_EVT =1U <<18 , +PERF_AUX_ACTION_MASK = ~PERF_AUX_ACTION_START_PAUSED , +PERF_AUX_PAUSE_RESUME_MASK =PERF_AUX_ACTION_PAUSE |PERF_AUX_ACTION_RESUME , +}; +#define PERF_ATTR_SIZE_VER0 64/* sizeof first published struct */ #define PERF_ATTR_SIZE_VER1 72/* add: config2 */ #define PERF_ATTR_SIZE_VER2 80/* add: branch_sample_type */
@@ -515,10 +531,19 @@ struct perf_event_attr {union {__u32 aux_action;struct { -__u32 aux_start_paused :1 ,/* start AUX area tracing paused */ - aux_pause :1 ,/* on overflow, pause AUX area tracing */ - aux_resume :1 ,/* on overflow, resume AUX area tracing */ - __reserved_3 :29 ; +__u32 aux_start_paused :1 ,/* start AUX area tracing paused */ + aux_pause :1 ,/* on overflow, pause AUX area tracing */ + aux_resume :1 ,/* on overflow, resume AUX area tracing */ + aux_emit :1 ,/* generate AUX records instead of events */ + aux_nr :5 ,/* AUX area tracing reference number */ + aux_no_ip :1 ,/* suppress IP in AUX records */ +/* Following apply to event occurrence not overflows */ + aux_pause_on_evt :1 ,/* on event, pause AUX area tracing */ + aux_resume_on_evt :1 ,/* on event, resume AUX area tracing */ + aux_emit_on_evt :1 ,/* generate AUX records instead of events */ + aux_nr_on_evt :5 ,/* AUX area tracing reference number */ + aux_no_ip_on_evt :1 ,/* suppress IP in AUX records */ + __reserved_3 :13 ; }; };
Changes in V16:
Rebased
Changes in V15:
perf/x86/intel/pt: Fix buffer full but size is 0 case
perf/core: Add aux_pause, aux_resume, aux_start_paused perf/x86/intel/pt: Add support for pause / resume
perf/x86/intel: Do not enable large PEBS for events with aux actions or aux sampling
Dropped kernel patches because they have been applied
perf tools: Add missing_features for aux_start_paused, aux_pause, aux_resume
Re-base on new API probe method of missing feature detection and add probe for aux_action.
Changes in V14:
Dropped KVM patches
perf/x86/intel/pt: Add support for pause / resume
Set pt->handle_nmi after configuration is completed instead of during
Changes in V13:
perf/core: Add aux_pause, aux_resume, aux_start_paused
Do aux_resume at the end of __perf_event_overflow() so as to trace less of perf itself
perf tools: Add missing_features for aux_start_paused, aux_pause, aux_resume
Add error message also in EOPNOTSUPP case (Leo)
Changes in V12:
Add previously sent patch "perf/x86/intel/pt: Fix buffer full but size is 0 case"
Add previously sent patch set "KVM: x86: Fix Intel PT Host/Guest mode when host tracing"
Rebase on current tip plus patch set "KVM: x86: Fix Intel PT Host/Guest mode when host tracing"
Changes in V11:
perf/core: Add aux_pause, aux_resume, aux_start_paused
Make assignment to event->hw.aux_paused conditional on (pmu->capabilities & PERF_PMU_CAP_AUX_PAUSE).
perf/x86/intel: Do not enable large PEBS for events with aux actions or aux sampling
Remove definition of has_aux_action() because it has
already been added as an inline function.
perf/x86/intel/pt: Fix sampling synchronization
perf tools: Enable evsel__is_aux_event() to work for ARM/ARM64 perf tools: Enable evsel__is_aux_event() to work for S390_CPUMSF
Dropped because they have already been applied
Changes in V10:
perf/core: Add aux_pause, aux_resume, aux_start_paused
Move aux_paused into a union within struct hw_perf_event. Additional comment wrt PERF_EF_PAUSE/PERF_EF_RESUME. Factor out has_aux_action() as an inline function. Use scoped_guard for irqsave.
Move calls of perf_event_aux_pause() from __perf_event_output() to __perf_event_overflow().
Changes in V9:
perf/x86/intel/pt: Fix sampling synchronization
New patch
perf/core: Add aux_pause, aux_resume, aux_start_paused
Move aux_paused to struct hw_perf_event
perf/x86/intel/pt: Add support for pause / resume
Add more comments and barriers for resume_allowed and pause_allowed
Always use WRITE_ONCE with resume_allowed
Changes in V8:
perf tools: Parse aux-action
Fix clang warning:
util/auxtrace.c:821:7: error: missing field 'aux_action' initializer [-Werror,-Wmissing-field-initializers]
821 | {NULL},
| ^
Changes in V7:
Add Andi's Reviewed-by for patches 2-12
Re-base
Changes in V6:
perf/core: Add aux_pause, aux_resume, aux_start_paused
Removed READ/WRITE_ONCE from __perf_event_aux_pause() Expanded comment about guarding against NMI
Changes in V5:
perf/core: Add aux_pause, aux_resume, aux_start_paused
Added James' Ack
perf/x86/intel: Do not enable large PEBS for events with aux actions or aux sampling
New patch
perf tools
Added Ian's Ack
Changes in V4:
perf/core: Add aux_pause, aux_resume, aux_start_paused
Rename aux_output_cfg -> aux_action Reorder aux_action bits from:
aux_pause, aux_resume, aux_start_paused
to:
aux_start_paused, aux_pause, aux_resume
Fix aux_action bits __u64 -> __u32
coresight: Have a stab at support for pause / resume
Dropped
perf tools
All new patches
Changes in RFC V3:
coresight: Have a stab at support for pause / resume
'mode' -> 'flags' so it at least compiles
Changes in RFC V2:
Use ->stop() / ->start() instead of ->pause_resume() Move aux_start_paused bit into aux_output_cfg Tighten up when Intel PT pause / resume is allowed Add an example of how it might work for CoreSight
Adrian Hunter (7):
perf tools: Add aux_start_paused, aux_pause and aux_resume
perf tools: Add aux-action config term
perf tools: Parse aux-action
perf tools: Add missing_features for aux_start_paused, aux_pause, aux_resume perf intel-pt: Improve man page format
perf intel-pt: Add documentation for pause / resume
perf intel-pt: Add a test for pause / resume
tools/perf/Documentation/perf-intel-pt.txt | 596 ++++++++++++++++++----------- tools/perf/Documentation/perf-record.txt | 4 + tools/perf/builtin-record.c | 4 +- tools/perf/tests/shell/test_intel_pt.sh | 28 ++ tools/perf/util/auxtrace.c | 67 +++- tools/perf/util/auxtrace.h | 6 +- tools/perf/util/evsel.c | 101 ++++- tools/perf/util/evsel.h | 1 + tools/perf/util/evsel_config.h | 1 + tools/perf/util/parse-events.c | 10 + tools/perf/util/parse-events.h | 1 + tools/perf/util/parse-events.l | 1 + tools/perf/util/perf_event_attr_fprintf.c | 3 + tools/perf/util/pmu.c | 1 + 14 files changed, 584 insertions(+), 240 deletions(-)
Regards
Adrian
Adrian Hunter
reviewerPATCH v16 1/7 perf tools: Add aux_start_paused, aux_pause and aux_resume
Add struct perf_event_attr members to support pause and resume of AUX area tracing.
Signed-off-by Adrian Hunter <[email protected]>
Acked-by Ian Rogers <[email protected]>
Reviewed-by Andi Kleen <[email protected]>
---
Changes in V16:
Changes to tools/include/uapi/linux/perf_event.h already applied
tools/perf/util/perf_event_attr_fprintf.c | 3 +++ 1 file changed, 3 insertions(+)
tools/perf/util/perf_event_attr_fprintf.c +3 -0
@@ -331,6 +331,9 @@ int perf_event_attr__fprintf(FILE *fp, struct perf_event_attr *attr,PRINT_ATTRf (sample_max_stack, p_unsigned);PRINT_ATTRf (aux_sample_size, p_unsigned);PRINT_ATTRf (sig_data, p_unsigned); +PRINT_ATTRf (aux_start_paused, p_unsigned); +PRINT_ATTRf (aux_pause, p_unsigned); +PRINT_ATTRf (aux_resume, p_unsigned);return ret; }
Signature · 1 line
2.43.0
Adrian Hunter
reviewerPATCH v16 2/7 perf tools: Add aux-action config term
In reply to Adrian HunterAdd a new common config term "aux-action" to use for configuring AUX area trace pause / resume. The value is a string that will be parsed in a subsequent patch.
Signed-off-by Adrian Hunter <[email protected]>
Acked-by Ian Rogers <[email protected]>
Reviewed-by Andi Kleen <[email protected]>
---
Changes in V7:
Add aux-action to perf_pmu__for_each_format
tools/perf/util/evsel.c | 2 ++ tools/perf/util/evsel_config.h | 1 + tools/perf/util/parse-events.c | 10 ++++++++++ tools/perf/util/parse-events.h | 1 + tools/perf/util/parse-events.l | 1 + tools/perf/util/pmu.c | 1 + 6 files changed, 16 insertions(+)
tools/perf/util/evsel.c +2 -0
@@ -1151,6 +1151,8 @@ static void evsel__apply_config_terms(struct evsel *evsel,case EVSEL__CONFIG_TERM_AUX_OUTPUT : attr->aux_output = term->val.aux_output ?1 :0 ;break ; +case EVSEL__CONFIG_TERM_AUX_ACTION : +break ;case EVSEL__CONFIG_TERM_AUX_SAMPLE_SIZE :/* Already applied by auxtrace */ break ;
tools/perf/util/evsel_config.h +1 -0
@@ -25,6 +25,7 @@ enum evsel_term_type {EVSEL__CONFIG_TERM_BRANCH ,EVSEL__CONFIG_TERM_PERCORE ,EVSEL__CONFIG_TERM_AUX_OUTPUT , +EVSEL__CONFIG_TERM_AUX_ACTION ,EVSEL__CONFIG_TERM_AUX_SAMPLE_SIZE , EVSEL__CONFIG_TERM_CFG_CHG, };
tools/perf/util/parse-events.c +10 -0
@@ -793,6 +793,7 @@ const char *parse_events__term_type_str(enum parse_events__term_type term_type) [PARSE_EVENTS__TERM_TYPE_DRV_CFG] = "driver-config", [PARSE_EVENTS__TERM_TYPE_PERCORE] = "percore", [PARSE_EVENTS__TERM_TYPE_AUX_OUTPUT] = "aux-output", + [PARSE_EVENTS__TERM_TYPE_AUX_ACTION] = "aux-action", [PARSE_EVENTS__TERM_TYPE_AUX_SAMPLE_SIZE] = "aux-sample-size", [PARSE_EVENTS__TERM_TYPE_METRIC_ID] = "metric-id", [PARSE_EVENTS__TERM_TYPE_RAW] = "raw",
@@ -842,6 +843,7 @@ config_term_avail(enum parse_events__term_type term_type, struct parse_events_ercase PARSE_EVENTS__TERM_TYPE_OVERWRITE :case PARSE_EVENTS__TERM_TYPE_DRV_CFG :case PARSE_EVENTS__TERM_TYPE_AUX_OUTPUT : +case PARSE_EVENTS__TERM_TYPE_AUX_ACTION :case PARSE_EVENTS__TERM_TYPE_AUX_SAMPLE_SIZE :case PARSE_EVENTS__TERM_TYPE_RAW :case PARSE_EVENTS__TERM_TYPE_LEGACY_CACHE :
@@ -961,6 +963,9 @@ do { \case PARSE_EVENTS__TERM_TYPE_AUX_OUTPUT :CHECK_TYPE_VAL (NUM );break ; +case PARSE_EVENTS__TERM_TYPE_AUX_ACTION : +CHECK_TYPE_VAL (STR ); +break ;case PARSE_EVENTS__TERM_TYPE_AUX_SAMPLE_SIZE :CHECK_TYPE_VAL (NUM );if (term->val.num >UINT_MAX ) {
@@ -1078,6 +1083,7 @@ static int config_term_tracepoint(struct perf_event_attr *attr,case PARSE_EVENTS__TERM_TYPE_OVERWRITE :case PARSE_EVENTS__TERM_TYPE_NOOVERWRITE :case PARSE_EVENTS__TERM_TYPE_AUX_OUTPUT : +case PARSE_EVENTS__TERM_TYPE_AUX_ACTION :case PARSE_EVENTS__TERM_TYPE_AUX_SAMPLE_SIZE :return config_term_common (attr, term, err);case PARSE_EVENTS__TERM_TYPE_USER :
@@ -1213,6 +1219,9 @@ do { \ADD_CONFIG_TERM_VAL (AUX_OUTPUT , aux_output, term->val.num ?1 :0 , term->weak);break ; +case PARSE_EVENTS__TERM_TYPE_AUX_ACTION : +ADD_CONFIG_TERM_STR (AUX_ACTION , term->val.str, term->weak); +break ;case PARSE_EVENTS__TERM_TYPE_AUX_SAMPLE_SIZE :ADD_CONFIG_TERM_VAL (AUX_SAMPLE_SIZE , aux_sample_size, term->val.num, term->weak);
@@ -1275,6 +1284,7 @@ static int get_config_chgs(struct perf_pmu *pmu, struct parse_events_terms *headcase PARSE_EVENTS__TERM_TYPE_DRV_CFG :case PARSE_EVENTS__TERM_TYPE_PERCORE :case PARSE_EVENTS__TERM_TYPE_AUX_OUTPUT : +case PARSE_EVENTS__TERM_TYPE_AUX_ACTION :case PARSE_EVENTS__TERM_TYPE_AUX_SAMPLE_SIZE :case PARSE_EVENTS__TERM_TYPE_METRIC_ID :case PARSE_EVENTS__TERM_TYPE_RAW :
tools/perf/util/parse-events.h +1 -0
@@ -74,6 +74,7 @@ enum parse_events__term_type {PARSE_EVENTS__TERM_TYPE_DRV_CFG ,PARSE_EVENTS__TERM_TYPE_PERCORE ,PARSE_EVENTS__TERM_TYPE_AUX_OUTPUT , +PARSE_EVENTS__TERM_TYPE_AUX_ACTION ,PARSE_EVENTS__TERM_TYPE_AUX_SAMPLE_SIZE ,PARSE_EVENTS__TERM_TYPE_METRIC_ID , PARSE_EVENTS__TERM_TYPE_RAW,
tools/perf/util/parse-events.l +1 -0
@@ -321,6 +321,7 @@ overwrite { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_OVERWRITE); } no-overwrite { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_NOOVERWRITE); } percore { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_PERCORE); } aux-output { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_AUX_OUTPUT); } +aux-action { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_AUX_ACTION); } aux-sample-size { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_AUX_SAMPLE_SIZE); } metric-id { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_METRIC_ID); } cpu-cycles|cycles { return hw_term(yyscanner, PERF_COUNT_HW_CPU_CYCLES); }
tools/perf/util/pmu.c +1 -0
@@ -1773,6 +1773,7 @@ int perf_pmu__for_each_format(struct perf_pmu *pmu, void *state, pmu_format_call"no-overwrite" ,"percore" ,"aux-output" , +"aux-action=(pause|resume|start-paused)" , "aux-sample-size=number", };struct perf_pmu_format *format;
Signature · 1 line
2.43.0
Adrian Hunter
reviewerPATCH v16 3/7 perf tools: Parse aux-action
In reply to Adrian HunterAdd parsing for aux-action to accept "pause", "resume" or "start-paused" values.
"start-paused" is valid only for AUX area events.
"pause" and "resume" are valid only for events grouped with an AUX area event as the group leader. However, like with aux-output, the events will be automatically grouped if they are not currently in a group, and the AUX area event precedes the other events.
Signed-off-by Adrian Hunter <[email protected]>
Acked-by Ian Rogers <[email protected]>
Reviewed-by Andi Kleen <[email protected]>
---
Changes in V8:
Fix clang warning:
util/auxtrace.c:821:7: error: missing field 'aux_action' initializer [-Werror,-Wmissing-field-initializers]
821 | {NULL},
| ^
tools/perf/Documentation/perf-record.txt | 4 ++ tools/perf/builtin-record.c | 4 +- tools/perf/util/auxtrace.c | 67 ++++++++++++++++++++++-- tools/perf/util/auxtrace.h | 6 ++- tools/perf/util/evsel.c | 1 + 5 files changed, 74 insertions(+), 8 deletions(-)
tools/perf/Documentation/perf-record.txt +4 -0
@@ -68,6 +68,10 @@ OPTIONS like this: name=\'CPU_CLK_UNHALTED.THREAD:cmask=0x1\'. - 'aux-output': Generate AUX records instead of events. This requires that an AUX area event is also provided. + - 'aux-action': "pause" or "resume" to pause or resume an AUX + area event (the group leader) when this event occurs. + "start-paused" on an AUX area event itself, will + start in a paused state. - 'aux-sample-size': Set sample size for AUX area sampling. If the '--aux-sample' option has been used, set aux-sample-size=0 to disable AUX area sampling for the event.
tools/perf/builtin-record.c +3 -1
@@ -860,7 +860,9 @@ static int record__auxtrace_init(struct record *rec)if (err)return err; -auxtrace_regroup_aux_output (rec->evlist); + err =auxtrace_parse_aux_action (rec->evlist); +if (err) +return err;return auxtrace_parse_filters (rec->evlist); }
tools/perf/util/auxtrace.c +62 -5
81 lines not on this page · raw message
tools/perf/util/auxtrace.h +4 -2
@@ -578,7 +578,7 @@ int auxtrace_parse_snapshot_options(struct auxtrace_record *itr,int auxtrace_parse_sample_options (struct auxtrace_record *itr,struct evlist *evlist,struct record_opts *opts,const char *str); -void auxtrace_regroup_aux_output (struct evlist *evlist); +int auxtrace_parse_aux_action (struct evlist *evlist);int auxtrace_record__options (struct auxtrace_record *itr,struct evlist *evlist,struct record_opts *opts);
@@ -799,8 +799,10 @@ int auxtrace_parse_sample_options(struct auxtrace_record *itr __maybe_unused, }static inline -void auxtrace_regroup_aux_output (struct evlist *evlist__maybe_unused ) +int auxtrace_parse_aux_action (struct evlist *evlist__maybe_unused ) { +pr_err ("AUX area tracing not supported\n" ); +return -EINVAL ; } static inline
tools/perf/util/evsel.c +1 -0
@@ -1152,6 +1152,7 @@ static void evsel__apply_config_terms(struct evsel *evsel, attr->aux_output = term->val.aux_output ?1 :0 ;break ;case EVSEL__CONFIG_TERM_AUX_ACTION : +/* Already applied by auxtrace */ break ;case EVSEL__CONFIG_TERM_AUX_SAMPLE_SIZE :/* Already applied by auxtrace */
Signature · 1 line
2.43.0
Adrian Hunter
reviewerPATCH v16 4/7 perf tools: Add missing_features for aux_start_paused, aux_pause, aux_resume
In reply to Adrian HunterDisplay "feature is not supported" error message if aux_start_paused, aux_pause or aux_resume result in a perf_event_open() error.
Signed-off-by Adrian Hunter <[email protected]>
---
Changes in V15:
Re-base on new API probe method of missing feature detection and add probe for aux_action.
Changes in V13:
Add error message also in EOPNOTSUPP case (Leo)
tools/perf/util/evsel.c | 98 ++++++++++++++++++++++++++++++++++++----- tools/perf/util/evsel.h | 1 + 2 files changed, 87 insertions(+), 12 deletions(-)
tools/perf/util/evsel.c +86 -12
@@ -2145,16 +2145,17 @@ int evsel__prepare_open(struct evsel *evsel, struct perf_cpu_map *cpus,return err; } -static bool has_attr_feature (struct perf_event_attr *attr,unsigned long flags) +static bool __has_attr_feature (struct perf_event_attr *attr, +struct perf_cpu cpu,unsigned long flags) { -int fd =syscall (SYS_perf_event_open, attr,/*pid=*/ 0 ,/*cpu=*/ -1 , +int fd =syscall (SYS_perf_event_open, attr,/*pid=*/ 0 , cpu.cpu,/*group_fd=*/ -1 , flags);close (fd);if (fd <0 ) { attr->exclude_kernel =1 ; - fd =syscall (SYS_perf_event_open, attr,/*pid=*/ 0 ,/*cpu=*/ -1 , + fd =syscall (SYS_perf_event_open, attr,/*pid=*/ 0 , cpu.cpu,/*group_fd=*/ -1 , flags);close (fd); }
@@ -2162,7 +2163,7 @@ static bool has_attr_feature(struct perf_event_attr *attr, unsigned long flags)if (fd <0 ) { attr->exclude_hv =1 ; - fd =syscall (SYS_perf_event_open, attr,/*pid=*/ 0 ,/*cpu=*/ -1 , + fd =syscall (SYS_perf_event_open, attr,/*pid=*/ 0 , cpu.cpu,/*group_fd=*/ -1 , flags);close (fd); }
@@ -2170,7 +2171,7 @@ static bool has_attr_feature(struct perf_event_attr *attr, unsigned long flags)if (fd <0 ) { attr->exclude_guest =1 ; - fd =syscall (SYS_perf_event_open, attr,/*pid=*/ 0 ,/*cpu=*/ -1 , + fd =syscall (SYS_perf_event_open, attr,/*pid=*/ 0 , cpu.cpu,/*group_fd=*/ -1 , flags);close (fd); }
@@ -2182,6 +2183,13 @@ static bool has_attr_feature(struct perf_event_attr *attr, unsigned long flags)return fd >=0 ; } +static bool has_attr_feature (struct perf_event_attr *attr,unsigned long flags) +{ +struct perf_cpu cpu = {.cpu =-1 }; + +return __has_attr_feature (attr, cpu, flags); +} +static void evsel__detect_missing_pmu_features (struct evsel *evsel) {struct perf_event_attr attr = {
@@ -2270,7 +2278,65 @@ static void evsel__detect_missing_brstack_features(struct evsel *evsel) errno = old_errno; } -static bool evsel__detect_missing_features (struct evsel *evsel) +static bool evsel__probe_aux_action (struct evsel *evsel,struct perf_cpu cpu) +{ +struct perf_event_attr attr = evsel->core.attr; +int old_errno = errno; + + attr.disabled =1 ; + attr.aux_start_paused =1 ; + +if (__has_attr_feature (&attr, cpu,/*flags=*/ 0 )) { + errno = old_errno; +return true; + } + +/* +* EOPNOTSUPP means the kernel supports the feature but the PMU does +* not, so keep that distinction if possible. +*/ +if (errno !=EOPNOTSUPP ) + errno = old_errno; + +return false; +} + +static void evsel__detect_missing_aux_action_feature (struct evsel *evsel,struct perf_cpu cpu) +{ +static bool detection_done; +struct evsel *leader; + +/* +* Don't bother probing aux_action if it is not being used or has been +* probed before. +*/ +if (!evsel->core.attr.aux_action || detection_done) +return ; + + detection_done = true; + +/* +* The leader is an AUX area event. If it has failed, assume the feature +* is not supported. +*/ + leader =evsel__leader (evsel); +if (evsel == leader) { + perf_missing_features.aux_action = true; +return ; + } + +/* +* AUX area event with aux_action must have been opened successfully +* already, so feature is supported. +*/ +if (leader->core.attr.aux_action) +return ; + +if (!evsel__probe_aux_action (leader, cpu)) + perf_missing_features.aux_action = true; +} + +static bool evsel__detect_missing_features (struct evsel *evsel,struct perf_cpu cpu) {static bool detection_done = false;struct perf_event_attr attr = {
@@ -2280,6 +2346,8 @@ static bool evsel__detect_missing_features(struct evsel *evsel) };int old_errno; +evsel__detect_missing_aux_action_feature (evsel, cpu); +evsel__detect_missing_pmu_features (evsel);if (evsel__has_br_stack (evsel))
@@ -2494,6 +2562,7 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,int idx, thread, nthreads;int pid =-1 , err, old_errno;enum rlimit_action set_rlimit =NO_CHANGE ; +struct perf_cpu cpu;if (evsel__is_retire_lat (evsel))return tpebs_start (evsel->evlist);
@@ -2531,6 +2600,7 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus, }for (idx = start_cpu_map_idx; idx < end_cpu_map_idx; idx++) { + cpu =perf_cpu_map__cpu (cpus, idx);for (thread =0 ; thread < nthreads; thread++) {int fd, group_fd;
@@ -2551,10 +2621,9 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,/* Debug message used by test scripts */ pr_debug2_peo ("sys_perf_event_open: pid %d cpu %d group_fd %d flags %#lx" , - pid,perf_cpu_map__cpu (cpus, idx).cpu, group_fd, evsel->open_flags); + pid, cpu.cpu, group_fd, evsel->open_flags); - fd =sys_perf_event_open (&evsel->core.attr, pid, -perf_cpu_map__cpu (cpus, idx).cpu, + fd =sys_perf_event_open (&evsel->core.attr, pid, cpu.cpu, group_fd, evsel->open_flags);FD (evsel, idx, thread) = fd;
@@ -2570,8 +2639,7 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,bpf_counter__install_pe (evsel, idx, fd);if (unlikely (test_attr__enabled ())) { -test_attr__open (&evsel->core.attr, pid, -perf_cpu_map__cpu (cpus, idx), +test_attr__open (&evsel->core.attr, pid, cpu, fd, group_fd, evsel->open_flags); }
@@ -2626,7 +2694,7 @@ static int evsel__open_cpu(struct evsel *evsel, struct perf_cpu_map *cpus,if (err == -EMFILE &&rlimit__increase_nofile (&set_rlimit)) goto retry_open; -if (err == -EINVAL &&evsel__detect_missing_features (evsel)) +if (err == -EINVAL &&evsel__detect_missing_features (evsel, cpu)) goto fallback_missing_features;if (evsel__precise_ip_fallback (evsel))
@@ -3585,6 +3653,10 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,return scnprintf (msg, size,"%s: PMU Hardware doesn't support 'aux_output' feature" ,evsel__name (evsel)); +if (evsel->core.attr.aux_action) +return scnprintf (msg, size, +"%s: PMU Hardware doesn't support 'aux_action' feature" , +evsel__name (evsel));if (evsel->core.attr.sample_period !=0 ) return scnprintf(msg, size, "%s: PMU Hardware doesn't support sampling/overflow-interrupts. Try 'perf stat'",
@@ -3615,6 +3687,8 @@ int evsel__open_strerror(struct evsel *evsel, struct target *target,return scnprintf (msg, size,"clockid feature not supported." );if (perf_missing_features.clockid_wrong)return scnprintf (msg, size,"wrong clockid (%d)." , clockid); +if (perf_missing_features.aux_action) +return scnprintf (msg, size,"The 'aux_action' feature is not supported, update the kernel." );if (perf_missing_features.aux_output)return scnprintf (msg, size,"The 'aux_output' feature is not supported, update the kernel." );if (!target__has_cpu (target))
tools/perf/util/evsel.h +1 -0
@@ -207,6 +207,7 @@ struct perf_missing_features {bool weight_struct;bool read_lost;bool branch_counters; +bool aux_action;bool inherit_sample_read; };
Signature · 1 line
2.43.0
Adrian Hunter
reviewerPATCH v16 5/7 perf intel-pt: Improve man page format
In reply to Adrian HunterImprove format of config terms and section references.
Signed-off-by Adrian Hunter <[email protected]>
Acked-by Ian Rogers <[email protected]>
Reviewed-by Andi Kleen <[email protected]>
--- tools/perf/Documentation/perf-intel-pt.txt | 486 +++++++++++---------- 1 file changed, 267 insertions(+), 219 deletions(-)
tools/perf/Documentation/perf-intel-pt.txt +267 -219
@@ -151,7 +151,7 @@ displayed as follows: There are two ways that instructions-per-cycle (IPC) can be calculated depending on the recording. -If the 'cyc' config term (see config terms section below) was used, then IPC +If the 'cyc' config term (see <<_config_terms,config terms>> section below) was used, then IPC and cycle events are calculated using the cycle count from CYC packets, otherwise MTC packets are used - refer to the 'mtc' config term. When MTC is used, however, the values are less accurate because the timing is less accurate.
@@ -239,7 +239,7 @@ which is the same as -e intel_pt/tsc=1,noretcomp=0/ -Note there are now new config terms - see section 'config terms' further below. +Note there are other config terms - see section <<_config_terms,config terms>> further below. The config terms are listed in /sys/devices/intel_pt/format. They are bit fields within the config member of the struct perf_event_attr which is
@@ -311,217 +311,264 @@ perf_event_attr is displayed if the -vv option is used e.g. config terms ~~~~~~~~~~~~ -The June 2015 version of Intel 64 and IA-32 Architectures Software Developer -Manuals, Chapter 36 Intel Processor Trace, defined new Intel PT features. -Some of the features are reflect in new config terms. All the config terms are -described below. - -tsc Always supported. Produces TSC timestamp packets to provide - timing information. In some cases it is possible to decode - without timing information, for example a per-thread context - that does not overlap executable memory maps. - - The default config selects tsc (i.e. tsc=1). - -noretcomp Always supported. Disables "return compression" so a TIP packet - is produced when a function returns. Causes more packets to be - produced but might make decoding more reliable. - - The default config does not select noretcomp (i.e. noretcomp=0). - -psb_period Allows the frequency of PSB packets to be specified. - - The PSB packet is a synchronization packet that provides a - starting point for decoding or recovery from errors. - - Support for psb_period is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/psb_cyc - - which contains "1" if the feature is supported and "0" - otherwise. - - Valid values are given by: - - /sys/bus/event_source/devices/intel_pt/caps/psb_periods - - which contains a hexadecimal value, the bits of which represent - valid values e.g. bit 2 set means value 2 is valid. - - The psb_period value is converted to the approximate number of - trace bytes between PSB packets as: - - 2 ^ (value + 11) - - e.g. value 3 means 16KiB bytes between PSBs - - If an invalid value is entered, the error message - will give a list of valid values e.g. - - $ perf record -e intel_pt/psb_period=15/u uname - Invalid psb_period for intel_pt. Valid values are: 0-5 - - If MTC packets are selected, the default config selects a value - of 3 (i.e. psb_period=3) or the nearest lower value that is - supported (0 is always supported). Otherwise the default is 0. - - If decoding is expected to be reliable and the buffer is large - then a large PSB period can be used. - - Because a TSC packet is produced with PSB, the PSB period can - also affect the granularity to timing information in the absence - of MTC or CYC. - -mtc Produces MTC timing packets. - - MTC packets provide finer grain timestamp information than TSC - packets. MTC packets record time using the hardware crystal - clock (CTC) which is related to TSC packets using a TMA packet. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/mtc - - which contains "1" if the feature is supported and - "0" otherwise. - - The frequency of MTC packets can also be specified - see - mtc_period below. - -mtc_period Specifies how frequently MTC packets are produced - see mtc - above for how to determine if MTC packets are supported. - - Valid values are given by: - - /sys/bus/event_source/devices/intel_pt/caps/mtc_periods - - which contains a hexadecimal value, the bits of which represent - valid values e.g. bit 2 set means value 2 is valid. - - The mtc_period value is converted to the MTC frequency as: - - CTC-frequency / (2 ^ value) - - e.g. value 3 means one eighth of CTC-frequency - - Where CTC is the hardware crystal clock, the frequency of which - can be related to TSC via values provided in cpuid leaf 0x15. - - If an invalid value is entered, the error message - will give a list of valid values e.g. - - $ perf record -e intel_pt/mtc_period=15/u uname - Invalid mtc_period for intel_pt. Valid values are: 0,3,6,9 - - The default value is 3 or the nearest lower value - that is supported (0 is always supported). - -cyc Produces CYC timing packets. - - CYC packets provide even finer grain timestamp information than - MTC and TSC packets. A CYC packet contains the number of CPU - cycles since the last CYC packet. Unlike MTC and TSC packets, - CYC packets are only sent when another packet is also sent. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/psb_cyc - - which contains "1" if the feature is supported and - "0" otherwise. - - The number of CYC packets produced can be reduced by specifying - a threshold - see cyc_thresh below. - -cyc_thresh Specifies how frequently CYC packets are produced - see cyc - above for how to determine if CYC packets are supported. - - Valid cyc_thresh values are given by: - - /sys/bus/event_source/devices/intel_pt/caps/cycle_thresholds - - which contains a hexadecimal value, the bits of which represent - valid values e.g. bit 2 set means value 2 is valid. - - The cyc_thresh value represents the minimum number of CPU cycles - that must have passed before a CYC packet can be sent. The - number of CPU cycles is: - - 2 ^ (value - 1) - - e.g. value 4 means 8 CPU cycles must pass before a CYC packet - can be sent. Note a CYC packet is still only sent when another - packet is sent, not at, e.g. every 8 CPU cycles. - - If an invalid value is entered, the error message - will give a list of valid values e.g. - - $ perf record -e intel_pt/cyc,cyc_thresh=15/u uname - Invalid cyc_thresh for intel_pt. Valid values are: 0-12 - - CYC packets are not requested by default. - -pt Specifies pass-through which enables the 'branch' config term. - - The default config selects 'pt' if it is available, so a user will - never need to specify this term. - -branch Enable branch tracing. Branch tracing is enabled by default so to - disable branch tracing use 'branch=0'. - - The default config selects 'branch' if it is available. - -ptw Enable PTWRITE packets which are produced when a ptwrite instruction - is executed. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/ptwrite - - which contains "1" if the feature is supported and - "0" otherwise. - - As an alternative, refer to "Emulated PTWRITE" further below. - -fup_on_ptw Enable a FUP packet to follow the PTWRITE packet. The FUP packet - provides the address of the ptwrite instruction. In the absence of - fup_on_ptw, the decoder will use the address of the previous branch - if branch tracing is enabled, otherwise the address will be zero. - Note that fup_on_ptw will work even when branch tracing is disabled. - -pwr_evt Enable power events. The power events provide information about - changes to the CPU C-state. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/power_event_trace - - which contains "1" if the feature is supported and - "0" otherwise. - -event Enable Event Trace. The events provide information about asynchronous - events. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/event_trace - - which contains "1" if the feature is supported and - "0" otherwise. - -notnt Disable TNT packets. Without TNT packets, it is not possible to walk - executable code to reconstruct control flow, however FUP, TIP, TIP.PGE - and TIP.PGD packets still indicate asynchronous control flow, and (if - return compression is disabled - see noretcomp) return statements. - The advantage of eliminating TNT packets is reducing the size of the - trace and corresponding tracing overhead. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/tnt_disable - - which contains "1" if the feature is supported and - "0" otherwise. +Config terms are parameters specified with the -e intel_pt// event option, +for example: + + -e intel_pt/cyc/ + +which selects cycle accurate mode. Each config term can have a value which +defaults to 1, so the above is the same as: + + -e intel_pt/cyc=1/ + +Some terms are set by default, so must be set to 0 to turn them off. For +example, to turn off branch tracing: + + -e intel_pt/branch=0/ + +Multiple config terms are separated by commas, for example: + + -e intel_pt/cyc,mtc_period=9/ + +There are also common config terms, see linkperf:perf-record[1] documentation. + +Intel PT config terms are described below. + +*tsc*:: +Always supported. Produces TSC timestamp packets to provide +timing information. In some cases it is possible to decode +without timing information, for example a per-thread context +that does not overlap executable memory maps. ++ +The default config selects tsc (i.e. tsc=1). + +*noretcomp*:: +Always supported. Disables "return compression" so a TIP packet +is produced when a function returns. Causes more packets to be +produced but might make decoding more reliable. ++ +The default config does not select noretcomp (i.e. noretcomp=0). + +*psb_period*:: +Allows the frequency of PSB packets to be specified. ++ +The PSB packet is a synchronization packet that provides a +starting point for decoding or recovery from errors. ++ +Support for psb_period is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/psb_cyc ++ +which contains "1" if the feature is supported and "0" +otherwise. ++ +Valid values are given by: ++ + /sys/bus/event_source/devices/intel_pt/caps/psb_periods ++ +which contains a hexadecimal value, the bits of which represent +valid values e.g. bit 2 set means value 2 is valid. ++ +The psb_period value is converted to the approximate number of +trace bytes between PSB packets as: ++ + 2 ^ (value + 11) ++ +e.g. value 3 means 16KiB bytes between PSBs ++ +If an invalid value is entered, the error message +will give a list of valid values e.g. ++ + $ perf record -e intel_pt/psb_period=15/u uname + Invalid psb_period for intel_pt. Valid values are: 0-5 ++ +If MTC packets are selected, the default config selects a value +of 3 (i.e. psb_period=3) or the nearest lower value that is +supported (0 is always supported). Otherwise the default is 0. ++ +If decoding is expected to be reliable and the buffer is large +then a large PSB period can be used. ++ +Because a TSC packet is produced with PSB, the PSB period can +also affect the granularity to timing information in the absence +of MTC or CYC. + +*mtc*:: +Produces MTC timing packets. ++ +MTC packets provide finer grain timestamp information than TSC +packets. MTC packets record time using the hardware crystal +clock (CTC) which is related to TSC packets using a TMA packet. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/mtc ++ +which contains "1" if the feature is supported and +"0" otherwise. ++ +The frequency of MTC packets can also be specified - see +mtc_period below. + +*mtc_period*:: +Specifies how frequently MTC packets are produced - see mtc +above for how to determine if MTC packets are supported. ++ +Valid values are given by: ++ + /sys/bus/event_source/devices/intel_pt/caps/mtc_periods ++ +which contains a hexadecimal value, the bits of which represent +valid values e.g. bit 2 set means value 2 is valid. ++ +The mtc_period value is converted to the MTC frequency as: + + CTC-frequency / (2 ^ value) ++ +e.g. value 3 means one eighth of CTC-frequency ++ +Where CTC is the hardware crystal clock, the frequency of which +can be related to TSC via values provided in cpuid leaf 0x15. ++ +If an invalid value is entered, the error message +will give a list of valid values e.g. ++ + $ perf record -e intel_pt/mtc_period=15/u uname + Invalid mtc_period for intel_pt. Valid values are: 0,3,6,9 ++ +The default value is 3 or the nearest lower value +that is supported (0 is always supported). + +*cyc*:: +Produces CYC timing packets. ++ +CYC packets provide even finer grain timestamp information than +MTC and TSC packets. A CYC packet contains the number of CPU +cycles since the last CYC packet. Unlike MTC and TSC packets, +CYC packets are only sent when another packet is also sent. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/psb_cyc ++ +which contains "1" if the feature is supported and +"0" otherwise. ++ +The number of CYC packets produced can be reduced by specifying +a threshold - see cyc_thresh below. + +*cyc_thresh*:: +Specifies how frequently CYC packets are produced - see cyc +above for how to determine if CYC packets are supported. ++ +Valid cyc_thresh values are given by: ++ + /sys/bus/event_source/devices/intel_pt/caps/cycle_thresholds ++ +which contains a hexadecimal value, the bits of which represent +valid values e.g. bit 2 set means value 2 is valid. ++ +The cyc_thresh value represents the minimum number of CPU cycles +that must have passed before a CYC packet can be sent. The +number of CPU cycles is: ++ + 2 ^ (value - 1) ++ +e.g. value 4 means 8 CPU cycles must pass before a CYC packet +can be sent. Note a CYC packet is still only sent when another +packet is sent, not at, e.g. every 8 CPU cycles. ++ +If an invalid value is entered, the error message +will give a list of valid values e.g. ++ + $ perf record -e intel_pt/cyc,cyc_thresh=15/u uname + Invalid cyc_thresh for intel_pt. Valid values are: 0-12 ++ +CYC packets are not requested by default. + +*pt*:: +Specifies pass-through which enables the 'branch' config term. ++ +The default config selects 'pt' if it is available, so a user will +never need to specify this term. + +*branch*:: +Enable branch tracing. Branch tracing is enabled by default so to +disable branch tracing use 'branch=0'. ++ +The default config selects 'branch' if it is available. + +*ptw*:: +Enable PTWRITE packets which are produced when a ptwrite instruction +is executed. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/ptwrite ++ +which contains "1" if the feature is supported and +"0" otherwise. ++ +As an alternative, refer to "Emulated PTWRITE" further below. + +*fup_on_ptw*:: +Enable a FUP packet to follow the PTWRITE packet. The FUP packet +provides the address of the ptwrite instruction. In the absence of +fup_on_ptw, the decoder will use the address of the previous branch +if branch tracing is enabled, otherwise the address will be zero. +Note that fup_on_ptw will work even when branch tracing is disabled. + +*pwr_evt*:: +Enable power events. The power events provide information about +changes to the CPU C-state. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/power_event_trace ++ +which contains "1" if the feature is supported and +"0" otherwise. + +*event*:: +Enable Event Trace. The events provide information about asynchronous +events. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/event_trace ++ +which contains "1" if the feature is supported and +"0" otherwise. + +*notnt*:: +Disable TNT packets. Without TNT packets, it is not possible to walk +executable code to reconstruct control flow, however FUP, TIP, TIP.PGE +and TIP.PGD packets still indicate asynchronous control flow, and (if +return compression is disabled - see noretcomp) return statements. +The advantage of eliminating TNT packets is reducing the size of the +trace and corresponding tracing overhead. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/tnt_disable ++ +which contains "1" if the feature is supported and +"0" otherwise. + + +config terms on other events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Some Intel PT features work with other events, features such as AUX area sampling +and PEBS-via-PT. In those cases, the other events can have config terms below: + +*aux-sample-size*:: + Used to set the AUX area sample size, refer to the section + <<_aux_area_sampling_option,AUX area sampling option>> + +*aux-output*:: + Used to select PEBS-via-PT, refer to the + section <<_pebs_via_intel_pt,PEBS via Intel PT>> AUX area sampling option
@@ -596,7 +643,8 @@ The default snapshot size is the auxtrace mmap size. If neither auxtrace mmap s nor snapshot size is specified, then the default is 4MiB for privileged users (or if /proc/sys/kernel/perf_event_paranoid < 0), 128KiB for unprivileged users. If an unprivileged user does not specify mmap pages, the mmap pages will be -reduced as described in the 'new auxtrace mmap size option' section below. +reduced as described in the <<_new_auxtrace_mmap_size_option,new auxtrace mmap size option>> +section below. The snapshot size is displayed if the option -vv is used e.g.
@@ -952,11 +1000,11 @@ transaction start, commit or abort. Note that "instructions", "cycles", "branches" and "transactions" events depend on code flow packets which can be disabled by using the config term -"branch=0". Refer to the config terms section above. +"branch=0". Refer to the <<_config_terms,config terms>> section above. "ptwrite" events record the payload of the ptwrite instruction and whether "fup_on_ptw" was used. "ptwrite" events depend on PTWRITE packets which are -recorded only if the "ptw" config term was used. Refer to the config terms +recorded only if the "ptw" config term was used. Refer to the <<_config_terms,config terms>> section above. perf script "synth" field displays "ptwrite" information like this: "ip: 0 payload: 0x123456789abcdef0" where "ip" is 1 if "fup_on_ptw" was used.
@@ -964,7 +1012,7 @@ used. "Power" events correspond to power event packets and CBR (core-to-bus ratio) packets. While CBR packets are always recorded when tracing is enabled, power event packets are recorded only if the "pwr_evt" config term was used. Refer to -the config terms section above. The power events record information about +the <<_config_terms,config terms>> section above. The power events record information about C-state changes, whereas CBR is indicative of CPU frequency. perf script "event,synth" fields display information like this:
@@ -1120,7 +1168,7 @@ What *will* be decoded with the (single) q option: - asynchronous branches such as interrupts - indirect branches - function return target address *if* the noretcomp config term (refer - config terms section) was used + <<_config_terms,config terms>> section) was used - start of (control-flow) tracing - end of (control-flow) tracing, if it is not out of context - power events, ptwrite, transaction start and abort
@@ -1133,7 +1181,7 @@ Repeating the q option (double-q i.e. qq) results in even faster decoding and ev less detail. The decoder decodes only extended PSB (PSB+) packets, getting the instruction pointer if there is a FUP packet within PSB+ (i.e. between PSB and PSBEND). Note PSB packets occur regularly in the trace based on the psb_period -config term (refer config terms section). There will be a FUP packet if the +config term (refer <<_config_terms,config terms>> section). There will be a FUP packet if the PSB+ occurs while control flow is being traced. What will *not* be decoded with the qq option:
Signature · 1 line
2.43.0
Adrian Hunter
reviewerPATCH v16 6/7 perf intel-pt: Add documentation for pause / resume
In reply to Adrian HunterDocument the use of aux-action config term and provide a simple example.
Signed-off-by Adrian Hunter <[email protected]>
Acked-by Ian Rogers <[email protected]>
Reviewed-by Andi Kleen <[email protected]>
---
Changes in V5:
Added more examples
tools/perf/Documentation/perf-intel-pt.txt | 108 +++++++++++++++++++++ 1 file changed, 108 insertions(+)
tools/perf/Documentation/perf-intel-pt.txt +108 -0
@@ -555,6 +555,9 @@ Support for this feature is indicated by: which contains "1" if the feature is supported and "0" otherwise. +*aux-action=start-paused*:: +Start tracing paused, refer to the section <<_pause_or_resume_tracing,Pause or Resume Tracing>> + config terms on other events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -570,6 +573,9 @@ and PEBS-via-PT. In those cases, the other events can have config terms below: Used to select PEBS-via-PT, refer to the section <<_pebs_via_intel_pt,PEBS via Intel PT>> +*aux-action*:: + Used to pause or resume tracing, refer to the section + <<_pause_or_resume_tracing,Pause or Resume Tracing>> AUX area sampling option ~~~~~~~~~~~~~~~~~~~~~~~~
@@ -1915,6 +1921,108 @@ For pipe mode, the order of events and timestamps can presumably be messed up. +Pause or Resume Tracing +----------------------- + +With newer Kernels, it is possible to use other selected events to pause +or resume Intel PT tracing. This is configured by using the "aux-action" +config term: + +"aux-action=pause" is used with events that are to pause Intel PT tracing. + +"aux-action=resume" is used with events that are to resume Intel PT tracing. + +"aux-action=start-paused" is used with the Intel PT event to start in a +paused state. + +For example, to trace only the uname system call (sys_newuname) when running the +command line utility uname: + + $ perf record --kcore -e intel_pt/aux-action=start-paused/k,syscalls:sys_enter_newuname/aux-action=resume/,syscalls:sys_exit_newuname/aux-action=pause/ uname + Linux + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 0.043 MB perf.data ] + $ perf script --call-trace + uname 30805 [000] 24001.058782799: name: 0x7ffc9c1865b0 + uname 30805 [000] 24001.058784424: psb offs: 0 + uname 30805 [000] 24001.058784424: cbr: 39 freq: 3904 MHz (139%) + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) __x64_sys_newuname + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) down_read + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) __cond_resched + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) preempt_count_add + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) in_lock_functions + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) preempt_count_sub + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) up_read + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) preempt_count_add + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) in_lock_functions + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) preempt_count_sub + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) _copy_to_user + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) syscall_exit_to_user_mode + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) syscall_exit_work + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) perf_syscall_exit + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_trace_buf_alloc + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_swevent_get_recursion_context + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_tp_event + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_trace_buf_update + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) tracing_gen_ctx_irq_test + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_swevent_event + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __perf_event_account_interrupt + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __this_cpu_preempt_check + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_event_output_forward + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_event_aux_pause + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) ring_buffer_get + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __rcu_read_lock + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __rcu_read_unlock + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) pt_event_stop + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) native_write_msr + uname 30805 [000] 24001.058785463: ([kernel.kallsyms]) native_write_msr + uname 30805 [000] 24001.058785639: 0x0 + +The example above uses tracepoints, but any kind of sampled event can be used. + +For example: + + Tracing between arch_cpu_idle_enter() and arch_cpu_idle_exit() using breakpoint events: + + $ sudo cat /proc/kallsyms | sort | grep ' arch_cpu_idle_enter\| arch_cpu_idle_exit' + ffffffffb605bf60 T arch_cpu_idle_enter + ffffffffb614d8a0 W arch_cpu_idle_exit + $ sudo perf record --kcore -a -e intel_pt/aux-action=start-paused/k -e mem:0xffffffffb605bf60:x/aux-action=resume/ -e mem:0xffffffffb614d8a0:x/aux-action=pause/ -- sleep 1 + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 1.387 MB perf.data ] + + Tracing __alloc_pages() using kprobes: + + $ sudo perf probe --add '__alloc_pages order' + Added new event: probe:__alloc_pages (on __alloc_pages with order) + $ sudo perf probe --add __alloc_pages%return + Added new event: probe:__alloc_pages__return (on __alloc_pages%return) + $ sudo perf record --kcore -aR -e intel_pt/aux-action=start-paused/k -e probe:__alloc_pages/aux-action=resume/ -e probe:__alloc_pages__return/aux-action=pause/ -- sleep 1 + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 1.490 MB perf.data ] + + Tracing starting at main() using a uprobe event: + + $ sudo perf probe -x /usr/bin/uname main + Added new event: probe_uname:main (on main in /usr/bin/uname) + $ sudo perf record -e intel_pt/-aux-action=start-paused/u -e probe_uname:main/aux-action=resume/ -- uname + Linux + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 0.031 MB perf.data ] + + Tracing occasionally using cycles events with different periods: + + $ perf record --kcore -a -m,64M -e intel_pt/aux-action=start-paused/k -e cycles/aux-action=pause,period=1000000/Pk -e cycles/aux-action=resume,period=10500000/Pk -- firefox + [ perf record: Woken up 19 times to write data ] + [ perf record: Captured and wrote 16.561 MB perf.data ] + + EXAMPLE -------
Signature · 1 line
2.43.0
Adrian Hunter
reviewerPATCH v16 7/7 perf intel-pt: Add a test for pause / resume
In reply to Adrian HunterAdd a simple sub-test to the "Miscellaneous Intel PT testing" test to check pause / resume.
Signed-off-by Adrian Hunter <[email protected]>
Acked-by Ian Rogers <[email protected]>
Reviewed-by Andi Kleen <[email protected]>
--- tools/perf/tests/shell/test_intel_pt.sh | 28 +++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
tools/perf/tests/shell/test_intel_pt.sh +28 -0
@@ -644,6 +644,33 @@ test_pipe() return 0 } +test_pause_resume() +{ + echo "--- Test with pause / resume ---" + if ! perf_record_no_decode -o "${perfdatafile}" -e intel_pt/aux-action=start-paused/u uname ; then + echo "SKIP: pause / resume is not supported" + return 2 + fi + if ! perf_record_no_bpf -o "${perfdatafile}" \ + -e intel_pt/aux-action=start-paused/u \ + -e instructions/period=50000,aux-action=resume,name=Resume/u \ + -e instructions/period=100000,aux-action=pause,name=Pause/u uname ; then + echo "perf record with pause / resume failed" + return 1 + fi + if ! perf script -i "${perfdatafile}" --itrace=b -Fperiod,event | \ + awk 'BEGIN {paused=1;branches=0} + /Resume/ {paused=0} + /branches/ {if (paused) exit 1;branches=1} + /Pause/ {paused=1} + END {if (!branches) exit 1}' ; then + echo "perf record with pause / resume failed" + return 1 + fi + echo OK + return 0 +} + count_result() { if [ "$1" -eq 2 ] ; then
@@ -672,6 +699,7 @@ test_power_event || ret=$? ; count_result $ret ; ret=0 test_no_tnt || ret=$? ; count_result $ret ; ret=0 test_event_trace || ret=$? ; count_result $ret ; ret=0 test_pipe || ret=$? ; count_result $ret ; ret=0 +test_pause_resume || ret=$? ; count_result $ret ; ret=0 cleanup
Signature · 1 line
2.43.0
Ian Rogers
reviewerRe: [PATCH V16 7/7] perf intel-pt: Add a test for pause / resume
Quoted 33 lines · Adrian Hunter
Add a simple sub-test to the "Miscellaneous Intel PT testing" test to check pause / resume.
Signed-off-by Adrian Hunter <[email protected]>Acked-by Ian Rogers <[email protected]>Reviewed-by Andi Kleen <[email protected]>--- tools/perf/tests/shell/test_intel_pt.sh | 28 +++++++++++++++++++++++++ 1 file changed, 28 insertions(+)tools/perf/tests/shell/test_intel_pt.sh +14
-0@@ -644,6 +644,33 @@ test_pipe() return 0 } +test_pause_resume() +{ + echo "--- Test with pause / resume ---" + if ! perf_record_no_decode -o "${perfdatafile}" -e intel_pt/aux-action=start-paused/u uname ; then + echo "SKIP: pause / resume is not supported" + return 2 + fi + if ! perf_record_no_bpf -o "${perfdatafile}" \ + -e intel_pt/aux-action=start-paused/u \ + -e instructions/period=50000,aux-action=resume,name=Resume/u \ + -e instructions/period=100000,aux-action=pause,name=Pause/u uname ; then + echo "perf record with pause / resume failed" + return 1 + fi
tools/perf/tests/shell/test_intel_pt.sh test_pipe() · line 670
Quoted 10 lines
tools/perf/tests/shell/test_intel_pt.sh +10
-0+ if ! perf script -i "${perfdatafile}" --itrace=b -Fperiod,event | \ + awk 'BEGIN {paused=1;branches=0} + /Resume/ {paused=0} + /branches/ {if (paused) exit 1;branches=1} + /Pause/ {paused=1} + END {if (!branches) exit 1}' ; then + echo "perf record with pause / resume failed" + return 1 + fi + echo OK
Hi,
this new test is now constantly making "Miscellaneous Intel PT testing" fail:
``` ... --- Test with pause / resume --- Error: Failure to open event 'intel_pt/aux-action=start-paused/u' on PMU 'intel_pt' which will be removed. The 'aux_action' feature is not supported, update the kernel. Linux [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.003 MB /tmp/perf-test-intel-pt-sh.Hs8jcq0ADc/test-perf.data ] Error: Failure to open event 'intel_pt/aux-action=start-paused/u' on PMU 'intel_pt' which will be removed. The 'aux_action' feature is not supported, update the kernel. Error: Failure to open event 'Resume' on PMU 'cpu' which will be removed. The 'aux_action' feature is not supported, update the kernel. Error: Failure to open event 'Pause' on PMU 'cpu' which will be removed. The 'aux_action' feature is not supported, update the kernel. Linux [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.005 MB /tmp/perf-test-intel-pt-sh.Hs8jcq0ADc/test-perf.data ] perf record with pause / resume failed --- Cleaning up --- ... ```
Should the fail be turned into a skip for missing kernel support?
Thanks,
Ian
Quoted 17 lines
tools/perf/tests/shell/test_intel_pt.sh +4
-1+ return 0 +} + count_result() { if [ "$1" -eq 2 ] ; then@@ -672,6 +699,7 @@ test_power_event || ret=$? ; count_result $ret ; ret=0 test_no_tnt || ret=$? ; count_result $ret ; ret=0 test_event_trace || ret=$? ; count_result $ret ; ret=0 test_pipe || ret=$? ; count_result $ret ; ret=0 +test_pause_resume || ret=$? ; count_result $ret ; ret=0 cleanup --2.43.0
Adrian Hunter
reviewerRe: [PATCH V16 7/7] perf intel-pt: Add a test for pause / resume
Quoted 79 lines · Ian Rogers
Quoted 33 lines · Adrian Hunter
Add a simple sub-test to the "Miscellaneous Intel PT testing" test to check pause / resume.
Signed-off-by Adrian Hunter <[email protected]>Acked-by Ian Rogers <[email protected]>Reviewed-by Andi Kleen <[email protected]>--- tools/perf/tests/shell/test_intel_pt.sh | 28 +++++++++++++++++++++++++ 1 file changed, 28 insertions(+)tools/perf/tests/shell/test_intel_pt.sh +14
-0@@ -644,6 +644,33 @@ test_pipe() return 0 } +test_pause_resume() +{ + echo "--- Test with pause / resume ---" + if ! perf_record_no_decode -o "${perfdatafile}" -e intel_pt/aux-action=start-paused/u uname ; then + echo "SKIP: pause / resume is not supported" + return 2 + fi + if ! perf_record_no_bpf -o "${perfdatafile}" \ + -e intel_pt/aux-action=start-paused/u \ + -e instructions/period=50000,aux-action=resume,name=Resume/u \ + -e instructions/period=100000,aux-action=pause,name=Pause/u uname ; then + echo "perf record with pause / resume failed" + return 1 + fitools/perf/tests/shell/test_intel_pt.sh test_pipe() · line 670
Quoted 10 lines
tools/perf/tests/shell/test_intel_pt.sh +10
-0+ if ! perf script -i "${perfdatafile}" --itrace=b -Fperiod,event | \ + awk 'BEGIN {paused=1;branches=0} + /Resume/ {paused=0} + /branches/ {if (paused) exit 1;branches=1} + /Pause/ {paused=1} + END {if (!branches) exit 1}' ; then + echo "perf record with pause / resume failed" + return 1 + fi + echo OKHi,
this new test is now constantly making "Miscellaneous Intel PT testing" fail:
``` ... --- Test with pause / resume --- Error: Failure to open event 'intel_pt/aux-action=start-paused/u' on PMU 'intel_pt' which will be removed. The 'aux_action' feature is not supported, update the kernel. Linux [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.003 MB /tmp/perf-test-intel-pt-sh.Hs8jcq0ADc/test-perf.data ] Error: Failure to open event 'intel_pt/aux-action=start-paused/u' on PMU 'intel_pt' which will be removed. The 'aux_action' feature is not supported, update the kernel. Error: Failure to open event 'Resume' on PMU 'cpu' which will be removed. The 'aux_action' feature is not supported, update the kernel. Error: Failure to open event 'Pause' on PMU 'cpu' which will be removed. The 'aux_action' feature is not supported, update the kernel. Linux [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.005 MB /tmp/perf-test-intel-pt-sh.Hs8jcq0ADc/test-perf.data ] perf record with pause / resume failed --- Cleaning up --- ... ```
Quoted 1 line
Should the fail be turned into a skip for missing kernel support?
Seems to skip for me with perf-tools-next:
--- Test with pause / resume ---
Error:
The 'aux_action' feature is not supported, update the kernel.
SKIP: pause / resume is not supported
perf version 6.13.rc2.g39c2547579aa
Quoted 21 lines
Thanks,
IanQuoted 17 lines
tools/perf/tests/shell/test_intel_pt.sh +4
-1+ return 0 +} + count_result() { if [ "$1" -eq 2 ] ; then@@ -672,6 +699,7 @@ test_power_event || ret=$? ; count_result $ret ; ret=0 test_no_tnt || ret=$? ; count_result $ret ; ret=0 test_event_trace || ret=$? ; count_result $ret ; ret=0 test_pipe || ret=$? ; count_result $ret ; ret=0 +test_pause_resume || ret=$? ; count_result $ret ; ret=0 cleanup --2.43.0
Ian Rogers
reviewerRe: [PATCH V16 7/7] perf intel-pt: Add a test for pause / resume
Quoted 83 lines · Adrian Hunter
Quoted 79 lines · Ian Rogers
Quoted 33 lines · Adrian Hunter
Add a simple sub-test to the "Miscellaneous Intel PT testing" test to check pause / resume.
Signed-off-by Adrian Hunter <[email protected]>Acked-by Ian Rogers <[email protected]>Reviewed-by Andi Kleen <[email protected]>--- tools/perf/tests/shell/test_intel_pt.sh | 28 +++++++++++++++++++++++++ 1 file changed, 28 insertions(+)tools/perf/tests/shell/test_intel_pt.sh +14
-0@@ -644,6 +644,33 @@ test_pipe() return 0 } +test_pause_resume() +{ + echo "--- Test with pause / resume ---" + if ! perf_record_no_decode -o "${perfdatafile}" -e intel_pt/aux-action=start-paused/u uname ; then + echo "SKIP: pause / resume is not supported" + return 2 + fi + if ! perf_record_no_bpf -o "${perfdatafile}" \ + -e intel_pt/aux-action=start-paused/u \ + -e instructions/period=50000,aux-action=resume,name=Resume/u \ + -e instructions/period=100000,aux-action=pause,name=Pause/u uname ; then + echo "perf record with pause / resume failed" + return 1 + fitools/perf/tests/shell/test_intel_pt.sh test_pipe() · line 670
Quoted 10 lines
tools/perf/tests/shell/test_intel_pt.sh +10
-0+ if ! perf script -i "${perfdatafile}" --itrace=b -Fperiod,event | \ + awk 'BEGIN {paused=1;branches=0} + /Resume/ {paused=0} + /branches/ {if (paused) exit 1;branches=1} + /Pause/ {paused=1} + END {if (!branches) exit 1}' ; then + echo "perf record with pause / resume failed" + return 1 + fi + echo OKHi,
this new test is now constantly making "Miscellaneous Intel PT testing" fail:
``` ... --- Test with pause / resume --- Error: Failure to open event 'intel_pt/aux-action=start-paused/u' on PMU 'intel_pt' which will be removed. The 'aux_action' feature is not supported, update the kernel. Linux [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.003 MB /tmp/perf-test-intel-pt-sh.Hs8jcq0ADc/test-perf.data ] Error: Failure to open event 'intel_pt/aux-action=start-paused/u' on PMU 'intel_pt' which will be removed. The 'aux_action' feature is not supported, update the kernel. Error: Failure to open event 'Resume' on PMU 'cpu' which will be removed. The 'aux_action' feature is not supported, update the kernel. Error: Failure to open event 'Pause' on PMU 'cpu' which will be removed. The 'aux_action' feature is not supported, update the kernel. Linux [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.005 MB /tmp/perf-test-intel-pt-sh.Hs8jcq0ADc/test-perf.data ] perf record with pause / resume failed --- Cleaning up --- ... ```Quoted 1 line
Should the fail be turned into a skip for missing kernel support?
Quoted 8 lines
Seems to skip for me with perf-tools-next:
--- Test with pause / resume ---
Error:
The 'aux_action' feature is not supported, update the kernel.
SKIP: pause / resume is not supportedperf version 6.13.rc2.g39c2547579aa
My mistake, I thought I tested a clean client but I was testing with:
https://lore.kernel.org/lkml/[email protected]/
The issue there is that intel-pt is opening multiple dummy events in this test:
```
Breakpoint 2, evlist__new () at util/evlist.c:89
89 struct evlist *evlist = zalloc(sizeof(*evlist));
(gdb) n
91 if (evlist != NULL)
(gdb) p evlist->core.nr_entries
$8 = 0
(gdb) p &evlist->core.nr_entries
$9 = (int *) 0x55555618d150
(gdb) watch *$9
Hardware watchpoint 3: *$9
(gdb) c
Continuing.
Hardware watchpoint 3: *$9
Old value = 0
New value = 1
perf_evlist__add (evlist=0x55555618d140, evsel=0x5555561a6910) at evlist.c:112
112 if (evlist->needs_map_propagation)
(gdb) bt
#0 perf_evlist__add (evlist=0x55555618d140, evsel=0x5555561a6910) at
evlist.c:112
#1 0x00005555557c91ba in evlist__add (evlist=0x55555618d140,
entry=0x5555561a6910)
at util/evlist.c:197
#2 0x00005555557c9320 in evlist__splice_list_tail
(evlist=0x55555618d140, list=0x7fffffffbaa8)
at util/evlist.c:218
#3 0x00005555557eb886 in __parse_events (evlist=0x55555618d140,
str=0x7fffffffe5c1 "intel_pt/aux-action=start-paused/u",
pmu_filter=0x0, err=0x7fffffffbb38,
fake_pmu=false, warn_if_reordered=true, fake_tp=false) at
util/parse-events.c:2166
#4 0x00005555557ec4b4 in parse_events_option (opt=0x5555560e5bb0
<__record_options>,
str=0x7fffffffe5c1 "intel_pt/aux-action=start-paused/u", unset=0)
at util/parse-events.c:2352
#5 0x00005555556acede in get_value (p=0x7fffffffbde0,
opt=0x5555560e5bb0 <__record_options>,
flags=1) at parse-options.c:251
#6 0x00005555556ac13f in parse_short_opt (p=0x7fffffffbde0,
options=0x5555560e5bb0 <__record_options>) at parse-options.c:351
#7 0x00005555556aaecb in parse_options_step (ctx=0x7fffffffbde0,
options=0x5555560e5bb0 <__record_options>, usagestr=0x5555560b5bb0
<__record_usage>)
at parse-options.c:539
#8 0x00005555556aa940 in parse_options_subcommand (argc=4, argv=0x7fffffffe310,
options=0x5555560e5bb0 <__record_options>, subcommands=0x0,
usagestr=0x5555560b5bb0 <__record_usage>, flags=2) at parse-options.c:654
#9 0x00005555556ab3d7 in parse_options (argc=4, argv=0x7fffffffe310,
options=0x5555560e5bb0 <__record_options>, usagestr=0x5555560b5bb0
<__record_usage>, flags=2)
at parse-options.c:689
#10 0x00005555555f23dd in cmd_record (argc=4, argv=0x7fffffffe310) at
builtin-record.c:4021
#11 0x000055555569d42f in run_builtin (p=0x5555560f0cb0 <commands+288>, argc=4,
argv=0x7fffffffe310) at perf.c:351
#12 0x000055555569cae3 in handle_internal_command (argc=4,
argv=0x7fffffffe310) at perf.c:404
#13 0x000055555569d2ff in run_argv (argcp=0x7fffffffe15c,
argv=0x7fffffffe150) at perf.c:448
#14 0x000055555569c73f in main (argc=4, argv=0x7fffffffe310) at perf.c:560
(gdb) c
Continuing.
Breakpoint 2, evlist__new () at util/evlist.c:89
89 struct evlist *evlist = zalloc(sizeof(*evlist));
(gdb) bt
#0 evlist__new () at util/evlist.c:89
#1 0x00005555558a8365 in perf_do_probe_api (fn=0x5555558a7e50
<perf_probe_context_switch>,
cpu=..., str=0x5555559ba4fb "cycles:u") at util/perf_api_probe.c:22
#2 0x00005555558a7c94 in perf_probe_api (fn=0x5555558a7e50
<perf_probe_context_switch>)
at util/perf_api_probe.c:74
#3 0x00005555558a7e21 in perf_can_record_switch_events () at
util/perf_api_probe.c:124
#4 0x000055555597273e in intel_pt_recording_options
(itr=0x5555561a9100, evlist=0x55555618d140,
opts=0x5555560e7b58 <record+320>) at arch/x86/util/intel-pt.c:793
#5 0x00005555558be64d in auxtrace_record__options
(itr=0x5555561a9100, evlist=0x55555618d140,
opts=0x5555560e7b58 <record+320>) at util/auxtrace.c:619
#6 0x00005555555f2da3 in cmd_record (argc=1, argv=0x7fffffffe310) at
builtin-record.c:4226
#7 0x000055555569d42f in run_builtin (p=0x5555560f0cb0 <commands+288>, argc=4,
argv=0x7fffffffe310) at perf.c:351
#8 0x000055555569cae3 in handle_internal_command (argc=4,
argv=0x7fffffffe310) at perf.c:404
#9 0x000055555569d2ff in run_argv (argcp=0x7fffffffe15c,
argv=0x7fffffffe150) at perf.c:448
#10 0x000055555569c73f in main (argc=4, argv=0x7fffffffe310) at perf.c:560
(gdb) c
Continuing.
Hardware watchpoint 3: *$9
Old value = 1
New value = 2
perf_evlist__add (evlist=0x55555618d140, evsel=0x5555561a6c10) at evlist.c:112
112 if (evlist->needs_map_propagation)
(gdb) bt
#0 perf_evlist__add (evlist=0x55555618d140, evsel=0x5555561a6c10) at
evlist.c:112
#1 0x00005555557c91ba in evlist__add (evlist=0x55555618d140,
entry=0x5555561a6c10)
at util/evlist.c:197
#2 0x00005555557c980a in evlist__add_aux_dummy
(evlist=0x55555618d140, system_wide=true)
at util/evlist.c:299
#3 0x0000555555974297 in evlist__add_dummy_on_all_cpus (evlist=0x55555618d140)
at arch/x86/util/../../../util/evlist.h:111
#4 0x00005555559727ae in intel_pt_recording_options
(itr=0x5555561a9100, evlist=0x55555618d140,
opts=0x5555560e7b58 <record+320>) at arch/x86/util/intel-pt.c:800
#5 0x00005555558be64d in auxtrace_record__options
(itr=0x5555561a9100, evlist=0x55555618d140,
opts=0x5555560e7b58 <record+320>) at util/auxtrace.c:619
#6 0x00005555555f2da3 in cmd_record (argc=1, argv=0x7fffffffe310) at
builtin-record.c:4226
#7 0x000055555569d42f in run_builtin (p=0x5555560f0cb0 <commands+288>, argc=4,
argv=0x7fffffffe310) at perf.c:351
#8 0x000055555569cae3 in handle_internal_command (argc=4,
argv=0x7fffffffe310) at perf.c:404
#9 0x000055555569d2ff in run_argv (argcp=0x7fffffffe15c,
argv=0x7fffffffe150) at perf.c:448
#10 0x000055555569c73f in main (argc=4, argv=0x7fffffffe310) at perf.c:560
(gdb) c
Continuing.
Hardware watchpoint 3: *$9
Old value = 2
New value = 3
perf_evlist__add (evlist=0x55555618d140, evsel=0x5555561a6ef0) at evlist.c:112
112 if (evlist->needs_map_propagation)
(gdb) bt
#0 perf_evlist__add (evlist=0x55555618d140, evsel=0x5555561a6ef0) at
evlist.c:112
#1 0x00005555557c91ba in evlist__add (evlist=0x55555618d140,
entry=0x5555561a6ef0)
at util/evlist.c:197
#2 0x00005555557c980a in evlist__add_aux_dummy
(evlist=0x55555618d140, system_wide=false)
at util/evlist.c:299
#3 0x00005555559729f8 in intel_pt_recording_options
(itr=0x5555561a9100, evlist=0x55555618d140,
opts=0x5555560e7b58 <record+320>) at arch/x86/util/intel-pt.c:865
#4 0x00005555558be64d in auxtrace_record__options
(itr=0x5555561a9100, evlist=0x55555618d140,
opts=0x5555560e7b58 <record+320>) at util/auxtrace.c:619
#5 0x00005555555f2da3 in cmd_record (argc=1, argv=0x7fffffffe310) at
builtin-record.c:4226
#6 0x000055555569d42f in run_builtin (p=0x5555560f0cb0 <commands+288>, argc=4,
argv=0x7fffffffe310) at perf.c:351
#7 0x000055555569cae3 in handle_internal_command (argc=4,
argv=0x7fffffffe310) at perf.c:404
#8 0x000055555569d2ff in run_argv (argcp=0x7fffffffe15c,
argv=0x7fffffffe150) at perf.c:448
#9 0x000055555569c73f in main (argc=4, argv=0x7fffffffe310) at perf.c:560
(gdb)
```That is a dummy event is added in 2 places:
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/arch/x86/util/intel-pt.c?h=perf-tools-next#n800
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/arch/x86/util/intel-pt.c?h=perf-tools-next#n865
I'll need to update my patch set for things outside of record adding dummy events, but I'm curious if the intel-pt code could share a single dummy evsel?
Thanks,
Ian
Adrian Hunter
reviewerRe: [PATCH V16 7/7] perf intel-pt: Add a test for pause / resume
Quoted 98 lines · Ian Rogers
Quoted 83 lines · Adrian Hunter
Quoted 79 lines · Ian Rogers
Quoted 33 lines · Adrian Hunter
Add a simple sub-test to the "Miscellaneous Intel PT testing" test to check pause / resume.
Signed-off-by Adrian Hunter <[email protected]>Acked-by Ian Rogers <[email protected]>Reviewed-by Andi Kleen <[email protected]>--- tools/perf/tests/shell/test_intel_pt.sh | 28 +++++++++++++++++++++++++ 1 file changed, 28 insertions(+)tools/perf/tests/shell/test_intel_pt.sh +14
-0@@ -644,6 +644,33 @@ test_pipe() return 0 } +test_pause_resume() +{ + echo "--- Test with pause / resume ---" + if ! perf_record_no_decode -o "${perfdatafile}" -e intel_pt/aux-action=start-paused/u uname ; then + echo "SKIP: pause / resume is not supported" + return 2 + fi + if ! perf_record_no_bpf -o "${perfdatafile}" \ + -e intel_pt/aux-action=start-paused/u \ + -e instructions/period=50000,aux-action=resume,name=Resume/u \ + -e instructions/period=100000,aux-action=pause,name=Pause/u uname ; then + echo "perf record with pause / resume failed" + return 1 + fitools/perf/tests/shell/test_intel_pt.sh test_pipe() · line 670
Quoted 10 lines
tools/perf/tests/shell/test_intel_pt.sh +10
-0+ if ! perf script -i "${perfdatafile}" --itrace=b -Fperiod,event | \ + awk 'BEGIN {paused=1;branches=0} + /Resume/ {paused=0} + /branches/ {if (paused) exit 1;branches=1} + /Pause/ {paused=1} + END {if (!branches) exit 1}' ; then + echo "perf record with pause / resume failed" + return 1 + fi + echo OKHi,
this new test is now constantly making "Miscellaneous Intel PT testing" fail:
``` ... --- Test with pause / resume --- Error: Failure to open event 'intel_pt/aux-action=start-paused/u' on PMU 'intel_pt' which will be removed. The 'aux_action' feature is not supported, update the kernel. Linux [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.003 MB /tmp/perf-test-intel-pt-sh.Hs8jcq0ADc/test-perf.data ] Error: Failure to open event 'intel_pt/aux-action=start-paused/u' on PMU 'intel_pt' which will be removed. The 'aux_action' feature is not supported, update the kernel. Error: Failure to open event 'Resume' on PMU 'cpu' which will be removed. The 'aux_action' feature is not supported, update the kernel. Error: Failure to open event 'Pause' on PMU 'cpu' which will be removed. The 'aux_action' feature is not supported, update the kernel. Linux [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.005 MB /tmp/perf-test-intel-pt-sh.Hs8jcq0ADc/test-perf.data ] perf record with pause / resume failed --- Cleaning up --- ... ```Quoted 1 line
Should the fail be turned into a skip for missing kernel support?
Quoted 8 lines
Seems to skip for me with perf-tools-next:
--- Test with pause / resume ---
Error:
The 'aux_action' feature is not supported, update the kernel.
SKIP: pause / resume is not supportedperf version 6.13.rc2.g39c2547579aa
My mistake, I thought I tested a clean client but I was testing with:
https://lore.kernel.org/lkml/[email protected]/
The issue there is that intel-pt is opening multiple dummy events in this test:
Quoted 7 lines
That is a dummy event is added in 2 places:
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/arch/x86/util/intel-pt.c?h=perf-tools-next#n800
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/arch/x86/util/intel-pt.c?h=perf-tools-next#n865I'll need to update my patch set for things outside of record adding dummy events, but I'm curious if the intel-pt code could share a single dummy evsel?
I doubt it. There are cases where capturing auxiliary events like text_poke or context_switch want to be system-wide while the workload task events (comm, task, mmap2) are not.
Ian Rogers
reviewerRe: [PATCH V16 7/7] perf intel-pt: Add a test for pause / resume
Quoted 104 lines · Adrian Hunter
Quoted 102 lines · Ian Rogers
Quoted 83 lines · Adrian Hunter
Quoted 79 lines · Ian Rogers
Quoted 33 lines · Adrian Hunter
Add a simple sub-test to the "Miscellaneous Intel PT testing" test to check pause / resume.
Signed-off-by Adrian Hunter <[email protected]>Acked-by Ian Rogers <[email protected]>Reviewed-by Andi Kleen <[email protected]>--- tools/perf/tests/shell/test_intel_pt.sh | 28 +++++++++++++++++++++++++ 1 file changed, 28 insertions(+)tools/perf/tests/shell/test_intel_pt.sh +14
-0@@ -644,6 +644,33 @@ test_pipe() return 0 } +test_pause_resume() +{ + echo "--- Test with pause / resume ---" + if ! perf_record_no_decode -o "${perfdatafile}" -e intel_pt/aux-action=start-paused/u uname ; then + echo "SKIP: pause / resume is not supported" + return 2 + fi + if ! perf_record_no_bpf -o "${perfdatafile}" \ + -e intel_pt/aux-action=start-paused/u \ + -e instructions/period=50000,aux-action=resume,name=Resume/u \ + -e instructions/period=100000,aux-action=pause,name=Pause/u uname ; then + echo "perf record with pause / resume failed" + return 1 + fitools/perf/tests/shell/test_intel_pt.sh test_pipe() · line 670
Quoted 10 lines
tools/perf/tests/shell/test_intel_pt.sh +10
-0+ if ! perf script -i "${perfdatafile}" --itrace=b -Fperiod,event | \ + awk 'BEGIN {paused=1;branches=0} + /Resume/ {paused=0} + /branches/ {if (paused) exit 1;branches=1} + /Pause/ {paused=1} + END {if (!branches) exit 1}' ; then + echo "perf record with pause / resume failed" + return 1 + fi + echo OKHi,
this new test is now constantly making "Miscellaneous Intel PT testing" fail:
``` ... --- Test with pause / resume --- Error: Failure to open event 'intel_pt/aux-action=start-paused/u' on PMU 'intel_pt' which will be removed. The 'aux_action' feature is not supported, update the kernel. Linux [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.003 MB /tmp/perf-test-intel-pt-sh.Hs8jcq0ADc/test-perf.data ] Error: Failure to open event 'intel_pt/aux-action=start-paused/u' on PMU 'intel_pt' which will be removed. The 'aux_action' feature is not supported, update the kernel. Error: Failure to open event 'Resume' on PMU 'cpu' which will be removed. The 'aux_action' feature is not supported, update the kernel. Error: Failure to open event 'Pause' on PMU 'cpu' which will be removed. The 'aux_action' feature is not supported, update the kernel. Linux [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.005 MB /tmp/perf-test-intel-pt-sh.Hs8jcq0ADc/test-perf.data ] perf record with pause / resume failed --- Cleaning up --- ... ```Quoted 1 line
Should the fail be turned into a skip for missing kernel support?
Quoted 8 lines
Seems to skip for me with perf-tools-next:
--- Test with pause / resume ---
Error:
The 'aux_action' feature is not supported, update the kernel.
SKIP: pause / resume is not supportedperf version 6.13.rc2.g39c2547579aa
My mistake, I thought I tested a clean client but I was testing with:
https://lore.kernel.org/lkml/[email protected]/
The issue there is that intel-pt is opening multiple dummy events in this test:That is a dummy event is added in 2 places:
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/arch/x86/util/intel-pt.c?h=perf-tools-next#n800
https://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/arch/x86/util/intel-pt.c?h=perf-tools-next#n865
Quoted 7 lines
Quoted 3 lines
I'll need to update my patch set for things outside of record adding dummy events, but I'm curious if the intel-pt code could share a single dummy evsel?
I doubt it. There are cases where capturing auxiliary events like
text_pokeorcontext_switchwant to be system-wide while the workload task events (comm, task, mmap2) are not.
Thanks, I'll rework the dummy event detection because of this. I'm a little concerned about APIs like evlist__findnew_tracking_event, nit that we're inconsistent on naming (dummy vs tracking vs sideband), perhaps we need an enum to capture the different dummy/tracking/sideband event kinds for APIs like this, given the expectation that there can be >1. Something I hope to do for uncore evsels is to move the multiple PMUs into the evsel itself. My hope is that will help with perf stat aggregation logic and the like, as well as simplifying event sorting. Perhaps the evsel for a dummy/tracking/sideband event can cover >1 use-case to reduce the evlist's evsels.
Thanks,
Ian