perf tools: Pipe fixes
Jiri Olsa
reviewercover perf tools: Pipe fixes
hi,
sending changes that allows callchain detection in pipe mode, fixing followign use case:
# perf record --no-buffering --call-graph dwarf -e sdt_rtld:init_start -a -o - | perf --no-pager script -i -
plus change that allows to read pipe data from file:
# perf record -o - sleep 1 > /tmp/perf.pipe.data # perf report -i /tmp/perf.pipe.data
plus unrelated build fix.
Also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/pipe
thanks,
jirka
---
Jiri Olsa (5):
perf tools: Do not display extra info when there is nothing to build perf tools: Do not seek in pipe fd during tracing data processing perf session: Try to read pipe data from file
perf tools: Setup callchain properly in pipe mode
perf script: Enable IP fields for callchains
tools/perf/Makefile.perf | 8 +++++--- tools/perf/builtin-report.c | 33 ++++++++++++++++++++++----------- tools/perf/builtin-script.c | 23 +++++++++++++++++++---- tools/perf/util/callchain.c | 14 ++++++++++++++ tools/perf/util/callchain.h | 1 + tools/perf/util/header.c | 34 ++++++++++++++++++++++++++-------- tools/perf/util/session.c | 9 +++++++-- 7 files changed, 94 insertions(+), 28 deletions(-)
Jiri Olsa
reviewerPATCH 1/5 perf tools: Do not display extra info when there is nothing to build
Even with fully built tree, we still display extra output when make is invoked, like:
$ make
BUILD: Doing 'make -j8' parallel build
DESCEND plugins
make[3]: Nothing to be done for 'plugins/libtraceevent-dynamic-list'.Changing the make descend directly to plugins directory, which quiets those messages down.
--- tools/perf/Makefile.perf | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
tools/perf/Makefile.perf +5 -3
@@ -278,6 +278,7 @@ strip-libs = $(filter-out -l%,$(1)) ifneq ($(OUTPUT),) TE_PATH=$(OUTPUT) + PLUGINS_PATH=$(OUTPUT) BPF_PATH=$(OUTPUT) SUBCMD_PATH=$(OUTPUT) LIBPERF_PATH=$(OUTPUT)
@@ -288,6 +289,7 @@ else endif else TE_PATH=$(TRACE_EVENT_DIR) + PLUGINS_PATH=$(TRACE_EVENT_DIR)plugins/ API_PATH=$(LIB_DIR) BPF_PATH=$(BPF_DIR) SUBCMD_PATH=$(SUBCMD_DIR)
@@ -297,7 +299,7 @@ endif LIBTRACEEVENT = $(TE_PATH)libtraceevent.a export LIBTRACEEVENT -LIBTRACEEVENT_DYNAMIC_LIST = $(TE_PATH)plugins/libtraceevent-dynamic-list +LIBTRACEEVENT_DYNAMIC_LIST = $(PLUGINS_PATH)libtraceevent-dynamic-list # # The static build has no dynsym table, so this does not work for
@@ -756,10 +758,10 @@ $(LIBTRACEEVENT): FORCE $(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)libtraceevent.a libtraceevent_plugins: FORCE - $(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) plugins + $(Q)$(MAKE) -C $(TRACE_EVENT_DIR)plugins $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) plugins $(LIBTRACEEVENT_DYNAMIC_LIST): libtraceevent_plugins - $(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)plugins/libtraceevent-dynamic-list + $(Q)$(MAKE) -C $(TRACE_EVENT_DIR)plugins $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)libtraceevent-dynamic-list $(LIBTRACEEVENT)-clean: $(call QUIET_CLEAN, libtraceevent)
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH 2/5 perf tools: Do not seek in pipe fd during tracing data processing
In reply to Jiri OlsaThere's no need to set 'fd' position in pipe mode, the file descriptor is already in proper place. Moreover the lseek will fail on pipe descriptor and that's why it's been working properly.
I was tempted to remove the lseek calls completely, because it seems that tracing data event was always synthesized only in pipe mode, so there's no need for 'file' mode handling. But I guess there was a reason behind this and there might (however unlikely) be a perf.data that we could break processing for.
--- tools/perf/util/header.c | 18 ++++++++++++++---- tools/perf/util/session.c | 9 +++++++-- 2 files changed, 21 insertions(+), 6 deletions(-)
tools/perf/util/header.c +14 -4
@@ -3947,12 +3947,22 @@ int perf_event__process_tracing_data(struct perf_session *session, {ssize_t size_read, padding, size = event->tracing_data.size;int fd =perf_data__fd (session->data); -off_t offset =lseek (fd,0 ,SEEK_CUR );char buf[BUFSIZ ]; -/* setup for reading amidst mmap */ -lseek (fd, offset +sizeof (struct perf_record_header_tracing_data ), -SEEK_SET ); +/* +* The pipe fd is already in proper place and in any case +* we can't move it, and we'd screw the case where we read +* 'pipe' data from regular file. The trace_report reads +* data from 'fd' so we need to set it directly behind the +* event, where the tracing data starts. +*/ +if (!perf_data__is_pipe (session->data)) { +off_t offset =lseek (fd,0 ,SEEK_CUR ); + +/* setup for reading amidst mmap */ +lseek (fd, offset +sizeof (struct perf_record_header_tracing_data ), +SEEK_SET ); + } size_read =trace_report (fd, &session->tevent, session->repipe);
tools/perf/util/session.c +7 -2
@@ -1542,8 +1542,13 @@ static s64 perf_session__process_user_event(struct perf_session *session,*/ return 0 ;case PERF_RECORD_HEADER_TRACING_DATA : -/* setup for reading amidst mmap */ -lseek (fd, file_offset,SEEK_SET ); +/* +* Setup for reading amidst mmap, but only when we +* are in 'file' mode. The 'pipe' fd is in proper +* place already. +*/ +if (!perf_data__is_pipe (session->data)) +lseek (fd, file_offset,SEEK_SET );return tool->tracing_data (session, event);case PERF_RECORD_HEADER_BUILD_ID :return tool->build_id (session, event);
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH 3/5 perf session: Try to read pipe data from file
In reply to Jiri OlsaIan came with the idea of having support to read the pipe data also from file. Currently pipe mode files fail like:
$ perf record -o - sleep 1 > /tmp/perf.pipe.data $ perf report -i /tmp/perf.pipe.data incompatible file format (rerun with -v to learn more)
This patch adds the support to do that by trying the pipe header first, and if its successfully detected, switching the perf data to pipe mode.
--- tools/perf/util/header.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
tools/perf/util/header.c +12 -4
@@ -3574,7 +3574,7 @@ static int perf_header__read_pipe(struct perf_session *session)return -EINVAL ; } -return 0 ; +return f_header.size ==sizeof (f_header) ?0 :-1 ; } static int read_attr(int fd, struct perf_header *ph,
@@ -3676,7 +3676,7 @@ int perf_session__read_header(struct perf_session *session)struct perf_file_header f_header;struct perf_file_attr f_attr;u64 f_id; -int nr_attrs, nr_ids, i, j; +int nr_attrs, nr_ids, i, j, err;int fd =perf_data__fd (data); session->evlist =evlist__new ();
@@ -3685,8 +3685,16 @@ int perf_session__read_header(struct perf_session *session) session->evlist->env = &header->env; session->machines.host.env = &header->env; -if (perf_data__is_pipe (data)) -return perf_header__read_pipe (session); + +/* +* We can read 'pipe' data event from regular file, +* check for the pipe header regardless of source. +*/ + err =perf_header__read_pipe (session); +if (!err || (err &&perf_data__is_pipe (data))) { + data->is_pipe = true; +return err; + }if (perf_file_header__read (&f_header, header, fd) <0 )return -EINVAL ;
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH 4/5 perf tools: Setup callchain properly in pipe mode
In reply to Jiri OlsaCallchains are automatically initialized by checking on event's sample_type. For pipe mode we need to put this check into attr event code.
Moving the callchains setup code into callchain_param_setup function and calling it from attr event process code.
This enables pipe output having callchains, like:
# perf record -g -e 'raw_syscalls:sys_enter' true | perf script # perf record -g -e 'raw_syscalls:sys_enter' true | perf report
--- tools/perf/builtin-report.c | 33 ++++++++++++++++++++++----------- tools/perf/builtin-script.c | 14 ++++++++++++-- tools/perf/util/callchain.c | 14 ++++++++++++++ tools/perf/util/callchain.h | 1 + 4 files changed, 49 insertions(+), 13 deletions(-)
tools/perf/builtin-report.c +22 -11
@@ -402,16 +402,7 @@ static int report__setup_sample_type(struct report *rep) } } -if (symbol_conf.use_callchain || symbol_conf.cumulate_callchain) { -if ((sample_type &PERF_SAMPLE_REGS_USER ) && - (sample_type &PERF_SAMPLE_STACK_USER )) { - callchain_param.record_mode =CALLCHAIN_DWARF ; - dwarf_callchain_users = true; - }else if (sample_type &PERF_SAMPLE_BRANCH_STACK ) - callchain_param.record_mode =CALLCHAIN_LBR ; -else - callchain_param.record_mode =CALLCHAIN_FP ; - } +callchain_param_setup (sample_type);if (rep->stitch_lbr && (callchain_param.record_mode !=CALLCHAIN_LBR )) { ui__warning("Can't find LBR callchain. Switch off --stitch-lbr.\n"
@@ -1090,6 +1081,26 @@ parse_percent_limit(const struct option *opt, const char *str,return 0 ; } +static int process_attr (struct perf_tool *tool__maybe_unused , +union perf_event *event, +struct evlist **pevlist) +{ +u64 sample_type; +int err; + + err =perf_event__process_attr (tool, event, pevlist); +if (err) +return err; + +/* +* Check if we need to enable callchains based +* on events sample_type. +*/ + sample_type =perf_evlist__combined_sample_type (*pevlist); +callchain_param_setup (sample_type); +return 0 ; +} +int cmd_report (int argc,const char **argv) {struct perf_session *session;
@@ -1120,7 +1131,7 @@ int cmd_report(int argc, const char **argv) .fork = perf_event__process_fork, .lost = perf_event__process_lost, .read = process_read_event, - .attr = perf_event__process_attr, + .attr = process_attr, .tracing_data = perf_event__process_tracing_data, .build_id = perf_event__process_build_id, .id_index = perf_event__process_id_index,
tools/perf/builtin-script.c +12 -2
@@ -2085,6 +2085,7 @@ static int process_attr(struct perf_tool *tool, union perf_event *event, struct perf_script *scr = container_of(tool, struct perf_script, tool);struct evlist *evlist;struct evsel *evsel, *pos; +u64 sample_type;int err;static struct evsel_script *es;
@@ -2119,10 +2120,19 @@ static int process_attr(struct perf_tool *tool, union perf_event *event,set_print_ip_opts (&evsel->core.attr); -if (evsel->core.attr.sample_type) +if (evsel->core.attr.sample_type) { err =perf_evsel__check_attr (evsel, scr->session); +if (err) +return err; + } -return err; +/* +* Check if we need to enable callchains based +* on events sample_type. +*/ + sample_type =perf_evlist__combined_sample_type (evlist); +callchain_param_setup (sample_type); +return 0 ; } static int print_event_with_time(struct perf_tool *tool,
tools/perf/util/callchain.c +14 -0
@@ -1599,3 +1599,17 @@ void callchain_cursor_reset(struct callchain_cursor *cursor)for (node = cursor->first; node !=NULL ; node = node->next)map__zput (node->ms.map); } + +void callchain_param_setup (u64 sample_type) +{ +if (symbol_conf.use_callchain || symbol_conf.cumulate_callchain) { +if ((sample_type &PERF_SAMPLE_REGS_USER ) && + (sample_type &PERF_SAMPLE_STACK_USER )) { + callchain_param.record_mode =CALLCHAIN_DWARF ; + dwarf_callchain_users = true; + }else if (sample_type &PERF_SAMPLE_BRANCH_STACK ) + callchain_param.record_mode =CALLCHAIN_LBR ; +else + callchain_param.record_mode =CALLCHAIN_FP ; + } +}
tools/perf/util/callchain.h +1 -0
@@ -297,4 +297,5 @@ int callchain_branch_counts(struct callchain_root *root, u64 *branch_count, u64 *predicted_count, u64 *abort_count, u64 *cycles_count); +void callchain_param_setup (u64 sample_type);#endif /* __PERF_CALLCHAIN_H */
Signature · 1 line
2.25.4
Jiri Olsa
reviewerPATCH 5/5 perf script: Enable IP fields for callchains
In reply to Jiri OlsaIn case the callchains were deleted in pipe mode,
we need to ensure that the IP fields are enabled,
otherwise the callchain is not displayed.
Enabling IP and SYM, which should be enough for
callchains.
--- tools/perf/builtin-script.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
tools/perf/builtin-script.c +7 -2
@@ -2118,8 +2118,6 @@ static int process_attr(struct perf_tool *tool, union perf_event *event,return 0 ; } -set_print_ip_opts (&evsel->core.attr); -if (evsel->core.attr.sample_type) { err =perf_evsel__check_attr (evsel, scr->session);if (err)
@@ -2132,6 +2130,13 @@ static int process_attr(struct perf_tool *tool, union perf_event *event,*/ sample_type =perf_evlist__combined_sample_type (evlist);callchain_param_setup (sample_type); + +/* Enable fields for callchain entries, if it got enabled. */ +if (callchain_param.record_mode !=CALLCHAIN_NONE ) { + output[output_type (evsel->core.attr.type)].fields |=PERF_OUTPUT_IP | +PERF_OUTPUT_SYM ; + } +set_print_ip_opts (&evsel->core.attr);return 0 ; }
Signature · 1 line
2.25.4
Re: [PATCH 1/5] perf tools: Do not display extra info when there is nothing to build
In reply to Jiri OlsaEm Thu, May 07, 2020 at 11:50:20AM +0200, Jiri Olsa escreveu:
Quoted 10 lines
Even with fully built tree, we still display extra output when make is invoked, like:
$ make BUILD: Doing 'make -j8' parallel build DESCEND plugins make[3]: Nothing to be done for 'plugins/libtraceevent-dynamic-list'.Changing the make descend directly to plugins directory, which quiets those messages down.
Thanks a lot for getting rid of that nuisance :-)
- Arnaldo
Quoted 50 lines
Signed-off-by Jiri Olsa <[email protected]>--- tools/perf/Makefile.perf | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)tools/perf/Makefile.perf +5
-3@@ -278,6 +278,7 @@ strip-libs = $(filter-out -l%,$(1)) ifneq ($(OUTPUT),) TE_PATH=$(OUTPUT) + PLUGINS_PATH=$(OUTPUT) BPF_PATH=$(OUTPUT) SUBCMD_PATH=$(OUTPUT) LIBPERF_PATH=$(OUTPUT)@@ -288,6 +289,7 @@ else endif else TE_PATH=$(TRACE_EVENT_DIR) + PLUGINS_PATH=$(TRACE_EVENT_DIR)plugins/ API_PATH=$(LIB_DIR) BPF_PATH=$(BPF_DIR) SUBCMD_PATH=$(SUBCMD_DIR)@@ -297,7 +299,7 @@ endif LIBTRACEEVENT = $(TE_PATH)libtraceevent.a export LIBTRACEEVENT -LIBTRACEEVENT_DYNAMIC_LIST = $(TE_PATH)plugins/libtraceevent-dynamic-list +LIBTRACEEVENT_DYNAMIC_LIST = $(PLUGINS_PATH)libtraceevent-dynamic-list # # The static build has no dynsym table, so this does not work for@@ -756,10 +758,10 @@ $(LIBTRACEEVENT): FORCE $(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)libtraceevent.a libtraceevent_plugins: FORCE - $(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) plugins + $(Q)$(MAKE) -C $(TRACE_EVENT_DIR)plugins $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) plugins $(LIBTRACEEVENT_DYNAMIC_LIST): libtraceevent_plugins - $(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)plugins/libtraceevent-dynamic-list + $(Q)$(MAKE) -C $(TRACE_EVENT_DIR)plugins $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)libtraceevent-dynamic-list $(LIBTRACEEVENT)-clean: $(call QUIET_CLEAN, libtraceevent)Signature · 1 line
2.25.4
Signature · 1 line
- Arnaldo
Em Thu, May 07, 2020 at 11:50:19AM +0200, Jiri Olsa escreveu:
Quoted 8 lines
hi,
sending changes that allows callchain detection in pipe mode, fixing followign use case:# perf record --no-buffering --call-graph dwarf -e sdt_rtld:init_start -a -o - | perf --no-pager script -i -plus change that allows to read pipe data from file:
Quoted 8 lines
# perf record -o - sleep 1 > /tmp/perf.pipe.data # perf report -i /tmp/perf.pipe.dataplus unrelated build fix.
Also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/pipe
Thanks a lot, all tested, great improvements for pipe mode!
- Arnaldo
Quoted 21 lines
thanks,
jirka---
Jiri Olsa (5):
perf tools: Do not display extra info when there is nothing to build perf tools: Do not seek in pipe fd during tracing data processing perf session: Try to read pipe data from file
perf tools: Setup callchain properly in pipe mode
perf script: Enable IP fields for callchainstools/perf/Makefile.perf | 8 +++++--- tools/perf/builtin-report.c | 33 ++++++++++++++++++++++----------- tools/perf/builtin-script.c | 23 +++++++++++++++++++---- tools/perf/util/callchain.c | 14 ++++++++++++++ tools/perf/util/callchain.h | 1 + tools/perf/util/header.c | 34 ++++++++++++++++++++++++++-------- tools/perf/util/session.c | 9 +++++++-- 7 files changed, 94 insertions(+), 28 deletions(-)
Signature · 1 line
- Arnaldo