perf tool: Add support for multiple data file storage
v2 cover perf tool: Add support for multiple data file storage
hi,
sending the support for multiple file storage. Initial RFC is here:
http://marc.info/?l=linux-kernel&m=137408381902423&w=2
v2 changes:
- reworked perf mmap size setup to be able to get the mmap size value easily later
- added perf.data read/write test for v2 and v3 for both endianity
- added record '-M time' support
It's reachable here:git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_file
thanks,
jirka
---
Jiri Olsa (25):
perf tools: Check mmap pages value early
perf tools: Add possibility to specify mmap size
perf tools: Introduce perf_evlist__new_default function
perf tools: Adding throttle event data struct support
perf tests: Add simple session read/write test
perf tests: Add session reading test for little endian perf data perf tests: Add session reading test for big endian perf data perf doc: Add perf data file documentation
perf tools: Introduce perf data file version CHECK macro perf tools: Introduce swap_features function
perf tools: Introduce swap_header function
perf tools: Separate version 2 specific perf data header bits perf tools: Using evlist as a holder for event_desc feature perf tools: Introduce perf.data version 3 format
perf tools: Add perf data version 3 header swap
perf tools: Add perf data version 3 header read
perf tools: Add perf.data version 3 header write
perf tools: Get rid of post_processing_offset in record command perf tools: Move synthesizing into single function
perf tools: Add perf_data_file__open interface to data object perf tools: Separating data file properties from session perf tests: Add session reading test for little endian perf data v3 perf tests: Add session reading test for big endian perf data v3 perf tools: Add multi file '-M' option for record command perf tools: Have the process properly sythesized in subsequent data files
tools/perf/Documentation/perf-data-file-v2.txt | 265 ++++++++++++++++++++++++++++++++++ tools/perf/Documentation/perf-data-file-v3.txt | 63 ++++++++ tools/perf/Documentation/perf-kvm.txt | 4 +- tools/perf/Documentation/perf-record.txt | 18 ++- tools/perf/Documentation/perf-top.txt | 4 +- tools/perf/Documentation/perf-trace.txt | 4 +- tools/perf/Makefile | 6 + tools/perf/builtin-annotate.c | 11 +- tools/perf/builtin-buildid-cache.c | 8 +- tools/perf/builtin-buildid-list.c | 11 +- tools/perf/builtin-diff.c | 19 ++- tools/perf/builtin-evlist.c | 7 +- tools/perf/builtin-inject.c | 10 +- tools/perf/builtin-kmem.c | 7 +- tools/perf/builtin-kvm.c | 18 ++- tools/perf/builtin-lock.c | 8 +- tools/perf/builtin-mem.c | 9 +- tools/perf/builtin-record.c | 484 ++++++++++++++++++++++++++++++++++++++++++++------------------ tools/perf/builtin-report.c | 18 ++- tools/perf/builtin-sched.c | 6 +- tools/perf/builtin-script.c | 17 ++- tools/perf/builtin-timechart.c | 10 +- tools/perf/builtin-top.c | 12 +- tools/perf/builtin-trace.c | 13 +- tools/perf/perf.h | 15 +- tools/perf/tests/builtin-test.c | 4 + tools/perf/tests/perf-record.c | 12 +- tools/perf/tests/perf.data.v2.be.h | 265 ++++++++++++++++++++++++++++++++++ tools/perf/tests/perf.data.v2.le.h | 219 ++++++++++++++++++++++++++++ tools/perf/tests/perf.data.v3.be.h | 258 +++++++++++++++++++++++++++++++++ tools/perf/tests/perf.data.v3.le.h | 212 +++++++++++++++++++++++++++ tools/perf/tests/session-simple.c | 658 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/tests/task-exit.c | 14 +- tools/perf/tests/tests.h | 1 + tools/perf/util/data.c | 120 ++++++++++++++++ tools/perf/util/data.h | 48 +++++++ tools/perf/util/event.h | 7 + tools/perf/util/evlist.c | 78 ++++++++-- tools/perf/util/evlist.h | 9 +- tools/perf/util/evsel.c | 24 ++++ tools/perf/util/header.c | 446 +++++++++++++++++++++++++++++++++------------------------ tools/perf/util/header.h | 31 +++- tools/perf/util/python.c | 7 - tools/perf/util/session.c | 138 ++++++++---------- tools/perf/util/session.h | 11 +- tools/perf/util/util.c | 25 ++++ tools/perf/util/util.h | 14 ++ 47 files changed, 3137 insertions(+), 511 deletions(-) create mode 100644 tools/perf/Documentation/perf-data-file-v2.txt create mode 100644 tools/perf/Documentation/perf-data-file-v3.txt create mode 100644 tools/perf/tests/perf.data.v2.be.h create mode 100644 tools/perf/tests/perf.data.v2.le.h create mode 100644 tools/perf/tests/perf.data.v3.be.h create mode 100644 tools/perf/tests/perf.data.v3.le.h create mode 100644 tools/perf/tests/session-simple.c create mode 100644 tools/perf/util/data.c create mode 100644 tools/perf/util/data.h
PATCH v2 1/25 perf tools: Check mmap pages value early
Moving the check of the mmap_pages value into the options parsing time, so we could relay on this value on other parts of code. Related changes come in next patches.
Also changins perf_evlist::mmap_len to proper size_t type.
--- tools/perf/builtin-kvm.c | 5 +++-- tools/perf/builtin-record.c | 9 +++------ tools/perf/builtin-top.c | 5 +++-- tools/perf/builtin-trace.c | 5 +++-- tools/perf/util/evlist.c | 46 ++++++++++++++++++++++++++++++++++++--------- tools/perf/util/evlist.h | 6 +++++- 6 files changed, 54 insertions(+), 22 deletions(-)
tools/perf/builtin-kvm.c +3 -2
@@ -1426,8 +1426,9 @@ static int kvm_events_live(struct perf_kvm_stat *kvm,const struct option live_options[] = {OPT_STRING ('p' ,"pid" , &kvm->opts.target.pid,"pid" ,"record events on existing process id" ), -OPT_UINTEGER ('m' ,"mmap-pages" , &kvm->opts.mmap_pages, -"number of mmap data pages" ), +OPT_CALLBACK ('m' ,"mmap-pages" , &kvm->opts.mmap_pages,"pages" , +"number of mmap data pages" , + perf_evlist__parse_mmap_pages),OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose (show counter open errors, etc)" ), OPT_BOOLEAN('a', "all-cpus", &kvm->opts.target.system_wide,
tools/perf/builtin-record.c +3 -6
@@ -234,10 +234,6 @@ try_again:"or try again with a smaller value of -m/--mmap_pages.\n" "(current value: %d)\n", opts->mmap_pages); rc = -errno; - }else if (!is_power_of_2 (opts->mmap_pages) && - (opts->mmap_pages !=UINT_MAX )) { -pr_err ("--mmap_pages/-m value must be a power of two." ); - rc = -EINVAL ; }else {pr_err ("failed to mmap with %d (%s)\n" , errno,strerror (errno)); rc = -errno;
@@ -854,8 +850,9 @@ const struct option record_options[] = {OPT_BOOLEAN ('i' ,"no-inherit" , &record.opts.no_inherit,"child tasks do not inherit counters" ),OPT_UINTEGER ('F' ,"freq" , &record.opts.user_freq,"profile at this frequency" ), -OPT_UINTEGER ('m' ,"mmap-pages" , &record.opts.mmap_pages, -"number of mmap data pages" ), +OPT_CALLBACK ('m' ,"mmap-pages" , &record.opts.mmap_pages,"pages" , +"number of mmap data pages" , + perf_evlist__parse_mmap_pages),OPT_BOOLEAN (0 ,"group" , &record.opts.group,"put the counters into a counter group" ), OPT_CALLBACK_DEFAULT('g', "call-graph", &record.opts,
tools/perf/builtin-top.c +3 -2
@@ -1076,8 +1076,9 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused) "file", "vmlinux pathname"),OPT_BOOLEAN ('K' ,"hide_kernel_symbols" , &top.hide_kernel_symbols,"hide kernel symbols" ), -OPT_UINTEGER ('m' ,"mmap-pages" , &opts->mmap_pages, -"number of mmap data pages" ), +OPT_CALLBACK ('m' ,"mmap-pages" , &opts->mmap_pages,"pages" , +"number of mmap data pages" , + perf_evlist__parse_mmap_pages),OPT_INTEGER ('r' ,"realtime" , &top.realtime_prio,"collect data with this RT SCHED_FIFO priority" ), OPT_INTEGER('d', "delay", &top.delay_secs,
tools/perf/builtin-trace.c +3 -2
@@ -1042,8 +1042,9 @@ int cmd_trace(int argc, const char **argv, const char *prefix __maybe_unused) "list of cpus to monitor"),OPT_BOOLEAN (0 ,"no-inherit" , &trace.opts.no_inherit,"child tasks do not inherit counters" ), -OPT_UINTEGER ('m' ,"mmap-pages" , &trace.opts.mmap_pages, -"number of mmap data pages" ), +OPT_CALLBACK ('m' ,"mmap-pages" , &trace.opts.mmap_pages,"pages" , +"number of mmap data pages" , + perf_evlist__parse_mmap_pages),OPT_STRING ('u' ,"uid" , &trace.opts.target.uid_str,"user" ,"user to profile" ), OPT_CALLBACK(0, "duration", &trace, "float",
tools/perf/util/evlist.c +37 -9
70 lines not on this page · raw message
tools/perf/util/evlist.h +5 -1
@@ -31,7 +31,7 @@ struct perf_evlist {int nr_groups;int nr_fds;int nr_mmaps; -int mmap_len; +size_t mmap_len;int id_pos;int is_pos;u64 combined_sample_type;
@@ -103,6 +103,10 @@ int perf_evlist__prepare_workload(struct perf_evlist *evlist, bool want_signal);int perf_evlist__start_workload (struct perf_evlist *evlist); +int perf_evlist__parse_mmap_pages (const struct option *opt, +const char *str, +int unset); +int perf_evlist__mmap (struct perf_evlist *evlist,unsigned int pages,bool overwrite);void perf_evlist__munmap (struct perf_evlist *evlist);
Signature · 1 line
1.7.11.7
tip-bot for Jiri Olsa
merge bot[tip:perf/core] perf tools: Check mmap pages value early
perf tools: Check mmap pages value early
Commit-ID: 994a1f78b191df0c9d6caca3f3afb03e247aff26 Gitweb: http://git.kernel.org/tip/994a1f78b191df0c9d6caca3f3afb03e247aff26 Author: Jiri Olsa <[email protected]> AuthorDate: Sun, 1 Sep 2013 12:36:12 +0200 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Wed, 9 Oct 2013 11:24:10 -0300
perf tools: Check mmap pages value early
Move the check of the mmap_pages value to the options parsing time, so we could rely on this value on other parts of code.
Related changes come in the next patches.
Also changes perf_evlist::mmap_len to proper size_t type.
--- tools/perf/builtin-kvm.c | 5 +++-- tools/perf/builtin-record.c | 9 +++------ tools/perf/builtin-top.c | 5 +++-- tools/perf/builtin-trace.c | 5 +++-- tools/perf/util/evlist.c | 46 ++++++++++++++++++++++++++++++++++++--------- tools/perf/util/evlist.h | 6 +++++- 6 files changed, 54 insertions(+), 22 deletions(-)
tools/perf/builtin-kvm.c +3 -2
@@ -1426,8 +1426,9 @@ static int kvm_events_live(struct perf_kvm_stat *kvm,const struct option live_options[] = {OPT_STRING ('p' ,"pid" , &kvm->opts.target.pid,"pid" ,"record events on existing process id" ), -OPT_UINTEGER ('m' ,"mmap-pages" , &kvm->opts.mmap_pages, -"number of mmap data pages" ), +OPT_CALLBACK ('m' ,"mmap-pages" , &kvm->opts.mmap_pages,"pages" , +"number of mmap data pages" , + perf_evlist__parse_mmap_pages),OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose (show counter open errors, etc)" ), OPT_BOOLEAN('a', "all-cpus", &kvm->opts.target.system_wide,
tools/perf/builtin-record.c +3 -6
@@ -233,10 +233,6 @@ try_again:"or try again with a smaller value of -m/--mmap_pages.\n" "(current value: %d)\n", opts->mmap_pages); rc = -errno; - }else if (!is_power_of_2 (opts->mmap_pages) && - (opts->mmap_pages !=UINT_MAX )) { -pr_err ("--mmap_pages/-m value must be a power of two." ); - rc = -EINVAL ; }else {pr_err ("failed to mmap with %d (%s)\n" , errno,strerror (errno)); rc = -errno;
@@ -854,8 +850,9 @@ const struct option record_options[] = {OPT_BOOLEAN ('i' ,"no-inherit" , &record.opts.no_inherit,"child tasks do not inherit counters" ),OPT_UINTEGER ('F' ,"freq" , &record.opts.user_freq,"profile at this frequency" ), -OPT_UINTEGER ('m' ,"mmap-pages" , &record.opts.mmap_pages, -"number of mmap data pages" ), +OPT_CALLBACK ('m' ,"mmap-pages" , &record.opts.mmap_pages,"pages" , +"number of mmap data pages" , + perf_evlist__parse_mmap_pages),OPT_BOOLEAN (0 ,"group" , &record.opts.group,"put the counters into a counter group" ), OPT_CALLBACK_DEFAULT('g', "call-graph", &record.opts,
tools/perf/builtin-top.c +3 -2
@@ -1075,8 +1075,9 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused) "file", "vmlinux pathname"),OPT_BOOLEAN ('K' ,"hide_kernel_symbols" , &top.hide_kernel_symbols,"hide kernel symbols" ), -OPT_UINTEGER ('m' ,"mmap-pages" , &opts->mmap_pages, -"number of mmap data pages" ), +OPT_CALLBACK ('m' ,"mmap-pages" , &opts->mmap_pages,"pages" , +"number of mmap data pages" , + perf_evlist__parse_mmap_pages),OPT_INTEGER ('r' ,"realtime" , &top.realtime_prio,"collect data with this RT SCHED_FIFO priority" ), OPT_INTEGER('d', "delay", &top.delay_secs,
tools/perf/builtin-trace.c +3 -2
@@ -1528,8 +1528,9 @@ int cmd_trace(int argc, const char **argv, const char *prefix __maybe_unused) "list of cpus to monitor"),OPT_BOOLEAN (0 ,"no-inherit" , &trace.opts.no_inherit,"child tasks do not inherit counters" ), -OPT_UINTEGER ('m' ,"mmap-pages" , &trace.opts.mmap_pages, -"number of mmap data pages" ), +OPT_CALLBACK ('m' ,"mmap-pages" , &trace.opts.mmap_pages,"pages" , +"number of mmap data pages" , + perf_evlist__parse_mmap_pages),OPT_STRING ('u' ,"uid" , &trace.opts.target.uid_str,"user" ,"user to profile" ), OPT_CALLBACK(0, "duration", &trace, "float",
tools/perf/util/evlist.c +37 -9
70 lines not on this page · raw message
tools/perf/util/evlist.h +5 -1
@@ -31,7 +31,7 @@ struct perf_evlist {int nr_groups;int nr_fds;int nr_mmaps; -int mmap_len; +size_t mmap_len;int id_pos;int is_pos;u64 combined_sample_type;
@@ -103,6 +103,10 @@ int perf_evlist__prepare_workload(struct perf_evlist *evlist, bool want_signal);int perf_evlist__start_workload (struct perf_evlist *evlist); +int perf_evlist__parse_mmap_pages (const struct option *opt, +const char *str, +int unset); +int perf_evlist__mmap (struct perf_evlist *evlist,unsigned int pages,bool overwrite);void perf_evlist__munmap (struct perf_evlist *evlist);
PATCH v2 2/25 perf tools: Add possibility to specify mmap size
In reply to Jiri OlsaAdding possibility to specify mmap size via -m/--mmap-pages by appending unit size character (B/K/M/G) to the number, like:
$ perf record -m 8K ls $ perf record -m 2M ls
The size is rounded up appropriately to follow perf mmap restrictions.
If no unit is specified the number provides pages as of now, like:
$ perf record -m 8 ls
--- tools/perf/Documentation/perf-kvm.txt | 4 +++- tools/perf/Documentation/perf-record.txt | 4 +++- tools/perf/Documentation/perf-top.txt | 4 +++- tools/perf/Documentation/perf-trace.txt | 4 +++- tools/perf/util/evlist.c | 32 ++++++++++++++++++++++++++------ tools/perf/util/util.c | 25 +++++++++++++++++++++++++ tools/perf/util/util.h | 14 ++++++++++++++ 7 files changed, 77 insertions(+), 10 deletions(-)
tools/perf/Documentation/perf-kvm.txt +3 -1
@@ -109,7 +109,9 @@ STAT LIVE OPTIONS -m:: --mmap-pages=:: - Number of mmap data pages. Must be a power of two. + Number of mmap data pages (must be a power of two) or size + specification with appended unit character - B/K/M/G. The + size is rounded up to have nearest pages power of two value. -a:: --all-cpus::
tools/perf/Documentation/perf-record.txt +3 -1
@@ -87,7 +87,9 @@ OPTIONS -m:: --mmap-pages=:: - Number of mmap data pages. Must be a power of two. + Number of mmap data pages (must be a power of two) or size + specification with appended unit character - B/K/M/G. The + size is rounded up to have nearest pages power of two value. -g:: --call-graph::
tools/perf/Documentation/perf-top.txt +3 -1
@@ -68,7 +68,9 @@ Default is to monitor all CPUS. -m <pages>:: --mmap-pages=<pages>:: - Number of mmapped data pages. + Number of mmap data pages (must be a power of two) or size + specification with appended unit character - B/K/M/G. The + size is rounded up to have nearest pages power of two value. -p <pid>:: --pid=<pid>::
tools/perf/Documentation/perf-trace.txt +3 -1
@@ -59,7 +59,9 @@ OPTIONS -m:: --mmap-pages=:: - Number of mmap data pages. Must be a power of two. + Number of mmap data pages (must be a power of two) or size + specification with appended unit character - B/K/M/G. The + size is rounded up to have nearest pages power of two value. -C:: --cpu::
tools/perf/util/evlist.c +26 -6
@@ -634,14 +634,33 @@ static size_t perf_evlist__mmap_size(unsigned long pages)int perf_evlist__parse_mmap_pages (const struct option *opt,const char *str,int unset__maybe_unused ) { -unsigned int pages, *mmap_pages = opt->value; +unsigned int pages, val, *mmap_pages = opt->value;size_t size; -char *eptr; +static struct parse_tag tags[] = { + { .tag ='B' , .mult =1 }, + { .tag ='K' , .mult =1 <<10 }, + { .tag ='M' , .mult =1 <<20 }, + { .tag ='G' , .mult =1 <<30 }, + { .tag =0 }, + }; - pages =strtoul (str, &eptr,10 ); -if (*eptr !='\0' ) { -pr_err ("failed to parse --mmap_pages/-m value\n" ); -return -1 ; + val =parse_tag_value (str, tags); +if (val != (unsigned int )-1 ) { +/* we got file size value */ + pages =PERF_ALIGN (val, page_size) / page_size; +if (!is_power_of_2 (pages)) { + pages =next_pow2 (pages); +pr_info ("rounding mmap pages size to %u (%u pages)\n" , + pages * page_size, pages); + } + }else { +/* we got pages count value */ +char *eptr; + pages =strtoul (str, &eptr,10 ); +if (*eptr !='\0' ) { +pr_err ("failed to parse --mmap_pages/-m value\n" ); +return -1 ; + } } size =perf_evlist__mmap_size (pages);
@@ -685,6 +704,7 @@ int perf_evlist__mmap(struct perf_evlist *evlist, unsigned int pages, evlist->overwrite = overwrite; evlist->mmap_len =perf_evlist__mmap_size (pages); +pr_debug ("mmap size %luB\n" , evlist->mmap_len); mask = evlist->mmap_len - page_size -1 ;list_for_each_entry (evsel, &evlist->entries, node) {
tools/perf/util/util.c +25 -0
@@ -361,3 +361,28 @@ int parse_nsec_time(const char *str, u64 *ptime) *ptime = time_sec *NSEC_PER_SEC + time_nsec;return 0 ; } + +unsigned long parse_tag_value (const char *str,struct parse_tag *tags) +{ +struct parse_tag *i = tags; + +while (i->tag) { +char *s; + + s =strchr (str, i->tag); +if (s) { +unsigned long int value; +char *endptr; + + value =strtoul (str, &endptr,10 ); +if (s != endptr) +break ; + + value *= i->mult; +return value; + } + i++; + } + +return (unsigned long )-1 ; +}
tools/perf/util/util.h +14 -0
@@ -270,6 +270,13 @@ bool is_power_of_2(unsigned long n)return (n !=0 && ((n & (n -1 )) ==0 )); } +static inline unsigned next_pow2 (unsigned x) +{ +if (!x) +return 1 ; +return 1ULL << (32 -__builtin_clz (x -1 )); +} +size_t hex_width (u64 v);int hex2u64 (const char *ptr,u64 *val);
@@ -281,4 +288,11 @@ void dump_stack(void);extern unsigned int page_size;void get_term_dimensions (struct winsize *ws); + +struct parse_tag { +char tag; +int mult; +}; + +unsigned long parse_tag_value (const char *str,struct parse_tag *tags);#endif /* GIT_COMPAT_UTIL_H */
Signature · 1 line
1.7.11.7
tip-bot for Jiri Olsa
merge bot[tip:perf/core] perf tools: Add possibility to specify mmap size
perf tools: Add possibility to specify mmap size
Commit-ID: 27050f530dc4fd88dc93d85c177e000efe970d12 Gitweb: http://git.kernel.org/tip/27050f530dc4fd88dc93d85c177e000efe970d12 Author: Jiri Olsa <[email protected]> AuthorDate: Sun, 1 Sep 2013 12:36:13 +0200 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Wed, 9 Oct 2013 11:24:20 -0300
perf tools: Add possibility to specify mmap size
Adding possibility to specify mmap size via -m/--mmap-pages by appending unit size character (B/K/M/G) to the number, like:
$ perf record -m 8K ls $ perf record -m 2M ls
The size is rounded up appropriately to follow perf mmap restrictions.
If no unit is specified the number provides pages as of now, like:
$ perf record -m 8 ls
--- tools/perf/Documentation/perf-kvm.txt | 4 +++- tools/perf/Documentation/perf-record.txt | 4 +++- tools/perf/Documentation/perf-top.txt | 4 +++- tools/perf/Documentation/perf-trace.txt | 4 +++- tools/perf/util/evlist.c | 32 ++++++++++++++++++++++++++------ tools/perf/util/util.c | 25 +++++++++++++++++++++++++ tools/perf/util/util.h | 14 ++++++++++++++ 7 files changed, 77 insertions(+), 10 deletions(-)
tools/perf/Documentation/perf-kvm.txt +3 -1
@@ -109,7 +109,9 @@ STAT LIVE OPTIONS -m:: --mmap-pages=:: - Number of mmap data pages. Must be a power of two. + Number of mmap data pages (must be a power of two) or size + specification with appended unit character - B/K/M/G. The + size is rounded up to have nearest pages power of two value. -a:: --all-cpus::
tools/perf/Documentation/perf-record.txt +3 -1
@@ -87,7 +87,9 @@ OPTIONS -m:: --mmap-pages=:: - Number of mmap data pages. Must be a power of two. + Number of mmap data pages (must be a power of two) or size + specification with appended unit character - B/K/M/G. The + size is rounded up to have nearest pages power of two value. -g:: --call-graph::
tools/perf/Documentation/perf-top.txt +3 -1
@@ -68,7 +68,9 @@ Default is to monitor all CPUS. -m <pages>:: --mmap-pages=<pages>:: - Number of mmapped data pages. + Number of mmap data pages (must be a power of two) or size + specification with appended unit character - B/K/M/G. The + size is rounded up to have nearest pages power of two value. -p <pid>:: --pid=<pid>::
tools/perf/Documentation/perf-trace.txt +3 -1
@@ -59,7 +59,9 @@ OPTIONS -m:: --mmap-pages=:: - Number of mmap data pages. Must be a power of two. + Number of mmap data pages (must be a power of two) or size + specification with appended unit character - B/K/M/G. The + size is rounded up to have nearest pages power of two value. -C:: --cpu::
tools/perf/util/evlist.c +26 -6
@@ -687,14 +687,33 @@ static size_t perf_evlist__mmap_size(unsigned long pages)int perf_evlist__parse_mmap_pages (const struct option *opt,const char *str,int unset__maybe_unused ) { -unsigned int pages, *mmap_pages = opt->value; +unsigned int pages, val, *mmap_pages = opt->value;size_t size; -char *eptr; +static struct parse_tag tags[] = { + { .tag ='B' , .mult =1 }, + { .tag ='K' , .mult =1 <<10 }, + { .tag ='M' , .mult =1 <<20 }, + { .tag ='G' , .mult =1 <<30 }, + { .tag =0 }, + }; - pages =strtoul (str, &eptr,10 ); -if (*eptr !='\0' ) { -pr_err ("failed to parse --mmap_pages/-m value\n" ); -return -1 ; + val =parse_tag_value (str, tags); +if (val != (unsigned int )-1 ) { +/* we got file size value */ + pages =PERF_ALIGN (val, page_size) / page_size; +if (!is_power_of_2 (pages)) { + pages =next_pow2 (pages); +pr_info ("rounding mmap pages size to %u (%u pages)\n" , + pages * page_size, pages); + } + }else { +/* we got pages count value */ +char *eptr; + pages =strtoul (str, &eptr,10 ); +if (*eptr !='\0' ) { +pr_err ("failed to parse --mmap_pages/-m value\n" ); +return -1 ; + } } size =perf_evlist__mmap_size (pages);
@@ -738,6 +757,7 @@ int perf_evlist__mmap(struct perf_evlist *evlist, unsigned int pages, evlist->overwrite = overwrite; evlist->mmap_len =perf_evlist__mmap_size (pages); +pr_debug ("mmap size %luB\n" , evlist->mmap_len); mask = evlist->mmap_len - page_size -1 ;list_for_each_entry (evsel, &evlist->entries, node) {
tools/perf/util/util.c +25 -0
@@ -361,3 +361,28 @@ int parse_nsec_time(const char *str, u64 *ptime) *ptime = time_sec *NSEC_PER_SEC + time_nsec;return 0 ; } + +unsigned long parse_tag_value (const char *str,struct parse_tag *tags) +{ +struct parse_tag *i = tags; + +while (i->tag) { +char *s; + + s =strchr (str, i->tag); +if (s) { +unsigned long int value; +char *endptr; + + value =strtoul (str, &endptr,10 ); +if (s != endptr) +break ; + + value *= i->mult; +return value; + } + i++; + } + +return (unsigned long )-1 ; +}
tools/perf/util/util.h +14 -0
@@ -270,6 +270,13 @@ bool is_power_of_2(unsigned long n)return (n !=0 && ((n & (n -1 )) ==0 )); } +static inline unsigned next_pow2 (unsigned x) +{ +if (!x) +return 1 ; +return 1ULL << (32 -__builtin_clz (x -1 )); +} +size_t hex_width (u64 v);int hex2u64 (const char *ptr,u64 *val);
@@ -281,4 +288,11 @@ void dump_stack(void);extern unsigned int page_size;void get_term_dimensions (struct winsize *ws); + +struct parse_tag { +char tag; +int mult; +}; + +unsigned long parse_tag_value (const char *str,struct parse_tag *tags);#endif /* GIT_COMPAT_UTIL_H */
PATCH v2 3/25 perf tools: Introduce perf_evlist__new_default function
In reply to Jiri OlsaAdding new common function to create evlist with default event. It spares some code lines in automated tests.
--- tools/perf/tests/perf-record.c | 12 +----------- tools/perf/tests/task-exit.c | 14 ++------------ tools/perf/util/evlist.c | 12 ++++++++++++ tools/perf/util/evlist.h | 1 + 4 files changed, 16 insertions(+), 23 deletions(-)
tools/perf/tests/perf-record.c +1 -11
@@ -45,7 +45,7 @@ int test__PERF_RECORD(void) };cpu_set_t cpu_mask;size_t cpu_mask_size =sizeof (cpu_mask); -struct perf_evlist *evlist =perf_evlist__new (); +struct perf_evlist *evlist =perf_evlist__new_default ();struct perf_evsel *evsel;struct perf_sample sample;const char *cmd ="sleep" ;
@@ -66,16 +66,6 @@ int test__PERF_RECORD(void) }/* -* We need at least one evsel in the evlist, use the default -* one: "cycles". -*/ - err =perf_evlist__add_default (evlist); -if (err <0 ) { -pr_debug ("Not enough memory to create evsel\n" ); - goto out_delete_evlist; - } - -/* * Create maps of threads and cpus to monitor. In this case * we start with all threads and cpus (-1, -1) but then in * perf_evlist__prepare_workload we'll fill in the only thread
tools/perf/tests/task-exit.c +2 -12
@@ -37,20 +37,11 @@ int test__task_exit(void)signal (SIGCHLD , sig_handler);signal (SIGUSR1 , sig_handler); - evlist =perf_evlist__new (); + evlist =perf_evlist__new_default ();if (evlist ==NULL ) { -pr_debug ("perf_evlist__new\n" ); +pr_debug ("perf_evlist__new_default\n" );return -1 ; } -/* -* We need at least one evsel in the evlist, use the default -* one: "cycles". -*/ - err =perf_evlist__add_default (evlist); -if (err <0 ) { -pr_debug ("Not enough memory to create evsel\n" ); - goto out_free_evlist; - }/* * Create maps of threads and cpus to monitor. In this case
@@ -117,7 +108,6 @@ out_close_evlist:perf_evlist__close (evlist); out_delete_maps:perf_evlist__delete_maps (evlist); -out_free_evlist:perf_evlist__delete (evlist);return err; }
tools/perf/util/evlist.c +12 -0
@@ -50,6 +50,18 @@ struct perf_evlist *perf_evlist__new(void)return evlist; } +struct perf_evlist *perf_evlist__new_default (void ) +{ +struct perf_evlist *evlist =perf_evlist__new (); + +if (evlist &&perf_evlist__add_default (evlist)) { +perf_evlist__delete (evlist); + evlist =NULL ; + } + +return evlist; +} +/** * perf_evlist__set_id_pos - set the positions of event ids. * @evlist: selected event list
tools/perf/util/evlist.h +1 -0
@@ -53,6 +53,7 @@ struct perf_evsel_str_handler { };struct perf_evlist *perf_evlist__new (void ); +struct perf_evlist *perf_evlist__new_default (void );void perf_evlist__init (struct perf_evlist *evlist,struct cpu_map *cpus,struct thread_map *threads);void perf_evlist__exit (struct perf_evlist *evlist);
Signature · 1 line
1.7.11.7
tip-bot for Jiri Olsa
merge bot[tip:perf/core] perf evlist: Introduce perf_evlist__new_default function
perf evlist: Introduce perf_evlist__new_default function
Commit-ID: b22d54b09a5448d3706929c6f0eae36429f4ec5d Gitweb: http://git.kernel.org/tip/b22d54b09a5448d3706929c6f0eae36429f4ec5d Author: Jiri Olsa <[email protected]> AuthorDate: Sun, 1 Sep 2013 12:36:14 +0200 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Wed, 9 Oct 2013 11:24:34 -0300
perf evlist: Introduce perf_evlist__new_default function
Adding new common function to create evlist with default event. It spares some code lines in automated tests.
--- tools/perf/tests/perf-record.c | 12 +----------- tools/perf/tests/task-exit.c | 14 ++------------ tools/perf/util/evlist.c | 12 ++++++++++++ tools/perf/util/evlist.h | 1 + 4 files changed, 16 insertions(+), 23 deletions(-)
tools/perf/tests/perf-record.c +1 -11
@@ -45,7 +45,7 @@ int test__PERF_RECORD(void) };cpu_set_t cpu_mask;size_t cpu_mask_size =sizeof (cpu_mask); -struct perf_evlist *evlist =perf_evlist__new (); +struct perf_evlist *evlist =perf_evlist__new_default ();struct perf_evsel *evsel;struct perf_sample sample;const char *cmd ="sleep" ;
@@ -66,16 +66,6 @@ int test__PERF_RECORD(void) }/* -* We need at least one evsel in the evlist, use the default -* one: "cycles". -*/ - err =perf_evlist__add_default (evlist); -if (err <0 ) { -pr_debug ("Not enough memory to create evsel\n" ); - goto out_delete_evlist; - } - -/* * Create maps of threads and cpus to monitor. In this case * we start with all threads and cpus (-1, -1) but then in * perf_evlist__prepare_workload we'll fill in the only thread
tools/perf/tests/task-exit.c +2 -12
@@ -37,20 +37,11 @@ int test__task_exit(void)signal (SIGCHLD , sig_handler);signal (SIGUSR1 , sig_handler); - evlist =perf_evlist__new (); + evlist =perf_evlist__new_default ();if (evlist ==NULL ) { -pr_debug ("perf_evlist__new\n" ); +pr_debug ("perf_evlist__new_default\n" );return -1 ; } -/* -* We need at least one evsel in the evlist, use the default -* one: "cycles". -*/ - err =perf_evlist__add_default (evlist); -if (err <0 ) { -pr_debug ("Not enough memory to create evsel\n" ); - goto out_free_evlist; - }/* * Create maps of threads and cpus to monitor. In this case
@@ -117,7 +108,6 @@ out_close_evlist:perf_evlist__close (evlist); out_delete_maps:perf_evlist__delete_maps (evlist); -out_free_evlist:perf_evlist__delete (evlist);return err; }
tools/perf/util/evlist.c +12 -0
@@ -50,6 +50,18 @@ struct perf_evlist *perf_evlist__new(void)return evlist; } +struct perf_evlist *perf_evlist__new_default (void ) +{ +struct perf_evlist *evlist =perf_evlist__new (); + +if (evlist &&perf_evlist__add_default (evlist)) { +perf_evlist__delete (evlist); + evlist =NULL ; + } + +return evlist; +} +/** * perf_evlist__set_id_pos - set the positions of event ids. * @evlist: selected event list
tools/perf/util/evlist.h +1 -0
@@ -53,6 +53,7 @@ struct perf_evsel_str_handler { };struct perf_evlist *perf_evlist__new (void ); +struct perf_evlist *perf_evlist__new_default (void );void perf_evlist__init (struct perf_evlist *evlist,struct cpu_map *cpus,struct thread_map *threads);void perf_evlist__exit (struct perf_evlist *evlist);
PATCH v2 4/25 perf tools: Adding throttle event data struct support
In reply to Jiri OlsaMoving 'struct throttle_event' out of python code and making it global as any other event.
There's no usage of throttling events in any perf commands so far (besides python support), but we'll need this event data backup for upcomming test.
--- tools/perf/util/event.h | 7 +++++++ tools/perf/util/python.c | 7 ------- tools/perf/util/session.c | 13 +++++++++++++ 3 files changed, 20 insertions(+), 7 deletions(-)
tools/perf/util/event.h +7 -0
@@ -48,6 +48,12 @@ struct read_event {u64 id; }; +struct throttle_event { +struct perf_event_header header; +u64 time; +u64 id; +u64 stream_id; +};#define PERF_SAMPLE_MASK \ (PERF_SAMPLE_IP | PERF_SAMPLE_TID | \
@@ -163,6 +169,7 @@ union perf_event {struct fork_event fork;struct lost_event lost;struct read_event read; +struct throttle_event throttle;struct sample_event sample;struct attr_event attr;struct event_type_event event_type;
tools/perf/util/python.c +0 -7
@@ -33,13 +33,6 @@ int eprintf(int level, const char *fmt, ...)# define PyVarObject_HEAD_INIT (type, size) PyObject_HEAD_INIT(type) size,#endif -struct throttle_event { -struct perf_event_header header; -u64 time; -u64 id; -u64 stream_id; -}; -PyMODINIT_FUNC initperf (void );#define member_def (type, member, ptype, help) \
tools/perf/util/session.c +13 -0
@@ -376,6 +376,17 @@ static void perf_event__read_swap(union perf_event *event, bool sample_id_all)swap_sample_id_all (event, &event->read +1 ); } +static void perf_event__throttle_swap (union perf_event *event, +bool sample_id_all) +{ + event->throttle.time =bswap_64 (event->throttle.time); + event->throttle.id =bswap_64 (event->throttle.id); + event->throttle.stream_id =bswap_64 (event->throttle.stream_id); + +if (sample_id_all) +swap_sample_id_all (event, &event->throttle +1 ); +} +static u8 revbyte (u8 b) {int rev = (b >>4 ) | ((b &0xf ) <<4 );
@@ -460,6 +471,8 @@ static perf_event__swap_op perf_event__swap_ops[] = { [PERF_RECORD_EXIT] = perf_event__task_swap, [PERF_RECORD_LOST] = perf_event__all64_swap, [PERF_RECORD_READ] = perf_event__read_swap, + [PERF_RECORD_THROTTLE] = perf_event__throttle_swap, + [PERF_RECORD_UNTHROTTLE] = perf_event__throttle_swap, [PERF_RECORD_SAMPLE] = perf_event__all64_swap, [PERF_RECORD_HEADER_ATTR] = perf_event__hdr_attr_swap, [PERF_RECORD_HEADER_EVENT_TYPE] = perf_event__event_type_swap,
Signature · 1 line
1.7.11.7
tip-bot for Jiri Olsa
merge bot[tip:perf/core] perf tools: Adding throttle event data struct support
perf tools: Adding throttle event data struct support
Commit-ID: dd96c46b5c765a779d8c35cc7d1df7515b4c7baf Gitweb: http://git.kernel.org/tip/dd96c46b5c765a779d8c35cc7d1df7515b4c7baf Author: Jiri Olsa <[email protected]> AuthorDate: Sun, 1 Sep 2013 12:36:15 +0200 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Wed, 9 Oct 2013 11:39:18 -0300
perf tools: Adding throttle event data struct support
Moving 'struct throttle_event' out of python code and making it global as any other event.
There's no usage of throttling events in any perf commands so far (besides python support), but we'll need this event data backup for upcoming test.
--- tools/perf/util/event.h | 7 +++++++ tools/perf/util/python.c | 7 ------- tools/perf/util/session.c | 13 +++++++++++++ 3 files changed, 20 insertions(+), 7 deletions(-)
tools/perf/util/event.h +7 -0
@@ -61,6 +61,12 @@ struct read_event {u64 id; }; +struct throttle_event { +struct perf_event_header header; +u64 time; +u64 id; +u64 stream_id; +};#define PERF_SAMPLE_MASK \ (PERF_SAMPLE_IP | PERF_SAMPLE_TID | \
@@ -178,6 +184,7 @@ union perf_event {struct fork_event fork;struct lost_event lost;struct read_event read; +struct throttle_event throttle;struct sample_event sample;struct attr_event attr;struct event_type_event event_type;
tools/perf/util/python.c +0 -7
@@ -33,13 +33,6 @@ int eprintf(int level, const char *fmt, ...)# define PyVarObject_HEAD_INIT (type, size) PyObject_HEAD_INIT(type) size,#endif -struct throttle_event { -struct perf_event_header header; -u64 time; -u64 id; -u64 stream_id; -}; -PyMODINIT_FUNC initperf (void );#define member_def (type, member, ptype, help) \
tools/perf/util/session.c +13 -0
@@ -397,6 +397,17 @@ static void perf_event__read_swap(union perf_event *event, bool sample_id_all)swap_sample_id_all (event, &event->read +1 ); } +static void perf_event__throttle_swap (union perf_event *event, +bool sample_id_all) +{ + event->throttle.time =bswap_64 (event->throttle.time); + event->throttle.id =bswap_64 (event->throttle.id); + event->throttle.stream_id =bswap_64 (event->throttle.stream_id); + +if (sample_id_all) +swap_sample_id_all (event, &event->throttle +1 ); +} +static u8 revbyte (u8 b) {int rev = (b >>4 ) | ((b &0xf ) <<4 );
@@ -482,6 +493,8 @@ static perf_event__swap_op perf_event__swap_ops[] = { [PERF_RECORD_EXIT] = perf_event__task_swap, [PERF_RECORD_LOST] = perf_event__all64_swap, [PERF_RECORD_READ] = perf_event__read_swap, + [PERF_RECORD_THROTTLE] = perf_event__throttle_swap, + [PERF_RECORD_UNTHROTTLE] = perf_event__throttle_swap, [PERF_RECORD_SAMPLE] = perf_event__all64_swap, [PERF_RECORD_HEADER_ATTR] = perf_event__hdr_attr_swap, [PERF_RECORD_HEADER_EVENT_TYPE] = perf_event__event_type_swap,
PATCH v2 5/25 perf tests: Add simple session read/write test
In reply to Jiri OlsaAdding simple session read/write test to keep up
with file format changes.
Dumping the session to the file with several events and reading it again back.
--- tools/perf/Makefile | 1 + tools/perf/tests/builtin-test.c | 4 + tools/perf/tests/session-simple.c | 602 ++++++++++++++++++++++++++++++++++++++ tools/perf/tests/tests.h | 1 + 4 files changed, 608 insertions(+) create mode 100644 tools/perf/tests/session-simple.c
tools/perf/Makefile +1 -0
@@ -394,6 +394,7 @@ ifeq ($(ARCH),x86) LIB_OBJS += $(OUTPUT)tests/perf-time-to-tsc.o endif LIB_OBJS += $(OUTPUT)tests/code-reading.o +LIB_OBJS += $(OUTPUT)tests/session-simple.o BUILTIN_OBJS += $(OUTPUT)builtin-annotate.o BUILTIN_OBJS += $(OUTPUT)builtin-bench.o
tools/perf/tests/builtin-test.c +4 -0
@@ -108,6 +108,10 @@ static struct test { .func = test__sample_parsing, }, { + .desc = "Test session - simple read/write", + .func = test__session_simple, + }, + { .func = NULL, }, };
tools/perf/tests/session-simple.c new file +602 -0
602 lines not on this page · raw message
tools/perf/tests/tests.h +1 -0
@@ -38,5 +38,6 @@ int test__sw_clock_freq(void);int test__perf_time_to_tsc (void );int test__code_reading (void );int test__sample_parsing (void ); +int test__session_simple (void );#endif /* TESTS_H */
Signature · 1 line
1.7.11.7
PATCH v2 6/25 perf tests: Add session reading test for little endian perf data
In reply to Jiri OlsaAdding session reading test for little endian perf data included as binary data in the test object.
--- tools/perf/Makefile | 1 + tools/perf/tests/perf.data.v2.le.h | 219 +++++++++++++++++++++++++++++++++++++ tools/perf/tests/session-simple.c | 33 ++++++ 3 files changed, 253 insertions(+) create mode 100644 tools/perf/tests/perf.data.v2.le.h
tools/perf/Makefile +1 -0
@@ -292,6 +292,7 @@ LIB_H += ui/helpline.h LIB_H += ui/progress.h LIB_H += ui/util.h LIB_H += ui/ui.h +LIB_H += tests/perf.data.v2.le.h LIB_OBJS += $(OUTPUT)util/abspath.o LIB_OBJS += $(OUTPUT)util/alias.o
tools/perf/tests/perf.data.v2.le.h new file +219 -0
219 lines not on this page · raw message
tools/perf/tests/session-simple.c +33 -0
@@ -21,6 +21,10 @@#define EVENTS_UNTHROTTLE 3#define EVENTS_SAMPLE 4 +char data_file_v2_le[] = { +#include "perf.data.v2.le.h" +}; +static int events_mmap;static int events_lost;static int events_comm;
@@ -592,11 +596,40 @@ static int test_generated_data(void)return err; } +static int file_write (char *file,char *data,ssize_t size) +{ +int fd =open (file,O_TRUNC |O_RDWR ); +int err =0 ; + +if (size !=write (fd, data, size)) + err =-1 ; + +close (fd); +return err; +} + +static int test_file_data (char *data,ssize_t size) +{ +char *file =get_file (); +int err =0 ; + +TEST_ASSERT_VAL ("failed to get temporary file" , file); + + err =file_write (file, data, size); +if (!err) + err =session_read (file); + +unlink (file); +return err; +} +int test__session_simple (void ) {int err;pr_debug ("Testing generated data\n" ); err =test_generated_data (); +pr_debug ("Testing v2 LE data\n" ); + err |=test_file_data (data_file_v2_le,sizeof (data_file_v2_le));return err; }
Signature · 1 line
1.7.11.7
PATCH v2 7/25 perf tests: Add session reading test for big endian perf data
In reply to Jiri OlsaAdding session reading test for big endian perf data included as binary data in the test object.
--- tools/perf/Makefile | 1 + tools/perf/tests/perf.data.v2.be.h | 265 +++++++++++++++++++++++++++++++++++++ tools/perf/tests/session-simple.c | 6 + 3 files changed, 272 insertions(+) create mode 100644 tools/perf/tests/perf.data.v2.be.h
tools/perf/Makefile +1 -0
@@ -293,6 +293,7 @@ LIB_H += ui/progress.h LIB_H += ui/util.h LIB_H += ui/ui.h LIB_H += tests/perf.data.v2.le.h +LIB_H += tests/perf.data.v2.be.h LIB_OBJS += $(OUTPUT)util/abspath.o LIB_OBJS += $(OUTPUT)util/alias.o
tools/perf/tests/perf.data.v2.be.h new file +265 -0
265 lines not on this page · raw message
tools/perf/tests/session-simple.c +6 -0
@@ -25,6 +25,10 @@ char data_file_v2_le[] = {#include "perf.data.v2.le.h" }; +char data_file_v2_be[] = { +#include "perf.data.v2.be.h" +}; +static int events_mmap;static int events_lost;static int events_comm;
@@ -631,5 +635,7 @@ int test__session_simple(void) err =test_generated_data ();pr_debug ("Testing v2 LE data\n" ); err |=test_file_data (data_file_v2_le,sizeof (data_file_v2_le)); +pr_debug ("Testing v2 BE data\n" ); + err |=test_file_data (data_file_v2_be,sizeof (data_file_v2_be));return err; }
Signature · 1 line
1.7.11.7
PATCH v2 8/25 perf doc: Add perf data file documentation
In reply to Jiri OlsaAdding perf data file documentation.
TODO Finish the FEATURES section
--- tools/perf/Documentation/perf-data-file-v2.txt | 264 +++++++++++++++++++++++++ 1 file changed, 264 insertions(+) create mode 100644 tools/perf/Documentation/perf-data-file-v2.txt
tools/perf/Documentation/perf-data-file-v2.txt new file +264 -0
@@ -0,0 +1,264 @@ +perf-data-file-v2(5) +==================== + +NAME +---- +perf-data-file-v2 - The perf tool file format version 2 + + +DESCRIPTION +----------- +Following text describes version 2 of the perf data file format, +which is version that is currently used by perf tool. + +The perf data file format is composed of several sections +describing monitored events and the data itself. + +High level view of the format: + FILE HEADER + EVENT IDS + EVENT ATTRIBUTES + EVENT TYPES + EVENT DATA + FEATURES + + +COMMON PRIMITIVES +----------------- +Following definitions are used as primitives on multiple places +within the perf data file. + +struct perf_file_section:: + Used to determine location of data sections within the file. + + struct perf_file_section { + u64 offset; + u64 size; + }; + + offset section location (offset) within the file + size section size in bytes + + +FILE HEADER +----------- +Starting point of the data file with magic bytes and global +section information. + +The section contains following data: + struct perf_file_header header + +struct perf_file_header:: + struct perf_file_header { + u64 magic; + u64 size; + u64 attr_size; + struct perf_file_section attrs; + struct perf_file_section data; + struct perf_file_section event_types; + DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); + } + + magic ID 'PERFFILE|PERFILE2' + size header size sizeof(struct perf_file_header) + attr_size attribute size sizeof(struct perf_file_attr) + (see EVENT ATTRIBUTES) + attrs location of 'EVENT ATTRIBUTES' + data location of 'EVENT DATA' + event_types location of 'EVENT TYPES' + adds_features 'FEATURES' bitmask + + +EVENT IDS +--------- +This section contains CPU related IDs for each monitored event. +It is used only if there was more than 1 events monitored. The +reason is to be able to map event's data to event definition. + +The section contains following data: + u64 event1_ids[] + u64 event2_ids[] + ... + u64 eventX_ids[] + +eventX_ids:: + per-cpu event X IDs array + this array is referenced in 'EVENT ATTRIBUTES' + + +EVENT ATTRIBUTES +---------------- +This section contains attribute information together with the +location of the IDs array for each event. + +The section contains following data: + struct perf_file_attr attrs[] + +struct perf_file_attr:: + struct perf_file_attr { + struct perf_event_attr attr; + struct perf_file_section ids; + } + + attr perf event interface structure for each event + ids IDs location for event in 'EVENT IDS' section + + +EVENT TYPES +----------- +This section map event's config value (struct perf_event_attr::config) +with the event name. + +The section contains following data: + struct perf_trace_event_type types[] + +#define MAX_EVENT_NAME 64 + +struct perf_trace_event_type:: + struct perf_trace_event_type { + u64 event_id; + char name[MAX_EVENT_NAME]; + }; + + event_id struct perf_event_attr::config value + name event name + +NOTE This section got deprecated and its data are no longer stored. + The 'event_types' location in the header is zero-ed. + + +EVENT DATA +---------- +This section contains blob of all events data - auxiliary events +and samples. + + +FEATURES +-------- +This section contains various configuration data and its contents +depends on the header's adds_features bitmask. Each bit in this +bitmask represent a single feature (0 - NOT present, 1 present). + +The section contains following data: + struct perf_file_section feature_locations[nr_features]; + + nr_features number of present features bitmap_weight(adds_features) + feature_locations location of each feature data + +Available features (possible max 256): + enum { + HEADER_RESERVED = 0, + HEADER_TRACING_DATA = 1, + HEADER_BUILD_ID, + HEADER_HOSTNAME, + HEADER_OSRELEASE, + HEADER_VERSION, + HEADER_ARCH, + HEADER_NRCPUS, + HEADER_CPUDESC, + HEADER_CPUID, + HEADER_TOTAL_MEM, + HEADER_CMDLINE, + HEADER_EVENT_DESC, + HEADER_CPU_TOPOLOGY, + HEADER_NUMA_TOPOLOGY, + HEADER_BRANCH_STACK, + HEADER_PMU_MAPPINGS, + HEADER_GROUP_DESC, + HEADER_LAST_FEATURE, + HEADER_FEAT_BITS = 256, + } + + +TODO finish FEATURES::* sections descriptions. + +FEATURES::HEADER_RESERVED +------------------------- +allways zero bit + + +FEATURES::HEADER_TRACING_DATA +----------------------------- +tracing data + + +FEATURES::HEADER_BUILD_ID +------------------------- +build ids + + +FEATURES::HEADER_HOSTNAME +------------------------- +uname hostname + + +FEATURES::HEADER_OSRELEASE +-------------------------- +uname osrelease + + +FEATURES::HEADER_VERSION +------------------------ +perf version + + +FEATURES::HEADER_ARCH +--------------------- +uname arch + + +FEATURES::HEADER_NRCPUS +----------------------- +offset size(B) description +0 4 sysconf(_SC_NPROCESSORS_CONF) +4 4 sysconf(_SC_NPROCESSORS_ONLN) + + +FEATURES::HEADER_CPUDESC +------------------------ +/proc/cpuinfo data for 'Model name' + + +FEATURES::HEADER_CPUID +---------------------- +'vendor,family,model,step' + + +FEATURES::HEADER_TOTAL_MEM +-------------------------- +/proc/meminfo data for 'MemTotal' + + +FEATURES::HEADER_CMDLINE +------------------------ +perf complete command line + + +FEATURES::HEADER_EVENT_DESC +--------------------------- +event attributes plus related info(ids) + + +FEATURES::HEADER_CPU_TOPOLOGY +----------------------------- +cpu topology + + +FEATURES::HEADER_NUMA_TOPOLOGY +------------------------------ +numa topology + + +FEATURES::HEADER_BRANCH_STACK +----------------------------- +nothing + + +FEATURES::HEADER_PMU_MAPPINGS +----------------------------- +available pmus + + +FEATURES::HEADER_GROUP_DESC +--------------------------- +event groups info
Signature · 1 line
1.7.11.7
PATCH v2 9/25 perf tools: Introduce perf data file version CHECK macro
In reply to Jiri OlsaIntroducing perf data file version CHECK macro, so the check is easily extensible for new versions.
--- tools/perf/util/header.c | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-)
tools/perf/util/header.c +16 -11
@@ -2496,18 +2496,23 @@ static int check_magic_endian(u64 magic, uint64_t hdr_sz,* - encode endianness of file */ -/* check magic number with one endianness */ -if (magic == __perf_magic2) -return 0 ; - -/* check magic number with opposite endianness */ -if (magic != __perf_magic2_sw) -return -1 ; +#define CHECK(__m, __v) \ +do { \ + /* check magic number with one endianness */ \ + if (magic == __m) \ + goto v ## __v; \ + /* check magic number with opposite endianness */ \ + if (magic != __m ## _sw) \ + break; \ + ph->needs_swap = true; \ + v ## __v: \ + ph->version = __v; \ + return 0; \ +} while (0) + +CHECK (__perf_magic2,PERF_HEADER_VERSION_2 ); - ph->needs_swap = true; - ph->version =PERF_HEADER_VERSION_2 ; - -return 0 ; +return -1 ; } int perf_file_header__read(struct perf_file_header *header,
Signature · 1 line
1.7.11.7
PATCH v2 10/25 perf tools: Introduce swap_features function
In reply to Jiri OlsaIntroducing swap_features function to make the swapping code more clear and extensible.
--- tools/perf/util/header.c | 69 ++++++++++++++++++++++++------------------------ 1 file changed, 35 insertions(+), 34 deletions(-)
tools/perf/util/header.c +35 -34
@@ -2515,6 +2515,39 @@ do { \return -1 ; } +static void swap_features (unsigned long *adds_features) +{ +/* +* feature bitmap is declared as an array of unsigned longs -- +* not good since its size can differ between the host that +* generated the data file and the host analyzing the file. +* +* We need to handle endianness, but we don't know the size of +* the unsigned long where the file was generated. Take a best +* guess at determining it: try 64-bit swap first (ie., file +* created on a 64-bit host), and check if the hostname feature +* bit is set (this feature bit is forced on as of fbe96f2). +* If the bit is not, undo the 64-bit swap and try a 32-bit +* swap. If the hostname bit is still not set (e.g., older data +* file), punt and fallback to the original behavior -- +* clearing all feature bits and setting buildid. +*/ +mem_bswap_64 (adds_features,BITS_TO_U64 (HEADER_FEAT_BITS )); + +if (!test_bit (HEADER_HOSTNAME , adds_features)) { +/* unswap as u64 */ +mem_bswap_64 (adds_features,BITS_TO_U64 (HEADER_FEAT_BITS )); + +/* unswap as u32 */ +mem_bswap_32 (adds_features,BITS_TO_U32 (HEADER_FEAT_BITS )); + } + +if (!test_bit (HEADER_HOSTNAME , adds_features)) { +bitmap_zero (adds_features,HEADER_FEAT_BITS ); +set_bit (HEADER_BUILD_ID , adds_features); + } +} +int perf_file_header__read (struct perf_file_header *header,struct perf_header *ph,int fd) {
@@ -2543,40 +2576,8 @@ int perf_file_header__read(struct perf_file_header *header,bitmap_zero (header->adds_features,HEADER_FEAT_BITS ); else return -1; - }else if (ph->needs_swap) { -/* -* feature bitmap is declared as an array of unsigned longs -- -* not good since its size can differ between the host that -* generated the data file and the host analyzing the file. -* -* We need to handle endianness, but we don't know the size of -* the unsigned long where the file was generated. Take a best -* guess at determining it: try 64-bit swap first (ie., file -* created on a 64-bit host), and check if the hostname feature -* bit is set (this feature bit is forced on as of fbe96f2). -* If the bit is not, undo the 64-bit swap and try a 32-bit -* swap. If the hostname bit is still not set (e.g., older data -* file), punt and fallback to the original behavior -- -* clearing all feature bits and setting buildid. -*/ -mem_bswap_64 (&header->adds_features, -BITS_TO_U64 (HEADER_FEAT_BITS )); - -if (!test_bit (HEADER_HOSTNAME , header->adds_features)) { -/* unswap as u64 */ -mem_bswap_64 (&header->adds_features, -BITS_TO_U64 (HEADER_FEAT_BITS )); - -/* unswap as u32 */ -mem_bswap_32 (&header->adds_features, -BITS_TO_U32 (HEADER_FEAT_BITS )); - } - -if (!test_bit (HEADER_HOSTNAME , header->adds_features)) { -bitmap_zero (header->adds_features,HEADER_FEAT_BITS ); -set_bit (HEADER_BUILD_ID , header->adds_features); - } - } + }else if (ph->needs_swap) +swap_features (header->adds_features);memcpy (&ph->adds_features, &header->adds_features,sizeof (ph->adds_features));
Signature · 1 line
1.7.11.7
PATCH v2 11/25 perf tools: Introduce swap_header function
In reply to Jiri OlsaIntroducing swap_header function to make the swapping code more clear and extensible.
--- tools/perf/util/header.c | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-)
tools/perf/util/header.c +18 -13
@@ -2548,6 +2548,22 @@ static void swap_features(unsigned long *adds_features) } } +static int swap_header (struct perf_file_header *header) +{ +mem_bswap_64 (header, offsetof(struct perf_file_header , adds_features)); + +if (header->size !=sizeof (*header)) { +/* Support the previous format */ + if (header->size == offsetof(typeof(*header), adds_features)) +bitmap_zero (header->adds_features,HEADER_FEAT_BITS ); +else +return -1 ; + }else +swap_features (header->adds_features); + +return 0 ; +} +int perf_file_header__read (struct perf_file_header *header,struct perf_header *ph,int fd) {
@@ -2565,19 +2581,8 @@ int perf_file_header__read(struct perf_file_header *header,return -1 ; } -if (ph->needs_swap) { -mem_bswap_64 (header, offsetof(struct perf_file_header , - adds_features)); - } - -if (header->size !=sizeof (*header)) { -/* Support the previous format */ - if (header->size == offsetof(typeof(*header), adds_features)) -bitmap_zero (header->adds_features,HEADER_FEAT_BITS ); -else -return -1 ; - }else if (ph->needs_swap) -swap_features (header->adds_features); +if (ph->needs_swap &&swap_header (header)) +return -1 ;memcpy (&ph->adds_features, &header->adds_features,sizeof (ph->adds_features));
Signature · 1 line
1.7.11.7
PATCH v2 12/25 perf tools: Separate version 2 specific perf data header bits
In reply to Jiri OlsaSeparating version 2 specific perf data header bits, so the code could be extented with new format version.
--- tools/perf/util/header.c | 122 +++++++++++++++++++++++++++++++---------------- tools/perf/util/header.h | 16 +++++-- 2 files changed, 92 insertions(+), 46 deletions(-)
tools/perf/util/header.c +80 -42
@@ -2319,18 +2319,21 @@ int perf_session__write_header(struct perf_session *session, .magic = PERF_MAGIC, .size = sizeof(f_header), .attr_size = sizeof(f_attr), - .attrs = { - .offset = attr_offset, - .size = evlist->nr_entries * sizeof(f_attr), - }, - .data = { - .offset = header->data_offset, - .size = header->data_size, - }, + .v2 = { + .attrs = { + .offset = attr_offset, + .size = evlist->nr_entries * sizeof(f_attr), + }, + .data = { + .offset = header->data_offset, + .size = header->data_size, + }, /* event_types is ignored, store zeros */ + }, }; -memcpy (&f_header.adds_features, &header->adds_features,sizeof (header->adds_features)); +memcpy (&f_header.v2.adds_features, &header->adds_features, +sizeof (header->adds_features));lseek (fd,0 ,SEEK_SET ); err =do_write (fd, &f_header,sizeof (f_header));
@@ -2548,22 +2551,34 @@ static void swap_features(unsigned long *adds_features) } } -static int swap_header (struct perf_file_header *header) +static int swap_header_v2 (struct perf_file_header *header) { -mem_bswap_64 (header, offsetof(struct perf_file_header , adds_features)); +struct perf_file_header_v2 *v2 = &header->v2; + +mem_bswap_64 (v2, offsetof(struct perf_file_header_v2 , + adds_features));if (header->size !=sizeof (*header)) {/* Support the previous format */ - if (header->size == offsetof(typeof(*header), adds_features)) -bitmap_zero (header->adds_features,HEADER_FEAT_BITS ); + if (header->size == offsetof(typeof(*header), v2.adds_features)) +bitmap_zero (v2->adds_features,HEADER_FEAT_BITS );else return -1 ; }else -swap_features (header->adds_features); +swap_features (v2->adds_features);return 0 ; } +static int swap_header (struct perf_file_header *header) +{ +/* swap the generic part */ +mem_bswap_64 (header, offsetof(struct perf_file_header , v2)); + +/* version specific swap */ +return swap_header_v2 (header); +} +int perf_file_header__read (struct perf_file_header *header,struct perf_header *ph,int fd) {
@@ -2581,16 +2596,7 @@ int perf_file_header__read(struct perf_file_header *header,return -1 ; } -if (ph->needs_swap &&swap_header (header)) -return -1 ; - -memcpy (&ph->adds_features, &header->adds_features, -sizeof (ph->adds_features)); - - ph->data_offset = header->data.offset; - ph->data_size = header->data.size; - ph->feat_offset = header->data.offset + header->data.size; -return 0 ; +return ph->needs_swap ?swap_header (header) :0 ; } static int perf_file_section__process(struct perf_file_section *section,
@@ -2742,12 +2748,13 @@ static int perf_evlist__prepare_tracepoint_events(struct perf_evlist *evlist,return 0 ; } -int perf_session__read_header (struct perf_session *session) +static int __perf_session__read_header_v2 (struct perf_session *session, +struct perf_file_header *header) { -struct perf_header *header = &session->header; -struct perf_file_header f_header; -struct perf_file_attr f_attr; -u64 f_id; +struct perf_header *ph = &session->header; +struct perf_file_header_v2 *v2 = &header->v2; +struct perf_file_attr f_attr; +u64 f_id;int nr_attrs, nr_ids, i, j;int fd = session->fd;
@@ -2755,23 +2762,17 @@ int perf_session__read_header(struct perf_session *session)if (session->evlist ==NULL )return -ENOMEM ; -if (session->fd_pipe) -return perf_header__read_pipe (session); - -if (perf_file_header__read (&f_header, header, fd) <0 ) -return -EINVAL ; - - nr_attrs = f_header.attrs.size / f_header.attr_size; -lseek (fd, f_header.attrs.offset,SEEK_SET ); + nr_attrs = v2->attrs.size / header->attr_size; +lseek (fd, v2->attrs.offset,SEEK_SET );for (i =0 ; i < nr_attrs; i++) {struct perf_evsel *evsel;off_t tmp; -if (read_attr (fd, header, &f_attr) <0 ) +if (read_attr (fd, ph, &f_attr) <0 ) goto out_errno; -if (header->needs_swap) +if (ph->needs_swap)perf_event__attr_swap (&f_attr.attr); tmp =lseek (fd,0 ,SEEK_CUR );
@@ -2780,7 +2781,7 @@ int perf_session__read_header(struct perf_session *session)if (evsel ==NULL ) goto out_delete_evlist; - evsel->needs_swap = header->needs_swap; + evsel->needs_swap = ph->needs_swap;/* * Do it before so that if perf_evsel__alloc_id fails, this * entry gets purged too at perf_evlist__delete().
@@ -2799,7 +2800,8 @@ int perf_session__read_header(struct perf_session *session)lseek (fd, f_attr.ids.offset,SEEK_SET );for (j =0 ; j < nr_ids; j++) { -if (perf_header__getbuffer64 (header, fd, &f_id,sizeof (f_id))) +if (perf_header__getbuffer64 (ph, fd, &f_id, +sizeof (f_id))) goto out_errno;perf_evlist__id_add (session->evlist, evsel,0 , j, f_id);
@@ -2810,7 +2812,7 @@ int perf_session__read_header(struct perf_session *session) symbol_conf.nr_events = nr_attrs; -perf_header__process_sections (header, fd, &session->pevent, +perf_header__process_sections (ph, fd, &session->pevent, perf_file_section__process); if (perf_evlist__prepare_tracepoint_events(session->evlist,
@@ -2827,6 +2829,42 @@ out_delete_evlist:return -ENOMEM ; } + +static int perf_session__read_header_v2 (struct perf_session *session, +struct perf_file_header *header) +{ +struct perf_header *ph = &session->header; +struct perf_file_header_v2 *v2 = &header->v2; + +memcpy (&ph->adds_features, &v2->adds_features, +sizeof (ph->adds_features)); + + ph->data_offset = v2->data.offset; + ph->data_size = v2->data.size; + ph->feat_offset = v2->data.offset + v2->data.size; + +return __perf_session__read_header_v2 (session, header); +} + +static int perf_header_read_file (struct perf_session *session) +{ +struct perf_file_header header; + +if (perf_file_header__read (&header, &session->header, session->fd)) +return -1 ; + +/* read v2 specific data */ +return perf_session__read_header_v2 (session, &header); +} + +int perf_session__read_header (struct perf_session *session) +{ +if (session->fd_pipe) +return perf_header__read_pipe (session); + +return perf_header_read_file (session) <0 ? -EINVAL :0 ; +} +int perf_event__synthesize_attr (struct perf_tool *tool,struct perf_event_attr *attr,u32 ids,u64 *id,perf_event__handler_t process)
tools/perf/util/header.h +12 -4
@@ -44,10 +44,7 @@ struct perf_file_section {u64 size; }; -struct perf_file_header { -u64 magic; -u64 size; -u64 attr_size; +struct perf_file_header_v2 {struct perf_file_section attrs;struct perf_file_section data;/* event_types is ignored */
@@ -55,6 +52,17 @@ struct perf_file_header {DECLARE_BITMAP (adds_features,HEADER_FEAT_BITS ); }; +struct perf_file_header { +u64 magic; +u64 size; +u64 attr_size; + +/* version specific data */ +union { +struct perf_file_header_v2 v2; + }; +}; +struct perf_pipe_file_header {u64 magic;u64 size;
Signature · 1 line
1.7.11.7
PATCH v2 13/25 perf tools: Using evlist as a holder for event_desc feature
In reply to Jiri OlsaUsing evlist as a holder for events stored via event_desc feature. The event_desc events data is same as the one stored via main header. We will use event_desc data to replace the main header data source in next patches.
--- tools/perf/util/header.c | 84 ++++++++++++++++++++++-------------------------- 1 file changed, 38 insertions(+), 46 deletions(-)
tools/perf/util/header.c +38 -46
@@ -1208,28 +1208,11 @@ static void print_cpu_topology(struct perf_header *ph, int fd __maybe_unused, } } -static void free_event_desc (struct perf_evsel *events) -{ -struct perf_evsel *evsel; - -if (!events) -return ; - -for (evsel = events; evsel->attr.size; evsel++) { -if (evsel->name) -free (evsel->name); -if (evsel->id) -free (evsel->id); - } - -free (events); -} - -static struct perf_evsel * +static struct perf_evlist *read_event_desc (struct perf_header *ph,int fd) { -struct perf_evsel *evsel, *events =NULL ; -u64 *id; +struct perf_evlist *evlist =NULL ; +struct perf_evsel *evsel;void *buf =NULL ;u32 nre, sz, nr, i, j;ssize_t ret;
@@ -1255,18 +1238,15 @@ read_event_desc(struct perf_header *ph, int fd)if (!buf) goto error; -/* the last event terminates with evsel->attr.size == 0: */ - events =calloc (nre +1 ,sizeof (*events)); -if (!events) + evlist =perf_evlist__new (); +if (!evlist) goto error; msz =sizeof (evsel->attr);if (sz < msz) msz = sz; -for (i =0 , evsel = events; i < nre; evsel++, i++) { - evsel->idx = i; - +for (i =0 ; i < nre; i++) {/* * must read entire on-file attr struct to * sync up with layout.
@@ -1275,6 +1255,10 @@ read_event_desc(struct perf_header *ph, int fd)if (ret != (ssize_t )sz) goto error; + evsel =perf_evsel__new (buf, i); +if (!evsel) + goto error; +if (ph->needs_swap)perf_event__attr_swap (buf);
@@ -1289,49 +1273,55 @@ read_event_desc(struct perf_header *ph, int fd) evsel->needs_swap = true; } +perf_evlist__add (evlist, evsel); + evsel->name =do_read_string (fd, ph);if (!nr)continue ; - id =calloc (nr,sizeof (*id)); -if (!id) +if (perf_evsel__alloc_id (evsel,1 , nr)) goto error; + evsel->ids = nr; - evsel->id = id;for (j =0 ; j < nr; j++) { - ret =readn (fd, id,sizeof (*id)); -if (ret != (ssize_t )sizeof (*id)) +u64 id; + + ret =readn (fd, &id,sizeof (id)); +if (ret != (ssize_t )sizeof (id)) goto error;if (ph->needs_swap) - *id =bswap_64 (*id); - id++; + id =bswap_64 (id); + +perf_evlist__id_add (evlist, evsel,0 , j, id); } } out:if (buf)free (buf); -return events; +return evlist; error: -if (events) -free_event_desc (events); - events =NULL ; +if (evlist) +perf_evlist__delete (evlist); + evlist =NULL ; goto out; }static void print_event_desc (struct perf_header *ph,int fd,FILE *fp) { -struct perf_evsel *evsel, *events =read_event_desc (ph, fd); +struct perf_evlist *evlist; +struct perf_evsel *evsel;u32 j;u64 *id; -if (!events) { + evlist =read_event_desc (ph, fd); +if (!evlist) {fprintf (fp,"# event desc: not available or unable to read\n" );return ; } -for (evsel = events; evsel->attr.size; evsel++) { +list_for_each_entry (evsel, &evlist->entries, node) {fprintf (fp,"# event : name = %s, " , evsel->name); fprintf(fp, "type = %d, config = 0x%"PRIx64
@@ -1364,7 +1354,7 @@ static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)fputc ('\n' , fp); } -free_event_desc (events); +perf_evlist__delete (evlist); } static void print_total_mem(struct perf_header *ph, int fd __maybe_unused,
@@ -1785,17 +1775,19 @@ process_event_desc(struct perf_file_section *section __maybe_unused, void *data __maybe_unused) {struct perf_session *session; -struct perf_evsel *evsel, *events =read_event_desc (header, fd); +struct perf_evlist *evlist; +struct perf_evsel *evsel; -if (!events) + evlist =read_event_desc (header, fd); +if (!evlist)return 0 ; session = container_of(header, struct perf_session, header); -for (evsel = events; evsel->attr.size; evsel++) -perf_evlist__set_event_name (session->evlist, evsel); -free_event_desc (events); +list_for_each_entry (evsel, &evlist->entries, node) +perf_evlist__set_event_name (session->evlist, evsel); +perf_evlist__delete (evlist);return 0 ; }
Signature · 1 line
1.7.11.7
PATCH v2 14/25 perf tools: Introduce perf.data version 3 format
In reply to Jiri OlsaIntroducing perf.data version 3 format. No functional change, just introducing doc, magic bytes and the struct.
--- tools/perf/Documentation/perf-data-file-v3.txt | 63 ++++++++++++++++++++++++++ tools/perf/util/header.c | 8 +++- tools/perf/util/header.h | 8 ++++ 3 files changed, 78 insertions(+), 1 deletion(-) create mode 100644 tools/perf/Documentation/perf-data-file-v3.txt
tools/perf/Documentation/perf-data-file-v3.txt new file +63 -0
@@ -0,0 +1,63 @@ +perf-data-file-v3(5) +==================== + +NAME +---- +perf-data-file-v3 - The perf tool file format version 3 + + +DESCRIPTION +----------- +Following text describes version 3 of the perf data file format, +which is version that is currently used by perf tool. + +In version 3 we got rid of following sections: + EVENT IDS + EVENT ATTRIBUTES + EVENT TYPES + +and keep only data and features sections. All the data +from above listed sections are now read from specific +FEATURES sections. + +High level view of the format: + FILE HEADER + EVENT DATA + FEATURES + + +FILE HEADER +----------- +Starting point of the data file with magic bytes and global +section information. + +The section contains following data: + struct perf_file_header header + +struct perf_file_header:: + struct perf_file_header { + u64 magic; + u64 size; + u64 attr_size; + struct perf_file_section data; + struct perf_file_section features; + DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); + } + + magic ID 'PERFILE3' + size header size sizeof(struct perf_file_header) + attr_size attribute size sizeof(struct perf_file_attr) + data location of 'EVENT DATA' + features location of 'EVENT TYPES' + adds_features 'FEATURES' bitmask + + +EVENT DATA +---------- +This section contains blob of all events' data - auxiliary events +and samples. + + +FEATURES +-------- +Same as in version 2.
tools/perf/util/header.c +7 -1
@@ -30,6 +30,7 @@ static const char **header_argv;/* * magic2 = "PERFILE2" +* magic3 = "PERFILE3" * must be a numerical value to let the endianness * determine the memory layout. That way we are able * to detect endianness when reading the perf.data file
@@ -40,6 +41,8 @@ static const char **header_argv;static const char *__perf_magic1 ="PERFFILE" ;static const u64 __perf_magic2 =0x32454c4946524550ULL ;static const u64 __perf_magic2_sw =0x50455246494c4532ULL ; +static const u64 __perf_magic3 =0x33454c4946524550ULL ; +static const u64 __perf_magic3_sw =0x50455246494c4533ULL ;#define PERF_MAGIC __perf_magic2
@@ -2464,7 +2467,9 @@ bool is_perf_magic(u64 magic) {if (!memcmp (&magic, __perf_magic1,sizeof (magic)) || magic == __perf_magic2 - || magic == __perf_magic2_sw) + || magic == __perf_magic2_sw + || magic == __perf_magic3 + || magic == __perf_magic3_sw)return true;return false;
@@ -2506,6 +2511,7 @@ do { \ }while (0 )CHECK (__perf_magic2,PERF_HEADER_VERSION_2 ); +CHECK (__perf_magic3,PERF_HEADER_VERSION_3 );return -1 ; }
tools/perf/util/header.h +8 -0
@@ -37,6 +37,7 @@ enum {enum perf_header_version {PERF_HEADER_VERSION_1 ,PERF_HEADER_VERSION_2 , +PERF_HEADER_VERSION_3 , };struct perf_file_section {
@@ -52,6 +53,12 @@ struct perf_file_header_v2 {DECLARE_BITMAP (adds_features,HEADER_FEAT_BITS ); }; +struct perf_file_header_v3 { +struct perf_file_section data; +struct perf_file_section features; + DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS); +}; +struct perf_file_header {u64 magic;u64 size;
@@ -60,6 +67,7 @@ struct perf_file_header {/* version specific data */ union {struct perf_file_header_v2 v2; +struct perf_file_header_v3 v3; }; };
Signature · 1 line
1.7.11.7
PATCH v2 15/25 perf tools: Add perf data version 3 header swap
In reply to Jiri OlsaAdding perf data version 3 header swap same way as it's done for v2.
--- tools/perf/util/header.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-)
tools/perf/util/header.c +18 -3
@@ -2568,13 +2568,28 @@ static int swap_header_v2(struct perf_file_header *header)return 0 ; } -static int swap_header (struct perf_file_header *header) +static int swap_header_v3 (struct perf_file_header *header) +{ +struct perf_file_header_v3 *v3 = &header->v3; + +mem_bswap_64 (v3, offsetof(struct perf_file_header_v3 , + adds_features)); + +swap_features (v3->adds_features); +return 0 ; +} + +static int swap_header (struct perf_header *ph, +struct perf_file_header *header) {/* swap the generic part */ mem_bswap_64 (header, offsetof(struct perf_file_header , v2));/* version specific swap */ -return swap_header_v2 (header); +if (ph->version <=PERF_HEADER_VERSION_2 ) +return swap_header_v2 (header); + +return swap_header_v3 (header); } int perf_file_header__read(struct perf_file_header *header,
@@ -2594,7 +2609,7 @@ int perf_file_header__read(struct perf_file_header *header,return -1 ; } -return ph->needs_swap ?swap_header (header) :0 ; +return ph->needs_swap ?swap_header (ph, header) :0 ; } static int perf_file_section__process(struct perf_file_section *section,
Signature · 1 line
1.7.11.7
PATCH v2 16/25 perf tools: Add perf data version 3 header read
In reply to Jiri OlsaAdding perf data version 3 header read code to load data for v3 format.
--- tools/perf/util/header.c | 46 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 39 insertions(+), 7 deletions(-)
tools/perf/util/header.c +39 -7
@@ -1780,17 +1780,26 @@ process_event_desc(struct perf_file_section *section __maybe_unused,struct perf_session *session;struct perf_evlist *evlist;struct perf_evsel *evsel; +struct perf_header *ph; + + session = container_of(header, struct perf_session, header); evlist =read_event_desc (header, fd);if (!evlist) -return 0 ; +return -1 ; - session = container_of(header, struct perf_session, header); + ph = &session->header; -list_for_each_entry (evsel, &evlist->entries, node) -perf_evlist__set_event_name (session->evlist, evsel); +if (ph->version <=PERF_HEADER_VERSION_2 ) { +list_for_each_entry (evsel, &evlist->entries, node) +perf_evlist__set_event_name (session->evlist, evsel); + +perf_evlist__delete (evlist); + }else { + session->evlist = session->evlist ?: evlist; + symbol_conf.nr_events = evlist->nr_entries; + } -perf_evlist__delete (evlist);return 0 ; }
@@ -2859,15 +2868,38 @@ static int perf_session__read_header_v2(struct perf_session *session,return __perf_session__read_header_v2 (session, header); } +static int perf_session__read_header_v3 (struct perf_session *session, +struct perf_file_header *header) +{ +struct perf_header *ph = &session->header; +struct perf_file_header_v3 *v3 = &header->v3; + +memcpy (&ph->adds_features, &v3->adds_features, +sizeof (ph->adds_features)); + + ph->data_offset = v3->data.offset; + ph->data_size = v3->data.size; + ph->feat_offset = v3->features.offset; + +perf_header__process_sections (ph, session->fd, &session->pevent, + perf_file_section__process); + +return 0 ; +} +static int perf_header_read_file (struct perf_session *session) {struct perf_file_header header; +struct perf_header *ph = &session->header;if (perf_file_header__read (&header, &session->header, session->fd))return -1 ; -/* read v2 specific data */ -return perf_session__read_header_v2 (session, &header); +/* read version specific data */ +if (ph->version <=PERF_HEADER_VERSION_2 ) +return perf_session__read_header_v2 (session, &header); + +return perf_session__read_header_v3 (session, &header); }int perf_session__read_header (struct perf_session *session)
Signature · 1 line
1.7.11.7
PATCH v2 17/25 perf tools: Add perf.data version 3 header write
In reply to Jiri OlsaAdding perf data version 3 header write code and switch perf tool storing to version 3.
--- tools/perf/Documentation/perf-data-file-v2.txt | 5 +- tools/perf/builtin-inject.c | 3 +- tools/perf/builtin-record.c | 5 +- tools/perf/tests/session-simple.c | 4 +- tools/perf/util/header.c | 70 ++++++++------------------ tools/perf/util/header.h | 6 ++- 6 files changed, 36 insertions(+), 57 deletions(-)
tools/perf/Documentation/perf-data-file-v2.txt +3 -2
@@ -8,8 +8,9 @@ perf-data-file-v2 - The perf tool file format version 2 DESCRIPTION ----------- -Following text describes version 2 of the perf data file format, -which is version that is currently used by perf tool. +Following text describes version 2 of the perf data file format. +The perf tool supports this format for reading, but for writing +it uses version 3 format perf-data-file-v3(1). The perf data file format is composed of several sections describing monitored events and the data itself.
tools/perf/builtin-inject.c +2 -1
@@ -376,7 +376,8 @@ static int __cmd_inject(struct perf_inject *inject)if (!inject->pipe_output) { session->header.data_size = inject->bytes_written; -perf_session__write_header (session, session->evlist, inject->output, true); +perf_session__write_header (session, session->evlist, + inject->output); }perf_session__delete (session);
tools/perf/builtin-record.c +2 -3
@@ -273,7 +273,7 @@ static void perf_record__exit(int status, void *arg)if (!rec->no_buildid)process_buildids (rec);perf_session__write_header (rec->session, rec->evlist, - rec->output, true); + rec->output);perf_session__delete (rec->session);perf_evlist__delete (rec->evlist);symbol__exit ();
@@ -444,8 +444,7 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv)if (err <0 ) goto out_delete_session; }else { - err =perf_session__write_header (session, evsel_list, - output, false); + err =perf_session__prepare_header (output);if (err <0 ) goto out_delete_session; }
tools/perf/tests/session-simple.c +2 -2
@@ -480,7 +480,7 @@ static int session_write(char *file)perf_header__clear_feat (&session->header,HEADER_BRANCH_STACK );TEST_ASSERT_VAL ("failed to write header" , - !perf_session__write_header (session, evlist, fd, false)); + !perf_session__prepare_header (fd));#define STORE_EVENTS (str, func, cnt) \ do { \
@@ -520,7 +520,7 @@ do { \ session->header.data_size += size;TEST_ASSERT_VAL ("failed to write header" , - !perf_session__write_header (session, evlist, fd, true)); + !perf_session__write_header (session, evlist, fd));perf_session__delete (session);perf_evlist__delete (evlist);
tools/perf/util/header.c +22 -48
120 lines not on this page · raw message
tools/perf/util/header.h +5 -1
@@ -53,6 +53,8 @@ struct perf_file_header_v2 {DECLARE_BITMAP (adds_features,HEADER_FEAT_BITS ); }; +#define PERF_FILE_HEADER__DATA_OFFSET (sizeof(struct perf_file_header)) +struct perf_file_header_v3 {struct perf_file_section data;struct perf_file_section features;
@@ -111,6 +113,7 @@ struct perf_header {u64 data_offset;u64 data_size;u64 feat_offset; +u64 feat_size;DECLARE_BITMAP (adds_features,HEADER_FEAT_BITS );struct perf_session_env env; };
@@ -119,9 +122,10 @@ struct perf_evlist;struct perf_session ;int perf_session__read_header (struct perf_session *session); +int perf_session__prepare_header (int fd);int perf_session__write_header (struct perf_session *session,struct perf_evlist *evlist, -int fd,bool at_exit); +int fd);int perf_header__write_pipe (int fd);void perf_header__set_feat (struct perf_header *header,int feat);
Signature · 1 line
1.7.11.7
PATCH v2 18/25 perf tools: Get rid of post_processing_offset in record command
In reply to Jiri OlsaNo need to use post_processing_offset variable now when we know where the data starts.
--- tools/perf/builtin-record.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
tools/perf/builtin-record.c +6 -7
@@ -75,7 +75,6 @@ struct perf_record {bool no_buildid;bool no_buildid_cache;long samples; -off_t post_processing_offset; };static void advance_output (struct perf_record *rec,size_t size)
@@ -249,15 +248,17 @@ out:static int process_buildids (struct perf_record *rec) { -u64 size =lseek (rec->output,0 ,SEEK_CUR ); +struct perf_session *session = rec->session; +u64 data_offset =PERF_FILE_HEADER__DATA_OFFSET ; +u64 size = session->header.data_size;if (size ==0 )return 0 ; rec->session->fd = rec->output; -return __perf_session__process_events (rec->session, rec->post_processing_offset, - size - rec->post_processing_offset, - size, &build_id__mark_dso_hit_ops); +return __perf_session__process_events (session, data_offset, + size - data_offset, size, + &build_id__mark_dso_hit_ops); }static void perf_record__exit (int status,void *arg)
@@ -457,8 +458,6 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv) goto out_delete_session; } - rec->post_processing_offset =lseek (output,0 ,SEEK_CUR ); - machine = &session->machines.host;if (opts->pipe_output) {
Signature · 1 line
1.7.11.7
PATCH v2 19/25 perf tools: Move synthesizing into single function
In reply to Jiri OlsaMoving synthesizing into single function, so it could be reused.
--- tools/perf/builtin-annotate.c | 9 +- tools/perf/builtin-buildid-cache.c | 8 +- tools/perf/builtin-buildid-list.c | 9 +- tools/perf/builtin-diff.c | 19 ++-- tools/perf/builtin-evlist.c | 7 +- tools/perf/builtin-inject.c | 7 +- tools/perf/builtin-kmem.c | 7 +- tools/perf/builtin-kvm.c | 13 ++- tools/perf/builtin-lock.c | 8 +- tools/perf/builtin-mem.c | 9 +- tools/perf/builtin-record.c | 204 +++++++++++++++++++++---------------- tools/perf/builtin-report.c | 10 +- tools/perf/builtin-sched.c | 6 +- tools/perf/builtin-script.c | 15 ++- tools/perf/builtin-timechart.c | 10 +- tools/perf/builtin-top.c | 6 +- tools/perf/builtin-trace.c | 8 +- tools/perf/perf.h | 1 - tools/perf/tests/session-simple.c | 30 ++++-- tools/perf/util/data.h | 29 ++++++ tools/perf/util/session.c | 12 +-- tools/perf/util/session.h | 6 +- 22 files changed, 290 insertions(+), 143 deletions(-) create mode 100644 tools/perf/util/data.h
tools/perf/builtin-annotate.c +7 -2
@@ -28,6 +28,7 @@#include "util/hist.h" #include "util/session.h" #include "util/tool.h" +#include "util/data.h" #include "arch/common.h" #include <linux/bitmap.h>
@@ -188,9 +189,13 @@ static int __cmd_annotate(struct perf_annotate *ann)struct perf_session *session;struct perf_evsel *pos;u64 total_nr_samples; +struct perf_data_file file = { + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = ann->force, + }; - session =perf_session__new (input_name,O_RDONLY , - ann->force, false, &ann->tool); + session =perf_session__new (&file, false, &ann->tool);if (session ==NULL )return -ENOMEM ;
tools/perf/builtin-buildid-cache.c +6 -2
@@ -82,8 +82,12 @@ static bool dso__missing_buildid_cache(struct dso *dso, int parm __maybe_unused)static int build_id_cache__fprintf_missing (const char *filename,bool force,FILE *fp) { -struct perf_session *session =perf_session__new (filename,O_RDONLY , - force, false,NULL ); +struct perf_data_file file = { + .path = filename, + .mode =PERF_DATA_MODE_READ , + .force = force, + }; +struct perf_session *session =perf_session__new (&file, false,NULL );if (session ==NULL )return -1 ;
tools/perf/builtin-buildid-list.c +7 -2
@@ -15,6 +15,7 @@#include "util/parse-options.h" #include "util/session.h" #include "util/symbol.h" +#include "util/data.h" static int sysfs__fprintf_build_id (FILE *fp) {
@@ -52,6 +53,11 @@ static bool dso__skip_buildid(struct dso *dso, int with_hits)static int perf_session__list_build_ids (bool force,bool with_hits) {struct perf_session *session; +struct perf_data_file file = { + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = force, + };symbol__elf_init ();/*
@@ -60,8 +66,7 @@ static int perf_session__list_build_ids(bool force, bool with_hits)if (filename__fprintf_build_id (input_name, stdout)) goto out; - session =perf_session__new (input_name,O_RDONLY , force, false, - &build_id__mark_dso_hit_ops); + session =perf_session__new (&file, false, &build_id__mark_dso_hit_ops);if (session ==NULL )return -1 ;/*
tools/perf/builtin-diff.c +12 -7
@@ -16,6 +16,7 @@#include "util/sort.h" #include "util/symbol.h" #include "util/util.h" +#include "util/data.h" #include <stdlib.h> #include <math.h>
@@ -42,7 +43,7 @@ struct diff_hpp_fmt {struct data__file {struct perf_session *session; -const char *file; +struct perf_data_file file;int idx;struct hists *hists;struct diff_hpp_fmt fmt[PERF_HPP_DIFF__MAX_INDEX ];
@@ -599,7 +600,7 @@ static void data__fprintf(void)data__for_each_file (i, d)fprintf (stdout,"# [%d] %s %s\n" , - d->idx, d->file, + d->idx, d->file.path, !d->idx ?"(Baseline)" :"" );fprintf (stdout,"#\n" );
@@ -661,17 +662,16 @@ static int __cmd_diff(void)int ret = -EINVAL , i;data__for_each_file (i, d) { - d->session =perf_session__new (d->file,O_RDONLY , force, - false, &tool); + d->session =perf_session__new (&d->file, false, &tool);if (!d->session) { -pr_err ("Failed to open %s\n" , d->file); +pr_err ("Failed to open %s\n" , d->file.path); ret = -ENOMEM ; goto out_delete; } ret =perf_session__process_events (d->session, &tool);if (ret) { -pr_err ("Failed to process %s\n" , d->file); +pr_err ("Failed to process %s\n" , d->file.path); goto out_delete; }
@@ -1014,7 +1014,12 @@ static int data_init(int argc, const char **argv)return -ENOMEM ;data__for_each_file (i, d) { - d->file = use_default ? defaults[i] : argv[i]; +struct perf_data_file *file = &d->file; + + file->path = use_default ? defaults[i] : argv[i]; + file->mode =PERF_DATA_MODE_READ , + file->force = force, + d->idx = i; }
tools/perf/builtin-evlist.c +6 -1
@@ -14,13 +14,18 @@#include "util/parse-events.h" #include "util/parse-options.h" #include "util/session.h" +#include "util/data.h" static int __cmd_evlist (const char *file_name,struct perf_attr_details *details) {struct perf_session *session;struct perf_evsel *pos; +struct perf_data_file file = { + .path = file_name, + .mode =PERF_DATA_MODE_READ , + }; - session =perf_session__new (file_name,O_RDONLY ,0 , false,NULL ); + session =perf_session__new (&file,0 ,NULL );if (session ==NULL )return -ENOMEM ;
tools/perf/builtin-inject.c +6 -1
@@ -15,6 +15,7 @@#include "util/tool.h" #include "util/debug.h" #include "util/build-id.h" +#include "util/data.h" #include "util/parse-options.h"
@@ -334,6 +335,10 @@ static int __cmd_inject(struct perf_inject *inject) {struct perf_session *session;int ret = -EINVAL ; +struct perf_data_file file = { + .path = inject->input_name, + .mode =PERF_DATA_MODE_READ , + };signal (SIGINT , sig_handler);
@@ -343,7 +348,7 @@ static int __cmd_inject(struct perf_inject *inject) inject->tool.tracing_data = perf_event__repipe_tracing_data; } - session =perf_session__new (inject->input_name,O_RDONLY , false, true, &inject->tool); + session =perf_session__new (&file, true, &inject->tool);if (session ==NULL )return -ENOMEM ;
tools/perf/builtin-kmem.c +6 -1
@@ -13,6 +13,7 @@#include "util/parse-options.h" #include "util/trace-event.h" +#include "util/data.h" #include "util/debug.h"
@@ -486,8 +487,12 @@ static int __cmd_kmem(void) { "kmem:kfree", perf_evsel__process_free_event, }, { "kmem:kmem_cache_free", perf_evsel__process_free_event, }, }; +struct perf_data_file file = { + .path = input_name, + .mode =PERF_DATA_MODE_READ , + }; - session =perf_session__new (input_name,O_RDONLY ,0 , false, &perf_kmem); + session =perf_session__new (&file, false, &perf_kmem);if (session ==NULL )return -ENOMEM ;
tools/perf/builtin-kvm.c +10 -3
@@ -17,6 +17,7 @@#include "util/tool.h" #include "util/stat.h" #include "util/top.h" +#include "util/data.h" #include <sys/prctl.h> #include <sys/timerfd.h>
@@ -1215,10 +1216,13 @@ static int read_events(struct perf_kvm_stat *kvm) .comm = perf_event__process_comm, .ordered_samples = true, }; +struct perf_data_file file = { + .path = input_name, + .mode =PERF_DATA_MODE_READ , + }; kvm->tool = eops; - kvm->session =perf_session__new (kvm->file_name,O_RDONLY ,0 , false, - &kvm->tool); + kvm->session =perf_session__new (&file, false, &kvm->tool);if (!kvm->session) {pr_err ("Initializing perf session failed\n" );return -EINVAL ;
@@ -1450,6 +1454,9 @@ static int kvm_events_live(struct perf_kvm_stat *kvm,"perf kvm stat live [<options>]" , NULL }; +struct perf_data_file file = { + .mode =PERF_DATA_MODE_WRITE , + };/* event handling */
@@ -1514,7 +1521,7 @@ static int kvm_events_live(struct perf_kvm_stat *kvm,/* * perf session */ - kvm->session =perf_session__new (NULL ,O_WRONLY , false, false, &kvm->tool); + kvm->session =perf_session__new (&file, false, &kvm->tool);if (kvm->session ==NULL ) { err = -ENOMEM ; goto out;
tools/perf/builtin-lock.c +7 -1
@@ -15,6 +15,7 @@#include "util/debug.h" #include "util/session.h" #include "util/tool.h" +#include "util/data.h" #include <sys/types.h> #include <sys/prctl.h>
@@ -836,7 +837,12 @@ static int read_events(void) .comm = perf_event__process_comm, .ordered_samples = true, }; - session =perf_session__new (input_name,O_RDONLY ,0 , false, &eops); +struct perf_data_file file = { + .path = input_name, + .mode =PERF_DATA_MODE_READ , + }; + + session =perf_session__new (&file, false, &eops);if (!session) {pr_err ("Initializing perf session failed\n" );return -1 ;
tools/perf/builtin-mem.c +7 -2
@@ -5,6 +5,7 @@#include "util/trace-event.h" #include "util/tool.h" #include "util/session.h" +#include "util/data.h" #define MEM_OPERATION_LOAD "load"#define MEM_OPERATION_STORE "store"
@@ -119,10 +120,14 @@ static int process_sample_event(struct perf_tool *tool,static int report_raw_events (struct perf_mem *mem) { +struct perf_data_file file = { + .path = input_name, + .mode =PERF_DATA_MODE_READ , + };int err = -EINVAL ;int ret; -struct perf_session *session =perf_session__new (input_name,O_RDONLY , -0 , false, &mem->tool); +struct perf_session *session =perf_session__new (&file, false, + &mem->tool);if (session ==NULL )return -ENOMEM ;
tools/perf/builtin-record.c +116 -88
323 lines not on this page · raw message
tools/perf/builtin-report.c +8 -2
@@ -33,6 +33,7 @@#include "util/thread.h" #include "util/sort.h" #include "util/hist.h" +#include "util/data.h" #include "arch/common.h" #include <linux/bitmap.h>
@@ -843,6 +844,9 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused) "Don't show entries under that percent", parse_percent_limit), OPT_END() }; +struct perf_data_file file = { + .mode =PERF_DATA_MODE_READ , + };perf_config (perf_report_config, &report);
@@ -872,9 +876,11 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)perf_hpp__init (); } + file.path = input_name; + file.force = report.force; + repeat: - session =perf_session__new (input_name,O_RDONLY , - report.force, false, &report.tool); + session =perf_session__new (&file, false, &report.tool);if (session ==NULL )return -ENOMEM ;
tools/perf/builtin-sched.c +5 -1
@@ -1446,8 +1446,12 @@ static int perf_sched__read_events(struct perf_sched *sched, { "sched:sched_migrate_task", process_sched_migrate_task_event, }, };struct perf_session *session; +struct perf_data_file file = { + .path = input_name, + .mode =PERF_DATA_MODE_READ , + }; - session =perf_session__new (input_name,O_RDONLY ,0 , false, &sched->tool); + session =perf_session__new (&file, false, &sched->tool);if (session ==NULL ) {pr_debug ("No Memory for session\n" );return -1 ;
tools/perf/builtin-script.c +12 -3
@@ -15,6 +15,7 @@#include "util/evlist.h" #include "util/evsel.h" #include "util/sort.h" +#include "util/data.h" #include <linux/bitmap.h> static char const *script_name;
@@ -1114,10 +1115,14 @@ int find_scripts(char **scripts_array, char **scripts_path_array)char scripts_path[MAXPATHLEN ], lang_path[MAXPATHLEN ];DIR *scripts_dir, *lang_dir;struct perf_session *session; +struct perf_data_file file = { + .path = input_name, + .mode =PERF_DATA_MODE_READ , + };char *temp;int i =0 ; - session =perf_session__new (input_name,O_RDONLY ,0 , false,NULL ); + session =perf_session__new (&file, false,NULL );if (!session)return -1 ;
@@ -1318,12 +1323,17 @@ int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)"perf script [<options>] <top-script> [script-args]" , NULL }; +struct perf_data_file file = { + .mode =PERF_DATA_MODE_READ , + };setup_scripting (); argc =parse_options (argc, argv, options, script_usage,PARSE_OPT_STOP_AT_NON_OPTION ); + file.path = input_name; +if (argc >1 && !strncmp (argv[0 ],"rec" ,strlen ("rec" ))) { rec_script_path =get_script_path (argv[1 ],RECORD_SUFFIX );if (!rec_script_path)
@@ -1487,8 +1497,7 @@ int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)if (!script_name)setup_pager (); - session =perf_session__new (input_name,O_RDONLY ,0 , false, - &perf_script); + session =perf_session__new (&file, false, &perf_script);if (session ==NULL )return -ENOMEM ;
tools/perf/builtin-timechart.c +8 -2
@@ -36,6 +36,7 @@#include "util/session.h" #include "util/svghelper.h" #include "util/tool.h" +#include "util/data.h" #define SUPPORT_OLD_POWER_EVENTS 1#define PWR_EVENT_EXIT -1
@@ -990,8 +991,13 @@ static int __cmd_timechart(const char *output_name) { "power:power_frequency", process_sample_power_frequency }, #endif }; - struct perf_session *session = perf_session__new(input_name, O_RDONLY, - 0, false, &perf_timechart); + struct perf_data_file file = { + .path = input_name, + .mode = PERF_DATA_MODE_READ, + }; + + struct perf_session *session = perf_session__new(&file, false, + &perf_timechart); int ret = -EINVAL; if (session == NULL)
tools/perf/builtin-top.c +5 -1
@@ -930,11 +930,15 @@ static int __cmd_top(struct perf_top *top)struct perf_record_opts *opts = &top->record_opts;pthread_t thread;int ret; +struct perf_data_file file = { + .mode =PERF_DATA_MODE_WRITE , + }; +/* * FIXME: perf_session__new should allow passing a O_MMAP, so that all this * mmap reading, etc is encapsulated in it. Use O_WRONLY for now. */ - top->session =perf_session__new (NULL ,O_WRONLY , false, false,NULL ); + top->session =perf_session__new (&file, false,NULL );if (top->session ==NULL )return -ENOMEM ;
tools/perf/builtin-trace.c +5 -3
@@ -873,7 +873,10 @@ static int trace__replay(struct trace *trace) { "raw_syscalls:sys_enter", trace__sys_enter, }, { "raw_syscalls:sys_exit", trace__sys_exit, }, }; - +struct perf_data_file file = { + .path = input_name, + .mode =PERF_DATA_MODE_READ , + };struct perf_session *session;int err =-1 ;
@@ -895,8 +898,7 @@ static int trace__replay(struct trace *trace)if (symbol__init () <0 )return -1 ; - session =perf_session__new (input_name,O_RDONLY ,0 , false, - &trace->tool); + session =perf_session__new (&file, false, &trace->tool);if (session ==NULL )return -ENOMEM ;
tools/perf/perf.h +0 -1
@@ -218,7 +218,6 @@ struct perf_record_opts {bool no_delay;bool no_inherit;bool no_samples; -bool pipe_output;bool raw_samples;bool sample_address;bool sample_weight;
tools/perf/tests/session-simple.c +19 -11
@@ -11,6 +11,7 @@#include "header.h" #include "util.h" #include "evlist.h" +#include "data.h" #define EVENTS_MMAP 5#define EVENTS_LOST 6
@@ -449,15 +450,19 @@ static int store_event(int fd, union perf_event *event, size_t *size)return write (fd, event, event->header.size) >0 ?0 :-1 ; } -static int session_write (char *file) +static int session_write (char *path) { +struct perf_data_file file = { + .mode =PERF_DATA_MODE_WRITE , + .path = path, + };struct perf_session *session;struct perf_evlist *evlist;size_t size =0 ; -int feat, fd; +int feat; - fd =open (file,O_RDWR ); -TEST_ASSERT_VAL ("failed to open data file" , fd >=0 ); + file.fd =open (file.path,O_RDWR ); +TEST_ASSERT_VAL ("failed to open data file" , file.fd >=0 ); evlist =perf_evlist__new_default ();TEST_ASSERT_VAL ("failed to get evlist" , evlist);
@@ -466,11 +471,10 @@ static int session_write(char *file)pr_debug ("session writing start\n" ); - session =perf_session__new (file,O_WRONLY , true, false,NULL ); + session =perf_session__new (&file, false,NULL );TEST_ASSERT_VAL ("failed to create session" , session); session->evlist = evlist; - session->fd = fd;for (feat =HEADER_FIRST_FEATURE ; feat <HEADER_LAST_FEATURE ; feat++)perf_header__set_feat (&session->header, feat);
@@ -480,14 +484,14 @@ static int session_write(char *file)perf_header__clear_feat (&session->header,HEADER_BRANCH_STACK );TEST_ASSERT_VAL ("failed to write header" , - !perf_session__prepare_header (fd)); + !perf_session__prepare_header (file.fd));#define STORE_EVENTS (str, func, cnt) \ do { \ int i; \ for (i = 0; i < cnt; i++) { \ TEST_ASSERT_VAL(str, \ - !store_event(fd, func(), &size)); \ + !store_event(file.fd, func(), &size)); \ } \ } while (0)
@@ -520,7 +524,7 @@ do { \ session->header.data_size += size;TEST_ASSERT_VAL ("failed to write header" , - !perf_session__write_header (session, evlist, fd)); + !perf_session__write_header (session, evlist, file.fd));perf_session__delete (session);perf_evlist__delete (evlist);
@@ -529,7 +533,7 @@ do { \return 0 ; } -static int __session_read (char *file) +static int __session_read (char *path) {struct perf_session *session;struct perf_tool tool = {
@@ -542,10 +546,14 @@ static int __session_read(char *file) .unthrottle = process_unthrottle, .sample = process_sample, }; +struct perf_data_file file = { + .mode =PERF_DATA_MODE_READ , + .path = path, + };pr_debug ("session reading start\n" ); - session =perf_session__new (file,O_RDONLY , false, false, &tool); + session =perf_session__new (&file, false, &tool);TEST_ASSERT_VAL ("failed to create session" , session); TEST_ASSERT_VAL("failed to process events",
tools/perf/util/data.h new file +29 -0
@@ -0,0 +1,29 @@ +#ifndef __PERF_DATA_H +#define __PERF_DATA_H + +#include <stdbool.h> + +enum perf_data_mode { +PERF_DATA_MODE_WRITE , +PERF_DATA_MODE_READ , +}; + +struct perf_data_file { +const char *path; +int fd; +bool is_pipe; +bool force; +enum perf_data_mode mode; +}; + +static inline bool perf_data_file__is_read (struct perf_data_file *file) +{ +return file->mode ==PERF_DATA_MODE_READ ; +} + +static inline bool perf_data_file__is_write (struct perf_data_file *file) +{ +return file->mode ==PERF_DATA_MODE_WRITE ; +} + +#endif /* __PERF_DATA_H */
tools/perf/util/session.c +6 -6
@@ -106,11 +106,11 @@ static void perf_session__destroy_kernel_maps(struct perf_session *self)machines__destroy_kernel_maps (&self->machines); } -struct perf_session *perf_session__new (const char *filename,int mode, -bool force,bool repipe, -struct perf_tool *tool) +struct perf_session *perf_session__new (struct perf_data_file *file, +bool repipe,struct perf_tool *tool) {struct perf_session *self; +const char *filename = file->path;struct stat st;size_t len;
@@ -134,11 +134,11 @@ struct perf_session *perf_session__new(const char *filename, int mode,INIT_LIST_HEAD (&self->ordered_samples.to_free);machines__init (&self->machines); -if (mode ==O_RDONLY ) { -if (perf_session__open (self, force) <0 ) +if (perf_data_file__is_read (file)) { +if (perf_session__open (self, file->force) <0 ) goto out_delete;perf_session__set_id_hdr_size (self); - }else if (mode ==O_WRONLY ) { + }else if (perf_data_file__is_write (file)) {/* * In O_RDONLY mode this will be performed when reading the * kernel MMAP event, in perf_event__process_mmap().
tools/perf/util/session.h +3 -3
@@ -7,6 +7,7 @@#include "machine.h" #include "symbol.h" #include "thread.h" +#include "data.h" #include <linux/rbtree.h> #include <linux/perf_event.h>
@@ -49,9 +50,8 @@ struct perf_session {struct perf_tool ; -struct perf_session *perf_session__new (const char *filename,int mode, -bool force,bool repipe, -struct perf_tool *tool); +struct perf_session *perf_session__new (struct perf_data_file *file, +bool repipe,struct perf_tool *tool);void perf_session__delete (struct perf_session *session);void perf_event_header__bswap (struct perf_event_header *self);
Signature · 1 line
1.7.11.7
PATCH v2 20/25 perf tools: Add perf_data_file__open interface to data object
In reply to Jiri OlsaAdding perf_data_file__open interface to data object to open the perf.data file for both read and write.
--- tools/perf/Makefile | 1 + tools/perf/builtin-record.c | 34 +---------- tools/perf/builtin-top.c | 9 +-- tools/perf/tests/session-simple.c | 3 - tools/perf/util/data.c | 120 ++++++++++++++++++++++++++++++++++++++ tools/perf/util/data.h | 4 ++ tools/perf/util/session.c | 95 ++++++++++-------------------- tools/perf/util/session.h | 2 +- 8 files changed, 158 insertions(+), 110 deletions(-) create mode 100644 tools/perf/util/data.c
tools/perf/Makefile +1 -0
@@ -363,6 +363,7 @@ LIB_OBJS += $(OUTPUT)util/intlist.o LIB_OBJS += $(OUTPUT)util/vdso.o LIB_OBJS += $(OUTPUT)util/stat.o LIB_OBJS += $(OUTPUT)util/record.o +LIB_OBJS += $(OUTPUT)util/data.o LIB_OBJS += $(OUTPUT)ui/setup.o LIB_OBJS += $(OUTPUT)ui/helpline.o
tools/perf/builtin-record.c +1 -33
@@ -433,15 +433,12 @@ out:static int __cmd_record (struct perf_record *rec,int argc,const char **argv) { -struct stat st; -int flags;int err, feat;unsigned long waking =0 ;const bool forks = argc >0 ;struct perf_record_opts *opts = &rec->opts;struct perf_evlist *evsel_list = rec->evlist;struct perf_data_file *file = rec->file; -const char *output_name = file->path;struct perf_session *session;bool disabled = false;
@@ -455,35 +452,6 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv)signal (SIGUSR1 , sig_handler);signal (SIGTERM , sig_handler); -if (!output_name) { -if (!fstat (STDOUT_FILENO , &st) &&S_ISFIFO (st.st_mode)) - file->is_pipe = true; -else - file->path = output_name ="perf.data" ; - } -if (output_name) { -if (!strcmp (output_name,"-" )) - file->is_pipe = true; -else if (!stat (output_name, &st) && st.st_size) { -char oldname[PATH_MAX ]; -snprintf (oldname,sizeof (oldname),"%s.old" , - output_name); -unlink (oldname); -rename (output_name, oldname); - } - } - - flags =O_CREAT |O_RDWR |O_TRUNC ; - -if (file->is_pipe) - file->fd =STDOUT_FILENO ; -else - file->fd =open (output_name, flags,S_IRUSR |S_IWUSR ); -if (file->fd <0 ) { -perror ("failed to create output file" ); -return -1 ; - } - session =perf_session__new (file, false,NULL );if (session ==NULL ) {pr_err ("Not enough memory for reading perf file header\n" );
@@ -611,7 +579,7 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv)fprintf (stderr,"[ perf record: Captured and wrote %.3f MB %s (~%" PRIu64" samples) ]\n" , (double )rec->bytes_written /1024.0 /1024.0 , - output_name, + file->path, rec->bytes_written /24 );return 0 ;
tools/perf/builtin-top.c +1 -8
@@ -930,15 +930,8 @@ static int __cmd_top(struct perf_top *top)struct perf_record_opts *opts = &top->record_opts;pthread_t thread;int ret; -struct perf_data_file file = { - .mode =PERF_DATA_MODE_WRITE , - }; -/* -* FIXME: perf_session__new should allow passing a O_MMAP, so that all this -* mmap reading, etc is encapsulated in it. Use O_WRONLY for now. -*/ - top->session =perf_session__new (&file, false,NULL ); + top->session =perf_session__new (NULL , false,NULL );if (top->session ==NULL )return -ENOMEM ;
tools/perf/tests/session-simple.c +0 -3
@@ -461,9 +461,6 @@ static int session_write(char *path)size_t size =0 ;int feat; - file.fd =open (file.path,O_RDWR ); -TEST_ASSERT_VAL ("failed to open data file" , file.fd >=0 ); - evlist =perf_evlist__new_default ();TEST_ASSERT_VAL ("failed to get evlist" , evlist);
tools/perf/util/data.c new file +120 -0
120 lines not on this page · raw message
tools/perf/util/data.h +4 -0
@@ -13,6 +13,7 @@ struct perf_data_file {int fd;bool is_pipe;bool force; +unsigned long size;enum perf_data_mode mode; };
@@ -26,4 +27,7 @@ static inline bool perf_data_file__is_write(struct perf_data_file *file) return file->mode == PERF_DATA_MODE_WRITE; } +int perf_data_file__open (struct perf_data_file *file); +void perf_data_file__close (struct perf_data_file *file); +#endif /* __PERF_DATA_H */
tools/perf/util/session.c +30 -65
148 lines not on this page · raw message
tools/perf/util/session.h +1 -1
@@ -39,7 +39,7 @@ struct perf_session {bool fd_pipe;bool repipe;struct ordered_samples ordered_samples; -char filename[1 ]; +const char *filename; };#define PRINT_IP_OPT_IP (1<<0)
Signature · 1 line
1.7.11.7
PATCH v2 21/25 perf tools: Separating data file properties from session
In reply to Jiri OlsaRemoving 'fd, fd_pipe, filename, size' from struct perf_session and replacing it with struct perf_data_file.
--- tools/perf/builtin-annotate.c | 2 +- tools/perf/builtin-buildid-list.c | 2 +- tools/perf/builtin-record.c | 2 -- tools/perf/builtin-report.c | 8 +++++--- tools/perf/builtin-script.c | 2 +- tools/perf/util/data.h | 15 +++++++++++++++ tools/perf/util/header.c | 30 ++++++++++++++++++++---------- tools/perf/util/header.h | 1 + tools/perf/util/session.c | 36 +++++++++++++++++++----------------- tools/perf/util/session.h | 5 +---- 10 files changed, 64 insertions(+), 39 deletions(-)
tools/perf/builtin-annotate.c +1 -1
@@ -248,7 +248,7 @@ static int __cmd_annotate(struct perf_annotate *ann) }if (total_nr_samples ==0 ) { -ui__error ("The %s file has no samples!\n" , session->filename); +ui__error ("The %s file has no samples!\n" , file.path); goto out_delete; }
tools/perf/builtin-buildid-list.c +1 -1
@@ -73,7 +73,7 @@ static int perf_session__list_build_ids(bool force, bool with_hits)* in pipe-mode, the only way to get the buildids is to parse * the record stream. Buildids are stored as RECORD_HEADER_BUILD_ID */ -if (with_hits || session->fd_pipe) +if (with_hits ||perf_data_file__is_pipe (&file))perf_session__process_events (session, &build_id__mark_dso_hit_ops);perf_session__fprintf_dsos_buildid (session, stdout, dso__skip_buildid, with_hits);
tools/perf/builtin-record.c +0 -2
@@ -251,7 +251,6 @@ out:static int process_buildids (struct perf_record *rec) { -struct perf_data_file *file = rec->file;struct perf_session *session = rec->session;u64 data_offset =PERF_FILE_HEADER__DATA_OFFSET ;u64 size = session->header.data_size;
@@ -259,7 +258,6 @@ static int process_buildids(struct perf_record *rec)if (size ==0 )return 0 ; - rec->session->fd = file->fd;return __perf_session__process_events (session, data_offset, size - data_offset, size, &build_id__mark_dso_hit_ops);
tools/perf/builtin-report.c +5 -3
@@ -367,8 +367,9 @@ static int perf_report__setup_sample_type(struct perf_report *rep) {struct perf_session *self = rep->session;u64 sample_type =perf_evlist__combined_sample_type (self->evlist); +bool is_pipe =perf_data_file__is_pipe (self->file); -if (!self->fd_pipe && !(sample_type &PERF_SAMPLE_CALLCHAIN )) { +if (!is_pipe && !(sample_type &PERF_SAMPLE_CALLCHAIN )) {if (sort__has_parent) {ui__error ("Selected --sort parent, but no " "callchain data. Did you call "
@@ -391,7 +392,7 @@ static int perf_report__setup_sample_type(struct perf_report *rep) }if (sort__mode ==SORT_MODE__BRANCH ) { -if (!self->fd_pipe && +if (!is_pipe && !(sample_type &PERF_SAMPLE_BRANCH_STACK )) {ui__error ("Selected -b but no branch data. " "Did you call perf record without -b?\n" );
@@ -489,6 +490,7 @@ static int __cmd_report(struct perf_report *rep)struct map *kernel_map;struct kmap *kernel_kmap;const char *help ="For a higher level overview, try: perf report --sort comm,dso" ; +struct perf_data_file *file = session->file;signal (SIGINT , sig_handler);
@@ -570,7 +572,7 @@ static int __cmd_report(struct perf_report *rep) }if (nr_samples ==0 ) { -ui__error ("The %s file has no samples!\n" , session->filename); +ui__error ("The %s file has no samples!\n" , file->path);return 0 ; }
tools/perf/builtin-script.c +1 -1
@@ -1524,7 +1524,7 @@ int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)return -1 ; } - input =open (session->filename,O_RDONLY );/* input_name */ + input =open (file.path,O_RDONLY );/* input_name */ if (input <0 ) {perror ("failed to open file" );return -1 ;
tools/perf/util/data.h +15 -0
@@ -27,6 +27,21 @@ static inline bool perf_data_file__is_write(struct perf_data_file *file)return file->mode ==PERF_DATA_MODE_WRITE ; } +static inline int perf_data_file__is_pipe (struct perf_data_file *file) +{ +return file->is_pipe; +} + +static inline int perf_data_file__fd (struct perf_data_file *file) +{ +return file->fd; +} + +static inline unsigned long perf_data_file__size (struct perf_data_file *file) +{ +return file->size; +} +int perf_data_file__open (struct perf_data_file *file);void perf_data_file__close (struct perf_data_file *file);
tools/perf/util/header.c +20 -10
@@ -2175,7 +2175,7 @@ int perf_header__fprintf_info(struct perf_session *session, FILE *fp, bool full) {struct header_print_data hd;struct perf_header *header = &session->header; -int fd = session->fd; +int fd =perf_data_file__fd (session->file); hd.fp = fp; hd.full = full;
@@ -2643,8 +2643,10 @@ static int perf_header__read_pipe(struct perf_session *session) {struct perf_header *header = &session->header;struct perf_pipe_file_header f_header; +struct perf_data_file *file = session->file; -if (perf_file_header__read_pipe (&f_header, header, session->fd, +if (perf_file_header__read_pipe (&f_header, header, +perf_data_file__fd (file), session->repipe) <0 ) {pr_debug ("incompatible file format\n" );return -EINVAL ;
@@ -2746,12 +2748,13 @@ static int perf_evlist__prepare_tracepoint_events(struct perf_evlist *evlist,static int __perf_session__read_header_v2 (struct perf_session *session,struct perf_file_header *header) { +struct perf_data_file *file = session->file;struct perf_header *ph = &session->header;struct perf_file_header_v2 *v2 = &header->v2;struct perf_file_attr f_attr;u64 f_id;int nr_attrs, nr_ids, i, j; -int fd = session->fd; +int fd =perf_data_file__fd (file); session->evlist =perf_evlist__new ();if (session->evlist ==NULL )
@@ -2846,6 +2849,8 @@ static int perf_session__read_header_v3(struct perf_session *session, {struct perf_header *ph = &session->header;struct perf_file_header_v3 *v3 = &header->v3; +struct perf_data_file *file = session->file; +int fd =perf_data_file__fd (file);memcpy (&ph->adds_features, &v3->adds_features,sizeof (ph->adds_features));
@@ -2855,7 +2860,7 @@ static int perf_session__read_header_v3(struct perf_session *session, ph->feat_offset = v3->features.offset; ph->feat_size = v3->features.size; -perf_header__process_sections (ph, session->fd, &session->pevent, +perf_header__process_sections (ph, fd, &session->pevent, perf_file_section__process);return 0 ;
@@ -2865,8 +2870,10 @@ static int perf_header_read_file(struct perf_session *session) {struct perf_file_header header;struct perf_header *ph = &session->header; +struct perf_data_file *file = session->file; -if (perf_file_header__read (&header, &session->header, session->fd)) +if (perf_file_header__read (&header, &session->header, +perf_data_file__fd (file)))return -1 ;/* read version specific data */
@@ -2878,7 +2885,9 @@ static int perf_header_read_file(struct perf_session *session)int perf_session__read_header (struct perf_session *session) { -if (session->fd_pipe) +struct perf_data_file *file = session->file; + +if (perf_data_file__is_pipe (file))return perf_header__read_pipe (session);return perf_header_read_file (session) <0 ? -EINVAL :0 ;
@@ -3028,18 +3037,19 @@ int perf_event__process_tracing_data(struct perf_tool *tool __maybe_unused, struct perf_session *session) {ssize_t size_read, padding, size = event->tracing_data.size; -off_t offset =lseek (session->fd,0 ,SEEK_CUR ); +int fd =perf_data_file__fd (session->file); +off_t offset =lseek (fd,0 ,SEEK_CUR );char buf[BUFSIZ ];/* setup for reading amidst mmap */ -lseek (session->fd, offset +sizeof (struct tracing_data_event ), +lseek (fd, offset +sizeof (struct tracing_data_event ),SEEK_SET ); - size_read =trace_report (session->fd, &session->pevent, + size_read =trace_report (fd, &session->pevent, session->repipe); padding =PERF_ALIGN (size_read,sizeof (u64)) - size_read; -if (readn (session->fd, buf, padding) <0 ) { +if (readn (fd, buf, padding) <0 ) {pr_err ("%s: reading input file" , __func__);return -1 ; }
tools/perf/util/header.h +1 -0
@@ -6,6 +6,7 @@#include <stdbool.h> #include "types.h" #include "event.h" +#include "data.h" #include <linux/bitmap.h>
tools/perf/util/session.c +19 -17
@@ -18,17 +18,16 @@static int perf_session__open (struct perf_session *self) { -if (self->fd_pipe) { -if (perf_session__read_header (self) <0 ) -pr_err ("incompatible file format (rerun with -v to learn more)" ); -return 0 ; - } +struct perf_data_file *file = self->file;if (perf_session__read_header (self) <0 ) {pr_err ("incompatible file format (rerun with -v to learn more)" );return -1 ; } +if (perf_data_file__is_pipe (file)) +return 0 ; +if (!perf_evlist__valid_sample_type (self->evlist)) {pr_err ("non matching sample_type" );return -1 ;
@@ -87,10 +86,7 @@ struct perf_session *perf_session__new(struct perf_data_file *file,if (perf_data_file__open (file)) goto out_delete; - self->fd = file->fd; - self->fd_pipe = file->is_pipe; - self->filename = file->path; - self->size = file->size; + self->file = file;if (perf_data_file__is_read (file)) {if (perf_session__open (self) <0 )
@@ -158,7 +154,8 @@ void perf_session__delete(struct perf_session *self)perf_session__delete_threads (self);perf_session_env__delete (&self->header.env);machines__exit (&self->machines); -close (self->fd); +if (self->file) +perf_data_file__close (self->file);free (self);vdso__exit (); }
@@ -980,6 +977,7 @@ static int perf_session_deliver_event(struct perf_session *session,static int perf_session__process_user_event (struct perf_session *session,union perf_event *event,struct perf_tool *tool,u64 file_offset) { +int fd =perf_data_file__fd (session->file);int err;dump_event (session, event, file_offset,NULL );
@@ -993,7 +991,7 @@ static int perf_session__process_user_event(struct perf_session *session, unionreturn err;case PERF_RECORD_HEADER_TRACING_DATA :/* setup for reading amidst mmap */ -lseek (session->fd, file_offset,SEEK_SET ); +lseek (fd, file_offset,SEEK_SET );return tool->tracing_data (tool, event, session);case PERF_RECORD_HEADER_BUILD_ID :return tool->build_id (tool, event, session);
@@ -1120,6 +1118,7 @@ volatile int session_done;static int __perf_session__process_pipe_events (struct perf_session *self,struct perf_tool *tool) { +int fd =perf_data_file__fd (self->file);union perf_event *event;uint32_t size, cur_size =0 ;void *buf =NULL ;
@@ -1138,7 +1137,7 @@ static int __perf_session__process_pipe_events(struct perf_session *self,return -errno; more: event = buf; - err =readn (self->fd, event,sizeof (struct perf_event_header )); + err =readn (fd, event,sizeof (struct perf_event_header ));if (err <=0 ) {if (err ==0 ) goto done;
@@ -1170,7 +1169,7 @@ more: p +=sizeof (struct perf_event_header );if (size -sizeof (struct perf_event_header )) { - err =readn (self->fd, p, size -sizeof (struct perf_event_header )); + err =readn (fd, p, size -sizeof (struct perf_event_header ));if (err <=0 ) {if (err ==0 ) {pr_err ("unexpected end of event stream\n" );
@@ -1249,6 +1248,7 @@ int __perf_session__process_events(struct perf_session *session, u64 data_offset, u64 data_size, u64 file_size, struct perf_tool *tool) { + int fd = perf_data_file__fd(session->file);u64 head, page_offset, file_offset, file_pos, progress_next;int err, mmap_prot, mmap_flags, map_idx =0 ;size_t mmap_size;
@@ -1281,7 +1281,7 @@ int __perf_session__process_events(struct perf_session *session, mmap_flags =MAP_PRIVATE ; } remap: - buf =mmap (NULL , mmap_size, mmap_prot, mmap_flags, session->fd, + buf =mmap (NULL , mmap_size, mmap_prot, mmap_flags, fd, file_offset);if (buf ==MAP_FAILED ) {pr_err ("failed to mmap file\n" );
@@ -1343,16 +1343,17 @@ out_err:int perf_session__process_events (struct perf_session *self,struct perf_tool *tool) { +u64 size =perf_data_file__size (self->file);int err;if (perf_session__register_idle_thread (self) ==NULL )return -ENOMEM ; -if (!self->fd_pipe) +if (!perf_data_file__is_pipe (self->file)) err =__perf_session__process_events (self, self->header.data_offset, self->header.data_size, - self->size, tool); + size, tool);else err =__perf_session__process_pipe_events (self, tool);
@@ -1575,13 +1576,14 @@ int perf_session__cpu_bitmap(struct perf_session *session,void perf_session__fprintf_info (struct perf_session *session,FILE *fp,bool full) { +int fd =perf_data_file__fd (session->file);struct stat st;int ret;if (session ==NULL || fp ==NULL )return ; - ret =fstat (session->fd, &st); + ret =fstat (fd, &st);if (ret ==-1 )return ;
tools/perf/util/session.h +1 -4
@@ -30,16 +30,13 @@ struct ordered_samples {struct perf_session {struct perf_header header; -unsigned long size;struct machines machines;struct perf_evlist *evlist;struct pevent *pevent;struct events_stats stats; -int fd; -bool fd_pipe;bool repipe;struct ordered_samples ordered_samples; -const char *filename; +struct perf_data_file *file; };#define PRINT_IP_OPT_IP (1<<0)
Signature · 1 line
1.7.11.7
PATCH v2 22/25 perf tests: Add session reading test for little endian perf data v3
In reply to Jiri OlsaAdding session reading test for little endian perf data version 3 included as binary data in the test object.
--- tools/perf/Makefile | 1 + tools/perf/tests/perf.data.v3.le.h | 212 +++++++++++++++++++++++++++++++++++++ tools/perf/tests/session-simple.c | 6 ++ 3 files changed, 219 insertions(+) create mode 100644 tools/perf/tests/perf.data.v3.le.h
tools/perf/Makefile +1 -0
@@ -294,6 +294,7 @@ LIB_H += ui/util.h LIB_H += ui/ui.h LIB_H += tests/perf.data.v2.le.h LIB_H += tests/perf.data.v2.be.h +LIB_H += tests/perf.data.v3.le.h LIB_OBJS += $(OUTPUT)util/abspath.o LIB_OBJS += $(OUTPUT)util/alias.o
tools/perf/tests/perf.data.v3.le.h new file +212 -0
212 lines not on this page · raw message
tools/perf/tests/session-simple.c +6 -0
@@ -30,6 +30,10 @@ char data_file_v2_be[] = {#include "perf.data.v2.be.h" }; +char data_file_v3_le[] = { +#include "perf.data.v3.le.h" +}; +static int events_mmap;static int events_lost;static int events_comm;
@@ -642,5 +646,7 @@ int test__session_simple(void) err |=test_file_data (data_file_v2_le,sizeof (data_file_v2_le));pr_debug ("Testing v2 BE data\n" ); err |=test_file_data (data_file_v2_be,sizeof (data_file_v2_be)); +pr_debug ("Testing v3 LE data\n" ); + err |=test_file_data (data_file_v3_le,sizeof (data_file_v3_le));return err; }
Signature · 1 line
1.7.11.7
PATCH v2 23/25 perf tests: Add session reading test for big endian perf data v3
In reply to Jiri OlsaAdding session reading test for big endian perf data version 3 included as binary data in the test object.
--- tools/perf/Makefile | 1 + tools/perf/tests/perf.data.v3.be.h | 258 +++++++++++++++++++++++++++++++++++++ tools/perf/tests/session-simple.c | 6 + 3 files changed, 265 insertions(+) create mode 100644 tools/perf/tests/perf.data.v3.be.h
tools/perf/Makefile +1 -0
@@ -295,6 +295,7 @@ LIB_H += ui/ui.h LIB_H += tests/perf.data.v2.le.h LIB_H += tests/perf.data.v2.be.h LIB_H += tests/perf.data.v3.le.h +LIB_H += tests/perf.data.v3.be.h LIB_OBJS += $(OUTPUT)util/abspath.o LIB_OBJS += $(OUTPUT)util/alias.o
tools/perf/tests/perf.data.v3.be.h new file +258 -0
258 lines not on this page · raw message
tools/perf/tests/session-simple.c +6 -0
@@ -34,6 +34,10 @@ char data_file_v3_le[] = {#include "perf.data.v3.le.h" }; +char data_file_v3_be[] = { +#include "perf.data.v3.be.h" +}; +static int events_mmap;static int events_lost;static int events_comm;
@@ -648,5 +652,7 @@ int test__session_simple(void) err |=test_file_data (data_file_v2_be,sizeof (data_file_v2_be));pr_debug ("Testing v3 LE data\n" ); err |=test_file_data (data_file_v3_le,sizeof (data_file_v3_le)); +pr_debug ("Testing v3 BE data\n" ); + err |=test_file_data (data_file_v3_be,sizeof (data_file_v3_be));return err; }
Signature · 1 line
1.7.11.7
PATCH v2 24/25 perf tools: Add multi file '-M' option for record command
In reply to Jiri Olsasplit event data into multiple files based on the file size or time delta specified as an argument to the option.
Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold.
The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like:
-M 5M for 5 megabytes threshold
-M 1h for 1 hour threshold
The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be:
perf.data-00000
perf.data-00001
perf.data-00002
...
Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise.
Example:
$ perf record -M 5M yes > /dev/null
^C[ perf record: Woken up 228 times to write data ]
[ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ]
yes: Interrupt
$ ls -l perf.data-0*
-rw------- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-00000
-rw------- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-00001
-rw------- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-00002
-rw------- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-00003
-rw------- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-00004
$ ./perf diff perf.data-0000*
# Event 'cycles'
#
# Data files:
# [0] perf.data-00000 (Baseline)
# [1] perf.data-00001
# [2] perf.data-00002
# [3] perf.data-00003
# [4] perf.data-00004
#
# Baseline/0 Delta/1 Delta/2 Delta/3 Delta/4 Shared Object Symbol
# .......... ....... ....... ....... ....... ................. ..........................................
#
37.70% -0.17% -0.42% -0.24% -0.31% libc-2.15.so [.] _IO_file_xsputn@@GLIBC_2.2.5
30.31% +0.28% +0.22% +0.07% +0.06% yes [.] main
16.73% +0.02% +0.10% -0.03% +0.11% libc-2.15.so [.] __strlen_sse2
14.22% -0.30% -0.10% -0.31% -0.14% libc-2.15.so [.] fputs_unlocked
0.39% -0.01% yes [.] fputs_unlocked@plt
0.06% [kernel.kallsyms] [k] system_call
0.06% +0.01% [kernel.kallsyms] [k] __srcu_read_lock
0.05% +0.01% [kernel.kallsyms] [k] __srcu_read_unlock
...--- tools/perf/Documentation/perf-record.txt | 14 ++ tools/perf/builtin-record.c | 246 ++++++++++++++++++++++++++++--- tools/perf/perf.h | 14 ++ tools/perf/util/evlist.c | 2 +- tools/perf/util/evlist.h | 2 + tools/perf/util/evsel.c | 24 +++ 6 files changed, 280 insertions(+), 22 deletions(-)
tools/perf/Documentation/perf-record.txt +14 -0
@@ -184,6 +184,20 @@ Enable weightened sampling. An additional weight is recorded per sample and can displayed with the weight and local_weight sort keys. This currently works for TSX abort events and some memory events in precise mode on modern Intel CPUs. +-M:: +--multi:: +Store output perf.data into multiple files based on the size threshold. +The threshold is specified either as size (B/K/M/G) or time (s/m/h) +by appending the size with appropriate unit, like: + -M 5M for 5 megabytes threshold + -M 1h for 1 hour threshold +The generated name for each data file is appended with sequential number +(prepended by 5 zeros). For default output file name it will be: + perf.data-00000 + perf.data-00001 + perf.data-00002 + ... + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/builtin-record.c +225 -21
313 lines not on this page · raw message
tools/perf/perf.h +14 -0
@@ -210,6 +210,17 @@ enum perf_call_graph_mode {CALLCHAIN_DWARF }; +enum perf_record_multi_limits { +MULTI_LIMIT__MIN_SIZE =100 *1024 ,/* 100K */ +MULTI_LIMIT__MIN_WATTERMARK =10 *1024 ,/* 10K */ +MULTI_LIMIT__MAX_WATTERMARK =MULTI_LIMIT__MIN_SIZE , +}; + +enum perf_record_multi_type { +MULTI_TYPE__SIZE , +MULTI_TYPE__TIME , +}; +struct perf_record_opts {struct perf_target target;int call_graph;
@@ -223,6 +234,7 @@ struct perf_record_opts {bool sample_weight;bool sample_time;bool period; +bool multi_limit;unsigned int freq;unsigned int mmap_pages;unsigned int user_freq;
@@ -230,6 +242,8 @@ struct perf_record_opts {u64 default_interval;u64 user_interval;u16 stack_dump_size; +u64 multi_value; + int multi_type; };#endif
tools/perf/util/evlist.c +1 -1
@@ -632,7 +632,7 @@ out_unmap:return -1 ; } -static size_t perf_evlist__mmap_size (unsigned long pages) +size_t perf_evlist__mmap_size (unsigned long pages) {/* 512 kiB: default amount of unprivileged mlocked memory */ if (pages ==UINT_MAX )
tools/perf/util/evlist.h +2 -0
@@ -104,6 +104,8 @@ int perf_evlist__prepare_workload(struct perf_evlist *evlist, bool want_signal);int perf_evlist__start_workload (struct perf_evlist *evlist); +size_t perf_evlist__mmap_size (unsigned long pages); +int perf_evlist__parse_mmap_pages (const struct option *opt,const char *str,int unset);
tools/perf/util/evsel.c +24 -0
@@ -672,6 +672,30 @@ void perf_evsel__config(struct perf_evsel *evsel, attr->branch_sample_type = opts->branch_stack; } +if (opts->multi_limit) { +u64 wm =MULTI_LIMIT__MIN_WATTERMARK ; + attr->watermark =1 ; + +if (opts->multi_type ==MULTI_TYPE__SIZE ) { +/* +* The watermark could not get under 10K because +* of the minimal file limit and we are guarded +* with 100K for max wattermark. +*/ + wm = opts->multi_value; + wm =min (wm /10 , (u64 )MULTI_LIMIT__MAX_WATTERMARK ); + +/* +* We also dont want to have watermark close to the size +* of the mmap to ensure data would always cross it and +* we get poll notification. +*/ + wm =min (wm, (u64 )perf_evlist__mmap_size (opts->mmap_pages) -100 ); + } + + attr->wakeup_watermark = wm; + } +if (opts->sample_weight) attr->sample_type |=PERF_SAMPLE_WEIGHT ;
Signature · 1 line
1.7.11.7
Adrian Hunter
reviewerRe: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
Quoted 24 lines · Jiri Olsa
split event data into multiple files based on the file size or time delta specified as an argument to the option.
Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold.
The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like:
-M 5M for 5 megabytes threshold
-M 1h for 1 hour threshold
The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be:
perf.data-00000
perf.data-00001
perf.data-00002
...Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise.
Example:
Quoted 10 lines
$ perf record -M 5M yes > /dev/null ^C[ perf record: Woken up 228 times to write data ] [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ] yes: Interrupt $ ls -l perf.data-0* -rw------- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-00000 -rw------- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-00001 -rw------- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-00002 -rw------- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-00003 -rw------- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-00004
Did you consider doing this with "perf inject" instead of "perf record"? That way you could get the split exactly right, and you could split the same file different ways.
Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
Quoted 36 lines · Adrian Hunter
Quoted 24 lines · Jiri Olsa
split event data into multiple files based on the file size or time delta specified as an argument to the option.
Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold.
The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like:
-M 5M for 5 megabytes threshold
-M 1h for 1 hour threshold
The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be:
perf.data-00000
perf.data-00001
perf.data-00002
...Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise.
Example:
Quoted 10 lines
$ perf record -M 5M yes > /dev/null ^C[ perf record: Woken up 228 times to write data ] [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ] yes: Interrupt $ ls -l perf.data-0* -rw------- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-00000 -rw------- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-00001 -rw------- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-00002 -rw------- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-00003 -rw------- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-00004
Quoted 3 lines
Did you consider doing this with "perf inject" instead of "perf record"? That way you could get the split exactly right, and you could split the same file different ways.
right, this could be added into inject command as well
jirka
Adrian Hunter
reviewerRe: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
Quoted 37 lines · Jiri Olsa
Quoted 36 lines · Adrian Hunter
Quoted 34 lines · Jiri Olsa
split event data into multiple files based on the file size or time delta specified as an argument to the option.
Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold.
The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like:
-M 5M for 5 megabytes threshold
-M 1h for 1 hour threshold
The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be:
perf.data-00000
perf.data-00001
perf.data-00002
...Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise.
Example:
$ perf record -M 5M yes > /dev/null ^C[ perf record: Woken up 228 times to write data ] [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ] yes: Interrupt $ ls -l perf.data-0* -rw------- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-00000 -rw------- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-00001 -rw------- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-00002 -rw------- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-00003 -rw------- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-00004
Quoted 5 lines
Quoted 3 lines
Did you consider doing this with "perf inject" instead of "perf record"? That way you could get the split exactly right, and you could split the same file different ways.
right, this could be added into inject command as well
I was really thinking "instead" not "as well" e.g.
perf record yes | perf inject -M 5M
Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command
Quoted 42 lines · Adrian Hunter
Quoted 41 lines · Jiri Olsa
Quoted 39 lines · Adrian Hunter
Quoted 24 lines · Jiri Olsa
split event data into multiple files based on the file size or time delta specified as an argument to the option.
Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold.
The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like:
-M 5M for 5 megabytes threshold
-M 1h for 1 hour threshold
The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be:
perf.data-00000
perf.data-00001
perf.data-00002
...Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise.
Example:
Quoted 10 lines
$ perf record -M 5M yes > /dev/null ^C[ perf record: Woken up 228 times to write data ] [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ] yes: Interrupt $ ls -l perf.data-0* -rw------- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-00000 -rw------- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-00001 -rw------- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-00002 -rw------- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-00003 -rw------- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-00004Did you consider doing this with "perf inject" instead of "perf record"? That way you could get the split exactly right, and you could split the same file different ways.
Quoted 6 lines
Quoted 1 line
right, this could be added into inject command as well
I was really thinking "instead" not "as well" e.g.
perf record yes | perf inject -M 5M
yep, got it.. this gives you more fine grained size resolution and lets you split already stored data files, nice
But while I think this is nice feature to have, two things keeps me thinking we should leave the support in record command as well:
- the extra inject command overhead, that needs to parse the whole stream, which might not be wanted in some cases
- AFAIU the vast majority of perf users do just record/report and would be restrained from this feature ;-)
jirka
PATCH v2 25/25 perf tools: Have the process properly sythesized in subsequent data files
In reply to Jiri OlsaChange the target so the sythesizing routine would think it needs to synthesized whole process, because there's no exec COMM event for subsequent data files.
--- tools/perf/builtin-record.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
tools/perf/builtin-record.c +11 -0
@@ -516,6 +516,8 @@ static int multi_file_finish(struct perf_record *rec)static int multi_file_init (struct perf_record *rec) {struct perf_data_file *file = rec->file; +struct perf_record_opts *opts = &rec->opts; +static bool target_change;int err;if (multi_file_name (rec->file, rec->multi_idx++))
@@ -529,6 +531,15 @@ static int multi_file_init(struct perf_record *rec)if (err) goto out_close; +/* +* XXX HACK - Make the target looks like we have pid +* so we get the process fully synthetised. +*/ +if (perf_target__none (&opts->target) && !target_change) { + target_change = true; + opts->target.pid ="multi" ; + } + err =synthesize_record_file (rec);if (err) goto out_close;
Signature · 1 line
1.7.11.7
Quoted 4 lines · Jiri Olsa
hi,
sending the support for multiple file storage. Initial RFC is here:
http://marc.info/?l=linux-kernel&m=137408381902423&w=2
Just a general comment. If you repost please always keep the overview description/motivation in 0/0.
-Andi
Peter Zijlstra
maintainerQuoted 11 lines · Jiri Olsa
hi,
sending the support for multiple file storage. Initial RFC is here:
http://marc.info/?l=linux-kernel&m=137408381902423&w=2v2 changes:
- reworked perf mmap size setup to be able to get the mmap size value easily later
- added perf.data read/write test for v2 and v3 for both endianity
- added record '-M time' support
So this 0/n post seems to have forgotten to list the rationale for doing all this..
The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.
However your earlier patches didn't do it because of that -- I forgot why they did it and you're now not telling.
Quoted 13 lines · Peter Zijlstra
Quoted 11 lines · Jiri Olsa
hi,
sending the support for multiple file storage. Initial RFC is here:
http://marc.info/?l=linux-kernel&m=137408381902423&w=2v2 changes:
- reworked perf mmap size setup to be able to get the mmap size value easily later
- added perf.data read/write test for v2 and v3 for both endianity
- added record '-M time' support
Quoted 2 lines
So this 0/n post seems to have forgotten to list the rationale for doing all this..
here it is:
This patchset is actually doing 3 essential things:
1) adds new perf data format version (v3)
2) adds class/object to handle perf data file
3) adds record '-M size' option
ad 1) Along the way I realized it'd be much easier to have the
format simplified. I ended up with just 'DATA' and 'FEATURES' sections making the file format more generic.
The easier/generic format will be for example useful for the cpu hotplug notification (and maybe even for ftrace data files features?)
The version 2 should be still readable by perf, while it produces data in version 3 from this patchset onwards.
ad 2) I needed to unify perf data file handling to make the -M
implementation clear.
Quoted 5 lines
The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.
I haven't considered the thread based storage yet, but I think having the simplified format and centralized file handling will only help.
jirka
Peter Zijlstra
maintainerQuoted 6 lines · Jiri Olsa
Quoted 1 line · Peter Zijlstra
On Sun, Sep 01, 2013 at 12:36:11PM +0200, Jiri Olsa wrote:
This patchset is actually doing 3 essential things:
1) adds new perf data format version (v3)
2) adds class/object to handle perf data file
3) adds record '-M size' option
So the -M thing is where the multiple files stuff comes from, right? What's the purpose of this? Why would I care how big these files get, surely filesystems handle 4g+ files these days.
Quoted 8 lines
Quoted 4 lines
The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.
I haven't considered the thread based storage yet, but I think having the simplified format and centralized file handling will only help.
Please do consider it now, it'd suck to have to change the file format thing yet again.
With both per-thread and size limited your filename generation needs some extra bits I suppose.
Quoted 11 lines · Peter Zijlstra
Quoted 6 lines · Jiri Olsa
Quoted 1 line · Peter Zijlstra
On Sun, Sep 01, 2013 at 12:36:11PM +0200, Jiri Olsa wrote:
This patchset is actually doing 3 essential things:
1) adds new perf data format version (v3)
2) adds class/object to handle perf data file
3) adds record '-M size' optionSo the -M thing is where the multiple files stuff comes from, right? What's the purpose of this? Why would I care how big these files get, surely filesystems handle 4g+ files these days.
my usage currently is to having this running during the day:
(well whenever I remember to run it.. ;-) )
[jolsa@krava perf]$ sudo ./perf record -g -M 1m -a
and checking report when the system or app get stuck
with multiple files I can just easily (or automaticaly) remove old ones without restarting session
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.
Quoted 6 lines
Quoted 3 lines
I haven't considered the thread based storage yet, but I think having the simplified format and centralized file handling will only help.
Please do consider it now, it'd suck to have to change the file format thing yet again.
ook
Quoted 3 lines
With both per-thread and size limited your filename generation needs some extra bits I suppose.
yep, and probably some merging at the end
jirka
Quoted 9 lines · Jiri Olsa
my usage currently is to having this running during the day:
(well whenever I remember to run it.. ;-) )[jolsa@krava perf]$ sudo ./perf record -g -M 1m -a
and checking report when the system or app get stuck
with multiple files I can just easily (or automaticaly) remove old ones without restarting session
Aren't you losing potentially important events by doing that -- FORK, COMM, MMAP?
I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.
David
Quoted 6 lines · David Ahern
Quoted 5 lines · Jiri Olsa
my usage currently is to having this running during the day:
(well whenever I remember to run it.. ;-) )[jolsa@krava perf]$ sudo ./perf record -g -M 1m -a
Quoted 7 lines
Quoted 4 lines
and checking report when the system or app get stuck
with multiple files I can just easily (or automaticaly) remove old ones without restarting session
Aren't you losing potentially important events by doing that -- FORK, COMM, MMAP?
those are synthesized for each file via synthesize_record function, see:
[PATCH 19/25] perf tools: Move synthesizing into single function
Quoted 5 lines
I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.
so far I noticed there could be race among EXIT and remaining SAMPLE events on another CPU mmap than EXIT event.. ending up with EXIT being stored in the old file, while SAMPLEs will get to the new one
I was thinking about some 'perf daemon' so I dont need to run that manually.. seems similar to what you did
jirka
Quoted 12 lines · Jiri Olsa
Quoted 11 lines · David Ahern
Quoted 9 lines · Jiri Olsa
my usage currently is to having this running during the day:
(well whenever I remember to run it.. ;-) )[jolsa@krava perf]$ sudo ./perf record -g -M 1m -a
and checking report when the system or app get stuck
with multiple files I can just easily (or automaticaly) remove old ones without restarting session
Quoted 7 lines
Quoted 2 lines
Aren't you losing potentially important events by doing that -- FORK, COMM, MMAP?
those are synthesized for each file via
synthesize_recordfunction, see:[PATCH 19/25] perf tools: Move synthesizing into single function
Ok. haven't had time to look through your 2 large patch sets.
Seems like a lot of repetitive work on a loaded system. The overhead of the task events will dominate compared to the samples.
Quoted 7 lines
Quoted 6 lines
I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.
Quoted 7 lines
so far I noticed there could be race among EXIT and remaining SAMPLE events on another CPU mmap than EXIT event.. ending up with EXIT being stored in the old file, while SAMPLEs will get to the new one
I was thinking about some 'perf daemon' so I dont need to run that manually.. seems similar to what you did
Right now I focus on scheduling events. This latest version of it can be easily recycled for other use cases. Some work would be needed to dump events to a file versus dumping processed information.
I am in San Jose this week. Not sure if I will have time to finish it to a point of pushing out patches, but maybe I can push to github in the next couple of days.
David
Quoted 13 lines · David Ahern
Quoted 5 lines
Quoted 4 lines
I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.
Quoted 7 lines
so far I noticed there could be race among EXIT and remaining SAMPLE events on another CPU mmap than EXIT event.. ending up with EXIT being stored in the old file, while SAMPLEs will get to the new one
I was thinking about some 'perf daemon' so I dont need to run that manually.. seems similar to what you did
Quoted 7 lines
Right now I focus on scheduling events. This latest version of it can be easily recycled for other use cases. Some work would be needed to dump events to a file versus dumping processed information.
I am in San Jose this week. Not sure if I will have time to finish it to a point of pushing out patches, but maybe I can push to github in the next couple of days.
https://github.com/dsahern/linux/tree/perf-sched-timehist-3.11
Take a look at tools/perf/schedmon.c. Task events are processed and sample events are stashed on a list (daemon__process_sample). Could easily do something similar for mmap, comm, fork events and then on exit event flush events for terminated tasks. Then you will have both the task and sample events which can be dumped to a file for later processing.
David
Quoted 6 lines · David Ahern
Quoted 5 lines · Jiri Olsa
my usage currently is to having this running during the day:
(well whenever I remember to run it.. ;-) )[jolsa@krava perf]$ sudo ./perf record -g -M 1m -a
Quoted 7 lines
Quoted 4 lines
and checking report when the system or app get stuck
with multiple files I can just easily (or automaticaly) remove old ones without restarting session
Aren't you losing potentially important events by doing that -- FORK, COMM, MMAP?
I suspect these could/should be tracked and emitted fully (in bulk) when a new data file is opened, so that each partial data file is fully consistent?
Quoted 4 lines
I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.
Could this be used to emit currently relevant task context?
Btw., I also think it would be useful to have kernel support for that - the 'collections' stuff I talked about a good while ago: the kernel would work with user-space to iterate over all MMAPs and all running COMMs at the opening of a tracing session.
That way we could avoid racy access to /proc, we could make sure that all information that is emitted by FORK/COMM/MMAP is also emitted for the 'bulk' data, etc.
Thanks,
Ingo
Quoted 6 lines · Ingo Molnar
Quoted 2 lines
Aren't you losing potentially important events by doing that -- FORK, COMM, MMAP?
I suspect these could/should be tracked and emitted fully (in bulk) when a new data file is opened, so that each partial data file is fully consistent?
In my case I am not saving task events, but processing them. In Jiri's case where events are written to a file it should be possible to stash the unprocessed events on a list, when the exit happens move them to a dead threads list which can be cleaned up from time to time and then on file dump requests dump the task events followed by the sample events.
Quoted 7 lines
Quoted 5 lines
I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.
Could this be used to emit currently relevant task context?
sure.
Quoted 9 lines
Btw., I also think it would be useful to have kernel support for that - the 'collections' stuff I talked about a good while ago: the kernel would work with user-space to iterate over all MMAPs and all running COMMs at the opening of a tracing session.
That way we could avoid racy access to /proc, we could make sure that all information that is emitted by FORK/COMM/MMAP is also emitted for the 'bulk' data, etc.
Walking the task list and emitting events would be better but wouldn't that be a performance hit holding the task lock (tasklist_lock?)? (I thought that one is needed when walking the task list.)
David
Adrian Hunter
reviewerQuoted 13 lines · Peter Zijlstra
Quoted 11 lines · Jiri Olsa
hi,
sending the support for multiple file storage. Initial RFC is here:
http://marc.info/?l=linux-kernel&m=137408381902423&w=2v2 changes:
- reworked perf mmap size setup to be able to get the mmap size value easily later
- added perf.data read/write test for v2 and v3 for both endianity
- added record '-M time' support
Quoted 6 lines
So this 0/n post seems to have forgotten to list the rationale for doing all this..
The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.
Do you need multiple files for that? Why not just feed writes to a pool of threads?
Peter Zijlstra
maintainerQuoted 8 lines · Adrian Hunter
Quoted 3 lines · Peter Zijlstra
The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.
Do you need multiple files for that? Why not just feed writes to a pool of threads?
Not sure I understand what you mean there. Since we have a (or multiple) event per cpu it doesn't make sense to read that data from another cpu and have that write it to disk. That completely destroys the locality.
Instead the suggestion was to have a thread per cpu (bound) to empty out the per cpu mmap buffer(s) and write them to disk.
Now the only way to coherently do that into a single file is to do some streams implementation, which I suppose is possible. Pre-allocate large-ish sections per thread and mark them as such, then let the thread spool data into it, when full allocate a new section.
However this would result in allocation-block-size*nr_cpus minimal file sizes, which wouldn't be a problem per-se since they can be sparse.
It will however pretty much mess up Jolsa's max-MBs per file thing and it is of course slightly more complex than having a file per cpu/thread. And is more likely to create kernel lock contention; the pagecache locks are typically per inode, bouncing those around the machine isn't nice either.
Namhyung Kim
maintainerHi Peter and Jiri,
Quoted 13 lines · Peter Zijlstra
Quoted 11 lines · Jiri Olsa
hi,
sending the support for multiple file storage. Initial RFC is here:
http://marc.info/?l=linux-kernel&m=137408381902423&w=2v2 changes:
- reworked perf mmap size setup to be able to get the mmap size value easily later
- added perf.data read/write test for v2 and v3 for both endianity
- added record '-M time' support
Quoted 6 lines
So this 0/n post seems to have forgotten to list the rationale for doing all this..
The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.
Per-thread or per-cpu? In my perf ftrace patchset, I used to per-cpu data file for this reason. Do you think per-thread approach is better than per-cpu one?
Jiri, one of my colleagues asked me about the multiple file support separated by time while ago. At that time I just added --time-filter option to perf report, but it'd better if perf record can support it. (Unfortunately the patch seems to buried in the list).
Anyway, as Peter said, please consider per-thread or per-cpu multiple file support with your series. It will help further developments.
Thanks,
Namhyung