perf record: Add support to store data in directory
Jiri Olsa
reviewerv3 cover perf record: Add support to store data in directory
hi,
this patchset adds the --dir option to record command (and all the other record command that overload cmd_record) that allows the data to be stored in directory with multiple data files.
It's next step for multiple threads implementation in record. It's now possible to make directory data via --dir option, like:
$ perf record --dir perf bench sched messaging $ ls -l perf.data total 344 -rw-------. 1 jolsa jolsa 43864 Jan 20 22:26 data.0 -rw-------. 1 jolsa jolsa 30464 Jan 20 22:26 data.1 -rw-------. 1 jolsa jolsa 53816 Jan 20 22:26 data.2 -rw-------. 1 jolsa jolsa 30368 Jan 20 22:26 data.3 -rw-------. 1 jolsa jolsa 40088 Jan 20 22:26 data.4 -rw-------. 1 jolsa jolsa 42592 Jan 20 22:26 data.5 -rw-------. 1 jolsa jolsa 56136 Jan 20 22:26 data.6 -rw-------. 1 jolsa jolsa 25992 Jan 20 22:26 data.7 -rw-------. 1 jolsa jolsa 8832 Jan 20 22:26 header
There's a data file created for every cpu and it's storing data for those cpu maps. The report command will read it transparently, sort it and display as single file data.
There's new DIR_FORMAT feature to describe directory data layout/format. In future we can describe different data files layout according to special needs.
It's possible to transform directory data into standard perf.data file via simple inject command:
$ perf inject -o perf.data.file -i perf.data
The old perf fails over the directory data with following message:
$ perf report incompatible file format (rerun with -v to learn more)
I'm now testing the record threads support, so I'd like to have some agreement on the directory data support before.
v3 changes:
- add rm_rf_perf_data to safely remove file/directory perf data
- allocation fix in perf_data__create_dir
v2 changes:
- rm_rf changes are already accepted with requested changes
- updated doc/man plus adding perf.data-directory-format.txt to describe directory format/layout
- the --switch-output options now works over directory data
- data rollback is not part of this patchset, updated my TODO though ;-)
- added --output-dir option to combine -o and --dir
- added DIR_FORMAT feature to describe directory data
- disabling directory output for aio for now
It's also available in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/dir
thanks,
jirka
---
Jiri Olsa (20):
perf tools: Add depth checking to rm_rf
perf tools: Add pattern name checking to rm_rf
perf tools: Add rm_rf_perf_data function
perf data: Move size to struct perf_data_file
perf data: Add global path holder
perf data: Make check_backup work over directories perf data: Fail check_backup in case of error
perf data: Add perf_data__(create_dir|close_dir) functions perf data: Add perf_data__open_dir_data function perf data: Add directory support
perf data: Don't store auxtrace index for directory data file perf data: Add perf_data__update_dir function
perf data: Make perf_data__size to work over directory perf header: Add DIR_FORMAT feature to describe directory data perf session: Add process callback to reader object perf session: Add __perf_session__process_dir_events function perf session: Add path to reader object
perf record: Add --dir option to store data in directory perf record: Add --output-dir option to store data in directory perf record: Describe perf.data directory format
tools/lib/subcmd/parse-options.h | 4 ++ tools/perf/Documentation/perf-record.txt | 6 +++ tools/perf/Documentation/perf.data-directory-format.txt | 54 ++++++++++++++++++++ tools/perf/builtin-annotate.c | 4 +- tools/perf/builtin-buildid-cache.c | 4 +- tools/perf/builtin-buildid-list.c | 8 ++- tools/perf/builtin-c2c.c | 4 +- tools/perf/builtin-diff.c | 12 ++--- tools/perf/builtin-evlist.c | 4 +- tools/perf/builtin-inject.c | 10 ++-- tools/perf/builtin-kmem.c | 2 +- tools/perf/builtin-kvm.c | 8 ++- tools/perf/builtin-lock.c | 8 ++- tools/perf/builtin-mem.c | 8 ++- tools/perf/builtin-record.c | 104 ++++++++++++++++++++++++++++++++++----- tools/perf/builtin-report.c | 6 +-- tools/perf/builtin-sched.c | 16 +++--- tools/perf/builtin-script.c | 12 ++--- tools/perf/builtin-stat.c | 6 +-- tools/perf/builtin-timechart.c | 8 ++- tools/perf/builtin-trace.c | 8 ++- tools/perf/util/data-convert-bt.c | 2 +- tools/perf/util/data.c | 257 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------- tools/perf/util/data.h | 26 +++++++--- tools/perf/util/header.c | 44 ++++++++++++++++- tools/perf/util/header.h | 5 ++ tools/perf/util/mmap.h | 23 ++++----- tools/perf/util/session.c | 125 +++++++++++++++++++++++++++++++++++++++++++--- tools/perf/util/util.c | 55 ++++++++++++++++++++- tools/perf/util/util.h | 1 + 30 files changed, 700 insertions(+), 134 deletions(-) create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt
Jiri Olsa
reviewerPATCH v3 1/20 perf tools: Add depth checking to rm_rf
Adding depth argument to rm_rf (and renaming it to rm_rf_depth) to specify the depth we will go searching for files to remove.
It will be used to specify single depth for perf.data directory removal in following patch.
--- tools/perf/util/util.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
tools/perf/util/util.c +12 -2
@@ -117,7 +117,12 @@ int mkdir_p(char *path, mode_t mode)return (stat (path, &st) &&mkdir (path, mode)) ?-1 :0 ; } -int rm_rf (const char *path) +/* +* The depth specify how deep the removal will go. +* 0 - will remove only files under the 'path' directory +* 1 .. x - will dive in x-level deep under the 'path' directory +*/ +static int rm_rf_depth (const char *path,int depth) {DIR *dir;int ret;
@@ -155,7 +160,7 @@ int rm_rf(const char *path) }if (S_ISDIR (statbuf.st_mode)) - ret =rm_rf (namebuf); + ret = depth ?rm_rf_depth (namebuf, depth -1 ) :0 ;else ret =unlink (namebuf); }
@@ -167,6 +172,11 @@ int rm_rf(const char *path)return rmdir (path); } +int rm_rf (const char *path) +{ +return rm_rf_depth (path,INT_MAX ); +} +/* A filter which removes dot files */ bool lsdir_no_dot_filter (const char *name__maybe_unused ,struct dirent *d) {
Signature · 1 line
2.17.2
tip-bot for Jiri Olsa
merge bot[tip:perf/core] perf tools: Add depth checking to rm_rf
perf tools: Add depth checking to rm_rf
Commit-ID: 05a486593977bfcf71de6bf5cad6d045c18829c6 Gitweb: https://git.kernel.org/tip/05a486593977bfcf71de6bf5cad6d045c18829c6 Author: Jiri Olsa <[email protected]> AuthorDate: Sun, 24 Feb 2019 20:06:37 +0100 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Mon, 25 Feb 2019 10:32:11 -0300
perf tools: Add depth checking to rm_rf
Adding depth argument to rm_rf (and renaming it to rm_rf_depth) to specify the depth we will go searching for files to remove.
It will be used to specify single depth for perf.data directory removal in following patch.
--- tools/perf/util/util.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
tools/perf/util/util.c +12 -2
@@ -117,7 +117,12 @@ int mkdir_p(char *path, mode_t mode)return (stat (path, &st) &&mkdir (path, mode)) ?-1 :0 ; } -int rm_rf (const char *path) +/* +* The depth specify how deep the removal will go. +* 0 - will remove only files under the 'path' directory +* 1 .. x - will dive in x-level deep under the 'path' directory +*/ +static int rm_rf_depth (const char *path,int depth) {DIR *dir;int ret;
@@ -155,7 +160,7 @@ int rm_rf(const char *path) }if (S_ISDIR (statbuf.st_mode)) - ret =rm_rf (namebuf); + ret = depth ?rm_rf_depth (namebuf, depth -1 ) :0 ;else ret =unlink (namebuf); }
@@ -167,6 +172,11 @@ int rm_rf(const char *path)return rmdir (path); } +int rm_rf (const char *path) +{ +return rm_rf_depth (path,INT_MAX ); +} +/* A filter which removes dot files */ bool lsdir_no_dot_filter (const char *name__maybe_unused ,struct dirent *d) {
Jiri Olsa
reviewerPATCH v3 2/20 perf tools: Add pattern name checking to rm_rf
In reply to Jiri OlsaAdding pattern argument to rm_rf_depth (and renaming it to rm_rf_depth_pat) to specify the name pattern files need to match inside the directory. The function fails if we find different file to remove.
--- tools/perf/util/util.c | 36 +++++++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-)
tools/perf/util/util.c +33 -3
@@ -21,6 +21,7 @@#include <linux/time64.h> #include <unistd.h> #include "strlist.h" +#include "string2.h" /* * XXX We need to find a better place for these things...
@@ -117,12 +118,38 @@ int mkdir_p(char *path, mode_t mode)return (stat (path, &st) &&mkdir (path, mode)) ?-1 :0 ; } +static bool match_pat (char *file,const char **pat) +{ +int i =0 ; + +if (!pat) +return true; + +while (pat[i]) { +if (strglobmatch (file, pat[i])) +return true; + + i++; + } + +return false; +} +/* * The depth specify how deep the removal will go. * 0 - will remove only files under the 'path' directory * 1 .. x - will dive in x-level deep under the 'path' directory +* +* If specified the pat is array of string patterns ended with NULL, +* which are checked upon every file/directory found. Only matching +* ones are removed. +* +* The function returns: +* 0 on success +* -1 on removal failure with errno set +* -2 on pattern failure */ -static int rm_rf_depth (const char *path,int depth) +static int rm_rf_depth_pat (const char *path,int depth,const char **pat) {DIR *dir;int ret;
@@ -149,6 +176,9 @@ static int rm_rf_depth(const char *path, int depth)if (!strcmp (d->d_name,"." ) || !strcmp (d->d_name,".." ))continue ; +if (!match_pat (d->d_name, pat)) +return -2 ; +scnprintf (namebuf,sizeof (namebuf),"%s/%s" , path, d->d_name);
@@ -160,7 +190,7 @@ static int rm_rf_depth(const char *path, int depth) }if (S_ISDIR (statbuf.st_mode)) - ret = depth ?rm_rf_depth (namebuf, depth -1 ) :0 ; + ret = depth ?rm_rf_depth_pat (namebuf, depth -1 , pat) :0 ;else ret =unlink (namebuf); }
@@ -174,7 +204,7 @@ static int rm_rf_depth(const char *path, int depth)int rm_rf (const char *path) { -return rm_rf_depth (path,INT_MAX ); +return rm_rf_depth_pat (path,INT_MAX ,NULL ); }/* A filter which removes dot files */
Signature · 1 line
2.17.2
tip-bot for Jiri Olsa
merge bot[tip:perf/core] perf tools: Add pattern name checking to rm_rf
perf tools: Add pattern name checking to rm_rf
Commit-ID: cdb6b0235f170a5ffcd74731178efc064bd4d24a Gitweb: https://git.kernel.org/tip/cdb6b0235f170a5ffcd74731178efc064bd4d24a Author: Jiri Olsa <[email protected]> AuthorDate: Sun, 24 Feb 2019 20:06:38 +0100 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Mon, 25 Feb 2019 10:33:04 -0300
perf tools: Add pattern name checking to rm_rf
Add pattern argument to rm_rf_depth() (and rename it to rm_rf_depth_pat()) to specify the name pattern files need to match inside the directory.
The function fails if we find different file to remove.
--- tools/perf/util/util.c | 36 +++++++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-)
tools/perf/util/util.c +33 -3
@@ -21,6 +21,7 @@#include <linux/time64.h> #include <unistd.h> #include "strlist.h" +#include "string2.h" /* * XXX We need to find a better place for these things...
@@ -117,12 +118,38 @@ int mkdir_p(char *path, mode_t mode)return (stat (path, &st) &&mkdir (path, mode)) ?-1 :0 ; } +static bool match_pat (char *file,const char **pat) +{ +int i =0 ; + +if (!pat) +return true; + +while (pat[i]) { +if (strglobmatch (file, pat[i])) +return true; + + i++; + } + +return false; +} +/* * The depth specify how deep the removal will go. * 0 - will remove only files under the 'path' directory * 1 .. x - will dive in x-level deep under the 'path' directory +* +* If specified the pat is array of string patterns ended with NULL, +* which are checked upon every file/directory found. Only matching +* ones are removed. +* +* The function returns: +* 0 on success +* -1 on removal failure with errno set +* -2 on pattern failure */ -static int rm_rf_depth (const char *path,int depth) +static int rm_rf_depth_pat (const char *path,int depth,const char **pat) {DIR *dir;int ret;
@@ -149,6 +176,9 @@ static int rm_rf_depth(const char *path, int depth)if (!strcmp (d->d_name,"." ) || !strcmp (d->d_name,".." ))continue ; +if (!match_pat (d->d_name, pat)) +return -2 ; +scnprintf (namebuf,sizeof (namebuf),"%s/%s" , path, d->d_name);
@@ -160,7 +190,7 @@ static int rm_rf_depth(const char *path, int depth) }if (S_ISDIR (statbuf.st_mode)) - ret = depth ?rm_rf_depth (namebuf, depth -1 ) :0 ; + ret = depth ?rm_rf_depth_pat (namebuf, depth -1 , pat) :0 ;else ret =unlink (namebuf); }
@@ -174,7 +204,7 @@ static int rm_rf_depth(const char *path, int depth)int rm_rf (const char *path) { -return rm_rf_depth (path,INT_MAX ); +return rm_rf_depth_pat (path,INT_MAX ,NULL ); }/* A filter which removes dot files */
Jiri Olsa
reviewerPATCH v3 3/20 perf tools: Add rm_rf_perf_data function
In reply to Jiri OlsaTo remove perf.data including the directory,
with checking on expected files and no other
directories inside.
--- tools/perf/util/util.c | 11 +++++++++++ tools/perf/util/util.h | 1 + 2 files changed, 12 insertions(+)
tools/perf/util/util.c +11 -0
@@ -202,6 +202,17 @@ static int rm_rf_depth_pat(const char *path, int depth, const char **pat)return rmdir (path); } +int rm_rf_perf_data (const char *path) +{ +const char *pat[] = { +"header" , +"data.*" , +NULL , + }; + +return rm_rf_depth_pat (path,0 , pat); +} +int rm_rf (const char *path) {return rm_rf_depth_pat (path,INT_MAX ,NULL );
tools/perf/util/util.h +1 -0
@@ -31,6 +31,7 @@ struct strlist;int mkdir_p (char *path,mode_t mode);int rm_rf (const char *path); +int rm_rf_perf_data (const char *path);struct strlist *lsdir (const char *name,bool (*filter)(const char *,struct dirent *));bool lsdir_no_dot_filter (const char *name,struct dirent *d);int copyfile (const char *from,const char *to);
Signature · 1 line
2.17.2
tip-bot for Jiri Olsa
merge bot[tip:perf/core] perf tools: Add rm_rf_perf_data function
perf tools: Add rm_rf_perf_data function
Commit-ID: c69e4c37b37c816c7dbd307a6e7d2d2a5cfe8788 Gitweb: https://git.kernel.org/tip/c69e4c37b37c816c7dbd307a6e7d2d2a5cfe8788 Author: Jiri Olsa <[email protected]> AuthorDate: Sun, 24 Feb 2019 20:06:39 +0100 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Mon, 25 Feb 2019 10:33:51 -0300
perf tools: Add rm_rf_perf_data function
To remove perf.data including the directory, with checking on expected files and no other directories inside.
--- tools/perf/util/util.c | 11 +++++++++++ tools/perf/util/util.h | 1 + 2 files changed, 12 insertions(+)
tools/perf/util/util.c +11 -0
@@ -202,6 +202,17 @@ static int rm_rf_depth_pat(const char *path, int depth, const char **pat)return rmdir (path); } +int rm_rf_perf_data (const char *path) +{ +const char *pat[] = { +"header" , +"data.*" , +NULL , + }; + +return rm_rf_depth_pat (path,0 , pat); +} +int rm_rf (const char *path) {return rm_rf_depth_pat (path,INT_MAX ,NULL );
tools/perf/util/util.h +1 -0
@@ -31,6 +31,7 @@ struct strlist;int mkdir_p (char *path,mode_t mode);int rm_rf (const char *path); +int rm_rf_perf_data (const char *path);struct strlist *lsdir (const char *name,bool (*filter)(const char *,struct dirent *));bool lsdir_no_dot_filter (const char *name,struct dirent *d);int copyfile (const char *from,const char *to);
Jiri Olsa
reviewerPATCH v3 4/20 perf data: Move size to struct perf_data_file
In reply to Jiri OlsaWe are about to add support for multiple files,
so we need each file to keep its size.
--- tools/perf/builtin-record.c | 5 ++--- tools/perf/util/data.c | 2 +- tools/perf/util/data.h | 4 ++-- 3 files changed, 5 insertions(+), 6 deletions(-)
tools/perf/builtin-record.c +2 -3
@@ -660,10 +660,9 @@ static int process_sample_event(struct perf_tool *tool,static int process_buildids (struct record *rec) { -struct perf_data *data = &rec->data;struct perf_session *session = rec->session; -if (data->size ==0 ) +if (perf_data__size (&rec->data) ==0 )return 0 ;/*
@@ -851,7 +850,7 @@ record__finish_output(struct record *rec)return ; rec->session->header.data_size += rec->bytes_written; - data->size =lseek (perf_data__fd (data),0 ,SEEK_CUR ); + data->file.size =lseek (perf_data__fd (data),0 ,SEEK_CUR );if (!rec->no_buildid) {process_buildids (rec);
tools/perf/util/data.c +1 -1
@@ -82,7 +82,7 @@ static int open_file_read(struct perf_data *data) goto out_close; } - data->size = st.st_size; + data->file.size = st.st_size;return fd; out_close:
tools/perf/util/data.h +2 -2
@@ -12,13 +12,13 @@ enum perf_data_mode {struct perf_data_file {const char *path;int fd; +unsigned long size; };struct perf_data {struct perf_data_file file;bool is_pipe;bool force; -unsigned long size;enum perf_data_mode mode; };
@@ -44,7 +44,7 @@ static inline int perf_data__fd(struct perf_data *data)static inline unsigned long perf_data__size (struct perf_data *data) { -return data->size; +return data->file.size; }int perf_data__open (struct perf_data *data);
Signature · 1 line
2.17.2
Jiri Olsa
reviewerPATCH v3 5/20 perf data: Add global path holder
In reply to Jiri OlsaAdding path to the struct perf_data. It will keep the configured path for the data (const char*). The path in struct perf_data_file is now dynamically allocated (duped) from it.
This scheme is useful/used in following patches where struct perf_data::path holds the 'configure' directory path and struct perf_data_file::path holds the allocated path for specific files.
Also it actually makes the code little simpler.
--- tools/perf/builtin-annotate.c | 4 +-- tools/perf/builtin-buildid-cache.c | 4 +-- tools/perf/builtin-buildid-list.c | 8 +++--- tools/perf/builtin-c2c.c | 4 +-- tools/perf/builtin-diff.c | 12 ++++----- tools/perf/builtin-evlist.c | 4 +-- tools/perf/builtin-inject.c | 10 +++----- tools/perf/builtin-kmem.c | 2 +- tools/perf/builtin-kvm.c | 8 +++--- tools/perf/builtin-lock.c | 8 +++--- tools/perf/builtin-mem.c | 8 +++--- tools/perf/builtin-record.c | 6 ++--- tools/perf/builtin-report.c | 6 ++--- tools/perf/builtin-sched.c | 16 +++++------- tools/perf/builtin-script.c | 12 ++++----- tools/perf/builtin-stat.c | 6 ++--- tools/perf/builtin-timechart.c | 8 +++--- tools/perf/builtin-trace.c | 8 +++--- tools/perf/util/data-convert-bt.c | 2 +- tools/perf/util/data.c | 39 +++++++++++++++++++++--------- tools/perf/util/data.h | 3 ++- 21 files changed, 86 insertions(+), 92 deletions(-)
tools/perf/builtin-annotate.c +2 -2
@@ -441,7 +441,7 @@ static int __cmd_annotate(struct perf_annotate *ann) }if (total_nr_samples ==0 ) { -ui__error ("The %s file has no samples!\n" , session->data->file.path); +ui__error ("The %s data has no samples!\n" , session->data->path); goto out; }
@@ -578,7 +578,7 @@ int cmd_annotate(int argc, const char **argv)if (quiet)perf_quiet_option (); - data.file.path = input_name; + data.path = input_name; annotate.session =perf_session__new (&data, false, &annotate.tool);if (annotate.session ==NULL )
tools/perf/builtin-buildid-cache.c +2 -2
@@ -416,8 +416,8 @@ int cmd_buildid_cache(int argc, const char **argv) nsi =nsinfo__new (ns_id);if (missing_filename) { - data.file.path = missing_filename; - data.force = force; + data.path = missing_filename; + data.force = force; session =perf_session__new (&data, false,NULL );if (session ==NULL )
tools/perf/builtin-buildid-list.c +3 -5
@@ -52,11 +52,9 @@ static int perf_session__list_build_ids(bool force, bool with_hits) {struct perf_session *session;struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = force, };symbol__elf_init ();
tools/perf/builtin-c2c.c +2 -2
@@ -2750,8 +2750,8 @@ static int perf_c2c__report(int argc, const char **argv)if (!input_name || !strlen (input_name)) input_name ="perf.data" ; - data.file.path = input_name; - data.force = symbol_conf.force; + data.path = input_name; + data.force = symbol_conf.force; err =setup_display (display);if (err)
tools/perf/builtin-diff.c +6 -6
@@ -708,7 +708,7 @@ static void data__fprintf(void)data__for_each_file (i, d)fprintf (stdout,"# [%d] %s %s\n" , - d->idx, d->data.file.path, + d->idx, d->data.path, !d->idx ?"(Baseline)" :"" );fprintf (stdout,"#\n" );
@@ -779,14 +779,14 @@ static int __cmd_diff(void)data__for_each_file (i, d) { d->session =perf_session__new (&d->data, false, &tool);if (!d->session) { -pr_err ("Failed to open %s\n" , d->data.file.path); +pr_err ("Failed to open %s\n" , d->data.path); ret =-1 ; goto out_delete; } ret =perf_session__process_events (d->session);if (ret) { -pr_err ("Failed to process %s\n" , d->data.file.path); +pr_err ("Failed to process %s\n" , d->data.path); goto out_delete; }
@@ -1289,9 +1289,9 @@ static int data_init(int argc, const char **argv)data__for_each_file (i, d) {struct perf_data *data = &d->data; - data->file.path = use_default ? defaults[i] : argv[i]; - data->mode =PERF_DATA_MODE_READ , - data->force = force, + data->path = use_default ? defaults[i] : argv[i]; + data->mode =PERF_DATA_MODE_READ , + data->force = force, d->idx = i; }
tools/perf/builtin-evlist.c +1 -3
@@ -23,9 +23,7 @@ static int __cmd_evlist(const char *file_name, struct perf_attr_details *detailsstruct perf_session *session;struct perf_evsel *pos;struct perf_data data = { - .file = { - .path = file_name, - }, + .path = file_name, .mode =PERF_DATA_MODE_READ , .force = details->force, };
tools/perf/builtin-inject.c +4 -6
@@ -770,10 +770,8 @@ int cmd_inject(int argc, const char **argv) .input_name = "-", .samples = LIST_HEAD_INIT(inject.samples), .output = { - .file = { - .path = "-", - }, - .mode = PERF_DATA_MODE_WRITE, + .path = "-", + .mode = PERF_DATA_MODE_WRITE, }, };struct perf_data data = {
@@ -786,7 +784,7 @@ int cmd_inject(int argc, const char **argv) "Inject build-ids into the output stream"),OPT_STRING ('i' ,"input" , &inject.input_name,"file" ,"input file name" ), -OPT_STRING ('o' ,"output" , &inject.output.file.path,"file" , +OPT_STRING ('o' ,"output" , &inject.output.path,"file" ,"output file name" ),OPT_BOOLEAN ('s' ,"sched-stat" , &inject.sched_stat, "Merge sched-stat and sched-switch for getting events "
@@ -834,7 +832,7 @@ int cmd_inject(int argc, const char **argv) inject.tool.ordered_events = inject.sched_stat; - data.file.path = inject.input_name; + data.path = inject.input_name; inject.session =perf_session__new (&data, true, &inject.tool);if (inject.session ==NULL )return -1 ;
tools/perf/builtin-kmem.c +1 -1
@@ -1949,7 +1949,7 @@ int cmd_kmem(int argc, const char **argv)return __cmd_record (argc, argv); } - data.file.path = input_name; + data.path = input_name; kmem_session = session =perf_session__new (&data, false, &perf_kmem);if (session ==NULL )
tools/perf/builtin-kvm.c +3 -5
@@ -1080,11 +1080,9 @@ static int read_events(struct perf_kvm_stat *kvm) .ordered_events = true, };struct perf_data file = { - .file = { - .path = kvm->file_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = kvm->force, + .path = kvm->file_name, + .mode =PERF_DATA_MODE_READ , + .force = kvm->force, }; kvm->tool = eops;
tools/perf/builtin-lock.c +3 -5
@@ -866,11 +866,9 @@ static int __cmd_report(bool display_info) .ordered_events = true, };struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = force, }; session =perf_session__new (&data, false, &eops);
tools/perf/builtin-mem.c +3 -5
@@ -239,11 +239,9 @@ static int process_sample_event(struct perf_tool *tool,static int report_raw_events (struct perf_mem *mem) {struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = mem->force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = mem->force, };int ret; struct perf_session *session = perf_session__new(&data, false,
tools/perf/builtin-record.c +3 -3
@@ -918,7 +918,7 @@ record__switch_output(struct record *rec, bool at_exit)if (!quiet)fprintf (stderr,"[ perf record: Dump %s.%s ]\n" , - data->file.path, timestamp); + data->path, timestamp);/* Output tracking events */ if (!at_exit) {
@@ -1461,7 +1461,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)fprintf (stderr,"[ perf record: Captured and wrote %.3f MB %s%s%s ]\n" ,perf_data__size (data) /1024.0 /1024.0 , - data->file.path, postfix, samples); + data->path, postfix, samples); } out_delete_session:
@@ -1862,7 +1862,7 @@ static struct option __record_options[] = {OPT_STRING ('C' ,"cpu" , &record.opts.target.cpu_list,"cpu" ,"list of cpus to monitor" ),OPT_U64 ('c' ,"count" , &record.opts.user_interval,"event period to sample" ), -OPT_STRING ('o' ,"output" , &record.data.file.path,"file" , +OPT_STRING ('o' ,"output" , &record.data.path,"file" ,"output file name" ), OPT_BOOLEAN_SET('i', "no-inherit", &record.opts.no_inherit, &record.opts.no_inherit_set,
tools/perf/builtin-report.c +3 -3
@@ -899,7 +899,7 @@ static int __cmd_report(struct report *rep) rep->nr_entries +=evsel__hists (pos)->nr_entries;if (rep->nr_entries ==0 ) { -ui__error ("The %s file has no samples!\n" , data->file.path); +ui__error ("The %s data has no samples!\n" , data->path);return 0 ; }
@@ -1207,8 +1207,8 @@ int cmd_report(int argc, const char **argv) input_name ="perf.data" ; } - data.file.path = input_name; - data.force = symbol_conf.force; + data.path = input_name; + data.force = symbol_conf.force; repeat: session =perf_session__new (&data, false, &report.tool);
tools/perf/builtin-sched.c +6 -10
@@ -1785,11 +1785,9 @@ static int perf_sched__read_events(struct perf_sched *sched) };struct perf_session *session;struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = sched->force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = sched->force, };int rc =-1 ;
@@ -2958,11 +2956,9 @@ static int perf_sched__timehist(struct perf_sched *sched) { "sched:sched_migrate_task", timehist_migrate_task_event, }, };struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = sched->force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = sched->force, };struct perf_session *session;
tools/perf/builtin-script.c +5 -7
@@ -2951,10 +2951,8 @@ int find_scripts(char **scripts_array, char **scripts_path_array)DIR *scripts_dir, *lang_dir;struct perf_session *session;struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , + .path = input_name, + .mode =PERF_DATA_MODE_READ , };char *temp;int i =0 ;
@@ -3427,8 +3425,8 @@ int cmd_script(int argc, const char **argv) argc =parse_options_subcommand (argc, argv, options, script_subcommands, script_usage,PARSE_OPT_STOP_AT_NON_OPTION ); - data.file.path = input_name; - data.force = symbol_conf.force; + data.path = input_name; + data.force = symbol_conf.force;if (argc >1 && !strncmp (argv[0 ],"rec" ,strlen ("rec" ))) { rec_script_path =get_script_path (argv[1 ],RECORD_SUFFIX );
@@ -3654,7 +3652,7 @@ int cmd_script(int argc, const char **argv) goto out_delete; } - input =open (data.file.path,O_RDONLY );/* input_name */ + input =open (data.path,O_RDONLY );/* input_name */ if (input <0 ) { err = -errno;perror ("failed to open file" );
tools/perf/builtin-stat.c +3 -3
@@ -1322,7 +1322,7 @@ static int __cmd_record(int argc, const char **argv) PARSE_OPT_STOP_AT_NON_OPTION);if (output_name) - data->file.path = output_name; + data->path = output_name;if (stat_config.run_count !=1 || forever) {pr_err ("Cannot use -r option with perf stat record.\n" );
@@ -1523,8 +1523,8 @@ static int __cmd_report(int argc, const char **argv) input_name ="perf.data" ; } - perf_stat.data.file.path = input_name; - perf_stat.data.mode =PERF_DATA_MODE_READ ; + perf_stat.data.path = input_name; + perf_stat.data.mode =PERF_DATA_MODE_READ ; session =perf_session__new (&perf_stat.data, false, &perf_stat.tool);if (session ==NULL )
tools/perf/builtin-timechart.c +3 -5
@@ -1602,11 +1602,9 @@ static int __cmd_timechart(struct timechart *tchart, const char *output_name) { "syscalls:sys_exit_select", process_exit_poll }, };struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = tchart->force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = tchart->force, }; struct perf_session *session = perf_session__new(&data, false,
tools/perf/builtin-trace.c +3 -5
@@ -3154,11 +3154,9 @@ static int trace__replay(struct trace *trace) { "probe:vfs_getname", trace__vfs_getname, }, };struct perf_data data = { - .file = { - .path = input_name, - }, - .mode =PERF_DATA_MODE_READ , - .force = trace->force, + .path = input_name, + .mode =PERF_DATA_MODE_READ , + .force = trace->force, };struct perf_session *session;struct perf_evsel *evsel;
tools/perf/util/data-convert-bt.c +1 -1
@@ -1650,7 +1650,7 @@ int bt_convert__perf2ctf(const char *input, const char *path,fprintf (stderr,"[ perf data convert: Converted '%s' into CTF data '%s' ]\n" , - data.file.path, path); + data.path, path); fprintf(stderr, "[ perf data convert: Converted and wrote %.3f MB (%" PRIu64 " samples",
tools/perf/util/data.c +27 -12
@@ -19,11 +19,11 @@ static bool check_pipe(struct perf_data *data)int fd =perf_data__is_read (data) ?STDIN_FILENO :STDOUT_FILENO ; -if (!data->file.path) { +if (!data->path) {if (!fstat (fd, &st) &&S_ISFIFO (st.st_mode)) is_pipe = true; }else { -if (!strcmp (data->file.path,"-" )) +if (!strcmp (data->path,"-" )) is_pipe = true; }
@@ -37,13 +37,13 @@ static int check_backup(struct perf_data *data) {struct stat st; -if (!stat (data->file.path, &st) && st.st_size) { +if (!stat (data->path, &st) && st.st_size) {/* TODO check errors properly */ char oldname[PATH_MAX ];snprintf (oldname,sizeof (oldname),"%s.old" , - data->file.path); + data->path);unlink (oldname); -rename (data->file.path, oldname); +rename (data->path, oldname); }return 0 ;
@@ -115,8 +115,22 @@ static int open_file(struct perf_data *data) fd =perf_data__is_read (data) ?open_file_read (data) :open_file_write (data); +if (fd <0 ) { +free (data->file.path); +return -1 ; + } + data->file.fd = fd; -return fd <0 ?-1 :0 ; +return 0 ; +} + +static int open_file_dup (struct perf_data *data) +{ + data->file.path =strdup (data->path); +if (!data->file.path) +return -ENOMEM ; + +return open_file (data); }int perf_data__open (struct perf_data *data)
@@ -124,14 +138,15 @@ int perf_data__open(struct perf_data *data)if (check_pipe (data))return 0 ; -if (!data->file.path) - data->file.path ="perf.data" ; +if (!data->path) + data->path ="perf.data" ; -return open_file (data); +return open_file_dup (data); }void perf_data__close (struct perf_data *data) { +free (data->file.path);close (data->file.fd); }
@@ -159,15 +174,15 @@ int perf_data__switch(struct perf_data *data,if (perf_data__is_read (data))return -EINVAL ; -if (asprintf (&new_filepath,"%s.%s" , data->file.path, postfix) <0 ) +if (asprintf (&new_filepath,"%s.%s" , data->path, postfix) <0 )return -ENOMEM ;/* * Only fire a warning, don't return error, continue fill * original file. */ -if (rename (data->file.path, new_filepath)) -pr_warning ("Failed to rename %s to %s\n" , data->file.path, new_filepath); +if (rename (data->path, new_filepath)) +pr_warning ("Failed to rename %s to %s\n" , data->path, new_filepath);if (!at_exit) {close (data->file.fd);
tools/perf/util/data.h +2 -1
@@ -10,12 +10,13 @@ enum perf_data_mode { };struct perf_data_file { -const char *path; +char *path;int fd;unsigned long size; };struct perf_data { +const char *path;struct perf_data_file file;bool is_pipe;bool force;
Signature · 1 line
2.17.2
Jiri Olsa
reviewerPATCH v3 6/20 perf data: Make check_backup work over directories
In reply to Jiri OlsaChanging check_backup to call rm_rf_perf_data instead of unlink to work over directory paths. Also moving the call earlier in the code, before we fork for file/dir, so it can backup also directory data.
--- tools/perf/util/data.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
tools/perf/util/data.c +7 -4
@@ -37,12 +37,15 @@ static int check_backup(struct perf_data *data) {struct stat st; +if (perf_data__is_read (data)) +return 0 ; +if (!stat (data->path, &st) && st.st_size) {/* TODO check errors properly */ char oldname[PATH_MAX ];snprintf (oldname,sizeof (oldname),"%s.old" , data->path); -unlink (oldname); +rm_rf_perf_data (oldname);rename (data->path, oldname); }
@@ -95,9 +98,6 @@ static int open_file_write(struct perf_data *data)int fd;char sbuf[STRERR_BUFSIZE ]; -if (check_backup (data)) -return -1 ; - fd =open (data->file.path,O_CREAT |O_RDWR |O_TRUNC |O_CLOEXEC ,S_IRUSR |S_IWUSR );
@@ -141,6 +141,9 @@ int perf_data__open(struct perf_data *data)if (!data->path) data->path ="perf.data" ; +if (check_backup (data)) +return -1 ; +return open_file_dup (data); }
Signature · 1 line
2.17.2
tip-bot for Jiri Olsa
merge bot[tip:perf/core] perf data: Make check_backup work over directories
perf data: Make check_backup work over directories
Commit-ID: 5021fc4e8c7c3be6f8735eff76fb2520485fcec4 Gitweb: https://git.kernel.org/tip/5021fc4e8c7c3be6f8735eff76fb2520485fcec4 Author: Jiri Olsa <[email protected]> AuthorDate: Sun, 24 Feb 2019 20:06:42 +0100 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Mon, 25 Feb 2019 10:35:19 -0300
perf data: Make check_backup work over directories
Change check_backup() to call rm_rf_perf_data() instead of unlink() to work over directory paths.
Also move the call earlier in the code, before we fork for file/dir, so it can backup also directory data.
--- tools/perf/util/data.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
tools/perf/util/data.c +7 -4
@@ -37,12 +37,15 @@ static int check_backup(struct perf_data *data) {struct stat st; +if (perf_data__is_read (data)) +return 0 ; +if (!stat (data->path, &st) && st.st_size) {/* TODO check errors properly */ char oldname[PATH_MAX ];snprintf (oldname,sizeof (oldname),"%s.old" , data->path); -unlink (oldname); +rm_rf_perf_data (oldname);rename (data->path, oldname); }
@@ -95,9 +98,6 @@ static int open_file_write(struct perf_data *data)int fd;char sbuf[STRERR_BUFSIZE ]; -if (check_backup (data)) -return -1 ; - fd =open (data->file.path,O_CREAT |O_RDWR |O_TRUNC |O_CLOEXEC ,S_IRUSR |S_IWUSR );
@@ -141,6 +141,9 @@ int perf_data__open(struct perf_data *data)if (!data->path) data->path ="perf.data" ; +if (check_backup (data)) +return -1 ; +return open_file_dup (data); }
Jiri Olsa
reviewerPATCH v3 7/20 perf data: Fail check_backup in case of error
In reply to Jiri OlsaAnd display the error message from removing
the old data file:
$ perf record ls Can't remove old data: Permission denied (perf.data.old) Perf session creation failed.
$ perf record ls Can't remove old data: Unknown file found (perf.data.old) Perf session creation failed.
Not sure how to make fail the rename (after we successfully remove the destination file/dir) to show the message, anyway let's have it there.
--- tools/perf/util/data.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-)
tools/perf/util/data.c +18 -3
@@ -41,12 +41,27 @@ static int check_backup(struct perf_data *data)return 0 ;if (!stat (data->path, &st) && st.st_size) { -/* TODO check errors properly */ char oldname[PATH_MAX ]; +int ret; +snprintf (oldname,sizeof (oldname),"%s.old" , data->path); -rm_rf_perf_data (oldname); -rename (data->path, oldname); + + ret =rm_rf_perf_data (oldname); +if (ret) { +pr_err ("Can't remove old data: %s (%s)\n" , + ret ==-2 ? +"Unknown file found" :strerror (errno), + oldname); +return -1 ; + } + +if (rename (data->path, oldname)) { +pr_err ("Can't move data: %s (%s to %s)\n" , +strerror (errno), + data->path, oldname); +return -1 ; + } }return 0 ;
Signature · 1 line
2.17.2
tip-bot for Jiri Olsa
merge bot[tip:perf/core] perf data: Fail check_backup in case of error
perf data: Fail check_backup in case of error
Commit-ID: ccb7a71dcea071c7fd68192909c4e54561c88043 Gitweb: https://git.kernel.org/tip/ccb7a71dcea071c7fd68192909c4e54561c88043 Author: Jiri Olsa <[email protected]> AuthorDate: Sun, 24 Feb 2019 20:06:43 +0100 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Mon, 25 Feb 2019 10:37:01 -0300
perf data: Fail check_backup in case of error
And display the error message from removing the old data file:
$ perf record ls Can't remove old data: Permission denied (perf.data.old) Perf session creation failed.
$ perf record ls Can't remove old data: Unknown file found (perf.data.old) Perf session creation failed.
Not sure how to make fail the rename (after we successfully remove the destination file/dir) to show the message, anyway let's have it there.
--- tools/perf/util/data.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-)
tools/perf/util/data.c +18 -3
@@ -41,12 +41,27 @@ static int check_backup(struct perf_data *data)return 0 ;if (!stat (data->path, &st) && st.st_size) { -/* TODO check errors properly */ char oldname[PATH_MAX ]; +int ret; +snprintf (oldname,sizeof (oldname),"%s.old" , data->path); -rm_rf_perf_data (oldname); -rename (data->path, oldname); + + ret =rm_rf_perf_data (oldname); +if (ret) { +pr_err ("Can't remove old data: %s (%s)\n" , + ret ==-2 ? +"Unknown file found" :strerror (errno), + oldname); +return -1 ; + } + +if (rename (data->path, oldname)) { +pr_err ("Can't move data: %s (%s to %s)\n" , +strerror (errno), + data->path, oldname); +return -1 ; + } }return 0 ;
Jiri Olsa
reviewerPATCH v3 8/20 perf data: Add perf_data__(create_dir|close_dir) functions
In reply to Jiri OlsaAdding perf_data__create_dir to create nr files inside struct perf_data path directory:
int perf_data__create_dir(struct perf_data *data, int nr);
and function to close that data:
void perf_data__close_dir(struct perf_data *data);
--- tools/perf/util/data.c | 47 ++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data.h | 8 +++++++ 2 files changed, 55 insertions(+)
tools/perf/util/data.c +47 -0
@@ -7,11 +7,58 @@#include <fcntl.h> #include <unistd.h> #include <string.h> +#include <asm/bug.h> #include "data.h" #include "util.h" #include "debug.h" +static void close_dir (struct perf_data_file *files,int nr) +{ +while (--nr >=1 ) { +close (files[nr].fd); +free (files[nr].path); + } +free (files); +} + +void perf_data__close_dir (struct perf_data *data) +{ +close_dir (data->dir.files, data->dir.nr); +} + +int perf_data__create_dir (struct perf_data *data,int nr) +{ +struct perf_data_file *files =NULL ; +int i, ret =-1 ; + + files =zalloc (nr *sizeof (*files)); +if (!files) +return -ENOMEM ; + + data->dir.files = files; + data->dir.nr = nr; + +for (i =0 ; i < nr; i++) { +struct perf_data_file *file = &files[i]; + +if (asprintf (&file->path,"%s/data.%d" , data->path, i) <0 ) + goto out_err; + + ret =open (file->path,O_RDWR |O_CREAT |O_TRUNC ,S_IRUSR |S_IWUSR ); +if (ret <0 ) + goto out_err; + + file->fd = ret; + } + +return 0 ; + +out_err: +close_dir (files, i); +return ret; +} +static bool check_pipe (struct perf_data *data) {struct stat st;
tools/perf/util/data.h +8 -0
@@ -21,6 +21,11 @@ struct perf_data {bool is_pipe;bool force;enum perf_data_mode mode; + +struct { +struct perf_data_file *files; +int nr; + } dir; };static inline bool perf_data__is_read (struct perf_data *data)
@@ -64,4 +69,7 @@ ssize_t perf_data_file__write(struct perf_data_file *file,int perf_data__switch (struct perf_data *data,const char *postfix,size_t pos,bool at_exit); + +int perf_data__create_dir (struct perf_data *data,int nr); +void perf_data__close_dir (struct perf_data *data);#endif /* __PERF_DATA_H */
Signature · 1 line
2.17.2
tip-bot for Jiri Olsa
merge bot[tip:perf/core] perf data: Add perf_data__(create_dir|close_dir) functions
perf data: Add perf_data__(create_dir|close_dir) functions
Commit-ID: 145520631130bd64820b591775733256473eac62 Gitweb: https://git.kernel.org/tip/145520631130bd64820b591775733256473eac62 Author: Jiri Olsa <[email protected]> AuthorDate: Sun, 24 Feb 2019 20:06:44 +0100 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Mon, 25 Feb 2019 10:42:05 -0300
perf data: Add perf_data__(create_dir|close_dir) functions
Add perf_data__create_dir() to create nr files inside 'struct perf_data' path directory:
int perf_data__create_dir(struct perf_data *data, int nr);
and function to close that data:
void perf_data__close_dir(struct perf_data *data);
--- tools/perf/util/data.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data.h | 8 ++++++++ 2 files changed, 55 insertions(+)
tools/perf/util/data.c +47 -0
@@ -7,11 +7,58 @@#include <fcntl.h> #include <unistd.h> #include <string.h> +#include <asm/bug.h> #include "data.h" #include "util.h" #include "debug.h" +static void close_dir (struct perf_data_file *files,int nr) +{ +while (--nr >=1 ) { +close (files[nr].fd); +free (files[nr].path); + } +free (files); +} + +void perf_data__close_dir (struct perf_data *data) +{ +close_dir (data->dir.files, data->dir.nr); +} + +int perf_data__create_dir (struct perf_data *data,int nr) +{ +struct perf_data_file *files =NULL ; +int i, ret =-1 ; + + files =zalloc (nr *sizeof (*files)); +if (!files) +return -ENOMEM ; + + data->dir.files = files; + data->dir.nr = nr; + +for (i =0 ; i < nr; i++) { +struct perf_data_file *file = &files[i]; + +if (asprintf (&file->path,"%s/data.%d" , data->path, i) <0 ) + goto out_err; + + ret =open (file->path,O_RDWR |O_CREAT |O_TRUNC ,S_IRUSR |S_IWUSR ); +if (ret <0 ) + goto out_err; + + file->fd = ret; + } + +return 0 ; + +out_err: +close_dir (files, i); +return ret; +} +static bool check_pipe (struct perf_data *data) {struct stat st;
tools/perf/util/data.h +8 -0
@@ -21,6 +21,11 @@ struct perf_data {bool is_pipe;bool force;enum perf_data_mode mode; + +struct { +struct perf_data_file *files; +int nr; + } dir; };static inline bool perf_data__is_read (struct perf_data *data)
@@ -64,4 +69,7 @@ ssize_t perf_data_file__write(struct perf_data_file *file,int perf_data__switch (struct perf_data *data,const char *postfix,size_t pos,bool at_exit); + +int perf_data__create_dir (struct perf_data *data,int nr); +void perf_data__close_dir (struct perf_data *data);#endif /* __PERF_DATA_H */
Jiri Olsa
reviewerPATCH v3 9/20 perf data: Add perf_data__open_dir_data function
In reply to Jiri OlsaAdding perf_data__open_dir_data to open files inside struct perf_data path directory:
static int perf_data__open_dir(struct perf_data *data);
--- tools/perf/util/data.c | 59 ++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data.h | 1 + 2 files changed, 60 insertions(+)
tools/perf/util/data.c +59 -0
@@ -8,6 +8,8 @@#include <unistd.h> #include <string.h> #include <asm/bug.h> +#include <sys/types.h> +#include <dirent.h> #include "data.h" #include "util.h"
@@ -59,6 +61,63 @@ int perf_data__create_dir(struct perf_data *data, int nr)return ret; } +int perf_data__open_dir (struct perf_data *data) +{ +struct perf_data_file *files =NULL ; +struct dirent *dent; +int ret =-1 ; +DIR *dir; +int nr =0 ; + + dir =opendir (data->path); +if (!dir) +return -EINVAL ; + +while ((dent =readdir (dir)) !=NULL ) { +struct perf_data_file *file; +char path[PATH_MAX ]; +struct stat st; + +snprintf (path,sizeof (path),"%s/%s" , data->path, dent->d_name); +if (stat (path, &st)) +continue ; + +if (!S_ISREG (st.st_mode) ||strncmp (dent->d_name,"data" ,4 )) +continue ; + + ret = -ENOMEM ; + + file =realloc (files, (nr +1 ) *sizeof (*files)); +if (!file) + goto out_err; + + files = file; + file = &files[nr++]; + + file->path =strdup (path); +if (!file->path) + goto out_err; + + ret =open (file->path,O_RDONLY ); +if (ret <0 ) + goto out_err; + + file->fd = ret; + file->size = st.st_size; + } + +if (!files) +return -EINVAL ; + + data->dir.files = files; + data->dir.nr = nr; +return 0 ; + +out_err: +close_dir (files, nr); +return ret; +} +static bool check_pipe (struct perf_data *data) {struct stat st;
tools/perf/util/data.h +1 -0
@@ -71,5 +71,6 @@ int perf_data__switch(struct perf_data *data, size_t pos, bool at_exit);int perf_data__create_dir (struct perf_data *data,int nr); +int perf_data__open_dir (struct perf_data *data);void perf_data__close_dir (struct perf_data *data);#endif /* __PERF_DATA_H */
Signature · 1 line
2.17.2
tip-bot for Jiri Olsa
merge bot[tip:perf/core] perf data: Add perf_data__open_dir_data function
perf data: Add perf_data__open_dir_data function
Commit-ID: eb6176709b235b96511c7b4745c30412568395c7 Gitweb: https://git.kernel.org/tip/eb6176709b235b96511c7b4745c30412568395c7 Author: Jiri Olsa <[email protected]> AuthorDate: Sun, 24 Feb 2019 20:06:45 +0100 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Mon, 25 Feb 2019 10:43:07 -0300
perf data: Add perf_data__open_dir_data function
Add perf_data__open_dir_data to open files inside 'struct perf_data' path directory:
static int perf_data__open_dir(struct perf_data *data);
--- tools/perf/util/data.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/data.h | 1 + 2 files changed, 60 insertions(+)
tools/perf/util/data.c +59 -0
@@ -8,6 +8,8 @@#include <unistd.h> #include <string.h> #include <asm/bug.h> +#include <sys/types.h> +#include <dirent.h> #include "data.h" #include "util.h"
@@ -59,6 +61,63 @@ out_err:return ret; } +int perf_data__open_dir (struct perf_data *data) +{ +struct perf_data_file *files =NULL ; +struct dirent *dent; +int ret =-1 ; +DIR *dir; +int nr =0 ; + + dir =opendir (data->path); +if (!dir) +return -EINVAL ; + +while ((dent =readdir (dir)) !=NULL ) { +struct perf_data_file *file; +char path[PATH_MAX ]; +struct stat st; + +snprintf (path,sizeof (path),"%s/%s" , data->path, dent->d_name); +if (stat (path, &st)) +continue ; + +if (!S_ISREG (st.st_mode) ||strncmp (dent->d_name,"data" ,4 )) +continue ; + + ret = -ENOMEM ; + + file =realloc (files, (nr +1 ) *sizeof (*files)); +if (!file) + goto out_err; + + files = file; + file = &files[nr++]; + + file->path =strdup (path); +if (!file->path) + goto out_err; + + ret =open (file->path,O_RDONLY ); +if (ret <0 ) + goto out_err; + + file->fd = ret; + file->size = st.st_size; + } + +if (!files) +return -EINVAL ; + + data->dir.files = files; + data->dir.nr = nr; +return 0 ; + +out_err: +close_dir (files, nr); +return ret; +} +static bool check_pipe (struct perf_data *data) {struct stat st;
tools/perf/util/data.h +1 -0
@@ -71,5 +71,6 @@ int perf_data__switch(struct perf_data *data, size_t pos, bool at_exit);int perf_data__create_dir (struct perf_data *data,int nr); +int perf_data__open_dir (struct perf_data *data);void perf_data__close_dir (struct perf_data *data);#endif /* __PERF_DATA_H */
Jiri Olsa
reviewerPATCH v3 10/20 perf data: Add directory support
In reply to Jiri OlsaAdding support to have directory as perf.data.
The caller needs to set 'struct perf_data::is_dir flag and the path will be treated as directory.
The 'struct perf_data::file' is initialized and open as 'path/header' file.
Adding check to direcory interface functions to check on is_dir flag.
--- tools/perf/util/data.c | 41 ++++++++++++++++++++++++++++++++++++++- tools/perf/util/data.h | 6 ++++++ tools/perf/util/session.c | 4 ++++ 3 files changed, 50 insertions(+), 1 deletion(-)
tools/perf/util/data.c +40 -1
@@ -34,6 +34,9 @@ int perf_data__create_dir(struct perf_data *data, int nr)struct perf_data_file *files =NULL ;int i, ret =-1 ; +if (WARN_ON (!data->is_dir)) +return -EINVAL ; + files =zalloc (nr *sizeof (*files));if (!files)return -ENOMEM ;
@@ -69,6 +72,9 @@ int perf_data__open_dir(struct perf_data *data)DIR *dir;int nr =0 ; +if (WARN_ON (!data->is_dir)) +return -EINVAL ; + dir =opendir (data->path);if (!dir)return -EINVAL ;
@@ -173,6 +179,16 @@ static int check_backup(struct perf_data *data)return 0 ; } +static bool is_dir (struct perf_data *data) +{ +struct stat st; + +if (stat (data->path, &st)) +return false; + +return (st.st_mode &S_IFMT ) ==S_IFDIR ; +} +static int open_file_read (struct perf_data *data) {struct stat st;
@@ -254,6 +270,22 @@ static int open_file_dup(struct perf_data *data)return open_file (data); } +static int open_dir (struct perf_data *data) +{ +if (perf_data__is_write (data) && +mkdir (data->path,S_IRWXU ) <0 ) +return -1 ; + +/* +* So far we open only the header, so we +* can read the data version and layout. +*/ +if (asprintf (&data->file.path,"%s/header" , data->path) <0 ) +return -ENOMEM ; + +return open_file (data); +} +int perf_data__open (struct perf_data *data) {if (check_pipe (data))
@@ -265,11 +297,18 @@ int perf_data__open(struct perf_data *data)if (check_backup (data))return -1 ; -return open_file_dup (data); +if (perf_data__is_read (data)) + data->is_dir =is_dir (data); + +return perf_data__is_dir (data) ? +open_dir (data) :open_file_dup (data); }void perf_data__close (struct perf_data *data) { +if (perf_data__is_dir (data)) +perf_data__close_dir (data); +free (data->file.path);close (data->file.fd); }
tools/perf/util/data.h +6 -0
@@ -19,6 +19,7 @@ struct perf_data {const char *path;struct perf_data_file file;bool is_pipe; +bool is_dir;bool force;enum perf_data_mode mode;
@@ -43,6 +44,11 @@ static inline int perf_data__is_pipe(struct perf_data *data)return data->is_pipe; } +static inline bool perf_data__is_dir (struct perf_data *data) +{ +return data->is_dir; +} +static inline int perf_data__fd (struct perf_data *data) {return data->file.fd;
tools/perf/util/session.c +4 -0
@@ -152,6 +152,10 @@ struct perf_session *perf_session__new(struct perf_data *data, }perf_evlist__init_trace_event_sample_raw (session->evlist); + +/* Open the directory data. */ +if (data->is_dir &&perf_data__open_dir (data)) + goto out_close; } }else { session->machines.host.env = &perf_env;
Signature · 1 line
2.17.2
Re: [PATCH 10/20] perf data: Add directory support
Em Sun, Feb 24, 2019 at 08:06:46PM +0100, Jiri Olsa escreveu:
Quoted 66 lines
Adding support to have directory as perf.data.
The caller needs to set 'struct
perf_data::is_dirflag and the path will be treated as directory.The 'struct
perf_data::file' is initialized and open as 'path/header' file.Adding check to direcory interface functions to check on
is_dirflag.Signed-off-by Jiri Olsa <[email protected]>--- tools/perf/util/data.c | 41 ++++++++++++++++++++++++++++++++++++++- tools/perf/util/data.h | 6 ++++++ tools/perf/util/session.c | 4 ++++ 3 files changed, 50 insertions(+), 1 deletion(-)tools/perf/util/data.c +18
-0@@ -34,6 +34,9 @@ int perf_data__create_dir(struct perf_data *data, int nr)struct perf_data_file *files =NULL ;int i, ret =-1 ; +if (WARN_ON (!data->is_dir)) +return -EINVAL ; + files =zalloc (nr *sizeof (*files));if (!files)return -ENOMEM ;@@ -69,6 +72,9 @@ int perf_data__open_dir(struct perf_data *data)DIR *dir;int nr =0 ; +if (WARN_ON (!data->is_dir)) +return -EINVAL ; + dir =opendir (data->path);if (!dir)return -EINVAL ;@@ -173,6 +179,16 @@ static int check_backup(struct perf_data *data)return 0 ; } +static bool is_dir (struct perf_data *data) +{ +struct stat st; + +if (stat (data->path, &st)) +return false; + +return (st.st_mode &S_IFMT ) ==S_IFDIR ; +} +static int open_file_read (struct perf_data *data) {struct stat st;@@ -254,6 +270,22 @@ static int open_file_dup(struct perf_data *data)return open_file (data); } +static int open_dir (struct perf_data *data) +{
tools/perf/util/data.c static int open_file_dup(struct perf_data *data) · line 284
Quoted 10 lines
tools/perf/util/data.c +10
-0+if (perf_data__is_write (data) && +mkdir (data->path,S_IRWXU ) <0 ) +return -1 ; + +/* +* So far we open only the header, so we +* can read the data version and layout. +*/ +if (asprintf (&data->file.path,"%s/header" , data->path) <0 ) +return -ENOMEM ;
so, if this fails, then we should unwind the mkdir, if it was performed, so that we leave things as they were before calling open_dir(), right?
I processed everything up to here.
- Arnaldo
Quoted 68 lines
tools/perf/util/data.c +12
-1+ +return open_file (data); +} +int perf_data__open (struct perf_data *data) {if (check_pipe (data))@@ -265,11 +297,18 @@ int perf_data__open(struct perf_data *data)if (check_backup (data))return -1 ; -return open_file_dup (data); +if (perf_data__is_read (data)) + data->is_dir =is_dir (data); + +return perf_data__is_dir (data) ? +open_dir (data) :open_file_dup (data); }void perf_data__close (struct perf_data *data) { +if (perf_data__is_dir (data)) +perf_data__close_dir (data); +free (data->file.path);close (data->file.fd); }tools/perf/util/data.h +6
-0@@ -19,6 +19,7 @@ struct perf_data {const char *path;struct perf_data_file file;bool is_pipe; +bool is_dir;bool force;enum perf_data_mode mode;@@ -43,6 +44,11 @@ static inline int perf_data__is_pipe(struct perf_data *data)return data->is_pipe; } +static inline bool perf_data__is_dir (struct perf_data *data) +{ +return data->is_dir; +} +static inline int perf_data__fd (struct perf_data *data) {return data->file.fd;tools/perf/util/session.c +4
-0@@ -152,6 +152,10 @@ struct perf_session *perf_session__new(struct perf_data *data, }perf_evlist__init_trace_event_sample_raw (session->evlist); + +/* Open the directory data. */ +if (data->is_dir &&perf_data__open_dir (data)) + goto out_close; } }else { session->machines.host.env = &perf_env;Signature · 1 line
2.17.2
Signature · 1 line
- Arnaldo
Re: [PATCH 10/20] perf data: Add directory support
Quoted 65 lines · Arnaldo Carvalho de Melo
Em Sun, Feb 24, 2019 at 08:06:46PM +0100, Jiri Olsa escreveu:
Quoted 64 lines
Adding support to have directory as perf.data.
The caller needs to set 'struct
perf_data::is_dirflag and the path will be treated as directory.The 'struct
perf_data::file' is initialized and open as 'path/header' file.Adding check to direcory interface functions to check on
is_dirflag.Signed-off-by Jiri Olsa <[email protected]>--- tools/perf/util/data.c | 41 ++++++++++++++++++++++++++++++++++++++- tools/perf/util/data.h | 6 ++++++ tools/perf/util/session.c | 4 ++++ 3 files changed, 50 insertions(+), 1 deletion(-)tools/perf/util/data.c +16
-0@@ -34,6 +34,9 @@ int perf_data__create_dir(struct perf_data *data, int nr)struct perf_data_file *files =NULL ;int i, ret =-1 ; +if (WARN_ON (!data->is_dir)) +return -EINVAL ; + files =zalloc (nr *sizeof (*files));if (!files)return -ENOMEM ;@@ -69,6 +72,9 @@ int perf_data__open_dir(struct perf_data *data)DIR *dir;int nr =0 ; +if (WARN_ON (!data->is_dir)) +return -EINVAL ; + dir =opendir (data->path);if (!dir)return -EINVAL ;@@ -173,6 +179,16 @@ static int check_backup(struct perf_data *data)return 0 ; } +static bool is_dir (struct perf_data *data) +{ +struct stat st; + +if (stat (data->path, &st)) +return false; + +return (st.st_mode &S_IFMT ) ==S_IFDIR ; +} +static int open_file_read (struct perf_data *data) {struct stat st;@@ -254,6 +270,22 @@ static int open_file_dup(struct perf_data *data)return open_file (data); }
Quoted 16 lines
tools/perf/util/data.c static int open_file_dup(struct perf_data *data) · line 284
Quoted 12 lines
tools/perf/util/data.c +12
-0+static int open_dir (struct perf_data *data) +{ +if (perf_data__is_write (data) && +mkdir (data->path,S_IRWXU ) <0 ) +return -1 ; + +/* +* So far we open only the header, so we +* can read the data version and layout. +*/ +if (asprintf (&data->file.path,"%s/header" , data->path) <0 ) +return -ENOMEM ;so, if this fails, then we should unwind the mkdir, if it was performed, so that we leave things as they were before calling
open_dir(), right?
I think we need to some global solution on this, currently we also leve halfway screwed perf.data if we fail in the middle
I think maybe we need to add something on upper (session) layer to cleanup if we screw up
jirka
Re: [PATCH 10/20] perf data: Add directory support
Em Mon, Feb 25, 2019 at 02:56:51PM +0100, Jiri Olsa escreveu:
Quoted 83 lines · Arnaldo Carvalho de Melo
Quoted 65 lines
Em Sun, Feb 24, 2019 at 08:06:46PM +0100, Jiri Olsa escreveu:
Quoted 64 lines
Adding support to have directory as perf.data.
The caller needs to set 'struct
perf_data::is_dirflag and the path will be treated as directory.The 'struct
perf_data::file' is initialized and open as 'path/header' file.Adding check to direcory interface functions to check on
is_dirflag.Signed-off-by Jiri Olsa <[email protected]>--- tools/perf/util/data.c | 41 ++++++++++++++++++++++++++++++++++++++- tools/perf/util/data.h | 6 ++++++ tools/perf/util/session.c | 4 ++++ 3 files changed, 50 insertions(+), 1 deletion(-)tools/perf/util/data.c +16
-0@@ -34,6 +34,9 @@ int perf_data__create_dir(struct perf_data *data, int nr)struct perf_data_file *files =NULL ;int i, ret =-1 ; +if (WARN_ON (!data->is_dir)) +return -EINVAL ; + files =zalloc (nr *sizeof (*files));if (!files)return -ENOMEM ;@@ -69,6 +72,9 @@ int perf_data__open_dir(struct perf_data *data)DIR *dir;int nr =0 ; +if (WARN_ON (!data->is_dir)) +return -EINVAL ; + dir =opendir (data->path);if (!dir)return -EINVAL ;@@ -173,6 +179,16 @@ static int check_backup(struct perf_data *data)return 0 ; } +static bool is_dir (struct perf_data *data) +{ +struct stat st; + +if (stat (data->path, &st)) +return false; + +return (st.st_mode &S_IFMT ) ==S_IFDIR ; +} +static int open_file_read (struct perf_data *data) {struct stat st;@@ -254,6 +270,22 @@ static int open_file_dup(struct perf_data *data)return open_file (data); }Quoted 16 lines
tools/perf/util/data.c static int open_file_dup(struct perf_data *data) · line 284
Quoted 12 lines
tools/perf/util/data.c +12
-0+static int open_dir (struct perf_data *data) +{ +if (perf_data__is_write (data) && +mkdir (data->path,S_IRWXU ) <0 ) +return -1 ; + +/* +* So far we open only the header, so we +* can read the data version and layout. +*/ +if (asprintf (&data->file.path,"%s/header" , data->path) <0 ) +return -ENOMEM ;so, if this fails, then we should unwind the mkdir, if it was performed, so that we leave things as they were before calling
open_dir(), right?
Quoted 1 line
I think we need to some global solution on this,
If we don't add more, that would be a good thing :-)
The other parts also need to be investigated to see what is best in that case, but here, undoing the mkdir() if the asprintf() fails is the right thing to do :-)
- Arnaldo
Quoted 5 lines
currently we also leve halfway screwed perf.data
if we fail in the middleI think maybe we need to add something on upper (session) layer to cleanup if we screw up
B
Re: [PATCH 10/20] perf data: Add directory support
On Mon, Feb 25, 2019 at 12:07:22PM -0300, Arnaldo Carvalho de Melo wrote:
SNIP
Quoted 26 lines
Quoted 26 lines
Quoted 25 lines
Quoted 11 lines
tools/perf/util/data.c +4
-0+} +static int open_file_read (struct perf_data *data) {struct stat st;@@ -254,6 +270,22 @@ static int open_file_dup(struct perf_data *data)return open_file (data); } +static int open_dir (struct perf_data *data) +{tools/perf/util/data.c static int open_file_dup(struct perf_data *data) · line 284
Quoted 10 lines
tools/perf/util/data.c +10
-0+if (perf_data__is_write (data) && +mkdir (data->path,S_IRWXU ) <0 ) +return -1 ; + +/* +* So far we open only the header, so we +* can read the data version and layout. +*/ +if (asprintf (&data->file.path,"%s/header" , data->path) <0 ) +return -ENOMEM ;so, if this fails, then we should unwind the mkdir, if it was performed, so that we leave things as they were before calling
open_dir(), right?
Quoted 7 lines
Quoted 1 line
I think we need to some global solution on this,
If we don't add more, that would be a good thing :-)
The other parts also need to be investigated to see what is best in that case, but here, undoing the
mkdir()if theasprintf()fails is the right thing to do :-)
true ;-) will resend
thanks,
jirka
Jiri Olsa
reviewerPATCH v3 11/20 perf data: Don't store auxtrace index for directory data file
In reply to Jiri OlsaWe can't store auxtrace index when we store to multiple files, because we keep only offset for it, not the file.
The auxtrace data will be processed correctly in the 'pipe' mode.
--- tools/perf/builtin-record.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/builtin-record.c +1 -1
@@ -392,7 +392,7 @@ static int record__process_auxtrace(struct perf_tool *tool,size_t padding;u8 pad[8 ] = {0 }; -if (!perf_data__is_pipe (data)) { +if (!perf_data__is_pipe (data) && !perf_data__is_dir (data)) {off_t file_offset;int fd =perf_data__fd (data);int err;
Signature · 1 line
2.17.2
Jiri Olsa
reviewerPATCH v3 12/20 perf data: Add perf_data__update_dir function
In reply to Jiri OlsaAdding perf_data__update_dir function to update size for every file within the perf.data directory.
--- tools/perf/util/data.c | 20 ++++++++++++++++++++ tools/perf/util/data.h | 1 + 2 files changed, 21 insertions(+)
tools/perf/util/data.c +20 -0
@@ -124,6 +124,26 @@ int perf_data__open_dir(struct perf_data *data)return ret; } +int perf_data__update_dir (struct perf_data *data) +{ +int i; + +if (WARN_ON (!data->is_dir)) +return -EINVAL ; + +for (i =0 ; i < data->dir.nr; i++) { +struct perf_data_file *file = &data->dir.files[i]; +struct stat st; + +if (fstat (file->fd, &st)) +return -1 ; + + file->size = st.st_size; + } + +return 0 ; +} +static bool check_pipe (struct perf_data *data) {struct stat st;
tools/perf/util/data.h +1 -0
@@ -79,4 +79,5 @@ int perf_data__switch(struct perf_data *data,int perf_data__create_dir (struct perf_data *data,int nr);int perf_data__open_dir (struct perf_data *data);void perf_data__close_dir (struct perf_data *data); +int perf_data__update_dir (struct perf_data *data);#endif /* __PERF_DATA_H */
Signature · 1 line
2.17.2
Jiri Olsa
reviewerPATCH v3 13/20 perf data: Make perf_data__size to work over directory
In reply to Jiri OlsaMaking perf_data__size to return proper size
for directory data.
--- tools/perf/util/data.c | 17 +++++++++++++++++ tools/perf/util/data.h | 6 +----- 2 files changed, 18 insertions(+), 5 deletions(-)
tools/perf/util/data.c +17 -0
@@ -385,3 +385,20 @@ int perf_data__switch(struct perf_data *data,free (new_filepath);return ret; } + +unsigned long perf_data__size (struct perf_data *data) +{ +u64 size = data->file.size; +int i; + +if (!data->is_dir) +return size; + +for (i =0 ; i < data->dir.nr; i++) { +struct perf_data_file *file = &data->dir.files[i]; + + size += file->size; + } + +return size; +}
tools/perf/util/data.h +1 -5
@@ -54,11 +54,6 @@ static inline int perf_data__fd(struct perf_data *data)return data->file.fd; } -static inline unsigned long perf_data__size (struct perf_data *data) -{ -return data->file.size; -} -int perf_data__open (struct perf_data *data);void perf_data__close (struct perf_data *data); ssize_t perf_data__write(struct perf_data *data,
@@ -80,4 +75,5 @@ int perf_data__create_dir(struct perf_data *data, int nr);int perf_data__open_dir (struct perf_data *data);void perf_data__close_dir (struct perf_data *data);int perf_data__update_dir (struct perf_data *data); +unsigned long perf_data__size (struct perf_data *data);#endif /* __PERF_DATA_H */
Signature · 1 line
2.17.2
Jiri Olsa
reviewerPATCH v3 14/20 perf header: Add DIR_FORMAT feature to describe directory data
In reply to Jiri OlsaThe data files layout is described by HEADER_DIR_FORMAT feature. Currently it holds only version number (1):
uint64_t version;
The current version holds only version value (1) means that data files:
- follow the 'data.*' name format
- contain raw events data in standard perf format as read from kernel (and need to be sorted)
Future versions are expected to describe different data files layout according to special needs.
--- tools/perf/builtin-record.c | 2 ++ tools/perf/util/data.c | 10 +++++++-- tools/perf/util/data.h | 1 + tools/perf/util/header.c | 44 ++++++++++++++++++++++++++++++++++++- tools/perf/util/header.h | 5 +++++ 5 files changed, 59 insertions(+), 3 deletions(-)
tools/perf/builtin-record.c +2 -0
@@ -837,6 +837,8 @@ static void record__init_features(struct record *rec)if (!(rec->opts.use_clockid && rec->opts.clockid_res_ns))perf_header__clear_feat (&session->header,HEADER_CLOCKID ); +perf_header__clear_feat (&session->header,HEADER_DIR_FORMAT ); +perf_header__clear_feat (&session->header,HEADER_STAT ); }
tools/perf/util/data.c +8 -2
@@ -14,6 +14,7 @@#include "data.h" #include "util.h" #include "debug.h" +#include "header.h" static void close_dir (struct perf_data_file *files,int nr) {
@@ -41,8 +42,9 @@ int perf_data__create_dir(struct perf_data *data, int nr)if (!files)return -ENOMEM ; - data->dir.files = files; - data->dir.nr = nr; + data->dir.version =PERF_DIR_VERSION ; + data->dir.files = files; + data->dir.nr = nr;for (i =0 ; i < nr; i++) {struct perf_data_file *file = &files[i];
@@ -75,6 +77,10 @@ int perf_data__open_dir(struct perf_data *data)if (WARN_ON (!data->is_dir))return -EINVAL ; +/* The version is provided by DIR_FORMAT feature. */ +if (WARN_ON (data->dir.version !=PERF_DIR_VERSION )) +return -1 ; + dir =opendir (data->path);if (!dir)return -EINVAL ;
tools/perf/util/data.h +1 -0
@@ -24,6 +24,7 @@ struct perf_data {enum perf_data_mode mode;struct { +u64 version;struct perf_data_file *files;int nr; } dir;
tools/perf/util/header.c +43 -1
68 lines not on this page · raw message
tools/perf/util/header.h +5 -0
@@ -39,6 +39,7 @@ enum {HEADER_SAMPLE_TIME ,HEADER_MEM_TOPOLOGY ,HEADER_CLOCKID , +HEADER_DIR_FORMAT ,HEADER_LAST_FEATURE , HEADER_FEAT_BITS = 256, };
@@ -48,6 +49,10 @@ enum perf_header_version { PERF_HEADER_VERSION_2, }; +enum perf_dir_version { +PERF_DIR_VERSION =1 , +}; +struct perf_file_section {u64 offset;u64 size;
Signature · 1 line
2.17.2
Jiri Olsa
reviewerPATCH v3 15/20 perf session: Add process callback to reader object
In reply to Jiri OlsaAdding callback function to reader object so
callers can process data in different ways.
--- tools/perf/util/session.c | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-)
tools/perf/util/session.c +19 -4
@@ -1849,10 +1849,17 @@ fetch_mmaped_event(struct perf_session *session,#define NUM_MMAPS 128#endif +struct reader ; + +typedef s64 (*reader_cb_t )(struct perf_session *session, +union perf_event *event, +u64 file_offset); +struct reader { -int fd; -u64 data_size; -u64 data_offset; +int fd; +u64 data_size; +u64 data_offset; +reader_cb_t process; }; static int
@@ -1923,7 +1930,7 @@ reader__process_events(struct reader *rd, struct perf_session *session, size = event->header.size;if (size <sizeof (struct perf_event_header ) || - (skip =perf_session__process_event (session, event, file_pos)) <0 ) { + (skip = rd->process (session, event, file_pos)) <0 ) {pr_err ("%#" PRIx64" [%#x]: failed to process type: %d\n" , file_offset + head, event->header.size, event->header.type);
@@ -1949,12 +1956,20 @@ reader__process_events(struct reader *rd, struct perf_session *session,return err; } +static s64 process_simple (struct perf_session *session, +union perf_event *event, +u64 file_offset) +{ +return perf_session__process_event (session, event, file_offset); +} +static int __perf_session__process_events (struct perf_session *session) {struct reader rd = { .fd =perf_data__fd (session->data), .data_size = session->header.data_size, .data_offset = session->header.data_offset, + .process = process_simple, };struct ordered_events *oe = &session->ordered_events;struct perf_tool *tool = session->tool;
Signature · 1 line
2.17.2
Jiri Olsa
reviewerPATCH v3 16/20 perf session: Add __perf_session__process_dir_events function
In reply to Jiri OlsaAdding __perf_session__process_dir_events function to process events over the directory data.
All directory events are pushed into sessions ordered data and flushed for processing.
--- tools/perf/util/session.c | 88 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 86 insertions(+), 2 deletions(-)
tools/perf/util/session.c +86 -2
@@ -1878,8 +1878,6 @@ reader__process_events(struct reader *rd, struct perf_session *session, file_offset = page_offset; head = rd->data_offset - page_offset; -ui_progress__init_size (prog, data_size,"Processing events..." ); - data_size += rd->data_offset; mmap_size =MMAP_SIZE ;
@@ -2008,6 +2006,89 @@ static int __perf_session__process_events(struct perf_session *session)return err; } +static s64 process_index (struct perf_session *session, +union perf_event *event, +u64 file_offset) +{ +struct perf_evlist *evlist = session->evlist; +u64 timestamp; +s64 ret; + +if (session->header.needs_swap) +event_swap (event,perf_evlist__sample_id_all (evlist)); + +if (event->header.type >=PERF_RECORD_HEADER_MAX ) +return -EINVAL ; + +events_stats__inc (&evlist->stats, event->header.type); + +if (event->header.type >=PERF_RECORD_USER_TYPE_START ) +return perf_session__process_user_event (session, event, file_offset); + + ret =perf_evlist__parse_sample_timestamp (evlist, event, ×tamp); +if (ret) +return ret; + +return ordered_events__queue (&session->ordered_events, event, + timestamp, file_offset); +} + +static int __perf_session__process_dir_events (struct perf_session *session) +{ +struct perf_data *data = session->data; +struct perf_tool *tool = session->tool; +struct reader rd = { + .fd =perf_data__fd (session->data), + .data_size = session->header.data_size, + .data_offset = session->header.data_offset, + .process = process_simple, + }; +int i, ret =0 ; +struct ui_progress prog; +u64 total_size =perf_data__size (session->data); + +perf_tool__fill_defaults (tool); + +ui_progress__init_size (&prog, total_size,"Processing events..." ); + +/* Read data from the header file.. */ + ret =reader__process_events (&rd, session, &prog); +if (ret) + goto out_err; + +/* ... and continue with data files. */ +for (i =0 ; i < data->dir.nr ; i++) { +struct perf_data_file *file = &data->dir.files[i]; + +if (file->size ==0 ) +continue ; + + rd = (struct reader ) { + .fd = file->fd, + .data_size = file->size, + .data_offset =0 , + .process = process_index, + }; + + ret =reader__process_events (&rd, session, &prog); +if (ret) + goto out_err; + } + + ret =ordered_events__flush (&session->ordered_events,OE_FLUSH__FINAL ); + +out_err: +if (!tool->no_warn) +perf_session__warn_about_errors (session); + +/* +* We may switching perf.data output, make ordered_events +* reusable. +*/ +ordered_events__reinit (&session->ordered_events); +return ret; +} +int perf_session__process_events (struct perf_session *session) {if (perf_session__register_idle_thread (session) <0 )
@@ -2016,6 +2097,9 @@ int perf_session__process_events(struct perf_session *session)if (perf_data__is_pipe (session->data))return __perf_session__process_pipe_events (session); +if (perf_data__is_dir (session->data)) +return __perf_session__process_dir_events (session); +return __perf_session__process_events (session); }
Signature · 1 line
2.17.2
Jiri Olsa
reviewerPATCH v3 17/20 perf session: Add path to reader object
In reply to Jiri OlsaAdding path to reader object, so we can display file the processing fails for (string in [] brackets).
$ perf report --stdio 0x5e0 [perf.data/data.3] [0xa200]: failed to process type: -1577027574
--- tools/perf/util/session.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
tools/perf/util/session.c +8 -2
@@ -1860,6 +1860,7 @@ struct reader {u64 data_size;u64 data_offset;reader_cb_t process; +char *path; }; static int
@@ -1874,6 +1875,8 @@ reader__process_events(struct reader *rd, struct perf_session *session,union perf_event *event;s64 skip; +pr_debug ("reader processing %s\n" , rd->path); + page_offset = page_size * (rd->data_offset / page_size); file_offset = page_offset; head = rd->data_offset - page_offset;
@@ -1929,8 +1932,8 @@ reader__process_events(struct reader *rd, struct perf_session *session,if (size <sizeof (struct perf_event_header ) || (skip = rd->process (session, event, file_pos)) <0 ) { -pr_err ("%#" PRIx64" [%#x]: failed to process type: %d\n" , - file_offset + head, event->header.size, +pr_err ("%#" PRIx64" [%s] [%#x]: failed to process type: %d\n" , + file_offset + head, rd->path, event->header.size, event->header.type); err = -EINVAL ; goto out;
@@ -1968,6 +1971,7 @@ static int __perf_session__process_events(struct perf_session *session) .data_size = session->header.data_size, .data_offset = session->header.data_offset, .process = process_simple, + .path = session->data->file.path, };struct ordered_events *oe = &session->ordered_events;struct perf_tool *tool = session->tool;
@@ -2042,6 +2046,7 @@ static int __perf_session__process_dir_events(struct perf_session *session) .data_size = session->header.data_size, .data_offset = session->header.data_offset, .process = process_simple, + .path = session->data->file.path, };int i, ret =0 ;struct ui_progress prog;
@@ -2068,6 +2073,7 @@ static int __perf_session__process_dir_events(struct perf_session *session) .data_size = file->size, .data_offset = 0, .process = process_index, + .path = file->path, }; ret =reader__process_events (&rd, session, &prog);
Signature · 1 line
2.17.2
Jiri Olsa
reviewerPATCH v3 18/20 perf record: Add --dir option to store data in directory
In reply to Jiri OlsaAdding --dir option to store data in directory. It's next step for multiple threads in record. It's now possible to make directory data via --dir option, like:
$ perf record --dir perf bench sched messaging $ ls -l perf.data total 344 -rw-------. 1 jolsa jolsa 43864 Jan 20 22:26 data.0 -rw-------. 1 jolsa jolsa 30464 Jan 20 22:26 data.1 -rw-------. 1 jolsa jolsa 53816 Jan 20 22:26 data.2 -rw-------. 1 jolsa jolsa 30368 Jan 20 22:26 data.3 -rw-------. 1 jolsa jolsa 40088 Jan 20 22:26 data.4 -rw-------. 1 jolsa jolsa 42592 Jan 20 22:26 data.5 -rw-------. 1 jolsa jolsa 56136 Jan 20 22:26 data.6 -rw-------. 1 jolsa jolsa 25992 Jan 20 22:26 data.7 -rw-------. 1 jolsa jolsa 8832 Jan 20 22:26 header
There's a data file created for every cpu and it's storing data for those cpu maps.
It's possible to transform directory data into standard perf.data file via following inject command:
$ perf inject -o perf.data.file -i perf.data
The --dir option enabled DIR_FORMAT feature to be stored in header file to indicate the directory layout.
Don't allow to use --dir with --aio yet. It needs to be investigated first.
--- tools/perf/Documentation/perf-record.txt | 3 + tools/perf/builtin-record.c | 80 ++++++++++++++++++++++-- tools/perf/util/mmap.h | 23 +++---- 3 files changed, 90 insertions(+), 16 deletions(-)
tools/perf/Documentation/perf-record.txt +3 -0
@@ -524,6 +524,9 @@ config terms. For example: 'cycles/overwrite/' and 'instructions/no-overwrite/'. Implies --tail-synthesize. +--dir:: +Store data into directory with one data file for cpu. + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/builtin-record.c +75 -5
156 lines not on this page · raw message
tools/perf/util/mmap.h +12 -11
@@ -19,17 +19,18 @@ struct aiocb;* @refcnt - e.g. code using PERF_EVENT_IOC_SET_OUTPUT to share this */ struct perf_mmap { -void *base; -int mask; -int fd; -int cpu; -refcount_t refcnt; -u64 prev; -u64 start; -u64 end; -bool overwrite; -struct auxtrace_mmap auxtrace_mmap; -char event_copy[PERF_SAMPLE_MAX_SIZE ]__aligned(8) ; +void *base; +int mask; +int fd; +int cpu; +refcount_t refcnt; +u64 prev; +u64 start; +u64 end; +bool overwrite; +struct auxtrace_mmap auxtrace_mmap; +struct perf_data_file *file; +char event_copy[PERF_SAMPLE_MAX_SIZE ]__aligned(8) ; #ifdef HAVE_AIO_SUPPORT struct { void **data;
Signature · 1 line
2.17.2
Jiri Olsa
reviewerPATCH v3 19/20 perf record: Add --output-dir option to store data in directory
In reply to Jiri OlsaAdding --output-dir option to mimic -o and --dir options. following commands do the same:
$ perf record -o perf.dir.data --dir ... $ perf record --output-dir perf.dir.data ...
User cannot use both -o and output-dir together,
error is displayed.
--- tools/lib/subcmd/parse-options.h | 4 ++++ tools/perf/Documentation/perf-record.txt | 3 +++ tools/perf/builtin-record.c | 13 +++++++++++-- 3 files changed, 18 insertions(+), 2 deletions(-)
tools/lib/subcmd/parse-options.h +4 -0
@@ -146,6 +146,10 @@ struct option { .value = check_vtype(v, const char **), .argh = (a), .help = (h), \ .flags = PARSE_OPT_OPTARG, .defval = (intptr_t)(d), \ .set = check_vtype(os, bool *)} +#define OPT_STRING_SET(s, l, v, os, a, h) \ + { .type = OPTION_STRING, .short_name = (s), .long_name = (l), \ + .value = check_vtype(v, const char **), .argh = (a), .help = (h), \ + .set = check_vtype(os, bool *)} #define OPT_STRING_NOEMPTY(s, l, v, a, h) { .type = OPTION_STRING, .short_name = (s), .long_name = (l), .value = check_vtype(v, const char **), .argh = (a), .help = (h), .flags = PARSE_OPT_NOEMPTY} #define OPT_DATE(s, l, v, h) \ { .type = OPTION_CALLBACK, .short_name = (s), .long_name = (l), .value = (v), .argh = "time", .help = (h), .callback = parse_opt_approxidate_cb }
tools/perf/Documentation/perf-record.txt +3 -0
@@ -527,6 +527,9 @@ Implies --tail-synthesize. --dir:: Store data into directory with one data file for cpu. +--output-dir:: +Same as --dir option, can't be used together with -o option. + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/builtin-record.c +11 -2
@@ -80,6 +80,7 @@ struct record {bool buildid_all;bool timestamp_filename;bool timestamp_boundary; +bool output_is_file;struct switch_output switch_output;unsigned long long samples;cpu_set_t affinity_mask;
@@ -1921,8 +1922,10 @@ static struct option __record_options[] = {OPT_STRING ('C' ,"cpu" , &record.opts.target.cpu_list,"cpu" ,"list of cpus to monitor" ),OPT_U64 ('c' ,"count" , &record.opts.user_interval,"event period to sample" ), -OPT_STRING ('o' ,"output" , &record.data.path,"file" , -"output file name" ), +OPT_STRING_SET ('o' ,"output" , &record.data.path, &record.output_is_file, +"file" ,"output file name" ), +OPT_STRING_SET (0 ,"output-dir" , &record.data.path, &record.data.is_dir, +"file" ,"output directory name" ),OPT_BOOLEAN_SET ('i' ,"no-inherit" , &record.opts.no_inherit, &record.opts.no_inherit_set, "child tasks do not inherit counters"),
@@ -2101,6 +2104,12 @@ int cmd_record(int argc, const char **argv) "cgroup monitoring only available in system-wide mode"); } + +if (perf_data__is_dir (&rec->data) && record.output_is_file) { +ui__error ("cannot use both -o and --output-dir\n" ); +return -EINVAL ; + } +if (rec->opts.record_switch_events && !perf_can_record_switch_events ()) {ui__error ("kernel does not support recording context switch events\n" );
Signature · 1 line
2.17.2
Jiri Olsa
reviewerPATCH v3 20/20 perf record: Describe perf.data directory format
In reply to Jiri OlsaAdding perf.data-directory-format.txt to describe the directory data layout.
--- .../perf.data-directory-format.txt | 54 +++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt
tools/perf/Documentation/perf.data-directory-format.txt new file +54 -0
@@ -0,0 +1,54 @@ +perf.data directory format + +DISCLAIMER This is not ABI yet and is subject to possible change + in following versions of perf. We will remove this + disclaimer once the directory format soaks in. + + +This document describes the on-disk perf.data format, generated +by perf record with --dir option and consumed by the other perf +tools. + +The directory perf.data is created by perf record command by +using the --dir option: + + $ perf record --dir perf bench sched messaging + $ ls -l perf.data + total 344 + -rw-------. 1 jolsa jolsa 43864 Jan 20 22:26 data.0 + -rw-------. 1 jolsa jolsa 30464 Jan 20 22:26 data.1 + -rw-------. 1 jolsa jolsa 53816 Jan 20 22:26 data.2 + -rw-------. 1 jolsa jolsa 30368 Jan 20 22:26 data.3 + -rw-------. 1 jolsa jolsa 40088 Jan 20 22:26 data.4 + -rw-------. 1 jolsa jolsa 42592 Jan 20 22:26 data.5 + -rw-------. 1 jolsa jolsa 56136 Jan 20 22:26 data.6 + -rw-------. 1 jolsa jolsa 25992 Jan 20 22:26 data.7 + -rw-------. 1 jolsa jolsa 8832 Jan 20 22:26 header + +The header file keeps the standard perf.data file header, +and the data.* files keep data. + +header file +----------- +The header file following the standard format describe in +Documentation/perf.data-file-format doc. Including its data +portion that is used to store manually synthesized events. + +data file +--------- +The data files layout is described by HEADER_DIR_FORMAT feature. +Currently it holds only version number (1): + + HEADER_DIR_FORMAT = 24 + + struct { + uint64_t version; + } + +The current only only version value 1 means that data files: + - follow the 'data.*' format + - contain raw events data in standard perf format as read + from kernel (and need to be sorted) + +Future versions are expected to describe different data files +layout according to special needs.
Signature · 1 line
2.17.2