Skip to contentSkip to conversation

perf tool: Add support for multiple data file storage

PATCH In review Jiri Olsa 8 participants 48 messages Sep 1, 2013 – Oct 15, 2013 perf events

lore
Share & tools
lore

Jiri Olsa

author
#0 lore raw

v2 cover perf tool: Add support for multiple data file storage

hi,
sending the support for multiple file storage. Initial RFC is here:
http://marc.info/?l=linux-kernel&m=137408381902423&w=2

v2 changes:
- reworked perf mmap size setup to be able to get the mmap size value easily later
- added perf.data read/write test for v2 and v3 for both endianity
- added record '-M time' support

It's reachable here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/core_file

thanks,
jirka

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>

---
Jiri Olsa (25):
perf tools: Check mmap pages value early
perf tools: Add possibility to specify mmap size
perf tools: Introduce perf_evlist__new_default function
perf tools: Adding throttle event data struct support
perf tests: Add simple session read/write test
perf tests: Add session reading test for little endian perf data perf tests: Add session reading test for big endian perf data perf doc: Add perf data file documentation
perf tools: Introduce perf data file version CHECK macro perf tools: Introduce swap_features function
perf tools: Introduce swap_header function
perf tools: Separate version 2 specific perf data header bits perf tools: Using evlist as a holder for event_desc feature perf tools: Introduce perf.data version 3 format
perf tools: Add perf data version 3 header swap
perf tools: Add perf data version 3 header read
perf tools: Add perf.data version 3 header write
perf tools: Get rid of post_processing_offset in record command perf tools: Move synthesizing into single function
perf tools: Add perf_data_file__open interface to data object perf tools: Separating data file properties from session perf tests: Add session reading test for little endian perf data v3 perf tests: Add session reading test for big endian perf data v3 perf tools: Add multi file '-M' option for record command perf tools: Have the process properly sythesized in subsequent data files

 tools/perf/Documentation/perf-data-file-v2.txt | 265 ++++++++++++++++++++++++++++++++++
 tools/perf/Documentation/perf-data-file-v3.txt |  63 ++++++++
 tools/perf/Documentation/perf-kvm.txt          |   4 +-
 tools/perf/Documentation/perf-record.txt       |  18 ++-
 tools/perf/Documentation/perf-top.txt          |   4 +-
 tools/perf/Documentation/perf-trace.txt        |   4 +-
 tools/perf/Makefile                            |   6 +
 tools/perf/builtin-annotate.c                  |  11 +-
 tools/perf/builtin-buildid-cache.c             |   8 +-
 tools/perf/builtin-buildid-list.c              |  11 +-
 tools/perf/builtin-diff.c                      |  19 ++-
 tools/perf/builtin-evlist.c                    |   7 +-
 tools/perf/builtin-inject.c                    |  10 +-
 tools/perf/builtin-kmem.c                      |   7 +-
 tools/perf/builtin-kvm.c                       |  18 ++-
 tools/perf/builtin-lock.c                      |   8 +-
 tools/perf/builtin-mem.c                       |   9 +-
 tools/perf/builtin-record.c                    | 484 ++++++++++++++++++++++++++++++++++++++++++++------------------
 tools/perf/builtin-report.c                    |  18 ++-
 tools/perf/builtin-sched.c                     |   6 +-
 tools/perf/builtin-script.c                    |  17 ++-
 tools/perf/builtin-timechart.c                 |  10 +-
 tools/perf/builtin-top.c                       |  12 +-
 tools/perf/builtin-trace.c                     |  13 +-
 tools/perf/perf.h                              |  15 +-
 tools/perf/tests/builtin-test.c                |   4 +
 tools/perf/tests/perf-record.c                 |  12 +-
 tools/perf/tests/perf.data.v2.be.h             | 265 ++++++++++++++++++++++++++++++++++
 tools/perf/tests/perf.data.v2.le.h             | 219 ++++++++++++++++++++++++++++
 tools/perf/tests/perf.data.v3.be.h             | 258 +++++++++++++++++++++++++++++++++
 tools/perf/tests/perf.data.v3.le.h             | 212 +++++++++++++++++++++++++++
 tools/perf/tests/session-simple.c              | 658 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/tests/task-exit.c                   |  14 +-
 tools/perf/tests/tests.h                       |   1 +
 tools/perf/util/data.c                         | 120 ++++++++++++++++
 tools/perf/util/data.h                         |  48 +++++++
 tools/perf/util/event.h                        |   7 +
 tools/perf/util/evlist.c                       |  78 ++++++++--
 tools/perf/util/evlist.h                       |   9 +-
 tools/perf/util/evsel.c                        |  24 ++++
 tools/perf/util/header.c                       | 446 +++++++++++++++++++++++++++++++++------------------------
 tools/perf/util/header.h                       |  31 +++-
 tools/perf/util/python.c                       |   7 -
 tools/perf/util/session.c                      | 138 ++++++++----------
 tools/perf/util/session.h                      |  11 +-
 tools/perf/util/util.c                         |  25 ++++
 tools/perf/util/util.h                         |  14 ++
 47 files changed, 3137 insertions(+), 511 deletions(-)
 create mode 100644 tools/perf/Documentation/perf-data-file-v2.txt
 create mode 100644 tools/perf/Documentation/perf-data-file-v3.txt
 create mode 100644 tools/perf/tests/perf.data.v2.be.h
 create mode 100644 tools/perf/tests/perf.data.v2.le.h
 create mode 100644 tools/perf/tests/perf.data.v3.be.h
 create mode 100644 tools/perf/tests/perf.data.v3.le.h
 create mode 100644 tools/perf/tests/session-simple.c
 create mode 100644 tools/perf/util/data.c
 create mode 100644 tools/perf/util/data.h
author
#1 lore raw

PATCH v2 1/25 perf tools: Check mmap pages value early

Moving the check of the mmap_pages value into the options parsing time, so we could relay on this value on other parts of code. Related changes come in next patches.

Also changins perf_evlist::mmap_len to proper size_t type.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/builtin-kvm.c    |  5 +++--
 tools/perf/builtin-record.c |  9 +++------
 tools/perf/builtin-top.c    |  5 +++--
 tools/perf/builtin-trace.c  |  5 +++--
 tools/perf/util/evlist.c    | 46 ++++++++++++++++++++++++++++++++++++---------
 tools/perf/util/evlist.h    |  6 +++++-
 6 files changed, 54 insertions(+), 22 deletions(-)
tools/perf/builtin-kvm.c +3 -2
index 47b3540..f89f7d6 100644
@@ -1426,8 +1426,9 @@ static int kvm_events_live(struct perf_kvm_stat *kvm,
 	const struct option live_options[] = {
 		OPT_STRING('p', "pid", &kvm->opts.target.pid, "pid",
 			"record events on existing process id"),
-		OPT_UINTEGER('m', "mmap-pages", &kvm->opts.mmap_pages,
-			"number of mmap data pages"),
+		OPT_CALLBACK('m', "mmap-pages", &kvm->opts.mmap_pages, "pages",
+			"number of mmap data pages",
+			perf_evlist__parse_mmap_pages),
 		OPT_INCR('v', "verbose", &verbose,
 			"be more verbose (show counter open errors, etc)"),
 		OPT_BOOLEAN('a', "all-cpus", &kvm->opts.target.system_wide,
tools/perf/builtin-record.c +3 -6
index a41ac415..351ede3 100644
@@ -234,10 +234,6 @@ try_again:
 			       "or try again with a smaller value of -m/--mmap_pages.\n"
 			       "(current value: %d)\n", opts->mmap_pages);
 			rc = -errno;
-		} else if (!is_power_of_2(opts->mmap_pages) &&
-			   (opts->mmap_pages != UINT_MAX)) {
-			pr_err("--mmap_pages/-m value must be a power of two.");
-			rc = -EINVAL;
 		} else {
 			pr_err("failed to mmap with %d (%s)\n", errno, strerror(errno));
 			rc = -errno;
@@ -854,8 +850,9 @@ const struct option record_options[] = {
 	OPT_BOOLEAN('i', "no-inherit", &record.opts.no_inherit,
 		    "child tasks do not inherit counters"),
 	OPT_UINTEGER('F', "freq", &record.opts.user_freq, "profile at this frequency"),
-	OPT_UINTEGER('m', "mmap-pages", &record.opts.mmap_pages,
-		     "number of mmap data pages"),
+	OPT_CALLBACK('m', "mmap-pages", &record.opts.mmap_pages, "pages",
+		     "number of mmap data pages",
+		     perf_evlist__parse_mmap_pages),
 	OPT_BOOLEAN(0, "group", &record.opts.group,
 		    "put the counters into a counter group"),
 	OPT_CALLBACK_DEFAULT('g', "call-graph", &record.opts,
tools/perf/builtin-top.c +3 -2
index 2122141..0a84334 100644
@@ -1076,8 +1076,9 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)
 		   "file", "vmlinux pathname"),
 	OPT_BOOLEAN('K', "hide_kernel_symbols", &top.hide_kernel_symbols,
 		    "hide kernel symbols"),
-	OPT_UINTEGER('m', "mmap-pages", &opts->mmap_pages,
-		     "number of mmap data pages"),
+	OPT_CALLBACK('m', "mmap-pages", &opts->mmap_pages, "pages",
+		     "number of mmap data pages",
+		     perf_evlist__parse_mmap_pages),
 	OPT_INTEGER('r', "realtime", &top.realtime_prio,
 		    "collect data with this RT SCHED_FIFO priority"),
 	OPT_INTEGER('d', "delay", &top.delay_secs,
tools/perf/builtin-trace.c +3 -2
index 69a065e..9967a56 100644
@@ -1042,8 +1042,9 @@ int cmd_trace(int argc, const char **argv, const char *prefix __maybe_unused)
 		    "list of cpus to monitor"),
 	OPT_BOOLEAN(0, "no-inherit", &trace.opts.no_inherit,
 		    "child tasks do not inherit counters"),
-	OPT_UINTEGER('m', "mmap-pages", &trace.opts.mmap_pages,
-		     "number of mmap data pages"),
+	OPT_CALLBACK('m', "mmap-pages", &trace.opts.mmap_pages, "pages",
+		     "number of mmap data pages",
+		     perf_evlist__parse_mmap_pages),
 	OPT_STRING('u', "uid", &trace.opts.target.uid_str, "user",
 		   "user to profile"),
 	OPT_CALLBACK(0, "duration", &trace, "float",
tools/perf/util/evlist.c +37 -9

70 lines not on this page · raw message

tools/perf/util/evlist.h +5 -1
index 841a394..cae0bf5 100644
@@ -31,7 +31,7 @@ struct perf_evlist {
 	int		 nr_groups;
 	int		 nr_fds;
 	int		 nr_mmaps;
-	int		 mmap_len;
+	size_t		 mmap_len;
 	int		 id_pos;
 	int		 is_pos;
 	u64		 combined_sample_type;
@@ -103,6 +103,10 @@ int perf_evlist__prepare_workload(struct perf_evlist *evlist,
 				  bool want_signal);
 int perf_evlist__start_workload(struct perf_evlist *evlist);
 
+int perf_evlist__parse_mmap_pages(const struct option *opt,
+				  const char *str,
+				  int unset);
+
 int perf_evlist__mmap(struct perf_evlist *evlist, unsigned int pages,
 		      bool overwrite);
 void perf_evlist__munmap(struct perf_evlist *evlist);
Signature · 1 line
1.7.11.7
#2 lore raw

[tip:perf/core] perf tools: Check mmap pages value early

perf tools: Check mmap pages value early
Commit-ID:  994a1f78b191df0c9d6caca3f3afb03e247aff26
Gitweb:     http://git.kernel.org/tip/994a1f78b191df0c9d6caca3f3afb03e247aff26
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 1 Sep 2013 12:36:12 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 9 Oct 2013 11:24:10 -0300

perf tools: Check mmap pages value early

Move the check of the mmap_pages value to the options parsing time, so we could rely on this value on other parts of code.

Related changes come in the next patches.

Also changes perf_evlist::mmap_len to proper size_t type.

Cc Andi Kleen <[email protected]>
Cc Corey Ashford <[email protected]>
Cc David Ahern <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 tools/perf/builtin-kvm.c    |  5 +++--
 tools/perf/builtin-record.c |  9 +++------
 tools/perf/builtin-top.c    |  5 +++--
 tools/perf/builtin-trace.c  |  5 +++--
 tools/perf/util/evlist.c    | 46 ++++++++++++++++++++++++++++++++++++---------
 tools/perf/util/evlist.h    |  6 +++++-
 6 files changed, 54 insertions(+), 22 deletions(-)
tools/perf/builtin-kvm.c +3 -2
index 935d522..cfa0b79 100644
@@ -1426,8 +1426,9 @@ static int kvm_events_live(struct perf_kvm_stat *kvm,
 	const struct option live_options[] = {
 		OPT_STRING('p', "pid", &kvm->opts.target.pid, "pid",
 			"record events on existing process id"),
-		OPT_UINTEGER('m', "mmap-pages", &kvm->opts.mmap_pages,
-			"number of mmap data pages"),
+		OPT_CALLBACK('m', "mmap-pages", &kvm->opts.mmap_pages, "pages",
+			"number of mmap data pages",
+			perf_evlist__parse_mmap_pages),
 		OPT_INCR('v', "verbose", &verbose,
 			"be more verbose (show counter open errors, etc)"),
 		OPT_BOOLEAN('a', "all-cpus", &kvm->opts.target.system_wide,
tools/perf/builtin-record.c +3 -6
index 0aacd62..92ca541 100644
@@ -233,10 +233,6 @@ try_again:
 			       "or try again with a smaller value of -m/--mmap_pages.\n"
 			       "(current value: %d)\n", opts->mmap_pages);
 			rc = -errno;
-		} else if (!is_power_of_2(opts->mmap_pages) &&
-			   (opts->mmap_pages != UINT_MAX)) {
-			pr_err("--mmap_pages/-m value must be a power of two.");
-			rc = -EINVAL;
 		} else {
 			pr_err("failed to mmap with %d (%s)\n", errno, strerror(errno));
 			rc = -errno;
@@ -854,8 +850,9 @@ const struct option record_options[] = {
 	OPT_BOOLEAN('i', "no-inherit", &record.opts.no_inherit,
 		    "child tasks do not inherit counters"),
 	OPT_UINTEGER('F', "freq", &record.opts.user_freq, "profile at this frequency"),
-	OPT_UINTEGER('m', "mmap-pages", &record.opts.mmap_pages,
-		     "number of mmap data pages"),
+	OPT_CALLBACK('m', "mmap-pages", &record.opts.mmap_pages, "pages",
+		     "number of mmap data pages",
+		     perf_evlist__parse_mmap_pages),
 	OPT_BOOLEAN(0, "group", &record.opts.group,
 		    "put the counters into a counter group"),
 	OPT_CALLBACK_DEFAULT('g', "call-graph", &record.opts,
tools/perf/builtin-top.c +3 -2
index b3e0229..e846695 100644
@@ -1075,8 +1075,9 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)
 		   "file", "vmlinux pathname"),
 	OPT_BOOLEAN('K', "hide_kernel_symbols", &top.hide_kernel_symbols,
 		    "hide kernel symbols"),
-	OPT_UINTEGER('m', "mmap-pages", &opts->mmap_pages,
-		     "number of mmap data pages"),
+	OPT_CALLBACK('m', "mmap-pages", &opts->mmap_pages, "pages",
+		     "number of mmap data pages",
+		     perf_evlist__parse_mmap_pages),
 	OPT_INTEGER('r', "realtime", &top.realtime_prio,
 		    "collect data with this RT SCHED_FIFO priority"),
 	OPT_INTEGER('d', "delay", &top.delay_secs,
tools/perf/builtin-trace.c +3 -2
index be65843..f3ddbb0 100644
@@ -1528,8 +1528,9 @@ int cmd_trace(int argc, const char **argv, const char *prefix __maybe_unused)
 		    "list of cpus to monitor"),
 	OPT_BOOLEAN(0, "no-inherit", &trace.opts.no_inherit,
 		    "child tasks do not inherit counters"),
-	OPT_UINTEGER('m', "mmap-pages", &trace.opts.mmap_pages,
-		     "number of mmap data pages"),
+	OPT_CALLBACK('m', "mmap-pages", &trace.opts.mmap_pages, "pages",
+		     "number of mmap data pages",
+		     perf_evlist__parse_mmap_pages),
 	OPT_STRING('u', "uid", &trace.opts.target.uid_str, "user",
 		   "user to profile"),
 	OPT_CALLBACK(0, "duration", &trace, "float",
tools/perf/util/evlist.c +37 -9

70 lines not on this page · raw message

tools/perf/util/evlist.h +5 -1
index 880d713..4edb500 100644
@@ -31,7 +31,7 @@ struct perf_evlist {
 	int		 nr_groups;
 	int		 nr_fds;
 	int		 nr_mmaps;
-	int		 mmap_len;
+	size_t		 mmap_len;
 	int		 id_pos;
 	int		 is_pos;
 	u64		 combined_sample_type;
@@ -103,6 +103,10 @@ int perf_evlist__prepare_workload(struct perf_evlist *evlist,
 				  bool want_signal);
 int perf_evlist__start_workload(struct perf_evlist *evlist);
 
+int perf_evlist__parse_mmap_pages(const struct option *opt,
+				  const char *str,
+				  int unset);
+
 int perf_evlist__mmap(struct perf_evlist *evlist, unsigned int pages,
 		      bool overwrite);
 void perf_evlist__munmap(struct perf_evlist *evlist);
author
#3 lore raw

PATCH v2 2/25 perf tools: Add possibility to specify mmap size

In reply to Jiri Olsa

Adding possibility to specify mmap size via -m/--mmap-pages by appending unit size character (B/K/M/G) to the number, like:

  $ perf record -m 8K ls
  $ perf record -m 2M ls

The size is rounded up appropriately to follow perf mmap restrictions.

If no unit is specified the number provides pages as of now, like:

  $ perf record -m 8 ls
Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/Documentation/perf-kvm.txt    |  4 +++-
 tools/perf/Documentation/perf-record.txt |  4 +++-
 tools/perf/Documentation/perf-top.txt    |  4 +++-
 tools/perf/Documentation/perf-trace.txt  |  4 +++-
 tools/perf/util/evlist.c                 | 32 ++++++++++++++++++++++++++------
 tools/perf/util/util.c                   | 25 +++++++++++++++++++++++++
 tools/perf/util/util.h                   | 14 ++++++++++++++
 7 files changed, 77 insertions(+), 10 deletions(-)
tools/perf/Documentation/perf-kvm.txt +3 -1
index ac84db2..6a06cef 100644
@@ -109,7 +109,9 @@ STAT LIVE OPTIONS
 
 -m::
 --mmap-pages=::
-    Number of mmap data pages. Must be a power of two.
+    Number of mmap data pages (must be a power of two) or size
+    specification with appended unit character - B/K/M/G. The
+    size is rounded up to have nearest pages power of two value.
 
 -a::
 --all-cpus::
tools/perf/Documentation/perf-record.txt +3 -1
index e297b74..400e9bb 100644
@@ -87,7 +87,9 @@ OPTIONS
 
 -m::
 --mmap-pages=::
-	Number of mmap data pages. Must be a power of two.
+	Number of mmap data pages (must be a power of two) or size
+	specification with appended unit character - B/K/M/G. The
+	size is rounded up to have nearest pages power of two value.
 
 -g::
 --call-graph::
tools/perf/Documentation/perf-top.txt +3 -1
index 58d6598..19ab69e 100644
@@ -68,7 +68,9 @@ Default is to monitor all CPUS.
 
 -m <pages>::
 --mmap-pages=<pages>::
-	Number of mmapped data pages.
+	Number of mmap data pages (must be a power of two) or size
+	specification with appended unit character - B/K/M/G. The
+	size is rounded up to have nearest pages power of two value.
 
 -p <pid>::
 --pid=<pid>::
tools/perf/Documentation/perf-trace.txt +3 -1
index daccd2c..d390efa 100644
@@ -59,7 +59,9 @@ OPTIONS
 
 -m::
 --mmap-pages=::
-	Number of mmap data pages. Must be a power of two.
+	Number of mmap data pages (must be a power of two) or size
+	specification with appended unit character - B/K/M/G. The
+	size is rounded up to have nearest pages power of two value.
 
 -C::
 --cpu::
tools/perf/util/evlist.c +26 -6
index 4780b9e..8ea933f 100644
@@ -634,14 +634,33 @@ static size_t perf_evlist__mmap_size(unsigned long pages)
 int perf_evlist__parse_mmap_pages(const struct option *opt, const char *str,
 				  int unset __maybe_unused)
 {
-	unsigned int pages, *mmap_pages = opt->value;
+	unsigned int pages, val, *mmap_pages = opt->value;
 	size_t size;
-	char *eptr;
+	static struct parse_tag tags[] = {
+		{ .tag  = 'B', .mult = 1       },
+		{ .tag  = 'K', .mult = 1 << 10 },
+		{ .tag  = 'M', .mult = 1 << 20 },
+		{ .tag  = 'G', .mult = 1 << 30 },
+		{ .tag  = 0 },
+	};
 
-	pages = strtoul(str, &eptr, 10);
-	if (*eptr != '\0') {
-		pr_err("failed to parse --mmap_pages/-m value\n");
-		return -1;
+	val = parse_tag_value(str, tags);
+	if (val != (unsigned int) -1) {
+		/* we got file size value */
+		pages = PERF_ALIGN(val, page_size) / page_size;
+		if (!is_power_of_2(pages)) {
+			pages = next_pow2(pages);
+			pr_info("rounding mmap pages size to %u (%u pages)\n",
+				pages * page_size, pages);
+		}
+	} else {
+		/* we got pages count value */
+		char *eptr;
+		pages = strtoul(str, &eptr, 10);
+		if (*eptr != '\0') {
+			pr_err("failed to parse --mmap_pages/-m value\n");
+			return -1;
+		}
 	}
 
 	size = perf_evlist__mmap_size(pages);
@@ -685,6 +704,7 @@ int perf_evlist__mmap(struct perf_evlist *evlist, unsigned int pages,
 
 	evlist->overwrite = overwrite;
 	evlist->mmap_len = perf_evlist__mmap_size(pages);
+	pr_debug("mmap size %luB\n", evlist->mmap_len);
 	mask = evlist->mmap_len - page_size - 1;
 
 	list_for_each_entry(evsel, &evlist->entries, node) {
tools/perf/util/util.c +25 -0
index 6d17b18..141317e 100644
@@ -361,3 +361,28 @@ int parse_nsec_time(const char *str, u64 *ptime)
 	*ptime = time_sec * NSEC_PER_SEC + time_nsec;
 	return 0;
 }
+
+unsigned long parse_tag_value(const char *str, struct parse_tag *tags)
+{
+	struct parse_tag *i = tags;
+
+	while (i->tag) {
+		char *s;
+
+		s = strchr(str, i->tag);
+		if (s) {
+			unsigned long int value;
+			char *endptr;
+
+			value = strtoul(str, &endptr, 10);
+			if (s != endptr)
+				break;
+
+			value *= i->mult;
+			return value;
+		}
+		i++;
+	}
+
+	return (unsigned long) -1;
+}
tools/perf/util/util.h +14 -0
index a535359..c29ecaa 100644
@@ -270,6 +270,13 @@ bool is_power_of_2(unsigned long n)
 	return (n != 0 && ((n & (n - 1)) == 0));
 }
 
+static inline unsigned next_pow2(unsigned x)
+{
+	if (!x)
+		return 1;
+	return 1ULL << (32 - __builtin_clz(x - 1));
+}
+
 size_t hex_width(u64 v);
 int hex2u64(const char *ptr, u64 *val);
 
@@ -281,4 +288,11 @@ void dump_stack(void);
 extern unsigned int page_size;
 
 void get_term_dimensions(struct winsize *ws);
+
+struct parse_tag {
+	char tag;
+	int mult;
+};
+
+unsigned long parse_tag_value(const char *str, struct parse_tag *tags);
 #endif /* GIT_COMPAT_UTIL_H */
Signature · 1 line
1.7.11.7
#4 lore raw

[tip:perf/core] perf tools: Add possibility to specify mmap size

perf tools: Add possibility to specify mmap size
Commit-ID:  27050f530dc4fd88dc93d85c177e000efe970d12
Gitweb:     http://git.kernel.org/tip/27050f530dc4fd88dc93d85c177e000efe970d12
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 1 Sep 2013 12:36:13 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 9 Oct 2013 11:24:20 -0300

perf tools: Add possibility to specify mmap size

Adding possibility to specify mmap size via -m/--mmap-pages by appending unit size character (B/K/M/G) to the number, like:

  $ perf record -m 8K ls
  $ perf record -m 2M ls

The size is rounded up appropriately to follow perf mmap restrictions.

If no unit is specified the number provides pages as of now, like:

  $ perf record -m 8 ls
Cc Andi Kleen <[email protected]>
Cc Corey Ashford <[email protected]>
Cc David Ahern <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 tools/perf/Documentation/perf-kvm.txt    |  4 +++-
 tools/perf/Documentation/perf-record.txt |  4 +++-
 tools/perf/Documentation/perf-top.txt    |  4 +++-
 tools/perf/Documentation/perf-trace.txt  |  4 +++-
 tools/perf/util/evlist.c                 | 32 ++++++++++++++++++++++++++------
 tools/perf/util/util.c                   | 25 +++++++++++++++++++++++++
 tools/perf/util/util.h                   | 14 ++++++++++++++
 7 files changed, 77 insertions(+), 10 deletions(-)
tools/perf/Documentation/perf-kvm.txt +3 -1
index ac84db2..6a06cef 100644
@@ -109,7 +109,9 @@ STAT LIVE OPTIONS
 
 -m::
 --mmap-pages=::
-    Number of mmap data pages. Must be a power of two.
+    Number of mmap data pages (must be a power of two) or size
+    specification with appended unit character - B/K/M/G. The
+    size is rounded up to have nearest pages power of two value.
 
 -a::
 --all-cpus::
tools/perf/Documentation/perf-record.txt +3 -1
index f732eaa..f10ab63 100644
@@ -87,7 +87,9 @@ OPTIONS
 
 -m::
 --mmap-pages=::
-	Number of mmap data pages. Must be a power of two.
+	Number of mmap data pages (must be a power of two) or size
+	specification with appended unit character - B/K/M/G. The
+	size is rounded up to have nearest pages power of two value.
 
 -g::
 --call-graph::
tools/perf/Documentation/perf-top.txt +3 -1
index 6d70fbf..f65777c 100644
@@ -68,7 +68,9 @@ Default is to monitor all CPUS.
 
 -m <pages>::
 --mmap-pages=<pages>::
-	Number of mmapped data pages.
+	Number of mmap data pages (must be a power of two) or size
+	specification with appended unit character - B/K/M/G. The
+	size is rounded up to have nearest pages power of two value.
 
 -p <pid>::
 --pid=<pid>::
tools/perf/Documentation/perf-trace.txt +3 -1
index 3777385..7f70d36 100644
@@ -59,7 +59,9 @@ OPTIONS
 
 -m::
 --mmap-pages=::
-	Number of mmap data pages. Must be a power of two.
+	Number of mmap data pages (must be a power of two) or size
+	specification with appended unit character - B/K/M/G. The
+	size is rounded up to have nearest pages power of two value.
 
 -C::
 --cpu::
tools/perf/util/evlist.c +26 -6
index 4283f49..d21ab08 100644
@@ -687,14 +687,33 @@ static size_t perf_evlist__mmap_size(unsigned long pages)
 int perf_evlist__parse_mmap_pages(const struct option *opt, const char *str,
 				  int unset __maybe_unused)
 {
-	unsigned int pages, *mmap_pages = opt->value;
+	unsigned int pages, val, *mmap_pages = opt->value;
 	size_t size;
-	char *eptr;
+	static struct parse_tag tags[] = {
+		{ .tag  = 'B', .mult = 1       },
+		{ .tag  = 'K', .mult = 1 << 10 },
+		{ .tag  = 'M', .mult = 1 << 20 },
+		{ .tag  = 'G', .mult = 1 << 30 },
+		{ .tag  = 0 },
+	};
 
-	pages = strtoul(str, &eptr, 10);
-	if (*eptr != '\0') {
-		pr_err("failed to parse --mmap_pages/-m value\n");
-		return -1;
+	val = parse_tag_value(str, tags);
+	if (val != (unsigned int) -1) {
+		/* we got file size value */
+		pages = PERF_ALIGN(val, page_size) / page_size;
+		if (!is_power_of_2(pages)) {
+			pages = next_pow2(pages);
+			pr_info("rounding mmap pages size to %u (%u pages)\n",
+				pages * page_size, pages);
+		}
+	} else {
+		/* we got pages count value */
+		char *eptr;
+		pages = strtoul(str, &eptr, 10);
+		if (*eptr != '\0') {
+			pr_err("failed to parse --mmap_pages/-m value\n");
+			return -1;
+		}
 	}
 
 	size = perf_evlist__mmap_size(pages);
@@ -738,6 +757,7 @@ int perf_evlist__mmap(struct perf_evlist *evlist, unsigned int pages,
 
 	evlist->overwrite = overwrite;
 	evlist->mmap_len = perf_evlist__mmap_size(pages);
+	pr_debug("mmap size %luB\n", evlist->mmap_len);
 	mask = evlist->mmap_len - page_size - 1;
 
 	list_for_each_entry(evsel, &evlist->entries, node) {
tools/perf/util/util.c +25 -0
index ccfdeb6..ab71d62 100644
@@ -361,3 +361,28 @@ int parse_nsec_time(const char *str, u64 *ptime)
 	*ptime = time_sec * NSEC_PER_SEC + time_nsec;
 	return 0;
 }
+
+unsigned long parse_tag_value(const char *str, struct parse_tag *tags)
+{
+	struct parse_tag *i = tags;
+
+	while (i->tag) {
+		char *s;
+
+		s = strchr(str, i->tag);
+		if (s) {
+			unsigned long int value;
+			char *endptr;
+
+			value = strtoul(str, &endptr, 10);
+			if (s != endptr)
+				break;
+
+			value *= i->mult;
+			return value;
+		}
+		i++;
+	}
+
+	return (unsigned long) -1;
+}
tools/perf/util/util.h +14 -0
index a535359..c29ecaa 100644
@@ -270,6 +270,13 @@ bool is_power_of_2(unsigned long n)
 	return (n != 0 && ((n & (n - 1)) == 0));
 }
 
+static inline unsigned next_pow2(unsigned x)
+{
+	if (!x)
+		return 1;
+	return 1ULL << (32 - __builtin_clz(x - 1));
+}
+
 size_t hex_width(u64 v);
 int hex2u64(const char *ptr, u64 *val);
 
@@ -281,4 +288,11 @@ void dump_stack(void);
 extern unsigned int page_size;
 
 void get_term_dimensions(struct winsize *ws);
+
+struct parse_tag {
+	char tag;
+	int mult;
+};
+
+unsigned long parse_tag_value(const char *str, struct parse_tag *tags);
 #endif /* GIT_COMPAT_UTIL_H */
author
#5 lore raw

PATCH v2 3/25 perf tools: Introduce perf_evlist__new_default function

In reply to Jiri Olsa

Adding new common function to create evlist with default event. It spares some code lines in automated tests.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/tests/perf-record.c | 12 +-----------
 tools/perf/tests/task-exit.c   | 14 ++------------
 tools/perf/util/evlist.c       | 12 ++++++++++++
 tools/perf/util/evlist.h       |  1 +
 4 files changed, 16 insertions(+), 23 deletions(-)
tools/perf/tests/perf-record.c +1 -11
index 72d8881..6143b06 100644
@@ -45,7 +45,7 @@ int test__PERF_RECORD(void)
 	};
 	cpu_set_t cpu_mask;
 	size_t cpu_mask_size = sizeof(cpu_mask);
-	struct perf_evlist *evlist = perf_evlist__new();
+	struct perf_evlist *evlist = perf_evlist__new_default();
 	struct perf_evsel *evsel;
 	struct perf_sample sample;
 	const char *cmd = "sleep";
@@ -66,16 +66,6 @@ int test__PERF_RECORD(void)
 	}
 
 	/*
-	 * We need at least one evsel in the evlist, use the default
-	 * one: "cycles".
-	 */
-	err = perf_evlist__add_default(evlist);
-	if (err < 0) {
-		pr_debug("Not enough memory to create evsel\n");
-		goto out_delete_evlist;
-	}
-
-	/*
 	 * Create maps of threads and cpus to monitor. In this case
 	 * we start with all threads and cpus (-1, -1) but then in
 	 * perf_evlist__prepare_workload we'll fill in the only thread
tools/perf/tests/task-exit.c +2 -12
index 28fe589..b07f8a1 100644
@@ -37,20 +37,11 @@ int test__task_exit(void)
 	signal(SIGCHLD, sig_handler);
 	signal(SIGUSR1, sig_handler);
 
-	evlist = perf_evlist__new();
+	evlist = perf_evlist__new_default();
 	if (evlist == NULL) {
-		pr_debug("perf_evlist__new\n");
+		pr_debug("perf_evlist__new_default\n");
 		return -1;
 	}
-	/*
-	 * We need at least one evsel in the evlist, use the default
-	 * one: "cycles".
-	 */
-	err = perf_evlist__add_default(evlist);
-	if (err < 0) {
-		pr_debug("Not enough memory to create evsel\n");
-		goto out_free_evlist;
-	}
 
 	/*
 	 * Create maps of threads and cpus to monitor. In this case
@@ -117,7 +108,6 @@ out_close_evlist:
 	perf_evlist__close(evlist);
 out_delete_maps:
 	perf_evlist__delete_maps(evlist);
-out_free_evlist:
 	perf_evlist__delete(evlist);
 	return err;
 }
tools/perf/util/evlist.c +12 -0
index 8ea933f..998e0d1 100644
@@ -50,6 +50,18 @@ struct perf_evlist *perf_evlist__new(void)
 	return evlist;
 }
 
+struct perf_evlist *perf_evlist__new_default(void)
+{
+	struct perf_evlist *evlist = perf_evlist__new();
+
+	if (evlist && perf_evlist__add_default(evlist)) {
+		perf_evlist__delete(evlist);
+		evlist = NULL;
+	}
+
+	return evlist;
+}
+
 /**
  * perf_evlist__set_id_pos - set the positions of event ids.
  * @evlist: selected event list
tools/perf/util/evlist.h +1 -0
index cae0bf5..ca016b1 100644
@@ -53,6 +53,7 @@ struct perf_evsel_str_handler {
 };
 
 struct perf_evlist *perf_evlist__new(void);
+struct perf_evlist *perf_evlist__new_default(void);
 void perf_evlist__init(struct perf_evlist *evlist, struct cpu_map *cpus,
 		       struct thread_map *threads);
 void perf_evlist__exit(struct perf_evlist *evlist);
Signature · 1 line
1.7.11.7
#6 lore raw

[tip:perf/core] perf evlist: Introduce perf_evlist__new_default function

perf evlist: Introduce perf_evlist__new_default function
Commit-ID:  b22d54b09a5448d3706929c6f0eae36429f4ec5d
Gitweb:     http://git.kernel.org/tip/b22d54b09a5448d3706929c6f0eae36429f4ec5d
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 1 Sep 2013 12:36:14 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 9 Oct 2013 11:24:34 -0300

perf evlist: Introduce perf_evlist__new_default function

Adding new common function to create evlist with default event. It spares some code lines in automated tests.

Cc Andi Kleen <[email protected]>
Cc Corey Ashford <[email protected]>
Cc David Ahern <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 tools/perf/tests/perf-record.c | 12 +-----------
 tools/perf/tests/task-exit.c   | 14 ++------------
 tools/perf/util/evlist.c       | 12 ++++++++++++
 tools/perf/util/evlist.h       |  1 +
 4 files changed, 16 insertions(+), 23 deletions(-)
tools/perf/tests/perf-record.c +1 -11
index b8a7056..82ac715 100644
@@ -45,7 +45,7 @@ int test__PERF_RECORD(void)
 	};
 	cpu_set_t cpu_mask;
 	size_t cpu_mask_size = sizeof(cpu_mask);
-	struct perf_evlist *evlist = perf_evlist__new();
+	struct perf_evlist *evlist = perf_evlist__new_default();
 	struct perf_evsel *evsel;
 	struct perf_sample sample;
 	const char *cmd = "sleep";
@@ -66,16 +66,6 @@ int test__PERF_RECORD(void)
 	}
 
 	/*
-	 * We need at least one evsel in the evlist, use the default
-	 * one: "cycles".
-	 */
-	err = perf_evlist__add_default(evlist);
-	if (err < 0) {
-		pr_debug("Not enough memory to create evsel\n");
-		goto out_delete_evlist;
-	}
-
-	/*
 	 * Create maps of threads and cpus to monitor. In this case
 	 * we start with all threads and cpus (-1, -1) but then in
 	 * perf_evlist__prepare_workload we'll fill in the only thread
tools/perf/tests/task-exit.c +2 -12
index 28fe589..b07f8a1 100644
@@ -37,20 +37,11 @@ int test__task_exit(void)
 	signal(SIGCHLD, sig_handler);
 	signal(SIGUSR1, sig_handler);
 
-	evlist = perf_evlist__new();
+	evlist = perf_evlist__new_default();
 	if (evlist == NULL) {
-		pr_debug("perf_evlist__new\n");
+		pr_debug("perf_evlist__new_default\n");
 		return -1;
 	}
-	/*
-	 * We need at least one evsel in the evlist, use the default
-	 * one: "cycles".
-	 */
-	err = perf_evlist__add_default(evlist);
-	if (err < 0) {
-		pr_debug("Not enough memory to create evsel\n");
-		goto out_free_evlist;
-	}
 
 	/*
 	 * Create maps of threads and cpus to monitor. In this case
@@ -117,7 +108,6 @@ out_close_evlist:
 	perf_evlist__close(evlist);
 out_delete_maps:
 	perf_evlist__delete_maps(evlist);
-out_free_evlist:
 	perf_evlist__delete(evlist);
 	return err;
 }
tools/perf/util/evlist.c +12 -0
index d21ab08..f0d71a9 100644
@@ -50,6 +50,18 @@ struct perf_evlist *perf_evlist__new(void)
 	return evlist;
 }
 
+struct perf_evlist *perf_evlist__new_default(void)
+{
+	struct perf_evlist *evlist = perf_evlist__new();
+
+	if (evlist && perf_evlist__add_default(evlist)) {
+		perf_evlist__delete(evlist);
+		evlist = NULL;
+	}
+
+	return evlist;
+}
+
 /**
  * perf_evlist__set_id_pos - set the positions of event ids.
  * @evlist: selected event list
tools/perf/util/evlist.h +1 -0
index 4edb500..871b55a 100644
@@ -53,6 +53,7 @@ struct perf_evsel_str_handler {
 };
 
 struct perf_evlist *perf_evlist__new(void);
+struct perf_evlist *perf_evlist__new_default(void);
 void perf_evlist__init(struct perf_evlist *evlist, struct cpu_map *cpus,
 		       struct thread_map *threads);
 void perf_evlist__exit(struct perf_evlist *evlist);
author
#7 lore raw

PATCH v2 4/25 perf tools: Adding throttle event data struct support

In reply to Jiri Olsa

Moving 'struct throttle_event' out of python code and making it global as any other event.

There's no usage of throttling events in any perf commands so far (besides python support), but we'll need this event data backup for upcomming test.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/util/event.h   |  7 +++++++
 tools/perf/util/python.c  |  7 -------
 tools/perf/util/session.c | 13 +++++++++++++
 3 files changed, 20 insertions(+), 7 deletions(-)
tools/perf/util/event.h +7 -0
index 93130d8..4971bfc 100644
@@ -48,6 +48,12 @@ struct read_event {
 	u64 id;
 };
 
+struct throttle_event {
+	struct perf_event_header header;
+	u64 time;
+	u64 id;
+	u64 stream_id;
+};
 
 #define PERF_SAMPLE_MASK				\
 	(PERF_SAMPLE_IP | PERF_SAMPLE_TID |		\
@@ -163,6 +169,7 @@ union perf_event {
 	struct fork_event		fork;
 	struct lost_event		lost;
 	struct read_event		read;
+	struct throttle_event		throttle;
 	struct sample_event		sample;
 	struct attr_event		attr;
 	struct event_type_event		event_type;
tools/perf/util/python.c +0 -7
index 381f4fd..aa76519 100644
@@ -33,13 +33,6 @@ int eprintf(int level, const char *fmt, ...)
 # define PyVarObject_HEAD_INIT(type, size) PyObject_HEAD_INIT(type) size,
 #endif
 
-struct throttle_event {
-	struct perf_event_header header;
-	u64			 time;
-	u64			 id;
-	u64			 stream_id;
-};
-
 PyMODINIT_FUNC initperf(void);
 
 #define member_def(type, member, ptype, help) \
tools/perf/util/session.c +13 -0
index 07642a7..787c234 100644
@@ -376,6 +376,17 @@ static void perf_event__read_swap(union perf_event *event, bool sample_id_all)
 		swap_sample_id_all(event, &event->read + 1);
 }
 
+static void perf_event__throttle_swap(union perf_event *event,
+				      bool sample_id_all)
+{
+	event->throttle.time	  = bswap_64(event->throttle.time);
+	event->throttle.id	  = bswap_64(event->throttle.id);
+	event->throttle.stream_id = bswap_64(event->throttle.stream_id);
+
+	if (sample_id_all)
+		swap_sample_id_all(event, &event->throttle + 1);
+}
+
 static u8 revbyte(u8 b)
 {
 	int rev = (b >> 4) | ((b & 0xf) << 4);
@@ -460,6 +471,8 @@ static perf_event__swap_op perf_event__swap_ops[] = {
 	[PERF_RECORD_EXIT]		  = perf_event__task_swap,
 	[PERF_RECORD_LOST]		  = perf_event__all64_swap,
 	[PERF_RECORD_READ]		  = perf_event__read_swap,
+	[PERF_RECORD_THROTTLE]		  = perf_event__throttle_swap,
+	[PERF_RECORD_UNTHROTTLE]	  = perf_event__throttle_swap,
 	[PERF_RECORD_SAMPLE]		  = perf_event__all64_swap,
 	[PERF_RECORD_HEADER_ATTR]	  = perf_event__hdr_attr_swap,
 	[PERF_RECORD_HEADER_EVENT_TYPE]	  = perf_event__event_type_swap,
Signature · 1 line
1.7.11.7
#8 lore raw

[tip:perf/core] perf tools: Adding throttle event data struct support

perf tools: Adding throttle event data struct support
Commit-ID:  dd96c46b5c765a779d8c35cc7d1df7515b4c7baf
Gitweb:     http://git.kernel.org/tip/dd96c46b5c765a779d8c35cc7d1df7515b4c7baf
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 1 Sep 2013 12:36:15 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Wed, 9 Oct 2013 11:39:18 -0300

perf tools: Adding throttle event data struct support

Moving 'struct throttle_event' out of python code and making it global as any other event.

There's no usage of throttling events in any perf commands so far (besides python support), but we'll need this event data backup for upcoming test.

Cc Andi Kleen <[email protected]>
Cc Corey Ashford <[email protected]>
Cc David Ahern <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 tools/perf/util/event.h   |  7 +++++++
 tools/perf/util/python.c  |  7 -------
 tools/perf/util/session.c | 13 +++++++++++++
 3 files changed, 20 insertions(+), 7 deletions(-)
tools/perf/util/event.h +7 -0
index 17d9e16..9b7d4d3 100644
@@ -61,6 +61,12 @@ struct read_event {
 	u64 id;
 };
 
+struct throttle_event {
+	struct perf_event_header header;
+	u64 time;
+	u64 id;
+	u64 stream_id;
+};
 
 #define PERF_SAMPLE_MASK				\
 	(PERF_SAMPLE_IP | PERF_SAMPLE_TID |		\
@@ -178,6 +184,7 @@ union perf_event {
 	struct fork_event		fork;
 	struct lost_event		lost;
 	struct read_event		read;
+	struct throttle_event		throttle;
 	struct sample_event		sample;
 	struct attr_event		attr;
 	struct event_type_event		event_type;
tools/perf/util/python.c +0 -7
index a24ce0a..06efd02 100644
@@ -33,13 +33,6 @@ int eprintf(int level, const char *fmt, ...)
 # define PyVarObject_HEAD_INIT(type, size) PyObject_HEAD_INIT(type) size,
 #endif
 
-struct throttle_event {
-	struct perf_event_header header;
-	u64			 time;
-	u64			 id;
-	u64			 stream_id;
-};
-
 PyMODINIT_FUNC initperf(void);
 
 #define member_def(type, member, ptype, help) \
tools/perf/util/session.c +13 -0
index b97f468..d1e4495 100644
@@ -397,6 +397,17 @@ static void perf_event__read_swap(union perf_event *event, bool sample_id_all)
 		swap_sample_id_all(event, &event->read + 1);
 }
 
+static void perf_event__throttle_swap(union perf_event *event,
+				      bool sample_id_all)
+{
+	event->throttle.time	  = bswap_64(event->throttle.time);
+	event->throttle.id	  = bswap_64(event->throttle.id);
+	event->throttle.stream_id = bswap_64(event->throttle.stream_id);
+
+	if (sample_id_all)
+		swap_sample_id_all(event, &event->throttle + 1);
+}
+
 static u8 revbyte(u8 b)
 {
 	int rev = (b >> 4) | ((b & 0xf) << 4);
@@ -482,6 +493,8 @@ static perf_event__swap_op perf_event__swap_ops[] = {
 	[PERF_RECORD_EXIT]		  = perf_event__task_swap,
 	[PERF_RECORD_LOST]		  = perf_event__all64_swap,
 	[PERF_RECORD_READ]		  = perf_event__read_swap,
+	[PERF_RECORD_THROTTLE]		  = perf_event__throttle_swap,
+	[PERF_RECORD_UNTHROTTLE]	  = perf_event__throttle_swap,
 	[PERF_RECORD_SAMPLE]		  = perf_event__all64_swap,
 	[PERF_RECORD_HEADER_ATTR]	  = perf_event__hdr_attr_swap,
 	[PERF_RECORD_HEADER_EVENT_TYPE]	  = perf_event__event_type_swap,
author
#9 lore raw

PATCH v2 5/25 perf tests: Add simple session read/write test

In reply to Jiri Olsa

Adding simple session read/write test to keep up
with file format changes.

Dumping the session to the file with several events and reading it again back.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/Makefile               |   1 +
 tools/perf/tests/builtin-test.c   |   4 +
 tools/perf/tests/session-simple.c | 602 ++++++++++++++++++++++++++++++++++++++
 tools/perf/tests/tests.h          |   1 +
 4 files changed, 608 insertions(+)
 create mode 100644 tools/perf/tests/session-simple.c
tools/perf/Makefile +1 -0
index ecebfd0..7484119 100644
@@ -394,6 +394,7 @@ ifeq ($(ARCH),x86)
 LIB_OBJS += $(OUTPUT)tests/perf-time-to-tsc.o
 endif
 LIB_OBJS += $(OUTPUT)tests/code-reading.o
+LIB_OBJS += $(OUTPUT)tests/session-simple.o
 
 BUILTIN_OBJS += $(OUTPUT)builtin-annotate.o
 BUILTIN_OBJS += $(OUTPUT)builtin-bench.o
tools/perf/tests/builtin-test.c +4 -0
index 8ad9415..e40aec9 100644
@@ -108,6 +108,10 @@ static struct test {
 		.func = test__sample_parsing,
 	},
 	{
+		.desc = "Test session - simple read/write",
+		.func = test__session_simple,
+	},
+	{
 		.func = NULL,
 	},
 };
tools/perf/tests/session-simple.c new file +602 -0

602 lines not on this page · raw message

tools/perf/tests/tests.h +1 -0
index 83d5b71..94b04cd 100644
@@ -38,5 +38,6 @@ int test__sw_clock_freq(void);
 int test__perf_time_to_tsc(void);
 int test__code_reading(void);
 int test__sample_parsing(void);
+int test__session_simple(void);
 
 #endif /* TESTS_H */
Signature · 1 line
1.7.11.7
author
#10 lore raw

PATCH v2 6/25 perf tests: Add session reading test for little endian perf data

In reply to Jiri Olsa

Adding session reading test for little endian perf data included as binary data in the test object.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/Makefile                |   1 +
 tools/perf/tests/perf.data.v2.le.h | 219 +++++++++++++++++++++++++++++++++++++
 tools/perf/tests/session-simple.c  |  33 ++++++
 3 files changed, 253 insertions(+)
 create mode 100644 tools/perf/tests/perf.data.v2.le.h
tools/perf/Makefile +1 -0
index 7484119..c4498c2 100644
@@ -292,6 +292,7 @@ LIB_H += ui/helpline.h
 LIB_H += ui/progress.h
 LIB_H += ui/util.h
 LIB_H += ui/ui.h
+LIB_H += tests/perf.data.v2.le.h
 
 LIB_OBJS += $(OUTPUT)util/abspath.o
 LIB_OBJS += $(OUTPUT)util/alias.o
tools/perf/tests/perf.data.v2.le.h new file +219 -0

219 lines not on this page · raw message

tools/perf/tests/session-simple.c +33 -0
index 612f84d..e975ec3 100644
@@ -21,6 +21,10 @@
 #define EVENTS_UNTHROTTLE 3
 #define EVENTS_SAMPLE 4
 
+char data_file_v2_le[] = {
+#include "perf.data.v2.le.h"
+};
+
 static int events_mmap;
 static int events_lost;
 static int events_comm;
@@ -592,11 +596,40 @@ static int test_generated_data(void)
 	return err;
 }
 
+static int file_write(char *file, char *data, ssize_t size)
+{
+	int fd = open(file, O_TRUNC|O_RDWR);
+	int err = 0;
+
+	if (size != write(fd, data, size))
+		err = -1;
+
+	close(fd);
+	return err;
+}
+
+static int test_file_data(char *data, ssize_t size)
+{
+	char *file = get_file();
+	int err = 0;
+
+	TEST_ASSERT_VAL("failed to get temporary file", file);
+
+	err = file_write(file, data, size);
+	if (!err)
+		err = session_read(file);
+
+	unlink(file);
+	return err;
+}
+
 int test__session_simple(void)
 {
 	int err;
 
 	pr_debug("Testing generated data\n");
 	err = test_generated_data();
+	pr_debug("Testing v2 LE data\n");
+	err |= test_file_data(data_file_v2_le, sizeof(data_file_v2_le));
 	return err;
 }
Signature · 1 line
1.7.11.7
author
#11 lore raw

PATCH v2 7/25 perf tests: Add session reading test for big endian perf data

In reply to Jiri Olsa

Adding session reading test for big endian perf data included as binary data in the test object.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/Makefile                |   1 +
 tools/perf/tests/perf.data.v2.be.h | 265 +++++++++++++++++++++++++++++++++++++
 tools/perf/tests/session-simple.c  |   6 +
 3 files changed, 272 insertions(+)
 create mode 100644 tools/perf/tests/perf.data.v2.be.h
tools/perf/Makefile +1 -0
index c4498c2..2510355 100644
@@ -293,6 +293,7 @@ LIB_H += ui/progress.h
 LIB_H += ui/util.h
 LIB_H += ui/ui.h
 LIB_H += tests/perf.data.v2.le.h
+LIB_H += tests/perf.data.v2.be.h
 
 LIB_OBJS += $(OUTPUT)util/abspath.o
 LIB_OBJS += $(OUTPUT)util/alias.o
tools/perf/tests/perf.data.v2.be.h new file +265 -0

265 lines not on this page · raw message

tools/perf/tests/session-simple.c +6 -0
index e975ec3..4f58c0f 100644
@@ -25,6 +25,10 @@ char data_file_v2_le[] = {
 #include "perf.data.v2.le.h"
 };
 
+char data_file_v2_be[] = {
+#include "perf.data.v2.be.h"
+};
+
 static int events_mmap;
 static int events_lost;
 static int events_comm;
@@ -631,5 +635,7 @@ int test__session_simple(void)
 	err = test_generated_data();
 	pr_debug("Testing v2 LE data\n");
 	err |= test_file_data(data_file_v2_le, sizeof(data_file_v2_le));
+	pr_debug("Testing v2 BE data\n");
+	err |= test_file_data(data_file_v2_be, sizeof(data_file_v2_be));
 	return err;
 }
Signature · 1 line
1.7.11.7
author
#12 lore raw

PATCH v2 8/25 perf doc: Add perf data file documentation

In reply to Jiri Olsa

Adding perf data file documentation.

TODO Finish the FEATURES section

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/Documentation/perf-data-file-v2.txt | 264 +++++++++++++++++++++++++
 1 file changed, 264 insertions(+)
 create mode 100644 tools/perf/Documentation/perf-data-file-v2.txt
tools/perf/Documentation/perf-data-file-v2.txt new file +264 -0
new file mode 100644
index 0000000..f5c1d5b8
@@ -0,0 +1,264 @@
+perf-data-file-v2(5)
+====================
+
+NAME
+----
+perf-data-file-v2 - The perf tool file format version 2
+
+
+DESCRIPTION
+-----------
+Following text describes version 2 of the perf data file format,
+which is version that is currently used by perf tool.
+
+The perf data file format is composed of several sections
+describing monitored events and the data itself.
+
+High level view of the format:
+  FILE HEADER
+  EVENT IDS
+  EVENT ATTRIBUTES
+  EVENT TYPES
+  EVENT DATA
+  FEATURES
+
+
+COMMON PRIMITIVES
+-----------------
+Following definitions are used as primitives on multiple places
+within the perf data file.
+
+struct perf_file_section::
+  Used to determine location of data sections within the file.
+
+  struct perf_file_section {
+        u64 offset;
+        u64 size;
+  };
+
+  offset  section location (offset) within the file
+  size    section size in bytes
+
+
+FILE HEADER
+-----------
+Starting point of the data file with magic bytes and global
+section information.
+
+The section contains following data:
+  struct perf_file_header header
+
+struct perf_file_header::
+  struct perf_file_header {
+        u64                             magic;
+        u64                             size;
+        u64                             attr_size;
+        struct perf_file_section        attrs;
+        struct perf_file_section        data;
+        struct perf_file_section        event_types;
+        DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
+  }
+
+  magic           ID 'PERFFILE|PERFILE2'
+  size            header size sizeof(struct perf_file_header)
+  attr_size       attribute size sizeof(struct perf_file_attr)
+                  (see EVENT ATTRIBUTES)
+  attrs           location of 'EVENT ATTRIBUTES'
+  data            location of 'EVENT DATA'
+  event_types     location of 'EVENT TYPES'
+  adds_features   'FEATURES' bitmask
+
+
+EVENT IDS
+---------
+This section contains CPU related IDs for each monitored event.
+It is used only if there was more than 1 events monitored. The
+reason is to be able to map event's data to event definition.
+
+The section contains following data:
+  u64  event1_ids[]
+  u64  event2_ids[]
+  ...
+  u64  eventX_ids[]
+
+eventX_ids::
+  per-cpu event X IDs array
+  this array is referenced in 'EVENT ATTRIBUTES'
+
+
+EVENT ATTRIBUTES
+----------------
+This section contains attribute information together with the
+location of the IDs array for each event.
+
+The section contains following data:
+  struct perf_file_attr  attrs[]
+
+struct perf_file_attr::
+  struct perf_file_attr {
+        struct perf_event_attr    attr;
+        struct perf_file_section  ids;
+  }
+
+  attr  perf event interface structure for each event
+  ids   IDs location for event in 'EVENT IDS' section
+
+
+EVENT TYPES
+-----------
+This section map event's config value (struct perf_event_attr::config)
+with the event name.
+
+The section contains following data:
+  struct perf_trace_event_type  types[]
+
+#define MAX_EVENT_NAME 64
+
+struct perf_trace_event_type::
+  struct perf_trace_event_type {
+        u64   event_id;
+        char  name[MAX_EVENT_NAME];
+  };
+
+  event_id  struct perf_event_attr::config value
+  name      event name
+
+NOTE This section got deprecated and its data are no longer stored.
+     The 'event_types' location in the header is zero-ed.
+
+
+EVENT DATA
+----------
+This section contains blob of all events data - auxiliary events
+and samples.
+
+
+FEATURES
+--------
+This section contains various configuration data and its contents
+depends on the header's adds_features bitmask. Each bit in this
+bitmask represent a single feature (0 - NOT present, 1 present).
+
+The section contains following data:
+  struct perf_file_section feature_locations[nr_features];
+
+  nr_features        number of present features bitmap_weight(adds_features)
+  feature_locations  location of each feature data
+
+Available features (possible max 256):
+  enum {
+        HEADER_RESERVED         = 0,
+        HEADER_TRACING_DATA     = 1,
+        HEADER_BUILD_ID,
+        HEADER_HOSTNAME,
+        HEADER_OSRELEASE,
+        HEADER_VERSION,
+        HEADER_ARCH,
+        HEADER_NRCPUS,
+        HEADER_CPUDESC,
+        HEADER_CPUID,
+        HEADER_TOTAL_MEM,
+        HEADER_CMDLINE,
+        HEADER_EVENT_DESC,
+        HEADER_CPU_TOPOLOGY,
+        HEADER_NUMA_TOPOLOGY,
+        HEADER_BRANCH_STACK,
+        HEADER_PMU_MAPPINGS,
+        HEADER_GROUP_DESC,
+        HEADER_LAST_FEATURE,
+        HEADER_FEAT_BITS        = 256,
+  }
+
+
+TODO finish FEATURES::* sections descriptions.
+
+FEATURES::HEADER_RESERVED
+-------------------------
+allways zero bit
+
+
+FEATURES::HEADER_TRACING_DATA
+-----------------------------
+tracing data
+
+
+FEATURES::HEADER_BUILD_ID
+-------------------------
+build ids
+
+
+FEATURES::HEADER_HOSTNAME
+-------------------------
+uname hostname
+
+
+FEATURES::HEADER_OSRELEASE
+--------------------------
+uname osrelease
+
+
+FEATURES::HEADER_VERSION
+------------------------
+perf version
+
+
+FEATURES::HEADER_ARCH
+---------------------
+uname arch
+
+
+FEATURES::HEADER_NRCPUS
+-----------------------
+offset          size(B) description
+0               4       sysconf(_SC_NPROCESSORS_CONF)
+4               4       sysconf(_SC_NPROCESSORS_ONLN)
+
+
+FEATURES::HEADER_CPUDESC
+------------------------
+/proc/cpuinfo data for 'Model name'
+
+
+FEATURES::HEADER_CPUID
+----------------------
+'vendor,family,model,step'
+
+
+FEATURES::HEADER_TOTAL_MEM
+--------------------------
+/proc/meminfo data for 'MemTotal'
+
+
+FEATURES::HEADER_CMDLINE
+------------------------
+perf complete command line
+
+
+FEATURES::HEADER_EVENT_DESC
+---------------------------
+event attributes plus related info(ids)
+
+
+FEATURES::HEADER_CPU_TOPOLOGY
+-----------------------------
+cpu topology
+
+
+FEATURES::HEADER_NUMA_TOPOLOGY
+------------------------------
+numa topology
+
+
+FEATURES::HEADER_BRANCH_STACK
+-----------------------------
+nothing
+
+
+FEATURES::HEADER_PMU_MAPPINGS
+-----------------------------
+available pmus
+
+
+FEATURES::HEADER_GROUP_DESC
+---------------------------
+event groups info
Signature · 1 line
1.7.11.7
author
#13 lore raw

PATCH v2 9/25 perf tools: Introduce perf data file version CHECK macro

In reply to Jiri Olsa

Introducing perf data file version CHECK macro, so the check is easily extensible for new versions.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/util/header.c | 27 ++++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)
tools/perf/util/header.c +16 -11
index a33197a..bcded62 100644
@@ -2496,18 +2496,23 @@ static int check_magic_endian(u64 magic, uint64_t hdr_sz,
 	 * - encode endianness of file
 	 */
 
-	/* check magic number with one endianness */
-	if (magic == __perf_magic2)
-		return 0;
-
-	/* check magic number with opposite endianness */
-	if (magic != __perf_magic2_sw)
-		return -1;
+#define CHECK(__m, __v)						\
+do {								\
+	/* check magic number with one endianness */		\
+	if (magic == __m)					\
+		goto v ## __v;					\
+	/* check magic number with opposite endianness */	\
+	if (magic != __m ## _sw)				\
+		break;						\
+	ph->needs_swap = true;					\
+ v ## __v:							\
+	ph->version = __v;					\
+	return 0;						\
+} while (0)
+
+	CHECK(__perf_magic2, PERF_HEADER_VERSION_2);
 
-	ph->needs_swap = true;
-	ph->version = PERF_HEADER_VERSION_2;
-
-	return 0;
+	return -1;
 }
 
 int perf_file_header__read(struct perf_file_header *header,
Signature · 1 line
1.7.11.7
author
#14 lore raw

PATCH v2 10/25 perf tools: Introduce swap_features function

In reply to Jiri Olsa

Introducing swap_features function to make the swapping code more clear and extensible.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/util/header.c | 69 ++++++++++++++++++++++++------------------------
 1 file changed, 35 insertions(+), 34 deletions(-)
tools/perf/util/header.c +35 -34
index bcded62..164510e 100644
@@ -2515,6 +2515,39 @@ do {								\
 	return -1;
 }
 
+static void swap_features(unsigned long *adds_features)
+{
+	/*
+	 * feature bitmap is declared as an array of unsigned longs --
+	 * not good since its size can differ between the host that
+	 * generated the data file and the host analyzing the file.
+	 *
+	 * We need to handle endianness, but we don't know the size of
+	 * the unsigned long where the file was generated. Take a best
+	 * guess at determining it: try 64-bit swap first (ie., file
+	 * created on a 64-bit host), and check if the hostname feature
+	 * bit is set (this feature bit is forced on as of fbe96f2).
+	 * If the bit is not, undo the 64-bit swap and try a 32-bit
+	 * swap. If the hostname bit is still not set (e.g., older data
+	 * file), punt and fallback to the original behavior --
+	 * clearing all feature bits and setting buildid.
+	 */
+	mem_bswap_64(adds_features, BITS_TO_U64(HEADER_FEAT_BITS));
+
+	if (!test_bit(HEADER_HOSTNAME, adds_features)) {
+		/* unswap as u64 */
+		mem_bswap_64(adds_features, BITS_TO_U64(HEADER_FEAT_BITS));
+
+		/* unswap as u32 */
+		mem_bswap_32(adds_features, BITS_TO_U32(HEADER_FEAT_BITS));
+	}
+
+	if (!test_bit(HEADER_HOSTNAME, adds_features)) {
+		bitmap_zero(adds_features, HEADER_FEAT_BITS);
+		set_bit(HEADER_BUILD_ID, adds_features);
+	}
+}
+
 int perf_file_header__read(struct perf_file_header *header,
 			   struct perf_header *ph, int fd)
 {
@@ -2543,40 +2576,8 @@ int perf_file_header__read(struct perf_file_header *header,
 			bitmap_zero(header->adds_features, HEADER_FEAT_BITS);
 		else
 			return -1;
-	} else if (ph->needs_swap) {
-		/*
-		 * feature bitmap is declared as an array of unsigned longs --
-		 * not good since its size can differ between the host that
-		 * generated the data file and the host analyzing the file.
-		 *
-		 * We need to handle endianness, but we don't know the size of
-		 * the unsigned long where the file was generated. Take a best
-		 * guess at determining it: try 64-bit swap first (ie., file
-		 * created on a 64-bit host), and check if the hostname feature
-		 * bit is set (this feature bit is forced on as of fbe96f2).
-		 * If the bit is not, undo the 64-bit swap and try a 32-bit
-		 * swap. If the hostname bit is still not set (e.g., older data
-		 * file), punt and fallback to the original behavior --
-		 * clearing all feature bits and setting buildid.
-		 */
-		mem_bswap_64(&header->adds_features,
-			    BITS_TO_U64(HEADER_FEAT_BITS));
-
-		if (!test_bit(HEADER_HOSTNAME, header->adds_features)) {
-			/* unswap as u64 */
-			mem_bswap_64(&header->adds_features,
-				    BITS_TO_U64(HEADER_FEAT_BITS));
-
-			/* unswap as u32 */
-			mem_bswap_32(&header->adds_features,
-				    BITS_TO_U32(HEADER_FEAT_BITS));
-		}
-
-		if (!test_bit(HEADER_HOSTNAME, header->adds_features)) {
-			bitmap_zero(header->adds_features, HEADER_FEAT_BITS);
-			set_bit(HEADER_BUILD_ID, header->adds_features);
-		}
-	}
+	} else if (ph->needs_swap)
+		swap_features(header->adds_features);
 
 	memcpy(&ph->adds_features, &header->adds_features,
 	       sizeof(ph->adds_features));
Signature · 1 line
1.7.11.7
author
#15 lore raw

PATCH v2 11/25 perf tools: Introduce swap_header function

In reply to Jiri Olsa

Introducing swap_header function to make the swapping code more clear and extensible.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/util/header.c | 31 ++++++++++++++++++-------------
 1 file changed, 18 insertions(+), 13 deletions(-)
tools/perf/util/header.c +18 -13
index 164510e..fba6a9a 100644
@@ -2548,6 +2548,22 @@ static void swap_features(unsigned long *adds_features)
 	}
 }
 
+static int swap_header(struct perf_file_header *header)
+{
+	mem_bswap_64(header, offsetof(struct perf_file_header, adds_features));
+
+	if (header->size != sizeof(*header)) {
+		/* Support the previous format */
+		if (header->size == offsetof(typeof(*header), adds_features))
+			bitmap_zero(header->adds_features, HEADER_FEAT_BITS);
+		else
+			return -1;
+	} else
+		swap_features(header->adds_features);
+
+	return 0;
+}
+
 int perf_file_header__read(struct perf_file_header *header,
 			   struct perf_header *ph, int fd)
 {
@@ -2565,19 +2581,8 @@ int perf_file_header__read(struct perf_file_header *header,
 		return -1;
 	}
 
-	if (ph->needs_swap) {
-		mem_bswap_64(header, offsetof(struct perf_file_header,
-			     adds_features));
-	}
-
-	if (header->size != sizeof(*header)) {
-		/* Support the previous format */
-		if (header->size == offsetof(typeof(*header), adds_features))
-			bitmap_zero(header->adds_features, HEADER_FEAT_BITS);
-		else
-			return -1;
-	} else if (ph->needs_swap)
-		swap_features(header->adds_features);
+	if (ph->needs_swap && swap_header(header))
+		return -1;
 
 	memcpy(&ph->adds_features, &header->adds_features,
 	       sizeof(ph->adds_features));
Signature · 1 line
1.7.11.7
author
#16 lore raw

PATCH v2 12/25 perf tools: Separate version 2 specific perf data header bits

In reply to Jiri Olsa

Separating version 2 specific perf data header bits, so the code could be extented with new format version.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/util/header.c | 122 +++++++++++++++++++++++++++++++----------------
 tools/perf/util/header.h |  16 +++++--
 2 files changed, 92 insertions(+), 46 deletions(-)
tools/perf/util/header.c +80 -42
index fba6a9a..3f58a3f 100644
@@ -2319,18 +2319,21 @@ int perf_session__write_header(struct perf_session *session,
 		.magic	   = PERF_MAGIC,
 		.size	   = sizeof(f_header),
 		.attr_size = sizeof(f_attr),
-		.attrs = {
-			.offset = attr_offset,
-			.size   = evlist->nr_entries * sizeof(f_attr),
-		},
-		.data = {
-			.offset = header->data_offset,
-			.size	= header->data_size,
-		},
+		.v2        = {
+			.attrs = {
+				.offset = attr_offset,
+				.size   = evlist->nr_entries * sizeof(f_attr),
+			},
+			.data = {
+				.offset = header->data_offset,
+				.size	= header->data_size,
+			},
 		/* event_types is ignored, store zeros */
+		},
 	};
 
-	memcpy(&f_header.adds_features, &header->adds_features, sizeof(header->adds_features));
+	memcpy(&f_header.v2.adds_features, &header->adds_features,
+	       sizeof(header->adds_features));
 
 	lseek(fd, 0, SEEK_SET);
 	err = do_write(fd, &f_header, sizeof(f_header));
@@ -2548,22 +2551,34 @@ static void swap_features(unsigned long *adds_features)
 	}
 }
 
-static int swap_header(struct perf_file_header *header)
+static int swap_header_v2(struct perf_file_header *header)
 {
-	mem_bswap_64(header, offsetof(struct perf_file_header, adds_features));
+	struct perf_file_header_v2 *v2 = &header->v2;
+
+	mem_bswap_64(v2, offsetof(struct perf_file_header_v2,
+		     adds_features));
 
 	if (header->size != sizeof(*header)) {
 		/* Support the previous format */
-		if (header->size == offsetof(typeof(*header), adds_features))
-			bitmap_zero(header->adds_features, HEADER_FEAT_BITS);
+		if (header->size == offsetof(typeof(*header), v2.adds_features))
+			bitmap_zero(v2->adds_features, HEADER_FEAT_BITS);
 		else
 			return -1;
 	} else
-		swap_features(header->adds_features);
+		swap_features(v2->adds_features);
 
 	return 0;
 }
 
+static int swap_header(struct perf_file_header *header)
+{
+	/* swap the generic part */
+	mem_bswap_64(header, offsetof(struct perf_file_header, v2));
+
+	/* version specific swap */
+	return swap_header_v2(header);
+}
+
 int perf_file_header__read(struct perf_file_header *header,
 			   struct perf_header *ph, int fd)
 {
@@ -2581,16 +2596,7 @@ int perf_file_header__read(struct perf_file_header *header,
 		return -1;
 	}
 
-	if (ph->needs_swap && swap_header(header))
-		return -1;
-
-	memcpy(&ph->adds_features, &header->adds_features,
-	       sizeof(ph->adds_features));
-
-	ph->data_offset  = header->data.offset;
-	ph->data_size	 = header->data.size;
-	ph->feat_offset  = header->data.offset + header->data.size;
-	return 0;
+	return ph->needs_swap ? swap_header(header) : 0;
 }
 
 static int perf_file_section__process(struct perf_file_section *section,
@@ -2742,12 +2748,13 @@ static int perf_evlist__prepare_tracepoint_events(struct perf_evlist *evlist,
 	return 0;
 }
 
-int perf_session__read_header(struct perf_session *session)
+static int __perf_session__read_header_v2(struct perf_session *session,
+					  struct perf_file_header *header)
 {
-	struct perf_header *header = &session->header;
-	struct perf_file_header	f_header;
-	struct perf_file_attr	f_attr;
-	u64			f_id;
+	struct perf_header		*ph = &session->header;
+	struct perf_file_header_v2	*v2 = &header->v2;
+	struct perf_file_attr		f_attr;
+	u64				f_id;
 	int nr_attrs, nr_ids, i, j;
 	int fd = session->fd;
 
@@ -2755,23 +2762,17 @@ int perf_session__read_header(struct perf_session *session)
 	if (session->evlist == NULL)
 		return -ENOMEM;
 
-	if (session->fd_pipe)
-		return perf_header__read_pipe(session);
-
-	if (perf_file_header__read(&f_header, header, fd) < 0)
-		return -EINVAL;
-
-	nr_attrs = f_header.attrs.size / f_header.attr_size;
-	lseek(fd, f_header.attrs.offset, SEEK_SET);
+	nr_attrs = v2->attrs.size / header->attr_size;
+	lseek(fd, v2->attrs.offset, SEEK_SET);
 
 	for (i = 0; i < nr_attrs; i++) {
 		struct perf_evsel *evsel;
 		off_t tmp;
 
-		if (read_attr(fd, header, &f_attr) < 0)
+		if (read_attr(fd, ph, &f_attr) < 0)
 			goto out_errno;
 
-		if (header->needs_swap)
+		if (ph->needs_swap)
 			perf_event__attr_swap(&f_attr.attr);
 
 		tmp = lseek(fd, 0, SEEK_CUR);
@@ -2780,7 +2781,7 @@ int perf_session__read_header(struct perf_session *session)
 		if (evsel == NULL)
 			goto out_delete_evlist;
 
-		evsel->needs_swap = header->needs_swap;
+		evsel->needs_swap = ph->needs_swap;
 		/*
 		 * Do it before so that if perf_evsel__alloc_id fails, this
 		 * entry gets purged too at perf_evlist__delete().
@@ -2799,7 +2800,8 @@ int perf_session__read_header(struct perf_session *session)
 		lseek(fd, f_attr.ids.offset, SEEK_SET);
 
 		for (j = 0; j < nr_ids; j++) {
-			if (perf_header__getbuffer64(header, fd, &f_id, sizeof(f_id)))
+			if (perf_header__getbuffer64(ph, fd, &f_id,
+						     sizeof(f_id)))
 				goto out_errno;
 
 			perf_evlist__id_add(session->evlist, evsel, 0, j, f_id);
@@ -2810,7 +2812,7 @@ int perf_session__read_header(struct perf_session *session)
 
 	symbol_conf.nr_events = nr_attrs;
 
-	perf_header__process_sections(header, fd, &session->pevent,
+	perf_header__process_sections(ph, fd, &session->pevent,
 				      perf_file_section__process);
 
 	if (perf_evlist__prepare_tracepoint_events(session->evlist,
@@ -2827,6 +2829,42 @@ out_delete_evlist:
 	return -ENOMEM;
 }
 
+
+static int perf_session__read_header_v2(struct perf_session *session,
+					struct perf_file_header *header)
+{
+	struct perf_header *ph = &session->header;
+	struct perf_file_header_v2 *v2 = &header->v2;
+
+	memcpy(&ph->adds_features, &v2->adds_features,
+	       sizeof(ph->adds_features));
+
+	ph->data_offset  = v2->data.offset;
+	ph->data_size	 = v2->data.size;
+	ph->feat_offset  = v2->data.offset + v2->data.size;
+
+	return __perf_session__read_header_v2(session, header);
+}
+
+static int perf_header_read_file(struct perf_session *session)
+{
+	struct perf_file_header header;
+
+	if (perf_file_header__read(&header, &session->header, session->fd))
+		return -1;
+
+	/* read v2 specific data */
+	return perf_session__read_header_v2(session, &header);
+}
+
+int perf_session__read_header(struct perf_session *session)
+{
+	if (session->fd_pipe)
+		return perf_header__read_pipe(session);
+
+	return perf_header_read_file(session) < 0 ? -EINVAL : 0;
+}
+
 int perf_event__synthesize_attr(struct perf_tool *tool,
 				struct perf_event_attr *attr, u32 ids, u64 *id,
 				perf_event__handler_t process)
tools/perf/util/header.h +12 -4
index 307c9ae..c6ea4603 100644
@@ -44,10 +44,7 @@ struct perf_file_section {
 	u64 size;
 };
 
-struct perf_file_header {
-	u64				magic;
-	u64				size;
-	u64				attr_size;
+struct perf_file_header_v2 {
 	struct perf_file_section	attrs;
 	struct perf_file_section	data;
 	/* event_types is ignored */
@@ -55,6 +52,17 @@ struct perf_file_header {
 	DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
 };
 
+struct perf_file_header {
+	u64	magic;
+	u64	size;
+	u64	attr_size;
+
+	/* version specific data */
+	union {
+		struct perf_file_header_v2 v2;
+	};
+};
+
 struct perf_pipe_file_header {
 	u64				magic;
 	u64				size;
Signature · 1 line
1.7.11.7
author
#17 lore raw

PATCH v2 13/25 perf tools: Using evlist as a holder for event_desc feature

In reply to Jiri Olsa

Using evlist as a holder for events stored via event_desc feature. The event_desc events data is same as the one stored via main header. We will use event_desc data to replace the main header data source in next patches.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/util/header.c | 84 ++++++++++++++++++++++--------------------------
 1 file changed, 38 insertions(+), 46 deletions(-)
tools/perf/util/header.c +38 -46
index 3f58a3f..e43879e 100644
@@ -1208,28 +1208,11 @@ static void print_cpu_topology(struct perf_header *ph, int fd __maybe_unused,
 	}
 }
 
-static void free_event_desc(struct perf_evsel *events)
-{
-	struct perf_evsel *evsel;
-
-	if (!events)
-		return;
-
-	for (evsel = events; evsel->attr.size; evsel++) {
-		if (evsel->name)
-			free(evsel->name);
-		if (evsel->id)
-			free(evsel->id);
-	}
-
-	free(events);
-}
-
-static struct perf_evsel *
+static struct perf_evlist *
 read_event_desc(struct perf_header *ph, int fd)
 {
-	struct perf_evsel *evsel, *events = NULL;
-	u64 *id;
+	struct perf_evlist *evlist = NULL;
+	struct perf_evsel *evsel;
 	void *buf = NULL;
 	u32 nre, sz, nr, i, j;
 	ssize_t ret;
@@ -1255,18 +1238,15 @@ read_event_desc(struct perf_header *ph, int fd)
 	if (!buf)
 		goto error;
 
-	/* the last event terminates with evsel->attr.size == 0: */
-	events = calloc(nre + 1, sizeof(*events));
-	if (!events)
+	evlist = perf_evlist__new();
+	if (!evlist)
 		goto error;
 
 	msz = sizeof(evsel->attr);
 	if (sz < msz)
 		msz = sz;
 
-	for (i = 0, evsel = events; i < nre; evsel++, i++) {
-		evsel->idx = i;
-
+	for (i = 0; i < nre; i++) {
 		/*
 		 * must read entire on-file attr struct to
 		 * sync up with layout.
@@ -1275,6 +1255,10 @@ read_event_desc(struct perf_header *ph, int fd)
 		if (ret != (ssize_t)sz)
 			goto error;
 
+		evsel = perf_evsel__new(buf, i);
+		if (!evsel)
+			goto error;
+
 		if (ph->needs_swap)
 			perf_event__attr_swap(buf);
 
@@ -1289,49 +1273,55 @@ read_event_desc(struct perf_header *ph, int fd)
 			evsel->needs_swap = true;
 		}
 
+		perf_evlist__add(evlist, evsel);
+
 		evsel->name = do_read_string(fd, ph);
 
 		if (!nr)
 			continue;
 
-		id = calloc(nr, sizeof(*id));
-		if (!id)
+		if (perf_evsel__alloc_id(evsel, 1, nr))
 			goto error;
+
 		evsel->ids = nr;
-		evsel->id = id;
 
 		for (j = 0 ; j < nr; j++) {
-			ret = readn(fd, id, sizeof(*id));
-			if (ret != (ssize_t)sizeof(*id))
+			u64 id;
+
+			ret = readn(fd, &id, sizeof(id));
+			if (ret != (ssize_t)sizeof(id))
 				goto error;
 			if (ph->needs_swap)
-				*id = bswap_64(*id);
-			id++;
+				id = bswap_64(id);
+
+			perf_evlist__id_add(evlist, evsel, 0, j, id);
 		}
 	}
 out:
 	if (buf)
 		free(buf);
-	return events;
+	return evlist;
 error:
-	if (events)
-		free_event_desc(events);
-	events = NULL;
+	if (evlist)
+		perf_evlist__delete(evlist);
+	evlist = NULL;
 	goto out;
 }
 
 static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)
 {
-	struct perf_evsel *evsel, *events = read_event_desc(ph, fd);
+	struct perf_evlist *evlist;
+	struct perf_evsel *evsel;
 	u32 j;
 	u64 *id;
 
-	if (!events) {
+	evlist = read_event_desc(ph, fd);
+	if (!evlist) {
 		fprintf(fp, "# event desc: not available or unable to read\n");
 		return;
 	}
 
-	for (evsel = events; evsel->attr.size; evsel++) {
+	list_for_each_entry(evsel, &evlist->entries, node) {
 		fprintf(fp, "# event : name = %s, ", evsel->name);
 
 		fprintf(fp, "type = %d, config = 0x%"PRIx64
@@ -1364,7 +1354,7 @@ static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)
 		fputc('\n', fp);
 	}
 
-	free_event_desc(events);
+	perf_evlist__delete(evlist);
 }
 
 static void print_total_mem(struct perf_header *ph, int fd __maybe_unused,
@@ -1785,17 +1775,19 @@ process_event_desc(struct perf_file_section *section __maybe_unused,
 		   void *data __maybe_unused)
 {
 	struct perf_session *session;
-	struct perf_evsel *evsel, *events = read_event_desc(header, fd);
+	struct perf_evlist *evlist;
+	struct perf_evsel *evsel;
 
-	if (!events)
+	evlist = read_event_desc(header, fd);
+	if (!evlist)
 		return 0;
 
 	session = container_of(header, struct perf_session, header);
-	for (evsel = events; evsel->attr.size; evsel++)
-		perf_evlist__set_event_name(session->evlist, evsel);
 
-	free_event_desc(events);
+	list_for_each_entry(evsel, &evlist->entries, node)
+		perf_evlist__set_event_name(session->evlist, evsel);
 
+	perf_evlist__delete(evlist);
 	return 0;
 }
 
Signature · 1 line
1.7.11.7
author
#18 lore raw

PATCH v2 14/25 perf tools: Introduce perf.data version 3 format

In reply to Jiri Olsa

Introducing perf.data version 3 format. No functional change, just introducing doc, magic bytes and the struct.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/Documentation/perf-data-file-v3.txt | 63 ++++++++++++++++++++++++++
 tools/perf/util/header.c                       |  8 +++-
 tools/perf/util/header.h                       |  8 ++++
 3 files changed, 78 insertions(+), 1 deletion(-)
 create mode 100644 tools/perf/Documentation/perf-data-file-v3.txt
tools/perf/Documentation/perf-data-file-v3.txt new file +63 -0
new file mode 100644
index 0000000..9bf9464
@@ -0,0 +1,63 @@
+perf-data-file-v3(5)
+====================
+
+NAME
+----
+perf-data-file-v3 - The perf tool file format version 3
+
+
+DESCRIPTION
+-----------
+Following text describes version 3 of the perf data file format,
+which is version that is currently used by perf tool.
+
+In version 3 we got rid of following sections:
+  EVENT IDS
+  EVENT ATTRIBUTES
+  EVENT TYPES
+
+and keep only data and features sections. All the data
+from above listed sections are now read from specific
+FEATURES sections.
+
+High level view of the format:
+  FILE HEADER
+  EVENT DATA
+  FEATURES
+
+
+FILE HEADER
+-----------
+Starting point of the data file with magic bytes and global
+section information.
+
+The section contains following data:
+  struct perf_file_header header
+
+struct perf_file_header::
+  struct perf_file_header {
+        u64  magic;
+        u64  size;
+        u64  attr_size;
+        struct perf_file_section  data;
+        struct perf_file_section  features;
+        DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
+  }
+
+  magic           ID 'PERFILE3'
+  size            header size sizeof(struct perf_file_header)
+  attr_size       attribute size sizeof(struct perf_file_attr)
+  data            location of 'EVENT DATA'
+  features        location of 'EVENT TYPES'
+  adds_features   'FEATURES' bitmask
+
+
+EVENT DATA
+----------
+This section contains blob of all events' data - auxiliary events
+and samples.
+
+
+FEATURES
+--------
+Same as in version 2.
tools/perf/util/header.c +7 -1
index e43879e..db6b131 100644
@@ -30,6 +30,7 @@ static const char **header_argv;
 
 /*
  * magic2 = "PERFILE2"
+ * magic3 = "PERFILE3"
  * must be a numerical value to let the endianness
  * determine the memory layout. That way we are able
  * to detect endianness when reading the perf.data file
@@ -40,6 +41,8 @@ static const char **header_argv;
 static const char *__perf_magic1 = "PERFFILE";
 static const u64 __perf_magic2    = 0x32454c4946524550ULL;
 static const u64 __perf_magic2_sw = 0x50455246494c4532ULL;
+static const u64 __perf_magic3    = 0x33454c4946524550ULL;
+static const u64 __perf_magic3_sw = 0x50455246494c4533ULL;
 
 #define PERF_MAGIC	__perf_magic2
 
@@ -2464,7 +2467,9 @@ bool is_perf_magic(u64 magic)
 {
 	if (!memcmp(&magic, __perf_magic1, sizeof(magic))
 		|| magic == __perf_magic2
-		|| magic == __perf_magic2_sw)
+		|| magic == __perf_magic2_sw
+		|| magic == __perf_magic3
+		|| magic == __perf_magic3_sw)
 		return true;
 
 	return false;
@@ -2506,6 +2511,7 @@ do {								\
 } while (0)
 
 	CHECK(__perf_magic2, PERF_HEADER_VERSION_2);
+	CHECK(__perf_magic3, PERF_HEADER_VERSION_3);
 
 	return -1;
 }
tools/perf/util/header.h +8 -0
index c6ea4603..bcd3e64 100644
@@ -37,6 +37,7 @@ enum {
 enum perf_header_version {
 	PERF_HEADER_VERSION_1,
 	PERF_HEADER_VERSION_2,
+	PERF_HEADER_VERSION_3,
 };
 
 struct perf_file_section {
@@ -52,6 +53,12 @@ struct perf_file_header_v2 {
 	DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
 };
 
+struct perf_file_header_v3 {
+	struct perf_file_section	data;
+	struct perf_file_section	features;
+	DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
+};
+
 struct perf_file_header {
 	u64	magic;
 	u64	size;
@@ -60,6 +67,7 @@ struct perf_file_header {
 	/* version specific data */
 	union {
 		struct perf_file_header_v2 v2;
+		struct perf_file_header_v3 v3;
 	};
 };
 
Signature · 1 line
1.7.11.7
author
#19 lore raw

PATCH v2 15/25 perf tools: Add perf data version 3 header swap

In reply to Jiri Olsa

Adding perf data version 3 header swap same way as it's done for v2.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/util/header.c | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)
tools/perf/util/header.c +18 -3
index db6b131..77d588f 100644
@@ -2568,13 +2568,28 @@ static int swap_header_v2(struct perf_file_header *header)
 	return 0;
 }
 
-static int swap_header(struct perf_file_header *header)
+static int swap_header_v3(struct perf_file_header *header)
+{
+	struct perf_file_header_v3 *v3 = &header->v3;
+
+	mem_bswap_64(v3, offsetof(struct perf_file_header_v3,
+		     adds_features));
+
+	swap_features(v3->adds_features);
+	return 0;
+}
+
+static int swap_header(struct perf_header *ph,
+		       struct perf_file_header *header)
 {
 	/* swap the generic part */
 	mem_bswap_64(header, offsetof(struct perf_file_header, v2));
 
 	/* version specific swap */
-	return swap_header_v2(header);
+	if (ph->version <= PERF_HEADER_VERSION_2)
+		return swap_header_v2(header);
+
+	return swap_header_v3(header);
 }
 
 int perf_file_header__read(struct perf_file_header *header,
@@ -2594,7 +2609,7 @@ int perf_file_header__read(struct perf_file_header *header,
 		return -1;
 	}
 
-	return ph->needs_swap ? swap_header(header) : 0;
+	return ph->needs_swap ? swap_header(ph, header) : 0;
 }
 
 static int perf_file_section__process(struct perf_file_section *section,
Signature · 1 line
1.7.11.7
author
#20 lore raw

PATCH v2 16/25 perf tools: Add perf data version 3 header read

In reply to Jiri Olsa

Adding perf data version 3 header read code to load data for v3 format.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/util/header.c | 46 +++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 39 insertions(+), 7 deletions(-)
tools/perf/util/header.c +39 -7
index 77d588f..0bf0164 100644
@@ -1780,17 +1780,26 @@ process_event_desc(struct perf_file_section *section __maybe_unused,
 	struct perf_session *session;
 	struct perf_evlist *evlist;
 	struct perf_evsel *evsel;
+	struct perf_header *ph;
+
+	session = container_of(header, struct perf_session, header);
 
 	evlist = read_event_desc(header, fd);
 	if (!evlist)
-		return 0;
+		return -1;
 
-	session = container_of(header, struct perf_session, header);
+	ph = &session->header;
 
-	list_for_each_entry(evsel, &evlist->entries, node)
-		perf_evlist__set_event_name(session->evlist, evsel);
+	if (ph->version <= PERF_HEADER_VERSION_2) {
+		list_for_each_entry(evsel, &evlist->entries, node)
+			perf_evlist__set_event_name(session->evlist, evsel);
+
+		perf_evlist__delete(evlist);
+	} else {
+		session->evlist = session->evlist ?: evlist;
+		symbol_conf.nr_events = evlist->nr_entries;
+	}
 
-	perf_evlist__delete(evlist);
 	return 0;
 }
 
@@ -2859,15 +2868,38 @@ static int perf_session__read_header_v2(struct perf_session *session,
 	return __perf_session__read_header_v2(session, header);
 }
 
+static int perf_session__read_header_v3(struct perf_session *session,
+					struct perf_file_header *header)
+{
+	struct perf_header *ph = &session->header;
+	struct perf_file_header_v3 *v3 = &header->v3;
+
+	memcpy(&ph->adds_features, &v3->adds_features,
+	       sizeof(ph->adds_features));
+
+	ph->data_offset  = v3->data.offset;
+	ph->data_size	 = v3->data.size;
+	ph->feat_offset  = v3->features.offset;
+
+	perf_header__process_sections(ph, session->fd, &session->pevent,
+				      perf_file_section__process);
+
+	return 0;
+}
+
 static int perf_header_read_file(struct perf_session *session)
 {
 	struct perf_file_header header;
+	struct perf_header *ph = &session->header;
 
 	if (perf_file_header__read(&header, &session->header, session->fd))
 		return -1;
 
-	/* read v2 specific data */
-	return perf_session__read_header_v2(session, &header);
+	/* read version specific data */
+	if (ph->version <= PERF_HEADER_VERSION_2)
+		return perf_session__read_header_v2(session, &header);
+
+	return perf_session__read_header_v3(session, &header);
 }
 
 int perf_session__read_header(struct perf_session *session)
Signature · 1 line
1.7.11.7
author
#21 lore raw

PATCH v2 17/25 perf tools: Add perf.data version 3 header write

In reply to Jiri Olsa

Adding perf data version 3 header write code and switch perf tool storing to version 3.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/Documentation/perf-data-file-v2.txt |  5 +-
 tools/perf/builtin-inject.c                    |  3 +-
 tools/perf/builtin-record.c                    |  5 +-
 tools/perf/tests/session-simple.c              |  4 +-
 tools/perf/util/header.c                       | 70 ++++++++------------------
 tools/perf/util/header.h                       |  6 ++-
 6 files changed, 36 insertions(+), 57 deletions(-)
tools/perf/Documentation/perf-data-file-v2.txt +3 -2
index f5c1d5b8..0d29753 100644
@@ -8,8 +8,9 @@ perf-data-file-v2 - The perf tool file format version 2
 
 DESCRIPTION
 -----------
-Following text describes version 2 of the perf data file format,
-which is version that is currently used by perf tool.
+Following text describes version 2 of the perf data file format.
+The perf tool supports this format for reading, but for writing
+it uses version 3 format perf-data-file-v3(1).
 
 The perf data file format is composed of several sections
 describing monitored events and the data itself.
tools/perf/builtin-inject.c +2 -1
index 9b336fd..6dd8ed1 100644
@@ -376,7 +376,8 @@ static int __cmd_inject(struct perf_inject *inject)
 
 	if (!inject->pipe_output) {
 		session->header.data_size = inject->bytes_written;
-		perf_session__write_header(session, session->evlist, inject->output, true);
+		perf_session__write_header(session, session->evlist,
+					   inject->output);
 	}
 
 	perf_session__delete(session);
tools/perf/builtin-record.c +2 -3
index 351ede3..fdbcd51 100644
@@ -273,7 +273,7 @@ static void perf_record__exit(int status, void *arg)
 		if (!rec->no_buildid)
 			process_buildids(rec);
 		perf_session__write_header(rec->session, rec->evlist,
-					   rec->output, true);
+					   rec->output);
 		perf_session__delete(rec->session);
 		perf_evlist__delete(rec->evlist);
 		symbol__exit();
@@ -444,8 +444,7 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv)
 		if (err < 0)
 			goto out_delete_session;
 	} else {
-		err = perf_session__write_header(session, evsel_list,
-						 output, false);
+		err = perf_session__prepare_header(output);
 		if (err < 0)
 			goto out_delete_session;
 	}
tools/perf/tests/session-simple.c +2 -2
index 4f58c0f..215b6dc 100644
@@ -480,7 +480,7 @@ static int session_write(char *file)
 	perf_header__clear_feat(&session->header, HEADER_BRANCH_STACK);
 
 	TEST_ASSERT_VAL("failed to write header",
-		!perf_session__write_header(session, evlist, fd, false));
+		!perf_session__prepare_header(fd));
 
 #define STORE_EVENTS(str, func, cnt)				\
 do {								\
@@ -520,7 +520,7 @@ do {								\
 	session->header.data_size += size;
 
 	TEST_ASSERT_VAL("failed to write header",
-		!perf_session__write_header(session, evlist, fd, true));
+		!perf_session__write_header(session, evlist, fd));
 
 	perf_session__delete(session);
 	perf_evlist__delete(evlist);
tools/perf/util/header.c +22 -48

120 lines not on this page · raw message

tools/perf/util/header.h +5 -1
index bcd3e64..4982e04 100644
@@ -53,6 +53,8 @@ struct perf_file_header_v2 {
 	DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
 };
 
+#define PERF_FILE_HEADER__DATA_OFFSET (sizeof(struct perf_file_header))
+
 struct perf_file_header_v3 {
 	struct perf_file_section	data;
 	struct perf_file_section	features;
@@ -111,6 +113,7 @@ struct perf_header {
 	u64				data_offset;
 	u64				data_size;
 	u64				feat_offset;
+	u64				feat_size;
 	DECLARE_BITMAP(adds_features, HEADER_FEAT_BITS);
 	struct perf_session_env 	env;
 };
@@ -119,9 +122,10 @@ struct perf_evlist;
 struct perf_session;
 
 int perf_session__read_header(struct perf_session *session);
+int perf_session__prepare_header(int fd);
 int perf_session__write_header(struct perf_session *session,
 			       struct perf_evlist *evlist,
-			       int fd, bool at_exit);
+			       int fd);
 int perf_header__write_pipe(int fd);
 
 void perf_header__set_feat(struct perf_header *header, int feat);
Signature · 1 line
1.7.11.7
author
#22 lore raw

PATCH v2 18/25 perf tools: Get rid of post_processing_offset in record command

In reply to Jiri Olsa

No need to use post_processing_offset variable now when we know where the data starts.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/builtin-record.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
tools/perf/builtin-record.c +6 -7
index fdbcd51..4011698 100644
@@ -75,7 +75,6 @@ struct perf_record {
 	bool			no_buildid;
 	bool			no_buildid_cache;
 	long			samples;
-	off_t			post_processing_offset;
 };
 
 static void advance_output(struct perf_record *rec, size_t size)
@@ -249,15 +248,17 @@ out:
 
 static int process_buildids(struct perf_record *rec)
 {
-	u64 size = lseek(rec->output, 0, SEEK_CUR);
+	struct perf_session *session = rec->session;
+	u64 data_offset              = PERF_FILE_HEADER__DATA_OFFSET;
+	u64 size                     = session->header.data_size;
 
 	if (size == 0)
 		return 0;
 
 	rec->session->fd = rec->output;
-	return __perf_session__process_events(rec->session, rec->post_processing_offset,
-					      size - rec->post_processing_offset,
-					      size, &build_id__mark_dso_hit_ops);
+	return __perf_session__process_events(session, data_offset,
+					      size - data_offset, size,
+					      &build_id__mark_dso_hit_ops);
 }
 
 static void perf_record__exit(int status, void *arg)
@@ -457,8 +458,6 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv)
 		goto out_delete_session;
 	}
 
-	rec->post_processing_offset = lseek(output, 0, SEEK_CUR);
-
 	machine = &session->machines.host;
 
 	if (opts->pipe_output) {
Signature · 1 line
1.7.11.7
author
#23 lore raw

PATCH v2 19/25 perf tools: Move synthesizing into single function

In reply to Jiri Olsa

Moving synthesizing into single function, so it could be reused.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/builtin-annotate.c      |   9 +-
 tools/perf/builtin-buildid-cache.c |   8 +-
 tools/perf/builtin-buildid-list.c  |   9 +-
 tools/perf/builtin-diff.c          |  19 ++--
 tools/perf/builtin-evlist.c        |   7 +-
 tools/perf/builtin-inject.c        |   7 +-
 tools/perf/builtin-kmem.c          |   7 +-
 tools/perf/builtin-kvm.c           |  13 ++-
 tools/perf/builtin-lock.c          |   8 +-
 tools/perf/builtin-mem.c           |   9 +-
 tools/perf/builtin-record.c        | 204 +++++++++++++++++++++----------------
 tools/perf/builtin-report.c        |  10 +-
 tools/perf/builtin-sched.c         |   6 +-
 tools/perf/builtin-script.c        |  15 ++-
 tools/perf/builtin-timechart.c     |  10 +-
 tools/perf/builtin-top.c           |   6 +-
 tools/perf/builtin-trace.c         |   8 +-
 tools/perf/perf.h                  |   1 -
 tools/perf/tests/session-simple.c  |  30 ++++--
 tools/perf/util/data.h             |  29 ++++++
 tools/perf/util/session.c          |  12 +--
 tools/perf/util/session.h          |   6 +-
 22 files changed, 290 insertions(+), 143 deletions(-)
 create mode 100644 tools/perf/util/data.h
tools/perf/builtin-annotate.c +7 -2
index f988d38..64eae2c 100644
@@ -28,6 +28,7 @@
 #include "util/hist.h"
 #include "util/session.h"
 #include "util/tool.h"
+#include "util/data.h"
 #include "arch/common.h"
 
 #include <linux/bitmap.h>
@@ -188,9 +189,13 @@ static int __cmd_annotate(struct perf_annotate *ann)
 	struct perf_session *session;
 	struct perf_evsel *pos;
 	u64 total_nr_samples;
+	struct perf_data_file file = {
+		.path  = input_name,
+		.mode  = PERF_DATA_MODE_READ,
+		.force = ann->force,
+	};
 
-	session = perf_session__new(input_name, O_RDONLY,
-				    ann->force, false, &ann->tool);
+	session = perf_session__new(&file, false, &ann->tool);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-buildid-cache.c +6 -2
index c96c8fa..581c62a 100644
@@ -82,8 +82,12 @@ static bool dso__missing_buildid_cache(struct dso *dso, int parm __maybe_unused)
 
 static int build_id_cache__fprintf_missing(const char *filename, bool force, FILE *fp)
 {
-	struct perf_session *session = perf_session__new(filename, O_RDONLY,
-							 force, false, NULL);
+	struct perf_data_file file = {
+		.path  = filename,
+		.mode  = PERF_DATA_MODE_READ,
+		.force = force,
+	};
+	struct perf_session *session = perf_session__new(&file, false, NULL);
 	if (session == NULL)
 		return -1;
 
tools/perf/builtin-buildid-list.c +7 -2
index e74366a..0164c1c 100644
@@ -15,6 +15,7 @@
 #include "util/parse-options.h"
 #include "util/session.h"
 #include "util/symbol.h"
+#include "util/data.h"
 
 static int sysfs__fprintf_build_id(FILE *fp)
 {
@@ -52,6 +53,11 @@ static bool dso__skip_buildid(struct dso *dso, int with_hits)
 static int perf_session__list_build_ids(bool force, bool with_hits)
 {
 	struct perf_session *session;
+	struct perf_data_file file = {
+		.path  = input_name,
+		.mode  = PERF_DATA_MODE_READ,
+		.force = force,
+	};
 
 	symbol__elf_init();
 	/*
@@ -60,8 +66,7 @@ static int perf_session__list_build_ids(bool force, bool with_hits)
 	if (filename__fprintf_build_id(input_name, stdout))
 		goto out;
 
-	session = perf_session__new(input_name, O_RDONLY, force, false,
-				    &build_id__mark_dso_hit_ops);
+	session = perf_session__new(&file, false, &build_id__mark_dso_hit_ops);
 	if (session == NULL)
 		return -1;
 	/*
tools/perf/builtin-diff.c +12 -7
index f28799e..3fb2d46 100644
@@ -16,6 +16,7 @@
 #include "util/sort.h"
 #include "util/symbol.h"
 #include "util/util.h"
+#include "util/data.h"
 
 #include <stdlib.h>
 #include <math.h>
@@ -42,7 +43,7 @@ struct diff_hpp_fmt {
 
 struct data__file {
 	struct perf_session	*session;
-	const char		*file;
+	struct perf_data_file	file;
 	int			 idx;
 	struct hists		*hists;
 	struct diff_hpp_fmt	 fmt[PERF_HPP_DIFF__MAX_INDEX];
@@ -599,7 +600,7 @@ static void data__fprintf(void)
 
 	data__for_each_file(i, d)
 		fprintf(stdout, "#  [%d] %s %s\n",
-			d->idx, d->file,
+			d->idx, d->file.path,
 			!d->idx ? "(Baseline)" : "");
 
 	fprintf(stdout, "#\n");
@@ -661,17 +662,16 @@ static int __cmd_diff(void)
 	int ret = -EINVAL, i;
 
 	data__for_each_file(i, d) {
-		d->session = perf_session__new(d->file, O_RDONLY, force,
-					       false, &tool);
+		d->session = perf_session__new(&d->file, false, &tool);
 		if (!d->session) {
-			pr_err("Failed to open %s\n", d->file);
+			pr_err("Failed to open %s\n", d->file.path);
 			ret = -ENOMEM;
 			goto out_delete;
 		}
 
 		ret = perf_session__process_events(d->session, &tool);
 		if (ret) {
-			pr_err("Failed to process %s\n", d->file);
+			pr_err("Failed to process %s\n", d->file.path);
 			goto out_delete;
 		}
 
@@ -1014,7 +1014,12 @@ static int data_init(int argc, const char **argv)
 		return -ENOMEM;
 
 	data__for_each_file(i, d) {
-		d->file = use_default ? defaults[i] : argv[i];
+		struct perf_data_file *file = &d->file;
+
+		file->path  = use_default ? defaults[i] : argv[i];
+		file->mode  = PERF_DATA_MODE_READ,
+		file->force = force,
+
 		d->idx  = i;
 	}
 
tools/perf/builtin-evlist.c +6 -1
index 05bd9df..20b0f12 100644
@@ -14,13 +14,18 @@
 #include "util/parse-events.h"
 #include "util/parse-options.h"
 #include "util/session.h"
+#include "util/data.h"
 
 static int __cmd_evlist(const char *file_name, struct perf_attr_details *details)
 {
 	struct perf_session *session;
 	struct perf_evsel *pos;
+	struct perf_data_file file = {
+		.path = file_name,
+		.mode = PERF_DATA_MODE_READ,
+	};
 
-	session = perf_session__new(file_name, O_RDONLY, 0, false, NULL);
+	session = perf_session__new(&file, 0, NULL);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-inject.c +6 -1
index 6dd8ed1..1ec5246 100644
@@ -15,6 +15,7 @@
 #include "util/tool.h"
 #include "util/debug.h"
 #include "util/build-id.h"
+#include "util/data.h"
 
 #include "util/parse-options.h"
 
@@ -334,6 +335,10 @@ static int __cmd_inject(struct perf_inject *inject)
 {
 	struct perf_session *session;
 	int ret = -EINVAL;
+	struct perf_data_file file = {
+		.path = inject->input_name,
+		.mode = PERF_DATA_MODE_READ,
+	};
 
 	signal(SIGINT, sig_handler);
 
@@ -343,7 +348,7 @@ static int __cmd_inject(struct perf_inject *inject)
 		inject->tool.tracing_data = perf_event__repipe_tracing_data;
 	}
 
-	session = perf_session__new(inject->input_name, O_RDONLY, false, true, &inject->tool);
+	session = perf_session__new(&file, true, &inject->tool);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-kmem.c +6 -1
index c2dff9c..a4bff1c 100644
@@ -13,6 +13,7 @@
 
 #include "util/parse-options.h"
 #include "util/trace-event.h"
+#include "util/data.h"
 
 #include "util/debug.h"
 
@@ -486,8 +487,12 @@ static int __cmd_kmem(void)
 		{ "kmem:kfree",			perf_evsel__process_free_event, },
     		{ "kmem:kmem_cache_free",	perf_evsel__process_free_event, },
 	};
+	struct perf_data_file file = {
+		.path = input_name,
+		.mode = PERF_DATA_MODE_READ,
+	};
 
-	session = perf_session__new(input_name, O_RDONLY, 0, false, &perf_kmem);
+	session = perf_session__new(&file, false, &perf_kmem);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-kvm.c +10 -3
index f89f7d6..187848c 100644
@@ -17,6 +17,7 @@
 #include "util/tool.h"
 #include "util/stat.h"
 #include "util/top.h"
+#include "util/data.h"
 
 #include <sys/prctl.h>
 #include <sys/timerfd.h>
@@ -1215,10 +1216,13 @@ static int read_events(struct perf_kvm_stat *kvm)
 		.comm			= perf_event__process_comm,
 		.ordered_samples	= true,
 	};
+	struct perf_data_file file = {
+		.path = input_name,
+		.mode = PERF_DATA_MODE_READ,
+	};
 
 	kvm->tool = eops;
-	kvm->session = perf_session__new(kvm->file_name, O_RDONLY, 0, false,
-					 &kvm->tool);
+	kvm->session = perf_session__new(&file, false, &kvm->tool);
 	if (!kvm->session) {
 		pr_err("Initializing perf session failed\n");
 		return -EINVAL;
@@ -1450,6 +1454,9 @@ static int kvm_events_live(struct perf_kvm_stat *kvm,
 		"perf kvm stat live [<options>]",
 		NULL
 	};
+	struct perf_data_file file = {
+		.mode = PERF_DATA_MODE_WRITE,
+	};
 
 
 	/* event handling */
@@ -1514,7 +1521,7 @@ static int kvm_events_live(struct perf_kvm_stat *kvm,
 	/*
 	 * perf session
 	 */
-	kvm->session = perf_session__new(NULL, O_WRONLY, false, false, &kvm->tool);
+	kvm->session = perf_session__new(&file, false, &kvm->tool);
 	if (kvm->session == NULL) {
 		err = -ENOMEM;
 		goto out;
tools/perf/builtin-lock.c +7 -1
index ee33ba2..c3b685c 100644
@@ -15,6 +15,7 @@
 #include "util/debug.h"
 #include "util/session.h"
 #include "util/tool.h"
+#include "util/data.h"
 
 #include <sys/types.h>
 #include <sys/prctl.h>
@@ -836,7 +837,12 @@ static int read_events(void)
 		.comm		 = perf_event__process_comm,
 		.ordered_samples = true,
 	};
-	session = perf_session__new(input_name, O_RDONLY, 0, false, &eops);
+	struct perf_data_file file = {
+		.path = input_name,
+		.mode = PERF_DATA_MODE_READ,
+	};
+
+	session = perf_session__new(&file, false, &eops);
 	if (!session) {
 		pr_err("Initializing perf session failed\n");
 		return -1;
tools/perf/builtin-mem.c +7 -2
index 791b432..1b703da 100644
@@ -5,6 +5,7 @@
 #include "util/trace-event.h"
 #include "util/tool.h"
 #include "util/session.h"
+#include "util/data.h"
 
 #define MEM_OPERATION_LOAD	"load"
 #define MEM_OPERATION_STORE	"store"
@@ -119,10 +120,14 @@ static int process_sample_event(struct perf_tool *tool,
 
 static int report_raw_events(struct perf_mem *mem)
 {
+	struct perf_data_file file = {
+		.path = input_name,
+		.mode = PERF_DATA_MODE_READ,
+	};
 	int err = -EINVAL;
 	int ret;
-	struct perf_session *session = perf_session__new(input_name, O_RDONLY,
-							 0, false, &mem->tool);
+	struct perf_session *session = perf_session__new(&file, false,
+							 &mem->tool);
 
 	if (session == NULL)
 		return -ENOMEM;
tools/perf/builtin-record.c +116 -88

323 lines not on this page · raw message

tools/perf/builtin-report.c +8 -2
index 9725aa3..815866d 100644
@@ -33,6 +33,7 @@
 #include "util/thread.h"
 #include "util/sort.h"
 #include "util/hist.h"
+#include "util/data.h"
 #include "arch/common.h"
 
 #include <linux/bitmap.h>
@@ -843,6 +844,9 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)
 		     "Don't show entries under that percent", parse_percent_limit),
 	OPT_END()
 	};
+	struct perf_data_file file = {
+		.mode  = PERF_DATA_MODE_READ,
+	};
 
 	perf_config(perf_report_config, &report);
 
@@ -872,9 +876,11 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)
 		perf_hpp__init();
 	}
 
+	file.path  = input_name;
+	file.force = report.force;
+
 repeat:
-	session = perf_session__new(input_name, O_RDONLY,
-				    report.force, false, &report.tool);
+	session = perf_session__new(&file, false, &report.tool);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-sched.c +5 -1
index d8c51b2..5a46b10 100644
@@ -1446,8 +1446,12 @@ static int perf_sched__read_events(struct perf_sched *sched,
 		{ "sched:sched_migrate_task", process_sched_migrate_task_event, },
 	};
 	struct perf_session *session;
+	struct perf_data_file file = {
+		.path = input_name,
+		.mode = PERF_DATA_MODE_READ,
+	};
 
-	session = perf_session__new(input_name, O_RDONLY, 0, false, &sched->tool);
+	session = perf_session__new(&file, false, &sched->tool);
 	if (session == NULL) {
 		pr_debug("No Memory for session\n");
 		return -1;
tools/perf/builtin-script.c +12 -3
index 93a34ce..32103c2 100644
@@ -15,6 +15,7 @@
 #include "util/evlist.h"
 #include "util/evsel.h"
 #include "util/sort.h"
+#include "util/data.h"
 #include <linux/bitmap.h>
 
 static char const		*script_name;
@@ -1114,10 +1115,14 @@ int find_scripts(char **scripts_array, char **scripts_path_array)
 	char scripts_path[MAXPATHLEN], lang_path[MAXPATHLEN];
 	DIR *scripts_dir, *lang_dir;
 	struct perf_session *session;
+	struct perf_data_file file = {
+		.path = input_name,
+		.mode = PERF_DATA_MODE_READ,
+	};
 	char *temp;
 	int i = 0;
 
-	session = perf_session__new(input_name, O_RDONLY, 0, false, NULL);
+	session = perf_session__new(&file, false, NULL);
 	if (!session)
 		return -1;
 
@@ -1318,12 +1323,17 @@ int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)
 		"perf script [<options>] <top-script> [script-args]",
 		NULL
 	};
+	struct perf_data_file file = {
+		.mode = PERF_DATA_MODE_READ,
+	};
 
 	setup_scripting();
 
 	argc = parse_options(argc, argv, options, script_usage,
 			     PARSE_OPT_STOP_AT_NON_OPTION);
 
+	file.path = input_name;
+
 	if (argc > 1 && !strncmp(argv[0], "rec", strlen("rec"))) {
 		rec_script_path = get_script_path(argv[1], RECORD_SUFFIX);
 		if (!rec_script_path)
@@ -1487,8 +1497,7 @@ int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)
 	if (!script_name)
 		setup_pager();
 
-	session = perf_session__new(input_name, O_RDONLY, 0, false,
-				    &perf_script);
+	session = perf_session__new(&file, false, &perf_script);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-timechart.c +8 -2
index c2e0231..e11c61d 100644
@@ -36,6 +36,7 @@
 #include "util/session.h"
 #include "util/svghelper.h"
 #include "util/tool.h"
+#include "util/data.h"
 
 #define SUPPORT_OLD_POWER_EVENTS 1
 #define PWR_EVENT_EXIT -1
@@ -990,8 +991,13 @@ static int __cmd_timechart(const char *output_name)
 		{ "power:power_frequency",	process_sample_power_frequency },
 #endif
 	};
-	struct perf_session *session = perf_session__new(input_name, O_RDONLY,
-							 0, false, &perf_timechart);
+	struct perf_data_file file = {
+		.path = input_name,
+		.mode = PERF_DATA_MODE_READ,
+	};
+
+	struct perf_session *session = perf_session__new(&file, false,
+							 &perf_timechart);
 	int ret = -EINVAL;
 
 	if (session == NULL)
tools/perf/builtin-top.c +5 -1
index 0a84334..46489cb 100644
@@ -930,11 +930,15 @@ static int __cmd_top(struct perf_top *top)
 	struct perf_record_opts *opts = &top->record_opts;
 	pthread_t thread;
 	int ret;
+	struct perf_data_file file = {
+		.mode = PERF_DATA_MODE_WRITE,
+	};
+
 	/*
 	 * FIXME: perf_session__new should allow passing a O_MMAP, so that all this
 	 * mmap reading, etc is encapsulated in it. Use O_WRONLY for now.
 	 */
-	top->session = perf_session__new(NULL, O_WRONLY, false, false, NULL);
+	top->session = perf_session__new(&file, false, NULL);
 	if (top->session == NULL)
 		return -ENOMEM;
 
tools/perf/builtin-trace.c +5 -3
index 9967a56..d80b628 100644
@@ -873,7 +873,10 @@ static int trace__replay(struct trace *trace)
 		{ "raw_syscalls:sys_enter",  trace__sys_enter, },
 		{ "raw_syscalls:sys_exit",   trace__sys_exit, },
 	};
-
+	struct perf_data_file file = {
+		.path  = input_name,
+		.mode  = PERF_DATA_MODE_READ,
+	};
 	struct perf_session *session;
 	int err = -1;
 
@@ -895,8 +898,7 @@ static int trace__replay(struct trace *trace)
 	if (symbol__init() < 0)
 		return -1;
 
-	session = perf_session__new(input_name, O_RDONLY, 0, false,
-				    &trace->tool);
+	session = perf_session__new(&file, false, &trace->tool);
 	if (session == NULL)
 		return -ENOMEM;
 
tools/perf/perf.h +0 -1
index cf20187..0914630 100644
@@ -218,7 +218,6 @@ struct perf_record_opts {
 	bool	     no_delay;
 	bool	     no_inherit;
 	bool	     no_samples;
-	bool	     pipe_output;
 	bool	     raw_samples;
 	bool	     sample_address;
 	bool	     sample_weight;
tools/perf/tests/session-simple.c +19 -11
index 215b6dc..e2b1037 100644
@@ -11,6 +11,7 @@
 #include "header.h"
 #include "util.h"
 #include "evlist.h"
+#include "data.h"
 
 #define EVENTS_MMAP 5
 #define EVENTS_LOST 6
@@ -449,15 +450,19 @@ static int store_event(int fd, union perf_event *event, size_t *size)
 	return write(fd, event, event->header.size) > 0 ? 0 : -1;
 }
 
-static int session_write(char *file)
+static int session_write(char *path)
 {
+	struct perf_data_file file = {
+		.mode = PERF_DATA_MODE_WRITE,
+		.path = path,
+	};
 	struct perf_session *session;
 	struct perf_evlist *evlist;
 	size_t size = 0;
-	int feat, fd;
+	int feat;
 
-	fd = open(file, O_RDWR);
-	TEST_ASSERT_VAL("failed to open data file", fd >= 0);
+	file.fd = open(file.path, O_RDWR);
+	TEST_ASSERT_VAL("failed to open data file", file.fd >= 0);
 
 	evlist = perf_evlist__new_default();
 	TEST_ASSERT_VAL("failed to get evlist", evlist);
@@ -466,11 +471,10 @@ static int session_write(char *file)
 
 	pr_debug("session writing start\n");
 
-	session = perf_session__new(file, O_WRONLY, true, false, NULL);
+	session = perf_session__new(&file, false, NULL);
 	TEST_ASSERT_VAL("failed to create session", session);
 
 	session->evlist = evlist;
-	session->fd     = fd;
 
 	for (feat = HEADER_FIRST_FEATURE; feat < HEADER_LAST_FEATURE; feat++)
 		perf_header__set_feat(&session->header, feat);
@@ -480,14 +484,14 @@ static int session_write(char *file)
 	perf_header__clear_feat(&session->header, HEADER_BRANCH_STACK);
 
 	TEST_ASSERT_VAL("failed to write header",
-		!perf_session__prepare_header(fd));
+		!perf_session__prepare_header(file.fd));
 
 #define STORE_EVENTS(str, func, cnt)				\
 do {								\
 	int i;							\
 	for (i = 0; i < cnt; i++) {				\
 		TEST_ASSERT_VAL(str,				\
-			!store_event(fd, func(), &size));	\
+			!store_event(file.fd, func(), &size));	\
 	}							\
 } while (0)
 
@@ -520,7 +524,7 @@ do {								\
 	session->header.data_size += size;
 
 	TEST_ASSERT_VAL("failed to write header",
-		!perf_session__write_header(session, evlist, fd));
+		!perf_session__write_header(session, evlist, file.fd));
 
 	perf_session__delete(session);
 	perf_evlist__delete(evlist);
@@ -529,7 +533,7 @@ do {								\
 	return 0;
 }
 
-static int __session_read(char *file)
+static int __session_read(char *path)
 {
 	struct perf_session *session;
 	struct perf_tool tool = {
@@ -542,10 +546,14 @@ static int __session_read(char *file)
 		.unthrottle = process_unthrottle,
 		.sample = process_sample,
 	};
+	struct perf_data_file file = {
+		.mode = PERF_DATA_MODE_READ,
+		.path = path,
+	};
 
 	pr_debug("session reading start\n");
 
-	session = perf_session__new(file, O_RDONLY, false, false, &tool);
+	session = perf_session__new(&file, false, &tool);
 	TEST_ASSERT_VAL("failed to create session", session);
 
 	TEST_ASSERT_VAL("failed to process events",
tools/perf/util/data.h new file +29 -0
new file mode 100644
index 0000000..ffa0186
@@ -0,0 +1,29 @@
+#ifndef __PERF_DATA_H
+#define __PERF_DATA_H
+
+#include <stdbool.h>
+
+enum perf_data_mode {
+	PERF_DATA_MODE_WRITE,
+	PERF_DATA_MODE_READ,
+};
+
+struct perf_data_file {
+	const char *path;
+	int fd;
+	bool is_pipe;
+	bool force;
+	enum perf_data_mode mode;
+};
+
+static inline bool perf_data_file__is_read(struct perf_data_file *file)
+{
+	return file->mode == PERF_DATA_MODE_READ;
+}
+
+static inline bool perf_data_file__is_write(struct perf_data_file *file)
+{
+	return file->mode == PERF_DATA_MODE_WRITE;
+}
+
+#endif /* __PERF_DATA_H */
tools/perf/util/session.c +6 -6
index 787c234..023a5b4 100644
@@ -106,11 +106,11 @@ static void perf_session__destroy_kernel_maps(struct perf_session *self)
 	machines__destroy_kernel_maps(&self->machines);
 }
 
-struct perf_session *perf_session__new(const char *filename, int mode,
-				       bool force, bool repipe,
-				       struct perf_tool *tool)
+struct perf_session *perf_session__new(struct perf_data_file *file,
+				       bool repipe, struct perf_tool *tool)
 {
 	struct perf_session *self;
+	const char *filename = file->path;
 	struct stat st;
 	size_t len;
 
@@ -134,11 +134,11 @@ struct perf_session *perf_session__new(const char *filename, int mode,
 	INIT_LIST_HEAD(&self->ordered_samples.to_free);
 	machines__init(&self->machines);
 
-	if (mode == O_RDONLY) {
-		if (perf_session__open(self, force) < 0)
+	if (perf_data_file__is_read(file)) {
+		if (perf_session__open(self, file->force) < 0)
 			goto out_delete;
 		perf_session__set_id_hdr_size(self);
-	} else if (mode == O_WRONLY) {
+	} else if (perf_data_file__is_write(file)) {
 		/*
 		 * In O_RDONLY mode this will be performed when reading the
 		 * kernel MMAP event, in perf_event__process_mmap().
tools/perf/util/session.h +3 -3
index 3aa75fb..1e4d813 100644
@@ -7,6 +7,7 @@
 #include "machine.h"
 #include "symbol.h"
 #include "thread.h"
+#include "data.h"
 #include <linux/rbtree.h>
 #include <linux/perf_event.h>
 
@@ -49,9 +50,8 @@ struct perf_session {
 
 struct perf_tool;
 
-struct perf_session *perf_session__new(const char *filename, int mode,
-				       bool force, bool repipe,
-				       struct perf_tool *tool);
+struct perf_session *perf_session__new(struct perf_data_file *file,
+				       bool repipe, struct perf_tool *tool);
 void perf_session__delete(struct perf_session *session);
 
 void perf_event_header__bswap(struct perf_event_header *self);
Signature · 1 line
1.7.11.7
author
#24 lore raw

PATCH v2 20/25 perf tools: Add perf_data_file__open interface to data object

In reply to Jiri Olsa

Adding perf_data_file__open interface to data object to open the perf.data file for both read and write.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/Makefile               |   1 +
 tools/perf/builtin-record.c       |  34 +----------
 tools/perf/builtin-top.c          |   9 +--
 tools/perf/tests/session-simple.c |   3 -
 tools/perf/util/data.c            | 120 ++++++++++++++++++++++++++++++++++++++
 tools/perf/util/data.h            |   4 ++
 tools/perf/util/session.c         |  95 ++++++++++--------------------
 tools/perf/util/session.h         |   2 +-
 8 files changed, 158 insertions(+), 110 deletions(-)
 create mode 100644 tools/perf/util/data.c
tools/perf/Makefile +1 -0
index 2510355..7c80443 100644
@@ -363,6 +363,7 @@ LIB_OBJS += $(OUTPUT)util/intlist.o
 LIB_OBJS += $(OUTPUT)util/vdso.o
 LIB_OBJS += $(OUTPUT)util/stat.o
 LIB_OBJS += $(OUTPUT)util/record.o
+LIB_OBJS += $(OUTPUT)util/data.o
 
 LIB_OBJS += $(OUTPUT)ui/setup.o
 LIB_OBJS += $(OUTPUT)ui/helpline.o
tools/perf/builtin-record.c +1 -33
index 1887b20..ace16f8 100644
@@ -433,15 +433,12 @@ out:
 
 static int __cmd_record(struct perf_record *rec, int argc, const char **argv)
 {
-	struct stat st;
-	int flags;
 	int err, feat;
 	unsigned long waking = 0;
 	const bool forks = argc > 0;
 	struct perf_record_opts *opts = &rec->opts;
 	struct perf_evlist *evsel_list = rec->evlist;
 	struct perf_data_file *file = rec->file;
-	const char *output_name = file->path;
 	struct perf_session *session;
 	bool disabled = false;
 
@@ -455,35 +452,6 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv)
 	signal(SIGUSR1, sig_handler);
 	signal(SIGTERM, sig_handler);
 
-	if (!output_name) {
-		if (!fstat(STDOUT_FILENO, &st) && S_ISFIFO(st.st_mode))
-			file->is_pipe = true;
-		else
-			file->path = output_name = "perf.data";
-	}
-	if (output_name) {
-		if (!strcmp(output_name, "-"))
-			file->is_pipe = true;
-		else if (!stat(output_name, &st) && st.st_size) {
-			char oldname[PATH_MAX];
-			snprintf(oldname, sizeof(oldname), "%s.old",
-				 output_name);
-			unlink(oldname);
-			rename(output_name, oldname);
-		}
-	}
-
-	flags = O_CREAT|O_RDWR|O_TRUNC;
-
-	if (file->is_pipe)
-		file->fd = STDOUT_FILENO;
-	else
-		file->fd = open(output_name, flags, S_IRUSR | S_IWUSR);
-	if (file->fd < 0) {
-		perror("failed to create output file");
-		return -1;
-	}
-
 	session = perf_session__new(file, false, NULL);
 	if (session == NULL) {
 		pr_err("Not enough memory for reading perf file header\n");
@@ -611,7 +579,7 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv)
 	fprintf(stderr,
 		"[ perf record: Captured and wrote %.3f MB %s (~%" PRIu64 " samples) ]\n",
 		(double)rec->bytes_written / 1024.0 / 1024.0,
-		output_name,
+		file->path,
 		rec->bytes_written / 24);
 
 	return 0;
tools/perf/builtin-top.c +1 -8
index 46489cb..1cfc14c 100644
@@ -930,15 +930,8 @@ static int __cmd_top(struct perf_top *top)
 	struct perf_record_opts *opts = &top->record_opts;
 	pthread_t thread;
 	int ret;
-	struct perf_data_file file = {
-		.mode = PERF_DATA_MODE_WRITE,
-	};
 
-	/*
-	 * FIXME: perf_session__new should allow passing a O_MMAP, so that all this
-	 * mmap reading, etc is encapsulated in it. Use O_WRONLY for now.
-	 */
-	top->session = perf_session__new(&file, false, NULL);
+	top->session = perf_session__new(NULL, false, NULL);
 	if (top->session == NULL)
 		return -ENOMEM;
 
tools/perf/tests/session-simple.c +0 -3
index e2b1037..6a459d9 100644
@@ -461,9 +461,6 @@ static int session_write(char *path)
 	size_t size = 0;
 	int feat;
 
-	file.fd = open(file.path, O_RDWR);
-	TEST_ASSERT_VAL("failed to open data file", file.fd >= 0);
-
 	evlist = perf_evlist__new_default();
 	TEST_ASSERT_VAL("failed to get evlist", evlist);
 
tools/perf/util/data.c new file +120 -0

120 lines not on this page · raw message

tools/perf/util/data.h +4 -0
index ffa0186..d6c262e 100644
@@ -13,6 +13,7 @@ struct perf_data_file {
 	int fd;
 	bool is_pipe;
 	bool force;
+	unsigned long size;
 	enum perf_data_mode mode;
 };
 
@@ -26,4 +27,7 @@ static inline bool perf_data_file__is_write(struct perf_data_file *file)
 	return file->mode == PERF_DATA_MODE_WRITE;
 }
 
+int perf_data_file__open(struct perf_data_file *file);
+void perf_data_file__close(struct perf_data_file *file);
+
 #endif /* __PERF_DATA_H */
tools/perf/util/session.c +30 -65

148 lines not on this page · raw message

tools/perf/util/session.h +1 -1
index 1e4d813..c6a21d9 100644
@@ -39,7 +39,7 @@ struct perf_session {
 	bool			fd_pipe;
 	bool			repipe;
 	struct ordered_samples	ordered_samples;
-	char			filename[1];
+	const char		*filename;
 };
 
 #define PRINT_IP_OPT_IP		(1<<0)
Signature · 1 line
1.7.11.7
author
#25 lore raw

PATCH v2 21/25 perf tools: Separating data file properties from session

In reply to Jiri Olsa

Removing 'fd, fd_pipe, filename, size' from struct perf_session and replacing it with struct perf_data_file.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/builtin-annotate.c     |  2 +-
 tools/perf/builtin-buildid-list.c |  2 +-
 tools/perf/builtin-record.c       |  2 --
 tools/perf/builtin-report.c       |  8 +++++---
 tools/perf/builtin-script.c       |  2 +-
 tools/perf/util/data.h            | 15 +++++++++++++++
 tools/perf/util/header.c          | 30 ++++++++++++++++++++----------
 tools/perf/util/header.h          |  1 +
 tools/perf/util/session.c         | 36 +++++++++++++++++++-----------------
 tools/perf/util/session.h         |  5 +----
 10 files changed, 64 insertions(+), 39 deletions(-)
tools/perf/builtin-annotate.c +1 -1
index 64eae2c..1cb8861 100644
@@ -248,7 +248,7 @@ static int __cmd_annotate(struct perf_annotate *ann)
 	}
 
 	if (total_nr_samples == 0) {
-		ui__error("The %s file has no samples!\n", session->filename);
+		ui__error("The %s file has no samples!\n", file.path);
 		goto out_delete;
 	}
 
tools/perf/builtin-buildid-list.c +1 -1
index 0164c1c..ed3873b 100644
@@ -73,7 +73,7 @@ static int perf_session__list_build_ids(bool force, bool with_hits)
 	 * in pipe-mode, the only way to get the buildids is to parse
 	 * the record stream. Buildids are stored as RECORD_HEADER_BUILD_ID
 	 */
-	if (with_hits || session->fd_pipe)
+	if (with_hits || perf_data_file__is_pipe(&file))
 		perf_session__process_events(session, &build_id__mark_dso_hit_ops);
 
 	perf_session__fprintf_dsos_buildid(session, stdout, dso__skip_buildid, with_hits);
tools/perf/builtin-record.c +0 -2
index ace16f8..046ddda 100644
@@ -251,7 +251,6 @@ out:
 
 static int process_buildids(struct perf_record *rec)
 {
-	struct perf_data_file *file  = rec->file;
 	struct perf_session *session = rec->session;
 	u64 data_offset              = PERF_FILE_HEADER__DATA_OFFSET;
 	u64 size                     = session->header.data_size;
@@ -259,7 +258,6 @@ static int process_buildids(struct perf_record *rec)
 	if (size == 0)
 		return 0;
 
-	rec->session->fd = file->fd;
 	return __perf_session__process_events(session, data_offset,
 					      size - data_offset, size,
 					      &build_id__mark_dso_hit_ops);
tools/perf/builtin-report.c +5 -3
index 815866d..af594c5 100644
@@ -367,8 +367,9 @@ static int perf_report__setup_sample_type(struct perf_report *rep)
 {
 	struct perf_session *self = rep->session;
 	u64 sample_type = perf_evlist__combined_sample_type(self->evlist);
+	bool is_pipe = perf_data_file__is_pipe(self->file);
 
-	if (!self->fd_pipe && !(sample_type & PERF_SAMPLE_CALLCHAIN)) {
+	if (!is_pipe && !(sample_type & PERF_SAMPLE_CALLCHAIN)) {
 		if (sort__has_parent) {
 			ui__error("Selected --sort parent, but no "
 				    "callchain data. Did you call "
@@ -391,7 +392,7 @@ static int perf_report__setup_sample_type(struct perf_report *rep)
 	}
 
 	if (sort__mode == SORT_MODE__BRANCH) {
-		if (!self->fd_pipe &&
+		if (!is_pipe &&
 		    !(sample_type & PERF_SAMPLE_BRANCH_STACK)) {
 			ui__error("Selected -b but no branch data. "
 				  "Did you call perf record without -b?\n");
@@ -489,6 +490,7 @@ static int __cmd_report(struct perf_report *rep)
 	struct map *kernel_map;
 	struct kmap *kernel_kmap;
 	const char *help = "For a higher level overview, try: perf report --sort comm,dso";
+	struct perf_data_file *file = session->file;
 
 	signal(SIGINT, sig_handler);
 
@@ -570,7 +572,7 @@ static int __cmd_report(struct perf_report *rep)
 	}
 
 	if (nr_samples == 0) {
-		ui__error("The %s file has no samples!\n", session->filename);
+		ui__error("The %s file has no samples!\n", file->path);
 		return 0;
 	}
 
tools/perf/builtin-script.c +1 -1
index 32103c2..a88063d 100644
@@ -1524,7 +1524,7 @@ int cmd_script(int argc, const char **argv, const char *prefix __maybe_unused)
 			return -1;
 		}
 
-		input = open(session->filename, O_RDONLY);	/* input_name */
+		input = open(file.path, O_RDONLY);	/* input_name */
 		if (input < 0) {
 			perror("failed to open file");
 			return -1;
tools/perf/util/data.h +15 -0
index d6c262e..8c2df80 100644
@@ -27,6 +27,21 @@ static inline bool perf_data_file__is_write(struct perf_data_file *file)
 	return file->mode == PERF_DATA_MODE_WRITE;
 }
 
+static inline int perf_data_file__is_pipe(struct perf_data_file *file)
+{
+	return file->is_pipe;
+}
+
+static inline int perf_data_file__fd(struct perf_data_file *file)
+{
+	return file->fd;
+}
+
+static inline unsigned long perf_data_file__size(struct perf_data_file *file)
+{
+	return file->size;
+}
+
 int perf_data_file__open(struct perf_data_file *file);
 void perf_data_file__close(struct perf_data_file *file);
 
tools/perf/util/header.c +20 -10
index 36206b5..080252a 100644
@@ -2175,7 +2175,7 @@ int perf_header__fprintf_info(struct perf_session *session, FILE *fp, bool full)
 {
 	struct header_print_data hd;
 	struct perf_header *header = &session->header;
-	int fd = session->fd;
+	int fd = perf_data_file__fd(session->file);
 	hd.fp = fp;
 	hd.full = full;
 
@@ -2643,8 +2643,10 @@ static int perf_header__read_pipe(struct perf_session *session)
 {
 	struct perf_header *header = &session->header;
 	struct perf_pipe_file_header f_header;
+	struct perf_data_file *file = session->file;
 
-	if (perf_file_header__read_pipe(&f_header, header, session->fd,
+	if (perf_file_header__read_pipe(&f_header, header,
+					perf_data_file__fd(file),
 					session->repipe) < 0) {
 		pr_debug("incompatible file format\n");
 		return -EINVAL;
@@ -2746,12 +2748,13 @@ static int perf_evlist__prepare_tracepoint_events(struct perf_evlist *evlist,
 static int __perf_session__read_header_v2(struct perf_session *session,
 					  struct perf_file_header *header)
 {
+	struct perf_data_file		*file = session->file;
 	struct perf_header		*ph = &session->header;
 	struct perf_file_header_v2	*v2 = &header->v2;
 	struct perf_file_attr		f_attr;
 	u64				f_id;
 	int nr_attrs, nr_ids, i, j;
-	int fd = session->fd;
+	int fd = perf_data_file__fd(file);
 
 	session->evlist = perf_evlist__new();
 	if (session->evlist == NULL)
@@ -2846,6 +2849,8 @@ static int perf_session__read_header_v3(struct perf_session *session,
 {
 	struct perf_header *ph = &session->header;
 	struct perf_file_header_v3 *v3 = &header->v3;
+	struct perf_data_file *file = session->file;
+	int fd = perf_data_file__fd(file);
 
 	memcpy(&ph->adds_features, &v3->adds_features,
 	       sizeof(ph->adds_features));
@@ -2855,7 +2860,7 @@ static int perf_session__read_header_v3(struct perf_session *session,
 	ph->feat_offset  = v3->features.offset;
 	ph->feat_size    = v3->features.size;
 
-	perf_header__process_sections(ph, session->fd, &session->pevent,
+	perf_header__process_sections(ph, fd, &session->pevent,
 				      perf_file_section__process);
 
 	return 0;
@@ -2865,8 +2870,10 @@ static int perf_header_read_file(struct perf_session *session)
 {
 	struct perf_file_header header;
 	struct perf_header *ph = &session->header;
+	struct perf_data_file *file = session->file;
 
-	if (perf_file_header__read(&header, &session->header, session->fd))
+	if (perf_file_header__read(&header, &session->header,
+				   perf_data_file__fd(file)))
 		return -1;
 
 	/* read version specific data */
@@ -2878,7 +2885,9 @@ static int perf_header_read_file(struct perf_session *session)
 
 int perf_session__read_header(struct perf_session *session)
 {
-	if (session->fd_pipe)
+	struct perf_data_file *file = session->file;
+
+	if (perf_data_file__is_pipe(file))
 		return perf_header__read_pipe(session);
 
 	return perf_header_read_file(session) < 0 ? -EINVAL : 0;
@@ -3028,18 +3037,19 @@ int perf_event__process_tracing_data(struct perf_tool *tool __maybe_unused,
 				     struct perf_session *session)
 {
 	ssize_t size_read, padding, size = event->tracing_data.size;
-	off_t offset = lseek(session->fd, 0, SEEK_CUR);
+	int fd = perf_data_file__fd(session->file);
+	off_t offset = lseek(fd, 0, SEEK_CUR);
 	char buf[BUFSIZ];
 
 	/* setup for reading amidst mmap */
-	lseek(session->fd, offset + sizeof(struct tracing_data_event),
+	lseek(fd, offset + sizeof(struct tracing_data_event),
 	      SEEK_SET);
 
-	size_read = trace_report(session->fd, &session->pevent,
+	size_read = trace_report(fd, &session->pevent,
 				 session->repipe);
 	padding = PERF_ALIGN(size_read, sizeof(u64)) - size_read;
 
-	if (readn(session->fd, buf, padding) < 0) {
+	if (readn(fd, buf, padding) < 0) {
 		pr_err("%s: reading input file", __func__);
 		return -1;
 	}
tools/perf/util/header.h +1 -0
index 4982e04..a5875f2 100644
@@ -6,6 +6,7 @@
 #include <stdbool.h>
 #include "types.h"
 #include "event.h"
+#include "data.h"
 
 #include <linux/bitmap.h>
 
tools/perf/util/session.c +19 -17
index b9ceccb..dfefe78 100644
@@ -18,17 +18,16 @@
 
 static int perf_session__open(struct perf_session *self)
 {
-	if (self->fd_pipe) {
-		if (perf_session__read_header(self) < 0)
-			pr_err("incompatible file format (rerun with -v to learn more)");
-		return 0;
-	}
+	struct perf_data_file *file = self->file;
 
 	if (perf_session__read_header(self) < 0) {
 		pr_err("incompatible file format (rerun with -v to learn more)");
 		return -1;
 	}
 
+	if (perf_data_file__is_pipe(file))
+		return 0;
+
 	if (!perf_evlist__valid_sample_type(self->evlist)) {
 		pr_err("non matching sample_type");
 		return -1;
@@ -87,10 +86,7 @@ struct perf_session *perf_session__new(struct perf_data_file *file,
 		if (perf_data_file__open(file))
 			goto out_delete;
 
-		self->fd       = file->fd;
-		self->fd_pipe  = file->is_pipe;
-		self->filename = file->path;
-		self->size     = file->size;
+		self->file = file;
 
 		if (perf_data_file__is_read(file)) {
 			if (perf_session__open(self) < 0)
@@ -158,7 +154,8 @@ void perf_session__delete(struct perf_session *self)
 	perf_session__delete_threads(self);
 	perf_session_env__delete(&self->header.env);
 	machines__exit(&self->machines);
-	close(self->fd);
+	if (self->file)
+		perf_data_file__close(self->file);
 	free(self);
 	vdso__exit();
 }
@@ -980,6 +977,7 @@ static int perf_session_deliver_event(struct perf_session *session,
 static int perf_session__process_user_event(struct perf_session *session, union perf_event *event,
 					    struct perf_tool *tool, u64 file_offset)
 {
+	int fd = perf_data_file__fd(session->file);
 	int err;
 
 	dump_event(session, event, file_offset, NULL);
@@ -993,7 +991,7 @@ static int perf_session__process_user_event(struct perf_session *session, union
 		return err;
 	case PERF_RECORD_HEADER_TRACING_DATA:
 		/* setup for reading amidst mmap */
-		lseek(session->fd, file_offset, SEEK_SET);
+		lseek(fd, file_offset, SEEK_SET);
 		return tool->tracing_data(tool, event, session);
 	case PERF_RECORD_HEADER_BUILD_ID:
 		return tool->build_id(tool, event, session);
@@ -1120,6 +1118,7 @@ volatile int session_done;
 static int __perf_session__process_pipe_events(struct perf_session *self,
 					       struct perf_tool *tool)
 {
+	int fd = perf_data_file__fd(self->file);
 	union perf_event *event;
 	uint32_t size, cur_size = 0;
 	void *buf = NULL;
@@ -1138,7 +1137,7 @@ static int __perf_session__process_pipe_events(struct perf_session *self,
 		return -errno;
 more:
 	event = buf;
-	err = readn(self->fd, event, sizeof(struct perf_event_header));
+	err = readn(fd, event, sizeof(struct perf_event_header));
 	if (err <= 0) {
 		if (err == 0)
 			goto done;
@@ -1170,7 +1169,7 @@ more:
 	p += sizeof(struct perf_event_header);
 
 	if (size - sizeof(struct perf_event_header)) {
-		err = readn(self->fd, p, size - sizeof(struct perf_event_header));
+		err = readn(fd, p, size - sizeof(struct perf_event_header));
 		if (err <= 0) {
 			if (err == 0) {
 				pr_err("unexpected end of event stream\n");
@@ -1249,6 +1248,7 @@ int __perf_session__process_events(struct perf_session *session,
 				   u64 data_offset, u64 data_size,
 				   u64 file_size, struct perf_tool *tool)
 {
+	int fd = perf_data_file__fd(session->file);
 	u64 head, page_offset, file_offset, file_pos, progress_next;
 	int err, mmap_prot, mmap_flags, map_idx = 0;
 	size_t	mmap_size;
@@ -1281,7 +1281,7 @@ int __perf_session__process_events(struct perf_session *session,
 		mmap_flags = MAP_PRIVATE;
 	}
 remap:
-	buf = mmap(NULL, mmap_size, mmap_prot, mmap_flags, session->fd,
+	buf = mmap(NULL, mmap_size, mmap_prot, mmap_flags, fd,
 		   file_offset);
 	if (buf == MAP_FAILED) {
 		pr_err("failed to mmap file\n");
@@ -1343,16 +1343,17 @@ out_err:
 int perf_session__process_events(struct perf_session *self,
 				 struct perf_tool *tool)
 {
+	u64 size = perf_data_file__size(self->file);
 	int err;
 
 	if (perf_session__register_idle_thread(self) == NULL)
 		return -ENOMEM;
 
-	if (!self->fd_pipe)
+	if (!perf_data_file__is_pipe(self->file))
 		err = __perf_session__process_events(self,
 						     self->header.data_offset,
 						     self->header.data_size,
-						     self->size, tool);
+						     size, tool);
 	else
 		err = __perf_session__process_pipe_events(self, tool);
 
@@ -1575,13 +1576,14 @@ int perf_session__cpu_bitmap(struct perf_session *session,
 void perf_session__fprintf_info(struct perf_session *session, FILE *fp,
 				bool full)
 {
+	int fd = perf_data_file__fd(session->file);
 	struct stat st;
 	int ret;
 
 	if (session == NULL || fp == NULL)
 		return;
 
-	ret = fstat(session->fd, &st);
+	ret = fstat(fd, &st);
 	if (ret == -1)
 		return;
 
tools/perf/util/session.h +1 -4
index c6a21d9..5748198 100644
@@ -30,16 +30,13 @@ struct ordered_samples {
 
 struct perf_session {
 	struct perf_header	header;
-	unsigned long		size;
 	struct machines		machines;
 	struct perf_evlist	*evlist;
 	struct pevent		*pevent;
 	struct events_stats	stats;
-	int			fd;
-	bool			fd_pipe;
 	bool			repipe;
 	struct ordered_samples	ordered_samples;
-	const char		*filename;
+	struct perf_data_file	*file;
 };
 
 #define PRINT_IP_OPT_IP		(1<<0)
Signature · 1 line
1.7.11.7
author
#26 lore raw

PATCH v2 22/25 perf tests: Add session reading test for little endian perf data v3

In reply to Jiri Olsa

Adding session reading test for little endian perf data version 3 included as binary data in the test object.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/Makefile                |   1 +
 tools/perf/tests/perf.data.v3.le.h | 212 +++++++++++++++++++++++++++++++++++++
 tools/perf/tests/session-simple.c  |   6 ++
 3 files changed, 219 insertions(+)
 create mode 100644 tools/perf/tests/perf.data.v3.le.h
tools/perf/Makefile +1 -0
index 7c80443..54f41f5 100644
@@ -294,6 +294,7 @@ LIB_H += ui/util.h
 LIB_H += ui/ui.h
 LIB_H += tests/perf.data.v2.le.h
 LIB_H += tests/perf.data.v2.be.h
+LIB_H += tests/perf.data.v3.le.h
 
 LIB_OBJS += $(OUTPUT)util/abspath.o
 LIB_OBJS += $(OUTPUT)util/alias.o
tools/perf/tests/perf.data.v3.le.h new file +212 -0

212 lines not on this page · raw message

tools/perf/tests/session-simple.c +6 -0
index 6a459d9..e397342 100644
@@ -30,6 +30,10 @@ char data_file_v2_be[] = {
 #include "perf.data.v2.be.h"
 };
 
+char data_file_v3_le[] = {
+#include "perf.data.v3.le.h"
+};
+
 static int events_mmap;
 static int events_lost;
 static int events_comm;
@@ -642,5 +646,7 @@ int test__session_simple(void)
 	err |= test_file_data(data_file_v2_le, sizeof(data_file_v2_le));
 	pr_debug("Testing v2 BE data\n");
 	err |= test_file_data(data_file_v2_be, sizeof(data_file_v2_be));
+	pr_debug("Testing v3 LE data\n");
+	err |= test_file_data(data_file_v3_le, sizeof(data_file_v3_le));
 	return err;
 }
Signature · 1 line
1.7.11.7
author
#27 lore raw

PATCH v2 23/25 perf tests: Add session reading test for big endian perf data v3

In reply to Jiri Olsa

Adding session reading test for big endian perf data version 3 included as binary data in the test object.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/Makefile                |   1 +
 tools/perf/tests/perf.data.v3.be.h | 258 +++++++++++++++++++++++++++++++++++++
 tools/perf/tests/session-simple.c  |   6 +
 3 files changed, 265 insertions(+)
 create mode 100644 tools/perf/tests/perf.data.v3.be.h
tools/perf/Makefile +1 -0
index 54f41f5..f2e31d5 100644
@@ -295,6 +295,7 @@ LIB_H += ui/ui.h
 LIB_H += tests/perf.data.v2.le.h
 LIB_H += tests/perf.data.v2.be.h
 LIB_H += tests/perf.data.v3.le.h
+LIB_H += tests/perf.data.v3.be.h
 
 LIB_OBJS += $(OUTPUT)util/abspath.o
 LIB_OBJS += $(OUTPUT)util/alias.o
tools/perf/tests/perf.data.v3.be.h new file +258 -0

258 lines not on this page · raw message

tools/perf/tests/session-simple.c +6 -0
index e397342..175efd5 100644
@@ -34,6 +34,10 @@ char data_file_v3_le[] = {
 #include "perf.data.v3.le.h"
 };
 
+char data_file_v3_be[] = {
+#include "perf.data.v3.be.h"
+};
+
 static int events_mmap;
 static int events_lost;
 static int events_comm;
@@ -648,5 +652,7 @@ int test__session_simple(void)
 	err |= test_file_data(data_file_v2_be, sizeof(data_file_v2_be));
 	pr_debug("Testing v3 LE data\n");
 	err |= test_file_data(data_file_v3_le, sizeof(data_file_v3_le));
+	pr_debug("Testing v3 BE data\n");
+	err |= test_file_data(data_file_v3_be, sizeof(data_file_v3_be));
 	return err;
 }
Signature · 1 line
1.7.11.7
author
#28 lore raw

PATCH v2 24/25 perf tools: Add multi file '-M' option for record command

In reply to Jiri Olsa

split event data into multiple files based on the file size or time delta specified as an argument to the option.

Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold.

The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like:
-M 5M for 5 megabytes threshold
-M 1h for 1 hour threshold
The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be:
perf.data-00000
perf.data-00001
perf.data-00002
...

Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise.

Example:

  $ perf record -M 5M yes > /dev/null
  ^C[ perf record: Woken up 228 times to write data ]
  [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ]
  yes: Interrupt
  $ ls -l perf.data-0*
  -rw------- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-00000
  -rw------- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-00001
  -rw------- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-00002
  -rw------- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-00003
  -rw------- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-00004
  $ ./perf diff perf.data-0000*
  # Event 'cycles'
  #
  # Data files:
  #  [0] perf.data-00000 (Baseline)
  #  [1] perf.data-00001
  #  [2] perf.data-00002
  #  [3] perf.data-00003
  #  [4] perf.data-00004
  #
  # Baseline/0  Delta/1  Delta/2  Delta/3  Delta/4      Shared Object                                      Symbol
  # ..........  .......  .......  .......  .......  .................  ..........................................
  #
        37.70%   -0.17%   -0.42%   -0.24%   -0.31%  libc-2.15.so       [.] _IO_file_xsputn@@GLIBC_2.2.5
        30.31%   +0.28%   +0.22%   +0.07%   +0.06%  yes                [.] main
        16.73%   +0.02%   +0.10%   -0.03%   +0.11%  libc-2.15.so       [.] __strlen_sse2
        14.22%   -0.30%   -0.10%   -0.31%   -0.14%  libc-2.15.so       [.] fputs_unlocked
         0.39%                              -0.01%  yes                [.] fputs_unlocked@plt
         0.06%                                      [kernel.kallsyms]  [k] system_call
         0.06%                     +0.01%           [kernel.kallsyms]  [k] __srcu_read_lock
         0.05%                     +0.01%           [kernel.kallsyms]  [k] __srcu_read_unlock
  ...
Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/Documentation/perf-record.txt |  14 ++
 tools/perf/builtin-record.c              | 246 ++++++++++++++++++++++++++++---
 tools/perf/perf.h                        |  14 ++
 tools/perf/util/evlist.c                 |   2 +-
 tools/perf/util/evlist.h                 |   2 +
 tools/perf/util/evsel.c                  |  24 +++
 6 files changed, 280 insertions(+), 22 deletions(-)
tools/perf/Documentation/perf-record.txt +14 -0
index 400e9bb..f77658b 100644
@@ -184,6 +184,20 @@ Enable weightened sampling. An additional weight is recorded per sample and can
 displayed with the weight and local_weight sort keys.  This currently works for TSX
 abort events and some memory events in precise mode on modern Intel CPUs.
 
+-M::
+--multi::
+Store output perf.data into multiple files based on the size threshold.
+The threshold is specified either as size (B/K/M/G) or time (s/m/h)
+by appending the size with appropriate unit, like:
+  -M 5M for 5 megabytes threshold
+  -M 1h for 1 hour threshold
+The generated name for each data file is appended with sequential number
+(prepended by 5 zeros).  For default output file name it will be:
+  perf.data-00000
+  perf.data-00001
+  perf.data-00002
+  ...
+
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/builtin-record.c +225 -21

313 lines not on this page · raw message

tools/perf/perf.h +14 -0
index 0914630..76b7ae0 100644
@@ -210,6 +210,17 @@ enum perf_call_graph_mode {
 	CALLCHAIN_DWARF
 };
 
+enum perf_record_multi_limits {
+	MULTI_LIMIT__MIN_SIZE       = 100 * 1024, /* 100K */
+	MULTI_LIMIT__MIN_WATTERMARK = 10  * 1024, /* 10K */
+	MULTI_LIMIT__MAX_WATTERMARK = MULTI_LIMIT__MIN_SIZE,
+};
+
+enum perf_record_multi_type {
+	MULTI_TYPE__SIZE,
+	MULTI_TYPE__TIME,
+};
+
 struct perf_record_opts {
 	struct perf_target target;
 	int	     call_graph;
@@ -223,6 +234,7 @@ struct perf_record_opts {
 	bool	     sample_weight;
 	bool	     sample_time;
 	bool	     period;
+	bool	     multi_limit;
 	unsigned int freq;
 	unsigned int mmap_pages;
 	unsigned int user_freq;
@@ -230,6 +242,8 @@ struct perf_record_opts {
 	u64	     default_interval;
 	u64	     user_interval;
 	u16	     stack_dump_size;
+	u64	     multi_value;
+	int	     multi_type;
 };
 
 #endif
tools/perf/util/evlist.c +1 -1
index 998e0d1..a862937 100644
@@ -632,7 +632,7 @@ out_unmap:
 	return -1;
 }
 
-static size_t perf_evlist__mmap_size(unsigned long pages)
+size_t perf_evlist__mmap_size(unsigned long pages)
 {
 	/* 512 kiB: default amount of unprivileged mlocked memory */
 	if (pages == UINT_MAX)
tools/perf/util/evlist.h +2 -0
index ca016b1..59bcf52 100644
@@ -104,6 +104,8 @@ int perf_evlist__prepare_workload(struct perf_evlist *evlist,
 				  bool want_signal);
 int perf_evlist__start_workload(struct perf_evlist *evlist);
 
+size_t perf_evlist__mmap_size(unsigned long pages);
+
 int perf_evlist__parse_mmap_pages(const struct option *opt,
 				  const char *str,
 				  int unset);
tools/perf/util/evsel.c +24 -0
index e8745fb..122511d 100644
@@ -672,6 +672,30 @@ void perf_evsel__config(struct perf_evsel *evsel,
 		attr->branch_sample_type = opts->branch_stack;
 	}
 
+	if (opts->multi_limit) {
+		u64 wm = MULTI_LIMIT__MIN_WATTERMARK;
+		attr->watermark = 1;
+
+		if (opts->multi_type == MULTI_TYPE__SIZE) {
+			/*
+			 * The watermark could not get under 10K because
+			 * of the minimal file limit and we are guarded
+			 * with 100K for max wattermark.
+			 */
+			wm = opts->multi_value;
+			wm = min(wm / 10, (u64) MULTI_LIMIT__MAX_WATTERMARK);
+
+			/*
+			 * We also dont want to have watermark close to the size
+			 * of the mmap to ensure data would always cross it and
+			 * we get poll notification.
+			 */
+			wm = min(wm, (u64) perf_evlist__mmap_size(opts->mmap_pages) - 100);
+		}
+
+		attr->wakeup_watermark = wm;
+	}
+
 	if (opts->sample_weight)
 		attr->sample_type	|= PERF_SAMPLE_WEIGHT;
 
Signature · 1 line
1.7.11.7
reviewer
#29 lore raw

Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command

Quoted 24 lines · Jiri Olsa

split event data into multiple files based on the file size or time delta specified as an argument to the option.

Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold.

The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like:
-M 5M for 5 megabytes threshold
-M 1h for 1 hour threshold
The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be:
perf.data-00000
perf.data-00001
perf.data-00002
...

Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise.

Example:

Quoted 10 lines
  $ perf record -M 5M yes > /dev/null
  ^C[ perf record: Woken up 228 times to write data ]
  [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ]
  yes: Interrupt
  $ ls -l perf.data-0*
  -rw------- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-00000
  -rw------- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-00001
  -rw------- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-00002
  -rw------- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-00003
  -rw------- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-00004

Did you consider doing this with "perf inject" instead of "perf record"? That way you could get the split exactly right, and you could split the same file different ways.

author
#30 lore raw

Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command

Quoted 36 lines · Adrian Hunter
Quoted 24 lines · Jiri Olsa

split event data into multiple files based on the file size or time delta specified as an argument to the option.

Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold.

The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like:
-M 5M for 5 megabytes threshold
-M 1h for 1 hour threshold
The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be:
perf.data-00000
perf.data-00001
perf.data-00002
...

Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise.

Example:

Quoted 10 lines
  $ perf record -M 5M yes > /dev/null
  ^C[ perf record: Woken up 228 times to write data ]
  [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ]
  yes: Interrupt
  $ ls -l perf.data-0*
  -rw------- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-00000
  -rw------- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-00001
  -rw------- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-00002
  -rw------- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-00003
  -rw------- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-00004
Quoted 3 lines

Did you consider doing this with "perf inject" instead of "perf record"? That way you could get the split exactly right, and you could split the same file different ways.

right, this could be added into inject command as well

jirka

reviewer
#31 lore raw

Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command

Quoted 37 lines · Jiri Olsa
Quoted 36 lines · Adrian Hunter
Quoted 34 lines · Jiri Olsa

split event data into multiple files based on the file size or time delta specified as an argument to the option.

Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold.

The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like:
-M 5M for 5 megabytes threshold
-M 1h for 1 hour threshold
The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be:
perf.data-00000
perf.data-00001
perf.data-00002
...

Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise.

Example:

  $ perf record -M 5M yes > /dev/null
  ^C[ perf record: Woken up 228 times to write data ]
  [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ]
  yes: Interrupt
  $ ls -l perf.data-0*
  -rw------- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-00000
  -rw------- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-00001
  -rw------- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-00002
  -rw------- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-00003
  -rw------- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-00004
Quoted 5 lines
Quoted 3 lines

Did you consider doing this with "perf inject" instead of "perf record"? That way you could get the split exactly right, and you could split the same file different ways.

right, this could be added into inject command as well

I was really thinking "instead" not "as well" e.g.

perf record yes | perf inject -M 5M

author
#32 lore raw

Re: [PATCH 24/25] perf tools: Add multi file '-M' option for record command

Quoted 42 lines · Adrian Hunter
Quoted 41 lines · Jiri Olsa
Quoted 39 lines · Adrian Hunter
Quoted 24 lines · Jiri Olsa

split event data into multiple files based on the file size or time delta specified as an argument to the option.

Adding multi file '-M' option for record command to store output perf.data into multiple files based on the size threshold.

The threshold is specified either as size (B/K/M/G) or time (s/m/h) by appending the size with appropriate unit, like:
-M 5M for 5 megabytes threshold
-M 1h for 1 hour threshold
The generated name for each data file is appended with sequential number (prepended by 5 zeros).For default output file name it will be:
perf.data-00000
perf.data-00001
perf.data-00002
...

Also watermark/wakeup_watermark is set accordingly to get wake ups more often so we could get close enough to the file size promise.

Example:

Quoted 10 lines
  $ perf record -M 5M yes > /dev/null
  ^C[ perf record: Woken up 228 times to write data ]
  [ perf record: Captured and wrote 20.246 MB perf.data-[0-4](~884542 samples) ]
  yes: Interrupt
  $ ls -l perf.data-0*
  -rw------- 1 jolsa jolsa 5289856 Aug 16 16:07 perf.data-00000
  -rw------- 1 jolsa jolsa 5296008 Aug 16 16:08 perf.data-00001
  -rw------- 1 jolsa jolsa 5344968 Aug 16 16:09 perf.data-00002
  -rw------- 1 jolsa jolsa 5309144 Aug 16 16:09 perf.data-00003
  -rw------- 1 jolsa jolsa 2358268 Aug 16 16:09 perf.data-00004

Did you consider doing this with "perf inject" instead of "perf record"? That way you could get the split exactly right, and you could split the same file different ways.

Quoted 6 lines
Quoted 1 line

right, this could be added into inject command as well

I was really thinking "instead" not "as well" e.g.

perf record yes | perf inject -M 5M

yep, got it.. this gives you more fine grained size resolution and lets you split already stored data files, nice

But while I think this is nice feature to have, two things keeps me thinking we should leave the support in record command as well:
- the extra inject command overhead, that needs to parse the whole stream, which might not be wanted in some cases
- AFAIU the vast majority of perf users do just record/report and would be restrained from this feature ;-)

jirka

author
#33 lore raw

PATCH v2 25/25 perf tools: Have the process properly sythesized in subsequent data files

In reply to Jiri Olsa

Change the target so the sythesizing routine would think it needs to synthesized whole process, because there's no exec COMM event for subsequent data files.

Cc Corey Ashford <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Andi Kleen <[email protected]>
Cc David Ahern <[email protected]>
---
 tools/perf/builtin-record.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
tools/perf/builtin-record.c +11 -0
index b0c5937..0dbf9a3 100644
@@ -516,6 +516,8 @@ static int multi_file_finish(struct perf_record *rec)
 static int multi_file_init(struct perf_record *rec)
 {
 	struct perf_data_file *file = rec->file;
+	struct perf_record_opts *opts = &rec->opts;
+	static bool target_change;
 	int err;
 
 	if (multi_file_name(rec->file, rec->multi_idx++))
@@ -529,6 +531,15 @@ static int multi_file_init(struct perf_record *rec)
 	if (err)
 		goto out_close;
 
+	/*
+	 * XXX HACK - Make the target looks like we have pid
+	 * so we get the process fully synthetised.
+	 */
+	if (perf_target__none(&opts->target) && !target_change) {
+		target_change = true;
+		opts->target.pid = "multi";
+	}
+
 	err = synthesize_record_file(rec);
 	if (err)
 		goto out_close;
Signature · 1 line
1.7.11.7
maintainer
#35 lore raw
In reply to Jiri Olsa
Quoted 11 lines · Jiri Olsa

hi,
sending the support for multiple file storage. Initial RFC is here:
http://marc.info/?l=linux-kernel&m=137408381902423&w=2

v2 changes:
- reworked perf mmap size setup to be able to get the mmap size value easily later
- added perf.data read/write test for v2 and v3 for both endianity
- added record '-M time' support

So this 0/n post seems to have forgotten to list the rationale for doing all this..

The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.

However your earlier patches didn't do it because of that -- I forgot why they did it and you're now not telling.

author
#36 lore raw
Quoted 13 lines · Peter Zijlstra
Quoted 11 lines · Jiri Olsa

hi,
sending the support for multiple file storage. Initial RFC is here:
http://marc.info/?l=linux-kernel&m=137408381902423&w=2

v2 changes:
- reworked perf mmap size setup to be able to get the mmap size value easily later
- added perf.data read/write test for v2 and v3 for both endianity
- added record '-M time' support

Quoted 2 lines

So this 0/n post seems to have forgotten to list the rationale for doing all this..

here it is:

This patchset is actually doing 3 essential things:
1) adds new perf data format version (v3)
2) adds class/object to handle perf data file
3) adds record '-M size' option

ad 1) Along the way I realized it'd be much easier to have the
format simplified. I ended up with just 'DATA' and 'FEATURES' sections making the file format more generic.

The easier/generic format will be for example useful for the cpu hotplug notification (and maybe even for ftrace data files features?)

The version 2 should be still readable by perf, while it produces data in version 3 from this patchset onwards.

ad 2) I needed to unify perf data file handling to make the -M
implementation clear.

Quoted 5 lines

The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.

I haven't considered the thread based storage yet, but I think having the simplified format and centralized file handling will only help.

jirka

maintainer
#37 lore raw
Quoted 6 lines · Jiri Olsa
Quoted 1 line · Peter Zijlstra

On Sun, Sep 01, 2013 at 12:36:11PM +0200, Jiri Olsa wrote:

This patchset is actually doing 3 essential things:
1) adds new perf data format version (v3)
2) adds class/object to handle perf data file
3) adds record '-M size' option

So the -M thing is where the multiple files stuff comes from, right? What's the purpose of this? Why would I care how big these files get, surely filesystems handle 4g+ files these days.

Quoted 8 lines
Quoted 4 lines

The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.

I haven't considered the thread based storage yet, but I think having the simplified format and centralized file handling will only help.

Please do consider it now, it'd suck to have to change the file format thing yet again.

With both per-thread and size limited your filename generation needs some extra bits I suppose.

author
#38 lore raw
Quoted 11 lines · Peter Zijlstra
Quoted 6 lines · Jiri Olsa
Quoted 1 line · Peter Zijlstra

On Sun, Sep 01, 2013 at 12:36:11PM +0200, Jiri Olsa wrote:

This patchset is actually doing 3 essential things:
1) adds new perf data format version (v3)
2) adds class/object to handle perf data file
3) adds record '-M size' option

So the -M thing is where the multiple files stuff comes from, right? What's the purpose of this? Why would I care how big these files get, surely filesystems handle 4g+ files these days.

my usage currently is to having this running during the day:
(well whenever I remember to run it.. ;-) )

[jolsa@krava perf]$ sudo ./perf record -g -M 1m -a

and checking report when the system or app get stuck

with multiple files I can just easily (or automaticaly) remove old ones without restarting session

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.

Quoted 6 lines
Quoted 3 lines

I haven't considered the thread based storage yet, but I think having the simplified format and centralized file handling will only help.

Please do consider it now, it'd suck to have to change the file format thing yet again.

ook

Quoted 3 lines

With both per-thread and size limited your filename generation needs some extra bits I suppose.

yep, and probably some merging at the end

jirka

#39 lore raw
Quoted 9 lines · Jiri Olsa

my usage currently is to having this running during the day:
(well whenever I remember to run it.. ;-) )

[jolsa@krava perf]$ sudo ./perf record -g -M 1m -a

and checking report when the system or app get stuck

with multiple files I can just easily (or automaticaly) remove old ones without restarting session

Aren't you losing potentially important events by doing that -- FORK, COMM, MMAP?

I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.

David

author
#40 lore raw
Quoted 6 lines · David Ahern
Quoted 5 lines · Jiri Olsa

my usage currently is to having this running during the day:
(well whenever I remember to run it.. ;-) )

[jolsa@krava perf]$ sudo ./perf record -g -M 1m -a

Quoted 7 lines
Quoted 4 lines

and checking report when the system or app get stuck

with multiple files I can just easily (or automaticaly) remove old ones without restarting session

Aren't you losing potentially important events by doing that -- FORK, COMM, MMAP?

those are synthesized for each file via synthesize_record function, see:

[PATCH 19/25] perf tools: Move synthesizing into single function

Quoted 5 lines

I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.

so far I noticed there could be race among EXIT and remaining SAMPLE events on another CPU mmap than EXIT event.. ending up with EXIT being stored in the old file, while SAMPLEs will get to the new one

I was thinking about some 'perf daemon' so I dont need to run that manually.. seems similar to what you did

jirka

#41 lore raw
Quoted 12 lines · Jiri Olsa
Quoted 11 lines · David Ahern
Quoted 9 lines · Jiri Olsa

my usage currently is to having this running during the day:
(well whenever I remember to run it.. ;-) )

[jolsa@krava perf]$ sudo ./perf record -g -M 1m -a

and checking report when the system or app get stuck

with multiple files I can just easily (or automaticaly) remove old ones without restarting session

Quoted 7 lines
Quoted 2 lines

Aren't you losing potentially important events by doing that -- FORK, COMM, MMAP?

those are synthesized for each file via synthesize_record function, see:

[PATCH 19/25] perf tools: Move synthesizing into single function

Ok. haven't had time to look through your 2 large patch sets.

Seems like a lot of repetitive work on a loaded system. The overhead of the task events will dominate compared to the samples.

Quoted 7 lines
Quoted 6 lines

I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.

Quoted 7 lines

so far I noticed there could be race among EXIT and remaining SAMPLE events on another CPU mmap than EXIT event.. ending up with EXIT being stored in the old file, while SAMPLEs will get to the new one

I was thinking about some 'perf daemon' so I dont need to run that manually.. seems similar to what you did

Right now I focus on scheduling events. This latest version of it can be easily recycled for other use cases. Some work would be needed to dump events to a file versus dumping processed information.

I am in San Jose this week. Not sure if I will have time to finish it to a point of pushing out patches, but maybe I can push to github in the next couple of days.

David

#42 lore raw
Quoted 13 lines · David Ahern
Quoted 5 lines
Quoted 4 lines

I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.

Quoted 7 lines

so far I noticed there could be race among EXIT and remaining SAMPLE events on another CPU mmap than EXIT event.. ending up with EXIT being stored in the old file, while SAMPLEs will get to the new one

I was thinking about some 'perf daemon' so I dont need to run that manually.. seems similar to what you did

Quoted 7 lines

Right now I focus on scheduling events. This latest version of it can be easily recycled for other use cases. Some work would be needed to dump events to a file versus dumping processed information.

I am in San Jose this week. Not sure if I will have time to finish it to a point of pushing out patches, but maybe I can push to github in the next couple of days.

https://github.com/dsahern/linux/tree/perf-sched-timehist-3.11

Take a look at tools/perf/schedmon.c. Task events are processed and sample events are stashed on a list (daemon__process_sample). Could easily do something similar for mmap, comm, fork events and then on exit event flush events for terminated tasks. Then you will have both the task and sample events which can be dumped to a file for later processing.

David

#43 lore raw
In reply to David Ahern
Quoted 6 lines · David Ahern
Quoted 5 lines · Jiri Olsa

my usage currently is to having this running during the day:
(well whenever I remember to run it.. ;-) )

[jolsa@krava perf]$ sudo ./perf record -g -M 1m -a

Quoted 7 lines
Quoted 4 lines

and checking report when the system or app get stuck

with multiple files I can just easily (or automaticaly) remove old ones without restarting session

Aren't you losing potentially important events by doing that -- FORK, COMM, MMAP?

I suspect these could/should be tracked and emitted fully (in bulk) when a new data file is opened, so that each partial data file is fully consistent?

Quoted 4 lines

I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.

Could this be used to emit currently relevant task context?

Btw., I also think it would be useful to have kernel support for that - the 'collections' stuff I talked about a good while ago: the kernel would work with user-space to iterate over all MMAPs and all running COMMs at the opening of a tracing session.

That way we could avoid racy access to /proc, we could make sure that all information that is emitted by FORK/COMM/MMAP is also emitted for the 'bulk' data, etc.

Thanks,

Ingo

#44 lore raw
Quoted 6 lines · Ingo Molnar
Quoted 2 lines

Aren't you losing potentially important events by doing that -- FORK, COMM, MMAP?

I suspect these could/should be tracked and emitted fully (in bulk) when a new data file is opened, so that each partial data file is fully consistent?

In my case I am not saving task events, but processing them. In Jiri's case where events are written to a file it should be possible to stash the unprocessed events on a list, when the exit happens move them to a dead threads list which can be cleaned up from time to time and then on file dump requests dump the task events followed by the sample events.

Quoted 7 lines
Quoted 5 lines

I have a flight recorder style command that address this problem (long-running/daemons) by processing task events and then stashing the sample events on a time-ordered list with chopping to maintain the time window.

Could this be used to emit currently relevant task context?

sure.

Quoted 9 lines

Btw., I also think it would be useful to have kernel support for that - the 'collections' stuff I talked about a good while ago: the kernel would work with user-space to iterate over all MMAPs and all running COMMs at the opening of a tracing session.

That way we could avoid racy access to /proc, we could make sure that all information that is emitted by FORK/COMM/MMAP is also emitted for the 'bulk' data, etc.

Walking the task list and emitting events would be better but wouldn't that be a performance hit holding the task lock (tasklist_lock?)? (I thought that one is needed when walking the task list.)

David

reviewer
#45 lore raw
In reply to Peter Zijlstra
Quoted 13 lines · Peter Zijlstra
Quoted 11 lines · Jiri Olsa

hi,
sending the support for multiple file storage. Initial RFC is here:
http://marc.info/?l=linux-kernel&m=137408381902423&w=2

v2 changes:
- reworked perf mmap size setup to be able to get the mmap size value easily later
- added perf.data read/write test for v2 and v3 for both endianity
- added record '-M time' support

Quoted 6 lines

So this 0/n post seems to have forgotten to list the rationale for doing all this..

The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.

Do you need multiple files for that? Why not just feed writes to a pool of threads?

maintainer
#46 lore raw
Quoted 8 lines · Adrian Hunter
Quoted 3 lines · Peter Zijlstra

The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.

Do you need multiple files for that? Why not just feed writes to a pool of threads?

Not sure I understand what you mean there. Since we have a (or multiple) event per cpu it doesn't make sense to read that data from another cpu and have that write it to disk. That completely destroys the locality.

Instead the suggestion was to have a thread per cpu (bound) to empty out the per cpu mmap buffer(s) and write them to disk.

Now the only way to coherently do that into a single file is to do some streams implementation, which I suppose is possible. Pre-allocate large-ish sections per thread and mark them as such, then let the thread spool data into it, when full allocate a new section.

However this would result in allocation-block-size*nr_cpus minimal file sizes, which wouldn't be a problem per-se since they can be sparse.

It will however pretty much mess up Jolsa's max-MBs per file thing and it is of course slightly more complex than having a file per cpu/thread. And is more likely to create kernel lock contention; the pagecache locks are typically per inode, bouncing those around the machine isn't nice either.

maintainer
#47 lore raw
In reply to Peter Zijlstra

Hi Peter and Jiri,

Quoted 13 lines · Peter Zijlstra
Quoted 11 lines · Jiri Olsa

hi,
sending the support for multiple file storage. Initial RFC is here:
http://marc.info/?l=linux-kernel&m=137408381902423&w=2

v2 changes:
- reworked perf mmap size setup to be able to get the mmap size value easily later
- added perf.data read/write test for v2 and v3 for both endianity
- added record '-M time' support

Quoted 6 lines

So this 0/n post seems to have forgotten to list the rationale for doing all this..

The only reason I wanted this is so that each thread can write its own data. The current one file thing is an immense bottle-neck for big machines.

Per-thread or per-cpu? In my perf ftrace patchset, I used to per-cpu data file for this reason. Do you think per-thread approach is better than per-cpu one?

Jiri, one of my colleagues asked me about the multiple file support separated by time while ago. At that time I just added --time-filter option to perf report, but it'd better if perf record can support it. (Unfortunately the patch seems to buried in the list).

Anyway, as Peter said, please consider per-thread or per-cpu multiple file support with your series. It will help further developments.

Thanks,
Namhyung