Skip to contentSkip to conversation

perf record: Put a copy of kcore into the perf.data directory

PATCH Applied Adrian Hunter 4 participants 24 messages Oct 4, 2019 – Nov 12, 2019 perf events

lore
Share & tools
lore

Adrian Hunter

author reviewer
#0 lore raw

cover perf record: Put a copy of kcore into the perf.data directory

Hi

Here are patches to add a new 'perf record' option '--kcore' which will put a copy of /proc/kcore, kallsyms and modules into a perf.data directory. Note, that without the --kcore option, output goes to a file as previously. The tools' -o and -i options work with either a file name or directory name.

Example:

 $ sudo perf record --kcore uname
 $ sudo tree perf.data
 perf.data
 ├── kcore_dir
 │   ├── kallsyms
 │   ├── kcore
 │   └── modules
 └── data
 $ sudo perf script -v
 build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
 build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
 Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
 Using CPUID GenuineIntel-6-8E-A
 Using perf.data/kcore_dir/kcore for kernel data
 Using perf.data/kcore_dir/kallsyms for symbols
             perf 19058 506778.423729:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423733:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423734:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423736:        117 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
             perf 19058 506778.423738:       2092 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
             perf 19058 506778.423740:      37380 cycles:  ffffffffa2f121d0 perf_event_addr_filters_exec+0x0 (vmlinux)
            uname 19058 506778.423751:     582673 cycles:  ffffffffa303a407 propagate_protected_usage+0x147 (vmlinux)
            uname 19058 506778.423892:    2241841 cycles:  ffffffffa2cae0c9 unwind_next_frame.part.5+0x79 (vmlinux)
            uname 19058 506778.424430:    2457397 cycles:  ffffffffa3019232 check_memory_region+0x52 (vmlinux)

Adrian Hunter (5):
perf data: Correctly identify directory data files perf data: Move perf_dir_version into data.h
perf data: Rename directory "header" file to "data" perf tools: Support single perf.data file directory perf record: Put a copy of kcore into the perf.data directory

 tools/perf/Documentation/perf-record.txt           |  3 ++
 .../Documentation/perf.data-directory-format.txt   | 63 ++++++++++++++++++++++
 tools/perf/builtin-record.c                        | 54 ++++++++++++++++++-
 tools/perf/util/data.c                             | 46 ++++++++++++++--
 tools/perf/util/data.h                             | 12 +++++
 tools/perf/util/header.h                           |  4 --
 tools/perf/util/record.h                           |  1 +
 tools/perf/util/session.c                          |  4 ++
 tools/perf/util/util.c                             | 19 ++++++-
 9 files changed, 197 insertions(+), 9 deletions(-)
 create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt

Regards
Adrian

author reviewer
#1 lore raw

PATCH 1/5 perf data: Correctly identify directory data files

In order to rename the "header" file to "data" without conflicting, correctly identify the non-header files as starting with "data."

---
 tools/perf/util/data.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/data.c +1 -1
index 88fba2ba549f..8993253c5564 100644
@@ -96,7 +96,7 @@ int perf_data__open_dir(struct perf_data *data)
 		if (stat(path, &st))
 			continue;
 
-		if (!S_ISREG(st.st_mode) || strncmp(dent->d_name, "data", 4))
+		if (!S_ISREG(st.st_mode) || strncmp(dent->d_name, "data.", 5))
 			continue;
 
 		ret = -ENOMEM;
Signature · 1 line
2.17.1
merge bot Applied
#2 lore raw

[tip: perf/core] perf data: Correctly identify directory data files

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     490e6db09a9035d7b40a328a2a004ef70b6bdee6
Gitweb:        https://git.kernel.org/tip/490e6db09a9035d7b40a328a2a004ef70b6bdee6
Author:        Adrian Hunter <[email protected]>
AuthorDate:    Fri, 04 Oct 2019 11:31:17 +03:00
Committer:     Arnaldo Carvalho de Melo <[email protected]>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf data: Correctly identify directory data files

In order to rename the "header" file to "data" without conflicting, correctly identify the non-header files as starting with "data."

---
 tools/perf/util/data.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/data.c +1 -1
index 88fba2b..8993253 100644
@@ -96,7 +96,7 @@ int perf_data__open_dir(struct perf_data *data)
 		if (stat(path, &st))
 			continue;
 
-		if (!S_ISREG(st.st_mode) || strncmp(dent->d_name, "data", 4))
+		if (!S_ISREG(st.st_mode) || strncmp(dent->d_name, "data.", 5))
 			continue;
 
 		ret = -ENOMEM;
author reviewer
#3 lore raw

PATCH 2/5 perf data: Move perf_dir_version into data.h

In reply to Adrian Hunter

perf_dir_version belongs to struct perf_data which is declared in data.h. To allow its use in inline perf_data functions, move perf_dir_version to data.h

---
 tools/perf/util/data.h   | 4 ++++
 tools/perf/util/header.h | 4 ----
 2 files changed, 4 insertions(+), 4 deletions(-)
tools/perf/util/data.h +4 -0
index 259868a39019..218fe9a16801 100644
@@ -9,6 +9,10 @@ enum perf_data_mode {
 	PERF_DATA_MODE_READ,
 };
 
+enum perf_dir_version {
+	PERF_DIR_VERSION	= 1,
+};
+
 struct perf_data_file {
 	char		*path;
 	int		 fd;
tools/perf/util/header.h +0 -4
index ca53a929e9fd..840f95cee349 100644
@@ -52,10 +52,6 @@ enum perf_header_version {
 	PERF_HEADER_VERSION_2,
 };
 
-enum perf_dir_version {
-	PERF_DIR_VERSION	= 1,
-};
-
 struct perf_file_section {
 	u64 offset;
 	u64 size;
Signature · 1 line
2.17.1
merge bot Applied
#4 lore raw

[tip: perf/core] perf data: Move perf_dir_version into data.h

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     3dedec4f5ccc8048b9a2cfe89838c3b3275b6b2b
Gitweb:        https://git.kernel.org/tip/3dedec4f5ccc8048b9a2cfe89838c3b3275b6b2b
Author:        Adrian Hunter <[email protected]>
AuthorDate:    Fri, 04 Oct 2019 11:31:18 +03:00
Committer:     Arnaldo Carvalho de Melo <[email protected]>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf data: Move perf_dir_version into data.h

perf_dir_version belongs to struct perf_data which is declared in data.h. To allow its use in inline perf_data functions, move perf_dir_version to data.h

---
 tools/perf/util/data.h   | 4 ++++
 tools/perf/util/header.h | 4 ----
 2 files changed, 4 insertions(+), 4 deletions(-)
tools/perf/util/data.h +4 -0
index 259868a..218fe9a 100644
@@ -9,6 +9,10 @@ enum perf_data_mode {
 	PERF_DATA_MODE_READ,
 };
 
+enum perf_dir_version {
+	PERF_DIR_VERSION	= 1,
+};
+
 struct perf_data_file {
 	char		*path;
 	int		 fd;
tools/perf/util/header.h +0 -4
index ca53a92..840f95c 100644
@@ -52,10 +52,6 @@ enum perf_header_version {
 	PERF_HEADER_VERSION_2,
 };
 
-enum perf_dir_version {
-	PERF_DIR_VERSION	= 1,
-};
-
 struct perf_file_section {
 	u64 offset;
 	u64 size;
author reviewer
#5 lore raw

PATCH 3/5 perf data: Rename directory "header" file to "data"

In reply to Adrian Hunter

In preparation to support a single file directory format, rename "header" to "data" because "header" is a mis-leading name when there is only 1 file. Note, in the multi-file case, the "header" file also contains data.

---
 tools/perf/util/data.c | 2 +-
 tools/perf/util/util.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/data.c +1 -1
index 8993253c5564..df173f0bf654 100644
@@ -306,7 +306,7 @@ static int open_dir(struct perf_data *data)
 	 * So far we open only the header, so we can read the data version and
 	 * layout.
 	 */
-	if (asprintf(&data->file.path, "%s/header", data->path) < 0)
+	if (asprintf(&data->file.path, "%s/data", data->path) < 0)
 		return -1;
 
 	if (perf_data__is_write(data) &&
tools/perf/util/util.c +1 -1
index 5eda6e19c947..56f30ad29ca7 100644
@@ -183,7 +183,7 @@ static int rm_rf_depth_pat(const char *path, int depth, const char **pat)
 int rm_rf_perf_data(const char *path)
 {
 	const char *pat[] = {
-		"header",
+		"data",
 		"data.*",
 		NULL,
 	};
Signature · 1 line
2.17.1
merge bot Applied
#6 lore raw

[tip: perf/core] perf data: Rename directory "header" file to "data"

The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     9b70b9db4e0cc03d224795a18088fdb916dec823
Gitweb:        https://git.kernel.org/tip/9b70b9db4e0cc03d224795a18088fdb916dec823
Author:        Adrian Hunter <[email protected]>
AuthorDate:    Fri, 04 Oct 2019 11:31:19 +03:00
Committer:     Arnaldo Carvalho de Melo <[email protected]>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf data: Rename directory "header" file to "data"

In preparation to support a single file directory format, rename "header" to "data" because "header" is a mis-leading name when there is only 1 file. Note, in the multi-file case, the "header" file also contains data.

---
 tools/perf/util/data.c | 2 +-
 tools/perf/util/util.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/data.c +1 -1
index 8993253..df173f0 100644
@@ -306,7 +306,7 @@ static int open_dir(struct perf_data *data)
 	 * So far we open only the header, so we can read the data version and
 	 * layout.
 	 */
-	if (asprintf(&data->file.path, "%s/header", data->path) < 0)
+	if (asprintf(&data->file.path, "%s/data", data->path) < 0)
 		return -1;
 
 	if (perf_data__is_write(data) &&
tools/perf/util/util.c +1 -1
index ae56c76..3096654 100644
@@ -185,7 +185,7 @@ static int rm_rf_depth_pat(const char *path, int depth, const char **pat)
 int rm_rf_perf_data(const char *path)
 {
 	const char *pat[] = {
-		"header",
+		"data",
 		"data.*",
 		NULL,
 	};
author reviewer
#7 lore raw

PATCH 4/5 perf tools: Support single perf.data file directory

In reply to Adrian Hunter

Support directory output that contains a regular perf.data file, named "data". By default the directory is named perf.data i.e.
perf.data
└── data

Most of the infrastucture to support a directory is already there. This patch makes the changes needed to support the format above.

Presently there is no 'perf record' option to output a directory.

This is preparation for adding support for putting a copy of /proc/kcore in the directory.

---
 .../perf.data-directory-format.txt            | 28 +++++++++++++++++++
 tools/perf/builtin-record.c                   |  2 +-
 tools/perf/util/data.c                        |  9 +++++-
 tools/perf/util/data.h                        |  6 ++++
 4 files changed, 43 insertions(+), 2 deletions(-)
 create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt
tools/perf/Documentation/perf.data-directory-format.txt new file +28 -0
new file mode 100644
index 000000000000..4bf08908178d
@@ -0,0 +1,28 @@
+perf.data directory format
+
+DISCLAIMER This is not ABI yet and is subject to possible change
+           in following versions of perf. We will remove this
+           disclaimer once the directory format soaks in.
+
+
+This document describes the on-disk perf.data directory format.
+
+The layout is described by HEADER_DIR_FORMAT feature.
+Currently it holds only version number (0):
+
+  HEADER_DIR_FORMAT = 24
+
+  struct {
+     uint64_t version;
+  }
+
+The current only version value 0 means that:
+  - there is a single perf.data file named 'data' within the directory.
+  e.g.
+
+    $ tree -ps perf.data
+    perf.data
+    └── [-rw-------       25912]  data
+
+Future versions are expected to describe different data files
+layout according to special needs.
tools/perf/builtin-record.c +1 -1
index 23332861de6e..fceac9d42b4e 100644
@@ -537,7 +537,7 @@ static int record__process_auxtrace(struct perf_tool *tool,
 	size_t padding;
 	u8 pad[8] = {0};
 
-	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
+	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
 		off_t file_offset;
 		int fd = perf_data__fd(data);
 		int err;
tools/perf/util/data.c +8 -1
index df173f0bf654..964ea101dba6 100644
@@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
 	DIR *dir;
 	int nr = 0;
 
+	/*
+	 * Directory containing a single regular perf data file which is already
+	 * open, means there is nothing more to do here.
+	 */
+	if (perf_data__is_single_file(data))
+		return 0;
+
 	if (WARN_ON(!data->is_dir))
 		return -EINVAL;
 
@@ -406,7 +413,7 @@ unsigned long perf_data__size(struct perf_data *data)
 	u64 size = data->file.size;
 	int i;
 
-	if (!data->is_dir)
+	if (perf_data__is_single_file(data))
 		return size;
 
 	for (i = 0; i < data->dir.nr; i++) {
tools/perf/util/data.h +6 -0
index 218fe9a16801..f68815f7e428 100644
@@ -10,6 +10,7 @@ enum perf_data_mode {
 };
 
 enum perf_dir_version {
+	PERF_DIR_SINGLE_FILE	= 0,
 	PERF_DIR_VERSION	= 1,
 };
 
@@ -54,6 +55,11 @@ static inline bool perf_data__is_dir(struct perf_data *data)
 	return data->is_dir;
 }
 
+static inline bool perf_data__is_single_file(struct perf_data *data)
+{
+	return data->dir.version == PERF_DIR_SINGLE_FILE;
+}
+
 static inline int perf_data__fd(struct perf_data *data)
 {
 	return data->file.fd;
Signature · 1 line
2.17.1
#8 lore raw

Re: [PATCH 4/5] perf tools: Support single perf.data file directory

On Fri, Oct 04, 2019 at 11:31:20AM +0300, Adrian Hunter wrote:

SNIP

Quoted 12 lines
(quoted diff) +1 -1
 	u8 pad[8] = {0};
 
-	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
+	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
 		off_t file_offset;
 		int fd = perf_data__fd(data);
 		int err;
tools/perf/util/data.c +0 -0
index df173f0bf654..964ea101dba6 100644

tools/perf/util/data.c int perf_data__open_dir(struct perf_data *data) · line 85

Quoted 10 lines
tools/perf/util/data.c +7 -0
@@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
 	DIR *dir;
 	int nr = 0;
 
+	/*
+	 * Directory containing a single regular perf data file which is already
+	 * open, means there is nothing more to do here.
+	 */
+	if (perf_data__is_single_file(data))
+		return 0;
+

cool, I like this approach much more than the previous flag

any change you (if there's repost) or Arnaldo
could squeeze in indent change below?

thanks,
jirka

---

tools/perf/util/session.c +2 -2
index bfa80fe8d369..7f567a521cea 100644
@@ -227,8 +227,8 @@ struct perf_session *perf_session__new(struct perf_data *data,
 			/* Open the directory data. */
 			if (data->is_dir) {
 				ret = perf_data__open_dir(data);
-			if (ret)
-				goto out_delete;
+				if (ret)
+					goto out_delete;
 			}
 
 			if (!symbol_conf.kallsyms_name &&
author reviewer
#9 lore raw

Re: [PATCH 4/5] perf tools: Support single perf.data file directory

Quoted 19 lines · Jiri Olsa

On Fri, Oct 04, 2019 at 11:31:20AM +0300, Adrian Hunter wrote:

SNIP

Quoted 15 lines
(quoted diff) +1 -1
 	u8 pad[8] = {0};
 
-	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
+	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
 		off_t file_offset;
 		int fd = perf_data__fd(data);
 		int err;
tools/perf/util/data.c +0 -0
index df173f0bf654..964ea101dba6 100644
@@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
 	DIR *dir;
 	int nr = 0;
 
Quoted 9 lines

tools/perf/util/data.c int perf_data__open_dir(struct perf_data *data) · line 85

Quoted 7 lines
tools/perf/util/data.c +7 -0
+	/*
+	 * Directory containing a single regular perf data file which is already
+	 * open, means there is nothing more to do here.
+	 */
+	if (perf_data__is_single_file(data))
+		return 0;
+

cool, I like this approach much more than the previous flag

Yes it is much nicer. Thanks for your direction on that.

Quoted 3 lines

any change you (if there's repost) or Arnaldo
could squeeze in indent change below?

Sent a patch, to be applied before these.

Quoted 22 lines

thanks,
jirka

---

tools/perf/util/session.c +2 -2
index bfa80fe8d369..7f567a521cea 100644
@@ -227,8 +227,8 @@ struct perf_session *perf_session__new(struct perf_data *data,
 			/* Open the directory data. */
 			if (data->is_dir) {
 				ret = perf_data__open_dir(data);
-			if (ret)
-				goto out_delete;
+				if (ret)
+					goto out_delete;
 			}
 
 			if (!symbol_conf.kallsyms_name &&
author reviewer
#10 lore raw

Re: [PATCH 4/5] perf tools: Support single perf.data file directory

Quoted 30 lines · Adrian Hunter
Quoted 19 lines · Jiri Olsa

On Fri, Oct 04, 2019 at 11:31:20AM +0300, Adrian Hunter wrote:

SNIP

Quoted 15 lines
(quoted diff) +1 -1
 	u8 pad[8] = {0};
 
-	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
+	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
 		off_t file_offset;
 		int fd = perf_data__fd(data);
 		int err;
tools/perf/util/data.c +0 -0
index df173f0bf654..964ea101dba6 100644
@@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
 	DIR *dir;
 	int nr = 0;
 
Quoted 9 lines

tools/perf/util/data.c int perf_data__open_dir(struct perf_data *data) · line 85

Quoted 7 lines
tools/perf/util/data.c +7 -0
+	/*
+	 * Directory containing a single regular perf data file which is already
+	 * open, means there is nothing more to do here.
+	 */
+	if (perf_data__is_single_file(data))
+		return 0;
+

cool, I like this approach much more than the previous flag

Quoted 8 lines

Yes it is much nicer. Thanks for your direction on that.

Quoted 3 lines

any change you (if there's repost) or Arnaldo
could squeeze in indent change below?

Sent a patch, to be applied before these.

That is:

"[PATCH] perf session: Fix indent in perf_session__new()"

Any comments on this patch set Arnaldo?

#11 lore raw

Re: [PATCH 4/5] perf tools: Support single perf.data file directory

Em Mon, Oct 21, 2019 at 03:42:39PM +0300, Adrian Hunter escreveu:

Quoted 37 lines · Adrian Hunter
Quoted 36 lines · Jiri Olsa
Quoted 19 lines · Adrian Hunter

SNIP

Quoted 15 lines
(quoted diff) +1 -1
 	u8 pad[8] = {0};
 
-	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
+	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
 		off_t file_offset;
 		int fd = perf_data__fd(data);
 		int err;
tools/perf/util/data.c +0 -0
index df173f0bf654..964ea101dba6 100644
@@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
 	DIR *dir;
 	int nr = 0;
 
Quoted 9 lines

tools/perf/util/data.c int perf_data__open_dir(struct perf_data *data) · line 85

Quoted 7 lines
tools/perf/util/data.c +7 -0
+	/*
+	 * Directory containing a single regular perf data file which is already
+	 * open, means there is nothing more to do here.
+	 */
+	if (perf_data__is_single_file(data))
+		return 0;
+

cool, I like this approach much more than the previous flag

Yes it is much nicer. Thanks for your direction on that.

Quoted 3 lines

any change you (if there's repost) or Arnaldo
could squeeze in indent change below?

Quoted 8 lines
Quoted 2 lines

Sent a patch, to be applied before these.

That is:

"[PATCH] perf session: Fix indent in perf_session__new()"

Any comments on this patch set Arnaldo?

I'll take a look at it later, I think Jiri is ok with it already?

- Arnaldo

#12 lore raw

Re: [PATCH 4/5] perf tools: Support single perf.data file directory

Quoted 41 lines · Arnaldo Carvalho de Melo

Em Mon, Oct 21, 2019 at 03:42:39PM +0300, Adrian Hunter escreveu:

Quoted 40 lines · Adrian Hunter
Quoted 38 lines · Jiri Olsa
Quoted 19 lines · Adrian Hunter

SNIP

Quoted 15 lines
(quoted diff) +1 -1
 	u8 pad[8] = {0};
 
-	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
+	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
 		off_t file_offset;
 		int fd = perf_data__fd(data);
 		int err;
tools/perf/util/data.c +0 -0
index df173f0bf654..964ea101dba6 100644
@@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
 	DIR *dir;
 	int nr = 0;
 
Quoted 9 lines

tools/perf/util/data.c int perf_data__open_dir(struct perf_data *data) · line 85

Quoted 7 lines
tools/perf/util/data.c +7 -0
+	/*
+	 * Directory containing a single regular perf data file which is already
+	 * open, means there is nothing more to do here.
+	 */
+	if (perf_data__is_single_file(data))
+		return 0;
+

cool, I like this approach much more than the previous flag

Yes it is much nicer. Thanks for your direction on that.

Quoted 3 lines

any change you (if there's repost) or Arnaldo
could squeeze in indent change below?

Sent a patch, to be applied before these.

Quoted 7 lines
Quoted 5 lines

That is:

"[PATCH] perf session: Fix indent in perf_session__new()"

Any comments on this patch set Arnaldo?

I'll take a look at it later, I think Jiri is ok with it already?

yes, it's just indent fix

jirka

#13 lore raw

Re: [PATCH 4/5] perf tools: Support single perf.data file directory

In reply to Adrian Hunter

Em Mon, Oct 21, 2019 at 03:42:39PM +0300, Adrian Hunter escreveu:

Quoted 37 lines · Adrian Hunter
Quoted 36 lines · Jiri Olsa
Quoted 19 lines · Adrian Hunter

SNIP

Quoted 15 lines
(quoted diff) +1 -1
 	u8 pad[8] = {0};
 
-	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
+	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
 		off_t file_offset;
 		int fd = perf_data__fd(data);
 		int err;
tools/perf/util/data.c +0 -0
index df173f0bf654..964ea101dba6 100644
@@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
 	DIR *dir;
 	int nr = 0;
 
Quoted 9 lines

tools/perf/util/data.c int perf_data__open_dir(struct perf_data *data) · line 85

Quoted 7 lines
tools/perf/util/data.c +7 -0
+	/*
+	 * Directory containing a single regular perf data file which is already
+	 * open, means there is nothing more to do here.
+	 */
+	if (perf_data__is_single_file(data))
+		return 0;
+

cool, I like this approach much more than the previous flag

Yes it is much nicer. Thanks for your direction on that.

Quoted 3 lines

any change you (if there's repost) or Arnaldo
could squeeze in indent change below?

Quoted 6 lines
Quoted 2 lines

Sent a patch, to be applied before these.

That is:

"[PATCH] perf session: Fix indent in perf_session__new()"

Done it as a separate patch, etc.

Quoted 1 line

Any comments on this patch set Arnaldo?

Signature · 1 line
- Arnaldo
merge bot Applied
#14 lore raw

[tip: perf/core] perf session: Fix indent in perf_session__new()"

In reply to Jiri Olsa
The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     01e97a59ea3e071e5c60ad46c40b4db748bfe928
Gitweb:        https://git.kernel.org/tip/01e97a59ea3e071e5c60ad46c40b4db748bfe928
Author:        Jiri Olsa <[email protected]>
AuthorDate:    Mon, 07 Oct 2019 13:20:27 +02:00
Committer:     Arnaldo Carvalho de Melo <[email protected]>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf session: Fix indent in perf_session__new()"

Fix up indentation.

---
 tools/perf/util/session.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/session.c +2 -2
index 6cc32f5..0266604 100644
@@ -227,8 +227,8 @@ struct perf_session *perf_session__new(struct perf_data *data,
 			/* Open the directory data. */
 			if (data->is_dir) {
 				ret = perf_data__open_dir(data);
-			if (ret)
-				goto out_delete;
+				if (ret)
+					goto out_delete;
 			}
 		}
 	} else  {
merge bot Applied
#15 lore raw

[tip: perf/core] perf data: Support single perf.data file directory

In reply to Adrian Hunter
The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     46e201efa15b70ec39df5237116fddebb4f5057c
Gitweb:        https://git.kernel.org/tip/46e201efa15b70ec39df5237116fddebb4f5057c
Author:        Adrian Hunter <[email protected]>
AuthorDate:    Fri, 04 Oct 2019 11:31:20 +03:00
Committer:     Arnaldo Carvalho de Melo <[email protected]>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf data: Support single perf.data file directory

Support directory output that contains a regular perf.data file, named "data". By default the directory is named perf.data i.e.
perf.data
└── data

Most of the infrastructure to support a directory is already there. This patch makes the changes needed to support the format above.

Presently there is no 'perf record' option to output a directory.

This is preparation for adding support for putting a copy of /proc/kcore in the directory.

---
 tools/perf/Documentation/perf.data-directory-format.txt | 28 ++++++++-
 tools/perf/builtin-record.c                             |  2 +-
 tools/perf/util/data.c                                  |  9 ++-
 tools/perf/util/data.h                                  |  6 ++-
 4 files changed, 43 insertions(+), 2 deletions(-)
 create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt
tools/perf/Documentation/perf.data-directory-format.txt new file +28 -0
new file mode 100644
index 0000000..4bf0890
@@ -0,0 +1,28 @@
+perf.data directory format
+
+DISCLAIMER This is not ABI yet and is subject to possible change
+           in following versions of perf. We will remove this
+           disclaimer once the directory format soaks in.
+
+
+This document describes the on-disk perf.data directory format.
+
+The layout is described by HEADER_DIR_FORMAT feature.
+Currently it holds only version number (0):
+
+  HEADER_DIR_FORMAT = 24
+
+  struct {
+     uint64_t version;
+  }
+
+The current only version value 0 means that:
+  - there is a single perf.data file named 'data' within the directory.
+  e.g.
+
+    $ tree -ps perf.data
+    perf.data
+    └── [-rw-------       25912]  data
+
+Future versions are expected to describe different data files
+layout according to special needs.
tools/perf/builtin-record.c +1 -1
index 2fb83aa..e402459 100644
@@ -537,7 +537,7 @@ static int record__process_auxtrace(struct perf_tool *tool,
 	size_t padding;
 	u8 pad[8] = {0};
 
-	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
+	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
 		off_t file_offset;
 		int fd = perf_data__fd(data);
 		int err;
tools/perf/util/data.c +8 -1
index df173f0..964ea10 100644
@@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
 	DIR *dir;
 	int nr = 0;
 
+	/*
+	 * Directory containing a single regular perf data file which is already
+	 * open, means there is nothing more to do here.
+	 */
+	if (perf_data__is_single_file(data))
+		return 0;
+
 	if (WARN_ON(!data->is_dir))
 		return -EINVAL;
 
@@ -406,7 +413,7 @@ unsigned long perf_data__size(struct perf_data *data)
 	u64 size = data->file.size;
 	int i;
 
-	if (!data->is_dir)
+	if (perf_data__is_single_file(data))
 		return size;
 
 	for (i = 0; i < data->dir.nr; i++) {
tools/perf/util/data.h +6 -0
index 218fe9a..f68815f 100644
@@ -10,6 +10,7 @@ enum perf_data_mode {
 };
 
 enum perf_dir_version {
+	PERF_DIR_SINGLE_FILE	= 0,
 	PERF_DIR_VERSION	= 1,
 };
 
@@ -54,6 +55,11 @@ static inline bool perf_data__is_dir(struct perf_data *data)
 	return data->is_dir;
 }
 
+static inline bool perf_data__is_single_file(struct perf_data *data)
+{
+	return data->dir.version == PERF_DIR_SINGLE_FILE;
+}
+
 static inline int perf_data__fd(struct perf_data *data)
 {
 	return data->file.fd;
author reviewer
#16 lore raw

PATCH 5/5 perf record: Put a copy of kcore into the perf.data directory

In reply to Adrian Hunter

Add a new 'perf record' option '--kcore' which will put a copy of /proc/kcore, kallsyms and modules into a perf.data directory. Note, that without the --kcore option, output goes to a file as previously. The tools' -o and -i options work with either a file name or directory name.

Example:

 $ sudo perf record --kcore uname
 $ sudo tree perf.data
 perf.data
 ├── kcore_dir
 │   ├── kallsyms
 │   ├── kcore
 │   └── modules
 └── data
 $ sudo perf script -v
 build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
 build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
 Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
 Using CPUID GenuineIntel-6-8E-A
 Using perf.data/kcore_dir/kcore for kernel data
 Using perf.data/kcore_dir/kallsyms for symbols
             perf 19058 506778.423729:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423733:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423734:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423736:        117 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
             perf 19058 506778.423738:       2092 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
             perf 19058 506778.423740:      37380 cycles:  ffffffffa2f121d0 perf_event_addr_filters_exec+0x0 (vmlinux)
            uname 19058 506778.423751:     582673 cycles:  ffffffffa303a407 propagate_protected_usage+0x147 (vmlinux)
            uname 19058 506778.423892:    2241841 cycles:  ffffffffa2cae0c9 unwind_next_frame.part.5+0x79 (vmlinux)
            uname 19058 506778.424430:    2457397 cycles:  ffffffffa3019232 check_memory_region+0x52 (vmlinux)
---
 tools/perf/Documentation/perf-record.txt      |  3 ++
 .../perf.data-directory-format.txt            | 35 +++++++++++++
 tools/perf/builtin-record.c                   | 52 +++++++++++++++++++
 tools/perf/util/data.c                        | 33 ++++++++++++
 tools/perf/util/data.h                        |  2 +
 tools/perf/util/record.h                      |  1 +
 tools/perf/util/session.c                     |  4 ++
 tools/perf/util/util.c                        | 17 ++++++
 8 files changed, 147 insertions(+)
tools/perf/Documentation/perf-record.txt +3 -0
index c6f9f31b6039..8a4506113d9f 100644
@@ -571,6 +571,9 @@ config terms. For example: 'cycles/overwrite/' and 'instructions/no-overwrite/'.
 
 Implies --tail-synthesize.
 
+--kcore::
+Make a copy of /proc/kcore and place it into a directory with the perf data file.
+
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/Documentation/perf.data-directory-format.txt +35 -0
index 4bf08908178d..f37fbd29112e 100644
@@ -26,3 +26,38 @@ The current only version value 0 means that:
 
 Future versions are expected to describe different data files
 layout according to special needs.
+
+Currently the only 'perf record' option to output to a directory is
+the --kcore option which puts a copy of /proc/kcore into the directory.
+e.g.
+
+  $ sudo perf record --kcore uname
+  Linux
+  [ perf record: Woken up 1 times to write data ]
+  [ perf record: Captured and wrote 0.015 MB perf.data (9 samples) ]
+  $ sudo tree -ps perf.data
+  perf.data
+  ├── [-rw-------       23744]  data
+  └── [drwx------        4096]  kcore_dir
+      ├── [-r--------     6731125]  kallsyms
+      ├── [-r--------    40230912]  kcore
+      └── [-r--------        5419]  modules
+
+  1 directory, 4 files
+  $ sudo perf script -v
+  build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
+  build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
+  build id event received for /lib/x86_64-linux-gnu/libc-2.28.so: 5b157f49586a3ca84d55837f97ff466767dd3445
+  Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
+  Using CPUID GenuineIntel-6-8E-A
+  Using perf.data/kcore_dir/kcore for kernel data
+  Using perf.data/kcore_dir/kallsyms for symbols
+              perf 15316 2060795.480902:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480906:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480908:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480910:        119 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
+              perf 15316 2060795.480912:       2109 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
+              perf 15316 2060795.480914:      37606 cycles:  ffffffffa2f121fe perf_event_addr_filters_exec+0x2e (vmlinux)
+             uname 15316 2060795.480924:     588287 cycles:  ffffffffa303a56d page_counter_try_charge+0x6d (vmlinux)
+             uname 15316 2060795.481067:    2261945 cycles:  ffffffffa301438f kmem_cache_free+0x4f (vmlinux)
+             uname 15316 2060795.481643:    2172167 cycles:      7f1a48c393c0 _IO_un_link+0x0 (/lib/x86_64-linux-gnu/libc-2.28.so)
tools/perf/builtin-record.c +52 -0

88 lines not on this page · raw message

tools/perf/util/data.c +33 -0
index 964ea101dba6..c47aa34fdc0a 100644
@@ -424,3 +424,36 @@ unsigned long perf_data__size(struct perf_data *data)
 
 	return size;
 }
+
+int perf_data__make_kcore_dir(struct perf_data *data, char *buf, size_t buf_sz)
+{
+	int ret;
+
+	if (!data->is_dir)
+		return -1;
+
+	ret = snprintf(buf, buf_sz, "%s/kcore_dir", data->path);
+	if (ret < 0 || (size_t)ret >= buf_sz)
+		return -1;
+
+	return mkdir(buf, S_IRWXU);
+}
+
+char *perf_data__kallsyms_name(struct perf_data *data)
+{
+	char *kallsyms_name;
+	struct stat st;
+
+	if (!data->is_dir)
+		return NULL;
+
+	if (asprintf(&kallsyms_name, "%s/kcore_dir/kallsyms", data->path) < 0)
+		return NULL;
+
+	if (stat(kallsyms_name, &st)) {
+		free(kallsyms_name);
+		return NULL;
+	}
+
+	return kallsyms_name;
+}
tools/perf/util/data.h +2 -0
index f68815f7e428..75947ef6bc17 100644
@@ -87,4 +87,6 @@ int perf_data__open_dir(struct perf_data *data);
 void perf_data__close_dir(struct perf_data *data);
 int perf_data__update_dir(struct perf_data *data);
 unsigned long perf_data__size(struct perf_data *data);
+int perf_data__make_kcore_dir(struct perf_data *data, char *buf, size_t buf_sz);
+char *perf_data__kallsyms_name(struct perf_data *data);
 #endif /* __PERF_DATA_H */
tools/perf/util/record.h +1 -0
index 00275afc524d..948bbcf9aef3 100644
@@ -44,6 +44,7 @@ struct record_opts {
 	bool	      strict_freq;
 	bool	      sample_id;
 	bool	      no_bpf_event;
+	bool	      kcore;
 	unsigned int  freq;
 	unsigned int  mmap_pages;
 	unsigned int  auxtrace_mmap_pages;
tools/perf/util/session.c +4 -0
index 061bb4d6a3f5..bfa80fe8d369 100644
@@ -230,6 +230,10 @@ struct perf_session *perf_session__new(struct perf_data *data,
 			if (ret)
 				goto out_delete;
 			}
+
+			if (!symbol_conf.kallsyms_name &&
+			    !symbol_conf.vmlinux_name)
+				symbol_conf.kallsyms_name = perf_data__kallsyms_name(data);
 		}
 	} else  {
 		session->machines.host.env = &perf_env;
tools/perf/util/util.c +17 -0
index 56f30ad29ca7..16f81a0a9208 100644
@@ -180,6 +180,21 @@ static int rm_rf_depth_pat(const char *path, int depth, const char **pat)
 	return rmdir(path);
 }
 
+static int rm_rf_kcore_dir(const char *path)
+{
+	char kcore_dir_path[PATH_MAX];
+	const char *pat[] = {
+		"kcore",
+		"kallsyms",
+		"modules",
+		NULL,
+	};
+
+	snprintf(kcore_dir_path, sizeof(kcore_dir_path), "%s/kcore_dir", path);
+
+	return rm_rf_depth_pat(kcore_dir_path, 0, pat);
+}
+
 int rm_rf_perf_data(const char *path)
 {
 	const char *pat[] = {
@@ -188,6 +203,8 @@ int rm_rf_perf_data(const char *path)
 		NULL,
 	};
 
+	rm_rf_kcore_dir(path);
+
 	return rm_rf_depth_pat(path, 0, pat);
 }
 
Signature · 1 line
2.17.1
#17 lore raw

On Fri, Oct 04, 2019 at 11:31:21AM +0300, Adrian Hunter wrote:

SNIP

Quoted 6 lines
tools/perf/builtin-record.c +2 -0
+}
+
 static int record__mmap_evlist(struct record *rec,
 			       struct evlist *evlist)
 {

tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 1425

Quoted 9 lines
tools/perf/builtin-record.c +6 -0
@@ -1383,6 +1417,12 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 	session->header.env.comp_type  = PERF_COMP_ZSTD;
 	session->header.env.comp_level = rec->opts.comp_level;
 
+	if (rec->opts.kcore &&
+	    !record__kcore_readable(&session->machines.host)) {
+		pr_err("ERROR: kcore is not readable.\n");
+		return -1;
+	}
+

is there any reason why this change is not merged with the change below?

Quoted 5 lines
tools/perf/builtin-record.c +0 -0
 	record__init_features(rec);
 
 	if (rec->opts.use_clockid && rec->opts.clockid_res_ns)
@@ -1414,6 +1454,14 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 	}

tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 1464

Quoted 10 lines
tools/perf/builtin-record.c +8 -0
 	session->header.env.comp_mmap_len = session->evlist->core.mmap_len;
 
+	if (rec->opts.kcore) {
+		err = record__kcore_copy(&session->machines.host, data);
+		if (err) {
+			pr_err("ERROR: Failed to copy kcore\n");
+			goto out_child;
+		}
+	}
+

thanks,
jirka

author reviewer
#18 lore raw
Quoted 13 lines · Jiri Olsa

On Fri, Oct 04, 2019 at 11:31:21AM +0300, Adrian Hunter wrote:

SNIP

Quoted 9 lines
tools/perf/builtin-record.c +2 -0
+}
+
 static int record__mmap_evlist(struct record *rec,
 			       struct evlist *evlist)
 {
@@ -1383,6 +1417,12 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 	session->header.env.comp_type  = PERF_COMP_ZSTD;
 	session->header.env.comp_level = rec->opts.comp_level;
 
Quoted 8 lines

tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 1425

Quoted 6 lines
tools/perf/builtin-record.c +6 -0
+	if (rec->opts.kcore &&
+	    !record__kcore_readable(&session->machines.host)) {
+		pr_err("ERROR: kcore is not readable.\n");
+		return -1;
+	}
+

is there any reason why this change is not merged with the change below?

It seemed better to do the validation before opening all the events.

Quoted 21 lines
Quoted 7 lines
tools/perf/builtin-record.c +0 -0
 	record__init_features(rec);
 
 	if (rec->opts.use_clockid && rec->opts.clockid_res_ns)
@@ -1414,6 +1454,14 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 	}

tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 1464

Quoted 10 lines
tools/perf/builtin-record.c +8 -0
 	session->header.env.comp_mmap_len = session->evlist->core.mmap_len;
 
+	if (rec->opts.kcore) {
+		err = record__kcore_copy(&session->machines.host, data);
+		if (err) {
+			pr_err("ERROR: Failed to copy kcore\n");
+			goto out_child;
+		}
+	}
+

thanks,
jirka

#19 lore raw
In reply to Adrian Hunter

On Fri, Oct 04, 2019 at 11:31:21AM +0300, Adrian Hunter wrote:

SNIP

Quoted 12 lines
tools/perf/util/record.h +1 -0
 	bool	      strict_freq;
 	bool	      sample_id;
 	bool	      no_bpf_event;
+	bool	      kcore;
 	unsigned int  freq;
 	unsigned int  mmap_pages;
 	unsigned int  auxtrace_mmap_pages;
tools/perf/util/session.c +0 -0
index 061bb4d6a3f5..bfa80fe8d369 100644

tools/perf/util/session.c struct perf_session *perf_session__new(struct perf_data *data, · line 236

Quoted 7 lines
tools/perf/util/session.c +4 -0
@@ -230,6 +230,10 @@ struct perf_session *perf_session__new(struct perf_data *data,
 			if (ret)
 				goto out_delete;
 			}
+
+			if (!symbol_conf.kallsyms_name &&
+			    !symbol_conf.vmlinux_name)
+				symbol_conf.kallsyms_name = perf_data__kallsyms_name(data);

hum, should this also depend on rec->opts.kcore ?

jirka

author reviewer
#20 lore raw
Quoted 17 lines · Jiri Olsa

On Fri, Oct 04, 2019 at 11:31:21AM +0300, Adrian Hunter wrote:

SNIP

Quoted 13 lines
tools/perf/util/record.h +1 -0
 	bool	      strict_freq;
 	bool	      sample_id;
 	bool	      no_bpf_event;
+	bool	      kcore;
 	unsigned int  freq;
 	unsigned int  mmap_pages;
 	unsigned int  auxtrace_mmap_pages;
tools/perf/util/session.c +0 -0
index 061bb4d6a3f5..bfa80fe8d369 100644
@@ -230,6 +230,10 @@ struct perf_session *perf_session__new(struct perf_data *data,
 			if (ret)
Quoted 8 lines

tools/perf/util/session.c struct perf_session *perf_session__new(struct perf_data *data, · line 236

Quoted 6 lines
tools/perf/util/session.c +4 -0
 				goto out_delete;
 			}
+
+			if (!symbol_conf.kallsyms_name &&
+			    !symbol_conf.vmlinux_name)
+				symbol_conf.kallsyms_name = perf_data__kallsyms_name(data);

hum, should this also depend on rec->opts.kcore ?

This is the bit that makes 'perf script' work. It has the same affect as though the --kallsyms option was used. i.e. it makes 'perf script' behave the same as:

	perf script --kallsyms perf.data/kcore_dir/kallsyms

Which works because perf looks for kcore in the same directory as kallsyms.

merge bot Applied
#21 lore raw
In reply to Adrian Hunter
The following commit has been merged into the perf/core branch of tip:

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     eeb399b531a1576e36016f8a7f0c50d10194e190
Gitweb:        https://git.kernel.org/tip/eeb399b531a1576e36016f8a7f0c50d10194e190
Author:        Adrian Hunter <[email protected]>
AuthorDate:    Fri, 04 Oct 2019 11:31:21 +03:00
Committer:     Arnaldo Carvalho de Melo <[email protected]>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf record: Put a copy of kcore into the perf.data directory

Add a new 'perf record' option '--kcore' which will put a copy of /proc/kcore, kallsyms and modules into a perf.data directory. Note, that without the --kcore option, output goes to a file as previously. The tools' -o and -i options work with either a file name or directory name.

Example:

  $ sudo perf record --kcore uname
  $ sudo tree perf.data
  perf.data
  ├── kcore_dir
  │   ├── kallsyms
  │   ├── kcore
  │   └── modules
  └── data
  $ sudo perf script -v
  build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
  build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
  Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
  Using CPUID GenuineIntel-6-8E-A
  Using perf.data/kcore_dir/kcore for kernel data
  Using perf.data/kcore_dir/kallsyms for symbols
             perf 19058 506778.423729:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423733:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423734:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423736:        117 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
             perf 19058 506778.423738:       2092 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
             perf 19058 506778.423740:      37380 cycles:  ffffffffa2f121d0 perf_event_addr_filters_exec+0x0 (vmlinux)
            uname 19058 506778.423751:     582673 cycles:  ffffffffa303a407 propagate_protected_usage+0x147 (vmlinux)
            uname 19058 506778.423892:    2241841 cycles:  ffffffffa2cae0c9 unwind_next_frame.part.5+0x79 (vmlinux)
            uname 19058 506778.424430:    2457397 cycles:  ffffffffa3019232 check_memory_region+0x52 (vmlinux)

Committer testing:

  # rm -rf perf.data*
  # perf record sleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.024 MB perf.data (7 samples) ]
  # ls -l perf.data
  -rw-------. 1 root root 34772 Oct 21 11:08 perf.data
  # perf record --kcore uname
  Linux
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.024 MB perf.data (7 samples) ]
  ls[root@quaco ~]# ls -lad perf.data*
  drwx------. 3 root root  4096 Oct 21 11:08 perf.data
  -rw-------. 1 root root 34772 Oct 21 11:08 perf.data.old
  # perf evlist -v
  cycles: size: 112, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD, read_format: ID, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, precise_ip: 3, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, ksymbol: 1, bpf_event: 1
  # perf evlist -v -i perf.data/data
  cycles: size: 112, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD, read_format: ID, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, precise_ip: 3, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, ksymbol: 1, bpf_event: 1
  #
---
 tools/perf/Documentation/perf-record.txt                |  3 +-
 tools/perf/Documentation/perf.data-directory-format.txt | 35 +++++-
 tools/perf/builtin-record.c                             | 52 ++++++++-
 tools/perf/util/data.c                                  | 33 +++++-
 tools/perf/util/data.h                                  |  2 +-
 tools/perf/util/record.h                                |  1 +-
 tools/perf/util/session.c                               |  4 +-
 tools/perf/util/util.c                                  | 17 +++-
 8 files changed, 147 insertions(+)
tools/perf/Documentation/perf-record.txt +3 -0
index c6f9f31..8a45061 100644
@@ -571,6 +571,9 @@ config terms. For example: 'cycles/overwrite/' and 'instructions/no-overwrite/'.
 
 Implies --tail-synthesize.
 
+--kcore::
+Make a copy of /proc/kcore and place it into a directory with the perf data file.
+
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/Documentation/perf.data-directory-format.txt +35 -0
index 4bf0890..f37fbd2 100644
@@ -26,3 +26,38 @@ The current only version value 0 means that:
 
 Future versions are expected to describe different data files
 layout according to special needs.
+
+Currently the only 'perf record' option to output to a directory is
+the --kcore option which puts a copy of /proc/kcore into the directory.
+e.g.
+
+  $ sudo perf record --kcore uname
+  Linux
+  [ perf record: Woken up 1 times to write data ]
+  [ perf record: Captured and wrote 0.015 MB perf.data (9 samples) ]
+  $ sudo tree -ps perf.data
+  perf.data
+  ├── [-rw-------       23744]  data
+  └── [drwx------        4096]  kcore_dir
+      ├── [-r--------     6731125]  kallsyms
+      ├── [-r--------    40230912]  kcore
+      └── [-r--------        5419]  modules
+
+  1 directory, 4 files
+  $ sudo perf script -v
+  build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
+  build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
+  build id event received for /lib/x86_64-linux-gnu/libc-2.28.so: 5b157f49586a3ca84d55837f97ff466767dd3445
+  Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
+  Using CPUID GenuineIntel-6-8E-A
+  Using perf.data/kcore_dir/kcore for kernel data
+  Using perf.data/kcore_dir/kallsyms for symbols
+              perf 15316 2060795.480902:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480906:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480908:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480910:        119 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
+              perf 15316 2060795.480912:       2109 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
+              perf 15316 2060795.480914:      37606 cycles:  ffffffffa2f121fe perf_event_addr_filters_exec+0x2e (vmlinux)
+             uname 15316 2060795.480924:     588287 cycles:  ffffffffa303a56d page_counter_try_charge+0x6d (vmlinux)
+             uname 15316 2060795.481067:    2261945 cycles:  ffffffffa301438f kmem_cache_free+0x4f (vmlinux)
+             uname 15316 2060795.481643:    2172167 cycles:      7f1a48c393c0 _IO_un_link+0x0 (/lib/x86_64-linux-gnu/libc-2.28.so)
tools/perf/builtin-record.c +52 -0

88 lines not on this page · raw message

tools/perf/util/data.c +33 -0
index 964ea10..c47aa34 100644
@@ -424,3 +424,36 @@ unsigned long perf_data__size(struct perf_data *data)
 
 	return size;
 }
+
+int perf_data__make_kcore_dir(struct perf_data *data, char *buf, size_t buf_sz)
+{
+	int ret;
+
+	if (!data->is_dir)
+		return -1;
+
+	ret = snprintf(buf, buf_sz, "%s/kcore_dir", data->path);
+	if (ret < 0 || (size_t)ret >= buf_sz)
+		return -1;
+
+	return mkdir(buf, S_IRWXU);
+}
+
+char *perf_data__kallsyms_name(struct perf_data *data)
+{
+	char *kallsyms_name;
+	struct stat st;
+
+	if (!data->is_dir)
+		return NULL;
+
+	if (asprintf(&kallsyms_name, "%s/kcore_dir/kallsyms", data->path) < 0)
+		return NULL;
+
+	if (stat(kallsyms_name, &st)) {
+		free(kallsyms_name);
+		return NULL;
+	}
+
+	return kallsyms_name;
+}
tools/perf/util/data.h +2 -0
index f68815f..75947ef 100644
@@ -87,4 +87,6 @@ int perf_data__open_dir(struct perf_data *data);
 void perf_data__close_dir(struct perf_data *data);
 int perf_data__update_dir(struct perf_data *data);
 unsigned long perf_data__size(struct perf_data *data);
+int perf_data__make_kcore_dir(struct perf_data *data, char *buf, size_t buf_sz);
+char *perf_data__kallsyms_name(struct perf_data *data);
 #endif /* __PERF_DATA_H */
tools/perf/util/record.h +1 -0
index 00275af..948bbcf 100644
@@ -44,6 +44,7 @@ struct record_opts {
 	bool	      strict_freq;
 	bool	      sample_id;
 	bool	      no_bpf_event;
+	bool	      kcore;
 	unsigned int  freq;
 	unsigned int  mmap_pages;
 	unsigned int  auxtrace_mmap_pages;
tools/perf/util/session.c +4 -0
index 0266604..f07b8ec 100644
@@ -230,6 +230,10 @@ struct perf_session *perf_session__new(struct perf_data *data,
 				if (ret)
 					goto out_delete;
 			}
+
+			if (!symbol_conf.kallsyms_name &&
+			    !symbol_conf.vmlinux_name)
+				symbol_conf.kallsyms_name = perf_data__kallsyms_name(data);
 		}
 	} else  {
 		session->machines.host.env = &perf_env;
tools/perf/util/util.c +17 -0
index 3096654..969ae56 100644
@@ -182,6 +182,21 @@ static int rm_rf_depth_pat(const char *path, int depth, const char **pat)
 	return rmdir(path);
 }
 
+static int rm_rf_kcore_dir(const char *path)
+{
+	char kcore_dir_path[PATH_MAX];
+	const char *pat[] = {
+		"kcore",
+		"kallsyms",
+		"modules",
+		NULL,
+	};
+
+	snprintf(kcore_dir_path, sizeof(kcore_dir_path), "%s/kcore_dir", path);
+
+	return rm_rf_depth_pat(kcore_dir_path, 0, pat);
+}
+
 int rm_rf_perf_data(const char *path)
 {
 	const char *pat[] = {
@@ -190,6 +205,8 @@ int rm_rf_perf_data(const char *path)
 		NULL,
 	};
 
+	rm_rf_kcore_dir(path);
+
 	return rm_rf_depth_pat(path, 0, pat);
 }
 
Reviewed-by on the v1 series
#22 lore raw
In reply to Adrian Hunter
Quoted 38 lines · Adrian Hunter

Hi

Here are patches to add a new 'perf record' option '--kcore' which will put a copy of /proc/kcore, kallsyms and modules into a perf.data directory. Note, that without the --kcore option, output goes to a file as previously. The tools' -o and -i options work with either a file name or directory name.

Example:

 $ sudo perf record --kcore uname
 $ sudo tree perf.data
 perf.data
 ├── kcore_dir
 │   ├── kallsyms
 │   ├── kcore
 │   └── modules
 └── data
 $ sudo perf script -v
 build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
 build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
 Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
 Using CPUID GenuineIntel-6-8E-A
 Using perf.data/kcore_dir/kcore for kernel data
 Using perf.data/kcore_dir/kallsyms for symbols
             perf 19058 506778.423729:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423733:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423734:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423736:        117 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
             perf 19058 506778.423738:       2092 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
             perf 19058 506778.423740:      37380 cycles:  ffffffffa2f121d0 perf_event_addr_filters_exec+0x0 (vmlinux)
            uname 19058 506778.423751:     582673 cycles:  ffffffffa303a407 propagate_protected_usage+0x147 (vmlinux)
            uname 19058 506778.423892:    2241841 cycles:  ffffffffa2cae0c9 unwind_next_frame.part.5+0x79 (vmlinux)
            uname 19058 506778.424430:    2457397 cycles:  ffffffffa3019232 check_memory_region+0x52 (vmlinux)
Quoted 6 lines

Adrian Hunter (5):
perf data: Correctly identify directory data files perf data: Move perf_dir_version into data.h
perf data: Rename directory "header" file to "data" perf tools: Support single perf.data file directory perf record: Put a copy of kcore into the perf.data directory

thanks,
jirka

Quoted 16 lines
 tools/perf/Documentation/perf-record.txt           |  3 ++
 .../Documentation/perf.data-directory-format.txt   | 63 ++++++++++++++++++++++
 tools/perf/builtin-record.c                        | 54 ++++++++++++++++++-
 tools/perf/util/data.c                             | 46 ++++++++++++++--
 tools/perf/util/data.h                             | 12 +++++
 tools/perf/util/header.h                           |  4 --
 tools/perf/util/record.h                           |  1 +
 tools/perf/util/session.c                          |  4 ++
 tools/perf/util/util.c                             | 19 ++++++-
 9 files changed, 197 insertions(+), 9 deletions(-)
 create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt

Regards
Adrian

#23 lore raw
In reply to Adrian Hunter

Em Fri, Oct 04, 2019 at 11:31:16AM +0300, Adrian Hunter escreveu:

Quoted 7 lines

Hi

Here are patches to add a new 'perf record' option '--kcore' which will put a copy of /proc/kcore, kallsyms and modules into a perf.data directory. Note, that without the --kcore option, output goes to a file as previously. The tools' -o and -i options work with either a file name or directory name.

Its great that you two got in agreement about adding the new feature using the infrastructure that is being put in place for perf.data as a directory, great team work.

Applied, thanks a lot,

- Arnaldo

Quoted 52 lines

Example:

 $ sudo perf record --kcore uname
 $ sudo tree perf.data
 perf.data
 ├── kcore_dir
 │   ├── kallsyms
 │   ├── kcore
 │   └── modules
 └── data
 $ sudo perf script -v
 build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
 build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
 Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
 Using CPUID GenuineIntel-6-8E-A
 Using perf.data/kcore_dir/kcore for kernel data
 Using perf.data/kcore_dir/kallsyms for symbols
             perf 19058 506778.423729:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423733:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423734:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423736:        117 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
             perf 19058 506778.423738:       2092 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
             perf 19058 506778.423740:      37380 cycles:  ffffffffa2f121d0 perf_event_addr_filters_exec+0x0 (vmlinux)
            uname 19058 506778.423751:     582673 cycles:  ffffffffa303a407 propagate_protected_usage+0x147 (vmlinux)
            uname 19058 506778.423892:    2241841 cycles:  ffffffffa2cae0c9 unwind_next_frame.part.5+0x79 (vmlinux)
            uname 19058 506778.424430:    2457397 cycles:  ffffffffa3019232 check_memory_region+0x52 (vmlinux)

Adrian Hunter (5):
perf data: Correctly identify directory data files perf data: Move perf_dir_version into data.h
perf data: Rename directory "header" file to "data" perf tools: Support single perf.data file directory perf record: Put a copy of kcore into the perf.data directory

 tools/perf/Documentation/perf-record.txt           |  3 ++
 .../Documentation/perf.data-directory-format.txt   | 63 ++++++++++++++++++++++
 tools/perf/builtin-record.c                        | 54 ++++++++++++++++++-
 tools/perf/util/data.c                             | 46 ++++++++++++++--
 tools/perf/util/data.h                             | 12 +++++
 tools/perf/util/header.h                           |  4 --
 tools/perf/util/record.h                           |  1 +
 tools/perf/util/session.c                          |  4 ++
 tools/perf/util/util.c                             | 19 ++++++-
 9 files changed, 197 insertions(+), 9 deletions(-)
 create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt

Regards
Adrian

Signature · 1 line
- Arnaldo