Skip to contentSkip to conversation

perf archive: unpack to correct dir given by perf

PATCH In review Haoze Xie 3 participants 5 messages Jul 7, 2024 – Jul 23, 2024 1 message not in the archive perf events

lore
Share & tools
lore

Haoze Xie

author
#1 lore raw

PATCH 1/1 perf archive: unpack to correct dir given by perf

In reply to a message not in the archive

In perf-archive.sh, the code segment that defines 'PERF_BUILDID_DIR' is advanced before 'unpack' operation for subsequent use, followed by a 'mkdir' operation to ensure that the dir exists. Symbols in 'unpack' will be extracted to correct dir given by perf.

When '--unpack' param is appointed, the symbols are extracted to '~/.debug' folder by default, without using 'PERF_BUILDID_DIR' given by perf. This will cause perf to be unable to find the correct buildid's path when users configured buildid.dir in 'perf config' or used '--buildid-dir' cli param, since perf will read these params and put them in 'PERF_BUILDID_DIR' env. 'perf script' and 'perf report' will use the env as the basis for buildid indexing.

Fixes e43c64c971e4 ("perf archive: Add new option '--unpack' to expand tarballs")
---
 tools/perf/perf-archive.sh | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)
tools/perf/perf-archive.sh +15 -13
index 6ed7e52ab881..f29bbc129056 100755
@@ -23,6 +23,19 @@ while [ $# -gt 0 ] ; do
 	fi
 done
 
+#
+# PERF_BUILDID_DIR environment variable set by perf
+# path to buildid directory, default to $HOME/.debug
+#
+if [ -z $PERF_BUILDID_DIR ]; then
+	PERF_BUILDID_DIR=~/.debug/
+else
+	# append / to make substitutions work
+	PERF_BUILDID_DIR=$PERF_BUILDID_DIR/
+fi
+
+mkdir -p $PERF_BUILDID_DIR
+
 if [ $UNPACK -eq 1 ]; then
 	if [ ! -z "$UNPACK_TAR" ]; then					# tar given as an argument
 		if [ ! -e "$UNPACK_TAR" ]; then
@@ -65,25 +78,14 @@ if [ $UNPACK -eq 1 ]; then
 		fi
 
 		# unzip the perf.data file in the current working directory	and debug symbols in ~/.debug directory
-		tar xvf $TARGET && tar xvf $PERF_SYMBOLS.tar.bz2 -C ~/.debug
+		tar xvf $TARGET && tar xvf $PERF_SYMBOLS.tar.bz2 -C $PERF_BUILDID_DIR
 
 	else																# perf tar generated by perf archive (contains only debug symbols)
-		tar xvf $TARGET -C ~/.debug
+		tar xvf $TARGET -C $PERF_BUILDID_DIR
 	fi
 	exit 0
 fi
 
-#
-# PERF_BUILDID_DIR environment variable set by perf
-# path to buildid directory, default to $HOME/.debug
-#
-if [ -z $PERF_BUILDID_DIR ]; then
-	PERF_BUILDID_DIR=~/.debug/
-else
-        # append / to make substitutions work
-        PERF_BUILDID_DIR=$PERF_BUILDID_DIR/
-fi
-
 BUILDIDS=$(mktemp /tmp/perf-archive-buildids.XXXXXX)
 
 perf buildid-list -i $PERF_DATA --with-hits | grep -v "^ " > $BUILDIDS
Signature · 1 line
2.25.1
maintainer
#2 lore raw

Hello,

Quoted 5 lines · Haoze Xie

In perf-archive.sh, the code segment that defines 'PERF_BUILDID_DIR' is advanced before 'unpack' operation for subsequent use, followed by a 'mkdir' operation to ensure that the dir exists. Symbols in 'unpack' will be extracted to correct dir given by perf.

Quoted 7 lines

When '--unpack' param is appointed, the symbols are extracted to '~/.debug' folder by default, without using 'PERF_BUILDID_DIR' given by perf. This will cause perf to be unable to find the correct buildid's path when users configured buildid.dir in 'perf config' or used '--buildid-dir' cli param, since perf will read these params and put them in 'PERF_BUILDID_DIR' env. 'perf script' and 'perf report' will use the env as the basis for buildid indexing.

Can you please add an example command line and the output for the error case? It'd be helpful to understand the problem more intuitively.

Quoted 33 lines
Fixes e43c64c971e4 ("perf archive: Add new option '--unpack' to expand tarballs")
---
 tools/perf/perf-archive.sh | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)
tools/perf/perf-archive.sh +13 -0
index 6ed7e52ab881..f29bbc129056 100755
@@ -23,6 +23,19 @@ while [ $# -gt 0 ] ; do
 	fi
 done
 
+#
+# PERF_BUILDID_DIR environment variable set by perf
+# path to buildid directory, default to $HOME/.debug
+#
+if [ -z $PERF_BUILDID_DIR ]; then
+	PERF_BUILDID_DIR=~/.debug/
+else
+	# append / to make substitutions work
+	PERF_BUILDID_DIR=$PERF_BUILDID_DIR/
+fi
+
+mkdir -p $PERF_BUILDID_DIR
+
 if [ $UNPACK -eq 1 ]; then
 	if [ ! -z "$UNPACK_TAR" ]; then					# tar given as an argument
 		if [ ! -e "$UNPACK_TAR" ]; then

tools/perf/perf-archive.sh if [ $UNPACK -eq 1 ]; then · line 83

Quoted 7 lines
tools/perf/perf-archive.sh +1 -1
@@ -65,25 +78,14 @@ if [ $UNPACK -eq 1 ]; then
 		fi
 
 		# unzip the perf.data file in the current working directory	and debug symbols in ~/.debug directory
-		tar xvf $TARGET && tar xvf $PERF_SYMBOLS.tar.bz2 -C ~/.debug
+		tar xvf $TARGET && tar xvf $PERF_SYMBOLS.tar.bz2 -C $PERF_BUILDID_DIR
 
 	else																# perf tar generated by perf archive (contains only debug symbols)

Off-topic. I'm surprised by the comment placement.
It'd be nice if you (or someone else) can update the whole file and remove the unnecessary whitespaces altogether.

Thanks,
Namhyung

Quoted 23 lines
tools/perf/perf-archive.sh +1 -12
-		tar xvf $TARGET -C ~/.debug
+		tar xvf $TARGET -C $PERF_BUILDID_DIR
 	fi
 	exit 0
 fi
 
-#
-# PERF_BUILDID_DIR environment variable set by perf
-# path to buildid directory, default to $HOME/.debug
-#
-if [ -z $PERF_BUILDID_DIR ]; then
-	PERF_BUILDID_DIR=~/.debug/
-else
-        # append / to make substitutions work
-        PERF_BUILDID_DIR=$PERF_BUILDID_DIR/
-fi
-
 BUILDIDS=$(mktemp /tmp/perf-archive-buildids.XXXXXX)
 
 perf buildid-list -i $PERF_DATA --with-hits | grep -v "^ " > $BUILDIDS
Signature · 1 line
2.25.1
author
#3 lore raw
Quoted 87 lines · Namhyung Kim

Hello,

Quoted 5 lines · Haoze Xie

In perf-archive.sh, the code segment that defines 'PERF_BUILDID_DIR' is advanced before 'unpack' operation for subsequent use, followed by a 'mkdir' operation to ensure that the dir exists. Symbols in 'unpack' will be extracted to correct dir given by perf.

Quoted 7 lines

When '--unpack' param is appointed, the symbols are extracted to '~/.debug' folder by default, without using 'PERF_BUILDID_DIR' given by perf. This will cause perf to be unable to find the correct buildid's path when users configured buildid.dir in 'perf config' or used '--buildid-dir' cli param, since perf will read these params and put them in 'PERF_BUILDID_DIR' env. 'perf script' and 'perf report' will use the env as the basis for buildid indexing.

Can you please add an example command line and the output for the error case? It'd be helpful to understand the problem more intuitively.

Quoted 33 lines
Fixes e43c64c971e4 ("perf archive: Add new option '--unpack' to expand tarballs")
---
 tools/perf/perf-archive.sh | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)
tools/perf/perf-archive.sh +13 -0
index 6ed7e52ab881..f29bbc129056 100755
@@ -23,6 +23,19 @@ while [ $# -gt 0 ] ; do
 	fi
 done
 
+#
+# PERF_BUILDID_DIR environment variable set by perf
+# path to buildid directory, default to $HOME/.debug
+#
+if [ -z $PERF_BUILDID_DIR ]; then
+	PERF_BUILDID_DIR=~/.debug/
+else
+	# append / to make substitutions work
+	PERF_BUILDID_DIR=$PERF_BUILDID_DIR/
+fi
+
+mkdir -p $PERF_BUILDID_DIR
+
 if [ $UNPACK -eq 1 ]; then
 	if [ ! -z "$UNPACK_TAR" ]; then					# tar given as an argument
 		if [ ! -e "$UNPACK_TAR" ]; then

tools/perf/perf-archive.sh if [ $UNPACK -eq 1 ]; then · line 83

Quoted 7 lines
tools/perf/perf-archive.sh +1 -1
@@ -65,25 +78,14 @@ if [ $UNPACK -eq 1 ]; then
 		fi
 
 		# unzip the perf.data file in the current working directory	and debug symbols in ~/.debug directory
-		tar xvf $TARGET && tar xvf $PERF_SYMBOLS.tar.bz2 -C ~/.debug
+		tar xvf $TARGET && tar xvf $PERF_SYMBOLS.tar.bz2 -C $PERF_BUILDID_DIR
 
 	else																# perf tar generated by perf archive (contains only debug symbols)

Off-topic. I'm surprised by the comment placement.
It'd be nice if you (or someone else) can update the whole file and remove the unnecessary whitespaces altogether.

Thanks,
Namhyung

Quoted 19 lines
tools/perf/perf-archive.sh +1 -12
-		tar xvf $TARGET -C ~/.debug
+		tar xvf $TARGET -C $PERF_BUILDID_DIR
 	fi
 	exit 0
 fi
 
-#
-# PERF_BUILDID_DIR environment variable set by perf
-# path to buildid directory, default to $HOME/.debug
-#
-if [ -z $PERF_BUILDID_DIR ]; then
-	PERF_BUILDID_DIR=~/.debug/
-else
-        # append / to make substitutions work
-        PERF_BUILDID_DIR=$PERF_BUILDID_DIR/
-fi
-
 BUILDIDS=$(mktemp /tmp/perf-archive-buildids.XXXXXX)
 
Quoted 4 lines
Quoted 4 lines
tools/perf/perf-archive.sh +0 -0
 perf buildid-list -i $PERF_DATA --with-hits | grep -v "^ " > $BUILDIDS
Signature · 1 line
2.25.1

I have two machines A and B, I use 'perf record' and 'perf archive' to get a buildid archive 'perf.data.tar.bz2' on A:

$ perf record -a -g -- sleep 1
$ perf archive

then I transfer 'perf.data' and its buildid archive to B. On machine B, I use 'perf archive' to extract the buildid:

$ perf archive --unpack
$   Found target file for unpacking: ./perf.data.tar.bz2
$   .build-id/d1/a727ab038085dbbb50e74c812e5a6e8502e8c7
$   ...

But when I use 'perf script' to view hotspots, no function names are shown:

$ perf script
$   No kallsyms or vmlinux with build-id 
$   251c1248b97a17df394058a189dffe381169ddcd was found
$   perf    1770 [000] 1022235.467607:          1 cycles:P: 
$           ffffffff8ae9ceb6 [unknown] ([kernel.kallsyms])
$           ffffffff8ae15af5 [unknown] ([kernel.kallsyms])
$           ffffffff8ae0f83b [unknown] ([kernel.kallsyms])
$   ......

The problem is, I used 'perf config' to change buildid search path before:

$ perf config --user buildid.dir=/usr/local/symbols
$ cat ~/.perfconfig
$   # this file is auto-generated.
$   [buildid]
$           dir = /usr/local/symbols

But 'perf archive --unpack' just extract them to '~/.debug', which makes perf can't found right path to search for buildid info. If I add 'buildid-dir' param to replace buildid path defined in perfconfig, problem can be solved temporarily:

$ ./perf --buildid-dir ~/.debug script
$   perf    1770 [000] 1022235.467607:          1 cycles:P: 
$           ffffffff8ae9ceb6 native_write_msr+0x6 ([kernel.kallsyms])
$           ffffffff8ae15af5 intel_pmu_enable_all+0x15 ([kernel.kallsyms])
$           ffffffff8ae0f83b x86_pmu_enable+0x1ab ([kernel.kallsyms])

The code that determines the buildid path is in the 'util/config.c:set_buildid_dir' function, which will generate 'PERF_BUILDID_DIR' var.

#4 lore raw
Quoted 21 lines · royenheart
Quoted 20 lines · Namhyung Kim

Hello,

Quoted 5 lines · Haoze Xie

In perf-archive.sh, the code segment that defines 'PERF_BUILDID_DIR' is advanced before 'unpack' operation for subsequent use, followed by a 'mkdir' operation to ensure that the dir exists. Symbols in 'unpack' will be extracted to correct dir given by perf.

Quoted 7 lines

When '--unpack' param is appointed, the symbols are extracted to '~/.debug' folder by default, without using 'PERF_BUILDID_DIR' given by perf. This will cause perf to be unable to find the correct buildid's path when users configured buildid.dir in 'perf config' or used '--buildid-dir' cli param, since perf will read these params and put them in 'PERF_BUILDID_DIR' env. 'perf script' and 'perf report' will use the env as the basis for buildid indexing.

Can you please add an example command line and the output for the error case? It'd be helpful to understand the problem more intuitively.

Quoted 1 line

[...]

Quoted 26 lines

I have two machines A and B, I use 'perf record' and 'perf archive' to get a buildid archive 'perf.data.tar.bz2' on A:

$ perf record -a -g -- sleep 1
$ perf archive

then I transfer 'perf.data' and its buildid archive to B. On machine B, I use 'perf archive' to extract the buildid:

$ perf archive --unpack
$   Found target file for unpacking: ./perf.data.tar.bz2
$   .build-id/d1/a727ab038085dbbb50e74c812e5a6e8502e8c7
$   ...

But when I use 'perf script' to view hotspots, no function names are shown:

$ perf script
$   No kallsyms or vmlinux with build-id 
$   251c1248b97a17df394058a189dffe381169ddcd was found
$   perf    1770 [000] 1022235.467607:          1 cycles:P: 
$           ffffffff8ae9ceb6 [unknown] ([kernel.kallsyms])
$           ffffffff8ae15af5 [unknown] ([kernel.kallsyms])
$           ffffffff8ae0f83b [unknown] ([kernel.kallsyms])
$   ......
Quoted 8 lines

The problem is, I used 'perf config' to change buildid search path before:

$ perf config --user buildid.dir=/usr/local/symbols
$ cat ~/.perfconfig
$   # this file is auto-generated.
$   [buildid]
$           dir = /usr/local/symbols

Shouldn't then perf-archive just determine the directory from perf-config without any additional env var?

Quoted 11 lines

But 'perf archive --unpack' just extract them to '~/.debug', which makes perf can't found right path to search for buildid info. If I add 'buildid-dir' param to replace buildid path defined in perfconfig, problem can be solved temporarily:

$ ./perf --buildid-dir ~/.debug script
$   perf    1770 [000] 1022235.467607:          1 cycles:P: 
$           ffffffff8ae9ceb6 native_write_msr+0x6 ([kernel.kallsyms])
$           ffffffff8ae15af5 intel_pmu_enable_all+0x15 ([kernel.kallsyms])
$           ffffffff8ae0f83b x86_pmu_enable+0x1ab ([kernel.kallsyms])
Quoted 3 lines

The code that determines the buildid path is in the 'util/config.c:set_buildid_dir' function, which will generate 'PERF_BUILDID_DIR' var.

As above, the var seems to be a duplicate to what is already stored in perf-config...

Quoted 2 lines

Regards,
Michael

author
#5 lore raw
Quoted 78 lines · Michael Petlan
Quoted 21 lines · royenheart
Quoted 20 lines · Namhyung Kim

Hello,

Quoted 5 lines · Haoze Xie

In perf-archive.sh, the code segment that defines 'PERF_BUILDID_DIR' is advanced before 'unpack' operation for subsequent use, followed by a 'mkdir' operation to ensure that the dir exists. Symbols in 'unpack' will be extracted to correct dir given by perf.

Quoted 7 lines

When '--unpack' param is appointed, the symbols are extracted to '~/.debug' folder by default, without using 'PERF_BUILDID_DIR' given by perf. This will cause perf to be unable to find the correct buildid's path when users configured buildid.dir in 'perf config' or used '--buildid-dir' cli param, since perf will read these params and put them in 'PERF_BUILDID_DIR' env. 'perf script' and 'perf report' will use the env as the basis for buildid indexing.

Can you please add an example command line and the output for the error case? It'd be helpful to understand the problem more intuitively.

Quoted 1 line

[...]

Quoted 26 lines

I have two machines A and B, I use 'perf record' and 'perf archive' to get a buildid archive 'perf.data.tar.bz2' on A:

$ perf record -a -g -- sleep 1
$ perf archive

then I transfer 'perf.data' and its buildid archive to B. On machine B, I use 'perf archive' to extract the buildid:

$ perf archive --unpack
$   Found target file for unpacking: ./perf.data.tar.bz2
$   .build-id/d1/a727ab038085dbbb50e74c812e5a6e8502e8c7
$   ...

But when I use 'perf script' to view hotspots, no function names are shown:

$ perf script
$   No kallsyms or vmlinux with build-id 
$   251c1248b97a17df394058a189dffe381169ddcd was found
$   perf    1770 [000] 1022235.467607:          1 cycles:P: 
$           ffffffff8ae9ceb6 [unknown] ([kernel.kallsyms])
$           ffffffff8ae15af5 [unknown] ([kernel.kallsyms])
$           ffffffff8ae0f83b [unknown] ([kernel.kallsyms])
$   ......
Quoted 8 lines

The problem is, I used 'perf config' to change buildid search path before:

$ perf config --user buildid.dir=/usr/local/symbols
$ cat ~/.perfconfig
$   # this file is auto-generated.
$   [buildid]
$           dir = /usr/local/symbols

Shouldn't then perf-archive just determine the directory from perf-config without any additional env var?

Quoted 11 lines

But 'perf archive --unpack' just extract them to '~/.debug', which makes perf can't found right path to search for buildid info. If I add 'buildid-dir' param to replace buildid path defined in perfconfig, problem can be solved temporarily:

$ ./perf --buildid-dir ~/.debug script
$   perf    1770 [000] 1022235.467607:          1 cycles:P: 
$           ffffffff8ae9ceb6 native_write_msr+0x6 ([kernel.kallsyms])
$           ffffffff8ae15af5 intel_pmu_enable_all+0x15 ([kernel.kallsyms])
$           ffffffff8ae0f83b x86_pmu_enable+0x1ab ([kernel.kallsyms])
Quoted 3 lines

The code that determines the buildid path is in the 'util/config.c:set_buildid_dir' function, which will generate 'PERF_BUILDID_DIR' var.

Quoted 9 lines

As above, the var seems to be a duplicate to what is already stored in perf-config...

Quoted 2 lines

Regards,
Michael

Sorry for the delay, I read about the codes, in 'perf.c:main', it will call 'perf_config' function at first, which will call 'config:set_buildid_dir' to set an env 'PERF_BUILDID_DIR' for external commands(including perf-archive). Codes are:

$ // perf.c
$ int main(int argc, const char **argv) {
$   // ...
$   err = perf_config(perf_default_config, NULL);
$   // ...
$ }
$
$ ...
$ // config.c
$ void set_buildid_dir(const char *dir) {
$   if (dir)
$	    scnprintf(buildid_dir, MAXPATHLEN, "%s", dir);
$
$	/* default to $HOME/.debug */
$	if (buildid_dir[0] == '\0') {
$		char *home = getenv("HOME");
$
$		if (home) {
$			snprintf(buildid_dir, MAXPATHLEN, "%s/%s",
$				 home, DEBUG_CACHE_DIR);
$		} else {
$			strncpy(buildid_dir, DEBUG_CACHE_DIR, MAXPATHLEN-1);
$		}
$		buildid_dir[MAXPATHLEN-1] = '\0';
$	}
$	/* for communicating with external commands */
$	setenv("PERF_BUILDID_DIR", buildid_dir, 1);
$ }

Or it will set to '${HOME}/.debug'('.debug' if ${HOME} not set). '--buildid_dir' param will override it. So perf has already read the config for us and set the 'PERF_BUILDID_DIR' env intentionally at first. I think it's ok to use the env for indicating where to extract.