perf test: Add an 'import perf' test shell script
Ian Rogers
reviewerPATCH 1/2 perf test: Add an 'import perf' test shell script
The 'import perf' test needs to set up a path to the python module as well as to know the python command to invoke. These are hard coded at build time to be build a directory and the python used in the build, which is less than desirable. Avoid the hard coded values by reusing the existing shell script python setup and determine a potential built python module via the path of the perf executable.
--- tools/perf/tests/shell/python-use.sh | 36 ++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100755 tools/perf/tests/shell/python-use.sh
tools/perf/tests/shell/python-use.sh new file +36 -0
@@ -0,0 +1,36 @@ +#!/bin/bash +# 'import perf' in python +# SPDX-License-Identifier: GPL-2.0 +# Just test if we can load the python binding. +set -e + +shelldir=$(dirname "$0") +# shellcheck source=lib/setup_python.sh +. "${shelldir}"/lib/setup_python.sh + +MODULE_DIR=$(dirname "$(which perf)")/python + +if [ -d "$MODULE_DIR" ] +then + CMD=$(cat <<EOF +import sys +sys.path.insert(0, '$MODULE_DIR') +import perf +print('success!') +EOF + ) +else + CMD=$(cat <<EOF +import perf +print('success!') +EOF + ) +fi + +echo -e "Testing 'import perf' with:\n$CMD" + +if ! echo "$CMD" | $PYTHON | grep -q "success!" +then + exit 1 +fi +exit 0
Signature · 1 line
2.51.0.338.gd7d06c2dae-goog
Ian Rogers
reviewerPATCH 2/2 perf test: Remove C python_use test
Removed in favor of the shell script version.
--- tools/perf/tests/Build | 2 -- tools/perf/tests/builtin-test.c | 1 - tools/perf/tests/python-use.c | 27 --------------------------- tools/perf/tests/tests.h | 1 - 4 files changed, 31 deletions(-) delete mode 100644 tools/perf/tests/python-use.c
tools/perf/tests/Build +0 -2
@@ -20,7 +20,6 @@ perf-test-y += hists_link.o perf-test-y += hists_filter.o perf-test-y += hists_output.o perf-test-y += hists_cumulate.o -perf-test-y += python-use.o perf-test-y += bp_signal.o perf-test-y += bp_signal_overflow.o perf-test-y += bp_account.o
@@ -75,7 +74,6 @@ ifeq ($(SRCARCH),$(filter $(SRCARCH),x86 arm arm64 powerpc)) perf-test-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o endif -CFLAGS_python-use.o += -DPYTHONPATH="BUILD_STR($(OUTPUT)python)" -DPYTHON="BUILD_STR($(PYTHON_WORD))" CFLAGS_dwarf-unwind.o += -fno-optimize-sibling-calls perf-test-y += workloads/
tools/perf/tests/builtin-test.c +0 -1
@@ -84,7 +84,6 @@ static struct test_suite *generic_tests[] = { &suite__syscall_openat_tp_fields, #endif &suite__hists_link, - &suite__python_use, &suite__bp_signal, &suite__bp_signal_overflow, &suite__bp_accounting,
tools/perf/tests/python-use.c deleted +0 -27
@@ -1,27 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* -* Just test if we can load the python binding. -*/ - -#include <stdio.h> -#include <stdlib.h> -#include <linux/compiler.h> -#include "tests.h" -#include "util/debug.h" - -static int test__python_use (struct test_suite *test__maybe_unused ,int subtest__maybe_unused ) -{ -char *cmd; -int ret; - -if (asprintf (&cmd,"echo \"import sys ; sys.path.insert(0, '%s'); import perf\" | %s %s" , -PYTHONPATH ,PYTHON , verbose >0 ?"" :"2> /dev/null" ) <0 ) -return -1 ; - -pr_debug ("python usage test: \"%s\"\n" , cmd); - ret =system (cmd) ?-1 :0 ; -free (cmd); -return ret; -} - -DEFINE_SUITE ("'import perf' in python" , python_use);
tools/perf/tests/tests.h +0 -1
@@ -120,7 +120,6 @@ DECLARE_SUITE(dso_data_cache);DECLARE_SUITE (dso_data_reopen);DECLARE_SUITE (parse_events);DECLARE_SUITE (hists_link); -DECLARE_SUITE (python_use);DECLARE_SUITE (bp_signal);DECLARE_SUITE (bp_signal_overflow);DECLARE_SUITE (bp_accounting);
Signature · 1 line
2.51.0.338.gd7d06c2dae-goog
Ian Rogers
reviewerQuoted 9 lines · Ian Rogers
The 'import perf' test needs to set up a path to the python module as well as to know the python command to invoke. These are hard coded at build time to be build a directory and the python used in the build, which is less than desirable. Avoid the hard coded values by reusing the existing shell script python setup and determine a potential built python module via the path of the perf executable.
Signed-off-by Ian Rogers <[email protected]>
Ping.
Thanks,
Ian
Quoted 50 lines
--- tools/perf/tests/shell/python-use.sh | 36 ++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100755 tools/perf/tests/shell/python-use.shtools/perf/tests/shell/python-use.sh new file +36
-1@@ -0,0 +1,36 @@ +#!/bin/bash +# 'import perf' in python +# SPDX-License-Identifier: GPL-2.0 +# Just test if we can load the python binding. +set -e + +shelldir=$(dirname "$0") +# shellcheck source=lib/setup_python.sh +. "${shelldir}"/lib/setup_python.sh + +MODULE_DIR=$(dirname "$(which perf)")/python + +if [ -d "$MODULE_DIR" ] +then + CMD=$(cat <<EOF +import sys +sys.path.insert(0, '$MODULE_DIR') +import perf +print('success!') +EOF + ) +else + CMD=$(cat <<EOF +import perf +print('success!') +EOF + ) +fi + +echo -e "Testing 'import perf' with:\n$CMD" + +if ! echo "$CMD" | $PYTHON | grep -q "success!" +then + exit 1 +fi +exit 0 --2.51.0.338.gd7d06c2dae-goog
Arnaldo Carvalho de Melo
maintainerQuoted 9 lines · Ian Rogers
Quoted 8 lines · Ian Rogers
The 'import perf' test needs to set up a path to the python module as well as to know the python command to invoke. These are hard coded at build time to be build a directory and the python used in the build, which is less than desirable. Avoid the hard coded values by reusing the existing shell script python setup and determine a potential built python module via the path of the perf executable.
Quoted 3 lines
Quoted 1 line
Signed-off-by Ian Rogers <[email protected]>Ping.
There is another patch in this area to make it work, right?
After applying this one I get a failure, only when I set PYTHONPATH it works:
⬢ [acme@toolbx perf-tools-next]$ export PYTHONPATH=/tmp/build/perf-tools-next/python_ext_build/lib/ ⬢ [acme@toolbx perf-tools-next]$ perf test -v pytho
Couldn't bump rlimit(MEMLOCK), failures may take place when creating BPF maps, etc
88: 'import perf' in python : Ok
⬢ [acme@toolbx perf-tools-next]$
- Arnaldo
Quoted 53 lines
Thanks,
IanQuoted 50 lines
--- tools/perf/tests/shell/python-use.sh | 36 ++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100755 tools/perf/tests/shell/python-use.shtools/perf/tests/shell/python-use.sh new file +36
-1@@ -0,0 +1,36 @@ +#!/bin/bash +# 'import perf' in python +# SPDX-License-Identifier: GPL-2.0 +# Just test if we can load the python binding. +set -e + +shelldir=$(dirname "$0") +# shellcheck source=lib/setup_python.sh +. "${shelldir}"/lib/setup_python.sh + +MODULE_DIR=$(dirname "$(which perf)")/python + +if [ -d "$MODULE_DIR" ] +then + CMD=$(cat <<EOF +import sys +sys.path.insert(0, '$MODULE_DIR') +import perf +print('success!') +EOF + ) +else + CMD=$(cat <<EOF +import perf +print('success!') +EOF + ) +fi + +echo -e "Testing 'import perf' with:\n$CMD" + +if ! echo "$CMD" | $PYTHON | grep -q "success!" +then + exit 1 +fi +exit 0 --2.51.0.338.gd7d06c2dae-goog
Arnaldo Carvalho de Melo
maintainerQuoted 16 lines · Arnaldo Carvalho de Melo
Quoted 9 lines · Ian Rogers
Quoted 8 lines · Ian Rogers
The 'import perf' test needs to set up a path to the python module as well as to know the python command to invoke. These are hard coded at build time to be build a directory and the python used in the build, which is less than desirable. Avoid the hard coded values by reusing the existing shell script python setup and determine a potential built python module via the path of the perf executable.
Quoted 3 lines
Quoted 1 line
Signed-off-by Ian Rogers <[email protected]>Ping.
There is another patch in this area to make it work, right?
Quoted 8 lines
After applying this one I get a failure, only when I set PYTHONPATH it works:
⬢ [acme@toolbx perf-tools-next]$ export PYTHONPATH=/tmp/build/perf-tools-next/python_ext_build/lib/ ⬢ [acme@toolbx perf-tools-next]$ perf test -v pytho
Couldn't bump rlimit(MEMLOCK), failures may take place when creating BPF maps, etc
88: 'import perf' in python : Ok
⬢ [acme@toolbx perf-tools-next]$
⬢ [acme@toolbx perf-tools-next]$ b4 am -ctsl --cc-trailers [email protected] Grabbing thread from lore.kernel.org/all/[email protected]/t.mbox.gz Checking for newer revisions
Grabbing search results from lore.kernel.org
Analyzing 5 messages in the thread
Analyzing 0 code-review messages
Checking attestation on all messages, may take a moment...
---
✗ [PATCH v1 1/2] perf test: Add an 'import perf' test shell script
+ Link: https://lore.kernel.org/r/[email protected]
+ Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
✗ [PATCH v1 2/2] perf test: Remove C python_use test
+ Link: https://lore.kernel.org/r/[email protected]
+ Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
---
✗ BADSIG: DKIM/google.com
---
Total patches: 2
---
Base: not specified
git am ./20250903_irogers_perf_test_add_an_import_perf_test_shell_script.mbx
⬢ [acme@toolbx perf-tools-next]$ git am ./20250903_irogers_perf_test_add_an_import_perf_test_shell_script.mbx
Applying: perf test: Add an 'import perf' test shell script
error: tools/perf/tests/shell/python-use.sh: already exists in index Patch failed at 0001 perf test: Add an 'import perf' test shell script
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false" ⬢ [acme@toolbx perf-tools-next]$
Ian Rogers
reviewerQuoted 17 lines · Arnaldo Carvalho de Melo
Quoted 9 lines · Ian Rogers
Quoted 8 lines · Ian Rogers
The 'import perf' test needs to set up a path to the python module as well as to know the python command to invoke. These are hard coded at build time to be build a directory and the python used in the build, which is less than desirable. Avoid the hard coded values by reusing the existing shell script python setup and determine a potential built python module via the path of the perf executable.
Quoted 3 lines
Quoted 1 line
Signed-off-by Ian Rogers <[email protected]>Ping.
There is another patch in this area to make it work, right?
Quoted 8 lines
After applying this one I get a failure, only when I set PYTHONPATH it works:
⬢ [acme@toolbx perf-tools-next]$ export PYTHONPATH=/tmp/build/perf-tools-next/python_ext_build/lib/ ⬢ [acme@toolbx perf-tools-next]$ perf test -v pytho
Couldn't bump rlimit(MEMLOCK), failures may take place when creating BPF maps, etc
88: 'import perf' in python : Ok
⬢ [acme@toolbx perf-tools-next]$
Hmm.. if you are running from the install directory then I guess the assumption should be that the install bin directory is in the PATH, lib in LD_LIBRARY_PATH and PYTHONPATH set up for the directory too? We could have the script look for a python module going from ..../bin to ..../lib and fix the PYTHONPATH for you. Wdyyt?
Thanks,
Ian
Quoted 57 lines
- Arnaldo
Quoted 53 lines
Thanks,
IanQuoted 50 lines
--- tools/perf/tests/shell/python-use.sh | 36 ++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100755 tools/perf/tests/shell/python-use.shtools/perf/tests/shell/python-use.sh new file +36
-1@@ -0,0 +1,36 @@ +#!/bin/bash +# 'import perf' in python +# SPDX-License-Identifier: GPL-2.0 +# Just test if we can load the python binding. +set -e + +shelldir=$(dirname "$0") +# shellcheck source=lib/setup_python.sh +. "${shelldir}"/lib/setup_python.sh + +MODULE_DIR=$(dirname "$(which perf)")/python + +if [ -d "$MODULE_DIR" ] +then + CMD=$(cat <<EOF +import sys +sys.path.insert(0, '$MODULE_DIR') +import perf +print('success!') +EOF + ) +else + CMD=$(cat <<EOF +import perf +print('success!') +EOF + ) +fi + +echo -e "Testing 'import perf' with:\n$CMD" + +if ! echo "$CMD" | $PYTHON | grep -q "success!" +then + exit 1 +fi +exit 0 --2.51.0.338.gd7d06c2dae-goog
Arnaldo Carvalho de Melo
maintainerQuoted 27 lines · Ian Rogers
Quoted 17 lines · Arnaldo Carvalho de Melo
Quoted 9 lines · Ian Rogers
Quoted 8 lines · Ian Rogers
The 'import perf' test needs to set up a path to the python module as well as to know the python command to invoke. These are hard coded at build time to be build a directory and the python used in the build, which is less than desirable. Avoid the hard coded values by reusing the existing shell script python setup and determine a potential built python module via the path of the perf executable.
Quoted 3 lines
Quoted 1 line
Signed-off-by Ian Rogers <[email protected]>Ping.
There is another patch in this area to make it work, right?
Quoted 8 lines
After applying this one I get a failure, only when I set PYTHONPATH it works:
⬢ [acme@toolbx perf-tools-next]$ export PYTHONPATH=/tmp/build/perf-tools-next/python_ext_build/lib/ ⬢ [acme@toolbx perf-tools-next]$ perf test -v pytho
Couldn't bump rlimit(MEMLOCK), failures may take place when creating BPF maps, etc
88: 'import perf' in python : Ok
⬢ [acme@toolbx perf-tools-next]$
Quoted 5 lines
Hmm.. if you are running from the install directory then I guess the assumption should be that the install bin directory is in the PATH, lib in
LD_LIBRARY_PATHand PYTHONPATH set up for the directory too? We could have the script look for a python module going from ..../bin to ..../lib and fix the PYTHONPATH for you. Wdyyt?
I reported it more in the sense that something that was working before stopped working but having not yet tried to see why it stopped working.
Another thing that I just noticed that may be related:
35 97.47 fedora:rawhide : Ok gcc (GCC) 15.2.1 20250924 (Red Hat 15.2.1-2) , clang version 21.1.2 (Fedora 21.1.2-1.fc44) flex 2.6.4
36 36.81 gentoo:stage3 : Ok gcc (Gentoo 13.3.1_p20240614 p17) 13.3.1 20240614 flex 2.6.4
37 77.37 manjaro:base : FAIL clang version 16.0.6
AR /tmp/build/perf/libpmu-events.a
LINK /tmp/build/perf/perf
GEN /tmp/build/perf/python/perf.cpython-311-x86_64-linux-gnu.so
/tmp/build/perf/python_ext_build/tmp/git/perf-6.17.0-rc6/tools/perf/util/python.o: file not recognized: file format not recognized
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
error: command '/usr/sbin/clang' failed with exit code 1
cp: cannot stat '/tmp/build/perf/python_ext_build/lib/perf*.so': No such file or directory
make[2]: *** [Makefile.perf:747: /tmp/build/perf/python/perf.cpython-311-x86_64-linux-gnu.so] Error 1
make[1]: *** [Makefile.perf:289: sub-make] Error 2
make: *** [Makefile:76: all] Error 2
make: Leaving directory '/git/perf-6.17.0-rc6/tools/perf'
+ exit 1
38 130.27 opensuse:15.4 : Ok gcc (SUSE Linux) 7.5.0 , clang version 15.0.7 flex 2.6.4
39 119.76 opensuse:15.5 : Ok gcc (SUSE Linux) 7.5.0 , clang version 15.0.7 flex 2.6.4
40 120.03 opensuse:15.6 : Ok gcc (SUSE Linux) 7.5.0 , clang version 17.0.6 flex 2.6.4o- Arnaldo
Ian Rogers
reviewerQuoted 29 lines · Arnaldo Carvalho de Melo
Quoted 27 lines · Ian Rogers
Quoted 25 lines · Arnaldo Carvalho de Melo
Quoted 9 lines · Ian Rogers
Quoted 8 lines · Ian Rogers
The 'import perf' test needs to set up a path to the python module as well as to know the python command to invoke. These are hard coded at build time to be build a directory and the python used in the build, which is less than desirable. Avoid the hard coded values by reusing the existing shell script python setup and determine a potential built python module via the path of the perf executable.
Quoted 3 lines
Quoted 1 line
Signed-off-by Ian Rogers <[email protected]>Ping.
There is another patch in this area to make it work, right?
After applying this one I get a failure, only when I set PYTHONPATH it works:
⬢ [acme@toolbx perf-tools-next]$ export PYTHONPATH=/tmp/build/perf-tools-next/python_ext_build/lib/ ⬢ [acme@toolbx perf-tools-next]$ perf test -v pytho
Couldn't bump rlimit(MEMLOCK), failures may take place when creating BPF maps, etc
88: 'import perf' in python : Ok
⬢ [acme@toolbx perf-tools-next]$
Quoted 8 lines
Quoted 5 lines
Hmm.. if you are running from the install directory then I guess the assumption should be that the install bin directory is in the PATH, lib in
LD_LIBRARY_PATHand PYTHONPATH set up for the directory too? We could have the script look for a python module going from ..../bin to ..../lib and fix the PYTHONPATH for you. Wdyyt?I reported it more in the sense that something that was working before stopped working but having not yet tried to see why it stopped working.
Not working is not good. I'm not sure what I should do for the fix. Aside from cleanup I was hoping we could get this working so that we could start testing things like ilist.py that depend on the python module.
Quoted 8 lines
Another thing that I just noticed that may be related:
35 97.47 fedora:rawhide : Ok gcc (GCC) 15.2.1 20250924 (Red Hat 15.2.1-2) , clang version 21.1.2 (Fedora 21.1.2-1.fc44) flex 2.6.4 36 36.81 gentoo:stage3 : Ok gcc (Gentoo 13.3.1_p20240614 p17) 13.3.1 20240614 flex 2.6.4 37 77.37 manjaro:base : FAIL clang version 16.0.6 AR /tmp/build/perf/libpmu-events.a LINK /tmp/build/perf/perf GEN /tmp/build/perf/python/perf.cpython-311-x86_64-linux-gnu.so
Quoted 9 lines
/tmp/build/perf/python_ext_build/tmp/git/perf-6.17.0-rc6/tools/perf/util/python.o: file not recognized: file format not recognized
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
error: command '/usr/sbin/clang' failed with exit code 1
cp: cannot stat '/tmp/build/perf/python_ext_build/lib/perf*.so': No such file or directory
make[2]: *** [Makefile.perf:747:/tmp/build/perf/python/perf.cpython-311-x86_64-linux-gnu.so] Error 1 make[1]: *** [Makefile.perf:289: sub-make] Error 2
make: *** [Makefile:76: all] Error 2
make: Leaving directory '/git/perf-6.17.0-rc6/tools/perf'
+ exit 1
Hmm.. maybe that relates to Leo's changes. He modified the clang options in setup.py which this aligns with.
Thanks,
Ian
Quoted 5 lines
38 130.27 opensuse:15.4 : Ok gcc (SUSE Linux) 7.5.0 , clang version 15.0.7 flex 2.6.4 39 119.76 opensuse:15.5 : Ok gcc (SUSE Linux) 7.5.0 , clang version 15.0.7 flex 2.6.4 40 120.03 opensuse:15.6 : Ok gcc (SUSE Linux) 7.5.0 , clang version 17.0.6 flex 2.6.4o- Arnaldo
Quoted 7 lines · Ian Rogers
The 'import perf' test needs to set up a path to the python module as well as to know the python command to invoke. These are hard coded at build time to be build a directory and the python used in the build, which is less than desirable. Avoid the hard coded values by reusing the existing shell script python setup and determine a potential built python module via the path of the perf executable.
Quoted 5 lines
Signed-off-by Ian Rogers <[email protected]>--- tools/perf/tests/shell/python-use.sh | 36 ++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100755 tools/perf/tests/shell/python-use.sh
Aside from avoiding hard coded build directories, this also looks much cleaner to me.
Reviewed-by Collin Funk <[email protected]>
Collin