perf tools: Use Python devtools for version autodetection
cover perf tools: Use Python devtools for version autodetection
As mentioned here [1], I was going to try to attempt to improve Python autodetection in the scenario where Python3 devtools are installed alongside the Python2 runtime which is a pretty common scenario on a base system after running "apt install python3-dev".
Thanks
James
[1]: https://lkml.org/lkml/2021/11/30/614
James Clark (1):
perf tools: Use Python devtools for version autodetection rather than
runtime
tools/perf/Makefile.config | 39 ++++++++++++++++++++++++++------------ 1 file changed, 27 insertions(+), 12 deletions(-)
Signature · 1 line
2.28.0
PATCH 1/1 perf tools: Use Python devtools for version autodetection rather than runtime
This fixes the issue where the build will fail if only the Python2 runtime is installed but the Python3 devtools are installed. Currently the workaround is 'make PYTHON=python3'.
Fix it by autodetecting Python based on whether python[x]-config exists rather than just python[x] because both are needed for the build. Then -config is stripped to find the Python runtime.
Testing
=======
* Auto detect links with Python3 when the v3 devtools are installed and only Python 2 runtime is installed
* Auto detect links with Python2 when both devtools are installed
* Sensible warning is printed if no Python devtools are installed
* 'make PYTHON=x' still automatically sets PYTHON_CONFIG=x-config
* 'make PYTHON=x' fails if x-config doesn't exist
* 'make PYTHON=python3' overrides Python2 devtools
* 'make PYTHON=python2' overrides Python3 devtools
* 'make PYTHON_CONFIG=x-config' works
* 'make PYTHON=x PYTHON_CONFIG=x' works
* 'make PYTHON=missing' reports an error
* 'make PYTHON_CONFIG=missing' reports an error
--- tools/perf/Makefile.config | 39 ++++++++++++++++++++++++++------------ 1 file changed, 27 insertions(+), 12 deletions(-)
tools/perf/Makefile.config +27 -12
@@ -239,18 +239,33 @@ ifdef PARSER_DEBUG endif # Try different combinations to accommodate systems that only have -# python[2][-config] in weird combinations but always preferring -# python2 and python2-config as per pep-0394. If python2 or python -# aren't found, then python3 is used. -PYTHON_AUTO := python -PYTHON_AUTO := $(if $(call get-executable,python3),python3,$(PYTHON_AUTO)) -PYTHON_AUTO := $(if $(call get-executable,python),python,$(PYTHON_AUTO)) -PYTHON_AUTO := $(if $(call get-executable,python2),python2,$(PYTHON_AUTO)) -override PYTHON := $(call get-executable-or-default,PYTHON,$(PYTHON_AUTO)) -PYTHON_AUTO_CONFIG := \ - $(if $(call get-executable,$(PYTHON)-config),$(PYTHON)-config,python-config) -override PYTHON_CONFIG := \ - $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON_AUTO_CONFIG)) +# python[2][3]-config in weird combinations in the following order of +# priority from lowest to highest: +# * python3-config +# * python-config +# * python2-config as per pep-0394. +# * $(PYTHON)-config (If PYTHON is user supplied but PYTHON_CONFIG isn't) +# +PYTHON_AUTO := python-config +PYTHON_AUTO := $(if $(call get-executable,python3-config),python3-config,$(PYTHON_AUTO)) +PYTHON_AUTO := $(if $(call get-executable,python-config),python-config,$(PYTHON_AUTO)) +PYTHON_AUTO := $(if $(call get-executable,python2-config),python2-config,$(PYTHON_AUTO)) + +# If PYTHON is defined but PYTHON_CONFIG isn't, then take $(PYTHON)-config as if it was the user +# supplied value for PYTHON_CONFIG. Because it's "user supplied", error out if it doesn't exist. +ifdef PYTHON + ifndef PYTHON_CONFIG + PYTHON_CONFIG_AUTO := $(call get-executable,$(PYTHON)-config) + PYTHON_CONFIG := $(if $(PYTHON_CONFIG_AUTO),$(PYTHON_CONFIG_AUTO),\ + $(call $(error $(PYTHON)-config not found))) + endif +endif + +# Select either auto detected python and python-config or use user supplied values if they are +# defined. get-executable-or-default fails with an error if the first argument is supplied but +# doesn't exist. +override PYTHON_CONFIG := $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON_AUTO)) +override PYTHON := $(call get-executable-or-default,PYTHON,$(subst -config,,$(PYTHON_AUTO))) grep-libs = $(filter -l%,$(1)) strip-libs = $(filter-out -l%,$(1))
Signature · 1 line
2.28.0
Re: [PATCH 1/1] perf tools: Use Python devtools for version autodetection rather than runtime
Hi James,
Quoted 11 lines · James Clark
This fixes the issue where the build will fail if only the Python2 runtime is installed but the Python3 devtools are installed. Currently the workaround is 'make PYTHON=python3'.
Fix it by autodetecting Python based on whether python[x]-config exists rather than just python[x] because both are needed for the build. Then -config is stripped to find the Python runtime.
Testing
=======
Quoted 6 lines
* Auto detect links with Python3 when the v3 devtools are installed and only Python 2 runtime is installed
* Auto detect links with Python2 when both devtools are installed
* Sensible warning is printed if no Python devtools are installed
* 'make PYTHON=x' still automatically sets PYTHON_CONFIG=x-config
* 'make PYTHON=x' fails if x-config doesn't exist
If x is a valid python but no x-config is found in the system, the build fails, instead of printing a warning like before. If we use this approach I think [1] in the Makefile is never hit and needs to be cleaned up.
Thanks,
German
Quoted 63 lines
* 'make PYTHON=python3' overrides Python2 devtools
* 'make PYTHON=python2' overrides Python3 devtools
* 'make PYTHON_CONFIG=x-config' works
* 'make PYTHON=x PYTHON_CONFIG=x' works
* 'make PYTHON=missing' reports an error
* 'make PYTHON_CONFIG=missing' reports an errorFixes 79373082fa9d ("perf python: Autodetect python3 binary")Signed-off-by James Clark <[email protected]>--- tools/perf/Makefile.config | 39 ++++++++++++++++++++++++++------------ 1 file changed, 27 insertions(+), 12 deletions(-)tools/perf/Makefile.config +27
-12@@ -239,18 +239,33 @@ ifdef PARSER_DEBUG endif # Try different combinations to accommodate systems that only have -# python[2][-config] in weird combinations but always preferring -# python2 and python2-config as per pep-0394. If python2 or python -# aren't found, then python3 is used. -PYTHON_AUTO := python -PYTHON_AUTO := $(if $(call get-executable,python3),python3,$(PYTHON_AUTO)) -PYTHON_AUTO := $(if $(call get-executable,python),python,$(PYTHON_AUTO)) -PYTHON_AUTO := $(if $(call get-executable,python2),python2,$(PYTHON_AUTO)) -override PYTHON := $(call get-executable-or-default,PYTHON,$(PYTHON_AUTO)) -PYTHON_AUTO_CONFIG := \ - $(if $(call get-executable,$(PYTHON)-config),$(PYTHON)-config,python-config) -override PYTHON_CONFIG := \ - $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON_AUTO_CONFIG)) +# python[2][3]-config in weird combinations in the following order of +# priority from lowest to highest: +# * python3-config +# * python-config +# * python2-config as per pep-0394. +# * $(PYTHON)-config (If PYTHON is user supplied but PYTHON_CONFIG isn't) +# +PYTHON_AUTO := python-config +PYTHON_AUTO := $(if $(call get-executable,python3-config),python3-config,$(PYTHON_AUTO)) +PYTHON_AUTO := $(if $(call get-executable,python-config),python-config,$(PYTHON_AUTO)) +PYTHON_AUTO := $(if $(call get-executable,python2-config),python2-config,$(PYTHON_AUTO)) + +# If PYTHON is defined but PYTHON_CONFIG isn't, then take $(PYTHON)-config as if it was the user +# supplied value for PYTHON_CONFIG. Because it's "user supplied", error out if it doesn't exist. +ifdef PYTHON + ifndef PYTHON_CONFIG + PYTHON_CONFIG_AUTO := $(call get-executable,$(PYTHON)-config) + PYTHON_CONFIG := $(if $(PYTHON_CONFIG_AUTO),$(PYTHON_CONFIG_AUTO),\ + $(call $(error $(PYTHON)-config not found))) + endif +endif + +# Select either auto detected python and python-config or use user supplied values if they are +# defined. get-executable-or-default fails with an error if the first argument is supplied but +# doesn't exist. +override PYTHON_CONFIG := $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON_AUTO)) +override PYTHON := $(call get-executable-or-default,PYTHON,$(subst -config,,$(PYTHON_AUTO))) grep-libs = $(filter -l%,$(1)) strip-libs = $(filter-out -l%,$(1))
Re: [PATCH 1/1] perf tools: Use Python devtools for version autodetection rather than runtime
Quoted 20 lines · German Gomez
Hi James,
Quoted 11 lines · James Clark
This fixes the issue where the build will fail if only the Python2 runtime is installed but the Python3 devtools are installed. Currently the workaround is 'make PYTHON=python3'.
Fix it by autodetecting Python based on whether python[x]-config exists rather than just python[x] because both are needed for the build. Then -config is stripped to find the Python runtime.
Testing
=======Quoted 6 lines
* Auto detect links with Python3 when the v3 devtools are installed and only Python 2 runtime is installed
* Auto detect links with Python2 when both devtools are installed
* Sensible warning is printed if no Python devtools are installed
* 'make PYTHON=x' still automatically sets PYTHON_CONFIG=x-config
* 'make PYTHON=x' fails if x-config doesn't exist
Quoted 3 lines
If x is a valid python but no x-config is found in the system, the build fails, instead of printing a warning like before. If we use this approach I think [1] in the
Makefileis never hit and needs to be cleaned up.
Sorry, small correction:
The line in the Makefile is never hit when PYTHON is explicitly defined (probably not a big deal).
Still I wanted to point the small change in behaviour with the build warning vs. build fail.