perf tools: Bump the minimum Python version to 3.9
Arnaldo Carvalho de Melo
maintainerPATCH v2 1/1 perf tools: Bump the minimum Python version to 3.9
perf's build only refused interpreters older than 3.6, the check added when jevents.py started using f-strings, and jevents.py carried a removesuffix() shim to stay usable on them.
The kernel itself requires Python 3.9.x since 5e25b972a22be224 ("docs:
changes: update Python minimal version"), so tools in the tree are expected to work with it, and staying compatible with older interpreters stopped paying off:
- the PEP 585 generics, list[...], used by the type annotations in jevents.py and util/setup.py are only subscriptable from Python 3.9 and, being evaluated at import or definition time, they break the build on RHEL 8's python3.6.8 with:
TypeError: 'type' object is not subscriptable
- ProcessPoolExecutor(initializer=, initargs=) requires Python 3.7, so generating pmu-events.c fails there as well.
Rather than spreading version guards and typing.List() workarounds for interpreters that cannot be supported anyway, make 3.9 the floor: ask for it once in Makefile.config and use the result in both places that run python while building, jevents generation and the python binding, as the binding is still built when NO_JEVENTS=1 skips the generation check, and drop the removesuffix() shim, since str.removesuffix() was added in 3.9.
RHEL 8 is in feature freeze, so those wanting a new perf and its python features have to vendor a newer interpreter, just as Documentation/process/changes.rst already asks for the kernel itself.
Tested with Python 3.9.22: perf builds, generating pmu-events.c and the python binding, with byte for byte identical output to the build made with the shim, while Python 3.8.20 and 3.6.15 are rejected with the "older than 3.9" error, by the jevents check in a regular build and by the binding check when NO_JEVENTS=1 makes them sail past it, where they used to die on the TypeError from util/setup.py. With NO_JEVENTS=1 and NO_LIBPYTHON=1 no interpreter runs while building, so neither check applies. vermin reports no perf build file as requiring more than 3.9, the only offender being the runtime script scripts/python/mem-phys-addr.py, which the build doesn't run.
---
What changed from v1:
- The 3.9 check now also covers the python binding, addressing sashiko-bot's [Low] finding on v1 (<[email protected]>): Makefile.config
probes the interpreter once, PYTHON_39_GOOD, and both jevents
generation and the binding check it, so NO_JEVENTS=1 no longer lets
an old python reach util/setup.py and die with the TypeError.
- With NO_JEVENTS=1 and NO_LIBPYTHON=1 no interpreter runs while
building, so neither check applies.
Command to see this delta: git diff perf-tools-next-before-122-python-binding-gate..HEAD
--- tools/perf/Makefile.config | 24 ++++++++++++++++++++---- tools/perf/pmu-events/jevents.py | 16 +++------------- 2 files changed, 23 insertions(+), 17 deletions(-)
tools/perf/Makefile.config +20 -4
@@ -834,6 +834,16 @@ define disable-python_code NO_LIBPYTHON := 1 endef +# Minimum Python version for the interpreters perf runs while building, +# the same one the kernel requires, see +# Documentation/process/changes.rst: jevents generation (pmu-events/ +# jevents.py) and the python binding (util/setup.py) both use PEP 585 +# generics, list[...] and friends, that are subscriptable only from +# Python 3.9. +ifdef PYTHON + PYTHON_39_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if sys.version_info >= (3, 9) else "0")' 2> /dev/null) +endif + PYTHON_EXTENSION_SUFFIX := '.so' ifdef NO_LIBPYTHON $(call disable-python,Python support disabled by user)
@@ -851,6 +861,14 @@ else ifneq ($(feature-libpython), 1) $(call disable-python,No 'Python.h' was found: disables Python support - please install python-devel/python-dev) else + # The binding is built even when jevents generation is turned off + # with NO_JEVENTS=1, which skips the check in that block below, so + # check the version here as well: otherwise util/setup.py dies on + # older interpreters with "TypeError: 'type' object is not + # subscriptable" instead of a Makefile error. + ifneq ($(PYTHON_39_GOOD), 1) + $(error ERROR: Python interpreter needed for the python binding too old (older than 3.9). Install a newer python or build with NO_LIBPYTHON=1.) + endif LDFLAGS += $(PYTHON_EMBED_LDFLAGS) EXTLIBS += $(PYTHON_EMBED_LIBADD) PYTHON_SETUPTOOLS_INSTALLED := $(shell $(PYTHON) -c 'import setuptools;' 2> /dev/null && echo "yes" || echo "no")
@@ -885,10 +903,8 @@ ifneq ($(NO_JEVENTS),1) ifndef PYTHON $(error ERROR: No python interpreter needed for jevents generation. Install python or build with NO_JEVENTS=1.) else - # jevents.py uses f-strings present in Python 3.6 released in Dec. 2016. - JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if(sys.version_info.major >= 3 and sys.version_info.minor >= 6) else "0")' 2> /dev/null) - ifneq ($(JEVENTS_PYTHON_GOOD), 1) - $(error ERROR: Python interpreter needed for jevents generation too old (older than 3.6). Install a newer python or build with NO_JEVENTS=1.) + ifneq ($(PYTHON_39_GOOD), 1) + $(error ERROR: Python interpreter needed for jevents generation too old (older than 3.9). Install a newer python or build with NO_JEVENTS=1.) endif endif endif
tools/perf/pmu-events/jevents.py +3 -13
@@ -65,15 +65,6 @@ _json_metric_attributes = [ _json_enum_attributes = ['aggr_mode', 'deprecated', 'event_grouping', 'perpkg', 'default_show_events'] -def removesuffix(s: str, suffix: str) -> str: - """Remove the suffix from a string - - The removesuffix function is added to str in Python 3.9. We aim for 3.6 - compatibility and so provide our own function here. - """ - return s[0:-len(suffix)] if s.endswith(suffix) else s - - def file_name_to_table_name(prefix: str, parents: Sequence[str], dirname: str) -> str: """Generate a C table name from directory names."""
@@ -214,9 +205,8 @@ class JsonEvent: """Fix formatting issue for the desc string.""" if s is None: return None - return removesuffix(removesuffix(removesuffix(s, '. '), - '. '), '.').replace('\n', '\\n').replace( - '\"', '\\"').replace('\r', '\\r') + return s.removesuffix('. ').removesuffix('. ').removesuffix('.').replace( + '\n', '\\n').replace('\"', '\\"').replace('\r', '\\r') def convert_aggr_mode(aggr_mode: Optional[str]) -> Optional[str]: """Returns the aggr_mode_class enum value associated with the JSON string."""
@@ -641,7 +631,7 @@ static const struct pmu_table_entry {_pending_metrics_tblname}[] = {{ def get_topic(topic: str) -> str: if topic.endswith('metrics.json'): return 'metrics' - return removesuffix(topic, '.json').replace('-', ' ') + return topic.removesuffix('.json').replace('-', ' ') def preprocess_one_file(parents: Sequence[str], item: os.DirEntry) -> None: if item.is_dir():
Signature · 1 line
2.53.0
[email protected]
AI reviewSashiko has reviewed this patch and found no issues. It looks great!
Quoted 14 lines
perf's build only refused interpreters older than 3.6, the check added when
jevents.pystarted using f-strings, andjevents.pycarried aremovesuffix()shim to stay usable on them.The kernel itself requires Python 3.9.x since
5e25b972a22be224("docs:
changes: update Python minimal version"), so tools in the tree are expected to work with it, and staying compatible with older interpreters stopped paying off:- the PEP 585 generics, list[...], used by the type annotations in
jevents.pyandutil/setup.pyare only subscriptable from Python 3.9 and, being evaluated at import or definition time, they break the build on RHEL 8's python3.6.8 with:
Quoted 7 lines
TypeError: 'type' object is not subscriptable
[ ... ]Suggested-by Ian Rogers <[email protected]>Cc Mauro Carvalho Chehab <[email protected]>Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/arcHnvhd56I8SrZH@x2?part=1