Skip to contentSkip to conversation

perf tools: Bump the minimum Python version to 3.9

PATCH Superseded Arnaldo Carvalho de Melo 5 participants 7 messages Sep 25 – Sep 26 perf events

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Arnaldo Carvalho de Melo

author maintainer
#0 lore raw

PATCH 1/1 perf tools: Bump the minimum Python version to 3.9

perf's build only refused interpreters older than 3.6, the check added when jevents.py started using f-strings, and jevents.py carried a removesuffix() shim to stay usable on them.

The kernel itself requires Python 3.9.x since 5e25b972a22be224 ("docs:
changes: update Python minimal version"), so tools in the tree are expected to work with it, and staying compatible with older interpreters stopped paying off:

- the PEP 585 generics, list[...], used by the type annotations in jevents.py and util/setup.py are only subscriptable from Python 3.9 and, being evaluated at import or definition time, they break the build on RHEL 8's python3.6.8 with:

TypeError: 'type' object is not subscriptable

- ProcessPoolExecutor(initializer=, initargs=) requires Python 3.7, so generating pmu-events.c fails there as well.

Rather than spreading version guards and typing.List() workarounds for interpreters that cannot be supported anyway, make 3.9 the floor: check for it in Makefile.config, where the jevents generation check already lives, and drop the removesuffix() shim, since str.removesuffix() was added in 3.9.

RHEL 8 is in feature freeze, so those wanting a new perf and its python features have to vendor a newer interpreter, just as Documentation/process/changes.rst already asks for the kernel itself.

Tested with Python 3.9.22: perf builds, generating pmu-events.c and the python binding, with byte for byte identical output to the build made with the shim, while Python 3.8.20 and 3.6.15 are rejected with the "older than 3.9" error. vermin reports no perf build file as requiring more than 3.9, the only offender being the runtime script scripts/python/mem-phys-addr.py, which the build doesn't run.

Cc Mauro Carvalho Chehab <[email protected]>
Assisted-by LLM
---
 tools/perf/Makefile.config       |  8 +++++---
 tools/perf/pmu-events/jevents.py | 16 +++-------------
 2 files changed, 8 insertions(+), 16 deletions(-)
tools/perf/Makefile.config +5 -3
index b288ae0191acf7ef..92ac7c620b3ccd49 100644
@@ -885,10 +885,12 @@ ifneq ($(NO_JEVENTS),1)
   ifndef PYTHON
     $(error ERROR: No python interpreter needed for jevents generation. Install python or build with NO_JEVENTS=1.)
   else
-    # jevents.py uses f-strings present in Python 3.6 released in Dec. 2016.
-    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if(sys.version_info.major >= 3 and sys.version_info.minor >= 6) else "0")' 2> /dev/null)
+    # Minimum Python version, the same one the kernel requires, see
+    # Documentation/process/changes.rst: jevents.py needs PEP 585 generics,
+    # list[...] and friends, that are subscriptable from Python 3.9.
+    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if sys.version_info >= (3, 9) else "0")' 2> /dev/null)
     ifneq ($(JEVENTS_PYTHON_GOOD), 1)
-      $(error ERROR: Python interpreter needed for jevents generation too old (older than 3.6). Install a newer python or build with NO_JEVENTS=1.)
+      $(error ERROR: Python interpreter needed for jevents generation too old (older than 3.9). Install a newer python or build with NO_JEVENTS=1.)
     endif
   endif
 endif
tools/perf/pmu-events/jevents.py +3 -13
index 074daae5308b110b..296bf68d10810952 100755
@@ -65,15 +65,6 @@ _json_metric_attributes = [
 _json_enum_attributes = ['aggr_mode', 'deprecated', 'event_grouping', 'perpkg',
                          'default_show_events']
 
-def removesuffix(s: str, suffix: str) -> str:
-  """Remove the suffix from a string
-
-  The removesuffix function is added to str in Python 3.9. We aim for 3.6
-  compatibility and so provide our own function here.
-  """
-  return s[0:-len(suffix)] if s.endswith(suffix) else s
-
-
 def file_name_to_table_name(prefix: str, parents: Sequence[str],
                             dirname: str) -> str:
   """Generate a C table name from directory names."""
@@ -214,9 +205,8 @@ class JsonEvent:
       """Fix formatting issue for the desc string."""
       if s is None:
         return None
-      return removesuffix(removesuffix(removesuffix(s, '.  '),
-                                       '. '), '.').replace('\n', '\\n').replace(
-                                           '\"', '\\"').replace('\r', '\\r')
+      return s.removesuffix('.  ').removesuffix('. ').removesuffix('.').replace(
+          '\n', '\\n').replace('\"', '\\"').replace('\r', '\\r')
 
     def convert_aggr_mode(aggr_mode: Optional[str]) -> Optional[str]:
       """Returns the aggr_mode_class enum value associated with the JSON string."""
@@ -641,7 +631,7 @@ static const struct pmu_table_entry {_pending_metrics_tblname}[] = {{
 def get_topic(topic: str) -> str:
   if topic.endswith('metrics.json'):
     return 'metrics'
-  return removesuffix(topic, '.json').replace('-', ' ')
+  return topic.removesuffix('.json').replace('-', ' ')
 
 def preprocess_one_file(parents: Sequence[str], item: os.DirEntry) -> None:
   if item.is_dir():
Signature · 1 line
2.55.0
#1 lore raw
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] perf: incomplete python 3.9 version check leads to build crash --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] perf: incomplete python 3.9 version check leads to build crash
--

commit 7a016e98c4330410787d9ac22be5c76fe463b5ce
Author: Arnaldo Carvalho de Melo <[email protected]>

perf tools: Bump the minimum Python version to 3.9

This commit bumps the minimum Python version required by the perf build from 3.6 to 3.9. It replaces the custom removesuffix() implementation with the built-in str.removesuffix() and updates the version check in Makefile.config.

Quoted 7 lines
tools/perf/Makefile.config +0 -0
index 0ba307e78fe1f..549a809a88ca2 100644
@@ -876,10 +876,12 @@ ifneq ($(NO_JEVENTS),1)
   ifndef PYTHON
     $(error ERROR: No python interpreter needed for jevents generation. Install python or build with NO_JEVENTS=1.)

tools/perf/Makefile.config ifneq ($(NO_JEVENTS),1) · line 884

Quoted 10 lines
tools/perf/Makefile.config +5 -3
   else
-    # jevents.py uses f-strings present in Python 3.6 released in Dec. 2016.
-    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if(sys.version_info.major >= 3 and sys.version_info.minor >= 6) else "0")' 2> /dev/null)
+    # Minimum Python version, the same one the kernel requires, see
+    # Documentation/process/changes.rst: jevents.py needs PEP 585 generics,
+    # list[...] and friends, that are subscriptable from Python 3.9.
+    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if sys.version_info >= (3, 9) else "0")' 2> /dev/null)
     ifneq ($(JEVENTS_PYTHON_GOOD), 1)
-      $(error ERROR: Python interpreter needed for jevents generation too old (older than 3.6). Install a newer python or build with NO_JEVENTS=1.)
+      $(error ERROR: Python interpreter needed for jevents generation too old (older than 3.9). Install a newer python or build with NO_JEVENTS=1.)

[Severity: Low]
If a user with an older Python version follows the suggestion to build with NO_JEVENTS=1, won't the build crash later on?

Because this version check is inside the ifneq ($(NO_JEVENTS),1) block, setting NO_JEVENTS=1 bypasses it. However, tools/perf/util/setup.py also requires Python 3.9 for the PEP 585 list[str] generic in functions like clang_has_option():

tools/perf/util/setup.py:clang_has_option() {
def clang_has_option(cc: str, cc_args: list[str], ...
}

When the build reaches setup.py to build the Python bindings (unless NO_LIBPYTHON=1 is also specified), won't it fail with a cryptic TypeError ("'type' object is not subscriptable") instead of a clean Makefile error?

Should this Python 3.9 version requirement be enforced globally rather than just for jevents generation, or should the suggestion to use NO_JEVENTS=1 be removed?

Quoted 3 lines
tools/perf/Makefile.config +0 -0
     endif
   endif
 endif
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/arb9OMEQ8HCcw-9O@x2?part=1
Reviewed-by on v1 1/1
#2 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 9 lines · On Sat, 26 Sep 2026 01:01:12 +0200 Arnaldo Carvalho de Melo <[email protected]>…

perf's build only refused interpreters older than 3.6, the check added when jevents.py started using f-strings, and jevents.py carried a removesuffix() shim to stay usable on them.

The kernel itself requires Python 3.9.x since 5e25b972a22be224 ("docs:
changes: update Python minimal version"), so tools in the tree are expected to work with it, and staying compatible with older interpreters stopped paying off:

Quoted 6 lines

- the PEP 585 generics, list[...], used by the type annotations in jevents.py and util/setup.py are only subscriptable from Python 3.9 and, being evaluated at import or definition time, they break the build on RHEL 8's python3.6.8 with:

TypeError: 'type' object is not subscriptable

If it is just for type, you probably could do something like:

from __future__ import annotations

if you have a too old python version and you use import typing (I didn't test it myself - no idea it it would make any difference).

But yeah, 3.6 is too old for my taste ;-)

Quoted 2 lines

- ProcessPoolExecutor(initializer=, initargs=) requires Python 3.7, so generating pmu-events.c fails there as well.

Double-checked with vermin:

 $ vermin -vv ./tools/perf/pmu-events/jevents.py
...
  'concurrent.futures.ProcessPoolExecutor(initargs)' requires !2, 3.7
  'concurrent.futures.ProcessPoolExecutor(initializer)' requires !2, 3.7
Quoted 17 lines

Rather than spreading version guards and typing.List() workarounds for interpreters that cannot be supported anyway, make 3.9 the floor: check for it in Makefile.config, where the jevents generation check already lives, and drop the removesuffix() shim, since str.removesuffix() was added in 3.9.

RHEL 8 is in feature freeze, so those wanting a new perf and its python features have to vendor a newer interpreter, just as Documentation/process/changes.rst already asks for the kernel itself.

Tested with Python 3.9.22: perf builds, generating pmu-events.c and the python binding, with byte for byte identical output to the build made with the shim, while Python 3.8.20 and 3.6.15 are rejected with the "older than 3.9" error. vermin reports no perf build file as requiring more than 3.9, the only offender being the runtime script scripts/python/mem-phys-addr.py, which the build doesn't run.

Quoted 4 lines
Cc Mauro Carvalho Chehab <[email protected]>
Assisted-by LLM

Patch LGTM.

Quoted 58 lines
---
 tools/perf/Makefile.config       |  8 +++++---
 tools/perf/pmu-events/jevents.py | 16 +++-------------
 2 files changed, 8 insertions(+), 16 deletions(-)
tools/perf/Makefile.config +5 -3
index b288ae0191acf7ef..92ac7c620b3ccd49 100644
@@ -885,10 +885,12 @@ ifneq ($(NO_JEVENTS),1)
   ifndef PYTHON
     $(error ERROR: No python interpreter needed for jevents generation. Install python or build with NO_JEVENTS=1.)
   else
-    # jevents.py uses f-strings present in Python 3.6 released in Dec. 2016.
-    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if(sys.version_info.major >= 3 and sys.version_info.minor >= 6) else "0")' 2> /dev/null)
+    # Minimum Python version, the same one the kernel requires, see
+    # Documentation/process/changes.rst: jevents.py needs PEP 585 generics,
+    # list[...] and friends, that are subscriptable from Python 3.9.
+    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if sys.version_info >= (3, 9) else "0")' 2> /dev/null)
     ifneq ($(JEVENTS_PYTHON_GOOD), 1)
-      $(error ERROR: Python interpreter needed for jevents generation too old (older than 3.6). Install a newer python or build with NO_JEVENTS=1.)
+      $(error ERROR: Python interpreter needed for jevents generation too old (older than 3.9). Install a newer python or build with NO_JEVENTS=1.)
     endif
   endif
 endif
tools/perf/pmu-events/jevents.py +2 -12
index 074daae5308b110b..296bf68d10810952 100755
@@ -65,15 +65,6 @@ _json_metric_attributes = [
 _json_enum_attributes = ['aggr_mode', 'deprecated', 'event_grouping', 'perpkg',
                          'default_show_events']
 
-def removesuffix(s: str, suffix: str) -> str:
-  """Remove the suffix from a string
-
-  The removesuffix function is added to str in Python 3.9. We aim for 3.6
-  compatibility and so provide our own function here.
-  """
-  return s[0:-len(suffix)] if s.endswith(suffix) else s
-
-
 def file_name_to_table_name(prefix: str, parents: Sequence[str],
                             dirname: str) -> str:
   """Generate a C table name from directory names."""
@@ -214,9 +205,8 @@ class JsonEvent:
       """Fix formatting issue for the desc string."""
       if s is None:
         return None
-      return removesuffix(removesuffix(removesuffix(s, '.  '),
-                                       '. '), '.').replace('\n', '\\n').replace(
-                                           '\"', '\\"').replace('\r', '\\r')
+      return s.removesuffix('.  ').removesuffix('. ').removesuffix('.').replace(
+          '\n', '\\n').replace('\"', '\\"').replace('\r', '\\r')
 
     def convert_aggr_mode(aggr_mode: Optional[str]) -> Optional[str]:
       """Returns the aggr_mode_class enum value associated with the JSON string."""

tools/perf/pmu-events/jevents.py static const struct pmu_table_entry {_pending_metrics_tblname}[] = {{ · line 637

Quoted 8 lines
tools/perf/pmu-events/jevents.py +1 -1
@@ -641,7 +631,7 @@ static const struct pmu_table_entry {_pending_metrics_tblname}[] = {{
 def get_topic(topic: str) -> str:
   if topic.endswith('metrics.json'):
     return 'metrics'
-  return removesuffix(topic, '.json').replace('-', ' ')
+  return topic.removesuffix('.json').replace('-', ' ')
 
 def preprocess_one_file(parents: Sequence[str], item: os.DirEntry) -> None:
   if item.is_dir():

Thanks,
Mauro

reviewer Reviewed-by on v1 1/1
#3 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 37 lines · Arnaldo Carvalho de Melo

perf's build only refused interpreters older than 3.6, the check added when jevents.py started using f-strings, and jevents.py carried a removesuffix() shim to stay usable on them.

The kernel itself requires Python 3.9.x since 5e25b972a22be224 ("docs:
changes: update Python minimal version"), so tools in the tree are expected to work with it, and staying compatible with older interpreters stopped paying off:

- the PEP 585 generics, list[...], used by the type annotations in jevents.py and util/setup.py are only subscriptable from Python 3.9 and, being evaluated at import or definition time, they break the build on RHEL 8's python3.6.8 with:

TypeError: 'type' object is not subscriptable

- ProcessPoolExecutor(initializer=, initargs=) requires Python 3.7, so generating pmu-events.c fails there as well.

Rather than spreading version guards and typing.List() workarounds for interpreters that cannot be supported anyway, make 3.9 the floor: check for it in Makefile.config, where the jevents generation check already lives, and drop the removesuffix() shim, since str.removesuffix() was added in 3.9.

RHEL 8 is in feature freeze, so those wanting a new perf and its python features have to vendor a newer interpreter, just as Documentation/process/changes.rst already asks for the kernel itself.

Tested with Python 3.9.22: perf builds, generating pmu-events.c and the python binding, with byte for byte identical output to the build made with the shim, while Python 3.8.20 and 3.6.15 are rejected with the "older than 3.9" error. vermin reports no perf build file as requiring more than 3.9, the only offender being the runtime script scripts/python/mem-phys-addr.py, which the build doesn't run.

Quoted 4 lines
Cc Mauro Carvalho Chehab <[email protected]>
Assisted-by LLM

Thanks!
Ian

Quoted 69 lines
---
 tools/perf/Makefile.config       |  8 +++++---
 tools/perf/pmu-events/jevents.py | 16 +++-------------
 2 files changed, 8 insertions(+), 16 deletions(-)
tools/perf/Makefile.config +5 -3
index b288ae0191acf7ef..92ac7c620b3ccd49 100644
@@ -885,10 +885,12 @@ ifneq ($(NO_JEVENTS),1)
   ifndef PYTHON
     $(error ERROR: No python interpreter needed for jevents generation. Install python or build with NO_JEVENTS=1.)
   else
-    # jevents.py uses f-strings present in Python 3.6 released in Dec. 2016.
-    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if(sys.version_info.major >= 3 and sys.version_info.minor >= 6) else "0")' 2> /dev/null)
+    # Minimum Python version, the same one the kernel requires, see
+    # Documentation/process/changes.rst: jevents.py needs PEP 585 generics,
+    # list[...] and friends, that are subscriptable from Python 3.9.
+    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if sys.version_info >= (3, 9) else "0")' 2> /dev/null)
     ifneq ($(JEVENTS_PYTHON_GOOD), 1)
-      $(error ERROR: Python interpreter needed for jevents generation too old (older than 3.6). Install a newer python or build with NO_JEVENTS=1.)
+      $(error ERROR: Python interpreter needed for jevents generation too old (older than 3.9). Install a newer python or build with NO_JEVENTS=1.)
     endif
   endif
 endif
tools/perf/pmu-events/jevents.py +3 -14
index 074daae5308b110b..296bf68d10810952 100755
@@ -65,15 +65,6 @@ _json_metric_attributes = [
 _json_enum_attributes = ['aggr_mode', 'deprecated', 'event_grouping', 'perpkg',
                          'default_show_events']
 
-def removesuffix(s: str, suffix: str) -> str:
-  """Remove the suffix from a string
-
-  The removesuffix function is added to str in Python 3.9. We aim for 3.6
-  compatibility and so provide our own function here.
-  """
-  return s[0:-len(suffix)] if s.endswith(suffix) else s
-
-
 def file_name_to_table_name(prefix: str, parents: Sequence[str],
                             dirname: str) -> str:
   """Generate a C table name from directory names."""
@@ -214,9 +205,8 @@ class JsonEvent:
       """Fix formatting issue for the desc string."""
       if s is None:
         return None
-      return removesuffix(removesuffix(removesuffix(s, '.  '),
-                                       '. '), '.').replace('\n', '\\n').replace(
-                                           '\"', '\\"').replace('\r', '\\r')
+      return s.removesuffix('.  ').removesuffix('. ').removesuffix('.').replace(
+          '\n', '\\n').replace('\"', '\\"').replace('\r', '\\r')
 
     def convert_aggr_mode(aggr_mode: Optional[str]) -> Optional[str]:
       """Returns the aggr_mode_class enum value associated with the JSON string."""
@@ -641,7 +631,7 @@ static const struct pmu_table_entry {_pending_metrics_tblname}[] = {{
 def get_topic(topic: str) -> str:
   if topic.endswith('metrics.json'):
     return 'metrics'
-  return removesuffix(topic, '.json').replace('-', ' ')
+  return topic.removesuffix('.json').replace('-', ' ')
 
 def preprocess_one_file(parents: Sequence[str], item: os.DirEntry) -> None:
   if item.is_dir():
--

2.55.0

#4 lore raw
Quoted 10 lines · Ian Rogers
Quoted 8 lines · Arnaldo Carvalho de Melo

perf's build only refused interpreters older than 3.6, the check added when jevents.py started using f-strings, and jevents.py carried a removesuffix() shim to stay usable on them.

Cc Mauro Carvalho Chehab <[email protected]>
Assisted-by LLM

Hi,

I addressed a sashiko comment avoir the python binding in v2, does
your reviewed-by stands for v2 as well? Mauro?

Cheers,

- Arnaldo

reviewer
#5 lore raw
Quoted 13 lines · Arnaldo Carvalho de Melo
Quoted 10 lines · Ian Rogers
Quoted 8 lines · Arnaldo Carvalho de Melo

perf's build only refused interpreters older than 3.6, the check added when jevents.py started using f-strings, and jevents.py carried a removesuffix() shim to stay usable on them.

Cc Mauro Carvalho Chehab <[email protected]>
Assisted-by LLM
Quoted 4 lines

Hi,

I addressed a sashiko comment avoir the python binding in v2, does
your reviewed-by stands for v2 as well? Mauro?

Mine, yes. Thanks,
Ian

Quoted 3 lines

Cheers,

- Arnaldo

author maintainer
#6 lore raw
Quoted 13 lines · Ian Rogers
Quoted 12 lines · Arnaldo Carvalho de Melo
Quoted 10 lines · Ian Rogers
Quoted 8 lines · Arnaldo Carvalho de Melo

perf's build only refused interpreters older than 3.6, the check added when jevents.py started using f-strings, and jevents.py carried a removesuffix() shim to stay usable on them.

Cc Mauro Carvalho Chehab <[email protected]>
Assisted-by LLM
Quoted 7 lines
Quoted 4 lines

Hi,

I addressed a sashiko comment avoir the python binding in v2, does
your reviewed-by stands for v2 as well? Mauro?

Mine, yes. Thanks,
Ian

Mauro, I'm keeping yours as well, as the change in v2 was just to disable something else that requires python 3.9 when it is not present, so more of the same.

- Arnaldo