Skip to contentSkip to conversation

Address some issues related to Python version

PATCH In review Mauro Carvalho Chehab 3 participants 10 messages Jan 29, 2025 – Feb 11, 2025

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

cover Address some issues related to Python version

This series remove compatibility with Python 2.x from scripts that have some backward compatibility logic on it. The rationale is that, since commit 627395716cc3 ("docs: document python version used for compilation"), the minimal Python version was set to 3.x. Also, Python 2.x is EOL since Jan, 2020.

Patch 1: fix a script that was compatible only with Python 2.x; Patches 2-4: remove backward-compat code;
Patches 5-6 solves forward-compat with modern Python which warns about using
raw strings without using "r" format.

Mauro Carvalho Chehab (6):
docs: trace: decode_msr.py: make it compatible with python 3
tools: perf: exported-sql-viewer: drop support for Python 2
tools: perf: tools: perf: exported-sql-viewer: drop support for Python

    2

tools: perf: task-analyzer: drop support for Python 2
tools: selftests/bpf: test_bpftool_synctypes: escape raw symbols
comedi: convert_csv_to_c.py: use r-string for a regex expression

 Documentation/trace/postprocess/decode_msr.py |  2 +-
 .../ni_routing/tools/convert_csv_to_c.py      |  2 +-
 .../scripts/python/exported-sql-viewer.py     |  5 ++--
 tools/perf/scripts/python/task-analyzer.py    | 23 ++++----------
 tools/perf/tests/shell/lib/attr.py            |  6 +---
 .../selftests/bpf/test_bpftool_synctypes.py   | 30 +++++++++----------
 6 files changed, 25 insertions(+), 43 deletions(-)
Signature · 1 line
2.48.1
#1 lore raw

PATCH 1/6 docs: trace: decode_msr.py: make it compatible with python 3

This script uses print <foo> instead of print(foo), which is incompatible with Python 3.

Fix it.

---
 Documentation/trace/postprocess/decode_msr.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/trace/postprocess/decode_msr.py +1 -1
index aa9cc7abd5c2..f5609b16f589 100644
@@ -32,6 +32,6 @@ for j in sys.stdin:
 					break
 		if r:
 			j = j.replace(" " + m.group(2), " " + r + "(" + m.group(2) + ")")
-	print j,
+	print(j)
 
 
Signature · 1 line
2.48.1
#2 lore raw

PATCH 2/6 tools: perf: exported-sql-viewer: drop support for Python 2

In reply to Mauro Carvalho Chehab

As stated at process/changes.rsy doc, the current minimal Python version is 3.x, so drop support for EOL python 2.x.

---
 tools/perf/tests/shell/lib/attr.py | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
tools/perf/tests/shell/lib/attr.py +1 -5
index 3db9a7d78715..3ca405f45d10 100644
@@ -12,11 +12,7 @@ import logging
 import re
 import shutil
 import subprocess
-
-try:
-    import configparser
-except ImportError:
-    import ConfigParser as configparser
+import configparser
 
 def data_equal(a, b):
     # Allow multiple values in assignment separated by '|'
Signature · 1 line
2.48.1
#3 lore raw

PATCH 3/6 tools: perf: tools: perf: exported-sql-viewer: drop support for Python 2

In reply to Mauro Carvalho Chehab

As stated at process/changes.rsy doc, the current minimal Python version is 3.x, so drop support for EOL python 2.x.

---
 tools/perf/scripts/python/exported-sql-viewer.py | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +2 -3
index 121cf61ba1b3..b096592fd35f 100755
@@ -3939,9 +3939,8 @@ def CopyTreeCellsToClipboard(view, as_csv=False, with_hdr=False):
 	indent_str = " " * indent_sz
 
 	expanded_mark_sz = 2
-	if sys.version_info[0] == 3:
-		expanded_mark = "\u25BC "
-		not_expanded_mark = "\u25B6 "
+    expanded_mark = "\u25BC "
+    not_expanded_mark = "\u25B6 "
 	else:
 		expanded_mark = unicode(chr(0xE2) + chr(0x96) + chr(0xBC) + " ", "utf-8")
 		not_expanded_mark =  unicode(chr(0xE2) + chr(0x96) + chr(0xB6) + " ", "utf-8")
Signature · 1 line
2.48.1
reviewer
#4 lore raw

Re: [PATCH 3/6] tools: perf: tools: perf: exported-sql-viewer: drop support for Python 2

Quoted 13 lines · Mauro Carvalho Chehab

As stated at process/changes.rsy doc, the current minimal Python version is 3.x, so drop support for EOL python 2.x.

---
 tools/perf/scripts/python/exported-sql-viewer.py | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +0 -0
index 121cf61ba1b3..b096592fd35f 100755

tools/perf/scripts/python/exported-sql-viewer.py def CopyTreeCellsToClipboard(view, as_csv=False, with_hdr=False): · line 3943

Quoted 8 lines
tools/perf/scripts/python/exported-sql-viewer.py +2 -3
@@ -3939,9 +3939,8 @@ def CopyTreeCellsToClipboard(view, as_csv=False, with_hdr=False):
 	indent_str = " " * indent_sz
 
 	expanded_mark_sz = 2
-	if sys.version_info[0] == 3:
-		expanded_mark = "\u25BC "
-		not_expanded_mark = "\u25B6 "
+    expanded_mark = "\u25BC "
+    not_expanded_mark = "\u25B6 "

You removed the 'if' but left the 'else'.

The white is messed up.

So presumably not tested at all:

python3 tools/perf/scripts/python/exported-sql-viewer.py

    File "/home/user/git/review2/tools/perf/scripts/python/exported-sql-viewer.py", line 3942
      expanded_mark = "\u25BC "
                             ^

IndentationError: unindent does not match any outer indentation level

Posting untested patches, especially tidy-ups,
should really be discouraged.

There are many other obvious python2 things in this file that have not been addressed. When asked, AI listed 10 things.

Quoted 3 lines
tools/perf/scripts/python/exported-sql-viewer.py +0 -0
 	else:
 		expanded_mark = unicode(chr(0xE2) + chr(0x96) + chr(0xBC) + " ", "utf-8")
 		not_expanded_mark =  unicode(chr(0xE2) + chr(0x96) + chr(0xB6) + " ", "utf-8")
#5 lore raw

Re: [PATCH 3/6] tools: perf: tools: perf: exported-sql-viewer: drop support for Python 2

Em Thu, 30 Jan 2025 09:00:26 +0200
Adrian Hunter <[email protected]> escreveu:

Quoted 33 lines · Mauro Carvalho Chehab
Quoted 13 lines

As stated at process/changes.rsy doc, the current minimal Python version is 3.x, so drop support for EOL python 2.x.

---
 tools/perf/scripts/python/exported-sql-viewer.py | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
tools/perf/scripts/python/exported-sql-viewer.py +0 -0
index 121cf61ba1b3..b096592fd35f 100755

tools/perf/scripts/python/exported-sql-viewer.py def CopyTreeCellsToClipboard(view, as_csv=False, with_hdr=False): · line 3943

Quoted 8 lines
tools/perf/scripts/python/exported-sql-viewer.py +2 -3
@@ -3939,9 +3939,8 @@ def CopyTreeCellsToClipboard(view, as_csv=False, with_hdr=False):
 	indent_str = " " * indent_sz
 
 	expanded_mark_sz = 2
-	if sys.version_info[0] == 3:
-		expanded_mark = "\u25BC "
-		not_expanded_mark = "\u25B6 "
+    expanded_mark = "\u25BC "
+    not_expanded_mark = "\u25B6 "  

You removed the 'if' but left the 'else'.

The white is messed up.

So presumably not tested at all:

python3 tools/perf/scripts/python/exported-sql-viewer.py

    File "/home/user/git/review2/tools/perf/scripts/python/exported-sql-viewer.py", line 3942
      expanded_mark = "\u25BC "
                             ^
Quoted 8 lines

IndentationError: unindent does not match any outer indentation level

Posting untested patches, especially tidy-ups,
should really be discouraged.

There are many other obvious python2 things in this file that have not been addressed. When asked, AI listed 10 things.

You're right. Btw my main goal was just to make vermin to detect the minimal version of this script.

I tried to run this script on Fedora 41, installing PySide with pip there (*):

	$ pip install --user PySide2
	...
	$ perf record  ls
	...
	$ perf script -s tools/perf/scripts/python/export-to-sqlite.py pt_example branches calls
	Segmentation fault (core dumped)

but it gives me segmentation fault on a required library:

	Program terminated with signal SIGSEGV, Segmentation fault.
	#0  0x00007fcab2e0d1ec in SbkObject_TypeF () from ~/.local/lib/python3.13/site-packages/shiboken2/libshiboken2.abi3.so.5.13

(*) Fedora 41 doesn't have pyside2 packages anymore.

Thanks,
Mauro

#6 lore raw

PATCH 4/6 tools: perf: task-analyzer: drop support for Python 2

In reply to Mauro Carvalho Chehab

As stated at process/changes.rsy doc, the current minimal Python version is 3.x, so drop support for EOL python 2.x.

---
 tools/perf/scripts/python/task-analyzer.py | 23 +++++-----------------
 1 file changed, 5 insertions(+), 18 deletions(-)
tools/perf/scripts/python/task-analyzer.py +5 -18
index 3f1df9894246..9dc18875630c 100755
@@ -50,25 +50,12 @@ LEN_IN_IN = len("99999999999.999")  # 15
 LEN_IN_OUT = len("99999999999.999")  # 15
 
 
-# py2/py3 compatibility layer, see PEP469
-try:
-    dict.iteritems
-except AttributeError:
-    # py3
-    def itervalues(d):
-        return iter(d.values())
-
-    def iteritems(d):
-        return iter(d.items())
-
-else:
-    # py2
-    def itervalues(d):
-        return d.itervalues()
-
-    def iteritems(d):
-        return d.iteritems()
+# py3
+def itervalues(d):
+    return iter(d.values())
 
+def iteritems(d):
+    return iter(d.items())
 
 def _check_color():
     global _COLORS
Signature · 1 line
2.48.1
#7 lore raw

PATCH 5/6 tools: selftests/bpf: test_bpftool_synctypes: escape raw symbols

In reply to Mauro Carvalho Chehab

Modern Python versions complain about usage of "\" inside normal strings, as they should use r-string notation.

Change the annotations there to avoid such warnings:

	tools/testing/selftests/bpf/test_bpftool_synctypes.py:319: SyntaxWarning: invalid escape sequence '\w' pattern = re.compile('([\w-]+) ?(?:\||}[ }\]"])')
---
 .../selftests/bpf/test_bpftool_synctypes.py   | 30 +++++++++----------
 1 file changed, 15 insertions(+), 15 deletions(-)
tools/testing/selftests/bpf/test_bpftool_synctypes.py +15 -15
index 0ed67b6b31dd..81f286991012 100755
@@ -66,7 +66,7 @@ class ArrayParser(BlockParser):
 
     def __init__(self, reader, array_name):
         self.array_name = array_name
-        self.start_marker = re.compile(f'(static )?const bool {self.array_name}\[.*\] = {{\n')
+        self.start_marker = re.compile(fr'(static )?const bool {self.array_name}\[.*\] = {{\n')
         super().__init__(reader)
 
     def search_block(self):
@@ -80,7 +80,7 @@ class ArrayParser(BlockParser):
         Parse a block and return data as a dictionary. Items to extract must be
         on separate lines in the file.
         """
-        pattern = re.compile('\[(BPF_\w*)\]\s*= (true|false),?$')
+        pattern = re.compile(r'\[(BPF_\w*)\]\s*= (true|false),?$')
         entries = set()
         while True:
             line = self.reader.readline()
@@ -177,9 +177,9 @@ class FileExtractor(object):
 
         @enum_name: name of the enum to parse
         """
-        start_marker = re.compile(f'enum {enum_name} {{\n')
-        pattern = re.compile('^\s*(BPF_\w+),?(\s+/\*.*\*/)?$')
-        end_marker = re.compile('^};')
+        start_marker = re.compile(fr'enum {enum_name} {{\n')
+        pattern = re.compile(r'^\s*(BPF_\w+),?(\s+/\*.*\*/)?$')
+        end_marker = re.compile(r'^};')
         parser = BlockParser(self.reader)
         parser.search_block(start_marker)
         return parser.parse(pattern, end_marker)
@@ -226,8 +226,8 @@ class FileExtractor(object):
 
         @block_name: name of the blog to parse, 'TYPE' in the example
         """
-        start_marker = re.compile(f'\*{block_name}\* := {{')
-        pattern = re.compile('\*\*([\w/-]+)\*\*')
+        start_marker = re.compile(fr'\*{block_name}\* := {{')
+        pattern = re.compile(r'\*\*([\w/-]+)\*\*')
         end_marker = re.compile('}\n')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -245,8 +245,8 @@ class FileExtractor(object):
 
         @block_name: name of the blog to parse, 'TYPE' in the example
         """
-        start_marker = re.compile(f'"\s*{block_name} := {{')
-        pattern = re.compile('([\w/]+) [|}]')
+        start_marker = re.compile(fr'"\s*{block_name} := {{')
+        pattern = re.compile(r'([\w/]+) [|}]')
         end_marker = re.compile('}')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -264,8 +264,8 @@ class FileExtractor(object):
 
         @macro: macro starting the block, 'HELP_SPEC_OPTIONS' in the example
         """
-        start_marker = re.compile(f'"\s*{macro}\s*" [|}}]')
-        pattern = re.compile('([\w-]+) ?(?:\||}[ }\]])')
+        start_marker = re.compile(fr'"\s*{macro}\s*" [|}}]')
+        pattern = re.compile(r'([\w-]+) ?(?:\||}[ }\]])')
         end_marker = re.compile('}\\\\n')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -284,7 +284,7 @@ class FileExtractor(object):
         @block_name: name of the blog to parse, 'TYPE' in the example
         """
         start_marker = re.compile(f'local {block_name}=\'')
-        pattern = re.compile('(?:.*=\')?([\w/]+)')
+        pattern = re.compile(r'(?:.*=\')?([\w/]+)')
         end_marker = re.compile('\'$')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -316,7 +316,7 @@ class MainHeaderFileExtractor(SourceFileExtractor):
             {'-p', '-d', '--pretty', '--debug', '--json', '-j'}
         """
         start_marker = re.compile(f'"OPTIONS :=')
-        pattern = re.compile('([\w-]+) ?(?:\||}[ }\]"])')
+        pattern = re.compile(r'([\w-]+) ?(?:\||}[ }\]"])')
         end_marker = re.compile('#define')
 
         parser = InlineListParser(self.reader)
@@ -338,8 +338,8 @@ class ManSubstitutionsExtractor(SourceFileExtractor):
 
             {'-p', '-d', '--pretty', '--debug', '--json', '-j'}
         """
-        start_marker = re.compile('\|COMMON_OPTIONS\| replace:: {')
-        pattern = re.compile('\*\*([\w/-]+)\*\*')
+        start_marker = re.compile(r'\|COMMON_OPTIONS\| replace:: {')
+        pattern = re.compile(r'\*\*([\w/-]+)\*\*')
         end_marker = re.compile('}$')
 
         parser = InlineListParser(self.reader)
Signature · 1 line
2.48.1
#8 lore raw

Re: [PATCH 5/6] tools: selftests/bpf: test_bpftool_synctypes: escape raw symbols

2025-01-29 18:39 UTC+0100 ~ Mauro Carvalho Chehab
<[email protected]>

Quoted 8 lines

Modern Python versions complain about usage of "\" inside normal strings, as they should use r-string notation.

Change the annotations there to avoid such warnings:

	tools/testing/selftests/bpf/test_bpftool_synctypes.py:319: SyntaxWarning: invalid escape sequence '\w' pattern = re.compile('([\w-]+) ?(?:\||}[ }\]"])')

Hi, and thanks! But please note we have a fix for this in the bpf-next tree already:

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=c5d2bac978c513e1f22273cba9c55db3778032e5

Thanks,
Quentin

#9 lore raw

PATCH 6/6 comedi: convert_csv_to_c.py: use r-string for a regex expression

In reply to Mauro Carvalho Chehab

Modern Python versions complain about usage of "\" inside normal strings, as they should use r-string notation.

Change the annotations there to avoid such warnings:

drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py:47: SyntaxWarning: invalid escape sequence '\('

---
 drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py +1 -1
index 90378fb50580..d19101fc2a94 100755
@@ -44,7 +44,7 @@ def routedict_to_structinit_single(name, D, return_name=False):
 
     lines.append('\t\t[B({})] = {{'.format(D0_sig))
     for D1_sig, value in D1:
-      if not re.match('[VIU]\([^)]*\)', value):
+      if not re.match(r'[VIU]\([^)]*\)', value):
         sys.stderr.write('Invalid register format: {}\n'.format(repr(value)))
         sys.stderr.write(
           'Register values should be formatted with V(),I(),or U()\n')
Signature · 1 line
2.48.1