Skip to contentSkip to conversation

Fix issues with Python3 scripting

PATCH In review Tony Jones 4 participants 19 messages Jan 24, 2019 – Feb 9, 2019 perf events

lore
Share & tools
lore

Tony Jones

author
#0 lore raw

cover Fix issues with Python3 scripting

Seeteena posted, earlier this week, some patches to add Python3 support to scripts/python/*.py. Unfortunately there were some issues with these patches (such as: https://lkml.org/lkml/2019/1/17/351)

Since I already had a tested set of patches in openSUSE:Factory and SLE15-SP1 and was about to submit them, Seeteena and I that agreed I should post my patches not involving scripts/python/*.py and Seeteena will later resubmit the patches for scripts/python/*.py incorporating my review feedback under a joint signed-off-by.

It should be noted that the use of "from __future__ import print_function" (see: https://lkml.org/lkml/2019/1/16/641) and "except as" (see change to:
tests/attr.py) implies Python2 >= 2.6 as the necessary support has not been backported to prior versions. I am not sure if it's worth detecting <2.6 at build time or whether it's sufficiently old as to be a non-issue?

The shebang changes were driven mostly by our build process as it scans all files within an rpm and the shebangs would result in a rpm requires on the python2 binary when BuildRequires was python3-devel. I think they make sense to apply upstream but understand totally if it's prefered we keep them local.

These changes have been tested with PYTHON=python2 (v2.7) and PYTHON=python3 (v3.6) on latest openSUSE Tumbleweed. I did notice that test #18 "'import perf' in python" is failing on my system without these changes. I'll look at it further but didn't want to hold up Seeteena's resubmit.

Tony Jones (6):
perf script python: Add trace_context extension module to sys,modules perf script python: Use PyBytes for attr in trace-event-python perf script python: remove explicit shebang from setup.py perf script python: remove explicit shebang from tests/attr.c perf script python: remove explicit shebang from Python scripts perf script python: add Python3 support to tests/attr.py

 tools/perf/Makefile.perf                      |  4 +--
 .../scripts/python/exported-sql-viewer.py     |  1 -
 tools/perf/scripts/python/sched-migration.py  |  2 --
 tools/perf/scripts/python/stat-cpi.py         |  1 -
 tools/perf/tests/attr.py                      | 33 +++++++++++--------
 .../scripting-engines/trace-event-python.c    |  9 ++---
 tools/perf/util/setup.py                      |  2 --
 7 files changed, 26 insertions(+), 26 deletions(-)
Signature · 1 line
2.20.1
author
#1 lore raw

PATCH 1/6 perf script python: Add trace_context extension module to sys,modules

In Python3, the result of PyModule_Create (called from scripts/python/Perf-Trace-Util/Context.c) is not automatically added to sys.modules. See: https://bugs.python.org/issue4592

Below is the observed behavior without the fix:

# ldd /usr/bin/perf | grep -i python
	libpython3.6m.so.1.0 => /usr/lib64/libpython3.6m.so.1.0 (0x00007f8e1dfb2000)
# perf record /bin/false
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.015 MB perf.data (17 samples) ]
# perf script -g python | cat
generated Python script: perf-script.py
# perf script -s ./perf-script.py
Traceback (most recent call last):
  File "./perf-script.py", line 18, in <module>
    from perf_trace_context import *
ModuleNotFoundError: No module named 'perf_trace_context'
Error running python script ./perf-script.py
Cc Jaroslav Škarvada <[email protected]>
Fixes 66dfdff03d196e51322c6a85c0d8db8bb2bdd655
---
 tools/perf/util/scripting-engines/trace-event-python.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
tools/perf/util/scripting-engines/trace-event-python.c +4 -2
index 87ef16a1b17e..40300d8e80a7 100644
@@ -1494,6 +1494,7 @@ static void _free_command_line(wchar_t **command_line, int num)
 static int python_start_script(const char *script, int argc, const char **argv)
 {
 	struct tables *tables = &tables_global;
+	PyMODINIT_FUNC (*initfunc)(void);
 #if PY_MAJOR_VERSION < 3
 	const char **command_line;
 #else
@@ -1504,24 +1505,25 @@ static int python_start_script(const char *script, int argc, const char **argv)
 	FILE *fp;
 
 #if PY_MAJOR_VERSION < 3
+	initfunc = initperf_trace_context;
 	command_line = malloc((argc + 1) * sizeof(const char *));
 	command_line[0] = script;
 	for (i = 1; i < argc + 1; i++)
 		command_line[i] = argv[i - 1];
 #else
+	initfunc = PyInit_perf_trace_context;
 	command_line = malloc((argc + 1) * sizeof(wchar_t *));
 	command_line[0] = Py_DecodeLocale(script, NULL);
 	for (i = 1; i < argc + 1; i++)
 		command_line[i] = Py_DecodeLocale(argv[i - 1], NULL);
 #endif
 
+	PyImport_AppendInittab("perf_trace_context", initfunc);
 	Py_Initialize();
 
 #if PY_MAJOR_VERSION < 3
-	initperf_trace_context();
 	PySys_SetArgv(argc + 1, (char **)command_line);
 #else
-	PyInit_perf_trace_context();
 	PySys_SetArgv(argc + 1, command_line);
 #endif
 
Signature · 1 line
2.20.1
#2 lore raw

[tip:perf/core] perf script python: Add trace_context extension module to sys.modules

perf script python: Add trace_context extension module to sys.modules
Commit-ID:  cc437642255224e4140fed1f3e3156fc8ad91903
Gitweb:     https://git.kernel.org/tip/cc437642255224e4140fed1f3e3156fc8ad91903
Author:     Tony Jones <[email protected]>
AuthorDate: Wed, 23 Jan 2019 16:52:24 -0800
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Fri, 25 Jan 2019 15:12:10 +0100

perf script python: Add trace_context extension module to sys.modules

In Python3, the result of PyModule_Create (called from scripts/python/Perf-Trace-Util/Context.c) is not automatically added to sys.modules. See: https://bugs.python.org/issue4592

Below is the observed behavior without the fix:

  # ldd /usr/bin/perf | grep -i python
	libpython3.6m.so.1.0 => /usr/lib64/libpython3.6m.so.1.0 (0x00007f8e1dfb2000)
  # perf record /bin/false
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.015 MB perf.data (17 samples) ]
  # perf script -g python | cat
  generated Python script: perf-script.py
  # perf script -s ./perf-script.py
  Traceback (most recent call last):
    File "./perf-script.py", line 18, in <module>
      from perf_trace_context import *
  ModuleNotFoundError: No module named 'perf_trace_context'
  Error running python script ./perf-script.py
  #

Committer notes:

To build with python3 use:

  $ make -C tools/perf PYTHON=python3

Use a non-const variable to pass the 'name' arg to PyImport_AppendInittab(), as python2.6 has that as 'char *', which ends up trowing this in some environments:

CC /tmp/build/perf/util/parse-branch-options.o
util/scripting-engines/trace-event-python.c: In function 'python_start_script':
util/scripting-engines/trace-event-python.c:1520:2: error: passing argument 1 of 'PyImport_AppendInittab' discards 'const' qualifier from pointer target type [-Werror]

    PyImport_AppendInittab("perf_trace_context", initfunc);
    ^
  In file included from /usr/include/python2.6/Python.h:130:0,
                   from util/scripting-engines/trace-event-python.c:22:
  /usr/include/python2.6/import.h:54:17: note: expected 'char *' but argument is of type 'const char *'
   PyAPI_FUNC(int) PyImport_AppendInittab(char *name, void (*initfunc)(void));
                   ^

cc1: all warnings being treated as errors

Cc Jaroslav Škarvada <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Ravi Bangoria <[email protected]>
Cc Seeteena Thoufeek <[email protected]>
Fixes 66dfdff03d19 ("perf tools: Add Python 3 support")
---
 tools/perf/util/scripting-engines/trace-event-python.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
tools/perf/util/scripting-engines/trace-event-python.c +9 -3
index 87ef16a1b17e..315905c748fa 100644
@@ -1494,34 +1494,40 @@ static void _free_command_line(wchar_t **command_line, int num)
 static int python_start_script(const char *script, int argc, const char **argv)
 {
 	struct tables *tables = &tables_global;
+	PyMODINIT_FUNC (*initfunc)(void);
 #if PY_MAJOR_VERSION < 3
 	const char **command_line;
 #else
 	wchar_t **command_line;
 #endif
-	char buf[PATH_MAX];
+	/*
+	 * Use a non-const name variable to cope with python 2.6's
+	 * PyImport_AppendInittab prototype
+	 */
+	char buf[PATH_MAX], name[19] = "perf_trace_context";
 	int i, err = 0;
 	FILE *fp;
 
 #if PY_MAJOR_VERSION < 3
+	initfunc = initperf_trace_context;
 	command_line = malloc((argc + 1) * sizeof(const char *));
 	command_line[0] = script;
 	for (i = 1; i < argc + 1; i++)
 		command_line[i] = argv[i - 1];
 #else
+	initfunc = PyInit_perf_trace_context;
 	command_line = malloc((argc + 1) * sizeof(wchar_t *));
 	command_line[0] = Py_DecodeLocale(script, NULL);
 	for (i = 1; i < argc + 1; i++)
 		command_line[i] = Py_DecodeLocale(argv[i - 1], NULL);
 #endif
 
+	PyImport_AppendInittab(name, initfunc);
 	Py_Initialize();
 
 #if PY_MAJOR_VERSION < 3
-	initperf_trace_context();
 	PySys_SetArgv(argc + 1, (char **)command_line);
 #else
-	PyInit_perf_trace_context();
 	PySys_SetArgv(argc + 1, command_line);
 #endif
 
author
#3 lore raw

PATCH 2/6 perf script python: Use PyBytes for attr in trace-event-python

In reply to Tony Jones

With Python3. PyUnicode_FromStringAndSize is unsafe to call on attr and will return NULL. Use _PyBytes_FromStringAndSize (as with raw_buf).

Below is the observed behavior without the fix. Note it is first necessary to apply the prior fix (Add trace_context extension module to sys,modules):

# ldd /usr/bin/perf | grep -i python
        libpython3.6m.so.1.0 => /usr/lib64/libpython3.6m.so.1.0 (0x00007f8e1dfb2000)
# perf record -e raw_syscalls:sys_enter /bin/false
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.018 MB perf.data (21 samples) ]
# perf script -g python | cat
generated Python script: perf-script.py
# perf script -s ./perf-script.py
in trace_begin
Segmentation fault (core dumped)
Cc Jaroslav Škarvada <[email protected]>
Fixes 66dfdff03d196e51322c6a85c0d8db8bb2bdd655
---
 tools/perf/util/scripting-engines/trace-event-python.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
tools/perf/util/scripting-engines/trace-event-python.c +1 -2
index 40300d8e80a7..777c97b13905 100644
@@ -733,8 +733,7 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample,
 		Py_FatalError("couldn't create Python dictionary");
 
 	pydict_set_item_string_decref(dict, "ev_name", _PyUnicode_FromString(perf_evsel__name(evsel)));
-	pydict_set_item_string_decref(dict, "attr", _PyUnicode_FromStringAndSize(
-			(const char *)&evsel->attr, sizeof(evsel->attr)));
+	pydict_set_item_string_decref(dict, "attr", _PyBytes_FromStringAndSize((const char *)&evsel->attr, sizeof(evsel->attr)));
 
 	pydict_set_item_string_decref(dict_sample, "pid",
 			_PyLong_FromLong(sample->pid));
Signature · 1 line
2.20.1
#4 lore raw

[tip:perf/core] perf script python: Use PyBytes for attr in trace-event-python

perf script python: Use PyBytes for attr in trace-event-python
Commit-ID:  72e0b15cb24a497d7d0d4707cf51ff40c185ae8c
Gitweb:     https://git.kernel.org/tip/72e0b15cb24a497d7d0d4707cf51ff40c185ae8c
Author:     Tony Jones <[email protected]>
AuthorDate: Wed, 23 Jan 2019 16:52:25 -0800
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Fri, 25 Jan 2019 15:12:10 +0100

perf script python: Use PyBytes for attr in trace-event-python

With Python3. PyUnicode_FromStringAndSize is unsafe to call on attr and will return NULL. Use _PyBytes_FromStringAndSize (as with raw_buf).

Below is the observed behavior without the fix. Note it is first necessary to apply the prior fix (Add trace_context extension module to sys,modules):

  # ldd /usr/bin/perf | grep -i python
          libpython3.6m.so.1.0 => /usr/lib64/libpython3.6m.so.1.0 (0x00007f8e1dfb2000)
  # perf record -e raw_syscalls:sys_enter /bin/false
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.018 MB perf.data (21 samples) ]
  # perf script -g python | cat
  generated Python script: perf-script.py
  # perf script -s ./perf-script.py
  in trace_begin
  Segmentation fault (core dumped)
Cc Jaroslav Škarvada <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Ravi Bangoria <[email protected]>
Cc Seeteena Thoufeek <[email protected]>
Fixes 66dfdff03d19 ("perf tools: Add Python 3 support")
---
 tools/perf/util/scripting-engines/trace-event-python.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
tools/perf/util/scripting-engines/trace-event-python.c +1 -2
index 315905c748fa..7059d1be2d09 100644
@@ -733,8 +733,7 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample,
 		Py_FatalError("couldn't create Python dictionary");
 
 	pydict_set_item_string_decref(dict, "ev_name", _PyUnicode_FromString(perf_evsel__name(evsel)));
-	pydict_set_item_string_decref(dict, "attr", _PyUnicode_FromStringAndSize(
-			(const char *)&evsel->attr, sizeof(evsel->attr)));
+	pydict_set_item_string_decref(dict, "attr", _PyBytes_FromStringAndSize((const char *)&evsel->attr, sizeof(evsel->attr)));
 
 	pydict_set_item_string_decref(dict_sample, "pid",
 			_PyLong_FromLong(sample->pid));
author
#5 lore raw

PATCH 3/6 perf script python: remove explicit shebang from setup.py

In reply to Tony Jones

Makefile.perf invokes setup.py via an explicit invocation of python (PYTHON_WORD) so there is therefore no need for an explicit shebang.

Also most distros follow pep-0394 which recommends that /usr/bin/python refer only to v2 and so may not exist on the system (if PYTHON=python3).

Cc Arnaldo Carvalho de Melo <[email protected]>
---
 tools/perf/util/setup.py | 2 --
 1 file changed, 2 deletions(-)
tools/perf/util/setup.py +0 -2
index 64d1f36dee99..d3ffc18424b5 100644
@@ -1,5 +1,3 @@
-#!/usr/bin/python
-
 from os import getenv
 from subprocess import Popen, PIPE
 from re import sub
Signature · 1 line
2.20.1
#6 lore raw

[tip:perf/core] perf script python: Remove explicit shebang from setup.py

perf script python: Remove explicit shebang from setup.py
Commit-ID:  099b79ca25c507ecbb25fb04f434e10415b68de0
Gitweb:     https://git.kernel.org/tip/099b79ca25c507ecbb25fb04f434e10415b68de0
Author:     Tony Jones <[email protected]>
AuthorDate: Wed, 23 Jan 2019 16:52:26 -0800
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Fri, 25 Jan 2019 15:12:10 +0100

perf script python: Remove explicit shebang from setup.py

Makefile.perf invokes setup.py via an explicit invocation of python (PYTHON_WORD) so there is therefore no need for an explicit shebang.

Also most distros follow pep-0394 which recommends that /usr/bin/python refer only to v2 and so may not exist on the system (if PYTHON=python3).

---
 tools/perf/util/setup.py | 2 --
 1 file changed, 2 deletions(-)
tools/perf/util/setup.py +0 -2
index 64d1f36dee99..d3ffc18424b5 100644
@@ -1,5 +1,3 @@
-#!/usr/bin/python
-
 from os import getenv
 from subprocess import Popen, PIPE
 from re import sub
author
#7 lore raw

PATCH 4/6 perf script python: remove explicit shebang from tests/attr.c

In reply to Tony Jones

tests/attr.c invokes attr.py via an explicit invocation of Python ($PYTHON) so there is therefore no need for an explicit shebang.

Also most distros follow pep-0394 which recommends that /usr/bin/python refer only to v2 and so may not exist on the system (if PYTHON=python3).

---
 tools/perf/tests/attr.py | 1 -
 1 file changed, 1 deletion(-)
tools/perf/tests/attr.py +0 -1
index 44090a9a19f3..3e07eee33b10 100644
@@ -1,4 +1,3 @@
-#! /usr/bin/python
 # SPDX-License-Identifier: GPL-2.0
 
 import os
Signature · 1 line
2.20.1
#8 lore raw

[tip:perf/core] perf script python: Remove explicit shebang from tests/attr.c

perf script python: Remove explicit shebang from tests/attr.c
Commit-ID:  d72eadbc1d2866fc047edd4535ffb0298fe240be
Gitweb:     https://git.kernel.org/tip/d72eadbc1d2866fc047edd4535ffb0298fe240be
Author:     Tony Jones <[email protected]>
AuthorDate: Wed, 23 Jan 2019 16:52:27 -0800
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Fri, 25 Jan 2019 15:12:10 +0100

perf script python: Remove explicit shebang from tests/attr.c

tests/attr.c invokes attr.py via an explicit invocation of Python ($PYTHON) so there is therefore no need for an explicit shebang.

Also most distros follow pep-0394 which recommends that /usr/bin/python refer only to v2 and so may not exist on the system (if PYTHON=python3).

---
 tools/perf/tests/attr.py | 1 -
 1 file changed, 1 deletion(-)
tools/perf/tests/attr.py +0 -1
index 44090a9a19f3..3e07eee33b10 100644
@@ -1,4 +1,3 @@
-#! /usr/bin/python
 # SPDX-License-Identifier: GPL-2.0
 
 import os
author
#9 lore raw

PATCH 5/6 perf script python: remove explicit shebang from Python scripts

In reply to Tony Jones

The scripts in scripts/python are intended to be run from 'perf script' and the Python version used is dictated by how perf was built (PYTHON=).

Also most distros follow pep-0394 which recommends that /usr/bin/python refer to Python2 and so may not exist on the system (if PYTHON=python3).

- Remove the explicit shebang
- Install the scripts as mode 644

Cc Frederic Weisbecker <[email protected]>
Cc Adrian Hunter <[email protected]>
Cc Jiri Olsa <[email protected]>
---
 tools/perf/Makefile.perf                         | 4 ++--
 tools/perf/scripts/python/exported-sql-viewer.py | 1 -
 tools/perf/scripts/python/sched-migration.py     | 2 --
 tools/perf/scripts/python/stat-cpi.py            | 1 -
 4 files changed, 2 insertions(+), 6 deletions(-)
tools/perf/Makefile.perf +2 -2
index 0ee6795d82cc..09df1c8a4ec9 100644
@@ -863,8 +863,8 @@ ifndef NO_LIBPYTHON
 	$(call QUIET_INSTALL, python-scripts) \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'; \
-		$(INSTALL) scripts/python/Perf-Trace-Util/lib/Perf/Trace/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \
-		$(INSTALL) scripts/python/*.py -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'; \
+		$(INSTALL) scripts/python/Perf-Trace-Util/lib/Perf/Trace/* -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \
+		$(INSTALL) scripts/python/*.py -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'; \
 		$(INSTALL) scripts/python/bin/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'
 endif
 	$(call QUIET_INSTALL, perf_completion-script) \
tools/perf/scripts/python/exported-sql-viewer.py +0 -1
index f278ce5ebab7..c3091401df91 100755
@@ -1,4 +1,3 @@
-#!/usr/bin/python2
 # SPDX-License-Identifier: GPL-2.0
 # exported-sql-viewer.py: view data from sql database
 # Copyright (c) 2014-2018, Intel Corporation.
tools/perf/scripts/python/sched-migration.py +0 -2
index 3473e7f66081..3984bf51f3c5 100644
@@ -1,5 +1,3 @@
-#!/usr/bin/python
-#
 # Cpu task migration overview toy
 #
 # Copyright (C) 2010 Frederic Weisbecker <[email protected]>
tools/perf/scripts/python/stat-cpi.py +0 -1
index 8410672efb8b..a81ad8835a74 100644
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # SPDX-License-Identifier: GPL-2.0
 
 data    = {}
Signature · 1 line
2.20.1
#10 lore raw

[tip:perf/core] perf script python: Remove explicit shebang from Python scripts

perf script python: Remove explicit shebang from Python scripts
Commit-ID:  a38352de4495a6a4662609a560b2db4b03d6b352
Gitweb:     https://git.kernel.org/tip/a38352de4495a6a4662609a560b2db4b03d6b352
Author:     Tony Jones <[email protected]>
AuthorDate: Wed, 23 Jan 2019 16:52:28 -0800
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Fri, 25 Jan 2019 15:12:10 +0100

perf script python: Remove explicit shebang from Python scripts

The scripts in scripts/python are intended to be run from 'perf script' and the Python version used is dictated by how perf was built (PYTHON=).

Also most distros follow pep-0394 which recommends that /usr/bin/python refer to Python2 and so may not exist on the system (if PYTHON=python3).

- Remove the explicit shebang
- Install the scripts as mode 644

---
 tools/perf/Makefile.perf                         | 4 ++--
 tools/perf/scripts/python/exported-sql-viewer.py | 1 -
 tools/perf/scripts/python/sched-migration.py     | 2 --
 tools/perf/scripts/python/stat-cpi.py            | 1 -
 4 files changed, 2 insertions(+), 6 deletions(-)
tools/perf/Makefile.perf +2 -2
index 0ee6795d82cc..09df1c8a4ec9 100644
@@ -863,8 +863,8 @@ ifndef NO_LIBPYTHON
 	$(call QUIET_INSTALL, python-scripts) \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'; \
-		$(INSTALL) scripts/python/Perf-Trace-Util/lib/Perf/Trace/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \
-		$(INSTALL) scripts/python/*.py -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'; \
+		$(INSTALL) scripts/python/Perf-Trace-Util/lib/Perf/Trace/* -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \
+		$(INSTALL) scripts/python/*.py -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'; \
 		$(INSTALL) scripts/python/bin/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'
 endif
 	$(call QUIET_INSTALL, perf_completion-script) \
tools/perf/scripts/python/exported-sql-viewer.py +0 -1
index f278ce5ebab7..c3091401df91 100755
@@ -1,4 +1,3 @@
-#!/usr/bin/python2
 # SPDX-License-Identifier: GPL-2.0
 # exported-sql-viewer.py: view data from sql database
 # Copyright (c) 2014-2018, Intel Corporation.
tools/perf/scripts/python/sched-migration.py +0 -2
index 3473e7f66081..3984bf51f3c5 100644
@@ -1,5 +1,3 @@
-#!/usr/bin/python
-#
 # Cpu task migration overview toy
 #
 # Copyright (C) 2010 Frederic Weisbecker <[email protected]>
tools/perf/scripts/python/stat-cpi.py +0 -1
index 8410672efb8b..a81ad8835a74 100644
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 # SPDX-License-Identifier: GPL-2.0
 
 data    = {}
author
#11 lore raw

PATCH 6/6 perf script python: add Python3 support to tests/attr.py

In reply to Tony Jones

Support both Python 2 and Python 3 in tests/attr.py

The use of "except as" syntax implies the minimum supported Python2 version is now v2.6

---
 tools/perf/tests/attr.py | 32 +++++++++++++++++++-------------
 1 file changed, 19 insertions(+), 13 deletions(-)
tools/perf/tests/attr.py +19 -13
index 3e07eee33b10..cb39ac46bc73 100644
@@ -1,5 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 
+from __future__ import print_function
+
 import os
 import sys
 import glob
@@ -7,7 +9,11 @@ import optparse
 import tempfile
 import logging
 import shutil
-import ConfigParser
+
+try:
+    import configparser
+except ImportError:
+    import ConfigParser as configparser
 
 def data_equal(a, b):
     # Allow multiple values in assignment separated by '|'
@@ -99,20 +105,20 @@ class Event(dict):
     def equal(self, other):
         for t in Event.terms:
             log.debug("      [%s] %s %s" % (t, self[t], other[t]));
-            if not self.has_key(t) or not other.has_key(t):
+            if t not in self or t not in other:
                 return False
             if not data_equal(self[t], other[t]):
                 return False
         return True
 
     def optional(self):
-        if self.has_key('optional') and self['optional'] == '1':
+        if 'optional' in self and self['optional'] == '1':
             return True
         return False
 
     def diff(self, other):
         for t in Event.terms:
-            if not self.has_key(t) or not other.has_key(t):
+            if t not in self or t not in other:
                 continue
             if not data_equal(self[t], other[t]):
                 log.warning("expected %s=%s, got %s" % (t, self[t], other[t]))
@@ -133,7 +139,7 @@ class Event(dict):
 #   - expected values assignments
 class Test(object):
     def __init__(self, path, options):
-        parser = ConfigParser.SafeConfigParser()
+        parser = configparser.SafeConfigParser()
         parser.read(path)
 
         log.warning("running '%s'" % path)
@@ -192,7 +198,7 @@ class Test(object):
         return True
 
     def load_events(self, path, events):
-        parser_event = ConfigParser.SafeConfigParser()
+        parser_event = configparser.SafeConfigParser()
         parser_event.read(path)
 
         # The event record section header contains 'event' word,
@@ -206,7 +212,7 @@ class Test(object):
             # Read parent event if there's any
             if (':' in section):
                 base = section[section.index(':') + 1:]
-                parser_base = ConfigParser.SafeConfigParser()
+                parser_base = configparser.SafeConfigParser()
                 parser_base.read(self.test_dir + '/' + base)
                 base_items = parser_base.items('event')
 
@@ -321,9 +327,9 @@ def run_tests(options):
     for f in glob.glob(options.test_dir + '/' + options.test):
         try:
             Test(f, options).run()
-        except Unsup, obj:
+        except Unsup as obj:
             log.warning("unsupp  %s" % obj.getMsg())
-        except Notest, obj:
+        except Notest as obj:
             log.warning("skipped %s" % obj.getMsg())
 
 def setup_log(verbose):
@@ -362,7 +368,7 @@ def main():
     parser.add_option("-p", "--perf",
                       action="store", type="string", dest="perf")
     parser.add_option("-v", "--verbose",
-                      action="count", dest="verbose")
+                      default=0, action="count", dest="verbose")
 
     options, args = parser.parse_args()
     if args:
@@ -372,7 +378,7 @@ def main():
     setup_log(options.verbose)
 
     if not options.test_dir:
-        print 'FAILED no -d option specified'
+        print('FAILED no -d option specified')
         sys.exit(-1)
 
     if not options.test:
@@ -381,8 +387,8 @@ def main():
     try:
         run_tests(options)
 
-    except Fail, obj:
-        print "FAILED %s" % obj.getMsg();
+    except Fail as obj:
+        print("FAILED %s" % obj.getMsg())
         sys.exit(-1)
 
     sys.exit(0)
Signature · 1 line
2.20.1
#12 lore raw

[tip:perf/core] perf script python: Add Python3 support to tests/attr.py

perf script python: Add Python3 support to tests/attr.py
Commit-ID:  35ea7e4bbb89ecd32057f5f6a2a8feb0d7224e51
Gitweb:     https://git.kernel.org/tip/35ea7e4bbb89ecd32057f5f6a2a8feb0d7224e51
Author:     Tony Jones <[email protected]>
AuthorDate: Wed, 23 Jan 2019 16:52:29 -0800
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Fri, 25 Jan 2019 15:12:10 +0100

perf script python: Add Python3 support to tests/attr.py

Support both Python 2 and Python 3 in tests/attr.py

The use of "except as" syntax implies the minimum supported Python2 version is now v2.6

Committer testing:

  $ make -C tools/perf PYTHON3=python install-bin

Before:

  # perf test attr
  16: Setup struct perf_event_attr                          : FAILED!
  48: Synthesize attr update                                : Ok
  [root@quaco ~]# perf test -v attr
  16: Setup struct perf_event_attr                          :
  --- start ---
  test child forked, pid 3121
    File "/home/acme/libexec/perf-core/tests/attr.py", line 324
      except Unsup, obj:
                ^
  SyntaxError: invalid syntax
  test child finished with -1
  ---- end ----
  Setup struct perf_event_attr: FAILED!
  48: Synthesize attr update                                :
  --- start ---
  test child forked, pid 3124
  test child finished with 0
  ---- end ----
  Synthesize attr update: Ok
  #

After:

   # perf test attr
  16: Setup struct perf_event_attr                          : Ok
  48: Synthesize attr update                                : Ok
  #
---
 tools/perf/tests/attr.py | 32 +++++++++++++++++++-------------
 1 file changed, 19 insertions(+), 13 deletions(-)
tools/perf/tests/attr.py +19 -13
index 3e07eee33b10..cb39ac46bc73 100644
@@ -1,5 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 
+from __future__ import print_function
+
 import os
 import sys
 import glob
@@ -7,7 +9,11 @@ import optparse
 import tempfile
 import logging
 import shutil
-import ConfigParser
+
+try:
+    import configparser
+except ImportError:
+    import ConfigParser as configparser
 
 def data_equal(a, b):
     # Allow multiple values in assignment separated by '|'
@@ -99,20 +105,20 @@ class Event(dict):
     def equal(self, other):
         for t in Event.terms:
             log.debug("      [%s] %s %s" % (t, self[t], other[t]));
-            if not self.has_key(t) or not other.has_key(t):
+            if t not in self or t not in other:
                 return False
             if not data_equal(self[t], other[t]):
                 return False
         return True
 
     def optional(self):
-        if self.has_key('optional') and self['optional'] == '1':
+        if 'optional' in self and self['optional'] == '1':
             return True
         return False
 
     def diff(self, other):
         for t in Event.terms:
-            if not self.has_key(t) or not other.has_key(t):
+            if t not in self or t not in other:
                 continue
             if not data_equal(self[t], other[t]):
                 log.warning("expected %s=%s, got %s" % (t, self[t], other[t]))
@@ -133,7 +139,7 @@ class Event(dict):
 #   - expected values assignments
 class Test(object):
     def __init__(self, path, options):
-        parser = ConfigParser.SafeConfigParser()
+        parser = configparser.SafeConfigParser()
         parser.read(path)
 
         log.warning("running '%s'" % path)
@@ -192,7 +198,7 @@ class Test(object):
         return True
 
     def load_events(self, path, events):
-        parser_event = ConfigParser.SafeConfigParser()
+        parser_event = configparser.SafeConfigParser()
         parser_event.read(path)
 
         # The event record section header contains 'event' word,
@@ -206,7 +212,7 @@ class Test(object):
             # Read parent event if there's any
             if (':' in section):
                 base = section[section.index(':') + 1:]
-                parser_base = ConfigParser.SafeConfigParser()
+                parser_base = configparser.SafeConfigParser()
                 parser_base.read(self.test_dir + '/' + base)
                 base_items = parser_base.items('event')
 
@@ -321,9 +327,9 @@ def run_tests(options):
     for f in glob.glob(options.test_dir + '/' + options.test):
         try:
             Test(f, options).run()
-        except Unsup, obj:
+        except Unsup as obj:
             log.warning("unsupp  %s" % obj.getMsg())
-        except Notest, obj:
+        except Notest as obj:
             log.warning("skipped %s" % obj.getMsg())
 
 def setup_log(verbose):
@@ -362,7 +368,7 @@ def main():
     parser.add_option("-p", "--perf",
                       action="store", type="string", dest="perf")
     parser.add_option("-v", "--verbose",
-                      action="count", dest="verbose")
+                      default=0, action="count", dest="verbose")
 
     options, args = parser.parse_args()
     if args:
@@ -372,7 +378,7 @@ def main():
     setup_log(options.verbose)
 
     if not options.test_dir:
-        print 'FAILED no -d option specified'
+        print('FAILED no -d option specified')
         sys.exit(-1)
 
     if not options.test:
@@ -381,8 +387,8 @@ def main():
     try:
         run_tests(options)
 
-    except Fail, obj:
-        print "FAILED %s" % obj.getMsg();
+    except Fail as obj:
+        print("FAILED %s" % obj.getMsg())
         sys.exit(-1)
 
     sys.exit(0)
#13 lore raw

[tip:perf/urgent] perf script python: Add Python3 support to tests/attr.py

In reply to Tony Jones
perf script python: Add Python3 support to tests/attr.py
Commit-ID:  8f2f350cbdb2c2fbff654cb778139144b48a59ba
Gitweb:     https://git.kernel.org/tip/8f2f350cbdb2c2fbff654cb778139144b48a59ba
Author:     Tony Jones <[email protected]>
AuthorDate: Wed, 23 Jan 2019 16:52:29 -0800
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 5 Feb 2019 10:31:08 -0300

perf script python: Add Python3 support to tests/attr.py

Support both Python 2 and Python 3 in tests/attr.py

The use of "except as" syntax implies the minimum supported Python2 version is now v2.6

Committer testing:

  $ make -C tools/perf PYTHON3=python install-bin

Before:

  # perf test attr
  16: Setup struct perf_event_attr                          : FAILED!
  48: Synthesize attr update                                : Ok
  [root@quaco ~]# perf test -v attr
  16: Setup struct perf_event_attr                          :
  --- start ---
  test child forked, pid 3121
    File "/home/acme/libexec/perf-core/tests/attr.py", line 324
      except Unsup, obj:
                ^
  SyntaxError: invalid syntax
  test child finished with -1
  ---- end ----
  Setup struct perf_event_attr: FAILED!
  48: Synthesize attr update                                :
  --- start ---
  test child forked, pid 3124
  test child finished with 0
  ---- end ----
  Synthesize attr update: Ok
  #

After:

   # perf test attr
  16: Setup struct perf_event_attr                          : Ok
  48: Synthesize attr update                                : Ok
  #
---
 tools/perf/tests/attr.py | 32 +++++++++++++++++++-------------
 1 file changed, 19 insertions(+), 13 deletions(-)
tools/perf/tests/attr.py +19 -13
index 44090a9a19f3..e952127e4fb0 100644
@@ -1,6 +1,8 @@
 #! /usr/bin/python
 # SPDX-License-Identifier: GPL-2.0
 
+from __future__ import print_function
+
 import os
 import sys
 import glob
@@ -8,7 +10,11 @@ import optparse
 import tempfile
 import logging
 import shutil
-import ConfigParser
+
+try:
+    import configparser
+except ImportError:
+    import ConfigParser as configparser
 
 def data_equal(a, b):
     # Allow multiple values in assignment separated by '|'
@@ -100,20 +106,20 @@ class Event(dict):
     def equal(self, other):
         for t in Event.terms:
             log.debug("      [%s] %s %s" % (t, self[t], other[t]));
-            if not self.has_key(t) or not other.has_key(t):
+            if t not in self or t not in other:
                 return False
             if not data_equal(self[t], other[t]):
                 return False
         return True
 
     def optional(self):
-        if self.has_key('optional') and self['optional'] == '1':
+        if 'optional' in self and self['optional'] == '1':
             return True
         return False
 
     def diff(self, other):
         for t in Event.terms:
-            if not self.has_key(t) or not other.has_key(t):
+            if t not in self or t not in other:
                 continue
             if not data_equal(self[t], other[t]):
                 log.warning("expected %s=%s, got %s" % (t, self[t], other[t]))
@@ -134,7 +140,7 @@ class Event(dict):
 #   - expected values assignments
 class Test(object):
     def __init__(self, path, options):
-        parser = ConfigParser.SafeConfigParser()
+        parser = configparser.SafeConfigParser()
         parser.read(path)
 
         log.warning("running '%s'" % path)
@@ -193,7 +199,7 @@ class Test(object):
         return True
 
     def load_events(self, path, events):
-        parser_event = ConfigParser.SafeConfigParser()
+        parser_event = configparser.SafeConfigParser()
         parser_event.read(path)
 
         # The event record section header contains 'event' word,
@@ -207,7 +213,7 @@ class Test(object):
             # Read parent event if there's any
             if (':' in section):
                 base = section[section.index(':') + 1:]
-                parser_base = ConfigParser.SafeConfigParser()
+                parser_base = configparser.SafeConfigParser()
                 parser_base.read(self.test_dir + '/' + base)
                 base_items = parser_base.items('event')
 
@@ -322,9 +328,9 @@ def run_tests(options):
     for f in glob.glob(options.test_dir + '/' + options.test):
         try:
             Test(f, options).run()
-        except Unsup, obj:
+        except Unsup as obj:
             log.warning("unsupp  %s" % obj.getMsg())
-        except Notest, obj:
+        except Notest as obj:
             log.warning("skipped %s" % obj.getMsg())
 
 def setup_log(verbose):
@@ -363,7 +369,7 @@ def main():
     parser.add_option("-p", "--perf",
                       action="store", type="string", dest="perf")
     parser.add_option("-v", "--verbose",
-                      action="count", dest="verbose")
+                      default=0, action="count", dest="verbose")
 
     options, args = parser.parse_args()
     if args:
@@ -373,7 +379,7 @@ def main():
     setup_log(options.verbose)
 
     if not options.test_dir:
-        print 'FAILED no -d option specified'
+        print('FAILED no -d option specified')
         sys.exit(-1)
 
     if not options.test:
@@ -382,8 +388,8 @@ def main():
     try:
         run_tests(options)
 
-    except Fail, obj:
-        print "FAILED %s" % obj.getMsg();
+    except Fail as obj:
+        print("FAILED %s" % obj.getMsg())
         sys.exit(-1)
 
     sys.exit(0)
Acked-by on the v1 series
#14 lore raw
In reply to Tony Jones
Quoted 16 lines · Tony Jones

Seeteena posted, earlier this week, some patches to add Python3 support to scripts/python/*.py. Unfortunately there were some issues with these patches (such as: https://lkml.org/lkml/2019/1/17/351)

Since I already had a tested set of patches in openSUSE:Factory and SLE15-SP1 and was about to submit them, Seeteena and I that agreed I should post my patches not involving scripts/python/*.py and Seeteena will later resubmit the patches for scripts/python/*.py incorporating my review feedback under a joint signed-off-by.

It should be noted that the use of "from __future__ import print_function" (see: https://lkml.org/lkml/2019/1/16/641) and "except as" (see change to:
tests/attr.py) implies Python2 >= 2.6 as the necessary support has not been backported to prior versions. I am not sure if it's worth detecting <2.6 at build time or whether it's sufficiently old as to be a non-issue?

Quoted 8 lines

The shebang changes were driven mostly by our build process as it scans all files within an rpm and the shebangs would result in a rpm requires on the python2 binary when BuildRequires was python3-devel. I think they make sense to apply upstream but understand totally if it's prefered we keep them local.

These changes have been tested with PYTHON=python2 (v2.7) and PYTHON=python3 (v3.6) on latest openSUSE Tumbleweed. I did notice that

please try latest code from Arnaldo, we just had fix in that area

Quoted 11 lines

test #18 "'import perf' in python" is failing on my system without these changes. I'll look at it further but didn't want to hold up Seeteena's resubmit.

Tony Jones (6):
perf script python: Add trace_context extension module to sys,modules perf script python: Use PyBytes for attr in trace-event-python perf script python: remove explicit shebang from setup.py perf script python: remove explicit shebang from tests/attr.c perf script python: remove explicit shebang from Python scripts perf script python: add Python3 support to tests/attr.py

thanks,
jirka

#15 lore raw

Em Thu, Jan 24, 2019 at 11:39:22AM +0100, Jiri Olsa escreveu:

Quoted 40 lines · Tony Jones
Quoted 16 lines

Seeteena posted, earlier this week, some patches to add Python3 support to scripts/python/*.py. Unfortunately there were some issues with these patches (such as: https://lkml.org/lkml/2019/1/17/351)

Since I already had a tested set of patches in openSUSE:Factory and SLE15-SP1 and was about to submit them, Seeteena and I that agreed I should post my patches not involving scripts/python/*.py and Seeteena will later resubmit the patches for scripts/python/*.py incorporating my review feedback under a joint signed-off-by.

It should be noted that the use of "from __future__ import print_function" (see: https://lkml.org/lkml/2019/1/16/641) and "except as" (see change to:
tests/attr.py) implies Python2 >= 2.6 as the necessary support has not been backported to prior versions. I am not sure if it's worth detecting <2.6 at build time or whether it's sufficiently old as to be a non-issue?

Quoted 8 lines

The shebang changes were driven mostly by our build process as it scans all files within an rpm and the shebangs would result in a rpm requires on the python2 binary when BuildRequires was python3-devel. I think they make sense to apply upstream but understand totally if it's prefered we keep them local.

These changes have been tested with PYTHON=python2 (v2.7) and PYTHON=python3 (v3.6) on latest openSUSE Tumbleweed. I did notice that

please try latest code from Arnaldo, we just had fix in that area

Quoted 11 lines

test #18 "'import perf' in python" is failing on my system without these changes. I'll look at it further but didn't want to hold up Seeteena's resubmit.

Tony Jones (6):
perf script python: Add trace_context extension module to sys,modules perf script python: Use PyBytes for attr in trace-event-python perf script python: remove explicit shebang from setup.py perf script python: remove explicit shebang from tests/attr.c perf script python: remove explicit shebang from Python scripts perf script python: add Python3 support to tests/attr.py

Quoted 1 line

Thanks, tested and applied.

- Arnaldo

#16 lore raw
In reply to Tony Jones

Em Wed, Jan 23, 2019 at 04:52:23PM -0800, Tony Jones escreveu:

Quoted 22 lines

Seeteena posted, earlier this week, some patches to add Python3 support to scripts/python/*.py. Unfortunately there were some issues with these patches (such as: https://lkml.org/lkml/2019/1/17/351)

Since I already had a tested set of patches in openSUSE:Factory and SLE15-SP1 and was about to submit them, Seeteena and I that agreed I should post my patches not involving scripts/python/*.py and Seeteena will later resubmit the patches for scripts/python/*.py incorporating my review feedback under a joint signed-off-by.

It should be noted that the use of "from __future__ import print_function" (see: https://lkml.org/lkml/2019/1/16/641) and "except as" (see change to:
tests/attr.py) implies Python2 >= 2.6 as the necessary support has not been backported to prior versions. I am not sure if it's worth detecting <2.6 at build time or whether it's sufficiently old as to be a non-issue?

The shebang changes were driven mostly by our build process as it scans all files within an rpm and the shebangs would result in a rpm requires on the python2 binary when BuildRequires was python3-devel. I think they make sense to apply upstream but understand totally if it's prefered we keep them local.

Quoted 5 lines

These changes have been tested with PYTHON=python2 (v2.7) and PYTHON=python3 (v3.6) on latest openSUSE Tumbleweed. I did notice that test #18 "'import perf' in python" is failing on my system without these changes. I'll look at it further but didn't want to hold up Seeteena's resubmit.

So it fails on AmazonLinux 1, that has python 2.6, please check if this is something we can workaround, if its difficult, I'll just use NO_PYTHON=1 there to disable it.

CC /tmp/build/perf/util/parse-branch-options.o
util/scripting-engines/trace-event-python.c: In function 'python_start_script':
util/scripting-engines/trace-event-python.c:1520:2: error: passing argument 1 of 'PyImport_AppendInittab' discards 'const' qualifier from pointer target type [-Werror]
PyImport_AppendInittab("perf_trace_context", initfunc);
^
In file included from /usr/include/python2.6/Python.h:130:0,
from util/scripting-engines/trace-event-python.c:22:
/usr/include/python2.6/import.h:54:17: note: expected 'char *' but argument is of type 'const char *'

 PyAPI_FUNC(int) PyImport_AppendInittab(char *name, void (*initfunc)(void));
                 ^

cc1: all warnings being treated as errors
mv: cannot stat '/tmp/build/perf/util/scripting-engines/.trace-event-python.o.tmp': No such file or directory make[5]: *** [/tmp/build/perf/util/scripting-engines/trace-event-python.o] Error 1

#17 lore raw

Em Fri, Jan 25, 2019 at 01:31:19PM +0100, Arnaldo Carvalho de Melo escreveu:

Quoted 41 lines

Em Wed, Jan 23, 2019 at 04:52:23PM -0800, Tony Jones escreveu:

Quoted 22 lines

Seeteena posted, earlier this week, some patches to add Python3 support to scripts/python/*.py. Unfortunately there were some issues with these patches (such as: https://lkml.org/lkml/2019/1/17/351)

Since I already had a tested set of patches in openSUSE:Factory and SLE15-SP1 and was about to submit them, Seeteena and I that agreed I should post my patches not involving scripts/python/*.py and Seeteena will later resubmit the patches for scripts/python/*.py incorporating my review feedback under a joint signed-off-by.

It should be noted that the use of "from __future__ import print_function" (see: https://lkml.org/lkml/2019/1/16/641) and "except as" (see change to:
tests/attr.py) implies Python2 >= 2.6 as the necessary support has not been backported to prior versions. I am not sure if it's worth detecting <2.6 at build time or whether it's sufficiently old as to be a non-issue?

The shebang changes were driven mostly by our build process as it scans all files within an rpm and the shebangs would result in a rpm requires on the python2 binary when BuildRequires was python3-devel. I think they make sense to apply upstream but understand totally if it's prefered we keep them local.

Quoted 5 lines

These changes have been tested with PYTHON=python2 (v2.7) and PYTHON=python3 (v3.6) on latest openSUSE Tumbleweed. I did notice that test #18 "'import perf' in python" is failing on my system without these changes. I'll look at it further but didn't want to hold up Seeteena's resubmit.

So it fails on AmazonLinux 1, that has python 2.6, please check if this is something we can workaround, if its difficult, I'll just use NO_PYTHON=1 there to disable it.

CC /tmp/build/perf/util/parse-branch-options.o
util/scripting-engines/trace-event-python.c: In function 'python_start_script':
util/scripting-engines/trace-event-python.c:1520:2: error: passing argument 1 of 'PyImport_AppendInittab' discards 'const' qualifier from pointer target type [-Werror]
PyImport_AppendInittab("perf_trace_context", initfunc);
^
In file included from /usr/include/python2.6/Python.h:130:0,
from util/scripting-engines/trace-event-python.c:22:
/usr/include/python2.6/import.h:54:17: note: expected 'char *' but argument is of type 'const char *'

Quoted 5 lines
 PyAPI_FUNC(int) PyImport_AppendInittab(char *name, void (*initfunc)(void));
                 ^

cc1: all warnings being treated as errors
mv: cannot stat '/tmp/build/perf/util/scripting-engines/.trace-event-python.o.tmp': No such file or directory make[5]: *** [/tmp/build/perf/util/scripting-engines/trace-event-python.o] Error 1

I did a quick hack to init an auto variable with that const string and then pass it, is passing everything so far:

$ dm
   1 alpine:3.4                    : Ok   gcc (Alpine 5.3.0) 5.3.0
   2 alpine:3.5                    : Ok   gcc (Alpine 6.2.1) 6.2.1 20160822
   3 alpine:3.6                    : Ok   gcc (Alpine 6.3.0) 6.3.0
   4 alpine:3.7                    : Ok   gcc (Alpine 6.4.0) 6.4.0
   5 alpine:3.8                    : Ok   gcc (Alpine 6.4.0) 6.4.0
   6 alpine:edge                   : Ok   gcc (Alpine 8.2.0) 8.2.0
   7 amazonlinux:1                 : Ok   gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
   8 amazonlinux:2                 : Ok   gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)
   9 android-ndk:r12b-arm          : Ok   gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1)
  10 android-ndk:r15c-arm          : Ok   gcc (GCC) 8.2.1 20181215 (Red Hat 8.2.1-6)
  11 centos:5                      : Ok   gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-55)
  12 centos:6                      : Ok   gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23)
  13 centos:7                      : Ok   gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36)
  14 clearlinux:latest             : Ok   gcc (Clear Linux OS for Intel Architecture) 8.2.1 20180502
  15 debian:7                      : Ok   gcc (Debian 4.7.2-5) 4.7.2
  16 debian:8                      : Ok   gcc (Debian 4.9.2-10+deb8u2) 4.9.2
  17 debian:9                      : Ok   gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
  18 debian:experimental           : Ok   gcc (Debian 8.2.0-13) 8.2.0
  19 debian:experimental-x-arm64   : Ok   gcc (Debian 8.2.0-13) 8.2.0
  20 debian:experimental-x-mips    : Ok   gcc (Debian 8.2.0-13) 8.2.0
  21 debian:experimental-x-mips64  : Ok   gcc (Debian 8.2.0-13) 8.2.0
  22 debian:experimental-x-mipsel  : Ok   gcc (Debian 8.2.0-13) 8.2.0
  23 fedora:20                     : Ok   gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-7)
  24 fedora:22                     : Ok   gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6)
  25 fedora:23                     : Ok   gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6)
  26: fedora:24
author
#18 lore raw
Quoted 33 lines · Arnaldo Carvalho de Melo

Em Fri, Jan 25, 2019 at 01:31:19PM +0100, Arnaldo Carvalho de Melo escreveu:

Quoted 32 lines

Em Wed, Jan 23, 2019 at 04:52:23PM -0800, Tony Jones escreveu:

Quoted 22 lines

Seeteena posted, earlier this week, some patches to add Python3 support to scripts/python/*.py. Unfortunately there were some issues with these patches (such as: https://lkml.org/lkml/2019/1/17/351)

Since I already had a tested set of patches in openSUSE:Factory and SLE15-SP1 and was about to submit them, Seeteena and I that agreed I should post my patches not involving scripts/python/*.py and Seeteena will later resubmit the patches for scripts/python/*.py incorporating my review feedback under a joint signed-off-by.

It should be noted that the use of "from __future__ import print_function" (see: https://lkml.org/lkml/2019/1/16/641) and "except as" (see change to:
tests/attr.py) implies Python2 >= 2.6 as the necessary support has not been backported to prior versions. I am not sure if it's worth detecting <2.6 at build time or whether it's sufficiently old as to be a non-issue?

The shebang changes were driven mostly by our build process as it scans all files within an rpm and the shebangs would result in a rpm requires on the python2 binary when BuildRequires was python3-devel. I think they make sense to apply upstream but understand totally if it's prefered we keep them local.

Quoted 5 lines

These changes have been tested with PYTHON=python2 (v2.7) and PYTHON=python3 (v3.6) on latest openSUSE Tumbleweed. I did notice that test #18 "'import perf' in python" is failing on my system without these changes. I'll look at it further but didn't want to hold up Seeteena's resubmit.

So it fails on AmazonLinux 1, that has python 2.6, please check if this is something we can workaround, if its difficult, I'll just use NO_PYTHON=1 there to disable it.

Sorry about this. I'll be sure to test more broadly next time.

Quoted 15 lines
Quoted 9 lines

CC /tmp/build/perf/util/parse-branch-options.o
util/scripting-engines/trace-event-python.c: In function 'python_start_script':
util/scripting-engines/trace-event-python.c:1520:2: error: passing argument 1 of 'PyImport_AppendInittab' discards 'const' qualifier from pointer target type [-Werror]
PyImport_AppendInittab("perf_trace_context", initfunc);
^
In file included from /usr/include/python2.6/Python.h:130:0,
from util/scripting-engines/trace-event-python.c:22:
/usr/include/python2.6/import.h:54:17: note: expected 'char *' but argument is of type 'const char *'

Quoted 5 lines
 PyAPI_FUNC(int) PyImport_AppendInittab(char *name, void (*initfunc)(void));
                 ^

cc1: all warnings being treated as errors
mv: cannot stat '/tmp/build/perf/util/scripting-engines/.trace-event-python.o.tmp': No such file or directory make[5]: *** [/tmp/build/perf/util/scripting-engines/trace-event-python.o] Error 1

Quoted 2 lines

I did a quick hack to init an auto variable with that const string and then pass it, is passing everything so far:

I see that you already amended in cc4376422552

Thanks!