Skip to contentSkip to conversation

perf python: Add support to access tracepoint fields

PATCH Superseded Jiri Olsa 5 participants 30 messages Jul 10, 2016 – Jul 13, 2016 perf events

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Jiri Olsa

author reviewer
#0 lore raw

cover perf python: Add support to access tracepoint fields

hi,
adding support to access tracepoint fields in python scripts.

With this patchset it's possible to access tracepoint fields in event python object like:

print "time %u prev_comm=%s prev_pid=%d prev_prio=%d prev_state=0x%x ==> next_comm=%s next_pid=%d next_prio=%d" % (
event.sample_time,
event.prev_comm,
event.prev_pid,
event.prev_prio,
event.prev_state,
event.next_comm,
event.next_pid,
event.next_prio)

Also available in:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/fixes

thanks,
jirka

---
Jiri Olsa (10):
perf tools: Make perf_evlist__event2evsel public
perf tools: Introduce trace_event__tp_format_id function perf python: Init perf_event_attr::size in perf.evsel constructor perf python: Fix pyrf_evlist__read_on_cpu event consuming perf python: Put perf.event objects into dictionary perf python: Add perf.tracepoint method
perf python: Add struct evsel into struct pyrf_event perf python: Add support to resolve tracepoint fields perf python: Add tracepoint example
perf script python: Fix string vs byte array resolving

 tools/perf/python/tracepoint.py                        |  47 ++++++++++++++++++++++
 tools/perf/util/evlist.c                               |   4 +-
 tools/perf/util/evlist.h                               |   3 ++
 tools/perf/util/python.c                               | 162 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 tools/perf/util/scripting-engines/trace-event-python.c |  34 +++++++++++++---
 tools/perf/util/trace-event.c                          |   9 +++++
 tools/perf/util/trace-event.h                          |   2 +
 7 files changed, 250 insertions(+), 11 deletions(-)
 create mode 100755 tools/perf/python/tracepoint.py
author reviewer
#1 lore raw

PATCH 1/10 perf tools: Make perf_evlist__event2evsel public

It will be used outside of evlist.c object
in folowing patches.

---
 tools/perf/util/evlist.c | 4 ++--
 tools/perf/util/evlist.h | 3 +++
 2 files changed, 5 insertions(+), 2 deletions(-)
tools/perf/util/evlist.c +2 -2
index 113507716044..3aca5c9acef1 100644
@@ -651,8 +651,8 @@ static int perf_evlist__event2id(struct perf_evlist *evlist,
 	return 0;
 }
 
-static struct perf_evsel *perf_evlist__event2evsel(struct perf_evlist *evlist,
-						   union perf_event *event)
+struct perf_evsel *perf_evlist__event2evsel(struct perf_evlist *evlist,
+					    union perf_event *event)
 {
 	struct perf_evsel *first = perf_evlist__first(evlist);
 	struct hlist_head *head;
tools/perf/util/evlist.h +3 -0
index 872912b392c9..afd087761a47 100644
@@ -323,4 +323,7 @@ void perf_event_attr__set_max_precise_ip(struct perf_event_attr *attr);
 
 struct perf_evsel *
 perf_evlist__find_evsel_by_str(struct perf_evlist *evlist, const char *str);
+
+struct perf_evsel *perf_evlist__event2evsel(struct perf_evlist *evlist,
+					    union perf_event *event);
 #endif /* __PERF_EVLIST_H */
Signature · 1 line
2.4.11
#2 lore raw

[tip:perf/core] perf evlist: Make event2evsel public

perf evlist: Make event2evsel public
Commit-ID:  7cb5c5acaba9fa0b90ca11275f19131d3eca35c2
Gitweb:     http://git.kernel.org/tip/7cb5c5acaba9fa0b90ca11275f19131d3eca35c2
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 10 Jul 2016 13:07:53 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 12 Jul 2016 16:13:58 -0300

perf evlist: Make event2evsel public

It will be used outside of evlist.c object in folowing patches.

---
 tools/perf/util/evlist.c | 4 ++--
 tools/perf/util/evlist.h | 3 +++
 2 files changed, 5 insertions(+), 2 deletions(-)
tools/perf/util/evlist.c +2 -2
index f2d478d..862e69c 100644
@@ -651,8 +651,8 @@ static int perf_evlist__event2id(struct perf_evlist *evlist,
 	return 0;
 }
 
-static struct perf_evsel *perf_evlist__event2evsel(struct perf_evlist *evlist,
-						   union perf_event *event)
+struct perf_evsel *perf_evlist__event2evsel(struct perf_evlist *evlist,
+					    union perf_event *event)
 {
 	struct perf_evsel *first = perf_evlist__first(evlist);
 	struct hlist_head *head;
tools/perf/util/evlist.h +3 -0
index 872912b..afd0877 100644
@@ -323,4 +323,7 @@ void perf_event_attr__set_max_precise_ip(struct perf_event_attr *attr);
 
 struct perf_evsel *
 perf_evlist__find_evsel_by_str(struct perf_evlist *evlist, const char *str);
+
+struct perf_evsel *perf_evlist__event2evsel(struct perf_evlist *evlist,
+					    union perf_event *event);
 #endif /* __PERF_EVLIST_H */
author reviewer
#3 lore raw

PATCH 2/10 perf tools: Introduce trace_event__tp_format_id function

In reply to Jiri Olsa

To get struct event_format object from tracepoint ID. It will be used in following patches.

---
 tools/perf/util/trace-event.c | 9 +++++++++
 tools/perf/util/trace-event.h | 2 ++
 2 files changed, 11 insertions(+)
tools/perf/util/trace-event.c +9 -0
index 8ae051e0ec79..883caa8941a9 100644
@@ -105,3 +105,12 @@ trace_event__tp_format(const char *sys, const char *name)
 
 	return tp_format(sys, name);
 }
+
+struct event_format*
+trace_event__tp_format_id(int id)
+{
+	if (!tevent_initialized && trace_event__init2())
+		return ERR_PTR(-ENOMEM);
+
+	return pevent_find_event(tevent.pevent, id);
+}
tools/perf/util/trace-event.h +2 -0
index bce5b1dac268..b0af9c81bb0d 100644
@@ -23,6 +23,8 @@ int trace_event__register_resolver(struct machine *machine,
 struct event_format*
 trace_event__tp_format(const char *sys, const char *name);
 
+struct event_format *trace_event__tp_format_id(int id);
+
 int bigendian(void);
 
 void event_format__fprintf(struct event_format *event,
Signature · 1 line
2.4.11
#4 lore raw

[tip:perf/core] perf tools: Introduce trace_event__tp_format_id()

perf tools: Introduce trace_event__tp_format_id()
Commit-ID:  71fe1052af98fc5b615c067425aeb6fe39a0368c
Gitweb:     http://git.kernel.org/tip/71fe1052af98fc5b615c067425aeb6fe39a0368c
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 10 Jul 2016 13:07:54 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 12 Jul 2016 16:14:52 -0300

perf tools: Introduce trace_event__tp_format_id()

To get struct event_format object from tracepoint ID. It will be used in following patches.

---
 tools/perf/util/trace-event.c | 8 ++++++++
 tools/perf/util/trace-event.h | 2 ++
 2 files changed, 10 insertions(+)
tools/perf/util/trace-event.c +8 -0
index 8ae051e..c330780 100644
@@ -105,3 +105,11 @@ trace_event__tp_format(const char *sys, const char *name)
 
 	return tp_format(sys, name);
 }
+
+struct event_format *trace_event__tp_format_id(int id)
+{
+	if (!tevent_initialized && trace_event__init2())
+		return ERR_PTR(-ENOMEM);
+
+	return pevent_find_event(tevent.pevent, id);
+}
tools/perf/util/trace-event.h +2 -0
index bce5b1d..b0af9c8 100644
@@ -23,6 +23,8 @@ int trace_event__register_resolver(struct machine *machine,
 struct event_format*
 trace_event__tp_format(const char *sys, const char *name);
 
+struct event_format *trace_event__tp_format_id(int id);
+
 int bigendian(void);
 
 void event_format__fprintf(struct event_format *event,
author reviewer
#5 lore raw

PATCH 3/10 perf python: Init perf_event_attr::size in perf.evsel constructor

In reply to Jiri Olsa

Currently 0 is passed as perf_event_attr::size,
which could block usage of new features.

---
 tools/perf/util/python.c | 1 +
 1 file changed, 1 insertion(+)
tools/perf/util/python.c +1 -0
index 65c6c73d28fe..d0c1267741ee 100644
@@ -653,6 +653,7 @@ static int pyrf_evsel__init(struct pyrf_evsel *pevsel,
 	attr.precise_ip	    = precise_ip;
 	attr.mmap_data	    = mmap_data;
 	attr.sample_id_all  = sample_id_all;
+	attr.size	    = sizeof(attr);
 
 	perf_evsel__init(&pevsel->evsel, &attr, idx);
 	return 0;
Signature · 1 line
2.4.11
#6 lore raw

[tip:perf/core] perf python: Init perf_event_attr::size in perf.evsel constructor

perf python: Init perf_event_attr::size in perf.evsel constructor
Commit-ID:  ad4e3c04587c01c2e2b00c0e6a414dbededa6c55
Gitweb:     http://git.kernel.org/tip/ad4e3c04587c01c2e2b00c0e6a414dbededa6c55
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 10 Jul 2016 13:07:55 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 12 Jul 2016 16:16:17 -0300

perf python: Init perf_event_attr::size in perf.evsel constructor

Currently 0 is passed as perf_event_attr::size, which could block usage of new features.

---
 tools/perf/util/python.c | 1 +
 1 file changed, 1 insertion(+)
tools/perf/util/python.c +1 -0
index 65c6c73..d0c1267 100644
@@ -653,6 +653,7 @@ static int pyrf_evsel__init(struct pyrf_evsel *pevsel,
 	attr.precise_ip	    = precise_ip;
 	attr.mmap_data	    = mmap_data;
 	attr.sample_id_all  = sample_id_all;
+	attr.size	    = sizeof(attr);
 
 	perf_evsel__init(&pevsel->evsel, &attr, idx);
 	return 0;
author reviewer
#7 lore raw

PATCH 4/10 perf python: Fix pyrf_evlist__read_on_cpu event consuming

In reply to Jiri Olsa

We can't consume the event before parsing it. Under heavy load we could get caught by kernel writer overwriting the event we're trying to parse.

---
 tools/perf/util/python.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
tools/perf/util/python.c +4 -2
index d0c1267741ee..c68ef0319114 100644
@@ -865,12 +865,14 @@ static PyObject *pyrf_evlist__read_on_cpu(struct pyrf_evlist *pevlist,
 		PyObject *pyevent = pyrf_event__new(event);
 		struct pyrf_event *pevent = (struct pyrf_event *)pyevent;
 
-		perf_evlist__mmap_consume(evlist, cpu);
-
 		if (pyevent == NULL)
 			return PyErr_NoMemory();
 
 		err = perf_evlist__parse_sample(evlist, event, &pevent->sample);
+
+		/* Consume the even only after we parsed it out. */
+		perf_evlist__mmap_consume(evlist, cpu);
+
 		if (err)
 			return PyErr_Format(PyExc_OSError,
 					    "perf: can't parse sample, err=%d", err);
Signature · 1 line
2.4.11
#8 lore raw

[tip:perf/core] perf python: Fix pyrf_evlist__read_on_cpu event consuming

perf python: Fix pyrf_evlist__read_on_cpu event consuming
Commit-ID:  e8968e654191390a1300f0847250353a1c9da30d
Gitweb:     http://git.kernel.org/tip/e8968e654191390a1300f0847250353a1c9da30d
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 10 Jul 2016 13:07:56 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 12 Jul 2016 16:16:44 -0300

perf python: Fix pyrf_evlist__read_on_cpu event consuming

We can't consume the event before parsing it. Under heavy load we could get caught by kernel writer overwriting the event we're trying to parse.

---
 tools/perf/util/python.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
tools/perf/util/python.c +4 -2
index d0c1267..c68ef03 100644
@@ -865,12 +865,14 @@ static PyObject *pyrf_evlist__read_on_cpu(struct pyrf_evlist *pevlist,
 		PyObject *pyevent = pyrf_event__new(event);
 		struct pyrf_event *pevent = (struct pyrf_event *)pyevent;
 
-		perf_evlist__mmap_consume(evlist, cpu);
-
 		if (pyevent == NULL)
 			return PyErr_NoMemory();
 
 		err = perf_evlist__parse_sample(evlist, event, &pevent->sample);
+
+		/* Consume the even only after we parsed it out. */
+		perf_evlist__mmap_consume(evlist, cpu);
+
 		if (err)
 			return PyErr_Format(PyExc_OSError,
 					    "perf: can't parse sample, err=%d", err);
author reviewer
#9 lore raw

PATCH 5/10 perf python: Put perf.event objects into dictionary

In reply to Jiri Olsa

Make perf.event object parts of the perf module
dictionary so we can address them by name.

Following objects/names are added:
mmap_event
lost_event
comm_event
task_event
throttle_event
task_event
read_event
sample_event
switch_event

We can now use it in python script like:
...
event = evlist.read_on_cpu(cpu)
...
if not isinstance(event, perf.sample_event):

---
 tools/perf/util/python.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
tools/perf/util/python.c +27 -0
index c68ef0319114..fc277e486d17 100644
@@ -1103,6 +1103,33 @@ PyMODINIT_FUNC initperf(void)
 	Py_INCREF(&pyrf_evsel__type);
 	PyModule_AddObject(module, "evsel", (PyObject*)&pyrf_evsel__type);
 
+	Py_INCREF(&pyrf_mmap_event__type);
+	PyModule_AddObject(module, "mmap_event", (PyObject *)&pyrf_mmap_event__type);
+
+	Py_INCREF(&pyrf_lost_event__type);
+	PyModule_AddObject(module, "lost_event", (PyObject *)&pyrf_lost_event__type);
+
+	Py_INCREF(&pyrf_comm_event__type);
+	PyModule_AddObject(module, "comm_event", (PyObject *)&pyrf_comm_event__type);
+
+	Py_INCREF(&pyrf_task_event__type);
+	PyModule_AddObject(module, "task_event", (PyObject *)&pyrf_task_event__type);
+
+	Py_INCREF(&pyrf_throttle_event__type);
+	PyModule_AddObject(module, "throttle_event", (PyObject *)&pyrf_throttle_event__type);
+
+	Py_INCREF(&pyrf_task_event__type);
+	PyModule_AddObject(module, "task_event", (PyObject *)&pyrf_task_event__type);
+
+	Py_INCREF(&pyrf_read_event__type);
+	PyModule_AddObject(module, "read_event", (PyObject *)&pyrf_read_event__type);
+
+	Py_INCREF(&pyrf_sample_event__type);
+	PyModule_AddObject(module, "sample_event", (PyObject *)&pyrf_sample_event__type);
+
+	Py_INCREF(&pyrf_context_switch_event__type);
+	PyModule_AddObject(module, "switch_event", (PyObject *)&pyrf_context_switch_event__type);
+
 	Py_INCREF(&pyrf_thread_map__type);
 	PyModule_AddObject(module, "thread_map", (PyObject*)&pyrf_thread_map__type);
 
Signature · 1 line
2.4.11
#10 lore raw

[tip:perf/core] perf python: Put perf.event objects into dictionary

perf python: Put perf.event objects into dictionary
Commit-ID:  85e37de3a993b9e407398f792b996acad27f4cdc
Gitweb:     http://git.kernel.org/tip/85e37de3a993b9e407398f792b996acad27f4cdc
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 10 Jul 2016 13:07:57 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 12 Jul 2016 16:17:14 -0300

perf python: Put perf.event objects into dictionary

Make perf.event object parts of the perf module dictionary so we can address them by name.

The following objects/names are added:

mmap_event
lost_event
comm_event
task_event
throttle_event
task_event
read_event
sample_event
switch_event

We can now use it in python script like:
...
event = evlist.read_on_cpu(cpu)
...
if not isinstance(event, perf.sample_event):

---
 tools/perf/util/python.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
tools/perf/util/python.c +27 -0
index c68ef03..fc277e4 100644
@@ -1103,6 +1103,33 @@ PyMODINIT_FUNC initperf(void)
 	Py_INCREF(&pyrf_evsel__type);
 	PyModule_AddObject(module, "evsel", (PyObject*)&pyrf_evsel__type);
 
+	Py_INCREF(&pyrf_mmap_event__type);
+	PyModule_AddObject(module, "mmap_event", (PyObject *)&pyrf_mmap_event__type);
+
+	Py_INCREF(&pyrf_lost_event__type);
+	PyModule_AddObject(module, "lost_event", (PyObject *)&pyrf_lost_event__type);
+
+	Py_INCREF(&pyrf_comm_event__type);
+	PyModule_AddObject(module, "comm_event", (PyObject *)&pyrf_comm_event__type);
+
+	Py_INCREF(&pyrf_task_event__type);
+	PyModule_AddObject(module, "task_event", (PyObject *)&pyrf_task_event__type);
+
+	Py_INCREF(&pyrf_throttle_event__type);
+	PyModule_AddObject(module, "throttle_event", (PyObject *)&pyrf_throttle_event__type);
+
+	Py_INCREF(&pyrf_task_event__type);
+	PyModule_AddObject(module, "task_event", (PyObject *)&pyrf_task_event__type);
+
+	Py_INCREF(&pyrf_read_event__type);
+	PyModule_AddObject(module, "read_event", (PyObject *)&pyrf_read_event__type);
+
+	Py_INCREF(&pyrf_sample_event__type);
+	PyModule_AddObject(module, "sample_event", (PyObject *)&pyrf_sample_event__type);
+
+	Py_INCREF(&pyrf_context_switch_event__type);
+	PyModule_AddObject(module, "switch_event", (PyObject *)&pyrf_context_switch_event__type);
+
 	Py_INCREF(&pyrf_thread_map__type);
 	PyModule_AddObject(module, "thread_map", (PyObject*)&pyrf_thread_map__type);
 
author reviewer
#11 lore raw

PATCH 6/10 perf python: Add perf.tracepoint method

In reply to Jiri Olsa

To get id of the tracepoint from subsystem and
name strings. The interface is:

id = perf.tracepoint(sys, name)

In case of error -1 is returned.

It will be used to get python tracepoint event's
config value for tracepoint event.

---
 tools/perf/util/python.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
tools/perf/util/python.c +26 -0
index fc277e486d17..45fdd4a68721 100644
@@ -2,6 +2,7 @@
 #include <structmember.h>
 #include <inttypes.h>
 #include <poll.h>
+#include <linux/err.h>
 #include "evlist.h"
 #include "evsel.h"
 #include "event.h"
@@ -1076,7 +1077,32 @@ static struct {
 	{ .name = NULL, },
 };
 
+static PyObject *pyrf__tracepoint(struct pyrf_evsel *pevsel,
+				  PyObject *args, PyObject *kwargs)
+{
+	struct event_format *tp_format;
+	static char *kwlist[] = { "sys", "name", NULL };
+	char *sys  = NULL;
+	char *name = NULL;
+
+	if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|ss", kwlist,
+					 &sys, &name))
+		return NULL;
+
+	tp_format = trace_event__tp_format(sys, name);
+	if (IS_ERR(tp_format))
+		return PyInt_FromLong(-1);
+
+	return PyInt_FromLong(tp_format->id);
+}
+
 static PyMethodDef perf__methods[] = {
+	{
+		.ml_name  = "tracepoint",
+		.ml_meth  = (PyCFunction) pyrf__tracepoint,
+		.ml_flags = METH_VARARGS | METH_KEYWORDS,
+		.ml_doc	  = PyDoc_STR("Get tracepoint config.")
+	},
 	{ .ml_name = NULL, }
 };
 
Signature · 1 line
2.4.11
#12 lore raw

[tip:perf/core] perf python: Add perf.tracepoint method

perf python: Add perf.tracepoint method
Commit-ID:  1075fbb22f095c857930190e30fd3ae422d424b6
Gitweb:     http://git.kernel.org/tip/1075fbb22f095c857930190e30fd3ae422d424b6
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 10 Jul 2016 13:07:58 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 12 Jul 2016 16:17:54 -0300

perf python: Add perf.tracepoint method

To get id of the tracepoint from subsystem and name strings. The interface is:

id = perf.tracepoint(sys, name)

In case of error -1 is returned.

It will be used to get python tracepoint event's config value for tracepoint event.

---
 tools/perf/util/python.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
tools/perf/util/python.c +26 -0
index fc277e4..45fdd4a 100644
@@ -2,6 +2,7 @@
 #include <structmember.h>
 #include <inttypes.h>
 #include <poll.h>
+#include <linux/err.h>
 #include "evlist.h"
 #include "evsel.h"
 #include "event.h"
@@ -1076,7 +1077,32 @@ static struct {
 	{ .name = NULL, },
 };
 
+static PyObject *pyrf__tracepoint(struct pyrf_evsel *pevsel,
+				  PyObject *args, PyObject *kwargs)
+{
+	struct event_format *tp_format;
+	static char *kwlist[] = { "sys", "name", NULL };
+	char *sys  = NULL;
+	char *name = NULL;
+
+	if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|ss", kwlist,
+					 &sys, &name))
+		return NULL;
+
+	tp_format = trace_event__tp_format(sys, name);
+	if (IS_ERR(tp_format))
+		return PyInt_FromLong(-1);
+
+	return PyInt_FromLong(tp_format->id);
+}
+
 static PyMethodDef perf__methods[] = {
+	{
+		.ml_name  = "tracepoint",
+		.ml_meth  = (PyCFunction) pyrf__tracepoint,
+		.ml_flags = METH_VARARGS | METH_KEYWORDS,
+		.ml_doc	  = PyDoc_STR("Get tracepoint config.")
+	},
 	{ .ml_name = NULL, }
 };
 
author reviewer
#13 lore raw

PATCH 7/10 perf python: Add struct evsel into struct pyrf_event

In reply to Jiri Olsa

To be able to find out event configuration info
during sample parsing.

---
 tools/perf/util/python.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
tools/perf/util/python.c +9 -1
index 45fdd4a68721..dc7adaa3a02c 100644
@@ -48,6 +48,7 @@ PyMODINIT_FUNC initperf(void);
 
 struct pyrf_event {
 	PyObject_HEAD
+	struct perf_evsel *evsel;
 	struct perf_sample sample;
 	union perf_event   event;
 };
@@ -865,11 +866,18 @@ static PyObject *pyrf_evlist__read_on_cpu(struct pyrf_evlist *pevlist,
 	if (event != NULL) {
 		PyObject *pyevent = pyrf_event__new(event);
 		struct pyrf_event *pevent = (struct pyrf_event *)pyevent;
+		struct perf_evsel *evsel;
 
 		if (pyevent == NULL)
 			return PyErr_NoMemory();
 
-		err = perf_evlist__parse_sample(evlist, event, &pevent->sample);
+		evsel = perf_evlist__event2evsel(evlist, event);
+		if (!evsel)
+			return Py_None;
+
+		pevent->evsel = evsel;
+
+		err = perf_evsel__parse_sample(evsel, event, &pevent->sample);
 
 		/* Consume the even only after we parsed it out. */
 		perf_evlist__mmap_consume(evlist, cpu);
Signature · 1 line
2.4.11
#14 lore raw

[tip:perf/core] perf python: Add struct evsel into struct pyrf_event

perf python: Add struct evsel into struct pyrf_event
Commit-ID:  377f698db12150a1cf79987dca1d3990fa14a1f8
Gitweb:     http://git.kernel.org/tip/377f698db12150a1cf79987dca1d3990fa14a1f8
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 10 Jul 2016 13:07:59 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 12 Jul 2016 16:18:36 -0300

perf python: Add struct evsel into struct pyrf_event

To be able to find out event configuration info during sample parsing.

---
 tools/perf/util/python.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
tools/perf/util/python.c +9 -1
index 45fdd4a..dc7adaa 100644
@@ -48,6 +48,7 @@ PyMODINIT_FUNC initperf(void);
 
 struct pyrf_event {
 	PyObject_HEAD
+	struct perf_evsel *evsel;
 	struct perf_sample sample;
 	union perf_event   event;
 };
@@ -865,11 +866,18 @@ static PyObject *pyrf_evlist__read_on_cpu(struct pyrf_evlist *pevlist,
 	if (event != NULL) {
 		PyObject *pyevent = pyrf_event__new(event);
 		struct pyrf_event *pevent = (struct pyrf_event *)pyevent;
+		struct perf_evsel *evsel;
 
 		if (pyevent == NULL)
 			return PyErr_NoMemory();
 
-		err = perf_evlist__parse_sample(evlist, event, &pevent->sample);
+		evsel = perf_evlist__event2evsel(evlist, event);
+		if (!evsel)
+			return Py_None;
+
+		pevent->evsel = evsel;
+
+		err = perf_evsel__parse_sample(evsel, event, &pevent->sample);
 
 		/* Consume the even only after we parsed it out. */
 		perf_evlist__mmap_consume(evlist, cpu);
author reviewer
#15 lore raw

PATCH 8/10 perf python: Add support to resolve tracepoint fields

In reply to Jiri Olsa

Adding tp_getattro callback for sample event. It resolves tracepoint fields in runtime.

It's now possible to access tracepoint fields in normal fashion like hardcoded ones (see the example in the next patch).

---
 tools/perf/util/python.c | 92 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 92 insertions(+)
tools/perf/util/python.c +92 -0
index dc7adaa3a02c..d32f97033718 100644
@@ -290,6 +290,97 @@ static PyObject *pyrf_sample_event__repr(struct pyrf_event *pevent)
 	return ret;
 }
 
+static bool is_tracepoint(struct pyrf_event *pevent)
+{
+	return pevent->evsel->attr.type == PERF_TYPE_TRACEPOINT;
+}
+
+static int is_printable_array(char *p, unsigned int len)
+{
+	unsigned int i;
+
+	for (i = 0; i < len; i++) {
+		if (!isprint(p[i]) && !isspace(p[i]))
+			return 0;
+	}
+
+	return 1;
+}
+
+static PyObject*
+tracepoint_field(struct pyrf_event *pe, struct format_field *field)
+{
+	struct pevent *pevent = field->event->pevent;
+	void *data = pe->sample.raw_data;
+	PyObject *ret = NULL;
+	unsigned long long val;
+	unsigned int offset, len;
+
+	if (field->flags & FIELD_IS_ARRAY) {
+		offset = field->offset;
+		len    = field->size;
+		if (field->flags & FIELD_IS_DYNAMIC) {
+			val     = pevent_read_number(pevent, data + offset, len);
+			offset  = val;
+			len     = offset >> 16;
+			offset &= 0xffff;
+		}
+		if (field->flags & FIELD_IS_STRING &&
+		    is_printable_array(data + offset, len)) {
+			ret = PyString_FromString((char *)data + offset);
+		} else {
+			ret = PyByteArray_FromStringAndSize((const char *) data + offset, len);
+			field->flags &= ~FIELD_IS_STRING;
+		}
+	} else {
+		val = pevent_read_number(pevent, data + field->offset,
+					 field->size);
+		if (field->flags & FIELD_IS_POINTER)
+			ret = PyLong_FromUnsignedLong((unsigned long) val);
+		else if (field->flags & FIELD_IS_SIGNED)
+			ret = PyLong_FromLong((long) val);
+		else
+			ret = PyLong_FromUnsignedLong((unsigned long) val);
+	}
+
+	return ret;
+}
+
+static PyObject*
+get_tracepoint_field(struct pyrf_event *pevent, PyObject *attr_name)
+{
+	const char *str = PyString_AsString(PyObject_Str(attr_name));
+	struct perf_evsel *evsel = pevent->evsel;
+	struct format_field *field;
+
+	if (!evsel->tp_format) {
+		struct event_format *tp_format;
+
+		tp_format = trace_event__tp_format_id(evsel->attr.config);
+		if (!tp_format)
+			return NULL;
+
+		evsel->tp_format = tp_format;
+	}
+
+	field = pevent_find_any_field(evsel->tp_format, str);
+	if (!field)
+		return NULL;
+
+	return tracepoint_field(pevent, field);
+}
+
+static PyObject*
+pyrf_sample_event__getattro(struct pyrf_event *pevent, PyObject *attr_name)
+{
+	PyObject *obj = NULL;
+
+	if (is_tracepoint(pevent))
+		obj = get_tracepoint_field(pevent, attr_name);
+
+	return obj ?: PyObject_GenericGetAttr((PyObject *) pevent, attr_name);
+}
+
 static PyTypeObject pyrf_sample_event__type = {
 	PyVarObject_HEAD_INIT(NULL, 0)
 	.tp_name	= "perf.sample_event",
@@ -298,6 +389,7 @@ static PyTypeObject pyrf_sample_event__type = {
 	.tp_doc		= pyrf_sample_event__doc,
 	.tp_members	= pyrf_sample_event__members,
 	.tp_repr	= (reprfunc)pyrf_sample_event__repr,
+	.tp_getattro	= (getattrofunc) pyrf_sample_event__getattro,
 };
 
 static char pyrf_context_switch_event__doc[] = PyDoc_STR("perf context_switch event object.");
Signature · 1 line
2.4.11
#16 lore raw

[tip:perf/core] perf python: Add support to resolve tracepoint fields

perf python: Add support to resolve tracepoint fields
Commit-ID:  bae57e3825a3dded15f61cd20c6757d60ad6c712
Gitweb:     http://git.kernel.org/tip/bae57e3825a3dded15f61cd20c6757d60ad6c712
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 10 Jul 2016 13:08:00 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 12 Jul 2016 16:19:16 -0300

perf python: Add support to resolve tracepoint fields

Adding tp_getattro callback for sample event. It resolves tracepoint fields in runtime.

It's now possible to access tracepoint fields in normal fashion like hardcoded ones (see the example in the next patch).

---
 tools/perf/util/python.c | 92 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 92 insertions(+)
tools/perf/util/python.c +92 -0
index dc7adaa..d32f970 100644
@@ -290,6 +290,97 @@ static PyObject *pyrf_sample_event__repr(struct pyrf_event *pevent)
 	return ret;
 }
 
+static bool is_tracepoint(struct pyrf_event *pevent)
+{
+	return pevent->evsel->attr.type == PERF_TYPE_TRACEPOINT;
+}
+
+static int is_printable_array(char *p, unsigned int len)
+{
+	unsigned int i;
+
+	for (i = 0; i < len; i++) {
+		if (!isprint(p[i]) && !isspace(p[i]))
+			return 0;
+	}
+
+	return 1;
+}
+
+static PyObject*
+tracepoint_field(struct pyrf_event *pe, struct format_field *field)
+{
+	struct pevent *pevent = field->event->pevent;
+	void *data = pe->sample.raw_data;
+	PyObject *ret = NULL;
+	unsigned long long val;
+	unsigned int offset, len;
+
+	if (field->flags & FIELD_IS_ARRAY) {
+		offset = field->offset;
+		len    = field->size;
+		if (field->flags & FIELD_IS_DYNAMIC) {
+			val     = pevent_read_number(pevent, data + offset, len);
+			offset  = val;
+			len     = offset >> 16;
+			offset &= 0xffff;
+		}
+		if (field->flags & FIELD_IS_STRING &&
+		    is_printable_array(data + offset, len)) {
+			ret = PyString_FromString((char *)data + offset);
+		} else {
+			ret = PyByteArray_FromStringAndSize((const char *) data + offset, len);
+			field->flags &= ~FIELD_IS_STRING;
+		}
+	} else {
+		val = pevent_read_number(pevent, data + field->offset,
+					 field->size);
+		if (field->flags & FIELD_IS_POINTER)
+			ret = PyLong_FromUnsignedLong((unsigned long) val);
+		else if (field->flags & FIELD_IS_SIGNED)
+			ret = PyLong_FromLong((long) val);
+		else
+			ret = PyLong_FromUnsignedLong((unsigned long) val);
+	}
+
+	return ret;
+}
+
+static PyObject*
+get_tracepoint_field(struct pyrf_event *pevent, PyObject *attr_name)
+{
+	const char *str = PyString_AsString(PyObject_Str(attr_name));
+	struct perf_evsel *evsel = pevent->evsel;
+	struct format_field *field;
+
+	if (!evsel->tp_format) {
+		struct event_format *tp_format;
+
+		tp_format = trace_event__tp_format_id(evsel->attr.config);
+		if (!tp_format)
+			return NULL;
+
+		evsel->tp_format = tp_format;
+	}
+
+	field = pevent_find_any_field(evsel->tp_format, str);
+	if (!field)
+		return NULL;
+
+	return tracepoint_field(pevent, field);
+}
+
+static PyObject*
+pyrf_sample_event__getattro(struct pyrf_event *pevent, PyObject *attr_name)
+{
+	PyObject *obj = NULL;
+
+	if (is_tracepoint(pevent))
+		obj = get_tracepoint_field(pevent, attr_name);
+
+	return obj ?: PyObject_GenericGetAttr((PyObject *) pevent, attr_name);
+}
+
 static PyTypeObject pyrf_sample_event__type = {
 	PyVarObject_HEAD_INIT(NULL, 0)
 	.tp_name	= "perf.sample_event",
@@ -298,6 +389,7 @@ static PyTypeObject pyrf_sample_event__type = {
 	.tp_doc		= pyrf_sample_event__doc,
 	.tp_members	= pyrf_sample_event__members,
 	.tp_repr	= (reprfunc)pyrf_sample_event__repr,
+	.tp_getattro	= (getattrofunc) pyrf_sample_event__getattro,
 };
 
 static char pyrf_context_switch_event__doc[] = PyDoc_STR("perf context_switch event object.");
author reviewer
#17 lore raw

PATCH 9/10 perf python: Add tracepoint example

In reply to Jiri Olsa

To show how to open tracepoint and access its fields.

---
 tools/perf/python/tracepoint.py | 47 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
 create mode 100755 tools/perf/python/tracepoint.py
tools/perf/python/tracepoint.py new file +47 -0
new file mode 100755
index 000000000000..eb4dbed57de7
@@ -0,0 +1,47 @@
+#! /usr/bin/python
+# -*- python -*-
+# -*- coding: utf-8 -*-
+
+import perf
+
+class tracepoint(perf.evsel):
+    def __init__(self, sys, name):
+        config = perf.tracepoint(sys, name)
+        perf.evsel.__init__(self,
+                            type   = perf.TYPE_TRACEPOINT,
+                            config = config,
+                            freq = 0, sample_period = 1, wakeup_events = 1,
+                            sample_type = perf.SAMPLE_PERIOD | perf.SAMPLE_TID | perf.SAMPLE_CPU | perf.SAMPLE_RAW | perf.SAMPLE_TIME)
+
+def main():
+    tp      = tracepoint("sched", "sched_switch")
+    cpus    = perf.cpu_map()
+    threads = perf.thread_map(-1)
+
+    evlist = perf.evlist(cpus, threads)
+    evlist.add(tp)
+    evlist.open()
+    evlist.mmap()
+
+    while True:
+        evlist.poll(timeout = -1)
+        for cpu in cpus:
+            event = evlist.read_on_cpu(cpu)
+            if not event:
+                continue
+
+            if not isinstance(event, perf.sample_event):
+                continue
+
+            print "time %u prev_comm=%s prev_pid=%d prev_prio=%d prev_state=0x%x ==> next_comm=%s next_pid=%d next_prio=%d" % (
+                   event.sample_time,
+                   event.prev_comm,
+                   event.prev_pid,
+                   event.prev_prio,
+                   event.prev_state,
+                   event.next_comm,
+                   event.next_pid,
+                   event.next_prio)
+
+if __name__ == '__main__':
+    main()
Signature · 1 line
2.4.11
#18 lore raw

[tip:perf/core] perf python: Add tracepoint example

perf python: Add tracepoint example
Commit-ID:  9881d7df9dddef24e34949a4510245e156746c21
Gitweb:     http://git.kernel.org/tip/9881d7df9dddef24e34949a4510245e156746c21
Author:     Jiri Olsa <[email protected]>
AuthorDate: Sun, 10 Jul 2016 13:08:01 +0200
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Tue, 12 Jul 2016 16:23:35 -0300

perf python: Add tracepoint example

To show how to enable a tracepoint and access its fields.

Committer note:

Testing it:

  # ls -l /tmp/build/perf/python/perf.so
  -rwxrwxr-x. 1 acme acme 1563256 Jul 12 16:19 /tmp/build/perf/python/perf.so
  # export PYTHONPATH=/tmp/build/perf/python/
  # tools/perf/python/tracepoint.py 2> /dev/null | head -200 | tail -10
  time 76345337296548 prev_comm=swapper/0 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=tracepoint.py- next_pid=18479 next_prio=120
  time 76345338520479 prev_comm=gnome-shelln-b prev_pid=2186 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/1 next_pid=0 next_prio=120
  time 76345337309942 prev_comm=tracepoint.py- prev_pid=18479 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/0 next_pid=0 next_prio=120
  time 76345337312302 prev_comm=swapper/0 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=tracepoint.py- next_pid=18479 next_prio=120
  time 76345337324927 prev_comm=tracepoint.py- prev_pid=18479 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/0 next_pid=0 next_prio=120
  time 76345337327115 prev_comm=swapper/0 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=tracepoint.py- next_pid=18479 next_prio=120
  time 76345338621750 prev_comm=swapper/2 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=rcuos/2 next_pid=29 next_prio=120
  time 76345338607922 prev_comm=swapper/3 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=rcu_sched next_pid=7 next_prio=120
  time 76345337338817 prev_comm=tracepoint.py- prev_pid=18479 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/0 next_pid=0 next_prio=120
  time 76345338627156 prev_comm=swapper/1 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=head-terminal- next_pid=18480 next_prio=120
  #
  # strip /tmp/build/perf/python/perf.so
  # ls -l /tmp/build/perf/python/perf.so
  -rwxrwxr-x. 1 acme acme 319616 Jul 12 16:25 /tmp/build/perf/python/perf.so
---
 tools/perf/python/tracepoint.py | 47 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
tools/perf/python/tracepoint.py new file +47 -0
new file mode 100755
index 0000000..eb4dbed
@@ -0,0 +1,47 @@
+#! /usr/bin/python
+# -*- python -*-
+# -*- coding: utf-8 -*-
+
+import perf
+
+class tracepoint(perf.evsel):
+    def __init__(self, sys, name):
+        config = perf.tracepoint(sys, name)
+        perf.evsel.__init__(self,
+                            type   = perf.TYPE_TRACEPOINT,
+                            config = config,
+                            freq = 0, sample_period = 1, wakeup_events = 1,
+                            sample_type = perf.SAMPLE_PERIOD | perf.SAMPLE_TID | perf.SAMPLE_CPU | perf.SAMPLE_RAW | perf.SAMPLE_TIME)
+
+def main():
+    tp      = tracepoint("sched", "sched_switch")
+    cpus    = perf.cpu_map()
+    threads = perf.thread_map(-1)
+
+    evlist = perf.evlist(cpus, threads)
+    evlist.add(tp)
+    evlist.open()
+    evlist.mmap()
+
+    while True:
+        evlist.poll(timeout = -1)
+        for cpu in cpus:
+            event = evlist.read_on_cpu(cpu)
+            if not event:
+                continue
+
+            if not isinstance(event, perf.sample_event):
+                continue
+
+            print "time %u prev_comm=%s prev_pid=%d prev_prio=%d prev_state=0x%x ==> next_comm=%s next_pid=%d next_prio=%d" % (
+                   event.sample_time,
+                   event.prev_comm,
+                   event.prev_pid,
+                   event.prev_prio,
+                   event.prev_state,
+                   event.next_comm,
+                   event.next_pid,
+                   event.next_prio)
+
+if __name__ == '__main__':
+    main()
author reviewer
#19 lore raw

PATCH 10/10 perf script python: Fix string vs byte array resolving

In reply to Jiri Olsa

Jirka reported that python code returns all arrays as strings. This makes impossible to get all items for byte array tracepoint field containing 0x00 value item.

Fixing this by scanning full length of the array and returning it as PyByteArray object in case non printable byte is found.

---
 .../util/scripting-engines/trace-event-python.c    | 34 ++++++++++++++++++----
 1 file changed, 28 insertions(+), 6 deletions(-)
tools/perf/util/scripting-engines/trace-event-python.c +28 -6
index ff134700bf30..75e9790ebb96 100644
@@ -386,6 +386,16 @@ exit:
 	return pylist;
 }
 
+static int is_printable_array(char *p, unsigned int len)
+{
+	unsigned int i;
+
+	for (i = 0; i < len; i++)
+		if (!isprint(p[i]) && !isspace(p[i]))
+			return 0;
+
+	return 1;
+}
 
 static void python_process_tracepoint(struct perf_sample *sample,
 				      struct perf_evsel *evsel,
@@ -457,14 +467,26 @@ static void python_process_tracepoint(struct perf_sample *sample,
 		pydict_set_item_string_decref(dict, "common_callchain", callchain);
 	}
 	for (field = event->format.fields; field; field = field->next) {
-		if (field->flags & FIELD_IS_STRING) {
-			int offset;
+		unsigned int offset, len;
+		unsigned long long val;
+
+		if (field->flags & FIELD_IS_ARRAY) {
+			offset = field->offset;
+			len    = field->size;
 			if (field->flags & FIELD_IS_DYNAMIC) {
-				offset = *(int *)(data + field->offset);
+				val     = pevent_read_number(scripting_context->pevent,
+							     data + offset, len);
+				offset  = val;
+				len     = offset >> 16;
 				offset &= 0xffff;
-			} else
-				offset = field->offset;
-			obj = PyString_FromString((char *)data + offset);
+			}
+			if (field->flags & FIELD_IS_STRING &&
+			    is_printable_array(data + offset, len)) {
+				obj = PyString_FromString((char *) data + offset);
+			} else {
+				obj = PyByteArray_FromStringAndSize((const char *) data + offset, len);
+				field->flags &= ~FIELD_IS_STRING;
+			}
 		} else { /* FIELD_IS_NUMERIC */
 			obj = get_field_numeric_entry(event, field, data);
 		}
Signature · 1 line
2.4.11
#20 lore raw

Re: [PATCH 10/10] perf script python: Fix string vs byte array resolving

Quoted 20 lines · On Sun, 10 Jul 2016 13:08:02 +0200 Jiri Olsa <[email protected]> wrote:

Jirka reported that python code returns all arrays as strings. This makes impossible to get all items for byte array tracepoint field containing 0x00 value item.

Fixing this by scanning full length of the array and returning it as PyByteArray object in case non printable byte is found.

---
 .../util/scripting-engines/trace-event-python.c    | 34 ++++++++++++++++++----
 1 file changed, 28 insertions(+), 6 deletions(-)
tools/perf/util/scripting-engines/trace-event-python.c +0 -0
index ff134700bf30..75e9790ebb96 100644

tools/perf/util/scripting-engines/trace-event-python.c exit: · line 395

Quoted 10 lines
tools/perf/util/scripting-engines/trace-event-python.c +7 -0
@@ -386,6 +386,16 @@ exit:
 	return pylist;
 }
 
+static int is_printable_array(char *p, unsigned int len)
+{
+	unsigned int i;
+
+	for (i = 0; i < len; i++)
+		if (!isprint(p[i]) && !isspace(p[i]))
+			return 0;

I believe this will return the wrong result if you pass in a string that has '\0' in it. Try it out:

	char a[] = "hello";

	printf("a=%s is %d\n", a, is_printable_array(a, sizeof(a)));

and see what you get?

-- Steve

Quoted 39 lines
tools/perf/util/scripting-engines/trace-event-python.c +21 -6
+
+	return 1;
+}
 
 static void python_process_tracepoint(struct perf_sample *sample,
 				      struct perf_evsel *evsel,
@@ -457,14 +467,26 @@ static void python_process_tracepoint(struct perf_sample *sample,
 		pydict_set_item_string_decref(dict, "common_callchain", callchain);
 	}
 	for (field = event->format.fields; field; field = field->next) {
-		if (field->flags & FIELD_IS_STRING) {
-			int offset;
+		unsigned int offset, len;
+		unsigned long long val;
+
+		if (field->flags & FIELD_IS_ARRAY) {
+			offset = field->offset;
+			len    = field->size;
 			if (field->flags & FIELD_IS_DYNAMIC) {
-				offset = *(int *)(data + field->offset);
+				val     = pevent_read_number(scripting_context->pevent,
+							     data + offset, len);
+				offset  = val;
+				len     = offset >> 16;
 				offset &= 0xffff;
-			} else
-				offset = field->offset;
-			obj = PyString_FromString((char *)data + offset);
+			}
+			if (field->flags & FIELD_IS_STRING &&
+			    is_printable_array(data + offset, len)) {
+				obj = PyString_FromString((char *) data + offset);
+			} else {
+				obj = PyByteArray_FromStringAndSize((const char *) data + offset, len);
+				field->flags &= ~FIELD_IS_STRING;
+			}
 		} else { /* FIELD_IS_NUMERIC */
 			obj = get_field_numeric_entry(event, field, data);
 		}
#21 lore raw

Re: [PATCH 10/10] perf script python: Fix string vs byte array resolving

Quoted 37 lines · Steven Rostedt
Quoted 20 lines · On Sun, 10 Jul 2016 13:08:02 +0200 Jiri Olsa <[email protected]> wrote:

Jirka reported that python code returns all arrays as strings. This makes impossible to get all items for byte array tracepoint field containing 0x00 value item.

Fixing this by scanning full length of the array and returning it as PyByteArray object in case non printable byte is found.

---
 .../util/scripting-engines/trace-event-python.c    | 34 ++++++++++++++++++----
 1 file changed, 28 insertions(+), 6 deletions(-)
tools/perf/util/scripting-engines/trace-event-python.c +0 -0
index ff134700bf30..75e9790ebb96 100644

tools/perf/util/scripting-engines/trace-event-python.c exit: · line 395

Quoted 10 lines
tools/perf/util/scripting-engines/trace-event-python.c +7 -0
@@ -386,6 +386,16 @@ exit:
 	return pylist;
 }
 
+static int is_printable_array(char *p, unsigned int len)
+{
+	unsigned int i;
+
+	for (i = 0; i < len; i++)
+		if (!isprint(p[i]) && !isspace(p[i]))
+			return 0;

I believe this will return the wrong result if you pass in a string that has '\0' in it. Try it out:

Quoted 5 lines
	char a[] = "hello";

	printf("a=%s is %d\n", a, is_printable_array(a, sizeof(a)));

and see what you get?

ugh, forgot the last 0 byte.. how about attached change?

I wonder it'd be less pain to present all arrays as PyByteArray, it might be no difference for python users anyway ;-)

thanks,
jirka

---

tools/perf/util/scripting-engines/trace-event-python.c +4 -2
index 75e9790ebb96..9f810e6c739a 100644
@@ -390,11 +390,13 @@ static int is_printable_array(char *p, unsigned int len)
 {
 	unsigned int i;
 
-	for (i = 0; i < len; i++)
+	/* Exclude the last zero byte of string.. */
+	for (i = 0; i < len - 1; i++)
 		if (!isprint(p[i]) && !isspace(p[i]))
 			return 0;
 
-	return 1;
+	/* ... and check its presence in the last byte. */
+	return p[i] == 0;
 }
 
 static void python_process_tracepoint(struct perf_sample *sample,
#22 lore raw

Re: [PATCH 10/10] perf script python: Fix string vs byte array resolving

Quoted 10 lines · On Tue, 12 Jul 2016 10:11:56 +0200 Jiri Olsa <[email protected]> wrote:

---

tools/perf/util/scripting-engines/trace-event-python.c +0 -1
index 75e9790ebb96..9f810e6c739a 100644
@@ -390,11 +390,13 @@ static int is_printable_array(char *p, unsigned int len)
 {
 	unsigned int i;
 
-	for (i = 0; i < len; i++)

tools/perf/util/scripting-engines/trace-event-python.c static int is_printable_array(char *p, unsigned int len) · line 401

Quoted 10 lines
tools/perf/util/scripting-engines/trace-event-python.c +4 -1
+	/* Exclude the last zero byte of string.. */
+	for (i = 0; i < len - 1; i++)
 		if (!isprint(p[i]) && !isspace(p[i]))
 			return 0;
 
-	return 1;
+	/* ... and check its presence in the last byte. */
+	return p[i] == 0;
 }
 

Of course this fails on:

	is_printable_array(a, strlen(a));

because the last byte wont be 0. What about:

	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			break;

	return i == len ||
		(i == len - 1 && p[i] == 0);

-- Steve

#23 lore raw

Re: [PATCH 10/10] perf script python: Fix string vs byte array resolving

Quoted 27 lines · Steven Rostedt
Quoted 10 lines · On Tue, 12 Jul 2016 10:11:56 +0200 Jiri Olsa <[email protected]> wrote:

---

tools/perf/util/scripting-engines/trace-event-python.c +0 -1
index 75e9790ebb96..9f810e6c739a 100644
@@ -390,11 +390,13 @@ static int is_printable_array(char *p, unsigned int len)
 {
 	unsigned int i;
 
-	for (i = 0; i < len; i++)

tools/perf/util/scripting-engines/trace-event-python.c static int is_printable_array(char *p, unsigned int len) · line 401

Quoted 10 lines
tools/perf/util/scripting-engines/trace-event-python.c +4 -1
+	/* Exclude the last zero byte of string.. */
+	for (i = 0; i < len - 1; i++)
 		if (!isprint(p[i]) && !isspace(p[i]))
 			return 0;
 
-	return 1;
+	/* ... and check its presence in the last byte. */
+	return p[i] == 0;
 }
 

Of course this fails on:

Quoted 1 line
	is_printable_array(a, strlen(a));

hum, so string is not passed to trace buffer with 0 at the end?

I'll check the option of passing everything as PyByteArray. If it's the same for user, we'll have simpler code in here.

Quoted 9 lines

because the last byte wont be 0. What about:

	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			break;

	return i == len ||
		(i == len - 1 && p[i] == 0);

thanks,
jirka

#24 lore raw

Re: [PATCH 10/10] perf script python: Fix string vs byte array resolving

Quoted 1 line · On Tue, 12 Jul 2016 14:52:45 +0200 Jiri Olsa <[email protected]> wrote:

hum, so string is not passed to trace buffer with 0 at the end?

strlen() returns only the length of the string (minus the \0 at the end).

Quoted 13 lines

I'll check the option of passing everything as PyByteArray. If it's the same for user, we'll have simpler code in here.

Quoted 9 lines

because the last byte wont be 0. What about:

	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			break;

	return i == len ||
		(i == len - 1 && p[i] == 0);  

Or you can do:

	if (p[len - 1] == 0)
		len--;

	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			return 0;
	return 1;

-- Steve

#25 lore raw

Re: [PATCH 10/10] perf script python: Fix string vs byte array resolving

Quoted 8 lines · Steven Rostedt
Quoted 1 line · On Tue, 12 Jul 2016 14:52:45 +0200 Jiri Olsa <[email protected]> wrote:

hum, so string is not passed to trace buffer with 0 at the end?

strlen() returns only the length of the string (minus the \0 at the end).

yea, is_printable_array is called here on tracepoint data that's why I asked.. is 0 part of the string data when stored within tracepoint?

Quoted 17 lines
Quoted 14 lines

I'll check the option of passing everything as PyByteArray. If it's the same for user, we'll have simpler code in here.

Quoted 9 lines

because the last byte wont be 0. What about:

	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			break;

	return i == len ||
		(i == len - 1 && p[i] == 0);  

Or you can do:

Quoted 7 lines
	if (p[len - 1] == 0)
		len--;

	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			return 0;
	return 1;

yep, seems good.. thanks

jirka

#26 lore raw

Re: [PATCH 10/10] perf script python: Fix string vs byte array resolving

Quoted 5 lines · On Tue, 12 Jul 2016 15:35:52 +0200 Jiri Olsa <[email protected]> wrote:
Quoted 4 lines · Steven Rostedt

On Tue, 12 Jul 2016 14:52:45 +0200
Jiri Olsa <[email protected]> wrote:

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

hum, so string is not passed to trace buffer with 0 at the end?

strlen() returns only the length of the string (minus the \0 at the end).

yea, is_printable_array is called here on tracepoint data that's why I asked.. is 0 part of the string data when stored within tracepoint?

Good question. It may or may not (haven't looked at the code), but we probably want to make sure that the function is robust enough for either case.

Quoted 21 lines
Quoted 21 lines
Quoted 15 lines

I'll check the option of passing everything as PyByteArray. If it's the same for user, we'll have simpler code in here.

Quoted 9 lines

because the last byte wont be 0. What about:

	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			break;

	return i == len ||
		(i == len - 1 && p[i] == 0);    

Or you can do:

	if (p[len - 1] == 0)
		len--;
Quoted 7 lines
Quoted 4 lines
	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			return 0;
	return 1;  

yep, seems good.. thanks

OK, great!

-- Steve

#27 lore raw

Re: [PATCH 10/10] perf script python: Fix string vs byte array resolving

Em Tue, Jul 12, 2016 at 09:48:36AM -0400, Steven Rostedt escreveu:

Quoted 13 lines
Quoted 10 lines · On Tue, 12 Jul 2016 15:35:52 +0200 Jiri Olsa <[email protected]> wrote:
Quoted 8 lines · Steven Rostedt
Quoted 1 line · On Tue, 12 Jul 2016 14:52:45 +0200 Jiri Olsa <[email protected]> wrote:

hum, so string is not passed to trace buffer with 0 at the end?

strlen() returns only the length of the string (minus the \0 at the end).

Quoted 7 lines
Quoted 3 lines

yea, is_printable_array is called here on tracepoint data that's why I asked.. is 0 part of the string data when stored within tracepoint?

Good question. It may or may not (haven't looked at the code), but we probably want to make sure that the function is robust enough for either case.

Ok, applied and tested up to patch 9, waiting for this discussion to come to an end to consider 10.

- Arnaldo

Quoted 32 lines
Quoted 21 lines
Quoted 21 lines
Quoted 15 lines

I'll check the option of passing everything as PyByteArray. If it's the same for user, we'll have simpler code in here.

Quoted 9 lines

because the last byte wont be 0. What about:

	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			break;

	return i == len ||
		(i == len - 1 && p[i] == 0);    

Or you can do:

	if (p[len - 1] == 0)
		len--;
Quoted 7 lines
Quoted 4 lines
	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			return 0;
	return 1;  

yep, seems good.. thanks

OK, great!

-- Steve

#28 lore raw

Re: [PATCH 10/10] perf script python: Fix string vs byte array resolving

Quoted 18 lines · On Tue, 12 Jul 2016 16:27:16 -0300 Arnaldo Carvalho de Melo <[email protected]>…

Em Tue, Jul 12, 2016 at 09:48:36AM -0400, Steven Rostedt escreveu:

Quoted 17 lines
Quoted 13 lines · On Tue, 12 Jul 2016 15:35:52 +0200 Jiri Olsa <[email protected]> wrote:
Quoted 8 lines · Steven Rostedt
Quoted 1 line · On Tue, 12 Jul 2016 14:52:45 +0200 Jiri Olsa <[email protected]> wrote:

hum, so string is not passed to trace buffer with 0 at the end?

strlen() returns only the length of the string (minus the \0 at the end).

yea, is_printable_array is called here on tracepoint data that's why I asked.. is 0 part of the string data when stored within tracepoint?

Quoted 25 lines
Quoted 3 lines

Good question. It may or may not (haven't looked at the code), but we probably want to make sure that the function is robust enough for either case.

Ok, applied and tested up to patch 9, waiting for this discussion to come to an end to consider 10.

- Arnaldo

Quoted 14 lines
Quoted 5 lines

Or you can do:

Quoted 4 lines
	if (p[len - 1] == 0)
		len--;
Quoted 7 lines
Quoted 4 lines
	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			return 0;
	return 1;    

yep, seems good.. thanks

OK, great!

I think Jiri is going to implement the above.

-- Steve

#29 lore raw

Re: [PATCH 10/10] perf script python: Fix string vs byte array resolving

Quoted 42 lines · Steven Rostedt
Quoted 39 lines · On Tue, 12 Jul 2016 16:27:16 -0300 Arnaldo Carvalho de Melo <[email protected]>…

Em Tue, Jul 12, 2016 at 09:48:36AM -0400, Steven Rostedt escreveu:

Quoted 13 lines
Quoted 10 lines · On Tue, 12 Jul 2016 15:35:52 +0200 Jiri Olsa <[email protected]> wrote:
Quoted 8 lines · Steven Rostedt
Quoted 1 line · On Tue, 12 Jul 2016 14:52:45 +0200 Jiri Olsa <[email protected]> wrote:

hum, so string is not passed to trace buffer with 0 at the end?

strlen() returns only the length of the string (minus the \0 at the end).

Quoted 7 lines
Quoted 3 lines

yea, is_printable_array is called here on tracepoint data that's why I asked.. is 0 part of the string data when stored within tracepoint?

Good question. It may or may not (haven't looked at the code), but we probably want to make sure that the function is robust enough for either case.

Ok, applied and tested up to patch 9, waiting for this discussion to come to an end to consider 10.

- Arnaldo

Quoted 10 lines
Quoted 10 lines

Or you can do:

Quoted 8 lines
	if (p[len - 1] == 0)
		len--;

	for (i = 0; i < len; i++)
		if (!isprint(p[i]) && !isspace(p[i]))
			return 0;
	return 1;    
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

yep, seems good.. thanks

OK, great!

I think Jiri is going to implement the above.

yep ;-)

jirka