Python improvements for a real use of parse_events
Ian Rogers
reviewerv2 cover Python improvements for a real use of parse_events
While parse_events access in python was added, it wasn't used by any python script. In enabling this for the tracepoint.py script a number of latent bugs and necessary improvements were discovered.
v2: Fix a whitespace issue on the evlist.config patch spotted by
Howard and add his reviewed-by tags.
Ian Rogers (11):
perf debug: Avoid stack overflow in recursive error message perf evlist: Add success path to evlist__create_syswide_maps perf evsel: tp_format accessing improvements
perf python: Add evlist enable and disable methods
perf python: Add member access to a number of evsel variables perf python: Add optional cpus and threads arguments to parse_events perf python: Update ungrouped evsel leader in clone
perf python: Avoid duplicated code in get_tracepoint_field perf python: Add evlist all_cpus accessor
perf python: Add evlist.config to set up record options perf python tracepoint: Switch to using parse_events
tools/perf/python/tracepoint.py | 23 +++--- tools/perf/util/debug.c | 2 +- tools/perf/util/evlist.c | 13 ++-- tools/perf/util/evsel.c | 16 +++- tools/perf/util/python.c | 127 ++++++++++++++++++++++++++++---- 5 files changed, 145 insertions(+), 36 deletions(-)
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerPATCH v2 1/11 perf debug: Avoid stack overflow in recursive error message
In debug_file, pr_warning_once is called on error. As that function calls debug_file the function will yield a stack overflow. Switch the location of the call so the recursion is avoided.
--- tools/perf/util/debug.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/debug.c +1 -1
@@ -46,8 +46,8 @@ int debug_type_profile;FILE *debug_file (void ) {if (!_debug_file) { -pr_warning_once ("debug_file not set" );debug_set_file (stderr); +pr_warning_once ("debug_file not set" ); }return _debug_file; }
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v2 01/11] perf debug: Avoid stack overflow in recursive error message
Quoted 6 lines · Ian Rogers
In
debug_file,pr_warning_onceis called on error. As that function callsdebug_filethe function will yield a stack overflow. Switch the location of the call so the recursion is avoided.Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Good to add this so that stable picks it:
- Arnaldo
Quoted 20 lines
--- tools/perf/util/debug.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/perf/util/debug.c +1
-1@@ -46,8 +46,8 @@ int debug_type_profile;FILE *debug_file (void ) {if (!_debug_file) { -pr_warning_once ("debug_file not set" );debug_set_file (stderr); +pr_warning_once ("debug_file not set" ); }return _debug_file; }Signature · 1 line
2.48.1.711.g2feabab25a-goog
Re: [PATCH v2 01/11] perf debug: Avoid stack overflow in recursive error message
Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 6 lines · Ian Rogers
In
debug_file,pr_warning_onceis called on error. As that function callsdebug_filethe function will yield a stack overflow. Switch the location of the call so the recursion is avoided.Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>Good to add this so that stable picks it:
Fixes ec49230cf6dda704 ("perf debug: Expose debug file")
Forgot to add:
- Arnaldo
Quoted 21 lines
Quoted 21 lines
--- tools/perf/util/debug.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/perf/util/debug.c +1
-1@@ -46,8 +46,8 @@ int debug_type_profile;FILE *debug_file (void ) {if (!_debug_file) { -pr_warning_once ("debug_file not set" );debug_set_file (stderr); +pr_warning_once ("debug_file not set" ); }return _debug_file; }Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerPATCH v2 2/11 perf evlist: Add success path to evlist__create_syswide_maps
In reply to Ian RogersOver various refactorings evlist__create_syswide_maps has been made to only ever return with -ENOMEM. Fix this so that when perf_evlist__set_maps is successfully called, 0 is returned.
--- tools/perf/util/evlist.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
tools/perf/util/evlist.c +6 -7
@@ -1373,19 +1373,18 @@ static int evlist__create_syswide_maps(struct evlist *evlist)*/ cpus =perf_cpu_map__new_online_cpus ();if (!cpus) - goto out; +return -ENOMEM ; threads =perf_thread_map__new_dummy (); -if (!threads) - goto out_put; +if (!threads) { +perf_cpu_map__put (cpus); +return -ENOMEM ; + }perf_evlist__set_maps (&evlist->core, cpus, threads); -perf_thread_map__put (threads); -out_put:perf_cpu_map__put (cpus); -out: -return -ENOMEM ; +return 0 ; }int evlist__open (struct evlist *evlist)
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Re: [PATCH v2 02/11] perf evlist: Add success path to evlist__create_syswide_maps
Quoted 3 lines · Ian Rogers
Over various refactorings
evlist__create_syswide_mapshas been made to only ever return with-ENOMEM. Fix this so that whenperf_evlist__set_mapsis successfully called, 0 is returned.
I think this was when the problem was introduced:
- Arnaldo
Quoted 38 lines
Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/evlist.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)tools/perf/util/evlist.c +6
-7@@ -1373,19 +1373,18 @@ static int evlist__create_syswide_maps(struct evlist *evlist)*/ cpus =perf_cpu_map__new_online_cpus ();if (!cpus) - goto out; +return -ENOMEM ; threads =perf_thread_map__new_dummy (); -if (!threads) - goto out_put; +if (!threads) { +perf_cpu_map__put (cpus); +return -ENOMEM ; + }perf_evlist__set_maps (&evlist->core, cpus, threads); -perf_thread_map__put (threads); -out_put:perf_cpu_map__put (cpus); -out: -return -ENOMEM ; +return 0 ; }int evlist__open (struct evlist *evlist)Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerPATCH v2 3/11 perf evsel: tp_format accessing improvements
In reply to Ian RogersEnsure evsel__clone copies the tp_sys and tp_name variables. In evsel__tp_format, if tp_sys isn't set, use the config value to find the tp_format. This succeeds in python code where pyrf__tracepoint has already found the format.
--- tools/perf/util/evsel.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)
tools/perf/util/evsel.c +15 -1
@@ -521,6 +521,16 @@ struct evsel *evsel__clone(struct evsel *dest, struct evsel *orig) } evsel->cgrp =cgroup__get (orig->cgrp);#ifdef HAVE_LIBTRACEEVENT +if (orig->tp_sys) { + evsel->tp_sys =strdup (orig->tp_sys); +if (evsel->tp_sys ==NULL ) + goto out_err; + } +if (orig->tp_name) { + evsel->tp_name =strdup (orig->tp_name); +if (evsel->tp_name ==NULL ) + goto out_err; + } evsel->tp_format = orig->tp_format;#endif evsel->handler = orig->handler;
@@ -644,7 +654,11 @@ struct tep_event *evsel__tp_format(struct evsel *evsel)if (evsel->core.attr.type !=PERF_TYPE_TRACEPOINT )return NULL ; - tp_format =trace_event__tp_format (evsel->tp_sys, evsel->tp_name); +if (!evsel->tp_sys) + tp_format =trace_event__tp_format_id (evsel->core.attr.config); +else + tp_format =trace_event__tp_format (evsel->tp_sys, evsel->tp_name); +if (IS_ERR (tp_format)) {int err = -PTR_ERR (evsel->tp_format);
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Re: [PATCH v2 03/11] perf evsel: tp_format accessing improvements
Quoted 4 lines · Ian Rogers
Ensure
evsel__clonecopies thetp_sysandtp_namevariables. Inevsel__tp_format, iftp_sysisn't set, use the config value to find thetp_format. This succeeds in python code wherepyrf__tracepointhas already found the format.
Here those two fields were introduced but evsel__clone() wasn't update to clone those:
- Arnaldo
Quoted 42 lines
Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/evsel.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)tools/perf/util/evsel.c +15
-1@@ -521,6 +521,16 @@ struct evsel *evsel__clone(struct evsel *dest, struct evsel *orig) } evsel->cgrp =cgroup__get (orig->cgrp);#ifdef HAVE_LIBTRACEEVENT +if (orig->tp_sys) { + evsel->tp_sys =strdup (orig->tp_sys); +if (evsel->tp_sys ==NULL ) + goto out_err; + } +if (orig->tp_name) { + evsel->tp_name =strdup (orig->tp_name); +if (evsel->tp_name ==NULL ) + goto out_err; + } evsel->tp_format = orig->tp_format;#endif evsel->handler = orig->handler;@@ -644,7 +654,11 @@ struct tep_event *evsel__tp_format(struct evsel *evsel)if (evsel->core.attr.type !=PERF_TYPE_TRACEPOINT )return NULL ; - tp_format =trace_event__tp_format (evsel->tp_sys, evsel->tp_name); +if (!evsel->tp_sys) + tp_format =trace_event__tp_format_id (evsel->core.attr.config); +else + tp_format =trace_event__tp_format (evsel->tp_sys, evsel->tp_name); +if (IS_ERR (tp_format)) {int err = -PTR_ERR (evsel->tp_format);Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerPATCH v2 4/11 perf python: Add evlist enable and disable methods
In reply to Ian RogersBy default the evsels from parse_events will be disabled. Add access to the evlist functions so they can be enabled/disabled.
--- tools/perf/util/python.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
tools/perf/util/python.c +26 -0
@@ -1028,6 +1028,20 @@ static PyObject *pyrf_evlist__open(struct pyrf_evlist *pevlist,return Py_None; } +static PyObject *pyrf_evlist__disable (struct pyrf_evlist *pevlist) +{ +evlist__disable (&pevlist->evlist); +Py_INCREF (Py_None); +return Py_None; +} + +static PyObject *pyrf_evlist__enable (struct pyrf_evlist *pevlist) +{ +evlist__enable (&pevlist->evlist); +Py_INCREF (Py_None); +return Py_None; +} +static PyMethodDef pyrf_evlist__methods[] = { { .ml_name ="mmap" ,
@@ -1065,6 +1079,18 @@ static PyMethodDef pyrf_evlist__methods[] = { .ml_flags = METH_VARARGS | METH_KEYWORDS, .ml_doc = PyDoc_STR("reads an event.") }, + { + .ml_name = "disable", + .ml_meth = (PyCFunction)pyrf_evlist__disable, + .ml_flags = METH_NOARGS, + .ml_doc = PyDoc_STR("Disable the evsels in the evlist.") + }, + { + .ml_name = "enable", + .ml_meth = (PyCFunction)pyrf_evlist__enable, + .ml_flags = METH_NOARGS, + .ml_doc = PyDoc_STR("Enable the evsels in the evlist.") + }, { .ml_name = NULL, } };
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Re: [PATCH v2 04/11] perf python: Add evlist enable and disable methods
Quoted 2 lines · Ian Rogers
By default the evsels from
parse_eventswill be disabled. Add access to the evlist functions so they can be enabled/disabled.
- Arnaldo
Quoted 52 lines
Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/python.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)tools/perf/util/python.c +26
-0@@ -1028,6 +1028,20 @@ static PyObject *pyrf_evlist__open(struct pyrf_evlist *pevlist,return Py_None; } +static PyObject *pyrf_evlist__disable (struct pyrf_evlist *pevlist) +{ +evlist__disable (&pevlist->evlist); +Py_INCREF (Py_None); +return Py_None; +} + +static PyObject *pyrf_evlist__enable (struct pyrf_evlist *pevlist) +{ +evlist__enable (&pevlist->evlist); +Py_INCREF (Py_None); +return Py_None; +} +static PyMethodDef pyrf_evlist__methods[] = { { .ml_name ="mmap" ,@@ -1065,6 +1079,18 @@ static PyMethodDef pyrf_evlist__methods[] = { .ml_flags = METH_VARARGS | METH_KEYWORDS, .ml_doc = PyDoc_STR("reads an event.") }, + { + .ml_name = "disable", + .ml_meth = (PyCFunction)pyrf_evlist__disable, + .ml_flags = METH_NOARGS, + .ml_doc = PyDoc_STR("Disable the evsels in the evlist.") + }, + { + .ml_name = "enable", + .ml_meth = (PyCFunction)pyrf_evlist__enable, + .ml_flags = METH_NOARGS, + .ml_doc = PyDoc_STR("Enable the evsels in the evlist.") + }, { .ml_name = NULL, } };Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerPATCH v2 5/11 perf python: Add member access to a number of evsel variables
In reply to Ian RogersMost variables are part of the perf_event_attr, so that they may be queried and modified.
--- tools/perf/util/python.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
tools/perf/util/python.c +23 -0
@@ -811,6 +811,28 @@ static PyMethodDef pyrf_evsel__methods[] = { { .ml_name = NULL, } }; +#define evsel_member_def (member, ptype, help) \ + { #member, ptype, \ + offsetof(struct pyrf_evsel, evsel.member), \ + 0, help } + +#define evsel_attr_member_def (member, ptype, help) \ + { #member, ptype, \ + offsetof(struct pyrf_evsel, evsel.core.attr.member), \ + 0, help } + +static PyMemberDef pyrf_evsel__members[] = { +evsel_member_def (tracking,T_BOOL ,"tracking event." ), +evsel_attr_member_def (type,T_UINT ,"attribute type." ), +evsel_attr_member_def (size,T_UINT ,"attribute size." ), +evsel_attr_member_def (config,T_ULONGLONG ,"attribute config." ), +evsel_attr_member_def (sample_period,T_ULONGLONG ,"attribute sample_period." ), +evsel_attr_member_def (sample_type,T_ULONGLONG ,"attribute sample_type." ), +evsel_attr_member_def (read_format,T_ULONGLONG ,"attribute read_format." ), +evsel_attr_member_def (wakeup_events,T_UINT ,"attribute wakeup_events." ), + { .name =NULL , }, +}; +static const char pyrf_evsel__doc[] =PyDoc_STR ("perf event selector list object." );static PyTypeObject pyrf_evsel__type = {
@@ -820,6 +842,7 @@ static PyTypeObject pyrf_evsel__type = { .tp_dealloc = (destructor)pyrf_evsel__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_evsel__doc, + .tp_members = pyrf_evsel__members, .tp_methods = pyrf_evsel__methods, .tp_init = (initproc)pyrf_evsel__init, .tp_str = pyrf_evsel__str,
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Re: [PATCH v2 05/11] perf python: Add member access to a number of evsel variables
Quoted 4 lines · Ian Rogers
Most variables are part of the
perf_event_attr, so that they may be queried and modified.Reviewed-by Howard Chu <[email protected]>
- Arnaldo
Quoted 49 lines
Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/python.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)tools/perf/util/python.c +23
-0@@ -811,6 +811,28 @@ static PyMethodDef pyrf_evsel__methods[] = { { .ml_name = NULL, } }; +#define evsel_member_def (member, ptype, help) \ + { #member, ptype, \ + offsetof(struct pyrf_evsel, evsel.member), \ + 0, help } + +#define evsel_attr_member_def (member, ptype, help) \ + { #member, ptype, \ + offsetof(struct pyrf_evsel, evsel.core.attr.member), \ + 0, help } + +static PyMemberDef pyrf_evsel__members[] = { +evsel_member_def (tracking,T_BOOL ,"tracking event." ), +evsel_attr_member_def (type,T_UINT ,"attribute type." ), +evsel_attr_member_def (size,T_UINT ,"attribute size." ), +evsel_attr_member_def (config,T_ULONGLONG ,"attribute config." ), +evsel_attr_member_def (sample_period,T_ULONGLONG ,"attribute sample_period." ), +evsel_attr_member_def (sample_type,T_ULONGLONG ,"attribute sample_type." ), +evsel_attr_member_def (read_format,T_ULONGLONG ,"attribute read_format." ), +evsel_attr_member_def (wakeup_events,T_UINT ,"attribute wakeup_events." ), + { .name =NULL , }, +}; +static const char pyrf_evsel__doc[] =PyDoc_STR ("perf event selector list object." );static PyTypeObject pyrf_evsel__type = {@@ -820,6 +842,7 @@ static PyTypeObject pyrf_evsel__type = { .tp_dealloc = (destructor)pyrf_evsel__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_evsel__doc, + .tp_members = pyrf_evsel__members, .tp_methods = pyrf_evsel__methods, .tp_init = (initproc)pyrf_evsel__init, .tp_str = pyrf_evsel__str,Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerPATCH v2 6/11 perf python: Add optional cpus and threads arguments to parse_events
In reply to Ian RogersUsed for the evlist initialization.
--- tools/perf/util/python.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
tools/perf/util/python.c +8 -2
@@ -1339,12 +1339,18 @@ static PyObject *pyrf__parse_events(PyObject *self, PyObject *args)struct evlist evlist = {};struct parse_events_error err;PyObject *result; +PyObject *pcpus =NULL , *pthreads =NULL ; +struct perf_cpu_map *cpus; +struct perf_thread_map *threads; -if (!PyArg_ParseTuple (args,"s" , &input)) +if (!PyArg_ParseTuple (args,"s|OO" , &input, &pcpus, &pthreads))return NULL ; + threads = pthreads ? ((struct pyrf_thread_map *)pthreads)->threads :NULL ; + cpus = pcpus ? ((struct pyrf_cpu_map *)pcpus)->cpus :NULL ; +parse_events_error__init (&err); -evlist__init (&evlist,NULL ,NULL ); +evlist__init (&evlist, cpus, threads);if (parse_events (&evlist, input, &err)) {parse_events_error__print (&err, input);PyErr_SetFromErrno (PyExc_OSError);
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Re: [PATCH v2 06/11] perf python: Add optional cpus and threads arguments to parse_events
Quoted 1 line · Ian Rogers
Used for the evlist initialization.
- Arnaldo
Quoted 33 lines
Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/python.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)tools/perf/util/python.c +8
-2@@ -1339,12 +1339,18 @@ static PyObject *pyrf__parse_events(PyObject *self, PyObject *args)struct evlist evlist = {};struct parse_events_error err;PyObject *result; +PyObject *pcpus =NULL , *pthreads =NULL ; +struct perf_cpu_map *cpus; +struct perf_thread_map *threads; -if (!PyArg_ParseTuple (args,"s" , &input)) +if (!PyArg_ParseTuple (args,"s|OO" , &input, &pcpus, &pthreads))return NULL ; + threads = pthreads ? ((struct pyrf_thread_map *)pthreads)->threads :NULL ; + cpus = pcpus ? ((struct pyrf_cpu_map *)pcpus)->cpus :NULL ; +parse_events_error__init (&err); -evlist__init (&evlist,NULL ,NULL ); +evlist__init (&evlist, cpus, threads);if (parse_events (&evlist, input, &err)) {parse_events_error__print (&err, input);PyErr_SetFromErrno (PyExc_OSError);Signature · 1 line
2.48.1.711.g2feabab25a-goog
Namhyung Kim
maintainerRe: [PATCH v2 06/11] perf python: Add optional cpus and threads arguments to parse_events
In reply to Ian RogersQuoted 16 lines · Ian Rogers
Used for the evlist initialization.
Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/python.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)tools/perf/util/python.c +0
-0@@ -1339,12 +1339,18 @@ static PyObject *pyrf__parse_events(PyObject *self, PyObject *args)struct evlist evlist = {};struct parse_events_error err;PyObject *result;
tools/perf/util/python.c static PyObject *pyrf__parse_events(PyObject *self, PyObject *args) · line 1350
Quoted 10 lines
tools/perf/util/python.c +6
-1+PyObject *pcpus =NULL , *pthreads =NULL ; +struct perf_cpu_map *cpus; +struct perf_thread_map *threads; -if (!PyArg_ParseTuple (args,"s" , &input)) +if (!PyArg_ParseTuple (args,"s|OO" , &input, &pcpus, &pthreads))return NULL ; + threads = pthreads ? ((struct pyrf_thread_map *)pthreads)->threads :NULL ; + cpus = pcpus ? ((struct pyrf_cpu_map *)pcpus)->cpus :NULL ;
I wonder if it needs any type checks before accessing them.
Thanks,
Namhyung
Quoted 10 lines
tools/perf/util/python.c +2
-1+parse_events_error__init (&err); -evlist__init (&evlist,NULL ,NULL ); +evlist__init (&evlist, cpus, threads);if (parse_events (&evlist, input, &err)) {parse_events_error__print (&err, input);PyErr_SetFromErrno (PyExc_OSError);Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerRe: [PATCH v2 06/11] perf python: Add optional cpus and threads arguments to parse_events
Quoted 22 lines · Namhyung Kim
Quoted 20 lines · Ian Rogers
Used for the evlist initialization.
Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/python.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)tools/perf/util/python.c +3
-0@@ -1339,12 +1339,18 @@ static PyObject *pyrf__parse_events(PyObject *self, PyObject *args)struct evlist evlist = {};struct parse_events_error err;PyObject *result; +PyObject *pcpus =NULL , *pthreads =NULL ; +struct perf_cpu_map *cpus; +struct perf_thread_map *threads;
Quoted 8 lines
tools/perf/util/python.c static PyObject *pyrf__parse_events(PyObject *self, PyObject *args) · line 1350
Quoted 6 lines
tools/perf/util/python.c +3
-1-if (!PyArg_ParseTuple (args,"s" , &input)) +if (!PyArg_ParseTuple (args,"s|OO" , &input, &pcpus, &pthreads))return NULL ; + threads = pthreads ? ((struct pyrf_thread_map *)pthreads)->threads :NULL ; + cpus = pcpus ? ((struct pyrf_cpu_map *)pcpus)->cpus :NULL ;I wonder if it needs any type checks before accessing them.
Agreed. We don't do it yet elsewhere:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/util/python.c?h=perf-tools-next#n769
although there keywords potentially avoid ambiguity. Given this I think improving the typing is follow up work.
Thanks,
Ian
Quoted 13 lines
Thanks,
NamhyungQuoted 10 lines
tools/perf/util/python.c +2
-2+parse_events_error__init (&err); -evlist__init (&evlist,NULL ,NULL ); +evlist__init (&evlist, cpus, threads);if (parse_events (&evlist, input, &err)) {parse_events_error__print (&err, input);PyErr_SetFromErrno (PyExc_OSError); --2.48.1.711.g2feabab25a-goog
Namhyung Kim
maintainerRe: [PATCH v2 06/11] perf python: Add optional cpus and threads arguments to parse_events
Hi Ian,
Quoted 32 lines · Ian Rogers
Quoted 22 lines · Namhyung Kim
Quoted 20 lines · Ian Rogers
Used for the evlist initialization.
Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/python.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)tools/perf/util/python.c +3
-0@@ -1339,12 +1339,18 @@ static PyObject *pyrf__parse_events(PyObject *self, PyObject *args)struct evlist evlist = {};struct parse_events_error err;PyObject *result; +PyObject *pcpus =NULL , *pthreads =NULL ; +struct perf_cpu_map *cpus; +struct perf_thread_map *threads;Quoted 8 lines
tools/perf/util/python.c static PyObject *pyrf__parse_events(PyObject *self, PyObject *args) · line 1350
Quoted 6 lines
tools/perf/util/python.c +3
-1-if (!PyArg_ParseTuple (args,"s" , &input)) +if (!PyArg_ParseTuple (args,"s|OO" , &input, &pcpus, &pthreads))return NULL ; + threads = pthreads ? ((struct pyrf_thread_map *)pthreads)->threads :NULL ; + cpus = pcpus ? ((struct pyrf_cpu_map *)pcpus)->cpus :NULL ;I wonder if it needs any type checks before accessing them.
Quoted 4 lines
Agreed. We don't do it yet elsewhere:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/util/python.c?h=perf-tools-next#n769
although there keywords potentially avoid ambiguity. Given this I think improving the typing is follow up work.
Sounds good.
Thanks,
Namhyung
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
tools/perf/util/python.c +2
-2+parse_events_error__init (&err); -evlist__init (&evlist,NULL ,NULL ); +evlist__init (&evlist, cpus, threads);if (parse_events (&evlist, input, &err)) {parse_events_error__print (&err, input);PyErr_SetFromErrno (PyExc_OSError); --2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerPATCH v2 7/11 perf python: Update ungrouped evsel leader in clone
In reply to Ian Rogersevsels are cloned in the python code as they form part of the Python object pyrf_evsel. The cloning doesn't update the evsel's leader, do this for the case of an evsel being ungrouped.
--- tools/perf/util/python.c | 2 ++ 1 file changed, 2 insertions(+)
tools/perf/util/python.c +2 -0
@@ -1312,6 +1312,8 @@ static PyObject *pyrf_evsel__from_evsel(struct evsel *evsel)evsel__init (&pevsel->evsel, &evsel->core.attr, evsel->core.idx);evsel__clone (&pevsel->evsel, evsel); +if (evsel__is_group_leader (evsel)) +evsel__set_leader (&pevsel->evsel, &pevsel->evsel);return (PyObject *)pevsel; }
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Re: [PATCH v2 07/11] perf python: Update ungrouped evsel leader in clone
Quoted 3 lines · Ian Rogers
evsels are cloned in the python code as they form part of the Python object
pyrf_evsel. The cloning doesn't update the evsel's leader, do this for the case of an evsel being ungrouped.
- Arnaldo
Quoted 21 lines
Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/python.c | 2 ++ 1 file changed, 2 insertions(+)tools/perf/util/python.c +2
-0@@ -1312,6 +1312,8 @@ static PyObject *pyrf_evsel__from_evsel(struct evsel *evsel)evsel__init (&pevsel->evsel, &evsel->core.attr, evsel->core.idx);evsel__clone (&pevsel->evsel, evsel); +if (evsel__is_group_leader (evsel)) +evsel__set_leader (&pevsel->evsel, &pevsel->evsel);return (PyObject *)pevsel; }Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerPATCH v2 8/11 perf python: Avoid duplicated code in get_tracepoint_field
In reply to Ian RogersThe code replicates computations done in evsel__tp_format, reuse evsel__tp_format to simplify the python C code.
--- tools/perf/util/python.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-)
tools/perf/util/python.c +4 -13
@@ -342,23 +342,14 @@ get_tracepoint_field(struct pyrf_event *pevent, PyObject *attr_name) {const char *str =_PyUnicode_AsString (PyObject_Str (attr_name));struct evsel *evsel = pevent->evsel; +struct tep_event *tp_format =evsel__tp_format (evsel);struct tep_format_field *field; -if (!evsel->tp_format) { -struct tep_event *tp_format; - - tp_format =trace_event__tp_format_id (evsel->core.attr.config); -if (IS_ERR_OR_NULL (tp_format)) -return NULL ; - - evsel->tp_format = tp_format; - } - - field =tep_find_any_field (evsel->tp_format, str); -if (!field) +if (IS_ERR_OR_NULL (tp_format))return NULL ; -return tracepoint_field (pevent, field); + field =tep_find_any_field (tp_format, str); +return field ?tracepoint_field (pevent, field) :NULL ; }#endif /* HAVE_LIBTRACEEVENT */
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Re: [PATCH v2 08/11] perf python: Avoid duplicated code in get_tracepoint_field
Quoted 2 lines · Ian Rogers
The code replicates computations done in
evsel__tp_format, reuseevsel__tp_formatto simplify the python C code.
- Arnaldo
Quoted 40 lines
Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/python.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-)tools/perf/util/python.c +4
-13@@ -342,23 +342,14 @@ get_tracepoint_field(struct pyrf_event *pevent, PyObject *attr_name) {const char *str =_PyUnicode_AsString (PyObject_Str (attr_name));struct evsel *evsel = pevent->evsel; +struct tep_event *tp_format =evsel__tp_format (evsel);struct tep_format_field *field; -if (!evsel->tp_format) { -struct tep_event *tp_format; - - tp_format =trace_event__tp_format_id (evsel->core.attr.config); -if (IS_ERR_OR_NULL (tp_format)) -return NULL ; - - evsel->tp_format = tp_format; - } - - field =tep_find_any_field (evsel->tp_format, str); -if (!field) +if (IS_ERR_OR_NULL (tp_format))return NULL ; -return tracepoint_field (pevent, field); + field =tep_find_any_field (tp_format, str); +return field ?tracepoint_field (pevent, field) :NULL ; }#endif /* HAVE_LIBTRACEEVENT */ Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerPATCH v2 9/11 perf python: Add evlist all_cpus accessor
In reply to Ian RogersAdd a means to get the reference counted all_cpus CPU map from an evlist in its python form.
--- tools/perf/util/python.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
tools/perf/util/python.c +16 -0
@@ -874,6 +874,16 @@ static void pyrf_evlist__delete(struct pyrf_evlist *pevlist)Py_TYPE (pevlist)->tp_free ((PyObject *)pevlist); } +static PyObject *pyrf_evlist__all_cpus (struct pyrf_evlist *pevlist) +{ + struct pyrf_cpu_map *pcpu_map = PyObject_New(struct pyrf_cpu_map, &pyrf_cpu_map__type); + +if (pcpu_map) + pcpu_map->cpus =perf_cpu_map__get (pevlist->evlist.core.all_cpus); + +return (PyObject *)pcpu_map; +} +static PyObject *pyrf_evlist__mmap (struct pyrf_evlist *pevlist,PyObject *args,PyObject *kwargs) {
@@ -1057,6 +1067,12 @@ static PyObject *pyrf_evlist__enable(struct pyrf_evlist *pevlist) }static PyMethodDef pyrf_evlist__methods[] = { + { + .ml_name ="all_cpus" , + .ml_meth = (PyCFunction )pyrf_evlist__all_cpus, + .ml_flags =METH_NOARGS , + .ml_doc =PyDoc_STR ("CPU map union of all evsel CPU maps." ) + }, { .ml_name ="mmap" , .ml_meth = (PyCFunction )pyrf_evlist__mmap,
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Re: [PATCH v2 09/11] perf python: Add evlist all_cpus accessor
Quoted 2 lines · Ian Rogers
Add a means to get the reference counted
all_cpusCPU map from an evlist in its python form.
- Arnaldo
Quoted 42 lines
Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/python.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)tools/perf/util/python.c +16
-0@@ -874,6 +874,16 @@ static void pyrf_evlist__delete(struct pyrf_evlist *pevlist)Py_TYPE (pevlist)->tp_free ((PyObject *)pevlist); } +static PyObject *pyrf_evlist__all_cpus (struct pyrf_evlist *pevlist) +{ + struct pyrf_cpu_map *pcpu_map = PyObject_New(struct pyrf_cpu_map, &pyrf_cpu_map__type); + +if (pcpu_map) + pcpu_map->cpus =perf_cpu_map__get (pevlist->evlist.core.all_cpus); + +return (PyObject *)pcpu_map; +} +static PyObject *pyrf_evlist__mmap (struct pyrf_evlist *pevlist,PyObject *args,PyObject *kwargs) {@@ -1057,6 +1067,12 @@ static PyObject *pyrf_evlist__enable(struct pyrf_evlist *pevlist) }static PyMethodDef pyrf_evlist__methods[] = { + { + .ml_name ="all_cpus" , + .ml_meth = (PyCFunction )pyrf_evlist__all_cpus, + .ml_flags =METH_NOARGS , + .ml_doc =PyDoc_STR ("CPU map union of all evsel CPU maps." ) + }, { .ml_name ="mmap" , .ml_meth = (PyCFunction )pyrf_evlist__mmap,Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerPATCH v2 10/11 perf python: Add evlist.config to set up record options
In reply to Ian RogersAdd access to evlist__config that is used to configure an evlist with record options.
--- tools/perf/util/python.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
tools/perf/util/python.c +33 -0
@@ -9,10 +9,12 @@#include <event-parse.h> #endif #include <perf/mmap.h> +#include "callchain.h" #include "evlist.h" #include "evsel.h" #include "event.h" #include "print_binary.h" +#include "record.h" #include "strbuf.h" #include "thread_map.h" #include "trace-event.h"
@@ -1052,6 +1054,31 @@ static PyObject *pyrf_evlist__open(struct pyrf_evlist *pevlist,return Py_None; } +static PyObject *pyrf_evlist__config (struct pyrf_evlist *pevlist) +{ +struct record_opts opts = { + .sample_time = true, + .mmap_pages =UINT_MAX , + .user_freq =UINT_MAX , + .user_interval =ULLONG_MAX , + .freq =4000 , + .target = { + .uses_mmap = true, + .default_per_cpu = true, + }, + .nr_threads_synthesize =1 , + .ctl_fd =-1 , + .ctl_fd_ack =-1 , + .no_buffering = true, + .no_inherit = true, + }; +struct evlist *evlist = &pevlist->evlist; + +evlist__config (evlist, &opts, &callchain_param); +Py_INCREF (Py_None); +return Py_None; +} +static PyObject *pyrf_evlist__disable (struct pyrf_evlist *pevlist) {evlist__disable (&pevlist->evlist);
@@ -1109,6 +1136,12 @@ static PyMethodDef pyrf_evlist__methods[] = { .ml_flags = METH_VARARGS | METH_KEYWORDS, .ml_doc =PyDoc_STR ("reads an event." ) }, + { + .ml_name = "config", + .ml_meth = (PyCFunction)pyrf_evlist__config, + .ml_flags = METH_NOARGS, + .ml_doc =PyDoc_STR ("Apply default record options to the evlist." ) + }, { .ml_name = "disable", .ml_meth = (PyCFunction)pyrf_evlist__disable,
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Re: [PATCH v2 10/11] perf python: Add evlist.config to set up record options
Quoted 2 lines · Ian Rogers
Add access to
evlist__configthat is used to configure an evlist with record options.
I guess nothing precludes adding support later for passing fields as an optional dictionary, so I think its ok to have it with the name "config" but being more of a "default_config".
- Arnaldo
Quoted 70 lines
Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/util/python.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)tools/perf/util/python.c +32
-0@@ -9,10 +9,12 @@#include <event-parse.h> #endif #include <perf/mmap.h> +#include "callchain.h" #include "evlist.h" #include "evsel.h" #include "event.h" #include "print_binary.h" +#include "record.h" #include "strbuf.h" #include "thread_map.h" #include "trace-event.h" @@ -1052,6 +1054,31 @@ static PyObject *pyrf_evlist__open(struct pyrf_evlist *pevlist,return Py_None; } +static PyObject *pyrf_evlist__config (struct pyrf_evlist *pevlist) +{ +struct record_opts opts = { + .sample_time = true, + .mmap_pages =UINT_MAX , + .user_freq =UINT_MAX , + .user_interval =ULLONG_MAX , + .freq =4000 , + .target = { + .uses_mmap = true, + .default_per_cpu = true, + }, + .nr_threads_synthesize =1 , + .ctl_fd =-1 , + .ctl_fd_ack =-1 , + .no_buffering = true, + .no_inherit = true, + }; +struct evlist *evlist = &pevlist->evlist; +evlist__config (evlist, &opts, &callchain_param); +Py_INCREF (Py_None); +return Py_None; +} +static PyObject *pyrf_evlist__disable (struct pyrf_evlist *pevlist) {evlist__disable (&pevlist->evlist);@@ -1109,6 +1136,12 @@ static PyMethodDef pyrf_evlist__methods[] = { .ml_flags = METH_VARARGS | METH_KEYWORDS, .ml_doc =PyDoc_STR ("reads an event." ) }, + { + .ml_name = "config", + .ml_meth = (PyCFunction)pyrf_evlist__config, + .ml_flags = METH_NOARGS, + .ml_doc =PyDoc_STR ("Apply default record options to the evlist." ) + }, { .ml_name = "disable", .ml_meth = (PyCFunction)pyrf_evlist__disable,Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerPATCH v2 11/11 perf python tracepoint: Switch to using parse_events
In reply to Ian RogersRather than manually configuring an evsel, switch to using parse_events for greater commonality with the rest of the perf code.
--- tools/perf/python/tracepoint.py | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-)
tools/perf/python/tracepoint.py +11 -12
@@ -5,24 +5,23 @@ import perf -class tracepoint(perf.evsel): - def __init__(self, sys, name): - config = perf.tracepoint(sys, name) - perf.evsel.__init__(self, - type = perf.TYPE_TRACEPOINT, - config = config, - freq = 0, sample_period = 1, wakeup_events = 1, - sample_type = perf.SAMPLE_PERIOD | perf.SAMPLE_TID | perf.SAMPLE_CPU | perf.SAMPLE_RAW | perf.SAMPLE_TIME) - def main(): - tp = tracepoint("sched", "sched_switch") cpus = perf.cpu_map() threads = perf.thread_map(-1) + evlist = perf.parse_events("sched:sched_switch", cpus, threads) + # Disable tracking of mmaps and similar that are unnecessary. + for ev in evlist: + ev.tracking = False + # Configure evsels with default record options. + evlist.config() + # Simplify the sample_type and read_format of evsels + for ev in evlist: + ev.sample_type = ev.sample_type & ~perf.SAMPLE_IP + ev.read_format = 0 - evlist = perf.evlist(cpus, threads) - evlist.add(tp) evlist.open() evlist.mmap() + evlist.enable(); while True: evlist.poll(timeout = -1)
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
Quoted 5 lines · Ian Rogers
Rather than manually configuring an evsel, switch to using
parse_eventsfor greater commonality with the rest of the perf code.Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Now will all in place I'm trying to test it and I am getting some strange results:
root@number:/home/acme/git/perf-tools-next# tools/perf/python/tracepoint.py
<SNIP lots of seemingly ok lines>
time 78318710956557 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/14 next_pid=0 next_prio=120
time 78318720082300 prev_comm=swapper/16 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=kworker/u112:8 next_pid=1752774 next_prio=120
time 78318706232435 prev_comm=kworker/u112:17 prev_pid=1551246 prev_prio=120 prev_state=0x80 ==> next_comm=swapper/21 next_pid=0 next_prio=120
time 78318708202121 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x2 ==> next_comm=swapper/25 next_pid=0 next_prio=120
time 78318748346989 prev_comm=swapper/26 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=gnome-terminal- next_pid=3551 next_prio=120
Traceback (most recent call last):
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 47, in <module>
main()
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 42, in main
event.next_comm,
^^^^^^^^^^^^^^^
AttributeError: 'perf.sample_event' object has no attribute 'next_comm'
root@number:/home/acme/git/perf-tools-next# But it shouldn't get there as there is this check:
if not isinstance(event, perf.sample_event):
continue
:-\
Trying to debug that...
- Arnaldo
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
Quoted 28 lines · Arnaldo Carvalho de Melo
Quoted 5 lines · Ian Rogers
Rather than manually configuring an evsel, switch to using
parse_eventsfor greater commonality with the rest of the perf code.Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>Now will all in place I'm trying to test it and I am getting some strange results:
root@number:/home/acme/git/perf-tools-next# tools/perf/python/tracepoint.py <SNIP lots of seemingly ok lines> time 78318710956557 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/14 next_pid=0 next_prio=120 time 78318720082300 prev_comm=swapper/16 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=kworker/u112:8 next_pid=1752774 next_prio=120 time 78318706232435 prev_comm=kworker/u112:17 prev_pid=1551246 prev_prio=120 prev_state=0x80 ==> next_comm=swapper/21 next_pid=0 next_prio=120 time 78318708202121 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x2 ==> next_comm=swapper/25 next_pid=0 next_prio=120 time 78318748346989 prev_comm=swapper/26 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=gnome-terminal- next_pid=3551 next_prio=120 Traceback (most recent call last): File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 47, in <module> main() File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 42, in main event.next_comm, ^^^^^^^^^^^^^^^ AttributeError: 'perf.sample_event' object has no attribute 'next_comm' root@number:/home/acme/git/perf-tools-next#But it shouldn't get there as there is this check:
Quoted 7 lines
if not
isinstance(event, perf.sample_event):
continue:-\
Trying to debug that...
And it doesn't seem related to your series, I checked and v6.13 has the same problem, I nuked the build dir, etc.
- Arnaldo
Ian Rogers
reviewerRe: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
Quoted 34 lines · Arnaldo Carvalho de Melo
Quoted 32 lines · Arnaldo Carvalho de Melo
Quoted 5 lines · Ian Rogers
Rather than manually configuring an evsel, switch to using
parse_eventsfor greater commonality with the rest of the perf code.Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>Now will all in place I'm trying to test it and I am getting some strange results:
root@number:/home/acme/git/perf-tools-next# tools/perf/python/tracepoint.py <SNIP lots of seemingly ok lines> time 78318710956557 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/14 next_pid=0 next_prio=120 time 78318720082300 prev_comm=swapper/16 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=kworker/u112:8 next_pid=1752774 next_prio=120 time 78318706232435 prev_comm=kworker/u112:17 prev_pid=1551246 prev_prio=120 prev_state=0x80 ==> next_comm=swapper/21 next_pid=0 next_prio=120 time 78318708202121 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x2 ==> next_comm=swapper/25 next_pid=0 next_prio=120 time 78318748346989 prev_comm=swapper/26 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=gnome-terminal- next_pid=3551 next_prio=120 Traceback (most recent call last): File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 47, in <module> main() File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 42, in main event.next_comm, ^^^^^^^^^^^^^^^ AttributeError: 'perf.sample_event' object has no attribute 'next_comm' root@number:/home/acme/git/perf-tools-next#But it shouldn't get there as there is this check:
if not
isinstance(event, perf.sample_event):
continue
Quoted 6 lines
Quoted 3 lines
:-\
Trying to debug that...
And it doesn't seem related to your series, I checked and v6.13 has the same problem, I nuked the build dir, etc.
Right. I'd seen the same issue.
Thanks,
Ian
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
Quoted 36 lines · Ian Rogers
Quoted 34 lines · Arnaldo Carvalho de Melo
Quoted 32 lines · Arnaldo Carvalho de Melo
Quoted 5 lines · Ian Rogers
Rather than manually configuring an evsel, switch to using
parse_eventsfor greater commonality with the rest of the perf code.Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>Now will all in place I'm trying to test it and I am getting some strange results:
root@number:/home/acme/git/perf-tools-next# tools/perf/python/tracepoint.py <SNIP lots of seemingly ok lines> time 78318710956557 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/14 next_pid=0 next_prio=120 time 78318720082300 prev_comm=swapper/16 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=kworker/u112:8 next_pid=1752774 next_prio=120 time 78318706232435 prev_comm=kworker/u112:17 prev_pid=1551246 prev_prio=120 prev_state=0x80 ==> next_comm=swapper/21 next_pid=0 next_prio=120 time 78318708202121 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x2 ==> next_comm=swapper/25 next_pid=0 next_prio=120 time 78318748346989 prev_comm=swapper/26 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=gnome-terminal- next_pid=3551 next_prio=120 Traceback (most recent call last): File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 47, in <module> main() File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 42, in main event.next_comm, ^^^^^^^^^^^^^^^ AttributeError: 'perf.sample_event' object has no attribute 'next_comm' root@number:/home/acme/git/perf-tools-next#But it shouldn't get there as there is this check:
if not
isinstance(event, perf.sample_event):
continue
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
:-\
Trying to debug that...
And it doesn't seem related to your series, I checked and v6.13 has the same problem, I nuked the build dir, etc.
Right. I'd seen the same issue.
time 79411977132102 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x2 ==> next_comm=swapper/12 next_pid=0 next_prio=120 { type: sample }
time 79411977200343 prev_comm=kworker/u112:17 prev_pid=1551246 prev_prio=120 prev_state=0x80 ==> next_comm=swapper/16 next_pid=0 next_prio=120 { type: sample }
time 79411964535268 prev_comm=kworker/u112:14 prev_pid=810939 prev_prio=120 prev_state=0x80 ==> next_comm=swapper/17 next_pid=0 next_prio=120 { type: sample }
time 79411964746511 prev_comm=swapper/18 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=kworker/u112:12 next_pid=2109251 next_prio=120 { type: sample }
Traceback (most recent call last):
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 48, in <module>
main()
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 43, in main
event.next_comm,
^^^^^^^^^^^^^^^AttributeError: 'perf.sample_event' object has no attribute 'next_comm' root@number:/home/acme/git/perf-tools-next#
And it says it is a sample...
Well, ran out of time, will try later or early tomorrow, will also try to review the syscalltbl series and Howard's off cpu profiling.
- Arnaldo
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
Quoted 60 lines · Arnaldo Carvalho de Melo
Quoted 36 lines · Ian Rogers
Quoted 34 lines · Arnaldo Carvalho de Melo
Quoted 32 lines · Arnaldo Carvalho de Melo
Quoted 5 lines · Ian Rogers
Rather than manually configuring an evsel, switch to using
parse_eventsfor greater commonality with the rest of the perf code.Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>Now will all in place I'm trying to test it and I am getting some strange results:
root@number:/home/acme/git/perf-tools-next# tools/perf/python/tracepoint.py <SNIP lots of seemingly ok lines> time 78318710956557 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/14 next_pid=0 next_prio=120 time 78318720082300 prev_comm=swapper/16 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=kworker/u112:8 next_pid=1752774 next_prio=120 time 78318706232435 prev_comm=kworker/u112:17 prev_pid=1551246 prev_prio=120 prev_state=0x80 ==> next_comm=swapper/21 next_pid=0 next_prio=120 time 78318708202121 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x2 ==> next_comm=swapper/25 next_pid=0 next_prio=120 time 78318748346989 prev_comm=swapper/26 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=gnome-terminal- next_pid=3551 next_prio=120 Traceback (most recent call last): File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 47, in <module> main() File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 42, in main event.next_comm, ^^^^^^^^^^^^^^^ AttributeError: 'perf.sample_event' object has no attribute 'next_comm' root@number:/home/acme/git/perf-tools-next#But it shouldn't get there as there is this check:
if not
isinstance(event, perf.sample_event):
continueQuoted 8 lines
Quoted 6 lines
Quoted 3 lines
:-\
Trying to debug that...
And it doesn't seem related to your series, I checked and v6.13 has the same problem, I nuked the build dir, etc.
Right. I'd seen the same issue.
time 79411977132102 prev_comm=sudo prev_pid=3133818 prev_prio=120 prev_state=0x2 ==> next_comm=swapper/12 next_pid=0 next_prio=120 { type: sample }
time 79411977200343 prev_comm=kworker/u112:17 prev_pid=1551246 prev_prio=120 prev_state=0x80 ==> next_comm=swapper/16 next_pid=0 next_prio=120 { type: sample }
time 79411964535268 prev_comm=kworker/u112:14 prev_pid=810939 prev_prio=120 prev_state=0x80 ==> next_comm=swapper/17 next_pid=0 next_prio=120 { type: sample }
time 79411964746511 prev_comm=swapper/18 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=kworker/u112:12 next_pid=2109251 next_prio=120 { type: sample }
Traceback (most recent call last):
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 48, in <module>main() File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 43, in main event.next_comm, ^^^^^^^^^^^^^^^
Quoted 7 lines
AttributeError: 'perf.sample_event' object has no attribute '
next_comm' root@number:/home/acme/git/perf-tools-next#And it says it is a sample...
Well, ran out of time, will try later or early tomorrow, will also try to review the syscalltbl series and Howard's off cpu profiling.
Its some sort of corruption, I added printing the sample_* fields and then up to the field before next_comm:
time 2462907120158 prev_comm=swapper/8 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=gnome-terminal- next_pid=3646 next_prio=120
ip 0 pid=0 tid=0 cpu=12
time 2462916970223 prev_comm=swapper/12 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=gnome-terminal- next_pid=3646 next_prio=120
ip 0 pid=0 tid=0 cpu=14
time 2462902120635 prev_comm= prev_pid=1919907691 prev_prio=796026219 prev_state=0x323a32313175 ==> Traceback (most recent call last):
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 51, in <module>
main()
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 46, in main
event.next_comm,
^^^^^^^^^^^^^^^AttributeError: 'perf.sample_event' object has no attribute 'next_comm' root@number:/home/acme/git/perf-tools-next#
See the empty prev_comm, bogus pref_pid, ditto for prev_state and pref_prio and then it simple doesn't have that next_comm...
Also I noticed that in tracepoint_field() (tools/perf/util/python.c) we can get the string tracepoint fields as a bytearray or as a string, ahere:
if (field->flags & TEP_FIELD_IS_STRING &&
is_printable_array(data + offset, len)) {
ret = PyUnicode_FromString((char *)data + offset);
} else {
ret = PyByteArray_FromStringAndSize((const char *) data + offset, len);
field->flags &= ~TEP_FIELD_IS_STRING;
}For some reason in sessions where bytearrays are returned, and all comes as bytearrays, the problem is not noticed.
But when it comes as a string it breaks after a short time, /me scratches head...
- Arnaldo
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
Quoted 9 lines · Arnaldo Carvalho de Melo
if (field->flags & TEP_FIELD_IS_STRING &&
is_printable_array(data + offset, len)) {
ret = PyUnicode_FromString((char *)data + offset);
} else {
ret = PyByteArray_FromStringAndSize((const char *) data + offset, len);
field->flags &= ~TEP_FIELD_IS_STRING;
}Quoted 5 lines
For some reason in sessions where bytearrays are returned, and all comes as bytearrays, the problem is not noticed.
But when it comes as a string it breaks after a short time, /me scratches head...
I think I'm getting closer, with some debugging sprinkled in the python binding:
ip 0 pid=74131 tid=74131 cpu=3
( field 'prev_comm' ret=0x7f0a66a1b970 ) ( field 'prev_pid' ret=0x7f0a66b1bed0 ) ( field 'prev_prio' ret=0x7f0a74f60d08 ) ( field 'prev_state' ret=0x7f0a74f60e08 ) time 4503271651784 prev_comm=kworker/u112:14 prev_pid=74131 prev_prio=120 prev_state=0x80 ==>
( field 'next_comm' ret=0x7f0a66a1b970 ) ( field 'next_pid' ret=0x7f0a74f5fe08 ) ( field 'next_prio' ret=0x7f0a74f60d08 ) next_comm=swapper/3 next_pid=0 next_prio=120
ip 0 pid=80209 tid=80209 cpu=4
( field 'prev_comm' ret=0x7f0a66a1b970 ) ( field 'prev_pid' ret=0x7f0a66b1bed0 ) ( field 'prev_prio' ret=0x7f0a74f60d08 ) ( field 'prev_state' ret=0x7f0a74f60e08 ) time 4503280531143 prev_comm=kworker/u112:3 prev_pid=80209 prev_prio=120 prev_state=0x80 ==>
( field 'next_comm' ret=0x7f0a66a1b970 ) ( field 'next_pid' ret=0x7f0a74f5fe08 ) ( field 'next_prio' ret=0x7f0a74f60d08 ) next_comm=swapper/4 next_pid=0 next_prio=120
ip 0 pid=74131 tid=74131 cpu=5
( XXX '�!' len=16) ( field 'prev_comm' ret=(nil) ) Traceback (most recent call last):
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 51, in <module>
main()
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 40, in main
event.prev_comm,
^^^^^^^^^^^^^^^AttributeError: 'perf.sample_event' object has no attribute 'prev_comm'
So the size of the field, in the tracepoint format file is 16, its a comm:
root@number:/home/acme/git/perf-tools-next# grep '_comm\[' /sys/kernel/tracing/events/sched/sched_switch/format field:char prev_comm[16]; offset:8; size:16; signed:0; field:char next_comm[16]; offset:40; size:16; signed:0; root@number:/home/acme/git/perf-tools-next#
But:
root@number:/home/acme/git/perf-tools-next# cat /proc/74131/comm kworker/u112:14-events_unbound root@number:/home/acme/git/perf-tools-next#
root@number:/home/acme/git/perf-tools-next# echo -n "kworker/u112:14-events_unbound" | wc -c 30 root@number:/home/acme/git/perf-tools-next#
Which should explain that:
( XXX '�!' len=16) ( field 'prev_comm' ret=(nil) )
That is printed by:
+++ b/tools/perf/util/python.c
@@ -318,16 +318,17 @@ tracepoint_field(const struct pyrf_event *pe, struct tep_format_field *field)
if (tep_field_is_relative(field->flags))
offset += field->offset + field->size;
}
- if (field->flags & TEP_FIELD_IS_STRING &&
- is_printable_array(data + offset, len)) {
+ if (field->flags & TEP_FIELD_IS_STRING) {
ret = PyUnicode_FromString((char *)data + offset);
+ if (ret == NULL) {
+ printf(" ( XXX '%.*s' len=%d) ", len, (char *)data + offset, len); fflush(stdout);
+ }
} else {
ret = PyByteArray_FromStringAndSize((const char *) data + offset, len);
- field->flags &= ~TEP_FIELD_IS_STRING;
}So now trying to figure out why when the comm lenght is more than 16 the raw_data appears to contain trash...
- Arnaldo
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
Quoted 76 lines · Arnaldo Carvalho de Melo
Quoted 9 lines · Arnaldo Carvalho de Melo
if (field->flags & TEP_FIELD_IS_STRING && is_printable_array(data + offset, len)) { ret = PyUnicode_FromString((char *)data + offset); } else { ret = PyByteArray_FromStringAndSize((const char *) data + offset, len); field->flags &= ~TEP_FIELD_IS_STRING; }Quoted 5 lines
For some reason in sessions where bytearrays are returned, and all comes as bytearrays, the problem is not noticed.
But when it comes as a string it breaks after a short time, /me scratches head...
I think I'm getting closer, with some debugging sprinkled in the python binding:
ip 0 pid=74131 tid=74131 cpu=3
( field 'prev_comm' ret=0x7f0a66a1b970 ) ( field 'prev_pid' ret=0x7f0a66b1bed0 ) ( field 'prev_prio' ret=0x7f0a74f60d08 ) ( field 'prev_state' ret=0x7f0a74f60e08 ) time 4503271651784 prev_comm=kworker/u112:14 prev_pid=74131 prev_prio=120 prev_state=0x80 ==> ( field 'next_comm' ret=0x7f0a66a1b970 ) ( field 'next_pid' ret=0x7f0a74f5fe08 ) ( field 'next_prio' ret=0x7f0a74f60d08 ) next_comm=swapper/3 next_pid=0 next_prio=120 ip 0 pid=80209 tid=80209 cpu=4 ( field 'prev_comm' ret=0x7f0a66a1b970 ) ( field 'prev_pid' ret=0x7f0a66b1bed0 ) ( field 'prev_prio' ret=0x7f0a74f60d08 ) ( field 'prev_state' ret=0x7f0a74f60e08 ) time 4503280531143 prev_comm=kworker/u112:3 prev_pid=80209 prev_prio=120 prev_state=0x80 ==> ( field 'next_comm' ret=0x7f0a66a1b970 ) ( field 'next_pid' ret=0x7f0a74f5fe08 ) ( field 'next_prio' ret=0x7f0a74f60d08 ) next_comm=swapper/4 next_pid=0 next_prio=120 ip 0 pid=74131 tid=74131 cpu=5 ( XXX '�!' len=16) ( field 'prev_comm' ret=(nil) ) Traceback (most recent call last): File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 51, in <module> main() File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 40, in main event.prev_comm, ^^^^^^^^^^^^^^^AttributeError: 'perf.sample_event' object has no attribute '
prev_comm'So the size of the field, in the tracepoint format file is 16, its a comm:
root@number:/home/acme/git/perf-tools-next# grep '_comm\[' /sys/kernel/tracing/events/sched/sched_switch/format field:char prev_comm[16]; offset:8; size:16; signed:0; field:char next_comm[16]; offset:40; size:16; signed:0; root@number:/home/acme/git/perf-tools-next#But:
root@number:/home/acme/git/perf-tools-next# cat /proc/74131/comm kworker/u112:14-events_unbound root@number:/home/acme/git/perf-tools-next#root@number:/home/acme/git/perf-tools-next# echo -n "kworker/u112:14-events_unbound" | wc -c 30 root@number:/home/acme/git/perf-tools-next#Which should explain that:
( XXX '�!' len=16) ( field '
prev_comm' ret=(nil) )That is printed by:
+++
b/tools/perf/util/python.c(quoted diff) +4
-3@@ -318,16 +318,17 @@ tracepoint_field(const struct pyrf_event *pe, struct tep_format_field *field) if (tep_field_is_relative(field->flags)) offset += field->offset + field->size; } - if (field->flags & TEP_FIELD_IS_STRING && - is_printable_array(data + offset, len)) { + if (field->flags & TEP_FIELD_IS_STRING) { ret = PyUnicode_FromString((char *)data + offset); + if (ret == NULL) { + printf(" ( XXX '%.*s' len=%d) ", len, (char *)data + offset, len); fflush(stdout); + } } else { ret = PyByteArray_FromStringAndSize((const char *) data + offset, len); - field->flags &= ~TEP_FIELD_IS_STRING; }
Quoted 2 lines
So now trying to figure out why when the comm lenght is more than 16 the
raw_dataappears to contain trash...
So it seems to be something just in the python binding, as perf trace seems to handle it well:
( field 'prev_comm' ret=0x7f7c31f65110, raw_size=68 ) ( field 'prev_pid' ret=0x7f7c23b1bed0, raw_size=68 ) ( field 'prev_prio' ret=0x7f7c239c0030, raw_size=68 ) ( field 'prev_state' ret=0x7f7c239c0250, raw_size=68 ) time 14771421785867 prev_comm= prev_pid=1919907691 prev_prio=796026219 prev_state=0x303a32313175 ==> ( XXX '��' len=16, raw_size=68) ( field 'next_comm' ret=(nil), raw_size=68 ) Traceback (most recent call last):
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 51, in <module>
main()
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 46, in main
event.next_comm,
^^^^^^^^^^^^^^^
AttributeError: 'perf.sample_event' object has no attribute 'next_comm'
root@number:/home/acme/git/perf-tools-next# cat /proc/125355/comm
kworker/u112:0-i915
root@number:/home/acme/git/perf-tools-next#
root@number:/home/acme/git/perf-tools-next#
root@number:/home/acme/git/perf-tools-next# perf trace -e sched:sched_switch -p 125355
0.000 sched:sched_switch(prev_comm: "kworker/u112:0", prev_pid: 125355 (kworker/u112:0-), prev_prio: 120, prev_state: 128, next_comm: "swapper/6", next_prio: 120)
^Croot@number:/home/acme/git/perf-tools-next#I.e. it chops up the prev_comm size to what is specified in the tracepoint format.
And that sample->raw_size is the same accross the sched:sched_switch raw_datas (seemingly suboptimal, most are less than 16 bytes, but probably its not guaranteed that the \0 will be there, so copy all the 16 bytes).
Now to try to figure out why simply using PyUnicode_FromStringAndSize doesn't work...
- Arnaldo
Re: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
Quoted 22 lines · Arnaldo Carvalho de Melo
So it seems to be something just in the python binding, as perf trace seems to handle it well:
( field '
prev_comm' ret=0x7f7c31f65110, raw_size=68 ) ( field 'prev_pid' ret=0x7f7c23b1bed0, raw_size=68 ) ( field 'prev_prio' ret=0x7f7c239c0030, raw_size=68 ) ( field 'prev_state' ret=0x7f7c239c0250, raw_size=68 ) time 14771421785867 prev_comm= prev_pid=1919907691 prev_prio=796026219 prev_state=0x303a32313175 ==> ( XXX '��' len=16, raw_size=68) ( field 'next_comm' ret=(nil), raw_size=68 ) Traceback (most recent call last):
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 51, in <module>main() File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 46, in main event.next_comm, ^^^^^^^^^^^^^^^ AttributeError: 'perf.sample_event' object has no attribute 'next_comm' root@number:/home/acme/git/perf-tools-next# cat /proc/125355/comm kworker/u112:0-i915 root@number:/home/acme/git/perf-tools-next# root@number:/home/acme/git/perf-tools-next# root@number:/home/acme/git/perf-tools-next# perf trace -e sched:sched_switch -p 125355 0.000 sched:sched_switch(prev_comm: "kworker/u112:0", prev_pid: 125355 (kworker/u112:0-), prev_prio: 120, prev_state: 128, next_comm: "swapper/6", next_prio: 120) ^Croot@number:/home/acme/git/perf-tools-next#I.e. it chops up the
prev_commsize to what is specified in the tracepoint format.
Quoted 7 lines
And that
sample->raw_sizeis the same accross the sched:sched_switchraw_datas(seemingly suboptimal, most are less than 16 bytes, but probably its not guaranteed that the \0 will be there, so copy all the 16 bytes).Now to try to figure out why simply using
PyUnicode_FromStringAndSizedoesn't work...
Didn't manage to make progress on this, I spent more time than I expected as I think this could be some sort of canary on some coal mine, but with the patch below, that gives up and just avoids touching the COMM fields and don't switch from string to bytearray in the binding, it runs forever, this is just a data point in case somebody wants to pursue.
That flipping from string to not string based on just one entry not being acceptable is questionable, and I think it should go away, but why when COMM fields are bigger what is alloted to them in the tracepoint ends up tripping up just the python binding is something I couldn't grasp in today's session.
Namhyung, this is something open, but not caused by Ian's patchset, for which I give my:
In addition to the tags I provided patch by patch.
- Arnaldo
tools/perf/python/tracepoint.py +6 -9
@@ -33,15 +33,12 @@ def main(): if not isinstance(event, perf.sample_event): continue - print("time %u prev_comm=%s prev_pid=%d prev_prio=%d prev_state=0x%x ==> next_comm=%s next_pid=%d next_prio=%d" % ( - event.sample_time, - event.prev_comm, - event.prev_pid, - event.prev_prio, - event.prev_state, - event.next_comm, - event.next_pid, - event.next_prio)) + try: + print("time %u prev_comm=%s prev_pid=%d prev_prio=%d prev_state=0x%x ==> next_comm=%s next_pid=%d next_prio=%d" % ( + event.sample_time, event.prev_comm, event.prev_pid, event.prev_prio, event.prev_state, event.next_comm, event.next_pid, event.next_prio)) + except: + print("time %u prev_pid=%d prev_prio=%d prev_state=0x%x ==> next_pid=%d next_prio=%d" % ( + event.sample_time, event.prev_pid, event.prev_prio, event.prev_state, event.next_pid, event.next_prio)) if __name__ == '__main__': main()
tools/perf/util/python.c +2 -5
@@ -318,13 +318,10 @@ tracepoint_field(const struct pyrf_event *pe, struct tep_format_field *field)if (tep_field_is_relative (field->flags)) offset += field->offset + field->size; } -if (field->flags &TEP_FIELD_IS_STRING && -is_printable_array (data + offset, len)) { +if (field->flags &TEP_FIELD_IS_STRING ) ret =PyUnicode_FromString ((char *)data + offset); - }else { +else ret =PyByteArray_FromStringAndSize ((const char *) data + offset, len); - field->flags &= ~TEP_FIELD_IS_STRING ; - } }else { val =tep_read_number (pevent, data + field->offset, field->size);
Namhyung Kim
maintainerRe: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
Quoted 44 lines · Arnaldo Carvalho de Melo
Quoted 22 lines · Arnaldo Carvalho de Melo
So it seems to be something just in the python binding, as perf trace seems to handle it well:
( field '
prev_comm' ret=0x7f7c31f65110, raw_size=68 ) ( field 'prev_pid' ret=0x7f7c23b1bed0, raw_size=68 ) ( field 'prev_prio' ret=0x7f7c239c0030, raw_size=68 ) ( field 'prev_state' ret=0x7f7c239c0250, raw_size=68 ) time 14771421785867 prev_comm= prev_pid=1919907691 prev_prio=796026219 prev_state=0x303a32313175 ==> ( XXX '��' len=16, raw_size=68) ( field 'next_comm' ret=(nil), raw_size=68 ) Traceback (most recent call last):
File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 51, in <module>main() File "/home/acme/git/perf-tools-next/tools/perf/python/tracepoint.py", line 46, in main event.next_comm, ^^^^^^^^^^^^^^^ AttributeError: 'perf.sample_event' object has no attribute 'next_comm' root@number:/home/acme/git/perf-tools-next# cat /proc/125355/comm kworker/u112:0-i915 root@number:/home/acme/git/perf-tools-next# root@number:/home/acme/git/perf-tools-next# root@number:/home/acme/git/perf-tools-next# perf trace -e sched:sched_switch -p 125355 0.000 sched:sched_switch(prev_comm: "kworker/u112:0", prev_pid: 125355 (kworker/u112:0-), prev_prio: 120, prev_state: 128, next_comm: "swapper/6", next_prio: 120) ^Croot@number:/home/acme/git/perf-tools-next#I.e. it chops up the
prev_commsize to what is specified in the tracepoint format.Quoted 7 lines
And that
sample->raw_sizeis the same accross the sched:sched_switchraw_datas(seemingly suboptimal, most are less than 16 bytes, but probably its not guaranteed that the \0 will be there, so copy all the 16 bytes).Now to try to figure out why simply using
PyUnicode_FromStringAndSizedoesn't work...Didn't manage to make progress on this, I spent more time than I expected as I think this could be some sort of canary on some coal mine, but with the patch below, that gives up and just avoids touching the COMM fields and don't switch from string to bytearray in the binding, it runs forever, this is just a data point in case somebody wants to pursue.
That flipping from string to not string based on just one entry not being acceptable is questionable, and I think it should go away, but why when COMM fields are bigger what is alloted to them in the tracepoint ends up tripping up just the python binding is something I couldn't grasp in today's session.
Quoted 4 lines
Namhyung, this is something open, but not caused by Ian's patchset, for which I give my:
Thanks for the analysis!
Quoted 2 lines
In addition to the tags I provided patch by patch.
Ian, can you check if it works well for you?
Thanks,
Namhyung
Quoted 46 lines
tools/perf/python/tracepoint.py +6
-9@@ -33,15 +33,12 @@ def main(): if not isinstance(event, perf.sample_event): continue - print("time %u prev_comm=%s prev_pid=%d prev_prio=%d prev_state=0x%x ==> next_comm=%s next_pid=%d next_prio=%d" % ( - event.sample_time, - event.prev_comm, - event.prev_pid, - event.prev_prio, - event.prev_state, - event.next_comm, - event.next_pid, - event.next_prio)) + try: + print("time %u prev_comm=%s prev_pid=%d prev_prio=%d prev_state=0x%x ==> next_comm=%s next_pid=%d next_prio=%d" % ( + event.sample_time, event.prev_comm, event.prev_pid, event.prev_prio, event.prev_state, event.next_comm, event.next_pid, event.next_prio)) + except: + print("time %u prev_pid=%d prev_prio=%d prev_state=0x%x ==> next_pid=%d next_prio=%d" % ( + event.sample_time, event.prev_pid, event.prev_prio, event.prev_state, event.next_pid, event.next_prio)) if __name__ == '__main__': main()tools/perf/util/python.c +2
-5@@ -318,13 +318,10 @@ tracepoint_field(const struct pyrf_event *pe, struct tep_format_field *field)if (tep_field_is_relative (field->flags)) offset += field->offset + field->size; } -if (field->flags &TEP_FIELD_IS_STRING && -is_printable_array (data + offset, len)) { +if (field->flags &TEP_FIELD_IS_STRING ) ret =PyUnicode_FromString ((char *)data + offset); - }else { +else ret =PyByteArray_FromStringAndSize ((const char *) data + offset, len); - field->flags &= ~TEP_FIELD_IS_STRING ; - } }else { val =tep_read_number (pevent, data + field->offset, field->size);
Namhyung Kim
maintainerRe: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
In reply to Ian RogersQuoted 26 lines · Ian Rogers
Rather than manually configuring an evsel, switch to using
parse_eventsfor greater commonality with the rest of the perf code.Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/python/tracepoint.py | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-)tools/perf/python/tracepoint.py +0
-9@@ -5,24 +5,23 @@ import perf -class tracepoint(perf.evsel): - def __init__(self, sys, name): - config = perf.tracepoint(sys, name) - perf.evsel.__init__(self, - type = perf.TYPE_TRACEPOINT, - config = config, - freq = 0, sample_period = 1, wakeup_events = 1, - sample_type = perf.SAMPLE_PERIOD | perf.SAMPLE_TID | perf.SAMPLE_CPU | perf.SAMPLE_RAW | perf.SAMPLE_TIME) -
tools/perf/python/tracepoint.py line 16
Quoted 10 lines
tools/perf/python/tracepoint.py +6
-1def main(): - tp = tracepoint("sched", "sched_switch") cpus = perf.cpu_map() threads = perf.thread_map(-1) + evlist = perf.parse_events("sched:sched_switch", cpus, threads) + # Disable tracking of mmaps and similar that are unnecessary. + for ev in evlist: + ev.tracking = False + # Configure evsels with default record options. + evlist.config()
I think the default option uses frequency of 4000 but tracepoints want to use period of 1. Also I'm not sure if it sets the proper sample type bits namely PERF_SAMPLE_RAW.
Thanks,
Namhyung
Quoted 16 lines
tools/perf/python/tracepoint.py +5
-2+ # Simplify the sample_type and read_format of evsels + for ev in evlist: + ev.sample_type = ev.sample_type & ~perf.SAMPLE_IP + ev.read_format = 0 - evlist = perf.evlist(cpus, threads) - evlist.add(tp) evlist.open() evlist.mmap() + evlist.enable(); while True: evlist.poll(timeout = -1)Signature · 1 line
2.48.1.711.g2feabab25a-goog
Ian Rogers
reviewerRe: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
Quoted 34 lines · Namhyung Kim
Quoted 32 lines · Ian Rogers
Rather than manually configuring an evsel, switch to using
parse_eventsfor greater commonality with the rest of the perf code.Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/python/tracepoint.py | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-)tools/perf/python/tracepoint.py +2
-10@@ -5,24 +5,23 @@ import perf -class tracepoint(perf.evsel): - def __init__(self, sys, name): - config = perf.tracepoint(sys, name) - perf.evsel.__init__(self, - type = perf.TYPE_TRACEPOINT, - config = config, - freq = 0, sample_period = 1, wakeup_events = 1, - sample_type = perf.SAMPLE_PERIOD | perf.SAMPLE_TID | perf.SAMPLE_CPU | perf.SAMPLE_RAW | perf.SAMPLE_TIME) - def main(): - tp = tracepoint("sched", "sched_switch") cpus = perf.cpu_map() threads = perf.thread_map(-1) + evlist = perf.parse_events("sched:sched_switch", cpus, threads) + # Disable tracking of mmaps and similar that are unnecessary.
Quoted 8 lines
tools/perf/python/tracepoint.py line 16
Quoted 4 lines
tools/perf/python/tracepoint.py +4
-0+ for ev in evlist: + ev.tracking = False + # Configure evsels with default record options. + evlist.config()I think the default option uses frequency of 4000 but tracepoints want to use period of 1. Also I'm not sure if it sets the proper sample type bits namely
PERF_SAMPLE_RAW.
I used trace to ensure they matched. Fwiw, the sample_period for a tracepoint is set to 1 in evsel__newtp_idx:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/util/evsel.c?h=perf-tools-next#n621
and the evsel__config won't overwrite an already set sample_period:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/util/evsel.c?h=perf-tools-next#n1341
Thanks, Ian
Quoted 20 lines
Thanks,
NamhyungQuoted 16 lines
tools/perf/python/tracepoint.py +5
-3+ # Simplify the sample_type and read_format of evsels + for ev in evlist: + ev.sample_type = ev.sample_type & ~perf.SAMPLE_IP + ev.read_format = 0 - evlist = perf.evlist(cpus, threads) - evlist.add(tp) evlist.open() evlist.mmap() + evlist.enable(); while True: evlist.poll(timeout = -1) --2.48.1.711.g2feabab25a-goog
Namhyung Kim
maintainerRe: [PATCH v2 11/11] perf python tracepoint: Switch to using parse_events
Quoted 44 lines · Ian Rogers
Quoted 34 lines · Namhyung Kim
Quoted 32 lines · Ian Rogers
Rather than manually configuring an evsel, switch to using
parse_eventsfor greater commonality with the rest of the perf code.Reviewed-by Howard Chu <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/python/tracepoint.py | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-)tools/perf/python/tracepoint.py +2
-10@@ -5,24 +5,23 @@ import perf -class tracepoint(perf.evsel): - def __init__(self, sys, name): - config = perf.tracepoint(sys, name) - perf.evsel.__init__(self, - type = perf.TYPE_TRACEPOINT, - config = config, - freq = 0, sample_period = 1, wakeup_events = 1, - sample_type = perf.SAMPLE_PERIOD | perf.SAMPLE_TID | perf.SAMPLE_CPU | perf.SAMPLE_RAW | perf.SAMPLE_TIME) - def main(): - tp = tracepoint("sched", "sched_switch") cpus = perf.cpu_map() threads = perf.thread_map(-1) + evlist = perf.parse_events("sched:sched_switch", cpus, threads) + # Disable tracking of mmaps and similar that are unnecessary.Quoted 8 lines
tools/perf/python/tracepoint.py line 16
Quoted 4 lines
tools/perf/python/tracepoint.py +4
-0+ for ev in evlist: + ev.tracking = False + # Configure evsels with default record options. + evlist.config()I think the default option uses frequency of 4000 but tracepoints want to use period of 1. Also I'm not sure if it sets the proper sample type bits namely
PERF_SAMPLE_RAW.
Quoted 5 lines
I used trace to ensure they matched. Fwiw, the
sample_periodfor a tracepoint is set to 1 inevsel__newtp_idx:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/util/evsel.c?h=perf-tools-next#n621
and theevsel__configwon't overwrite an already setsample_period:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/util/evsel.c?h=perf-tools-next#n1341
Ok, it only sets the default value and it will use the attr from evsel__newtp_idx(). Thanks for checking this.
Thanks,
Namhyung
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
tools/perf/python/tracepoint.py +5
-3+ # Simplify the sample_type and read_format of evsels + for ev in evlist: + ev.sample_type = ev.sample_type & ~perf.SAMPLE_IP + ev.read_format = 0 - evlist = perf.evlist(cpus, threads) - evlist.add(tp) evlist.open() evlist.mmap() + evlist.enable(); while True: evlist.poll(timeout = -1) --2.48.1.711.g2feabab25a-goog
Hello Ian,
Quoted 8 lines · Ian Rogers
While
parse_eventsaccess in python was added, it wasn't used by any python script. In enabling this for thetracepoint.pyscript a number of latent bugs and necessary improvements were discovered.v2: Fix a whitespace issue on the evlist.config patch spotted by
Howard and add his reviewed-by tags.
Quoted 12 lines
Ian Rogers (11):
perf debug: Avoid stack overflow in recursive error message perf evlist: Add success path toevlist__create_syswide_mapsperf evsel:tp_formataccessing improvements
perf python: Add evlist enable and disable methods
perf python: Add member access to a number of evsel variables perf python: Add optional cpus and threads arguments toparse_eventsperf python: Update ungrouped evsel leader in clone
perf python: Avoid duplicated code inget_tracepoint_fieldperf python: Add evlistall_cpusaccessor
perf python: Add evlist.config to set up record options perf python tracepoint: Switch to usingparse_events
Quoted 11 lines
tools/perf/python/tracepoint.py | 23 +++--- tools/perf/util/debug.c | 2 +- tools/perf/util/evlist.c | 13 ++-- tools/perf/util/evsel.c | 16 +++- tools/perf/util/python.c | 127 ++++++++++++++++++++++++++++---- 5 files changed, 145 insertions(+), 36 deletions(-)--
2.48.1.711.g2feabab25a-goog
Quoted 8 lines · Ian Rogers
While
parse_eventsaccess in python was added, it wasn't used by any python script. In enabling this for thetracepoint.pyscript a number of latent bugs and necessary improvements were discovered.v2: Fix a whitespace issue on the evlist.config patch spotted by
Howard and add his reviewed-by tags.[...]
Applied to perf-tools-next, thanks!
Best regards,
Namhyung