perf timechart/list/treport: Interactive Textual TUIs and perf python session improvements
Ian Rogers
reviewercover perf timechart/list/treport: Interactive Textual TUIs and perf python session improvements
This series adds interactive terminal user interfaces (TUIs) built with Python's textual framework for 'perf timechart' (--tui and --live) and 'perf list' (--tui), updates 'treport' to progressively display profiles while loading in the background, adds headless shell tests for all three TUIs, and improves the 'perf' Python extension module to support responsive background and live pipe event processing:
1. Session and Python extension improvements (patches 1-5):
- Skip flushing remaining deferred samples and auxtrace events once session_done() is set so stopping event processing is immediate.
- Quietly stop processing events when a Python session callback raises an exception (e.g., to cancel a background load on quit) without corrupting TUI output with reader error messages.
- Lazily copy perf_event and perf_sample in process_events() only when a Python callback retains a reference beyond its return, shrinking struct pyrf_event to fit in CPython's small-object allocator.
- Lazily unwind and resolve sample callchains only when event.callchain is accessed.
- Release the Python GIL around perf_session__new() and perf_session__process_events() (reacquiring it in callbacks and guarding concurrent thread/comm/symbol access) so blocking pipe reads do not freeze TUI threads.
2. 'perf list --tui' and ilist test (patches 6-7):
- Add 'perf list --tui' to launch the interactive ilist script via 'perf script', and add a shell test driving ilist headlessly.
3. Progressive loading in treport and test (patches 8-9):
- Load profiles in a background thread in treport.py so the report tree and flame graph are interactive while loading, with lazy tree node creation and a headless shell test.
4. 'perf timechart --tui' and '--live' and tests (patches 10-13):
- Add ttimechart.py and 'perf timechart --tui' to interactively view, zoom, and pan per-CPU (busy, frequency, idle), per-task (running, runnable, blocked, wakeups), and I/O timelines along with a summary table.
- Add 'perf timechart --live' to stream live scheduler, power, or I/O events from a 'perf record' subprocess over a pipe with a rolling time window, pause/resume via the control pipe, and optional output recording (-o).
- Add headless shell tests covering both '--tui' and '--live' modes.
Alice Rogers (6):
perf treport: Show the profile while it loads
perf test: Add a test for the treport script
perf timechart: Add an interactive --tui mode
perf test: Add a test for perf timechart --tui
perf timechart: Add a --live mode to the TUI
perf test: Test perf timechart --live
Ian Rogers (7):
perf session: Don't flush remaining events once processing is done perf python: Quietly stop processing events when a callback raises perf python: Lazily copy events and samples from process_events perf python: Lazily resolve sample callchains
perf python: Release the GIL while processing session events perf list: Add a --tui option to launch ilist
perf test: Add a test for the ilist script
tools/perf/Documentation/perf-list.txt | 10 + tools/perf/Documentation/perf-timechart.txt | 24 + tools/perf/builtin-list.c | 34 + tools/perf/builtin-timechart.c | 114 +- tools/perf/python/treport.py | 458 ++- tools/perf/python/ttimechart.py | 2467 +++++++++++++++++ tools/perf/tests/shell/test_ilist_python.sh | 181 ++ tools/perf/tests/shell/test_treport_python.sh | 247 ++ .../tests/shell/test_ttimechart_python.sh | 537 ++++ tools/perf/util/auxtrace.c | 7 + tools/perf/util/python.c | 1191 +++++--- tools/perf/util/session.c | 8 + 12 files changed, 4754 insertions(+), 524 deletions(-) create mode 100755 tools/perf/python/ttimechart.py create mode 100755 tools/perf/tests/shell/test_ilist_python.sh create mode 100755 tools/perf/tests/shell/test_treport_python.sh create mode 100755 tools/perf/tests/shell/test_ttimechart_python.sh
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
Ian Rogers
reviewerPATCH 1/13 perf session: Don't flush remaining events once processing is done
When session_done() is set, for example by SIGINT or by a python callback requesting processing stop, the reader loops stop but the deferred samples and auxtrace data are still flushed afterwards. For large traces decoding and delivering the remaining auxtrace data can be slow, and the events are delivered to tools that asked to stop.
Discard the remaining deferred samples, and skip auxtrace__flush_events(), when session_done() is set. The ordered_events flush already honors session_done().
--- tools/perf/util/auxtrace.c | 7 +++++++ tools/perf/util/session.c | 8 ++++++++ 2 files changed, 15 insertions(+)
tools/perf/util/auxtrace.c +7 -0
@@ -2978,6 +2978,13 @@ int auxtrace__flush_events(struct perf_session *session, const struct perf_toolif (!session->auxtrace)return 0 ; +/* +* Processing was stopped early, e.g. by SIGINT, so don't decode and +* deliver the remaining trace data which can be slow. +*/ +if (session_done ()) +return 0 ; +return session->auxtrace->flush_events (session, tool); }
tools/perf/util/session.c +8 -0
@@ -2018,6 +2018,14 @@ static int session__flush_deferred_samples(struct perf_session *session,struct perf_sample sample;struct evsel *new_evsel; +if (session_done ()) { +/* Processing was stopped early, discard the sample. */ +list_del (&de->list); +free (de->event); +free (de); +continue ; + } +perf_sample__init (&sample,/*all=*/ false); ret =evlist__parse_sample (evlist, de->event, &sample);if (ret <0 ) {
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 01/13] perf session: Don't flush remaining events once processing is done
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 6 lines
When
session_done()is set, for example by SIGINT or by a python callback requesting processing stop, the reader loops stop but the deferred samples and auxtrace data are still flushed afterwards. For large traces decoding and delivering the remaining auxtrace data can be slow, and the events are delivered to tools that asked to stop.
Quoted 6 lines
Discard the remaining deferred samples, and skip
auxtrace__flush_events(), whensession_done()is set. Theordered_eventsflush already honorssession_done().Assisted-by Antigravity:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH 2/13 perf python: Quietly stop processing events when a callback raises
In reply to Ian RogersWhen a session callback raises a python exception the callback returned -1 causing reader__read_event() to print a "processing failed for event of type: ..." error. This is redundant with the python exception that is raised from process_events(), and corrupts the display of a TUI that raises from a callback to deliberately stop processing, such as when cancelling a background load.
Instead set session_done so that processing stops and return success, the pending exception is then raised by process_events(). As a single event may still cause further callbacks, for example multiple call_return callbacks followed by a sample callback, skip calling python while an exception is pending.
The session whose callback raised resets session_done after processing so later sessions process all their events. As session_done is a global, a session processed concurrently in another thread may be stopped early too. Count the stop requests so that such a session raises an error rather than silently dropping events.
--- tools/perf/util/python.c | 149 ++++++++++++++++++++++++++++++++------- 1 file changed, 122 insertions(+), 27 deletions(-)
tools/perf/util/python.c +122 -27
@@ -4120,6 +4120,11 @@ struct pyrf_session {*/ u64 call_return_last_db_id;u64 sample_last_db_id; +/** +* @callback_raised: A python callback raised an exception and requested +* that event processing stops, see pyrf_session__callback_raised(). +*/ +bool callback_raised;char *vmlinux_name;char *kallsyms_name;char *symfs;
@@ -4439,6 +4444,55 @@ static PyObject *pyrf_call_return__new(struct call_return *cr)return (PyObject *)pevent; } +/* +* Number of times a python callback has requested that event processing +* stops, see pyrf_session__callback_raised(). +*/ +static unsigned int pyrf_session__stop_requests; + +/* +* A python callback raised an exception. +* +* Returning an error to the session code would cause reader__read_event() to +* print a "processing failed for event of type: ..." error to stderr. This is +* noise, and in the case of a TUI corrupts the display, when a callback raises +* to deliberately stop processing (e.g. cancelling a background load) and the +* error message is redundant with the python exception that is raised anyway. +* +* Instead request that event processing stops by setting session_done and +* return success. The exception remains pending (PyErr_Occurred()) and +* pyrf_session__find_thread_events() returns NULL so python raises it. +* +* Note, session_done is a process-global rather than per-session. Were +* multiple sessions processing events concurrently in different python +* threads then an exception in one would stop all of them. Only the session +* whose callback raised clears session_done, and the other sessions detect +* the stop request, using pyrf_session__stop_requests, and raise an error +* rather than silently dropping events. +*/ +static int pyrf_session__callback_raised (struct pyrf_session *psession) +{ + psession->callback_raised = true; + pyrf_session__stop_requests++; + session_done =1 ; +return 0 ; +} + +/* +* session_done stops the reader loops, the ordered_events and deferred sample +* flushes, and auxtrace__flush_events(), but some callbacks may still happen +* after a callback has raised. For example, a single sample may generate +* multiple call_return callbacks, and perf_session__flush_thread_stacks() +* delivers call returns for the remaining stack entries (skipping it would +* only defer those callbacks until the threads are deleted). Calling into +* python with an exception pending is an error, and the events would be +* discarded anyway, so callbacks check this and skip calling python. +*/ +static bool pyrf_session__exception_pending (void ) +{ +return PyErr_Occurred () !=NULL ; +} +static int pyrf_session__call_return_process (struct call_return *cr,u64 *parent_db_id,void *data)
@@ -4446,7 +4500,7 @@ static int pyrf_session__call_return_process(struct call_return *cr,struct pyrf_session *psession = data;PyObject *pyevent, *ret; -if (!psession->call_return) +if (!psession->call_return ||pyrf_session__exception_pending ())return 0 ;if (!cr->db_id)
@@ -4463,12 +4517,10 @@ static int pyrf_session__call_return_process(struct call_return *cr,return -1 ; ret =PyObject_CallFunctionObjArgs (psession->call_return, pyevent,NULL ); -if (!ret) { -Py_DECREF (pyevent); -return -1 ; - } -Py_DECREF (ret);Py_DECREF (pyevent); +if (!ret) +return pyrf_session__callback_raised (psession); +Py_DECREF (ret);return 0 ; }
@@ -4480,6 +4532,9 @@ static int pyrf_session_tool__sample(const struct perf_tool *tool, struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);u64 sample_db_id = ++psession->sample_last_db_id; +if (pyrf_session__exception_pending ()) +return 0 ; +if (psession->crp) {struct addr_location al, addr_al;struct thread *thread =NULL ;
@@ -4510,6 +4565,10 @@ static int pyrf_session_tool__sample(const struct perf_tool *tool,addr_location__exit (&addr_al);addr_location__exit (&al); + +/* A call_return callback may have raised. */ +if (pyrf_session__exception_pending ()) +return 0 ; }if (psession->sample) {
@@ -4524,7 +4583,7 @@ static int pyrf_session_tool__sample(const struct perf_tool *tool, ret =PyObject_CallFunction (psession->sample,"O" , pyevent);Py_DECREF (pyevent);if (!ret) -return -1 ; +return pyrf_session__callback_raised (psession);Py_DECREF (ret); }return 0 ;
@@ -4536,9 +4595,19 @@ static int pyrf_session_tool__context_switch(const struct perf_tool *tool, struct machine *machine) { struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool); -PyObject *pyevent =pyrf_event__new (event, sample->evsel, psession->session, machine,NULL ); +PyObject *pyevent;PyObject *ret; +if (pyrf_session__exception_pending ()) { +/* +* Skip calling python, but keep the machine's thread state +* consistent as perf_event__process_switch() would have been +* called had no python callback been registered. +*/ +return perf_event__process_switch (tool, event, sample, machine); + } + + pyevent =pyrf_event__new (event, sample->evsel, psession->session, machine,NULL );if (perf_event__process_switch (tool, event, sample, machine) <0 ) {Py_XDECREF (pyevent);return -1 ;
@@ -4548,12 +4617,10 @@ static int pyrf_session_tool__context_switch(const struct perf_tool *tool,return -ENOMEM ; ret =PyObject_CallFunction (psession->context_switch,"O" , pyevent); -if (!ret) { -Py_DECREF (pyevent); -return -1 ; - } -Py_DECREF (ret);Py_DECREF (pyevent); +if (!ret) +return pyrf_session__callback_raised (psession); +Py_DECREF (ret);return 0 ; }
@@ -4563,21 +4630,23 @@ static int pyrf_session_tool__stat(const struct perf_tool *tool, { struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);struct evsel *evsel =evlist__id2evsel (session->evlist, event->stat.id); -PyObject *pyevent =pyrf_event__new (event,/*evsel=*/ NULL , psession->session, -/*machine=*/ NULL ,NULL );const char *name = evsel ?evsel__name (evsel) :"unknown" ; +PyObject *pyevent;PyObject *ret; +if (pyrf_session__exception_pending ()) +return 0 ; + + pyevent =pyrf_event__new (event,/*evsel=*/ NULL , psession->session, +/*machine=*/ NULL ,NULL );if (pyevent ==NULL )return -ENOMEM ; ret =PyObject_CallFunction (psession->stat,"Oz" , pyevent, name); -if (!ret) { -Py_DECREF (pyevent); -return -1 ; - } -Py_DECREF (ret);Py_DECREF (pyevent); +if (!ret) +return pyrf_session__callback_raised (psession); +Py_DECREF (ret);return 0 ; }
@@ -4586,20 +4655,22 @@ static int pyrf_session_tool__stat_round(const struct perf_tool *tool, union perf_event *event) { struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool); -PyObject *pyevent =pyrf_event__new (event,/*evsel=*/ NULL , psession->session, -/*machine=*/ NULL ,NULL ); +PyObject *pyevent;PyObject *ret; +if (pyrf_session__exception_pending ()) +return 0 ; + + pyevent =pyrf_event__new (event,/*evsel=*/ NULL , psession->session, +/*machine=*/ NULL ,NULL );if (pyevent ==NULL )return -ENOMEM ; ret =PyObject_CallFunction (psession->stat,"Oz" , pyevent,NULL ); -if (!ret) { -Py_DECREF (pyevent); -return -1 ; - } -Py_DECREF (ret);Py_DECREF (pyevent); +if (!ret) +return pyrf_session__callback_raised (psession); +Py_DECREF (ret);return 0 ; }
@@ -4680,6 +4751,7 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObject psession->crp =NULL ; psession->call_return_last_db_id =0 ; psession->sample_last_db_id =0 ; + psession->callback_raised = false; psession->vmlinux_name =NULL ; psession->kallsyms_name =NULL ; psession->symfs =NULL ;
@@ -4847,11 +4919,34 @@ static void pyrf_session__delete(struct pyrf_session *psession)static PyObject *pyrf_session__find_thread_events (struct pyrf_session *psession) { +unsigned int stop_requests = pyrf_session__stop_requests;int err;CHECK_INITIALIZED (psession->session,"session" ); + psession->callback_raised = false; err =perf_session__process_events (psession->session); +if (psession->callback_raised) { +/* +* Clear the early stop requested by this session's callback, in +* pyrf_session__callback_raised(), so that later sessions process +* all their events. That function is the only writer of +* session_done in the python module, so resetting it here after +* processing is sufficient and there is no need to also reset it +* before processing. +*/ + session_done =0 ; + }else if (stop_requests != pyrf_session__stop_requests && !PyErr_Occurred ()) { +/* +* A callback of a session being processed concurrently, in +* another thread, raised and set session_done which may have +* stopped processing this session's events early. Raise an error +* rather than silently dropping events. +*/ +PyErr_SetString (PyExc_RuntimeError, +"Processing events may have stopped early as a callback of a concurrently processed session raised an exception" ); +return NULL ; + }if (PyErr_Occurred ())return NULL ;
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 02/13] perf python: Quietly stop processing events when a callback raises
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 13 lines
When a session callback raises a python exception the callback returned -1 causing
reader__read_event()to print a "processing failed for event of type: ..." error. This is redundant with the python exception that is raised fromprocess_events(), and corrupts the display of a TUI that raises from a callback to deliberately stop processing, such as when cancelling a background load.Instead set
session_doneso that processing stops and return success, the pending exception is then raised byprocess_events(). As a single event may still cause further callbacks, for example multiplecall_returncallbacks followed by a sample callback, skip calling python while an exception is pending.
Quoted 8 lines
The session whose callback raised resets
session_doneafter processing so later sessions process all their events. Assession_doneis a global, a session processed concurrently in another thread may be stopped early too. Count the stop requests so that such a session raises an error rather than silently dropping events.Assisted-by Antigravity:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerPATCH 3/13 perf python: Lazily copy events and samples from process_events
In reply to Ian RogersWhen perf.session.process_events() delivers an event to a Python callback, pyrf_event__new() previously copied the raw perf_event into a 4,160-byte union perf_event embedded in struct pyrf_event and called evsel__parse_sample() a second time to re-parse the sample against the copied buffer.
In the common case, the Python callback inspects the event and returns without storing a reference to it. Avoid the event memcpy, the duplicate evsel__parse_sample() call, and the large PyObject allocation by:
1. Storing pointers to the underlying union perf_event and struct perf_sample in struct pyrf_event (using PyGetSetDef descriptors backed by PyMember_GetOne() to read fields through those pointers), shrinking struct pyrf_event so it fits in CPython's small-object allocator pool.
2. Borrowing the event and pre-parsed sample pointers from process_events() for the duration of the Python callback.
3. Checking Py_REFCNT(pyevent) > 1 after the callback returns and only allocating event_copy and re-parsing into sample_storage when the Python callback retained a reference to the event object (or immediately in pyrf_evlist__read_on_cpu() where no pre-parsed sample is passed).
--- tools/perf/util/python.c | 664 ++++++++++++++++++++++----------------- 1 file changed, 384 insertions(+), 280 deletions(-)
tools/perf/util/python.c +384 -280
@@ -70,16 +70,6 @@ PyMODINIT_FUNC PyInit_perf(void);static PyObject *pyrf_evsel__from_evsel (struct evsel *evsel); -#define member_def (type, member, ptype, help) \ - { #member, ptype, \ - offsetof(struct pyrf_event, event) + offsetof(struct type, member), \ - 0, help } - -#define sample_member_def (name, member, ptype, help) \ - { #name, ptype, \ - offsetof(struct pyrf_event, sample) + offsetof(struct perf_sample, member), \ - 0, help } -#define CHECK_INITIALIZED (ptr, msg) \ do { \ if (!(ptr)) { \
@@ -98,8 +88,12 @@ static PyObject *pyrf_evsel__from_evsel(struct evsel *evsel);struct pyrf_event {PyObject_HEAD -/** @sample: The parsed sample from the event. */ - struct perf_sample sample; +/** @sample: The parsed sample, pointing to @sample_storage or a borrowed sample. */ + struct perf_sample *sample; +/** @sample_storage: Storage for @sample when owned by this object. */ +struct perf_sample sample_storage; +/** @evsel: The evsel associated with the event. */ +struct evsel *evsel;/** @machine: The machine associated with the sample/event. */ struct machine *machine;/** @al: The address location from machine__resolve, lazily computed. */
@@ -110,11 +104,85 @@ struct pyrf_event {PyObject *callchain;/** @brstack: Resolved branch stack, eagerly computed if requested. */ PyObject *brstack; -/** @event: The underlying perf_event that may be in a file or ring buffer. */ -union perf_event event; +/** @event: The underlying perf_event, pointing to @event_copy or a borrowed event. */ +const union perf_event *event; +/** @event_copy: Copy of @event when owned by this object. */ +union perf_event *event_copy; +}; + +static const union perf_event zero_event; + +static PyObject *pyrf_event__get_member (PyObject *self,void *closure) +{ +struct pyrf_event *pevent = (void *)self; +const union perf_event *event = pevent->event ?: &zero_event; + +return PyMember_GetOne ((const char *)event, closure); +} + +static PyObject *pyrf_event__get_sample_member (PyObject *self,void *closure) +{ +struct pyrf_event *pevent = (void *)self; +const struct perf_sample *sample = pevent->sample ?: &pevent->sample_storage; + +return PyMember_GetOne ((const char *)sample, closure); +} + +#define named_member_def (field, type, member, ptype, help) \ + { .name = #field, .get = pyrf_event__get_member, .doc = help, \ + .closure = &(struct PyMemberDef){ #field, ptype, \ + offsetof(struct type, member), READONLY, help } } + +#define member_def (type, member, ptype, help) \ + named_member_def(member, type, member, ptype, help) + +struct pyrf_string_member { +size_t offset; +size_t maxlen; }; +static size_t pyrf_event__str_len (const struct pyrf_event *pevent,size_t offset,size_t maxlen) +{ +const char *base = (const char *)(pevent->event ?: &zero_event); +size_t event_size = pevent->event ? pevent->event->header.size :0 ; + +if (event_size <= offset) +return 0 ; +return strnlen (base + offset,min (event_size - offset, maxlen)); +} + +static PyObject *pyrf_event__get_string_member (PyObject *self,void *closure) +{ +struct pyrf_event *pevent = (void *)self; +const struct pyrf_string_member *sm = closure; +const char *base = (const char *)(pevent->event ?: &zero_event); + +return PyUnicode_FromStringAndSize (base + sm->offset, +pyrf_event__str_len (pevent, sm->offset, sm->maxlen)); +} + +#define string_member_def (type, member, help) \ + { .name = #member, .get = pyrf_event__get_string_member, .doc = help, \ + .closure = &(struct pyrf_string_member){ offsetof(struct type, member), \ + sizeof(((struct type *)0)->member) } } + +#define sample_member_def (field, member, ptype, help) \ + { .name = #field, .get = pyrf_event__get_sample_member, .doc = help, \ + .closure = &(struct PyMemberDef){ #field, ptype, \ + offsetof(struct perf_sample, member), READONLY, help } } + +static PyObject *pyrf_event__get_evsel (PyObject *self,void *closure__maybe_unused ) +{ +struct pyrf_event *pevent = (void *)self; + +if (!pevent->sample || !pevent->sample->evsel) + Py_RETURN_NONE; + +return pyrf_evsel__from_evsel (pevent->sample->evsel); +} +#define sample_members \ + { .name = "evsel", .get = pyrf_event__get_evsel, .doc = "tracking event." }, \ sample_member_def(sample_pid, pid, T_INT, "event pid"), \ sample_member_def(sample_tid, tid, T_INT, "event tid"), \ sample_member_def(sample_time, time, T_ULONGLONG, "event timestamp"), \
@@ -125,39 +193,21 @@ struct pyrf_event {sample_member_def (sample_period, period,T_ULONGLONG ,"event period" ), \sample_member_def (sample_cpu, cpu,T_UINT ,"event cpu" ), -static PyObject *pyrf_event__get_evsel(PyObject *self, void *closure __maybe_unused) -{ -struct pyrf_event *pevent = (void *)self; - -if (!pevent->sample.evsel) - Py_RETURN_NONE; - -return pyrf_evsel__from_evsel (pevent->sample.evsel); -} - -static PyGetSetDef pyrf_event__getset[] = { - { - .name ="evsel" , - .get = pyrf_event__get_evsel, - .set =NULL , - .doc ="tracking event." , - }, - { .name =NULL , }, -}; - static void pyrf_event__delete(struct pyrf_event *pevent) {if (pevent->al_resolved)addr_location__exit (&pevent->al);Py_XDECREF (pevent->callchain);Py_XDECREF (pevent->brstack); -perf_sample__exit (&pevent->sample); +perf_sample__exit (&pevent->sample_storage); +evsel__put (pevent->evsel); +free (pevent->event_copy);Py_TYPE (pevent)->tp_free ((PyObject *)pevent); }static const char pyrf_mmap_event__doc[] =PyDoc_STR ("perf mmap event object." ); -static PyMemberDef pyrf_mmap_event__members[] = { +static PyGetSetDef pyrf_mmap_event__getset[] = { sample_membersmember_def (perf_event_header, type,T_UINT ,"event type" ),member_def (perf_event_header, misc,T_USHORT ,"event misc" ),
@@ -166,7 +216,7 @@ static PyMemberDef pyrf_mmap_event__members[] = {member_def (perf_record_mmap, start,T_ULONGLONG ,"start of the map" ),member_def (perf_record_mmap, len,T_ULONGLONG ,"map length" ),member_def (perf_record_mmap, pgoff,T_ULONGLONG ,"page offset" ), -member_def (perf_record_mmap, filename,T_STRING_INPLACE ,"backing store" ), +string_member_def (perf_record_mmap, filename,"backing store" ), { .name = NULL, }, };
@@ -174,13 +224,15 @@ static PyObject *pyrf_mmap_event__repr(const struct pyrf_event *pevent) {PyObject *ret;char *s; +int fn_len =pyrf_event__str_len (pevent, offsetof(struct perf_record_mmap , filename), +sizeof (pevent->event->mmap.filename));if (asprintf (&s,"{ type: mmap, pid: %u, tid: %u, start: %#" PRI_lx64", " "length: %#" PRI_lx64", offset: %#" PRI_lx64", " -"filename: %s }" , - pevent->event.mmap.pid, pevent->event.mmap.tid, - pevent->event.mmap.start, pevent->event.mmap.len, - pevent->event.mmap.pgoff, pevent->event.mmap.filename) <0 ) { +"filename: %.*s }" , + pevent->event->mmap.pid, pevent->event->mmap.tid, + pevent->event->mmap.start, pevent->event->mmap.len, + pevent->event->mmap.pgoff, fn_len, pevent->event->mmap.filename) <0 ) { ret =PyErr_NoMemory (); }else { ret =PyUnicode_FromString (s);
@@ -196,8 +248,7 @@ static PyTypeObject pyrf_mmap_event__type = { .tp_dealloc = (destructor)pyrf_event__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_mmap_event__doc, - .tp_members = pyrf_mmap_event__members, - .tp_getset = pyrf_event__getset, + .tp_getset = pyrf_mmap_event__getset, .tp_repr = (reprfunc)pyrf_mmap_event__repr, };
@@ -207,64 +258,69 @@ static PyObject *pyrf_mmap2_event__get_maj(PyObject *self, void *closure __maybe {struct pyrf_event *pevent = (void *)self; -if (pevent->event.header.misc &PERF_RECORD_MISC_MMAP_BUILD_ID ) +if (pevent->event->header.misc &PERF_RECORD_MISC_MMAP_BUILD_ID ) Py_RETURN_NONE; -return PyLong_FromUnsignedLong (pevent->event.mmap2.maj); +return PyLong_FromUnsignedLong (pevent->event->mmap2.maj); }static PyObject *pyrf_mmap2_event__get_min (PyObject *self,void *closure__maybe_unused ) {struct pyrf_event *pevent = (void *)self; -if (pevent->event.header.misc &PERF_RECORD_MISC_MMAP_BUILD_ID ) +if (pevent->event->header.misc &PERF_RECORD_MISC_MMAP_BUILD_ID ) Py_RETURN_NONE; -return PyLong_FromUnsignedLong (pevent->event.mmap2.min); +return PyLong_FromUnsignedLong (pevent->event->mmap2.min); }static PyObject *pyrf_mmap2_event__get_ino (PyObject *self,void *closure__maybe_unused ) {struct pyrf_event *pevent = (void *)self; -if (pevent->event.header.misc &PERF_RECORD_MISC_MMAP_BUILD_ID ) +if (pevent->event->header.misc &PERF_RECORD_MISC_MMAP_BUILD_ID ) Py_RETURN_NONE; -return PyLong_FromUnsignedLongLong (pevent->event.mmap2.ino); +return PyLong_FromUnsignedLongLong (pevent->event->mmap2.ino); }static PyObject *pyrf_mmap2_event__get_ino_generation (PyObject *self,void *closure__maybe_unused ) {struct pyrf_event *pevent = (void *)self; -if (pevent->event.header.misc &PERF_RECORD_MISC_MMAP_BUILD_ID ) +if (pevent->event->header.misc &PERF_RECORD_MISC_MMAP_BUILD_ID ) Py_RETURN_NONE; -return PyLong_FromUnsignedLongLong (pevent->event.mmap2.ino_generation); +return PyLong_FromUnsignedLongLong (pevent->event->mmap2.ino_generation); }static PyObject *pyrf_mmap2_event__get_build_id (PyObject *self,void *closure__maybe_unused ) {struct pyrf_event *pevent = (void *)self; -if (!(pevent->event.header.misc &PERF_RECORD_MISC_MMAP_BUILD_ID )) +if (!(pevent->event->header.misc &PERF_RECORD_MISC_MMAP_BUILD_ID )) Py_RETURN_NONE; -int size = pevent->event.mmap2.build_id_size; +int size = pevent->event->mmap2.build_id_size;if (size >20 ) size =20 ; -return PyBytes_FromStringAndSize ((const char *)pevent->event.mmap2.build_id, size); +return PyBytes_FromStringAndSize ((const char *)pevent->event->mmap2.build_id, size); }static PyGetSetDef pyrf_mmap2_event__getset[] = { - { - .name ="evsel" , - .get = pyrf_event__get_evsel, - .set =NULL , - .doc ="tracking event." , - }, + sample_members +member_def (perf_event_header, type,T_UINT ,"event type" ), +member_def (perf_event_header, misc,T_USHORT ,"event misc" ), +member_def (perf_record_mmap2, pid,T_UINT ,"event pid" ), +member_def (perf_record_mmap2, tid,T_UINT ,"event tid" ), +member_def (perf_record_mmap2, start,T_ULONGLONG ,"start of the map" ), +member_def (perf_record_mmap2, len,T_ULONGLONG ,"map length" ), +member_def (perf_record_mmap2, pgoff,T_ULONGLONG ,"page offset" ), +member_def (perf_record_mmap2, prot,T_UINT ,"protection" ), +member_def (perf_record_mmap2, flags,T_UINT ,"flags" ), +string_member_def (perf_record_mmap2, filename,"backing store" ), { .name ="maj" , .get = pyrf_mmap2_event__get_maj,
@@ -298,31 +354,18 @@ static PyGetSetDef pyrf_mmap2_event__getset[] = { { .name = NULL, }, }; -static PyMemberDef pyrf_mmap2_event__members[] = { - sample_members -member_def (perf_event_header, type,T_UINT ,"event type" ), -member_def (perf_event_header, misc,T_USHORT ,"event misc" ), -member_def (perf_record_mmap2, pid,T_UINT ,"event pid" ), -member_def (perf_record_mmap2, tid,T_UINT ,"event tid" ), -member_def (perf_record_mmap2, start,T_ULONGLONG ,"start of the map" ), -member_def (perf_record_mmap2, len,T_ULONGLONG ,"map length" ), -member_def (perf_record_mmap2, pgoff,T_ULONGLONG ,"page offset" ), -member_def (perf_record_mmap2, prot,T_UINT ,"protection" ), -member_def (perf_record_mmap2, flags,T_UINT ,"flags" ), -member_def (perf_record_mmap2, filename,T_STRING_INPLACE ,"backing store" ), - { .name =NULL , }, -}; -static PyObject *pyrf_mmap2_event__repr (const struct pyrf_event *pevent) {PyObject *ret;char *s; - -if (asprintf (&s,"{ type: mmap2, pid: %u, tid: %u, start: %#" PRI_lx64", length: %#" PRI_lx64", offset: %#" PRI_lx64", flags: %#x, prot: %#x, filename: %s }" , - pevent->event.mmap2.pid, pevent->event.mmap2.tid, - pevent->event.mmap2.start, pevent->event.mmap2.len, - pevent->event.mmap2.pgoff, pevent->event.mmap2.flags, - pevent->event.mmap2.prot, pevent->event.mmap2.filename) <0 ) +int fn_len =pyrf_event__str_len (pevent, offsetof(struct perf_record_mmap2 , filename), +sizeof (pevent->event->mmap2.filename)); + +if (asprintf (&s,"{ type: mmap2, pid: %u, tid: %u, start: %#" PRI_lx64", length: %#" PRI_lx64", offset: %#" PRI_lx64", flags: %#x, prot: %#x, filename: %.*s }" , + pevent->event->mmap2.pid, pevent->event->mmap2.tid, + pevent->event->mmap2.start, pevent->event->mmap2.len, + pevent->event->mmap2.pgoff, pevent->event->mmap2.flags, + pevent->event->mmap2.prot, fn_len, pevent->event->mmap2.filename) <0 )return PyErr_NoMemory (); ret =PyUnicode_FromString (s);
@@ -337,14 +380,13 @@ static PyTypeObject pyrf_mmap2_event__type = { .tp_dealloc = (destructor)pyrf_event__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_mmap2_event__doc, - .tp_members = pyrf_mmap2_event__members, .tp_getset = pyrf_mmap2_event__getset, .tp_repr = (reprfunc)pyrf_mmap2_event__repr, };static const char pyrf_task_event__doc[] =PyDoc_STR ("perf task (fork/exit) event object." ); -static PyMemberDef pyrf_task_event__members[] = { +static PyGetSetDef pyrf_task_event__getset[] = { sample_membersmember_def (perf_event_header, type,T_UINT ,"event type" ),member_def (perf_record_fork, pid,T_UINT ,"event pid" ),
@@ -359,12 +401,12 @@ static PyObject *pyrf_task_event__repr(const struct pyrf_event *pevent) {return PyUnicode_FromFormat ("{ type: %s, pid: %u, ppid: %u, tid: %u, " "ptid: %u, time: %" PRI_lu64"}" , - pevent->event.header.type ==PERF_RECORD_FORK ?"fork" :"exit" , - pevent->event.fork.pid, - pevent->event.fork.ppid, - pevent->event.fork.tid, - pevent->event.fork.ptid, - pevent->event.fork.time); + pevent->event->header.type ==PERF_RECORD_FORK ?"fork" :"exit" , + pevent->event->fork.pid, + pevent->event->fork.ppid, + pevent->event->fork.tid, + pevent->event->fork.ptid, + pevent->event->fork.time); }static PyTypeObject pyrf_task_event__type = {
@@ -374,28 +416,36 @@ static PyTypeObject pyrf_task_event__type = { .tp_dealloc = (destructor)pyrf_event__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_task_event__doc, - .tp_members = pyrf_task_event__members, - .tp_getset = pyrf_event__getset, + .tp_getset = pyrf_task_event__getset, .tp_repr = (reprfunc)pyrf_task_event__repr, };static const char pyrf_comm_event__doc[] =PyDoc_STR ("perf comm event object." ); -static PyMemberDef pyrf_comm_event__members[] = { +static PyGetSetDef pyrf_comm_event__getset[] = { sample_membersmember_def (perf_event_header, type,T_UINT ,"event type" ),member_def (perf_record_comm, pid,T_UINT ,"event pid" ),member_def (perf_record_comm, tid,T_UINT ,"event tid" ), -member_def (perf_record_comm, comm,T_STRING_INPLACE ,"process name" ), +string_member_def (perf_record_comm, comm,"process name" ), { .name =NULL , }, };static PyObject *pyrf_comm_event__repr (const struct pyrf_event *pevent) { -return PyUnicode_FromFormat ("{ type: comm, pid: %u, tid: %u, comm: %s }" , - pevent->event.comm.pid, - pevent->event.comm.tid, - pevent->event.comm.comm); +PyObject *ret; +char *s; +int comm_len =pyrf_event__str_len (pevent, offsetof(struct perf_record_comm , comm), +sizeof (pevent->event->comm.comm)); + +if (asprintf (&s,"{ type: comm, pid: %u, tid: %u, comm: %.*s }" , + pevent->event->comm.pid, pevent->event->comm.tid, + comm_len, pevent->event->comm.comm) <0 ) +return PyErr_NoMemory (); + + ret =PyUnicode_FromString (s); +free (s); +return ret; }static PyTypeObject pyrf_comm_event__type = {
@@ -405,14 +455,13 @@ static PyTypeObject pyrf_comm_event__type = { .tp_dealloc = (destructor)pyrf_event__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_comm_event__doc, - .tp_members = pyrf_comm_event__members, - .tp_getset = pyrf_event__getset, + .tp_getset = pyrf_comm_event__getset, .tp_repr = (reprfunc)pyrf_comm_event__repr, };static const char pyrf_throttle_event__doc[] =PyDoc_STR ("perf throttle event object." ); -static PyMemberDef pyrf_throttle_event__members[] = { +static PyGetSetDef pyrf_throttle_event__getset[] = { sample_membersmember_def (perf_event_header, type,T_UINT ,"event type" ),member_def (perf_record_throttle, time,T_ULONGLONG ,"timestamp" ),
@@ -424,11 +473,11 @@ static PyMemberDef pyrf_throttle_event__members[] = {static PyObject *pyrf_throttle_event__repr (const struct pyrf_event *pevent) {const struct perf_record_throttle *te = (const struct perf_record_throttle *) - (&pevent->event.header +1 ); + (&pevent->event->header +1 );return PyUnicode_FromFormat ("{ type: %sthrottle, time: %" PRI_lu64", id: %" PRI_lu64", stream_id: %" PRI_lu64" }" , - pevent->event.header.type ==PERF_RECORD_THROTTLE ?"" :"un" , + pevent->event->header.type ==PERF_RECORD_THROTTLE ?"" :"un" , te->time, te->id, te->stream_id); }
@@ -439,14 +488,13 @@ static PyTypeObject pyrf_throttle_event__type = { .tp_dealloc = (destructor)pyrf_event__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_throttle_event__doc, - .tp_members = pyrf_throttle_event__members, - .tp_getset = pyrf_event__getset, + .tp_getset = pyrf_throttle_event__getset, .tp_repr = (reprfunc)pyrf_throttle_event__repr, };static const char pyrf_lost_event__doc[] =PyDoc_STR ("perf lost event object." ); -static PyMemberDef pyrf_lost_event__members[] = { +static PyGetSetDef pyrf_lost_event__getset[] = { sample_membersmember_def (perf_event_header, type,T_UINT ,"event type" ),member_def (perf_record_lost, id,T_ULONGLONG ,"event id" ),
@@ -461,7 +509,7 @@ static PyObject *pyrf_lost_event__repr(const struct pyrf_event *pevent)if (asprintf (&s,"{ type: lost, id: %#" PRI_lx64", " "lost: %#" PRI_lx64" }" , - pevent->event.lost.id, pevent->event.lost.lost) <0 ) { + pevent->event->lost.id, pevent->event->lost.lost) <0 ) { ret =PyErr_NoMemory (); }else { ret =PyUnicode_FromString (s);
@@ -477,14 +525,13 @@ static PyTypeObject pyrf_lost_event__type = { .tp_dealloc = (destructor)pyrf_event__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_lost_event__doc, - .tp_members = pyrf_lost_event__members, - .tp_getset = pyrf_event__getset, + .tp_getset = pyrf_lost_event__getset, .tp_repr = (reprfunc)pyrf_lost_event__repr, };static const char pyrf_stat_event__doc[] =PyDoc_STR ("perf stat event object." ); -static PyMemberDef pyrf_stat_event__members[] = { +static PyGetSetDef pyrf_stat_event__getset[] = { sample_membersmember_def (perf_event_header, type,T_UINT ,"event type" ),member_def (perf_record_stat, id,T_ULONGLONG ,"event id" ),
@@ -498,14 +545,16 @@ static PyMemberDef pyrf_stat_event__members[] = {static PyObject *pyrf_stat_event__repr (const struct pyrf_event *pevent) { +const union perf_event *event = pevent->event ?: &zero_event; +return PyUnicode_FromFormat ("{ type: stat, id: %llu, cpu: %u, thread: %u, val: %llu, ena: %llu, run: %llu }" , - pevent->event.stat.id, - pevent->event.stat.cpu, - pevent->event.stat.thread, - pevent->event.stat.val, - pevent->event.stat.ena, - pevent->event.stat.run); + event->stat.id, + event->stat.cpu, + event->stat.thread, + event->stat.val, + event->stat.ena, + event->stat.run); }static PyTypeObject pyrf_stat_event__type = {
@@ -516,28 +565,27 @@ static PyTypeObject pyrf_stat_event__type = { .tp_dealloc = (destructor)pyrf_event__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_stat_event__doc, - .tp_members = pyrf_stat_event__members, - .tp_getset = pyrf_event__getset, + .tp_getset = pyrf_stat_event__getset, .tp_repr = (reprfunc)pyrf_stat_event__repr, };static const char pyrf_stat_round_event__doc[] =PyDoc_STR ("perf stat round event object." ); -static PyMemberDef pyrf_stat_round_event__members[] = { +static PyGetSetDef pyrf_stat_round_event__getset[] = { sample_membersmember_def (perf_event_header, type,T_UINT ,"event type" ), - { .name ="stat_round_type" , .type =T_ULONGLONG , - .offset = offsetof(struct pyrf_event , event) + offsetof(struct perf_record_stat_round , type), - .doc ="round type" }, +named_member_def (stat_round_type, perf_record_stat_round, type,T_ULONGLONG ,"round type" ),member_def (perf_record_stat_round, time,T_ULONGLONG ,"round time" ), { .name =NULL , }, };static PyObject *pyrf_stat_round_event__repr (const struct pyrf_event *pevent) { +const union perf_event *event = pevent->event ?: &zero_event; +return PyUnicode_FromFormat ("{ type: stat_round, type: %llu, time: %llu }" , - pevent->event.stat_round.type, - pevent->event.stat_round.time); + event->stat_round.type, + event->stat_round.time); }static PyTypeObject pyrf_stat_round_event__type = {
@@ -548,14 +596,13 @@ static PyTypeObject pyrf_stat_round_event__type = { .tp_dealloc = (destructor)pyrf_event__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_stat_round_event__doc, - .tp_members = pyrf_stat_round_event__members, - .tp_getset = pyrf_event__getset, + .tp_getset = pyrf_stat_round_event__getset, .tp_repr = (reprfunc)pyrf_stat_round_event__repr, };static const char pyrf_read_event__doc[] =PyDoc_STR ("perf read event object." ); -static PyMemberDef pyrf_read_event__members[] = { +static PyGetSetDef pyrf_read_event__getset[] = { sample_membersmember_def (perf_event_header, type,T_UINT ,"event type" ),member_def (perf_record_read, pid,T_UINT ,"event pid" ),
@@ -566,8 +613,8 @@ static PyMemberDef pyrf_read_event__members[] = {static PyObject *pyrf_read_event__repr (const struct pyrf_event *pevent) {return PyUnicode_FromFormat ("{ type: read, pid: %u, tid: %u }" , - pevent->event.read.pid, - pevent->event.read.tid); + pevent->event->read.pid, + pevent->event->read.tid);/* * FIXME: return the array of read values, * making this method useful ;-)
@@ -581,27 +628,12 @@ static PyTypeObject pyrf_read_event__type = { .tp_dealloc = (destructor)pyrf_event__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_read_event__doc, - .tp_members = pyrf_read_event__members, - .tp_getset = pyrf_event__getset, + .tp_getset = pyrf_read_event__getset, .tp_repr = (reprfunc)pyrf_read_event__repr, };static const char pyrf_sample_event__doc[] =PyDoc_STR ("perf sample event object." ); -static PyMemberDef pyrf_sample_event__members[] = { - sample_members -sample_member_def (sample_ip, ip,T_ULONGLONG ,"event ip" ), -sample_member_def (sample_addr, addr,T_ULONGLONG ,"event addr" ), -sample_member_def (sample_phys_addr, phys_addr,T_ULONGLONG ,"event physical addr" ), -sample_member_def (sample_weight, weight,T_ULONGLONG ,"event weight" ), -sample_member_def (sample_data_src, data_src,T_ULONGLONG ,"event data source" ), -sample_member_def (sample_insn_count, insn_cnt,T_ULONGLONG ,"event instruction count" ), -sample_member_def (sample_cyc_count, cyc_cnt,T_ULONGLONG ,"event cycle count" ), -sample_member_def (flags, flags,T_UINT ,"event flags" ), -member_def (perf_event_header, type,T_UINT ,"event type" ), - { .name =NULL , }, -}; -static PyObject *pyrf_sample_event__repr (const struct pyrf_event *pevent) {PyObject *ret;
@@ -619,16 +651,18 @@ static PyObject *pyrf_sample_event__repr(const struct pyrf_event *pevent)#ifdef HAVE_LIBTRACEEVENT static bool is_tracepoint (const struct pyrf_event *pevent) { -if (!pevent->sample.evsel) +struct evsel *evsel = pevent->sample->evsel ?: pevent->evsel; + +if (!evsel)return false; -return pevent->sample.evsel->core.attr.type ==PERF_TYPE_TRACEPOINT ; +return evsel->core.attr.type ==PERF_TYPE_TRACEPOINT ; }static PyObject *tracepoint_field (const struct pyrf_event *pe,struct tep_format_field *field) {struct tep_handle *pevent = field->event->tep; -void *data = pe->sample.raw_data; +void *data = pe->sample->raw_data;PyObject *ret =NULL ;unsigned long long val;unsigned int offset, len;
@@ -668,7 +702,7 @@ tracepoint_field(const struct pyrf_event *pe, struct tep_format_field *field)static PyObject *get_tracepoint_field (struct pyrf_event *pevent,PyObject *attr_name) { -struct evsel *evsel = pevent->sample.evsel; +struct evsel *evsel = pevent->sample->evsel ?: pevent->evsel;struct tep_event *tp_format =evsel__tp_format (evsel);struct tep_format_field *field;
@@ -693,7 +727,7 @@ get_tracepoint_field(struct pyrf_event *pevent, PyObject *attr_name)static int pyrf_sample_event__resolve_al (struct pyrf_event *pevent) { -struct evsel *evsel = pevent->sample.evsel; +struct evsel *evsel = pevent->sample->evsel ?: pevent->evsel;struct evlist *evlist = evsel ? evsel->evlist :NULL ;struct perf_session *session = evlist ?evlist__session (evlist) :NULL ;struct machine *machine;
@@ -714,7 +748,7 @@ static int pyrf_sample_event__resolve_al(struct pyrf_event *pevent) machine = pevent->machine ? pevent->machine : &session->machines.host;addr_location__init (&pevent->al); -if (machine__resolve (machine, &pevent->al, &pevent->sample) <0 ) { +if (machine__resolve (machine, &pevent->al, pevent->sample) <0 ) {addr_location__exit (&pevent->al);return -1 ; }
@@ -819,17 +853,17 @@ static PyObject *pyrf_sample_event__get_sym_end(struct pyrf_event *pevent,static PyObject *pyrf_sample_event__get_raw_buf (struct pyrf_event *pevent,void *closure__maybe_unused ) { -if (pevent->event.header.type !=PERF_RECORD_SAMPLE ) +if (pevent->event->header.type !=PERF_RECORD_SAMPLE ) Py_RETURN_NONE; -return PyBytes_FromStringAndSize ((const char *)pevent->sample.raw_data, - pevent->sample.raw_size); +return PyBytes_FromStringAndSize ((const char *)pevent->sample->raw_data, + pevent->sample->raw_size); }static PyObject *pyrf_sample_event__srccode (PyObject *self,PyObject *args) {struct pyrf_event *pevent = (void *)self; -u64 addr = pevent->sample.ip; +u64 addr = pevent->sample->ip;char *srcfile =NULL ;char *srccode =NULL ;unsigned int line =0 ;
@@ -838,8 +872,8 @@ static PyObject *pyrf_sample_event__srccode(PyObject *self, PyObject *args)struct addr_location al;/* -* addr defaults to pevent->sample.ip when omitted, and when -* addr != pevent->sample.ip a local addr_location is resolved so +* addr defaults to pevent->sample->ip when omitted, and when +* addr != pevent->sample->ip a local addr_location is resolved so * callers can inspect callchain/branch addresses without mutating * the sample's cached pevent->al. */
@@ -849,9 +883,9 @@ static PyObject *pyrf_sample_event__srccode(PyObject *self, PyObject *args)if (pyrf_sample_event__resolve_al (pevent) <0 ) Py_RETURN_NONE; -if (addr != pevent->sample.ip) { +if (addr != pevent->sample->ip) {addr_location__init (&al); -thread__find_symbol_fb (pevent->al.thread, pevent->sample.cpumode, addr, &al); +thread__find_symbol_fb (pevent->al.thread, pevent->sample->cpumode, addr, &al); }else {addr_location__init (&al); al.thread =thread__get (pevent->al.thread);
@@ -905,14 +939,14 @@ static PyObject *pyrf_sample_event__insn(PyObject *self, PyObject *args __maybe_if (!machine) Py_RETURN_NONE; -if (pevent->sample.ip && !pevent->sample.insn_len) -perf_sample__fetch_insn (&pevent->sample, thread, machine); +if (pevent->sample->ip && !pevent->sample->insn_len) +perf_sample__fetch_insn (pevent->sample, thread, machine); -if (!pevent->sample.insn_len) +if (!pevent->sample->insn_len) Py_RETURN_NONE; -return PyBytes_FromStringAndSize ((const char *)pevent->sample.insn, - pevent->sample.insn_len); +return PyBytes_FromStringAndSize ((const char *)pevent->sample->insn, + pevent->sample->insn_len); }struct pyrf_callchain_node {
@@ -1242,8 +1276,8 @@ static int pyrf_sample_event__resolve_addr_al(struct pyrf_event *pevent,if (pyrf_sample_event__resolve_al (pevent) <0 || !pevent->al.thread)return -1 ; -thread__find_symbol_fb (pevent->al.thread, pevent->sample.cpumode, - pevent->sample.addr, addr_al); +thread__find_symbol_fb (pevent->al.thread, pevent->sample->cpumode, + pevent->sample->addr, addr_al);return 0 ; }
@@ -1295,23 +1329,32 @@ static PyObject *pyrf_sample_event__get_addr_sym_offset(struct pyrf_event *pevenstatic PyObject *pyrf_sample_event__get_branch_type (struct pyrf_event *pevent,void *closure__maybe_unused ) { -return PyLong_FromUnsignedLong (pevent->sample.flags &PERF_BRANCH_MASK ); +return PyLong_FromUnsignedLong (pevent->sample->flags &PERF_BRANCH_MASK ); }static PyObject *pyrf_sample_event__get_in_tx (struct pyrf_event *pevent,void *closure__maybe_unused ) { -return PyLong_FromUnsignedLong (!!(pevent->sample.flags &PERF_IP_FLAG_IN_TX )); +return PyLong_FromUnsignedLong (!!(pevent->sample->flags &PERF_IP_FLAG_IN_TX )); }static PyObject *pyrf_sample_event__get_transaction (struct pyrf_event *pevent,void *closure__maybe_unused ) { -return PyLong_FromUnsignedLongLong (pevent->sample.transaction); +return PyLong_FromUnsignedLongLong (pevent->sample->transaction); }static PyGetSetDef pyrf_sample_event__getset[] = { - + sample_members +sample_member_def (sample_ip, ip,T_ULONGLONG ,"event ip" ), +sample_member_def (sample_addr, addr,T_ULONGLONG ,"event addr" ), +sample_member_def (sample_phys_addr, phys_addr,T_ULONGLONG ,"event physical addr" ), +sample_member_def (sample_weight, weight,T_ULONGLONG ,"event weight" ), +sample_member_def (sample_data_src, data_src,T_ULONGLONG ,"event data source" ), +sample_member_def (sample_insn_count, insn_cnt,T_ULONGLONG ,"event instruction count" ), +sample_member_def (sample_cyc_count, cyc_cnt,T_ULONGLONG ,"event cycle count" ), +sample_member_def (flags, flags,T_UINT ,"event flags" ), +member_def (perf_event_header, type,T_UINT ,"event type" ), { .name ="addr_dso" , .get = (getter )pyrf_sample_event__get_addr_dso,
@@ -1361,12 +1404,6 @@ static PyGetSetDef pyrf_sample_event__getset[] = { .set = NULL, .doc = "event raw buffer.", }, - { - .name = "evsel", - .get = pyrf_event__get_evsel, - .set = NULL, - .doc = "tracking event.", - }, { .name = "dso", .get = (getter)pyrf_sample_event__get_dso,
@@ -1453,7 +1490,6 @@ static PyTypeObject pyrf_sample_event__type = { .tp_dealloc = (destructor)pyrf_event__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_sample_event__doc, - .tp_members = pyrf_sample_event__members, .tp_getset = pyrf_sample_event__getset, .tp_methods = pyrf_sample_event__methods, .tp_repr = (reprfunc)pyrf_sample_event__repr,
@@ -1462,36 +1498,26 @@ static PyTypeObject pyrf_sample_event__type = {static const char pyrf_context_switch_event__doc[] =PyDoc_STR ("perf context_switch event object." ); -static PyMemberDef pyrf_context_switch_event__members[] = { - sample_members -member_def (perf_event_header, type,T_UINT ,"event type" ), -member_def (perf_event_header, misc,T_USHORT ,"event misc" ), - { .name =NULL , }, -}; -static PyObject *pyrf_context_switch_event__get_next_prev_pid (const struct pyrf_event *pevent,void *closure__maybe_unused ) { -if (pevent->event.header.type ==PERF_RECORD_SWITCH_CPU_WIDE ) -return PyLong_FromUnsignedLong (pevent->event.context_switch.next_prev_pid); +if (pevent->event->header.type ==PERF_RECORD_SWITCH_CPU_WIDE ) +return PyLong_FromUnsignedLong (pevent->event->context_switch.next_prev_pid); Py_RETURN_NONE; }static PyObject *pyrf_context_switch_event__get_next_prev_tid (const struct pyrf_event *pevent,void *closure__maybe_unused ) { -if (pevent->event.header.type ==PERF_RECORD_SWITCH_CPU_WIDE ) -return PyLong_FromUnsignedLong (pevent->event.context_switch.next_prev_tid); +if (pevent->event->header.type ==PERF_RECORD_SWITCH_CPU_WIDE ) +return PyLong_FromUnsignedLong (pevent->event->context_switch.next_prev_tid); Py_RETURN_NONE; }static PyGetSetDef pyrf_context_switch_event__getset[] = { - { - .name ="evsel" , - .get = pyrf_event__get_evsel, - .set =NULL , - .doc ="tracking event." , - }, + sample_members +member_def (perf_event_header, type,T_UINT ,"event type" ), +member_def (perf_event_header, misc,T_USHORT ,"event misc" ), { .name ="next_prev_pid" , .get = (getter )pyrf_context_switch_event__get_next_prev_pid,
@@ -1513,15 +1539,15 @@ static PyObject *pyrf_context_switch_event__repr(const struct pyrf_event *peventchar *s;int res; -if (pevent->event.header.type ==PERF_RECORD_SWITCH_CPU_WIDE ) { +if (pevent->event->header.type ==PERF_RECORD_SWITCH_CPU_WIDE ) { res =asprintf (&s,"{ type: context_switch, next_prev_pid: %u, next_prev_tid: %u, switch_out: %u }" , - pevent->event.context_switch.next_prev_pid, - pevent->event.context_switch.next_prev_tid, - !!(pevent->event.header.misc &PERF_RECORD_MISC_SWITCH_OUT )); + pevent->event->context_switch.next_prev_pid, + pevent->event->context_switch.next_prev_tid, + !!(pevent->event->header.misc &PERF_RECORD_MISC_SWITCH_OUT )); }else { res =asprintf (&s,"{ type: context_switch, switch_out: %u }" , - !!(pevent->event.header.misc &PERF_RECORD_MISC_SWITCH_OUT )); + !!(pevent->event->header.misc &PERF_RECORD_MISC_SWITCH_OUT )); }if (res <0 ) { ret =PyErr_NoMemory ();
@@ -1539,7 +1565,6 @@ static PyTypeObject pyrf_context_switch_event__type = { .tp_dealloc = (destructor)pyrf_event__delete, .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, .tp_doc = pyrf_context_switch_event__doc, - .tp_members = pyrf_context_switch_event__members, .tp_getset = pyrf_context_switch_event__getset, .tp_repr = (reprfunc)pyrf_context_switch_event__repr, };
@@ -1614,15 +1639,102 @@ static PyTypeObject *pyrf_event__type[] = { [PERF_RECORD_STAT_ROUND] = &pyrf_stat_round_event__type, }; +static int pyrf_event__copy (struct pyrf_event *pevent) +{ +const union perf_event *orig_event = pevent->event; +const struct perf_sample *orig_sample = pevent->sample; +size_t copy_size = orig_event->header.size; +int err = -EINVAL ; + +if (pevent->event_copy) +return 0 ; + +/* +* Clear borrowed pointers immediately so that even if copying fails, +* pevent does not retain dangling pointers to the caller's stack or +* ring buffer. +*/ + pevent->event = &zero_event; + pevent->sample = &pevent->sample_storage; + +/* +* Allocate an extra zero byte so string fields such as mmap/mmap2 +* filenames or comm strings are always NUL-terminated. +*/ + pevent->event_copy =calloc (1 , copy_size +1 ); +if (!pevent->event_copy) { +PyErr_NoMemory (); +return -ENOMEM ; + } + +memcpy (pevent->event_copy, orig_event, copy_size); + pevent->event = pevent->event_copy; + +if (pevent->evsel) { +bool needs_swap = pevent->evsel->needs_swap; + + pevent->evsel->needs_swap = false; + err =evsel__parse_sample (pevent->evsel, pevent->event_copy, + &pevent->sample_storage); + pevent->evsel->needs_swap = needs_swap; + } + +if (err ==0 ) { +if (orig_sample != &pevent->sample_storage) { +if (orig_sample->id) + pevent->sample_storage.id = orig_sample->id; +if (orig_sample->period) + pevent->sample_storage.period = orig_sample->period; +if (orig_sample->machine_pid) + pevent->sample_storage.machine_pid = + orig_sample->machine_pid; +if (orig_sample->vcpu != (u32)-1 ) + pevent->sample_storage.vcpu = orig_sample->vcpu; +if (orig_sample->insn_len && !pevent->sample_storage.insn_len) { +memcpy (pevent->sample_storage.insn, orig_sample->insn, +sizeof (pevent->sample_storage.insn)); + pevent->sample_storage.insn_len = orig_sample->insn_len; + } + } + pevent->sample = &pevent->sample_storage; + }else if (orig_sample != &pevent->sample_storage) { +/* +* Synthesized events (e.g. Intel PT itrace) or events without +* an evsel may not have raw sample buffers for +* evsel__parse_sample(); copy the scalar fields from the +* borrowed sample. +*/ +perf_sample__exit (&pevent->sample_storage); + pevent->sample_storage = *orig_sample; +if (pevent->sample_storage.evsel) + pevent->sample_storage.evsel = +evsel__get (pevent->sample_storage.evsel); + pevent->sample_storage.callchain =NULL ; + pevent->sample_storage.merged_callchain = false; + pevent->sample_storage.raw_data =NULL ; + pevent->sample_storage.raw_size =0 ; + pevent->sample_storage.branch_stack =NULL ; + pevent->sample_storage.user_regs =NULL ; + pevent->sample_storage.intr_regs =NULL ; + pevent->sample_storage.user_stack.size =0 ; + pevent->sample_storage.user_stack.data =NULL ; + pevent->sample_storage.aux_sample.size =0 ; + pevent->sample_storage.aux_sample.data =NULL ; + pevent->sample = &pevent->sample_storage; + }else if (pevent->evsel) { +PyErr_Format (PyExc_OSError,"perf: can't parse sample, err=%d" , err); +return err; + } +return 0 ; +} +static PyObject *pyrf_event__new (const union perf_event *event,struct evsel *evsel,struct perf_session *session,struct machine *machine,struct perf_sample *sample_arg) {struct pyrf_event *pevent;struct perf_sample *sample; -int err;u32 min_size; -bool needs_swap;if (event->header.type >=ARRAY_SIZE (pyrf_event__type) || pyrf_event__type[event->header.type] ==NULL ) {
@@ -1635,31 +1747,35 @@ static PyObject *pyrf_event__new(const union perf_event *event, struct evsel *ev event->header.size, event->header.type); } -size_t copy_size = event->header.size; - -if (copy_size >sizeof (pevent->event)) { -return PyErr_Format (PyExc_TypeError,"Unexpected event size: %zd < %zu" , -sizeof (pevent->event), copy_size); +if (event->header.size >sizeof (union perf_event )) { +return PyErr_Format (PyExc_TypeError,"Unexpected event size: %zu < %u" , +sizeof (union perf_event ), event->header.size); } pevent = PyObject_New(struct pyrf_event, pyrf_event__type[event->header.type]);if (pevent ==NULL )return PyErr_NoMemory (); -/* Copy the event for memory safety and initialize variables. */ -memcpy (&pevent->event, event, copy_size); -if (copy_size <sizeof (pevent->event)) -memset ((char *)&pevent->event + copy_size,0 ,sizeof (pevent->event) - copy_size); - -if (event->header.type ==PERF_RECORD_MMAP2 ) - pevent->event.mmap2.filename[sizeof (pevent->event.mmap2.filename) -1 ] ='\0' ; - -perf_sample__init (&pevent->sample,/*all=*/ true); +perf_sample__init (&pevent->sample_storage,/*all=*/ true); + pevent->sample = sample_arg ?: &pevent->sample_storage; + pevent->evsel = evsel ?evsel__get (evsel) :NULL ; pevent->machine = machine; pevent->callchain =NULL ; pevent->brstack =NULL ; pevent->al_resolved = false;addr_location__init (&pevent->al); + pevent->event = event; + pevent->event_copy =NULL ; + +/* +* When called outside a session callback (e.g. pyrf_evlist__read_on_cpu) +* or without a pre-parsed sample for an evsel, copy the event and parse +* its sample into sample_storage immediately. +*/ +if ((!session || (evsel && !sample_arg)) &&pyrf_event__copy (pevent) <0 ) { +Py_DECREF (pevent); +return NULL ; + }if (!evsel) {if (!pevent->machine && session)
@@ -1667,53 +1783,25 @@ static PyObject *pyrf_event__new(const union perf_event *event, struct evsel *evreturn (PyObject *)pevent; } -/* Parse the sample again so that pointers are within the copied event. */ - needs_swap = evsel->needs_swap; - - evsel->needs_swap = false; - err =evsel__parse_sample (evsel, &pevent->event, &pevent->sample); - evsel->needs_swap = needs_swap; -if (err <0 ) { -/* -* Synthesized events (e.g. Intel PT itrace) may not have raw sample -* buffers for evsel__parse_sample(); use the pre-parsed sample_arg. -*/ -if (sample_arg) { -perf_sample__exit (&pevent->sample); - pevent->sample = *sample_arg; -if (pevent->sample.evsel) - pevent->sample.evsel =evsel__get (pevent->sample.evsel); - pevent->sample.merged_callchain = false; - - pevent->sample.user_regs =NULL ; - pevent->sample.intr_regs =NULL ; - pevent->sample.raw_data =NULL ; - pevent->sample.raw_size =0 ; - }else { -Py_DECREF (pevent); -return PyErr_Format (PyExc_OSError, -"perf: can't parse sample, err=%d" , err); - } - } -if (session && session->evlist && perf_guest && pevent->sample.id) { -struct perf_sample_id *sid =evlist__id2sid (session->evlist, pevent->sample.id); + sample = pevent->sample; +if (session && session->evlist && perf_guest && sample->id) { +struct perf_sample_id *sid =evlist__id2sid (session->evlist, sample->id);if (sid) { - pevent->sample.machine_pid = sid->machine_pid; - pevent->sample.vcpu = sid->vcpu.cpu; + sample->machine_pid = sid->machine_pid; + sample->vcpu = sid->vcpu.cpu; } }if (!machine && session) { - machine = pevent->sample.machine_pid ? -machines__find (&session->machines, pevent->sample.machine_pid) : + machine = sample->machine_pid ? +machines__find (&session->machines, sample->machine_pid) : &session->machines.host;if (!machine) machine = &session->machines.host; } pevent->machine = machine; -if (machine && machine->pid >0 && !pevent->sample.machine_pid) - pevent->sample.machine_pid = machine->pid; - sample = &pevent->sample; +if (machine && machine->pid >0 && !sample->machine_pid) + sample->machine_pid = machine->pid;if (machine && sample->callchain) {struct addr_location al;struct callchain_cursor *cursor;
@@ -4524,6 +4612,41 @@ static int pyrf_session__call_return_process(struct call_return *cr,return 0 ; } +static int pyrf_session__call_event (struct pyrf_session *psession,PyObject *callback, +PyObject *pyevent,const char *extra_arg,bool has_extra_arg) +{ +PyObject *ret; +int err =0 ; + +if (has_extra_arg) + ret =PyObject_CallFunction (callback,"Oz" , pyevent, extra_arg); +else + ret =PyObject_CallFunctionObjArgs (callback, pyevent,NULL ); + +Py_XDECREF (ret); +if (Py_REFCNT (pyevent) >1 ) { +PyObject *exc_type =NULL , *exc_value =NULL , *exc_tb =NULL ; + +/* +* The Python callback retained a reference to the event +* beyond the callback's return (or in an exception frame); +* copy the underlying event and sample before process_events +* reuses or unmaps its buffer. +*/ +if (!ret) +PyErr_Fetch (&exc_type, &exc_value, &exc_tb); + err =pyrf_event__copy ((struct pyrf_event *)pyevent); +if (!ret) { +PyErr_Clear (); +PyErr_Restore (exc_type, exc_value, exc_tb); + } + } +Py_DECREF (pyevent); +if (!ret || err <0 ) +return pyrf_session__callback_raised (psession); +return 0 ; +} + static int pyrf_session_tool__sample(const struct perf_tool *tool, union perf_event *event, struct perf_sample *sample,
@@ -4575,16 +4698,12 @@ static int pyrf_session_tool__sample(const struct perf_tool *tool,PyObject *pyevent =pyrf_event__new (event, sample->evsel, psession->session, machine, sample); -PyObject *ret;if (pyevent ==NULL )return -ENOMEM ; - ret =PyObject_CallFunction (psession->sample,"O" , pyevent); -Py_DECREF (pyevent); -if (!ret) -return pyrf_session__callback_raised (psession); -Py_DECREF (ret); +return pyrf_session__call_event (psession, psession->sample, pyevent, +/*extra_arg=*/ NULL ,/*has_extra_arg=*/ false); }return 0 ; }
@@ -4596,7 +4715,6 @@ static int pyrf_session_tool__context_switch(const struct perf_tool *tool, { struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);PyObject *pyevent; -PyObject *ret;if (pyrf_session__exception_pending ()) {/*
@@ -4607,7 +4725,7 @@ static int pyrf_session_tool__context_switch(const struct perf_tool *tool,return perf_event__process_switch (tool, event, sample, machine); } - pyevent =pyrf_event__new (event, sample->evsel, psession->session, machine,NULL ); + pyevent =pyrf_event__new (event, sample->evsel, psession->session, machine, sample);if (perf_event__process_switch (tool, event, sample, machine) <0 ) {Py_XDECREF (pyevent);return -1 ;
@@ -4616,12 +4734,8 @@ static int pyrf_session_tool__context_switch(const struct perf_tool *tool,if (pyevent ==NULL )return -ENOMEM ; - ret =PyObject_CallFunction (psession->context_switch,"O" , pyevent); -Py_DECREF (pyevent); -if (!ret) -return pyrf_session__callback_raised (psession); -Py_DECREF (ret); -return 0 ; +return pyrf_session__call_event (psession, psession->context_switch, pyevent, +/*extra_arg=*/ NULL ,/*has_extra_arg=*/ false); } static int pyrf_session_tool__stat(const struct perf_tool *tool,
@@ -4632,7 +4746,6 @@ static int pyrf_session_tool__stat(const struct perf_tool *tool,struct evsel *evsel =evlist__id2evsel (session->evlist, event->stat.id);const char *name = evsel ?evsel__name (evsel) :"unknown" ;PyObject *pyevent; -PyObject *ret;if (pyrf_session__exception_pending ())return 0 ;
@@ -4642,12 +4755,8 @@ static int pyrf_session_tool__stat(const struct perf_tool *tool,if (pyevent ==NULL )return -ENOMEM ; - ret =PyObject_CallFunction (psession->stat,"Oz" , pyevent, name); -Py_DECREF (pyevent); -if (!ret) -return pyrf_session__callback_raised (psession); -Py_DECREF (ret); -return 0 ; +return pyrf_session__call_event (psession, psession->stat, pyevent, + name,/*has_extra_arg=*/ true); } static int pyrf_session_tool__stat_round(const struct perf_tool *tool,
@@ -4656,7 +4765,6 @@ static int pyrf_session_tool__stat_round(const struct perf_tool *tool, { struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);PyObject *pyevent; -PyObject *ret;if (pyrf_session__exception_pending ())return 0 ;
@@ -4666,12 +4774,8 @@ static int pyrf_session_tool__stat_round(const struct perf_tool *tool,if (pyevent ==NULL )return -ENOMEM ; - ret =PyObject_CallFunction (psession->stat,"Oz" , pyevent,NULL ); -Py_DECREF (pyevent); -if (!ret) -return pyrf_session__callback_raised (psession); -Py_DECREF (ret); -return 0 ; +return pyrf_session__call_event (psession, psession->stat, pyevent, +/*extra_arg=*/ NULL ,/*has_extra_arg=*/ true); }static PyObject *pyrf_session__find_thread (struct pyrf_session *psession,PyObject *args)
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 03/13] perf python: Lazily copy events and samples from process_events
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 21 lines
When
perf.session.process_events()delivers an event to a Python callback,pyrf_event__new()previously copied the rawperf_eventinto a 4,160-byte unionperf_eventembedded in structpyrf_eventand calledevsel__parse_sample()a second time to re-parse the sample against the copied buffer.In the common case, the Python callback inspects the event and returns without storing a reference to it. Avoid the event memcpy, the duplicate
evsel__parse_sample()call, and the large PyObject allocation by:
1. Storing pointers to the underlying unionperf_eventand structperf_samplein structpyrf_event(using PyGetSetDef descriptors backed byPyMember_GetOne()to read fields through those pointers), shrinking structpyrf_eventso it fits in CPython's small-object allocator pool.
2. Borrowing the event and pre-parsed sample pointers fromprocess_events()for the duration of the Python callback.
3. CheckingPy_REFCNT(pyevent)> 1 after the callback returns and only allocatingevent_copyand re-parsing intosample_storagewhen the Python callback retained a reference to the event object (or immediately inpyrf_evlist__read_on_cpu()where no pre-parsed sample is passed).
Quoted 2 lines
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Ian Rogers
reviewerPATCH 4/13 perf python: Lazily resolve sample callchains
In reply to Ian RogersPreviously, pyrf_event__new() eagerly resolved the sample's address location and callchain (including DWARF/FP unwinding and symbol lookup) for every sample with a callchain, even when the Python callback never accessed event.callchain (such as in ttimechart.py).
Move callchain resolution to pyrf_sample_event__get_callchain() so that callchains are only unwound and resolved when event.callchain is accessed, reusing the sample's cached pevent->al via pyrf_sample_event__resolve_al().
Because process_events() borrows the event and sample during the callback, callchains are never copied in the common case. Only when a callback retains a reference to the event beyond its return (Py_REFCNT(pyevent) > 1), the callchain was not already resolved during the callback, and the callchain is synthesized or LBR-merged (not backed by event_copy) does pyrf_event__copy() allocate a copy of sample->callchain.
--- tools/perf/util/python.c | 160 +++++++++++++++++++-------------------- 1 file changed, 79 insertions(+), 81 deletions(-)
tools/perf/util/python.c +79 -81
@@ -100,7 +100,7 @@ struct pyrf_event {struct addr_location al;/** @al_resolved: True when machine__resolve been called. */ bool al_resolved; -/** @callchain: Resolved callchain, eagerly computed if requested. */ +/** @callchain: Resolved callchain, lazily computed. */ PyObject *callchain;/** @brstack: Resolved branch stack, eagerly computed if requested. */ PyObject *brstack;
@@ -727,28 +727,19 @@ get_tracepoint_field(struct pyrf_event *pevent, PyObject *attr_name)static int pyrf_sample_event__resolve_al (struct pyrf_event *pevent) { -struct evsel *evsel = pevent->sample->evsel ?: pevent->evsel; -struct evlist *evlist = evsel ? evsel->evlist :NULL ; -struct perf_session *session = evlist ?evlist__session (evlist) :NULL ; -struct machine *machine; -if (pevent->al_resolved)return 0 ; -if (!session) -return -1 ; -/* -* Use pevent->machine, which pyrf_event__new() initializes either from -* the machine resolved by perf_session (machines__find_for_cpumode(), -* preserving DEFAULT_GUEST_KERNEL_ID == 0 for default guests) or via -* sample.machine_pid when pyrf_event__new() is called with a NULL -* machine. +* Use pevent->machine, which pyrf_event__new() initializes while the +* session callback is active and pyrf_event__copy() clears after +* resolving pevent->al if a reference is retained beyond the callback. */ - machine = pevent->machine ? pevent->machine : &session->machines.host; +if (!pevent->machine) +return -1 ;addr_location__init (&pevent->al); -if (machine__resolve (machine, &pevent->al, pevent->sample) <0 ) { +if (machine__resolve (pevent->machine, &pevent->al, pevent->sample) <0 ) {addr_location__exit (&pevent->al);return -1 ; }
@@ -924,23 +915,15 @@ static PyObject *pyrf_sample_event__srccode(PyObject *self, PyObject *args)static PyObject *pyrf_sample_event__insn (PyObject *self,PyObject *args__maybe_unused ) {struct pyrf_event *pevent = (void *)self; -struct thread *thread; -struct machine *machine; -if (pyrf_sample_event__resolve_al (pevent) <0 ) - Py_RETURN_NONE; - - thread = pevent->al.thread; - -if (!thread || !thread__maps (thread)) - Py_RETURN_NONE; - - machine =maps__machine (thread__maps (thread)); -if (!machine) - Py_RETURN_NONE; +if (pevent->sample->ip && !pevent->sample->insn_len) { +if (pyrf_sample_event__resolve_al (pevent) <0 || + !pevent->al.thread || !pevent->machine) + Py_RETURN_NONE; -if (pevent->sample->ip && !pevent->sample->insn_len) -perf_sample__fetch_insn (pevent->sample, thread, machine); +perf_sample__fetch_insn (pevent->sample, pevent->al.thread, + pevent->machine); + }if (!pevent->sample->insn_len) Py_RETURN_NONE;
@@ -1097,10 +1080,53 @@ static PyTypeObject pyrf_callchain__type = { .tp_as_sequence = &pyrf_callchain__sequence_methods, }; +static int pyrf_sample_event__resolve_callchain (struct pyrf_event *pevent) +{ +struct callchain_cursor *cursor; +struct pyrf_callchain *pchain; +struct callchain_cursor_node *node; + +if (pevent->callchain || !pevent->sample->callchain || !pevent->machine) +return 0 ; + +if (pyrf_sample_event__resolve_al (pevent) <0 ) +return 0 ; + + cursor =get_tls_callchain_cursor (); +if (thread__resolve_callchain (pevent->al.thread, cursor, pevent->sample, +NULL ,NULL ,PERF_MAX_STACK_DEPTH ) !=0 ) +return 0 ; + +callchain_cursor_commit (cursor); + pchain = PyObject_New(struct pyrf_callchain, &pyrf_callchain__type); +if (!pchain) +return -ENOMEM ; + + pchain->nr_frames = cursor->nr; + pchain->frames =calloc (pchain->nr_frames,sizeof (*pchain->frames)); +if (!pchain->frames) { +Py_DECREF (pchain); +PyErr_NoMemory (); +return -ENOMEM ; + } +for (u64 i =0 ; i < pchain->nr_frames; i++) { + node =callchain_cursor_current (cursor); + pchain->frames[i].ip = node->ip; + pchain->frames[i].map =map__get (node->ms.map); + pchain->frames[i].sym = node->ms.sym; +callchain_cursor_advance (cursor); + } + pevent->callchain = (PyObject *)pchain; +return 0 ; +} +static PyObject *pyrf_sample_event__get_callchain (PyObject *self,void *closure__maybe_unused ) {struct pyrf_event *pevent = (void *)self; +if (pyrf_sample_event__resolve_callchain (pevent) <0 ) +return NULL ; +if (!pevent->callchain) Py_RETURN_NONE;
@@ -1646,13 +1672,31 @@ static int pyrf_event__copy(struct pyrf_event *pevent)size_t copy_size = orig_event->header.size;int err = -EINVAL ; +/* +* If a session callback retained a reference to pevent, resolve its +* address location and callchain now while pevent->machine is still +* live and thread->maps reflects the point in time of the sample. +*/ +if (pevent->machine && pevent->evsel) { +pyrf_sample_event__resolve_al (pevent); +if (pyrf_sample_event__resolve_callchain (pevent) <0 ) { + pevent->machine =NULL ; +if (!pevent->event_copy) { + pevent->event = &zero_event; + pevent->sample = &pevent->sample_storage; + } +return -ENOMEM ; + } + } + pevent->machine =NULL ; +if (pevent->event_copy)return 0 ;/* * Clear borrowed pointers immediately so that even if copying fails, -* pevent does not retain dangling pointers to the caller's stack or -* ring buffer. +* pevent does not retain dangling pointers to the caller's stack, +* ring buffer, or session. */ pevent->event = &zero_event; pevent->sample = &pevent->sample_storage;
@@ -1759,7 +1803,7 @@ static PyObject *pyrf_event__new(const union perf_event *event, struct evsel *evperf_sample__init (&pevent->sample_storage,/*all=*/ true); pevent->sample = sample_arg ?: &pevent->sample_storage; pevent->evsel = evsel ?evsel__get (evsel) :NULL ; - pevent->machine = machine; + pevent->machine =NULL ; pevent->callchain =NULL ; pevent->brstack =NULL ; pevent->al_resolved = false;
@@ -1777,11 +1821,8 @@ static PyObject *pyrf_event__new(const union perf_event *event, struct evsel *evreturn NULL ; } -if (!evsel) { -if (!pevent->machine && session) - pevent->machine = &session->machines.host; +if (!evsel)return (PyObject *)pevent; - } sample = pevent->sample;if (session && session->evlist && perf_guest && sample->id) {
@@ -1802,49 +1843,6 @@ static PyObject *pyrf_event__new(const union perf_event *event, struct evsel *ev pevent->machine = machine;if (machine && machine->pid >0 && !sample->machine_pid) sample->machine_pid = machine->pid; -if (machine && sample->callchain) { -struct addr_location al; -struct callchain_cursor *cursor; -u64 i; -struct pyrf_callchain *pchain; - -addr_location__init (&al); -if (machine__resolve (machine, &al, sample) >=0 ) { - cursor =get_tls_callchain_cursor (); -if (thread__resolve_callchain (al.thread, cursor, sample, -NULL ,NULL ,PERF_MAX_STACK_DEPTH ) ==0 ) { -callchain_cursor_commit (cursor); - - pchain = PyObject_New(struct pyrf_callchain, &pyrf_callchain__type); -if (!pchain) { -addr_location__exit (&al); -Py_DECREF (pevent); -return NULL ; - } - pchain->nr_frames = cursor->nr; - pchain->frames =calloc (pchain->nr_frames, -sizeof (*pchain->frames)); -if (!pchain->frames) { -Py_DECREF (pchain); -addr_location__exit (&al); -Py_DECREF (pevent); -return PyErr_NoMemory (); - } -struct callchain_cursor_node *node; - -for (i =0 ; i < pchain->nr_frames; i++) { - node =callchain_cursor_current (cursor); - pchain->frames[i].ip = node->ip; - pchain->frames[i].map = -map__get (node->ms.map); - pchain->frames[i].sym = node->ms.sym; -callchain_cursor_advance (cursor); - } - pevent->callchain = (PyObject *)pchain; - } -addr_location__exit (&al); - } - }if (sample->branch_stack) {struct branch_stack *bs = sample->branch_stack;struct branch_entry *entries =perf_sample__branch_entries (sample);
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 04/13] perf python: Lazily resolve sample callchains
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 18 lines
Previously,
pyrf_event__new()eagerly resolved the sample's address location and callchain (including DWARF/FP unwinding and symbol lookup) for every sample with a callchain, even when the Python callback never accessed event.callchain (such as inttimechart.py).Move callchain resolution to
pyrf_sample_event__get_callchain()so that callchains are only unwound and resolved when event.callchain is accessed, reusing the sample's cachedpevent->alviapyrf_sample_event__resolve_al().Because
process_events()borrows the event and sample during the callback, callchains are never copied in the common case. Only when a callback retains a reference to the event beyond its return (Py_REFCNT(pyevent)> 1), the callchain was not already resolved during the callback, and the callchain is synthesized or LBR-merged (not backed byevent_copy) doespyrf_event__copy()allocate a copy ofsample->callchain.
Quoted 2 lines
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Ian Rogers
reviewerPATCH 5/13 perf python: Release the GIL while processing session events
In reply to Ian RogersWhen a python session reads from a pipe, perf_session__new() and perf_session__process_events() block waiting for data with the GIL held, freezing any other python threads such as a TUI thread. Release the GIL around both calls and acquire it with PyGILState_Ensure() in the tool and call-return callbacks that invoke python.
--- tools/perf/util/python.c | 258 ++++++++++++++++++++++++++++++++------- 1 file changed, 215 insertions(+), 43 deletions(-)
tools/perf/util/python.c +215 -43
@@ -52,6 +52,7 @@#include "srccode.h" #include "srcline.h" #include "strbuf.h" +#include "strlist.h" #include "symbol.h" #include "stat.h" #include "header.h"
@@ -4104,16 +4105,38 @@ static void pyrf_thread__delete(struct pyrf_thread *pthread)static PyObject *pyrf_thread__comm (PyObject *obj) {struct pyrf_thread *pthread = (void *)obj; -const char *str; +struct thread *thread = pthread->thread; +bool has_comm = false; +char *str =NULL ; +PyObject *ret; -CHECK_INITIALIZED (pthread->thread,"perf.thread" ); +CHECK_INITIALIZED (thread,"perf.thread" ); - str =thread__comm_str (pthread->thread); +/* +* Copy the comm string under comm_lock so a concurrent PERF_RECORD_COMM +* in perf_session__process_events() (which runs with the GIL released) +* cannot free the old comm_str via comm__override() while it is being +* read, and drop comm_lock before calling into the Python allocator. +*/ +down_read (thread__comm_lock (thread)); +if (!list_empty (thread__comm_list (thread))) { +const struct comm *comm; + + comm = list_first_entry(thread__comm_list(thread), struct comm, list); +if (comm &&comm__str (comm)) { + has_comm = true; + str =strdup (comm__str (comm)); + } + } +up_read (thread__comm_lock (thread)); -if (!str) +if (!has_comm) Py_RETURN_NONE; - -return PyUnicode_FromString (str); +if (!str) +return PyErr_NoMemory (); + ret =PyUnicode_FromString (str); +free (str); +return ret; }static PyMethodDef pyrf_thread__methods[] = {
@@ -4211,6 +4234,8 @@ struct pyrf_session {* that event processing stops, see pyrf_session__callback_raised(). */ bool callback_raised; +unsigned int callback_depth; +unsigned long callback_thread;char *vmlinux_name;char *kallsyms_name;char *symfs;
@@ -4579,11 +4604,10 @@ static bool pyrf_session__exception_pending(void)return PyErr_Occurred () !=NULL ; } -static int pyrf_session__call_return_process (struct call_return *cr, -u64 *parent_db_id, -void *data) +static int __pyrf_session__call_return_process (struct call_return *cr, +u64 *parent_db_id, +struct pyrf_session *psession) { -struct pyrf_session *psession = data;PyObject *pyevent, *ret;if (!psession->call_return ||pyrf_session__exception_pending ())
@@ -4610,6 +4634,38 @@ static int pyrf_session__call_return_process(struct call_return *cr,return 0 ; } +static PyGILState_STATE pyrf_session__callback_enter (struct pyrf_session *psession) +{ +PyGILState_STATE gstate =PyGILState_Ensure (); + + psession->callback_thread =PyThread_get_thread_ident (); + psession->callback_depth++; +return gstate; +} + +static void pyrf_session__callback_leave (struct pyrf_session *psession, +PyGILState_STATE gstate) +{ + psession->callback_depth--; +PyGILState_Release (gstate); +} + +/* +* Call returns may be processed during a sample callback, or when thread +* stacks are flushed, so acquire the GIL, see pyrf_session_tool__sample(). +*/ +static int pyrf_session__call_return_process (struct call_return *cr, +u64 *parent_db_id, +void *data) +{ +struct pyrf_session *psession = data; +PyGILState_STATE gstate =pyrf_session__callback_enter (psession); +int ret =__pyrf_session__call_return_process (cr, parent_db_id, psession); + +pyrf_session__callback_leave (psession, gstate); +return ret; +} +static int pyrf_session__call_event (struct pyrf_session *psession,PyObject *callback,PyObject *pyevent,const char *extra_arg,bool has_extra_arg) {
@@ -4645,10 +4701,10 @@ static int pyrf_session__call_event(struct pyrf_session *psession, PyObject *calreturn 0 ; } -static int pyrf_session_tool__sample (const struct perf_tool *tool, -union perf_event *event, -struct perf_sample *sample, -struct machine *machine) +static int __pyrf_session_tool__sample (const struct perf_tool *tool, +union perf_event *event, +struct perf_sample *sample, +struct machine *machine) { struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);u64 sample_db_id = ++psession->sample_last_db_id;
@@ -4706,10 +4762,10 @@ static int pyrf_session_tool__sample(const struct perf_tool *tool,return 0 ; } -static int pyrf_session_tool__context_switch (const struct perf_tool *tool, -union perf_event *event, -struct perf_sample *sample, -struct machine *machine) +static int __pyrf_session_tool__context_switch (const struct perf_tool *tool, +union perf_event *event, +struct perf_sample *sample, +struct machine *machine) { struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);PyObject *pyevent;
@@ -4736,9 +4792,9 @@ static int pyrf_session_tool__context_switch(const struct perf_tool *tool, /*extra_arg=*/NULL, /*has_extra_arg=*/false); } -static int pyrf_session_tool__stat (const struct perf_tool *tool, -struct perf_session *session, -union perf_event *event) +static int __pyrf_session_tool__stat (const struct perf_tool *tool, +struct perf_session *session, +union perf_event *event) { struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);struct evsel *evsel =evlist__id2evsel (session->evlist, event->stat.id);
@@ -4757,9 +4813,9 @@ static int pyrf_session_tool__stat(const struct perf_tool *tool, name, /*has_extra_arg=*/true); } -static int pyrf_session_tool__stat_round (const struct perf_tool *tool, -struct perf_session *session__maybe_unused , -union perf_event *event) +static int __pyrf_session_tool__stat_round (const struct perf_tool *tool, +struct perf_session *session__maybe_unused , +union perf_event *event) { struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);PyObject *pyevent;
@@ -4776,6 +4832,84 @@ static int pyrf_session_tool__stat_round(const struct perf_tool *tool, /*extra_arg=*/NULL, /*has_extra_arg=*/true); } +/* +* The session currently inside perf_session__new() or +* perf_session__process_events() with the GIL released. Because symbol_conf +* and session_done are process-globals and perf_session is not thread-safe, +* only one session may be active at a time. Modified with the GIL held. +*/ +static struct pyrf_session *pyrf_session__active; + +static bool pyrf_session__check_not_busy (struct pyrf_session *psession) +{ +if (pyrf_session__active == psession && + (!psession->callback_depth || + psession->callback_thread !=PyThread_get_thread_ident ())) { +PyErr_SetString (PyExc_RuntimeError, +"perf.session is busy processing events in another thread" ); +return false; + } +return true; +} + +/* +* The GIL is released while events are processed, see +* pyrf_session__find_thread_events(), so that other python threads can run, +* such as a UI thread while a session reads from a pipe that may block. The +* tool callbacks that use python acquire the GIL, PyGILState_Ensure() is +* reentrant so nested callbacks, like call returns during a sample, work. +*/ + +static int pyrf_session_tool__sample (const struct perf_tool *tool, +union perf_event *event, +struct perf_sample *sample, +struct machine *machine) +{ + struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool); +PyGILState_STATE gstate =pyrf_session__callback_enter (psession); +int ret =__pyrf_session_tool__sample (tool, event, sample, machine); + +pyrf_session__callback_leave (psession, gstate); +return ret; +} + +static int pyrf_session_tool__context_switch (const struct perf_tool *tool, +union perf_event *event, +struct perf_sample *sample, +struct machine *machine) +{ + struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool); +PyGILState_STATE gstate =pyrf_session__callback_enter (psession); +int ret =__pyrf_session_tool__context_switch (tool, event, sample, machine); + +pyrf_session__callback_leave (psession, gstate); +return ret; +} + +static int pyrf_session_tool__stat (const struct perf_tool *tool, +struct perf_session *session, +union perf_event *event) +{ + struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool); +PyGILState_STATE gstate =pyrf_session__callback_enter (psession); +int ret =__pyrf_session_tool__stat (tool, session, event); + +pyrf_session__callback_leave (psession, gstate); +return ret; +} + +static int pyrf_session_tool__stat_round (const struct perf_tool *tool, +struct perf_session *session, +union perf_event *event) +{ + struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool); +PyGILState_STATE gstate =pyrf_session__callback_enter (psession); +int ret =__pyrf_session_tool__stat_round (tool, session, event); + +pyrf_session__callback_leave (psession, gstate); +return ret; +} +static PyObject *pyrf_session__find_thread (struct pyrf_session *psession,PyObject *args) {struct machine *machine;
@@ -4784,6 +4918,8 @@ static PyObject *pyrf_session__find_thread(struct pyrf_session *psession, PyObjeint pid, tid =-1 ;CHECK_INITIALIZED (psession->session,"session" ); +if (!pyrf_session__check_not_busy (psession)) +return NULL ;if (!PyArg_ParseTuple (args,"i|i" , &pid, &tid))return NULL ;
@@ -4854,6 +4990,8 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObject psession->call_return_last_db_id =0 ; psession->sample_last_db_id =0 ; psession->callback_raised = false; + psession->callback_depth =0 ; + psession->callback_thread =0 ; psession->vmlinux_name =NULL ; psession->kallsyms_name =NULL ; psession->symfs =NULL ;
@@ -4936,7 +5074,6 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObjectPyErr_NoMemory (); goto err_out; } - symbol_conf.vmlinux_name = psession->vmlinux_name; }if (!kallsyms_str)
@@ -4947,7 +5084,6 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObjectPyErr_NoMemory (); goto err_out; } - symbol_conf.kallsyms_name = psession->kallsyms_name; }if (!symfs_str)
@@ -4958,16 +5094,7 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObjectPyErr_NoMemory (); goto err_out; } - symbol_conf.symfs = psession->symfs; - } - - session =perf_session__new (&pdata->data, &psession->tool); -if (IS_ERR (session)) { -PyErr_Format (PyExc_IOError,"failed to create session: %ld" ,PTR_ERR (session)); - goto err_out; } - psession->session = session; - psession->session->itrace_synth_opts = &psession->itrace_opts;if (!itrace_str) itrace_str =getenv ("PERF_ITRACE" );
@@ -4975,18 +5102,43 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObjectmemset (&psession->itrace_opts,0 ,sizeof (psession->itrace_opts));if (itrace_do_parse_synth_opts (&psession->itrace_opts, itrace_str,0 ) <0 ) {PyErr_SetString (PyExc_ValueError,"Failed to parse itrace options" ); -Py_DECREF (psession); -return NULL ; + goto err_out; } } +if (pyrf_session__active) { +PyErr_SetString (PyExc_RuntimeError, +"Another perf.session is currently active" ); + goto err_out; + } + + symbol_conf.vmlinux_name = psession->vmlinux_name; + symbol_conf.kallsyms_name = psession->kallsyms_name; + symbol_conf.symfs = psession->symfs ?:"" ; symbol_conf.use_callchain = true; symbol_conf.show_kernel_path = true; symbol_conf.inline_name = false; + +/* Reading the header of a pipe may block, let other python threads run. */ + pyrf_session__active = psession; +Py_BEGIN_ALLOW_THREADS + session =perf_session__new (&pdata->data, &psession->tool); +Py_END_ALLOW_THREADS + pyrf_session__active =NULL ; +if (!psession->kallsyms_name && symbol_conf.kallsyms_name) + psession->kallsyms_name = (char *)symbol_conf.kallsyms_name; +if (IS_ERR (session)) { +PyErr_Format (PyExc_IOError,"failed to create session: %ld" ,PTR_ERR (session)); + goto err_out; + } + psession->session = session; + psession->session->itrace_synth_opts = &psession->itrace_opts;if (symbol__init (perf_session__env (session)) <0 ) {PyErr_SetString (PyExc_OSError,"perf: symbol__init failed" ); goto err_out; } +if (psession->symfs && !symbol_conf.symfs[0 ]) +zfree (&psession->symfs);return (PyObject *)psession; err_out:
@@ -4997,14 +5149,16 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObjectstatic void pyrf_session__delete (struct pyrf_session *psession) {perf_session__delete (psession->session); -if (symbol_conf.vmlinux_name == psession->vmlinux_name) - symbol_conf.vmlinux_name =NULL ; +if (!pyrf_session__active) { +if (symbol_conf.vmlinux_name == psession->vmlinux_name) + symbol_conf.vmlinux_name =NULL ; +if (symbol_conf.kallsyms_name == psession->kallsyms_name) + symbol_conf.kallsyms_name =NULL ; +if (symbol_conf.symfs == psession->symfs) + symbol_conf.symfs ="" ; + }free (psession->vmlinux_name); -if (symbol_conf.kallsyms_name == psession->kallsyms_name) - symbol_conf.kallsyms_name =NULL ;free (psession->kallsyms_name); -if (symbol_conf.symfs == psession->symfs) - symbol_conf.symfs ="" ;free (psession->symfs);free (psession->itrace_opts.vm_tm_corr_args);free (psession->itrace_opts.cpu_bitmap);
@@ -5025,9 +5179,23 @@ static PyObject *pyrf_session__find_thread_events(struct pyrf_session *psession)int err;CHECK_INITIALIZED (psession->session,"session" ); +if (pyrf_session__active) { +PyErr_SetString (PyExc_RuntimeError, +"Another perf.session is currently active" ); +return NULL ; + } + + symbol_conf.vmlinux_name = psession->vmlinux_name; + symbol_conf.kallsyms_name = psession->kallsyms_name; + symbol_conf.symfs = psession->symfs ?:"" ; psession->callback_raised = false; +/* Callbacks that use python acquire the GIL, see pyrf_session_tool__sample(). */ + pyrf_session__active = psession; +Py_BEGIN_ALLOW_THREADS err =perf_session__process_events (psession->session); +Py_END_ALLOW_THREADS + pyrf_session__active =NULL ;if (psession->callback_raised) {/* * Clear the early stop requested by this session's callback, in
@@ -5089,6 +5257,8 @@ static PyObject *pyrf_session__getattro(struct pyrf_session *psession, PyObjectPyErr_SetString (PyExc_ValueError,"session not initialized" );return NULL ; } +if (!pyrf_session__check_not_busy (psession)) +return NULL ;if (!strcmp (name_str,"e_machine" ))return PyLong_FromLong (perf_session__e_machine (psession->session,/*e_flags=*/ NULL ));
@@ -5139,6 +5309,8 @@ static int pyrf_session__setattro(struct pyrf_session *psession, PyObject *attr_PyErr_SetString (PyExc_ValueError,"session not initialized" );return -1 ; } +if (!pyrf_session__check_not_busy (psession)) +return -1 ;return PyObject_GenericSetAttr ((PyObject *) psession, attr_name, value); }
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 05/13] perf python: Release the GIL while processing session events
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
When a python session reads from a pipe,
perf_session__new()andperf_session__process_events()block waiting for data with the GIL held, freezing any other python threads such as a TUI thread. Release the GIL around both calls and acquire it withPyGILState_Ensure()in the tool and call-return callbacks that invoke python.Assisted-by Antigravity:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Ian Rogers
reviewerPATCH 6/13 perf list: Add a --tui option to launch ilist
In reply to Ian RogersAdd a --tui option to perf list that launches the textual based ilist script through perf script, similar to perf timechart --tui. Rather than printing the events, the PMUs, events and metrics can be browsed interactively and the selected event or metric is counted on each CPU. Arguments after '--' are passed to the script.
--- tools/perf/Documentation/perf-list.txt | 10 ++++++++ tools/perf/builtin-list.c | 34 ++++++++++++++++++++++++++ 2 files changed, 44 insertions(+)
tools/perf/Documentation/perf-list.txt +10 -0
@@ -10,6 +10,7 @@ SYNOPSIS [verse] 'perf list' [<options>] [hw|sw|cache|tracepoint|pmu|sdt|metric|metricgroup|event_glob] +'perf list' --tui [-- <ilist options>] DESCRIPTION -----------
@@ -51,6 +52,15 @@ Output in JSON format. --output=:: Output file name. By default output is written to stdout. +--tui:: +Rather than printing the events, interactively browse the PMUs, events and +metrics in the terminal using the 'ilist' python script (see +linkperf:perf-script[1]). Selecting an event or metric shows its description +and counts it on each CPU. The script requires the perf python module and the +python 'textual' library. Other options are ignored, arguments after '--' are +passed to the script, for example 'perf list --tui -- -I 0.5' to update the +counts every 0.5 seconds. + [[EVENT_MODIFIERS]] EVENT MODIFIERS ---------------
tools/perf/builtin-list.c +34 -0
@@ -24,8 +24,10 @@#include <subcmd/parse-options.h> #include <linux/zalloc.h> #include <ctype.h> +#include <errno.h> #include <stdarg.h> #include <stdio.h> +#include <stdlib.h> /** * struct print_state - State and configuration passed to the default_print
@@ -538,6 +540,31 @@ static bool default_skip_duplicate_pmus(void *ps)return !print_state->long_desc; } +/* +* Launch the interactive textual based ilist python script via 'perf script' +* that finds the script and sets up the python environment. Arguments are +* passed to the script. +*/ +static int list__tui (int argc,const char **argv) +{ +const char **script_argv; +int script_argc =0 , ret; + + script_argv =calloc (argc +4 ,sizeof (*script_argv)); +if (!script_argv) +return -ENOMEM ; + + script_argv[script_argc++] ="script" ; + script_argv[script_argc++] ="ilist" ; + script_argv[script_argc++] ="--" ; +for (int i =0 ; i < argc; i++) + script_argv[script_argc++] = argv[i]; + + ret =cmd_script (script_argc, script_argv); +free (script_argv); +return ret; +} +int cmd_list (int argc,const char **argv) {int i, ret =0 ;
@@ -562,6 +589,7 @@ int cmd_list(int argc, const char **argv)const char *unit_name =NULL ;const char *output_path =NULL ;bool json = false; +bool use_tui = false;struct option list_options[] = {OPT_BOOLEAN (0 ,"raw-dump" , &default_ps.name_only,"Dump raw events" ),OPT_BOOLEAN ('j' ,"json" , &json,"JSON encode events and metrics" ),
@@ -578,6 +606,8 @@ int cmd_list(int argc, const char **argv) "Limit PMU or metric printing to the given PMU (e.g. cpu, core or atom)."),OPT_STRING (0 ,"unit" , &unit_name,"PMU name" ,"Limit PMU or metric printing to the specified PMU." ), +OPT_BOOLEAN (0 ,"tui" , &use_tui, +"Interactively browse and count events and metrics with ilist" ),OPT_INCR (0 ,"debug" , &verbose,"Enable debugging output" ),OPT_END ()
@@ -588,6 +618,7 @@ int cmd_list(int argc, const char **argv)#else "perf list [<options>] [hw|sw|cache|tracepoint|pmu|sdt|metric|metricgroup|event_glob]" , #endif +"perf list --tui [-- <ilist options>]" ,NULL };
@@ -598,6 +629,9 @@ int cmd_list(int argc, const char **argv) argc =parse_options (argc, argv, list_options, list_usage,PARSE_OPT_STOP_AT_NON_OPTION ); +if (use_tui) +return list__tui (argc, argv); +if (json) ps = &json_ps.common;
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 06/13] perf list: Add a --tui option to launch ilist
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
Add a --tui option to perf list that launches the textual based ilist script through perf script, similar to perf timechart --tui. Rather than printing the events, the PMUs, events and metrics can be browsed interactively and the selected event or metric is counted on each CPU. Arguments after '--' are passed to the script.
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Ian Rogers
reviewerPATCH 7/13 perf test: Add a test for the ilist script
In reply to Ian RogersTest launching ilist through perf script and perf list --tui, and drive the textual app headless: check the tree of PMUs and metrics, search for and select the software task-clock event checking its counters are shown, check a failed search shows an error and select a metric. Failing to open counters, for example for want of permissions, is tolerated as an error dialog is shown.
--- tools/perf/tests/shell/test_ilist_python.sh | 181 ++++++++++++++++++++ 1 file changed, 181 insertions(+) create mode 100755 tools/perf/tests/shell/test_ilist_python.sh
tools/perf/tests/shell/test_ilist_python.sh new file +181 -0
@@ -0,0 +1,181 @@ +#!/bin/bash +# perf script ilist and perf list --tui test +# SPDX-License-Identifier: GPL-2.0 + +set -e + +shelldir=$(dirname "$0") +# shellcheck source=lib/setup_python.sh +. "${shelldir}"/lib/setup_python.sh + +if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then + echo "Skipping test, perf python module not found" + exit 2 +fi + +if ! "$PYTHON" -c 'import textual' > /dev/null 2>&1; then + echo "Skipping test, python textual library not found" + exit 2 +fi + +err=0 +output=$(mktemp /tmp/__perf_ilist_test.output.XXXXX) + +cleanup() { + rm -f "${output}" + trap - EXIT TERM INT +} + +trap_cleanup() { + echo "Unexpected signal in ${FUNCNAME[1]}" + cleanup + exit 1 +} +trap trap_cleanup EXIT TERM INT + +test_plumbing() { + echo "ilist plumbing test" + # The script should be found and run by perf script, --help doesn't + # need a terminal. + if ! perf script ilist --help > "${output}" 2>&1; then + echo "ilist plumbing test [Failed: script failed]" + cat "${output}" + err=1 + return + fi + if ! grep -q -- "--interval" "${output}"; then + echo "ilist plumbing test [Failed: script not launched]" + cat "${output}" + err=1 + return + fi + echo "ilist plumbing test [Success]" +} + +test_list_tui() { + echo "perf list --tui plumbing test" + # Arguments after -- are passed to the script launched by perf list. + if ! perf list --tui -- --help > "${output}" 2>&1; then + echo "perf list --tui plumbing test [Failed: script failed]" + cat "${output}" + err=1 + return + fi + if ! grep -q -- "--interval" "${output}"; then + echo "perf list --tui plumbing test [Failed: script not launched]" + cat "${output}" + err=1 + return + fi + echo "perf list --tui plumbing test [Success]" +} + +test_headless_ui() { + echo "ilist headless UI test" + # Drive the textual app without a terminal. Opening counters may fail + # for want of permissions, in which case an error dialog is shown. + if ! "$PYTHON" > "${output}" 2>&1 <<'EOF' +import asyncio +import ilist +from textual.widgets import Label, Tree + +async def dismiss_error(app, pilot) -> bool: + """Close the error dialog if shown, returning whether it was.""" + if not isinstance(app.screen, ilist.ErrorScreen): + return False + await pilot.click("#error") + await pilot.pause() + if isinstance(app.screen, ilist.ErrorScreen): + raise RuntimeError("Error dialog not dismissed") + return True + +async def search(pilot, text: str) -> None: + await pilot.press("s") + await pilot.pause() + await pilot.press(*text, "enter") + await pilot.pause() + +def find_leaf(node, cls): + """Find the first node whose data is of type cls.""" + if isinstance(node.data, cls): + return node + for child in node.children: + found = find_leaf(child, cls) + if found: + return found + return None + +async def check_counting(app, pilot, name: str) -> None: + """Check the selected event is shown and counting, or that it failed to open.""" + await pilot.pause(0.5) + if await dismiss_error(app, pilot): + print(f"{name}: failed to open") + return + shown = str(app.query_one("#event_name", Label).render()) + if shown != name: + raise RuntimeError(f"Selected {shown} rather than {name}") + if not app.query(ilist.CounterSparkline) or not app.query("#counter_total"): + raise RuntimeError(f"No counters shown for {name}") + print(f"{name}: total {app.query_one('#counter_total', Label).render()}") + +async def run() -> None: + app = ilist.IListApp(0.1) + async with app.run_test(size=(120, 40)) as pilot: + await pilot.pause() + tree = app.query_one("#root", Tree) + top = [str(node.label) for node in tree.root.children] + if top != ["PMUs", "Metrics"]: + raise RuntimeError(f"Unexpected tree: {top}") + pmus = [str(node.label) for node in tree.root.children[0].children] + if "software" not in pmus: + raise RuntimeError(f"No software PMU in: {pmus}") + + # Search for a software event, which exists everywhere. The search + # result is selected which opens the event. + await search(pilot, "task-clock") + found = app.cur_search_result + if not found or not isinstance(found.data, ilist.PmuEvent) or \ + "task-clock" not in str(found.label): + raise RuntimeError(f"Search didn't find task-clock: {found}") + await check_counting(app, pilot, found.data.name()) + if not app.query_one("#active_search", Label).display: + raise RuntimeError("Active search not shown") + await pilot.press("n", "p", "down", "up", "c") + await pilot.pause(0.2) + await dismiss_error(app, pilot) + + # Searching for something that doesn't exist shows an error. + await search(pilot, "no-such-event-xyzzy") + if not await dismiss_error(app, pilot): + raise RuntimeError("No error for a failed search") + + # Select a metric, if there are any. + metric = find_leaf(tree.root, ilist.Metric) + if metric: + app.expand_and_select(metric) + await check_counting(app, pilot, metric.data.name()) + print("headless UI ok") + +asyncio.run(run()) +EOF + then + echo "ilist headless UI test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "headless UI ok" "${output}"; then + echo "ilist headless UI test [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "ilist headless UI test [Success]" +} + +test_plumbing +test_list_tui +test_headless_ui + +cleanup +exit $err
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 07/13] perf test: Add a test for the ilist script
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
Test launching ilist through perf script and perf list --tui, and drive the textual app headless: check the tree of PMUs and metrics, search for and select the software task-clock event checking its counters are shown, check a failed search shows an error and select a metric. Failing to open counters, for example for want of permissions, is tolerated as an error dialog is shown.
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Ian Rogers
reviewerPATCH 8/13 perf treport: Show the profile while it loads
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Reading a large perf.data file caused a long stall before the treport app started. Start the app immediately and build the profile in a background thread, showing the report tree and flame graph as they are built with the progress in the header. The time between updates grows with the load time as updating the views costs more as the profile grows. A lock guards the profile shared between the threads. Quitting while loading cancels the load.
Tree nodes are now created lazily as they are expanded, expanded nodes and the cursor are kept across updates, and events are kept in the order they are first seen as their values aren't comparable. Node names are escaped so they aren't interpreted as markup.
--- tools/perf/python/treport.py | 458 ++++++++++++++++++++++++++--------- 1 file changed, 350 insertions(+), 108 deletions(-)
tools/perf/python/treport.py +350 -108
@@ -2,14 +2,17 @@ # SPDX-License-Identifier: MIT """treport.py - perf report like tool written using textual.""" from abc import ABC, abstractmethod -from typing import Dict, Optional +from typing import Callable, Dict, List, Optional, Set import argparse import os import sys +import threading +import time import perf +from rich.markup import escape from rich.segment import Segment from rich.style import Style -from textual import events +from textual import events, work from textual.app import App, ComposeResult from textual.binding import Binding from textual.color import Color
@@ -18,9 +21,6 @@ from textual.strip import Strip from textual.widgets import Footer, Header, TabbedContent, TabPane, Tree from textual.widgets.tree import TreeNode -# Global session. -session :Optional[perf.session] = None - def make_fixed_length_string(s: str, length: int, pad_char=' '): """Make the string s a fixed length.
@@ -61,7 +61,7 @@ class ProfileNode: children (Dict[str, ProfileNode]): A dictionary of child nodes, keyed by their names. """ - def __init__(self, name: str, parent: "ProfileNode"): + def __init__(self, name: str, parent: Optional["ProfileNode"]): """Initializes a ProfileNode.""" self.name = name self.value: int = 0
@@ -82,20 +82,13 @@ class ProfileNode: return 1 return max(child.depth() for child in self.children.values()) + 1 - def process_event(self, sample) -> None: + def process_event(self, sample, comm: str) -> None: """Processes a single profiling event to update the call stack tree. Args: sample: a single profiling sample. + comm: the command name of the sampled thread. """ - pid = sample.sample_pid - try: - assert session - thread = session.find_thread(pid, sample.sample_tid) - comm = (thread.comm() if thread else None) or f"unknown ({pid})" - except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError): - comm = f"unknown ({pid})" - period = sample.sample_period self.value += period
@@ -120,47 +113,36 @@ class ProfileNode: node = node.find_or_create_node(name) node.value += period - def add_to_tree(self, node: TreeNode, root_value: int) -> None: - """Recursively adds this node and its children to a textual TreeNode. + def sorted_children(self) -> List["ProfileNode"]: + """The children, largest value first. - Args: - node (TreeNode): The textual `TreeNode` object to which this - ProfileNode should be added. - root_value (int): Value at the root of the tree. + The root's children are events whose values aren't comparable, each is + 100% of itself, so they are kept in the order they were first seen + rather than reordering as they load. """ - if root_value == 0: - root_value = self.value + if self.parent is self: + return list(self.children.values()) + return sorted(self.children.values(), key=lambda pnode: pnode.value, reverse=True) - # Calculate the percentage for the node, highlighting the - # percentage with reversed colors. - if root_value != 0: - percent = self.value / root_value * 100 - label = f"{self.name} [r]{percent:.3g}%[/]" - else: - label = self.name - - # Add a standalone leaf. - if not self.children: - node.add_leaf(label) - return + def tree_label(self, root_value: int) -> str: + """Label for the node in a Tree, with a percentage of root_value. - # Recursively add children. - new_node = node.add(label) - for pnode in sorted(self.children.values(), - key=lambda pnode: pnode.value, reverse=True): - pnode.add_to_tree(new_node, root_value) + The percentage is highlighted with reversed colors. + """ + if root_value == 0: + return escape(self.name) + return f"{escape(self.name)} [r]{self.value / root_value * 100:.3g}%[/]" def largest_child(self) -> "ProfileNode": """Finds the child with the highest value (sample count).""" if self.children: - return max(self.children.values(), key=lambda node: node.value) + return self.sorted_children()[0] return self def child_after(self, sought: "ProfileNode") -> "ProfileNode": """Finds the next sibling after the given node, sorted by value.""" found = False - for child in sorted(self.children.values(), key=lambda node: node.value, - reverse=True): + for child in self.sorted_children(): if child == sought: found = True elif found:
@@ -170,8 +152,7 @@ class ProfileNode: def child_before(self, sought: "ProfileNode") -> "ProfileNode": """Finds the previous sibling before the given node, sorted by value.""" last = None - for child in sorted(self.children.values(), key=lambda node: node.value, - reverse=True): + for child in self.sorted_children(): if child == sought: return last if last else sought last = child
@@ -226,8 +207,7 @@ class ProfileNode: # left_over is used to check for a gap after the children due # to samples being in the parent. left_over = parent_width - for child in sorted(self.children.values(), key=lambda node: node.value, - reverse=True): + for child in self.sorted_children(): if parent_selected: if self.value: desired_width = int((parent_width * child.value) / self.value)
@@ -412,16 +392,22 @@ class FlameGraph(ScrollView): } """ - def __init__(self, root: ProfileNode, *pos_args, **kwargs): - """Initialize the FlameGraph widget.""" + def __init__(self, root: ProfileNode, lock: threading.Lock, *pos_args, **kwargs): + """Initialize the FlameGraph widget. + + The lock must be held when reading the profile as it may be being built + in another thread. + """ super().__init__(*pos_args, **kwargs) self.root = root + self.profile_lock = lock self.cursor = root self.selected = root def action_move_down(self) -> None: """Handle key press down.""" - self.cursor = self.cursor.largest_child() + with self.profile_lock: + self.cursor = self.cursor.largest_child() self.refresh() def action_move_up(self) -> None:
@@ -432,12 +418,14 @@ class FlameGraph(ScrollView): def action_move_right(self) -> None: """Handle key press right.""" - self.cursor = self.cursor.parent.child_after(self.cursor) + with self.profile_lock: + self.cursor = self.cursor.parent.child_after(self.cursor) self.refresh() def action_move_left(self) -> None: """Handle key press left.""" - self.cursor = self.cursor.parent.child_before(self.cursor) + with self.profile_lock: + self.cursor = self.cursor.parent.child_before(self.cursor) self.refresh() def action_zoom_in(self) -> None:
@@ -454,24 +442,148 @@ class FlameGraph(ScrollView): """Render a single line (row) of the flame graph.""" _, scroll_y = self.scroll_offset y += scroll_y - return self.root.make_flame_strip(y, self.size.width, self.cursor, - self.selected, self.app.theme_variables) + with self.profile_lock: + return self.root.make_flame_strip(y, self.size.width, self.cursor, + self.selected, self.app.theme_variables) + + def profile_changed(self) -> None: + """Resize and redraw after the profile changed.""" + with self.profile_lock: + self.styles.height = self.root.depth() + self.refresh() def on_mount(self) -> None: """Set the height of the widget when it is displayed.""" - self.styles.height = self.root.depth() + self.profile_changed() def on_click(self, click: events.Click) -> None: """Handles a mouse click and update the cursor position.""" _, scroll_y = self.scroll_offset y = scroll_y + click.y - clicked_node = self.root.find_node(click.x, y, self.size.width, - self.selected) + with self.profile_lock: + clicked_node = self.root.find_node(click.x, y, self.size.width, + self.selected) if clicked_node: self.cursor = clicked_node self.refresh() +class ProfileTree(Tree): + """A tree view of the profile that can be updated while it is built. + + Tree nodes are created lazily when expanded, the data of each tree node is + its ProfileNode. The lock must be held when reading the profile. + """ + + def __init__(self, root: ProfileNode, lock: threading.Lock, *pos_args, **kwargs): + super().__init__("Profile", *pos_args, **kwargs) + self.profile = root + self.profile_lock = lock + # Events whose initial expansion has been done. + self.seen_events: Set[str] = set() + + def pnode_of(self, tnode: TreeNode) -> ProfileNode: + """The ProfileNode shown by tnode, the tree's root shows the profile's root.""" + return tnode.data if tnode.data is not None else self.profile + + def event_value(self, pnode: ProfileNode) -> int: + """Value of the event containing pnode, percentages are relative to it.""" + while pnode.parent is not self.profile and pnode.parent is not pnode: + pnode = pnode.parent + return pnode.value + + @staticmethod + def expanded_descendants(tnode: TreeNode, expanded: Set[ProfileNode]) -> None: + """Add the profile nodes of the expanded descendants of tnode to expanded.""" + for tchild in tnode.children: + if tchild.is_expanded and tchild.data is not None: + expanded.add(tchild.data) + if tchild.children: + ProfileTree.expanded_descendants(tchild, expanded) + + def sync_node(self, tnode: TreeNode, expanded: Optional[Set[ProfileNode]] = None) -> None: + """Update the children of tnode, and expanded descendants, from the profile. + + expanded holds the profile nodes of descendants that were expanded + before an ancestor's children were recreated. The lock must be held. + """ + children = self.pnode_of(tnode).sorted_children() + if expanded is not None or [tchild.data for tchild in tnode.children] != children: + # Recreate the children in the new order keeping the expanded + # ones, and their expanded descendants. + if expanded is None: + expanded = set() + self.expanded_descendants(tnode, expanded) + tnode.remove_children() + ancestors: Set[ProfileNode] = set() + for pnode in expanded: + parent = pnode.parent + while parent is not None and parent not in ancestors: + ancestors.add(parent) + parent = parent.parent + for child in children: + label = child.tree_label(self.event_value(child)) + new = tnode.add(label, child, allow_expand=bool(child.children)) + if child in expanded: + new.expand() + if child in expanded or child in ancestors: + self.sync_node(new, expanded) + return + for tchild in tnode.children: + child = self.pnode_of(tchild) + tchild.set_label(child.tree_label(self.event_value(child))) + tchild.allow_expand = bool(child.children) + if tchild.is_expanded or tchild.children: + self.sync_node(tchild) + + def expand_largest(self, tnode: TreeNode) -> None: + """Expand the chain of largest children below tnode. + + The lock must be held. + """ + while self.pnode_of(tnode).children: + self.sync_node(tnode) + tnode = tnode.children[0] + tnode.expand() + + def find(self, tnode: TreeNode, pnode: ProfileNode) -> Optional[TreeNode]: + """Find the visible tree node for pnode.""" + for tchild in tnode.children: + if tchild.data is pnode: + return tchild + if tchild.is_expanded: + found = self.find(tchild, pnode) + if found: + return found + return None + + def sync(self) -> None: + """Update the tree from the profile keeping the expanded nodes and cursor.""" + cursor = self.cursor_node.data if self.cursor_node else None + with self.profile_lock: + self.root.expand() + self.sync_node(self.root) + for tnode in self.root.children: + name = self.pnode_of(tnode).name + if name in self.seen_events: + continue + self.seen_events.add(name) + self.expand_largest(tnode) + # If there is only one event, expand it also. + if len(self.root.children) == 1: + tnode.expand() + if cursor is not None: + found = self.find(self.root, cursor) + if found: + self.move_cursor(found) + + def on_tree_node_expanded(self, event: Tree.NodeExpanded) -> None: + """Create the children of a node when it is expanded.""" + if event.node is not self.root: + with self.profile_lock: + self.sync_node(event.node) + + class ReportApp(App): """A Textual application to display profiling data."""
@@ -481,84 +593,214 @@ class ReportApp(App): tooltip="Quit the app"), ] - def __init__(self, root: ProfileNode): - """Initialize the application.""" - super().__init__() - self.root = root + def __init__(self, input_file: str, root: Optional[ProfileNode] = None): + """Initialize the application. - def make_report_tree(self) -> Tree: - """Make a Tree widget from the profile data.""" - tree: Tree[None] = Tree("Profile") - # Add events to tree skipping the root. - for pnode in sorted(self.root.children.values(), - key=lambda node: node.value, reverse=True): - pnode.add_to_tree(tree.root, root_value=0) - - # Expand the root tree (shows all events) and the largest of the children - # for each event. - def expand_first_child(tnode: TreeNode) -> None: - """Recursively expand the first child node""" - if not tnode.children: - return - first = tnode.children[0] - first.expand() - expand_first_child(first) - tree.root.expand() - for tnode in tree.root.children: - expand_first_child(tnode) - - # If there is only one event, expand it also. - if len(tree.root.children) == 1: - tree.root.children[0].expand() - - return tree + If root isn't given the profile is loaded from input_file in the + background after the application starts, the views show the profile + as it is built. + """ + super().__init__() + self.input_file = input_file + self.root = root if root else ProfileNode("root", parent=None) + self.loaded = root is not None + # Held when reading or changing the profile, which is built in a + # background thread. + self.profile_lock = threading.Lock() + # The profile being built in a background thread. + self.loading: Optional[ProfileBuilder] = None def compose(self) -> ComposeResult: """Composes the user interface of the application.""" yield Header() with TabbedContent(initial="report"): with TabPane("Report", id="report"): - yield self.make_report_tree() + yield ProfileTree(self.root, self.profile_lock) with TabPane("Flame Graph", id="flame"): - yield FlameGraph(self.root) + yield FlameGraph(self.root, self.profile_lock) yield Footer() + def on_mount(self) -> None: + """Start loading the profile unless it was given.""" + self.sub_title = self.input_file + if self.loaded: + self.update_views() + else: + self.sub_title = f"Loading {self.input_file}" + self.loading = ProfileBuilder(self.root, self.profile_lock) + self.load_profile() + + def update_views(self) -> None: + """Show the latest state of the profile.""" + self.query_one(ProfileTree).sync() + self.query_one(FlameGraph).profile_changed() + + @work(thread=True, exclusive=True) + def load_profile(self) -> None: + """Build the profile in a thread so the UI stays responsive.""" + profile = self.loading + assert profile is not None + + def progress() -> None: + try: + self.call_from_thread(self.update_progress, profile) + except RuntimeError: + # The app is no longer running. + profile.cancelled = True + + profile.progress = progress + try: + profile.read(self.input_file) + except LoadCancelled: + return + except (OSError, ValueError, RuntimeError) as e: + self.call_from_thread(self.exit, None, 1, + f"Error processing {self.input_file}: {e}") + return + finally: + profile.progress = None + self.loading = None + if not profile.cancelled: + self.call_from_thread(self.profile_loaded, profile) + + def update_progress(self, profile: "ProfileBuilder") -> None: + """Show how much of the file has been processed and the profile so far.""" + self.sub_title = (f"Loading {self.input_file}: {profile.nr_samples:,} samples" + f"{profile.duration_str()}") + self.update_views() + + def profile_loaded(self, profile: "ProfileBuilder") -> None: + """Called on the UI thread when loading completes.""" + self.loaded = True + self.sub_title = (f"{self.input_file}: {profile.nr_samples:,} samples" + f"{profile.duration_str()}") + self.update_views() + + def cancel_loading(self) -> None: + """Stop a background load, noticed at the next progress interval.""" + loading = self.loading + if loading is not None: + loading.cancelled = True + + async def action_quit(self) -> None: + """Quit, stopping any background load.""" + self.cancel_loading() + await super().action_quit() + + def on_unmount(self) -> None: + """Stop any background load when the app exits.""" + self.cancel_loading() + + +class LoadCancelled(Exception): + """Raised from the sample callback to stop processing events early.""" + class ProfileBuilder: """Constructs a profile tree from a stream of events.""" - def __init__(self): - self.root = ProfileNode("root", parent=None) + # Number of samples between checks for cancellation and progress. + PROGRESS_INTERVAL = 1000 + # Minimum and maximum time between calls to the progress callback. + PROGRESS_SECONDS = 1.0 + PROGRESS_MAX_SECONDS = 10.0 + # Updating the views costs more as the profile grows, so the time between + # progress calls grows as this fraction of the time spent loading. This + # bounds the fraction of the load time spent updating the views. + PROGRESS_FRACTION = 0.25 + + def __init__(self, root: Optional[ProfileNode] = None, + lock: Optional[threading.Lock] = None) -> None: + """Build into root, holding lock when changing it.""" + self.root = root if root else ProfileNode("root", parent=None) + self.profile_lock = lock if lock else threading.Lock() + self.session: Optional[perf.session] = None + self.nr_samples = 0 + self.first_time = 0 + self.last_time = 0 + # Set, possibly from another thread, to stop processing events. + self.cancelled = False + # Called periodically, see PROGRESS_FRACTION, while processing events. + self.progress: Optional[Callable[[], None]] = None + self.start_progress = time.monotonic() + self.last_progress = self.start_progress + + def duration_str(self) -> str: + """Description of the time covered by the processed samples.""" + if not self.last_time: + return "" + return f", {(self.last_time - self.first_time) / 1e9:.3f}s of trace" + + def comm(self, sample) -> str: + """The command name of the thread of the sample.""" + pid = sample.sample_pid + try: + assert self.session + thread = self.session.find_thread(pid, sample.sample_tid) + return (thread.comm() if thread else None) or f"unknown ({pid})" + except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError, + AssertionError): + return f"unknown ({pid})" def process_event(self, sample) -> None: """Called by session.process_events to update the profile tree.""" + self.nr_samples += 1 + if self.nr_samples % self.PROGRESS_INTERVAL == 0: + if self.cancelled: + raise LoadCancelled() + now = time.monotonic() + interval = min(max(self.PROGRESS_SECONDS, + (now - self.start_progress) * self.PROGRESS_FRACTION), + self.PROGRESS_MAX_SECONDS) + if self.progress and now - self.last_progress >= interval: + # Must not hold the lock as the callback may read the profile. + self.progress() + # Time from when the callback, that may block, returns. + self.last_progress = time.monotonic() + sample_time = sample.sample_time + if sample_time: + if not self.first_time or sample_time < self.first_time: + self.first_time = sample_time + self.last_time = max(self.last_time, sample_time) ev_name = str(sample.evsel)[6:-1] - ev_root = self.root.find_or_create_node(ev_name) - ev_root.process_event(sample) + comm = self.comm(sample) + with self.profile_lock: + ev_root = self.root.find_or_create_node(ev_name) + ev_root.process_event(sample, comm) + + def read(self, input_file: str) -> None: + """Process the events in input_file, raising on errors.""" + if self.cancelled: + raise LoadCancelled() + self.start_progress = self.last_progress = time.monotonic() + try: + self.session = perf.session(perf.data(input_file), sample=self.process_event) + self.session.process_events() + finally: + # Break the reference cycle between the session and the callback. + self.session = None -if __name__ == "__main__": + +def main() -> None: + """Parse arguments and run the app.""" parser = argparse.ArgumentParser( description="TUI report and flame graph using perf python module.") parser.add_argument("-i", "--input", help="input perf.data file") args = parser.parse_args() input_file = args.input or "perf.data" + if input_file == "-": + # The interactive UI reads the keyboard from stdin. + print("Error: reading perf.data from stdin isn't supported.", file=sys.stderr) + sys.exit(1) if not os.path.exists(input_file): print(f"Error: {input_file} not found. (try 'perf record' first)", file=sys.stderr) sys.exit(1) - profile = ProfileBuilder() - try: - session = perf.session(perf.data(input_file), sample=profile.process_event) - except (OSError, ValueError, RuntimeError) as e: - print(f"Error opening session: {e}", file=sys.stderr) - sys.exit(1) + # The app starts immediately and builds the profile in the background. + app = ReportApp(input_file) + app.run() + sys.exit(app.return_code or 0) - # profile.process_event is called for each perf event to build the profile. - try: - session.process_events() - finally: - session = None - # Visualize data. - app = ReportApp(profile.root) - app.run() +if __name__ == "__main__": + main()
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 08/13] perf treport: Show the profile while it loads
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 15 lines
From: Alice Rogers <[email protected]>
Reading a large perf.data file caused a long stall before the treport app started. Start the app immediately and build the profile in a background thread, showing the report tree and flame graph as they are built with the progress in the header. The time between updates grows with the load time as updating the views costs more as the profile grows. A lock guards the profile shared between the threads. Quitting while loading cancels the load.
Tree nodes are now created lazily as they are expanded, expanded nodes and the cursor are kept across updates, and events are kept in the order they are first seen as their values aren't comparable. Node names are escaped so they aren't interpreted as markup.
Quoted 4 lines
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v1 08/13] perf treport: Show the profile while it loads
In reply to Ian RogersQuoted 6 lines · Ian Rogers
From: Alice Rogers <[email protected]>
Reading a large perf.data file caused a long stall before the treport app started. Start the app immediately and build the profile in a background thread, showing the report tree and flame graph as they are built with the progress in the header. The time between updates grows
Nifty! That would be great to have in the main 'perf report' as well, and it is something I thought about doing for a long time, it would be a mixture of 'perf top', that goes on updating the screen as it gets live events, with perf report, i.e. the end result would do just like this patch does for treport.py: perf report would not wait till it processes all events, but would refresh the screen with the state it has periodically, as the samples go on being processed.
Anyway, I tested this and it works as advertised, applied.
- Arnaldo
Quoted 633 lines
with the load time as updating the views costs more as the profile grows. A lock guards the profile shared between the threads. Quitting while loading cancels the load.
Tree nodes are now created lazily as they are expanded, expanded nodes and the cursor are kept across updates, and events are kept in the order they are first seen as their values aren't comparable. Node names are escaped so they aren't interpreted as markup.
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/python/treport.py | 458 ++++++++++++++++++++++++++--------- 1 file changed, 350 insertions(+), 108 deletions(-)tools/perf/python/treport.py +350
-108@@ -2,14 +2,17 @@ # SPDX-License-Identifier: MIT """treport.py - perf report like tool written using textual.""" from abc import ABC, abstractmethod -from typing import Dict, Optional +from typing import Callable, Dict, List, Optional, Set import argparse import os import sys +import threading +import time import perf +from rich.markup import escape from rich.segment import Segment from rich.style import Style -from textual import events +from textual import events, work from textual.app import App, ComposeResult from textual.binding import Binding from textual.color import Color@@ -18,9 +21,6 @@ from textual.strip import Strip from textual.widgets import Footer, Header, TabbedContent, TabPane, Tree from textual.widgets.tree import TreeNode -# Global session. -session :Optional[perf.session] = None - def make_fixed_length_string(s: str, length: int, pad_char=' '): """Make the string s a fixed length.@@ -61,7 +61,7 @@ class ProfileNode: children (Dict[str, ProfileNode]): A dictionary of child nodes, keyed by their names. """ - def __init__(self, name: str, parent: "ProfileNode"): + def __init__(self, name: str, parent: Optional["ProfileNode"]): """Initializes a ProfileNode.""" self.name = name self.value: int = 0@@ -82,20 +82,13 @@ class ProfileNode: return 1 return max(child.depth() for child in self.children.values()) + 1 - def process_event(self, sample) -> None: + def process_event(self, sample, comm: str) -> None: """Processes a single profiling event to update the call stack tree. Args: sample: a single profiling sample. + comm: the command name of the sampled thread. """ - pid = sample.sample_pid - try: - assert session - thread = session.find_thread(pid, sample.sample_tid) - comm = (thread.comm() if thread else None) or f"unknown ({pid})" - except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError): - comm = f"unknown ({pid})" - period = sample.sample_period self.value += period@@ -120,47 +113,36 @@ class ProfileNode: node = node.find_or_create_node(name) node.value += period - def add_to_tree(self, node: TreeNode, root_value: int) -> None: - """Recursively adds this node and its children to a textual TreeNode. + def sorted_children(self) -> List["ProfileNode"]: + """The children, largest value first. - Args: - node (TreeNode): The textual `TreeNode` object to which this - ProfileNode should be added. - root_value (int): Value at the root of the tree. + The root's children are events whose values aren't comparable, each is + 100% of itself, so they are kept in the order they were first seen + rather than reordering as they load. """ - if root_value == 0: - root_value = self.value + if self.parent is self: + return list(self.children.values()) + return sorted(self.children.values(), key=lambda pnode: pnode.value, reverse=True) - # Calculate the percentage for the node, highlighting the - # percentage with reversed colors. - if root_value != 0: - percent = self.value / root_value * 100 - label = f"{self.name} [r]{percent:.3g}%[/]" - else: - label = self.name - - # Add a standalone leaf. - if not self.children: - node.add_leaf(label) - return + def tree_label(self, root_value: int) -> str: + """Label for the node in a Tree, with a percentage of root_value. - # Recursively add children. - new_node = node.add(label) - for pnode in sorted(self.children.values(), - key=lambda pnode: pnode.value, reverse=True): - pnode.add_to_tree(new_node, root_value) + The percentage is highlighted with reversed colors. + """ + if root_value == 0: + return escape(self.name) + return f"{escape(self.name)} [r]{self.value / root_value * 100:.3g}%[/]" def largest_child(self) -> "ProfileNode": """Finds the child with the highest value (sample count).""" if self.children: - return max(self.children.values(), key=lambda node: node.value) + return self.sorted_children()[0] return self def child_after(self, sought: "ProfileNode") -> "ProfileNode": """Finds the next sibling after the given node, sorted by value.""" found = False - for child in sorted(self.children.values(), key=lambda node: node.value, - reverse=True): + for child in self.sorted_children(): if child == sought: found = True elif found:@@ -170,8 +152,7 @@ class ProfileNode: def child_before(self, sought: "ProfileNode") -> "ProfileNode": """Finds the previous sibling before the given node, sorted by value.""" last = None - for child in sorted(self.children.values(), key=lambda node: node.value, - reverse=True): + for child in self.sorted_children(): if child == sought: return last if last else sought last = child@@ -226,8 +207,7 @@ class ProfileNode: # left_over is used to check for a gap after the children due # to samples being in the parent. left_over = parent_width - for child in sorted(self.children.values(), key=lambda node: node.value, - reverse=True): + for child in self.sorted_children(): if parent_selected: if self.value: desired_width = int((parent_width * child.value) / self.value)@@ -412,16 +392,22 @@ class FlameGraph(ScrollView): } """ - def __init__(self, root: ProfileNode, *pos_args, **kwargs): - """Initialize the FlameGraph widget.""" + def __init__(self, root: ProfileNode, lock: threading.Lock, *pos_args, **kwargs): + """Initialize the FlameGraph widget. + + The lock must be held when reading the profile as it may be being built + in another thread. + """ super().__init__(*pos_args, **kwargs) self.root = root + self.profile_lock = lock self.cursor = root self.selected = root def action_move_down(self) -> None: """Handle key press down.""" - self.cursor = self.cursor.largest_child() + with self.profile_lock: + self.cursor = self.cursor.largest_child() self.refresh() def action_move_up(self) -> None:@@ -432,12 +418,14 @@ class FlameGraph(ScrollView): def action_move_right(self) -> None: """Handle key press right.""" - self.cursor = self.cursor.parent.child_after(self.cursor) + with self.profile_lock: + self.cursor = self.cursor.parent.child_after(self.cursor) self.refresh() def action_move_left(self) -> None: """Handle key press left.""" - self.cursor = self.cursor.parent.child_before(self.cursor) + with self.profile_lock: + self.cursor = self.cursor.parent.child_before(self.cursor) self.refresh() def action_zoom_in(self) -> None:@@ -454,24 +442,148 @@ class FlameGraph(ScrollView): """Render a single line (row) of the flame graph.""" _, scroll_y = self.scroll_offset y += scroll_y - return self.root.make_flame_strip(y, self.size.width, self.cursor, - self.selected, self.app.theme_variables) + with self.profile_lock: + return self.root.make_flame_strip(y, self.size.width, self.cursor, + self.selected, self.app.theme_variables) + + def profile_changed(self) -> None: + """Resize and redraw after the profile changed.""" + with self.profile_lock: + self.styles.height = self.root.depth() + self.refresh() def on_mount(self) -> None: """Set the height of the widget when it is displayed.""" - self.styles.height = self.root.depth() + self.profile_changed() def on_click(self, click: events.Click) -> None: """Handles a mouse click and update the cursor position.""" _, scroll_y = self.scroll_offset y = scroll_y + click.y - clicked_node = self.root.find_node(click.x, y, self.size.width, - self.selected) + with self.profile_lock: + clicked_node = self.root.find_node(click.x, y, self.size.width, + self.selected) if clicked_node: self.cursor = clicked_node self.refresh() +class ProfileTree(Tree): + """A tree view of the profile that can be updated while it is built. + + Tree nodes are created lazily when expanded, the data of each tree node is + its ProfileNode. The lock must be held when reading the profile. + """ + + def __init__(self, root: ProfileNode, lock: threading.Lock, *pos_args, **kwargs): + super().__init__("Profile", *pos_args, **kwargs) + self.profile = root + self.profile_lock = lock + # Events whose initial expansion has been done. + self.seen_events: Set[str] = set() + + def pnode_of(self, tnode: TreeNode) -> ProfileNode: + """The ProfileNode shown by tnode, the tree's root shows the profile's root.""" + return tnode.data if tnode.data is not None else self.profile + + def event_value(self, pnode: ProfileNode) -> int: + """Value of the event containing pnode, percentages are relative to it.""" + while pnode.parent is not self.profile and pnode.parent is not pnode: + pnode = pnode.parent + return pnode.value + + @staticmethod + def expanded_descendants(tnode: TreeNode, expanded: Set[ProfileNode]) -> None: + """Add the profile nodes of the expanded descendants of tnode to expanded.""" + for tchild in tnode.children: + if tchild.is_expanded and tchild.data is not None: + expanded.add(tchild.data) + if tchild.children: + ProfileTree.expanded_descendants(tchild, expanded) + + def sync_node(self, tnode: TreeNode, expanded: Optional[Set[ProfileNode]] = None) -> None: + """Update the children of tnode, and expanded descendants, from the profile. + + expanded holds the profile nodes of descendants that were expanded + before an ancestor's children were recreated. The lock must be held. + """ + children = self.pnode_of(tnode).sorted_children() + if expanded is not None or [tchild.data for tchild in tnode.children] != children: + # Recreate the children in the new order keeping the expanded + # ones, and their expanded descendants. + if expanded is None: + expanded = set() + self.expanded_descendants(tnode, expanded) + tnode.remove_children() + ancestors: Set[ProfileNode] = set() + for pnode in expanded: + parent = pnode.parent + while parent is not None and parent not in ancestors: + ancestors.add(parent) + parent = parent.parent + for child in children: + label = child.tree_label(self.event_value(child)) + new = tnode.add(label, child, allow_expand=bool(child.children)) + if child in expanded: + new.expand() + if child in expanded or child in ancestors: + self.sync_node(new, expanded) + return + for tchild in tnode.children: + child = self.pnode_of(tchild) + tchild.set_label(child.tree_label(self.event_value(child))) + tchild.allow_expand = bool(child.children) + if tchild.is_expanded or tchild.children: + self.sync_node(tchild) + + def expand_largest(self, tnode: TreeNode) -> None: + """Expand the chain of largest children below tnode. + + The lock must be held. + """ + while self.pnode_of(tnode).children: + self.sync_node(tnode) + tnode = tnode.children[0] + tnode.expand() + + def find(self, tnode: TreeNode, pnode: ProfileNode) -> Optional[TreeNode]: + """Find the visible tree node for pnode.""" + for tchild in tnode.children: + if tchild.data is pnode: + return tchild + if tchild.is_expanded: + found = self.find(tchild, pnode) + if found: + return found + return None + + def sync(self) -> None: + """Update the tree from the profile keeping the expanded nodes and cursor.""" + cursor = self.cursor_node.data if self.cursor_node else None + with self.profile_lock: + self.root.expand() + self.sync_node(self.root) + for tnode in self.root.children: + name = self.pnode_of(tnode).name + if name in self.seen_events: + continue + self.seen_events.add(name) + self.expand_largest(tnode) + # If there is only one event, expand it also. + if len(self.root.children) == 1: + tnode.expand() + if cursor is not None: + found = self.find(self.root, cursor) + if found: + self.move_cursor(found) + + def on_tree_node_expanded(self, event: Tree.NodeExpanded) -> None: + """Create the children of a node when it is expanded.""" + if event.node is not self.root: + with self.profile_lock: + self.sync_node(event.node) + + class ReportApp(App): """A Textual application to display profiling data."""@@ -481,84 +593,214 @@ class ReportApp(App): tooltip="Quit the app"), ] - def __init__(self, root: ProfileNode): - """Initialize the application.""" - super().__init__() - self.root = root + def __init__(self, input_file: str, root: Optional[ProfileNode] = None): + """Initialize the application. - def make_report_tree(self) -> Tree: - """Make a Tree widget from the profile data.""" - tree: Tree[None] = Tree("Profile") - # Add events to tree skipping the root. - for pnode in sorted(self.root.children.values(), - key=lambda node: node.value, reverse=True): - pnode.add_to_tree(tree.root, root_value=0) - - # Expand the root tree (shows all events) and the largest of the children - # for each event. - def expand_first_child(tnode: TreeNode) -> None: - """Recursively expand the first child node""" - if not tnode.children: - return - first = tnode.children[0] - first.expand() - expand_first_child(first) - tree.root.expand() - for tnode in tree.root.children: - expand_first_child(tnode) - - # If there is only one event, expand it also. - if len(tree.root.children) == 1: - tree.root.children[0].expand() - - return tree + If root isn't given the profile is loaded from input_file in the + background after the application starts, the views show the profile + as it is built. + """ + super().__init__() + self.input_file = input_file + self.root = root if root else ProfileNode("root", parent=None) + self.loaded = root is not None + # Held when reading or changing the profile, which is built in a + # background thread. + self.profile_lock = threading.Lock() + # The profile being built in a background thread. + self.loading: Optional[ProfileBuilder] = None def compose(self) -> ComposeResult: """Composes the user interface of the application.""" yield Header() with TabbedContent(initial="report"): with TabPane("Report", id="report"): - yield self.make_report_tree() + yield ProfileTree(self.root, self.profile_lock) with TabPane("Flame Graph", id="flame"): - yield FlameGraph(self.root) + yield FlameGraph(self.root, self.profile_lock) yield Footer() + def on_mount(self) -> None: + """Start loading the profile unless it was given.""" + self.sub_title = self.input_file + if self.loaded: + self.update_views() + else: + self.sub_title = f"Loading {self.input_file}" + self.loading = ProfileBuilder(self.root, self.profile_lock) + self.load_profile() + + def update_views(self) -> None: + """Show the latest state of the profile.""" + self.query_one(ProfileTree).sync() + self.query_one(FlameGraph).profile_changed() + + @work(thread=True, exclusive=True) + def load_profile(self) -> None: + """Build the profile in a thread so the UI stays responsive.""" + profile = self.loading + assert profile is not None + + def progress() -> None: + try: + self.call_from_thread(self.update_progress, profile) + except RuntimeError: + # The app is no longer running. + profile.cancelled = True + + profile.progress = progress + try: + profile.read(self.input_file) + except LoadCancelled: + return + except (OSError, ValueError, RuntimeError) as e: + self.call_from_thread(self.exit, None, 1, + f"Error processing {self.input_file}: {e}") + return + finally: + profile.progress = None + self.loading = None + if not profile.cancelled: + self.call_from_thread(self.profile_loaded, profile) + + def update_progress(self, profile: "ProfileBuilder") -> None: + """Show how much of the file has been processed and the profile so far.""" + self.sub_title = (f"Loading {self.input_file}: {profile.nr_samples:,} samples" + f"{profile.duration_str()}") + self.update_views() + + def profile_loaded(self, profile: "ProfileBuilder") -> None: + """Called on the UI thread when loading completes.""" + self.loaded = True + self.sub_title = (f"{self.input_file}: {profile.nr_samples:,} samples" + f"{profile.duration_str()}") + self.update_views() + + def cancel_loading(self) -> None: + """Stop a background load, noticed at the next progress interval.""" + loading = self.loading + if loading is not None: + loading.cancelled = True + + async def action_quit(self) -> None: + """Quit, stopping any background load.""" + self.cancel_loading() + await super().action_quit() + + def on_unmount(self) -> None: + """Stop any background load when the app exits.""" + self.cancel_loading() + + +class LoadCancelled(Exception): + """Raised from the sample callback to stop processing events early.""" + class ProfileBuilder: """Constructs a profile tree from a stream of events.""" - def __init__(self): - self.root = ProfileNode("root", parent=None) + # Number of samples between checks for cancellation and progress. + PROGRESS_INTERVAL = 1000 + # Minimum and maximum time between calls to the progress callback. + PROGRESS_SECONDS = 1.0 + PROGRESS_MAX_SECONDS = 10.0 + # Updating the views costs more as the profile grows, so the time between + # progress calls grows as this fraction of the time spent loading. This + # bounds the fraction of the load time spent updating the views. + PROGRESS_FRACTION = 0.25 + + def __init__(self, root: Optional[ProfileNode] = None, + lock: Optional[threading.Lock] = None) -> None: + """Build into root, holding lock when changing it.""" + self.root = root if root else ProfileNode("root", parent=None) + self.profile_lock = lock if lock else threading.Lock() + self.session: Optional[perf.session] = None + self.nr_samples = 0 + self.first_time = 0 + self.last_time = 0 + # Set, possibly from another thread, to stop processing events. + self.cancelled = False + # Called periodically, see PROGRESS_FRACTION, while processing events. + self.progress: Optional[Callable[[], None]] = None + self.start_progress = time.monotonic() + self.last_progress = self.start_progress + + def duration_str(self) -> str: + """Description of the time covered by the processed samples.""" + if not self.last_time: + return "" + return f", {(self.last_time - self.first_time) / 1e9:.3f}s of trace" + + def comm(self, sample) -> str: + """The command name of the thread of the sample.""" + pid = sample.sample_pid + try: + assert self.session + thread = self.session.find_thread(pid, sample.sample_tid) + return (thread.comm() if thread else None) or f"unknown ({pid})" + except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError, + AssertionError): + return f"unknown ({pid})" def process_event(self, sample) -> None: """Called by session.process_events to update the profile tree.""" + self.nr_samples += 1 + if self.nr_samples % self.PROGRESS_INTERVAL == 0: + if self.cancelled: + raise LoadCancelled() + now = time.monotonic() + interval = min(max(self.PROGRESS_SECONDS, + (now - self.start_progress) * self.PROGRESS_FRACTION), + self.PROGRESS_MAX_SECONDS) + if self.progress and now - self.last_progress >= interval: + # Must not hold the lock as the callback may read the profile. + self.progress() + # Time from when the callback, that may block, returns. + self.last_progress = time.monotonic() + sample_time = sample.sample_time + if sample_time: + if not self.first_time or sample_time < self.first_time: + self.first_time = sample_time + self.last_time = max(self.last_time, sample_time) ev_name = str(sample.evsel)[6:-1] - ev_root = self.root.find_or_create_node(ev_name) - ev_root.process_event(sample) + comm = self.comm(sample) + with self.profile_lock: + ev_root = self.root.find_or_create_node(ev_name) + ev_root.process_event(sample, comm) + + def read(self, input_file: str) -> None: + """Process the events in input_file, raising on errors.""" + if self.cancelled: + raise LoadCancelled() + self.start_progress = self.last_progress = time.monotonic() + try: + self.session = perf.session(perf.data(input_file), sample=self.process_event) + self.session.process_events() + finally: + # Break the reference cycle between the session and the callback. + self.session = None -if __name__ == "__main__": + +def main() -> None: + """Parse arguments and run the app.""" parser = argparse.ArgumentParser( description="TUI report and flame graph using perf python module.") parser.add_argument("-i", "--input", help="input perf.data file") args = parser.parse_args() input_file = args.input or "perf.data" + if input_file == "-": + # The interactive UI reads the keyboard from stdin. + print("Error: reading perf.data from stdin isn't supported.", file=sys.stderr) + sys.exit(1) if not os.path.exists(input_file): print(f"Error: {input_file} not found. (try 'perf record' first)", file=sys.stderr) sys.exit(1) - profile = ProfileBuilder() - try: - session = perf.session(perf.data(input_file), sample=profile.process_event) - except (OSError, ValueError, RuntimeError) as e: - print(f"Error opening session: {e}", file=sys.stderr) - sys.exit(1) + # The app starts immediately and builds the profile in the background. + app = ReportApp(input_file) + app.run() + sys.exit(app.return_code or 0) - # profile.process_event is called for each perf event to build the profile. - try: - session.process_events() - finally: - session = None - # Visualize data. - app = ReportApp(profile.root) - app.run() +if __name__ == "__main__": + main()Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
Ian Rogers
reviewerPATCH 9/13 perf test: Add a test for the treport script
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Test launching treport through perf script, the error for a file that isn't a perf.data file, drive the textual app headless checking the profile matches one built without the UI and exercising the key bindings, and that quitting while loading stops the background load.
--- tools/perf/tests/shell/test_treport_python.sh | 247 ++++++++++++++++++ 1 file changed, 247 insertions(+) create mode 100755 tools/perf/tests/shell/test_treport_python.sh
tools/perf/tests/shell/test_treport_python.sh new file +247 -0
@@ -0,0 +1,247 @@ +#!/bin/bash +# perf script treport test +# SPDX-License-Identifier: GPL-2.0 + +set -e + +shelldir=$(dirname "$0") +# shellcheck source=lib/setup_python.sh +. "${shelldir}"/lib/setup_python.sh + +if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then + echo "Skipping test, perf python module not found" + exit 2 +fi + +if ! "$PYTHON" -c 'import textual' > /dev/null 2>&1; then + echo "Skipping test, python textual library not found" + exit 2 +fi + +err=0 +perfdata= +output= +badfile= + +cleanup() { + [ -n "${perfdata}" ] && rm -f "${perfdata}"* + rm -f "${output}" "${badfile}" + trap - EXIT TERM INT +} + +trap_cleanup() { + echo "Unexpected signal in ${FUNCNAME[1]}" + cleanup + exit 1 +} +trap trap_cleanup EXIT TERM INT + +perfdata=$(mktemp /tmp/__perf_treport_test.perf.data.XXXXX) +output=$(mktemp /tmp/__perf_treport_test.output.XXXXX) +badfile=$(mktemp /tmp/__perf_treport_test.bad.XXXXX) + +test_plumbing() { + echo "treport plumbing test" + # A missing input file should be reported by the script launched + # through perf script, this doesn't need a terminal. + if perf script treport -i "${perfdata}.missing" > "${output}" 2>&1; then + echo "treport plumbing test [Failed: missing file not an error]" + err=1 + return + fi + if ! grep -q "not found" "${output}"; then + echo "treport plumbing test [Failed: script not launched]" + cat "${output}" + err=1 + return + fi + echo "treport plumbing test [Success]" +} + +test_bad_file() { + echo "treport bad file test" + echo "not a perf.data file" > "${badfile}" + # The app starts before the file is read, so the error is reported by + # the app exiting with a failure. + if ! "$PYTHON" - "${badfile}" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import treport + +async def run() -> None: + app = treport.ReportApp(sys.argv[1]) + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + if app.return_code is not None: + break + await pilot.pause(0.1) + if app.return_code != 1: + raise RuntimeError(f"Unexpected return code {app.return_code}") + print("bad file ok") + +asyncio.run(run()) +EOF + then + echo "treport bad file test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "bad file ok" "${output}"; then + echo "treport bad file test [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "treport bad file test [Success]" +} + +record() { + # A software event with callchains that doesn't need privileges. + perf record -e task-clock -g -o "${perfdata}" -- perf test -w noploop \ + > /dev/null 2>&1 +} + +test_headless_ui() { + echo "treport headless UI test" + # Drive the textual app without a terminal, check the profile shown + # matches one built without the UI and exercise the key bindings. + if ! "$PYTHON" - "${perfdata}" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import treport + +def labels(tree, tnode, out): + for child in tnode.children: + out.append(str(child.label)) + if child.is_expanded: + labels(tree, child, out) + return out + +async def run() -> None: + expected = treport.ProfileBuilder() + expected.read(sys.argv[1]) + if not expected.root.children: + raise RuntimeError("No samples in the profile") + + # The app starts before the profile is built in a background thread. + app = treport.ReportApp(sys.argv[1]) + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if app.loaded: + break + await pilot.pause(0.1) + if not app.loaded: + raise RuntimeError("Timed out loading data") + await pilot.pause() + if app.sub_title.startswith("Loading") or "samples" not in app.sub_title: + raise RuntimeError(f"Unexpected sub-title: {app.sub_title}") + totals = {name: node.value for name, node in app.root.children.items()} + want = {name: node.value for name, node in expected.root.children.items()} + if totals != want: + raise RuntimeError(f"Profile {totals} differs from {want}") + tree = app.query_one(treport.ProfileTree) + shown = labels(tree, tree.root, []) + if not any("noploop" in label for label in shown): + raise RuntimeError(f"noploop not shown in: {shown}") + await pilot.press("down", "down", "enter", "up", "enter") + tabs = app.query_one(treport.TabbedContent) + tabs.active = "flame" + await pilot.pause() + await pilot.press("down", "down", "right", "enter", "escape", "up", "left") + await pilot.pause() + print("headless UI ok") + +asyncio.run(run()) +EOF + then + echo "treport headless UI test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "headless UI ok" "${output}"; then + echo "treport headless UI test [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "treport headless UI test [Success]" +} + +test_cancel() { + echo "treport cancel test" + # Quitting while loading should stop the background thread without a + # "processing failed" error from the session. + if ! "$PYTHON" - "${perfdata}" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import time +import treport + +# Check for cancellation often and slow processing each sample so that the +# load can't finish before it is cancelled. +treport.ProfileBuilder.PROGRESS_INTERVAL = 1 +process_event = treport.ProfileBuilder.process_event + +def slow_process_event(self, sample) -> None: + time.sleep(0.001) + process_event(self, sample) + +treport.ProfileBuilder.process_event = slow_process_event + +async def run() -> None: + app = treport.ReportApp(sys.argv[1]) + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + loading = app.loading + if loading is not None and loading.nr_samples > 0: + break + await pilot.pause(0.1) + if loading is None or app.loaded: + raise RuntimeError("Background load not in progress") + await pilot.press("ctrl+q") + for _ in range(100): + if app.loading is None: + break + time.sleep(0.1) + if app.loading is not None: + raise RuntimeError("Background load didn't stop") + if not loading.cancelled or app.loaded: + raise RuntimeError("Background load wasn't cancelled") + print("cancel ok") + +asyncio.run(run()) +EOF + then + echo "treport cancel test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "cancel ok" "${output}" || grep -q "processing failed" "${output}"; then + echo "treport cancel test [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "treport cancel test [Success]" +} + +test_plumbing +test_bad_file + +if ! record; then + echo "Skipping remaining tests, failed to record samples" + if [ $err -eq 0 ]; then + err=2 + fi + cleanup + exit $err +fi + +test_headless_ui +test_cancel + +cleanup +exit $err
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 09/13] perf test: Add a test for the treport script
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 11 lines
From: Alice Rogers <[email protected]>
Test launching treport through perf script, the error for a file that isn't a perf.data file, drive the textual app headless checking the profile matches one built without the UI and exercising the key bindings, and that quitting while loading stops the background load.
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v1 09/13] perf test: Add a test for the treport script
In reply to Ian RogersQuoted 6 lines · Ian Rogers
From: Alice Rogers <[email protected]>
Test launching treport through perf script, the error for a file that isn't a perf.data file, drive the textual app headless checking the profile matches one built without the UI and exercising the key bindings, and that quitting while loading stops the background load.
+ tabs.active = "flame"
+ await pilot.pause()
+ await pilot.press("down", "down", "right", "enter", "escape", "up", "left")
+ await pilot.pause()
+ print("headless UI ok")This is super great, testing a TUI app in 'perf test', well done.
- Arnaldo
Quoted 264 lines
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>--- tools/perf/tests/shell/test_treport_python.sh | 247 ++++++++++++++++++ 1 file changed, 247 insertions(+) create mode 100755 tools/perf/tests/shell/test_treport_python.shtools/perf/tests/shell/test_treport_python.sh new file +247
-0@@ -0,0 +1,247 @@ +#!/bin/bash +# perf script treport test +# SPDX-License-Identifier: GPL-2.0 + +set -e + +shelldir=$(dirname "$0") +# shellcheck source=lib/setup_python.sh +. "${shelldir}"/lib/setup_python.sh + +if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then + echo "Skipping test, perf python module not found" + exit 2 +fi + +if ! "$PYTHON" -c 'import textual' > /dev/null 2>&1; then + echo "Skipping test, python textual library not found" + exit 2 +fi + +err=0 +perfdata= +output= +badfile= + +cleanup() { + [ -n "${perfdata}" ] && rm -f "${perfdata}"* + rm -f "${output}" "${badfile}" + trap - EXIT TERM INT +} + +trap_cleanup() { + echo "Unexpected signal in ${FUNCNAME[1]}" + cleanup + exit 1 +} +trap trap_cleanup EXIT TERM INT + +perfdata=$(mktemp /tmp/__perf_treport_test.perf.data.XXXXX) +output=$(mktemp /tmp/__perf_treport_test.output.XXXXX) +badfile=$(mktemp /tmp/__perf_treport_test.bad.XXXXX) + +test_plumbing() { + echo "treport plumbing test" + # A missing input file should be reported by the script launched + # through perf script, this doesn't need a terminal. + if perf script treport -i "${perfdata}.missing" > "${output}" 2>&1; then + echo "treport plumbing test [Failed: missing file not an error]" + err=1 + return + fi + if ! grep -q "not found" "${output}"; then + echo "treport plumbing test [Failed: script not launched]" + cat "${output}" + err=1 + return + fi + echo "treport plumbing test [Success]" +} + +test_bad_file() { + echo "treport bad file test" + echo "not a perf.data file" > "${badfile}" + # The app starts before the file is read, so the error is reported by + # the app exiting with a failure. + if ! "$PYTHON" - "${badfile}" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import treport + +async def run() -> None: + app = treport.ReportApp(sys.argv[1]) + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + if app.return_code is not None: + break + await pilot.pause(0.1) + if app.return_code != 1: + raise RuntimeError(f"Unexpected return code {app.return_code}") + print("bad file ok") + +asyncio.run(run()) +EOF + then + echo "treport bad file test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "bad file ok" "${output}"; then + echo "treport bad file test [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "treport bad file test [Success]" +} + +record() { + # A software event with callchains that doesn't need privileges. + perf record -e task-clock -g -o "${perfdata}" -- perf test -w noploop \ + > /dev/null 2>&1 +} + +test_headless_ui() { + echo "treport headless UI test" + # Drive the textual app without a terminal, check the profile shown + # matches one built without the UI and exercise the key bindings. + if ! "$PYTHON" - "${perfdata}" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import treport + +def labels(tree, tnode, out): + for child in tnode.children: + out.append(str(child.label)) + if child.is_expanded: + labels(tree, child, out) + return out + +async def run() -> None: + expected = treport.ProfileBuilder() + expected.read(sys.argv[1]) + if not expected.root.children: + raise RuntimeError("No samples in the profile") + + # The app starts before the profile is built in a background thread. + app = treport.ReportApp(sys.argv[1]) + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if app.loaded: + break + await pilot.pause(0.1) + if not app.loaded: + raise RuntimeError("Timed out loading data") + await pilot.pause() + if app.sub_title.startswith("Loading") or "samples" not in app.sub_title: + raise RuntimeError(f"Unexpected sub-title: {app.sub_title}") + totals = {name: node.value for name, node in app.root.children.items()} + want = {name: node.value for name, node in expected.root.children.items()} + if totals != want: + raise RuntimeError(f"Profile {totals} differs from {want}") + tree = app.query_one(treport.ProfileTree) + shown = labels(tree, tree.root, []) + if not any("noploop" in label for label in shown): + raise RuntimeError(f"noploop not shown in: {shown}") + await pilot.press("down", "down", "enter", "up", "enter") + tabs = app.query_one(treport.TabbedContent) + tabs.active = "flame" + await pilot.pause() + await pilot.press("down", "down", "right", "enter", "escape", "up", "left") + await pilot.pause() + print("headless UI ok") + +asyncio.run(run()) +EOF + then + echo "treport headless UI test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "headless UI ok" "${output}"; then + echo "treport headless UI test [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "treport headless UI test [Success]" +} + +test_cancel() { + echo "treport cancel test" + # Quitting while loading should stop the background thread without a + # "processing failed" error from the session. + if ! "$PYTHON" - "${perfdata}" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import time +import treport + +# Check for cancellation often and slow processing each sample so that the +# load can't finish before it is cancelled. +treport.ProfileBuilder.PROGRESS_INTERVAL = 1 +process_event = treport.ProfileBuilder.process_event + +def slow_process_event(self, sample) -> None: + time.sleep(0.001) + process_event(self, sample) + +treport.ProfileBuilder.process_event = slow_process_event + +async def run() -> None: + app = treport.ReportApp(sys.argv[1]) + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + loading = app.loading + if loading is not None and loading.nr_samples > 0: + break + await pilot.pause(0.1) + if loading is None or app.loaded: + raise RuntimeError("Background load not in progress") + await pilot.press("ctrl+q") + for _ in range(100): + if app.loading is None: + break + time.sleep(0.1) + if app.loading is not None: + raise RuntimeError("Background load didn't stop") + if not loading.cancelled or app.loaded: + raise RuntimeError("Background load wasn't cancelled") + print("cancel ok") + +asyncio.run(run()) +EOF + then + echo "treport cancel test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "cancel ok" "${output}" || grep -q "processing failed" "${output}"; then + echo "treport cancel test [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "treport cancel test [Success]" +} + +test_plumbing +test_bad_file + +if ! record; then + echo "Skipping remaining tests, failed to record samples" + if [ $err -eq 0 ]; then + err=2 + fi + cleanup + exit $err +fi + +test_headless_ui +test_cancel + +cleanup +exit $errSignature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
Ian Rogers
reviewerPATCH 10/13 perf timechart: Add an interactive --tui mode
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Add ttimechart.py, a textual based interactive timechart, and a --tui option to perf timechart that launches it through perf script. Rather than writing an SVG file, per-CPU (busy, frequency and idle state), per-task (running, runnable and blocked) and I/O timelines are shown in the terminal along with a summary table. The timelines can be zoomed, panned and the state of the selected row at the cursor is described, including the task that woke it. The -i, -P, -T and -p options are passed to the script.
The data is loaded in a background thread and the timelines are shown and updated as it loads. A --dump option prints a text summary without the UI.
--- tools/perf/Documentation/perf-timechart.txt | 8 + tools/perf/builtin-timechart.c | 44 + tools/perf/python/ttimechart.py | 1807 +++++++++++++++++++ 3 files changed, 1859 insertions(+) create mode 100755 tools/perf/python/ttimechart.py
tools/perf/Documentation/perf-timechart.txt +8 -0
@@ -53,6 +53,14 @@ TIMECHART OPTIONS -f:: --force:: Don't complain, do it. +--tui:: + Rather than writing an SVG file, interactively display the timechart + in the terminal using the 'ttimechart' python script (see + linkperf:perf-script[1]). The script requires the perf python module + and the python 'textual' library. The CPU, task, I/O and summary views + can be zoomed ('+'/'-'), panned (shift+arrows) and the state of the + selected row at the cursor is described. The -i, -P, -T and -p options + are passed to the script, other output options are ignored. --symfs=<directory[,layout]>:: Look for files with symbols relative to this directory. The optional layout can be 'hierarchy' (default, matches full path) or 'flat'
tools/perf/builtin-timechart.c +44 -0
74 lines not on this page · raw message
tools/perf/python/ttimechart.py new file +1807 -0
1,807 lines not on this page · raw message
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 10/13] perf timechart: Add an interactive --tui mode
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 11 lines
From: Alice Rogers <[email protected]>
Add
ttimechart.py, a textual based interactive timechart, and a --tui option to perf timechart that launches it through perf script. Rather than writing an SVG file, per-CPU (busy, frequency and idle state), per-task (running, runnable and blocked) and I/O timelines are shown in the terminal along with a summary table. The timelines can be zoomed, panned and the state of the selected row at the cursor is described, including the task that woke it. The -i, -P, -T and -p options are passed to the script.
Quoted 8 lines
The data is loaded in a background thread and the timelines are shown and updated as it loads. A --dump option prints a text summary without the UI.
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
Ian Rogers
reviewerPATCH 11/13 perf test: Add a test for perf timechart --tui
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Test launching ttimechart through perf timechart --tui, the error when there are no scheduler events, the --dump output with the -P, -T and -p options, and drive the textual app headless exercising the views and key bindings.
--- .../tests/shell/test_ttimechart_python.sh | 237 ++++++++++++++++++ 1 file changed, 237 insertions(+) create mode 100755 tools/perf/tests/shell/test_ttimechart_python.sh
tools/perf/tests/shell/test_ttimechart_python.sh new file +237 -0
@@ -0,0 +1,237 @@ +#!/bin/bash +# perf timechart --tui (ttimechart.py) test +# SPDX-License-Identifier: GPL-2.0 + +set -e + +shelldir=$(dirname "$0") +# shellcheck source=lib/setup_python.sh +. "${shelldir}"/lib/setup_python.sh + +if ! perf check feature -q libtraceevent; then + echo "Skipping test, libtraceevent is disabled" + exit 2 +fi + +if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then + echo "Skipping test, perf python module not found" + exit 2 +fi + +if ! "$PYTHON" -c 'import textual' > /dev/null 2>&1; then + echo "Skipping test, python textual library not found" + exit 2 +fi + +err=0 +perfdata= +output= +clockdata= + +cleanup() { + [ -n "${perfdata}" ] && rm -f "${perfdata}"* + [ -n "${clockdata}" ] && rm -f "${clockdata}"* + rm -f "${output}" + trap - EXIT TERM INT +} + +trap_cleanup() { + echo "Unexpected signal in ${FUNCNAME[1]}" + cleanup + exit 1 +} +trap trap_cleanup EXIT TERM INT + +perfdata=$(mktemp /tmp/__perf_ttimechart_test.perf.data.XXXXX) +output=$(mktemp /tmp/__perf_ttimechart_test.output.XXXXX) +clockdata=$(mktemp /tmp/__perf_ttimechart_test.clock.data.XXXXX) + +workload() { + for _ in 1 2 3 4 5; do + sleep 0.01 + done +} + +record() { + # perf timechart record is system wide and so may need privileges, + # fall back to recording just the workload's scheduler events. + if perf timechart record -o "${perfdata}" -- \ + bash -c "$(declare -f workload); workload" > /dev/null 2>&1; then + return 0 + fi + if perf record -R -c 1 -e sched:sched_switch -e sched:sched_wakeup \ + -o "${perfdata}" -- bash -c "$(declare -f workload); workload" \ + > /dev/null 2>&1; then + return 0 + fi + return 1 +} + +test_plumbing() { + echo "perf timechart --tui plumbing test" + # A missing input file should be reported by the script launched + # through perf script, this doesn't need a terminal. + if perf timechart --tui -i "${perfdata}.missing" > "${output}" 2>&1; then + echo "perf timechart --tui plumbing test [Failed: missing file not an error]" + err=1 + return + fi + if ! grep -q "not found" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: script not launched]" + cat "${output}" + err=1 + return + fi + echo "perf timechart --tui plumbing test [Success]" +} + +test_no_events() { + echo "ttimechart no events test" + if ! perf record -e task-clock -o "${clockdata}" -- true > /dev/null 2>&1; then + echo "ttimechart no events test [Skipped: perf record failed]" + return + fi + if perf script ttimechart -i "${clockdata}" --dump > "${output}" 2>&1; then + echo "ttimechart no events test [Failed: expected an error]" + err=1 + return + fi + if ! grep -q "no scheduler, power or I/O events" "${output}"; then + echo "ttimechart no events test [Failed: unexpected error message]" + cat "${output}" + err=1 + return + fi + # The same should be reported when reading the data from a pipe. + if perf record -e task-clock -o - -- true 2> /dev/null | \ + perf script ttimechart -i - --dump > "${output}" 2>&1; then + echo "ttimechart no events test [Failed: expected an error from a pipe]" + err=1 + return + fi + if ! grep -q "no scheduler, power or I/O events" "${output}"; then + echo "ttimechart no events test [Failed: unexpected error message from a pipe]" + cat "${output}" + err=1 + return + fi + echo "ttimechart no events test [Success]" +} + +test_dump() { + echo "ttimechart dump test" + if ! perf script ttimechart -i "${perfdata}" --dump > "${output}" 2>&1; then + echo "ttimechart dump test [Failed: script failed]" + cat "${output}" + err=1 + return + fi + if ! grep -q "^Duration: .*sched events: [1-9]" "${output}" || + ! grep -q "^CPU [0-9]*: busy" "${output}" || + ! grep -q "^Task .*Running .*Waiting .*Blocked" "${output}" || + ! grep -q "sleep" "${output}"; then + echo "ttimechart dump test [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + + # Tasks only shouldn't show CPUs, power only shouldn't show tasks. + perf script ttimechart -i "${perfdata}" --dump -T > "${output}" 2>&1 || true + if grep -q "^CPU [0-9]*: " "${output}" || + ! grep -q "^Task .*Running .*Waiting .*Blocked" "${output}"; then + echo "ttimechart dump test [Failed: -T output]" + cat "${output}" + err=1 + return + fi + perf script ttimechart -i "${perfdata}" --dump -P > "${output}" 2>&1 || true + if grep -q "^Task .*Running .*Waiting .*Blocked" "${output}" || + ! grep -q "^CPU [0-9]*: " "${output}"; then + echo "ttimechart dump test [Failed: -P output]" + cat "${output}" + err=1 + return + fi + + # Filtering on the workload shouldn't show other tasks. + perf script ttimechart -i "${perfdata}" --dump -p sleep > "${output}" 2>&1 || true + if ! grep -q "^sleep " "${output}" || grep -q "^bash " "${output}"; then + echo "ttimechart dump test [Failed: -p output]" + cat "${output}" + err=1 + return + fi + echo "ttimechart dump test [Success]" +} + +test_headless_ui() { + echo "ttimechart headless UI test" + # Drive the textual app without a terminal, exercising the views and + # key bindings. + if ! "$PYTHON" - "${perfdata}" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import ttimechart as tc + +async def run() -> None: + # The app starts before the data is loaded in a background thread. + app = tc.TimechartApp(sys.argv[1], False, False, []) + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if app.loaded: + break + await pilot.pause(0.1) + if not app.loaded: + raise RuntimeError("Timed out loading data") + await pilot.pause() + tabs = app.query_one(tc.TabbedContent) + for tab in ["cpus", "tasks", "summary"]: + tabs.active = tab + await pilot.pause() + await pilot.press("down", "plus", "plus", "right", "n", "p", "minus", + "shift+right", "shift+left", "s", "w", "0") + await pilot.pause() + tabs.active = "summary" + await pilot.pause() + await pilot.press("enter") + await pilot.pause() + details = str(app.query_one("#details", tc.Static).render()) + if "Cursor" not in details: + raise RuntimeError(f"Unexpected details: {details}") + print("headless UI ok") + +asyncio.run(run()) +EOF + then + echo "ttimechart headless UI test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "headless UI ok" "${output}"; then + echo "ttimechart headless UI test [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "ttimechart headless UI test [Success]" +} + +test_plumbing +test_no_events + +if ! record; then + echo "Skipping remaining tests, failed to record scheduler events (permissions?)" + if [ $err -eq 0 ]; then + err=2 + fi + cleanup + exit $err +fi + +test_dump +test_headless_ui + +cleanup +exit $err
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 11/13] perf test: Add a test for perf timechart --tui
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 11 lines
From: Alice Rogers <[email protected]>
Test launching ttimechart through perf timechart --tui, the error when there are no scheduler events, the --dump output with the -P, -T and -p options, and drive the textual app headless exercising the views and key bindings.
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Ian Rogers
reviewerPATCH 12/13 perf timechart: Add a --live mode to the TUI
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Add a --live option to perf timechart and ttimechart.py that runs 'perf record' writing CLOCK_MONOTONIC scheduler and power events, or I/O syscall events when -I/--io-only is given, to a pipe and displays them in the TUI as they happen. Only the most recent --window seconds of history (default 10) are kept in memory, pruning older segments and tasks that have no remaining history in the window.
Pressing space pauses and resumes recording using perf record's --control pipe. Because perf record only reads its ring buffers when a watermark is reached, periodic 'ping' commands are sent on the control pipe to flush events promptly. The refresh interval adapts to the measured screen refresh cost and backs off if unprocessed data builds up in the pipe, with two flushes per refresh to account for ordered-event buffering across rounds.
An optional workload may be given after '--' to record just that command rather than the whole system, and '-o' optionally saves a copy of the recorded perf.data stream.
--- tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 80 +- tools/perf/python/ttimechart.py | 786 ++++++++++++++++++-- 3 files changed, 811 insertions(+), 75 deletions(-)
tools/perf/Documentation/perf-timechart.txt +18 -2
@@ -47,6 +47,9 @@ TIMECHART OPTIONS -T:: --tasks-only:: Don't output processor state transitions +-I:: +--io-only:: + In '--live' or '--tui' mode, only record or show I/O activity -p:: --process:: Select the processes to display, by name or PID
@@ -59,8 +62,21 @@ TIMECHART OPTIONS linkperf:perf-script[1]). The script requires the perf python module and the python 'textual' library. The CPU, task, I/O and summary views can be zoomed ('+'/'-'), panned (shift+arrows) and the state of the - selected row at the cursor is described. The -i, -P, -T and -p options - are passed to the script, other output options are ignored. + selected row at the cursor is described. The -i, -P, -T, -I and -p + options are passed to the script, other output options are ignored. +--live:: + Record scheduler and power events, or I/O events with '-I', with + linkperf:perf-record[1] and display them as they happen in the + terminal timechart, implying --tui. Only the most recent '--window' + seconds of events are kept and shown. Pressing space pauses and + resumes recording; zooming or panning stops the view following the + latest events until '0' is pressed. If a command is given after '--' + then only that workload is recorded and recording stops when it exits, + otherwise the whole system is recorded until the UI is quit. If '-o' + is given the recorded perf data is also saved to that file. +--window=<seconds>:: + In '--live' mode, the number of seconds of recent events kept in + memory and shown in the timeline (default: 10). --symfs=<directory[,layout]>:: Look for files with symbols relative to this directory. The optional layout can be 'hierarchy' (default, matches full path) or 'flat'
tools/perf/builtin-timechart.c +70 -10
159 lines not on this page · raw message
tools/perf/python/ttimechart.py +723 -63
1,028 lines not on this page · raw message
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 12/13] perf timechart: Add a --live mode to the TUI
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] perf: missing close for live mode pipe fd in ttimechart --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] perf: missing close for live mode pipe fd in ttimechart
--
commit dd0096e018bbc0f7d639c04dde6e4a581907e0c1
Author: Alice Rogers <[email protected]>
perf timechart: Add a --live mode to the TUI
Add a --live option to perf timechart and ttimechart.py that runs 'perf record' writing CLOCK_MONOTONIC scheduler and power events, or I/O syscall events when -I/--io-only is given, to a pipe and displays them in the TUI as they happen.
Quoted 4 lines
tools/perf/python/ttimechart.py +0
-0
[ ... ]
Quoted 1 line
tools/perf/python/ttimechart.py +0
-0@@ -1727,52 +2110,329 @@ class TimechartApp(App):
[ ... ]
Quoted 1 line
+ def start(self) -> None:
[ ... ]
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2226
Quoted 11 lines
tools/perf/python/ttimechart.py +11
-0+ self.data_fd = self.proc.stdout.fileno() + self.grow_pipe(self.data_fd) + if output: + # Save the data as it is passed on to be processed. + tee_r, tee_w = os.pipe() + self.tee_thread = threading.Thread(target=self.tee, + args=(self.data_fd, tee_w, output), + daemon=True) + self.tee_thread.start() + self.data_fd = tee_r + self.grow_pipe(self.data_fd)
[Severity: Medium]
Does this code leak the file descriptor for the read end of the pipe?
When running in --live mode with an output file (-o), tee_r from os.pipe() is assigned to self.data_fd. While self.data_fd is later passed to the C extension (perf.data(fd=...)) where it is duplicated:
tools/perf/util/python.c:pyrf_data__init() {
...
fd = dup(fd);
if (fd < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
...
}the original file descriptor is never closed in the Python code. This causes the original tee_r (and potentially proc.stdout.fileno()) to remain open for the lifetime of the TUI process.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12
Ian Rogers
reviewerPATCH 13/13 perf test: Test perf timechart --live
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Extend test_ttimechart_python.sh to cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file, a synthetic live UI test exercising the rolling window, pruning, pause/resume and zoom following, and a live workload recording test when scheduler tracepoints are accessible.
--- .../tests/shell/test_ttimechart_python.sh | 300 ++++++++++++++++++ 1 file changed, 300 insertions(+)
tools/perf/tests/shell/test_ttimechart_python.sh +300 -0
@@ -82,6 +82,34 @@ test_plumbing() { err=1 return fi + if perf timechart --live -i "${perfdata}" > "${output}" 2>&1 || + ! grep -q -- "--live can't be used with -i" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --live with -i not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --live --window 0 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "--window must be a positive" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window 0 not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --tui --window 5 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "require --live" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window without --live]" + cat "${output}" + err=1 + return + fi + if perf timechart --live -I -P < /dev/null > "${output}" 2>&1 || + ! grep -q -- "-I cannot be used with -P or -T" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: -I with -P not rejected]" + cat "${output}" + err=1 + return + fi echo "perf timechart --tui plumbing test [Success]" }
@@ -115,6 +143,43 @@ test_no_events() { err=1 return fi + # Exercise LiveRecorder's perf record subprocess, control pipe and + # -o output file using task-clock when tracepoints may be unavailable. + rm -f "${clockdata}" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${clockdata}" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import os +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.events = lambda: ["task-clock"] + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + await pilot.press("space") + await pilot.pause(0.1) + await pilot.press("space") + for _ in range(300): + if app.return_code is not None: + break + await pilot.pause(0.05) + finally: + rec.stop() + if app.return_code != 1 or not app.data.nr_samples or not os.path.getsize(sys.argv[2]): + raise RuntimeError(f"Unexpected result: rc={app.return_code} samples={app.data.nr_samples}") + +asyncio.run(run()) +EOF + then + echo "ttimechart no events test [Failed: LiveRecorder task-clock test]" + cat "${output}" + err=1 + return + fi echo "ttimechart no events test [Success]" }
@@ -218,8 +283,242 @@ EOF echo "ttimechart headless UI test [Success]" } +test_live_synthetic() { + echo "ttimechart live UI test (synthetic)" + # Exercise live mode's rolling window, pruning, pause/resume and + # following with a fake recorder, which doesn't need tracepoint + # permissions. + if ! "$PYTHON" - > "${output}" 2>&1 <<'EOF' +import asyncio +import threading +import time +from time import monotonic_ns +import ttimechart as tc + +SEC = 1_000_000_000 + +class Ev: + def __init__(self, name): + self.name = name + def __str__(self): + return f"evsel({self.name})" + +class Sample: + def __init__(self, name, t, cpu, tid, **kw): + self.evsel = Ev(name) + self.sample_time = t + self.sample_cpu = cpu + self.sample_tid = tid + self.sample_pid = tid + self.sample_id = hash(name) + for k, v in kw.items(): + setattr(self, k, v) + +class FakeRecorder: + def __init__(self): + self.origin = monotonic_ns() + self.data_fd = -1 + self.flush_interval = 0.05 + self.paused = False + self.pause_calls = [] + self.stopped = threading.Event() + self.finish = False + + def backlog(self): + return 0.0 + + def set_paused(self, paused): + self.paused = paused + self.pause_calls.append(paused) + + def stop(self): + self.stopped.set() + + def error_text(self): + return "" + +async def run() -> None: + rec = FakeRecorder() + + def fake_read(data, _fd): + t0 = rec.origin + # Task 300 only runs at the start so pruning should drop it once the + # 1s window advances past t0 + 3000. + data.process_event(Sample("sched:sched_wakeup", t0 + 1000, 0, 1, + pid=300, comm="early", common_pid=1, common_flags=0)) + data.process_event(Sample("sched:sched_switch", t0 + 2000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=300, next_comm="early")) + data.process_event(Sample("sched:sched_switch", t0 + 3000, 0, 300, + prev_pid=300, prev_comm="early", prev_state=1, + next_pid=0, next_comm="swapper")) + cur = 0 + while not rec.stopped.is_set() and not rec.finish: + now = monotonic_ns() + if rec.paused: + time.sleep(0.01) + continue + nxt = 100 if cur == 0 else 0 + comm = "worker" if nxt == 100 else "swapper" + prev_comm = "worker" if cur == 100 else "swapper" + if nxt: + data.process_event(Sample("sched:sched_wakeup", now, 0, cur or 1, + pid=nxt, comm=comm, common_pid=cur or 1, + common_flags=0)) + data.process_event(Sample("sched:sched_switch", now, 0, cur, + prev_pid=cur, prev_comm=prev_comm, prev_state=1, + next_pid=nxt, next_comm=comm)) + cur = nxt + time.sleep(0.005) + + tc.read_events = fake_read + app = tc.TimechartApp("live", False, False, [], live=rec, window=1.0) + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(300): + with app.data.lock: + if app.data.pruned_to > rec.origin + 3000: + break + await pilot.pause(0.05) + with app.data.lock: + if app.data.pruned_to <= rec.origin + 3000 or 300 in app.data.tasks: + raise RuntimeError("Task outside the window was not pruned") + starts = [t.segs.starts[0] for t in app.data.tasks.values() if len(t.segs)] + if not starts or min(starts) < app.data.pruned_to: + raise RuntimeError(f"Unpruned segments: {starts} < {app.data.pruned_to}") + if app.window.last - app.window.first != SEC: + raise RuntimeError(f"Unexpected window span: {app.window}") + + await pilot.press("space") + await pilot.pause(0.2) + if rec.pause_calls != [True] or "Paused" not in app.sub_title: + raise RuntimeError(f"Did not pause: {rec.pause_calls} {app.sub_title}") + frozen = app.window.last + await pilot.pause(0.4) + if app.window.last != frozen: + raise RuntimeError("Window moved while paused") + await pilot.press("space") + await pilot.pause(0.3) + if rec.pause_calls != [True, False] or not app.following: + raise RuntimeError("Did not resume") + + await pilot.press("plus") + await pilot.pause(0.2) + if app.following: + raise RuntimeError("Zooming did not stop following") + await pilot.press("0") + await pilot.pause(0.2) + if not app.following: + raise RuntimeError("Resetting zoom did not resume following") + + rec.finish = True + for _ in range(100): + if not app.recording: + break + await pilot.pause(0.05) + if app.recording or "Finished" not in app.sub_title: + raise RuntimeError(f"Did not finish: {app.sub_title}") + + # Also test live I/O-only mode (-I). + io_events = tc.LiveRecorder("perf", False, False, [], None, io_only=True).events() + if not io_events or not all(e.startswith("syscalls:sys_") for e in io_events): + raise RuntimeError(f"Unexpected I/O events: {io_events}") + io_rec = FakeRecorder() + + def fake_io_read(data, _fd): + now = monotonic_ns() + # Self I/O should be ignored in live mode. + own = tc.os.getpid() + data.process_event(Sample("syscalls:sys_enter_read", now, 0, own, fd=3)) + data.process_event(Sample("syscalls:sys_exit_read", now + 1000, 0, own, ret=64)) + # Workload I/O should be recorded. + data.process_event(Sample("syscalls:sys_enter_read", now + 2000, 0, 4242, fd=4)) + data.process_event(Sample("syscalls:sys_exit_read", now + 5000, 0, 4242, ret=4096)) + data.process_event(Sample("syscalls:sys_enter_write", now + 6000, 0, 4242, fd=5)) + data.process_event(Sample("syscalls:sys_exit_write", now + 9000, 0, 4242, ret=2048)) + + tc.read_events = fake_io_read + io_app = tc.TimechartApp("live", False, False, [], live=io_rec, window=1.0, io_only=True) + async with io_app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + if not io_app.recording: + break + await pilot.pause(0.05) + await pilot.pause() + tabs = io_app.query_one(tc.TabbedContent) + if tabs.active != "io" or not io_app.io_tasks or io_app.data.io_events != 4: + raise RuntimeError(f"Unexpected I/O state: active={tabs.active} " + f"tasks={len(io_app.io_tasks)} events={io_app.data.io_events}") + tabs.active = "summary" + await pilot.pause() + await pilot.press("enter") + await pilot.pause() + if tabs.active != "io": + raise RuntimeError(f"Summary enter did not activate I/O tab: {tabs.active}") + print("live synthetic ok") + +asyncio.run(run()) +EOF + then + echo "ttimechart live UI test (synthetic) [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "live synthetic ok" "${output}"; then + echo "ttimechart live UI test (synthetic) [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live UI test (synthetic) [Success]" +} + +test_live_record() { + echo "ttimechart live record test" + rm -f "${perfdata}.live" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${perfdata}.live" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if not app.recording: + break + await pilot.pause(0.1) + if app.recording or not app.data.has_events(): + raise RuntimeError("Live recording of workload did not complete") + print("live record ok") + finally: + rec.stop() + +asyncio.run(run()) +EOF + then + echo "ttimechart live record test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! perf script ttimechart -i "${perfdata}.live" --dump > "${output}" 2>&1 || + ! grep -q "^Duration: .*sched events: [1-9]" "${output}"; then + echo "ttimechart live record test [Failed: saved perf.data unreadable]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live record test [Success]" +} + test_plumbing test_no_events +test_live_synthetic if ! record; then echo "Skipping remaining tests, failed to record scheduler events (permissions?)"
@@ -232,6 +531,7 @@ fi test_dump test_headless_ui +test_live_record cleanup exit $err
Signature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
[email protected]
AI reviewRe: [PATCH v1 13/13] perf test: Test perf timechart --live
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
From: Alice Rogers <[email protected]>
Extend
test_ttimechart_python.shto cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file, a synthetic live UI test exercising the rolling window, pruning, pause/resume and zoom following, and a live workload recording test when scheduler tracepoints are accessible.
Quoted 4 lines
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=13
Arnaldo Carvalho de Melo
maintainerQuoted 7 lines · Ian Rogers
This series adds interactive terminal user interfaces (TUIs) built with Python's textual framework for 'perf timechart' (--tui and --live) and 'perf list' (--tui), updates 'treport' to progressively display profiles while loading in the background, adds headless shell tests for all three TUIs, and improves the 'perf' Python extension module to support responsive background and live pipe event processing:
Quoted 15 lines
1. Session and Python extension improvements (patches 1-5):
- Skip flushing remaining deferred samples and auxtrace events oncesession_done()is set so stopping event processing is immediate.
- Quietly stop processing events when a Python session callback raises an exception (e.g., to cancel a background load on quit) without corrupting TUI output with reader error messages.
- Lazily copyperf_eventandperf_sampleinprocess_events()only when a Python callback retains a reference beyond its return, shrinking structpyrf_eventto fit in CPython's small-object allocator.
- Lazily unwind and resolve sample callchains only when event.callchain is accessed.
- Release the Python GIL aroundperf_session__new()andperf_session__process_events()(reacquiring it in callbacks and guarding concurrent thread/comm/symbol access) so blocking pipe reads do not freeze TUI threads.
Nice optimizations, applied and tested by running some of the live python scripts (twatch, ilist, etc)
Quoted 3 lines
2. 'perf list --tui' and ilist test (patches 6-7):
- Add 'perf list --tui' to launch the interactive ilist script via 'perf script', and add a shell test driving ilist headlessly.
Applied and tested
Quoted 4 lines
3. Progressive loading in treport and test (patches 8-9):
- Load profiles in a background thread intreport.pyso the report tree and flame graph are interactive while loading, with lazy tree node creation and a headless shell test.
Ditto
Quoted 10 lines
4. 'perf timechart --tui' and '--live' and tests (patches 10-13):
- Addttimechart.pyand 'perf timechart --tui' to interactively view, zoom, and pan per-CPU (busy, frequency, idle), per-task (running, runnable, blocked, wakeups), and I/O timelines along with a summary table.
- Add 'perf timechart --live' to stream live scheduler, power, or I/O events from a 'perf record' subprocess over a pipe with a rolling time window, pause/resume via the control pipe, and optional output recording (-o).
- Add headless shell tests covering both '--tui' and '--live' modes.
Applied up to patch 11, 12 has an issue, you can resubmit once addressing the issue.
Thanks,
- Arnaldo
Quoted 37 lines
Alice Rogers (6):
perf treport: Show the profile while it loads
perf test: Add a test for the treport script
perf timechart: Add an interactive --tui mode
perf test: Add a test for perf timechart --tui
perf timechart: Add a --live mode to the TUI
perf test: Test perf timechart --liveIan Rogers (7):
perf session: Don't flush remaining events once processing is done perf python: Quietly stop processing events when a callback raises perf python: Lazily copy events and samples fromprocess_eventsperf python: Lazily resolve sample callchains
perf python: Release the GIL while processing session events perf list: Add a --tui option to launch ilist
perf test: Add a test for the ilist scripttools/perf/Documentation/perf-list.txt | 10 + tools/perf/Documentation/perf-timechart.txt | 24 + tools/perf/builtin-list.c | 34 + tools/perf/builtin-timechart.c | 114 +- tools/perf/python/treport.py | 458 ++- tools/perf/python/ttimechart.py | 2467 +++++++++++++++++ tools/perf/tests/shell/test_ilist_python.sh | 181 ++ tools/perf/tests/shell/test_treport_python.sh | 247 ++ .../tests/shell/test_ttimechart_python.sh | 537 ++++ tools/perf/util/auxtrace.c | 7 + tools/perf/util/python.c | 1191 +++++--- tools/perf/util/session.c | 8 + 12 files changed, 4754 insertions(+), 524 deletions(-) create mode 100755 tools/perf/python/ttimechart.py create mode 100755 tools/perf/tests/shell/test_ilist_python.sh create mode 100755 tools/perf/tests/shell/test_treport_python.sh create mode 100755 tools/perf/tests/shell/test_ttimechart_python.shSignature · 1 line
2.56.0.rc1.315.gc6ed9934b7-goog
Arnaldo Carvalho de Melo
maintainerQuoted 6 lines · Arnaldo Carvalho de Melo
Quoted 3 lines · Ian Rogers
2. 'perf list --tui' and ilist test (patches 6-7):
- Add 'perf list --tui' to launch the interactive ilist script via 'perf script', and add a shell test driving ilist headlessly.Applied and tested
This one seems to be pre-existent:
root@x2:/home/acme/git/perf-tools-next# tools/perf/python/sctop.py syscall events:
event count
---------------------------------------- ----------
Traceback (most recent call last):
File "/home/acme/git/perf-tools-next/tools/perf/python/sctop.py", line 253, in <module>
main()
~~~~^^
File "/home/acme/git/perf-tools-next/tools/perf/python/sctop.py", line 234, in main
live_session.run()
~~~~~~~~~~~~~~~~^^
File "/home/acme/git/perf-tools-next/tools/perf/python/perf_live.py", line 61, in run
self.sample_callback(event)
~~~~~~~~~~~~~~~~~~~~^^^^^^^ File "/home/acme/git/perf-tools-next/tools/perf/python/sctop.py", line 66, in process_event
name = str(sample.evsel)
^^^^^^^^^^^^
AttributeError: 'perf.sample_event' object has no attribute 'evsel' root@x2:/home/acme/git/perf-tools-next#
Arnaldo Carvalho de Melo
maintainerQuoted 14 lines · Arnaldo Carvalho de Melo
Quoted 6 lines · Arnaldo Carvalho de Melo
Quoted 3 lines · Ian Rogers
2. 'perf list --tui' and ilist test (patches 6-7):
- Add 'perf list --tui' to launch the interactive ilist script via 'perf script', and add a shell test driving ilist headlessly.Applied and tested
This one seems to be pre-existent:
root@x2:/home/acme/git/perf-tools-next# tools/perf/python/sctop.py syscall events:
Quoted 6 lines
event count
---------------------------------------- ----------
Traceback (most recent call last):
File "/home/acme/git/perf-tools-next/tools/perf/python/sctop.py", line 253, in <module>main() ~~~~^^
Nevermind, root was finding some old python binding, after:
root@x2:/home/acme/git/perf-tools-next# export PYTHONPATH=/tmp/build/perf-tools-next/
it works,
- Arnaldo
Ian Rogers
reviewerv2 cover perf timechart: Add --live mode to the TUI and test
Add a '--live' option to 'perf timechart' and 'ttimechart.py' that streams live scheduler, power, or I/O (-I/--io-only) events from a 'perf record' subprocess over a pipe and displays them in the Textual TUI with a rolling time window (--window, default 10s), pause/resume via the control pipe, an optional workload after '--', and optional output recording (-o), along with headless shell tests in test_ttimechart_python.sh.
v2: Rebase the remaining 2 patches onto perf-tools-next after patches 1-11
of v1 were merged.
Alice Rogers (2):
perf timechart: Add a --live mode to the TUI
perf test: Test perf timechart --live
tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 80 +- tools/perf/python/ttimechart.py | 786 ++++++++++++++++-- .../tests/shell/test_ttimechart_python.sh | 300 +++++++ 4 files changed, 1111 insertions(+), 75 deletions(-)
Signature · 1 line
2.56.0.385.gd3acb90ef8-goog
Ian Rogers
reviewerPATCH v2 1/2 perf timechart: Add a --live mode to the TUI
From: Alice Rogers <[email protected]>
Add a --live option to perf timechart and ttimechart.py that runs 'perf record' writing CLOCK_MONOTONIC scheduler and power events, or I/O syscall events when -I/--io-only is given, to a pipe and displays them in the TUI as they happen. Only the most recent --window seconds of history (default 10) are kept in memory, pruning older segments and tasks that have no remaining history in the window.
Pressing space pauses and resumes recording using perf record's --control pipe. Because perf record only reads its ring buffers when a watermark is reached, periodic 'ping' commands are sent on the control pipe to flush events promptly. The refresh interval adapts to the measured screen refresh cost and backs off if unprocessed data builds up in the pipe, with two flushes per refresh to account for ordered-event buffering across rounds.
An optional workload may be given after '--' to record just that command rather than the whole system, and '-o' optionally saves a copy of the recorded perf.data stream.
--- tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 80 +- tools/perf/python/ttimechart.py | 786 ++++++++++++++++++-- 3 files changed, 811 insertions(+), 75 deletions(-)
tools/perf/Documentation/perf-timechart.txt +18 -2
@@ -47,6 +47,9 @@ TIMECHART OPTIONS -T:: --tasks-only:: Don't output processor state transitions +-I:: +--io-only:: + In '--live' or '--tui' mode, only record or show I/O activity -p:: --process:: Select the processes to display, by name or PID
@@ -59,8 +62,21 @@ TIMECHART OPTIONS linkperf:perf-script[1]). The script requires the perf python module and the python 'textual' library. The CPU, task, I/O and summary views can be zoomed ('+'/'-'), panned (shift+arrows) and the state of the - selected row at the cursor is described. The -i, -P, -T and -p options - are passed to the script, other output options are ignored. + selected row at the cursor is described. The -i, -P, -T, -I and -p + options are passed to the script, other output options are ignored. +--live:: + Record scheduler and power events, or I/O events with '-I', with + linkperf:perf-record[1] and display them as they happen in the + terminal timechart, implying --tui. Only the most recent '--window' + seconds of events are kept and shown. Pressing space pauses and + resumes recording; zooming or panning stops the view following the + latest events until '0' is pressed. If a command is given after '--' + then only that workload is recorded and recording stops when it exits, + otherwise the whole system is recorded until the UI is quit. If '-o' + is given the recorded perf data is also saved to that file. +--window=<seconds>:: + In '--live' mode, the number of seconds of recent events kept in + memory and shown in the timeline (default: 10). --symfs=<directory[,layout]>:: Look for files with symbols relative to this directory. The optional layout can be 'hierarchy' (default, matches full path) or 'flat'
tools/perf/builtin-timechart.c +70 -10
159 lines not on this page · raw message
tools/perf/python/ttimechart.py +723 -63
1,028 lines not on this page · raw message
Signature · 1 line
2.56.0.385.gd3acb90ef8-goog
[email protected]
AI reviewRe: [PATCH v2 1/2] perf timechart: Add a --live mode to the TUI
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] perf: missing <string.h> inclusion for string functions --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] perf: missing <string.h> inclusion for string functions
--
commit affd0db9a1d96743f8d4a342d96e560618e1bb09
Author: Alice Rogers <[email protected]>
perf timechart: Add a --live mode to the TUI
This commit adds a --live option to perf timechart and ttimechart.py that runs perf record and displays the events interactively in the TUI. It limits the display to the most recent window of history and allows pausing or recording specific workloads.
Quoted 4 lines
tools/perf/builtin-timechart.c +0
-0
[ ... ]
tools/perf/builtin-timechart.c int cmd_timechart(int argc, const char **argv) · line 2216
Quoted 12 lines
tools/perf/builtin-timechart.c +6
-1@@ -2207,13 +2207,24 @@ int cmd_timechart(int argc, const char **argv) ret =-1 ; goto out; } +if (tchart.io_only && (tchart.power_only || tchart.tasks_only)) { +pr_err ("-I cannot be used with -P or -T.\n" ); + ret =-1 ; + goto out; + } -if (argc &&strlen (argv[0 ]) >2 &&strstarts ("record" , argv[0 ])) { +if (!tchart.live && argc &&strlen (argv[0 ]) >2 &&strstarts ("record" , argv[0 ])) {
[Severity: Low]
Does this file need to explicitly include <string.h> for musl compatibility? The perf subsystem guide mandates explicit inclusions of POSIX headers for libc functions to ensure the tools can be compiled under musl, which strictly separates declarations, unlike glibc. Since strlen() and strstarts() are used here, should <string.h> be added to the includes at the top of the file?
Quoted 3 lines
tools/perf/builtin-timechart.c +0
-0argc =parse_options (argc, argv, timechart_record_options, timechart_record_usage,PARSE_OPT_STOP_AT_NON_OPTION );
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v2 2/2 perf test: Test perf timechart --live
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Extend test_ttimechart_python.sh to cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file, a synthetic live UI test exercising the rolling window, pruning, pause/resume and zoom following, and a live workload recording test when scheduler tracepoints are accessible.
--- .../tests/shell/test_ttimechart_python.sh | 300 ++++++++++++++++++ 1 file changed, 300 insertions(+)
tools/perf/tests/shell/test_ttimechart_python.sh +300 -0
@@ -82,6 +82,34 @@ test_plumbing() { err=1 return fi + if perf timechart --live -i "${perfdata}" > "${output}" 2>&1 || + ! grep -q -- "--live can't be used with -i" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --live with -i not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --live --window 0 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "--window must be a positive" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window 0 not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --tui --window 5 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "require --live" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window without --live]" + cat "${output}" + err=1 + return + fi + if perf timechart --live -I -P < /dev/null > "${output}" 2>&1 || + ! grep -q -- "-I cannot be used with -P or -T" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: -I with -P not rejected]" + cat "${output}" + err=1 + return + fi echo "perf timechart --tui plumbing test [Success]" }
@@ -115,6 +143,43 @@ test_no_events() { err=1 return fi + # Exercise LiveRecorder's perf record subprocess, control pipe and + # -o output file using task-clock when tracepoints may be unavailable. + rm -f "${clockdata}" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${clockdata}" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import os +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.events = lambda: ["task-clock"] + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + await pilot.press("space") + await pilot.pause(0.1) + await pilot.press("space") + for _ in range(300): + if app.return_code is not None: + break + await pilot.pause(0.05) + finally: + rec.stop() + if app.return_code != 1 or not app.data.nr_samples or not os.path.getsize(sys.argv[2]): + raise RuntimeError(f"Unexpected result: rc={app.return_code} samples={app.data.nr_samples}") + +asyncio.run(run()) +EOF + then + echo "ttimechart no events test [Failed: LiveRecorder task-clock test]" + cat "${output}" + err=1 + return + fi echo "ttimechart no events test [Success]" }
@@ -218,8 +283,242 @@ EOF echo "ttimechart headless UI test [Success]" } +test_live_synthetic() { + echo "ttimechart live UI test (synthetic)" + # Exercise live mode's rolling window, pruning, pause/resume and + # following with a fake recorder, which doesn't need tracepoint + # permissions. + if ! "$PYTHON" - > "${output}" 2>&1 <<'EOF' +import asyncio +import threading +import time +from time import monotonic_ns +import ttimechart as tc + +SEC = 1_000_000_000 + +class Ev: + def __init__(self, name): + self.name = name + def __str__(self): + return f"evsel({self.name})" + +class Sample: + def __init__(self, name, t, cpu, tid, **kw): + self.evsel = Ev(name) + self.sample_time = t + self.sample_cpu = cpu + self.sample_tid = tid + self.sample_pid = tid + self.sample_id = hash(name) + for k, v in kw.items(): + setattr(self, k, v) + +class FakeRecorder: + def __init__(self): + self.origin = monotonic_ns() + self.data_fd = -1 + self.flush_interval = 0.05 + self.paused = False + self.pause_calls = [] + self.stopped = threading.Event() + self.finish = False + + def backlog(self): + return 0.0 + + def set_paused(self, paused): + self.paused = paused + self.pause_calls.append(paused) + + def stop(self): + self.stopped.set() + + def error_text(self): + return "" + +async def run() -> None: + rec = FakeRecorder() + + def fake_read(data, _fd): + t0 = rec.origin + # Task 300 only runs at the start so pruning should drop it once the + # 1s window advances past t0 + 3000. + data.process_event(Sample("sched:sched_wakeup", t0 + 1000, 0, 1, + pid=300, comm="early", common_pid=1, common_flags=0)) + data.process_event(Sample("sched:sched_switch", t0 + 2000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=300, next_comm="early")) + data.process_event(Sample("sched:sched_switch", t0 + 3000, 0, 300, + prev_pid=300, prev_comm="early", prev_state=1, + next_pid=0, next_comm="swapper")) + cur = 0 + while not rec.stopped.is_set() and not rec.finish: + now = monotonic_ns() + if rec.paused: + time.sleep(0.01) + continue + nxt = 100 if cur == 0 else 0 + comm = "worker" if nxt == 100 else "swapper" + prev_comm = "worker" if cur == 100 else "swapper" + if nxt: + data.process_event(Sample("sched:sched_wakeup", now, 0, cur or 1, + pid=nxt, comm=comm, common_pid=cur or 1, + common_flags=0)) + data.process_event(Sample("sched:sched_switch", now, 0, cur, + prev_pid=cur, prev_comm=prev_comm, prev_state=1, + next_pid=nxt, next_comm=comm)) + cur = nxt + time.sleep(0.005) + + tc.read_events = fake_read + app = tc.TimechartApp("live", False, False, [], live=rec, window=1.0) + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(300): + with app.data.lock: + if app.data.pruned_to > rec.origin + 3000: + break + await pilot.pause(0.05) + with app.data.lock: + if app.data.pruned_to <= rec.origin + 3000 or 300 in app.data.tasks: + raise RuntimeError("Task outside the window was not pruned") + starts = [t.segs.starts[0] for t in app.data.tasks.values() if len(t.segs)] + if not starts or min(starts) < app.data.pruned_to: + raise RuntimeError(f"Unpruned segments: {starts} < {app.data.pruned_to}") + if app.window.last - app.window.first != SEC: + raise RuntimeError(f"Unexpected window span: {app.window}") + + await pilot.press("space") + await pilot.pause(0.2) + if rec.pause_calls != [True] or "Paused" not in app.sub_title: + raise RuntimeError(f"Did not pause: {rec.pause_calls} {app.sub_title}") + frozen = app.window.last + await pilot.pause(0.4) + if app.window.last != frozen: + raise RuntimeError("Window moved while paused") + await pilot.press("space") + await pilot.pause(0.3) + if rec.pause_calls != [True, False] or not app.following: + raise RuntimeError("Did not resume") + + await pilot.press("plus") + await pilot.pause(0.2) + if app.following: + raise RuntimeError("Zooming did not stop following") + await pilot.press("0") + await pilot.pause(0.2) + if not app.following: + raise RuntimeError("Resetting zoom did not resume following") + + rec.finish = True + for _ in range(100): + if not app.recording: + break + await pilot.pause(0.05) + if app.recording or "Finished" not in app.sub_title: + raise RuntimeError(f"Did not finish: {app.sub_title}") + + # Also test live I/O-only mode (-I). + io_events = tc.LiveRecorder("perf", False, False, [], None, io_only=True).events() + if not io_events or not all(e.startswith("syscalls:sys_") for e in io_events): + raise RuntimeError(f"Unexpected I/O events: {io_events}") + io_rec = FakeRecorder() + + def fake_io_read(data, _fd): + now = monotonic_ns() + # Self I/O should be ignored in live mode. + own = tc.os.getpid() + data.process_event(Sample("syscalls:sys_enter_read", now, 0, own, fd=3)) + data.process_event(Sample("syscalls:sys_exit_read", now + 1000, 0, own, ret=64)) + # Workload I/O should be recorded. + data.process_event(Sample("syscalls:sys_enter_read", now + 2000, 0, 4242, fd=4)) + data.process_event(Sample("syscalls:sys_exit_read", now + 5000, 0, 4242, ret=4096)) + data.process_event(Sample("syscalls:sys_enter_write", now + 6000, 0, 4242, fd=5)) + data.process_event(Sample("syscalls:sys_exit_write", now + 9000, 0, 4242, ret=2048)) + + tc.read_events = fake_io_read + io_app = tc.TimechartApp("live", False, False, [], live=io_rec, window=1.0, io_only=True) + async with io_app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + if not io_app.recording: + break + await pilot.pause(0.05) + await pilot.pause() + tabs = io_app.query_one(tc.TabbedContent) + if tabs.active != "io" or not io_app.io_tasks or io_app.data.io_events != 4: + raise RuntimeError(f"Unexpected I/O state: active={tabs.active} " + f"tasks={len(io_app.io_tasks)} events={io_app.data.io_events}") + tabs.active = "summary" + await pilot.pause() + await pilot.press("enter") + await pilot.pause() + if tabs.active != "io": + raise RuntimeError(f"Summary enter did not activate I/O tab: {tabs.active}") + print("live synthetic ok") + +asyncio.run(run()) +EOF + then + echo "ttimechart live UI test (synthetic) [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "live synthetic ok" "${output}"; then + echo "ttimechart live UI test (synthetic) [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live UI test (synthetic) [Success]" +} + +test_live_record() { + echo "ttimechart live record test" + rm -f "${perfdata}.live" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${perfdata}.live" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if not app.recording: + break + await pilot.pause(0.1) + if app.recording or not app.data.has_events(): + raise RuntimeError("Live recording of workload did not complete") + print("live record ok") + finally: + rec.stop() + +asyncio.run(run()) +EOF + then + echo "ttimechart live record test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! perf script ttimechart -i "${perfdata}.live" --dump > "${output}" 2>&1 || + ! grep -q "^Duration: .*sched events: [1-9]" "${output}"; then + echo "ttimechart live record test [Failed: saved perf.data unreadable]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live record test [Success]" +} + test_plumbing test_no_events +test_live_synthetic if ! record; then echo "Skipping remaining tests, failed to record scheduler events (permissions?)"
@@ -232,6 +531,7 @@ fi test_dump test_headless_ui +test_live_record cleanup exit $err
Signature · 1 line
2.56.0.385.gd3acb90ef8-goog
[email protected]
AI reviewRe: [PATCH v2 2/2] perf test: Test perf timechart --live
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] perf/tests: test hang due to missing FakeRecorder cleanup on error paths --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] perf/tests: test hang due to missing FakeRecorder cleanup on error paths --
commit 160f73df934f65800928fbfdebb9b1aabb7e4e95
Author: Alice Rogers <[email protected]>
perf test: Test perf timechart --live
This commit extends test_ttimechart_python.sh to validate the --live option. It exercises LiveRecorder's perf record subprocess, control pipe, output file, and introduces a synthetic live UI test for checking the rolling window, pruning, and zoom tracking features.
Quoted 4 lines
tools/perf/tests/shell/test_ttimechart_python.sh +0
-0
[ ... ]
Quoted 8 lines
tools/perf/tests/shell/test_ttimechart_python.sh +8
-0+test_live_synthetic() { + echo "ttimechart live UI test (synthetic)" + # Exercise live mode's rolling window, pruning, pause/resume and + # following with a fake recorder, which doesn't need tracepoint + # permissions. + if ! "$PYTHON" - > "${output}" 2>&1 <<'EOF' +import asyncio +import threading
[ ... ]
Quoted 6 lines
tools/perf/tests/shell/test_ttimechart_python.sh +6
-0+ cur = 0 + while not rec.stopped.is_set() and not rec.finish: + now = monotonic_ns() + if rec.paused: + time.sleep(0.01) + continue
[ ... ]
tools/perf/tests/shell/test_ttimechart_python.sh EOF · line 384
Quoted 11 lines
tools/perf/tests/shell/test_ttimechart_python.sh +11
-0+ tc.read_events = fake_read + app = tc.TimechartApp("live", False, False, [], live=rec, window=1.0) + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(300): + with app.data.lock: + if app.data.pruned_to > rec.origin + 3000: + break + await pilot.pause(0.05) + with app.data.lock: + if app.data.pruned_to <= rec.origin + 3000 or 300 in app.data.tasks: + raise RuntimeError("Task outside the window was not pruned")
[Severity: Medium]
Will this cause the test suite to hang if an exception is raised here?
Because fake_read runs in a background thread and its loop relies on rec.stopped.is_set() or rec.finish, an exception (like RuntimeError on a failed assertion) will abort the test logic without executing rec.finish = True or calling rec.stop().
This lack of cleanup leaves the background thread looping indefinitely and prevents the Python interpreter from exiting. Other tests introduced in this patch like test_no_events and test_live_record use try...finally blocks to ensure rec.stop() is always called.
Should a similar pattern be applied here to guarantee termination?
Quoted 3 lines
tools/perf/tests/shell/test_ttimechart_python.sh +3
-0+ starts = [t.segs.starts[0] for t in app.data.tasks.values() if len(t.segs)] + if not starts or min(starts) < app.data.pruned_to: + raise RuntimeError(f"Unpruned segments: {starts} < {app.data.pruned_to}")
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerv3 cover perf timechart: Add --live mode to the TUI and test
In reply to Ian RogersAdd a '--live' option to 'perf timechart' and 'ttimechart.py' that streams live scheduler, power, or I/O (-I/--io-only) events from a 'perf record' subprocess over a pipe and displays them in the Textual TUI with a rolling time window (--window, default 10s), pause/resume via the control pipe, an optional workload after '--', and optional output recording (-o), along with headless shell tests in test_ttimechart_python.sh.
v3: Address Sashiko review feedback:
- Patch 1: Explicitly include <string.h> in builtin-timechart.c.
- Patch 1: Reject non-finite --window values (e.g. 'inf') that
overflowed converting to nanoseconds.
- Patch 2: Stop the fake recorder in a try/finally in the synthetic
live UI test so a failed check can't leave its background thread
running and hang the test.
Rebase onto the latest perf-tools-next.
v2: https://lore.kernel.org/linux-perf-users/[email protected]/
v2: Rebase the remaining 2 patches onto perf-tools-next after patches 1-11
of v1 were merged.
Alice Rogers (2):
perf timechart: Add a --live mode to the TUI
perf test: Test perf timechart --live
tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 81 +- tools/perf/python/ttimechart.py | 786 ++++++++++++++++-- .../tests/shell/test_ttimechart_python.sh | 303 +++++++ 4 files changed, 1115 insertions(+), 75 deletions(-)
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
Ian Rogers
reviewerPATCH v3 1/2 perf timechart: Add a --live mode to the TUI
From: Alice Rogers <[email protected]>
Add a --live option to perf timechart and ttimechart.py that runs 'perf record' writing CLOCK_MONOTONIC scheduler and power events, or I/O syscall events when -I/--io-only is given, to a pipe and displays them in the TUI as they happen. Only the most recent --window seconds of history (default 10) are kept in memory, pruning older segments and tasks that have no remaining history in the window.
Pressing space pauses and resumes recording using perf record's --control pipe. Because perf record only reads its ring buffers when a watermark is reached, periodic 'ping' commands are sent on the control pipe to flush events promptly. The refresh interval adapts to the measured screen refresh cost and backs off if unprocessed data builds up in the pipe, with two flushes per refresh to account for ordered-event buffering across rounds.
An optional workload may be given after '--' to record just that command rather than the whole system, and '-o' optionally saves a copy of the recorded perf.data stream.
--- tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 81 +- tools/perf/python/ttimechart.py | 786 ++++++++++++++++++-- 3 files changed, 812 insertions(+), 75 deletions(-)
tools/perf/Documentation/perf-timechart.txt +18 -2
@@ -47,6 +47,9 @@ TIMECHART OPTIONS -T:: --tasks-only:: Don't output processor state transitions +-I:: +--io-only:: + In '--live' or '--tui' mode, only record or show I/O activity -p:: --process:: Select the processes to display, by name or PID
@@ -59,8 +62,21 @@ TIMECHART OPTIONS linkperf:perf-script[1]). The script requires the perf python module and the python 'textual' library. The CPU, task, I/O and summary views can be zoomed ('+'/'-'), panned (shift+arrows) and the state of the - selected row at the cursor is described. The -i, -P, -T and -p options - are passed to the script, other output options are ignored. + selected row at the cursor is described. The -i, -P, -T, -I and -p + options are passed to the script, other output options are ignored. +--live:: + Record scheduler and power events, or I/O events with '-I', with + linkperf:perf-record[1] and display them as they happen in the + terminal timechart, implying --tui. Only the most recent '--window' + seconds of events are kept and shown. Pressing space pauses and + resumes recording; zooming or panning stops the view following the + latest events until '0' is pressed. If a command is given after '--' + then only that workload is recorded and recording stops when it exits, + otherwise the whole system is recorded until the UI is quit. If '-o' + is given the recorded perf data is also saved to that file. +--window=<seconds>:: + In '--live' mode, the number of seconds of recent events kept in + memory and shown in the timeline (default: 10). --symfs=<directory[,layout]>:: Look for files with symbols relative to this directory. The optional layout can be 'hierarchy' (default, matches full path) or 'flat'
tools/perf/builtin-timechart.c +71 -10
160 lines not on this page · raw message
tools/perf/python/ttimechart.py +723 -63
1,028 lines not on this page · raw message
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
[email protected]
AI reviewRe: [PATCH v3 1/2] perf timechart: Add a --live mode to the TUI
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider: - [Medium] perf timechart: checkpoint() uses last_time while prune() uses monotonic_ns() - [Medium] perf…
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [Medium] perf timechart: checkpoint() uses last_time while prune() uses monotonic_ns()
- [Medium] perf timechart: recording_finished() jumps live_bounds() to pruned history
- [Medium] perf timechart: action_toggle_pause() leaves task and CPU states open
- [Medium] perf timechart: LiveRecorder leaks tee_r and stdout pipe file descriptors
- [Medium] perf timechart: LiveRecorder.start() leaks perf record process on pipe error --
commit 082000f3148508f20fda16552cb18e198341c26e
Author: Alice Rogers <[email protected]>
perf timechart: Add a --live mode to the TUI
This commit adds a --live option to perf timechart and ttimechart.py to record scheduler, power, or I/O events with perf record into a pipe and display a sliding window of recent history in the interactive TUI.
Quoted 4 lines
tools/perf/python/ttimechart.py +0
-0
[ ... ]
Quoted 17 lines
tools/perf/python/ttimechart.py +13
-0@@ -265,6 +317,38 @@ class Task: return True return any(f == str(self.tid) or f in self.comms for f in filters) + def prune(self, time: int) -> None: + """Discard history ending at or before time, removing it from the totals.""" + segs = self.segs + num = segs.num_ending_by(time) + for i in range(num): + state = segs.keys[i] + self.totals[state] -= segs.ends[i] - segs.starts[i] + if state == STATE_RUNNING: + # Running segments end when the task is switched out. + self.switches = max(self.switches - 1, 0) + if num < len(segs.starts) and segs.starts[num] < time: + self.totals[segs.keys[num]] -= time - segs.starts[num] + segs.prune(time)
[ ... ]
Quoted 4 lines
tools/perf/python/ttimechart.py +4
-0+ if self.since <= time: + self.since = time + if not segs and self.state != STATE_RUNNING: + self.state = STATE_UNKNOWN
[ ... ]
Quoted 14 lines
tools/perf/python/ttimechart.py +10
-0@@ -281,6 +365,17 @@ class Cpu: self.pstate = SegmentList() self.pstate_cur: Optional[Tuple[int, int]] = None + def prune(self, time: int) -> None: + """Discard history ending at or before time.""" + self.run.prune(time) + self.cstate.prune(time) + self.pstate.prune(time) + self.since = max(self.since, time) + if self.cstate_cur and self.cstate_cur[0] < time: + self.cstate_cur = (time, self.cstate_cur[1]) + if self.pstate_cur and self.pstate_cur[0] < time: + self.pstate_cur = (time, self.pstate_cur[1])
[ ... ]
Quoted 29 lines
tools/perf/python/ttimechart.py +25
-0@@ -562,6 +688,32 @@ class TimechartData: except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError): pass + def checkpoint(self) -> None: + """Extend open running, idle and frequency states to last_time. + + Live mode calls this before displaying and pruning so that states that + haven't changed recently are shown up to the latest event, rather than + only once they end. The lock must be held. + """ + end = self.last_time + if not end: + return + for task in self.tasks.values(): + if task.state == STATE_RUNNING and end > task.since: + task.segs.add(task.since, end, STATE_RUNNING, task.cpu) + task.totals[STATE_RUNNING] += end - task.since + task.since = end + for c in self.cpus.values(): + if c.cur_tid > 0 and end > c.since: + c.run.add(c.since, end, 1, c.cur_tid) + c.since = end + if c.cstate_cur and end > c.cstate_cur[0]: + c.cstate.add(c.cstate_cur[0], end, 1, c.cstate_cur[1]) + c.cstate_cur = (end, c.cstate_cur[1]) + if c.pstate_cur and end > c.pstate_cur[0]: + c.pstate.add(c.pstate_cur[0], end, 1, c.pstate_cur[1]) + c.pstate_cur = (end, c.pstate_cur[1])
[ ... ]
Quoted 1 line
tools/perf/python/ttimechart.py +0
-0@@ -1486,6 +1674,186 @@ class TimechartApp(App):
[ ... ]
Quoted 15 lines
tools/perf/python/ttimechart.py +15
-0+ def recording_finished(self) -> None: + """Called on the UI thread when perf record exits, such as when the workload ends.""" + self.data.finish() + self.recording = False + self.paused_at = 0 + self.loaded = True + self.refresh_bindings() + self.update_views() + self.update_live_status() + + def live_bounds(self) -> Tuple[int, int]: + """The time range kept and shown in live mode, the last window of the recording. + + Until a window of time has been recorded the range is the first window, + so the timeline fills from the left, after that it moves with time.
tools/perf/python/ttimechart.py class TimechartApp(App): · line 1743
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ """ + assert self.live + if self.paused_at: + now = self.paused_at + elif self.recording: + now = monotonic_ns() + else: + now = self.data.last_time + last = max(now, self.live.origin + self.live_window) + return max(self.live.origin, last - self.live_window), last
[Severity: Medium]
In tools/perf/python/ttimechart.py, when perf record exits and TimechartApp.recording_finished() sets self.recording = False before calling TimechartApp.update_views() -> TimechartApp.live_bounds(), can the displayed window jump backwards into already-pruned history or leave final flushed events unpruned?
While self.recording is True, TimechartApp.live_bounds() computes the window from monotonic_ns(), and TimechartApp.live_refresh() prunes history prior to monotonic_ns() - self.live_window (stored in self.data.pruned_to).
Once self.recording becomes False, live_bounds() switches now to self.data.last_time without clamping the lower bound to at least self.data.pruned_to. If no events arrived shortly before the workload exited (for example, during a sleep or between I/O bursts with -I), last - self.live_window falls earlier than self.data.pruned_to, shifting the view back into a time range whose segments were already discarded.
Conversely, if perf record flushes buffered events on exit in read_events() after the last timer refresh, recording_finished() does not call self.data.prune(self.live_bounds()[0]) and self.prune_rows(), leaving history older than the final window in the task totals and segments.
Quoted 7 lines
tools/perf/python/ttimechart.py +7
-0+ def live_refresh(self) -> None: + """Show newly recorded events and discard those no longer in the window.""" + start = monotonic() + with self.data.lock: + samples = self.data.nr_samples + # When paused, once the buffered events are shown there's nothing new. + if not self.paused_at or samples != self.refreshed_samples:
tools/perf/python/ttimechart.py class TimechartApp(App): · line 1761
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ self.refreshed_samples = samples + with self.data.lock: + self.data.checkpoint() + if self.recording and not self.paused_at: + self.data.prune(self.live_bounds()[0]) + self.prune_rows() + self.update_views() + self.update_live_status() + # Measure the cost of the refresh including drawing the screen. + self.call_after_refresh(self.live_refreshed, start)
[Severity: Medium]
What happens in TimechartApp.live_refresh() when self.data.last_time lags behind the prune timestamp returned by self.live_bounds()[0]?
In TimechartApp.live_refresh(), self.data.checkpoint() extends open running, idle, and frequency states only up to end = self.last_time (the timestamp of the last processed sample). Immediately afterwards, self.data.prune() is called with self.live_bounds()[0], which is computed from monotonic_ns() - self.live_window.
When no new events have been processed for --window seconds (for example, a CPU-bound single-threaded workload without context switches, steady power states under -P, or an idle period), the prune timestamp time becomes greater than or equal to self.data.last_time:
1. In Task.prune(), the checkpointed running segment for a still-running task ends at self.last_time <= time, so segs.num_ending_by(time) includes it. Task.prune() deletes that segment, subtracts its duration from self.totals, and decrements self.switches as if the task had switched out, causing the active running task to lose all segments and disappear from TimechartData.sched_tasks().
2. Both Task.prune() and Cpu.prune() then advance self.since, self.cstate_cur[0], and self.pstate_cur[0] to time without checkpointing the open state across [self.last_time, time], permanently dropping active task and CPU state across that interval.
Should TimechartData.checkpoint() or TimechartData.prune() ensure open states are extended at least up to the prune timestamp (or avoid pruning past self.data.last_time) and avoid decrementing Task.switches for the open edge of a still-running segment?
[ ... ]
tools/perf/python/ttimechart.py class TimechartApp(App): · line 1835
Quoted 13 lines
tools/perf/python/ttimechart.py +13
-0+ def action_toggle_pause(self) -> None: + """Pause or resume live recording, resuming shows the latest events again.""" + if not self.live or not self.recording: + return + if self.paused_at: + self.paused_at = 0 + self.following = True + self.follow_cursor = True + else: + self.paused_at = monotonic_ns() + self.live.set_paused(bool(self.paused_at)) + self.update_views() + self.update_live_status()
[Severity: Medium]
When pausing and resuming live recording in TimechartApp.action_toggle_pause() via LiveRecorder.set_paused(), how are open task and CPU states handled across the unrecorded gap?
While perf record is disabled via the control pipe, no sched_switch, sched_wakeup, cpu_idle, or syscall tracepoints are recorded, but open states in TimechartData (task.state, task.since, c.cur_tid, c.since, c.cstate_cur, and task.io_pending) are left open at their pre-pause timestamps.
When recording is resumed and new events arrive:
1. TimechartData.checkpoint() and subsequent event handlers extend open segments from the pre-pause timestamp to the post-resume timestamp, attributing the entire paused period as continuous running or idle time.
2. If a task that was in STATE_RUNNING on a CPU when paused switches out or exits while recording is disabled, its sched_switch out is missed. When events resume on that CPU with a different prev_tid:
TimechartData.sched_switch() {
...
if c.cur_tid > 0:
c.run.add(c.since, time, 1, c.cur_tid)
c.cur_tid = next_tid
c.since = time
if prev_tid != 0:
prev = self.task(prev_tid, prev_comm)
...}
only prev_tid and next_tid are updated, while the old task (c.cur_tid) remains in STATE_RUNNING forever and is extended up to self.last_time on every subsequent TimechartData.checkpoint() call.
Should pausing (or resuming) close or reset open task, CPU, and pending I/O states across the disabled interval, and should TimechartData.sched_switch() also handle the case where c.cur_tid > 0 and c.cur_tid != prev_tid?
[ ... ]
Quoted 1 line
tools/perf/python/ttimechart.py +0
-0@@ -1727,52 +2110,329 @@ class TimechartApp(App):
[ ... ]
Quoted 12 lines
tools/perf/python/ttimechart.py +12
-0+ def start(self) -> None: + """Start perf record, the data is then read from data_fd.""" + output = open(self.output, "wb") if self.output else None + ctl_r, self.ctl_fd = os.pipe() + try: + self.ack_fd, ack_w = os.pipe() + except OSError: + os.close(ctl_r) + self.close_control_fds() + if output: + output.close() + raise
[ ... ]
Quoted 15 lines
tools/perf/python/ttimechart.py +15
-0+ try: + # A new session so that signals for the terminal aren't received. + self.proc = subprocess.Popen(cmd, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, + stderr=self.stderr, pass_fds=(ctl_r, ack_w), + start_new_session=True) + except OSError: + self.close_control_fds() + if output: + output.close() + raise + finally: + os.close(ctl_r) + os.close(ack_w) + assert self.proc.stdout + self.data_fd = self.proc.stdout.fileno()
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2226
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ self.grow_pipe(self.data_fd) + if output: + # Save the data as it is passed on to be processed. + tee_r, tee_w = os.pipe() + self.tee_thread = threading.Thread(target=self.tee, + args=(self.data_fd, tee_w, output), + daemon=True) + self.tee_thread.start() + self.data_fd = tee_r + self.grow_pipe(self.data_fd)
[Severity: Medium]
In LiveRecorder.start(), if os.pipe() for (tee_r, tee_w) raises OSError after subprocess.Popen() has already launched perf record with start_new_session=True, will the detached perf record subprocess and open output file be leaked when main() catches OSError and exits?
Also, if the first os.pipe() call for (ctl_r, self.ctl_fd) raises OSError, output is not closed before the exception propagates.
Quoted 33 lines
tools/perf/python/ttimechart.py +33
-0+ self.control_thread = threading.Thread(target=self.control_loop, daemon=True) + self.control_thread.start() + + @staticmethod + def grow_pipe(fd: int) -> None: + """Make the pipe bigger, if allowed, to absorb bursts of events.""" + try: + fcntl.fcntl(fd, fcntl.F_SETPIPE_SZ, LIVE_PIPE_SIZE) + except OSError: + pass + + def tee(self, src: int, dst: int, output: Any) -> None: + """Copy the data from src to dst and the output file.""" + os.set_blocking(dst, False) + forwarding = True + try: + while True: + buf = os.read(src, 1 << 16) + if not buf: + break + output.write(buf) + view = memoryview(buf) + while forwarding and not self.stopping.is_set() and view: + try: + view = view[os.write(dst, view):] + except BlockingIOError: + select.select([], [dst], [], 0.1) + except OSError: + # The data is no longer processed, keep saving it. + forwarding = False + finally: + output.close() + os.close(dst)
[ ... ]
Quoted 8 lines
tools/perf/python/ttimechart.py +8
-0+ def stop(self) -> None: + """Stop recording, perf record then writes any remaining data and exits.""" + self.stopping.set() + self.commands.put("") + proc = self.proc + if proc is not None and proc.poll() is None: + try: + proc.send_signal(signal.SIGINT)
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2330
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ proc.wait(timeout=1) + except subprocess.TimeoutExpired: + # Probably blocked writing data that is no longer processed. + proc.kill() + proc.wait() + if self.tee_thread and self.tee_thread is not threading.current_thread(): + self.tee_thread.join(timeout=1) + if self.control_thread and self.control_thread is not threading.current_thread(): + self.control_thread.join(timeout=1) + self.close_control_fds()
[Severity: Medium]
Does LiveRecorder leak the tee_r pipe file descriptor (assigned to self.data_fd) and self.proc.stdout?
When read_events() creates perf.data(fd=source), pyrf_data__init() in tools/perf/util/python.c duplicates the file descriptor with dup(fd), so destroying the perf.data object only closes the duplicated descriptor and leaves live.data_fd open.
Because tee_r (self.data_fd) and self.proc.stdout are never closed in read_events() or LiveRecorder.stop(), LiveRecorder.tee() will also never see BrokenPipeError on os.write(dst, view) if read_events() stops reading early, and instead will hit BlockingIOError once the pipe fills until self.stopping is set.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v3 2/2 perf test: Test perf timechart --live
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Extend test_ttimechart_python.sh to cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file, a synthetic live UI test exercising the rolling window, pruning, pause/resume and zoom following, and a live workload recording test when scheduler tracepoints are accessible.
--- .../tests/shell/test_ttimechart_python.sh | 303 ++++++++++++++++++ 1 file changed, 303 insertions(+)
tools/perf/tests/shell/test_ttimechart_python.sh +303 -0
@@ -82,6 +82,34 @@ test_plumbing() { err=1 return fi + if perf timechart --live -i "${perfdata}" > "${output}" 2>&1 || + ! grep -q -- "--live can't be used with -i" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --live with -i not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --live --window 0 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "--window must be a positive" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window 0 not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --tui --window 5 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "require --live" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window without --live]" + cat "${output}" + err=1 + return + fi + if perf timechart --live -I -P < /dev/null > "${output}" 2>&1 || + ! grep -q -- "-I cannot be used with -P or -T" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: -I with -P not rejected]" + cat "${output}" + err=1 + return + fi echo "perf timechart --tui plumbing test [Success]" }
@@ -115,6 +143,43 @@ test_no_events() { err=1 return fi + # Exercise LiveRecorder's perf record subprocess, control pipe and + # -o output file using task-clock when tracepoints may be unavailable. + rm -f "${clockdata}" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${clockdata}" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import os +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.events = lambda: ["task-clock"] + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + await pilot.press("space") + await pilot.pause(0.1) + await pilot.press("space") + for _ in range(300): + if app.return_code is not None: + break + await pilot.pause(0.05) + finally: + rec.stop() + if app.return_code != 1 or not app.data.nr_samples or not os.path.getsize(sys.argv[2]): + raise RuntimeError(f"Unexpected result: rc={app.return_code} samples={app.data.nr_samples}") + +asyncio.run(run()) +EOF + then + echo "ttimechart no events test [Failed: LiveRecorder task-clock test]" + cat "${output}" + err=1 + return + fi echo "ttimechart no events test [Success]" }
@@ -218,8 +283,245 @@ EOF echo "ttimechart headless UI test [Success]" } +test_live_synthetic() { + echo "ttimechart live UI test (synthetic)" + # Exercise live mode's rolling window, pruning, pause/resume and + # following with a fake recorder, which doesn't need tracepoint + # permissions. + if ! "$PYTHON" - > "${output}" 2>&1 <<'EOF' +import asyncio +import threading +import time +from time import monotonic_ns +import ttimechart as tc + +SEC = 1_000_000_000 + +class Ev: + def __init__(self, name): + self.name = name + def __str__(self): + return f"evsel({self.name})" + +class Sample: + def __init__(self, name, t, cpu, tid, **kw): + self.evsel = Ev(name) + self.sample_time = t + self.sample_cpu = cpu + self.sample_tid = tid + self.sample_pid = tid + self.sample_id = hash(name) + for k, v in kw.items(): + setattr(self, k, v) + +class FakeRecorder: + def __init__(self): + self.origin = monotonic_ns() + self.data_fd = -1 + self.flush_interval = 0.05 + self.paused = False + self.pause_calls = [] + self.stopped = threading.Event() + self.finish = False + + def backlog(self): + return 0.0 + + def set_paused(self, paused): + self.paused = paused + self.pause_calls.append(paused) + + def stop(self): + self.stopped.set() + + def error_text(self): + return "" + +async def run() -> None: + rec = FakeRecorder() + + def fake_read(data, _fd): + t0 = rec.origin + # Task 300 only runs at the start so pruning should drop it once the + # 1s window advances past t0 + 3000. + data.process_event(Sample("sched:sched_wakeup", t0 + 1000, 0, 1, + pid=300, comm="early", common_pid=1, common_flags=0)) + data.process_event(Sample("sched:sched_switch", t0 + 2000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=300, next_comm="early")) + data.process_event(Sample("sched:sched_switch", t0 + 3000, 0, 300, + prev_pid=300, prev_comm="early", prev_state=1, + next_pid=0, next_comm="swapper")) + cur = 0 + while not rec.stopped.is_set() and not rec.finish: + now = monotonic_ns() + if rec.paused: + time.sleep(0.01) + continue + nxt = 100 if cur == 0 else 0 + comm = "worker" if nxt == 100 else "swapper" + prev_comm = "worker" if cur == 100 else "swapper" + if nxt: + data.process_event(Sample("sched:sched_wakeup", now, 0, cur or 1, + pid=nxt, comm=comm, common_pid=cur or 1, + common_flags=0)) + data.process_event(Sample("sched:sched_switch", now, 0, cur, + prev_pid=cur, prev_comm=prev_comm, prev_state=1, + next_pid=nxt, next_comm=comm)) + cur = nxt + time.sleep(0.005) + + tc.read_events = fake_read + app = tc.TimechartApp("live", False, False, [], live=rec, window=1.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(300): + with app.data.lock: + if app.data.pruned_to > rec.origin + 3000: + break + await pilot.pause(0.05) + with app.data.lock: + if app.data.pruned_to <= rec.origin + 3000 or 300 in app.data.tasks: + raise RuntimeError("Task outside the window was not pruned") + starts = [t.segs.starts[0] for t in app.data.tasks.values() if len(t.segs)] + if not starts or min(starts) < app.data.pruned_to: + raise RuntimeError(f"Unpruned segments: {starts} < {app.data.pruned_to}") + if app.window.last - app.window.first != SEC: + raise RuntimeError(f"Unexpected window span: {app.window}") + + await pilot.press("space") + await pilot.pause(0.2) + if rec.pause_calls != [True] or "Paused" not in app.sub_title: + raise RuntimeError(f"Did not pause: {rec.pause_calls} {app.sub_title}") + frozen = app.window.last + await pilot.pause(0.4) + if app.window.last != frozen: + raise RuntimeError("Window moved while paused") + await pilot.press("space") + await pilot.pause(0.3) + if rec.pause_calls != [True, False] or not app.following: + raise RuntimeError("Did not resume") + + await pilot.press("plus") + await pilot.pause(0.2) + if app.following: + raise RuntimeError("Zooming did not stop following") + await pilot.press("0") + await pilot.pause(0.2) + if not app.following: + raise RuntimeError("Resetting zoom did not resume following") + + rec.finish = True + for _ in range(100): + if not app.recording: + break + await pilot.pause(0.05) + if app.recording or "Finished" not in app.sub_title: + raise RuntimeError(f"Did not finish: {app.sub_title}") + finally: + rec.stop() + + # Also test live I/O-only mode (-I). + io_events = tc.LiveRecorder("perf", False, False, [], None, io_only=True).events() + if not io_events or not all(e.startswith("syscalls:sys_") for e in io_events): + raise RuntimeError(f"Unexpected I/O events: {io_events}") + io_rec = FakeRecorder() + + def fake_io_read(data, _fd): + now = monotonic_ns() + # Self I/O should be ignored in live mode. + own = tc.os.getpid() + data.process_event(Sample("syscalls:sys_enter_read", now, 0, own, fd=3)) + data.process_event(Sample("syscalls:sys_exit_read", now + 1000, 0, own, ret=64)) + # Workload I/O should be recorded. + data.process_event(Sample("syscalls:sys_enter_read", now + 2000, 0, 4242, fd=4)) + data.process_event(Sample("syscalls:sys_exit_read", now + 5000, 0, 4242, ret=4096)) + data.process_event(Sample("syscalls:sys_enter_write", now + 6000, 0, 4242, fd=5)) + data.process_event(Sample("syscalls:sys_exit_write", now + 9000, 0, 4242, ret=2048)) + + tc.read_events = fake_io_read + io_app = tc.TimechartApp("live", False, False, [], live=io_rec, window=1.0, io_only=True) + async with io_app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + if not io_app.recording: + break + await pilot.pause(0.05) + await pilot.pause() + tabs = io_app.query_one(tc.TabbedContent) + if tabs.active != "io" or not io_app.io_tasks or io_app.data.io_events != 4: + raise RuntimeError(f"Unexpected I/O state: active={tabs.active} " + f"tasks={len(io_app.io_tasks)} events={io_app.data.io_events}") + tabs.active = "summary" + await pilot.pause() + await pilot.press("enter") + await pilot.pause() + if tabs.active != "io": + raise RuntimeError(f"Summary enter did not activate I/O tab: {tabs.active}") + print("live synthetic ok") + +asyncio.run(run()) +EOF + then + echo "ttimechart live UI test (synthetic) [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "live synthetic ok" "${output}"; then + echo "ttimechart live UI test (synthetic) [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live UI test (synthetic) [Success]" +} + +test_live_record() { + echo "ttimechart live record test" + rm -f "${perfdata}.live" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${perfdata}.live" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if not app.recording: + break + await pilot.pause(0.1) + if app.recording or not app.data.has_events(): + raise RuntimeError("Live recording of workload did not complete") + print("live record ok") + finally: + rec.stop() + +asyncio.run(run()) +EOF + then + echo "ttimechart live record test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! perf script ttimechart -i "${perfdata}.live" --dump > "${output}" 2>&1 || + ! grep -q "^Duration: .*sched events: [1-9]" "${output}"; then + echo "ttimechart live record test [Failed: saved perf.data unreadable]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live record test [Success]" +} + test_plumbing test_no_events +test_live_synthetic if ! record; then echo "Skipping remaining tests, failed to record scheduler events (permissions?)"
@@ -232,6 +534,7 @@ fi test_dump test_headless_ui +test_live_record cleanup exit $err
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
[email protected]
AI reviewRe: [PATCH v3 2/2] perf test: Test perf timechart --live
Sashiko has reviewed this patch and found no issues.
Quoted 8 lines
From: Alice Rogers <[email protected]>
Extend
test_ttimechart_python.shto cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file, a synthetic live UI test exercising the rolling window, pruning, pause/resume and zoom following, and a live workload recording test when scheduler tracepoints are accessible.
Quoted 4 lines
Assisted-by Antigravity:gemini-3.1-proSigned-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerv4 cover perf timechart: Add --live mode to the TUI and test
In reply to Ian RogersAdd a '--live' option to 'perf timechart' and 'ttimechart.py' that streams live scheduler, power, or I/O (-I/--io-only) events from a 'perf record' subprocess over a pipe and displays them in the Textual TUI with a rolling time window (--window, default 10s), pause/resume via the control pipe, an optional workload after '--', and optional output recording (-o), along with headless shell tests in test_ttimechart_python.sh.
v4: Address Sashiko review feedback on patch 1:
- Don't prune live history past the last event, so open states aren't
lost when no events arrive for a window, and don't count a still
running task's open segment as a switch.
- Keep showing the window at the end of recording, rather than jumping back to already pruned history, and prune events flushed as perf
record exits.
- Close open task, CPU and I/O states when resuming after a pause,
rather than extending them across the unrecorded gap, and close the running task when a switch away from it was lost.
- Stop perf record and close the files and pipes if
LiveRecorder.start() fails after starting it, and close the data pipe read ends in LiveRecorder.stop().
v3: https://lore.kernel.org/linux-perf-users/[email protected]/
v3: Address Sashiko review feedback:
- Patch 1: Explicitly include <string.h> in builtin-timechart.c.
- Patch 1: Reject non-finite --window values (e.g. 'inf') that
overflowed converting to nanoseconds.
- Patch 2: Stop the fake recorder in a try/finally in the synthetic
live UI test so a failed check can't leave its background thread
running and hang the test.
Rebase onto the latest perf-tools-next.
v2: https://lore.kernel.org/linux-perf-users/[email protected]/
v2: Rebase the remaining 2 patches onto perf-tools-next after patches 1-11
of v1 were merged.
Alice Rogers (2):
perf timechart: Add a --live mode to the TUI
perf test: Test perf timechart --live
tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 81 +- tools/perf/python/ttimechart.py | 883 ++++++++++++++++-- .../tests/shell/test_ttimechart_python.sh | 303 ++++++ 4 files changed, 1212 insertions(+), 75 deletions(-)
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
Ian Rogers
reviewerPATCH v4 1/2 perf timechart: Add a --live mode to the TUI
From: Alice Rogers <[email protected]>
Add a --live option to perf timechart and ttimechart.py that runs 'perf record' writing CLOCK_MONOTONIC scheduler and power events, or I/O syscall events when -I/--io-only is given, to a pipe and displays them in the TUI as they happen. Only the most recent --window seconds of history (default 10) are kept in memory, pruning older segments and tasks that have no remaining history in the window.
Pressing space pauses and resumes recording using perf record's --control pipe. Because perf record only reads its ring buffers when a watermark is reached, periodic 'ping' commands are sent on the control pipe to flush events promptly. The refresh interval adapts to the measured screen refresh cost and backs off if unprocessed data builds up in the pipe, with two flushes per refresh to account for ordered-event buffering across rounds.
An optional workload may be given after '--' to record just that command rather than the whole system, and '-o' optionally saves a copy of the recorded perf.data stream.
--- tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 81 +- tools/perf/python/ttimechart.py | 883 ++++++++++++++++++-- 3 files changed, 909 insertions(+), 75 deletions(-)
tools/perf/Documentation/perf-timechart.txt +18 -2
@@ -47,6 +47,9 @@ TIMECHART OPTIONS -T:: --tasks-only:: Don't output processor state transitions +-I:: +--io-only:: + In '--live' or '--tui' mode, only record or show I/O activity -p:: --process:: Select the processes to display, by name or PID
@@ -59,8 +62,21 @@ TIMECHART OPTIONS linkperf:perf-script[1]). The script requires the perf python module and the python 'textual' library. The CPU, task, I/O and summary views can be zoomed ('+'/'-'), panned (shift+arrows) and the state of the - selected row at the cursor is described. The -i, -P, -T and -p options - are passed to the script, other output options are ignored. + selected row at the cursor is described. The -i, -P, -T, -I and -p + options are passed to the script, other output options are ignored. +--live:: + Record scheduler and power events, or I/O events with '-I', with + linkperf:perf-record[1] and display them as they happen in the + terminal timechart, implying --tui. Only the most recent '--window' + seconds of events are kept and shown. Pressing space pauses and + resumes recording; zooming or panning stops the view following the + latest events until '0' is pressed. If a command is given after '--' + then only that workload is recorded and recording stops when it exits, + otherwise the whole system is recorded until the UI is quit. If '-o' + is given the recorded perf data is also saved to that file. +--window=<seconds>:: + In '--live' mode, the number of seconds of recent events kept in + memory and shown in the timeline (default: 10). --symfs=<directory[,layout]>:: Look for files with symbols relative to this directory. The optional layout can be 'hierarchy' (default, matches full path) or 'flat'
tools/perf/builtin-timechart.c +71 -10
160 lines not on this page · raw message
tools/perf/python/ttimechart.py +820 -63
1,133 lines not on this page · raw message
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
[email protected]
AI reviewRe: [PATCH v4 1/2] perf timechart: Add a --live mode to the TUI
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Medium] perf timechart: missing -a in LiveRecorder.start() breaks workload timelines - [Medium] perf…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] perf timechart: missing -a in LiveRecorder.start() breaks workload timelines
- [Medium] perf timechart: backward prev.since reset in sched_switch() corrupts totals
- [Medium] perf timechart: unsynchronized concurrent teardown in LiveRecorder.stop() --
commit 154f81d15ad61319be3006e45c11ac5951330583
Author: Alice Rogers <[email protected]>
perf timechart: Add a --live mode to the TUI
This commit adds a --live option to perf timechart and ttimechart.py to record scheduler, power, or I/O events with perf record and display the most recent window of events interactively in the terminal UI.
Quoted 4 lines
tools/perf/python/ttimechart.py +0
-0
[ ... ]
Quoted 17 lines
tools/perf/python/ttimechart.py +6
-1@@ -363,9 +524,14 @@ class TimechartData: if c.cur_tid == -1 and prev_tid != 0: # Assume the task was running from the start of the trace. c.cur_tid = prev_tid - c.since = self.first_time + c.since = self.state_start() if c.cur_tid > 0: c.run.add(c.since, time, 1, c.cur_tid) + if c.cur_tid != prev_tid: + # The switch away from the task that was running was lost. + old = self.tasks.get(c.cur_tid) + if old and old.state == STATE_RUNNING and old.cpu == cpu: + old.change_state(time, STATE_UNKNOWN) c.cur_tid = next_tid c.since = time
tools/perf/python/ttimechart.py class TimechartData: · line 543
Quoted 6 lines
tools/perf/python/ttimechart.py +1
-1@@ -373,7 +539,7 @@ class TimechartData: prev = self.task(prev_tid, prev_comm) if prev.state == STATE_UNKNOWN: prev.state = STATE_RUNNING - prev.since = self.first_time + prev.since = self.state_start() prev.cpu = cpu
[Severity: Medium]
Can resetting prev.since to self.state_start() in TimechartData.sched_switch() move prev.since backwards before existing segments and corrupt Task.totals[STATE_RUNNING]?
When c.cur_tid != prev_tid above, a previously running task old is transitioned to STATE_UNKNOWN via old.change_state(time, STATE_UNKNOWN), which appends a segment ending at time (old.segs.ends[-1] == time) and sets old.since = time.
If that task's subsequent switch-in event is also missed and the task is next seen switching out as prev_tid at time2 > time while still in STATE_UNKNOWN, setting prev.since = self.state_start() moves prev.since backwards to self.state_start() < prev.segs.ends[-1].
When prev.change_state(time2, new_state) then runs:
if self.state in (STATE_RUNNING, STATE_WAITING, STATE_BLOCKED) and time > self.since:
self.segs.add(self.since, time, self.state, self.cpu) self.totals[self.state] += time - self.since
SegmentList.add() clips start to self.ends[-1] (time), so the added segment has length (time2 - time), but Task.change_state() adds the unclipped (time2 - self.state_start()) to self.totals[STATE_RUNNING]. When Task.prune() later discards that segment, it only subtracts (time2 - time), leaving self.totals[STATE_RUNNING] permanently inflated by (time - self.state_start()).
Should prev.since be set to max(prev.since, self.state_start()) here?
[ ... ]
Quoted 5 lines
tools/perf/python/ttimechart.py +1
-0@@ -1727,52 +2178,358 @@ class TimechartApp(App): self.goto_task(waker, wake_time) +class LiveRecorder:
[ ... ]
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ def start(self) -> None: + """Start perf record, the data is then read from data_fd.""" + output = open(self.output, "wb") if self.output else None + # perf record's ends of the control pipes, closed once it is started. + child_fds: List[int] = [] + try: + ctl_r, self.ctl_fd = os.pipe() + child_fds.append(ctl_r) + self.ack_fd, ack_w = os.pipe() + child_fds.append(ack_w)
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2268
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ # Quiet so that only errors are written to stderr, see error_text(). + cmd = [self.perf_exe, "record", "-q", "-o", "-", "-R", "-c", "1", "-k", "mono", + "--synth=task", "--control", f"fd:{ctl_r},{ack_w}"] + filt = f"common_pid != {os.getpid()}" + for name in self.events(): + cmd += ["-e", name] + if name.startswith("syscalls:"): + cmd += ["--filter", filt] + # Without a workload record the whole system until stopped. + cmd += ["--", *self.workload] if self.workload else ["-a"]
[Severity: Medium]
Does omitting -a in LiveRecorder.start() when self.workload is specified break task, CPU, and power timelines for the recorded workload?
Unlike timechart__record() and timechart__io_record() in tools/perf/builtin-timechart.c, which always pass -a, LiveRecorder.start() omits -a when a workload is given. That causes perf record to open sched:sched_switch, sched:sched_wakeup, sched:sched_wakeup_new, power:cpu_idle, and power:cpu_frequency as per-task events attached only to the workload tasks.
In the kernel's __schedule() in kernel/sched/core.c:
trace_sched_switch(preempt, prev, next, prev_state);
/* Also unlocks the rq: */
rq = context_switch(rq, prev, next, &rf);trace_sched_switch() runs before context_switch() activates next's per-task perf_event_context, and TRACE_EVENT(sched_switch) does not target next via __perf_task(). As a result, per-task sched:sched_switch events only fire when a workload task switches out as prev, and never when an untraced task (such as swapper/idle or another process) switches into the workload task as next.
In TimechartData.sched_switch(), when a workload task wakes up via sched:sched_wakeup (which does use __perf_task(p)) or is preempted, its state becomes STATE_WAITING. Because the subsequent switch-in from idle or an untraced task is never recorded, task.state remains STATE_WAITING across the entire CPU execution interval until the workload switches out, recording its running time as STATE_WAITING instead of STATE_RUNNING and leaving c.cur_tid on the Cpu object at 0 or at the previous untraced next_tid.
In addition, trace_cpu_idle() only runs in the idle task (PID 0), so per-task recording on a workload captures no power:cpu_idle events, and running perf timechart --live -P -- <workload> exits with "Error: no scheduler, power or I/O events were recorded." if no CPU frequency change occurs within the workload's own task context.
Should -a always be passed to perf record even when self.workload is present?
[ ... ]
Quoted 28 lines
tools/perf/python/ttimechart.py +28
-0+ def stop(self) -> None: + """Stop recording, perf record then writes any remaining data and exits.""" + self.stopping.set() + self.commands.put("") + proc = self.proc + if proc is not None and proc.poll() is None: + try: + proc.send_signal(signal.SIGINT) + proc.wait(timeout=1) + except subprocess.TimeoutExpired: + # Probably blocked writing data that is no longer processed. + proc.kill() + proc.wait() + if self.tee_thread and self.tee_thread is not threading.current_thread(): + self.tee_thread.join(timeout=1) + if self.control_thread and self.control_thread is not threading.current_thread(): + self.control_thread.join(timeout=1) + self.close_control_fds() + self.close_data_fds() + + def close_data_fds(self) -> None: + """Close the read ends of the data pipes. + + perf.data() duplicates the file descriptor it reads from, so they + aren't needed once processing has started. Closing them means writes + to them fail, rather than block, once they are no longer read. + """ + self.data_fd = -1
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2427
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ tee_r, self.tee_r = self.tee_r, -1 + if tee_r >= 0: + try: + os.close(tee_r) + except OSError: + pass + # Unless the tee thread is still reading from it. + if self.proc and self.proc.stdout and not (self.tee_thread and + self.tee_thread.is_alive()): + self.proc.stdout.close()
[Severity: Medium]
Can concurrent calls to LiveRecorder.stop() from the UI thread and worker thread race during teardown?
When quitting during live recording, TimechartApp.action_quit() (or TimechartApp.on_unmount()) calls TimechartApp.cancel_loading() on the UI thread:
loading = self.loading
if loading is not None:
loading.cancelled = True
if self.live:
self.live.stop()
At the same time, the background worker in TimechartApp.record_events() exits read_events() (either via LoadCancelled or when perf record closes the pipe) and runs its finally block:
finally:
self.loading = None
live.stop()
In LiveRecorder.stop(), both threads can concurrently inspect proc.poll() and call proc.send_signal(), proc.wait(), and proc.kill() without synchronization. If one thread reaps proc in wait() while the other calls send_signal() or kill(), an uncaught ProcessLookupError can be raised. Similarly, in LiveRecorder.close_data_fds(), self.tee_r and self.proc.stdout.close() are accessed without self.fd_lock (unlike LiveRecorder.close_control_fds()), allowing both threads to race on closing the pipe file descriptors and buffered stdout stream.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v4 2/2 perf test: Test perf timechart --live
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Extend test_ttimechart_python.sh to cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file, a synthetic live UI test exercising the rolling window, pruning, pause/resume and zoom following, and a live workload recording test when scheduler tracepoints are accessible.
--- .../tests/shell/test_ttimechart_python.sh | 303 ++++++++++++++++++ 1 file changed, 303 insertions(+)
tools/perf/tests/shell/test_ttimechart_python.sh +303 -0
@@ -82,6 +82,34 @@ test_plumbing() { err=1 return fi + if perf timechart --live -i "${perfdata}" > "${output}" 2>&1 || + ! grep -q -- "--live can't be used with -i" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --live with -i not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --live --window 0 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "--window must be a positive" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window 0 not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --tui --window 5 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "require --live" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window without --live]" + cat "${output}" + err=1 + return + fi + if perf timechart --live -I -P < /dev/null > "${output}" 2>&1 || + ! grep -q -- "-I cannot be used with -P or -T" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: -I with -P not rejected]" + cat "${output}" + err=1 + return + fi echo "perf timechart --tui plumbing test [Success]" }
@@ -115,6 +143,43 @@ test_no_events() { err=1 return fi + # Exercise LiveRecorder's perf record subprocess, control pipe and + # -o output file using task-clock when tracepoints may be unavailable. + rm -f "${clockdata}" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${clockdata}" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import os +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.events = lambda: ["task-clock"] + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + await pilot.press("space") + await pilot.pause(0.1) + await pilot.press("space") + for _ in range(300): + if app.return_code is not None: + break + await pilot.pause(0.05) + finally: + rec.stop() + if app.return_code != 1 or not app.data.nr_samples or not os.path.getsize(sys.argv[2]): + raise RuntimeError(f"Unexpected result: rc={app.return_code} samples={app.data.nr_samples}") + +asyncio.run(run()) +EOF + then + echo "ttimechart no events test [Failed: LiveRecorder task-clock test]" + cat "${output}" + err=1 + return + fi echo "ttimechart no events test [Success]" }
@@ -218,8 +283,245 @@ EOF echo "ttimechart headless UI test [Success]" } +test_live_synthetic() { + echo "ttimechart live UI test (synthetic)" + # Exercise live mode's rolling window, pruning, pause/resume and + # following with a fake recorder, which doesn't need tracepoint + # permissions. + if ! "$PYTHON" - > "${output}" 2>&1 <<'EOF' +import asyncio +import threading +import time +from time import monotonic_ns +import ttimechart as tc + +SEC = 1_000_000_000 + +class Ev: + def __init__(self, name): + self.name = name + def __str__(self): + return f"evsel({self.name})" + +class Sample: + def __init__(self, name, t, cpu, tid, **kw): + self.evsel = Ev(name) + self.sample_time = t + self.sample_cpu = cpu + self.sample_tid = tid + self.sample_pid = tid + self.sample_id = hash(name) + for k, v in kw.items(): + setattr(self, k, v) + +class FakeRecorder: + def __init__(self): + self.origin = monotonic_ns() + self.data_fd = -1 + self.flush_interval = 0.05 + self.paused = False + self.pause_calls = [] + self.stopped = threading.Event() + self.finish = False + + def backlog(self): + return 0.0 + + def set_paused(self, paused): + self.paused = paused + self.pause_calls.append(paused) + + def stop(self): + self.stopped.set() + + def error_text(self): + return "" + +async def run() -> None: + rec = FakeRecorder() + + def fake_read(data, _fd): + t0 = rec.origin + # Task 300 only runs at the start so pruning should drop it once the + # 1s window advances past t0 + 3000. + data.process_event(Sample("sched:sched_wakeup", t0 + 1000, 0, 1, + pid=300, comm="early", common_pid=1, common_flags=0)) + data.process_event(Sample("sched:sched_switch", t0 + 2000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=300, next_comm="early")) + data.process_event(Sample("sched:sched_switch", t0 + 3000, 0, 300, + prev_pid=300, prev_comm="early", prev_state=1, + next_pid=0, next_comm="swapper")) + cur = 0 + while not rec.stopped.is_set() and not rec.finish: + now = monotonic_ns() + if rec.paused: + time.sleep(0.01) + continue + nxt = 100 if cur == 0 else 0 + comm = "worker" if nxt == 100 else "swapper" + prev_comm = "worker" if cur == 100 else "swapper" + if nxt: + data.process_event(Sample("sched:sched_wakeup", now, 0, cur or 1, + pid=nxt, comm=comm, common_pid=cur or 1, + common_flags=0)) + data.process_event(Sample("sched:sched_switch", now, 0, cur, + prev_pid=cur, prev_comm=prev_comm, prev_state=1, + next_pid=nxt, next_comm=comm)) + cur = nxt + time.sleep(0.005) + + tc.read_events = fake_read + app = tc.TimechartApp("live", False, False, [], live=rec, window=1.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(300): + with app.data.lock: + if app.data.pruned_to > rec.origin + 3000: + break + await pilot.pause(0.05) + with app.data.lock: + if app.data.pruned_to <= rec.origin + 3000 or 300 in app.data.tasks: + raise RuntimeError("Task outside the window was not pruned") + starts = [t.segs.starts[0] for t in app.data.tasks.values() if len(t.segs)] + if not starts or min(starts) < app.data.pruned_to: + raise RuntimeError(f"Unpruned segments: {starts} < {app.data.pruned_to}") + if app.window.last - app.window.first != SEC: + raise RuntimeError(f"Unexpected window span: {app.window}") + + await pilot.press("space") + await pilot.pause(0.2) + if rec.pause_calls != [True] or "Paused" not in app.sub_title: + raise RuntimeError(f"Did not pause: {rec.pause_calls} {app.sub_title}") + frozen = app.window.last + await pilot.pause(0.4) + if app.window.last != frozen: + raise RuntimeError("Window moved while paused") + await pilot.press("space") + await pilot.pause(0.3) + if rec.pause_calls != [True, False] or not app.following: + raise RuntimeError("Did not resume") + + await pilot.press("plus") + await pilot.pause(0.2) + if app.following: + raise RuntimeError("Zooming did not stop following") + await pilot.press("0") + await pilot.pause(0.2) + if not app.following: + raise RuntimeError("Resetting zoom did not resume following") + + rec.finish = True + for _ in range(100): + if not app.recording: + break + await pilot.pause(0.05) + if app.recording or "Finished" not in app.sub_title: + raise RuntimeError(f"Did not finish: {app.sub_title}") + finally: + rec.stop() + + # Also test live I/O-only mode (-I). + io_events = tc.LiveRecorder("perf", False, False, [], None, io_only=True).events() + if not io_events or not all(e.startswith("syscalls:sys_") for e in io_events): + raise RuntimeError(f"Unexpected I/O events: {io_events}") + io_rec = FakeRecorder() + + def fake_io_read(data, _fd): + now = monotonic_ns() + # Self I/O should be ignored in live mode. + own = tc.os.getpid() + data.process_event(Sample("syscalls:sys_enter_read", now, 0, own, fd=3)) + data.process_event(Sample("syscalls:sys_exit_read", now + 1000, 0, own, ret=64)) + # Workload I/O should be recorded. + data.process_event(Sample("syscalls:sys_enter_read", now + 2000, 0, 4242, fd=4)) + data.process_event(Sample("syscalls:sys_exit_read", now + 5000, 0, 4242, ret=4096)) + data.process_event(Sample("syscalls:sys_enter_write", now + 6000, 0, 4242, fd=5)) + data.process_event(Sample("syscalls:sys_exit_write", now + 9000, 0, 4242, ret=2048)) + + tc.read_events = fake_io_read + io_app = tc.TimechartApp("live", False, False, [], live=io_rec, window=1.0, io_only=True) + async with io_app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + if not io_app.recording: + break + await pilot.pause(0.05) + await pilot.pause() + tabs = io_app.query_one(tc.TabbedContent) + if tabs.active != "io" or not io_app.io_tasks or io_app.data.io_events != 4: + raise RuntimeError(f"Unexpected I/O state: active={tabs.active} " + f"tasks={len(io_app.io_tasks)} events={io_app.data.io_events}") + tabs.active = "summary" + await pilot.pause() + await pilot.press("enter") + await pilot.pause() + if tabs.active != "io": + raise RuntimeError(f"Summary enter did not activate I/O tab: {tabs.active}") + print("live synthetic ok") + +asyncio.run(run()) +EOF + then + echo "ttimechart live UI test (synthetic) [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "live synthetic ok" "${output}"; then + echo "ttimechart live UI test (synthetic) [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live UI test (synthetic) [Success]" +} + +test_live_record() { + echo "ttimechart live record test" + rm -f "${perfdata}.live" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${perfdata}.live" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if not app.recording: + break + await pilot.pause(0.1) + if app.recording or not app.data.has_events(): + raise RuntimeError("Live recording of workload did not complete") + print("live record ok") + finally: + rec.stop() + +asyncio.run(run()) +EOF + then + echo "ttimechart live record test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! perf script ttimechart -i "${perfdata}.live" --dump > "${output}" 2>&1 || + ! grep -q "^Duration: .*sched events: [1-9]" "${output}"; then + echo "ttimechart live record test [Failed: saved perf.data unreadable]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live record test [Success]" +} + test_plumbing test_no_events +test_live_synthetic if ! record; then echo "Skipping remaining tests, failed to record scheduler events (permissions?)"
@@ -232,6 +534,7 @@ fi test_dump test_headless_ui +test_live_record cleanup exit $err
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
[email protected]
AI reviewRe: [PATCH v4 2/2] perf test: Test perf timechart --live
Sashiko has reviewed this patch and found no issues.
Quoted 11 lines
From: Alice Rogers <[email protected]>
Extend
test_ttimechart_python.shto cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file, a synthetic live UI test exercising the rolling window, pruning, pause/resume and zoom following, and a live workload recording test when scheduler tracepoints are accessible.Signed-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerv5 cover perf timechart: Add --live mode to the TUI and test
In reply to Ian RogersAdd a '--live' option to 'perf timechart' and 'ttimechart.py' that streams live scheduler, power, or I/O (-I/--io-only) events from a 'perf record' subprocess over a pipe and displays them in the Textual TUI with a rolling time window (--window, default 10s), pause/resume via the control pipe, an optional workload after '--' that stops recording when it exits, and optional output recording (-o), along with headless shell tests in test_ttimechart_python.sh.
v5: Address Sashiko review feedback:
- Patch 1: Always record system wide, as 'perf timechart record'
does, as switches into a task and idle states aren't seen by
per-task events when only a workload was recorded.
- Patch 1: Don't move a task's state start back before its recorded history when its switch out and back in were lost, which inflated
its running time.
- Patch 1: Serialize LiveRecorder.stop() calls from the UI thread and
the thread processing events.
- Patch 2: Only run the live recording test when scheduler events can
be recorded system wide.
v4: https://lore.kernel.org/linux-perf-users/[email protected]/
v4: Address Sashiko review feedback on patch 1:
- Don't prune live history past the last event, so open states aren't
lost when no events arrive for a window, and don't count a still
running task's open segment as a switch.
- Keep showing the window at the end of recording, rather than jumping back to already pruned history, and prune events flushed as perf
record exits.
- Close open task, CPU and I/O states when resuming after a pause,
rather than extending them across the unrecorded gap, and close the running task when a switch away from it was lost.
- Stop perf record and close the files and pipes if
LiveRecorder.start() fails after starting it, and close the data pipe read ends in LiveRecorder.stop().
v3: https://lore.kernel.org/linux-perf-users/[email protected]/
v3: Address Sashiko review feedback:
- Patch 1: Explicitly include <string.h> in builtin-timechart.c.
- Patch 1: Reject non-finite --window values (e.g. 'inf') that
overflowed converting to nanoseconds.
- Patch 2: Stop the fake recorder in a try/finally in the synthetic
live UI test so a failed check can't leave its background thread
running and hang the test.
Rebase onto the latest perf-tools-next.
v2: https://lore.kernel.org/linux-perf-users/[email protected]/
v2: Rebase the remaining 2 patches onto perf-tools-next after patches 1-11
of v1 were merged.
Alice Rogers (2):
perf timechart: Add a --live mode to the TUI
perf test: Test perf timechart --live
tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 81 +- tools/perf/python/ttimechart.py | 901 ++++++++++++++++-- .../tests/shell/test_ttimechart_python.sh | 310 ++++++ 4 files changed, 1237 insertions(+), 75 deletions(-)
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
Ian Rogers
reviewerPATCH v5 1/2 perf timechart: Add a --live mode to the TUI
From: Alice Rogers <[email protected]>
Add a --live option to perf timechart and ttimechart.py that runs 'perf record' writing CLOCK_MONOTONIC scheduler and power events, or I/O syscall events when -I/--io-only is given, to a pipe and displays them in the TUI as they happen. Only the most recent --window seconds of history (default 10) are kept in memory, pruning older segments and tasks that have no remaining history in the window.
Pressing space pauses and resumes recording using perf record's --control pipe. Because perf record only reads its ring buffers when a watermark is reached, periodic 'ping' commands are sent on the control pipe to flush events promptly. The refresh interval adapts to the measured screen refresh cost and backs off if unprocessed data builds up in the pipe, with two flushes per refresh to account for ordered-event buffering across rounds.
Like 'perf timechart record', the whole system is recorded, as switches into a task and idle states aren't seen by per-task events. An optional workload may be given after '--' to stop recording when it exits, and '-o' optionally saves a copy of the recorded perf.data stream.
--- tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 81 +- tools/perf/python/ttimechart.py | 901 ++++++++++++++++++-- 3 files changed, 927 insertions(+), 75 deletions(-)
tools/perf/Documentation/perf-timechart.txt +18 -2
@@ -47,6 +47,9 @@ TIMECHART OPTIONS -T:: --tasks-only:: Don't output processor state transitions +-I:: +--io-only:: + In '--live' or '--tui' mode, only record or show I/O activity -p:: --process:: Select the processes to display, by name or PID
@@ -59,8 +62,21 @@ TIMECHART OPTIONS linkperf:perf-script[1]). The script requires the perf python module and the python 'textual' library. The CPU, task, I/O and summary views can be zoomed ('+'/'-'), panned (shift+arrows) and the state of the - selected row at the cursor is described. The -i, -P, -T and -p options - are passed to the script, other output options are ignored. + selected row at the cursor is described. The -i, -P, -T, -I and -p + options are passed to the script, other output options are ignored. +--live:: + Record scheduler and power events, or I/O events with '-I', with + linkperf:perf-record[1] and display them as they happen in the + terminal timechart, implying --tui. Only the most recent '--window' + seconds of events are kept and shown. Pressing space pauses and + resumes recording; zooming or panning stops the view following the + latest events until '0' is pressed. The whole system is recorded, if a + command is given after '--' then recording stops when it exits, + otherwise it continues until the UI is quit. If '-o' + is given the recorded perf data is also saved to that file. +--window=<seconds>:: + In '--live' mode, the number of seconds of recent events kept in + memory and shown in the timeline (default: 10). --symfs=<directory[,layout]>:: Look for files with symbols relative to this directory. The optional layout can be 'hierarchy' (default, matches full path) or 'flat'
tools/perf/builtin-timechart.c +71 -10
160 lines not on this page · raw message
tools/perf/python/ttimechart.py +838 -63
1,151 lines not on this page · raw message
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
[email protected]
AI reviewRe: [PATCH v5 1/2] perf timechart: Add a --live mode to the TUI
Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider: - [Medium] perf timechart: broken live cursor following on zoom, reset, and goto_task() - [Medium] perf…
Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
- [Medium] perf timechart: broken live cursor following on zoom, reset, and goto_task()
- [Medium] perf timechart: mark_gap() overwrites pending gap_at on rapid pause/resume
- [Medium] perf timechart: unlocked nr_samples increment races with live_refresh()
- [Medium] perf timechart: missing finished_round leaves live events buffered
- [Medium] perf timechart: incomplete syscall filter in LiveRecorder.start()
- [Medium] perf timechart: live mode stop() blocks perf record and orphans workload --
commit dc903a5155a83951be9359f3695ae64ac3fc69e5
Author: Alice Rogers <[email protected]>
perf timechart: Add a --live mode to the TUI
This commit adds a --live mode to perf timechart and ttimechart.py that runs perf record into a pipe and displays a rolling window of scheduler, power, or I/O events interactively in the terminal UI. It also supports pausing and resuming recording via perf record's control pipe and adaptive refresh intervals.
Quoted 4 lines
tools/perf/python/ttimechart.py +0
-0
[ ... ]
Quoted 1 line
tools/perf/python/ttimechart.py +0
-0@@ -326,21 +425,83 @@ class TimechartData:
[ ... ]
Quoted 27 lines
tools/perf/python/ttimechart.py +27
-0+ def mark_gap(self, time: int) -> None: + """Note that recording, paused for a while, resumes at time, the lock must be held. + + Events weren't recorded while paused, so the states at the pause aren't + known to have continued. They are closed before the first event after + time is processed, the events are processed in time order. + """ + self.gap_at = time + + def _close_gap(self) -> None: + """Close open states at the last event before the gap, see mark_gap().""" + end = self.last_time + for task in self.tasks.values(): + task.change_state(end, STATE_UNKNOWN) + task.io_pending = None + for c in self.cpus.values(): + if c.cur_tid > 0: + c.run.add(c.since, end, 1, c.cur_tid) + c.cur_tid = -1 + if c.cstate_cur: + c.cstate.add(c.cstate_cur[0], end, 1, c.cstate_cur[1]) + c.cstate_cur = None + if c.pstate_cur: + c.pstate.add(c.pstate_cur[0], end, 1, c.pstate_cur[1]) + c.pstate_cur = None + self.resumed_at = self.gap_at + self.gap_at = 0
[ ... ]
tools/perf/python/ttimechart.py class TimechartData: · line 708
Quoted 9 lines
tools/perf/python/ttimechart.py +2
-0@@ -527,6 +701,8 @@ class TimechartData: def _process_event(self, sample: perf.sample_event) -> None: """Update the data from a sample, the lock must be held.""" time = sample.sample_time + if self.gap_at and time >= self.gap_at: + self._close_gap() if not self.first_time or time < self.first_time: self.first_time = time self.last_time = max(self.last_time, time)
[Severity: Medium]
What happens in TimechartData.mark_gap() and TimechartData._process_event() if the user pauses and resumes live recording again before an event from the first resumed interval has been processed?
In TimechartApp.action_toggle_pause(), resuming from pause calls self.data.mark_gap(monotonic_ns()), which sets self.gap_at = time. Open task and CPU states from before the pause are only closed when TimechartData._process_event() sees a sample with time >= self.gap_at and calls self._close_gap().
Because perf record flushes periodically on ping and perf.session buffers events in ordered_events until a subsequent round marker, a user can resume at T2, pause at T3, and resume at T4 before any sample with time >= T2 reaches TimechartData._process_event(). When mark_gap(T4) overwrites self.gap_at = T4, buffered samples from [T2, T3) have sample_time < T4 and bypass _close_gap(), stretching pre-pause task and CPU states across the first pause gap.
Should mark_gap() avoid overwriting a smaller non-zero self.gap_at, or close the pending gap immediately before recording the new timestamp?
[ ... ]
Quoted 1 line
tools/perf/python/ttimechart.py +0
-0@@ -1486,6 +1726,204 @@ class TimechartApp(App):
[ ... ]
Quoted 6 lines
tools/perf/python/ttimechart.py +6
-0+ def live_refresh(self) -> None: + """Show newly recorded events and discard those no longer in the window.""" + start = monotonic() + with self.data.lock: + samples = self.data.nr_samples + # When paused, once the buffered events are shown there's nothing new.
tools/perf/python/ttimechart.py class TimechartApp(App): · line 1828
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ if not self.paused_at or samples != self.refreshed_samples: + self.refreshed_samples = samples + with self.data.lock: + self.data.checkpoint() + if self.recording and not self.paused_at: + self.live_prune() + self.update_views() + self.update_live_status() + # Measure the cost of the refresh including drawing the screen. + self.call_after_refresh(self.live_refreshed, start)
[Severity: Medium]
Can TimechartApp.live_refresh() race with TimechartData.process_event() and miss rendering the final buffered event when paused?
In TimechartData.process_event(), the worker thread increments self.nr_samples before acquiring self.lock to call _process_event(sample):
def process_event(self, sample: perf.sample_event) -> None:
"""Callback from perf.session for each sample."""
self.nr_samples += 1
...
with self.lock:
self._process_event(sample)Meanwhile, TimechartApp.live_refresh() reads self.data.nr_samples under self.data.lock, releases the lock, updates self.refreshed_samples = samples, and then re-acquires self.data.lock to call self.data.checkpoint() and self.update_views().
If live recording is paused (self.paused_at != 0) and live_refresh() reads the incremented self.data.nr_samples before the worker thread acquires self.lock and finishes _process_event(sample), live_refresh() records self.refreshed_samples = samples and renders the views before the sample is added to self.data. Subsequent live_refresh() ticks while paused then see samples == self.refreshed_samples and skip refreshing.
Should self.nr_samples be incremented inside with self.lock: in TimechartData.process_event(), and read inside the same lock acquisition in TimechartApp.live_refresh()?
[ ... ]
Quoted 19 lines
tools/perf/python/ttimechart.py +19
-0+ def user_window(self, window: TimeWindow) -> None: + """Change the window for the user. + + In live mode, zooming or panning so that the whole window isn't shown + stops the view following the latest events. + """ + if self.live: + whole = window.start == window.first and window.end == window.last + if window.cursor != self.window.cursor: + self.follow_cursor = False + elif whole and not self.following: + self.follow_cursor = True + if self.data.last_time: + window = replace(window, cursor=min(max(self.data.last_time, window.first), + window.last)) + if self.following != whole: + self.following = whole + self.update_live_status() + self.window = window
[ ... ]
Quoted 1 line
tools/perf/python/ttimechart.py +0
-0@@ -1569,14 +2012,23 @@ class TimechartApp(App):
[ ... ]
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2032
Quoted 11 lines
tools/perf/python/ttimechart.py +9
-1- if first or last: + if self.live: + live_first, live_last = self.live_bounds() + win = self.window.extend(live_first, live_last) + if self.following: + win = win.reset() + if self.follow_cursor and last: + win = replace(win, cursor=min(max(last, live_first), live_last)) + self.window = win + elif first or last: self.window = self.window.extend(first, last)
[Severity: Medium]
Does cursor following in TimechartApp.user_window(), TimechartApp.update_views(), and TimechartApp.goto_task() stay in sync when zooming, resetting zoom, or jumping to a waker in live mode?
There appear to be three related cases where self.follow_cursor and self.following diverge:
1. When zooming in or out via TimelineView.action_zoom_in() or action_zoom_out(), TimeWindow.zoom() keeps window.cursor == self.window.cursor whenever the cursor is within [self.start, self.end]. In TimechartApp.user_window(), whole becomes False and self.following is set to False, but self.follow_cursor = False is only run if window.cursor != self.window.cursor. Because self.follow_cursor remains True, every subsequent live_refresh() -> update_views() overwrites win.cursor with last, dragging the cursor out of the zoomed view.
2. If the user moves the cursor without zooming or panning, user_window() sets self.follow_cursor = False while whole and self.following remain True. Pressing 0 (TimelineView.action_zoom_reset()) then calls user_window(self.window.reset()), where elif whole and not self.following:
is skipped because self.following is already True, leaving self.follow_cursor stuck at False.
3. Pressing w (TimechartApp.action_goto_waker() -> goto_task()) assigns self.window = self.window.with_cursor(time) directly instead of calling self.user_window(), leaving self.follow_cursor (and self.following) True so the next refresh snaps the cursor and view back to the latest event.
[ ... ]
Quoted 1 line
tools/perf/python/ttimechart.py +0
-0@@ -1727,52 +2180,374 @@ class TimechartApp(App):
[ ... ]
Quoted 8 lines
tools/perf/python/ttimechart.py +8
-0+ def start(self) -> None: + """Start perf record, the data is then read from data_fd.""" + output = open(self.output, "wb") if self.output else None + # perf record's ends of the control pipes, closed once it is started. + child_fds: List[int] = [] + try: + ctl_r, self.ctl_fd = os.pipe() + child_fds.append(ctl_r)
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2270
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ self.ack_fd, ack_w = os.pipe() + child_fds.append(ack_w) + # Quiet so that only errors are written to stderr, see error_text(). + cmd = [self.perf_exe, "record", "-q", "-o", "-", "-R", "-c", "1", "-k", "mono", + "--synth=task", "--control", f"fd:{ctl_r},{ack_w}"] + filt = f"common_pid != {os.getpid()}" + for name in self.events(): + cmd += ["-e", name] + if name.startswith("syscalls:"): + cmd += ["--filter", filt]
[Severity: Medium]
Does the tracepoint filter common_pid != os.getpid() in LiveRecorder.start() exclude I/O syscalls made by the child perf record process and ttimechart.py's worker threads when running with --live -I?
In kernel tracepoints, common_pid is current->pid (the thread ID, not the process TGID). Unlike builtin-timechart.c:timechart__io_record(), which runs cmd_record() in-process on a single thread, LiveRecorder.start() spawns perf record as a child process and runs control_thread, tee_thread, record_events, and Textual workers on separate Python threads whose TIDs differ from os.getpid().
While TimechartApp.__init__() adds os.getpid() and proc.pid to self.data.ignore_pids to hide them from live TUI rows, all of their read/write/poll/select syscalls are still recorded by the kernel. Those events increment self.nr_samples in TimechartData.process_event() before io_enter() and io_exit() check ignore_pids, and tee_thread writes them into any file saved with -o, polluting the saved perf.data when later viewed with perf timechart or perf timechart --tui.
[ ... ]
Quoted 9 lines
tools/perf/python/ttimechart.py +9
-0+ def control_loop(self) -> None: + """Send queued commands, otherwise pings that flush perf record's buffers.""" + try: + while not self.stopping.is_set(): + try: + cmd = self.commands.get(timeout=self.flush_interval) + except queue.Empty: + if self.paused: + continue
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2386
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ cmd = "ping" + if not cmd or self.stopping.is_set() or not self.send(cmd): + break + finally: + self.close_control_fds() + + def set_paused(self, paused: bool) -> None: + """Pause or resume recording.""" + self.paused = paused + self.commands.put("disable" if paused else "enable")
[Severity: Medium]
When pausing recording or running in --power-only mode, can the latest round of events remain buffered in perf.session's ordered_events queue without being delivered to TimechartData.process_event()?
In tools/perf/util/ordered-events.c:__ordered_events__flush(), handling OE_FLUSH__ROUND (on PERF_RECORD_FINISHED_ROUND) calls do_flush() up to the previous oe->next_flush before advancing oe->next_flush = oe->max_timestamp:
err = do_flush(oe, show_progress);
if (!err) {
if (how == OE_FLUSH__ROUND)
oe->next_flush = oe->max_timestamp;
oe->last_flush_type = how;
}TimechartApp.tune_refresh() sets self.live.flush_interval = self.refresh_interval / 2 so that two pings occur per refresh. However, in tools/perf/builtin-record.c:record__mmap_read_evlist(), perf record only writes finished_round_event when new bytes were actually read from the ring buffers during that pass:
if (!record__threads_enabled(rec) && bytes_written != rec->bytes_written)
rc = record__write(rec, NULL, &finished_round_event, sizeof(finished_round_event));When LiveRecorder.set_paused(True) queues "disable" and stops sending "ping" in LiveRecorder.control_loop(), perf record disables the evlist, drains the final pre-pause events in one pass, and writes only one PERF_RECORD_FINISHED_ROUND (which flushes the prior round and sets oe->next_flush). The next pass reads 0 bytes from the disabled ring buffers and emits no second PERF_RECORD_FINISHED_ROUND, leaving the final round of pre-pause events buffered in ordered_events while paused. The same happens in --power-only mode when a subsequent ping pass sees no new power events.
[ ... ]
Quoted 37 lines
tools/perf/python/ttimechart.py +37
-0+ def stop(self) -> None: + """Stop recording, perf record then writes any remaining data and exits. + + Called from both the UI thread and the thread processing the events, + so they don't both signal, reap and close perf record's pipes at once. + """ + self.stopping.set() + self.commands.put("") + with self.stop_lock: + self._stop() + + def _stop(self) -> None: + """Stop perf record and the threads, stop_lock must be held.""" + proc = self.proc + if proc is not None and proc.poll() is None: + try: + proc.send_signal(signal.SIGINT) + proc.wait(timeout=1) + except subprocess.TimeoutExpired: + # Probably blocked writing data that is no longer processed. + proc.kill() + proc.wait() + if self.tee_thread and self.tee_thread is not threading.current_thread(): + self.tee_thread.join(timeout=1) + if self.control_thread and self.control_thread is not threading.current_thread(): + self.control_thread.join(timeout=1) + self.close_control_fds() + self.close_data_fds() + + def close_data_fds(self) -> None: + """Close the read ends of the data pipes, stop_lock must be held. + + perf.data() duplicates the file descriptor it reads from, so they + aren't needed once processing has started. Closing them means writes + to them fail, rather than block, once they are no longer read. + """ + self.data_fd = -1
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2444
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ tee_r, self.tee_r = self.tee_r, -1 + if tee_r >= 0: + try: + os.close(tee_r) + except OSError: + pass + # Unless the tee thread is still reading from it. + if self.proc and self.proc.stdout and not (self.tee_thread and + self.tee_thread.is_alive()): + self.proc.stdout.close()
[Severity: Medium]
When quitting live mode, can TimechartApp.cancel_loading() and LiveRecorder._stop() cause perf record to block on pipe writes, get killed with SIGKILL, and leave the workload child process running as an orphan?
When quitting, TimechartApp.cancel_loading() sets loading.cancelled = True before calling self.live.stop():
loading = self.loading
if loading is not None:
loading.cancelled = True
if self.live:
self.live.stop()
On the worker thread, TimechartData.process_event() raises LoadCancelled at the next PROGRESS_INTERVAL (1000 samples), causing read_events() to exit and close perf.data's duplicated pipe fd while LiveRecorder._stop() sends SIGINT to perf record and calls proc.wait(timeout=1). However, self.close_data_fds() is not called until the end of _stop(), so self.proc.stdout remains open while nobody is reading from it.
In tools/perf/builtin-record.c:__cmd_record(), perf record flushes all ring buffers to stdout before sending SIGTERM to its workload child:
record__mmap_read_all(rec, true);
goto out_free_threads;
...
if (forks) {
int exit_status;
if (!child_finished)
kill(evlist__workload_pid(rec->evlist), SIGTERM);Once the pipe fills, perf record blocks in write(), hits subprocess.TimeoutExpired after 1 second, and proc.kill() sends SIGKILL only to proc.pid (even though perf record was started with start_new_session=True), leaving any workload started after -- running in the background. (Even if close_data_fds() were called before proc.wait(), an early EPIPE/SIGPIPE during record__mmap_read_all() would still terminate perf record before it reaches the kill(..., SIGTERM) call.)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v5 2/2 perf test: Test perf timechart --live
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Extend test_ttimechart_python.sh to cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file, a synthetic live UI test exercising the rolling window, pruning, pause/resume and zoom following, and a live workload recording test when scheduler events can be recorded system wide.
--- .../tests/shell/test_ttimechart_python.sh | 310 ++++++++++++++++++ 1 file changed, 310 insertions(+)
tools/perf/tests/shell/test_ttimechart_python.sh +310 -0
@@ -24,6 +24,7 @@ if ! "$PYTHON" -c 'import textual' > /dev/null 2>&1; then fi err=0 +system_wide=0 perfdata= output= clockdata=
@@ -57,6 +58,7 @@ record() { # fall back to recording just the workload's scheduler events. if perf timechart record -o "${perfdata}" -- \ bash -c "$(declare -f workload); workload" > /dev/null 2>&1; then + system_wide=1 return 0 fi if perf record -R -c 1 -e sched:sched_switch -e sched:sched_wakeup \
@@ -82,6 +84,34 @@ test_plumbing() { err=1 return fi + if perf timechart --live -i "${perfdata}" > "${output}" 2>&1 || + ! grep -q -- "--live can't be used with -i" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --live with -i not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --live --window 0 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "--window must be a positive" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window 0 not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --tui --window 5 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "require --live" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window without --live]" + cat "${output}" + err=1 + return + fi + if perf timechart --live -I -P < /dev/null > "${output}" 2>&1 || + ! grep -q -- "-I cannot be used with -P or -T" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: -I with -P not rejected]" + cat "${output}" + err=1 + return + fi echo "perf timechart --tui plumbing test [Success]" }
@@ -115,6 +145,43 @@ test_no_events() { err=1 return fi + # Exercise LiveRecorder's perf record subprocess, control pipe and + # -o output file using task-clock when tracepoints may be unavailable. + rm -f "${clockdata}" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${clockdata}" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import os +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.events = lambda: ["task-clock"] + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + await pilot.press("space") + await pilot.pause(0.1) + await pilot.press("space") + for _ in range(300): + if app.return_code is not None: + break + await pilot.pause(0.05) + finally: + rec.stop() + if app.return_code != 1 or not app.data.nr_samples or not os.path.getsize(sys.argv[2]): + raise RuntimeError(f"Unexpected result: rc={app.return_code} samples={app.data.nr_samples}") + +asyncio.run(run()) +EOF + then + echo "ttimechart no events test [Failed: LiveRecorder task-clock test]" + cat "${output}" + err=1 + return + fi echo "ttimechart no events test [Success]" }
@@ -218,8 +285,250 @@ EOF echo "ttimechart headless UI test [Success]" } +test_live_synthetic() { + echo "ttimechart live UI test (synthetic)" + # Exercise live mode's rolling window, pruning, pause/resume and + # following with a fake recorder, which doesn't need tracepoint + # permissions. + if ! "$PYTHON" - > "${output}" 2>&1 <<'EOF' +import asyncio +import threading +import time +from time import monotonic_ns +import ttimechart as tc + +SEC = 1_000_000_000 + +class Ev: + def __init__(self, name): + self.name = name + def __str__(self): + return f"evsel({self.name})" + +class Sample: + def __init__(self, name, t, cpu, tid, **kw): + self.evsel = Ev(name) + self.sample_time = t + self.sample_cpu = cpu + self.sample_tid = tid + self.sample_pid = tid + self.sample_id = hash(name) + for k, v in kw.items(): + setattr(self, k, v) + +class FakeRecorder: + def __init__(self): + self.origin = monotonic_ns() + self.data_fd = -1 + self.flush_interval = 0.05 + self.paused = False + self.pause_calls = [] + self.stopped = threading.Event() + self.finish = False + + def backlog(self): + return 0.0 + + def set_paused(self, paused): + self.paused = paused + self.pause_calls.append(paused) + + def stop(self): + self.stopped.set() + + def error_text(self): + return "" + +async def run() -> None: + rec = FakeRecorder() + + def fake_read(data, _fd): + t0 = rec.origin + # Task 300 only runs at the start so pruning should drop it once the + # 1s window advances past t0 + 3000. + data.process_event(Sample("sched:sched_wakeup", t0 + 1000, 0, 1, + pid=300, comm="early", common_pid=1, common_flags=0)) + data.process_event(Sample("sched:sched_switch", t0 + 2000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=300, next_comm="early")) + data.process_event(Sample("sched:sched_switch", t0 + 3000, 0, 300, + prev_pid=300, prev_comm="early", prev_state=1, + next_pid=0, next_comm="swapper")) + cur = 0 + while not rec.stopped.is_set() and not rec.finish: + now = monotonic_ns() + if rec.paused: + time.sleep(0.01) + continue + nxt = 100 if cur == 0 else 0 + comm = "worker" if nxt == 100 else "swapper" + prev_comm = "worker" if cur == 100 else "swapper" + if nxt: + data.process_event(Sample("sched:sched_wakeup", now, 0, cur or 1, + pid=nxt, comm=comm, common_pid=cur or 1, + common_flags=0)) + data.process_event(Sample("sched:sched_switch", now, 0, cur, + prev_pid=cur, prev_comm=prev_comm, prev_state=1, + next_pid=nxt, next_comm=comm)) + cur = nxt + time.sleep(0.005) + + tc.read_events = fake_read + app = tc.TimechartApp("live", False, False, [], live=rec, window=1.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(300): + with app.data.lock: + if app.data.pruned_to > rec.origin + 3000: + break + await pilot.pause(0.05) + with app.data.lock: + if app.data.pruned_to <= rec.origin + 3000 or 300 in app.data.tasks: + raise RuntimeError("Task outside the window was not pruned") + starts = [t.segs.starts[0] for t in app.data.tasks.values() if len(t.segs)] + if not starts or min(starts) < app.data.pruned_to: + raise RuntimeError(f"Unpruned segments: {starts} < {app.data.pruned_to}") + if app.window.last - app.window.first != SEC: + raise RuntimeError(f"Unexpected window span: {app.window}") + + await pilot.press("space") + await pilot.pause(0.2) + if rec.pause_calls != [True] or "Paused" not in app.sub_title: + raise RuntimeError(f"Did not pause: {rec.pause_calls} {app.sub_title}") + frozen = app.window.last + await pilot.pause(0.4) + if app.window.last != frozen: + raise RuntimeError("Window moved while paused") + await pilot.press("space") + await pilot.pause(0.3) + if rec.pause_calls != [True, False] or not app.following: + raise RuntimeError("Did not resume") + + await pilot.press("plus") + await pilot.pause(0.2) + if app.following: + raise RuntimeError("Zooming did not stop following") + await pilot.press("0") + await pilot.pause(0.2) + if not app.following: + raise RuntimeError("Resetting zoom did not resume following") + + rec.finish = True + for _ in range(100): + if not app.recording: + break + await pilot.pause(0.05) + if app.recording or "Finished" not in app.sub_title: + raise RuntimeError(f"Did not finish: {app.sub_title}") + finally: + rec.stop() + + # Also test live I/O-only mode (-I). + io_events = tc.LiveRecorder("perf", False, False, [], None, io_only=True).events() + if not io_events or not all(e.startswith("syscalls:sys_") for e in io_events): + raise RuntimeError(f"Unexpected I/O events: {io_events}") + io_rec = FakeRecorder() + + def fake_io_read(data, _fd): + now = monotonic_ns() + # Self I/O should be ignored in live mode. + own = tc.os.getpid() + data.process_event(Sample("syscalls:sys_enter_read", now, 0, own, fd=3)) + data.process_event(Sample("syscalls:sys_exit_read", now + 1000, 0, own, ret=64)) + # Workload I/O should be recorded. + data.process_event(Sample("syscalls:sys_enter_read", now + 2000, 0, 4242, fd=4)) + data.process_event(Sample("syscalls:sys_exit_read", now + 5000, 0, 4242, ret=4096)) + data.process_event(Sample("syscalls:sys_enter_write", now + 6000, 0, 4242, fd=5)) + data.process_event(Sample("syscalls:sys_exit_write", now + 9000, 0, 4242, ret=2048)) + + tc.read_events = fake_io_read + io_app = tc.TimechartApp("live", False, False, [], live=io_rec, window=1.0, io_only=True) + async with io_app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + if not io_app.recording: + break + await pilot.pause(0.05) + await pilot.pause() + tabs = io_app.query_one(tc.TabbedContent) + if tabs.active != "io" or not io_app.io_tasks or io_app.data.io_events != 4: + raise RuntimeError(f"Unexpected I/O state: active={tabs.active} " + f"tasks={len(io_app.io_tasks)} events={io_app.data.io_events}") + tabs.active = "summary" + await pilot.pause() + await pilot.press("enter") + await pilot.pause() + if tabs.active != "io": + raise RuntimeError(f"Summary enter did not activate I/O tab: {tabs.active}") + print("live synthetic ok") + +asyncio.run(run()) +EOF + then + echo "ttimechart live UI test (synthetic) [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "live synthetic ok" "${output}"; then + echo "ttimechart live UI test (synthetic) [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live UI test (synthetic) [Success]" +} + +test_live_record() { + echo "ttimechart live record test" + if [ "${system_wide}" != 1 ]; then + # Live mode records system wide, like perf timechart record. + echo "ttimechart live record test [Skipped: can't record system wide]" + return + fi + rm -f "${perfdata}.live" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${perfdata}.live" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if not app.recording: + break + await pilot.pause(0.1) + if app.recording or not app.data.has_events(): + raise RuntimeError("Live recording of workload did not complete") + print("live record ok") + finally: + rec.stop() + +asyncio.run(run()) +EOF + then + echo "ttimechart live record test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! perf script ttimechart -i "${perfdata}.live" --dump > "${output}" 2>&1 || + ! grep -q "^Duration: .*sched events: [1-9]" "${output}"; then + echo "ttimechart live record test [Failed: saved perf.data unreadable]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live record test [Success]" +} + test_plumbing test_no_events +test_live_synthetic if ! record; then echo "Skipping remaining tests, failed to record scheduler events (permissions?)"
@@ -232,6 +541,7 @@ fi test_dump test_headless_ui +test_live_record cleanup exit $err
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
[email protected]
AI reviewRe: [PATCH v5 2/2] perf test: Test perf timechart --live
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] perf test: unprivileged failure in test_no_events() due to system-wide -a --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] perf test: unprivileged failure in test_no_events() due to system-wide -a --
commit a678a998e849226efd51ba949028c49bad18c75e
Author: Alice Rogers <[email protected]>
perf test: Test perf timechart --live
This commit extends test_ttimechart_python.sh to test --live option validation, LiveRecorder's perf record subprocess, synthetic live UI behavior, and live workload recording.
Quoted 4 lines
tools/perf/tests/shell/test_ttimechart_python.sh +0
-0
[ ... ]
Quoted 31 lines
tools/perf/tests/shell/test_ttimechart_python.sh +27
-0@@ -115,6 +145,43 @@ test_no_events() { err=1 return fi + # Exercise LiveRecorder's perf record subprocess, control pipe and + # -o output file using task-clock when tracepoints may be unavailable. + rm -f "${clockdata}" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${clockdata}" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import os +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.events = lambda: ["task-clock"] + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + await pilot.press("space") + await pilot.pause(0.1) + await pilot.press("space") + for _ in range(300): + if app.return_code is not None: + break + await pilot.pause(0.05) + finally: + rec.stop() + if app.return_code != 1 or not app.data.nr_samples or not os.path.getsize(sys.argv[2]):
tools/perf/tests/shell/test_ttimechart_python.sh test_no_events() { · line 184
Quoted 10 lines
tools/perf/tests/shell/test_ttimechart_python.sh +10
-0+ raise RuntimeError(f"Unexpected result: rc={app.return_code} samples={app.data.nr_samples}") + +asyncio.run(run()) +EOF + then + echo "ttimechart no events test [Failed: LiveRecorder task-clock test]" + cat "${output}" + err=1 + return + fi
[Severity: Medium]
Can this cause test_no_events() to fail instead of skip when run by an unprivileged user on a machine with the default kernel.perf_event_paranoid setting of 1 or 2?
At the start of test_no_events(), the permission check runs a per-task recording without -a:
tools/perf/tests/shell/test_ttimechart_python.sh:test_no_events() {
if ! perf record -e task-clock -o "${clockdata}" -- true > /dev/null 2>&1; then
echo "ttimechart no events test [Skipped: perf record failed]"
return
fi
...
}Because that check opens a per-task software event, it succeeds for unprivileged users under kernel.perf_event_paranoid = 1 or 2.
However, LiveRecorder.start() unconditionally passes -a for system-wide recording:
tools/perf/python/ttimechart.py:LiveRecorder.start() {
...
cmd.append("-a")
if self.workload:
cmd += ["--", *self.workload]
...
}When kernel.perf_event_paranoid > 0 and the user lacks CAP_PERFMON, system-wide perf record -a -e task-clock fails with EACCES via find_get_context() -> perf_allow_cpu(). That leaves app.data.nr_samples and the output file empty, raising RuntimeError and setting err=1, which causes the script to exit with 1 (failure) rather than 2 (skip) when record() subsequently skips.
Should the LiveRecorder task-clock subtest check whether perf record -a -e task-clock is permitted before running, or only run when system-wide recording is available?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerv6 cover perf timechart: Add --live mode to the TUI and test
In reply to Ian RogersAdd a '--live' option to 'perf timechart' and 'ttimechart.py' that streams live scheduler, power, or I/O (-I/--io-only) events from a 'perf record' subprocess over a pipe and displays them in the Textual TUI with a rolling time window (--window, default 10s), pause/resume via the control pipe, an optional workload after '--' that stops recording when it exits, and optional output recording (-o), along with headless shell tests in test_ttimechart_python.sh.
v6: Rebase onto perf-tools-next and address Sashiko review feedback:
- Patch 1: Queue pause/resume gap timestamps in a deque so rapid pause/resume cycles before buffered events arrive still close
pre-pause open task and CPU states at each gap.
- Patch 1: Increment nr_samples inside TimechartData.lock and check/update refreshed_samples in the same lock acquisition as checkpoint() in live_refresh() so final buffered events aren't missed when paused.
- Patch 1: Keep cursor following in sync when zooming in/out, resetting zoom with '0' after moving the cursor, or jumping to a waker ('w').
- Patch 1: Filter out all threads of ttimechart.py (via comm) and the child perf record process (via $$) on syscall tracepoints in --live -I mode, and ignore filtered I/O events before counting nr_samples.
- Patch 1: Track record boundaries in LiveRecorder's pipe pump and emit a synthetic PERF_RECORD_FINISHED_ROUND when perf record goes idle or pauses so ordered_events does not hold back the latest round.
- Patch 1: Always drain perf record's stdout on stop so perf record can flush and terminate its workload cleanly without blocking on pipe writes, and kill its process group if wait times out.
- Patch 2: Gate the LiveRecorder task-clock subtest on system-wide task-clock permissions so unprivileged runs under perf_event_paranoid > 0 skip cleanly, and add synthetic tests for multiple pause gaps, round tracking, and cursor following.
v5: https://lore.kernel.org/linux-perf-users/[email protected]/
v5: Address Sashiko review feedback:
- Patch 1: Always record system wide, as 'perf timechart record'
does, as switches into a task and idle states aren't seen by
per-task events when only a workload was recorded.
- Patch 1: Don't move a task's state start back before its recorded history when its switch out and back in were lost, which inflated
its running time.
- Patch 1: Serialize LiveRecorder.stop() calls from the UI thread and
the thread processing events.
- Patch 2: Only run the live recording test when scheduler events can
be recorded system wide.
v4: https://lore.kernel.org/linux-perf-users/[email protected]/
v4: Address Sashiko review feedback on patch 1:
- Don't prune live history past the last event, so open states aren't
lost when no events arrive for a window, and don't count a still
running task's open segment as a switch.
- Keep showing the window at the end of recording, rather than jumping back to already pruned history, and prune events flushed as perf
record exits.
- Close open task, CPU and I/O states when resuming after a pause,
rather than extending them across the unrecorded gap, and close the running task when a switch away from it was lost.
- Stop perf record and close the files and pipes if
LiveRecorder.start() fails after starting it, and close the data pipe read ends in LiveRecorder.stop().
v3: https://lore.kernel.org/linux-perf-users/[email protected]/
v3: Address Sashiko review feedback:
- Patch 1: Explicitly include <string.h> in builtin-timechart.c.
- Patch 1: Reject non-finite --window values (e.g. 'inf') that
overflowed converting to nanoseconds.
- Patch 2: Stop the fake recorder in a try/finally in the synthetic
live UI test so a failed check can't leave its background thread
running and hang the test.
Rebase onto the latest perf-tools-next.
v2: https://lore.kernel.org/linux-perf-users/[email protected]/
v2: Rebase the remaining 2 patches onto perf-tools-next after patches 1-11
of v1 were merged.
Alice Rogers (2):
perf timechart: Add a --live mode to the TUI
perf test: Test perf timechart --live
tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 80 +- tools/perf/python/ttimechart.py | 1232 +++++++++++++++-- .../tests/shell/test_ttimechart_python.sh | 355 +++++ 4 files changed, 1590 insertions(+), 97 deletions(-)
Signature · 1 line
2.56.1.703.gf848ed807c-goog
Ian Rogers
reviewerPATCH v6 1/2 perf timechart: Add a --live mode to the TUI
From: Alice Rogers <[email protected]>
Add a --live option to perf timechart and ttimechart.py that runs 'perf record' writing CLOCK_MONOTONIC scheduler and power events, or I/O syscall events when -I/--io-only is given, to a pipe and displays them in the TUI as they happen. Only the most recent --window seconds of history (default 10) are kept in memory, pruning older segments and tasks that have no remaining history in the window.
Pressing space pauses and resumes recording using perf record's --control pipe. Because perf record only reads its ring buffers when a watermark is reached, periodic 'ping' commands are sent on the control pipe to flush events promptly, and a synthetic finished-round event is injected on record boundaries when perf record goes idle or pauses so ordered-event buffering does not hold onto the latest round. The refresh interval adapts to the measured screen refresh cost and backs off if unprocessed data builds up in the pipe.
Like 'perf timechart record', the whole system is recorded, as switches into a task and idle states aren't seen by per-task events. An optional workload may be given after '--' to stop recording when it exits, and '-o' optionally saves a copy of the recorded perf.data stream.
--- tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 80 +- tools/perf/python/ttimechart.py | 1232 +++++++++++++++++-- 3 files changed, 1235 insertions(+), 97 deletions(-)
tools/perf/Documentation/perf-timechart.txt +18 -2
@@ -47,6 +47,9 @@ TIMECHART OPTIONS -T:: --tasks-only:: Don't output processor state transitions +-I:: +--io-only:: + In '--live' or '--tui' mode, only record or show I/O activity -p:: --process:: Select the processes to display, by name or PID
@@ -59,8 +62,21 @@ TIMECHART OPTIONS linkperf:perf-script[1]). The script requires the perf python module and the python 'textual' library. The CPU, task, I/O and summary views can be zoomed ('+'/'-'), panned (shift+arrows) and the state of the - selected row at the cursor is described. The -i, -P, -T and -p options - are passed to the script, other output options are ignored. + selected row at the cursor is described. The -i, -P, -T, -I and -p + options are passed to the script, other output options are ignored. +--live:: + Record scheduler and power events, or I/O events with '-I', with + linkperf:perf-record[1] and display them as they happen in the + terminal timechart, implying --tui. Only the most recent '--window' + seconds of events are kept and shown. Pressing space pauses and + resumes recording; zooming or panning stops the view following the + latest events until '0' is pressed. The whole system is recorded, if a + command is given after '--' then recording stops when it exits, + otherwise it continues until the UI is quit. If '-o' + is given the recorded perf data is also saved to that file. +--window=<seconds>:: + In '--live' mode, the number of seconds of recent events kept in + memory and shown in the timeline (default: 10). --symfs=<directory[,layout]>:: Look for files with symbols relative to this directory. The optional layout can be 'hierarchy' (default, matches full path) or 'flat'
tools/perf/builtin-timechart.c +70 -10
159 lines not on this page · raw message
tools/perf/python/ttimechart.py +1147 -85
1,542 lines not on this page · raw message
Signature · 1 line
2.56.1.703.gf848ed807c-goog
[email protected]
AI reviewRe: [PATCH v6 1/2] perf timechart: Add a --live mode to the TUI
Thank you for your contribution! Sashiko AI review found 2 potential issues to consider: - [Medium] perf timechart: unescaped quotes in syscall filter in LiveRecorder.start() - [Low] perf timechart…
Thank you for your contribution! Sashiko AI review found 2 potential issues to consider:
- [Medium] perf timechart: unescaped quotes in syscall filter in LiveRecorder.start()
- [Low] perf timechart: unstarted control_thread.join() in LiveRecorder._stop()
--
commit 5178221692a3b293f71ade63d89dfc1258413569
Author: Alice Rogers <[email protected]>
perf timechart: Add a --live mode to the TUI
This commit adds a --live option to perf timechart and ttimechart.py that runs perf record to stream scheduler, power, or I/O syscall events over a pipe and displays a rolling window of recent activity in the TUI.
Quoted 4 lines
tools/perf/python/ttimechart.py +0
-0
[ ... ]
Quoted 5 lines
tools/perf/python/ttimechart.py +1
-0@@ -1726,52 +2210,630 @@ class TimechartApp(App): self.goto_task(waker, wake_time) +class PerfPipeTracker:
[ ... ]
Quoted 1 line
+class LiveRecorder:
[ ... ]
Quoted 20 lines
tools/perf/python/ttimechart.py +20
-0+ def start(self) -> None: + """Start perf record, the data is then read from data_fd.""" + output = open(self.output, "wb") if self.output else None + # perf record's ends of the control pipes, closed once it is started. + child_fds: List[int] = [] + try: + ctl_r, self.ctl_fd = os.pipe() + child_fds.append(ctl_r) + self.ack_fd, ack_w = os.pipe() + child_fds.append(ack_w) + self.wake_r, self.wake_w = os.pipe() + os.set_blocking(self.wake_r, False) + os.set_blocking(self.wake_w, False) + ev_names = self.events() + if any(name.startswith("syscalls:") for name in ev_names): + own_comm = self.set_own_comm() + # common_pid is a thread ID; matching comm excludes all threads + # of this process, and $$ excludes the perf record process that + # replaces sh via exec. + filt = (f'comm != "{own_comm}" && common_pid != $$' if own_comm else
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2473
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ f"common_pid != {os.getpid()} && common_pid != $$") + ev_args = " ".join( + f'-e {shlex.quote(name)} --filter "{filt}"' + if name.startswith("syscalls:") else f"-e {shlex.quote(name)}" + for name in ev_names + ) + script = ('perf_exe=$1; ctl=$2; shift 2; ' + 'exec "$perf_exe" record -q -o - -R -c 1 -k mono ' + f'--synth=task --control "$ctl" {ev_args} -a "$@"') + cmd = ["/bin/sh", "-c", script, "sh", self.perf_exe, f"fd:{ctl_r},{ack_w}"]
[Severity: Medium]
Does /bin/sh strip the inner double quotes around own_comm when expanding --filter "{filt}"?
When set_own_comm() succeeds, interpolating filt into --filter "{filt}" produces --filter "comm != "ttimechart:..." && common_pid != $$", so the shell removes the quotes around the comm string before execing perf record. The kernel tracepoint filter parser requires string operands for comm to be quoted and rejects the filter with -EINVAL, causing perf timechart --live -I to fail on startup.
Quoted 40 lines
tools/perf/python/ttimechart.py +40
-0+ if self.workload: + cmd += ["--", *self.workload] + else: + # Quiet so that only errors are written to stderr, see error_text(). + cmd = [self.perf_exe, "record", "-q", "-o", "-", "-R", "-c", "1", "-k", "mono", + "--synth=task", "--control", f"fd:{ctl_r},{ack_w}"] + for name in ev_names: + cmd += ["-e", name] + # Record system wide, as timechart__record() does. Switches into a + # task and idle states aren't seen by events following tasks. With + # a workload, recording stops when it exits, otherwise when stopped. + cmd.append("-a") + if self.workload: + cmd += ["--", *self.workload] + self.origin = monotonic_ns() + # A new session so that signals for the terminal aren't received. + self.proc = subprocess.Popen(cmd, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, + stderr=self.stderr, pass_fds=(ctl_r, ack_w), + start_new_session=True) + assert self.proc.stdout + proc_fd = self.proc.stdout.fileno() + self.grow_pipe(proc_fd) + self.tee_r, tee_w = os.pipe() + self.pump_thread = threading.Thread(target=self.pump, + args=(proc_fd, tee_w, output), + daemon=True) + try: + self.pump_thread.start() + except BaseException: + self.pump_thread = None + os.close(tee_w) + raise + # Closed by the pump thread. + output = None + self.data_fd = self.tee_r + self.grow_pipe(self.data_fd) + self.control_thread = threading.Thread(target=self.control_loop, daemon=True) + self.control_thread.start() + except BaseException: + self.stop()
[ ... ]
Quoted 13 lines
tools/perf/python/ttimechart.py +13
-0+ def _stop(self) -> None: + """Stop perf record and the threads, stop_lock must be held.""" + proc = self.proc + if proc is not None and proc.poll() is None: + try: + proc.send_signal(signal.SIGINT) + proc.wait(timeout=2) + except subprocess.TimeoutExpired: + # Kill the whole session so a workload isn't left running. + try: + os.killpg(proc.pid, signal.SIGKILL) + except OSError: + pass
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2711
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ proc.kill() + proc.wait() + if self.pump_thread and self.pump_thread is not threading.current_thread(): + self.pump_thread.join(timeout=1) + if self.control_thread and self.control_thread is not threading.current_thread(): + self.control_thread.join(timeout=1) + self.close_control_fds() + self.close_wake_fds() + self.close_data_fds() + self.restore_own_comm()
[Severity: Low]
If self.control_thread.start() fails in LiveRecorder.start(), can self.control_thread.join() here raise RuntimeError and skip the cleanup that follows?
Unlike self.pump_thread, self.control_thread is not reset to None if starting the thread raises an exception, so self.stop() -> self._stop() attempts to join an unstarted thread and aborts before closing the pipe file descriptors and restoring /proc/self/comm.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v6 2/2 perf test: Test perf timechart --live
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Extend test_ttimechart_python.sh to cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file when system-wide recording is permitted, a synthetic live UI test exercising the rolling window, pruning, pause/resume gaps, round-flush framing and zoom/cursor following, and a live workload recording test when scheduler events can be recorded system wide.
--- .../tests/shell/test_ttimechart_python.sh | 355 ++++++++++++++++++ 1 file changed, 355 insertions(+)
tools/perf/tests/shell/test_ttimechart_python.sh +355 -0
@@ -24,6 +24,7 @@ if ! "$PYTHON" -c 'import textual' > /dev/null 2>&1; then fi err=0 +system_wide=0 perfdata= output= clockdata=
@@ -57,6 +58,7 @@ record() { # fall back to recording just the workload's scheduler events. if perf timechart record -o "${perfdata}" -- \ bash -c "$(declare -f workload); workload" > /dev/null 2>&1; then + system_wide=1 return 0 fi if perf record -R -c 1 -e sched:sched_switch -e sched:sched_wakeup \
@@ -82,6 +84,34 @@ test_plumbing() { err=1 return fi + if perf timechart --live -i "${perfdata}" > "${output}" 2>&1 || + ! grep -q -- "--live can't be used with -i" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --live with -i not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --live --window 0 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "--window must be a positive" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window 0 not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --tui --window 5 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "require --live" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window without --live]" + cat "${output}" + err=1 + return + fi + if perf timechart --live -I -P < /dev/null > "${output}" 2>&1 || + ! grep -q -- "-I cannot be used with -P or -T" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: -I with -P not rejected]" + cat "${output}" + err=1 + return + fi echo "perf timechart --tui plumbing test [Success]" }
@@ -115,6 +145,45 @@ test_no_events() { err=1 return fi + # Exercise LiveRecorder's perf record subprocess, control pipe and + # -o output file using task-clock when system-wide recording is allowed. + if perf record -a -e task-clock -o "${clockdata}" -- true > /dev/null 2>&1; then + rm -f "${clockdata}" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${clockdata}" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import os +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.events = lambda: ["task-clock"] + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + await pilot.press("space") + await pilot.pause(0.1) + await pilot.press("space") + for _ in range(300): + if app.return_code is not None: + break + await pilot.pause(0.05) + finally: + rec.stop() + if app.return_code != 1 or not app.data.nr_samples or not os.path.getsize(sys.argv[2]): + raise RuntimeError(f"Unexpected result: rc={app.return_code} samples={app.data.nr_samples}") + +asyncio.run(run()) +EOF + then + echo "ttimechart no events test [Failed: LiveRecorder task-clock test]" + cat "${output}" + err=1 + return + fi + fi echo "ttimechart no events test [Success]" }
@@ -218,8 +287,293 @@ EOF echo "ttimechart headless UI test [Success]" } +test_live_synthetic() { + echo "ttimechart live UI test (synthetic)" + # Exercise live mode's rolling window, pruning, pause/resume and + # following with a fake recorder, which doesn't need tracepoint + # permissions. + if ! "$PYTHON" - > "${output}" 2>&1 <<'EOF' +import asyncio +import threading +import time +from time import monotonic_ns +import ttimechart as tc + +SEC = 1_000_000_000 + +class Ev: + def __init__(self, name): + self.name = name + def __str__(self): + return f"evsel({self.name})" + +class Sample: + def __init__(self, name, t, cpu, tid, **kw): + self.evsel = Ev(name) + self.sample_time = t + self.sample_cpu = cpu + self.sample_tid = tid + self.sample_pid = tid + self.sample_id = hash(name) + for k, v in kw.items(): + setattr(self, k, v) + +class FakeRecorder: + def __init__(self): + self.origin = monotonic_ns() + self.data_fd = -1 + self.flush_interval = 0.05 + self.paused = False + self.pause_calls = [] + self.stopped = threading.Event() + self.finish = False + + def backlog(self): + return 0.0 + + def set_paused(self, paused): + self.paused = paused + self.pause_calls.append(paused) + + def stop(self): + self.stopped.set() + + def error_text(self): + return "" + +async def run() -> None: + rec = FakeRecorder() + + def fake_read(data, _fd): + t0 = rec.origin + # Task 300 only runs at the start so pruning should drop it once the + # 1s window advances past t0 + 3000. + data.process_event(Sample("sched:sched_wakeup", t0 + 1000, 0, 1, + pid=300, comm="early", common_pid=1, common_flags=0)) + data.process_event(Sample("sched:sched_switch", t0 + 2000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=300, next_comm="early")) + data.process_event(Sample("sched:sched_switch", t0 + 3000, 0, 300, + prev_pid=300, prev_comm="early", prev_state=1, + next_pid=0, next_comm="swapper")) + cur = 0 + while not rec.stopped.is_set() and not rec.finish: + now = monotonic_ns() + if rec.paused: + time.sleep(0.01) + continue + nxt = 100 if cur == 0 else 0 + comm = "worker" if nxt == 100 else "swapper" + prev_comm = "worker" if cur == 100 else "swapper" + if nxt: + data.process_event(Sample("sched:sched_wakeup", now, 0, cur or 1, + pid=nxt, comm=comm, common_pid=cur or 1, + common_flags=0)) + data.process_event(Sample("sched:sched_switch", now, 0, cur, + prev_pid=cur, prev_comm=prev_comm, prev_state=1, + next_pid=nxt, next_comm=comm)) + cur = nxt + time.sleep(0.005) + + tc.read_events = fake_read + app = tc.TimechartApp("live", False, False, [], live=rec, window=1.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(300): + with app.data.lock: + if app.data.pruned_to > rec.origin + 3000: + break + await pilot.pause(0.05) + with app.data.lock: + if app.data.pruned_to <= rec.origin + 3000 or 300 in app.data.tasks: + raise RuntimeError("Task outside the window was not pruned") + starts = [t.segs.starts[0] for t in app.data.tasks.values() if len(t.segs)] + if not starts or min(starts) < app.data.pruned_to: + raise RuntimeError(f"Unpruned segments: {starts} < {app.data.pruned_to}") + if app.window.last - app.window.first != SEC: + raise RuntimeError(f"Unexpected window span: {app.window}") + + await pilot.press("space") + await pilot.pause(0.2) + if rec.pause_calls != [True] or "Paused" not in app.sub_title: + raise RuntimeError(f"Did not pause: {rec.pause_calls} {app.sub_title}") + frozen = app.window.last + await pilot.pause(0.4) + if app.window.last != frozen: + raise RuntimeError("Window moved while paused") + await pilot.press("space") + await pilot.pause(0.3) + if rec.pause_calls != [True, False] or not app.following: + raise RuntimeError("Did not resume") + + await pilot.press("plus") + await pilot.pause(0.2) + if app.following or app.follow_cursor: + raise RuntimeError("Zooming did not stop following") + await pilot.press("0") + await pilot.pause(0.2) + if not app.following or not app.follow_cursor: + raise RuntimeError("Resetting zoom did not resume following") + await pilot.press("left") + await pilot.pause(0.1) + if not app.following or app.follow_cursor: + raise RuntimeError("Moving cursor did not stop cursor following") + await pilot.press("0") + await pilot.pause(0.1) + if not app.follow_cursor: + raise RuntimeError("Resetting zoom did not resume cursor following") + + rec.finish = True + for _ in range(100): + if not app.recording: + break + await pilot.pause(0.05) + if app.recording or "Finished" not in app.sub_title: + raise RuntimeError(f"Did not finish: {app.sub_title}") + finally: + rec.stop() + + # Multiple pause/resume cycles before buffered events arrive must still + # close states at each unrecorded gap, and pipe round tracking must detect + # unflushed rounds across tracing metadata. + gap_data = tc.TimechartData() + gap_data.process_event(Sample("sched:sched_switch", 1000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=50, next_comm="runner")) + with gap_data.lock: + gap_data.mark_gap(2000) + gap_data.mark_gap(4000) + gap_data.process_event(Sample("sched:sched_switch", 2500, 0, 50, + prev_pid=50, prev_comm="runner", prev_state=1, + next_pid=0, next_comm="swapper")) + gap_data.process_event(Sample("sched:sched_switch", 4500, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=50, next_comm="runner")) + gap_data.finish() + if list(zip(gap_data.tasks[50].segs.starts, gap_data.tasks[50].segs.ends)) != [(2000, 2500)]: + raise RuntimeError(f"Unexpected gap segments: {gap_data.tasks[50].segs.starts}") + + tracker = tc.PerfPipeTracker() + pipe_hdr = tc.struct.pack("=QQ", tracker.PERF_MAGIC, 16) + td_rec = tc.struct.pack("=IHHI", tracker.HEADER_TRACING_DATA, 0, 12, 8) + b"12345678" + sample_rec = tc.struct.pack("=IHHQ", 9, 0, 16, 0) + # Feed byte-by-byte to exercise split headers. + for b in pipe_hdr + td_rec + sample_rec + tracker.FINISHED_ROUND_EVENT: + tracker.feed(bytes([b])) + if not tracker.at_boundary() or not tracker.unflushed_round or tracker.rounds != 1: + raise RuntimeError("PerfPipeTracker failed to track unflushed round") + tracker.feed(tracker.FINISHED_ROUND_EVENT) + if tracker.unflushed_round or tracker.rounds != 2: + raise RuntimeError("PerfPipeTracker failed to clear unflushed round") + + # Also test live I/O-only mode (-I). + io_events = tc.LiveRecorder("perf", False, False, [], None, io_only=True).events() + if not io_events or not all(e.startswith("syscalls:sys_") for e in io_events): + raise RuntimeError(f"Unexpected I/O events: {io_events}") + io_rec = FakeRecorder() + + def fake_io_read(data, _fd): + now = monotonic_ns() + # Self I/O should be ignored in live mode without counting as samples. + own = tc.os.getpid() + data.process_event(Sample("syscalls:sys_enter_read", now, 0, own, fd=3)) + data.process_event(Sample("syscalls:sys_exit_read", now + 1000, 0, own, ret=64)) + # Workload I/O should be recorded. + data.process_event(Sample("syscalls:sys_enter_read", now + 2000, 0, 4242, fd=4)) + data.process_event(Sample("syscalls:sys_exit_read", now + 5000, 0, 4242, ret=4096)) + data.process_event(Sample("syscalls:sys_enter_write", now + 6000, 0, 4242, fd=5)) + data.process_event(Sample("syscalls:sys_exit_write", now + 9000, 0, 4242, ret=2048)) + + tc.read_events = fake_io_read + io_app = tc.TimechartApp("live", False, False, [], live=io_rec, window=1.0, io_only=True) + async with io_app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + if not io_app.recording: + break + await pilot.pause(0.05) + await pilot.pause() + tabs = io_app.query_one(tc.TabbedContent) + if (tabs.active != "io" or not io_app.io_tasks or io_app.data.io_events != 4 or + io_app.data.nr_samples != 4): + raise RuntimeError(f"Unexpected I/O state: active={tabs.active} " + f"tasks={len(io_app.io_tasks)} events={io_app.data.io_events} " + f"samples={io_app.data.nr_samples}") + tabs.active = "summary" + await pilot.pause() + await pilot.press("enter") + await pilot.pause() + if tabs.active != "io": + raise RuntimeError(f"Summary enter did not activate I/O tab: {tabs.active}") + print("live synthetic ok") + +asyncio.run(run()) +EOF + then + echo "ttimechart live UI test (synthetic) [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "live synthetic ok" "${output}"; then + echo "ttimechart live UI test (synthetic) [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live UI test (synthetic) [Success]" +} + +test_live_record() { + echo "ttimechart live record test" + if [ "${system_wide}" != 1 ]; then + # Live mode records system wide, like perf timechart record. + echo "ttimechart live record test [Skipped: can't record system wide]" + return + fi + rm -f "${perfdata}.live" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${perfdata}.live" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if not app.recording: + break + await pilot.pause(0.1) + if app.recording or not app.data.has_events(): + raise RuntimeError("Live recording of workload did not complete") + print("live record ok") + finally: + rec.stop() + +asyncio.run(run()) +EOF + then + echo "ttimechart live record test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! perf script ttimechart -i "${perfdata}.live" --dump > "${output}" 2>&1 || + ! grep -q "^Duration: .*sched events: [1-9]" "${output}"; then + echo "ttimechart live record test [Failed: saved perf.data unreadable]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live record test [Success]" +} + test_plumbing test_no_events +test_live_synthetic if ! record; then echo "Skipping remaining tests, failed to record scheduler events (permissions?)"
@@ -232,6 +586,7 @@ fi test_dump test_headless_ui +test_live_record cleanup exit $err
Signature · 1 line
2.56.1.703.gf848ed807c-goog
[email protected]
AI reviewRe: [PATCH v6 2/2] perf test: Test perf timechart --live
Sashiko has reviewed this patch and found no issues.
Quoted 9 lines
From: Alice Rogers <[email protected]>
Extend
test_ttimechart_python.shto cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file when system-wide recording is permitted, a synthetic live UI test exercising the rolling window, pruning, pause/resume gaps, round-flush framing and zoom/cursor following, and a live workload recording test when scheduler events can be recorded system wide.
Quoted 3 lines
Signed-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerv7 cover perf timechart: Add --live mode to the TUI and test
In reply to Ian RogersAdd a '--live' option to 'perf timechart' and 'ttimechart.py' that streams live scheduler, power, or I/O (-I/--io-only) events from a 'perf record' subprocess over a pipe and displays them in the Textual TUI with a rolling time window (--window, default 10s), pause/resume via the control pipe, an optional workload after '--' that stops recording when it exits, and optional output recording (-o), along with headless shell tests in test_ttimechart_python.sh.
v7: Address Sashiko review feedback:
- Patch 1: Pass the comm/PID filter expression as a positional argument to /bin/sh so the shell preserves the quotes around comm in --live -I mode instead of stripping them before execing perf record.
- Patch 1: Clear LiveRecorder.control_thread if starting it raises so stop() does not call join() on an unstarted thread and skip FD/comm cleanup, and ignore OSError if perf record exits concurrently during stop().
- Patch 2: Test the /bin/sh --filter quoting and control_thread startup failure cleanup in the synthetic live test, and add a live -I workload recording subtest when syscall tracepoints are permitted.
v6: https://lore.kernel.org/lkml/[email protected]/
v6: Rebase onto perf-tools-next and address Sashiko review feedback:
- Patch 1: Queue pause/resume gap timestamps in a deque so rapid pause/resume cycles before buffered events arrive still close
pre-pause open task and CPU states at each gap.
- Patch 1: Increment nr_samples inside TimechartData.lock and check/update refreshed_samples in the same lock acquisition as checkpoint() in live_refresh() so final buffered events aren't missed when paused.
- Patch 1: Keep cursor following in sync when zooming in/out, resetting zoom with '0' after moving the cursor, or jumping to a waker ('w').
- Patch 1: Filter out all threads of ttimechart.py (via comm) and the child perf record process (via $$) on syscall tracepoints in --live -I mode, and ignore filtered I/O events before counting nr_samples.
- Patch 1: Track record boundaries in LiveRecorder's pipe pump and emit a synthetic PERF_RECORD_FINISHED_ROUND when perf record goes idle or pauses so ordered_events does not hold back the latest round.
- Patch 1: Always drain perf record's stdout on stop so perf record can flush and terminate its workload cleanly without blocking on pipe writes, and kill its process group if wait times out.
- Patch 2: Gate the LiveRecorder task-clock subtest on system-wide task-clock permissions so unprivileged runs under perf_event_paranoid > 0 skip cleanly, and add synthetic tests for multiple pause gaps, round tracking, and cursor following.
v5: https://lore.kernel.org/linux-perf-users/[email protected]/
v5: Address Sashiko review feedback:
- Patch 1: Always record system wide, as 'perf timechart record'
does, as switches into a task and idle states aren't seen by
per-task events when only a workload was recorded.
- Patch 1: Don't move a task's state start back before its recorded history when its switch out and back in were lost, which inflated
its running time.
- Patch 1: Serialize LiveRecorder.stop() calls from the UI thread and
the thread processing events.
- Patch 2: Only run the live recording test when scheduler events can
be recorded system wide.
v4: https://lore.kernel.org/linux-perf-users/[email protected]/
v4: Address Sashiko review feedback on patch 1:
- Don't prune live history past the last event, so open states aren't
lost when no events arrive for a window, and don't count a still
running task's open segment as a switch.
- Keep showing the window at the end of recording, rather than jumping back to already pruned history, and prune events flushed as perf
record exits.
- Close open task, CPU and I/O states when resuming after a pause,
rather than extending them across the unrecorded gap, and close the running task when a switch away from it was lost.
- Stop perf record and close the files and pipes if
LiveRecorder.start() fails after starting it, and close the data pipe read ends in LiveRecorder.stop().
v3: https://lore.kernel.org/linux-perf-users/[email protected]/
v3: Address Sashiko review feedback:
- Patch 1: Explicitly include <string.h> in builtin-timechart.c.
- Patch 1: Reject non-finite --window values (e.g. 'inf') that
overflowed converting to nanoseconds.
- Patch 2: Stop the fake recorder in a try/finally in the synthetic
live UI test so a failed check can't leave its background thread
running and hang the test.
Rebase onto the latest perf-tools-next.
v2: https://lore.kernel.org/linux-perf-users/[email protected]/
v2: Rebase the remaining 2 patches onto perf-tools-next after patches 1-11
of v1 were merged.
Alice Rogers (2):
perf timechart: Add a --live mode to the TUI
perf test: Test perf timechart --live
tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 80 +- tools/perf/python/ttimechart.py | 1244 +++++++++++++++-- .../tests/shell/test_ttimechart_python.sh | 436 ++++++ 4 files changed, 1683 insertions(+), 97 deletions(-)
Signature · 1 line
2.56.1.703.gf848ed807c-goog
Ian Rogers
reviewerPATCH v7 1/2 perf timechart: Add a --live mode to the TUI
From: Alice Rogers <[email protected]>
Add a --live option to perf timechart and ttimechart.py that runs 'perf record' writing CLOCK_MONOTONIC scheduler and power events, or I/O syscall events when -I/--io-only is given, to a pipe and displays them in the TUI as they happen. Only the most recent --window seconds of history (default 10) are kept in memory, pruning older segments and tasks that have no remaining history in the window.
Pressing space pauses and resumes recording using perf record's --control pipe. Because perf record only reads its ring buffers when a watermark is reached, periodic 'ping' commands are sent on the control pipe to flush events promptly, and a synthetic finished-round event is injected on record boundaries when perf record goes idle or pauses so ordered-event buffering does not hold onto the latest round. The refresh interval adapts to the measured screen refresh cost and backs off if unprocessed data builds up in the pipe.
Like 'perf timechart record', the whole system is recorded, as switches into a task and idle states aren't seen by per-task events. An optional workload may be given after '--' to stop recording when it exits, and '-o' optionally saves a copy of the recorded perf.data stream.
--- tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 80 +- tools/perf/python/ttimechart.py | 1244 +++++++++++++++++-- 3 files changed, 1247 insertions(+), 97 deletions(-)
tools/perf/Documentation/perf-timechart.txt +18 -2
@@ -47,6 +47,9 @@ TIMECHART OPTIONS -T:: --tasks-only:: Don't output processor state transitions +-I:: +--io-only:: + In '--live' or '--tui' mode, only record or show I/O activity -p:: --process:: Select the processes to display, by name or PID
@@ -59,8 +62,21 @@ TIMECHART OPTIONS linkperf:perf-script[1]). The script requires the perf python module and the python 'textual' library. The CPU, task, I/O and summary views can be zoomed ('+'/'-'), panned (shift+arrows) and the state of the - selected row at the cursor is described. The -i, -P, -T and -p options - are passed to the script, other output options are ignored. + selected row at the cursor is described. The -i, -P, -T, -I and -p + options are passed to the script, other output options are ignored. +--live:: + Record scheduler and power events, or I/O events with '-I', with + linkperf:perf-record[1] and display them as they happen in the + terminal timechart, implying --tui. Only the most recent '--window' + seconds of events are kept and shown. Pressing space pauses and + resumes recording; zooming or panning stops the view following the + latest events until '0' is pressed. The whole system is recorded, if a + command is given after '--' then recording stops when it exits, + otherwise it continues until the UI is quit. If '-o' + is given the recorded perf data is also saved to that file. +--window=<seconds>:: + In '--live' mode, the number of seconds of recent events kept in + memory and shown in the timeline (default: 10). --symfs=<directory[,layout]>:: Look for files with symbols relative to this directory. The optional layout can be 'hierarchy' (default, matches full path) or 'flat'
tools/perf/builtin-timechart.c +70 -10
159 lines not on this page · raw message
tools/perf/python/ttimechart.py +1159 -85
1,554 lines not on this page · raw message
Signature · 1 line
2.56.1.703.gf848ed807c-goog
[email protected]
AI reviewRe: [PATCH v7 1/2] perf timechart: Add a --live mode to the TUI
Thank you for your contribution! Sashiko AI review found 9 potential issues to consider: - [Medium] perf timechart: live_prune() at last_time deletes all open states - [Medium] perf timechart…
Thank you for your contribution! Sashiko AI review found 9 potential issues to consider:
- [Medium] perf timechart: live_prune() at last_time deletes all open states
- [Medium] perf timechart: checkpoint() and prune() drop blocked and waiting states
- [Medium] perf timechart: stale pre-exec comm in task() during live I/O mode
- [Medium] perf timechart: io_bytes and switches drift in add() and prune()
- [Low] perf timechart: Task.totals drift after checkpoint() and change_state()
- [Low] perf timechart: update_views() and dump() use sched tasks with -I
- [Low] perf timechart: fix switch count underflow in Task.prune() after gaps
- [Low] perf timechart: race in LiveRecorder.pump() clearing flush_round_target
- [Low] perf timechart: race on data_fd in LiveRecorder.backlog() during stop()
--
commit bf97da434173bd983d47c0d1625c5c56ef3fbfc7
Author: Alice Rogers <[email protected]>
perf timechart: Add a --live mode to the TUI
This commit adds a --live mode to perf timechart and ttimechart.py that streams events from perf record into the TUI and prunes history outside a sliding time window.
Quoted 4 lines
tools/perf/python/ttimechart.py +0
-0
[ ... ]
Quoted 8 lines
tools/perf/python/ttimechart.py +4
-0@@ -265,6 +319,42 @@ class Task: return True return any(f == str(self.tid) or f in self.comms for f in filters) + def prune(self, time: int) -> None: + """Discard history ending at or before time, removing it from the totals.""" + segs = self.segs + num = segs.num_ending_by(time)
tools/perf/python/ttimechart.py class Task: · line 335
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ # A still running task's last segment, extended by checkpoint(), ends + # at since. It hasn't been switched out yet. + open_edge = len(segs) - 1 if self.state == STATE_RUNNING and segs and \ + segs.ends[-1] == self.since else -1 + for i in range(num): + state = segs.keys[i] + self.totals[state] -= segs.ends[i] - segs.starts[i] + if state == STATE_RUNNING and i != open_edge: + # Running segments end when the task is switched out. + self.switches = max(self.switches - 1, 0)
[Severity: Low]
Does decrementing self.switches for every pruned STATE_RUNNING segment cause the switch count to drift when running segments are closed without a switch?
TimechartData._close_gap(), TimechartData.finish() and the lost-switch path in TimechartData.sched_switch() transition a running task to STATE_UNKNOWN and add a STATE_RUNNING segment without incrementing task.switches. When those segments are later pruned, i != open_edge is true and Task.prune() decrements self.switches anyway. Conversely, if a switch-in was lost so prev.state is not STATE_RUNNING in sched_switch(), prev.switches is incremented without a matching STATE_RUNNING segment to decrement it on prune.
tools/perf/python/ttimechart.py class Task: · line 344
Quoted 9 lines
tools/perf/python/ttimechart.py +9
-0+ if num < len(segs.starts) and segs.starts[num] < time: + self.totals[segs.keys[num]] -= time - segs.starts[num] + segs.prune(time) + io = self.io + for i in range(io.num_ending_by(time)): + ret = io.data[i][1] + if ret > 0 and io.keys[i] in (IOTYPE_READ, IOTYPE_WRITE, IOTYPE_TX, IOTYPE_RX): + self.io_bytes -= ret + io.prune(time)
[Severity: Medium]
Can SegmentList.add() merging or clipping segments cause Task.io_bytes and syscall counts to drift across Task.prune()?
SegmentList.add() merges a new segment into the previous one when start == self.ends[-1], key == self.keys[-1] and data == self.data[-1], and drops segments that clip to end <= start. In TimechartData.io_exit(), task.io.add() uses iotype as key and (fd, ret) as data, while task.io_bytes is incremented by ret on every call. If two consecutive syscalls on the same fd have the same type and return value and meet at self.ends[-1] (or share the same timestamp so the second is clipped out), only one segment is kept in task.io while task.io_bytes is incremented twice. len(task.io) then undercounts syscalls and Task.prune() only subtracts ret once, leaking ret bytes in task.io_bytes.
tools/perf/python/ttimechart.py class Task: · line 351
Quoted 7 lines
tools/perf/python/ttimechart.py +7
-0+ if self.io_pending and self.io_pending[0] < time: + self.io_pending = (time, self.io_pending[1], self.io_pending[2]) + del self.wakeups[:bisect.bisect_right(self.wakeups, (time, sys.maxsize))] + if self.since <= time: + self.since = time + if not segs and self.state != STATE_RUNNING: + self.state = STATE_UNKNOWN
[Severity: Medium]
Does pruning a task whose state is STATE_BLOCKED or STATE_WAITING drop its current state and history?
TimechartData.checkpoint() only extends open segments for tasks in STATE_RUNNING, leaving STATE_BLOCKED and STATE_WAITING tasks unextended. While a task remains blocked or waiting, that state is not shown in the live timeline until a later event transitions the task, and once the prune window reaches self.since, segs becomes empty and Task.prune() resets self.state to STATE_UNKNOWN, allowing TimechartData.prune() to delete the task altogether.
[ ... ]
Quoted 10 lines
tools/perf/python/ttimechart.py +6
-0@@ -342,13 +500,13 @@ class TimechartData: def task(self, tid: int, comm: Optional[str] = None) -> Task: """Find or create a task.""" task = self.tasks.get(tid) + if not comm and self.session and (task is None or task.comm == f"[{tid}]"): + try: + thread = self.session.find_thread(tid, tid) + comm = thread.comm() if thread else None + except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError): + comm = None
tools/perf/python/ttimechart.py class TimechartData: · line 512
Quoted 10 lines
tools/perf/python/ttimechart.py +0
-6if task is None: - if not comm and self.session: - try: - thread = self.session.find_thread(tid, tid) - comm = thread.comm() if thread else None - except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError): - comm = None task = Task(tid, comm or f"[{tid}]") self.tasks[tid] = task else:
[Severity: Medium]
In live I/O-only mode, does this leave a task's pre-exec comm stuck after execve()?
Syscall tracepoints call self.task(tid) with comm=None. When a child process (such as a workload started by perf record with comm "perf-exec") performs an I/O syscall before execve(), self.session.find_thread() sets task.comm to the pre-exec name. After execve() emits PERF_RECORD_COMM, task.comm is neither None nor "[{tid}]", so self.task() never queries self.session.find_thread() again, and TimechartData.update_comms() is never called in live mode because self.progress is None.
[ ... ]
Quoted 9 lines
tools/perf/python/ttimechart.py +5
-0@@ -562,6 +752,32 @@ class TimechartData: except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError): pass + def checkpoint(self) -> None: + """Extend open running, idle and frequency states to last_time. + + Live mode calls this before displaying and pruning so that states that + haven't changed recently are shown up to the latest event, rather than
tools/perf/python/ttimechart.py class TimechartData: · line 769
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ only once they end. The lock must be held. + """ + end = self.last_time + if not end: + return + for task in self.tasks.values(): + if task.state == STATE_RUNNING and end > task.since: + task.segs.add(task.since, end, STATE_RUNNING, task.cpu) + task.totals[STATE_RUNNING] += end - task.since + task.since = end
[Severity: Low]
Can extending all running tasks to self.last_time cause Task.totals to drift when a subsequent flush delivers an earlier event on another CPU?
checkpoint() advances task.segs.ends[-1] and task.since to self.last_time for every running task. If the next round flushes an event for one of those tasks with sample_time < self.last_time, Task.change_state() sets self.since = time (which is now earlier than self.segs.ends[-1]). On the following state change, change_state() adds the unclipped time - self.since to self.totals, whereas self.segs.add() clips the segment start to self.segs.ends[-1]; when Task.prune() later removes that segment, it subtracts the clipped length from self.totals, leaking the difference.
[ ... ]
Quoted 11 lines
tools/perf/python/ttimechart.py +2
-1@@ -604,7 +820,8 @@ class TimechartData: if len(t.io) and t.passes_filter(filters)), key=lambda t: -len(t.io)) - def dump(self, power_only: bool, tasks_only: bool, filters: Sequence[str]) -> None: + def dump(self, power_only: bool, tasks_only: bool, filters: Sequence[str], + io_only: bool = False) -> None: """Print a plain text summary, for use without a terminal UI.""" duration = self.last_time - self.first_time span = max(duration, 1)
tools/perf/python/ttimechart.py class TimechartData: · line 832
Quoted 3 lines
tools/perf/python/ttimechart.py +0
-0@@ -613,7 +830,7 @@ class TimechartData: print(f"Duration: {fmt_duration(duration)}, CPUs: {len(self.cpus)}, " f"tasks: {len(tasks) or len(io_tasks)}, sched events: {self.sched_events}, " f"power events: {self.power_events}, I/O events: {self.io_events}")
[Severity: Low]
When io_only is true on a perf.data file that also contains scheduler events, shouldn't dump() and TimechartApp.update_views() ignore sched_tasks()?
In both dump() and TimechartApp.update_views(), tasks is still populated from data.sched_tasks(filters) when io_only is set. If a recorded file contains scheduler events as well as I/O events, -I reports the scheduler task count and fills the Summary tab with scheduler tasks instead of io_tasks, and pressing Enter on a task without I/O fails to jump to the hidden Tasks tab.
[ ... ]
Quoted 1 line
tools/perf/python/ttimechart.py +0
-0@@ -1486,6 +1743,218 @@ class TimechartApp(App):
[ ... ]
Quoted 31 lines
tools/perf/python/ttimechart.py +31
-0+ def recording_finished(self) -> None: + """Called on the UI thread when perf record exits, such as when the workload ends.""" + # Keep showing the window at the end of recording, or when paused, + # which is after the history already discarded. + self.finished_at = self.paused_at or monotonic_ns() + self.data.finish() + self.recording = False + self.paused_at = 0 + self.loaded = True + with self.data.lock: + # Discard history from events flushed when perf record exited. + self.live_prune() + self.refresh_bindings() + self.update_views() + self.update_live_status() + + def live_bounds(self) -> Tuple[int, int]: + """The time range kept and shown in live mode, the last window of the recording. + + Until a window of time has been recorded the range is the first window, + so the timeline fills from the left, after that it moves with time. + """ + assert self.live + if self.paused_at: + now = self.paused_at + elif self.recording: + now = monotonic_ns() + else: + now = self.finished_at + last = max(now, self.live.origin + self.live_window) + return max(self.live.origin, last - self.live_window), last
tools/perf/python/ttimechart.py class TimechartApp(App): · line 1828
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ + def live_prune(self) -> None: + """Discard history before the live window, the data's lock must be held. + + Open states are only extended up to the last event, see checkpoint(), + so history after it isn't discarded. Otherwise, when events haven't + been seen for a window, a still open state would lose its history. + """ + self.data.prune(min(self.live_bounds()[0], self.data.last_time)) + self.prune_rows()
[Severity: Medium]
What happens in live_prune() when no new events arrive for --window seconds so self.live_bounds()[0] reaches self.data.last_time?
TimechartData.checkpoint() and TimechartData.finish() only extend open segments up to self.last_time, and SegmentList.num_ending_by(time) uses bisect.bisect_right(self.ends, time), which selects all segments with end <= time. Once self.live_bounds()[0] >= self.data.last_time, clamping the prune time to self.data.last_time passes self.data.last_time to self.data.prune(), which deletes every segment in the trace (including open states just extended to self.data.last_time by checkpoint()). In recording_finished(), self.data.finish() also sets every task's state to STATE_UNKNOWN before self.live_prune() runs, so all tasks become empty and are deleted as well.
[ ... ]
Quoted 1 line
tools/perf/python/ttimechart.py +0
-0@@ -1726,52 +2210,642 @@ class TimechartApp(App):
[ ... ]
Quoted 1 line
+ def
pump(self, src: int, dst: int, output: Any)-> None:
[ ... ]
Quoted 19 lines
tools/perf/python/ttimechart.py +19
-0+ while True: + try: + buf = os.read(src, 1 << 16) + except BlockingIOError: + break + except OSError: + eof = True + break + if not buf: + eof = True + break + if output: + output.write(buf) + with self.round_lock: + tracker.feed(buf) + if forwarding and not self.stopping.is_set(): + # Keep draining src if dst is no longer read. + forwarding = self.write_pump(dst, buf) + if eof or not forwarding or self.stopping.is_set():
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2601
Quoted 10 lines
tools/perf/python/ttimechart.py +10
-0+ continue + flush = False + with self.round_lock: + target = self.flush_round_target + self.flush_round_target = None + if (target is not None and tracker.at_boundary() and + tracker.unflushed_round and + (target < 0 or tracker.rounds == target)): + tracker.unflushed_round = False + flush = True
[Severity: Low]
Can pump() clear self.flush_round_target before reading the data that control_loop() wanted to flush?
pump thread control thread
----------- --------------os.read(src) -> BlockingIOError
send("ping")
perf record writes round + FINISHED_ROUND perf record writes ack
with self.round_lock:
self.flush_round_target = -1
wake_pump()
with self.round_lock:
target = self.flush_round_target
self.flush_round_target = None <-- cleared before new data in src is read
On the next loop iteration pump() drains the new round from src (setting tracker.unflushed_round = True), but self.flush_round_target has already been reset to None, leaving the final round unflushed in ordered_events while paused. Snapshotting self.flush_round_target before draining src (and only clearing it if it wasn't updated in the meantime) would avoid losing the flush request.
[ ... ]
tools/perf/python/ttimechart.py class TimechartApp(App): · line 2681
Quoted 9 lines
tools/perf/python/ttimechart.py +9
-0+ def backlog(self) -> float: + """The fraction of the data pipe filled with data waiting to be processed.""" + try: + pending = array.array("i", [0]) + fcntl.ioctl(self.data_fd, termios.FIONREAD, pending) + size = fcntl.fcntl(self.data_fd, fcntl.F_GETPIPE_SZ) + except OSError: + return 0.0 + return pending[0] / max(size, 1)
[Severity: Low]
Can backlog() race with close_data_fds() when recording stops and raise an uncaught ValueError on the UI thread?
worker thread UI thread
------------- ---------record_events()
live.stop()
close_data_fds()
self.data_fd = -1
os.close(tee_r)
live_refreshed()
tune_refresh()
live.backlog()
fcntl.ioctl(-1, ...) <-- raises ValueError call_from_thread(recording_finished)
self.recording = False
In Python, passing -1 to fcntl.ioctl() or fcntl.fcntl() raises ValueError rather than OSError, which is not caught here. In addition, if close_data_fds() closes tee_r right after backlog() reads self.data_fd, a concurrent open() (such as restore_own_comm() opening /proc/self/comm) can reuse that file descriptor number before ioctl() runs.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v7 2/2 perf test: Test perf timechart --live
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Extend test_ttimechart_python.sh to cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file when system-wide recording is permitted, a synthetic live UI test exercising the rolling window, pruning, pause/resume gaps, round-flush framing and zoom/cursor following, and a live workload recording test when scheduler events can be recorded system wide.
--- .../tests/shell/test_ttimechart_python.sh | 436 ++++++++++++++++++ 1 file changed, 436 insertions(+)
tools/perf/tests/shell/test_ttimechart_python.sh +436 -0
@@ -24,6 +24,7 @@ if ! "$PYTHON" -c 'import textual' > /dev/null 2>&1; then fi err=0 +system_wide=0 perfdata= output= clockdata=
@@ -57,6 +58,7 @@ record() { # fall back to recording just the workload's scheduler events. if perf timechart record -o "${perfdata}" -- \ bash -c "$(declare -f workload); workload" > /dev/null 2>&1; then + system_wide=1 return 0 fi if perf record -R -c 1 -e sched:sched_switch -e sched:sched_wakeup \
@@ -82,6 +84,34 @@ test_plumbing() { err=1 return fi + if perf timechart --live -i "${perfdata}" > "${output}" 2>&1 || + ! grep -q -- "--live can't be used with -i" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --live with -i not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --live --window 0 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "--window must be a positive" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window 0 not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --tui --window 5 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "require --live" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window without --live]" + cat "${output}" + err=1 + return + fi + if perf timechart --live -I -P < /dev/null > "${output}" 2>&1 || + ! grep -q -- "-I cannot be used with -P or -T" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: -I with -P not rejected]" + cat "${output}" + err=1 + return + fi echo "perf timechart --tui plumbing test [Success]" }
@@ -115,6 +145,45 @@ test_no_events() { err=1 return fi + # Exercise LiveRecorder's perf record subprocess, control pipe and + # -o output file using task-clock when system-wide recording is allowed. + if perf record -a -e task-clock -o "${clockdata}" -- true > /dev/null 2>&1; then + rm -f "${clockdata}" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${clockdata}" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import os +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.events = lambda: ["task-clock"] + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + await pilot.press("space") + await pilot.pause(0.1) + await pilot.press("space") + for _ in range(300): + if app.return_code is not None: + break + await pilot.pause(0.05) + finally: + rec.stop() + if app.return_code != 1 or not app.data.nr_samples or not os.path.getsize(sys.argv[2]): + raise RuntimeError(f"Unexpected result: rc={app.return_code} samples={app.data.nr_samples}") + +asyncio.run(run()) +EOF + then + echo "ttimechart no events test [Failed: LiveRecorder task-clock test]" + cat "${output}" + err=1 + return + fi + fi echo "ttimechart no events test [Success]" }
@@ -218,8 +287,374 @@ EOF echo "ttimechart headless UI test [Success]" } +test_live_synthetic() { + echo "ttimechart live UI test (synthetic)" + # Exercise live mode's rolling window, pruning, pause/resume and + # following with a fake recorder, which doesn't need tracepoint + # permissions. + if ! "$PYTHON" - > "${output}" 2>&1 <<'EOF' +import asyncio +import threading +import time +from time import monotonic_ns +import ttimechart as tc + +SEC = 1_000_000_000 + +class Ev: + def __init__(self, name): + self.name = name + def __str__(self): + return f"evsel({self.name})" + +class Sample: + def __init__(self, name, t, cpu, tid, **kw): + self.evsel = Ev(name) + self.sample_time = t + self.sample_cpu = cpu + self.sample_tid = tid + self.sample_pid = tid + self.sample_id = hash(name) + for k, v in kw.items(): + setattr(self, k, v) + +class FakeRecorder: + def __init__(self): + self.origin = monotonic_ns() + self.data_fd = -1 + self.flush_interval = 0.05 + self.paused = False + self.pause_calls = [] + self.stopped = threading.Event() + self.finish = False + + def backlog(self): + return 0.0 + + def set_paused(self, paused): + self.paused = paused + self.pause_calls.append(paused) + + def stop(self): + self.stopped.set() + + def error_text(self): + return "" + +async def run() -> None: + rec = FakeRecorder() + + def fake_read(data, _fd): + t0 = rec.origin + # Task 300 only runs at the start so pruning should drop it once the + # 1s window advances past t0 + 3000. + data.process_event(Sample("sched:sched_wakeup", t0 + 1000, 0, 1, + pid=300, comm="early", common_pid=1, common_flags=0)) + data.process_event(Sample("sched:sched_switch", t0 + 2000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=300, next_comm="early")) + data.process_event(Sample("sched:sched_switch", t0 + 3000, 0, 300, + prev_pid=300, prev_comm="early", prev_state=1, + next_pid=0, next_comm="swapper")) + cur = 0 + while not rec.stopped.is_set() and not rec.finish: + now = monotonic_ns() + if rec.paused: + time.sleep(0.01) + continue + nxt = 100 if cur == 0 else 0 + comm = "worker" if nxt == 100 else "swapper" + prev_comm = "worker" if cur == 100 else "swapper" + if nxt: + data.process_event(Sample("sched:sched_wakeup", now, 0, cur or 1, + pid=nxt, comm=comm, common_pid=cur or 1, + common_flags=0)) + data.process_event(Sample("sched:sched_switch", now, 0, cur, + prev_pid=cur, prev_comm=prev_comm, prev_state=1, + next_pid=nxt, next_comm=comm)) + cur = nxt + time.sleep(0.005) + + tc.read_events = fake_read + app = tc.TimechartApp("live", False, False, [], live=rec, window=1.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(300): + with app.data.lock: + if app.data.pruned_to > rec.origin + 3000: + break + await pilot.pause(0.05) + with app.data.lock: + if app.data.pruned_to <= rec.origin + 3000 or 300 in app.data.tasks: + raise RuntimeError("Task outside the window was not pruned") + starts = [t.segs.starts[0] for t in app.data.tasks.values() if len(t.segs)] + if not starts or min(starts) < app.data.pruned_to: + raise RuntimeError(f"Unpruned segments: {starts} < {app.data.pruned_to}") + if app.window.last - app.window.first != SEC: + raise RuntimeError(f"Unexpected window span: {app.window}") + + await pilot.press("space") + await pilot.pause(0.2) + if rec.pause_calls != [True] or "Paused" not in app.sub_title: + raise RuntimeError(f"Did not pause: {rec.pause_calls} {app.sub_title}") + frozen = app.window.last + await pilot.pause(0.4) + if app.window.last != frozen: + raise RuntimeError("Window moved while paused") + await pilot.press("space") + await pilot.pause(0.3) + if rec.pause_calls != [True, False] or not app.following: + raise RuntimeError("Did not resume") + + await pilot.press("plus") + await pilot.pause(0.2) + if app.following or app.follow_cursor: + raise RuntimeError("Zooming did not stop following") + await pilot.press("0") + await pilot.pause(0.2) + if not app.following or not app.follow_cursor: + raise RuntimeError("Resetting zoom did not resume following") + await pilot.press("left") + await pilot.pause(0.1) + if not app.following or app.follow_cursor: + raise RuntimeError("Moving cursor did not stop cursor following") + await pilot.press("0") + await pilot.pause(0.1) + if not app.follow_cursor: + raise RuntimeError("Resetting zoom did not resume cursor following") + + rec.finish = True + for _ in range(100): + if not app.recording: + break + await pilot.pause(0.05) + if app.recording or "Finished" not in app.sub_title: + raise RuntimeError(f"Did not finish: {app.sub_title}") + finally: + rec.stop() + + # Multiple pause/resume cycles before buffered events arrive must still + # close states at each unrecorded gap, and pipe round tracking must detect + # unflushed rounds across tracing metadata. + gap_data = tc.TimechartData() + gap_data.process_event(Sample("sched:sched_switch", 1000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=50, next_comm="runner")) + with gap_data.lock: + gap_data.mark_gap(2000) + gap_data.mark_gap(4000) + gap_data.process_event(Sample("sched:sched_switch", 2500, 0, 50, + prev_pid=50, prev_comm="runner", prev_state=1, + next_pid=0, next_comm="swapper")) + gap_data.process_event(Sample("sched:sched_switch", 4500, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=50, next_comm="runner")) + gap_data.finish() + if list(zip(gap_data.tasks[50].segs.starts, gap_data.tasks[50].segs.ends)) != [(2000, 2500)]: + raise RuntimeError(f"Unexpected gap segments: {gap_data.tasks[50].segs.starts}") + + tracker = tc.PerfPipeTracker() + pipe_hdr = tc.struct.pack("=QQ", tracker.PERF_MAGIC, 16) + td_rec = tc.struct.pack("=IHHI", tracker.HEADER_TRACING_DATA, 0, 12, 8) + b"12345678" + sample_rec = tc.struct.pack("=IHHQ", 9, 0, 16, 0) + # Feed byte-by-byte to exercise split headers. + for b in pipe_hdr + td_rec + sample_rec + tracker.FINISHED_ROUND_EVENT: + tracker.feed(bytes([b])) + if not tracker.at_boundary() or not tracker.unflushed_round or tracker.rounds != 1: + raise RuntimeError("PerfPipeTracker failed to track unflushed round") + tracker.feed(tracker.FINISHED_ROUND_EVENT) + if tracker.unflushed_round or tracker.rounds != 2: + raise RuntimeError("PerfPipeTracker failed to clear unflushed round") + + # Also test live I/O-only mode (-I), including the shell filter quoting. + io_events = tc.LiveRecorder("perf", False, False, [], None, io_only=True).events() + if not io_events or not all(e.startswith("syscalls:sys_") for e in io_events): + raise RuntimeError(f"Unexpected I/O events: {io_events}") + orig_comm = open("/proc/self/comm", encoding="utf-8").read().strip() + with tc.tempfile.NamedTemporaryFile("w", delete=False) as fake_perf: + fake_perf.write(f"#!{tc.sys.executable}\n" + "import json, os, sys\n" + "sys.stderr.write(json.dumps([os.getpid(), sys.argv[1:]]) + '\\n')\n") + fake_perf_path = fake_perf.name + try: + tc.os.chmod(fake_perf_path, 0o755) + cmd_rec = tc.LiveRecorder(fake_perf_path, False, False, ["echo", "a b"], None, + io_only=True) + cmd_rec.start() + cmd_rec.proc.wait(timeout=5) + cmd_rec.stop() + import json + exec_pid, exec_argv = json.loads(cmd_rec.error_text()) + expected_filt = (f'comm != "ttimechart:{tc.os.getpid() & 0xffff:x}" && ' + f"common_pid != {exec_pid}") + filt_idx = exec_argv.index("--filter") + if (exec_pid != cmd_rec.proc.pid or exec_argv[filt_idx + 1] != expected_filt or + exec_argv[-3:] != ["--", "echo", "a b"]): + raise RuntimeError(f"Unexpected I/O perf record argv: {exec_pid} {exec_argv}") + if open("/proc/self/comm", encoding="utf-8").read().strip() != orig_comm: + raise RuntimeError("Process comm was not restored after stop()") + # Verify start() cleans up if starting control_thread fails. + real_start = threading.Thread.start + calls = 0 + def fail_second_thread(self_t): + nonlocal calls + calls += 1 + if calls == 2: + raise RuntimeError("simulated thread start failure") + real_start(self_t) + threading.Thread.start = fail_second_thread + try: + fail_rec = tc.LiveRecorder(fake_perf_path, False, False, [], None, io_only=True) + try: + fail_rec.start() + raise RuntimeError("Expected start() to fail") + except RuntimeError as e: + if "simulated thread start failure" not in str(e): + raise + finally: + threading.Thread.start = real_start + if (fail_rec.ctl_fd != -1 or fail_rec.wake_r != -1 or fail_rec.tee_r != -1 or + open("/proc/self/comm", encoding="utf-8").read().strip() != orig_comm): + raise RuntimeError("Failed start() left unclosed fds or unmended comm") + finally: + tc.os.unlink(fake_perf_path) + io_rec = FakeRecorder() + + def fake_io_read(data, _fd): + now = monotonic_ns() + # Self I/O should be ignored in live mode without counting as samples. + own = tc.os.getpid() + data.process_event(Sample("syscalls:sys_enter_read", now, 0, own, fd=3)) + data.process_event(Sample("syscalls:sys_exit_read", now + 1000, 0, own, ret=64)) + # Workload I/O should be recorded. + data.process_event(Sample("syscalls:sys_enter_read", now + 2000, 0, 4242, fd=4)) + data.process_event(Sample("syscalls:sys_exit_read", now + 5000, 0, 4242, ret=4096)) + data.process_event(Sample("syscalls:sys_enter_write", now + 6000, 0, 4242, fd=5)) + data.process_event(Sample("syscalls:sys_exit_write", now + 9000, 0, 4242, ret=2048)) + + tc.read_events = fake_io_read + io_app = tc.TimechartApp("live", False, False, [], live=io_rec, window=1.0, io_only=True) + async with io_app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + if not io_app.recording: + break + await pilot.pause(0.05) + await pilot.pause() + tabs = io_app.query_one(tc.TabbedContent) + if (tabs.active != "io" or not io_app.io_tasks or io_app.data.io_events != 4 or + io_app.data.nr_samples != 4): + raise RuntimeError(f"Unexpected I/O state: active={tabs.active} " + f"tasks={len(io_app.io_tasks)} events={io_app.data.io_events} " + f"samples={io_app.data.nr_samples}") + tabs.active = "summary" + await pilot.pause() + await pilot.press("enter") + await pilot.pause() + if tabs.active != "io": + raise RuntimeError(f"Summary enter did not activate I/O tab: {tabs.active}") + print("live synthetic ok") + +asyncio.run(run()) +EOF + then + echo "ttimechart live UI test (synthetic) [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "live synthetic ok" "${output}"; then + echo "ttimechart live UI test (synthetic) [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live UI test (synthetic) [Success]" +} + +test_live_record() { + echo "ttimechart live record test" + if [ "${system_wide}" != 1 ]; then + # Live mode records system wide, like perf timechart record. + echo "ttimechart live record test [Skipped: can't record system wide]" + return + fi + rm -f "${perfdata}.live" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${perfdata}.live" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if not app.recording: + break + await pilot.pause(0.1) + if app.recording or not app.data.has_events(): + raise RuntimeError("Live recording of workload did not complete") + print("live record ok") + finally: + rec.stop() + +asyncio.run(run()) +EOF + then + echo "ttimechart live record test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! perf script ttimechart -i "${perfdata}.live" --dump > "${output}" 2>&1 || + ! grep -q "^Duration: .*sched events: [1-9]" "${output}"; then + echo "ttimechart live record test [Failed: saved perf.data unreadable]" + cat "${output}" + err=1 + return + fi + if perf timechart record -I -o "${perfdata}.live" -- true > /dev/null 2>&1; then + rm -f "${perfdata}.live" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${perfdata}.live" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, + ["bash", "-c", "cat /etc/hostname > /dev/null"], + sys.argv[2], io_only=True) + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0, io_only=True) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if not app.recording: + break + await pilot.pause(0.1) + if app.recording or not app.data.io_events: + raise RuntimeError("Live I/O recording of workload did not complete") + finally: + rec.stop() + +asyncio.run(run()) +EOF + then + echo "ttimechart live record test [Failed: live -I exception]" + cat "${output}" + err=1 + return + fi + fi + echo "ttimechart live record test [Success]" +} + test_plumbing test_no_events +test_live_synthetic if ! record; then echo "Skipping remaining tests, failed to record scheduler events (permissions?)"
@@ -232,6 +667,7 @@ fi test_dump test_headless_ui +test_live_record cleanup exit $err
Signature · 1 line
2.56.1.703.gf848ed807c-goog
[email protected]
AI reviewRe: [PATCH v7 2/2] perf test: Test perf timechart --live
Sashiko has reviewed this patch and found no issues.
Quoted 9 lines
From: Alice Rogers <[email protected]>
Extend
test_ttimechart_python.shto cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file when system-wide recording is permitted, a synthetic live UI test exercising the rolling window, pruning, pause/resume gaps, round-flush framing and zoom/cursor following, and a live workload recording test when scheduler events can be recorded system wide.
Quoted 3 lines
Signed-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerv8 cover perf timechart: Add --live mode to the TUI and test
In reply to Ian RogersAdd a '--live' option to 'perf timechart' and 'ttimechart.py' that streams live scheduler, power, or I/O (-I/--io-only) events from a 'perf record' subprocess over a pipe and displays them in the Textual TUI with a rolling time window (--window, default 10s), pause/resume via the control pipe, an optional workload after '--' that stops recording when it exits, and optional output recording (-o), along with headless shell tests in test_ttimechart_python.sh.
v8: Address Sashiko review feedback:
- Patch 1: Track switch-out timestamps explicitly on Task rather than inferring switch counts from pruned STATE_RUNNING segments, and accumulate Task.totals from the actual clipped duration returned by SegmentList.add() while keeping Task.since monotonic.
- Patch 1: Keep each I/O syscall as its own unmerged segment and avoid same-timestamp clipping so Task.io_bytes and syscall counts never drift across pruning.
- Patch 1: Extend open STATE_WAITING and STATE_BLOCKED task states in checkpoint(), preserve non-sleeping states across prune(), and clamp live_prune() to last_time - 1 so open or finished states at last_time are not deleted when idle past --window.
- Patch 1: Refresh comm names from perf.session on syscall lookups and periodic live intervals (ignoring perf's unset ':tid' placeholders) so pre-exec names update after execve(), and ignore scheduler-only tasks in dump() and the UI when -I is set.
- Patch 1: Snapshot flush_round_target with a sequence counter before draining perf record's pipe so concurrent or split-record flushes are not dropped, and synchronize LiveRecorder.backlog() with close_data_fds() under fd_lock.
- Patch 2: Add synthetic regression tests for all cases.
v7: Address Sashiko review feedback:
- Patch 1: Pass the comm/PID filter expression as a positional argument to /bin/sh so the shell preserves the quotes around comm in --live -I mode instead of stripping them before execing perf record.
- Patch 1: Clear LiveRecorder.control_thread if starting it raises so stop() does not call join() on an unstarted thread and skip FD/comm cleanup, and ignore OSError if perf record exits concurrently during stop().
- Patch 2: Test the /bin/sh --filter quoting and control_thread startup failure cleanup in the synthetic live test, and add a live -I workload recording subtest when syscall tracepoints are permitted.
v6: https://lore.kernel.org/lkml/[email protected]/
v6: Rebase onto perf-tools-next and address Sashiko review feedback:
- Patch 1: Queue pause/resume gap timestamps in a deque so rapid pause/resume cycles before buffered events arrive still close
pre-pause open task and CPU states at each gap.
- Patch 1: Increment nr_samples inside TimechartData.lock and check/update refreshed_samples in the same lock acquisition as checkpoint() in live_refresh() so final buffered events aren't missed when paused.
- Patch 1: Keep cursor following in sync when zooming in/out, resetting zoom with '0' after moving the cursor, or jumping to a waker ('w').
- Patch 1: Filter out all threads of ttimechart.py (via comm) and the child perf record process (via $$) on syscall tracepoints in --live -I mode, and ignore filtered I/O events before counting nr_samples.
- Patch 1: Track record boundaries in LiveRecorder's pipe pump and emit a synthetic PERF_RECORD_FINISHED_ROUND when perf record goes idle or pauses so ordered_events does not hold back the latest round.
- Patch 1: Always drain perf record's stdout on stop so perf record can flush and terminate its workload cleanly without blocking on pipe writes, and kill its process group if wait times out.
- Patch 2: Gate the LiveRecorder task-clock subtest on system-wide task-clock permissions so unprivileged runs under perf_event_paranoid > 0 skip cleanly, and add synthetic tests for multiple pause gaps, round tracking, and cursor following.
v5: https://lore.kernel.org/linux-perf-users/[email protected]/
v5: Address Sashiko review feedback:
- Patch 1: Always record system wide, as 'perf timechart record'
does, as switches into a task and idle states aren't seen by
per-task events when only a workload was recorded.
- Patch 1: Don't move a task's state start back before its recorded history when its switch out and back in were lost, which inflated
its running time.
- Patch 1: Serialize LiveRecorder.stop() calls from the UI thread and
the thread processing events.
- Patch 2: Only run the live recording test when scheduler events can
be recorded system wide.
v4: https://lore.kernel.org/linux-perf-users/[email protected]/
v4: Address Sashiko review feedback on patch 1:
- Don't prune live history past the last event, so open states aren't
lost when no events arrive for a window, and don't count a still
running task's open segment as a switch.
- Keep showing the window at the end of recording, rather than jumping back to already pruned history, and prune events flushed as perf
record exits.
- Close open task, CPU and I/O states when resuming after a pause,
rather than extending them across the unrecorded gap, and close the running task when a switch away from it was lost.
- Stop perf record and close the files and pipes if
LiveRecorder.start() fails after starting it, and close the data pipe read ends in LiveRecorder.stop().
v3: https://lore.kernel.org/linux-perf-users/[email protected]/
v3: Address Sashiko review feedback:
- Patch 1: Explicitly include <string.h> in builtin-timechart.c.
- Patch 1: Reject non-finite --window values (e.g. 'inf') that
overflowed converting to nanoseconds.
- Patch 2: Stop the fake recorder in a try/finally in the synthetic
live UI test so a failed check can't leave its background thread
running and hang the test.
Rebase onto the latest perf-tools-next.
v2: https://lore.kernel.org/linux-perf-users/[email protected]/
v2: Rebase the remaining 2 patches onto perf-tools-next after patches 1-11
of v1 were merged.
Alice Rogers (2):
perf timechart: Add a --live mode to the TUI
perf test: Test perf timechart --live
tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 80 +- tools/perf/python/ttimechart.py | 1334 +++++++++++++++-- .../tests/shell/test_ttimechart_python.sh | 591 ++++++++ 4 files changed, 1899 insertions(+), 126 deletions(-)
Signature · 1 line
2.56.1.703.gf848ed807c-goog
Ian Rogers
reviewerPATCH v8 1/2 perf timechart: Add a --live mode to the TUI
From: Alice Rogers <[email protected]>
Add a --live option to perf timechart and ttimechart.py that runs 'perf record' writing CLOCK_MONOTONIC scheduler and power events, or I/O syscall events when -I/--io-only is given, to a pipe and displays them in the TUI as they happen. Only the most recent --window seconds of history (default 10) are kept in memory, pruning older segments and tasks that have no remaining history in the window.
Pressing space pauses and resumes recording using perf record's --control pipe. Because perf record only reads its ring buffers when a watermark is reached, periodic 'ping' commands are sent on the control pipe to flush events promptly, and a synthetic finished-round event is injected on record boundaries when perf record goes idle or pauses so ordered-event buffering does not hold onto the latest round. The refresh interval adapts to the measured screen refresh cost and backs off if unprocessed data builds up in the pipe.
Like 'perf timechart record', the whole system is recorded, as switches into a task and idle states aren't seen by per-task events. An optional workload may be given after '--' to stop recording when it exits, and '-o' optionally saves a copy of the recorded perf.data stream.
--- tools/perf/Documentation/perf-timechart.txt | 20 +- tools/perf/builtin-timechart.c | 80 +- tools/perf/python/ttimechart.py | 1334 +++++++++++++++++-- 3 files changed, 1308 insertions(+), 126 deletions(-)
tools/perf/Documentation/perf-timechart.txt +18 -2
@@ -47,6 +47,9 @@ TIMECHART OPTIONS -T:: --tasks-only:: Don't output processor state transitions +-I:: +--io-only:: + In '--live' or '--tui' mode, only record or show I/O activity -p:: --process:: Select the processes to display, by name or PID
@@ -59,8 +62,21 @@ TIMECHART OPTIONS linkperf:perf-script[1]). The script requires the perf python module and the python 'textual' library. The CPU, task, I/O and summary views can be zoomed ('+'/'-'), panned (shift+arrows) and the state of the - selected row at the cursor is described. The -i, -P, -T and -p options - are passed to the script, other output options are ignored. + selected row at the cursor is described. The -i, -P, -T, -I and -p + options are passed to the script, other output options are ignored. +--live:: + Record scheduler and power events, or I/O events with '-I', with + linkperf:perf-record[1] and display them as they happen in the + terminal timechart, implying --tui. Only the most recent '--window' + seconds of events are kept and shown. Pressing space pauses and + resumes recording; zooming or panning stops the view following the + latest events until '0' is pressed. The whole system is recorded, if a + command is given after '--' then recording stops when it exits, + otherwise it continues until the UI is quit. If '-o' + is given the recorded perf data is also saved to that file. +--window=<seconds>:: + In '--live' mode, the number of seconds of recent events kept in + memory and shown in the timeline (default: 10). --symfs=<directory[,layout]>:: Look for files with symbols relative to this directory. The optional layout can be 'hierarchy' (default, matches full path) or 'flat'
tools/perf/builtin-timechart.c +70 -10
159 lines not on this page · raw message
tools/perf/python/ttimechart.py +1220 -114
1,703 lines not on this page · raw message
Signature · 1 line
2.56.1.703.gf848ed807c-goog
[email protected]
AI reviewRe: [PATCH v8 1/2] perf timechart: Add a --live mode to the TUI
Sashiko has reviewed this patch and found no issues.
Quoted 18 lines
From: Alice Rogers <[email protected]>
Add a --live option to perf timechart and
ttimechart.pythat runs 'perf record' writingCLOCK_MONOTONICscheduler and power events, or I/O syscall events when -I/--io-only is given, to a pipe and displays them in the TUI as they happen. Only the most recent --window seconds of history (default 10) are kept in memory, pruning older segments and tasks that have no remaining history in the window.Pressing space pauses and resumes recording using perf record's --control pipe. Because perf record only reads its ring buffers when a watermark is reached, periodic 'ping' commands are sent on the control pipe to flush events promptly, and a synthetic finished-round event is injected on record boundaries when perf record goes idle or pauses so ordered-event buffering does not hold onto the latest round. The refresh interval adapts to the measured screen refresh cost and backs off if unprocessed data builds up in the pipe.
Quoted 8 lines
Like 'perf timechart record', the whole system is recorded, as switches into a task and idle states aren't seen by per-task events. An optional workload may be given after '--' to stop recording when it exits, and '-o' optionally saves a copy of the recorded perf.data stream.
Signed-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v8 2/2 perf test: Test perf timechart --live
In reply to Ian RogersFrom: Alice Rogers <[email protected]>
Extend test_ttimechart_python.sh to cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file when system-wide recording is permitted, a synthetic live UI test exercising the rolling window, pruning, pause/resume gaps, round-flush framing and zoom/cursor following, and a live workload recording test when scheduler events can be recorded system wide.
--- .../tests/shell/test_ttimechart_python.sh | 591 ++++++++++++++++++ 1 file changed, 591 insertions(+)
tools/perf/tests/shell/test_ttimechart_python.sh +591 -0
@@ -24,6 +24,7 @@ if ! "$PYTHON" -c 'import textual' > /dev/null 2>&1; then fi err=0 +system_wide=0 perfdata= output= clockdata=
@@ -57,6 +58,7 @@ record() { # fall back to recording just the workload's scheduler events. if perf timechart record -o "${perfdata}" -- \ bash -c "$(declare -f workload); workload" > /dev/null 2>&1; then + system_wide=1 return 0 fi if perf record -R -c 1 -e sched:sched_switch -e sched:sched_wakeup \
@@ -82,6 +84,34 @@ test_plumbing() { err=1 return fi + if perf timechart --live -i "${perfdata}" > "${output}" 2>&1 || + ! grep -q -- "--live can't be used with -i" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --live with -i not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --live --window 0 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "--window must be a positive" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window 0 not rejected]" + cat "${output}" + err=1 + return + fi + if perf timechart --tui --window 5 < /dev/null > "${output}" 2>&1 || + ! grep -q -- "require --live" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: --window without --live]" + cat "${output}" + err=1 + return + fi + if perf timechart --live -I -P < /dev/null > "${output}" 2>&1 || + ! grep -q -- "-I cannot be used with -P or -T" "${output}"; then + echo "perf timechart --tui plumbing test [Failed: -I with -P not rejected]" + cat "${output}" + err=1 + return + fi echo "perf timechart --tui plumbing test [Success]" }
@@ -115,6 +145,45 @@ test_no_events() { err=1 return fi + # Exercise LiveRecorder's perf record subprocess, control pipe and + # -o output file using task-clock when system-wide recording is allowed. + if perf record -a -e task-clock -o "${clockdata}" -- true > /dev/null 2>&1; then + rm -f "${clockdata}" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${clockdata}" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import os +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.events = lambda: ["task-clock"] + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + await pilot.press("space") + await pilot.pause(0.1) + await pilot.press("space") + for _ in range(300): + if app.return_code is not None: + break + await pilot.pause(0.05) + finally: + rec.stop() + if app.return_code != 1 or not app.data.nr_samples or not os.path.getsize(sys.argv[2]): + raise RuntimeError(f"Unexpected result: rc={app.return_code} samples={app.data.nr_samples}") + +asyncio.run(run()) +EOF + then + echo "ttimechart no events test [Failed: LiveRecorder task-clock test]" + cat "${output}" + err=1 + return + fi + fi echo "ttimechart no events test [Success]" }
@@ -218,8 +287,529 @@ EOF echo "ttimechart headless UI test [Success]" } +test_live_synthetic() { + echo "ttimechart live UI test (synthetic)" + # Exercise live mode's rolling window, pruning, pause/resume and + # following with a fake recorder, which doesn't need tracepoint + # permissions. + if ! "$PYTHON" - > "${output}" 2>&1 <<'EOF' +import asyncio +import threading +import time +from time import monotonic_ns +import ttimechart as tc + +SEC = 1_000_000_000 + +class Ev: + def __init__(self, name): + self.name = name + def __str__(self): + return f"evsel({self.name})" + +class Sample: + def __init__(self, name, t, cpu, tid, **kw): + self.evsel = Ev(name) + self.sample_time = t + self.sample_cpu = cpu + self.sample_tid = tid + self.sample_pid = tid + self.sample_id = hash(name) + for k, v in kw.items(): + setattr(self, k, v) + +class FakeRecorder: + def __init__(self): + self.origin = monotonic_ns() + self.data_fd = -1 + self.flush_interval = 0.05 + self.paused = False + self.pause_calls = [] + self.stopped = threading.Event() + self.finish = False + + def backlog(self): + return 0.0 + + def set_paused(self, paused): + self.paused = paused + self.pause_calls.append(paused) + + def stop(self): + self.stopped.set() + + def error_text(self): + return "" + +async def run() -> None: + rec = FakeRecorder() + + def fake_read(data, _fd): + t0 = rec.origin + # Task 300 only runs at the start so pruning should drop it once the + # 1s window advances past t0 + 3000. + data.process_event(Sample("sched:sched_wakeup", t0 + 1000, 0, 1, + pid=300, comm="early", common_pid=1, common_flags=0)) + data.process_event(Sample("sched:sched_switch", t0 + 2000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=300, next_comm="early")) + data.process_event(Sample("sched:sched_switch", t0 + 3000, 0, 300, + prev_pid=300, prev_comm="early", prev_state=1, + next_pid=0, next_comm="swapper")) + cur = 0 + while not rec.stopped.is_set() and not rec.finish: + now = monotonic_ns() + if rec.paused: + time.sleep(0.01) + continue + nxt = 100 if cur == 0 else 0 + comm = "worker" if nxt == 100 else "swapper" + prev_comm = "worker" if cur == 100 else "swapper" + if nxt: + data.process_event(Sample("sched:sched_wakeup", now, 0, cur or 1, + pid=nxt, comm=comm, common_pid=cur or 1, + common_flags=0)) + data.process_event(Sample("sched:sched_switch", now, 0, cur, + prev_pid=cur, prev_comm=prev_comm, prev_state=1, + next_pid=nxt, next_comm=comm)) + cur = nxt + time.sleep(0.005) + + tc.read_events = fake_read + app = tc.TimechartApp("live", False, False, [], live=rec, window=1.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(300): + with app.data.lock: + if app.data.pruned_to > rec.origin + 3000: + break + await pilot.pause(0.05) + with app.data.lock: + if app.data.pruned_to <= rec.origin + 3000 or 300 in app.data.tasks: + raise RuntimeError("Task outside the window was not pruned") + starts = [t.segs.starts[0] for t in app.data.tasks.values() if len(t.segs)] + if not starts or min(starts) < app.data.pruned_to: + raise RuntimeError(f"Unpruned segments: {starts} < {app.data.pruned_to}") + if app.window.last - app.window.first != SEC: + raise RuntimeError(f"Unexpected window span: {app.window}") + + await pilot.press("space") + await pilot.pause(0.2) + if rec.pause_calls != [True] or "Paused" not in app.sub_title: + raise RuntimeError(f"Did not pause: {rec.pause_calls} {app.sub_title}") + frozen = app.window.last + await pilot.pause(0.4) + if app.window.last != frozen: + raise RuntimeError("Window moved while paused") + await pilot.press("space") + await pilot.pause(0.3) + if rec.pause_calls != [True, False] or not app.following: + raise RuntimeError("Did not resume") + + await pilot.press("plus") + await pilot.pause(0.2) + if app.following or app.follow_cursor: + raise RuntimeError("Zooming did not stop following") + await pilot.press("0") + await pilot.pause(0.2) + if not app.following or not app.follow_cursor: + raise RuntimeError("Resetting zoom did not resume following") + await pilot.press("left") + await pilot.pause(0.1) + if not app.following or app.follow_cursor: + raise RuntimeError("Moving cursor did not stop cursor following") + await pilot.press("0") + await pilot.pause(0.1) + if not app.follow_cursor: + raise RuntimeError("Resetting zoom did not resume cursor following") + + rec.finish = True + for _ in range(100): + if not app.recording: + break + await pilot.pause(0.05) + if app.recording or "Finished" not in app.sub_title: + raise RuntimeError(f"Did not finish: {app.sub_title}") + finally: + rec.stop() + + # Multiple pause/resume cycles before buffered events arrive must still + # close states at each unrecorded gap, and pipe round tracking must detect + # unflushed rounds across tracing metadata. + gap_data = tc.TimechartData() + gap_data.process_event(Sample("sched:sched_switch", 1000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=50, next_comm="runner")) + with gap_data.lock: + gap_data.mark_gap(2000) + gap_data.mark_gap(4000) + gap_data.process_event(Sample("sched:sched_switch", 2500, 0, 50, + prev_pid=50, prev_comm="runner", prev_state=1, + next_pid=0, next_comm="swapper")) + gap_data.process_event(Sample("sched:sched_switch", 4500, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=50, next_comm="runner")) + gap_data.finish() + if list(zip(gap_data.tasks[50].segs.starts, gap_data.tasks[50].segs.ends)) != [(2000, 2500)]: + raise RuntimeError(f"Unexpected gap segments: {gap_data.tasks[50].segs.starts}") + gap_data.prune(2000) + if gap_data.tasks[50].switches != 1: + raise RuntimeError(f"Gap prune corrupted switches: {gap_data.tasks[50].switches}") + gap_data.prune(2500) + if 50 in gap_data.tasks: + raise RuntimeError("Pruned task 50 should be empty") + + # Verify totals, switches, blocked/waiting checkpoints, out-of-order flushes, + # unmerged I/O syscalls, and comm updates across exec. + inv = tc.TimechartData() + cur_comm = "perf-exec" + class FakeThread: + def comm(self): + return cur_comm + class FakeSession: + def find_thread(self, _tid): + return FakeThread() + inv.session = FakeSession() + inv.process_event(Sample("sched:sched_switch", 1000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=10, next_comm="blocker")) + # Task 10 blocks at 2000; task 11 runs to 5000 so checkpoint extends task 10. + inv.process_event(Sample("sched:sched_switch", 2000, 0, 10, + prev_pid=10, prev_comm="blocker", prev_state=2, + next_pid=11, next_comm="other")) + inv.process_event(Sample("sched:sched_wakeup", 5000, 1, 11, + pid=12, comm="waiter", common_pid=11, common_flags=0)) + with inv.lock: + inv.checkpoint() + inv.prune(3000) + if (inv.tasks[10].state != tc.STATE_BLOCKED or + inv.tasks[10].totals[tc.STATE_BLOCKED] != 2000 or + inv.tasks[12].state != tc.STATE_WAITING): + raise RuntimeError(f"Blocked/waiting state lost: {inv.tasks[10].totals}") + # Earlier event on another CPU after checkpoint must not leak totals. + inv.process_event(Sample("sched:sched_switch", 4800, 0, 11, + prev_pid=11, prev_comm="other", prev_state=0, + next_pid=0, next_comm="swapper")) + inv.process_event(Sample("sched:sched_switch", 6000, 0, 11, + prev_pid=11, prev_comm="other", prev_state=1, + next_pid=0, next_comm="swapper")) + with inv.lock: + inv.prune(6000) + if 11 in inv.tasks: + raise RuntimeError(f"Leaked totals or switches: {inv.tasks[11].totals} " + f"{inv.tasks[11].switches}") + # Adjacent and same-timestamp I/O syscalls must stay separate and update comm, + # while ignoring perf's unset ':tid' thread placeholder. + cur_comm = ":20" + inv.process_event(Sample("sched:sched_wakeup", 6500, 0, 1, + pid=20, comm="launcher", common_pid=1, common_flags=0)) + inv.process_event(Sample("syscalls:sys_enter_read", 7000, 0, 20, fd=4)) + inv.process_event(Sample("syscalls:sys_exit_read", 7100, 0, 20, ret=4096)) + if inv.tasks[20].comm != "launcher": + raise RuntimeError(f"Unset :tid placeholder overwrote comm: {inv.tasks[20].comm}") + cur_comm = "cat" + inv.process_event(Sample("syscalls:sys_enter_read", 7100, 0, 20, fd=4)) + inv.process_event(Sample("syscalls:sys_exit_read", 7200, 0, 20, ret=4096)) + inv.process_event(Sample("syscalls:sys_enter_read", 8000, 0, 20, fd=4)) + inv.process_event(Sample("syscalls:sys_exit_read", 8000, 0, 20, ret=512)) + inv.process_event(Sample("syscalls:sys_enter_read", 8000, 0, 20, fd=4)) + inv.process_event(Sample("syscalls:sys_exit_read", 8000, 0, 20, ret=512)) + t20 = inv.tasks[20] + if (len(t20.io) != 4 or t20.io_bytes != 9216 or t20.comm != "cat" or + t20.comms != ["launcher", "cat"]): + raise RuntimeError(f"Unexpected I/O task state: {len(t20.io)} {t20.io_bytes} {t20.comms}") + inv.process_event(Sample("sched:sched_switch", 8500, 0, 20, + prev_pid=20, prev_comm="cat", prev_state=1, + next_pid=0, next_comm="swapper")) + with inv.lock: + inv.prune(9000) + if 20 in inv.tasks: + raise RuntimeError(f"Leaked I/O bytes on prune: {t20.io_bytes}") + inv.session = None + + tracker = tc.PerfPipeTracker() + pipe_hdr = tc.struct.pack("=QQ", tracker.PERF_MAGIC, 16) + td_rec = tc.struct.pack("=IHHI", tracker.HEADER_TRACING_DATA, 0, 12, 8) + b"12345678" + sample_rec = tc.struct.pack("=IHHQ", 9, 0, 16, 0) + # Feed byte-by-byte to exercise split headers. + for b in pipe_hdr + td_rec + sample_rec + tracker.FINISHED_ROUND_EVENT: + tracker.feed(bytes([b])) + if not tracker.at_boundary() or not tracker.unflushed_round or tracker.rounds != 1: + raise RuntimeError("PerfPipeTracker failed to track unflushed round") + tracker.feed(tracker.FINISHED_ROUND_EVENT) + if tracker.unflushed_round or tracker.rounds != 2: + raise RuntimeError("PerfPipeTracker failed to clear unflushed round") + + # Also test live I/O-only mode (-I), including the shell filter quoting. + io_events = tc.LiveRecorder("perf", False, False, [], None, io_only=True).events() + if not io_events or not all(e.startswith("syscalls:sys_") for e in io_events): + raise RuntimeError(f"Unexpected I/O events: {io_events}") + orig_comm = open("/proc/self/comm", encoding="utf-8").read().strip() + with tc.tempfile.NamedTemporaryFile("w", delete=False) as fake_perf: + fake_perf.write(f"#!{tc.sys.executable}\n" + "import json, os, sys\n" + "sys.stderr.write(json.dumps([os.getpid(), sys.argv[1:]]) + '\\n')\n") + fake_perf_path = fake_perf.name + try: + tc.os.chmod(fake_perf_path, 0o755) + cmd_rec = tc.LiveRecorder(fake_perf_path, False, False, ["echo", "a b"], None, + io_only=True) + cmd_rec.start() + cmd_rec.proc.wait(timeout=5) + cmd_rec.stop() + if cmd_rec.backlog() != 0.0: + raise RuntimeError("backlog() after stop() should be 0.0") + import json + exec_pid, exec_argv = json.loads(cmd_rec.error_text()) + expected_filt = (f'comm != "ttimechart:{tc.os.getpid() & 0xffff:x}" && ' + f"common_pid != {exec_pid}") + filt_idx = exec_argv.index("--filter") + if (exec_pid != cmd_rec.proc.pid or exec_argv[filt_idx + 1] != expected_filt or + exec_argv[-3:] != ["--", "echo", "a b"]): + raise RuntimeError(f"Unexpected I/O perf record argv: {exec_pid} {exec_argv}") + if open("/proc/self/comm", encoding="utf-8").read().strip() != orig_comm: + raise RuntimeError("Process comm was not restored after stop()") + # Verify start() cleans up if starting control_thread fails. + real_start = threading.Thread.start + calls = 0 + def fail_second_thread(self_t): + nonlocal calls + calls += 1 + if calls == 2: + raise RuntimeError("simulated thread start failure") + real_start(self_t) + threading.Thread.start = fail_second_thread + try: + fail_rec = tc.LiveRecorder(fake_perf_path, False, False, [], None, io_only=True) + try: + fail_rec.start() + raise RuntimeError("Expected start() to fail") + except RuntimeError as e: + if "simulated thread start failure" not in str(e): + raise + finally: + threading.Thread.start = real_start + if (fail_rec.ctl_fd != -1 or fail_rec.wake_r != -1 or fail_rec.tee_r != -1 or + open("/proc/self/comm", encoding="utf-8").read().strip() != orig_comm): + raise RuntimeError("Failed start() left unclosed fds or unmended comm") + finally: + tc.os.unlink(fake_perf_path) + # Pump must preserve flush_round_target across partial record reads. + pump_rec = tc.LiveRecorder("perf", False, False, [], None) + pump_rec.wake_r, pump_rec.wake_w = tc.os.pipe() + tc.os.set_blocking(pump_rec.wake_r, False) + tc.os.set_blocking(pump_rec.wake_w, False) + src_r, src_w = tc.os.pipe() + dst_r, dst_w = tc.os.pipe() + pump_t = threading.Thread(target=pump_rec.pump, args=(src_r, dst_w, None), daemon=True) + pump_t.start() + try: + full_round = pipe_hdr + sample_rec + tracker.FINISHED_ROUND_EVENT + with pump_rec.round_lock: + pump_rec.flush_round_target = -1 + pump_rec.flush_seq = 1 + # Write a partial record first so tracker is not at a boundary yet. + tc.os.write(src_w, full_round[:20]) + time.sleep(0.05) + tc.os.write(src_w, full_round[20:]) + pump_rec.wake_pump() + expected_len = len(full_round) + len(tracker.FINISHED_ROUND_EVENT) + pumped = b"" + deadline = time.monotonic() + 2.0 + while len(pumped) < expected_len and time.monotonic() < deadline: + ready, _, _ = tc.select.select([dst_r], [], [], 0.1) + if ready: + chunk = tc.os.read(dst_r, 4096) + if not chunk: + break + pumped += chunk + tc.os.close(src_w) + src_w = -1 + pump_t.join(timeout=2) + if pumped != full_round + tracker.FINISHED_ROUND_EVENT: + raise RuntimeError(f"Pump missed split-record flush: {len(pumped)} != {expected_len}") + finally: + if src_w >= 0: + tc.os.close(src_w) + tc.os.close(src_r) + tc.os.close(dst_r) + pump_rec.close_wake_fds() + + # Idle live window past last_time must not wipe open or finished states. + idle_rec = FakeRecorder() + idle_rec.origin = monotonic_ns() - 5 * SEC + idle_app = tc.TimechartApp("live", False, False, [], live=idle_rec, window=1.0) + idle_app.data.process_event(Sample("sched:sched_switch", idle_rec.origin + 1000, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=77, next_comm="quiet")) + idle_app.data.process_event(Sample("power:cpu_idle", idle_rec.origin + 1500, 1, 0, + cpu_id=1, state=1)) + idle_app.data.process_event(Sample("power:cpu_frequency", idle_rec.origin + 2000, 0, 77, + cpu_id=0, state=2000000)) + with idle_app.data.lock: + idle_app.data.checkpoint() + idle_app.live_prune() + if (77 not in idle_app.data.tasks or not len(idle_app.data.tasks[77].segs) or + not len(idle_app.data.cpus[1].cstate)): + raise RuntimeError("live_prune() wiped open states when idle past window") + idle_app.data.finish() + with idle_app.data.lock: + idle_app.live_prune() + if 77 not in idle_app.data.tasks or not len(idle_app.data.tasks[77].segs): + raise RuntimeError("live_prune() after finish() wiped tasks") + + io_rec = FakeRecorder() + + def fake_io_read(data, _fd): + now = monotonic_ns() + # Self I/O should be ignored in live mode without counting as samples. + own = tc.os.getpid() + data.process_event(Sample("syscalls:sys_enter_read", now, 0, own, fd=3)) + data.process_event(Sample("syscalls:sys_exit_read", now + 1000, 0, own, ret=64)) + # Scheduler-only tasks must be ignored when io_only is set. + data.process_event(Sample("sched:sched_switch", now + 1500, 0, 0, + prev_pid=0, prev_comm="swapper", prev_state=1, + next_pid=9999, next_comm="schedonly")) + data.process_event(Sample("sched:sched_switch", now + 1800, 0, 9999, + prev_pid=9999, prev_comm="schedonly", prev_state=1, + next_pid=0, next_comm="swapper")) + # Workload I/O should be recorded. + data.process_event(Sample("syscalls:sys_enter_read", now + 2000, 0, 4242, fd=4)) + data.process_event(Sample("syscalls:sys_exit_read", now + 5000, 0, 4242, ret=4096)) + data.process_event(Sample("syscalls:sys_enter_write", now + 6000, 0, 4242, fd=5)) + data.process_event(Sample("syscalls:sys_exit_write", now + 9000, 0, 4242, ret=2048)) + + tc.read_events = fake_io_read + io_app = tc.TimechartApp("live", False, False, [], live=io_rec, window=1.0, io_only=True) + async with io_app.run_test(size=(120, 40)) as pilot: + for _ in range(100): + if not io_app.recording: + break + await pilot.pause(0.05) + await pilot.pause() + tabs = io_app.query_one(tc.TabbedContent) + if (tabs.active != "io" or not io_app.data.sched_tasks([]) or io_app.tasks or + len(io_app.io_tasks) != 1 or io_app.data.io_events != 4 or + io_app.data.nr_samples != 6): + raise RuntimeError(f"Unexpected I/O state: active={tabs.active} " + f"tasks={len(io_app.tasks)} io_tasks={len(io_app.io_tasks)} " + f"events={io_app.data.io_events} samples={io_app.data.nr_samples}") + import io as pyio + from contextlib import redirect_stdout + buf = pyio.StringIO() + with redirect_stdout(buf): + io_app.data.dump(False, False, [], io_only=True) + if "tasks: 1," not in buf.getvalue() or "schedonly" in buf.getvalue(): + raise RuntimeError(f"Unexpected -I dump output: {buf.getvalue()}") + tabs.active = "summary" + await pilot.pause() + await pilot.press("enter") + await pilot.pause() + if tabs.active != "io": + raise RuntimeError(f"Summary enter did not activate I/O tab: {tabs.active}") + print("live synthetic ok") + +asyncio.run(run()) +EOF + then + echo "ttimechart live UI test (synthetic) [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! grep -q "live synthetic ok" "${output}"; then + echo "ttimechart live UI test (synthetic) [Failed: unexpected output]" + cat "${output}" + err=1 + return + fi + echo "ttimechart live UI test (synthetic) [Success]" +} + +test_live_record() { + echo "ttimechart live record test" + if [ "${system_wide}" != 1 ]; then + # Live mode records system wide, like perf timechart record. + echo "ttimechart live record test [Skipped: can't record system wide]" + return + fi + rm -f "${perfdata}.live" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${perfdata}.live" \ + "$(declare -f workload); workload" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, ["bash", "-c", sys.argv[3]], sys.argv[2]) + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if not app.recording: + break + await pilot.pause(0.1) + if app.recording or not app.data.has_events(): + raise RuntimeError("Live recording of workload did not complete") + print("live record ok") + finally: + rec.stop() + +asyncio.run(run()) +EOF + then + echo "ttimechart live record test [Failed: exception]" + cat "${output}" + err=1 + return + fi + if ! perf script ttimechart -i "${perfdata}.live" --dump > "${output}" 2>&1 || + ! grep -q "^Duration: .*sched events: [1-9]" "${output}"; then + echo "ttimechart live record test [Failed: saved perf.data unreadable]" + cat "${output}" + err=1 + return + fi + if perf timechart record -I -o "${perfdata}.live" -- true > /dev/null 2>&1; then + rm -f "${perfdata}.live" + if ! "$PYTHON" - "${PERF_BIN:-perf}" "${perfdata}.live" > "${output}" 2>&1 <<'EOF' +import asyncio +import sys +import ttimechart as tc + +async def run() -> None: + rec = tc.LiveRecorder(sys.argv[1], False, False, + ["bash", "-c", "cat /etc/hostname > /dev/null"], + sys.argv[2], io_only=True) + rec.start() + app = tc.TimechartApp("live", False, False, [], live=rec, window=5.0, io_only=True) + try: + async with app.run_test(size=(120, 40)) as pilot: + for _ in range(600): + if not app.recording: + break + await pilot.pause(0.1) + if app.recording or not app.data.io_events: + raise RuntimeError("Live I/O recording of workload did not complete") + finally: + rec.stop() + +asyncio.run(run()) +EOF + then + echo "ttimechart live record test [Failed: live -I exception]" + cat "${output}" + err=1 + return + fi + fi + echo "ttimechart live record test [Success]" +} + test_plumbing test_no_events +test_live_synthetic if ! record; then echo "Skipping remaining tests, failed to record scheduler events (permissions?)"
@@ -232,6 +822,7 @@ fi test_dump test_headless_ui +test_live_record cleanup exit $err
Signature · 1 line
2.56.1.703.gf848ed807c-goog
[email protected]
AI reviewRe: [PATCH v8 2/2] perf test: Test perf timechart --live
Sashiko has reviewed this patch and found no issues.
Quoted 9 lines
From: Alice Rogers <[email protected]>
Extend
test_ttimechart_python.shto cover --live option validation, LiveRecorder's perf record subprocess, control pipe and -o output file when system-wide recording is permitted, a synthetic live UI test exercising the rolling window, pruning, pause/resume gaps, round-flush framing and zoom/cursor following, and a live workload recording test when scheduler events can be recorded system wide.
Quoted 3 lines
Signed-off-by Alice Rogers <[email protected]>Co-developed-by Ian Rogers <[email protected]>Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2