perf tools: Add progress diagnostics and a false-sharing workload
Arnaldo Carvalho de Melo
maintainerv3 cover perf tools: Add progress diagnostics and a false-sharing workload
Hi,
This series adds progress and stuck-process diagnostics to perf, and a workload that makes false sharing visible to data type profiling.
The changes are:
- move perf_config__set_variable() to util/config.c and serialize config parser and read-modify-write state, so non-builtin perf code can persist configuration changes safely;
- add 'perf report --progress' for stdio users, showing the current phase,
percentage, and counts while a session is processed;
- add the prototype tools/perf/scripts/perf-stuck.sh helper, which samples a running perf process and can invoke the accompanying GDB commands when progress stops;
- add 'perf test -w false_sharing', a synthetic TCP-shaped workload with identity and packet counters sharing a cacheline, and include it in the data type profiling shell test.
The progress backend handles nested phases and completes them on error paths as well as successful paths. The false-sharing workload uses a shared volatile instance so the data type profiler can resolve direct accesses to the individual members.
The perf-stuck helper is intentionally a prototype and requires the usual /proc access; its optional GDB mode additionally requires gdb and suitable debug information. When only one CPU is available, the false-sharing workload warns that it cannot produce cross-CPU traffic but still runs.
Testing:
make -C tools/perf -j4 tools/perf/perf test -w false_sharing taskset -c 0 tools/perf/perf test -w false_sharing bash -n tools/perf/scripts/perf-stuck.sh bash -n tools/perf/tests/shell/data_type_profiling.sh git diff --check 0ae6fc78c5ce0dfd..485532296710225862daf8ffb19802ae327efe2a
Best regards,
- Arnaldo
What changed from v2:
- tools/perf/util/config.c: perf_etc_perfconfig() returned NULL when the allocation in system_path() fails, and none of its callers check for that. Reported by Sashiko while reviewing v2: the deref reached from this series was fixed, and since the same unchecked deref is already reachable from the three callers predating it, perf_config_from_file() and 'perf daemon' among them, the opportunity was taken to make the accessor total as well, falling back to the unresolved path, which for the usual absolute ETC_PERFCONFIG is the same string;
- tools/perf/scripts/perf-stuck.gdb: don't dereference map_symbol.sym without checking it for NULL, printing "(no symbol)" instead. Reported by Sashiko while reviewing v2;
- tools/perf/scripts/perf-stuck.gdb: note next to the structure walks that a REFCNT_CHECKING build wraps some structs in a proxy holding the original, where this has to read map->orig->dso->orig->name;
- tools/perf/scripts/perf-stuck.sh: count the samples showing no progress when there is no progress to look at, as an empty or missing progress log reset the counter on every sample and never fired -g;
- tools/perf/scripts/perf-stuck.sh: use -- for pgrep and tail, as a process name starting with a hyphen was taken as an option and had the script attach to an unrelated process, gdb included when -g is used;
- tools/perf/scripts/perf-stuck.sh: bound the gdb run with timeout --signal=INT 30, as the script makes inferior calls and one into a perf wedged in a loop never returns, while SIGINT lets gdb release the inferior instead of leaving perf stopped.
What changed from v1:
- avoid calling CPU_SET() with -1 when false_sharing runs with only one CPU available in its affinity mask.
tools/perf/Documentation/perf-report.txt | 13 + tools/perf/builtin-config.c | 70 +---- tools/perf/builtin-report.c | 10 + tools/perf/scripts/perf-stuck.gdb | 109 ++++++++ tools/perf/scripts/perf-stuck.sh | 184 ++++++++++++++ tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/shell/data_type_profiling.sh | 9 +- tools/perf/tests/tests.h | 1 + tools/perf/tests/workloads/Build | 2 + tools/perf/tests/workloads/false_sharing.c | 240 ++++++++++++++++++ tools/perf/ui/Build | 1 + tools/perf/ui/progress.h | 2 + tools/perf/ui/stdio/progress.c | 162 ++++++++++++ tools/perf/util/config.c | 131 +++++++++- tools/perf/util/config.h | 2 + tools/perf/util/ordered-events.c | 16 +- tools/perf/util/session.c | 12 +- 17 files changed, 882 insertions(+), 83 deletions(-)
create mode 100644 tools/perf/scripts/perf-stuck.gdb create mode 100755 tools/perf/scripts/perf-stuck.sh create mode 100644 tools/perf/tests/workloads/false_sharing.c create mode 100644 tools/perf/ui/stdio/progress.c
base-commit: 0ae6fc78c5ce0dfd
v1-head: 45d7917f7e05e8a29828ed5f0bbdc94fc938f79f
v2-head: d4f84e4de8890194924ccd897a8e6773e7d4240b
--
Arnaldo Carvalho de Melo
maintainerPATCH v3 1/4 perf config: Move perf_config__set_variable() to util/config.c
From: Arnaldo Carvalho de Melo <[email protected]>
Move perf_config__set_variable() out of the 'perf config' builtin so that opt-in features can persist their choice from outside it, e.g. util/debuginfo.c writing core.debuginfod=false when the user disables debuginfod for the rest of the session. Theset_config() body becomes perf_config_set__write(), with the system_config choice as an argument, as the builtin's use_system_config/use_user_config statics are not available outside it.
All config file access shares the static parser state and can now run on more than one thread, with a feature writing the config while perf top's display thread reads it, so serialize parsing and rewriting with a mutex, and the whole read-modify-write of perf_config__set_variable() against itself.
perf_etc_perfconfig() caches what system_path() returns, and that allocates, so on failure every caller dereferenced NULL, the pre-existing ones in perf_config_from_file() and in the daemon included. Pre-existing, not introduced here, but this rewrites the accessor anyway, so make it total: the unresolved path is the same string for the usual absolute ETC_PERFCONFIG.
--- tools/perf/builtin-config.c | 70 +------------------ tools/perf/util/config.c | 131 ++++++++++++++++++++++++++++++++++-- tools/perf/util/config.h | 2 + 3 files changed, 129 insertions(+), 74 deletions(-)
tools/perf/builtin-config.c +1 -69
@@ -41,37 +41,7 @@ static struct option config_options[] = {static int set_config (struct perf_config_set *set,const char *file_name) { -struct perf_config_section *section =NULL ; -struct perf_config_item *item =NULL ; -const char *first_line ="# this file is auto-generated." ; -FILE *fp; - -if (set ==NULL ) -return -1 ; - - fp =fopen (file_name,"w" ); -if (!fp) -return -1 ; - -fprintf (fp,"%s\n" , first_line); - -/* overwrite configvariables */ -perf_config_items__for_each_entry (&set->sections, section) { -if (!use_system_config && section->from_system_config) -continue ; -fprintf (fp,"[%s]\n" , section->name); - -perf_config_items__for_each_entry (§ion->items, item) { -if (!use_system_config && item->from_system_config) -continue ; -if (item->value) -fprintf (fp,"\t%s = %s\n" , - item->name, item->value); - } - } -fclose (fp); - -return 0 ; +return perf_config_set__write (set, file_name, use_system_config); }static int show_spec_config (struct perf_config_set *set,const char *var)
@@ -158,44 +128,6 @@ static int parse_config_arg(char *arg, char **var, char **value)return 0 ; } -int perf_config__set_variable (const char *var,const char *value) -{ -char path[PATH_MAX ]; -char *user_config =mkpath (path,sizeof (path),"%s/.perfconfig" ,getenv ("HOME" )); -const char *config_filename; -struct perf_config_set *set; -int ret =-1 ; - -if (use_system_config) - config_exclusive_filename =perf_etc_perfconfig (); -else if (use_user_config) - config_exclusive_filename = user_config; - -if (!config_exclusive_filename) - config_filename = user_config; -else - config_filename = config_exclusive_filename; - - set =perf_config_set__new (); -if (!set) - goto out_err; - -if (perf_config_set__collect (set, config_filename, var, value) <0 ) { -pr_err ("Failed to add '%s=%s'\n" , var, value); - goto out_err; - } - -if (set_config (set, config_filename) <0 ) { -pr_err ("Failed to set the configs on %s\n" , config_filename); - goto out_err; - } - - ret =0 ; -out_err: -perf_config_set__delete (set); -return ret; -} -int cmd_config (int argc,const char **argv) {int i, ret =-1 ;
tools/perf/util/config.c +126 -5
172 lines not on this page · raw message
tools/perf/util/config.h +2 -0
@@ -33,6 +33,8 @@ int perf_config_scan(const char *name, const char *fmt, ...) __scanf(2, 3);const char *perf_config_get (const char *name);int perf_config_set (struct perf_config_set *set,config_fn_t fn,void *data); +int perf_config_set__write (struct perf_config_set *set, +const char *file_name,bool system_config);int perf_config_int (int *dest,const char *,const char *);int perf_config_u8 (u8 *dest,const char *name,const char *value);int perf_config_u64 (u64 *dest,const char *,const char *);
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH 1/4] perf config: Move perf_config__set_variable() to util/config.c
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] perf: data race in bad_config() causes stack UAF --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] perf: data race in bad_config() causes stack UAF
--
commit 861fbcb5738f68f4bf3fa7cf60e666d3e7639e45
Author: Arnaldo Carvalho de Melo <[email protected]>
perf config: Move perf_config__set_variable() to util/config.c
This commit moves perf_config__set_variable() out of the builtin so opt-in features can persist their choice from outside it. It also serializes parsing and rewriting with a mutex since config file access can now run on more than one thread.
Quoted 4 lines
tools/perf/util/config.c +0
-0
[ ... ]
Quoted 6 lines
tools/perf/util/config.c +2
-0@@ -926,6 +926,84 @@ void perf_config__exit(void) config_set =NULL ; } +int perf_config_set__write (struct perf_config_set *set, +const char *file_name,bool system_config)
[ ... ]
Quoted 10 lines
tools/perf/util/config.c +10
-0+int perf_config__set_variable (const char *var,const char *value) +{ +char path[PATH_MAX ]; +char *user_config =mkpath (path,sizeof (path),"%s/.perfconfig" ,getenv ("HOME" )); +const char *config_filename = config_exclusive_filename ?: user_config; +/* +* When rewriting the system wide file all entries are marked as coming +* from it and must be kept, or it would be truncated down to its +* header. +*/
tools/perf/util/config.c void perf_config__exit(void) · line 990
Quoted 10 lines
tools/perf/util/config.c +10
-0+bool system_config =strcmp (config_filename,perf_etc_perfconfig ()) ==0 ; +struct perf_config_set *set; +int ret =-1 ; + +pthread_mutex_lock (&config_update_mutex); + set =perf_config_set__new (); +if (!set) + goto out_err; + +if (perf_config_set__collect (set, config_filename, var, value) <0 ) {
[Severity: High]
Can this lead to a stack use-after-free during concurrent config parsing?
A writer thread running perf_config__set_variable() assigns the global config_file_name to point to its local stack buffer (path) under config_mutex in perf_config_set__collect().
Concurrently, a reader thread iterating cached configurations via perf_config() might encounter a malformed value and invoke bad_config(). bad_config() reads config_file_name locklessly:
tools/perf/util/config.c:bad_config() {
...
if (config_file_name)
pr_warning("bad config value for '%s' in %s, ignoring...\n",
name, config_file_name);
...
}If the reader reads the pointer locklessly, and the writer finishes and returns before the reader executes pr_warning(), won't the reader dereference the destroyed stack frame for path?
Quoted 15 lines
tools/perf/util/config.c +15
-0+pr_err ("Failed to add '%s=%s'\n" , var, value); + goto out_err; + } + +if (perf_config_set__write (set, config_filename, system_config) <0 ) { +pr_err ("Failed to set the configs on %s\n" , config_filename); + goto out_err; + } + + ret =0 ; +out_err: +perf_config_set__delete (set); +pthread_mutex_unlock (&config_update_mutex); +return ret; +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Arnaldo Carvalho de Melo
maintainerPATCH v3 2/4 perf report: Add --progress option
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Processing a large session with stdio output gives no feedback about which phase perf is in or how far along it is: ui_progress updates are only shown by the TUI. Add --progress, installing a stdio backend (ui/stdio/progress.c) that prints the phase title, percentage and counts:
Processing events... [42.3%] 317M / 746M
Phases can be nested, so the backend tracks the ones started to complete the right one on ui_progress__finish(); that requires init()/finish() pairs, fixed in ordered-events.c and the pipe and directory event processing.
--- tools/perf/Documentation/perf-report.txt | 13 ++ tools/perf/builtin-report.c | 10 ++ tools/perf/ui/Build | 1 + tools/perf/ui/progress.h | 2 + tools/perf/ui/stdio/progress.c | 162 +++++++++++++++++++++++ tools/perf/util/ordered-events.c | 16 ++- tools/perf/util/session.c | 12 +- 7 files changed, 209 insertions(+), 7 deletions(-) create mode 100644 tools/perf/ui/stdio/progress.c
tools/perf/Documentation/perf-report.txt +13 -0
@@ -29,6 +29,19 @@ OPTIONS --quiet:: Do not show any warnings or messages. (Suppress -v) +--progress:: + Show progress for each of the processing phases, printing the + percentage done and the current/total counts: the first phase + counts the bytes of the perf.data file processed so far, the + merge and sort phases count hist entries, e.g.: + + Processing events... [ 42.3%] 4G / 10G + Merging related events... [ 7.1%] 1024 / 14387 + Sorting events for output... [ 98.2%] 14132 / 14387 + + It is a no-op when using the TUI or GTK browsers, that already + present progress information. + -n:: --show-nr-samples:: Show the number of samples for each symbol
tools/perf/builtin-report.c +10 -0
@@ -87,6 +87,7 @@ struct report {bool use_gtk;#endif bool use_stdio; +bool progress;bool show_full_info;bool show_threads;bool inverted_callchain;
@@ -1384,6 +1385,8 @@ int cmd_report(int argc, const char **argv) "Use the stdio interface"),OPT_BOOLEAN (0 ,"weights" , &symbol_conf.annotate_weight,"Show or hide weight columns in annotation. Default show if non-zero." ), +OPT_BOOLEAN (0 ,"progress" , &report.progress, +"Show progress while processing the perf.data file" ),OPT_BOOLEAN (0 ,"header" , &report.header,"Show data header." ),OPT_BOOLEAN (0 ,"header-only" , &report.header_only, "Show only data header."),
@@ -1790,6 +1793,13 @@ int cmd_report(int argc, const char **argv)else use_browser =0 ; +/* +* For the stdio case: print the percentage of the perf.data file +* processed so far for each processing phase. +*/ +if (report.progress && use_browser ==0 ) +stdio_progress__init (); +if (report.data_type && use_browser ==1 ) { symbol_conf.annotate_data_member = true; symbol_conf.annotate_data_sample = true;
tools/perf/ui/Build +1 -0
@@ -4,6 +4,7 @@ perf-ui-y += progress.o perf-ui-y += util.o perf-ui-y += hist.o perf-ui-y += stdio/hist.o +perf-ui-y += stdio/progress.o CFLAGS_setup.o += -DLIBDIR="BUILD_STR($(LIBDIR))"
tools/perf/ui/progress.h +2 -0
@@ -23,6 +23,8 @@ void __ui_progress__init(struct ui_progress *p, u64 total,void ui_progress__update (struct ui_progress *p,u64 adv); +void stdio_progress__init (void ); +struct ui_progress_ops {void (*init)(struct ui_progress *p);void (*update)(struct ui_progress *p);
tools/perf/ui/stdio/progress.c new file +162 -0
162 lines not on this page · raw message
tools/perf/util/ordered-events.c +12 -4
@@ -237,14 +237,16 @@ static int do_flush(struct ordered_events *oe, bool show_progress)ui_progress__init (&prog, oe->nr_events,"Processing time ordered events..." );list_for_each_entry_safe (iter, tmp, head, list) { -if (session_done ()) -return 0 ; +if (session_done ()) { + ret =0 ; + goto out_progress; + }if (iter->timestamp > limit)break ; ret = oe->deliver (oe, iter);if (ret <0 ) -return ret; + goto out_progress;ordered_events__delete (oe, iter); oe->last_flush = iter->timestamp;
@@ -258,10 +260,16 @@ static int do_flush(struct ordered_events *oe, bool show_progress)else if (last_ts <= limit) oe->last = list_entry(head->prev, struct ordered_event, list); + ret =0 ; +out_progress: +/* +* Always pair ui_progress__init() with ui_progress__finish(), the +* stdio backend tracks the phases on a stack. +*/ if (show_progress)ui_progress__finish (); -return 0 ; +return ret; } static int __ordered_events__flush(struct ordered_events *oe, enum oe_flush how,
tools/perf/util/session.c +9 -3
@@ -3253,8 +3253,12 @@ static int __perf_session__process_pipe_events(struct perf_session *session) cur_size =sizeof (union perf_event ); buf =malloc (cur_size); -if (!buf) -return -errno; +if (!buf) { + err = -errno; +if (update_prog) +ui_progress__finish (); +return err; + }ordered_events__set_copy_on_queue (oe, true); more: event = buf;
@@ -3753,8 +3757,10 @@ static int __perf_session__process_dir_events(struct perf_session *session) } rd =calloc (nr_readers,sizeof (struct reader )); -if (!rd) +if (!rd) { +ui_progress__finish ();return -ENOMEM ; + } rd[0 ] = (struct reader ) { .fd =perf_data__fd (session->data),
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH 2/4] perf report: Add --progress option
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
From: Arnaldo Carvalho de Melo <[email protected]>
Processing a large session with stdio output gives no feedback about which phase perf is in or how far along it is:
ui_progressupdates are only shown by the TUI. Add --progress, installing a stdio backend (ui/stdio/progress.c) that prints the phase title, percentage and counts:Processing events... [42.3%] 317M / 746M
Quoted 7 lines
Phases can be nested, so the backend tracks the ones started to complete the right one on
ui_progress__finish(); that requires init()/finish() pairs, fixed inordered-events.cand the pipe and directory event processing.Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Namhyung Kim
maintainerRe: [PATCH 2/4] perf report: Add --progress option
In reply to Arnaldo Carvalho de MeloQuoted 38 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
Processing a large session with stdio output gives no feedback about which phase perf is in or how far along it is:
ui_progressupdates are only shown by the TUI. Add --progress, installing a stdio backend (ui/stdio/progress.c) that prints the phase title, percentage and counts:Processing events... [42.3%] 317M / 746M
Phases can be nested, so the backend tracks the ones started to complete the right one on
ui_progress__finish(); that requires init()/finish() pairs, fixed inordered-events.cand the pipe and directory event processing.Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/Documentation/perf-report.txt | 13 ++ tools/perf/builtin-report.c | 10 ++ tools/perf/ui/Build | 1 + tools/perf/ui/progress.h | 2 + tools/perf/ui/stdio/progress.c | 162 +++++++++++++++++++++++ tools/perf/util/ordered-events.c | 16 ++- tools/perf/util/session.c | 12 +- 7 files changed, 209 insertions(+), 7 deletions(-) create mode 100644 tools/perf/ui/stdio/progress.ctools/perf/Documentation/perf-report.txt +2
-0@@ -29,6 +29,19 @@ OPTIONS --quiet:: Do not show any warnings or messages. (Suppress -v) +--progress:: + Show progress for each of the processing phases, printing the
tools/perf/Documentation/perf-report.txt OPTIONS · line 43
Quoted 10 lines
tools/perf/Documentation/perf-report.txt +10
-0+ percentage done and the current/total counts: the first phase + counts the bytes of the perf.data file processed so far, the + merge and sort phases count hist entries, e.g.: + + Processing events... [ 42.3%] 4G / 10G + Merging related events... [ 7.1%] 1024 / 14387 + Sorting events for output... [ 98.2%] 14132 / 14387 + + It is a no-op when using the TUI or GTK browsers, that already + present progress information.
I'm not sure what to do if --quiet and --progress were given together.
Also maybe meaningful to handle --no-progress for TUI and GTK.
Thanks,
Namhyung
Quoted 308 lines
tools/perf/Documentation/perf-report.txt +1
-0+ -n:: --show-nr-samples:: Show the number of samples for each symboltools/perf/builtin-report.c +10
-0@@ -87,6 +87,7 @@ struct report {bool use_gtk;#endif bool use_stdio; +bool progress;bool show_full_info;bool show_threads;bool inverted_callchain;@@ -1384,6 +1385,8 @@ int cmd_report(int argc, const char **argv) "Use the stdio interface"),OPT_BOOLEAN (0 ,"weights" , &symbol_conf.annotate_weight,"Show or hide weight columns in annotation. Default show if non-zero." ), +OPT_BOOLEAN (0 ,"progress" , &report.progress, +"Show progress while processing the perf.data file" ),OPT_BOOLEAN (0 ,"header" , &report.header,"Show data header." ),OPT_BOOLEAN (0 ,"header-only" , &report.header_only, "Show only data header."),@@ -1790,6 +1793,13 @@ int cmd_report(int argc, const char **argv)else use_browser =0 ; +/* +* For the stdio case: print the percentage of the perf.data file +* processed so far for each processing phase. +*/ +if (report.progress && use_browser ==0 ) +stdio_progress__init (); +if (report.data_type && use_browser ==1 ) { symbol_conf.annotate_data_member = true; symbol_conf.annotate_data_sample = true;tools/perf/ui/Build +1
-0@@ -4,6 +4,7 @@ perf-ui-y += progress.o perf-ui-y += util.o perf-ui-y += hist.o perf-ui-y += stdio/hist.o +perf-ui-y += stdio/progress.o CFLAGS_setup.o += -DLIBDIR="BUILD_STR($(LIBDIR))"tools/perf/ui/progress.h +2
-0@@ -23,6 +23,8 @@ void __ui_progress__init(struct ui_progress *p, u64 total,void ui_progress__update (struct ui_progress *p,u64 adv); +void stdio_progress__init (void ); +struct ui_progress_ops {void (*init)(struct ui_progress *p);void (*update)(struct ui_progress *p);tools/perf/ui/stdio/progress.c new file +162
-0@@ -0,0 +1,162 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Progress feedback for the stdio (non-TUI/GTK) case, enabled with +* 'perf report --progress'. +*/ +#include <inttypes.h> +#include <stdio.h> +#include <unistd.h> +#include <linux/kernel.h> +#include "../../util/debug.h" +#include "../../util/units.h" +#include "../progress.h" + +/* +* Phases can be nested, so keep track of the ones started so far to be +* able to complete the right one on ui_progress__finish(), which gets +* no arguments. +*/ +#define STDIO_PROGRESS__MAX_DEPTH 8 + +struct stdio_progress_phase { +struct ui_progress *p; +u64 last_printed; +size_t last_len; +}; + +static struct stdio_progress_phase stdio_progress__stack[STDIO_PROGRESS__MAX_DEPTH ]; +static int stdio_progress__depth; +static bool stdio_progress__is_tty; +/* Phases that didn't fit on the stack are not shown. */ +static int stdio_progress__dropped; + +static void stdio_progress__print_phase (struct stdio_progress_phase *phase, +u64 curr) +{ +struct ui_progress *p = phase->p; +char buf_cur[20 ], buf_tot[20 ], buf[128 ]; +double percent = p->total ?100.0 * (double )curr / (double )p->total :0.0 ; +size_t len; + +/* +* Only the completion line shows 100.0%: a 99.99% progress would round +* up to it and look like a duplicate at finish time. +*/ +if (curr < p->total && percent >99.9 ) + percent =99.9 ; + +if (p->size) { +unit_number__scnprintf (buf_cur,sizeof (buf_cur), curr); +unit_number__scnprintf (buf_tot,sizeof (buf_tot), p->total); + len =scnprintf (buf,sizeof (buf),"%s [%5.1f%%] %s / %s" , + p->title, percent, buf_cur, buf_tot); + }else { + len =scnprintf (buf,sizeof (buf),"%s [%5.1f%%] %" PRIu64" / %" PRIu64, + p->title, percent, curr, p->total); + } + +if (!stdio_progress__is_tty) { +fprintf (stderr,"%s\n" , buf); + goto out; + } + +/* Pad to the length of the previous line to erase its leftovers. */ +fprintf (stderr,"\r%s%*s" , buf, + (int )(len < phase->last_len ? phase->last_len - len :0 ),"" ); + phase->last_len = len; +out: + phase->last_printed = curr; +fflush (stderr); +} + +static void __stdio_progress__init (struct ui_progress *p) +{ +/* The default step is meant for the TUI bar, use 1% steps for stdio. */ + p->next = p->step = p->total /100 ?:1 ; + +if (stdio_progress__depth ==STDIO_PROGRESS__MAX_DEPTH ) { +/* +* Out of room: don't start this phase, its finish() is swallowed and +* its updates ignored below. +*/ +pr_warning ("progress phases nested deeper than %d, not showing progress for %s\n" , +STDIO_PROGRESS__MAX_DEPTH , p->title); + stdio_progress__dropped++; +return ; + } + +/* Start a nested phase in a line of its own. */ +if (stdio_progress__depth && stdio_progress__is_tty) +fputc ('\n' , stderr); + + stdio_progress__stack[stdio_progress__depth++] = + (struct stdio_progress_phase ) { + .p = p, + .last_printed =0 , + .last_len =0 , + }; + +stdio_progress__print_phase (&stdio_progress__stack[stdio_progress__depth -1 ], + p->curr); +} + +static void stdio_progress__update (struct ui_progress *p) +{ +/* +* An update that doesn't match the innermost phase means something is +* out of sync: print nothing rather than another phase's numbers, or +* read past the stack. +*/ +if (!stdio_progress__depth || + stdio_progress__stack[stdio_progress__depth -1 ].p != p) +return ; + +stdio_progress__print_phase (&stdio_progress__stack[stdio_progress__depth -1 ], + p->curr); +} + +static void stdio_progress__finish (void ) +{ +struct stdio_progress_phase *phase; + +/* +* Being the innermost phase, its finish() comes first: swallow it, or +* it would complete the phase that encloses it. +*/ +if (stdio_progress__dropped) { + stdio_progress__dropped--; +return ; + } + +if (!stdio_progress__depth) +return ; + + phase = &stdio_progress__stack[--stdio_progress__depth]; + +/* +* The last line may have stopped short of the total, close this phase +* showing it as complete unless that was already printed. +*/ +if (phase->last_printed != phase->p->total) +stdio_progress__print_phase (phase, phase->p->total); + + phase->last_printed =0 ; + phase->last_len =0 ; + +if (stdio_progress__is_tty) +fputc ('\n' , stderr); + +fflush (stderr); +} + +static struct ui_progress_ops stdio_progress__ops = { + .init = __stdio_progress__init, + .update = stdio_progress__update, + .finish = stdio_progress__finish, +}; + +void stdio_progress__init (void ) +{ + stdio_progress__is_tty =isatty (STDERR_FILENO ) ==1 ; + ui_progress__ops = &stdio_progress__ops; +}tools/perf/util/ordered-events.c +12
-4@@ -237,14 +237,16 @@ static int do_flush(struct ordered_events *oe, bool show_progress)ui_progress__init (&prog, oe->nr_events,"Processing time ordered events..." );list_for_each_entry_safe (iter, tmp, head, list) { -if (session_done ()) -return 0 ; +if (session_done ()) { + ret =0 ; + goto out_progress; + }if (iter->timestamp > limit)break ; ret = oe->deliver (oe, iter);if (ret <0 ) -return ret; + goto out_progress;ordered_events__delete (oe, iter); oe->last_flush = iter->timestamp;@@ -258,10 +260,16 @@ static int do_flush(struct ordered_events *oe, bool show_progress)else if (last_ts <= limit) oe->last = list_entry(head->prev, struct ordered_event, list); + ret =0 ; +out_progress: +/* +* Always pair ui_progress__init() with ui_progress__finish(), the +* stdio backend tracks the phases on a stack. +*/ if (show_progress)ui_progress__finish (); -return 0 ; +return ret; } static int __ordered_events__flush(struct ordered_events *oe, enum oe_flush how,tools/perf/util/session.c +9
-3@@ -3253,8 +3253,12 @@ static int __perf_session__process_pipe_events(struct perf_session *session) cur_size =sizeof (union perf_event ); buf =malloc (cur_size); -if (!buf) -return -errno; +if (!buf) { + err = -errno; +if (update_prog) +ui_progress__finish (); +return err; + }ordered_events__set_copy_on_queue (oe, true); more: event = buf;@@ -3753,8 +3757,10 @@ static int __perf_session__process_dir_events(struct perf_session *session) } rd =calloc (nr_readers,sizeof (struct reader )); -if (!rd) +if (!rd) { +ui_progress__finish ();return -ENOMEM ; + } rd[0 ] = (struct reader ) { .fd =perf_data__fd (session->data),Signature · 1 line
2.55.0
Arnaldo Carvalho de Melo
maintainerPATCH v3 3/4 perf scripts: Add perf-stuck, to tell where a running perf is stuck
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
A perf that takes forever is hard to tell apart from one stuck in a loop, and there is no way to see where without attaching gdb. perf-stuck.sh samples a running process' /proc entries and its progress line at a fixed interval, and with -g runs gdb (perf-stuck.gdb, adding the perf-die-chain command) when no progress is made across two samples, printing the DIE chain a DWARF type chase is stuck in.
It is a prototype: the plan is to turn it into a first class 'perf stuck' command. The process name is resolved with pgrep among the caller's own processes only, as root an unscoped one would attach gdb to the first process of any user with a matching name.
--- tools/perf/scripts/perf-stuck.gdb | 109 ++++++++++++++++++ tools/perf/scripts/perf-stuck.sh | 184 ++++++++++++++++++++++++++++++ 2 files changed, 293 insertions(+) create mode 100644 tools/perf/scripts/perf-stuck.gdb create mode 100755 tools/perf/scripts/perf-stuck.sh
tools/perf/scripts/perf-stuck.gdb new file +109 -0
@@ -0,0 +1,109 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# gdb commands for a stuck perf, used by perf-stuck.sh -g and usable directly: +# +# gdb -p $(pgrep -x perf) -batch -x perf-stuck.gdb -ex bt +# +# PROTOTYPE: part of the perf-stuck.sh stopgap, wants to become a first +# class 'perf stuck' command printing these DIE chains without gdb. +# +# The commands are for the DWARF type chasers in util/dwarf-aux.c: +# +# perf-die-chain <function> <die variable> [iterations] +# perf-die-chain-all [iterations] +# perf-dso +# +# For each iteration of the chasing loop they print the DIE address, the +# CU, its file offset, tag and name: a cycle shows as the same (addr, cu) +# pairs repeating, and a CU changing between iterations means the chase +# hops between a debug file and its dwz common file. + +set pagination off +set confirm off +set debuginfod enabled off +set print pretty on +set height 0 +set width 0 + +define perf-die-chain + if $argc < 2 + printf "usage: perf-die-chain <function> <die variable> [iterations]\n" + else + frame function $arg0 + if $argc == 3 + set $perf_die_chain_n = $arg2 + else + set $perf_die_chain_n = 10 + end + set $perf_die_chain_head = $pc + set $perf_die_chain_i = 0 + while $perf_die_chain_i < $perf_die_chain_n + # Pointer type DIEs have no DW_AT_name, so dwarf_diename() can + # return NULL: printf %s of it would error out and abort this + # batch script, handle it. + set $perf_die_chain_name = (char *) dwarf_diename($arg1) + printf "chain[%d] die=%p addr=%p cu=%p off=0x%lx tag=%d name=", $perf_die_chain_i, $arg1, $arg1->addr, $arg1->cu, ((Dwarf_Off) dwarf_dieoffset($arg1)), ((int) dwarf_tag($arg1)) + if $perf_die_chain_name == 0 + printf "(null)\n" + else + printf "%s\n", $perf_die_chain_name + end + until *$perf_die_chain_head + set $perf_die_chain_i = $perf_die_chain_i + 1 + end + end +end + +document perf-die-chain +Print the DIE chain being walked by a DWARF type chasing loop. +usage: perf-die-chain <function> <die variable> [iterations] + perf-die-chain die_get_pointer_type type_die + perf-die-chain __die_get_real_type vr_die + perf-die-chain die_get_real_type vr_die +end + +define perf-die-chain-all + if $argc == 0 + set $perf_die_chain_n = 10 + else + set $perf_die_chain_n = $arg0 + end + if $_any_caller_is("die_get_pointer_type", 20) + printf "stuck in die_get_pointer_type():\n" + perf-die-chain die_get_pointer_type type_die $perf_die_chain_n + else + if $_any_caller_is("__die_get_real_type", 20) + printf "stuck in __die_get_real_type():\n" + perf-die-chain __die_get_real_type vr_die $perf_die_chain_n + else + if $_any_caller_is("die_get_real_type", 20) + printf "stuck in die_get_real_type():\n" + perf-die-chain die_get_real_type vr_die $perf_die_chain_n + else + printf "not in a DWARF type chaser, try: bt\n" + end + end + end +end + +document perf-die-chain-all +Find which DWARF type chaser the process is in and print the DIE chain. +usage: perf-die-chain-all [iterations] +end + +define perf-dso + if $_any_caller_is("find_data_type", 20) + frame function find_data_type + # REFCNT_CHECKING, implied by an ASan/LSan build, wraps struct map and + # struct dso in a proxy keeping the real object in ->orig, so there the + # dso is map->orig->dso->orig, giving map->orig->dso->orig->name here; + # struct symbol is not wrapped, so sym->name needs no ->orig. + printf "dso=%s ip=0x%lx sym=%s\n", dloc->ms->map->dso->name, dloc->ip, dloc->ms->sym ? dloc->ms->sym->name : "(no symbol)" + else + printf "not in find_data_type()\n" + end +end + +document perf-dso +Print the dso, ip and symbol of the data location being resolved. +end
tools/perf/scripts/perf-stuck.sh new file +184 -0
184 lines not on this page · raw message
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH 3/4] perf scripts: Add perf-stuck, to tell where a running perf is stuck
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] perf scripts: perf-dso GDB command fails on ASan/LSan builds --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] perf scripts: perf-dso GDB command fails on ASan/LSan builds
--
commit 3e959f0b96fd50b6f00f007448b2245cac82c305
Author: Arnaldo Carvalho de Melo <[email protected]>
perf scripts: Add perf-stuck, to tell where a running perf is stuck
This commit introduces a prototype perf-stuck script and accompanying GDB script to identify where a running perf process is stuck. It samples /proc entries and can optionally use GDB to dump the DWARF DIE chain.
Quoted 5 lines
tools/perf/scripts/perf-stuck.gdb new file +0
-0
[ ... ]
tools/perf/scripts/perf-stuck.gdb line 101
Quoted 8 lines
tools/perf/scripts/perf-stuck.gdb new file +8
-0+define perf-dso + if $_any_caller_is("find_data_type", 20) + frame function find_data_type + # REFCNT_CHECKING, implied by an ASan/LSan build, wraps struct map and + # struct dso in a proxy keeping the real object in ->orig, so there the + # dso is map->orig->dso->orig, giving map->orig->dso->orig->name here; + # struct symbol is not wrapped, so sym->name needs no ->orig. + printf "dso=%s ip=0x%lx sym=%s\n", dloc->ms->map->dso->name, dloc->ip, dloc->ms->sym ? dloc->ms->sym->name : "(no symbol)"
[Severity: Low]
Does this command fail on ASan/LSan builds?
The comment correctly notes that REFCNT_CHECKING wraps struct map and struct dso in a proxy requiring ->orig access, but the printf statement hardcodes the standard non-ASan access path (dloc->ms->map->dso->name).
This causes the perf-dso GDB macro to fail to evaluate and abort when running perf-stuck.sh -g against an ASan-instrumented perf process.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Namhyung Kim
maintainerRe: [PATCH 3/4] perf scripts: Add perf-stuck, to tell where a running perf is stuck
In reply to Arnaldo Carvalho de MeloQuoted 8 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
A perf that takes forever is hard to tell apart from one stuck in a loop, and there is no way to see where without attaching gdb.
perf-stuck.shsamples a running process' /proc entries and its progress line at a fixed interval, and with -g runs gdb (perf-stuck.gdb, adding the perf-die-chain command) when no progress is made across two samples, printing the DIE chain a DWARF type chase is stuck in.
Probably you need to check if gdb is available first.
Thanks,
Namhyung
Quoted 323 lines
It is a prototype: the plan is to turn it into a first class 'perf stuck' command. The process name is resolved with pgrep among the caller's own processes only, as root an unscoped one would attach gdb to the first process of any user with a matching name.
Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/scripts/perf-stuck.gdb | 109 ++++++++++++++++++ tools/perf/scripts/perf-stuck.sh | 184 ++++++++++++++++++++++++++++++ 2 files changed, 293 insertions(+) create mode 100644 tools/perf/scripts/perf-stuck.gdb create mode 100755 tools/perf/scripts/perf-stuck.shtools/perf/scripts/perf-stuck.gdb new file +109
-0@@ -0,0 +1,109 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# gdb commands for a stuck perf, used by perf-stuck.sh -g and usable directly: +# +# gdb -p $(pgrep -x perf) -batch -x perf-stuck.gdb -ex bt +# +# PROTOTYPE: part of the perf-stuck.sh stopgap, wants to become a first +# class 'perf stuck' command printing these DIE chains without gdb. +# +# The commands are for the DWARF type chasers in util/dwarf-aux.c: +# +# perf-die-chain <function> <die variable> [iterations] +# perf-die-chain-all [iterations] +# perf-dso +# +# For each iteration of the chasing loop they print the DIE address, the +# CU, its file offset, tag and name: a cycle shows as the same (addr, cu) +# pairs repeating, and a CU changing between iterations means the chase +# hops between a debug file and its dwz common file. + +set pagination off +set confirm off +set debuginfod enabled off +set print pretty on +set height 0 +set width 0 + +define perf-die-chain + if $argc < 2 + printf "usage: perf-die-chain <function> <die variable> [iterations]\n" + else + frame function $arg0 + if $argc == 3 + set $perf_die_chain_n = $arg2 + else + set $perf_die_chain_n = 10 + end + set $perf_die_chain_head = $pc + set $perf_die_chain_i = 0 + while $perf_die_chain_i < $perf_die_chain_n + # Pointer type DIEs have no DW_AT_name, so dwarf_diename() can + # return NULL: printf %s of it would error out and abort this + # batch script, handle it. + set $perf_die_chain_name = (char *) dwarf_diename($arg1) + printf "chain[%d] die=%p addr=%p cu=%p off=0x%lx tag=%d name=", $perf_die_chain_i, $arg1, $arg1->addr, $arg1->cu, ((Dwarf_Off) dwarf_dieoffset($arg1)), ((int) dwarf_tag($arg1)) + if $perf_die_chain_name == 0 + printf "(null)\n" + else + printf "%s\n", $perf_die_chain_name + end + until *$perf_die_chain_head + set $perf_die_chain_i = $perf_die_chain_i + 1 + end + end +end + +document perf-die-chain +Print the DIE chain being walked by a DWARF type chasing loop. +usage: perf-die-chain <function> <die variable> [iterations] + perf-die-chain die_get_pointer_type type_die + perf-die-chain __die_get_real_type vr_die + perf-die-chain die_get_real_type vr_die +end + +define perf-die-chain-all + if $argc == 0 + set $perf_die_chain_n = 10 + else + set $perf_die_chain_n = $arg0 + end + if $_any_caller_is("die_get_pointer_type", 20) + printf "stuck in die_get_pointer_type():\n" + perf-die-chain die_get_pointer_type type_die $perf_die_chain_n + else + if $_any_caller_is("__die_get_real_type", 20) + printf "stuck in __die_get_real_type():\n" + perf-die-chain __die_get_real_type vr_die $perf_die_chain_n + else + if $_any_caller_is("die_get_real_type", 20) + printf "stuck in die_get_real_type():\n" + perf-die-chain die_get_real_type vr_die $perf_die_chain_n + else + printf "not in a DWARF type chaser, try: bt\n" + end + end + end +end + +document perf-die-chain-all +Find which DWARF type chaser the process is in and print the DIE chain. +usage: perf-die-chain-all [iterations] +end + +define perf-dso + if $_any_caller_is("find_data_type", 20) + frame function find_data_type + # REFCNT_CHECKING, implied by an ASan/LSan build, wraps struct map and + # struct dso in a proxy keeping the real object in ->orig, so there the + # dso is map->orig->dso->orig, giving map->orig->dso->orig->name here; + # struct symbol is not wrapped, so sym->name needs no ->orig. + printf "dso=%s ip=0x%lx sym=%s\n", dloc->ms->map->dso->name, dloc->ip, dloc->ms->sym ? dloc->ms->sym->name : "(no symbol)" + else + printf "not in find_data_type()\n" + end +end + +document perf-dso +Print the dso, ip and symbol of the data location being resolved. +endtools/perf/scripts/perf-stuck.sh new file +184
-0@@ -0,0 +1,184 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# perf-stuck - tell a spinning perf apart from a blocked or recursing one +# +# Arnaldo Carvalho de Melo <[email protected]> +# +# PROTOTYPE: wants to become a first class 'perf stuck' command, sampling +# a process from inside perf, with the knowledge of perf's phases and of +# the DWARF type chasing loops built in, instead of poking /proc and +# shelling out to gdb. +# +# Samples /proc/<pid> at a fixed interval and prints the CPU time used +# since the previous sample, the [stack] mapping start and size, and the +# last line of a progress log when one is given, e.g. the stderr of +# 'perf report --progress': burning a full interval with a constant +# stack is a loop, a [stack] start moving down is runaway recursion. +# +# With -g it runs gdb (perf-stuck.gdb) when no progress is made for two +# consecutive samples, printing the DIE chain a DWARF type chase is +# walking. +# +# usage: perf-stuck.sh [options] <pid|process-name> + +set -u + +usage() { + cat <<-EOF + usage: perf-stuck.sh [options] <pid|process-name> + + -i <secs> sampling interval (default: 10) + -n <count> stop after this many samples (default: watch till it exits) + -l <file> progress log, its last line is printed with every sample + -g run gdb with perf-stuck.gdb when no progress is made for + two consecutive samples, writing the output to a temp file + -x <file> use this gdb command file instead of perf-stuck.gdb + -h this help + EOF + exit "${1:-0}" +} + +interval=10 +count=0 +progress_log= +use_gdb= +gdb_cmds= + +while getopts "i:n:l:gx:h" opt; do + case "$opt" in + i) interval=$OPTARG ;; + n) count=$OPTARG ;; + l) progress_log=$OPTARG ;; + g) use_gdb=1 ;; + x) gdb_cmds=$OPTARG ;; + h) usage 0 ;; + *) usage 1 ;; + esac +done +shift $((OPTIND - 1)) + +[ $# -eq 1 ] || usage 1 + +if [[ "$1" =~ ^[0-9]+$ ]]; then + pid=$1 +else + # Resolve the name against the caller's own processes: as root, + # unscoped pgrep picks the first match of any user, e.g. one planted + # to get gdb attached to it, use an explicit pid to look at a perf of + # another user. + pid=$(pgrep -x -u "$(id -u)" -- "$1" | head -1) + [ -n "$pid" ] || { echo "no process named '$1' owned by $(id -un)"; exit 1; } +fi + +[ -d /proc/"$pid" ] || { echo "no process $pid"; exit 1; } + +if [ -n "$use_gdb" ] && [ -z "$gdb_cmds" ]; then + gdb_cmds=$(dirname "$0")/perf-stuck.gdb + [ -r "$gdb_cmds" ] || { echo "cannot read $gdb_cmds"; exit 1; } +fi + +hz=$(getconf CLK_TCK) +psz=$(getconf PAGESIZE) +prev_cpu= +prev_stack= +prev_progress= +stuck=0 +gdb_done= +nsample=0 + +# The command line is whatever the process was started with, so drop the +# control characters from it: a process started with escape sequences in +# its arguments, e.g. one replaying a log line, would otherwise get them +# replayed on the terminal of whoever runs this. +cmdline=$(tr '\0' ' ' < /proc/"$pid"/cmdline | tr -d '[:cntrl:]') + +echo "watching $pid ($cmdline) every ${interval}s" + +while :; do + if [ ! -d /proc/"$pid" ]; then + echo "$(date +%T) process gone" + break + fi + + # Field 2, the command name, is in parentheses and can contain + # spaces, so drop it together with the pid before splitting so the + # fields line up. %d keeps the CPU time out of scientific + # notation, that bash arithmetic can't parse past six digits. + if ! stat_line=$(awk '{ sub(/^[^ ]+ \(.*\) /, ""); + printf "%s %d %d\n", $1, $12 + $13, $22 }' \ + /proc/"$pid"/stat 2>/dev/null); then + echo "$(date +%T) process gone" + break + fi + + # The process can be gone between the check above and this read, in + # which case there is nothing to report: 'set -u' would otherwise + # turn the unbound fields into an aborted script. + if [ -z "$stat_line" ]; then + echo "$(date +%T) process gone" + break + fi + + stat=($stat_line) + state=${stat[0]} + cpu=${stat[1]} + # field 24 of /proc/<pid>/stat, the resident set size in pages + rss=$(( stat[2] * psz / 1024 )) + + stack=$(awk '/\[stack\]/{print $1; exit}' /proc/"$pid"/maps 2>/dev/null) + if [ -n "$stack" ]; then + stack_start=0x${stack%-*} + stack_size=$(( 0x${stack#*-} - stack_start )) + stack_txt="$stack size=$((stack_size / 1024))kB" + else + stack_start= + stack_txt="-" + fi + + progress= + [ -n "$progress_log" ] && [ -s "$progress_log" ] && progress=$(tail -1 -- "$progress_log") + + if [ -n "$prev_cpu" ]; then + cpu_delta=$(( cpu - prev_cpu )) + # No progress log, or one with nothing written to it yet, leaves + # no progress to look at, so count the samples that show none: + # -g then looks at where the process is after two intervals. + # Otherwise count the repeats of the same last line. + if [ -z "$progress" ] || [ "$progress" = "$prev_progress" ]; then + stuck=$((stuck + 1)) + else + stuck=0 + fi + stuck_txt="stuck=${stuck}" + [ "$stack_start" != "$prev_stack" ] && stuck_txt="$stuck_txt STACK" + else + cpu_delta=0 + stuck_txt="" + fi + + printf '%s state=%s cpu=+%d (%d.%02ds) rss=%dkB stack=%s %s %s\n' \ + "$(date +%T)" "$state" "$cpu_delta" \ + $(( cpu_delta / hz )) $(( (cpu_delta % hz) * 100 / hz )) \ + "$rss" "$stack_txt" "$stuck_txt" "${progress:-(no progress log)}" + + if [ -n "$use_gdb" ] && [ -z "$gdb_done" ] && [ "$stuck" -ge 2 ]; then + gdb_log=$(mktemp /tmp/perf-stuck-gdb.XXXXXX) + # The gdb script makes inferior calls, and a call into a perf + # wedged in a loop never returns, so bound the run; SIGINT + # releases the inferior instead of leaving perf stopped. + timeout --signal=INT 30 gdb -p "$pid" -batch -x "$gdb_cmds" -ex bt \ + -ex 'perf-die-chain-all' -ex perf-dso -ex detach > "$gdb_log" 2>&1 + gdb_done=1 + echo "... gdb output of $pid in $gdb_log" + fi + + prev_cpu=$cpu + prev_stack=$stack_start + prev_progress=$progress + + nsample=$((nsample + 1)) + [ "$count" -gt 0 ] && [ "$nsample" -ge "$count" ] && break + + sleep "$interval" +doneSignature · 1 line
2.55.0
Arnaldo Carvalho de Melo
maintainerPATCH v3 4/4 perf test: Add false_sharing workload exhibiting cross-CPU false sharing
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Add a 'perf test -w false_sharing' workload that hammers one shared struct from several CPUs, shaped as a TCP connection: a read-mostly identity (five-tuple) shares a cacheline with per-packet rx counters (the false-sharing line), a second line has packet-path private tx and congestion control counters, and a third the connection config.
The packet path runs in the main thread and up to four lookup threads hash the five-tuple and pull the config, reading one volatile shared instance directly so the accesses are PC-relative and resolvable by the data type profiler.
--- tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/shell/data_type_profiling.sh | 9 +- tools/perf/tests/tests.h | 1 + tools/perf/tests/workloads/Build | 2 + tools/perf/tests/workloads/false_sharing.c | 240 ++++++++++++++++++ 5 files changed, 251 insertions(+), 2 deletions(-) create mode 100644 tools/perf/tests/workloads/false_sharing.c
tools/perf/tests/builtin-test.c +1 -0
@@ -175,6 +175,7 @@ static struct test_workload *workloads[] = { &workload__context_switch_loop, &workload__deterministic, &workload__callchain, + &workload__false_sharing, #ifdef HAVE_RUST_SUPPORT &workload__code_with_type,
tools/perf/tests/shell/data_type_profiling.sh +7 -2
@@ -8,8 +8,8 @@ set -e # data type profiling manifestation # Values in testtypes and testprogs should match -testtypes=("# data-type: struct Buf" "# data-type: struct buf") -testprogs=("perf test -w code_with_type" "perf test -w datasym") +testtypes=("# data-type: struct Buf" "# data-type: struct buf" "# data-type: struct net_conn") +testprogs=("perf test -w code_with_type" "perf test -w datasym" "perf test -w false_sharing") err=0 perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
@@ -59,6 +59,9 @@ test_basic_annotate() { "xC") index=1 ;; + + "xFS") + index=2 ;; esac # Under 'set -e' a bare failing command aborts the script through the EXIT
@@ -114,6 +117,8 @@ test_basic_annotate Basic Rust test_basic_annotate Pipe Rust test_basic_annotate Basic C test_basic_annotate Pipe C +test_basic_annotate Basic FS +test_basic_annotate Pipe FS cleanup exit $err
tools/perf/tests/tests.h +1 -0
@@ -250,6 +250,7 @@ DECLARE_WORKLOAD(jitdump);DECLARE_WORKLOAD (context_switch_loop);DECLARE_WORKLOAD (deterministic);DECLARE_WORKLOAD (callchain); +DECLARE_WORKLOAD (false_sharing);#ifdef HAVE_RUST_SUPPORT DECLARE_WORKLOAD (code_with_type);
tools/perf/tests/workloads/Build +2 -0
@@ -14,6 +14,7 @@ perf-test-y += jitdump.o perf-test-y += context_switch_loop.o perf-test-y += deterministic.o perf-test-y += callchain.o +perf-test-y += false_sharing.o ifeq ($(CONFIG_RUST_SUPPORT),y) perf-test-y += code_with_type.o
@@ -29,3 +30,4 @@ CFLAGS_inlineloop.o = -g -O2 CFLAGS_deterministic.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_named_threads.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_callchain.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE +CFLAGS_false_sharing.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE
tools/perf/tests/workloads/false_sharing.c new file +240 -0
240 lines not on this page · raw message
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH 4/4] perf test: Add false_sharing workload exhibiting cross-CPU false sharing
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] perf test: fs_reader padding calculation ignores implicit padding --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] perf test: fs_reader padding calculation ignores implicit padding --
commit 3c1c77347e93202ea6f68ec871d8e60e5a4defed
Author: Arnaldo Carvalho de Melo <[email protected]>
perf test: Add false_sharing workload exhibiting cross-CPU false sharing
This commit adds a 'perf test -w false_sharing' workload that hammers a shared struct from several CPUs, shaped as a TCP connection. The workload creates threads sharing cache lines to test the data type profiler's ability to resolve false sharing.
Quoted 5 lines
tools/perf/tests/workloads/false_sharing.c new file +0
-0
[ ... ]
tools/perf/tests/workloads/false_sharing.c line 66
Quoted 5 lines
tools/perf/tests/workloads/false_sharing.c new file +5
-0+struct fs_reader { +pthread_t thread; +int cpu; +unsigned long sum; +char __pad[64 -sizeof (pthread_t) -sizeof (int ) -sizeof (unsigned long )];
[Severity: Low]
Does the manual padding computation in struct fs_reader ignore implicit compiler padding?
On 64-bit architectures, there are 4 bytes of implicit compiler padding required between the 4-byte cpu and the 8-byte sum. As a result, the struct fields occupy 68 bytes before the __attribute__((aligned(64))) rounds the final struct size up to 128 bytes, effectively doubling the intended footprint and wasting 64 bytes per array element.
Quoted 1 line
+} __attribute__((aligned(64)));
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4