perf tools: Add progress diagnostics and a false-sharing workload
Arnaldo Carvalho de Melo
maintainerv12 cover perf tools: Add progress diagnostics and a false-sharing workload
Hi,
This series adds progress diagnostics to perf, and a workload that makes false sharing visible to data type profiling.
The changes are:
- move perf_config__set_variable() to util/config.c and serialize config parser and read-modify-write state, so non-builtin perf code can persist configuration changes safely;
- add 'perf report --progress' for stdio users, showing the current phase,
percentage, and counts while a session is processed;
- wire up 'perf report --no-progress', the counterpart of the option above, for the TUI and GTK browsers, whose progress there is no other way to turn off;
- add 'perf test -w false_sharing', a synthetic TCP-shaped workload with identity and packet counters sharing a cacheline, and include it in the data type profiling shell test.
Follow up work: the DO_ONCE() one-time init primitive added here mirrors what the eight pre-existing raw pthread_once() users in util/ (annotate.c, callchain.c, comm.c, dso.c, fncache.c, intel-tpebs.c, libbfd.c, pmus.c) need, conversions that will also exercise the DEFINE_MUTEX() static initializer; converting them is left for after this series. Best regards,
- Arnaldo
What changed from v11:
- The 'perf-stuck' patch is removed for the time being, to be submitted
separately later on;
What changed from v10:
- tools/perf/scripts/perf-stuck.sh: pin the watched process by its start
time, so a recycled PID can't get samples or gdb; Sashiko, v10;
- tools/perf/scripts/perf-stuck.sh: drop the control characters from the
shown progress updates, no terminal escape injection; Sashiko, v10;
- tools/perf/tests/shell/perf_stuck.sh: test the script, from option
validation to the gdb DIE chain runs on stand-in workloads;
What changed from v9:
- tools/perf/ui/stdio/progress.c: progress updates now work well with
the pager, printed to /dev/tty so they update in place;
- tools/perf/scripts/perf-stuck.sh: take the last progress update
from a bounded tail window, dropping the \r's. Sashiko, v9;
What changed from v8:
- tools/perf/util/mutex.h: DEFINE_MUTEX() statics keep mutex_init()'s
!NDEBUG errorcheck type;
- patch 4/9 commit log: clarify the acquire/release pairing, what the dropped static key used to guarantee.
What changed from v7:
- tools/perf/util/mutex.h: the DO_ONCE() lockless fast path pairs its
__ATOMIC_ACQUIRE load with the __ATOMIC_RELEASE store. sashiko, v7;
What changed from v6:
- patch 1/5 split into four: the move, perf_etc_perfconfig() never returning NULL, the DEFINE_MUTEX() prep and the serialization. Namhyung Kim asked, v6;
- tools/perf/util/{config.c,mutex.h}: perf's own mutex type, adding
the DEFINE_MUTEX() initializer it lacked. Namhyung Kim, v6;
- tools/perf/util/mutex.h: add DO_ONCE() one-time init, from the
kernel's once.h, moving the lazy inits in config.c to it;
- tools/perf/builtin-report.c: drop the option negation and stdio
hook comments Namhyung Kim asked to drop, reviewing v6;
- tools/perf/scripts/perf-stuck.gdb: break the long printf() line.
Namhyung Kim, reviewing v6;
- the false_sharing cset now carries the data type profiling output with cacheline info. Namhyung Kim asked for it, reviewing v6.
What changed from v5:
- tools/perf/util/config.c: drop the config_file_name read in
bad_config(), locking would buy a consistent NULL. Sashiko, v5;
- tools/perf/util/config.c: new config_set_mutex around the shared
set's init/teardown, home init via pthread_once. Sashiko, v5;
- tools/perf/builtin-report.c, perf-report.txt: --no-progress is --progress's auto negation, last wins. Namhyung Kim, reviewing v5.
What changed from v4:
- tools/perf/builtin-report.c: mark --progress PARSE_OPT_NOAUTONEG,
parse_long_opt() claimed --no-progress first. Sashiko, v4;
- tools/perf/util/config.c: format the path buffer inside the critical
section. Sashiko pointed out the window, reviewing v4;
- tools/perf/tests/workloads/false_sharing.c: walk every bit of the affinity mask, not _SC_NPROCESSORS_CONF. Sashiko, reviewing v4.
What changed from v3:
- tools/perf/util/config.c: keep the buffer handed to the parser in
static storage. Sashiko, reviewing v3;
- tools/perf/scripts/perf-stuck.gdb: perf-dso walks each dso candidate
until one evaluates, for REFCNT_CHECKING builds. Sashiko, v3;
- tools/perf/tests/workloads/false_sharing.c: sum before cpu in struct
fs_reader, the padding after cpu rounded it to 128. Sashiko, v3;
- tools/perf/builtin-report.c: --quiet wins over --progress, the phases
stay uncounted, documented next to it. Namhyung Kim, v3;
- tools/perf/builtin-report.c, tools/perf/ui/progress.c: --no-progress
installs the no-op ops. Suggested by Namhyung Kim, reviewing v3;
- tools/perf/scripts/perf-stuck.sh: check gdb is there before watching. Namhyung Kim, reviewing v3.
What changed from v2:
- tools/perf/util/config.c: make perf_etc_perfconfig() total, falling
back to the unresolved path. Sashiko, reviewing v2;
- tools/perf/scripts/perf-stuck.gdb: don't deref map_symbol.sym without
a NULL check. Sashiko, reviewing v2;
- tools/perf/scripts/perf-stuck.gdb: note the REFCNT_CHECKING proxy
next to the structure walks;
- tools/perf/scripts/perf-stuck.sh: count samples with no progress to
look at, an empty log never fired -g;
- tools/perf/scripts/perf-stuck.sh: -- for pgrep and tail against
names starting with a hyphen;
- tools/perf/scripts/perf-stuck.sh: bound the gdb run with
`timeout --signal=INT 30`.
What changed from v1:
- avoid calling CPU_SET() with -1 when false_sharing runs with only one CPU available in its affinity mask.
tools/perf/Documentation/perf-report.txt | 19 ++ tools/perf/builtin-config.c | 70 +---- tools/perf/builtin-report.c | 9 + tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/shell/data_type_profiling.sh | 9 +- tools/perf/tests/tests.h | 1 + tools/perf/tests/workloads/Build | 2 + tools/perf/tests/workloads/false_sharing.c | 251 ++++++++++++++++++ tools/perf/ui/Build | 1 + tools/perf/ui/progress.c | 6 + tools/perf/ui/progress.h | 4 + tools/perf/ui/stdio/progress.c | 170 ++++++++++++ tools/perf/util/config.c | 166 ++++++++++-- tools/perf/util/config.h | 2 + tools/perf/util/mutex.h | 36 +++ tools/perf/util/ordered-events.c | 16 +- tools/perf/util/session.c | 12 +- 17 files changed, 675 insertions(+), 100 deletions(-)
create mode 100644 tools/perf/tests/workloads/false_sharing.c create mode 100644 tools/perf/ui/stdio/progress.c
base-commit: 705da5b15ab89ba9
v1-head: 45d7917f7e05e8a29828ed5f0bbdc94fc938f79f
v2-head: d4f84e4de8890194924ccd897a8e6773e7d4240b
v3-head: 485532296710225862daf8ffb19802ae327efe2a
v4-head: 38193635508e0a5f04a6ebf70db27534b68b2d5b
v5-head: e99bc48f6c72b9cc67b4e4143a0364acac5523bb
v6-head: 7dc99dbb6504f199c3c39487d91370d1bbb7a827
v7-head: b357a76666d08f74eb99c755cca41855a96f6169
v8-head: 257b5ea6115ad5126d2c881523ea6c660cf5d28c
v9-head: 70235104c6cf7d18d91f46de1a233abbeb65893a
v10-head: 1ecf850c2694d20ca9cd0014f2a42528ef4dbfec
v11-head: a3ddbaab18aa24fba520927f71d0ea66edfc5593
--
Arnaldo Carvalho de Melo
maintainerPATCH v12 1/8 perf config: Move perf_config__set_variable() to util/config.c
From: Arnaldo Carvalho de Melo <[email protected]>
Move perf_config__set_variable() out of the 'perf config' builtin so that opt-in features can persist their choice from outside it, e.g. util/debuginfo.c writing core.debuginfod=false when the user disables debuginfod for the rest of the session. The set_config() body becomes perf_config_set__write(), with the system_config choice as an argument, as the builtin's use_system_config/use_user_config statics are not available outside it.
perf_config_set__write() checked fopen() but none of the fprintf()s or fclose(), so a write failure after truncating the file was reported as success. Harmless for the interactive 'perf config' this came from, but this makes it an entry point a background feature can call with no other feedback, so propagate those errors too.
--- tools/perf/builtin-config.c | 70 +--------------------------------- tools/perf/util/config.c | 76 +++++++++++++++++++++++++++++++++++++ tools/perf/util/config.h | 2 + 3 files changed, 79 insertions(+), 69 deletions(-)
tools/perf/builtin-config.c +1 -69
@@ -41,37 +41,7 @@ static struct option config_options[] = {static int set_config (struct perf_config_set *set,const char *file_name) { -struct perf_config_section *section =NULL ; -struct perf_config_item *item =NULL ; -const char *first_line ="# this file is auto-generated." ; -FILE *fp; - -if (set ==NULL ) -return -1 ; - - fp =fopen (file_name,"w" ); -if (!fp) -return -1 ; - -fprintf (fp,"%s\n" , first_line); - -/* overwrite configvariables */ -perf_config_items__for_each_entry (&set->sections, section) { -if (!use_system_config && section->from_system_config) -continue ; -fprintf (fp,"[%s]\n" , section->name); - -perf_config_items__for_each_entry (§ion->items, item) { -if (!use_system_config && item->from_system_config) -continue ; -if (item->value) -fprintf (fp,"\t%s = %s\n" , - item->name, item->value); - } - } -fclose (fp); - -return 0 ; +return perf_config_set__write (set, file_name, use_system_config); }static int show_spec_config (struct perf_config_set *set,const char *var)
@@ -158,44 +128,6 @@ static int parse_config_arg(char *arg, char **var, char **value)return 0 ; } -int perf_config__set_variable (const char *var,const char *value) -{ -char path[PATH_MAX ]; -char *user_config =mkpath (path,sizeof (path),"%s/.perfconfig" ,getenv ("HOME" )); -const char *config_filename; -struct perf_config_set *set; -int ret =-1 ; - -if (use_system_config) - config_exclusive_filename =perf_etc_perfconfig (); -else if (use_user_config) - config_exclusive_filename = user_config; - -if (!config_exclusive_filename) - config_filename = user_config; -else - config_filename = config_exclusive_filename; - - set =perf_config_set__new (); -if (!set) - goto out_err; - -if (perf_config_set__collect (set, config_filename, var, value) <0 ) { -pr_err ("Failed to add '%s=%s'\n" , var, value); - goto out_err; - } - -if (set_config (set, config_filename) <0 ) { -pr_err ("Failed to set the configs on %s\n" , config_filename); - goto out_err; - } - - ret =0 ; -out_err: -perf_config_set__delete (set); -return ret; -} -int cmd_config (int argc,const char **argv) {int i, ret =-1 ;
tools/perf/util/config.c +76 -0
88 lines not on this page · raw message
tools/perf/util/config.h +2 -0
@@ -33,6 +33,8 @@ int perf_config_scan(const char *name, const char *fmt, ...) __scanf(2, 3);const char *perf_config_get (const char *name);int perf_config_set (struct perf_config_set *set,config_fn_t fn,void *data); +int perf_config_set__write (struct perf_config_set *set, +const char *file_name,bool system_config);int perf_config_int (int *dest,const char *,const char *);int perf_config_u8 (u8 *dest,const char *name,const char *value);int perf_config_u64 (u64 *dest,const char *,const char *);
Signature · 1 line
2.55.0
Arnaldo Carvalho de Melo
maintainerPATCH v12 2/8 perf config: Make perf_etc_perfconfig() never return NULL
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
perf_etc_perfconfig() returns what system_path() gives, and that allocates, so on failure every caller dereferenced NULL, the pre-existing ones in perf_config_set__init() and in the daemon included. For the usual absolute ETC_PERFCONFIG system_path() just returns a strdup of it, so use ETC_PERFCONFIG itself when that fails.
--- tools/perf/util/config.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
tools/perf/util/config.c +7 -1
@@ -571,8 +571,14 @@ static int perf_config_from_file(config_fn_t fn, const char *filename, void *datconst char *perf_etc_perfconfig (void ) {static const char *system_wide; +if (!system_wide) - system_wide =system_path (ETC_PERFCONFIG ); +/* +* ETC_PERFCONFIG is absolute, so its unresolved path is +* the same string, better than the callers crashing. +*/ + system_wide =system_path (ETC_PERFCONFIG ) ?:ETC_PERFCONFIG ; +return system_wide; }
Signature · 1 line
2.55.0
Arnaldo Carvalho de Melo
maintainerPATCH v12 3/8 perf mutex: Add DEFINE_MUTEX() static initializer
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
For file scope mutexes, so that they don't need a constructor function to initialize them at startup, mirroring PTHREAD_MUTEX_INITIALIZER while keeping the clang -Wthread-safety annotations of struct mutex.
DEBUG=1 builds have mutex_init() set PTHREAD_MUTEX_ERRORCHECK, making the CHECK_ERR() paths in mutex_lock()/mutex_unlock() report relocking a held mutex and unlocking one that isn't held. PTHREAD_MUTEX_INITIALIZER gives a default type mutex, so statically initialized ones would silently lose that: use PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP, a glibc extension, falling back to the default type in libcs that lack it.
--- tools/perf/util/mutex.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)
tools/perf/util/mutex.h +12 -0
@@ -92,6 +92,18 @@ struct LOCKABLE mutex {pthread_mutex_t lock; }; +/* +* Statically initialized mutex, for the file scope ones. Error checking +* in !NDEBUG builds, like mutex_init(); the glibc-only errorcheck +* initializer falls back to the default type elsewhere. +*/ +#if !defined(NDEBUG ) && defined(PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP ) +#define __PERF_MUTEX_INITIALIZER PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP +#else +#define __PERF_MUTEX_INITIALIZER PTHREAD_MUTEX_INITIALIZER +#endif +#define DEFINE_MUTEX (name) struct mutex name = { .lock = __PERF_MUTEX_INITIALIZER } +/* A wrapper around the condition variable implementation. */ struct cond {pthread_cond_t cond;
Signature · 1 line
2.55.0
Arnaldo Carvalho de Melo
maintainerPATCH v12 4/8 perf mutex: Add DO_ONCE() for one-time initialization
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
For lazy init that more than one thread can race into, without pthread_once() so that it stays within perf's locking primitives and clang's -Wthread-safety keeps an eye on the mutexes involved.
Mirrors the kernel's include/linux/once.h DO_ONCE(), dropping its static key fast path, unnecessary at these cold init paths: a per call site static bool plus a statically initialized mutex, double checked. As there, code reachable from more than one call site must go through a common helper.
The lockless fast path reads ___done with an acquire, paired with a release store after fn(), so that fn()'s side effects are visible to threads taking the fast path, which the dropped static key used to guarantee.
The __atomic builtins are used rather than smp_load_acquire()/ smp_store_release() because ThreadSanitizer understands them, avoiding reports on ___done and on what fn() initialized.
--- tools/perf/util/mutex.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
tools/perf/util/mutex.h +24 -0
@@ -104,6 +104,30 @@ struct LOCKABLE mutex {#endif #define DEFINE_MUTEX (name) struct mutex name = { .lock = __PERF_MUTEX_INITIALIZER } +/* +* Runs fn() exactly once however many threads race to it, the rest wait +* for it to finish; per call site state, so multiple sites wanting the +* same once use a common helper, as in the kernel's DO_ONCE(). +*/ +#define DO_ONCE (fn, ...) \ + ({ \ + static bool ___done; \ + static DEFINE_MUTEX(___once_lock); \ + bool ___ret = false; \ + \ + if (!__atomic_load_n(&___done, __ATOMIC_ACQUIRE)) { \ + mutex_lock(&___once_lock); \ + if (!___done) { \ + fn(__VA_ARGS__); \ + __atomic_store_n(&___done, true, \ + __ATOMIC_RELEASE); \ + ___ret = true; \ + } \ + mutex_unlock(&___once_lock); \ + } \ + ___ret; \ + }) +/* A wrapper around the condition variable implementation. */ struct cond {pthread_cond_t cond;
Signature · 1 line
2.55.0
Arnaldo Carvalho de Melo
maintainerPATCH v12 5/8 perf config: Serialize config file access with a mutex
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Config file access shares the static parser state and, since perf_config__set_variable() became an entry point for background features, can run on more than one thread: a feature writing the config while perf top's display thread reads it. Serialize parsing and rewriting with config_mutex and the whole read-modify-write of perf_config__set_variable() with config_update_mutex; config_set_mutex comes before it, as building the set parses the config files.
perf has its own mutex type, util/mutex.h; the file scope mutexes here use the DEFINE_MUTEX() static initializer added in the previous patch. The two lazy inits that system_path() and home_perfconfig() do would leak all but one of the strings racing on two threads, so they move to DO_ONCE(), added in the previous patch as well. bad_config() runs on the dispatching thread, outside config_mutex, so it stops reading config_file_name, which the parsing thread owns.
--- tools/perf/util/config.c | 106 +++++++++++++++++++++++++++------------ 1 file changed, 73 insertions(+), 33 deletions(-)
tools/perf/util/config.c +73 -33
@@ -31,6 +31,7 @@#include "callchain.h" #include "debug.h" #include "header.h" +#include "mutex.h" #include "path.h" #include "srcline.h" #include "unwind.h"
@@ -371,10 +372,8 @@ static int perf_parse_long(const char *value, long *ret)static void bad_config (const char *name) { -if (config_file_name) -pr_warning ("bad config value for '%s' in %s, ignoring...\n" , name, config_file_name); -else -pr_warning ("bad config value for '%s', ignoring...\n" , name); +/* config_file_name is owned by the parsing thread, under config_mutex. */ +pr_warning ("bad config value for '%s', ignoring...\n" , name); }int perf_config_u64 (u64 *dest,const char *name,const char *value)
@@ -550,11 +549,18 @@ int perf_default_config(const char *var, const char *value,return 0 ; } +/* Parsing and rewriting share the static parser state. */ +static DEFINE_MUTEX (config_mutex); + +/* Serializes whole perf_config__set_variable() updates. */ +static DEFINE_MUTEX (config_update_mutex); +static int perf_config_from_file (config_fn_t fn,const char *filename,void *data) {int ret;FILE *f =fopen (filename,"r" ); +mutex_lock (&config_mutex); ret =-1 ;if (f) { config_file = f;
@@ -565,21 +571,24 @@ static int perf_config_from_file(config_fn_t fn, const char *filename, void *datfclose (f); config_file_name =NULL ; } +mutex_unlock (&config_mutex);return ret; } -const char *perf_etc_perfconfig (void ) -{ -static const char *system_wide; +/* system_path() allocates, so it is computed once. */ +static const char *etc_perfconfig; -if (!system_wide) -/* -* ETC_PERFCONFIG is absolute, so its unresolved path is -* the same string, better than the callers crashing. -*/ - system_wide =system_path (ETC_PERFCONFIG ) ?:ETC_PERFCONFIG ; +static void perf_etc_perfconfig__init (void ) +{ + etc_perfconfig =system_path (ETC_PERFCONFIG ); +if (!etc_perfconfig) + etc_perfconfig =ETC_PERFCONFIG ; +} -return system_wide; +const char *perf_etc_perfconfig (void ) +{ +DO_ONCE (perf_etc_perfconfig__init); +return etc_perfconfig; }static int perf_env_bool (const char *k,int def)
@@ -637,19 +646,18 @@ static char *home_perfconfig(void)return NULL ; } -const char *perf_home_perfconfig (void ) -{ -static const char *config; -static bool failed; - -if (failed || config) -return config; +/* home_perfconfig() allocates and warns, so it is computed once. */ +static const char *home_config; - config =home_perfconfig (); -if (!config) - failed = true; +static void perf_home_perfconfig__init (void ) +{ + home_config =home_perfconfig (); +} -return config; +const char *perf_home_perfconfig (void ) +{ +DO_ONCE (perf_home_perfconfig__init); +return home_config; } static struct perf_config_section *find_section(struct list_head *sections,
@@ -790,8 +798,15 @@ static int collect_config(const char *var, const char *value,int perf_config_set__collect (struct perf_config_set *set,const char *file_name,const char *var,const char *value) { +int ret; + +mutex_lock (&config_mutex); config_file_name = file_name; -return collect_config (var, value, set); + ret =collect_config (var, value, set); +/* Don't leave the static parser state pointing at the caller's buffer. */ + config_file_name =NULL ; +mutex_unlock (&config_mutex); +return ret; }static int perf_config_set__init (struct perf_config_set *set)
@@ -838,6 +853,10 @@ struct perf_config_set *perf_config_set__load_file(const char *file)return set; } +/* Not config_mutex: building the set parses the config files, which takes it. */ +static DEFINE_MUTEX (config_set_mutex); + +/* Called with config_set_mutex held. */ static int perf_config__init (void ) {if (config_set ==NULL )
@@ -878,16 +897,26 @@ int perf_config_set(struct perf_config_set *set,int perf_config (config_fn_t fn,void *data) { -if (config_set ==NULL &&perf_config__init ()) +struct perf_config_set *set; + +/* Not held across the dispatch: a callback can call perf_config() again. */ +mutex_lock (&config_set_mutex); +if (perf_config__init ()) { +mutex_unlock (&config_set_mutex);return -1 ; + } + set = config_set; +mutex_unlock (&config_set_mutex); -return perf_config_set (config_set, fn, data); +return perf_config_set (set, fn, data); }void perf_config__exit (void ) { +mutex_lock (&config_set_mutex);perf_config_set__delete (config_set); config_set =NULL ; +mutex_unlock (&config_set_mutex); } int perf_config_set__write(struct perf_config_set *set,
@@ -898,9 +927,12 @@ int perf_config_set__write(struct perf_config_set *set,int ret =0 ;FILE *fp; +mutex_lock (&config_mutex); fp =fopen (file_name,"w" ); -if (!fp) +if (!fp) { +mutex_unlock (&config_mutex);return -1 ; + }if (fprintf (fp,"# this file is auto-generated.\n" ) <0 ) ret =-1 ;
@@ -922,6 +954,7 @@ int perf_config_set__write(struct perf_config_set *set, }if (fclose (fp) !=0 ) ret =-1 ; +mutex_unlock (&config_mutex);return ret; }
@@ -933,14 +966,20 @@ int perf_config_set__write(struct perf_config_set *set,*/ int perf_config__set_variable (const char *var,const char *value) { -char path[PATH_MAX ]; -char *user_config =mkpath (path,sizeof (path),"%s/.perfconfig" ,getenv ("HOME" ));const char *config_filename;bool system_config; -struct perf_config_set *set; +struct perf_config_set *set =NULL ;int ret =-1 ; - config_filename = config_exclusive_filename ?: user_config; +mutex_lock (&config_update_mutex); + +/* Static: the parser publishes it as config_file_name. */ + { +static char path[PATH_MAX ]; +char *user_config =mkpath (path,sizeof (path),"%s/.perfconfig" ,getenv ("HOME" )); + + config_filename = config_exclusive_filename ?: user_config; + }/* Rewriting the system wide file keeps its entries, or it is truncated. */ system_config =strcmp (config_filename,perf_etc_perfconfig ()) ==0 ;
@@ -962,6 +1001,7 @@ int perf_config__set_variable(const char *var, const char *value) ret =0 ; out_err:perf_config_set__delete (set); +mutex_unlock (&config_update_mutex);return ret; }
Signature · 1 line
2.55.0
Arnaldo Carvalho de Melo
maintainerPATCH v12 6/8 perf report: Add --progress option
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Processing a large session with stdio output gives no feedback about which phase perf is in or how far along it is: ui_progress updates are only shown by the TUI. Add --progress, installing a stdio backend (ui/stdio/progress.c) that prints the phase title, percentage and counts:
Processing events... [ 42.3%] 317M / 746M
Phases can be nested, so the backend tracks the ones started to complete the right one on ui_progress__finish(); that requires init()/finish() pairs, fixed in ordered-events.c and the pipe and directory event processing.
With a pager both stdout and stderr lead to it, so isatty(stderr) turns false and the updates end up printed one per line in the pager's output: use /dev/tty in that case, so that the updates keep updating in place, untouched by the pager.
--- tools/perf/Documentation/perf-report.txt | 14 ++ tools/perf/builtin-report.c | 6 + tools/perf/ui/Build | 1 + tools/perf/ui/progress.h | 2 + tools/perf/ui/stdio/progress.c | 170 +++++++++++++++++++++++ tools/perf/util/ordered-events.c | 16 ++- tools/perf/util/session.c | 12 +- 7 files changed, 214 insertions(+), 7 deletions(-) create mode 100644 tools/perf/ui/stdio/progress.c
tools/perf/Documentation/perf-report.txt +14 -0
@@ -29,6 +29,20 @@ OPTIONS --quiet:: Do not show any warnings or messages. (Suppress -v) +--progress:: + Show progress for each of the processing phases, printing the + percentage done and the current/total counts: the first phase + counts the bytes of the perf.data file processed so far, the + merge and sort phases count hist entries, e.g.: + + Processing events... [ 42.3%] 4G / 10G + Merging related events... [ 7.1%] 1024 / 14387 + Sorting events for output... [ 98.2%] 14132 / 14387 + + It is a no-op when using the TUI or GTK browsers, that already + present progress information, or when --quiet is used, that asks + for no messages at all. + -n:: --show-nr-samples:: Show the number of samples for each symbol
tools/perf/builtin-report.c +6 -0
@@ -87,6 +87,7 @@ struct report {bool use_gtk;#endif bool use_stdio; +bool progress;bool show_full_info;bool show_threads;bool inverted_callchain;
@@ -1384,6 +1385,8 @@ int cmd_report(int argc, const char **argv) "Use the stdio interface"),OPT_BOOLEAN (0 ,"weights" , &symbol_conf.annotate_weight,"Show or hide weight columns in annotation. Default show if non-zero." ), +OPT_BOOLEAN (0 ,"progress" , &report.progress, +"Show progress while processing the perf.data file" ),OPT_BOOLEAN (0 ,"header" , &report.header,"Show data header." ),OPT_BOOLEAN (0 ,"header-only" , &report.header_only, "Show only data header."),
@@ -1790,6 +1793,9 @@ int cmd_report(int argc, const char **argv)else use_browser =0 ; +if (report.progress && !quiet && use_browser ==0 ) +stdio_progress__init (); +if (report.data_type && use_browser ==1 ) { symbol_conf.annotate_data_member = true; symbol_conf.annotate_data_sample = true;
tools/perf/ui/Build +1 -0
@@ -4,6 +4,7 @@ perf-ui-y += progress.o perf-ui-y += util.o perf-ui-y += hist.o perf-ui-y += stdio/hist.o +perf-ui-y += stdio/progress.o CFLAGS_setup.o += -DLIBDIR="BUILD_STR($(LIBDIR))"
tools/perf/ui/progress.h +2 -0
@@ -23,6 +23,8 @@ void __ui_progress__init(struct ui_progress *p, u64 total,void ui_progress__update (struct ui_progress *p,u64 adv); +void stdio_progress__init (void ); +struct ui_progress_ops {void (*init)(struct ui_progress *p);void (*update)(struct ui_progress *p);
tools/perf/ui/stdio/progress.c new file +170 -0
170 lines not on this page · raw message
tools/perf/util/ordered-events.c +12 -4
@@ -237,14 +237,16 @@ static int do_flush(struct ordered_events *oe, bool show_progress)ui_progress__init (&prog, oe->nr_events,"Processing time ordered events..." );list_for_each_entry_safe (iter, tmp, head, list) { -if (session_done ()) -return 0 ; +if (session_done ()) { + ret =0 ; + goto out_progress; + }if (iter->timestamp > limit)break ; ret = oe->deliver (oe, iter);if (ret <0 ) -return ret; + goto out_progress;ordered_events__delete (oe, iter); oe->last_flush = iter->timestamp;
@@ -258,10 +260,16 @@ static int do_flush(struct ordered_events *oe, bool show_progress)else if (last_ts <= limit) oe->last = list_entry(head->prev, struct ordered_event, list); + ret =0 ; +out_progress: +/* +* Always pair ui_progress__init() with ui_progress__finish(), the +* stdio backend tracks the phases on a stack. +*/ if (show_progress)ui_progress__finish (); -return 0 ; +return ret; } static int __ordered_events__flush(struct ordered_events *oe, enum oe_flush how,
tools/perf/util/session.c +9 -3
@@ -3253,8 +3253,12 @@ static int __perf_session__process_pipe_events(struct perf_session *session) cur_size =sizeof (union perf_event ); buf =malloc (cur_size); -if (!buf) -return -errno; +if (!buf) { + err = -errno; +if (update_prog) +ui_progress__finish (); +return err; + }ordered_events__set_copy_on_queue (oe, true); more: event = buf;
@@ -3753,8 +3757,10 @@ static int __perf_session__process_dir_events(struct perf_session *session) } rd =calloc (nr_readers,sizeof (struct reader )); -if (!rd) +if (!rd) { +ui_progress__finish ();return -ENOMEM ; + } rd[0 ] = (struct reader ) { .fd =perf_data__fd (session->data),
Signature · 1 line
2.55.0
Arnaldo Carvalho de Melo
maintainerPATCH v12 7/8 perf report: Add --no-progress option
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Now that --progress is being added for the stdio case, wire up its counterpart for the browsers: the TUI and GTK ones present progress of their own and there is no way to turn it off. Install the no-op ui_progress ops, the ones already used until a backend sets theirs, after setup_browser() installed the ones of the browser in use. The phases are still counted, nothing is shown for them, and no second option is needed for it: parse-options provides --no-progress as the negation of --progress, report.progress_set saying that it was asked for, report.progress being false both when nothing was asked for and when --no-progress was.
--- tools/perf/Documentation/perf-report.txt | 5 +++++ tools/perf/builtin-report.c | 9 ++++++--- tools/perf/ui/progress.c | 6 ++++++ tools/perf/ui/progress.h | 2 ++ 4 files changed, 19 insertions(+), 3 deletions(-)
tools/perf/Documentation/perf-report.txt +5 -0
@@ -43,6 +43,11 @@ OPTIONS present progress information, or when --quiet is used, that asks for no messages at all. +--no-progress:: + Do not show progress while processing the perf.data file. It + also turns off the progress the TUI and GTK browsers present, + which is their own. + -n:: --show-nr-samples:: Show the number of samples for each symbol
tools/perf/builtin-report.c +6 -3
@@ -88,6 +88,7 @@ struct report {#endif bool use_stdio;bool progress; +bool progress_set;bool show_full_info;bool show_threads;bool inverted_callchain;
@@ -1385,8 +1386,8 @@ int cmd_report(int argc, const char **argv) "Use the stdio interface"),OPT_BOOLEAN (0 ,"weights" , &symbol_conf.annotate_weight,"Show or hide weight columns in annotation. Default show if non-zero." ), -OPT_BOOLEAN (0 ,"progress" , &report.progress, -"Show progress while processing the perf.data file" ), +OPT_BOOLEAN_SET (0 ,"progress" , &report.progress, &report.progress_set, +"Show progress while processing the perf.data file" ),OPT_BOOLEAN (0 ,"header" , &report.header,"Show data header." ),OPT_BOOLEAN (0 ,"header-only" , &report.header_only, "Show only data header."),
@@ -1793,7 +1794,9 @@ int cmd_report(int argc, const char **argv)else use_browser =0 ; -if (report.progress && !quiet && use_browser ==0 ) +if (report.progress_set && !report.progress) +ui_progress__noop_init (); +else if (report.progress && !quiet && use_browser ==0 )stdio_progress__init ();if (report.data_type && use_browser ==1 ) {
tools/perf/ui/progress.c +6 -0
@@ -13,6 +13,12 @@ static struct ui_progress_ops null_progress__ops =struct ui_progress_ops *ui_progress__ops = &null_progress__ops; +/* Everything counts but nothing is shown, the way it starts out. */ +void ui_progress__noop_init (void ) +{ + ui_progress__ops = &null_progress__ops; +} +void ui_progress__update (struct ui_progress *p,u64 adv) {u64 last = p->curr;
tools/perf/ui/progress.h +2 -0
@@ -25,6 +25,8 @@ void ui_progress__update(struct ui_progress *p, u64 adv);void stdio_progress__init (void ); +void ui_progress__noop_init (void ); +struct ui_progress_ops {void (*init)(struct ui_progress *p);void (*update)(struct ui_progress *p);
Signature · 1 line
2.55.0
Arnaldo Carvalho de Melo
maintainerPATCH v12 8/8 perf test: Add false_sharing workload exhibiting cross-CPU false sharing
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Add a 'perf test -w false_sharing' workload that hammers one shared struct from several CPUs, shaped as a TCP connection: a read-mostly identity (five-tuple) shares a cacheline with per-packet rx counters (the false-sharing line), a second line has packet-path private tx and congestion control counters, and a third the connection config.
The packet path runs in the main thread and up to four lookup threads sum the five-tuple and pull the config, reading one volatile shared instance directly so the accesses are PC-relative and resolvable by the data type profiler.
Data type profiling of the workload with cacheline info:
$ perf mem record -- perf test -r5 -w false_sharing
[ perf record: Woken up 19 times to write data ]
[ perf record: Captured and wrote 5.612 MB perf.data (72454 samples) ]
$ perf report -s type,typecln -H --group --stdio
# Total Lost Samples: 0
#
# Samples: 72K of events 'cpu/mem-loads,ldlat=30/P, cpu/mem-stores/P'
# Event count (approx.): 1261814616
#
# Overhead Data Type / Data Type Cacheline
# ................... ...............................
#
88.99% 35.88% struct net_conn
56.43% 17.50% struct net_conn: cache-line 0
32.55% 0.00% struct net_conn: cache-line 2
0.01% 18.38% struct net_conn: cache-line 1
10.08% 63.03% (unknown)
10.08% 63.03% (unknown): cache-line 0
0.82% 0.13% int
0.82% 0.13% int: cache-line 0
0.01% 0.02% struct folio
0.01% 0.02% struct folio: cache-line 0
0.01% 0.03% Elf64_Addr
0.01% 0.03% Elf64_Addr: cache-line 0
0.01% 0.03% struct sched_entity
0.01% 0.03% struct sched_entity: cache-line 1
0.00% 0.00% struct sched_entity: cache-line 4
0.00% 0.00% struct sched_entity: cache-line 2
0.00% 0.00% struct sched_entity: cache-line 3
0.00% 0.00% struct sched_entity: cache-line 0
0.01% 0.00% struct task_group
0.01% 0.00% struct task_group: cache-line 4
0.00% 0.00% struct task_group: cache-line 5
0.00% 0.00% struct css_rstat_cpu
0.00% 0.00% struct css_rstat_cpu: cache-line 0
$ perf report -s type,typecln,typeoff -H --group --stdio
# Total Lost Samples: 0
#
# Samples: 72K of events 'cpu/mem-loads,ldlat=30/P, cpu/mem-stores/P'
# Event count (approx.): 1261814616
#
# Overhead Data Type / Data Type Cacheline / Data Type Offset
# ...................... ..................................................
#
88.99% 35.88% struct net_conn
56.43% 17.50% struct net_conn: cache-line 0
9.81% 0.00% struct net_conn +0xa (dport)
9.54% 0.00% struct net_conn +0x8 (sport)
9.53% 0.00% struct net_conn +0xc (state)
9.48% 0.00% struct net_conn +0xd (protocol)
9.27% 0.00% struct net_conn +0x4 (daddr)
8.79% 0.00% struct net_conn +0 (saddr)
0.00% 12.08% struct net_conn +0x10 (bytes_rx)
0.00% 1.71% struct net_conn +0x18 (packets_rx)
0.00% 2.80% struct net_conn +0x20 (rx_queue)
0.00% 0.91% struct net_conn +0x3c (last_ack)
32.55% 0.00% struct net_conn: cache-line 2
5.70% 0.00% struct net_conn +0x83 (rcv_wscale)
5.46% 0.00% struct net_conn +0x84 (keepalive_int)
5.43% 0.00% struct net_conn +0x82 (snd_wscale)
5.39% 0.00% struct net_conn +0x80 (mss)
5.34% 0.00% struct net_conn +0x88 (mark)
5.23% 0.00% struct net_conn +0x8c (priority)
0.01% 18.38% struct net_conn: cache-line 1
0.01% 0.05% struct net_conn +0x5c (retrans)
0.00% 4.83% struct net_conn +0x40 (bytes_tx)
0.00% 10.63% struct net_conn +0x48 (packets_tx)
0.00% 1.54% struct net_conn +0x58 (rtt_us)
0.00% 1.07% struct net_conn +0x50 (cwnd)
0.00% 0.25% struct net_conn +0x54 (ssthresh)
10.08% 63.03% (unknown)
10.08% 63.03% (unknown): cache-line 0
10.08% 63.03% (unknown)
0.82% 0.13% int
0.82% 0.13% int: cache-line 0
0.82% 0.13% int +0 (no field)
0.01% 0.02% struct folio
0.01% 0.02% struct folio: cache-line 0
0.01% 0.00% struct folio +0 (flags.f)
0.00% 0.01% struct folio +0x34 (_refcount.counter)
0.00% 0.00% struct folio +0x18 (mapping)--- tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/shell/data_type_profiling.sh | 9 +- tools/perf/tests/tests.h | 1 + tools/perf/tests/workloads/Build | 2 + tools/perf/tests/workloads/false_sharing.c | 251 ++++++++++++++++++ 5 files changed, 262 insertions(+), 2 deletions(-) create mode 100644 tools/perf/tests/workloads/false_sharing.c
tools/perf/tests/builtin-test.c +1 -0
@@ -175,6 +175,7 @@ static struct test_workload *workloads[] = { &workload__context_switch_loop, &workload__deterministic, &workload__callchain, + &workload__false_sharing, #ifdef HAVE_RUST_SUPPORT &workload__code_with_type,
tools/perf/tests/shell/data_type_profiling.sh +7 -2
@@ -8,8 +8,8 @@ set -e # data type profiling manifestation # Values in testtypes and testprogs should match -testtypes=("# data-type: struct Buf" "# data-type: struct buf") -testprogs=("perf test -w code_with_type" "perf test -w datasym") +testtypes=("# data-type: struct Buf" "# data-type: struct buf" "# data-type: struct net_conn") +testprogs=("perf test -w code_with_type" "perf test -w datasym" "perf test -w false_sharing") err=0 perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
@@ -59,6 +59,9 @@ test_basic_annotate() { "xC") index=1 ;; + + "xFS") + index=2 ;; esac # Under 'set -e' a bare failing command aborts the script through the EXIT
@@ -114,6 +117,8 @@ test_basic_annotate Basic Rust test_basic_annotate Pipe Rust test_basic_annotate Basic C test_basic_annotate Pipe C +test_basic_annotate Basic FS +test_basic_annotate Pipe FS cleanup exit $err
tools/perf/tests/tests.h +1 -0
@@ -250,6 +250,7 @@ DECLARE_WORKLOAD(jitdump);DECLARE_WORKLOAD (context_switch_loop);DECLARE_WORKLOAD (deterministic);DECLARE_WORKLOAD (callchain); +DECLARE_WORKLOAD (false_sharing);#ifdef HAVE_RUST_SUPPORT DECLARE_WORKLOAD (code_with_type);
tools/perf/tests/workloads/Build +2 -0
@@ -14,6 +14,7 @@ perf-test-y += jitdump.o perf-test-y += context_switch_loop.o perf-test-y += deterministic.o perf-test-y += callchain.o +perf-test-y += false_sharing.o ifeq ($(CONFIG_RUST_SUPPORT),y) perf-test-y += code_with_type.o
@@ -29,3 +30,4 @@ CFLAGS_inlineloop.o = -g -O2 CFLAGS_deterministic.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_named_threads.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE CFLAGS_callchain.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE +CFLAGS_false_sharing.o = -g -O0 -fno-inline -U_FORTIFY_SOURCE
tools/perf/tests/workloads/false_sharing.c new file +251 -0
251 lines not on this page · raw message
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH 3/8] perf mutex: Add DEFINE_MUTEX() static initializer
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 13 lines
From: Arnaldo Carvalho de Melo <[email protected]>
For file scope mutexes, so that they don't need a constructor function to initialize them at startup, mirroring
PTHREAD_MUTEX_INITIALIZERwhile keeping the clang -Wthread-safety annotations of struct mutex.DEBUG=1 builds have
mutex_init()setPTHREAD_MUTEX_ERRORCHECK, making theCHECK_ERR()paths in mutex_lock()/mutex_unlock() report relocking a held mutex and unlocking one that isn't held.PTHREAD_MUTEX_INITIALIZERgives a default type mutex, so statically initialized ones would silently lose that: usePTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP, a glibc extension, falling back to the default type in libcs that lack it.
Quoted 2 lines
Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
[email protected]
AI reviewRe: [PATCH 8/8] perf test: Add false_sharing workload exhibiting cross-CPU false sharing
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 13 lines
From: Arnaldo Carvalho de Melo <[email protected]>
Add a 'perf test -w
false_sharing' workload that hammers one shared struct from several CPUs, shaped as a TCP connection: a read-mostly identity (five-tuple) shares a cacheline with per-packet rx counters (the false-sharing line), a second line has packet-path private tx and congestion control counters, and a third the connection config.The packet path runs in the main thread and up to four lookup threads sum the five-tuple and pull the config, reading one volatile shared instance directly so the accesses are PC-relative and resolvable by the data type profiler.
Quoted 8 lines
Data type profiling of the workload with cacheline info:
[ ... ]
0.00% 0.01% struct folio +0x34 (_refcount.counter) 0.00% 0.00% struct folio +0x18 (mapping)Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
[email protected]
AI reviewRe: [PATCH 6/8] perf report: Add --progress option
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 15 lines
From: Arnaldo Carvalho de Melo <[email protected]>
Processing a large session with stdio output gives no feedback about which phase perf is in or how far along it is:
ui_progressupdates are only shown by the TUI. Add --progress, installing a stdio backend (ui/stdio/progress.c) that prints the phase title, percentage and counts:Processing events... [ 42.3%] 317M / 746M
Phases can be nested, so the backend tracks the ones started to complete the right one on
ui_progress__finish(); that requires init()/finish() pairs, fixed inordered-events.cand the pipe and directory event processing.
Quoted 7 lines
With a pager both stdout and stderr lead to it, so isatty(stderr) turns false and the updates end up printed one per line in the pager's output: use
/dev/ttyin that case, so that the updates keep updating in place, untouched by the pager.Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
[email protected]
AI reviewRe: [PATCH 4/8] perf mutex: Add DO_ONCE() for one-time initialization
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 17 lines
From: Arnaldo Carvalho de Melo <[email protected]>
For lazy init that more than one thread can race into, without
pthread_once()so that it stays within perf's locking primitives and clang's -Wthread-safety keeps an eye on the mutexes involved.Mirrors the kernel's
include/linux/once.hDO_ONCE(), dropping its static key fast path, unnecessary at these cold init paths: a per call site static bool plus a statically initialized mutex, double checked. As there, code reachable from more than one call site must go through a common helper.The lockless fast path reads
___donewith an acquire, paired with a release store afterfn(), so that fn()'s side effects are visible to threads taking the fast path, which the dropped static key used to guarantee.
Quoted 6 lines
The
__atomicbuiltins are used rather than smp_load_acquire()/smp_store_release()because ThreadSanitizer understands them, avoiding reports on___doneand on whatfn()initialized.Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
[email protected]
AI reviewRe: [PATCH 2/8] perf config: Make perf_etc_perfconfig() never return NULL
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
From: Arnaldo Carvalho de Melo <[email protected]>
perf_etc_perfconfig()returns whatsystem_path()gives, and that allocates, so on failure every caller dereferenced NULL, the pre-existing ones inperf_config_set__init()and in the daemon included. For the usual absoluteETC_PERFCONFIGsystem_path()just returns a strdup of it, so useETC_PERFCONFIGitself when that fails.Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
[email protected]
AI reviewRe: [PATCH 7/8] perf report: Add --no-progress option
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 13 lines
From: Arnaldo Carvalho de Melo <[email protected]>
Now that --progress is being added for the stdio case, wire up its counterpart for the browsers: the TUI and GTK ones present progress of their own and there is no way to turn it off. Install the no-op
ui_progressops, the ones already used until a backend sets theirs, aftersetup_browser()installed the ones of the browser in use. The phases are still counted, nothing is shown for them, and no second option is needed for it: parse-options provides --no-progress as the negation of --progress, report.progress_set saying that it was asked for, report.progress being false both when nothing was asked for and when --no-progress was.
Quoted 3 lines
Suggested-by Namhyung Kim <[email protected]>Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
[email protected]
AI reviewRe: [PATCH 5/8] perf config: Serialize config file access with a mutex
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 18 lines
From: Arnaldo Carvalho de Melo <[email protected]>
Config file access shares the static parser state and, since
perf_config__set_variable()became an entry point for background features, can run on more than one thread: a feature writing the config while perf top's display thread reads it. Serialize parsing and rewriting withconfig_mutexand the whole read-modify-write ofperf_config__set_variable()withconfig_update_mutex;config_set_mutexcomes before it, as building the set parses the config files.perf has its own mutex type,
util/mutex.h; the file scope mutexes here use theDEFINE_MUTEX()static initializer added in the previous patch. The two lazy inits thatsystem_path()andhome_perfconfig()do would leak all but one of the strings racing on two threads, so they move toDO_ONCE(), added in the previous patch as well.bad_config()runs on the dispatching thread, outsideconfig_mutex, so it stops readingconfig_file_name, which the parsing thread owns.
Quoted 2 lines
Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
[email protected]
AI reviewRe: [PATCH 1/8] perf config: Move perf_config__set_variable() to util/config.c
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
From: Arnaldo Carvalho de Melo <[email protected]>
Move
perf_config__set_variable()out of the 'perf config' builtin so that opt-in features can persist their choice from outside it, e.g.util/debuginfo.cwriting core.debuginfod=false when the user disables debuginfod for the rest of the session. Theset_config()body becomesperf_config_set__write(), with thesystem_configchoice as an argument, as the builtin's use_system_config/use_user_config statics are not available outside it.
Quoted 8 lines
perf_config_set__write()checkedfopen()but none of the fprintf()s orfclose(), so a write failure after truncating the file was reported as success. Harmless for the interactive 'perf config' this came from, but this makes it an entry point a background feature can call with no other feedback, so propagate those errors too.Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 22 lines · Arnaldo Carvalho de Melo
Hi,
This series adds progress diagnostics to perf, and a workload that makes false sharing visible to data type profiling.
The changes are:
- move
perf_config__set_variable()toutil/config.cand serialize config parser and read-modify-write state, so non-builtin perf code can persist configuration changes safely;- add 'perf report --progress' for stdio users, showing the current phase,
percentage, and counts while a session is processed;- wire up 'perf report --no-progress', the counterpart of the option above, for the TUI and GTK browsers, whose progress there is no other way to turn off;
- add 'perf test -w
false_sharing', a synthetic TCP-shaped workload with identity and packet counters sharing a cacheline, and include it in the data type profiling shell test.
Quoted 6 lines
Follow up work: the
DO_ONCE()one-time init primitive added here mirrors what the eight pre-existing rawpthread_once()users in util/ (annotate.c,callchain.c,comm.c,dso.c,fncache.c,intel-tpebs.c,libbfd.c,pmus.c) need, conversions that will also exercise theDEFINE_MUTEX()static initializer; converting them is left for after this series. Best regards,
Thanks,
Namhyung
Quoted 161 lines
- Arnaldo
What changed from v11:
- The 'perf-stuck' patch is removed for the time being, to be submitted
separately later on;What changed from v10:
-
tools/perf/scripts/perf-stuck.sh: pin the watched process by its starttime, so a recycled PID can't get samples or gdb; Sashiko, v10;-
tools/perf/scripts/perf-stuck.sh: drop the control characters from theshown progress updates, no terminal escape injection; Sashiko, v10;-
tools/perf/tests/shell/perf_stuck.sh: test the script, from optionvalidation to the gdb DIE chain runs on stand-in workloads;What changed from v9:
-
tools/perf/ui/stdio/progress.c: progress updates now work well withthe pager, printed to /dev/tty so they update in place;-
tools/perf/scripts/perf-stuck.sh: take the last progress updatefrom a bounded tail window, dropping the \r's. Sashiko, v9;What changed from v8:
-
tools/perf/util/mutex.h:DEFINE_MUTEX()statics keep mutex_init()'s!NDEBUG errorcheck type;- patch 4/9 commit log: clarify the acquire/release pairing, what the dropped static key used to guarantee.
What changed from v7:
-
tools/perf/util/mutex.h: theDO_ONCE()lockless fast path pairs its__ATOMIC_ACQUIRE load with the __ATOMIC_RELEASE store. sashiko, v7;What changed from v6:
- patch 1/5 split into four: the move,
perf_etc_perfconfig()never returning NULL, theDEFINE_MUTEX()prep and the serialization. Namhyung Kim asked, v6;-
tools/perf/util/{config.c,mutex.h}: perf's own mutex type, addingthe DEFINE_MUTEX() initializer it lacked. Namhyung Kim, v6;-
tools/perf/util/mutex.h: addDO_ONCE()one-time init, from thekernel's once.h, moving the lazy inits in config.c to it;-
tools/perf/builtin-report.c: drop the option negation and stdiohook comments Namhyung Kim asked to drop, reviewing v6;-
tools/perf/scripts/perf-stuck.gdb: break the longprintf()line.Namhyung Kim, reviewing v6;- the
false_sharingcset now carries the data type profiling output with cacheline info. Namhyung Kim asked for it, reviewing v6.What changed from v5:
-
tools/perf/util/config.c: drop theconfig_file_nameread inbad_config(), locking would buy a consistent NULL. Sashiko, v5;-
tools/perf/util/config.c: newconfig_set_mutexaround the sharedset's init/teardown, home init via pthread_once. Sashiko, v5;-
tools/perf/builtin-report.c, perf-report.txt: --no-progress is --progress's auto negation, last wins. Namhyung Kim, reviewing v5.What changed from v4:
-
tools/perf/builtin-report.c: mark --progressPARSE_OPT_NOAUTONEG,parse_long_opt() claimed --no-progress first. Sashiko, v4;-
tools/perf/util/config.c: format the path buffer inside the criticalsection. Sashiko pointed out the window, reviewing v4;-
tools/perf/tests/workloads/false_sharing.c: walk every bit of the affinity mask, not_SC_NPROCESSORS_CONF. Sashiko, reviewing v4.What changed from v3:
-
tools/perf/util/config.c: keep the buffer handed to the parser instatic storage. Sashiko, reviewing v3;-
tools/perf/scripts/perf-stuck.gdb: perf-dso walks each dso candidateuntil one evaluates, for REFCNT_CHECKING builds. Sashiko, v3;-
tools/perf/tests/workloads/false_sharing.c: sum before cpu in structfs_reader, the padding after cpu rounded it to 128. Sashiko, v3;-
tools/perf/builtin-report.c: --quiet wins over --progress, the phasesstay uncounted, documented next to it. Namhyung Kim, v3;-
tools/perf/builtin-report.c,tools/perf/ui/progress.c: --no-progressinstalls the no-op ops. Suggested by Namhyung Kim, reviewing v3;-
tools/perf/scripts/perf-stuck.sh: check gdb is there before watching. Namhyung Kim, reviewing v3.What changed from v2:
-
tools/perf/util/config.c: makeperf_etc_perfconfig()total, fallingback to the unresolved path. Sashiko, reviewing v2;-
tools/perf/scripts/perf-stuck.gdb: don't deref map_symbol.sym withouta NULL check. Sashiko, reviewing v2;-
tools/perf/scripts/perf-stuck.gdb: note theREFCNT_CHECKINGproxynext to the structure walks;-
tools/perf/scripts/perf-stuck.sh: count samples with no progress tolook at, an empty log never fired -g;-
tools/perf/scripts/perf-stuck.sh:--for pgrep and tail againstnames starting with a hyphen;-
tools/perf/scripts/perf-stuck.sh: bound the gdb run with`timeout --signal=INT 30`.What changed from v1:
- avoid calling
CPU_SET()with -1 whenfalse_sharingruns with only one CPU available in its affinity mask.tools/perf/Documentation/perf-report.txt | 19 ++ tools/perf/builtin-config.c | 70 +---- tools/perf/builtin-report.c | 9 + tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/shell/data_type_profiling.sh | 9 +- tools/perf/tests/tests.h | 1 + tools/perf/tests/workloads/Build | 2 + tools/perf/tests/workloads/false_sharing.c | 251 ++++++++++++++++++ tools/perf/ui/Build | 1 + tools/perf/ui/progress.c | 6 + tools/perf/ui/progress.h | 4 + tools/perf/ui/stdio/progress.c | 170 ++++++++++++ tools/perf/util/config.c | 166 ++++++++++-- tools/perf/util/config.h | 2 + tools/perf/util/mutex.h | 36 +++ tools/perf/util/ordered-events.c | 16 +- tools/perf/util/session.c | 12 +- 17 files changed, 675 insertions(+), 100 deletions(-)create mode 100644
tools/perf/tests/workloads/false_sharing.ccreate mode 100644tools/perf/ui/stdio/progress.c
base-commit:705da5b15ab89ba9
v1-head:45d7917f7e05e8a29828ed5f0bbdc94fc938f79f
v2-head:d4f84e4de8890194924ccd897a8e6773e7d4240b
v3-head:485532296710225862daf8ffb19802ae327efe2a
v4-head:38193635508e0a5f04a6ebf70db27534b68b2d5b
v5-head:e99bc48f6c72b9cc67b4e4143a0364acac5523bb
v6-head:7dc99dbb6504f199c3c39487d91370d1bbb7a827
v7-head:b357a76666d08f74eb99c755cca41855a96f6169
v8-head:257b5ea6115ad5126d2c881523ea6c660cf5d28c
v9-head:70235104c6cf7d18d91f46de1a233abbeb65893a
v10-head:1ecf850c2694d20ca9cd0014f2a42528ef4dbfec
v11-head:a3ddbaab18aa24fba520927f71d0ea66edfc5593
--
Quoted 19 lines · Namhyung Kim
Quoted 5 lines · Arnaldo Carvalho de Melo
- move
perf_config__set_variable()toutil/config.cand serialize config parser and read-modify-write state, so non-builtin perf code can persist configuration changes safely;
- add 'perf report --progress' for stdio users, showing the current phase,percentage, and counts while a session is processed;Quoted 12 lines
- wire up 'perf report --no-progress', the counterpart of the option above, for the TUI and GTK browsers, whose progress there is no other way to turn off;
- add 'perf test -wfalse_sharing', a synthetic TCP-shaped workload with identity and packet counters sharing a cacheline, and include it in the data type profiling shell test.
Follow up work: theDO_ONCE()one-time init primitive added here mirrors what the eight pre-existing rawpthread_once()users in util/ (annotate.c,callchain.c,comm.c,dso.c,fncache.c,intel-tpebs.c,libbfd.c,pmus.c) need, conversions that will also exercise theDEFINE_MUTEX()static initializer; converting them is left for after this series. Best regards,
Quoted 1 line
Reviewed-by Namhyung Kim <[email protected]>
Thanks, applied to perf-tools-next, for v7.4.
- Arnaldo