tracing: Final updates for 5.17
Subsystems (7)
Steven Rostedt
maintainercover tracing: Final updates for 5.17
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git
for-next
Head SHA1: 0a85bdad9ae1c5f68d83e16433f4b88c264843b2
Daniel Bristot de Oliveira (14):
rtla: Real-Time Linux Analysis tool
rtla: Helper functions for rtla
rtla: Add osnoise tool
rtla/osnoise: Add osnoise top mode rtla/osnoise: Add the hist mode
rtla: Add timerlat tool and timelart top mode rtla/timerlat: Add timerlat hist mode
rtla: Add Documentation
rtla: Add rtla osnoise man page
rtla: Add rtla osnoise top documentation
rtla: Add rtla osnoise hist documentation
rtla: Add rtla timerlat documentation
rtla: Add rtla timerlat top documentation
rtla: Add rtla timerlat hist documentation
Geliang Tang (1):
tracing: Fix mismatched comment in __string_len
Nikita Yushchenko (1):
tracing/osnoise: Properly unhook events if start_per_cpu_kthreads() fails
Sebastian Andrzej Siewior (1):
tracing: Account bottom half disabled sections.
Steven Rostedt (2):
tracing: Have syscall trace events use trace_event_buffer_lock_reserve()
tracing: Add test for user space strings when filtering on string pointers
Steven Rostedt (VMware) (1):
ftrace: Add test to make sure compiled time sorts work
Tom Zanussi (6):
tracing: Change event_command func() to parse()
tracing: Change event_trigger_ops func() to trigger()
tracing: Remove ops param from event_command reg()/unreg() callbacks
tracing: Add helper functions to simplify event_command.parse() callback handling
tracing: Have existing event_command.parse() implementations use helpers
tracing: Remove redundant trigger_ops params
Xiangyang Zhang (1):
tracing/kprobes: 'nmissed' not showed correctly for kretprobe
Xiaoke Wang (2):
tracing/uprobes: Check the return value of kstrdup() for tu->filename tracing/probes: check the return value of kstrndup() for pbuf
Yinan Liu (1):
scripts: ftrace - move the sort-processing in ftrace_init
Yuntao Wang (1):
tracing: Remove duplicate warnings when calling trace_create_file()
----
Documentation/tools/rtla/Makefile | 41 + Documentation/tools/rtla/common_appendix.rst | 12 + Documentation/tools/rtla/common_hist_options.rst | 23 + Documentation/tools/rtla/common_options.rst | 24 + .../tools/rtla/common_osnoise_description.rst | 8 + .../tools/rtla/common_osnoise_options.rst | 17 + .../tools/rtla/common_timerlat_description.rst | 10 + .../tools/rtla/common_timerlat_options.rst | 16 + Documentation/tools/rtla/common_top_options.rst | 3 + Documentation/tools/rtla/rtla-osnoise-hist.rst | 66 ++ Documentation/tools/rtla/rtla-osnoise-top.rst | 61 ++ Documentation/tools/rtla/rtla-osnoise.rst | 59 ++ Documentation/tools/rtla/rtla-timerlat-hist.rst | 106 ++ Documentation/tools/rtla/rtla-timerlat-top.rst | 145 +++ Documentation/tools/rtla/rtla-timerlat.rst | 57 ++ Documentation/tools/rtla/rtla.rst | 48 + Documentation/trace/events.rst | 10 + include/linux/trace_events.h | 1 + kernel/trace/Kconfig | 14 + kernel/trace/ftrace.c | 34 +- kernel/trace/trace.c | 6 +- kernel/trace/trace.h | 81 +- kernel/trace/trace_eprobe.c | 30 +- kernel/trace/trace_events.c | 12 +- kernel/trace/trace_events_filter.c | 59 +- kernel/trace/trace_events_hist.c | 147 ++- kernel/trace/trace_events_trigger.c | 626 ++++++++---- kernel/trace/trace_kprobe.c | 5 +- kernel/trace/trace_osnoise.c | 20 +- kernel/trace/trace_output.c | 4 + kernel/trace/trace_probe.c | 2 + kernel/trace/trace_syscalls.c | 6 +- kernel/trace/trace_uprobe.c | 5 + samples/trace_events/trace-events-sample.h | 2 +- scripts/Makefile | 6 +- scripts/link-vmlinux.sh | 6 +- scripts/sorttable.c | 2 + scripts/sorttable.h | 120 ++- tools/tracing/rtla/Makefile | 102 ++ tools/tracing/rtla/README.txt | 36 + tools/tracing/rtla/src/osnoise.c | 1017 ++++++++++++++++++++ tools/tracing/rtla/src/osnoise.h | 96 ++ tools/tracing/rtla/src/osnoise_hist.c | 799 +++++++++++++++ tools/tracing/rtla/src/osnoise_top.c | 577 +++++++++++ tools/tracing/rtla/src/rtla.c | 87 ++ tools/tracing/rtla/src/timerlat.c | 72 ++ tools/tracing/rtla/src/timerlat.h | 4 + tools/tracing/rtla/src/timerlat_hist.c | 820 ++++++++++++++++ tools/tracing/rtla/src/timerlat_top.c | 615 ++++++++++++ tools/tracing/rtla/src/trace.c | 192 ++++ tools/tracing/rtla/src/trace.h | 27 + tools/tracing/rtla/src/utils.c | 433 +++++++++ tools/tracing/rtla/src/utils.h | 56 ++ 53 files changed, 6477 insertions(+), 350 deletions(-) create mode 100644 Documentation/tools/rtla/Makefile create mode 100644 Documentation/tools/rtla/common_appendix.rst create mode 100644 Documentation/tools/rtla/common_hist_options.rst create mode 100644 Documentation/tools/rtla/common_options.rst create mode 100644 Documentation/tools/rtla/common_osnoise_description.rst create mode 100644 Documentation/tools/rtla/common_osnoise_options.rst create mode 100644 Documentation/tools/rtla/common_timerlat_description.rst create mode 100644 Documentation/tools/rtla/common_timerlat_options.rst create mode 100644 Documentation/tools/rtla/common_top_options.rst create mode 100644 Documentation/tools/rtla/rtla-osnoise-hist.rst create mode 100644 Documentation/tools/rtla/rtla-osnoise-top.rst create mode 100644 Documentation/tools/rtla/rtla-osnoise.rst create mode 100644 Documentation/tools/rtla/rtla-timerlat-hist.rst create mode 100644 Documentation/tools/rtla/rtla-timerlat-top.rst create mode 100644 Documentation/tools/rtla/rtla-timerlat.rst create mode 100644 Documentation/tools/rtla/rtla.rst create mode 100644 tools/tracing/rtla/Makefile create mode 100644 tools/tracing/rtla/README.txt create mode 100644 tools/tracing/rtla/src/osnoise.c create mode 100644 tools/tracing/rtla/src/osnoise.h create mode 100644 tools/tracing/rtla/src/osnoise_hist.c create mode 100644 tools/tracing/rtla/src/osnoise_top.c create mode 100644 tools/tracing/rtla/src/rtla.c create mode 100644 tools/tracing/rtla/src/timerlat.c create mode 100644 tools/tracing/rtla/src/timerlat.h create mode 100644 tools/tracing/rtla/src/timerlat_hist.c create mode 100644 tools/tracing/rtla/src/timerlat_top.c create mode 100644 tools/tracing/rtla/src/trace.c create mode 100644 tools/tracing/rtla/src/trace.h create mode 100644 tools/tracing/rtla/src/utils.c create mode 100644 tools/tracing/rtla/src/utils.h
Steven Rostedt
maintainerPATCH 1/31 tracing: Change event_command func() to parse()
From: Tom Zanussi <[email protected]>
The name of the func() callback on event_command is too generic and is easily confused with other callbacks with that name, so change it to something that reflects its actual purpose.
In this case, the main purpose of the callback is to parse an event command, so call it parse() instead.
--- kernel/trace/trace.h | 19 +++++++++++------- kernel/trace/trace_eprobe.c | 8 ++++---- kernel/trace/trace_events_hist.c | 26 ++++++++++++------------- kernel/trace/trace_events_trigger.c | 30 ++++++++++++++--------------- 4 files changed, 44 insertions(+), 39 deletions(-)
kernel/trace/trace.h +12 -7
@@ -1578,9 +1578,9 @@ extern int event_enable_trigger_print(struct seq_file *m, struct event_trigger_data *data);extern void event_enable_trigger_free (struct event_trigger_ops *ops,struct event_trigger_data *data); -extern int event_enable_trigger_func (struct event_command *cmd_ops, -struct trace_event_file *file, -char *glob,char *cmd,char *param); +extern int event_enable_trigger_parse (struct event_command *cmd_ops, +struct trace_event_file *file, +char *glob,char *cmd,char *param); extern int event_enable_register_trigger(char *glob, struct event_trigger_ops *ops, struct event_trigger_data *data,
@@ -1702,7 +1702,7 @@ struct event_trigger_ops {* All the methods below, except for @set_filter() and @unreg_all(), * must be implemented. * -* @func: The callback function responsible for parsing and +* @parse: The callback function responsible for parsing and * registering the trigger written to the 'trigger' file by the * user. It allocates the trigger instance and registers it with * the appropriate trace event. It makes use of the other
@@ -1737,15 +1737,20 @@ struct event_trigger_ops {* * @get_trigger_ops: The callback function invoked to retrieve the * event_trigger_ops implementation associated with the command. +* This callback function allows a single event_command to +* support multiple trigger implementations via different sets of +* event_trigger_ops, depending on the value of the @param +* string. */ struct event_command {struct list_head list;char *name;enum event_trigger_type trigger_type;int flags; -int (*func)(struct event_command *cmd_ops, -struct trace_event_file *file, -char *glob,char *cmd,char *params); +int (*parse)(struct event_command *cmd_ops, +struct trace_event_file *file, +char *glob,char *cmd, +char *param_and_filter); int (*reg)(char *glob, struct event_trigger_ops *ops, struct event_trigger_data *data,
kernel/trace/trace_eprobe.c +4 -4
@@ -549,9 +549,9 @@ static struct event_trigger_ops eprobe_trigger_ops = { .free = eprobe_trigger_free, }; -static int eprobe_trigger_cmd_func (struct event_command *cmd_ops, -struct trace_event_file *file, -char *glob,char *cmd,char *param) +static int eprobe_trigger_cmd_parse (struct event_command *cmd_ops, +struct trace_event_file *file, +char *glob,char *cmd,char *param) {return -1 ; }
@@ -580,7 +580,7 @@ static struct event_command event_trigger_cmd = { .name = "eprobe", .trigger_type = ETT_EVENT_EPROBE, .flags = EVENT_CMD_FL_NEEDS_REC, - .func = eprobe_trigger_cmd_func, + .parse = eprobe_trigger_cmd_parse, .reg = eprobe_trigger_reg_func, .unreg = eprobe_trigger_unreg_func, .unreg_all = NULL,
kernel/trace/trace_events_hist.c +13 -13
@@ -2761,9 +2761,9 @@ static char *find_trigger_filter(struct hist_trigger_data *hist_data, }static struct event_command trigger_hist_cmd; -static int event_hist_trigger_func (struct event_command *cmd_ops, -struct trace_event_file *file, -char *glob,char *cmd,char *param); +static int event_hist_trigger_parse (struct event_command *cmd_ops, +struct trace_event_file *file, +char *glob,char *cmd,char *param); static bool compatible_keys(struct hist_trigger_data *target_hist_data, struct hist_trigger_data *hist_data,
@@ -2966,8 +2966,8 @@ create_field_var_hist(struct hist_trigger_data *target_hist_data, var_hist->hist_data = hist_data;/* Create the new histogram with our variable */ - ret =event_hist_trigger_func (&trigger_hist_cmd, file, -"" ,"hist" , cmd); + ret =event_hist_trigger_parse (&trigger_hist_cmd, file, +"" ,"hist" , cmd);if (ret) {kfree (cmd);kfree (var_hist->cmd);
@@ -5729,8 +5729,8 @@ static void unregister_field_var_hists(struct hist_trigger_data *hist_data)for (i =0 ; i < hist_data->n_field_var_hists; i++) { file = hist_data->field_var_hists[i]->hist_data->event_file; cmd = hist_data->field_var_hists[i]->cmd; - ret =event_hist_trigger_func (&trigger_hist_cmd, file, -"!hist" ,"hist" , cmd); + ret =event_hist_trigger_parse (&trigger_hist_cmd, file, +"!hist" ,"hist" , cmd);WARN_ON_ONCE (ret <0 ); } }
@@ -6146,9 +6146,9 @@ static void hist_unreg_all(struct trace_event_file *file) } } -static int event_hist_trigger_func (struct event_command *cmd_ops, -struct trace_event_file *file, -char *glob,char *cmd,char *param) +static int event_hist_trigger_parse (struct event_command *cmd_ops, +struct trace_event_file *file, +char *glob,char *cmd,char *param) {unsigned int hist_trigger_bits =TRACING_MAP_BITS_DEFAULT ;struct event_trigger_data *trigger_data;
@@ -6331,7 +6331,7 @@ static struct event_command trigger_hist_cmd = { .name = "hist", .trigger_type = ETT_EVENT_HIST, .flags = EVENT_CMD_FL_NEEDS_REC, - .func = event_hist_trigger_func, + .parse = event_hist_trigger_parse, .reg = hist_register_trigger, .unreg = hist_unregister_trigger, .unreg_all = hist_unreg_all,
@@ -6446,7 +6446,7 @@ static void hist_enable_unreg_all(struct trace_event_file *file)static struct event_command trigger_hist_enable_cmd = { .name =ENABLE_HIST_STR , .trigger_type =ETT_HIST_ENABLE , - .func = event_enable_trigger_func, + .parse = event_enable_trigger_parse, .reg = event_enable_register_trigger, .unreg = event_enable_unregister_trigger, .unreg_all = hist_enable_unreg_all,
@@ -6457,7 +6457,7 @@ static struct event_command trigger_hist_enable_cmd = {static struct event_command trigger_hist_disable_cmd = { .name =DISABLE_HIST_STR , .trigger_type =ETT_HIST_ENABLE , - .func = event_enable_trigger_func, + .parse = event_enable_trigger_parse, .reg = event_enable_register_trigger, .unreg = event_enable_unregister_trigger, .unreg_all = hist_enable_unreg_all,
kernel/trace/trace_events_trigger.c +15 -15
@@ -245,7 +245,7 @@ int trigger_process_regex(struct trace_event_file *file, char *buff)mutex_lock (&trigger_cmd_mutex);list_for_each_entry (p, &trigger_commands, list) {if (strcmp (p->name, command) ==0 ) { - ret = p->func (p, file, buff, command, next); + ret = p->parse (p, file, buff, command, next); goto out_unlock; } }
@@ -622,7 +622,7 @@ static void unregister_trigger(char *glob, struct event_trigger_ops *ops, }/** -* event_trigger_callback - Generic event_command @func implementation +* event_trigger_parse - Generic event_command @parse implementation * @cmd_ops: The command ops, used for trigger registration * @file: The trace_event_file associated with the event * @glob: The raw string used to register the trigger
@@ -632,15 +632,15 @@ static void unregister_trigger(char *glob, struct event_trigger_ops *ops,* Common implementation for event command parsing and trigger * instantiation. * -* Usually used directly as the @func method in event command +* Usually used directly as the @parse method in event command * implementations. * * Return: 0 on success, errno otherwise */ static int -event_trigger_callback (struct event_command *cmd_ops, -struct trace_event_file *file, -char *glob,char *cmd,char *param) +event_trigger_parse (struct event_command *cmd_ops, +struct trace_event_file *file, +char *glob,char *cmd,char *param) {struct event_trigger_data *trigger_data;struct event_trigger_ops *trigger_ops;
@@ -1069,7 +1069,7 @@ onoff_get_trigger_ops(char *cmd, char *param)static struct event_command trigger_traceon_cmd = { .name ="traceon" , .trigger_type =ETT_TRACE_ONOFF , - .func = event_trigger_callback, + .parse = event_trigger_parse, .reg = register_trigger, .unreg = unregister_trigger, .get_trigger_ops = onoff_get_trigger_ops,
@@ -1080,7 +1080,7 @@ static struct event_command trigger_traceoff_cmd = { .name = "traceoff", .trigger_type = ETT_TRACE_ONOFF, .flags = EVENT_CMD_FL_POST_TRIGGER, - .func = event_trigger_callback, + .parse = event_trigger_parse, .reg = register_trigger, .unreg = unregister_trigger, .get_trigger_ops = onoff_get_trigger_ops,
@@ -1157,7 +1157,7 @@ snapshot_get_trigger_ops(char *cmd, char *param)static struct event_command trigger_snapshot_cmd = { .name ="snapshot" , .trigger_type =ETT_SNAPSHOT , - .func = event_trigger_callback, + .parse = event_trigger_parse, .reg = register_snapshot_trigger, .unreg = unregister_trigger, .get_trigger_ops = snapshot_get_trigger_ops,
@@ -1249,7 +1249,7 @@ static struct event_command trigger_stacktrace_cmd = { .name = "stacktrace", .trigger_type = ETT_STACKTRACE, .flags = EVENT_CMD_FL_POST_TRIGGER, - .func = event_trigger_callback, + .parse = event_trigger_parse, .reg = register_trigger, .unreg = unregister_trigger, .get_trigger_ops = stacktrace_get_trigger_ops,
@@ -1380,9 +1380,9 @@ static struct event_trigger_ops event_disable_count_trigger_ops = { .free = event_enable_trigger_free, }; -int event_enable_trigger_func (struct event_command *cmd_ops, -struct trace_event_file *file, -char *glob,char *cmd,char *param) +int event_enable_trigger_parse (struct event_command *cmd_ops, +struct trace_event_file *file, +char *glob,char *cmd,char *param) {struct trace_event_file *event_enable_file;struct enable_trigger_data *enable_data;
@@ -1628,7 +1628,7 @@ event_enable_get_trigger_ops(char *cmd, char *param)static struct event_command trigger_enable_cmd = { .name =ENABLE_EVENT_STR , .trigger_type =ETT_EVENT_ENABLE , - .func = event_enable_trigger_func, + .parse = event_enable_trigger_parse, .reg = event_enable_register_trigger, .unreg = event_enable_unregister_trigger, .get_trigger_ops = event_enable_get_trigger_ops,
@@ -1638,7 +1638,7 @@ static struct event_command trigger_enable_cmd = {static struct event_command trigger_disable_cmd = { .name =DISABLE_EVENT_STR , .trigger_type =ETT_EVENT_ENABLE , - .func = event_enable_trigger_func, + .parse = event_enable_trigger_parse, .reg = event_enable_register_trigger, .unreg = event_enable_unregister_trigger, .get_trigger_ops = event_enable_get_trigger_ops,
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 2/31 tracing: Change event_trigger_ops func() to trigger()
In reply to Steven RostedtFrom: Tom Zanussi <[email protected]>
The name of the func() callback on event_trigger_ops is too generic and is easily confused with other callbacks with that name, so change it to something that reflects its actual purpose.
In this case, the main purpose of the callback is to implement an event trigger, so call it trigger() instead.
Also add some more documentation to event_trigger_ops describing the callbacks a bit better.
--- kernel/trace/trace.h | 19 ++++++++++++++---- kernel/trace/trace_eprobe.c | 2 +- kernel/trace/trace_events_hist.c | 12 ++++++------ kernel/trace/trace_events_trigger.c | 30 ++++++++++++++--------------- 4 files changed, 37 insertions(+), 26 deletions(-)
kernel/trace/trace.h +15 -4
@@ -1619,10 +1619,20 @@ extern int register_trigger_hist_enable_disable_cmds(void);* The methods in this structure provide per-event trigger hooks for * various trigger operations. * +* The @init and @free methods are used during trigger setup and +* teardown, typically called from an event_command's @parse() +* function implementation. +* +* The @print method is used to print the trigger spec. +* +* The @trigger method is the function that actually implements the +* trigger and is called in the context of the triggering event +* whenever that event occurs. +* * All the methods below, except for @init() and @free(), must be * implemented. * -* @func: The trigger 'probe' function called when the triggering +* @trigger: The trigger 'probe' function called when the triggering * event occurs. The data passed into this callback is the data * that was supplied to the event_command @reg() function that * registered the trigger (see struct event_command) along with
@@ -1651,9 +1661,10 @@ extern int register_trigger_hist_enable_disable_cmds(void);* (see trace_event_triggers.c). */ struct event_trigger_ops { -void (*func)(struct event_trigger_data *data, -struct trace_buffer *buffer,void *rec, -struct ring_buffer_event *rbe); +void (*trigger)(struct event_trigger_data *data, +struct trace_buffer *buffer, +void *rec, +struct ring_buffer_event *rbe);int (*init)(struct event_trigger_ops *ops,struct event_trigger_data *data); void (*free)(struct event_trigger_ops *ops,
kernel/trace/trace_eprobe.c +1 -1
@@ -543,7 +543,7 @@ static void eprobe_trigger_func(struct event_trigger_data *data, }static struct event_trigger_ops eprobe_trigger_ops = { - .func = eprobe_trigger_func, + .trigger = eprobe_trigger_func, .print = eprobe_trigger_print, .init = eprobe_trigger_init, .free = eprobe_trigger_free,
kernel/trace/trace_events_hist.c +6 -6
@@ -5759,7 +5759,7 @@ static void event_hist_trigger_free(struct event_trigger_ops *ops, }static struct event_trigger_ops event_hist_trigger_ops = { - .func = event_hist_trigger, + .trigger = event_hist_trigger, .print = event_hist_trigger_print, .init = event_hist_trigger_init, .free = event_hist_trigger_free,
@@ -5793,7 +5793,7 @@ static void event_hist_trigger_named_free(struct event_trigger_ops *ops, }static struct event_trigger_ops event_hist_trigger_named_ops = { - .func = event_hist_trigger, + .trigger = event_hist_trigger, .print = event_hist_trigger_print, .init = event_hist_trigger_named_init, .free = event_hist_trigger_named_free,
@@ -6383,28 +6383,28 @@ hist_enable_count_trigger(struct event_trigger_data *data, }static struct event_trigger_ops hist_enable_trigger_ops = { - .func = hist_enable_trigger, + .trigger = hist_enable_trigger, .print = event_enable_trigger_print, .init = event_trigger_init, .free = event_enable_trigger_free, };static struct event_trigger_ops hist_enable_count_trigger_ops = { - .func = hist_enable_count_trigger, + .trigger = hist_enable_count_trigger, .print = event_enable_trigger_print, .init = event_trigger_init, .free = event_enable_trigger_free, };static struct event_trigger_ops hist_disable_trigger_ops = { - .func = hist_enable_trigger, + .trigger = hist_enable_trigger, .print = event_enable_trigger_print, .init = event_trigger_init, .free = event_enable_trigger_free, };static struct event_trigger_ops hist_disable_count_trigger_ops = { - .func = hist_enable_count_trigger, + .trigger = hist_enable_count_trigger, .print = event_enable_trigger_print, .init = event_trigger_init, .free = event_enable_trigger_free,
kernel/trace/trace_events_trigger.c +15 -15
@@ -68,7 +68,7 @@ event_triggers_call(struct trace_event_file *file,if (data->paused)continue ;if (!rec) { - data->ops->func (data, buffer, rec, event); + data->ops->trigger (data, buffer, rec, event);continue ; } filter =rcu_dereference_sched (data->filter);
@@ -78,7 +78,7 @@ event_triggers_call(struct trace_event_file *file, tt |= data->cmd_ops->trigger_type;continue ; } - data->ops->func (data, buffer, rec, event); + data->ops->trigger (data, buffer, rec, event); }return tt; }
@@ -106,7 +106,7 @@ event_triggers_post_call(struct trace_event_file *file,if (data->paused)continue ;if (data->cmd_ops->trigger_type & tt) - data->ops->func (data,NULL ,NULL ,NULL ); + data->ops->trigger (data,NULL ,NULL ,NULL ); } }EXPORT_SYMBOL_GPL (event_triggers_post_call);
@@ -1023,28 +1023,28 @@ traceoff_trigger_print(struct seq_file *m, struct event_trigger_ops *ops, }static struct event_trigger_ops traceon_trigger_ops = { - .func = traceon_trigger, + .trigger = traceon_trigger, .print = traceon_trigger_print, .init = event_trigger_init, .free = event_trigger_free, };static struct event_trigger_ops traceon_count_trigger_ops = { - .func = traceon_count_trigger, + .trigger = traceon_count_trigger, .print = traceon_trigger_print, .init = event_trigger_init, .free = event_trigger_free, };static struct event_trigger_ops traceoff_trigger_ops = { - .func = traceoff_trigger, + .trigger = traceoff_trigger, .print = traceoff_trigger_print, .init = event_trigger_init, .free = event_trigger_free, };static struct event_trigger_ops traceoff_count_trigger_ops = { - .func = traceoff_count_trigger, + .trigger = traceoff_count_trigger, .print = traceoff_trigger_print, .init = event_trigger_init, .free = event_trigger_free,
@@ -1135,14 +1135,14 @@ snapshot_trigger_print(struct seq_file *m, struct event_trigger_ops *ops, }static struct event_trigger_ops snapshot_trigger_ops = { - .func = snapshot_trigger, + .trigger = snapshot_trigger, .print = snapshot_trigger_print, .init = event_trigger_init, .free = event_trigger_free, };static struct event_trigger_ops snapshot_count_trigger_ops = { - .func = snapshot_count_trigger, + .trigger = snapshot_count_trigger, .print = snapshot_trigger_print, .init = event_trigger_init, .free = event_trigger_free,
@@ -1226,14 +1226,14 @@ stacktrace_trigger_print(struct seq_file *m, struct event_trigger_ops *ops, }static struct event_trigger_ops stacktrace_trigger_ops = { - .func = stacktrace_trigger, + .trigger = stacktrace_trigger, .print = stacktrace_trigger_print, .init = event_trigger_init, .free = event_trigger_free, };static struct event_trigger_ops stacktrace_count_trigger_ops = { - .func = stacktrace_count_trigger, + .trigger = stacktrace_count_trigger, .print = stacktrace_trigger_print, .init = event_trigger_init, .free = event_trigger_free,
@@ -1353,28 +1353,28 @@ void event_enable_trigger_free(struct event_trigger_ops *ops, }static struct event_trigger_ops event_enable_trigger_ops = { - .func = event_enable_trigger, + .trigger = event_enable_trigger, .print = event_enable_trigger_print, .init = event_trigger_init, .free = event_enable_trigger_free, };static struct event_trigger_ops event_enable_count_trigger_ops = { - .func = event_enable_count_trigger, + .trigger = event_enable_count_trigger, .print = event_enable_trigger_print, .init = event_trigger_init, .free = event_enable_trigger_free, };static struct event_trigger_ops event_disable_trigger_ops = { - .func = event_enable_trigger, + .trigger = event_enable_trigger, .print = event_enable_trigger_print, .init = event_trigger_init, .free = event_enable_trigger_free, };static struct event_trigger_ops event_disable_count_trigger_ops = { - .func = event_enable_count_trigger, + .trigger = event_enable_count_trigger, .print = event_enable_trigger_print, .init = event_trigger_init, .free = event_enable_trigger_free,
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 3/31 tracing: Remove ops param from event_command reg()/unreg() callbacks
In reply to Steven RostedtFrom: Tom Zanussi <[email protected]>
The event_trigger_ops for an event_command are already accessible via event_trigger_data.ops so remove the redundant ops from the callback.
--- kernel/trace/trace.h | 4 ---- kernel/trace/trace_eprobe.c | 12 ++++++------ kernel/trace/trace_events_hist.c | 10 +++++----- kernel/trace/trace_events_trigger.c | 22 +++++++++------------- 4 files changed, 20 insertions(+), 28 deletions(-)
kernel/trace/trace.h +0 -4
@@ -1582,11 +1582,9 @@ extern int event_enable_trigger_parse(struct event_command *cmd_ops,struct trace_event_file *file, char *glob, char *cmd, char *param);extern int event_enable_register_trigger (char *glob, -struct event_trigger_ops *ops,struct event_trigger_data *data,struct trace_event_file *file);extern void event_enable_unregister_trigger (char *glob, -struct event_trigger_ops *ops,struct event_trigger_data *test,struct trace_event_file *file);extern void trigger_data_free (struct event_trigger_data *data);
@@ -1763,11 +1761,9 @@ struct event_command { char *glob, char *cmd, char *param_and_filter);int (*reg)(char *glob, -struct event_trigger_ops *ops,struct event_trigger_data *data,struct trace_event_file *file);void (*unreg)(char *glob, -struct event_trigger_ops *ops,struct event_trigger_data *data,struct trace_event_file *file);void (*unreg_all)(struct trace_event_file *file);
kernel/trace/trace_eprobe.c +6 -6
@@ -556,16 +556,16 @@ static int eprobe_trigger_cmd_parse(struct event_command *cmd_ops,return -1 ; } -static int eprobe_trigger_reg_func (char *glob,struct event_trigger_ops *ops, -struct event_trigger_data *data, -struct trace_event_file *file) +static int eprobe_trigger_reg_func (char *glob, +struct event_trigger_data *data, +struct trace_event_file *file) {return -1 ; } -static void eprobe_trigger_unreg_func (char *glob,struct event_trigger_ops *ops, -struct event_trigger_data *data, -struct trace_event_file *file) +static void eprobe_trigger_unreg_func (char *glob, +struct event_trigger_data *data, +struct trace_event_file *file) { }
kernel/trace/trace_events_hist.c +5 -5
@@ -5910,7 +5910,7 @@ static bool hist_trigger_match(struct event_trigger_data *data,return true; } -static int hist_register_trigger (char *glob,struct event_trigger_ops *ops, +static int hist_register_trigger (char *glob,struct event_trigger_data *data,struct trace_event_file *file) {
@@ -6062,7 +6062,7 @@ static bool hist_trigger_check_refs(struct event_trigger_data *data,return false; } -static void hist_unregister_trigger (char *glob,struct event_trigger_ops *ops, +static void hist_unregister_trigger (char *glob,struct event_trigger_data *data,struct trace_event_file *file) {
@@ -6262,7 +6262,7 @@ static int event_hist_trigger_parse(struct event_command *cmd_ops, goto out_free; } - cmd_ops->unreg (glob+1 , trigger_ops, trigger_data, file); + cmd_ops->unreg (glob+1 , trigger_data, file); se_name =trace_event_name (file->event_call); se =find_synth_event (se_name);if (se)
@@ -6271,7 +6271,7 @@ static int event_hist_trigger_parse(struct event_command *cmd_ops, goto out_free; } - ret = cmd_ops->reg (glob, trigger_ops, trigger_data, file); + ret = cmd_ops->reg (glob, trigger_data, file);/* * The above returns on success the # of triggers registered, * but if it didn't register any it returns zero. Consider no
@@ -6314,7 +6314,7 @@ static int event_hist_trigger_parse(struct event_command *cmd_ops,return ret; out_unreg: - cmd_ops->unreg (glob+1 , trigger_ops, trigger_data, file); + cmd_ops->unreg (glob+1 , trigger_data, file); out_free:if (cmd_ops->set_filter) cmd_ops->set_filter (NULL , trigger_data,NULL );
kernel/trace/trace_events_trigger.c +9 -13
@@ -540,7 +540,6 @@ void update_cond_flag(struct trace_event_file *file)/** * register_trigger - Generic event_command @reg implementation * @glob: The raw string used to register the trigger -* @ops: The trigger ops associated with the trigger * @data: Trigger-specific data to associate with the trigger * @file: The trace_event_file associated with the event *
@@ -551,7 +550,7 @@ void update_cond_flag(struct trace_event_file *file)* * Return: 0 on success, errno otherwise */ -static int register_trigger (char *glob,struct event_trigger_ops *ops, +static int register_trigger (char *glob,struct event_trigger_data *data,struct trace_event_file *file) {
@@ -589,7 +588,6 @@ static int register_trigger(char *glob, struct event_trigger_ops *ops,/** * unregister_trigger - Generic event_command @unreg implementation * @glob: The raw string used to register the trigger -* @ops: The trigger ops associated with the trigger * @test: Trigger-specific data used to find the trigger to remove * @file: The trace_event_file associated with the event *
@@ -598,7 +596,7 @@ static int register_trigger(char *glob, struct event_trigger_ops *ops,* Usually used directly as the @unreg method in event command * implementations. */ -static void unregister_trigger (char *glob,struct event_trigger_ops *ops, +static void unregister_trigger (char *glob,struct event_trigger_data *test,struct trace_event_file *file) {
@@ -673,7 +671,7 @@ event_trigger_parse(struct event_command *cmd_ops,INIT_LIST_HEAD (&trigger_data->named_list);if (glob[0 ] =='!' ) { - cmd_ops->unreg (glob+1 , trigger_ops, trigger_data, file); + cmd_ops->unreg (glob+1 , trigger_data, file);kfree (trigger_data); ret =0 ; goto out;
@@ -708,14 +706,14 @@ event_trigger_parse(struct event_command *cmd_ops, out_reg:/* Up the trigger_data count to make sure reg doesn't free it on failure */ event_trigger_init (trigger_ops, trigger_data); - ret = cmd_ops->reg (glob, trigger_ops, trigger_data, file); + ret = cmd_ops->reg (glob, trigger_data, file);/* * The above returns on success the # of functions enabled, * but if it didn't find any functions it returns zero. * Consider no functions a failure too. */ if (!ret) { - cmd_ops->unreg (glob, trigger_ops, trigger_data, file); + cmd_ops->unreg (glob, trigger_data, file); ret = -ENOENT ; }else if (ret >0 ) ret =0 ;
@@ -1116,14 +1114,14 @@ snapshot_count_trigger(struct event_trigger_data *data, }static int -register_snapshot_trigger (char *glob,struct event_trigger_ops *ops, +register_snapshot_trigger (char *glob,struct event_trigger_data *data,struct trace_event_file *file) {if (tracing_alloc_snapshot_instance (file->tr) !=0 )return 0 ; -return register_trigger (glob, ops, data, file); +return register_trigger (glob, data, file); } static int
@@ -1455,7 +1453,7 @@ int event_enable_trigger_parse(struct event_command *cmd_ops, trigger_data->private_data = enable_data;if (glob[0 ] =='!' ) { - cmd_ops->unreg (glob+1 , trigger_ops, trigger_data, file); + cmd_ops->unreg (glob+1 , trigger_data, file);kfree (trigger_data);kfree (enable_data); ret =0 ;
@@ -1502,7 +1500,7 @@ int event_enable_trigger_parse(struct event_command *cmd_ops, ret =trace_event_enable_disable (event_enable_file,1 ,1 );if (ret <0 ) goto out_put; - ret = cmd_ops->reg (glob, trigger_ops, trigger_data, file); + ret = cmd_ops->reg (glob, trigger_data, file);/* * The above returns on success the # of functions enabled, * but if it didn't find any functions it returns zero.
@@ -1532,7 +1530,6 @@ int event_enable_trigger_parse(struct event_command *cmd_ops, }int event_enable_register_trigger (char *glob, -struct event_trigger_ops *ops,struct event_trigger_data *data,struct trace_event_file *file) {
@@ -1574,7 +1571,6 @@ int event_enable_register_trigger(char *glob, }void event_enable_unregister_trigger (char *glob, -struct event_trigger_ops *ops,struct event_trigger_data *test,struct trace_event_file *file) {
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 4/31 tracing: Add helper functions to simplify event_command.parse() callback handling
In reply to Steven RostedtFrom: Tom Zanussi <[email protected]>
The event_command.parse() callback is responsible for parsing and registering triggers. The existing command implementions for this callback duplicate a lot of the same code, so to clean up and consolidate those implementations, introduce a handful of helper functions for implementors to use.
This also makes it easier for new commands to be implemented and allows them to focus more on the customizations they provide rather than obscuring and complicating it with boilerplate code.
--- kernel/trace/trace.h | 24 ++ kernel/trace/trace_events_trigger.c | 342 ++++++++++++++++++++++++++++ 2 files changed, 366 insertions(+)
kernel/trace/trace.h +24 -0
@@ -1610,6 +1610,30 @@ get_named_trigger_data(struct event_trigger_data *data);extern int register_event_command (struct event_command *cmd);extern int unregister_event_command (struct event_command *cmd);extern int register_trigger_hist_enable_disable_cmds (void ); +extern bool event_trigger_check_remove (const char *glob); +extern bool event_trigger_empty_param (const char *param); +extern int event_trigger_separate_filter (char *param_and_filter,char **param, +char **filter,bool param_required); +extern struct event_trigger_data * +event_trigger_alloc (struct event_command *cmd_ops, +char *cmd, +char *param, +void *private_data); +extern int event_trigger_parse_num (char *trigger, +struct event_trigger_data *trigger_data); +extern int event_trigger_set_filter (struct event_command *cmd_ops, +struct trace_event_file *file, +char *param, +struct event_trigger_data *trigger_data); +extern void event_trigger_reset_filter (struct event_command *cmd_ops, +struct event_trigger_data *trigger_data); +extern int event_trigger_register (struct event_command *cmd_ops, +struct trace_event_file *file, +char *glob, +char *cmd, +char *trigger, +struct event_trigger_data *trigger_data, +int *n_registered);/** * struct event_trigger_ops - callbacks for trace event triggers
kernel/trace/trace_events_trigger.c +342 -0
348 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 5/31 tracing: Have existing event_command.parse() implementations use helpers
In reply to Steven RostedtFrom: Tom Zanussi <[email protected]>
Simplify the existing event_command.parse() implementations by having them make use of the helper functions previously introduced.
--- kernel/trace/trace.h | 3 +- kernel/trace/trace_eprobe.c | 3 +- kernel/trace/trace_events_hist.c | 76 +++++------ kernel/trace/trace_events_trigger.c | 192 ++++++++-------------------- 4 files changed, 86 insertions(+), 188 deletions(-)
kernel/trace/trace.h +2 -1
@@ -1580,7 +1580,8 @@ extern void event_enable_trigger_free(struct event_trigger_ops *ops, struct event_trigger_data *data);extern int event_enable_trigger_parse (struct event_command *cmd_ops,struct trace_event_file *file, -char *glob,char *cmd,char *param); +char *glob,char *cmd, +char *param_and_filter);extern int event_enable_register_trigger (char *glob,struct event_trigger_data *data,struct trace_event_file *file);
kernel/trace/trace_eprobe.c +2 -1
@@ -551,7 +551,8 @@ static struct event_trigger_ops eprobe_trigger_ops = {static int eprobe_trigger_cmd_parse (struct event_command *cmd_ops,struct trace_event_file *file, -char *glob,char *cmd,char *param) +char *glob,char *cmd, +char *param_and_filter) {return -1 ; }
kernel/trace/trace_events_hist.c +29 -47
163 lines not on this page · raw message
kernel/trace/trace_events_trigger.c +53 -139
290 lines not on this page · raw message
Signature · 1 line
2.33.0
Re: [for-next][PATCH 05/31] tracing: Have existing event_command.parse() implementations use helpers
Hi
Quoted 4 lines · Steven Rostedt
From: Tom Zanussi <[email protected]>
Simplify the existing
event_command.parse()implementations by having them make use of the helper functions previously introduced.
While testing rtla with all for-next changes, I noticed this patch breaks:
# rtla osnoise hist -d 5
Before this patch, osnoise hist is able to enable histograms and collect data. After this patch it does not work.
The event I am creating the histogram is osnoise:sample_threshold, and the trigger command is:
hist:keys=duration.buckets=1000,common_cpu:vals=hitcount:sort=hitcount:size=2048
I did some debug, and found that the histogram is working. The problem is that, to read the histogram I pause it to have consistent data:
in tools/tracing/rtla/osnoise_hist.c:osnoise_read_trace_hist() {
[...]
tracefs_hist_pause(tool->trace.inst, data->trace_hist);
content = tracefs_event_file_read(tool->trace.inst, "osnoise",
"sample_threshold",
"hist", NULL); [...]
}
and, as far as I got, after this patch, pausing the histogram makes it to clear up. If I comment the "tracefs_hist_pause" line, "rtla osnoise hist" start working back again.
Thoughts?
-- Daniel
Steven Rostedt
maintainerRe: [for-next][PATCH 05/31] tracing: Have existing event_command.parse() implementations use helpers
Quoted 11 lines · On Thu, 13 Jan 2022 18:03:07 +0100 Daniel Bristot de Oliveira…
I did some debug, and found that the histogram is working. The problem is that, to read the histogram I pause it to have consistent data:
in
tools/tracing/rtla/osnoise_hist.c:osnoise_read_trace_hist(){
[...]tracefs_hist_pause(tool->trace.inst, data->trace_hist); content = tracefs_event_file_read(tool->trace.inst, "osnoise", "sample_threshold", "hist", NULL);
Quoted 8 lines
[...]
}and, as far as I got, after this patch, pausing the histogram makes it to clear up. If I comment the "
tracefs_hist_pause" line, "rtla osnoise hist" start working back again.Thoughts?
This is all messed up. I'm removing this patch completely.
Tom, can you fix this. The issue is that it's putting too much policy into the helper functions, which is big no no.
Specifically, we have:
int event_trigger_register(struct event_command *cmd_ops,
struct trace_event_file *file, char *glob,
char *cmd,
char *param,
struct event_trigger_data *trigger_data, int *n_registered)
{
int ret;
if (n_registered)
*n_registered = 0;
ret = cmd_ops->reg(glob, trigger_data, file);
/*
* The above returns on success the # of functions enabled,
* but if it didn't find any functions it returns zero.
* Consider no functions a failure too.
*/
if (!ret) {
cmd_ops->unreg(glob, trigger_data, file);
ret = -ENOENT;
} else if (ret > 0) {
if (n_registered)
*n_registered = ret;
/* Just return zero, not the number of enabled functions */
ret = 0;
}
return ret;
}And in the case of pause, this *will* have ret = 0 on return. And what happens is that it removes the trigger completely.
Look at the code in the histogram on the return:
ret = event_trigger_register(cmd_ops, file, glob, cmd, param, trigger_data, &n_registered);
if (ret < 0)
goto out_free;
if ((ret == 0) && (n_registered == 0)) {
if (!(attrs->pause || attrs->cont || attrs->clear))
ret = -ENOENT;
goto out_free;
}It checks for 0 and 0 and only errors if it's not pause, cont, or clear. Hence, all three are now broken due to this patch.
I will not be adding this to this merge window.
-- Steve
Steven Rostedt
maintainerRe: [for-next][PATCH 05/31] tracing: Have existing event_command.parse() implementations use helpers
Quoted 1 line · On Thu, 13 Jan 2022 16:20:58 -0500 Steven Rostedt <[email protected]> wrote:
I will not be adding this to this merge window.
And it appears that "tracing: Remove redundant trigger_ops params" does not apply cleanly without this patch, so I'm dropping that one too.
-- Steve
Re: [for-next][PATCH 05/31] tracing: Have existing event_command.parse() implementations use helpers
In reply to Steven RostedtHi Steve,
Quoted 85 lines · Steven Rostedt
Quoted 16 lines · On Thu, 13 Jan 2022 18:03:07 +0100 Daniel Bristot de Oliveira…
I did some debug, and found that the histogram is working. The problem is that,
to read the histogram I pause it to have consistent data:in
tools/tracing/rtla/osnoise_hist.c:osnoise_read_trace_hist(){
[...]tracefs_hist_pause(tool->trace.inst, data->trace_hist); content = tracefs_event_file_read(tool->trace.inst, "osnoise", "sample_threshold", "hist", NULL);[...]
}Quoted 7 lines
and, as far as I got, after this patch, pausing the histogram makes it to clear
up. If I comment the "tracefs_hist_pause" line, "rtla osnoise hist" start
working back again.Thoughts?
This is all messed up. I'm removing this patch completely.
Tom, can you fix this. The issue is that it's putting too much policy into
the helper functions, which is big no no.Specifically, we have:
int event_trigger_register(struct
event_command*cmd_ops,
structtrace_event_file*file, char *glob,
char *cmd,
char *param,
structevent_trigger_data*trigger_data, int*n_registered){ int ret; if (n_registered) *n_registered = 0; ret = cmd_ops->reg(glob, trigger_data, file); /* * The above returns on success the # of functions enabled, * but if it didn't find any functions it returns zero. * Consider no functions a failure too. */ if (!ret) { cmd_ops->unreg(glob, trigger_data, file); ret = -ENOENT; } else if (ret > 0) { if (n_registered) *n_registered = ret; /* Just return zero, not the number of enabled functions */ ret = 0; } return ret; }And in the case of pause, this *will* have ret = 0 on return. And what
happens is that it removes the trigger completely.Look at the code in the histogram on the return:
ret = event_trigger_register(cmd_ops, file, glob, cmd, param,
trigger_data, &n_registered); if (ret < 0) goto out_free; if ((ret == 0) && (n_registered == 0)) { if (!(attrs->pause || attrs->cont || attrs->clear)) ret = -ENOENT; goto out_free; }
Quoted 5 lines
It checks for 0 and 0 and only errors if it's not pause, cont, or clear.
Hence, all three are now broken due to this patch.I will not be adding this to this merge window.
Yes, you're right, event_trigger_register() is trying to do a little too much, and shouldn't be doing the unreg().
Thanks for finding and figuring that out. I'll fix this and send a new version after the merge window.
Tom
Quoted 2 lines
-- Steve
Steven Rostedt
maintainerPATCH 6/31 tracing: Remove redundant trigger_ops params
In reply to Steven RostedtFrom: Tom Zanussi <[email protected]>
Since event_trigger_data contains the .ops trigger_ops field, there's no reason to pass the trigger_ops separately. Remove it as a param from functions whenever event_trigger_data is passed.
--- kernel/trace/trace.h | 14 +++------ kernel/trace/trace_eprobe.c | 7 ++--- kernel/trace/trace_events_hist.c | 29 ++++++++---------- kernel/trace/trace_events_trigger.c | 46 +++++++++++------------------ 4 files changed, 36 insertions(+), 60 deletions(-)
kernel/trace/trace.h +4 -10
@@ -1574,10 +1574,8 @@ struct enable_trigger_data { };extern int event_enable_trigger_print (struct seq_file *m, -struct event_trigger_ops *ops, -struct event_trigger_data *data); -extern void event_enable_trigger_free (struct event_trigger_ops *ops,struct event_trigger_data *data); +extern void event_enable_trigger_free (struct event_trigger_data *data); extern int event_enable_trigger_parse(struct event_command *cmd_ops, struct trace_event_file *file, char *glob, char *cmd,
@@ -1589,8 +1587,7 @@ extern void event_enable_unregister_trigger(char *glob,struct event_trigger_data *test, struct trace_event_file *file);extern void trigger_data_free (struct event_trigger_data *data); -extern int event_trigger_init (struct event_trigger_ops *ops, -struct event_trigger_data *data); +extern int event_trigger_init (struct event_trigger_data *data);extern int trace_event_trigger_enable_disable (struct trace_event_file *file,int trigger_enable);extern void update_cond_flag (struct trace_event_file *file);
@@ -1688,12 +1685,9 @@ struct event_trigger_ops {struct trace_buffer *buffer, void *rec, struct ring_buffer_event *rbe); -int (*init)(struct event_trigger_ops *ops, -struct event_trigger_data *data); -void (*free)(struct event_trigger_ops *ops, -struct event_trigger_data *data); +int (*init)(struct event_trigger_data *data); +void (*free)(struct event_trigger_data *data);int (*print)(struct seq_file *m, -struct event_trigger_ops *ops,struct event_trigger_data *data); };
kernel/trace/trace_eprobe.c +2 -5
@@ -513,20 +513,17 @@ __eprobe_trace_func(struct eprobe_data *edata, void *rec)* functions are just stubs to fulfill what is needed to use the trigger * infrastructure. */ -static int eprobe_trigger_init (struct event_trigger_ops *ops, -struct event_trigger_data *data) +static int eprobe_trigger_init (struct event_trigger_data *data) {return 0 ; } -static void eprobe_trigger_free (struct event_trigger_ops *ops, -struct event_trigger_data *data) +static void eprobe_trigger_free (struct event_trigger_data *data) { }static int eprobe_trigger_print (struct seq_file *m, -struct event_trigger_ops *ops,struct event_trigger_data *data) {/* Do not print eprobe event triggers */
kernel/trace/trace_events_hist.c +12 -17
@@ -5230,7 +5230,7 @@ static void hist_trigger_show(struct seq_file *m,seq_puts (m,"\n\n" );seq_puts (m,"# event histogram\n#\n# trigger info: " ); - data->ops->seq_puts (m,"#\n\n" ); hist_data = data->private_data;
@@ -5462,7 +5462,7 @@ static void hist_trigger_debug_show(struct seq_file *m,seq_puts (m,"\n\n" );seq_puts (m,"# event histogram\n#\n# trigger info: " ); - data->ops->seq_puts (m,"#\n\n" ); hist_data = data->private_data;
@@ -5599,7 +5599,6 @@ static void hist_field_print(struct seq_file *m, struct hist_field *hist_field) }static int event_hist_trigger_print (struct seq_file *m, -struct event_trigger_ops *ops,struct event_trigger_data *data) {struct hist_trigger_data *hist_data = data->private_data;
@@ -5707,8 +5706,7 @@ static int event_hist_trigger_print(struct seq_file *m,return 0 ; } -static int event_hist_trigger_init (struct event_trigger_ops *ops, -struct event_trigger_data *data) +static int event_hist_trigger_init (struct event_trigger_data *data) {struct hist_trigger_data *hist_data = data->private_data;
@@ -5736,8 +5734,7 @@ static void unregister_field_var_hists(struct hist_trigger_data *hist_data) } } -static void event_hist_trigger_free (struct event_trigger_ops *ops, -struct event_trigger_data *data) +static void event_hist_trigger_free (struct event_trigger_data *data) {struct hist_trigger_data *hist_data = data->private_data;
@@ -5766,25 +5763,23 @@ static struct event_trigger_ops event_hist_trigger_ops = { .free = event_hist_trigger_free, }; -static int event_hist_trigger_named_init (struct event_trigger_ops *ops, -struct event_trigger_data *data) +static int event_hist_trigger_named_init (struct event_trigger_data *data) { data->ref++;save_named_trigger (data->named_data->name, data); -event_hist_trigger_init (ops, data->named_data); +event_hist_trigger_init (data->named_data);return 0 ; } -static void event_hist_trigger_named_free (struct event_trigger_ops *ops, -struct event_trigger_data *data) +static void event_hist_trigger_named_free (struct event_trigger_data *data) {if (WARN_ON_ONCE (data->ref <=0 ))return ; -event_hist_trigger_free (ops, data->named_data); +event_hist_trigger_free (data->named_data); data->ref--;if (!data->ref) {
@@ -5971,7 +5966,7 @@ static int hist_register_trigger(char *glob, }if (data->ops->init) { - ret = data->ops->init (data->ops, data); + ret = data->ops->init (data);if (ret <0 ) goto out; }
@@ -6089,7 +6084,7 @@ static void hist_unregister_trigger(char *glob, }if (unregistered && test->ops->free) - test->ops->free (test->ops, test); + test->ops->free (test);if (hist_data->enable_timestamps) {if (!hist_data->remove || unregistered)
@@ -6142,7 +6137,7 @@ static void hist_unreg_all(struct trace_event_file *file)if (hist_data->enable_timestamps)tracing_set_filter_buffering (file->tr, false);if (test->ops->free) - test->ops->free (test->ops, test); + test->ops->free (test); } } }
@@ -6420,7 +6415,7 @@ static void hist_enable_unreg_all(struct trace_event_file *file)update_cond_flag (file);trace_event_trigger_enable_disable (file,0 );if (test->ops->free) - test->ops->free (test->ops, test); + test->ops->free (test); } } }
kernel/trace/trace_events_trigger.c +18 -28
172 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 7/31 tracing: Account bottom half disabled sections.
In reply to Steven RostedtFrom: Sebastian Andrzej Siewior <[email protected]>
Disabling only bottom halves via local_bh_disable() disables also preemption but this remains invisible to tracing. On a CONFIG_PREEMPT kernel one might wonder why there is no scheduling happening despite the N flag in the trace. The reason might be the a rcu_read_lock_bh() section.
Add a 'b' to the tracing output if in task context with disabled bottom halves.
--- include/linux/trace_events.h | 1 + kernel/trace/trace.c | 6 ++++-- kernel/trace/trace_output.c | 4 ++++ 3 files changed, 9 insertions(+), 2 deletions(-)
include/linux/trace_events.h +1 -0
@@ -172,6 +172,7 @@ enum trace_flag_type {TRACE_FLAG_SOFTIRQ =0x10 ,TRACE_FLAG_PREEMPT_RESCHED =0x20 ,TRACE_FLAG_NMI =0x40 , + TRACE_FLAG_BH_OFF = 0x80, };#ifdef CONFIG_TRACE_IRQFLAGS_SUPPORT
kernel/trace/trace.c +4 -2
@@ -2603,6 +2603,8 @@ unsigned int tracing_gen_ctx_irq_test(unsigned int irqs_status) trace_flags |=TRACE_FLAG_HARDIRQ ;if (in_serving_softirq ()) trace_flags |=TRACE_FLAG_SOFTIRQ ; +if (softirq_count () >> (SOFTIRQ_SHIFT +1 )) + trace_flags |=TRACE_FLAG_BH_OFF ;if (tif_need_resched ()) trace_flags |=TRACE_FLAG_NEED_RESCHED ;
@@ -4190,7 +4192,7 @@ unsigned long trace_total_entries(struct trace_array *tr)static void print_lat_help_header (struct seq_file *m) {seq_puts (m,"# _------=> CPU# \n" -"# / _-----=> irqs-off \n" +"# / _-----=> irqs-off/BH-disabled\n" "# | / _----=> need-resched \n" "# || / _---=> hardirq/softirq \n" "# ||| / _--=> preempt-depth \n"
@@ -4231,7 +4233,7 @@ static void print_func_help_header_irq(struct array_buffer *buf, struct seq_fileprint_event_info (buf, m); -seq_printf (m,"# %.*s _-----=> irqs-off\n" , prec, space); +seq_printf (m,"# %.*s _-----=> irqs-off/BH-disabled\n" , prec, space);seq_printf (m,"# %.*s / _----=> need-resched\n" , prec, space);seq_printf (m,"# %.*s| / _---=> hardirq/softirq\n" , prec, space);seq_printf (m,"# %.*s|| / _--=> preempt-depth\n" , prec, space);
kernel/trace/trace_output.c +4 -0
@@ -445,14 +445,18 @@ int trace_print_lat_fmt(struct trace_seq *s, struct trace_entry *entry)char irqs_off;int hardirq;int softirq; +int bh_off;int nmi; nmi = entry->flags &TRACE_FLAG_NMI ; hardirq = entry->flags &TRACE_FLAG_HARDIRQ ; softirq = entry->flags &TRACE_FLAG_SOFTIRQ ; + bh_off = entry->flags &TRACE_FLAG_BH_OFF ; irqs_off = + (entry->flags &TRACE_FLAG_IRQS_OFF && bh_off) ?'D' : (entry->flags &TRACE_FLAG_IRQS_OFF ) ?'d' : + bh_off ?'b' : (entry->flags &TRACE_FLAG_IRQS_NOSUPPORT ) ?'X' :'.' ;
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 8/31 tracing/uprobes: Check the return value of kstrdup() for tu->filename
In reply to Steven RostedtFrom: Xiaoke Wang <[email protected]>
kstrdup() returns NULL when some internal memory errors happen, it is better to check the return value of it so to catch the memory error in time.
--- kernel/trace/trace_uprobe.c | 5 +++++ 1 file changed, 5 insertions(+)
kernel/trace/trace_uprobe.c +5 -0
@@ -1609,6 +1609,11 @@ create_local_trace_uprobe(char *name, unsigned long offs, tu->path = path; tu->ref_ctr_offset = ref_ctr_offset; tu->filename =kstrdup (name,GFP_KERNEL ); +if (!tu->filename) { + ret = -ENOMEM ; + goto error; + } +init_trace_event_call (tu); ptype =is_ret_probe (tu) ?PROBE_PRINT_RETURN :PROBE_PRINT_NORMAL ;
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 9/31 tracing/probes: check the return value of kstrndup() for pbuf
In reply to Steven RostedtFrom: Xiaoke Wang <[email protected]>
kstrndup() is a memory allocation-related function, it returns NULL when some internal memory errors happen. It is better to check the return value of it so to catch the memory error in time.
--- kernel/trace/trace_probe.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/trace/trace_probe.c +2 -0
@@ -356,6 +356,8 @@ static int __parse_imm_string(char *str, char **pbuf, int offs)return -EINVAL ; } *pbuf =kstrndup (str, len -1 ,GFP_KERNEL ); +if (!*pbuf) +return -ENOMEM ;return 0 ; }
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 10/31 scripts: ftrace - move the sort-processing in ftrace_init
In reply to Steven RostedtFrom: Yinan Liu <[email protected]>
When the kernel starts, the initialization of ftrace takes up a portion of the time (approximately 6~8ms) to sort mcount addresses. We can save this time by moving mcount-sorting to compile time.
--- kernel/trace/ftrace.c | 11 +++- scripts/Makefile | 6 +- scripts/link-vmlinux.sh | 6 +- scripts/sorttable.c | 2 + scripts/sorttable.h | 120 +++++++++++++++++++++++++++++++++++++++- 5 files changed, 137 insertions(+), 8 deletions(-)
kernel/trace/ftrace.c +9 -2
@@ -6406,8 +6406,15 @@ static int ftrace_process_locs(struct module *mod,if (!count)return 0 ; -sort (start, count,sizeof (*start), - ftrace_cmp_ips,NULL ); +/* +* Sorting mcount in vmlinux at build time depend on +* CONFIG_BUILDTIME_TABLE_SORT, while mcount loc in +* modules can not be sorted at build time. +*/ +if (!IS_ENABLED (CONFIG_BUILDTIME_TABLE_SORT ) || mod) { +sort (start, count,sizeof (*start), + ftrace_cmp_ips,NULL ); + } start_pg =ftrace_allocate_pages (count);if (!start_pg)
scripts/Makefile +5 -1
@@ -17,6 +17,7 @@ hostprogs-always-$(CONFIG_SYSTEM_EXTRA_CERTIFICATE) += insert-sys-cert hostprogs-always-$(CONFIG_SYSTEM_REVOCATION_LIST) += extract-cert HOSTCFLAGS_sorttable.o = -I$(srctree)/tools/include +HOSTLDLIBS_sorttable = -lpthread HOSTCFLAGS_asn1_compiler.o = -I$(srctree)/include HOSTCFLAGS_sign-file.o = $(CRYPTO_CFLAGS) HOSTLDLIBS_sign-file = $(CRYPTO_LIBS)
@@ -29,7 +30,10 @@ ARCH := x86 endif HOSTCFLAGS_sorttable.o += -I$(srctree)/tools/arch/x86/include HOSTCFLAGS_sorttable.o += -DUNWINDER_ORC_ENABLED -HOSTLDLIBS_sorttable = -lpthread +endif + +ifdef CONFIG_DYNAMIC_FTRACE +HOSTCFLAGS_sorttable.o += -DMCOUNT_SORT_ENABLED endif # The following programs are only built on demand
scripts/link-vmlinux.sh +3 -3
@@ -400,6 +400,9 @@ if [ -n "${CONFIG_DEBUG_INFO_BTF}" -a -n "${CONFIG_BPF}" ]; then ${RESOLVE_BTFIDS} vmlinux fi +info SYSMAP System.map +mksysmap vmlinux System.map + if [ -n "${CONFIG_BUILDTIME_TABLE_SORT}" ]; then info SORTTAB vmlinux if ! sorttable vmlinux; then
@@ -408,9 +411,6 @@ if [ -n "${CONFIG_BUILDTIME_TABLE_SORT}" ]; then fi fi -info SYSMAP System.map -mksysmap vmlinux System.map - # step a (see comment above) if [ -n "${CONFIG_KALLSYMS}" ]; then mksysmap ${kallsyms_vmlinux} .tmp_System.map
scripts/sorttable.c +2 -0
@@ -30,6 +30,8 @@#include <stdlib.h> #include <string.h> #include <unistd.h> +#include <errno.h> +#include <pthread.h> #include <tools/be_byteshift.h> #include <tools/le_byteshift.h>
scripts/sorttable.h +118 -2
174 lines not on this page · raw message
Signature · 1 line
2.33.0
Nathan Chancellor
maintainerRe: [for-next][PATCH 10/31] scripts: ftrace - move the sort-processing in ftrace_init
Hi Steven and Yinan,
Quoted 7 lines · Steven Rostedt
From: Yinan Liu <[email protected]>
When the kernel starts, the initialization of ftrace takes up a portion of the time (approximately 6~8ms) to sort mcount addresses. We can save this time by moving mcount-sorting to compile time.
Quoted 6 lines
Signed-off-by Yinan Liu <[email protected]>Reported-by kernel test robot <[email protected]>Reported-by kernel test robot <[email protected]>Signed-off-by Steven Rostedt <[email protected]>
This change as commit 72b3942a173c ("scripts: ftrace - move the sort-processing in ftrace_init") in -next causes a bunch of warnings at the beginning of the build when using clang as the host compiler:
$ make -skj"$(nproc)" LLVM=1 distclean allmodconfig init/main.o
In file included from scripts/sorttable.c:195:
scripts/sorttable.h:380:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:380:2: note: remove the 'if' if its condition is always false
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:380:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever '||' condition is true [-Wsometimes-uninitialized]
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:380:6: note: remove the '||' if its condition is always false
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:380:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever '||' condition is true [-Wsometimes-uninitialized]
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:380:6: note: remove the '||' if its condition is always false
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:288:30: note: initialize the variable 'mcount_sort_thread' to silence this warning
pthread_t mcount_sort_thread;
^
= 0
In file included from scripts/sorttable.c:197:
scripts/sorttable.h:380:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:380:2: note: remove the 'if' if its condition is always false
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:380:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever '||' condition is true [-Wsometimes-uninitialized]
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:380:6: note: remove the '||' if its condition is always false
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:380:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever '||' condition is true [-Wsometimes-uninitialized]
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:380:6: note: remove the '||' if its condition is always false
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
^~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:370:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
if (pthread_create(&orc_sort_thread, NULL,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:370:2: note: remove the 'if' if its condition is always false
if (pthread_create(&orc_sort_thread, NULL,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:360:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
if (orc_ip_size % sizeof(int) != 0 ||
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:360:2: note: remove the 'if' if its condition is always false
if (orc_ip_size % sizeof(int) != 0 ||
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:360:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever '||' condition is true [-Wsometimes-uninitialized]
if (orc_ip_size % sizeof(int) != 0 ||
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:360:6: note: remove the '||' if its condition is always false
if (orc_ip_size % sizeof(int) != 0 ||
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:360:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever '||' condition is true [-Wsometimes-uninitialized]
if (orc_ip_size % sizeof(int) != 0 ||
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:360:6: note: remove the '||' if its condition is always false
if (orc_ip_size % sizeof(int) != 0 ||
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:353:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
if (!g_orc_ip_table || !g_orc_table) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:353:2: note: remove the 'if' if its condition is always false
if (!g_orc_ip_table || !g_orc_table) {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sorttable.h:353:6: warning: variable 'mcount_sort_thread' is used uninitialized whenever '||' condition is true [-Wsometimes-uninitialized]
if (!g_orc_ip_table || !g_orc_table) {
^~~~~~~~~~~~~~~
scripts/sorttable.h:479:6: note: uninitialized use occurs here
if (mcount_sort_thread) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:353:6: note: remove the '||' if its condition is always false
if (!g_orc_ip_table || !g_orc_table) {
^~~~~~~~~~~~~~~~~~
scripts/sorttable.h:288:30: note: initialize the variable 'mcount_sort_thread' to silence this warning
pthread_t mcount_sort_thread;
^
= 0
12 warnings generated.Should mcount_sort_thread be zero initialized or is there something else going on here? I am currently hunting down a bunch of other regressions so apologies for just the report rather than a patch to fix it.
Cheers,
Nathan
Steven Rostedt
maintainerRe: [for-next][PATCH 10/31] scripts: ftrace - move the sort-processing in ftrace_init
Quoted 24 lines · On Sat, 15 Jan 2022 13:36:04 -0700 Nathan Chancellor <[email protected]> wrote:
Hi Steven and Yinan,
Quoted 7 lines · Steven Rostedt
From: Yinan Liu <[email protected]>
When the kernel starts, the initialization of ftrace takes up a portion of the time (approximately 6~8ms) to sort mcount addresses. We can save this time by moving mcount-sorting to compile time.
Quoted 6 lines
Signed-off-by Yinan Liu <[email protected]>Reported-by kernel test robot <[email protected]>Reported-by kernel test robot <[email protected]>Signed-off-by Steven Rostedt <[email protected]>This change as commit
72b3942a173c("scripts: ftrace - move the sort-processing inftrace_init") in -next causes a bunch of warnings at the beginning of the build when using clang as the host compiler:
Quoted 3 lines
Should
mcount_sort_threadbe zero initialized or is there something else going on here? I am currently hunting down a bunch of other regressions so apologies for just the report rather than a patch to fix it.
Can this really happen? We have:
if (pthread_create(&mcount_sort_thread, NULL, &sort_mcount_loc, &mstruct)) {
fprintf(stderr,
"pthread_create mcount_sort_thread failed '%s': %s\n",
strerror(errno), fname);
goto out;
}[..]
if (mcount_sort_thread) {
void *retval = NULL;
/* wait for mcount sort done */
rc = pthread_join(mcount_sort_thread, &retval);
if (rc) {
fprintf(stderr,
"pthread_join failed '%s': %s\n",
strerror(errno), fname);
} else if (retval) {
rc = -1;
fprintf(stderr,
"failed to sort mcount '%s': %s\n",
(char *)retval, fname);
}
}Shouldn't the pthread_create() initialize it? And I'm not even sure if we need that if statement?
Or is there a path to get there without pthread_create() initializing it?
-- Steve
Nathan Chancellor
maintainerRe: [for-next][PATCH 10/31] scripts: ftrace - move the sort-processing in ftrace_init
Quoted 27 lines · Steven Rostedt
Quoted 24 lines · On Sat, 15 Jan 2022 13:36:04 -0700 Nathan Chancellor <[email protected]> wrote:
Hi Steven and Yinan,
Quoted 7 lines · Steven Rostedt
From: Yinan Liu <[email protected]>
When the kernel starts, the initialization of ftrace takes up a portion of the time (approximately 6~8ms) to sort mcount addresses. We can save this time by moving mcount-sorting to compile time.
Quoted 6 lines
Signed-off-by Yinan Liu <[email protected]>Reported-by kernel test robot <[email protected]>Reported-by kernel test robot <[email protected]>Signed-off-by Steven Rostedt <[email protected]>This change as commit
72b3942a173c("scripts: ftrace - move the sort-processing inftrace_init") in -next causes a bunch of warnings at the beginning of the build when using clang as the host compiler:
Quoted 5 lines
Quoted 3 lines
Should
mcount_sort_threadbe zero initialized or is there something else going on here? I am currently hunting down a bunch of other regressions so apologies for just the report rather than a patch to fix it.Can this really happen? We have:
The way the code is written now, yes.
Quoted 24 lines
if (pthread_create(&mcount_sort_thread, NULL, &sort_mcount_loc, &mstruct)) { fprintf(stderr, "pthread_create mcount_sort_thread failed '%s': %s\n", strerror(errno), fname); goto out; }[..]
if (mcount_sort_thread) { void *retval = NULL; /* wait for mcount sort done */ rc = pthread_join(mcount_sort_thread, &retval); if (rc) { fprintf(stderr, "pthread_join failed '%s': %s\n", strerror(errno), fname); } else if (retval) { rc = -1; fprintf(stderr, "failed to sort mcount '%s': %s\n", (char *)retval, fname); } }
Quoted 4 lines
Shouldn't the
pthread_create()initialize it? And I'm not even sure if we need that if statement?Or is there a path to get there without
pthread_create()initializing it?
Yes. If the if statment right above the pthread_create() call triggers, we jump to the out label, which hits the if (mcount_sort_thread), and mcount_sort_thread won't be initialized.
if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) {
fprintf(stderr,
"incomplete mcount's sort in file: %s\n",
fname);
goto out;
}
if (pthread_create(&mcount_sort_thread, ...)) {...
out:
...
if (mcount_sort_thread) {If I am misunderstanding something, please let me know.
Cheers,
Nathan
cover fix initialization problems
In reply to Steven RostedtHi,steven and Nathan
Thanks to Nathan, I realize that there are some problems with initialization. Initialization of pthreads and mstructs may fail or be skipped, in which case it is possible to use uninitialized values in "out" logic. I'm not sure if such an error would happen, but it's certainly safe to initialize it.
I use mcount_sort_pthread as a global variable, which is consistent with orc_sort_pthread. And initialize the mstruct in function do_sort.
Yinan Liu (1):
script/sorttable: fix some initialization problems
scripts/sorttable.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
Signature · 1 line
2.19.1.6.gb485710b
PATCH 1/1 script/sorttable: fix some initialization problems
elf_mcount_loc and mcount_sort_thread definitions are not initialized immediately within the function, which can cause the judgment logic to use uninitialized values when the initialization logic of subsequent code fails.
--- scripts/sorttable.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
scripts/sorttable.h +3 -2
@@ -199,6 +199,8 @@ static int compare_extable(const void *a, const void *b)return 0 ; } #ifdef MCOUNT_SORT_ENABLED +pthread_t mcount_sort_thread; + struct elf_mcount_loc { Elf_Ehdr *ehdr; Elf_Shdr *init_data_sec;
@@ -282,10 +284,9 @@ static int do_sort(Elf_Ehdr *ehdr,unsigned int shnum;unsigned int shstrndx;#ifdef MCOUNT_SORT_ENABLED -struct elf_mcount_loc mstruct; +struct elf_mcount_loc mstruct = {NULL ,NULL ,0 ,0 };uint_t _start_mcount_loc =0 ;uint_t _stop_mcount_loc =0 ; -pthread_t mcount_sort_thread;#endif #if defined(SORTTABLE_64 ) && defined(UNWINDER_ORC_ENABLED )unsigned int orc_ip_size =0 ;
Signature · 1 line
2.19.1.6.gb485710b
Re: [PATCH 1/1] script/sorttable: fix some initialization problems
Quoted 8 lines · Yinan Liu
elf_mcount_locandmcount_sort_threaddefinitions are not initialized immediately within the function, which can cause the judgment logic to use uninitialized values when the initialization logic of subsequent code fails.Fixes 72b3942a173c (scripts: ftrace - move the sort-processing in ftrace_init)
This should be:
You can add an alias to always get the format right, like:
$ git config --global alias.fixes 'show -s --format="Fixes: %h (\"%s\")"'
$ git fixes 72b3942a173c387b27860ba1069636726e208777
Fixes: 72b3942a173c ("scripts: ftrace - move the sort-processing in ftrace_init")Quoted 1 line
Signed-off-by Yinan Liu <[email protected]>
This resolves the warnings I reported and does not introduce any new ones. It seems reasonable to me.
Quoted 18 lines
--- scripts/sorttable.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)scripts/sorttable.h +2
-0@@ -199,6 +199,8 @@ static int compare_extable(const void *a, const void *b)return 0 ; } #ifdef MCOUNT_SORT_ENABLED +pthread_t mcount_sort_thread; + struct elf_mcount_loc { Elf_Ehdr *ehdr; Elf_Shdr *init_data_sec;
scripts/sorttable.h static int do_sort(Elf_Ehdr *ehdr, · line 287
Quoted 5 lines
scripts/sorttable.h +1
-1@@ -282,10 +284,9 @@ static int do_sort(Elf_Ehdr *ehdr,unsigned int shnum;unsigned int shstrndx;#ifdef MCOUNT_SORT_ENABLED -struct elf_mcount_loc mstruct; +struct elf_mcount_loc mstruct = {NULL ,NULL ,0 ,0 };
Wonder if this would be better using either '= {}' or '= {0}'?
Quoted 10 lines
scripts/sorttable.h +0
-1uint_t _start_mcount_loc =0 ;uint_t _stop_mcount_loc =0 ; -pthread_t mcount_sort_thread;#endif #if defined(SORTTABLE_64 ) && defined(UNWINDER_ORC_ENABLED )unsigned int orc_ip_size =0 ;Signature · 1 line
2.19.1.6.gb485710b
PATCH v2 1/1 script/sorttable: fix some initialization problems
In reply to Steven Rostedtelf_mcount_loc and mcount_sort_thread definitions are not initialized immediately within the function, which can cause the judgment logic to use uninitialized values when the initialization logic of subsequent code fails.
--- scripts/sorttable.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
scripts/sorttable.h +3 -2
@@ -199,6 +199,8 @@ static int compare_extable(const void *a, const void *b)return 0 ; } #ifdef MCOUNT_SORT_ENABLED +pthread_t mcount_sort_thread; + struct elf_mcount_loc { Elf_Ehdr *ehdr; Elf_Shdr *init_data_sec;
@@ -282,10 +284,9 @@ static int do_sort(Elf_Ehdr *ehdr,unsigned int shnum;unsigned int shstrndx;#ifdef MCOUNT_SORT_ENABLED -struct elf_mcount_loc mstruct; +struct elf_mcount_loc mstruct = {0 };uint_t _start_mcount_loc =0 ;uint_t _stop_mcount_loc =0 ; -pthread_t mcount_sort_thread;#endif #if defined(SORTTABLE_64 ) && defined(UNWINDER_ORC_ENABLED )unsigned int orc_ip_size =0 ;
Signature · 1 line
2.19.1.6.gb485710b
Steven Rostedt
maintainerPATCH 11/31 ftrace: Add test to make sure compiled time sorts work
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Now that ftrace function pointers are sorted at compile time, add a test that makes sure they are sorted at run time. This test is only run if it is configured in.
--- kernel/trace/Kconfig | 14 ++++++++++++++ kernel/trace/ftrace.c | 23 +++++++++++++++++++++++ 2 files changed, 37 insertions(+)
kernel/trace/Kconfig +14 -0
@@ -915,6 +915,20 @@ config EVENT_TRACE_TEST_SYSCALLS TBD - enable a way to actually call the syscalls as we test their events +config FTRACE_SORT_STARTUP_TEST + bool "Verify compile time sorting of ftrace functions" + depends on DYNAMIC_FTRACE + depends on BUILDTIME_TABLE_SORT + help + Sorting of the mcount_loc sections that is used to find the + where the ftrace knows where to patch functions for tracing + and other callbacks is done at compile time. But if the sort + is not done correctly, it will cause non-deterministic failures. + When this is set, the sorted sections will be verified that they + are in deed sorted and will warn if they are not. + + If unsure, say N + config RING_BUFFER_STARTUP_TEST bool "Ring buffer startup self test" depends on RING_BUFFER
kernel/trace/ftrace.c +23 -0
@@ -6388,6 +6388,27 @@ static int ftrace_cmp_ips(const void *a, const void *b)return 0 ; } +#ifdef CONFIG_FTRACE_SORT_STARTUP_TEST +static void test_is_sorted (unsigned long *start,unsigned long count) +{ +int i; + +for (i =1 ; i < count; i++) { +if (WARN (start[i -1 ] > start[i], +"[%d] %pS at %lx is not sorted with %pS at %lx\n" , i, + (void *)start[i -1 ], start[i -1 ], + (void *)start[i], start[i])) +break ; + } +if (i == count) +pr_info ("ftrace section at %px sorted properly\n" , start); +} +#else +static void test_is_sorted (unsigned long *start,unsigned long count) +{ +} +#endif +static int ftrace_process_locs (struct module *mod,unsigned long *start,unsigned long *end)
@@ -6414,6 +6435,8 @@ static int ftrace_process_locs(struct module *mod,if (!IS_ENABLED (CONFIG_BUILDTIME_TABLE_SORT ) || mod) {sort (start, count,sizeof (*start), ftrace_cmp_ips,NULL ); + }else { +test_is_sorted (start, count); } start_pg =ftrace_allocate_pages (count);
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 12/31 tracing: Fix mismatched comment in __string_len
In reply to Steven RostedtFrom: Geliang Tang <[email protected]>
Here __assign_str_len() should be used for the __string_len type, instead of __assign_str() in the comment.
--- samples/trace_events/trace-events-sample.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
samples/trace_events/trace-events-sample.h +1 -1
@@ -155,7 +155,7 @@* * To assign this string, use the helper macro __assign_str_len(). * -* __assign_str(foo, bar, len); +* __assign_str_len(foo, bar, len); * * Then len + 1 is allocated to the ring buffer, and a nul terminating * byte is added. This is similar to:
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 13/31 tracing: Have syscall trace events use trace_event_buffer_lock_reserve()
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Currently, the syscall trace events call trace_buffer_lock_reserve() directly, which means that it misses out on some of the filtering optimizations provided by the helper function trace_event_buffer_lock_reserve(). Have the syscall trace events call that instead, as it was missed when adding the update to use the temp buffer when filtering.
--- kernel/trace/trace_syscalls.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
kernel/trace/trace_syscalls.c +2 -4
@@ -323,8 +323,7 @@ static void ftrace_syscall_enter(void *data, struct pt_regs *regs, long id) trace_ctx =tracing_gen_ctx (); - buffer = tr->array_buffer.buffer; - event =trace_buffer_lock_reserve (buffer, + event =trace_event_buffer_lock_reserve (&buffer, trace_file, sys_data->enter_event->event.type, size, trace_ctx);if (!event)return ;
@@ -367,8 +366,7 @@ static void ftrace_syscall_exit(void *data, struct pt_regs *regs, long ret) trace_ctx =tracing_gen_ctx (); - buffer = tr->array_buffer.buffer; - event =trace_buffer_lock_reserve (buffer, + event =trace_event_buffer_lock_reserve (&buffer, trace_file, sys_data->exit_event->event.type,sizeof (*entry), trace_ctx);if (!event)
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 14/31 tracing: Add test for user space strings when filtering on string pointers
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Pingfan reported that the following causes a fault:
echo "filename ~ \"cpu\"" > events/syscalls/sys_enter_openat/filter echo 1 > events/syscalls/sys_enter_at/enable
The reason is that trace event filter treats the user space pointer defined by "filename" as a normal pointer to compare against the "cpu" string. If the string is not loaded into memory yet, it will trigger a fault in kernel space:
kvm-03-guest16 login: [72198.026181] BUG: unable to handle page fault for address: 00007fffaae8ef60 #PF: supervisor read access in kernel mode
#PF: error_code(0x0001) - permissions violation
PGD 80000001008b7067 P4D 80000001008b7067 PUD 2393f1067 PMD 2393ec067 PTE 8000000108f47867
Oops: 0001 [#1] PREEMPT SMP PTI
CPU: 1 PID: 1 Comm: systemd Kdump: loaded Not tainted 5.14.0-32.el9.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
RIP: 0010:strlen+0x0/0x20
Code: 48 89 f9 74 09 48 83 c1 01 80 39 00 75 f7 31 d2 44 0f b6 04 16 44 88 04 11
48 83 c2 01 45 84 c0 75 ee c3 0f 1f 80 00 00 00 00 <80> 3f 00 74 10 48 89 f8
48 83 c0 01 80 38 00 75 f7 48 29 f8 c3 31
RSP: 0018:ffffb5b900013e48 EFLAGS: 00010246
RAX: 0000000000000018 RBX: ffff8fc1c49ede00 RCX: 0000000000000000
RDX: 0000000000000020 RSI: ffff8fc1c02d601c RDI: 00007fffaae8ef60
RBP: 00007fffaae8ef60 R08: 0005034f4ddb8ea4 R09: 0000000000000000
R10: ffff8fc1c02d601c R11: 0000000000000000 R12: ffff8fc1c8a6e380
R13: 0000000000000000 R14: ffff8fc1c02d6010 R15: ffff8fc1c00453c0
FS: 00007fa86123db40(0000) GS:ffff8fc2ffd00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fffaae8ef60 CR3: 0000000102880001 CR4: 00000000007706e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
filter_pred_pchar+0x18/0x40
filter_match_preds+0x31/0x70
ftrace_syscall_enter+0x27a/0x2c0
syscall_trace_enter.constprop.0+0x1aa/0x1d0
do_syscall_64+0x16/0x90
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7fa861d88664To be even more robust, test both kernel and user space strings. If the string fails to read, then simply have the filter fail.
--- Documentation/trace/events.rst | 10 +++++ kernel/trace/trace_events_filter.c | 59 ++++++++++++++++++++++++++++-- 2 files changed, 66 insertions(+), 3 deletions(-)
Documentation/trace/events.rst +10 -0
@@ -230,6 +230,16 @@ Currently the caret ('^') for an error always appears at the beginning of the filter string; the error message should still be useful though even without more accurate position info. +5.2.1 Filter limitations +------------------------ + +If a filter is placed on a string pointer ``(char *)`` that does not point +to a string on the ring buffer, but instead points to kernel or user space +memory, then, for safety reasons, at most 1024 bytes of the content is +copied onto a temporary buffer to do the compare. If the copy of the memory +faults (the pointer points to memory that should not be accessed), then the +string compare will be treated as not matching. + 5.3 Clearing filters --------------------
kernel/trace/trace_events_filter.c +56 -3
92 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 15/31 tracing/kprobes: nmissed not showed correctly for kretprobe
In reply to Steven RostedtFrom: Xiangyang Zhang <[email protected]>
The 'nmissed' column of the 'kprobe_profile' file for kretprobe is not showed correctly, kretprobe can be skipped by two reasons, shortage of kretprobe_instance which is counted by tk->rp.nmissed, and kprobe itself is missed by some reason, so to show the sum.
--- kernel/trace/trace_kprobe.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
kernel/trace/trace_kprobe.c +4 -1
@@ -1170,15 +1170,18 @@ static int probes_profile_seq_show(struct seq_file *m, void *v) {struct dyn_event *ev = v;struct trace_kprobe *tk; +unsigned long nmissed;if (!is_trace_kprobe (ev))return 0 ; tk =to_trace_kprobe (ev); + nmissed =trace_kprobe_is_return (tk) ? + tk->rp.kp.nmissed + tk->rp.nmissed : tk->rp.kp.nmissed;seq_printf (m," %-44s %15lu %15lu\n" ,trace_probe_name (&tk->tp),trace_kprobe_nhit (tk), - tk->rp.kp.nmissed); + nmissed);return 0 ; }
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 16/31 rtla: Real-Time Linux Analysis tool
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
The rtla is a meta-tool that includes a set of commands that aims to analyze the real-time properties of Linux. But instead of testing Linux as a black box, rtla leverages kernel tracing capabilities to provide precise information about the properties and root causes of unexpected results.
rtla --help works and provide information about the available options.
This is just the "main" and the Makefile, no function yet.
--- tools/tracing/rtla/Makefile | 76 +++++++++++++++++++++++++++++++++++ tools/tracing/rtla/README.txt | 36 +++++++++++++++++ tools/tracing/rtla/src/rtla.c | 72 +++++++++++++++++++++++++++++++++ 3 files changed, 184 insertions(+) create mode 100644 tools/tracing/rtla/Makefile create mode 100644 tools/tracing/rtla/README.txt create mode 100644 tools/tracing/rtla/src/rtla.c
tools/tracing/rtla/Makefile new file +76 -0
@@ -0,0 +1,76 @@ +NAME := rtla +VERSION := 0.4 + +# From libtracefs: +# Makefiles suck: This macro sets a default value of $(2) for the +# variable named by $(1), unless the variable has been set by +# environment or command line. This is necessary for CC and AR +# because make sets default values, so the simpler ?= approach +# won't work as expected. +define allow-override + $(if $(or $(findstring environment,$(origin $(1))),\ + $(findstring command line,$(origin $(1)))),,\ + $(eval $(1) = $(2))) +endef + +# Allow setting CC and AR, or setting CROSS_COMPILE as a prefix. +$(call allow-override,CC,$(CROSS_COMPILE)gcc) +$(call allow-override,AR,$(CROSS_COMPILE)ar) +$(call allow-override,STRIP,$(CROSS_COMPILE)strip) +$(call allow-override,PKG_CONFIG,pkg-config) +$(call allow-override,LD_SO_CONF_PATH,/etc/ld.so.conf.d/) +$(call allow-override,LDCONFIG,ldconfig) + +INSTALL = install +FOPTS := -flto=auto -ffat-lto-objects -fexceptions -fstack-protector-strong \ + -fasynchronous-unwind-tables -fstack-clash-protection +WOPTS := -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -Wno-maybe-uninitialized + +TRACEFS_HEADERS := $$($(PKG_CONFIG) --cflags libtracefs) + +CFLAGS := -O -g -DVERSION=\"$(VERSION)\" $(FOPTS) $(MOPTS) $(WOPTS) $(TRACEFS_HEADERS) +LDFLAGS := -ggdb +LIBS := $$($(PKG_CONFIG) --libs libtracefs) -lprocps + +SRC := $(wildcard src/*.c) +HDR := $(wildcard src/*.h) +OBJ := $(SRC:.c=.o) +DIRS := src +FILES := Makefile README.txt +CEXT := bz2 +TARBALL := $(NAME)-$(VERSION).tar.$(CEXT) +TAROPTS := -cvjf $(TARBALL) +BINDIR := /usr/bin +DATADIR := /usr/share +DOCDIR := $(DATADIR)/doc +MANDIR := $(DATADIR)/man +LICDIR := $(DATADIR)/licenses + +.PHONY: all +all: rtla + +rtla: $(OBJ) + $(CC) -o rtla $(LDFLAGS) $(OBJ) $(LIBS) + +static: $(OBJ) + $(CC) -o rtla-static $(LDFLAGS) --static $(OBJ) $(LIBS) -lpthread -ldl + +.PHONY: install +install: + $(INSTALL) -d -m 755 $(DESTDIR)$(BINDIR) + $(INSTALL) rtla -m 755 $(DESTDIR)$(BINDIR) + $(STRIP) $(DESTDIR)$(BINDIR)/rtla + +.PHONY: clean tarball +clean: + @test ! -f rtla || rm rtla + @test ! -f rtla-static || rm rtla-static + @test ! -f src/rtla.o || rm src/rtla.o + @test ! -f $(TARBALL) || rm -f $(TARBALL) + @rm -rf *~ $(OBJ) *.tar.$(CEXT) + +tarball: clean + rm -rf $(NAME)-$(VERSION) && mkdir $(NAME)-$(VERSION) + cp -r $(DIRS) $(FILES) $(NAME)-$(VERSION) + tar $(TAROPTS) --exclude='*~' $(NAME)-$(VERSION) + rm -rf $(NAME)-$(VERSION)
tools/tracing/rtla/README.txt new file +36 -0
@@ -0,0 +1,36 @@ +RTLA: Real-Time Linux Analysis tools + +The rtla is a meta-tool that includes a set of commands that +aims to analyze the real-time properties of Linux. But, instead of +testing Linux as a black box, rtla leverages kernel tracing +capabilities to provide precise information about the properties +and root causes of unexpected results. + +Installing RTLA + +RTLA depends on some libraries and tools. More precisely, it depends on the +following libraries: + + - libtracefs + - libtraceevent + - procps + +It also depends on python3-docutils to compile man pages. + +For development, we suggest the following steps for compiling rtla: + + $ git clone git://git.kernel.org/pub/scm/libs/libtrace/libtraceevent.git + $ cd libtraceevent/ + $ make + $ sudo make install + $ cd .. + $ git clone git://git.kernel.org/pub/scm/libs/libtrace/libtracefs.git + $ cd libtracefs/ + $ make + $ sudo make install + $ cd .. + $ cd $rtla_src + $ make + $ sudo make install + +For further information, please refer to the rtla man page.
tools/tracing/rtla/src/rtla.c new file +72 -0
72 lines not on this page · raw message
Signature · 1 line
2.33.0
Re: [for-next][PATCH 16/31] rtla: Real-Time Linux Analysis tool
Hi Steven!
Quoted 8 lines · Steven Rostedt
From: Daniel Bristot de Oliveira <[email protected]>
The rtla is a meta-tool that includes a set of commands that aims to analyze the real-time properties of Linux. But instead of testing Linux as a black box, rtla leverages kernel tracing capabilities to provide precise information about the properties and root causes of unexpected results.
Quoted 5 lines
rtla --help works and provide information about the available options.
This is just the "main" and the
Makefile, no function yet.
Thanks for picking RTLA!
Just one problem, you are picking the V8, but we actually had a V9:
https://lore.kernel.org/lkml/[email protected]/
-- Daniel
Steven Rostedt
maintainerRe: [for-next][PATCH 16/31] rtla: Real-Time Linux Analysis tool
Quoted 6 lines · On Wed, 12 Jan 2022 12:15:13 +0100 Daniel Bristot de Oliveira…
Quoted 1 line
Thanks for picking RTLA!
Just one problem, you are picking the V8, but we actually had a V9:
https://lore.kernel.org/lkml/[email protected]/
Strange. I should have picked it up from Patchwork.
https://patchwork.kernel.org/project/linux-trace-devel/list/?series=604588
Not sure how that happened. As it's not kernel code, I may rebase to fix it.
-- Steve
Steven Rostedt
maintainerRe: [for-next][PATCH 16/31] rtla: Real-Time Linux Analysis tool
Quoted 4 lines · On Wed, 12 Jan 2022 11:21:59 -0500 Steven Rostedt <[email protected]> wrote:
Strange. I should have picked it up from Patchwork.
https://patchwork.kernel.org/project/linux-trace-devel/list/?series=604588
The above is the wrong v9 :-p (Now I see how this could have happened!)
Should have been:
https://patchwork.kernel.org/project/linux-trace-devel/list/?series=593861
-- Steve
Quoted 4 lines
Not sure how that happened. As it's not kernel code, I may rebase to fix it.
Re: [for-next][PATCH 16/31] rtla: Real-Time Linux Analysis tool
Quoted 11 lines · Steven Rostedt
Quoted 3 lines · On Wed, 12 Jan 2022 11:21:59 -0500 Steven Rostedt <[email protected]> wrote:
Strange. I should have picked it up from Patchwork.
https://patchwork.kernel.org/project/linux-trace-devel/list/?series=604588
The above is the wrong v9 :-p (Now I see how this could have happened!)
Should have been:
https://patchwork.kernel.org/project/linux-trace-devel/list/?series=593861
That is it, thanks Steve!
-- Daniel
Steven Rostedt
maintainerRe: [for-next][PATCH 16/31] rtla: Real-Time Linux Analysis tool
In reply to Steven RostedtQuoted 6 lines · On Wed, 12 Jan 2022 11:30:54 -0500 Steven Rostedt <[email protected]> wrote:
Quoted 3 lines · On Wed, 12 Jan 2022 11:21:59 -0500 Steven Rostedt <[email protected]> wrote:
Strange. I should have picked it up from Patchwork.
Quoted 8 lines
The above is the wrong v9 :-p (Now I see how this could have happened!)
Should have been:
https://patchwork.kernel.org/project/linux-trace-devel/list/?series=593861
And by making this mistake, I noticed something in doing the diff. You put rtla into Documentation/tools. It should be in tools/ itself, not the Documentation directory.
-- Steve
Steven Rostedt
maintainerRe: [for-next][PATCH 16/31] rtla: Real-Time Linux Analysis tool
Quoted 3 lines · On Wed, 12 Jan 2022 12:14:26 -0500 Steven Rostedt <[email protected]> wrote:
And by making this mistake, I noticed something in doing the diff. You put rtla into
Documentation/tools. It should be intools/itself, not the Documentation directory.
At least it use to be in tools/tracing/rtla, but this series has code in Documentation/tools/rtla, was it not fully moved over?
-- Steve
Re: [for-next][PATCH 16/31] rtla: Real-Time Linux Analysis tool
Quoted 8 lines · Steven Rostedt
Quoted 3 lines · On Wed, 12 Jan 2022 12:14:26 -0500 Steven Rostedt <[email protected]> wrote:
And by making this mistake, I noticed something in doing the diff. You put rtla into
Documentation/tools. It should be intools/itself, not the Documentation directory.At least it use to be in
tools/tracing/rtla, but this series has code inDocumentation/tools/rtla, was it not fully moved over?
The code is in tools/tracing/rtla/. The documentation is in Documentation/tools/rtla. It was requested by Corbet in the v3 [1].
Am I missing something?
[1] https://lore.kernel.org/lkml/[email protected]/
-- Daniel
Steven Rostedt
maintainerPATCH 17/31 rtla: Helper functions for rtla
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
This is a set of utils and tracer helper functions. They are used by rtla mostly to parse config, display data and some trace operations that are not part of libtracefs (because they are only useful it for this case).
--- tools/tracing/rtla/src/trace.c | 192 +++++++++++++++ tools/tracing/rtla/src/trace.h | 27 ++ tools/tracing/rtla/src/utils.c | 433 +++++++++++++++++++++++++++++++++ tools/tracing/rtla/src/utils.h | 56 +++++ 4 files changed, 708 insertions(+) create mode 100644 tools/tracing/rtla/src/trace.c create mode 100644 tools/tracing/rtla/src/trace.h create mode 100644 tools/tracing/rtla/src/utils.c create mode 100644 tools/tracing/rtla/src/utils.h
tools/tracing/rtla/src/trace.c new file +192 -0
@@ -0,0 +1,192 @@ +// SPDX-License-Identifier: GPL-2.0 +#define _GNU_SOURCE +#include <sys/sendfile.h> +#include <tracefs.h> +#include <signal.h> +#include <stdlib.h> +#include <unistd.h> +#include <errno.h> + +#include "trace.h" +#include "utils.h" + +/* +* enable_tracer_by_name - enable a tracer on the given instance +*/ +int enable_tracer_by_name (struct tracefs_instance *inst,const char *tracer) +{ +enum tracefs_tracers t; +int retval; + + t =TRACEFS_TRACER_CUSTOM ; + +debug_msg ("enabling %s tracer\n" , tracer); + + retval =tracefs_tracer_set (inst, t, tracer); +if (retval <0 ) { +if (errno ==ENODEV ) +err_msg ("tracer %s not found!\n" , tracer); + +err_msg ("failed to enable the tracer %s\n" , tracer); +return -1 ; + } + +return 0 ; +} + +/* +* disable_tracer - set nop tracer to the insta +*/ +void disable_tracer (struct tracefs_instance *inst) +{ +enum tracefs_tracers t =TRACEFS_TRACER_NOP ; +int retval; + + retval =tracefs_tracer_set (inst, t); +if (retval <0 ) +err_msg ("oops, error disabling tracer\n" ); +} + +/* +* create_instance - create a trace instance with *instance_name +*/ +struct tracefs_instance *create_instance (char *instance_name) +{ +return tracefs_instance_create (instance_name); +} + +/* +* destroy_instance - remove a trace instance and free the data +*/ +void destroy_instance (struct tracefs_instance *inst) +{ +tracefs_instance_destroy (inst); +tracefs_instance_free (inst); +} + +/* +* save_trace_to_file - save the trace output of the instance to the file +*/ +int save_trace_to_file (struct tracefs_instance *inst,const char *filename) +{ +const char *file ="trace" ; +mode_t mode =0644 ; +char buffer[4096 ]; +int out_fd, in_fd; +int retval =-1 ; + + in_fd =tracefs_instance_file_open (inst, file,O_RDONLY ); +if (in_fd <0 ) { +err_msg ("Failed to open trace file\n" ); +return -1 ; + } + + out_fd =creat (filename, mode); +if (out_fd <0 ) { +err_msg ("Failed to create output file %s\n" , filename); + goto out_close_in; + } + +do { + retval =read (in_fd, buffer,sizeof (buffer)); +if (retval <=0 ) + goto out_close; + + retval =write (out_fd, buffer, retval); +if (retval <0 ) + goto out_close; + }while (retval >0 ); + + retval =0 ; +out_close: +close (out_fd); +out_close_in: +close (in_fd); +return retval; +} + +/* +* collect_registered_events - call the existing callback function for the event +* +* If an event has a registered callback function, call it. +* Otherwise, ignore the event. +*/ +int +collect_registered_events (struct tep_event *event,struct tep_record *record, +int cpu,void *context) +{ +struct trace_instance *trace = context; +struct trace_seq *s = trace->seq; + +if (!event->handler) +return 0 ; + + event->handler (s, record, event, context); + +return 0 ; +} + +/* +* trace_instance_destroy - destroy and free a rtla trace instance +*/ +void trace_instance_destroy (struct trace_instance *trace) +{ +if (trace->inst) { +disable_tracer (trace->inst); +destroy_instance (trace->inst); + } + +if (trace->seq) +free (trace->seq); + +if (trace->tep) +tep_free (trace->tep); +} + +/* +* trace_instance_init - create an rtla trace instance +* +* It is more than the tracefs instance, as it contains other +* things required for the tracing, such as the local events and +* a seq file. +* +* Note that the trace instance is returned disabled. This allows +* the tool to apply some other configs, like setting priority +* to the kernel threads, before starting generating trace entries. +*/ +int trace_instance_init (struct trace_instance *trace,char *tool_name) +{ + trace->seq =calloc (1 ,sizeof (*trace->seq)); +if (!trace->seq) + goto out_err; + +trace_seq_init (trace->seq); + + trace->inst =create_instance (tool_name); +if (!trace->inst) + goto out_err; + + trace->tep =tracefs_local_events (NULL ); +if (!trace->tep) + goto out_err; + +/* +* Let the main enable the record after setting some other +* things such as the priority of the tracer's threads. +*/ +tracefs_trace_off (trace->inst); + +return 0 ; + +out_err: +trace_instance_destroy (trace); +return 1 ; +} + +/* +* trace_instance_start - start tracing a given rtla instance +*/ +int trace_instance_start (struct trace_instance *trace) +{ +return tracefs_trace_on (trace->inst); +}
tools/tracing/rtla/src/trace.h new file +27 -0
@@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <tracefs.h> +#include <stddef.h> + +struct trace_instance { +struct tracefs_instance *inst; +struct tep_handle *tep; +struct trace_seq *seq; +}; + +int trace_instance_init (struct trace_instance *trace,char *tool_name); +int trace_instance_start (struct trace_instance *trace); +void trace_instance_destroy (struct trace_instance *trace); + +struct trace_seq *get_trace_seq (void ); +int enable_tracer_by_name (struct tracefs_instance *inst,const char *tracer); +void disable_tracer (struct tracefs_instance *inst); + +int enable_osnoise (struct trace_instance *trace); +int enable_timerlat (struct trace_instance *trace); + +struct tracefs_instance *create_instance (char *instance_name); +void destroy_instance (struct tracefs_instance *inst); + +int save_trace_to_file (struct tracefs_instance *inst,const char *filename); +int collect_registered_events (struct tep_event *tep,struct tep_record *record, +int cpu,void *context);
tools/tracing/rtla/src/utils.c new file +433 -0
433 lines not on this page · raw message
tools/tracing/rtla/src/utils.h new file +56 -0
56 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 18/31 rtla: Add osnoise tool
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
The osnoise tool is the interface for the osnoise tracer. The osnoise tool will have multiple "modes" with different outputs. At this point, no mode is included.
The osnoise.c includes the osnoise_context abstraction. It serves to read-save-change-restore the default values from tracing/osnoise/ directory. When the context is deleted, the default values are restored.
It also includes some other helper functions for managing osnoise tracer sessions.
With these bits and pieces in place, we can start adding some functionality to rtla.
--- tools/tracing/rtla/Makefile | 2 + tools/tracing/rtla/src/osnoise.c | 1013 ++++++++++++++++++++++++++++++ tools/tracing/rtla/src/osnoise.h | 95 +++ tools/tracing/rtla/src/rtla.c | 10 + 4 files changed, 1120 insertions(+) create mode 100644 tools/tracing/rtla/src/osnoise.c create mode 100644 tools/tracing/rtla/src/osnoise.h
tools/tracing/rtla/Makefile +2 -0
@@ -60,6 +60,8 @@ install: $(INSTALL) -d -m 755 $(DESTDIR)$(BINDIR) $(INSTALL) rtla -m 755 $(DESTDIR)$(BINDIR) $(STRIP) $(DESTDIR)$(BINDIR)/rtla + @test ! -f $(DESTDIR)$(BINDIR)/osnoise || rm $(DESTDIR)$(BINDIR)/osnoise + ln -s $(DESTDIR)$(BINDIR)/rtla $(DESTDIR)$(BINDIR)/osnoise .PHONY: clean tarball clean:
tools/tracing/rtla/src/osnoise.c new file +1013 -0
1,013 lines not on this page · raw message
tools/tracing/rtla/src/osnoise.h new file +95 -0
95 lines not on this page · raw message
tools/tracing/rtla/src/rtla.c +10 -0
@@ -8,6 +8,8 @@#include <string.h> #include <stdio.h> +#include "osnoise.h" +/* * rtla_usage - print rtla usage */
@@ -22,6 +24,7 @@ static void rtla_usage(void)" usage: rtla COMMAND ..." ,"" ," commands:" , +" osnoise - gives information about the operating system noise (osnoise)" ,"" , NULL, };
@@ -39,7 +42,14 @@ static void rtla_usage(void)*/ int run_command (int argc,char **argv,int start_position) { +if (strcmp (argv[start_position],"osnoise" ) ==0 ) { +osnoise_main (argc-start_position, &argv[start_position]); + goto ran; + } +return 0 ; +ran: +return 1 ; }int main (int argc,char *argv[])
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 19/31 rtla/osnoise: Add osnoise top mode
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
The rtla osnoise tool is an interface for the osnoise tracer. The osnoise tracer dispatches a kernel thread per-cpu. These threads read the time in a loop while with preemption, softirqs and IRQs enabled, thus allowing all the sources of osnoise during its execution. The osnoise threads take note of the entry and exit point of any source of interferences, increasing a per-cpu interference counter. The osnoise tracer also saves an interference counter for each source of interference.
The rtla osnoise top mode displays information about the periodic summary from the osnoise tracer.
One example of rtla osnoise top output is:
[root@alien ~]# rtla osnoise top -c 0-3 -d 1m -q -r 900000 -P F:1
Operating System Noise
duration: 0 00:01:00 | time is in us
CPU Period Runtime Noise % CPU Aval Max Noise Max Single HW NMI IRQ Softirq Thread
0 #58 52200000 1031 99.99802 91 60 0 0 52285 0 101 1 #59 53100000 5 99.99999 5 5 0 9 53122 0 18 2 #59 53100000 7 99.99998 7 7 0 8 53115 0 18 3 #59 53100000 8274 99.98441 277 23 0 9 53778 0 660
"rtla osnoise top --help" works and provide information about the available options.
--- tools/tracing/rtla/src/osnoise_top.c | 577 +++++++++++++++++++++++++++ 1 file changed, 577 insertions(+) create mode 100644 tools/tracing/rtla/src/osnoise_top.c
tools/tracing/rtla/src/osnoise_top.c new file +577 -0
@@ -0,0 +1,577 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (C) 2021 Red Hat Inc, Daniel Bristot de Oliveira <[email protected]> +*/ + +#include <getopt.h> +#include <stdlib.h> +#include <string.h> +#include <signal.h> +#include <unistd.h> +#include <stdio.h> +#include <time.h> + +#include "osnoise.h" +#include "utils.h" + +/* +* osnoise top parameters +*/ +struct osnoise_top_params { +char *cpus; +char *monitored_cpus; +char *trace_output; +unsigned long long runtime; +unsigned long long period; +long long stop_us; +long long stop_total_us; +int sleep_time; +int duration; +int quiet; +int set_sched; +struct sched_attr sched_param; +}; + +struct osnoise_top_cpu { +unsigned long long sum_runtime; +unsigned long long sum_noise; +unsigned long long max_noise; +unsigned long long max_sample; + +unsigned long long hw_count; +unsigned long long nmi_count; +unsigned long long irq_count; +unsigned long long softirq_count; +unsigned long long thread_count; + +int sum_cycles; +}; + +struct osnoise_top_data { +struct osnoise_top_cpu *cpu_data; +int nr_cpus; +}; + +/* +* osnoise_free_top - free runtime data +*/ +static void +osnoise_free_top (struct osnoise_top_data *data) +{ +free (data->cpu_data); +free (data); +} + +/* +* osnoise_alloc_histogram - alloc runtime data +*/ +static struct osnoise_top_data *osnoise_alloc_top (int nr_cpus) +{ +struct osnoise_top_data *data; + + data =calloc (1 ,sizeof (*data)); +if (!data) +return NULL ; + + data->nr_cpus = nr_cpus; + +/* one set of histograms per CPU */ + data->cpu_data =calloc (1 ,sizeof (*data->cpu_data) * nr_cpus); +if (!data->cpu_data) + goto cleanup; + +return data; + +cleanup: +osnoise_free_top (data); +return NULL ; +} + +/* +* osnoise_top_handler - this is the handler for osnoise tracer events +*/ +static int +osnoise_top_handler (struct trace_seq *s,struct tep_record *record, +struct tep_event *event,void *context) +{ +struct trace_instance *trace = context; +struct osnoise_tool *tool; +unsigned long long val; +struct osnoise_top_cpu *cpu_data; +struct osnoise_top_data *data; +int cpu = record->cpu; + + tool = container_of(trace, struct osnoise_tool, trace); + + data = tool->data; + cpu_data = &data->cpu_data[cpu]; + + cpu_data->sum_cycles++; + +tep_get_field_val (s, event,"runtime" , record, &val,1 ); +update_sum (&cpu_data->sum_runtime, &val); + +tep_get_field_val (s, event,"noise" , record, &val,1 ); +update_max (&cpu_data->max_noise, &val); +update_sum (&cpu_data->sum_noise, &val); + +tep_get_field_val (s, event,"max_sample" , record, &val,1 ); +update_max (&cpu_data->max_sample, &val); + +tep_get_field_val (s, event,"hw_count" , record, &val,1 ); +update_sum (&cpu_data->hw_count, &val); + +tep_get_field_val (s, event,"nmi_count" , record, &val,1 ); +update_sum (&cpu_data->nmi_count, &val); + +tep_get_field_val (s, event,"irq_count" , record, &val,1 ); +update_sum (&cpu_data->irq_count, &val); + +tep_get_field_val (s, event,"softirq_count" , record, &val,1 ); +update_sum (&cpu_data->softirq_count, &val); + +tep_get_field_val (s, event,"thread_count" , record, &val,1 ); +update_sum (&cpu_data->thread_count, &val); + +return 0 ; +} + +/* +* osnoise_top_header - print the header of the tool output +*/ +static void osnoise_top_header (struct osnoise_tool *top) +{ +struct trace_seq *s = top->trace.seq; +char duration[26 ]; + +get_duration (top->start_time, duration,sizeof (duration)); + +trace_seq_printf (s,"\033[2;37;40m" ); +trace_seq_printf (s," Operating System Noise" ); +trace_seq_printf (s," " ); +trace_seq_printf (s," " ); +trace_seq_printf (s,"\033[0;0;0m" ); +trace_seq_printf (s,"\n" ); + +trace_seq_printf (s,"duration: %9s | time is in us\n" , duration); + +trace_seq_printf (s,"\033[2;30;47m" ); +trace_seq_printf (s,"CPU Period Runtime " ); +trace_seq_printf (s," Noise " ); +trace_seq_printf (s," %% CPU Aval " ); +trace_seq_printf (s," Max Noise Max Single " ); +trace_seq_printf (s," HW NMI IRQ Softirq Thread" ); +trace_seq_printf (s,"\033[0;0;0m" ); +trace_seq_printf (s,"\n" ); +} + +/* +* clear_terminal - clears the output terminal +*/ +static void clear_terminal (struct trace_seq *seq) +{ +if (!config_debug) +trace_seq_printf (seq,"\033c" ); +} + +/* +* osnoise_top_print - prints the output of a given CPU +*/ +static void osnoise_top_print (struct osnoise_tool *tool,int cpu) +{ +struct trace_seq *s = tool->trace.seq; +struct osnoise_top_cpu *cpu_data; +struct osnoise_top_data *data; +int percentage; +int decimal; + + data = tool->data; + cpu_data = &data->cpu_data[cpu]; + +if (!cpu_data->sum_runtime) +return ; + + percentage = ((cpu_data->sum_runtime - cpu_data->sum_noise) *10000000 ) + / cpu_data->sum_runtime; + decimal = percentage %100000 ; + percentage = percentage /100000 ; + +trace_seq_printf (s,"%3d #%-6d %12llu " , cpu, cpu_data->sum_cycles, cpu_data->sum_runtime); +trace_seq_printf (s,"%12llu " , cpu_data->sum_noise); +trace_seq_printf (s," %3d.%05d" , percentage, decimal); +trace_seq_printf (s,"%12llu %12llu" , cpu_data->max_noise, cpu_data->max_sample); + +trace_seq_printf (s,"%12llu " , cpu_data->hw_count); +trace_seq_printf (s,"%12llu " , cpu_data->nmi_count); +trace_seq_printf (s,"%12llu " , cpu_data->irq_count); +trace_seq_printf (s,"%12llu " , cpu_data->softirq_count); +trace_seq_printf (s,"%12llu\n" , cpu_data->thread_count); +} + +/* +* osnoise_print_stats - print data for all cpus +*/ +static void +osnoise_print_stats (struct osnoise_top_params *params,struct osnoise_tool *top) +{ +struct trace_instance *trace = &top->trace; +static int nr_cpus =-1 ; +int i; + +if (nr_cpus ==-1 ) + nr_cpus =sysconf (_SC_NPROCESSORS_CONF); + +if (!params->quiet) +clear_terminal (trace->seq); + +osnoise_top_header (top); + +for (i =0 ; i < nr_cpus; i++) { +if (params->cpus && !params->monitored_cpus[i]) +continue ; +osnoise_top_print (top, i); + } + +trace_seq_do_printf (trace->seq); +trace_seq_reset (trace->seq); +} + +/* +* osnoise_top_usage - prints osnoise top usage message +*/ +void osnoise_top_usage (char *usage) +{ +int i; + +static const char *const msg[] = { +" usage: rtla osnoise [top] [-h] [-q] [-p us] [-r us] [-s us] [-S us] [-T[=file]] \\" , +" [-c cpu-list] -P priority" , +"" , +" -h/--help: print this menu" , +" -p/--period us: osnoise period in us" , +" -r/--runtime us: osnoise runtime in us" , +" -s/--stop us: stop trace if a single sample is higher than the argument in us" , +" -S/--stop-total us: stop trace if the total sample is higher than the argument in us" , +" -c/--cpus cpu-list: list of cpus to run osnoise threads" , +" -d/--duration time[s|m|h|d]: duration of the session" , +" -t/--trace[=file]: save the stopped trace to [file|osnoise_trace.txt]" , +" -q/--quiet print only a summary at the end" , +" -P/--priority o:prio|r:prio|f:prio|d:runtime:period : set scheduling parameters" , +" o:prio - use SCHED_OTHER with prio" , +" r:prio - use SCHED_RR with prio" , +" f:prio - use SCHED_FIFO with prio" , +" d:runtime[us|ms|s]:period[us|ms|s] - use SCHED_DEADLINE with runtime and period" , +" in nanoseconds" , +NULL , + }; + +if (usage) +fprintf (stderr,"%s\n" , usage); + +fprintf (stderr,"rtla osnoise top: a per-cpu summary of the OS noise (version %s)\n" , +VERSION ); + +for (i =0 ; msg[i]; i++) +fprintf (stderr,"%s\n" , msg[i]); +exit (1 ); +} + +/* +* osnoise_top_parse_args - allocs, parse and fill the cmd line parameters +*/ +struct osnoise_top_params *osnoise_top_parse_args (int argc,char **argv) +{ +struct osnoise_top_params *params; +int retval; +int c; + + params =calloc (1 ,sizeof (*params)); +if (!params) +exit (1 ); + +while (1 ) { +static struct option long_options[] = { + {"cpus" , required_argument,0 ,'c' }, + {"debug" , no_argument,0 ,'D' }, + {"duration" , required_argument,0 ,'d' }, + {"help" , no_argument,0 ,'h' }, + {"period" , required_argument,0 ,'p' }, + {"priority" , required_argument,0 ,'P' }, + {"quiet" , no_argument,0 ,'q' }, + {"runtime" , required_argument,0 ,'r' }, + {"stop" , required_argument,0 ,'s' }, + {"stop-total" , required_argument,0 ,'S' }, + {"trace" , optional_argument,0 ,'t' }, + {0 ,0 ,0 ,0 } + }; + +/* getopt_long stores the option index here. */ +int option_index =0 ; + + c =getopt_long (argc, argv,"c:d:Dhp:P:qr:s:S:t::" , + long_options, &option_index); + +/* Detect the end of the options. */ +if (c ==-1 ) +break ; + +switch (c) { +case 'c' : + retval =parse_cpu_list (optarg, ¶ms->monitored_cpus); +if (retval) +osnoise_top_usage ("\nInvalid -c cpu list\n" ); + params->cpus = optarg; +debug_msg ("-c for %s\n" , params->cpus); +break ; +case 'D' : + config_debug =1 ; +break ; +case 'd' : + params->duration =parse_seconds_duration (optarg); +if (!params->duration) +osnoise_top_usage ("Invalid -D duration\n" ); +break ; +case 'h' : +case '?' : +osnoise_top_usage (NULL ); +break ; +case 'p' : + params->period =get_llong_from_str (optarg); +if (params->period >10000000 ) +osnoise_top_usage ("Period longer than 10 s\n" ); +break ; +case 'P' : + retval =parse_prio (optarg, ¶ms->sched_param); +if (retval ==-1 ) +osnoise_top_usage ("Invalid -P priority" ); + params->set_sched =1 ; +break ; +case 'q' : + params->quiet =1 ; +break ; +case 'r' : + params->runtime =get_llong_from_str (optarg); +if (params->runtime <100 ) +osnoise_top_usage ("Runtime shorter than 100 us\n" ); +break ; +case 's' : + params->stop_us =get_llong_from_str (optarg); +break ; +case 'S' : + params->stop_total_us =get_llong_from_str (optarg); +break ; +case 't' : +if (optarg) +/* skip = */ + params->trace_output = &optarg[1 ]; +else + params->trace_output ="osnoise_trace.txt" ; +break ; +default : +osnoise_top_usage ("Invalid option" ); + } + } + +if (geteuid ()) { +err_msg ("osnoise needs root permission\n" ); +exit (EXIT_FAILURE ); + } + +return params; +} + +/* +* osnoise_top_apply_config - apply the top configs to the initialized tool +*/ +static int +osnoise_top_apply_config (struct osnoise_tool *tool,struct osnoise_top_params *params) +{ +int retval; + +if (!params->sleep_time) + params->sleep_time =1 ; + +if (params->cpus) { + retval =osnoise_set_cpus (tool->context, params->cpus); +if (retval) { +err_msg ("Failed to apply CPUs config\n" ); + goto out_err; + } + } + +if (params->runtime || params->period) { + retval =osnoise_set_runtime_period (tool->context, + params->runtime, + params->period); +if (retval) { +err_msg ("Failed to set runtime and/or period\n" ); + goto out_err; + } + } + +if (params->stop_us) { + retval =osnoise_set_stop_us (tool->context, params->stop_us); +if (retval) { +err_msg ("Failed to set stop us\n" ); + goto out_err; + } + } + +if (params->stop_total_us) { + retval =osnoise_set_stop_total_us (tool->context, params->stop_total_us); +if (retval) { +err_msg ("Failed to set stop total us\n" ); + goto out_err; + } + } + +return 0 ; + +out_err: +return -1 ; +} + +/* +* osnoise_init_top - initialize a osnoise top tool with parameters +*/ +struct osnoise_tool *osnoise_init_top (struct osnoise_top_params *params) +{ +struct osnoise_tool *tool; +int nr_cpus; + + nr_cpus =sysconf (_SC_NPROCESSORS_CONF); + + tool =osnoise_init_tool ("osnoise_top" ); +if (!tool) +return NULL ; + + tool->data =osnoise_alloc_top (nr_cpus); +if (!tool->data) + goto out_err; + + tool->params = params; + +tep_register_event_handler (tool->trace.tep,-1 ,"ftrace" ,"osnoise" , + osnoise_top_handler,NULL ); + +return tool; + +out_err: +osnoise_free_top (tool->data); +osnoise_destroy_tool (tool); +return NULL ; +} + +static int stop_tracing; +static void stop_top (int sig) +{ + stop_tracing =1 ; +} + +/* +* osnoise_top_set_signals - handles the signal to stop the tool +*/ +static void osnoise_top_set_signals (struct osnoise_top_params *params) +{ +signal (SIGINT , stop_top); +if (params->duration) { +signal (SIGALRM , stop_top); +alarm (params->duration); + } +} + +int osnoise_top_main (int argc,char **argv) +{ +struct osnoise_top_params *params; +struct trace_instance *trace; +struct osnoise_tool *record; +struct osnoise_tool *tool; +int return_value =1 ; +int retval; + + params =osnoise_top_parse_args (argc, argv); +if (!params) +exit (1 ); + + tool =osnoise_init_top (params); +if (!tool) { +err_msg ("Could not init osnoise top\n" ); + goto out_exit; + } + + retval =osnoise_top_apply_config (tool, params); +if (retval) { +err_msg ("Could not apply config\n" ); + goto out_top; + } + + trace = &tool->trace; + + retval =enable_osnoise (trace); +if (retval) { +err_msg ("Failed to enable osnoise tracer\n" ); + goto out_top; + } + +if (params->set_sched) { + retval =set_comm_sched_attr ("osnoise/" , ¶ms->sched_param); +if (retval) +osnoise_top_usage ("Failed to set sched parameters\n" ); + } + +trace_instance_start (trace); + +if (params->trace_output) { + record =osnoise_init_trace_tool ("osnoise" ); +if (!record) { +err_msg ("Failed to enable the trace instance\n" ); + goto out_top; + } +trace_instance_start (&record->trace); + } + + tool->start_time =time (NULL ); +osnoise_top_set_signals (params); + +do { +sleep (params->sleep_time); + + retval =tracefs_iterate_raw_events (trace->tep, + trace->inst, +NULL , +0 , + collect_registered_events, + trace); +if (retval <0 ) { +err_msg ("Error iterating on events\n" ); + goto out_top; + } + +if (!params->quiet) +osnoise_print_stats (params, tool); + +if (!tracefs_trace_is_on (trace->inst)) +break ; + + }while (!stop_tracing); + +osnoise_print_stats (params, tool); + + return_value =0 ; + +if (!tracefs_trace_is_on (trace->inst)) { +printf ("osnoise hit stop tracing\n" ); +if (params->trace_output) { +printf (" Saving trace to %s\n" , params->trace_output); +save_trace_to_file (record->trace.inst, params->trace_output); + } + } + +out_top: +osnoise_free_top (tool->data); +osnoise_destroy_tool (tool); +if (params->trace_output) +osnoise_destroy_tool (record); +out_exit: +exit (return_value); +}
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 20/31 rtla/osnoise: Add the hist mode
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
The rtla osnoise hist tool collects all osnoise:sample_threshold occurrence in a histogram, displaying the results in a user-friendly way. The tool also allows many configurations of the osnoise tracer and the collection of the tracer output.
Here is one example of the rtla osnoise hist tool output:
---------- %< ----------
[root@f34 ~]# rtla osnoise hist --bucket-size 10 --entries 100 -c 0-8 -d 1M -r 9000 -P F:1 # RTLA osnoise histogram
# Time unit is microseconds (us)
# Duration: 0 00:01:00
Index CPU-000 CPU-001 CPU-002 CPU-003 CPU-004 CPU-005 CPU-006 CPU-007 CPU-008
0 430 434 352 455 440 463 467 436 484 10 88 88 92 141 120 100 126 166 100 20 19 7 12 22 8 8 13 13 16 30 6 0 2 0 1 2 2 1 0 50 0 0 0 0 0 0 1 0 0 over: 0 0 0 0 0 0 0 0 0 count: 543 529 458 618 569 573 609 616 600 min: 0 0 0 0 0 0 0 0 0 avg: 0 0 0 0 0 0 0 0 0
max: 30 20 30 20 30 30 50 30 20
---------- >% ----------
Running
- rtla osnoise hist --help
provides information about the available options.
--- tools/tracing/rtla/src/osnoise.c | 6 +- tools/tracing/rtla/src/osnoise.h | 1 + tools/tracing/rtla/src/osnoise_hist.c | 799 ++++++++++++++++++++++++++ 3 files changed, 805 insertions(+), 1 deletion(-) create mode 100644 tools/tracing/rtla/src/osnoise_hist.c
tools/tracing/rtla/src/osnoise.c +5 -1
@@ -970,7 +970,8 @@ static void osnoise_usage(void)" usage: [rtla] osnoise [MODE] ..." ,"" ," modes:" , -" top - prints the summary from osnoise tracer" , +" top - prints the summary from osnoise tracer" , +" hist - prints a histogram of osnoise samples" ,"" ,"if no MODE is given, the top mode is called, passing the arguments" , NULL,
@@ -1005,6 +1006,9 @@ int osnoise_main(int argc, char *argv[]) }else if (strcmp (argv[1 ],"top" ) ==0 ) {osnoise_top_main (argc-1 , &argv[1 ]);exit (0 ); + }else if (strcmp (argv[1 ],"hist" ) ==0 ) { +osnoise_hist_main (argc-1 , &argv[1 ]); +exit (0 ); } usage:
tools/tracing/rtla/src/osnoise.h +1 -0
@@ -91,5 +91,6 @@ void osnoise_destroy_tool(struct osnoise_tool *top);struct osnoise_tool *osnoise_init_tool (char *tool_name);struct osnoise_tool *osnoise_init_trace_tool (char *tracer); +int osnoise_hist_main (int argc,char *argv[]);int osnoise_top_main (int argc,char **argv);int osnoise_main (int argc,char **argv);
tools/tracing/rtla/src/osnoise_hist.c new file +799 -0
799 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 21/31 rtla: Add timerlat tool and timelart top mode
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
The rtla timerlat tool is an interface for the timerlat tracer. The timerlat tracer dispatches a kernel thread per-cpu. These threads set a periodic timer to wake themselves up and go back to sleep. After the wakeup, they collect and generate useful information for the debugging of operating system timer latency.
The timerlat tracer outputs information in two ways. It periodically prints the timer latency at the timer IRQ handler and the Thread handler. It also provides information for each noise via the osnoise tracepoints.
The rtla timerlat top mode displays a summary of the periodic output from the timerlat tracer.
Here is one example of the rtla timerlat tool output:
---------- %< ----------
[root@alien ~]# rtla timerlat top -c 0-3 -d 1m
Timer Latency
0 00:01:00 | IRQ Timer Latency (us) | Thread Timer Latency (us) CPU COUNT | cur min avg max | cur min avg max 0 #60001 | 0 0 0 3 | 1 1 1 6 1 #60001 | 0 0 0 3 | 2 1 1 5 2 #60001 | 0 0 1 6 | 1 1 2 7 3 #60001 | 0 0 0 7 | 1 1 1 11
---------- >% ----------
Running:
# rtla timerlat --help # rtla timerlat top --help provides information about the available options.
--- tools/tracing/rtla/Makefile | 2 + tools/tracing/rtla/src/rtla.c | 5 + tools/tracing/rtla/src/timerlat.c | 68 +++ tools/tracing/rtla/src/timerlat.h | 4 + tools/tracing/rtla/src/timerlat_top.c | 615 ++++++++++++++++++++++++++ 5 files changed, 694 insertions(+) create mode 100644 tools/tracing/rtla/src/timerlat.c create mode 100644 tools/tracing/rtla/src/timerlat.h create mode 100644 tools/tracing/rtla/src/timerlat_top.c
tools/tracing/rtla/Makefile +2 -0
@@ -62,6 +62,8 @@ install: $(STRIP) $(DESTDIR)$(BINDIR)/rtla @test ! -f $(DESTDIR)$(BINDIR)/osnoise || rm $(DESTDIR)$(BINDIR)/osnoise ln -s $(DESTDIR)$(BINDIR)/rtla $(DESTDIR)$(BINDIR)/osnoise + @test ! -f $(DESTDIR)$(BINDIR)/timerlat || rm $(DESTDIR)$(BINDIR)/timerlat + ln -s $(DESTDIR)$(BINDIR)/rtla $(DESTDIR)$(BINDIR)/timerlat .PHONY: clean tarball clean:
tools/tracing/rtla/src/rtla.c +5 -0
@@ -9,6 +9,7 @@#include <stdio.h> #include "osnoise.h" +#include "timerlat.h" /* * rtla_usage - print rtla usage
@@ -25,6 +26,7 @@ static void rtla_usage(void)"" ," commands:" ," osnoise - gives information about the operating system noise (osnoise)" , +" timerlat - measures the timer irq and thread latency" ,"" , NULL, };
@@ -45,6 +47,9 @@ int run_command(int argc, char **argv, int start_position)if (strcmp (argv[start_position],"osnoise" ) ==0 ) {osnoise_main (argc-start_position, &argv[start_position]); goto ran; + }else if (strcmp (argv[start_position],"timerlat" ) ==0 ) { +timerlat_main (argc-start_position, &argv[start_position]); + goto ran; }return 0 ;
tools/tracing/rtla/src/timerlat.c new file +68 -0
68 lines not on this page · raw message
tools/tracing/rtla/src/timerlat.h new file +4 -0
@@ -0,0 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 + +int timerlat_top_main (int argc,char *argv[]); +int timerlat_main (int argc,char *argv[]);
tools/tracing/rtla/src/timerlat_top.c new file +615 -0
615 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 22/31 rtla/timerlat: Add timerlat hist mode
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
The rtla hist hist mode displays a histogram of each tracer event occurrence, both for IRQ and timer latencies. The tool also allows many configurations of the timerlat tracer and the collection of the tracer output.
Here is one example of the rtla timerlat hist mode output:
---------- %< ----------
[root@alien ~]# rtla timerlat hist -c 0-3 -d 1M # RTLA timerlat histogram
# Time unit is microseconds (us)
# Duration: 0 00:01:00
Index IRQ-000 Thr-000 IRQ-001 Thr-001 IRQ-002 Thr-002 IRQ-003 Thr-003
0 58572 0 59373 0 58691 0 58895 0
1 1422 57021 628 57241 1310 56160 1102 56805
2 6 2931 0 2695 0 3567 4 3031 3 1 40 0 53 0 260 0 142 4 0 7 0 5 0 6 0 17 5 0 2 0 5 0 7 0 4 6 0 0 0 2 0 1 0 1 8 0 0 0 0 0 0 0 1 over: 0 0 0 0 0 0 0 0 count: 60001 60001 60001 60001 60001 60001 60001 60001 min: 0 1 0 1 0 1 0 1 avg: 0 1 0 1 0 1 0 1 max: 3 5 1 6 1 6 2 8
---------- >% ----------
Running
- rtla timerlat hist --help
provides information about the available options.
--- tools/tracing/rtla/src/timerlat.c | 6 +- tools/tracing/rtla/src/timerlat.h | 2 +- tools/tracing/rtla/src/timerlat_hist.c | 820 +++++++++++++++++++++++++ 3 files changed, 826 insertions(+), 2 deletions(-) create mode 100644 tools/tracing/rtla/src/timerlat_hist.c
tools/tracing/rtla/src/timerlat.c +5 -1
@@ -25,7 +25,8 @@ static void timerlat_usage(void)" usage: [rtla] timerlat [MODE] ..." ,"" ," modes:" , -" top - prints the summary from timerlat tracer" , +" top - prints the summary from timerlat tracer" , +" hist - prints a histogram of timer latencies" ,"" ,"if no MODE is given, the top mode is called, passing the arguments" , NULL,
@@ -60,6 +61,9 @@ int timerlat_main(int argc, char *argv[]) }else if (strcmp (argv[1 ],"top" ) ==0 ) {timerlat_top_main (argc-1 , &argv[1 ]);exit (0 ); + }else if (strcmp (argv[1 ],"hist" ) ==0 ) { +timerlat_hist_main (argc-1 , &argv[1 ]); +exit (0 ); } usage:
tools/tracing/rtla/src/timerlat.h +1 -1
@@ -1,4 +1,4 @@// SPDX-License-Identifier: GPL-2.0 - +int timerlat_hist_main (int argc,char *argv[]);int timerlat_top_main (int argc,char *argv[]);int timerlat_main (int argc,char *argv[]);
tools/tracing/rtla/src/timerlat_hist.c new file +820 -0
820 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 23/31 rtla: Add Documentation
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
Adds the basis for rtla documentation. This patch also includes the rtla(1) man page.
As suggested by Jonathan Corbet, we are placing these man pages at Documentation/tools/rtla, using rst format. It is not linked to the official documentation, though.
The Makefile is based on bpftool's Documentation one.
--- Documentation/tools/rtla/Makefile | 41 +++++++++++++++++ Documentation/tools/rtla/common_appendix.rst | 12 +++++ Documentation/tools/rtla/rtla.rst | 48 ++++++++++++++++++++ tools/tracing/rtla/Makefile | 30 ++++++++++-- 4 files changed, 127 insertions(+), 4 deletions(-) create mode 100644 Documentation/tools/rtla/Makefile create mode 100644 Documentation/tools/rtla/common_appendix.rst create mode 100644 Documentation/tools/rtla/rtla.rst
Documentation/tools/rtla/Makefile new file +41 -0
@@ -0,0 +1,41 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Based on bpftool's Documentation Makefile + +INSTALL ?= install +RM ?= rm -f +RMDIR ?= rmdir --ignore-fail-on-non-empty + +PREFIX ?= /usr/share +MANDIR ?= $(PREFIX)/man +MAN1DIR = $(MANDIR)/man1 + +MAN1_RST = $(wildcard rtla*.rst) + +_DOC_MAN1 = $(patsubst %.rst,%.1,$(MAN1_RST)) +DOC_MAN1 = $(addprefix $(OUTPUT),$(_DOC_MAN1)) + +RST2MAN_DEP := $(shell command -v rst2man 2>/dev/null) +RST2MAN_OPTS += --verbose + +$(OUTPUT)%.1: %.rst +ifndef RST2MAN_DEP + $(error "rst2man not found, but required to generate man pages") +endif + rst2man $(RST2MAN_OPTS) $< > $@ + +man1: $(DOC_MAN1) +man: man1 + +clean: + $(RM) $(DOC_MAN1) + +install: man + $(INSTALL) -d -m 755 $(DESTDIR)$(MAN1DIR) + $(INSTALL) -m 644 $(DOC_MAN1) $(DESTDIR)$(MAN1DIR) + +uninstall: + $(RM) $(addprefix $(DESTDIR)$(MAN1DIR)/,$(_DOC_MAN1)) + $(RMDIR) $(DESTDIR)$(MAN1DIR) + +.PHONY: man man1 clean install uninstall +.DEFAULT_GOAL := man
Documentation/tools/rtla/common_appendix.rst new file +12 -0
@@ -0,0 +1,12 @@ +REPORTING BUGS +============== +Report bugs to <[email protected]> + +LICENSE +======= +**rtla** is Free Software licensed under the GNU GPLv2 + +COPYING +======= +Copyright \(C) 2021 Red Hat, Inc. Free use of this software is granted under +the terms of the GNU Public License (GPL).
Documentation/tools/rtla/rtla.rst new file +48 -0
48 lines not on this page · raw message
tools/tracing/rtla/Makefile +26 -4
@@ -45,18 +45,28 @@ DATADIR := /usr/share DOCDIR := $(DATADIR)/doc MANDIR := $(DATADIR)/man LICDIR := $(DATADIR)/licenses +SRCTREE := $(if $(BUILD_SRC),$(BUILD_SRC),$(CURDIR)) + +# If running from the tarball, man pages are stored in the Documentation +# dir. If running from the kernel source, man pages are stored in +# Documentation/tools/rtla/. +ifneq ($(wildcard Documentation/.*),) +DOCSRC = Documentation/ +else +DOCSRC = $(SRCTREE)/../../../Documentation/tools/rtla/ +endif .PHONY: all all: rtla -rtla: $(OBJ) +rtla: $(OBJ) doc $(CC) -o rtla $(LDFLAGS) $(OBJ) $(LIBS) static: $(OBJ) $(CC) -o rtla-static $(LDFLAGS) --static $(OBJ) $(LIBS) -lpthread -ldl .PHONY: install -install: +install: doc_install $(INSTALL) -d -m 755 $(DESTDIR)$(BINDIR) $(INSTALL) rtla -m 755 $(DESTDIR)$(BINDIR) $(STRIP) $(DESTDIR)$(BINDIR)/rtla
@@ -66,15 +76,27 @@ install: ln -s $(DESTDIR)$(BINDIR)/rtla $(DESTDIR)$(BINDIR)/timerlat .PHONY: clean tarball -clean: +clean: doc_clean @test ! -f rtla || rm rtla @test ! -f rtla-static || rm rtla-static @test ! -f src/rtla.o || rm src/rtla.o @test ! -f $(TARBALL) || rm -f $(TARBALL) @rm -rf *~ $(OBJ) *.tar.$(CEXT) -tarball: clean +tarball: clean rm -rf $(NAME)-$(VERSION) && mkdir $(NAME)-$(VERSION) cp -r $(DIRS) $(FILES) $(NAME)-$(VERSION) + mkdir $(NAME)-$(VERSION)/Documentation/ + cp -rp $(SRCTREE)/../../../Documentation/tools/rtla/* $(NAME)-$(VERSION)/Documentation/ tar $(TAROPTS) --exclude='*~' $(NAME)-$(VERSION) rm -rf $(NAME)-$(VERSION) + +.PHONY: doc doc_clean doc_install +doc: + $(MAKE) -C $(DOCSRC) + +doc_clean: + $(MAKE) -C $(DOCSRC) clean + +doc_install: + $(MAKE) -C $(DOCSRC) install
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 24/31 rtla: Add rtla osnoise man page
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
Man page for rtla osnoise command.
--- .../tools/rtla/common_osnoise_description.rst | 8 +++ Documentation/tools/rtla/rtla-osnoise.rst | 59 +++++++++++++++++++ 2 files changed, 67 insertions(+) create mode 100644 Documentation/tools/rtla/common_osnoise_description.rst create mode 100644 Documentation/tools/rtla/rtla-osnoise.rst
Documentation/tools/rtla/common_osnoise_description.rst new file +8 -0
@@ -0,0 +1,8 @@ +The **rtla osnoise** tool is an interface for the *osnoise* tracer. The +*osnoise* tracer dispatches a kernel thread per-cpu. These threads read the +time in a loop while with preemption, softirq and IRQs enabled, thus +allowing all the sources of operating systme noise during its execution. +The *osnoise*'s tracer threads take note of the delta between each time +read, along with an interference counter of all sources of interference. +At the end of each period, the *osnoise* tracer displays a summary of +the results.
Documentation/tools/rtla/rtla-osnoise.rst new file +59 -0
59 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 25/31 rtla: Add rtla osnoise top documentation
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
Man page for rtla osnoise top mode.
--- Documentation/tools/rtla/common_options.rst | 24 ++++++++ .../tools/rtla/common_osnoise_options.rst | 17 ++++++ .../tools/rtla/common_top_options.rst | 3 + Documentation/tools/rtla/rtla-osnoise-top.rst | 61 +++++++++++++++++++ 4 files changed, 105 insertions(+) create mode 100644 Documentation/tools/rtla/common_options.rst create mode 100644 Documentation/tools/rtla/common_osnoise_options.rst create mode 100644 Documentation/tools/rtla/common_top_options.rst create mode 100644 Documentation/tools/rtla/rtla-osnoise-top.rst
Documentation/tools/rtla/common_options.rst new file +24 -0
@@ -0,0 +1,24 @@ +**-c**, **--cpus** *cpu-list* + + Set the osnoise tracer to run the sample threads in the cpu-list. + +**-d**, **--duration** *time[s|m|h|d]* + + Set the duration of the session. + +**-t**, **--trace**\[*=file*] + + Save the stopped trace to [*file|osnoise_trace.txt*]. + +**-P**, **--priority** *o:prio|r:prio|f:prio|d:runtime:period* + + Set scheduling parameters to the osnoise tracer threads, the format to set the priority are: + + - *o:prio* - use SCHED_OTHER with *prio*; + - *r:prio* - use SCHED_RR with *prio*; + - *f:prio* - use SCHED_FIFO with *prio*; + - *d:runtime[us|ms|s]:period[us|ms|s]* - use SCHED_DEADLINE with *runtime* and *period* in nanoseconds. + +**-h**, **--help** + + Print help menu.
Documentation/tools/rtla/common_osnoise_options.rst new file +17 -0
@@ -0,0 +1,17 @@ +**-p**, **--period** *us* + + Set the *osnoise* tracer period in microseconds. + +**-r**, **--runtime** *us* + + Set the *osnoise* tracer runtime in microseconds. + +**-s**, **--stop** *us* + + Stop the trace if a single sample is higher than the argument in microseconds. + If **-T** is set, it will also save the trace to the output. + +**-S**, **--stop-total** *us* + + Stop the trace if the total sample is higher than the argument in microseconds. + If **-T** is set, it will also save the trace to the output.
Documentation/tools/rtla/common_top_options.rst new file +3 -0
@@ -0,0 +1,3 @@ +**-q**, **--quiet** + + Print only a summary at the end of the session.
Documentation/tools/rtla/rtla-osnoise-top.rst new file +61 -0
61 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 26/31 rtla: Add rtla osnoise hist documentation
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
Man page for rtla osnoise hist mode.
--- .../tools/rtla/common_hist_options.rst | 23 +++++++ .../tools/rtla/rtla-osnoise-hist.rst | 66 +++++++++++++++++++ 2 files changed, 89 insertions(+) create mode 100644 Documentation/tools/rtla/common_hist_options.rst create mode 100644 Documentation/tools/rtla/rtla-osnoise-hist.rst
Documentation/tools/rtla/common_hist_options.rst new file +23 -0
@@ -0,0 +1,23 @@ +**-b**, **--bucket-size** *N* + + Set the histogram bucket size (default *1*). + +**-e**, **--entries** *N* + + Set the number of entries of the histogram (default 256). + +**--no-header** + + Do not print header. + +**--no-summary** + + Do not print summary. + +**--no-index** + + Do not print index. + +**--with-zeros** + + Print zero only entries.
Documentation/tools/rtla/rtla-osnoise-hist.rst new file +66 -0
66 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 27/31 rtla: Add rtla timerlat documentation
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
Man page for rtla timerlat tool.
--- .../rtla/common_timerlat_description.rst | 10 ++++ Documentation/tools/rtla/rtla-timerlat.rst | 57 +++++++++++++++++++ 2 files changed, 67 insertions(+) create mode 100644 Documentation/tools/rtla/common_timerlat_description.rst create mode 100644 Documentation/tools/rtla/rtla-timerlat.rst
Documentation/tools/rtla/common_timerlat_description.rst new file +10 -0
@@ -0,0 +1,10 @@ +The **rtla timerlat** tool is an interface for the *timerlat* tracer. The +*timerlat* tracer dispatches a kernel thread per-cpu. These threads +set a periodic timer to wake themselves up and go back to sleep. After +the wakeup, they collect and generate useful information for the +debugging of operating system timer latency. + +The *timerlat* tracer outputs information in two ways. It periodically +prints the timer latency at the timer *IRQ* handler and the *Thread* +handler. It also enable the trace of the most relevant information via +**osnoise:** tracepoints.
Documentation/tools/rtla/rtla-timerlat.rst new file +57 -0
57 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 28/31 rtla: Add rtla timerlat top documentation
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
Man page for rtla timerlat top mode.
--- .../tools/rtla/common_timerlat_options.rst | 16 ++ .../tools/rtla/rtla-timerlat-top.rst | 145 ++++++++++++++++++ 2 files changed, 161 insertions(+) create mode 100644 Documentation/tools/rtla/common_timerlat_options.rst create mode 100644 Documentation/tools/rtla/rtla-timerlat-top.rst
Documentation/tools/rtla/common_timerlat_options.rst new file +16 -0
@@ -0,0 +1,16 @@ +**-p**, **--period** *us* + + Set the *timerlat* tracer period in microseconds. + +**-i**, **--irq** *us* + + Stop trace if the *IRQ* latency is higher than the argument in us. + +**-T**, **--thread** *us* + + Stop trace if the *Thread* latency is higher than the argument in us. + +**-s**, **--stack** *us* + + Save the stack trace at the *IRQ* if a *Thread* latency is higher than the + argument in us.
Documentation/tools/rtla/rtla-timerlat-top.rst new file +145 -0
145 lines not on this page · raw message
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 29/31 rtla: Add rtla timerlat hist documentation
In reply to Steven RostedtFrom: Daniel Bristot de Oliveira <[email protected]>
Man page for rtla timerlat hist mode.
--- .../tools/rtla/rtla-timerlat-hist.rst | 106 ++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 Documentation/tools/rtla/rtla-timerlat-hist.rst
Documentation/tools/rtla/rtla-timerlat-hist.rst new file +106 -0
@@ -0,0 +1,106 @@ +===================== +rtla-timerlat-hist +===================== +------------------------------------------------ +Histograms of the operating system timer latency +------------------------------------------------ + +:Manual section: 1 + +SYNOPSIS +======== +**rtla timerlat hist** [*OPTIONS*] ... + +DESCRIPTION +=========== + +.. include:: common_timerlat_description.rst + +The **rtla timerlat hist** displays a histogram of each tracer event +occurrence. This tool uses the periodic information, and the +**osnoise:** tracepoints are enabled when using the **-T** option. + +OPTIONS +======= + +.. include:: common_timerlat_options.rst + +.. include:: common_hist_options.rst + +.. include:: common_options.rst + +EXAMPLE +======= +In the example below, **rtla timerlat hist** is set to run for *10* minutes, +in the cpus *0-4*, *skipping zero* only lines. Moreover, **rtla timerlat +hist** will change the priority of the *timelat* threads to run under +*SCHED_DEADLINE* priority, with a *10us* runtime every *1ms* period. The +*1ms* period is also passed to the *timerlat* tracer:: + + [root@alien ~]# timerlat hist -d 10m -c 0-4 -P d:100us:1ms -p 1ms + # RTLA timerlat histogram + # Time unit is microseconds (us) + # Duration: 0 00:10:00 + Index IRQ-000 Thr-000 IRQ-001 Thr-001 IRQ-002 Thr-002 IRQ-003 Thr-003 IRQ-004 Thr-004 + 0 276489 0 206089 0 466018 0 481102 0 205546 0 + 1 318327 35487 388149 30024 94531 48382 83082 71078 388026 55730 + 2 3282 122584 4019 126527 28231 109012 23311 89309 4568 98739 + 3 940 11815 837 9863 6209 16227 6895 17196 910 9780 + 4 444 17287 424 11574 2097 38443 2169 36736 462 13476 + 5 206 43291 255 25581 1223 101908 1304 101137 236 28913 + 6 132 101501 96 64584 635 213774 757 215471 99 73453 + 7 74 169347 65 124758 350 57466 441 53639 69 148573 + 8 53 85183 31 156751 229 9052 306 9026 39 139907 + 9 22 10387 12 42762 161 2554 225 2689 19 26192 + 10 13 1898 8 5770 114 1247 128 1405 13 3772 + 11 9 560 9 924 71 686 76 765 8 713 + 12 4 256 2 360 50 411 64 474 3 278 + 13 2 167 2 172 43 256 53 350 4 180 + 14 1 88 1 116 15 198 42 223 0 115 + 15 2 63 3 94 11 139 20 150 0 58 + 16 2 37 0 56 5 78 10 102 0 39 + 17 0 18 0 28 4 57 8 80 0 15 + 18 0 8 0 17 2 50 6 56 0 12 + 19 0 9 0 5 0 19 0 48 0 18 + 20 0 4 0 8 0 11 2 27 0 4 + 21 0 2 0 3 1 9 1 18 0 6 + 22 0 1 0 3 1 7 0 3 0 5 + 23 0 2 0 4 0 2 0 7 0 2 + 24 0 2 0 2 1 3 0 3 0 5 + 25 0 0 0 1 0 1 0 1 0 3 + 26 0 1 0 0 0 2 0 2 0 0 + 27 0 0 0 3 0 1 0 0 0 1 + 28 0 0 0 3 0 0 0 1 0 0 + 29 0 0 0 2 0 2 0 1 0 3 + 30 0 1 0 0 0 0 0 0 0 0 + 31 0 1 0 0 0 0 0 2 0 2 + 32 0 0 0 1 0 2 0 0 0 0 + 33 0 0 0 2 0 0 0 0 0 1 + 34 0 0 0 0 0 0 0 0 0 2 + 35 0 1 0 1 0 0 0 0 0 1 + 36 0 1 0 0 0 1 0 1 0 0 + 37 0 0 0 1 0 0 0 0 0 0 + 40 0 0 0 0 0 1 0 1 0 0 + 41 0 0 0 0 0 0 0 0 0 1 + 42 0 0 0 0 0 0 0 0 0 1 + 44 0 0 0 0 0 1 0 0 0 0 + 46 0 0 0 0 0 0 0 1 0 0 + 47 0 0 0 0 0 0 0 0 0 1 + 50 0 0 0 0 0 0 0 0 0 1 + 54 0 0 0 1 0 0 0 0 0 0 + 58 0 0 0 1 0 0 0 0 0 0 + over: 0 0 0 0 0 0 0 0 0 0 + count: 600002 600002 600002 600002 600002 600002 600002 600002 600002 600002 + min: 0 1 0 1 0 1 0 1 0 1 + avg: 0 5 0 5 0 4 0 4 0 5 + max: 16 36 15 58 24 44 21 46 13 50 + +SEE ALSO +======== +**rtla-timerlat**\(1), **rtla-timerlat-top**\(1) + +*timerlat* tracer documentation: <https://www.kernel.org/doc/html/latest/trace/timerlat-tracer.html> + +AUTHOR +====== +Written by Daniel Bristot de Oliveira <[email protected]>
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 30/31 tracing: Remove duplicate warnings when calling trace_create_file()
In reply to Steven RostedtFrom: Yuntao Wang <[email protected]>
Since the same warning message is already printed in the trace_create_file() function, there is no need to print it again.
--- kernel/trace/trace_events.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-)
kernel/trace/trace_events.c +3 -9
@@ -3461,10 +3461,8 @@ create_event_toplevel_files(struct dentry *parent, struct trace_array *tr) entry =trace_create_file ("enable" ,TRACE_MODE_WRITE , d_events, tr, &ftrace_tr_enable_fops); -if (!entry) { -pr_warn ("Could not create tracefs 'enable' entry\n" ); +if (!entry)return -ENOMEM ; - }/* There are not as crucial, just warn if they are not created */
@@ -3480,17 +3478,13 @@ create_event_toplevel_files(struct dentry *parent, struct trace_array *tr)pr_warn ("Could not create tracefs 'set_event_notrace_pid' entry\n" );/* ring buffer internal formats */ - entry =trace_create_file ("header_page" ,TRACE_MODE_READ , d_events, +trace_create_file ("header_page" ,TRACE_MODE_READ , d_events, ring_buffer_print_page_header, &ftrace_show_header_fops); -if (!entry) -pr_warn ("Could not create tracefs 'header_page' entry\n" ); - entry =trace_create_file ("header_event" ,TRACE_MODE_READ , d_events, +trace_create_file ("header_event" ,TRACE_MODE_READ , d_events, ring_buffer_print_entry_header, &ftrace_show_header_fops); -if (!entry) -pr_warn ("Could not create tracefs 'header_event' entry\n" ); tr->event_dir = d_events;
Signature · 1 line
2.33.0
Steven Rostedt
maintainerPATCH 31/31 tracing/osnoise: Properly unhook events if start_per_cpu_kthreads() fails
In reply to Steven RostedtFrom: Nikita Yushchenko <[email protected]>
If start_per_cpu_kthreads() called from osnoise_workload_start() returns error, event hooks are left in broken state: unhook_irq_events() called but unhook_thread_events() and unhook_softirq_events() not called, and trace_osnoise_callback_enabled flag not cleared.
On the next tracer enable, hooks get not installed due to trace_osnoise_callback_enabled flag.
And on the further tracer disable an attempt to remove non-installed hooks happened, hitting a WARN_ON_ONCE() in tracepoint_remove_func().
Fix the error path by adding the missing part of cleanup. While at this, introduce osnoise_unhook_events() to avoid code duplication between this error path and normal tracer disable.
--- kernel/trace/trace_osnoise.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-)
kernel/trace/trace_osnoise.c +16 -4
@@ -2122,6 +2122,13 @@ static int osnoise_hook_events(void)return -EINVAL ; } +static void osnoise_unhook_events (void ) +{ +unhook_thread_events (); +unhook_softirq_events (); +unhook_irq_events (); +} +/* * osnoise_workload_start - start the workload and hook to events */
@@ -2154,7 +2161,14 @@ static int osnoise_workload_start(void) retval =start_per_cpu_kthreads ();if (retval) { -unhook_irq_events (); + trace_osnoise_callback_enabled = false; +/* +* Make sure that ftrace_nmi_enter/exit() see +* trace_osnoise_callback_enabled as false before continuing. +*/ +barrier (); + +osnoise_unhook_events ();return retval; }
@@ -2185,9 +2199,7 @@ static void osnoise_workload_stop(void)stop_per_cpu_kthreads (); -unhook_irq_events (); -unhook_softirq_events (); -unhook_thread_events (); +osnoise_unhook_events (); }static void osnoise_tracer_start (struct trace_array *tr)
Signature · 1 line
2.33.0