tracing: Updates for 4.18
Subsystems (7)
Steven Rostedt
maintainercover tracing: Updates for 4.18
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git
for-next
Head SHA1: 086d713d2535b389de1893d2f3874e9cfb6dbdf3
Andi Kleen (1):
trace: Use -mcount-record for dynamic ftrace
Lee, Chun-Yi (1):
tracepoints: Fix the descriptions of tracepoint_probe_register{_prio}Steven Rostedt (VMware) (16):
tracing: Do not reference event data in post call triggers
tracing: Add __find_event_file() to find event files without restrictions
tracing: Have event_trace_init() called by trace_init_tracefs()
tracing: Add brackets in ftrace event dynamic arrays
tracing: Do not show filter file for ftrace internal events
tracing: Add trigger file for trace_markers tracefs/ftrace/print
tracing: Have zero size length in filter logic be full string
tracing: Prevent further users of zero size static arrays in trace events
tracing: Allow histogram triggers to access ftrace internal events
tracing: Document trace_marker triggers
ftrace/selftest: Have the reset_trigger code be a bit more careful ftrace/selftest: Fix reset_trigger() to handle triggers with filters tracing/selftest: Add selftests to test trace_marker histogram triggers tracing/selftest: Add test to test hist trigger between kernel event and trace_marker tracing/selftest: Add test to test simple snapshot trigger for trace_marker
ring-buffer: Fix a bunch of typos in comments
Vasyl Gomonovych (1):
ring-buffer: Fix typo in comment
----
Documentation/trace/events.rst | 6 +- Documentation/trace/ftrace.rst | 5 + Documentation/trace/histogram.txt | 545 ++++++++++++++++++++- include/linux/ring_buffer.h | 2 +- include/linux/trace_events.h | 3 +- include/trace/trace_events.h | 1 + kernel/trace/ring_buffer.c | 20 +- kernel/trace/trace.c | 19 + kernel/trace/trace.h | 9 +- kernel/trace/trace_entries.h | 6 +- kernel/trace/trace_events.c | 36 +- kernel/trace/trace_events_filter.c | 23 +- kernel/trace/trace_events_hist.c | 2 +- kernel/trace/trace_events_trigger.c | 6 +- kernel/trace/trace_export.c | 9 +- kernel/tracepoint.c | 3 +- scripts/Makefile.build | 6 + tools/testing/selftests/ftrace/test.d/functions | 23 +- .../test.d/trigger/trigger-trace-marker-hist.tc | 49 ++ .../trigger/trigger-trace-marker-snapshot.tc | 74 +++ .../trigger-trace-marker-synthetic-kernel.tc | 68 +++ .../trigger/trigger-trace-marker-synthetic.tc | 66 +++ 22 files changed, 927 insertions(+), 54 deletions(-) create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-hist.tc create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-snapshot.tc create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-synthetic-kernel.tc create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-synthetic.tc
Steven Rostedt
maintainerPATCH 1/19 trace: Use -mcount-record for dynamic ftrace
From: Andi Kleen <[email protected]>
gcc 5 supports a new -mcount-record option to generate ftrace tables directly. This avoids the need to run record_mcount manually.
Use this option when available.
So far doesn't use -mcount-nop, which also exists now.
This is needed to make ftrace work with LTO because the normal record-mcount script doesn't run over the link time output.
It should also improve build times slightly in the general case.
--- scripts/Makefile.build | 6 ++++++ 1 file changed, 6 insertions(+)
scripts/Makefile.build +6 -0
@@ -207,6 +207,11 @@ cmd_modversions_c = \ endif ifdef CONFIG_FTRACE_MCOUNT_RECORD +# gcc 5 supports generating the mcount tables directly +ifneq ($(call cc-option,-mrecord-mcount,y),y) +KBUILD_CFLAGS += -mrecord-mcount +else +# else do it all manually ifdef BUILD_C_RECORDMCOUNT ifeq ("$(origin RECORDMCOUNT_WARN)", "command line") RECORDMCOUNT_FLAGS = -w
@@ -259,6 +264,7 @@ ifneq ($(RETPOLINE_CFLAGS),) objtool_args += --retpoline endif endif +endif ifdef CONFIG_MODVERSIONS
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 2/19 tracepoints: Fix the descriptions of tracepoint_probe_register{_prio}
In reply to Steven RostedtFrom: "Lee, Chun-Yi" <[email protected]>
The description of tracepoint_probe_register duplicates with tracepoint_probe_register_prio. This patch cleans up the descriptions.
--- kernel/tracepoint.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
kernel/tracepoint.c +1 -2
@@ -257,7 +257,7 @@ static int tracepoint_remove_func(struct tracepoint *tp, }/** -* tracepoint_probe_register - Connect a probe to a tracepoint +* tracepoint_probe_register_prio - Connect a probe to a tracepoint with priority * @tp: tracepoint * @probe: probe handler * @data: tracepoint data
@@ -290,7 +290,6 @@ EXPORT_SYMBOL_GPL(tracepoint_probe_register_prio);* @tp: tracepoint * @probe: probe handler * @data: tracepoint data -* @prio: priority of this function over other registered functions * * Returns 0 if ok, error value on error. * Note: if @tp is within a module, the caller is responsible for
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 3/19 tracing: Do not reference event data in post call triggers
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Trace event triggers can be called before or after the event has been committed. If it has been called after the commit, there's a possibility that the event no longer exists. Currently, the two post callers is the trigger to disable tracing (traceoff) and the one that will record a stack dump (stacktrace). Neither of them reference the trace event entry record, as that would lead to a race condition that could pass in corrupted data.
To prevent any other users of the post data triggers from using the trace event record, pass in NULL to the post call trigger functions for the event record, as they should never need to use them in the first place.
This does not fix any bug, but prevents bugs from happening by new post call trigger users.
--- include/linux/trace_events.h | 3 +-- kernel/trace/trace.h | 4 ++-- kernel/trace/trace_events_trigger.c | 6 ++---- 3 files changed, 5 insertions(+), 8 deletions(-)
include/linux/trace_events.h +1 -2
@@ -435,8 +435,7 @@ event_triggers_call(struct trace_event_file *file, void *rec, struct ring_buffer_event *event);extern void event_triggers_post_call (struct trace_event_file *file, -enum event_trigger_type tt, -void *rec,struct ring_buffer_event *event); +enum event_trigger_type tt);bool trace_event_ignore_this_pid (struct trace_event_file *trace_file);
kernel/trace/trace.h +2 -2
@@ -1334,7 +1334,7 @@ event_trigger_unlock_commit(struct trace_event_file *file,trace_buffer_unlock_commit (file->tr, buffer, event, irq_flags, pc);if (tt) -event_triggers_post_call (file, tt, entry, event); +event_triggers_post_call (file, tt); }/**
@@ -1367,7 +1367,7 @@ event_trigger_unlock_commit_regs(struct trace_event_file *file, irq_flags, pc, regs);if (tt) -event_triggers_post_call (file, tt, entry, event); +event_triggers_post_call (file, tt); }#define FILTER_PRED_INVALID ((unsigned short)-1)
kernel/trace/trace_events_trigger.c +2 -4
@@ -97,7 +97,6 @@ EXPORT_SYMBOL_GPL(event_triggers_call);* event_triggers_post_call - Call 'post_triggers' for a trace event * @file: The trace_event_file associated with the event * @tt: enum event_trigger_type containing a set bit for each trigger to invoke -* @rec: The trace entry for the event * * For each trigger associated with an event, invoke the trigger * function registered with the associated trigger command, if the
@@ -108,8 +107,7 @@ EXPORT_SYMBOL_GPL(event_triggers_call);*/ void event_triggers_post_call (struct trace_event_file *file, -enum event_trigger_type tt, -void *rec,struct ring_buffer_event *event) +enum event_trigger_type tt) {struct event_trigger_data *data;
@@ -117,7 +115,7 @@ event_triggers_post_call(struct trace_event_file *file,if (data->paused)continue ;if (data->cmd_ops->trigger_type & tt) - data->ops->func (data, rec, event); + data->ops->func (data,NULL ,NULL ); } }EXPORT_SYMBOL_GPL (event_triggers_post_call);
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 4/19 tracing: Add __find_event_file() to find event files without restrictions
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
By adding the function __find_event_file() that can search for files without restrictions, such as if the event associated with the file has a reg function, or if it has the "ignore" flag set, the files that are associated to ftrace internal events (like trace_marker and function events) can be found and used.
find_event_file() still returns a "filtered" file, as most callers need a valid trace event file. One created by the trace_events.h macros and not one created for parsing ftrace specific events.
--- kernel/trace/trace.h | 3 +++ kernel/trace/trace_events.c | 22 +++++++++++++++++----- 2 files changed, 20 insertions(+), 5 deletions(-)
kernel/trace/trace.h +3 -0
@@ -1454,6 +1454,9 @@ extern void trace_event_enable_tgid_record(bool enable);extern int event_trace_add_tracer (struct dentry *parent,struct trace_array *tr);extern int event_trace_del_tracer (struct trace_array *tr); +extern struct trace_event_file *__find_event_file (struct trace_array *tr, +const char *system, +const char *event);extern struct trace_event_file *find_event_file (struct trace_array *tr,const char *system,const char *event);
kernel/trace/trace_events.c +17 -5
@@ -2473,8 +2473,9 @@ __trace_add_event_dirs(struct trace_array *tr) } } +/* Returns any file that matches the system and event */ struct trace_event_file * -find_event_file (struct trace_array *tr,const char *system,const char *event) +__find_event_file (struct trace_array *tr,const char *system,const char *event) {struct trace_event_file *file;struct trace_event_call *call;
@@ -2485,10 +2486,7 @@ find_event_file(struct trace_array *tr, const char *system, const char *event) call = file->event_call; name =trace_event_name (call); -if (!name || !call->class || !call->class->reg) -continue ; - -if (call->flags &TRACE_EVENT_FL_IGNORE_ENABLE ) +if (!name || !call->class)continue ; if (strcmp(event, name) == 0 &&
@@ -2498,6 +2496,20 @@ find_event_file(struct trace_array *tr, const char *system, const char *event)return NULL ; } +/* Returns valid trace event files that match system and event */ +struct trace_event_file * +find_event_file (struct trace_array *tr,const char *system,const char *event) +{ +struct trace_event_file *file; + + file =__find_event_file (tr, system, event); +if (!file || !file->event_call->class->reg || + file->event_call->flags &TRACE_EVENT_FL_IGNORE_ENABLE ) +return NULL ; + +return file; +} +#ifdef CONFIG_DYNAMIC_FTRACE /* Avoid typos */
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 5/19 tracing: Have event_trace_init() called by trace_init_tracefs()
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Instead of having both trace_init_tracefs() and event_trace_init() be called by fs_initcall() routines, have event_trace_init() called directly by trace_init_tracefs(). This will guarantee order of how the events are created with respect to the rest of the ftrace infrastructure. This is needed to be able to assoctiate event files with ftrace internal events, such as the trace_marker.
--- kernel/trace/trace.c | 2 ++ kernel/trace/trace.h | 1 + kernel/trace/trace_events.c | 4 +--- 3 files changed, 4 insertions(+), 3 deletions(-)
kernel/trace/trace.c +2 -0
@@ -8111,6 +8111,8 @@ static __init int tracer_init_tracefs(void)if (IS_ERR (d_tracer))return 0 ; +event_trace_init (); +init_tracer_tracefs (&global_trace, d_tracer);ftrace_init_tracefs_toplevel (&global_trace, d_tracer);
kernel/trace/trace.h +1 -0
@@ -1451,6 +1451,7 @@ trace_find_event_field(struct trace_event_call *call, char *name);extern void trace_event_enable_cmd_record (bool enable);extern void trace_event_enable_tgid_record (bool enable); +extern int event_trace_init (void );extern int event_trace_add_tracer (struct dentry *parent,struct trace_array *tr);extern int event_trace_del_tracer (struct trace_array *tr);
kernel/trace/trace_events.c +1 -3
@@ -3144,7 +3144,7 @@ static __init int event_trace_enable_again(void)early_initcall (event_trace_enable_again); -static __init int event_trace_init (void ) +__init int event_trace_init (void ) {struct trace_array *tr;struct dentry *d_tracer;
@@ -3189,8 +3189,6 @@ void __init trace_event_init(void)event_trace_enable (); } -fs_initcall (event_trace_init); -#ifdef CONFIG_FTRACE_STARTUP_TEST static DEFINE_SPINLOCK (test_spinlock);
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 6/19 tracing: Add brackets in ftrace event dynamic arrays
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
The dynamic arrays defined for ftrace internal events, such as the buf field for trace_marker (ftrace/print) did not have brackets which makes the filter code not accept it as a string. This is not currently an issues because the filter code doesn't do anything for these events, but they will in the future, and this needs to be fixed for when it does.
--- kernel/trace/trace_export.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/trace/trace_export.c +1 -1
@@ -117,7 +117,7 @@ static void __always_unused ____ftrace_check_##name(void) \#undef __dynamic_array#define __dynamic_array (type, item) \ - ret = trace_define_field(event_call, #type, #item, \ + ret = trace_define_field(event_call, #type "[]", #item, \ offsetof(typeof(field), item), \ 0, is_signed_type(type), filter_type);\ if (ret) \
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 7/19 tracing: Do not show filter file for ftrace internal events
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
The filter file in the ftrace internal events, like in /sys/kernel/tracing/events/ftrace/function/filter is not attached to any functionality. Do not create them as they are meaningless.
In the future, if an ftrace internal event gets filter functionality, then it will need to create it directly.
--- kernel/trace/trace_events.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
kernel/trace/trace_events.c +6 -4
@@ -2007,16 +2007,18 @@ event_create_dir(struct dentry *parent, struct trace_event_file *file)return -1 ; } } -trace_create_file ("filter" ,0644 , file->dir, file, - &ftrace_event_filter_fops);/* * Only event directories that can be enabled should have -* triggers. +* triggers or filters. */ -if (!(call->flags &TRACE_EVENT_FL_IGNORE_ENABLE )) +if (!(call->flags &TRACE_EVENT_FL_IGNORE_ENABLE )) { +trace_create_file ("filter" ,0644 , file->dir, file, + &ftrace_event_filter_fops); +trace_create_file ("trigger" ,0644 , file->dir, file, &event_trigger_fops); + }#ifdef CONFIG_HIST_TRIGGERS trace_create_file("hist", 0444, file->dir, file,
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 8/19 tracing: Add trigger file for trace_markers tracefs/ftrace/print
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Allow writing to the trace_markers file initiate triggers defined in tracefs/ftrace/print/trigger file. This will allow of user space to trigger the same type of triggers (including histograms) that the trace events use.
Had to create a ftrace_event_register() function that will become the trace_marker print event's reg() function. This is required because of how triggers are enabled:
event_trigger_write() {
event_trigger_regex_write() {
trigger_process_regex() {
for p in trigger_commands {
p->func(); /* trigger_snapshot_cmd->func */
event_trigger_callback() {
cmd_ops->reg() /* register_trigger() */ {
trace_event_trigger_enable_disable() {
trace_event_enable_disable() {
call->class->reg();Without the reg() function, the trigger code will call a NULL pointer and crash the system.
--- kernel/trace/trace.c | 17 +++++++++++++++++ kernel/trace/trace.h | 1 + kernel/trace/trace_entries.h | 6 ++++-- kernel/trace/trace_export.c | 7 +++++++ 4 files changed, 29 insertions(+), 2 deletions(-)
kernel/trace/trace.c +17 -0
@@ -6074,6 +6074,7 @@ tracing_mark_write(struct file *filp, const char __user *ubuf, {struct trace_array *tr = filp->private_data;struct ring_buffer_event *event; +enum event_trigger_type tt =ETT_NONE ;struct ring_buffer *buffer;struct print_entry *entry;unsigned long irq_flags;
@@ -6122,6 +6123,12 @@ tracing_mark_write(struct file *filp, const char __user *ubuf, written = cnt; len = cnt; +if (tr->trace_marker_file && !list_empty (&tr->trace_marker_file->triggers)) { +/* do not add \n before testing triggers, but add \0 */ + entry->buf[cnt] ='\0' ; + tt =event_triggers_call (tr->trace_marker_file, entry, event); + } +if (entry->buf[cnt -1 ] !='\n' ) { entry->buf[cnt] ='\n' ; entry->buf[cnt +1 ] ='\0' ;
@@ -6130,6 +6137,9 @@ tracing_mark_write(struct file *filp, const char __user *ubuf,__buffer_unlock_commit (buffer, event); +if (tt) +event_triggers_post_call (tr->trace_marker_file, tt); +if (written >0 ) *fpos += written;
@@ -7896,6 +7906,7 @@ static __init void create_trace_instances(struct dentry *d_tracer)static void init_tracer_tracefs (struct trace_array *tr,struct dentry *d_tracer) { +struct trace_event_file *file;int cpu; trace_create_file("available_tracers", 0444, d_tracer,
@@ -7928,6 +7939,12 @@ init_tracer_tracefs(struct trace_array *tr, struct dentry *d_tracer)trace_create_file ("trace_marker" ,0220 , d_tracer, tr, &tracing_mark_fops); + file =__find_event_file (tr,"ftrace" ,"print" ); +if (file && file->dir) +trace_create_file ("trigger" ,0644 , file->dir, file, + &event_trigger_fops); + tr->trace_marker_file = file; +trace_create_file ("trace_marker_raw" ,0220 , d_tracer, tr, &tracing_mark_raw_fops);
kernel/trace/trace.h +1 -0
@@ -259,6 +259,7 @@ struct trace_array {struct trace_options *topts;struct list_head systems;struct list_head events; +struct trace_event_file *trace_marker_file;cpumask_var_t tracing_cpumask;/* only trace on set CPUs */ int ref;#ifdef CONFIG_FUNCTION_TRACER
kernel/trace/trace_entries.h +4 -2
@@ -230,7 +230,7 @@ FTRACE_ENTRY(bprint, bprint_entry,FILTER_OTHER ); -FTRACE_ENTRY(print, print_entry, +FTRACE_ENTRY_REG(print, print_entry, TRACE_PRINT,
@@ -242,7 +242,9 @@ FTRACE_ENTRY(print, print_entry,F_printk ("%ps: %s" , (void *)__entry->ip, __entry->buf), -FILTER_OTHER +FILTER_OTHER , + + ftrace_event_register ); FTRACE_ENTRY(raw_data, raw_data_entry,
kernel/trace/trace_export.c +7 -0
@@ -14,6 +14,13 @@#include "trace_output.h" +/* Stub function for events with triggers */ +static int ftrace_event_register (struct trace_event_call *call, +enum trace_reg type,void *data) +{ +return 0 ; +} +#undef TRACE_SYSTEM#define TRACE_SYSTEM ftrace
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 9/19 tracing: Have zero size length in filter logic be full string
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
As strings in trace events may not have a nul terminating character, the filter string compares use the defined string length for the field for the compares.
The trace_marker records data slightly different than do normal events. It's size is zero, meaning that the string is the rest of the array, and that the string also ends with '\0'.
If the size is zero, assume that the string is nul terminated and read the string in question as is.
--- kernel/trace/trace_events_filter.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-)
kernel/trace/trace_events_filter.c +12 -11
@@ -750,31 +750,32 @@ static int filter_pred_none(struct filter_pred *pred, void *event)* * Note: * - @str might not be NULL-terminated if it's of type DYN_STRING -* or STATIC_STRING +* or STATIC_STRING, unless @len is zero. */ static int regex_match_full (char *str,struct regex *r,int len) { -if (strncmp (str, r->pattern, len) ==0 ) -return 1 ; -return 0 ; +/* len of zero means str is dynamic and ends with '\0' */ +if (!len) +return strcmp (str, r->pattern) ==0 ; + +return strncmp (str, r->pattern, len) ==0 ; }static int regex_match_front (char *str,struct regex *r,int len) { -if (len < r->len) +if (len && len < r->len)return 0 ; -if (strncmp (str, r->pattern, r->len) ==0 ) -return 1 ; -return 0 ; +return strncmp (str, r->pattern, r->len) ==0 ; }static int regex_match_middle (char *str,struct regex *r,int len) { -if (strnstr (str, r->pattern, len)) -return 1 ; -return 0 ; +if (!len) +return strstr (str, r->pattern) !=NULL ; + +return strnstr (str, r->pattern, len) !=NULL ; }static int regex_match_end (char *str,struct regex *r,int len)
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 10/19 tracing: Prevent further users of zero size static arrays in trace events
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
A zero size static array has special meaning in the ftrace infrastructure. Trace events are for recording data in the trace buffers that is normally difficult to obtain via probes or function tracing. There is no reason for any trace event to declare a zero size static array.
If one does, BUILD_BUG_ON() will trigger and prevent the kernel from compiling.
--- include/trace/trace_events.h | 1 + 1 file changed, 1 insertion(+)
include/trace/trace_events.h +1 -0
@@ -422,6 +422,7 @@ static struct trace_event_functions trace_event_type_funcs_##call = { \ do { \ char *type_str = #type"["__stringify(len)"]"; \ BUILD_BUG_ON(len > MAX_FILTER_STR_VAL); \ + BUILD_BUG_ON(len <= 0); \ ret = trace_define_field(event_call, type_str, #item, \ offsetof(typeof(field), item), \ sizeof(field.item), \
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 11/19 tracing: Allow histogram triggers to access ftrace internal events
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Now that trace_marker can have triggers, including a histogram triggers, the onmatch() and onmax() access the trace event. To do so, the search routine to find the event file needs to use the raw __find_event_file() that does not filter out ftrace events.
--- kernel/trace/trace_events_hist.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/trace/trace_events_hist.c +1 -1
@@ -2865,7 +2865,7 @@ static struct trace_event_file *event_file(struct trace_array *tr, {struct trace_event_file *file; - file =find_event_file (tr, system, event_name); + file =__find_event_file (tr, system, event_name);if (!file)return ERR_PTR (-EINVAL );
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 12/19 tracing: Document trace_marker triggers
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Add documentation and an example on how to use trace_marker triggers.
--- Documentation/trace/events.rst | 6 +- Documentation/trace/ftrace.rst | 5 + Documentation/trace/histogram.txt | 545 +++++++++++++++++++++++++++++- 3 files changed, 554 insertions(+), 2 deletions(-)
Documentation/trace/events.rst +5 -1
@@ -338,10 +338,14 @@ used for conditionally invoking triggers. The syntax for event triggers is roughly based on the syntax for set_ftrace_filter 'ftrace filter commands' (see the 'Filter commands' -section of Documentation/trace/ftrace.txt), but there are major +section of Documentation/trace/ftrace.rst), but there are major differences and the implementation isn't currently tied to it in any way, so beware about making generalizations between the two. +Note: Writing into trace_marker (See Documentation/trace/ftrace.rst) + can also enable triggers that are written into + /sys/kernel/tracing/events/ftrace/print/trigger + 6.1 Expression syntax ---------------------
Documentation/trace/ftrace.rst +5 -0
@@ -507,6 +507,11 @@ of ftrace. Here is a list of some of the key files: trace_fd = open("trace_marker", WR_ONLY); + Note: Writing into the trace_marker file can also initiate triggers + that are written into /sys/kernel/tracing/events/ftrace/print/trigger + See "Event triggers" in Documentation/trace/events.rst and an + example in Documentation/trace/histogram.rst (Section 3.) + trace_marker_raw: This is similar to trace_marker above, but is meant for for binary data
Documentation/trace/histogram.txt +544 -1
554 lines not on this page · raw message
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 13/19 ftrace/selftest: Have the reset_trigger code be a bit more careful
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
The trigger code is picky in how it can be disabled as there may be dependencies between different events and synthetic events. Change the order on how triggers are reset.
1) Reset triggers of all synthetic events first
2) Remove triggers with actions attached to them
3) Remove all other triggers
If this order isn't followed, then some triggers will not be reset, and an error may happen because a trigger is busy.
--- .../testing/selftests/ftrace/test.d/functions | 21 ++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-)
tools/testing/selftests/ftrace/test.d/functions +18 -3
@@ -15,14 +15,29 @@ reset_tracer() { # reset the current tracer echo nop > current_tracer } -reset_trigger() { # reset all current setting triggers - grep -v ^# events/*/*/trigger | +reset_trigger_file() { + # remove action triggers first + grep -H ':on[^:]*(' $@ | + while read line; do + cmd=`echo $line | cut -f2- -d: | cut -f1 -d" "` + file=`echo $line | cut -f1 -d:` + echo "!$cmd" >> $file + done + grep -Hv ^# $@ | while read line; do cmd=`echo $line | cut -f2- -d: | cut -f1 -d" "` - echo "!$cmd" > `echo $line | cut -f1 -d:` + file=`echo $line | cut -f1 -d:` + echo "!$cmd" > $file done } +reset_trigger() { # reset all current setting triggers + if [ -d events/synthetic ]; then + reset_trigger_file events/synthetic/*/trigger + fi + reset_trigger_file events/*/*/trigger +} + reset_events_filter() { # reset all current setting filters grep -v ^none events/*/*/filter | while read line; do
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 14/19 ftrace/selftest: Fix reset_trigger() to handle triggers with filters
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
The reset_trigger() function breaks up the command by a space ' '. This is useful to ignore the '[active]' word for triggers when removing them. But if the trigger has a filter (ie. "if prio < 10") then the filter needs to be attached to the line that is written into the trigger file to remove it. But the truncation removes the filter and the triggers are not cleared properly.
Before, reset_trigger() did this:
# echo 'hist:keys=common_pid if prev_prio < 10' > events/sched/sched_switch/trigger # echo 'hist:keys=common_pid if next_prio < 10' >> events/sched/sched_switch/trigger # cat events/sched/sched_switch/trigger hist:keys=common_pid:vals=hitcount:sort=hitcount:size=2048 if prev_prio < 10 [active] hist:keys=common_pid:vals=hitcount:sort=hitcount:size=2048 if next_prio < 10 [active]
reset_trigger() {
echo '!hist:keys=common_pid:vals=hitcount:sort=hitcount:size=2048' >> events/sched/sched_switch/trigger
}
# cat events/sched/sched_switch/trigger hist:keys=common_pid:vals=hitcount:sort=hitcount:size=2048 if prev_prio < 10 [active] hist:keys=common_pid:vals=hitcount:sort=hitcount:size=2048 if next_prio < 10 [active]
After, where it includes the filter:
reset_trigger() {
echo '!hist:keys=common_pid:vals=hitcount:sort=hitcount:size=2048 if prev_prio < 10' >> events/sched/sched_switch/trigger
}
# cat events/sched/sched_switch/trigger hist:keys=common_pid:vals=hitcount:sort=hitcount:size=2048 if next_prio < 10 [active]
--- tools/testing/selftests/ftrace/test.d/functions | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/ftrace/test.d/functions +2 -2
@@ -19,13 +19,13 @@ reset_trigger_file() { # remove action triggers first grep -H ':on[^:]*(' $@ | while read line; do - cmd=`echo $line | cut -f2- -d: | cut -f1 -d" "` + cmd=`echo $line | cut -f2- -d: | cut -f1 -d"["` file=`echo $line | cut -f1 -d:` echo "!$cmd" >> $file done grep -Hv ^# $@ | while read line; do - cmd=`echo $line | cut -f2- -d: | cut -f1 -d" "` + cmd=`echo $line | cut -f2- -d: | cut -f1 -d"["` file=`echo $line | cut -f1 -d:` echo "!$cmd" > $file done
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 15/19 tracing/selftest: Add selftests to test trace_marker histogram triggers
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Add a couple of tests that test the trace_marker histogram triggers. One does a straight histogram test, the other will create a synthetic event and test the latency between two different writes (using filters to differentiate between them).
--- .../trigger/trigger-trace-marker-hist.tc | 49 ++++++++++++++ .../trigger/trigger-trace-marker-synthetic.tc | 66 +++++++++++++++++++ 2 files changed, 115 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-hist.tc create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-synthetic.tc
tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-hist.tc new file +49 -0
@@ -0,0 +1,49 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: trace_marker trigger - test histogram trigger +# flags: instance + +do_reset() { + reset_trigger + echo > set_event + clear_trace +} + +fail() { #msg + do_reset + echo $1 + exit_fail +} + +if [ ! -f set_event ]; then + echo "event tracing is not supported" + exit_unsupported +fi + +if [ ! -d events/ftrace/print ]; then + echo "event trace_marker is not supported" + exit_unsupported +fi + +if [ ! -f events/ftrace/print/trigger ]; then + echo "event trigger is not supported" + exit_unsupported +fi + +if [ ! -f events/ftrace/print/hist ]; then + echo "hist trigger is not supported" + exit_unsupported +fi + +do_reset + +echo "Test histogram trace_marker tigger" + +echo 'hist:keys=common_pid' > events/ftrace/print/trigger +for i in `seq 1 10` ; do echo "hello" > trace_marker; done +grep 'hitcount: *10$' events/ftrace/print/hist > /dev/null || \ + fail "hist trigger did not trigger correct times on trace_marker" + +do_reset + +exit 0
tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-synthetic.tc new file +66 -0
66 lines not on this page · raw message
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 16/19 tracing/selftest: Add test to test hist trigger between kernel event and trace_marker
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Add a test that tests a trigger that is initiated by a kernel event (sched_waking) and compared to a write to the trace_marker.
--- .../trigger-trace-marker-synthetic-kernel.tc | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-synthetic-kernel.tc
tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-synthetic-kernel.tc new file +68 -0
@@ -0,0 +1,68 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: trace_marker trigger - test histogram with synthetic event against kernel event +# flags: + +do_reset() { + reset_trigger + echo > set_event + echo > synthetic_events + clear_trace +} + +fail() { #msg + do_reset + echo $1 + exit_fail +} + +if [ ! -f set_event ]; then + echo "event tracing is not supported" + exit_unsupported +fi + +if [ ! -f synthetic_events ]; then + echo "synthetic events not supported" + exit_unsupported +fi + +if [ ! -d events/ftrace/print ]; then + echo "event trace_marker is not supported" + exit_unsupported +fi + +if [ ! -d events/sched/sched_waking ]; then + echo "event sched_waking is not supported" + exit_unsupported +fi + +if [ ! -f events/ftrace/print/trigger ]; then + echo "event trigger is not supported" + exit_unsupported +fi + +if [ ! -f events/ftrace/print/hist ]; then + echo "hist trigger is not supported" + exit_unsupported +fi + +do_reset + +echo "Test histogram kernel event to trace_marker latency histogram trigger" + +echo 'latency u64 lat' > synthetic_events +echo 'hist:keys=pid:ts0=common_timestamp.usecs' > events/sched/sched_waking/trigger +echo 'hist:keys=common_pid:lat=common_timestamp.usecs-$ts0:onmatch(sched.sched_waking).latency($lat)' > events/ftrace/print/trigger +echo 'hist:keys=common_pid,lat:sort=lat' > events/synthetic/latency/trigger +sleep 1 +echo "hello" > trace_marker + +grep 'hitcount: *1$' events/ftrace/print/hist > /dev/null || \ + fail "hist trigger did not trigger correct times on trace_marker" + +grep 'hitcount: *1$' events/synthetic/latency/hist > /dev/null || \ + fail "hist trigger did not trigger " + +do_reset + +exit 0
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 17/19 tracing/selftest: Add test to test simple snapshot trigger for trace_marker
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Several complex trigger tests were added for trace_marker, but not a simple one. This could be used to help diagnose a problem with the code by giving a reference between how complex a trigger is that fails.
--- .../trigger/trigger-trace-marker-snapshot.tc | 74 +++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-snapshot.tc
tools/testing/selftests/ftrace/test.d/trigger/trigger-trace-marker-snapshot.tc new file +74 -0
@@ -0,0 +1,74 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: trace_marker trigger - test snapshot trigger +# flags: instance + +do_reset() { + reset_trigger + echo > set_event + echo 0 > snapshot + clear_trace +} + +fail() { #msg + do_reset + echo $1 + exit_fail +} + +if [ ! -f set_event ]; then + echo "event tracing is not supported" + exit_unsupported +fi + +if [ ! -f snapshot ]; then + echo "snapshot is not supported" + exit_unsupported +fi + +if [ ! -d events/ftrace/print ]; then + echo "event trace_marker is not supported" + exit_unsupported +fi + +if [ ! -f events/ftrace/print/trigger ]; then + echo "event trigger is not supported" + exit_unsupported +fi + +test_trace() { + file=$1 + x=$2 + + cat $file | while read line; do + comment=`echo $line | sed -e 's/^#//'` + if [ "$line" != "$comment" ]; then + continue + fi + echo "testing $line for >$x<" + match=`echo $line | sed -e "s/>$x<//"` + if [ "$line" == "$match" ]; then + fail "$line does not have >$x< in it" + fi + let x=$x+2 + done +} + +do_reset + +echo "Test snapshot trace_marker tigger" + +echo 'snapshot' > events/ftrace/print/trigger + +# make sure the snapshot is allocated + +grep -q 'Snapshot is allocated' snapshot + +for i in `seq 1 10` ; do echo "hello >$i<" > trace_marker; done + +test_trace trace 1 +test_trace snapshot 2 + +do_reset + +exit 0
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 18/19 ring-buffer: Fix a bunch of typos in comments
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
An anonymous source sent me a bunch of typo fixes in the comments of ring_buffer.c file. That source did not want to be associated to this patch because they don't want to be known as "one of those" commiters (you know who you are!). They gave me permission to sign this off in my own name.
--- kernel/trace/ring_buffer.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
kernel/trace/ring_buffer.c +10 -10
@@ -809,7 +809,7 @@ EXPORT_SYMBOL_GPL(ring_buffer_normalize_time_stamp);* * You can see, it is legitimate for the previous pointer of * the head (or any page) not to point back to itself. But only -* temporarially. +* temporarily. */ #define RB_PAGE_NORMAL 0UL
@@ -906,7 +906,7 @@ static void rb_list_head_clear(struct list_head *list) }/* -* rb_head_page_dactivate - clears head page ptr (for free list) +* rb_head_page_deactivate - clears head page ptr (for free list) */ static void rb_head_page_deactivate (struct ring_buffer_per_cpu *cpu_buffer)
@@ -1780,7 +1780,7 @@ int ring_buffer_resize(struct ring_buffer *buffer, unsigned long size,put_online_cpus (); }else { -/* Make sure this CPU has been intitialized */ +/* Make sure this CPU has been initialized */ if (!cpumask_test_cpu (cpu_id, buffer->cpumask)) goto out;
@@ -2325,7 +2325,7 @@ rb_update_event(struct ring_buffer_per_cpu *cpu_buffer,/* * If we need to add a timestamp, then we -* add it to the start of the resevered space. +* add it to the start of the reserved space. */ if (unlikely (info->add_timestamp)) {bool abs =ring_buffer_time_stamp_abs (cpu_buffer->buffer);
@@ -2681,7 +2681,7 @@ trace_recursive_unlock(struct ring_buffer_per_cpu *cpu_buffer)* ring_buffer_nest_start - Allow to trace while nested * @buffer: The ring buffer to modify * -* The ring buffer has a safty mechanism to prevent recursion. +* The ring buffer has a safety mechanism to prevent recursion. * But there may be a case where a trace needs to be done while * tracing something else. In this case, calling this function * will allow this function to nest within a currently active
@@ -2699,7 +2699,7 @@ void ring_buffer_nest_start(struct ring_buffer *buffer)preempt_disable_notrace (); cpu =raw_smp_processor_id (); cpu_buffer = buffer->buffers[cpu]; -/* This is the shift value for the above recusive locking */ +/* This is the shift value for the above recursive locking */ cpu_buffer->nest +=NESTED_BITS ; }
@@ -2718,7 +2718,7 @@ void ring_buffer_nest_end(struct ring_buffer *buffer)/* disabled by ring_buffer_nest_start() */ cpu =raw_smp_processor_id (); cpu_buffer = buffer->buffers[cpu]; -/* This is the shift value for the above recusive locking */ +/* This is the shift value for the above recursive locking */ cpu_buffer->nest -=NESTED_BITS ;preempt_enable_notrace (); }
@@ -2907,7 +2907,7 @@ rb_reserve_next_event(struct ring_buffer *buffer,* @buffer: the ring buffer to reserve from * @length: the length of the data to reserve (excluding event header) * -* Returns a reseverd event on the ring buffer to copy directly to. +* Returns a reserved event on the ring buffer to copy directly to. * The user of this interface will need to get the body to write into * and can use the ring_buffer_event_data() interface. *
@@ -3009,7 +3009,7 @@ rb_decrement_entry(struct ring_buffer_per_cpu *cpu_buffer,* This function lets the user discard an event in the ring buffer * and then that event will not be read later. * -* This function only works if it is called before the the item has been +* This function only works if it is called before the item has been * committed. It will try to free the event from the ring buffer * if another event has not been added behind it. *
@@ -4127,7 +4127,7 @@ EXPORT_SYMBOL_GPL(ring_buffer_consume);* through the buffer. Memory is allocated, buffer recording * is disabled, and the iterator pointer is returned to the caller. * -* Disabling buffer recordng prevents the reading from being +* Disabling buffer recording prevents the reading from being * corrupted. This is not a consuming read, so a producer is not * expected. *
Signature · 1 line
2.17.0
Steven Rostedt
maintainerPATCH 19/19 ring-buffer: Fix typo in comment
In reply to Steven RostedtFrom: Vasyl Gomonovych <[email protected]>
Fix typo of the word 'been'
--- include/linux/ring_buffer.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/ring_buffer.h +1 -1
@@ -65,7 +65,7 @@ u64 ring_buffer_event_time_stamp(struct ring_buffer_event *event);/* * ring_buffer_discard_commit will remove an event that has not -* ben committed yet. If this is used, then ring_buffer_unlock_commit +* been committed yet. If this is used, then ring_buffer_unlock_commit * must not be called on the discarded event. This function * will try to remove the event from the ring buffer completely * if another event has not been written after it.
Signature · 1 line
2.17.0