tracing: Updates for 5.15
Subsystems (4)
Steven Rostedt
maintainercover tracing: Updates for 5.15
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git
for-next
Head SHA1: db396be6ddc4b953db3a725e19945fd4f151cfe9
Huang Shijie (1):
tracing: Fix a typo in tracepoint.h
Masahiro Yamada (2):
tracing: Simplify the Kconfig dependency of FTRACE
tracing: Refactor TRACE_IRQFLAGS_SUPPORT in Kconfig
Masami Hiramatsu (10):
tracing/boot: Fix a hist trigger dependency for boot time tracing tracing/boot: Add per-event histogram action options tracing/boot: Support multiple handlers for per-event histogram tracing/boot: Support multiple histograms for each event tracing/boot: Show correct histogram error command
Documentation: tracing: Add histogram syntax to boot-time tracing tools/bootconfig: Support per-group/all event enabling option tools/bootconfig: Add histogram syntax support to bconf2ftrace.sh tools/bootconfig: Use per-group/all enable option in ftrace2bconf script bootconfig/tracing/ktest: Update ktest example for boot-time tracing
Steven Rostedt (VMware) (5):
tracing: Add linear buckets to histogram logic tracing/histogram: Update the documentation for the buckets modifier
tracing: Have histogram types be constant when possible
tracing: Allow execnames to be passed as args for synthetic events
MAINTAINERS: Add an entry for os noise/latency
zhaoxiao (1):
tracepoint: Fix kerneldoc comments
----
Documentation/trace/boottime-trace.rst | 85 +++++- Documentation/trace/histogram.rst | 92 ++++++- MAINTAINERS | 14 + arch/Kconfig | 3 + arch/arc/Kconfig | 4 +- arch/arm/Kconfig | 5 +- arch/arm64/Kconfig | 4 +- arch/csky/Kconfig | 4 +- arch/hexagon/Kconfig | 4 +- arch/microblaze/Kconfig | 1 + arch/microblaze/Kconfig.debug | 5 - arch/mips/Kconfig | 1 + arch/mips/Kconfig.debug | 4 - arch/nds32/Kconfig | 4 +- arch/nios2/Kconfig | 3 - arch/openrisc/Kconfig | 4 +- arch/parisc/Kconfig | 1 + arch/parisc/Kconfig.debug | 3 - arch/powerpc/Kconfig | 5 +- arch/riscv/Kconfig | 4 +- arch/s390/Kconfig | 1 + arch/s390/Kconfig.debug | 3 - arch/sh/Kconfig | 1 + arch/sh/Kconfig.debug | 3 - arch/sparc/Kconfig | 1 + arch/sparc/Kconfig.debug | 4 - arch/um/Kconfig | 5 +- arch/x86/Kconfig | 1 + arch/x86/Kconfig.debug | 3 - arch/xtensa/Kconfig | 4 +- include/linux/tracepoint.h | 2 +- kernel/trace/Kconfig | 6 +- kernel/trace/trace.c | 1 + kernel/trace/trace_boot.c | 302 ++++++++++++++++++++- kernel/trace/trace_events_hist.c | 143 ++++++++-- kernel/tracepoint.c | 4 +- tools/bootconfig/scripts/bconf2ftrace.sh | 101 +++++++ tools/bootconfig/scripts/ftrace2bconf.sh | 24 +- tools/bootconfig/scripts/xbc.sh | 4 +- .../ktest/examples/bootconfigs/boottrace.bconf | 20 +- .../ktest/examples/bootconfigs/verify-boottrace.sh | 2 +- 41 files changed, 753 insertions(+), 132 deletions(-)
Steven Rostedt
maintainerPATCH 1/19 tracing/boot: Fix a hist trigger dependency for boot time tracing
From: Masami Hiramatsu <[email protected]>
Fixes a build error when CONFIG_HIST_TRIGGERS=n with boot-time tracing. Since the trigger_process_regex() is defined only when CONFIG_HIST_TRIGGERS=y, if it is disabled, the 'actions' event option also must be disabled.
--- kernel/trace/trace_boot.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
kernel/trace/trace_boot.c +9 -6
@@ -205,12 +205,15 @@ trace_boot_init_one_event(struct trace_array *tr, struct xbc_node *gnode,pr_err ("Failed to apply filter: %s\n" , buf); } -xbc_node_for_each_array_value (enode,"actions" , anode, p) { -if (strlcpy (buf, p,ARRAY_SIZE (buf)) >=ARRAY_SIZE (buf)) -pr_err ("action string is too long: %s\n" , p); -else if (trigger_process_regex (file, buf) <0 ) -pr_err ("Failed to apply an action: %s\n" , buf); - } +if (IS_ENABLED (CONFIG_HIST_TRIGGERS )) { +xbc_node_for_each_array_value (enode,"actions" , anode, p) { +if (strlcpy (buf, p,ARRAY_SIZE (buf)) >=ARRAY_SIZE (buf)) +pr_err ("action string is too long: %s\n" , p); +else if (trigger_process_regex (file, buf) <0 ) +pr_err ("Failed to apply an action: %s\n" , buf); + } + }else if (xbc_node_find_value (enode,"actions" ,NULL )) +pr_err ("Failed to apply event actions because CONFIG_HIST_TRIGGERS is not set.\n" );if (xbc_node_find_value (enode,"enable" ,NULL )) {if (trace_event_enable_disable (file,1 ,0 ) <0 )
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 2/19 tracing: Add linear buckets to histogram logic
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
There's been several times I wished the histogram logic had a "grouping" feature for the buckets. Currently, each bucket has a size of one. That is, if you trace the amount of requested allocations, each allocation is its own bucket, even if you are interested in what allocates 100 bytes or less, 100 to 200, 200 to 300, etc.
Also, without grouping, it fills up the allocated histogram buckets quickly. If you are tracking latency, and don't care if something is 200 microseconds off, or 201 microseconds off, but want to track them by say 10 microseconds each. This can not currently be done.
There is a log2 but that grouping get's too big too fast for a lot of cases.
Introduce a "buckets=SIZE" command to each field where it will record in a rounded number. For example:
Quoted 2 lines
# echo 'hist:keys=bytes_req.buckets=100:sort=bytes_req' > events/kmem/kmalloc/trigger # cat events/kmem/kmalloc/hist
# event histogram # # trigger info: hist:keys=bytes_req.buckets=100:vals=hitcount:sort=bytes_req.buckets=100:size=2048 [active] #
{ bytes_req: ~ 0-99 } hitcount: 3149
{ bytes_req: ~ 100-199 } hitcount: 1468
{ bytes_req: ~ 200-299 } hitcount: 39
{ bytes_req: ~ 300-399 } hitcount: 306
{ bytes_req: ~ 400-499 } hitcount: 364
{ bytes_req: ~ 500-599 } hitcount: 32
{ bytes_req: ~ 600-699 } hitcount: 69
{ bytes_req: ~ 700-799 } hitcount: 37 { bytes_req: ~ 1200-1299 } hitcount: 16
{ bytes_req: ~ 1400-1499 } hitcount: 30
{ bytes_req: ~ 2000-2099 } hitcount: 6
{ bytes_req: ~ 4000-4099 } hitcount: 2168
{ bytes_req: ~ 5000-5099 } hitcount: 6
Totals:
Hits: 7690
Entries: 13
Dropped: 0--- kernel/trace/trace_events_hist.c | 65 ++++++++++++++++++++++++++++---- 1 file changed, 58 insertions(+), 7 deletions(-)
kernel/trace/trace_events_hist.c +58 -7
@@ -121,6 +121,7 @@ struct hist_field {unsigned int size;unsigned int offset;unsigned int is_signed; +unsigned long buckets;const char *type;struct hist_field *operands[HIST_FIELD_OPERANDS_MAX ];struct hist_trigger_data *hist_data;
@@ -219,6 +220,27 @@ static u64 hist_field_log2(struct hist_field *hist_field,return (u64 )ilog2 (roundup_pow_of_two (val)); } +static u64 hist_field_bucket (struct hist_field *hist_field, +struct tracing_map_elt *elt, +struct trace_buffer *buffer, +struct ring_buffer_event *rbe, +void *event) +{ +struct hist_field *operand = hist_field->operands[0 ]; +unsigned long buckets = hist_field->buckets; + +u64 val = operand->fn (operand, elt, buffer, rbe, event); + +if (WARN_ON_ONCE (!buckets)) +return val; + +if (val >=LONG_MAX ) + val =div64_ul (val, buckets); +else + val = (u64)((unsigned long )val / buckets); +return val * buckets; +} + static u64 hist_field_plus(struct hist_field *hist_field, struct tracing_map_elt *elt, struct trace_buffer *buffer,
@@ -318,6 +340,7 @@ enum hist_field_flags {HIST_FIELD_FL_VAR_REF =1 <<14 ,HIST_FIELD_FL_CPU =1 <<15 ,HIST_FIELD_FL_ALIAS =1 <<16 , + HIST_FIELD_FL_BUCKET = 1 << 17, }; struct var_defs {
@@ -1109,7 +1132,8 @@ static const char *hist_field_name(struct hist_field *field,if (field->field) field_name = field->field->name;else if (field->flags &HIST_FIELD_FL_LOG2 || - field->flags &HIST_FIELD_FL_ALIAS ) + field->flags &HIST_FIELD_FL_ALIAS || + field->flags &HIST_FIELD_FL_BUCKET ) field_name =hist_field_name (field->operands[0 ], ++level);else if (field->flags &HIST_FIELD_FL_CPU ) field_name ="common_cpu" ;
@@ -1470,6 +1494,8 @@ static const char *get_hist_field_flags(struct hist_field *hist_field) flags_str ="syscall" ; else if (hist_field->flags & HIST_FIELD_FL_LOG2) flags_str ="log2" ; + else if (hist_field->flags & HIST_FIELD_FL_BUCKET) + flags_str ="buckets" ; else if (hist_field->flags & HIST_FIELD_FL_TIMESTAMP_USECS) flags_str ="usecs" ;
@@ -1658,9 +1684,10 @@ static struct hist_field *create_hist_field(struct hist_trigger_data *hist_data, goto out; } -if (flags &HIST_FIELD_FL_LOG2 ) { -unsigned long fl = flags & ~HIST_FIELD_FL_LOG2 ; - hist_field->fn = hist_field_log2; +if (flags & (HIST_FIELD_FL_LOG2 |HIST_FIELD_FL_BUCKET )) { +unsigned long fl = flags & ~(HIST_FIELD_FL_LOG2 |HIST_FIELD_FL_BUCKET ); + hist_field->fn = flags &HIST_FIELD_FL_LOG2 ? hist_field_log2 : + hist_field_bucket; hist_field->operands[0 ] =create_hist_field (hist_data, field, fl,NULL ); hist_field->size = hist_field->operands[0 ]->size; hist_field->type =kstrdup (hist_field->operands[0 ]->type,GFP_KERNEL );
@@ -1953,7 +1980,7 @@ static struct hist_field *parse_var_ref(struct hist_trigger_data *hist_data,static struct ftrace_event_field *parse_field (struct hist_trigger_data *hist_data,struct trace_event_file *file, -char *field_str,unsigned long *flags) +char *field_str,unsigned long *flags,unsigned long *buckets) {struct ftrace_event_field *field =NULL ;char *field_name, *modifier, *str;
@@ -1980,7 +2007,22 @@ parse_field(struct hist_trigger_data *hist_data, struct trace_event_file *file, *flags |=HIST_FIELD_FL_LOG2 ; else if (strcmp(modifier, "usecs") == 0) *flags |= HIST_FIELD_FL_TIMESTAMP_USECS; - else { + else if (strncmp(modifier, "bucket", 6) == 0) { + int ret; + + modifier += 6; + + if (*modifier == 's') + modifier++; + if (*modifier != '=') + goto error; + modifier++; + ret = kstrtoul(modifier, 0, buckets); + if (ret || !(*buckets)) + goto error; + *flags |= HIST_FIELD_FL_BUCKET; + } else { + error: hist_err(tr, HIST_ERR_BAD_FIELD_MODIFIER, errpos(modifier)); field = ERR_PTR(-EINVAL); goto out;
@@ -2049,6 +2091,7 @@ static struct hist_field *parse_atom(struct hist_trigger_data *hist_data,char *s, *ref_system =NULL , *ref_event =NULL , *ref_var = str;struct ftrace_event_field *field =NULL ;struct hist_field *hist_field =NULL ; +unsigned long buckets =0 ;int ret =0 ; s =strchr (str,'.' );
@@ -2086,7 +2129,7 @@ static struct hist_field *parse_atom(struct hist_trigger_data *hist_data, }else str = s; - field =parse_field (hist_data, file, str, flags); + field =parse_field (hist_data, file, str, flags, &buckets);if (IS_ERR (field)) { ret =PTR_ERR (field); goto out;
@@ -2097,6 +2140,7 @@ static struct hist_field *parse_atom(struct hist_trigger_data *hist_data, ret = -ENOMEM ; goto out; } + hist_field->buckets = buckets;return hist_field; out:
@@ -4698,6 +4742,11 @@ static void hist_trigger_print_key(struct seq_file *m, }else if (key_field->flags &HIST_FIELD_FL_LOG2 ) {seq_printf (m,"%s: ~ 2^%-2llu" , field_name, *(u64 *)(key + key_field->offset)); + }else if (key_field->flags &HIST_FIELD_FL_BUCKET ) { +unsigned long buckets = key_field->buckets; + uval = *(u64 *)(key + key_field->offset); +seq_printf (m,"%s: ~ %llu-%llu" , field_name, + uval, uval + buckets -1 ); }else if (key_field->flags &HIST_FIELD_FL_STRING ) {seq_printf (m,"%s: %-50s" , field_name, (char *)(key + key_field->offset));
@@ -5137,6 +5186,8 @@ static void hist_field_print(struct seq_file *m, struct hist_field *hist_field)seq_printf (m,".%s" , flags); } } +if (hist_field->buckets) +seq_printf (m,"=%ld" , hist_field->buckets); } static int event_hist_trigger_print(struct seq_file *m,
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 3/19 tracing/histogram: Update the documentation for the buckets modifier
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Update both the tracefs README file as well as the histogram.rst to include an explanation of what the buckets modifier is and how to use it. Include an example with the wakeup_latency example for both log2 and the buckets modifiers as there was no existing log2 example.
--- Documentation/trace/histogram.rst | 92 +++++++++++++++++++++++++++++-- kernel/trace/trace.c | 1 + 2 files changed, 87 insertions(+), 6 deletions(-)
Documentation/trace/histogram.rst +86 -6
@@ -77,6 +77,7 @@ Documentation written by Tom Zanussi .syscall display a syscall id as a system call name .execname display a common_pid as a program name .log2 display log2 value rather than raw number + .buckets=size display grouping of values rather than raw number .usecs display a common_timestamp in microseconds =========== ==========================================
@@ -228,7 +229,7 @@ Extended error information that lists the total number of bytes requested for each function in the kernel that made one or more calls to kmalloc:: - # echo 'hist:key=call_site:val=bytes_req' > \ + # echo 'hist:key=call_site:val=bytes_req.buckets=32' > \ /sys/kernel/debug/tracing/events/kmem/kmalloc/trigger This tells the tracing system to create a 'hist' trigger using the
@@ -1823,20 +1824,99 @@ and variables defined on other events (see Section 2.2.3 below on how that is done using hist trigger 'onmatch' action). Once that is done, the 'wakeup_latency' synthetic event instance is created. -A histogram can now be defined for the new synthetic event:: - - # echo 'hist:keys=pid,prio,lat.log2:sort=pid,lat' >> \ - /sys/kernel/debug/tracing/events/synthetic/wakeup_latency/trigger - The new event is created under the tracing/events/synthetic/ directory and looks and behaves just like any other event:: # ls /sys/kernel/debug/tracing/events/synthetic/wakeup_latency enable filter format hist id trigger +A histogram can now be defined for the new synthetic event:: + + # echo 'hist:keys=pid,prio,lat.log2:sort=lat' >> \ + /sys/kernel/debug/tracing/events/synthetic/wakeup_latency/trigger + +The above shows the latency "lat" in a power of 2 grouping. + Like any other event, once a histogram is enabled for the event, the output can be displayed by reading the event's 'hist' file. + # cat /sys/kernel/debug/tracing/events/synthetic/wakeup_latency/hist + + # event histogram + # + # trigger info: hist:keys=pid,prio,lat.log2:vals=hitcount:sort=lat.log2:size=2048 [active] + # + + { pid: 2035, prio: 9, lat: ~ 2^2 } hitcount: 43 + { pid: 2034, prio: 9, lat: ~ 2^2 } hitcount: 60 + { pid: 2029, prio: 9, lat: ~ 2^2 } hitcount: 965 + { pid: 2034, prio: 120, lat: ~ 2^2 } hitcount: 9 + { pid: 2033, prio: 120, lat: ~ 2^2 } hitcount: 5 + { pid: 2030, prio: 9, lat: ~ 2^2 } hitcount: 335 + { pid: 2030, prio: 120, lat: ~ 2^2 } hitcount: 10 + { pid: 2032, prio: 120, lat: ~ 2^2 } hitcount: 1 + { pid: 2035, prio: 120, lat: ~ 2^2 } hitcount: 2 + { pid: 2031, prio: 9, lat: ~ 2^2 } hitcount: 176 + { pid: 2028, prio: 120, lat: ~ 2^2 } hitcount: 15 + { pid: 2033, prio: 9, lat: ~ 2^2 } hitcount: 91 + { pid: 2032, prio: 9, lat: ~ 2^2 } hitcount: 125 + { pid: 2029, prio: 120, lat: ~ 2^2 } hitcount: 4 + { pid: 2031, prio: 120, lat: ~ 2^2 } hitcount: 3 + { pid: 2029, prio: 120, lat: ~ 2^3 } hitcount: 2 + { pid: 2035, prio: 9, lat: ~ 2^3 } hitcount: 41 + { pid: 2030, prio: 120, lat: ~ 2^3 } hitcount: 1 + { pid: 2032, prio: 9, lat: ~ 2^3 } hitcount: 32 + { pid: 2031, prio: 9, lat: ~ 2^3 } hitcount: 44 + { pid: 2034, prio: 9, lat: ~ 2^3 } hitcount: 40 + { pid: 2030, prio: 9, lat: ~ 2^3 } hitcount: 29 + { pid: 2033, prio: 9, lat: ~ 2^3 } hitcount: 31 + { pid: 2029, prio: 9, lat: ~ 2^3 } hitcount: 31 + { pid: 2028, prio: 120, lat: ~ 2^3 } hitcount: 18 + { pid: 2031, prio: 120, lat: ~ 2^3 } hitcount: 2 + { pid: 2028, prio: 120, lat: ~ 2^4 } hitcount: 1 + { pid: 2029, prio: 9, lat: ~ 2^4 } hitcount: 4 + { pid: 2031, prio: 120, lat: ~ 2^7 } hitcount: 1 + { pid: 2032, prio: 120, lat: ~ 2^7 } hitcount: 1 + + Totals: + Hits: 2122 + Entries: 30 + Dropped: 0 + + +The latency values can also be grouped linearly by a given size with +the ".buckets" modifier and specify a size (in this case groups of 10). + + # echo 'hist:keys=pid,prio,lat.buckets=10:sort=lat' >> \ + /sys/kernel/debug/tracing/events/synthetic/wakeup_latency/trigger + + # event histogram + # + # trigger info: hist:keys=pid,prio,lat.buckets=10:vals=hitcount:sort=lat.buckets=10:size=2048 [active] + # + + { pid: 2067, prio: 9, lat: ~ 0-9 } hitcount: 220 + { pid: 2068, prio: 9, lat: ~ 0-9 } hitcount: 157 + { pid: 2070, prio: 9, lat: ~ 0-9 } hitcount: 100 + { pid: 2067, prio: 120, lat: ~ 0-9 } hitcount: 6 + { pid: 2065, prio: 120, lat: ~ 0-9 } hitcount: 2 + { pid: 2066, prio: 120, lat: ~ 0-9 } hitcount: 2 + { pid: 2069, prio: 9, lat: ~ 0-9 } hitcount: 122 + { pid: 2069, prio: 120, lat: ~ 0-9 } hitcount: 8 + { pid: 2070, prio: 120, lat: ~ 0-9 } hitcount: 1 + { pid: 2068, prio: 120, lat: ~ 0-9 } hitcount: 7 + { pid: 2066, prio: 9, lat: ~ 0-9 } hitcount: 365 + { pid: 2064, prio: 120, lat: ~ 0-9 } hitcount: 35 + { pid: 2065, prio: 9, lat: ~ 0-9 } hitcount: 998 + { pid: 2071, prio: 9, lat: ~ 0-9 } hitcount: 85 + { pid: 2065, prio: 9, lat: ~ 10-19 } hitcount: 2 + { pid: 2064, prio: 120, lat: ~ 10-19 } hitcount: 2 + + Totals: + Hits: 2112 + Entries: 16 + Dropped: 0 + 2.2.3 Hist trigger 'handlers' and 'actions' -------------------------------------------
kernel/trace/trace.c +1 -0
@@ -5654,6 +5654,7 @@ static const char readme_msg[] ="\t .execname display a common_pid as a program name\n" "\t .syscall display a syscall id as a syscall name\n" "\t .log2 display log2 value rather than raw number\n" +"\t .buckets=size display values in groups of size rather than raw number\n" "\t .usecs display a common_timestamp in microseconds\n\n" "\t The 'pause' parameter can be used to pause an existing hist\n" "\t trigger or to start a hist trigger but not log any events\n"
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 4/19 tracing: Have histogram types be constant when possible
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Instead of kstrdup("const", GFP_KERNEL), have the hist_field type simply assign the constant hist_field->type = "const"; And when the value passed to it is a variable, use "kstrdup_const(var, GFP_KERNEL);" which will just copy the value if the variable is already a constant. This saves on having to allocate when not needed.
All frees of the hist_field->type will need to use kfree_const().
--- kernel/trace/trace_events_hist.c | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-)
kernel/trace/trace_events_hist.c +14 -18
@@ -1616,7 +1616,9 @@ static void __destroy_hist_field(struct hist_field *hist_field)kfree (hist_field->var.name);kfree (hist_field->name); -kfree (hist_field->type); + +/* Can likely be a const */ +kfree_const (hist_field->type);kfree (hist_field->system);kfree (hist_field->event_name);
@@ -1673,9 +1675,7 @@ static struct hist_field *create_hist_field(struct hist_trigger_data *hist_data,if (flags &HIST_FIELD_FL_HITCOUNT ) { hist_field->fn = hist_field_counter; hist_field->size =sizeof (u64); - hist_field->type =kstrdup ("u64" ,GFP_KERNEL ); -if (!hist_field->type) - goto free; + hist_field->type ="u64" ; goto out; }
@@ -1690,7 +1690,7 @@ static struct hist_field *create_hist_field(struct hist_trigger_data *hist_data, hist_field_bucket; hist_field->operands[0 ] =create_hist_field (hist_data, field, fl,NULL ); hist_field->size = hist_field->operands[0 ]->size; - hist_field->type =kstrdup (hist_field->operands[0 ]->type,GFP_KERNEL ); + hist_field->type =kstrdup_const (hist_field->operands[0 ]->type,GFP_KERNEL );if (!hist_field->type) goto free; goto out;
@@ -1699,18 +1699,14 @@ static struct hist_field *create_hist_field(struct hist_trigger_data *hist_data,if (flags &HIST_FIELD_FL_TIMESTAMP ) { hist_field->fn = hist_field_timestamp; hist_field->size =sizeof (u64); - hist_field->type =kstrdup ("u64" ,GFP_KERNEL ); -if (!hist_field->type) - goto free; + hist_field->type ="u64" ; goto out; }if (flags &HIST_FIELD_FL_CPU ) { hist_field->fn = hist_field_cpu; hist_field->size =sizeof (int ); - hist_field->type =kstrdup ("unsigned int" ,GFP_KERNEL ); -if (!hist_field->type) - goto free; + hist_field->type ="unsigned int" ; goto out; }
@@ -1723,7 +1719,7 @@ static struct hist_field *create_hist_field(struct hist_trigger_data *hist_data, flags |=HIST_FIELD_FL_STRING ; hist_field->size =MAX_FILTER_STR_VAL ; - hist_field->type =kstrdup (field->type,GFP_KERNEL ); + hist_field->type =kstrdup_const (field->type,GFP_KERNEL );if (!hist_field->type) goto free;
@@ -1736,7 +1732,7 @@ static struct hist_field *create_hist_field(struct hist_trigger_data *hist_data, }else { hist_field->size = field->size; hist_field->is_signed = field->is_signed; - hist_field->type =kstrdup (field->type,GFP_KERNEL ); + hist_field->type =kstrdup_const (field->type,GFP_KERNEL );if (!hist_field->type) goto free;
@@ -1822,7 +1818,7 @@ static int init_var_ref(struct hist_field *ref_field, } } - ref_field->type =kstrdup (var_field->type,GFP_KERNEL ); + ref_field->type =kstrdup_const (var_field->type,GFP_KERNEL );if (!ref_field->type) { err = -ENOMEM ; goto free;
@@ -2215,7 +2211,7 @@ static struct hist_field *parse_unary(struct hist_trigger_data *hist_data, expr->operands[0 ] = operand1; expr->operator =FIELD_OP_UNARY_MINUS ; expr->name =expr_str (expr,0 ); - expr->type =kstrdup (operand1->type,GFP_KERNEL ); + expr->type =kstrdup_const (operand1->type,GFP_KERNEL );if (!expr->type) { ret = -ENOMEM ; goto free;
@@ -2355,7 +2351,7 @@ static struct hist_field *parse_expr(struct hist_trigger_data *hist_data, expr->operator = field_op; expr->name =expr_str (expr,0 ); - expr->type =kstrdup (operand1->type,GFP_KERNEL ); + expr->type =kstrdup_const (operand1->type,GFP_KERNEL );if (!expr->type) { ret = -ENOMEM ; goto free;
@@ -2743,10 +2739,10 @@ static struct hist_field *create_var(struct hist_trigger_data *hist_data, var->var.hist_data = var->hist_data = hist_data; var->size = size; var->var.name =kstrdup (name,GFP_KERNEL ); - var->type =kstrdup (type,GFP_KERNEL ); + var->type =kstrdup_const (type,GFP_KERNEL );if (!var->var.name || !var->type) { +kfree_const (var->type);kfree (var->var.name); -kfree (var->type);kfree (var); var =ERR_PTR (-ENOMEM ); }
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 5/19 tracing: Allow execnames to be passed as args for synthetic events
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
Allow common_pid.execname to be saved in a variable in one histogram to be passed to another histogram that can pass it as a parameter to a synthetic event.
Quoted 1 line
# echo 'hist:keys=pid:__arg__1=common_timestamp.usecs:arg2=common_pid.execname' \
> events/sched/sched_waking/trigger
Quoted 2 lines
# echo 'wakeup_lat s32 pid; u64 delta; char wake_comm[]' > synthetic_events # echo 'hist:keys=next_pid:pid=next_pid,delta=common_timestamp.usecs-$__arg__1,exec=$arg2'\
':onmatch(sched.sched_waking).trace(wakeup_lat,$pid,$delta,$exec)' \
Quoted 1 line
events/sched/sched_switch/trigger
The above is a wake up latency synthetic event setup that passes the execname of the common_pid that woke the task to the scheduling of that task, which triggers a synthetic event that passes the original execname as a parameter to display it.
Quoted 2 lines
# echo 1 > events/synthetic/enable # cat trace
<idle>-0 [006] d..4 186.863801: wakeup_lat: pid=1306 delta=65 wake_comm=kworker/u16:3
<idle>-0 [000] d..4 186.863858: wakeup_lat: pid=163 delta=27 wake_comm=<idle>
<idle>-0 [001] d..4 186.863903: wakeup_lat: pid=1307 delta=36 wake_comm=kworker/u16:4
<idle>-0 [000] d..4 186.863927: wakeup_lat: pid=163 delta=5 wake_comm=<idle>
<idle>-0 [006] d..4 186.863957: wakeup_lat: pid=1306 delta=24 wake_comm=kworker/u16:3
sshd-1306 [006] d..4 186.864051: wakeup_lat: pid=61 delta=62 wake_comm=<idle>
<idle>-0 [000] d..4 186.965030: wakeup_lat: pid=609 delta=18 wake_comm=<idle>
<idle>-0 [006] d..4 186.987582: wakeup_lat: pid=1306 delta=65 wake_comm=kworker/u16:3
<idle>-0 [000] d..4 186.987639: wakeup_lat: pid=163 delta=27 wake_comm=<idle>--- kernel/trace/trace_events_hist.c | 46 +++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 4 deletions(-)
kernel/trace/trace_events_hist.c +42 -4
@@ -1420,17 +1420,17 @@ static int hist_trigger_elt_data_alloc(struct tracing_map_elt *elt)struct hist_trigger_data *hist_data = elt->map->private_data;unsigned int size =TASK_COMM_LEN ;struct hist_elt_data *elt_data; -struct hist_field *key_field; +struct hist_field *hist_field;unsigned int i, n_str; elt_data =kzalloc (sizeof (*elt_data),GFP_KERNEL );if (!elt_data)return -ENOMEM ; -for_each_hist_key_field (i, hist_data) { - key_field = hist_data->fields[i]; +for_each_hist_field (i, hist_data) { + hist_field = hist_data->fields[i]; -if (key_field->flags &HIST_FIELD_FL_EXECNAME ) { +if (hist_field->flags &HIST_FIELD_FL_EXECNAME ) { elt_data->comm =kzalloc (size,GFP_KERNEL );if (!elt_data->comm) {kfree (elt_data);
@@ -3771,6 +3771,41 @@ static int create_val_field(struct hist_trigger_data *hist_data,return __create_val_field (hist_data, val_idx, file,NULL , field_str,0 ); } +static const char *no_comm ="(no comm)" ; + +static u64 hist_field_execname (struct hist_field *hist_field, +struct tracing_map_elt *elt, +struct trace_buffer *buffer, +struct ring_buffer_event *rbe, +void *event) +{ +struct hist_elt_data *elt_data; + +if (WARN_ON_ONCE (!elt)) +return (u64 )(unsigned long )no_comm; + + elt_data = elt->private_data; + +if (WARN_ON_ONCE (!elt_data->comm)) +return (u64 )(unsigned long )no_comm; + +return (u64 )(unsigned long )(elt_data->comm); +} + +/* Convert a var that points to common_pid.execname to a string */ +static void update_var_execname (struct hist_field *hist_field) +{ + hist_field->flags =HIST_FIELD_FL_STRING |HIST_FIELD_FL_VAR | +HIST_FIELD_FL_EXECNAME ; + hist_field->size =MAX_FILTER_STR_VAL ; + hist_field->is_signed =0 ; + +kfree_const (hist_field->type); + hist_field->type ="char[]" ; + + hist_field->fn = hist_field_execname; +} + static int create_var_field(struct hist_trigger_data *hist_data, unsigned int val_idx, struct trace_event_file *file,
@@ -3795,6 +3830,9 @@ static int create_var_field(struct hist_trigger_data *hist_data, ret =__create_val_field (hist_data, val_idx, file, var_name, expr_str, flags); +if (!ret && hist_data->fields[val_idx]->flags &HIST_FIELD_FL_EXECNAME ) +update_var_execname (hist_data->fields[val_idx]); +if (!ret && hist_data->fields[val_idx]->flags &HIST_FIELD_FL_STRING ) hist_data->fields[val_idx]->var_str_idx = hist_data->n_var_str++;
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 6/19 tracing: Simplify the Kconfig dependency of FTRACE
In reply to Steven RostedtFrom: Masahiro Yamada <[email protected]>
The entire FTRACE block is surrounded by 'if TRACING_SUPPORT' ... 'endif'.
Using 'depends on' is a simpler way to guard FTRACE.
--- kernel/trace/Kconfig | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
kernel/trace/Kconfig +1 -5
@@ -135,10 +135,9 @@ config TRACING_SUPPORT depends on STACKTRACE_SUPPORT default y -if TRACING_SUPPORT - menuconfig FTRACE bool "Tracers" + depends on TRACING_SUPPORT default y if DEBUG_KERNEL help Enable the kernel tracing infrastructure.
@@ -1037,6 +1036,3 @@ config HIST_TRIGGERS_DEBUG If unsure, say N. endif # FTRACE - -endif # TRACING_SUPPORT -
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 8/19 tracing: Fix a typo in tracepoint.h
In reply to Steven RostedtFrom: Huang Shijie <[email protected]>
It should be @prev_pid, not @prev_prid.
--- include/linux/tracepoint.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/tracepoint.h +1 -1
@@ -475,7 +475,7 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)* * * * The declared 'local variable' is called '__entry' * * -* * __field(pid_t, prev_prid) is equivalent to a standard declaration: +* * __field(pid_t, prev_pid) is equivalent to a standard declaration: * * * * pid_t prev_pid; * *
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 9/19 tracing/boot: Add per-event histogram action options
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add a hist-trigger action syntax support to boot-time tracing. Currently, boot-time tracing supports per-event actions as option strings. However, for the histogram action, it has a special syntax and usually needs a long action definition.
To make it readable and fit to the bootconfig syntax, this introduces a new options for histogram.
Here are the histogram action options for boot-time tracing.
ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist {
keys = <KEY>[,...]
values = <VAL>[,...]
sort = <SORT-KEY>[,...]
size = <ENTRIES>
name = <HISTNAME>
var { <VAR> = <EXPR> ... }
pause|continue|clear
onmax|onchange { var = <VAR>; <ACTION> [= <PARAM>] }
onmatch { event = <EVENT>; <ACTION> [= <PARAM>] }
filter = <FILTER>
}Where <ACTION> is one of below;
trace = <EVENT>, <ARG1>[, ...]
save = <ARG1>[, ...]
snapshot--- kernel/trace/trace_boot.c | 231 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 231 insertions(+)
kernel/trace/trace_boot.c +231 -0
@@ -171,6 +171,231 @@ trace_boot_add_synth_event(struct xbc_node *node, const char *event) }#endif +#ifdef CONFIG_HIST_TRIGGERS +static int __init __printf(3, 4) +append_printf(char **bufp, char *end, const char *fmt, ...) +{ + va_list args; +int ret; + + if (*bufp == end) + return -ENOSPC; + + va_start(args, fmt); + ret =vsnprintf (*bufp, end - *bufp, fmt, args); + if (ret < end - *bufp) { + *bufp += ret; + } else { + *bufp = end; + ret = -ERANGE ; + } +va_end (args); + +return ret; +} + +static int __init +append_str_nospace (char **bufp,char *end,const char *str) +{ +char *p = *bufp; +int len; + +while (p < end -1 && *str !='\0' ) { +if (!isspace (*str)) + *(p++) = *str; + str++; + } + *p ='\0' ; +if (p == end -1 ) { + *bufp = end; +return -ENOSPC ; + } + len = p - *bufp; + *bufp = p; +return (int )len; +} + +static int __init +trace_boot_hist_add_array (struct xbc_node *hnode,char **bufp, +char *end,const char *key) +{ +struct xbc_node *knode, *anode; +const char *p; +char sep; + + knode =xbc_node_find_child (hnode, key); +if (knode) { + anode =xbc_node_get_child (knode); +if (!anode) { +pr_err ("hist.%s requires value(s).\n" , key); +return -EINVAL ; + } + +append_printf (bufp, end,":%s" , key); + sep ='=' ; +xbc_array_for_each_value (anode, p) { +append_printf (bufp, end,"%c%s" , sep, p); +if (sep =='=' ) + sep =',' ; + } + }else +return -ENOENT ; + +return 0 ; +} + +static int __init +trace_boot_hist_add_handler (struct xbc_node *hnode,char **bufp, +char *end,const char *param) +{ +struct xbc_node *knode, *anode; +const char *p; +char sep; + +/* Compose 'handler' parameter */ + p =xbc_node_find_value (hnode, param,NULL ); +if (!p) { +pr_err ("hist.%s requires '%s' option.\n" , +xbc_node_get_data (hnode), param); +return -EINVAL ; + } +append_printf (bufp, end,":%s(%s)" ,xbc_node_get_data (hnode), p); + +/* Compose 'action' parameter */ + knode =xbc_node_find_child (hnode,"trace" ); +if (!knode) + knode =xbc_node_find_child (hnode,"save" ); + +if (knode) { + anode =xbc_node_get_child (knode); +if (!anode || !xbc_node_is_value (anode)) { +pr_err ("hist.%s.%s requires value(s).\n" , +xbc_node_get_data (hnode), +xbc_node_get_data (knode)); +return -EINVAL ; + } + +append_printf (bufp, end,".%s" ,xbc_node_get_data (knode)); + sep ='(' ; +xbc_array_for_each_value (anode, p) { +append_printf (bufp, end,"%c%s" , sep, p); +if (sep =='(' ) + sep =',' ; + } +append_printf (bufp, end,")" ); + }else if (xbc_node_find_child (hnode,"snapshot" )) { +append_printf (bufp, end,".snapshot()" ); + }else { +pr_err ("hist.%s requires an action.\n" , +xbc_node_get_data (hnode)); +return -EINVAL ; + } + +return 0 ; +} + +/* +* Histogram boottime tracing syntax. +* +* ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist { +* keys = <KEY>[,...] +* values = <VAL>[,...] +* sort = <SORT-KEY>[,...] +* size = <ENTRIES> +* name = <HISTNAME> +* var { <VAR> = <EXPR> ... } +* pause|continue|clear +* onmax|onchange { var = <VAR>; <ACTION> [= <PARAM>] } +* onmatch { event = <EVENT>; <ACTION> [= <PARAM>] } +* filter = <FILTER> +* } +* +* Where <ACTION> are; +* +* trace = <EVENT>, <ARG1>[, ...] +* save = <ARG1>[, ...] +* snapshot +*/ +static int __init +trace_boot_compose_hist_cmd (struct xbc_node *hnode,char *buf,size_t size) +{ +struct xbc_node *node, *knode; +char *end = buf + size; +const char *p; +int ret =0 ; + +append_printf (&buf, end,"hist" ); + + ret =trace_boot_hist_add_array (hnode, &buf, end,"keys" ); +if (ret <0 ) { +if (ret == -ENOENT ) +pr_err ("hist requires keys.\n" ); +return -EINVAL ; + } + + ret =trace_boot_hist_add_array (hnode, &buf, end,"values" ); +if (ret == -EINVAL ) +return ret; + ret =trace_boot_hist_add_array (hnode, &buf, end,"sort" ); +if (ret == -EINVAL ) +return ret; + + p =xbc_node_find_value (hnode,"size" ,NULL ); +if (p) +append_printf (&buf, end,":size=%s" , p); + + p =xbc_node_find_value (hnode,"name" ,NULL ); +if (p) +append_printf (&buf, end,":name=%s" , p); + + node =xbc_node_find_child (hnode,"var" ); +if (node) { +xbc_node_for_each_key_value (node, knode, p) { +/* Expression must not include spaces. */ +append_printf (&buf, end,":%s=" , +xbc_node_get_data (knode)); +append_str_nospace (&buf, end, p); + } + } + +/* Histogram control attributes (mutual exclusive) */ +if (xbc_node_find_child (hnode,"pause" )) +append_printf (&buf, end,":pause" ); +else if (xbc_node_find_child (hnode,"continue" )) +append_printf (&buf, end,":continue" ); +else if (xbc_node_find_child (hnode,"clear" )) +append_printf (&buf, end,":clear" ); + +/* Histogram handler and actions */ + node =xbc_node_find_child (hnode,"onmax" ); +if (node &&trace_boot_hist_add_handler (node, &buf, end,"var" ) <0 ) +return -EINVAL ; + node =xbc_node_find_child (hnode,"onchange" ); +if (node &&trace_boot_hist_add_handler (node, &buf, end,"var" ) <0 ) +return -EINVAL ; + node =xbc_node_find_child (hnode,"onmatch" ); +if (node &&trace_boot_hist_add_handler (node, &buf, end,"event" ) <0 ) +return -EINVAL ; + + p =xbc_node_find_value (hnode,"filter" ,NULL ); +if (p) +append_printf (&buf, end," if %s" , p); + +if (buf == end) { +pr_err ("hist exceeds the max command length.\n" ); +return -E2BIG ; + } + +return 0 ; +} +#else +static int __init +trace_boot_compose_hist_cmd (struct xbc_node *hnode,char *buf,size_t size) +{ +return -EOPNOTSUPP ; +} +#endif +static void __init trace_boot_init_one_event (struct trace_array *tr,struct xbc_node *gnode,struct xbc_node *enode)
@@ -212,6 +437,12 @@ trace_boot_init_one_event(struct trace_array *tr, struct xbc_node *gnode,else if (trigger_process_regex (file, buf) <0 )pr_err ("Failed to apply an action: %s\n" , buf); } + anode =xbc_node_find_child (enode,"hist" ); +if (anode && +trace_boot_compose_hist_cmd (anode, buf,ARRAY_SIZE (buf)) ==0 ) { +if (trigger_process_regex (file, buf) <0 ) +pr_err ("Failed to apply hist trigger: %s\n" , buf); + } }else if (xbc_node_find_value (enode,"actions" ,NULL ))pr_err ("Failed to apply event actions because CONFIG_HIST_TRIGGERS is not set.\n" );
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 10/19 tracing/boot: Support multiple handlers for per-event histogram
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Support multiple handlers for per-event histogram in boot-time tracing. Since the histogram can register multiple same handler-actions with different parameters, this expands the syntax to support such cases.
With this update, the 'onmax', 'onchange' and 'onmatch' handler subkeys under per-event histogram option will take a number subkeys optionally as below. (see [.N])
ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist {
onmax|onchange[.N] { var = <VAR>; <ACTION> [= <PARAM>] }
onmatch[.N] { event = <EVENT>; <ACTION> [= <PARAM>] }
}The 'N' must be a digit (or digit started word).
Thus user can add several handler-actions to the histogram, for example,
ftrace.event.SOMEGROUP.SOMEEVENT.hist {
keys = SOME_ID; lat = common_timestamp.usecs-$ts0 onmatch.1 {
event = GROUP1.STARTEVENT1
trace = latency_event, SOME_ID, $lat
}
onmatch.2 {
event = GROUP2.STARTEVENT2
trace = latency_event, SOME_ID, $lat
}
}
Then, it can trace the elapsed time from GROUP1.STARTEVENT1 to SOMEGROUP.SOMEEVENT, and from GROUP2.STARTEVENT2 to SOMEGROUP.SOMEEVENT with SOME_ID key.
--- kernel/trace/trace_boot.c | 43 +++++++++++++++++++++++++++++++-------- 1 file changed, 35 insertions(+), 8 deletions(-)
kernel/trace/trace_boot.c +35 -8
@@ -245,8 +245,9 @@ trace_boot_hist_add_array(struct xbc_node *hnode, char **bufp, }static int __init -trace_boot_hist_add_handler (struct xbc_node *hnode,char **bufp, -char *end,const char *param) +trace_boot_hist_add_one_handler (struct xbc_node *hnode,char **bufp, +char *end,const char *handler, +const char *param) {struct xbc_node *knode, *anode;const char *p;
@@ -259,7 +260,7 @@ trace_boot_hist_add_handler(struct xbc_node *hnode, char **bufp, xbc_node_get_data(hnode), param);return -EINVAL ; } -append_printf (bufp, end,":%s(%s)" ,xbc_node_get_data (hnode), p); +append_printf (bufp, end,":%s(%s)" , handler, p);/* Compose 'action' parameter */ knode =xbc_node_find_child (hnode,"trace" );
@@ -294,6 +295,32 @@ trace_boot_hist_add_handler(struct xbc_node *hnode, char **bufp,return 0 ; } +static int __init +trace_boot_hist_add_handlers (struct xbc_node *hnode,char **bufp, +char *end,const char *param) +{ +struct xbc_node *node; +const char *p, *handler; +int ret; + + handler =xbc_node_get_data (hnode); + +xbc_node_for_each_subkey (hnode, node) { + p =xbc_node_get_data (node); +if (!isdigit (p[0 ])) +continue ; +/* All digit started node should be instances. */ + ret =trace_boot_hist_add_one_handler (node, bufp, end, handler, param); +if (ret <0 ) +break ; + } + +if (xbc_node_find_child (hnode, param)) + ret =trace_boot_hist_add_one_handler (hnode, bufp, end, handler, param); + +return ret; +} +/* * Histogram boottime tracing syntax. *
@@ -305,8 +332,8 @@ trace_boot_hist_add_handler(struct xbc_node *hnode, char **bufp,* name = <HISTNAME> * var { <VAR> = <EXPR> ... } * pause|continue|clear -* onmax|onchange { var = <VAR>; <ACTION> [= <PARAM>] } -* onmatch { event = <EVENT>; <ACTION> [= <PARAM>] } +* onmax|onchange[.N] { var = <VAR>; <ACTION> [= <PARAM>] } +* onmatch[.N] { event = <EVENT>; <ACTION> [= <PARAM>] } * filter = <FILTER> * } *
@@ -368,13 +395,13 @@ trace_boot_compose_hist_cmd(struct xbc_node *hnode, char *buf, size_t size)/* Histogram handler and actions */ node =xbc_node_find_child (hnode,"onmax" ); -if (node &&trace_boot_hist_add_handler (node, &buf, end,"var" ) <0 ) +if (node &&trace_boot_hist_add_handlers (node, &buf, end,"var" ) <0 )return -EINVAL ; node =xbc_node_find_child (hnode,"onchange" ); -if (node &&trace_boot_hist_add_handler (node, &buf, end,"var" ) <0 ) +if (node &&trace_boot_hist_add_handlers (node, &buf, end,"var" ) <0 )return -EINVAL ; node =xbc_node_find_child (hnode,"onmatch" ); -if (node &&trace_boot_hist_add_handler (node, &buf, end,"event" ) <0 ) +if (node &&trace_boot_hist_add_handlers (node, &buf, end,"event" ) <0 )return -EINVAL ; p =xbc_node_find_value (hnode,"filter" ,NULL );
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 11/19 tracing/boot: Support multiple histograms for each event
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add multiple histograms support for each event. This allows user to set multiple histograms to an event.
ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist[.N] {
...
}
The 'N' is a digit started string and it can be omitted for the default histogram.
For example, multiple hist triggers example in the Documentation/trace/histogram.rst can be written as below;
ftrace.event.net.netif_receive_skb.hist {
1 {
keys = skbaddr.hex
values = len
filter = len < 0
}
2 {
keys = skbaddr.hex
values = len
filter = len > 4096
}
3 {
keys = skbaddr.hex
values = len
filter = len == 256
}
4 {
keys = skbaddr.hex
values = len
}
5 {
keys = len
values = common_preempt_count
}
}--- kernel/trace/trace_boot.c | 41 ++++++++++++++++++++++++++++++--------- 1 file changed, 32 insertions(+), 9 deletions(-)
kernel/trace/trace_boot.c +32 -9
@@ -324,7 +324,7 @@ trace_boot_hist_add_handlers(struct xbc_node *hnode, char **bufp,/* * Histogram boottime tracing syntax. * -* ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist { +* ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist[.N] { * keys = <KEY>[,...] * values = <VAL>[,...] * sort = <SORT-KEY>[,...]
@@ -415,11 +415,37 @@ trace_boot_compose_hist_cmd(struct xbc_node *hnode, char *buf, size_t size)return 0 ; } + +static void __init +trace_boot_init_histograms (struct trace_event_file *file, +struct xbc_node *hnode,char *buf,size_t size) +{ +struct xbc_node *node; +const char *p; + +xbc_node_for_each_subkey (hnode, node) { + p =xbc_node_get_data (node); +if (!isdigit (p[0 ])) +continue ; +/* All digit started node should be instances. */ +if (trace_boot_compose_hist_cmd (node, buf, size) ==0 ) { +if (trigger_process_regex (file, buf) <0 ) +pr_err ("Failed to apply hist trigger: %s\n" , buf); + } + } + +if (xbc_node_find_child (hnode,"keys" )) { +if (trace_boot_compose_hist_cmd (hnode, buf, size) ==0 ) +if (trigger_process_regex (file, buf) <0 ) +pr_err ("Failed to apply hist trigger: %s\n" , buf); + } +}#else -static int __init -trace_boot_compose_hist_cmd (struct xbc_node *hnode,char *buf,size_t size) +static void __init +trace_boot_init_histograms (struct trace_event_file *file, +struct xbc_node *hnode,char *buf,size_t size) { -return -EOPNOTSUPP ; +/* do nothing */ }#endif
@@ -465,11 +491,8 @@ trace_boot_init_one_event(struct trace_array *tr, struct xbc_node *gnode,pr_err ("Failed to apply an action: %s\n" , buf); } anode =xbc_node_find_child (enode,"hist" ); -if (anode && -trace_boot_compose_hist_cmd (anode, buf,ARRAY_SIZE (buf)) ==0 ) { -if (trigger_process_regex (file, buf) <0 ) -pr_err ("Failed to apply hist trigger: %s\n" , buf); - } +if (anode) +trace_boot_init_histograms (file, anode, buf,ARRAY_SIZE (buf)); }else if (xbc_node_find_value (enode,"actions" ,NULL ))pr_err ("Failed to apply event actions because CONFIG_HIST_TRIGGERS is not set.\n" );
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 12/19 tracing/boot: Show correct histogram error command
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Since trigger_process_regex() modifies given trigger actions while parsing, the error message couldn't show what command was passed to the trigger_process_regex() when it returns an error.
To fix that, show the backed up trigger action command instead of parsed buffer.
--- kernel/trace/trace_boot.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
kernel/trace/trace_boot.c +10 -4
@@ -422,6 +422,7 @@ trace_boot_init_histograms(struct trace_event_file *file, {struct xbc_node *node;const char *p; +char *tmp;xbc_node_for_each_subkey (hnode, node) { p =xbc_node_get_data (node);
@@ -429,15 +430,20 @@ trace_boot_init_histograms(struct trace_event_file *file,continue ;/* All digit started node should be instances. */ if (trace_boot_compose_hist_cmd (node, buf, size) ==0 ) { + tmp =kstrdup (buf,GFP_KERNEL );if (trigger_process_regex (file, buf) <0 ) -pr_err ("Failed to apply hist trigger: %s\n" , buf); +pr_err ("Failed to apply hist trigger: %s\n" , tmp); +kfree (tmp); } }if (xbc_node_find_child (hnode,"keys" )) { -if (trace_boot_compose_hist_cmd (hnode, buf, size) ==0 ) +if (trace_boot_compose_hist_cmd (hnode, buf, size) ==0 ) { + tmp =kstrdup (buf,GFP_KERNEL );if (trigger_process_regex (file, buf) <0 ) -pr_err ("Failed to apply hist trigger: %s\n" , buf); +pr_err ("Failed to apply hist trigger: %s\n" , tmp); +kfree (tmp); + } } }#else
@@ -488,7 +494,7 @@ trace_boot_init_one_event(struct trace_array *tr, struct xbc_node *gnode,if (strlcpy (buf, p,ARRAY_SIZE (buf)) >=ARRAY_SIZE (buf))pr_err ("action string is too long: %s\n" , p);else if (trigger_process_regex (file, buf) <0 ) -pr_err ("Failed to apply an action: %s\n" , buf); +pr_err ("Failed to apply an action: %s\n" , p); } anode =xbc_node_find_child (enode,"hist" );if (anode)
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 13/19 Documentation: tracing: Add histogram syntax to boot-time tracing
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add the documentation about histogram syntax in boot-time tracing. This will allow user to write the histogram setting in a structured parameters.
--- Documentation/trace/boottime-trace.rst | 85 ++++++++++++++++++++++++-- 1 file changed, 80 insertions(+), 5 deletions(-)
Documentation/trace/boottime-trace.rst +80 -5
@@ -125,6 +125,71 @@ Note that kprobe and synthetic event definitions can be written under instance node, but those are also visible from other instances. So please take care for event name conflict. +Ftrace Histogram Options +------------------------ + +Since it is too long to write a histogram action as a string for per-event +action option, there are tree-style options under per-event 'hist' subkey +for the histogram actions. For the detail of the each parameter, +please read the event histogram document [3]_. + +.. [3] See :ref:`Documentation/trace/histogram.rst <histogram>` + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]keys = KEY1[, KEY2[...]] + Set histogram key parameters. (Mandatory) + The 'N' is a digit string for the multiple histogram. You can omit it + if there is one histogram on the event. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]values = VAL1[, VAL2[...]] + Set histogram value parameters. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]sort = SORT1[, SORT2[...]] + Set histogram sort parameter options. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]size = NR_ENTRIES + Set histogram size (number of entries). + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]name = NAME + Set histogram name. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]var.VARIABLE = EXPR + Define a new VARIABLE by EXPR expression. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]<pause|continue|clear> + Set histogram control parameter. You can set one of them. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onmatch.[M.]event = GROUP.EVENT + Set histogram 'onmatch' handler matching event parameter. + The 'M' is a digit string for the multiple 'onmatch' handler. You can omit it + if there is one 'onmatch' handler on this histogram. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onmatch.[M.]trace = EVENT[, ARG1[...]] + Set histogram 'trace' action for 'onmatch'. + EVENT must be a synthetic event name, and ARG1... are parameters + for that event. Mandatory if 'onmatch.event' option is set. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onmax.[M.]var = VAR + Set histogram 'onmax' handler variable parameter. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]onchange.[M.]var = VAR + Set histogram 'onchange' handler variable parameter. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]<onmax|onchange>.[M.]save = ARG1[, ARG2[...]] + Set histogram 'save' action parameters for 'onmax' or 'onchange' handler. + This option or below 'snapshot' option is mandatory if 'onmax.var' or + 'onchange.var' option is set. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.[N.]<onmax|onchange>.[M.]snapshot + Set histogram 'snapshot' action for 'onmax' or 'onchange' handler. + This option or above 'save' option is mandatory if 'onmax.var' or + 'onchange.var' option is set. + +ftrace.[instance.INSTANCE.]event.GROUP.EVENT.hist.filter = FILTER_EXPR + Set histogram filter expression. You don't need 'if' in the FILTER_EXPR. + +Note that this 'hist' option can conflict with the per-event 'actions' +option if the 'actions' option has a histogram action. + When to Start =============
@@ -159,13 +224,23 @@ below:: } synthetic.initcall_latency { fields = "unsigned long func", "u64 lat" - actions = "hist:keys=func.sym,lat:vals=lat:sort=lat" + hist { + keys = func.sym, lat + values = lat + sort = lat + } } - initcall.initcall_start { - actions = "hist:keys=func:ts0=common_timestamp.usecs" + initcall.initcall_start.hist { + keys = func + var.ts0 = common_timestamp.usecs } - initcall.initcall_finish { - actions = "hist:keys=func:lat=common_timestamp.usecs-$ts0:onmatch(initcall.initcall_start).initcall_latency(func,$lat)" + initcall.initcall_finish.hist { + keys = func + var.lat = common_timestamp.usecs - $ts0 + onmatch { + event = initcall.initcall_start + trace = initcall_latency, func, $lat + } } }
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 14/19 tools/bootconfig: Support per-group/all event enabling option
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add group or all event enabling syntax support to bconf2ftrace.sh. User can pass a bootconfig file which includes
ftrace[.instance.INSTANCE].event.enable
and
ftrace[.instance.INSTANCE].event.GROUP.enable
correctly.
--- tools/bootconfig/scripts/bconf2ftrace.sh | 13 +++++++++++++ 1 file changed, 13 insertions(+)
tools/bootconfig/scripts/bconf2ftrace.sh +13 -0
@@ -101,6 +101,12 @@ setup_event() { # prefix group event [instance] else eventdir="$TRACEFS/events/$2/$3" fi + # group enable + if [ "$3" = "enable" ]; then + run_cmd "echo 1 > ${eventdir}" + return + fi + case $2 in kprobes) xbc_get_val ${branch}.probes | while read line; do
@@ -127,6 +133,13 @@ setup_events() { # prefix("ftrace" or "ftrace.instance.INSTANCE") [instance] setup_event $prefix ${grpev%.*} ${grpev#*.} $2 done fi + if xbc_has_branch ${1}.event.enable; then + if [ "$2" ]; then + run_cmd "echo 1 > $TRACEFS/instances/$2/events/enable" + else + run_cmd "echo 1 > $TRACEFS/events/enable" + fi + fi } size2kb() { # size[KB|MB]
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 15/19 tools/bootconfig: Add histogram syntax support to bconf2ftrace.sh
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Add histogram syntax support to bconf2ftrace.sh script.
--- tools/bootconfig/scripts/bconf2ftrace.sh | 88 ++++++++++++++++++++++++ tools/bootconfig/scripts/xbc.sh | 4 +- 2 files changed, 90 insertions(+), 2 deletions(-)
tools/bootconfig/scripts/bconf2ftrace.sh +88 -0
@@ -94,6 +94,92 @@ compose_synth() { # event_name branch xbc_get_val $2 | while read field; do echo -n "$field; "; done } +print_hist_array() { # prefix key + __sep="=" + if xbc_has_key ${1}.${2}; then + echo -n ":$2" + xbc_get_val ${1}.${2} | while read field; do + echo -n "$__sep$field"; __sep="," + done + fi +} + +print_hist_action_array() { # prefix key + __sep="(" + echo -n ".$2" + xbc_get_val ${1}.${2} | while read field; do + echo -n "$__sep$field"; __sep="," + done + echo -n ")" +} + +print_hist_one_action() { # prefix handler param + echo -n ":${2}("`xbc_get_val ${1}.${3}`")" + if xbc_has_key "${1}.trace"; then + print_hist_action_array ${1} "trace" + elif xbc_has_key "${1}.save"; then + print_hist_action_array ${1} "save" + elif xbc_has_key "${1}.snapshot"; then + echo -n ".snapshot()" + fi +} + +print_hist_actions() { # prefix handler param + for __hdr in `xbc_subkeys ${1}.${2} 1 ".[0-9]"`; do + print_hist_one_action ${1}.${2}.$__hdr ${2} ${3} + done + if xbc_has_key ${1}.${2}.${3} ; then + print_hist_one_action ${1}.${2} ${2} ${3} + fi +} + +print_hist_var() { # prefix varname + echo -n ":${2}="`xbc_get_val ${1}.var.${2} | tr -d [:space:]` +} + +print_one_histogram() { # prefix + echo -n "hist" + print_hist_array $1 "keys" + print_hist_array $1 "values" + print_hist_array $1 "sort" + if xbc_has_key "${1}.size"; then + echo -n ":size="`xbc_get_val ${1}.size` + fi + if xbc_has_key "${1}.name"; then + echo -n ":name="`xbc_get_val ${1}.name` + fi + for __var in `xbc_subkeys "${1}.var" 1`; do + print_hist_var ${1} ${__var} + done + if xbc_has_key "${1}.pause"; then + echo -n ":pause" + elif xbc_has_key "${1}.continue"; then + echo -n ":continue" + elif xbc_has_key "${1}.clear"; then + echo -n ":clear" + fi + print_hist_actions ${1} "onmax" "var" + print_hist_actions ${1} "onchange" "var" + print_hist_actions ${1} "onmatch" "event" + + if xbc_has_key "${1}.filter"; then + echo -n " if "`xbc_get_val ${1}.filter` + fi +} + +setup_one_histogram() { # prefix trigger-file + run_cmd "echo '`print_one_histogram ${1}`' >> ${2}" +} + +setup_histograms() { # prefix trigger-file + for __hist in `xbc_subkeys ${1} 1 ".[0-9]"`; do + setup_one_histogram ${1}.$__hist ${2} + done + if xbc_has_key ${1}.keys; then + setup_one_histogram ${1} ${2} + fi +} + setup_event() { # prefix group event [instance] branch=$1.$2.$3 if [ "$4" ]; then
@@ -121,6 +207,8 @@ setup_event() { # prefix group event [instance] set_value_of ${branch}.filter ${eventdir}/filter set_array_of ${branch}.actions ${eventdir}/trigger + setup_histograms ${branch}.hist ${eventdir}/trigger + if xbc_has_key ${branch}.enable; then run_cmd "echo 1 > ${eventdir}/enable" fi
tools/bootconfig/scripts/xbc.sh +2 -2
@@ -49,8 +49,8 @@ xbc_has_branch() { # prefix-key grep -q "^$1" $XBC_TMPFILE } -xbc_subkeys() { # prefix-key depth +xbc_subkeys() { # prefix-key depth [subkey-pattern] __keys=`echo $1 | sed "s/\./ /g"` __s=`nr_args $__keys` - grep "^$1" $XBC_TMPFILE | cut -d= -f1| cut -d. -f$((__s + 1))-$((__s + $2)) | uniq + grep "^$1$3" $XBC_TMPFILE | cut -d= -f1| cut -d. -f$((__s + 1))-$((__s + $2)) | uniq }
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 16/19 tools/bootconfig: Use per-group/all enable option in ftrace2bconf script
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Use per-group/all enable option instead of ftrace.events option. This will make the bootconfig file more readable.
--- tools/bootconfig/scripts/ftrace2bconf.sh | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-)
tools/bootconfig/scripts/ftrace2bconf.sh +18 -6
@@ -92,6 +92,10 @@ referred_vars() { grep "^hist" $1/trigger | grep -o '$[a-zA-Z0-9]*' } +event_is_enabled() { # enable-file + test -f $1 & grep -q "1" $1 +} + per_event_options() { # event-dir evdir=$1 # Check the special event which has no filter and no trigger
@@ -113,7 +117,9 @@ per_event_options() { # event-dir emit_kv $PREFIX.event.$group.$event.actions += \'$action\' done - # enable is not checked; this is done by set_event in the instance. + if [ $GROUP_ENABLED -eq 0 ] && event_is_enabled $evdir/enable; then + emit_kv $PREFIX.event.$group.$event.enable + fi val=`cat $evdir/filter` if [ "$val" != "none" ]; then emit_kv $PREFIX.event.$group.$event.filter = "$val"
@@ -137,8 +143,19 @@ event_options() { kprobe_event_options synth_event_options fi + ALL_ENABLED=0 + if event_is_enabled $INSTANCE/events/enable; then + emit_kv $PREFIX.event.enable + ALL_ENABLED=1 + fi for group in `ls $INSTANCE/events/` ; do [ ! -d $INSTANCE/events/$group ] && continue + GROUP_ENABLED=$ALL_ENABLED + if [ $ALL_ENABLED -eq 0 ] && \ + event_is_enabled $INSTANCE/events/$group/enable ;then + emit_kv $PREFIX.event.$group.enable + GROUP_ENABLED=1 + fi for event in `ls $INSTANCE/events/$group/` ;do [ ! -d $INSTANCE/events/$group/$event ] && continue per_event_options $INSTANCE/events/$group/$event
@@ -226,11 +243,6 @@ instance_options() { # [instance-name] emit_kv $PREFIX.tracing_on = $val fi - val= - for i in `cat $INSTANCE/set_event`; do - val="$val, $i" - done - [ "$val" ] && emit_kv $PREFIX.events = "${val#,}" val=`cat $INSTANCE/current_tracer` [ $val != nop ] && emit_kv $PREFIX.tracer = $val if grep -qv "^#" $INSTANCE/set_ftrace_filter $INSTANCE/set_ftrace_notrace; then
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 17/19 bootconfig/tracing/ktest: Update ktest example for boot-time tracing
In reply to Steven RostedtFrom: Masami Hiramatsu <[email protected]>
Update ktest example for the boot-time tracing with histogram options. Note that since the histogram option uses "trace()" action instead of "EVENT()", this updates the matching pattern too.
--- .../examples/bootconfigs/boottrace.bconf | 20 ++++++++++++++----- .../examples/bootconfigs/verify-boottrace.sh | 2 +- 2 files changed, 16 insertions(+), 6 deletions(-)
tools/testing/ktest/examples/bootconfigs/boottrace.bconf +15 -5
@@ -10,13 +10,23 @@ ftrace.event { } synthetic.initcall_latency { fields = "unsigned long func", "u64 lat" - actions = "hist:keys=func.sym,lat:vals=lat:sort=lat" + hist { + keys = func.sym,lat + values = lat + sort = lat + } } - initcall.initcall_start { - actions = "hist:keys=func:ts0=common_timestamp.usecs" + initcall.initcall_start.hist { + keys = func; + var.ts0 = common_timestamp.usecs } - initcall.initcall_finish { - actions = "hist:keys=func:lat=common_timestamp.usecs-$ts0:onmatch(initcall.initcall_start).initcall_latency(func,$lat)" + initcall.initcall_finish.hist { + keys = func + var.lat = common_timestamp.usecs - $ts0 + onmatch { + event = initcall.initcall_start + trace = initcall_latency, func, $lat + } } }
tools/testing/ktest/examples/bootconfigs/verify-boottrace.sh +1 -1
@@ -58,7 +58,7 @@ compare_file_partial "events/synthetic/initcall_latency/enable" "0" compare_file_partial "events/initcall/initcall_start/trigger" "hist:keys=func:vals=hitcount:ts0=common_timestamp.usecs" compare_file_partial "events/initcall/initcall_start/enable" "1" -compare_file_partial "events/initcall/initcall_finish/trigger" 'hist:keys=func:vals=hitcount:lat=common_timestamp.usecs-\$ts0:sort=hitcount:size=2048:clock=global:onmatch(initcall.initcall_start).initcall_latency(func,\$lat)' +compare_file_partial "events/initcall/initcall_finish/trigger" 'hist:keys=func:vals=hitcount:lat=common_timestamp.usecs-\$ts0:sort=hitcount:size=2048:clock=global:onmatch(initcall.initcall_start).trace(initcall_latency,func,\$lat)' compare_file_partial "events/initcall/initcall_finish/enable" "1" compare_file "instances/foo/current_tracer" "function"
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 18/19 tracepoint: Fix kerneldoc comments
In reply to Steven RostedtFrom: zhaoxiao <[email protected]>
Fix function name in tracepoint.c kernel-doc comment to remove a warning found by clang_w1.
kernel/tracepoint.c:589: warning: expecting prototype for register_tracepoint_notifier(). Prototype was for register_tracepoint_module_notifier() instead kernel/tracepoint.c:613: warning: expecting prototype for unregister_tracepoint_notifier(). Prototype was for unregister_tracepoint_module_notifier() instead
--- kernel/tracepoint.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/tracepoint.c +2 -2
@@ -577,7 +577,7 @@ bool trace_module_has_bad_taint(struct module *mod)static BLOCKING_NOTIFIER_HEAD (tracepoint_notify_list);/** -* register_tracepoint_notifier - register tracepoint coming/going notifier +* register_tracepoint_module_notifier - register tracepoint coming/going notifier * @nb: notifier block * * Notifiers registered with this function are called on module
@@ -603,7 +603,7 @@ int register_tracepoint_module_notifier(struct notifier_block *nb)EXPORT_SYMBOL_GPL (register_tracepoint_module_notifier);/** -* unregister_tracepoint_notifier - unregister tracepoint coming/going notifier +* unregister_tracepoint_module_notifier - unregister tracepoint coming/going notifier * @nb: notifier block * * The notifier block callback should expect a "struct tp_module" data
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 19/19 MAINTAINERS: Add an entry for os noise/latency
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
The "latency" tracers have some different requirements than normal tracing, and also includes Daniel as a maintainer. Add a section in the MAINTAINERS file to help direct patches and bug reports to these tracers to the right people.
--- MAINTAINERS | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
MAINTAINERS +14 -0
@@ -18798,6 +18798,20 @@ F: arch/x86/mm/testmmiotrace.c F: include/linux/mmiotrace.h F: kernel/trace/trace_mmiotrace.c +TRACING OS NOISE / LATENCY TRACERS +M: Steven Rostedt <[email protected]> +M: Daniel Bristot de Oliveira <[email protected]> +S: Maintained +F: kernel/trace/trace_osnoise.c +F: include/trace/events/osnoise.h +F: kernel/trace/trace_hwlat.c +F: kernel/trace/trace_irqsoff.c +F: kernel/trace/trace_sched_wakeup.c +F: Documentation/trace/osnoise-tracer.rst +F: Documentation/trace/timerlat-tracer.rst +F: Documentation/trace/hwlat_detector.rst +F: arch/*/kernel/trace.c + TRIVIAL PATCHES M: Jiri Kosina <[email protected]> S: Maintained
Signature · 1 line
2.30.2