few fixes for synthetic trace events
v3 cover few fixes for synthetic trace events
Hi Steven,
I'm now sending these patches in one patchset, because the second patch has a dependeny on the union vs. cast fix.
Changes in v3:
- remove superfluous struct around union trace_synth_field
Changes in v2:
- cosmetic changes
- add struct trace_dynamic_info to include/linux/trace_events.h
Sven Schnelle (3):
tracing/synthetic: use union instead of casts tracing/synthetic: skip first entry for stack traces tracing/synthetic: allocate one additional element for size
include/linux/trace_events.h | 11 ++++ kernel/trace/trace.h | 8 +++ kernel/trace/trace_events_synth.c | 103 ++++++++++++------------------ 3 files changed, 60 insertions(+), 62 deletions(-)
Signature · 1 line
2.39.2
PATCH v3 1/3 tracing/synthetic: use union instead of casts
The current code uses a lot of casts to access the fields member in struct synth_trace_events with different sizes. This makes the code hard to read, and had already introduced an endianness bug. Use a union and struct instead.
--- include/linux/trace_events.h | 11 ++++ kernel/trace/trace.h | 8 +++ kernel/trace/trace_events_synth.c | 87 +++++++++++++------------------ 3 files changed, 56 insertions(+), 50 deletions(-)
include/linux/trace_events.h +11 -0
@@ -59,6 +59,17 @@ int trace_raw_output_prep(struct trace_iterator *iter, extern __printf(2, 3) void trace_event_printf(struct trace_iterator *iter, const char *fmt, ...); +/* Used to find the offset and length of dynamic fields in trace events */ +struct trace_dynamic_info { +#ifdef CONFIG_CPU_BIG_ENDIAN + u16 offset; + u16 len; +#else + u16 len; + u16 offset; +#endif +}; + /* * The trace entry - the most basic unit of tracing. This is what * is printed in the end as a single line in the trace output, such as:
kernel/trace/trace.h +8 -0
@@ -1295,6 +1295,14 @@ static inline void trace_branch_disable(void)/* set ring buffers to default size if not already done so */ int tracing_update_buffers (void ); +union trace_synth_field { +u8 as_u8; +u16 as_u16; +u32 as_u32; +u64 as_u64; +struct trace_dynamic_info as_dynamic; +}; +struct ftrace_event_field {struct list_head link;const char *name;
kernel/trace/trace_events_synth.c +37 -50
223 lines not on this page · raw message
Signature · 1 line
2.39.2
Steven Rostedt
maintainerRe: [PATCH v3 1/3] tracing/synthetic: use union instead of casts
Quoted 7 lines · On Thu, 10 Aug 2023 08:05:36 +0200 Sven Schnelle <[email protected]> wrote:
include/linux/trace_events.h +1
-0@@ -59,6 +59,17 @@ int trace_raw_output_prep(struct trace_iterator *iter, extern __printf(2, 3) void trace_event_printf(struct trace_iterator *iter, const char *fmt, ...); +/* Used to find the offset and length of dynamic fields in trace events */
include/linux/trace_events.h int trace_raw_output_prep(struct trace_iterator *iter, · line 72
Quoted 10 lines
include/linux/trace_events.h +10
-0+struct trace_dynamic_info { +#ifdef CONFIG_CPU_BIG_ENDIAN +u16 offset; +u16 len; +#else +u16 len; +u16 offset; +#endif +}; +
So this patch broke synthetic events. I found that it works with:
include/linux/trace_events.h +2 -2
@@ -65,10 +65,10 @@ struct trace_dynamic_info {u16 offset;u16 len;#else -u16 len;u16 offset; +u16 len;#endif -}; +} __packed;/* * The trace entry - the most basic unit of tracing. This is what
That is, I had to swap the order of len vs offset, and to also be safe, I added "__packed".
I'm guessing that the BIG ENDIAN is broken too? So I'm going to swap that as well.
It causes the following test to fail:
./ftracetest test.d/trigger/inter-event/trigger-synthetic-event-dynstring.tc
Which I didn't notice because I found that test to be broken, and needs this patch:
tools/testing/selftests/ftrace/test.d/trigger/inter-event/trigger-synthetic-event-dynstring.tc +1 -1
@@ -1,7 +1,7 @@ #!/bin/sh # SPDX-License-Identifier: GPL-2.0 # description: event trigger - test inter-event histogram trigger trace action with dynamic string param -# requires: set_event synthetic_events events/sched/sched_process_exec/hist "char name[]' >> synthetic_events":README ping:program +# requires: set_event synthetic_events events/sched/sched_process_exec/hist "' >> synthetic_events":README ping:program fail() { #msg echo $1
-- Steve
PATCH v3 2/3 tracing/synthetic: skip first entry for stack traces
In reply to Sven SchnelleWhile debugging another issue i noticed that the stack trace output contains the number of entries on top:
<idle>-0 [000] d..4. 203.322502: wake_lat: pid=0 delta=2268270616 stack=STACK:
=> 0x10
=> __schedule+0xac6/0x1a98
=> schedule+0x126/0x2c0
=> schedule_timeout+0x242/0x2c0
=> __wait_for_common+0x434/0x680
=> __wait_rcu_gp+0x198/0x3e0
=> synchronize_rcu+0x112/0x138
=> ring_buffer_reset_online_cpus+0x140/0x2e0
=> tracing_reset_online_cpus+0x15c/0x1d0
=> tracing_set_clock+0x180/0x1d8
=> hist_register_trigger+0x486/0x670
=> event_hist_trigger_parse+0x494/0x1318
=> trigger_process_regex+0x1d4/0x258
=> event_trigger_write+0xb4/0x170
=> vfs_write+0x210/0xad0
=> ksys_write+0x122/0x208
Fix this by skipping the first element. Also replace the pointer logic with an index variable which is easier to read.
--- kernel/trace/trace_events_synth.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-)
kernel/trace/trace_events_synth.c +4 -13
@@ -350,7 +350,7 @@ static enum print_line_t print_synth_event(struct trace_iterator *iter,struct trace_seq *s = &iter->seq;struct synth_trace_event *entry;struct synth_event *se; -unsigned int i, n_u64; +unsigned int i, j, n_u64;char print_fmt[32 ];const char *fmt;
@@ -389,18 +389,13 @@ static enum print_line_t print_synth_event(struct trace_iterator *iter, n_u64 +=STR_VAR_LEN_MAX /sizeof (u64); } }else if (se->fields[i]->is_stack) { -unsigned long *p, *end;union trace_synth_field *data = &entry->fields[n_u64]; - - p = (void *)entry + data->as_dynamic.offset; - end = (void *)p + data->as_dynamic.len - (sizeof (long ) -1 ); +unsigned long *p = (void *)entry + data->as_dynamic.offset;trace_seq_printf (s,"%s=STACK:\n" , se->fields[i]->name); - -for (; *p && p < end; p++) -trace_seq_printf (s,"=> %pS\n" , (void *)*p); +for (j =1 ; j < data->as_dynamic.len /sizeof (long ); j++) +trace_seq_printf (s,"=> %pS\n" , (void *)p[j]); n_u64++; - }else {struct trace_print_flags __flags[] = { __def_gfpflag_names, {-1 ,NULL } };
@@ -490,10 +485,6 @@ static unsigned int trace_stack(struct synth_trace_event *entry,break ; } -/* Include the zero'd element if it fits */ -if (len <HIST_STACKTRACE_DEPTH ) - len++; - len *=sizeof (long );/* Find the dynamic section to copy the stack into. */
Signature · 1 line
2.39.2
PATCH v3 3/3 tracing/synthetic: allocate one additional element for size
In reply to Sven SchnelleWhile debugging another issue i noticed that the stack trace contains one invalid entry at the end:
<idle>-0 [008] d..4. 26.484201: wake_lat: pid=0 delta=2629976084 000000009cc24024 stack=STACK:
=> __schedule+0xac6/0x1a98
=> schedule+0x126/0x2c0
=> schedule_timeout+0x150/0x2c0
=> kcompactd+0x9ca/0xc20
=> kthread+0x2f6/0x3d8
=> __ret_from_fork+0x8a/0xe8
=> 0x6b6b6b6b6b6b6b6b
This is because the code failed to add the one element containing the number of entries to field_size.
--- kernel/trace/trace_events_synth.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/trace/trace_events_synth.c +2 -1
@@ -528,7 +528,8 @@ static notrace void trace_event_raw_event_synth(void *__data, str_val = (char *)(long )var_ref_vals[val_idx];if (event->dynamic_fields[i]->is_stack) { - len = *((unsigned long *)str_val); +/* reserve one extra element for size */ + len = *((unsigned long *)str_val) +1 ; len *=sizeof (unsigned long ); }else { len =fetch_store_strlen ((unsigned long )str_val);
Signature · 1 line
2.39.2
Quoted 17 lines · Sven Schnelle
Hi Steven,
I'm now sending these patches in one patchset, because the second patch has a dependeny on the union vs. cast fix.
Changes in v3:
- remove superfluous struct around uniontrace_synth_fieldChanges in v2:
- cosmetic changes
- add structtrace_dynamic_infotoinclude/linux/trace_events.hSven Schnelle (3):
tracing/synthetic: use union instead of casts tracing/synthetic: skip first entry for stack traces tracing/synthetic: allocate one additional element for size
Quoted 4 lines
include/linux/trace_events.h | 11 ++++ kernel/trace/trace.h | 8 +++ kernel/trace/trace_events_synth.c | 103 ++++++++++++------------------ 3 files changed, 60 insertions(+), 62 deletions(-)
Gentle ping? We're still seeing a lot of KASAN warnings.
Steven Rostedt
maintainerQuoted 19 lines · On Wed, 16 Aug 2023 11:08:52 +0200 Sven Schnelle <[email protected]> wrote:
Quoted 17 lines · Sven Schnelle
Hi Steven,
I'm now sending these patches in one patchset, because the second patch has a dependeny on the union vs. cast fix.
Changes in v3:
- remove superfluous struct around uniontrace_synth_fieldChanges in v2:
- cosmetic changes
- add structtrace_dynamic_infotoinclude/linux/trace_events.hSven Schnelle (3):
tracing/synthetic: use union instead of casts tracing/synthetic: skip first entry for stack traces tracing/synthetic: allocate one additional element for size
Quoted 6 lines
Quoted 4 lines
include/linux/trace_events.h | 11 ++++ kernel/trace/trace.h | 8 +++ kernel/trace/trace_events_synth.c | 103 ++++++++++++------------------ 3 files changed, 60 insertions(+), 62 deletions(-)Gentle ping? We're still seeing a lot of KASAN warnings.
Can you resend and include a Cc to [email protected]. That will put it into patchwork:
https://patchwork.kernel.org/project/linux-trace-kernel/list/
Also, include everyone in the MAINTAINERS file too please.
That's what I use to know what to pull in for tracing. I have a personal patchwork on my INBOX that this did go into, but that's always considered lower priority.
-- Steve