few fixes for synthetic trace events
v3 cover few fixes for synthetic trace events
Hi Steven,
I'm now sending these patches in one patchset, because the second patch has a dependeny on the union vs. cast fix.
Changes in v3:
- remove superfluous struct around union trace_synth_field
Changes in v2:
- cosmetic changes
- add struct trace_dynamic_info to include/linux/trace_events.h
Sven Schnelle (3):
tracing/synthetic: use union instead of casts tracing/synthetic: skip first entry for stack traces tracing/synthetic: allocate one additional element for size
include/linux/trace_events.h | 11 ++++ kernel/trace/trace.h | 8 +++ kernel/trace/trace_events_synth.c | 103 ++++++++++++------------------ 3 files changed, 60 insertions(+), 62 deletions(-)
Signature · 1 line
2.39.2
PATCH RESEND v3 1/3 tracing/synthetic: use union instead of casts
The current code uses a lot of casts to access the fields member in struct synth_trace_events with different sizes. This makes the code hard to read, and had already introduced an endianness bug. Use a union and struct instead.
--- include/linux/trace_events.h | 11 ++++ kernel/trace/trace.h | 8 +++ kernel/trace/trace_events_synth.c | 87 +++++++++++++------------------ 3 files changed, 56 insertions(+), 50 deletions(-)
include/linux/trace_events.h +11 -0
@@ -59,6 +59,17 @@ int trace_raw_output_prep(struct trace_iterator *iter, extern __printf(2, 3) void trace_event_printf(struct trace_iterator *iter, const char *fmt, ...); +/* Used to find the offset and length of dynamic fields in trace events */ +struct trace_dynamic_info { +#ifdef CONFIG_CPU_BIG_ENDIAN + u16 offset; + u16 len; +#else + u16 len; + u16 offset; +#endif +}; + /* * The trace entry - the most basic unit of tracing. This is what * is printed in the end as a single line in the trace output, such as:
kernel/trace/trace.h +8 -0
@@ -1295,6 +1295,14 @@ static inline void trace_branch_disable(void)/* set ring buffers to default size if not already done so */ int tracing_update_buffers (void ); +union trace_synth_field { +u8 as_u8; +u16 as_u16; +u32 as_u32; +u64 as_u64; +struct trace_dynamic_info as_dynamic; +}; +struct ftrace_event_field {struct list_head link;const char *name;
kernel/trace/trace_events_synth.c +37 -50
223 lines not on this page · raw message
Signature · 1 line
2.39.2
kernel test robot
build botRe: [PATCH RESEND v3 1/3] tracing/synthetic: use union instead of casts
kernel test robot noticed "BUG:unable_to_handle_page_fault_for_address" on:
Hello,
kernel test robot noticed "BUG:unable_to_handle_page_fault_for_address" on:
commit: e2c9745169808b98f235c09d1366cf8b53ce0d3c ("[PATCH RESEND v3 1/3] tracing/synthetic: use union instead of casts")
url: https://github.com/intel-lab-lkp/linux/commits/Sven-Schnelle/tracing-synthetic-use-union-instead-of-casts/20230817-002758
base: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git 4853c74bd7ab7fdb83f319bd9ace8a08c031e9b6 patch link: https://lore.kernel.org/all/[email protected]/
patch subject: [PATCH RESEND v3 1/3] tracing/synthetic: use union instead of casts
in testcase: boot
compiler: clang-16
test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 4G
(please refer to attached dmesg/kmsg for entire log/backtrace)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-lkp/[email protected]
[ 64.204176][ T48] Dumping ftrace buffer: [ 64.204632][ T48] --------------------------------- [ 64.205082][ T48] BUG: unable to handle page fault for address: 001c24ca [ 64.205641][ T48] #PF: supervisor read access in kernel mode [ 64.206115][ T48] #PF: error_code(0x0000) - not-present page [ 64.206588][ T48] *pde = 00000000 [ 64.206897][ T48] Oops: 0000 [#1] SMP [ 64.207221][ T48] CPU: 0 PID: 48 Comm: rcu_scale_write Tainted: G T 6.5.0-rc6-00037-ge2c974516980 #1 [ 64.208074][ T48] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 [ 64.208892][ T48] EIP: string+0x98/0x100 [ 64.209237][ T48] Code: 94 c2 75 b4 a1 8c c2 75 b4 40 89 44 24 18 31 d2 eb 0e 47 89 3d 94 c2 75 b4 42 39 54 24 0c 74 32 8b 44 24 08 8d 0c 10 8b 45 08 <0f> b6 04 10 84 c0 74 2c 8b 74 24 18 01 d6 89 35 8c c2 75 b4 3b 0c [ 64.210761][ T48] EAX: 001c24ca EBX: 000405c2 ECX: b516e406 EDX: 00000000 [ 64.211325][ T48] ESI: ffff0a00 EDI: 00005af2 EBP: b600bcd4 ESP: b600bca8 [ 64.211876][ T48] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00010046 [ 64.212468][ T48] CR0: 80050033 CR2: 001c24ca CR3: 08d2a000 CR4: 000406d0 [ 64.213022][ T48] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 [ 64.213577][ T48] DR6: fffe0ff0 DR7: 00000400 [ 64.213945][ T48] Call Trace: [ 64.214208][ T48] ? __die_body+0x69/0xc0 [ 64.214546][ T48] ? __die+0x7e/0x90 [ 64.214863][ T48] ? page_fault_oops+0x22d/0x2b0 [ 64.215242][ T48] ? is_prefetch+0x4a/0x220 [ 64.215605][ T48] ? kernelmode_fixup_or_oops+0xfa/0x140 [ 64.216051][ T48] ? __bad_area_nosemaphore+0x64/0x2a0 [ 64.216485][ T48] ? bad_area_nosemaphore+0x12/0x20 [ 64.216899][ T48] ? do_user_addr_fault+0x650/0x7e0 [ 64.217313][ T48] ? pvclock_clocksource_read_nowd+0x7c/0x230 [ 64.217800][ T48] ? exc_page_fault+0xc5/0x358 [ 64.218183][ T48] ? pvclock_clocksource_read_nowd+0x230/0x230 [ 64.218666][ T48] ? handle_exception+0x14b/0x14b [ 64.219059][ T48] ? number+0x9b/0x630 [ 64.219378][ T48] ? pvclock_clocksource_read_nowd+0x230/0x230 [ 64.219866][ T48] ? string+0x98/0x100 [ 64.220187][ T48] ? pvclock_clocksource_read_nowd+0x230/0x230 [ 64.220656][ T48] ? string+0x98/0x100 [ 64.220986][ T48] vsnprintf+0x420/0x580 [ 64.221323][ T48] ? vsnprintf+0x3e7/0x580 [ 64.221669][ T48] seq_buf_vprintf+0x79/0xc0 [ 64.222029][ T48] trace_seq_printf+0x35/0xa0 [ 64.222400][ T48] print_synth_event+0x26f/0x300 [ 64.222805][ T48] ? trace_event_raw_event_synth+0x410/0x410 [ 64.223272][ T48] print_trace_fmt+0xfe/0x170 [ 64.223651][ T48] print_trace_line+0x10d/0x1c0 [ 64.224046][ T48] ftrace_dump+0x2fa/0x410 [ 64.224407][ T48] rcu_scale_writer+0x59c/0x640 [ 64.224808][ T48] kthread+0x158/0x170 [ 64.225146][ T48] ? rcu_scale_reader+0x180/0x180 [ 64.225559][ T48] ? kthread_unuse_mm+0x160/0x160 [ 64.225968][ T48] ? kthread_unuse_mm+0x160/0x160 [ 64.226379][ T48] ret_from_fork+0x43/0x70 [ 64.226742][ T48] ret_from_fork_asm+0x12/0x1c [ 64.227130][ T48] entry_INT80_32+0x108/0x108 [ 64.227516][ T48] Modules linked in: [ 64.227832][ T48] CR2: 00000000001c24ca [ 64.228166][ T48] ---[ end trace 0000000000000000 ]--- [ 64.228586][ T48] EIP: string+0x98/0x100 [ 64.228921][ T48] Code: 94 c2 75 b4 a1 8c c2 75 b4 40 89 44 24 18 31 d2 eb 0e 47 89 3d 94 c2 75 b4 42 39 54 24 0c 74 32 8b 44 24 08 8d 0c 10 8b 45 08 <0f> b6 04 10 84 c0 74 2c 8b 74 24 18 01 d6 89 35 8c c2 75 b4 3b 0c [ 64.230401][ T48] EAX: 001c24ca EBX: 000405c2 ECX: b516e406 EDX: 00000000 [ 64.230918][ T48] ESI: ffff0a00 EDI: 00005af2 EBP: b600bcd4 ESP: b600bca8 [ 64.231432][ T48] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00010046 [ 64.231993][ T48] CR0: 80050033 CR2: 001c24ca CR3: 08d2a000 CR4: 000406d0 [ 64.232527][ T48] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000 [ 64.233094][ T48] DR6: fffe0ff0 DR7: 00000400 [ 64.233474][ T48] Kernel panic - not syncing: Fatal exception [ 64.234050][ T48] Kernel Offset: disabled
The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20230825/[email protected]
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
PATCH RESEND v3 2/3 tracing/synthetic: skip first entry for stack traces
In reply to Sven SchnelleWhile debugging another issue i noticed that the stack trace output contains the number of entries on top:
<idle>-0 [000] d..4. 203.322502: wake_lat: pid=0 delta=2268270616 stack=STACK:
=> 0x10
=> __schedule+0xac6/0x1a98
=> schedule+0x126/0x2c0
=> schedule_timeout+0x242/0x2c0
=> __wait_for_common+0x434/0x680
=> __wait_rcu_gp+0x198/0x3e0
=> synchronize_rcu+0x112/0x138
=> ring_buffer_reset_online_cpus+0x140/0x2e0
=> tracing_reset_online_cpus+0x15c/0x1d0
=> tracing_set_clock+0x180/0x1d8
=> hist_register_trigger+0x486/0x670
=> event_hist_trigger_parse+0x494/0x1318
=> trigger_process_regex+0x1d4/0x258
=> event_trigger_write+0xb4/0x170
=> vfs_write+0x210/0xad0
=> ksys_write+0x122/0x208
Fix this by skipping the first element. Also replace the pointer logic with an index variable which is easier to read.
--- kernel/trace/trace_events_synth.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-)
kernel/trace/trace_events_synth.c +4 -13
@@ -350,7 +350,7 @@ static enum print_line_t print_synth_event(struct trace_iterator *iter,struct trace_seq *s = &iter->seq;struct synth_trace_event *entry;struct synth_event *se; -unsigned int i, n_u64; +unsigned int i, j, n_u64;char print_fmt[32 ];const char *fmt;
@@ -389,18 +389,13 @@ static enum print_line_t print_synth_event(struct trace_iterator *iter, n_u64 +=STR_VAR_LEN_MAX /sizeof (u64); } }else if (se->fields[i]->is_stack) { -unsigned long *p, *end;union trace_synth_field *data = &entry->fields[n_u64]; - - p = (void *)entry + data->as_dynamic.offset; - end = (void *)p + data->as_dynamic.len - (sizeof (long ) -1 ); +unsigned long *p = (void *)entry + data->as_dynamic.offset;trace_seq_printf (s,"%s=STACK:\n" , se->fields[i]->name); - -for (; *p && p < end; p++) -trace_seq_printf (s,"=> %pS\n" , (void *)*p); +for (j =1 ; j < data->as_dynamic.len /sizeof (long ); j++) +trace_seq_printf (s,"=> %pS\n" , (void *)p[j]); n_u64++; - }else {struct trace_print_flags __flags[] = { __def_gfpflag_names, {-1 ,NULL } };
@@ -490,10 +485,6 @@ static unsigned int trace_stack(struct synth_trace_event *entry,break ; } -/* Include the zero'd element if it fits */ -if (len <HIST_STACKTRACE_DEPTH ) - len++; - len *=sizeof (long );/* Find the dynamic section to copy the stack into. */
Signature · 1 line
2.39.2
PATCH RESEND v3 3/3 tracing/synthetic: allocate one additional element for size
In reply to Sven SchnelleWhile debugging another issue i noticed that the stack trace contains one invalid entry at the end:
<idle>-0 [008] d..4. 26.484201: wake_lat: pid=0 delta=2629976084 000000009cc24024 stack=STACK:
=> __schedule+0xac6/0x1a98
=> schedule+0x126/0x2c0
=> schedule_timeout+0x150/0x2c0
=> kcompactd+0x9ca/0xc20
=> kthread+0x2f6/0x3d8
=> __ret_from_fork+0x8a/0xe8
=> 0x6b6b6b6b6b6b6b6b
This is because the code failed to add the one element containing the number of entries to field_size.
--- kernel/trace/trace_events_synth.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/trace/trace_events_synth.c +2 -1
@@ -528,7 +528,8 @@ static notrace void trace_event_raw_event_synth(void *__data, str_val = (char *)(long )var_ref_vals[val_idx];if (event->dynamic_fields[i]->is_stack) { - len = *((unsigned long *)str_val); +/* reserve one extra element for size */ + len = *((unsigned long *)str_val) +1 ; len *=sizeof (unsigned long ); }else { len =fetch_store_strlen ((unsigned long )str_val);
Signature · 1 line
2.39.2
Steven Rostedt
maintainerQuoted 4 lines · On Wed, 16 Aug 2023 17:49:25 +0200 Sven Schnelle <[email protected]> wrote:
Hi Steven,
I'm now sending these patches in one patchset, because the second patch has a dependeny on the union vs. cast fix.
Thanks. I'm currently waiting on some other fixes before running them all through my tests before sending them off to Linus. If they are not ready by tomorrow, I'll just kick off my tests without them.
-- Steve
Quoted 18 lines
Changes in v3:
- remove superfluous struct around uniontrace_synth_fieldChanges in v2:
- cosmetic changes
- add structtrace_dynamic_infotoinclude/linux/trace_events.hSven Schnelle (3):
tracing/synthetic: use union instead of casts tracing/synthetic: skip first entry for stack traces tracing/synthetic: allocate one additional element for sizeinclude/linux/trace_events.h | 11 ++++ kernel/trace/trace.h | 8 +++ kernel/trace/trace_events_synth.c | 103 ++++++++++++------------------ 3 files changed, 60 insertions(+), 62 deletions(-)