tracing/osnoise: Fix crash in timerlat_dump_stack()
Subsystems (2)
PATCH 1/1 tracing/osnoise: Fix crash in timerlat_dump_stack()
We have observed kernel panics when using timerlat with stack saving, with the following dmesg output:
memcpy: detected buffer overflow: 88 byte write of buffer size 0
WARNING: CPU: 2 PID: 8153 at lib/string_helpers.c:1032 __fortify_report+0x55/0xa0 CPU: 2 UID: 0 PID: 8153 Comm: timerlatu/2 Kdump: loaded Not tainted 6.15.3-200.fc42.x86_64 #1 PREEMPT(lazy) Call Trace: <TASK> ? trace_buffer_lock_reserve+0x2a/0x60 __fortify_panic+0xd/0xf __timerlat_dump_stack.cold+0xd/0xd timerlat_dump_stack.part.0+0x47/0x80 timerlat_fd_read+0x36d/0x390 vfs_read+0xe2/0x390 ? syscall_exit_to_user_mode+0x1d5/0x210 ksys_read+0x73/0xe0 do_syscall_64+0x7b/0x160 ? exc_page_fault+0x7e/0x1a0 entry_SYSCALL_64_after_hwframe+0x76/0x7e
__timerlat_dump_stack() constructs the ftrace stack entry like this:
struct stack_entry *entry;
...memcpy(&entry->caller, fstack->calls, size);entry->size = fstack->nr_entries;
Since commit e7186af7fb26 ("tracing: Add back FORTIFY_SOURCE logic to kernel_stack event structure"), struct stack_entry marks its caller field with __counted_by(size). At the time of the memcpy, entry->size contains garbage from the ringbuffer, which under some circumstances is zero, triggering a kernel panic by buffer overflow.
Populate the size field before the memcpy so that the out-of-bounds check knows the correct size. This is analogous to __ftrace_trace_stack().
--- kernel/trace/trace_osnoise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Note: This has been so far only reproduced on laptops (three different machines). Not sure what the reason for that is, but it is clearly a bug.
kernel/trace/trace_osnoise.c +1 -1
@@ -637,8 +637,8 @@ __timerlat_dump_stack(struct trace_buffer *buffer, struct trace_stack *fstack, u entry =ring_buffer_event_data (event); -memcpy (&entry->caller, fstack->calls, size); entry->size = fstack->nr_entries; +memcpy (&entry->caller, fstack->calls, size);trace_buffer_unlock_commit_nostack (buffer, event); }
Signature · 1 line
2.47.1
Quoted 33 lines · On Wed, 16 Jul 2025 16:36:01 +0200 Tomas Glozar <[email protected]> wrote:
We have observed kernel panics when using timerlat with stack saving, with the following dmesg output:
memcpy: detected buffer overflow: 88 byte write of buffer size 0
WARNING: CPU: 2 PID: 8153 at lib/string_helpers.c:1032 __fortify_report+0x55/0xa0 CPU: 2 UID: 0 PID: 8153 Comm: timerlatu/2 Kdump: loaded Not tainted 6.15.3-200.fc42.x86_64 #1 PREEMPT(lazy) Call Trace: <TASK> ? trace_buffer_lock_reserve+0x2a/0x60 __fortify_panic+0xd/0xf __timerlat_dump_stack.cold+0xd/0xd timerlat_dump_stack.part.0+0x47/0x80 timerlat_fd_read+0x36d/0x390 vfs_read+0xe2/0x390 ? syscall_exit_to_user_mode+0x1d5/0x210 ksys_read+0x73/0xe0 do_syscall_64+0x7b/0x160 ? exc_page_fault+0x7e/0x1a0 entry_SYSCALL_64_after_hwframe+0x76/0x7e
__timerlat_dump_stack()constructs the ftrace stack entry like this:struct
stack_entry*entry;
...memcpy(&entry->caller, fstack->calls, size);entry->size=fstack->nr_entries;Since commit
e7186af7fb26("tracing: Add backFORTIFY_SOURCElogic tokernel_stackevent structure"), structstack_entrymarks its caller field with__counted_by(size). At the time of the memcpy,entry->sizecontains garbage from the ringbuffer, which under some circumstances is zero, triggering a kernel panic by buffer overflow.
Quoted 4 lines
Populate the size field before the memcpy so that the out-of-bounds check knows the correct size. This is analogous to
__ftrace_trace_stack().
Looks good to me.
Thank you!
Quoted 29 lines
Fixes e7186af7fb26 ("tracing: Add back FORTIFY_SOURCE logic to kernel_stack event structure")Signed-off-by Tomas Glozar <[email protected]>--- kernel/trace/trace_osnoise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Note: This has been so far only reproduced on laptops (three different machines). Not sure what the reason for that is, but it is clearly a bug.
kernel/trace/trace_osnoise.c +1
-1@@ -637,8 +637,8 @@ __timerlat_dump_stack(struct trace_buffer *buffer, struct trace_stack *fstack, u entry =ring_buffer_event_data (event); -memcpy (&entry->caller, fstack->calls, size); entry->size = fstack->nr_entries; +memcpy (&entry->caller, fstack->calls, size);trace_buffer_unlock_commit_nostack (buffer, event); }Signature · 1 line
2.47.1
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>