tracing: Simplify and document the trace event filtering temp buffer code
Steven Rostedt
maintainercover tracing: Simplify and document the trace event filtering temp buffer code
When filtering trace events, a temp buffer is used because the extra copy from the temp buffer into the ring buffer is still faster than the direct write into the ring buffer followed by a discard if the filter does not match.
But the data that can be stored in the temp buffer is a PAGE_SIZE minus the ring buffer event header. The calculation of that header size is complex, but using the helper macro "struct_size()" can simplify it.
Also, add more documentation about what is going on.
Steven Rostedt (VMware) (2):
tracing: Simplify the max length test when using the filtering temp buffer
tracing: Add better comments for the filtering temp buffer use case
----
kernel/trace/trace.c | 40 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-)
Steven Rostedt
maintainerPATCH 1/2 tracing: Simplify the max length test when using the filtering temp buffer
From: "Steven Rostedt (VMware)" <[email protected]>
When filtering trace events, a temp buffer is used because the extra copy from the temp buffer into the ring buffer is still faster than the direct write into the ring buffer followed by a discard if the filter does not match.
But the data that can be stored in the temp buffer is a PAGE_SIZE minus the ring buffer event header. The calculation of that header size is complex, but using the helper macro "struct_size()" can simplify it.
--- kernel/trace/trace.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/trace/trace.c +3 -1
@@ -2737,8 +2737,10 @@ trace_event_buffer_lock_reserve(struct trace_buffer **current_rb, (trace_file->flags & (EVENT_FILE_FL_SOFT_DISABLED | EVENT_FILE_FL_FILTERED)) && (entry = this_cpu_read(trace_buffered_event))) {/* Try to use the per cpu buffer first */ +int max_len =PAGE_SIZE -struct_size (entry, array,1 ); + val =this_cpu_inc_return (trace_buffered_event_cnt); -if ((len < (PAGE_SIZE -sizeof (*entry) -sizeof (entry->array[0 ]))) && val ==1 ) { +if (val ==1 &&unlikely (len < max_len)) {trace_event_setup (entry, type, trace_ctx); entry->array[0 ] = len;return entry;
Signature · 1 line
2.30.2
Steven Rostedt
maintainerPATCH 2/2 tracing: Add better comments for the filtering temp buffer use case
In reply to Steven RostedtFrom: "Steven Rostedt (VMware)" <[email protected]>
When filtering is enabled, the event is copied into a temp buffer instead of being written into the ring buffer directly, because the discarding of events from the ring buffer is very expensive, and doing the extra copy is much faster than having to discard most of the time.
As that logic is subtle, add comments to explain in more detail to what is going on and how it works.
--- kernel/trace/trace.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-)
kernel/trace/trace.c +35 -1
@@ -2736,10 +2736,44 @@ trace_event_buffer_lock_reserve(struct trace_buffer **current_rb,if (!tr->no_filter_buffering_ref && (trace_file->flags & (EVENT_FILE_FL_SOFT_DISABLED |EVENT_FILE_FL_FILTERED )) && (entry =this_cpu_read (trace_buffered_event))) { -/* Try to use the per cpu buffer first */ +/* +* Filtering is on, so try to use the per cpu buffer first. +* This buffer will simulate a ring_buffer_event, +* where the type_len is zero and the array[0] will +* hold the full length. +* (see include/linux/ring-buffer.h for details on +* how the ring_buffer_event is structured). +* +* Using a temp buffer during filtering and copying it +* on a matched filter is quicker than writing directly +* into the ring buffer and then discarding it when +* it doesn't match. That is because the discard +* requires several atomic operations to get right. +* Copying on match and doing nothing on a failed match +* is still quicker than no copy on match, but having +* to discard out of the ring buffer on a failed match. +*/ int max_len =PAGE_SIZE -struct_size (entry, array,1 ); val =this_cpu_inc_return (trace_buffered_event_cnt); + +/* +* Preemption is disabled, but interrupts and NMIs +* can still come in now. If that happens after +* the above increment, then it will have to go +* back to the old method of allocating the event +* on the ring buffer, and if the filter fails, it +* will have to call ring_buffer_discard_commit() +* to remove it. +* +* Need to also check the unlikely case that the +* length is bigger than the temp buffer size. +* If that happens, then the reserve is pretty much +* guaranteed to fail, as the ring buffer currently +* only allows events less than a page. But that may +* change in the future, so let the ring buffer reserve +* handle the failure in that case. +*/ if (val ==1 &&unlikely (len < max_len)) {trace_event_setup (entry, type, trace_ctx); entry->array[0 ] = len;
Signature · 1 line
2.30.2
Re: [PATCH 1/2] tracing: Simplify the max length test when using the filtering temp buffer
In reply to Steven Rostedtkernel/trace/trace.c trace_event_buffer_lock_reserve(struct trace_buffer **current_rb, · line 2743
Quoted 4 lines · Steven Rostedt
val = this_cpu_inc_return(trace_buffered_event_cnt);kernel/trace/trace.c +1
-1-if ((len < (PAGE_SIZE -sizeof (*entry) -sizeof (entry->array[0 ]))) && val ==1 ) { +if (val ==1 &&unlikely (len < max_len)) {
unlikely? No.
Also, I still think that "len < max_len" should actually be "len <= max_len". It should be ok to use the whole page, no?
Unless there is some *other* overflow issue, and "len" doesn't contain a terminating NUL character or something like that.
Linus
Steven Rostedt
maintainerRe: [PATCH 1/2] tracing: Simplify the max length test when using the filtering temp buffer
Quoted 7 lines · On Wed, 9 Jun 2021 15:21:52 -0700 Linus Torvalds…
kernel/trace/trace.c trace_event_buffer_lock_reserve(struct trace_buffer **current_rb, · line 2743
Quoted 4 lines · Steven Rostedt
val = this_cpu_inc_return(trace_buffered_event_cnt);kernel/trace/trace.c +1
-1-if ((len < (PAGE_SIZE -sizeof (*entry) -sizeof (entry->array[0 ]))) && val ==1 ) { +if (val ==1 &&unlikely (len < max_len)) {unlikely? No.
Bah, that was suppose to be "likely()" :-p
Quoted 3 lines
Also, I still think that "len <
max_len" should actually be "len <=max_len". It should be ok to use the whole page, no?
Sure, I was still being paranoid. ;-)
Quoted 4 lines
Unless there is some *other* overflow issue, and "len" doesn't contain a terminating NUL character or something like that.
No, this is kind of like an internal "malloc", where the caller is passing in the length via "len" of what it needs.
-- Steve