Skip to contentSkip to conversation

tracing: Simplify and document the trace event filtering temp buffer code

PATCH Superseded Steven Rostedt 2 participants 5 messages Jun 9, 2021 Tracing

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Steven Rostedt

author maintainer
#0 lore raw

cover tracing: Simplify and document the trace event filtering temp buffer code

When filtering trace events, a temp buffer is used because the extra copy from the temp buffer into the ring buffer is still faster than the direct write into the ring buffer followed by a discard if the filter does not match.

But the data that can be stored in the temp buffer is a PAGE_SIZE minus the ring buffer event header. The calculation of that header size is complex, but using the helper macro "struct_size()" can simplify it.

Also, add more documentation about what is going on.

Steven Rostedt (VMware) (2):
tracing: Simplify the max length test when using the filtering temp buffer
tracing: Add better comments for the filtering temp buffer use case

----

 kernel/trace/trace.c | 40 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 38 insertions(+), 2 deletions(-)
author maintainer
#1 lore raw

PATCH 1/2 tracing: Simplify the max length test when using the filtering temp buffer

From: "Steven Rostedt (VMware)" <[email protected]>

When filtering trace events, a temp buffer is used because the extra copy from the temp buffer into the ring buffer is still faster than the direct write into the ring buffer followed by a discard if the filter does not match.

But the data that can be stored in the temp buffer is a PAGE_SIZE minus the ring buffer event header. The calculation of that header size is complex, but using the helper macro "struct_size()" can simplify it.

---
 kernel/trace/trace.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/trace/trace.c +3 -1
index f6c4d9cf0377..7f198d8959f2 100644
@@ -2737,8 +2737,10 @@ trace_event_buffer_lock_reserve(struct trace_buffer **current_rb,
 	    (trace_file->flags & (EVENT_FILE_FL_SOFT_DISABLED | EVENT_FILE_FL_FILTERED)) &&
 	    (entry = this_cpu_read(trace_buffered_event))) {
 		/* Try to use the per cpu buffer first */
+		int max_len = PAGE_SIZE - struct_size(entry, array, 1);
+
 		val = this_cpu_inc_return(trace_buffered_event_cnt);
-		if ((len < (PAGE_SIZE - sizeof(*entry) - sizeof(entry->array[0]))) && val == 1) {
+		if (val == 1 && unlikely(len < max_len)) {
 			trace_event_setup(entry, type, trace_ctx);
 			entry->array[0] = len;
 			return entry;
Signature · 1 line
2.30.2
author maintainer
#4 lore raw

PATCH 2/2 tracing: Add better comments for the filtering temp buffer use case

In reply to Steven Rostedt

From: "Steven Rostedt (VMware)" <[email protected]>

When filtering is enabled, the event is copied into a temp buffer instead of being written into the ring buffer directly, because the discarding of events from the ring buffer is very expensive, and doing the extra copy is much faster than having to discard most of the time.

As that logic is subtle, add comments to explain in more detail to what is going on and how it works.

---
 kernel/trace/trace.c | 36 +++++++++++++++++++++++++++++++++++-
 1 file changed, 35 insertions(+), 1 deletion(-)
kernel/trace/trace.c +35 -1
index 7f198d8959f2..6a83ab5c5147 100644
@@ -2736,10 +2736,44 @@ trace_event_buffer_lock_reserve(struct trace_buffer **current_rb,
 	if (!tr->no_filter_buffering_ref &&
 	    (trace_file->flags & (EVENT_FILE_FL_SOFT_DISABLED | EVENT_FILE_FL_FILTERED)) &&
 	    (entry = this_cpu_read(trace_buffered_event))) {
-		/* Try to use the per cpu buffer first */
+		/*
+		 * Filtering is on, so try to use the per cpu buffer first.
+		 * This buffer will simulate a ring_buffer_event,
+		 * where the type_len is zero and the array[0] will
+		 * hold the full length.
+		 * (see include/linux/ring-buffer.h for details on
+		 *  how the ring_buffer_event is structured).
+		 *
+		 * Using a temp buffer during filtering and copying it
+		 * on a matched filter is quicker than writing directly
+		 * into the ring buffer and then discarding it when
+		 * it doesn't match. That is because the discard
+		 * requires several atomic operations to get right.
+		 * Copying on match and doing nothing on a failed match
+		 * is still quicker than no copy on match, but having
+		 * to discard out of the ring buffer on a failed match.
+		 */
 		int max_len = PAGE_SIZE - struct_size(entry, array, 1);
 
 		val = this_cpu_inc_return(trace_buffered_event_cnt);
+
+		/*
+		 * Preemption is disabled, but interrupts and NMIs
+		 * can still come in now. If that happens after
+		 * the above increment, then it will have to go
+		 * back to the old method of allocating the event
+		 * on the ring buffer, and if the filter fails, it
+		 * will have to call ring_buffer_discard_commit()
+		 * to remove it.
+		 *
+		 * Need to also check the unlikely case that the
+		 * length is bigger than the temp buffer size.
+		 * If that happens, then the reserve is pretty much
+		 * guaranteed to fail, as the ring buffer currently
+		 * only allows events less than a page. But that may
+		 * change in the future, so let the ring buffer reserve
+		 * handle the failure in that case.
+		 */
 		if (val == 1 && unlikely(len < max_len)) {
 			trace_event_setup(entry, type, trace_ctx);
 			entry->array[0] = len;
Signature · 1 line
2.30.2
#2 lore raw

Re: [PATCH 1/2] tracing: Simplify the max length test when using the filtering temp buffer

In reply to Steven Rostedt

kernel/trace/trace.c trace_event_buffer_lock_reserve(struct trace_buffer **current_rb, · line 2743

Quoted 4 lines · Steven Rostedt
                val = this_cpu_inc_return(trace_buffered_event_cnt);
kernel/trace/trace.c +1 -1
-               if ((len < (PAGE_SIZE - sizeof(*entry) - sizeof(entry->array[0]))) && val == 1) {
+               if (val == 1 && unlikely(len < max_len)) {

unlikely? No.

Also, I still think that "len < max_len" should actually be "len <= max_len". It should be ok to use the whole page, no?

Unless there is some *other* overflow issue, and "len" doesn't contain a terminating NUL character or something like that.

Linus

author maintainer
#3 lore raw

Re: [PATCH 1/2] tracing: Simplify the max length test when using the filtering temp buffer

Quoted 7 lines · On Wed, 9 Jun 2021 15:21:52 -0700 Linus Torvalds…

kernel/trace/trace.c trace_event_buffer_lock_reserve(struct trace_buffer **current_rb, · line 2743

Quoted 4 lines · Steven Rostedt
                val = this_cpu_inc_return(trace_buffered_event_cnt);
kernel/trace/trace.c +1 -1
-               if ((len < (PAGE_SIZE - sizeof(*entry) - sizeof(entry->array[0]))) && val == 1) {
+               if (val == 1 && unlikely(len < max_len)) {  

unlikely? No.

Bah, that was suppose to be "likely()" :-p

Quoted 3 lines

Also, I still think that "len < max_len" should actually be "len <= max_len". It should be ok to use the whole page, no?

Sure, I was still being paranoid. ;-)

Quoted 4 lines

Unless there is some *other* overflow issue, and "len" doesn't contain a terminating NUL character or something like that.

No, this is kind of like an internal "malloc", where the caller is passing in the length via "len" of what it needs.

-- Steve