tracing: Add conditional to tracepoints
Steven Rostedt
maintainercover tracing: Add conditional to tracepoints
This is an RFC that adds the TP_CONDITION() to the TRACE_EVENT() code.
There are certain cases that a tracepoint only makes sense if a specific condition is met. But because we do not want to dirty the fast path (the non tracing case) with if statements that are there only to avoid tracing, we just pass the condition variables to the tracepoint, and let the user filter them out if needed.
A perfect example is the tracepoint sched_wakeup. It traces all calls to try_to_wake_up() even if it fails to wake up. But if we add:
if (success) trace_sched_wakeup(p);
We have that "if (success)" tested for every time we call try_to_wake_up(). Even when tracing is not (or never will be) enabled.
This patch set adds a variant TRACE_EVENT_CONDITIONAL()
(and DECLARE_EVENT_CLASS_CONDITIONAL()) that has a "cond" argument. This argument is encapsulated with "TP_CONDITIONAL()" which turns into:
if (!cond) return;
This is placed inside the called tracepoint routine, and is only tested when the trace is enabled. Otherwise it is a nop as tracepoints normally are when disabled.
Note, another variant of this, is to move the test directly into the _DO_TRACE() macro, and not call any registered event callbacks. This would even speed it up faster when tracing is enabled. I did not do this orginially because I just thought of it now as I wrote this change log ;-) I'm posting this version now just in case people prefer it instead.
The following patches are in:
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
branch: rfc/trace-conditional
Steven Rostedt (2):
tracing: Add TRACE_EVENT_CONDITIONAL()
tracing: Only trace sched_wakeup if it actually work something up
----
include/linux/tracepoint.h | 3 ++ include/trace/define_trace.h | 6 +++ include/trace/events/sched.h | 12 +++--- include/trace/ftrace.h | 70 +++++++++++++++++++++++++++++++---------- 4 files changed, 68 insertions(+), 23 deletions(-)
Steven Rostedt
maintainerRFC PATCH 1/2 tracing: Add TRACE_EVENT_CONDITIONAL()
From: Steven Rostedt <[email protected]>
There are instances in the kernel that we only want to trace a tracepoint when a certain condition is set. But we do not want to test for that condition in the core kernel.
If we test for that condition before calling the tracepoin, then we will be performing that test even when tracing is not enabled. This is 99.99% of the time.
We currently can just filter out on that condition, but that happens after we write to the trace buffer. We just wasted time writing to the ring buffer for an event we never cared about.
This patch adds:
TRACE_EVENT_CONDITION() and DECLARE_TRACE_CLASS_CONDITION()
These have a new TP_CONDITION() argument that comes right after the TP_ARGS(). This condition can use the parameters of the TRACE_EVENT() to determine if the tracepoint should be traced or not. The TP_CONDITION() will be placed in a if (cond) trace;
For example, for the tracepoint sched_wakeup, it is useless to trace an wakeup event where the caller never actually work anything up (success = 0). So adding:
TP_CONDITION(success),
which uses the "success" parameter of the wakeup tracepoint will have it only trace when we have successfully woken up a task.
--- include/linux/tracepoint.h | 3 ++ include/trace/define_trace.h | 6 +++ include/trace/events/sched.h | 2 +- include/trace/ftrace.h | 70 +++++++++++++++++++++++++++++++---------- 4 files changed, 63 insertions(+), 18 deletions(-)
include/linux/tracepoint.h +3 -0
@@ -344,12 +344,15 @@ do_trace: \* TRACE_EVENT_FN to perform any (un)registration work. */ +#define DECLARE_EVENT_CLASS_CONDITION (name, proto, args, cond, tstruct, assign, print)#define DECLARE_EVENT_CLASS (name, proto, args, tstruct, assign, print)#define DEFINE_EVENT (template, name, proto, args) \ DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))#define DEFINE_EVENT_PRINT (template, name, proto, args, print) \ DECLARE_TRACE(name, PARAMS(proto), PARAMS(args)) +#define TRACE_EVENT_CONDITION (name, proto, args, cond, struct, assign, print) \ + DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))#define TRACE_EVENT (name, proto, args, struct, assign, print) \ DECLARE_TRACE(name, PARAMS(proto), PARAMS(args)) #define TRACE_EVENT_FN(name, proto, args, struct, \
include/trace/define_trace.h +6 -0
@@ -26,6 +26,10 @@#define TRACE_EVENT (name, proto, args, tstruct, assign, print) \ DEFINE_TRACE(name) +#undef TRACE_EVENT_CONDITION +#define TRACE_EVENT_CONDITION (name, proto, args, cond, tstruct, assign, print) \ + DEFINE_TRACE(name) +#undef TRACE_EVENT_FN#define TRACE_EVENT_FN (name, proto, args, tstruct, \ assign, print, reg, unreg) \
@@ -75,7 +79,9 @@#undef TRACE_EVENT#undef TRACE_EVENT_FN +#undef TRACE_EVENT_CONDITION#undef DECLARE_EVENT_CLASS +#undef DECLARE_EVENT_CLASS_CONDITION#undef DEFINE_EVENT#undef DEFINE_EVENT_PRINT#undef TRACE_HEADER_MULTI_READ
include/trace/events/sched.h +1 -1
@@ -10,7 +10,7 @@/* * Tracepoint for calling kthread_stop, performed to end a kthread: */ -TRACE_EVENT(sched_kthread_stop, +TRACE_EVENT_CONDITION(sched_kthread_stop, TP_PROTO(struct task_struct *t),
include/trace/ftrace.h +53 -17
143 lines not on this page · raw message
Signature · 1 line
1.7.2.3
Re: [RFC][PATCH 1/2] [PATCH 1/2] tracing: Add TRACE_EVENT_CONDITIONAL()
Quoted 17 lines · Steven Rostedt
From: Steven Rostedt<[email protected]>
There are instances in the kernel that we only want to trace a tracepoint when a certain condition is set. But we do not want to test for that condition in the core kernel.
If we test for that condition before calling the tracepoin, then we will be performing that test even when tracing is not enabled. This is 99.99% of the time.We currently can just filter out on that condition, but that happens after we write to the trace buffer. We just wasted time writing to the ring buffer for an event we never cared about.
This patch adds:
TRACE_EVENT_CONDITION()andDECLARE_TRACE_CLASS_CONDITION()
Quoted 8 lines
These have a new
TP_CONDITION()argument that comes right after theTP_ARGS(). This condition can use the parameters of theTRACE_EVENT()to determine if the tracepoint should be traced or not. TheTP_CONDITION()will be placed in a if (cond) trace;For example, for the tracepoint
sched_wakeup, it is useless to trace an wakeup event where the caller never actually work anything up (success = 0). So adding:
^^^
s/=/==/As much as I hate to be a pedant, I would suggest changing this change log for the sake of clarity.
David Daney
Steven Rostedt
maintainerRe: [RFC][PATCH 1/2] [PATCH 1/2] tracing: Add TRACE_EVENT_CONDITIONAL()
Quoted 10 lines · David Daney
Quoted 3 lines · Steven Rostedt
For example, for the tracepoint
sched_wakeup, it is useless to trace an wakeup event where the caller never actually work anything up (success = 0). So adding:^^^ s/=/==/As much as I hate to be a pedant, I would suggest changing this change log for the sake of clarity.
Eek! Yeah I'll fix that, along with s/work anything/woke anything/.
Thanks,
-- Steve
Steven Rostedt
maintainerRFC PATCH 2/2 tracing: Only trace sched_wakeup if it actually work something up
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Currently the tracepoint sched_wakeup traces the wakeup event even if the wakeup failed to wake anything up. This is quite stupid but it happens because we did not want to add a conditional to the core kernel code that would just slow down the wakeup events.
This patch changes the wakeup tracepoints to use the
DECLARE_EVENT_CLASS_CONDITIONAL()
to test the "success" parameter and will only trace the event if the wakeup was successfull.
The success field in the tracepoint is removed since it is no longer needed.
--- include/trace/events/sched.h | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-)
include/trace/events/sched.h +5 -5
@@ -52,17 +52,18 @@ TRACE_EVENT(sched_kthread_stop_ret,/* * Tracepoint for waking up a task: */ -DECLARE_EVENT_CLASS(sched_wakeup_template, +DECLARE_EVENT_CLASS_CONDITION(sched_wakeup_template, TP_PROTO(struct task_struct *p, int success), TP_ARGS(p, success), + TP_CONDITION(success), + TP_STRUCT__entry( __array( char, comm, TASK_COMM_LEN ) __field( pid_t, pid ) __field( int, prio ) - __field( int, success ) __field( int, target_cpu ) ),
@@ -70,13 +71,12 @@ DECLARE_EVENT_CLASS(sched_wakeup_template,memcpy (__entry->comm, p->comm,TASK_COMM_LEN ); __entry->pid = p->pid; __entry->prio = p->prio; - __entry->success = success; __entry->target_cpu =task_cpu (p); ), -TP_printk ("comm=%s pid=%d prio=%d success=%d target_cpu=%03d" , +TP_printk ("comm=%s pid=%d prio=%d target_cpu=%03d" , __entry->comm, __entry->pid, __entry->prio, - __entry->success, __entry->target_cpu) + __entry->target_cpu) ); DEFINE_EVENT(sched_wakeup_template, sched_wakeup,
Signature · 1 line
1.7.2.3
Quoted 25 lines · Steven Rostedt
This is an RFC that adds the
TP_CONDITION()to theTRACE_EVENT()code.There are certain cases that a tracepoint only makes sense if a specific condition is met. But because we do not want to dirty the fast path (the non tracing case) with if statements that are there only to avoid tracing, we just pass the condition variables to the tracepoint, and let the user filter them out if needed.
A perfect example is the tracepoint
sched_wakeup. It traces all calls totry_to_wake_up()even if it fails to wake up. But if we add:if (success) trace_sched_wakeup(p);We have that "if (success)" tested for every time we call
try_to_wake_up(). Even when tracing is not (or never will be) enabled.This patch set adds a variant
TRACE_EVENT_CONDITIONAL()
(andDECLARE_EVENT_CLASS_CONDITIONAL()) that has a "cond" argument. This argument is encapsulated with "TP_CONDITIONAL()" which turns into:if (!cond) return;
Quoted 8 lines
This is placed inside the called tracepoint routine, and is only tested when the trace is enabled. Otherwise it is a nop as tracepoints normally are when disabled.
Note, another variant of this, is to move the test directly into the
_DO_TRACE()macro, and not call any registered event callbacks. This would even speed it up faster when tracing is enabled. I did not do this orginially because I just thought of it now as I wrote this change log ;-)
Hehe :)
Yeah indeed. And that looks fairly possible.
Quoted 19 lines
I'm posting this version now just in case people prefer it instead.
The following patches are in:
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.gitbranch: rfc/trace-conditional
Steven Rostedt (2):
tracing: AddTRACE_EVENT_CONDITIONAL()
tracing: Only tracesched_wakeupif it actually work something up----
include/linux/tracepoint.h | 3 ++ include/trace/define_trace.h | 6 +++ include/trace/events/sched.h | 12 +++--- include/trace/ftrace.h | 70 +++++++++++++++++++++++++++++++---------- 4 files changed, 68 insertions(+), 23 deletions(-)
Quoted 10 lines · * Frederic Weisbecker ([email protected]) wrote:
Quoted 5 lines · Steven Rostedt
Note, another variant of this, is to move the test directly into the
_DO_TRACE()macro, and not call any registered event callbacks. This would even speed it up faster when tracing is enabled. I did not do this orginially because I just thought of it now as I wrote this change log ;-)Hehe :)
Yeah indeed. And that looks fairly possible.
I'd very much prefer if the test is performed before the call, within the block that contains the stack setup and the tracepoint function call. Having an utterly low performance impact for the events that are filtered out is very important for my client's use-cases. Also, moving it outside of the tracepoint probe function allows us to filter only once for all the registered handlers.
All it would require is to skip over the function call rather than doing a "return".
For the rest, it looks nice. :-)
Thanks,
Mathieu
Signature · 4 lines
Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com