tracing: trace_puts() and faster trace_printk()
Steven Rostedt
maintainercover tracing: trace_puts() and faster trace_printk()
I thought I was done with my ftrace 3.10 patches, but oh well. There's a few more things that I want to do for 3.10.
I noticed that I used trace_printk() in my tracing_snapshot() in case it was used incorrectly, but then I realized that those caused the kernel to preallocate the trace_printk() context buffers all the time (1 page per context [4 of them], and per cpu). This was not what I wanted. Thus is started creating a special "internal to ftrace" use that would not need the context buffers. I added a trace_puts() for this and then realized that this would be nice for others to use as trace_printk(), even when doing a trace_bprintk(), still requires a scan of the fmt string when there are no arguments in use.
Then I thought it would be nice if the compiler just picked trace_puts() when trace_printk() had no args. Thus, I pulled out my old macro wizard robe, hat and wand and started playing again. And I came up with patch #2. :-)
These 4 patches are specific for using ftrace by developers, as the functions an not called anywhere and are only to be used when a developer is debugging their code.
Enjoy,
-- Steve
Steven Rostedt (Red Hat) (4):
tracing: Add trace_puts() for even faster trace_printk() tracing
tracing: Optimize trace_printk() with one arg to use trace_puts()
tracing: Add internal ftrace trace_puts() for ftrace to use
tracing: Let tracing_snapshot() be used by modules but not NMI
----
include/linux/kernel.h | 64 +++++++++++++++++++++++++++- kernel/trace/trace.c | 94 ++++++++++++++++++++++++++++++++++++++++-- kernel/trace/trace.h | 13 ++++++ kernel/trace/trace_entries.h | 23 +++++++++-- kernel/trace/trace_output.c | 75 +++++++++++++++++++++++++++++++++ kernel/trace/trace_output.h | 2 + 6 files changed, 261 insertions(+), 10 deletions(-)
Steven Rostedt
maintainerPATCH 1/4 tracing: Add trace_puts() for even faster trace_printk() tracing
From: "Steven Rostedt (Red Hat)" <[email protected]>
The trace_printk() is extremely fast and is very handy as it can be used in any context (including NMIs!). But it still requires scanning the fmt string for parsing the args. Even the trace_bprintk() requires a scan to know what args will be saved, although it doesn't copy the format string itself.
Several times trace_printk() has no args, and wastes cpu cycles scanning the fmt string.
Adding trace_puts() allows the developer to use an even faster tracing method that only saves the pointer to the string in the ring buffer without doing any format parsing at all. This will help remove even more of the "Heisenbug" effect, when debugging.
Also fixed up the F_printk()s for the ftrace internal bprint and print events.
--- include/linux/kernel.h | 41 ++++++++++++++++++++++- kernel/trace/trace.c | 76 ++++++++++++++++++++++++++++++++++++++++++ kernel/trace/trace.h | 2 ++ kernel/trace/trace_entries.h | 23 ++++++++++--- kernel/trace/trace_output.c | 75 +++++++++++++++++++++++++++++++++++++++++ kernel/trace/trace_output.h | 2 ++ 6 files changed, 214 insertions(+), 5 deletions(-)
include/linux/kernel.h +40 -1
@@ -514,7 +514,8 @@ do { \* * This is intended as a debugging tool for the developer only. * Please refrain from leaving trace_printks scattered around in -* your code. +* your code. (Extra memory is used for special buffers that are +* allocated when trace_printk() is used) */ #define trace_printk(fmt, args...) \
@@ -537,6 +538,44 @@ int __trace_bprintk(unsigned long ip, const char *fmt, ...); extern __printf(2, 3) int __trace_printk(unsigned long ip, const char *fmt, ...); +/** +* trace_puts - write a string into the ftrace buffer +* @str: the string to record +* +* Note: __trace_bputs is an internal function for trace_puts and +* the @ip is passed in via the trace_puts macro. +* +* This is similar to trace_printk() but is made for those really fast +* paths that a developer wants the least amount of "Heisenbug" affects, +* where the processing of the print format is still too much. +* +* This function allows a kernel developer to debug fast path sections +* that printk is not appropriate for. By scattering in various +* printk like tracing in the code, a developer can quickly see +* where problems are occurring. +* +* This is intended as a debugging tool for the developer only. +* Please refrain from leaving trace_puts scattered around in +* your code. (Extra memory is used for special buffers that are +* allocated when trace_puts() is used) +* +* Returns: 0 if nothing was written, positive # if string was. +* (1 when __trace_bputs is used, strlen(str) when __trace_puts is used) +*/ + +extern int __trace_bputs(unsigned long ip, const char *str); +extern int __trace_puts (unsigned long ip,const char *str,int size); +#define trace_puts (str) ({ \ + static const char *trace_printk_fmt \ + __attribute__((section("__trace_printk_fmt"))) = \ + __builtin_constant_p(str) ? str : NULL; \ + \ + if (__builtin_constant_p(str)) \ + __trace_bputs(_THIS_IP_, trace_printk_fmt); \ + else \ + __trace_puts(_THIS_IP_, str, strlen(str)); \ +}) +extern void trace_dump_stack (void );/*
kernel/trace/trace.c +76 -0
88 lines not on this page · raw message
kernel/trace/trace.h +2 -0
@@ -34,6 +34,7 @@ enum trace_type {TRACE_GRAPH_ENT ,TRACE_USER_STACK ,TRACE_BLK , +TRACE_BPUTS , __TRACE_LAST_TYPE, };
@@ -277,6 +278,7 @@ extern void __ftrace_bad_type(void); IF_ASSIGN(var, ent, struct userstack_entry, TRACE_USER_STACK);\ IF_ASSIGN(var, ent, struct print_entry, TRACE_PRINT); \ IF_ASSIGN(var, ent, struct bprint_entry, TRACE_BPRINT); \ + IF_ASSIGN(var, ent, struct bputs_entry, TRACE_BPUTS); \ IF_ASSIGN(var, ent, struct trace_mmiotrace_rw, \TRACE_MMIO_RW ); \ IF_ASSIGN(var, ent, struct trace_mmiotrace_map, \
kernel/trace/trace_entries.h +19 -4
@@ -223,8 +223,8 @@ FTRACE_ENTRY(bprint, bprint_entry,__dynamic_array ( u32, buf ) ), -F_printk ("%08lx fmt:%p" , - __entry->ip, __entry->fmt), +F_printk ("%pf: %s" , + (void *)__entry->ip, __entry->fmt),FILTER_OTHER );
@@ -238,8 +238,23 @@ FTRACE_ENTRY(print, print_entry,__dynamic_array ( char, buf ) ), -F_printk ("%08lx %s" , - __entry->ip, __entry->buf), +F_printk ("%pf: %s" , + (void *)__entry->ip, __entry->buf), + +FILTER_OTHER +); + +FTRACE_ENTRY (bputs, bputs_entry, + +TRACE_BPUTS , + +F_STRUCT ( + __field( unsigned long, ip ) + __field( const char *, str ) + ), + +F_printk ("%pf: %s" , + (void *)__entry->ip, __entry->str),FILTER_OTHER );
kernel/trace/trace_output.c +75 -0
93 lines not on this page · raw message
kernel/trace/trace_output.h +2 -0
@@ -5,6 +5,8 @@#include "trace.h" extern enum print_line_t +trace_print_bputs_msg_only (struct trace_iterator *iter); +extern enum print_line_t trace_print_bprintk_msg_only (struct trace_iterator *iter);extern enum print_line_t trace_print_printk_msg_only (struct trace_iterator *iter);
Signature · 1 line
1.7.10.4
Steven Rostedt
maintainerPATCH 2/4 tracing: Optimize trace_printk() with one arg to use trace_puts()
In reply to Steven RostedtFrom: "Steven Rostedt (Red Hat)" <[email protected]>
Although trace_printk() is extremely fast, especially when it uses trace_bprintk() (writes args straight to buffer instead of inserting into string), it still has the overhead of calling one of the printf sprintf() functions, that need to scan the fmt string to determine what, if any args it has.
This is a waste of precious CPU cycles if the printk format has no args but a single constant string. It is better to use trace_puts() which does not have the overhead of the fmt scanning.
But wouldn't it be nice if the developer didn't have to think about such things, and the compile would just do it for them?
trace_printk("this string has no args\n");
[...]
trace_printk("this sting does %p %d\n", foo, bar);As tracing is critical to have the least amount of overhead, especially when dealing with race conditions, and you want to eliminate any "Heisenbugs", you want the trace_printk() to use the fastest possible means of tracing.
Currently the macro magic determines if it will use trace_bprintk() or if the fmt is a dynamic string (a variable), it will fall back to the slow trace_printk() method that does a full snprintf() before copying it into the buffer, where as trace_bprintk() only copys the pointer to the fmt and the args into the buffer.
Well, now there's a way to spend some more Hogwarts cash and come up with new fancy macro magic.
#define trace_printk(fmt, ...) \
do { \
char _______STR[] = __stringify((__VA_ARGS__)); \
if (sizeof(_______STR) > 3) \
do_trace_printk(fmt, ##__VA_ARGS__); \
else \
trace_puts(fmt); \} while (0)
The above needs a bit of explaining (both here and in the comments).
By stringifying the __VA_ARGS__, we can, at compile time, determine the number of args that are being passed to trace_printk(). The extra parenthesis are required, otherwise the compiler complains about too many parameters for __stringify if there is more than one arg.
When there are no args, the __stringify((__VA_ARGS__)) converts into "()\0", a string of 3 characters. Anything else, will be a string containing more than 3 characters. Now we assign that string to a dynamic char array, and then take the sizeof() of that array. If it is greater than 3 characters, we know trace_printk() has args and we need to do the full "do_trace_printk()" on them, otherwise it was only passed a single arg and we can optimize to use trace_puts().
--- include/linux/kernel.h | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)
include/linux/kernel.h +22 -1
@@ -516,9 +516,30 @@ do { \* Please refrain from leaving trace_printks scattered around in * your code. (Extra memory is used for special buffers that are * allocated when trace_printk() is used) +* +* A little optization trick is done here. If there's only one +* argument, there's no need to scan the string for printf formats. +* The trace_puts() will suffice. But how can we take advantage of +* using trace_puts() when trace_printk() has only one argument? +* By stringifying the args and checking the size we can tell +* whether or not there are args. __stringify((__VA_ARGS__)) will +* turn into "()\0" with a size of 3 when there are no args, anything +* else will be bigger. All we need to do is define a string to this, +* and then take its size and compare to 3. If it's bigger, use +* do_trace_printk() otherwise, optimize it to trace_puts(). Then just +* let gcc optimize the rest. */ -#define trace_printk(fmt, args...) \ +#define trace_printk (fmt, ...) \ +do { \ + char _______STR[] = __stringify((__VA_ARGS__)); \ + if (sizeof(_______STR) > 3) \ + do_trace_printk(fmt, ##__VA_ARGS__); \ + else \ + trace_puts(fmt); \ +} while (0) + +#define do_trace_printk(fmt, args...) \ do { \ static const char *trace_printk_fmt \ __attribute__((section("__trace_printk_fmt"))) = \
Signature · 1 line
1.7.10.4
Steven Rostedt
maintainerPATCH 3/4 tracing: Add internal ftrace trace_puts() for ftrace to use
In reply to Steven RostedtFrom: "Steven Rostedt (Red Hat)" <[email protected]>
There's a few places that ftrace uses trace_printk() for internal use, but this requires context (normal, softirq, irq, NMI) buffers to keep things lockless. But the trace_puts() does not, as it can write the string directly into the ring buffer. Make a internal helper for trace_puts() and have the internal functions use that.
This way the extra context buffers are not used.
--- kernel/trace/trace.c | 8 ++++---- kernel/trace/trace.h | 11 +++++++++++ 2 files changed, 15 insertions(+), 4 deletions(-)
kernel/trace/trace.c +4 -4
@@ -443,16 +443,16 @@ void tracing_snapshot(void)unsigned long flags;if (!tr->allocated_snapshot) { -trace_printk ("*** SNAPSHOT NOT ALLOCATED ***\n" ); -trace_printk ("*** stopping trace here! ***\n" ); +internal_trace_puts ("*** SNAPSHOT NOT ALLOCATED ***\n" ); +internal_trace_puts ("*** stopping trace here! ***\n" );tracing_off ();return ; }/* Note, snapshot can not be used when the tracer uses it */ if (tracer->use_max_tr) { -trace_printk ("*** LATENCY TRACER ACTIVE ***\n" ); -trace_printk ("*** Can not use snapshot (sorry) ***\n" ); +internal_trace_puts ("*** LATENCY TRACER ACTIVE ***\n" ); +internal_trace_puts ("*** Can not use snapshot (sorry) ***\n" );return ; }
kernel/trace/trace.h +11 -0
@@ -1034,6 +1034,17 @@ extern const char *__stop___trace_bprintk_fmt[];void trace_printk_init_buffers (void );void trace_printk_start_comm (void ); +/* +* Normal trace_printk() and friends allocates special buffers +* to do the manipulation, as well as saves the print formats +* into sections to display. But the trace infrastructure wants +* to use these without the added overhead at the price of being +* a bit slower (used mainly for warnings, where we don't care +* about performance). The internal_trace_puts() is for such +* a purpose. +*/ +#define internal_trace_puts (str) __trace_puts(_THIS_IP_, str, strlen(str)) +#undef FTRACE_ENTRY#define FTRACE_ENTRY (call, struct_name, id, tstruct, print, filter) \ extern struct ftrace_event_call \
Signature · 1 line
1.7.10.4
Steven Rostedt
maintainerPATCH 4/4 tracing: Let tracing_snapshot() be used by modules but not NMI
In reply to Steven RostedtFrom: "Steven Rostedt (Red Hat)" <[email protected]>
Add EXPORT_SYMBOL_GPL() to let the tracing_snapshot() functions be called from modules.
Also add a test to see if the snapshot was called from NMI context and just warn in the tracing buffer if so, and return.
--- kernel/trace/trace.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
kernel/trace/trace.c +10 -0
@@ -442,6 +442,12 @@ void tracing_snapshot(void)struct tracer *tracer = tr->current_trace;unsigned long flags; +if (in_nmi ()) { +internal_trace_puts ("*** SNAPSHOT CALLED FROM NMI CONTEXT ***\n" ); +internal_trace_puts ("*** snapshot is being ignored ***\n" ); +return ; + } +if (!tr->allocated_snapshot) {internal_trace_puts ("*** SNAPSHOT NOT ALLOCATED ***\n" );internal_trace_puts ("*** stopping trace here! ***\n" );
@@ -460,6 +466,7 @@ void tracing_snapshot(void)update_max_tr (tr, current,smp_processor_id ());local_irq_restore (flags); } +EXPORT_SYMBOL_GPL (tracing_snapshot);static int resize_buffer_duplicate_size (struct trace_buffer *trace_buf,struct trace_buffer *size_buf,int cpu_id);
@@ -493,16 +500,19 @@ void tracing_snapshot_alloc(void) tracing_snapshot(); } +EXPORT_SYMBOL_GPL (tracing_snapshot_alloc);#else void tracing_snapshot (void ) {WARN_ONCE (1 ,"Snapshot feature not enabled, but internal snapshot used" ); } +EXPORT_SYMBOL_GPL (tracing_snapshot);void tracing_snapshot_alloc (void ) {/* Give warning */ tracing_snapshot (); } +EXPORT_SYMBOL_GPL (tracing_snapshot_alloc);#endif /* CONFIG_TRACER_SNAPSHOT */ /**
Signature · 1 line
1.7.10.4