for tip/tracing/ftrace
Subsystems (3)
Steven Rostedt
maintainerIngo,
This is a patch series that offers a variety of features.
The first is to add:
#### all functions enabled ####
to set_ftrace_filter when all functions will be traced when function tracing is on.
The next is to add a :command: interface, and the module command:
echo '*:mod:ext4' > set_ftrace_filter
Will select all functions in the ext4 module.
echo '!*write*:mod:ext4' > set_ftrace_filter
will remove all functions with the characters 'write' and are in ext4.
Next we extend the :command: interface as well as add a way to hook tracers to specific functions. Instead of tracing all functions the same, this patch series adds a hash to be able to trace different functions differently.
I implement a 'traceon' and 'traceoff' to have a function start or stop tracing when it is hit. It also includes a counter for the number of times to implement the trace on/off.
echo '*_lock:traceon:4' > set_ftrace_filter
echo '*_unlock:traceoff:5' > set_ftrace_filter
This will have all functions that end with _lock turn on tracing and all that end with _unlock disable it, 4 times each.
The following patches are in:
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
branch: tip/tracing/ftrace
Steven Rostedt (15):
ftrace: state that all functions are enabled in set_ftrace_filter
ftrace: add do_for_each_ftrace_rec and while_for_each_ftrace_rec
ftrace: rename ftrace_match to ftrace_match_records
ftrace: break up ftrace_match_records into smaller components
ftrace: add module command function filter selection
ftrace: enable filtering only when a function is filtered on
ftrace: add command interface for function selection
ftrace: convert ftrace_lock from a spinlock to mutex
ftrace: consolidate mutexes
ftrace: trace different functions with a different tracer
ring-buffer: add tracing_is_on to test if ring buffer is enabled
ftrace: add traceon traceoff commands to enable/disable the buffers
ftrace: show selected functions in set_ftrace_filter
ftrace: add pretty print to selected fuction traces
ftrace: add pretty print function for traceon and traceoff hooks
----
include/linux/ftrace.h | 40 ++ include/linux/ring_buffer.h | 2 + kernel/trace/ftrace.c | 929 +++++++++++++++++++++++++++++++--------- kernel/trace/ring_buffer.c | 9 + kernel/trace/trace_functions.c | 163 +++++++ 5 files changed, 946 insertions(+), 197 deletions(-)
Steven Rostedt
maintainerPATCH 1/15 ftrace: state that all functions are enabled in set_ftrace_filter
From: Steven Rostedt <[email protected]>
Impact: clean up, make set_ftrace_filter less confusing
The set_ftrace_filter shows only the functions that will be traced. But when it is empty, it will trace all functions. This can be a bit confusing.
This patch makes set_ftrace_filter show:
#### all functions enabled ####
When all functions will be traced, and we do not filter only a select few.
--- kernel/trace/ftrace.c | 23 +++++++++++++++++++++++ 1 files changed, 23 insertions(+), 0 deletions(-)
kernel/trace/ftrace.c +23 -0
@@ -773,6 +773,7 @@ enum {FTRACE_ITER_CONT = (1 <<1 ),FTRACE_ITER_NOTRACE = (1 <<2 ),FTRACE_ITER_FAILURES = (1 <<3 ), + FTRACE_ITER_PRINTALL = (1 << 4), };#define FTRACE_BUFF_MAX (KSYM_SYMBOL_LEN+4)/* room for wildcards */
@@ -794,6 +795,9 @@ t_next(struct seq_file *m, void *v, loff_t *pos) (*pos)++; +if (iter->flags &FTRACE_ITER_PRINTALL ) +return NULL ; +/* should not be called from interrupt context */ spin_lock (&ftrace_lock); retry:
@@ -834,6 +838,19 @@ static void *t_start(struct seq_file *m, loff_t *pos)struct ftrace_iterator *iter = m->private;void *p =NULL ; +/* +* For set_ftrace_filter reading, if we have the filter +* off, we can short cut and just print out that all +* functions are enabled. +*/ +if (iter->flags &FTRACE_ITER_FILTER && !ftrace_filtered) { +if (*pos >0 ) +return NULL ; + iter->flags |=FTRACE_ITER_PRINTALL ; + (*pos)++; +return iter; + } +if (*pos >0 ) {if (iter->idx <0 )return p;
@@ -852,9 +869,15 @@ static void t_stop(struct seq_file *m, void *p)static int t_show (struct seq_file *m,void *v) { +struct ftrace_iterator *iter = m->private;struct dyn_ftrace *rec = v;char str[KSYM_SYMBOL_LEN ]; +if (iter->flags &FTRACE_ITER_PRINTALL ) { +seq_printf (m,"#### all functions enabled ####\n" ); +return 0 ; + } +if (!rec)return 0 ;
Signature · 3 lines
1.5.6.5 --
Steven Rostedt
maintainerPATCH 2/15 ftrace: add do_for_each_ftrace_rec and while_for_each_ftrace_rec
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Impact: clean up
To iterate over all the functions that dynamic trace knows about it requires two for loops. One to iterate over the pages and the other to iterate over the records within the page.
There are several duplications of these loops in ftrace.c. This patch creates the macros do_for_each_ftrace_rec and while_for_each_ftrace_rec to handle this logic, and removes the duplicate code.
While making this change, I also discovered and fixed a small bug that one of the iterations should exit the loop after it found the record it was searching for. This used a break when it should have used a goto, since there were two loops it needed to break out from. No real harm was done by this bug since it would only continue to search the other records, and the code was in a slow path anyway.
--- kernel/trace/ftrace.c | 208 ++++++++++++++++++++++++------------------------- 1 files changed, 101 insertions(+), 107 deletions(-)
kernel/trace/ftrace.c +101 -107
@@ -297,6 +297,19 @@ static struct ftrace_page *ftrace_pages;static struct dyn_ftrace *ftrace_free_records; +/* +* This is a double for. Do not use 'break' to break out of the loop, +* you must use a goto. +*/ +#define do_for_each_ftrace_rec (pg, rec) \ + for (pg = ftrace_pages_start; pg; pg = pg->next) { \ + int _____i; \ + for (_____i = 0; _____i < pg->index; _____i++) { \ + rec = &pg->records[_____i]; + +#define while_for_each_ftrace_rec () \ + } \ + }#ifdef CONFIG_KPROBES
@@ -341,7 +354,6 @@ void ftrace_release(void *start, unsigned long size)struct ftrace_page *pg;unsigned long s = (unsigned long )start;unsigned long e = s + size; -int i;if (ftrace_disabled || !start)return ;
@@ -349,14 +361,11 @@ void ftrace_release(void *start, unsigned long size)/* should not be called from interrupt context */ spin_lock (&ftrace_lock); -for (pg = ftrace_pages_start; pg; pg = pg->next) { -for (i =0 ; i < pg->index; i++) { - rec = &pg->records[i]; +do_for_each_ftrace_rec (pg, rec) { +if ((rec->ip >= s) && (rec->ip < e)) +ftrace_free_rec (rec); + }while_for_each_ftrace_rec (); -if ((rec->ip >= s) && (rec->ip < e)) -ftrace_free_rec (rec); - } - }spin_unlock (&ftrace_lock); }
@@ -523,41 +532,37 @@ __ftrace_replace_code(struct dyn_ftrace *rec, int enable)static void ftrace_replace_code (int enable) { -int i, failed; +int failed;struct dyn_ftrace *rec;struct ftrace_page *pg; -for (pg = ftrace_pages_start; pg; pg = pg->next) { -for (i =0 ; i < pg->index; i++) { - rec = &pg->records[i]; - -/* -* Skip over free records and records that have -* failed. -*/ -if (rec->flags &FTRACE_FL_FREE || - rec->flags &FTRACE_FL_FAILED ) -continue ; - -/* ignore updates to this record's mcount site */ -if (get_kprobe ((void *)rec->ip)) { -freeze_record (rec); -continue ; - }else { -unfreeze_record (rec); - } +do_for_each_ftrace_rec (pg, rec) { +/* +* Skip over free records and records that have +* failed. +*/ +if (rec->flags &FTRACE_FL_FREE || + rec->flags &FTRACE_FL_FAILED ) +continue ; - failed =__ftrace_replace_code (rec, enable); -if (failed && (rec->flags &FTRACE_FL_CONVERTED )) { - rec->flags |=FTRACE_FL_FAILED ; -if ((system_state ==SYSTEM_BOOTING ) || - !core_kernel_text (rec->ip)) { -ftrace_free_rec (rec); - }else -ftrace_bug (failed, rec->ip); - } +/* ignore updates to this record's mcount site */ +if (get_kprobe ((void *)rec->ip)) { +freeze_record (rec); +continue ; + }else { +unfreeze_record (rec); } - } + + failed =__ftrace_replace_code (rec, enable); +if (failed && (rec->flags &FTRACE_FL_CONVERTED )) { + rec->flags |=FTRACE_FL_FAILED ; +if ((system_state ==SYSTEM_BOOTING ) || + !core_kernel_text (rec->ip)) { +ftrace_free_rec (rec); + }else +ftrace_bug (failed, rec->ip); + } + }while_for_each_ftrace_rec (); } static int
@@ -956,22 +961,17 @@ static void ftrace_filter_reset(int enable)struct ftrace_page *pg;struct dyn_ftrace *rec;unsigned long type = enable ?FTRACE_FL_FILTER :FTRACE_FL_NOTRACE ; -unsigned i;/* should not be called from interrupt context */ spin_lock (&ftrace_lock);if (enable) ftrace_filtered =0 ; - pg = ftrace_pages_start; -while (pg) { -for (i =0 ; i < pg->index; i++) { - rec = &pg->records[i]; -if (rec->flags &FTRACE_FL_FAILED ) -continue ; - rec->flags &= ~type; - } - pg = pg->next; - } +do_for_each_ftrace_rec (pg, rec) { +if (rec->flags &FTRACE_FL_FAILED ) +continue ; + rec->flags &= ~type; + }while_for_each_ftrace_rec (); +spin_unlock (&ftrace_lock); }
@@ -1094,44 +1094,39 @@ ftrace_match(unsigned char *buff, int len, int enable)spin_lock (&ftrace_lock);if (enable) ftrace_filtered =1 ; - pg = ftrace_pages_start; -while (pg) { -for (i =0 ; i < pg->index; i++) { -int matched =0 ; -char *ptr; - - rec = &pg->records[i]; -if (rec->flags &FTRACE_FL_FAILED ) -continue ; -kallsyms_lookup (rec->ip,NULL ,NULL ,NULL , str); -switch (type) { -case MATCH_FULL : -if (strcmp (str, buff) ==0 ) - matched =1 ; -break ; -case MATCH_FRONT_ONLY : -if (memcmp (str, buff, match) ==0 ) - matched =1 ; -break ; -case MATCH_MIDDLE_ONLY : -if (strstr (str, search)) - matched =1 ; -break ; -case MATCH_END_ONLY : - ptr =strstr (str, search); -if (ptr && (ptr[search_len] ==0 )) - matched =1 ; -break ; - } -if (matched) { -if (not) - rec->flags &= ~flag; -else - rec->flags |= flag; - } +do_for_each_ftrace_rec (pg, rec) { +int matched =0 ; +char *ptr; + +if (rec->flags &FTRACE_FL_FAILED ) +continue ; +kallsyms_lookup (rec->ip,NULL ,NULL ,NULL , str); +switch (type) { +case MATCH_FULL : +if (strcmp (str, buff) ==0 ) + matched =1 ; +break ; +case MATCH_FRONT_ONLY : +if (memcmp (str, buff, match) ==0 ) + matched =1 ; +break ; +case MATCH_MIDDLE_ONLY : +if (strstr (str, search)) + matched =1 ; +break ; +case MATCH_END_ONLY : + ptr =strstr (str, search); +if (ptr && (ptr[search_len] ==0 )) + matched =1 ; +break ; } - pg = pg->next; - } +if (matched) { +if (not) + rec->flags &= ~flag; +else + rec->flags |= flag; + } + }while_for_each_ftrace_rec ();spin_unlock (&ftrace_lock); }
@@ -1452,7 +1447,7 @@ ftrace_set_func(unsigned long *array, int idx, char *buffer)struct dyn_ftrace *rec;struct ftrace_page *pg;int found =0 ; -int i, j; +int j;if (ftrace_disabled)return -ENODEV ;
@@ -1460,27 +1455,26 @@ ftrace_set_func(unsigned long *array, int idx, char *buffer)/* should not be called from interrupt context */ spin_lock (&ftrace_lock); -for (pg = ftrace_pages_start; pg; pg = pg->next) { -for (i =0 ; i < pg->index; i++) { - rec = &pg->records[i]; - -if (rec->flags & (FTRACE_FL_FAILED |FTRACE_FL_FREE )) -continue ; - -kallsyms_lookup (rec->ip,NULL ,NULL ,NULL , str); -if (strcmp (str, buffer) ==0 ) { - found =1 ; -for (j =0 ; j < idx; j++) -if (array[j] == rec->ip) { - found =0 ; -break ; - } -if (found) - array[idx] = rec->ip; -break ; - } +do_for_each_ftrace_rec (pg, rec) { + +if (rec->flags & (FTRACE_FL_FAILED |FTRACE_FL_FREE )) +continue ; + +kallsyms_lookup (rec->ip,NULL ,NULL ,NULL , str); +if (strcmp (str, buffer) ==0 ) { +/* Return 1 if we add it to the array */ + found =1 ; +for (j =0 ; j < idx; j++) +if (array[j] == rec->ip) { + found =0 ; +break ; + } +if (found) + array[idx] = rec->ip; + goto out; } - } + }while_for_each_ftrace_rec (); + out:spin_unlock (&ftrace_lock);return found ?0 : -EINVAL ;
Signature · 3 lines
1.5.6.5 --
Steven Rostedt
maintainerPATCH 3/15 ftrace: rename ftrace_match to ftrace_match_records
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Impact: clean up
ftrace_match is too generic of a name. What it really does is search all records and matches the records with the given string, and either sets or unsets the functions to be traced depending on if the parameter 'enable' is set or not.
This allows us to make another function called ftrace_match that can be used to test a single record.
--- kernel/trace/ftrace.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)
kernel/trace/ftrace.c +4 -4
@@ -1054,7 +1054,7 @@ enum { };static void -ftrace_match (unsigned char *buff,int len,int enable) +ftrace_match_records (unsigned char *buff,int len,int enable) {char str[KSYM_SYMBOL_LEN ];char *search =NULL ;
@@ -1197,7 +1197,7 @@ ftrace_regex_write(struct file *file, const char __user *ubuf,if (isspace (ch)) { iter->filtered++; iter->buffer[iter->buffer_idx] =0 ; -ftrace_match (iter->buffer, iter->buffer_idx, enable); +ftrace_match_records (iter->buffer, iter->buffer_idx, enable); iter->buffer_idx =0 ; }else iter->flags |=FTRACE_ITER_CONT ;
@@ -1236,7 +1236,7 @@ ftrace_set_regex(unsigned char *buf, int len, int reset, int enable)if (reset)ftrace_filter_reset (enable);if (buf) -ftrace_match (buf, len, enable); +ftrace_match_records (buf, len, enable);mutex_unlock (&ftrace_regex_lock); }
@@ -1286,7 +1286,7 @@ ftrace_regex_release(struct inode *inode, struct file *file, int enable)if (iter->buffer_idx) { iter->filtered++; iter->buffer[iter->buffer_idx] =0 ; -ftrace_match (iter->buffer, iter->buffer_idx, enable); +ftrace_match_records (iter->buffer, iter->buffer_idx, enable); }mutex_lock (&ftrace_sysctl_lock);
Signature · 3 lines
1.5.6.5 --
Steven Rostedt
maintainerPATCH 4/15 ftrace: break up ftrace_match_records into smaller components
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Impact: clean up
ftrace_match_records does a lot of things that other features can use. This patch breaks up ftrace_match_records and pulls out ftrace_setup_glob and ftrace_match_record.
ftrace_setup_glob prepares a simple glob expression for use with ftrace_match_record. ftrace_match_record compares a single record with a glob type.
Breaking this up will allow for more features to run on individual records.
--- kernel/trace/ftrace.c | 115 ++++++++++++++++++++++++++++++++----------------- 1 files changed, 75 insertions(+), 40 deletions(-)
kernel/trace/ftrace.c +75 -40
@@ -1053,79 +1053,114 @@ enum { MATCH_END_ONLY, }; -static void -ftrace_match_records (unsigned char *buff,int len,int enable) +/* +* (static function - no need for kernel doc) +* +* Pass in a buffer containing a glob and this function will +* set search to point to the search part of the buffer and +* return the type of search it is (see enum above). +* This does modify buff. +* +* Returns enum type. +* search returns the pointer to use for comparison. +* not returns 1 if buff started with a '!' +* 0 otherwise. +*/ +static int +ftrace_setup_glob (unsigned char *buff,int len,char **search,int *not) { -char str[KSYM_SYMBOL_LEN ]; -char *search =NULL ; -struct ftrace_page *pg; -struct dyn_ftrace *rec;int type =MATCH_FULL ; -unsigned long flag = enable ?FTRACE_FL_FILTER :FTRACE_FL_NOTRACE ; -unsigned i, match =0 , search_len =0 ; -int not =0 ; +int i;if (buff[0 ] =='!' ) { - not =1 ; + *not =1 ; buff++; len--; - } + }else + *not =0 ; + + *search = buff;for (i =0 ; i < len; i++) {if (buff[i] =='*' ) {if (!i) { - search = buff + i +1 ; + *search = buff +1 ; type =MATCH_END_ONLY ; - search_len = len - (i +1 ); }else { -if (type ==MATCH_END_ONLY ) { +if (type ==MATCH_END_ONLY ) type =MATCH_MIDDLE_ONLY ; - }else { - match = i; +else type =MATCH_FRONT_ONLY ; - } buff[i] =0 ;break ; } } } +return type; +} + +static int +ftrace_match_record (struct dyn_ftrace *rec,char *regex,int len,int type) +{ +char str[KSYM_SYMBOL_LEN ]; +int matched =0 ; +char *ptr; + +kallsyms_lookup (rec->ip,NULL ,NULL ,NULL , str); +switch (type) { +case MATCH_FULL : +if (strcmp (str, regex) ==0 ) + matched =1 ; +break ; +case MATCH_FRONT_ONLY : +if (strncmp (str, regex, len) ==0 ) + matched =1 ; +break ; +case MATCH_MIDDLE_ONLY : +if (strstr (str, regex)) + matched =1 ; +break ; +case MATCH_END_ONLY : + ptr =strstr (str, regex); +if (ptr && (ptr[len] ==0 )) + matched =1 ; +break ; + } + +return matched; +} + +static void ftrace_match_records (char *buff,int len,int enable) +{ +char *search; +struct ftrace_page *pg; +struct dyn_ftrace *rec; +int type; +unsigned long flag = enable ?FTRACE_FL_FILTER :FTRACE_FL_NOTRACE ; +unsigned search_len; +int not; + + type =ftrace_setup_glob (buff, len, &search, ¬); + + search_len =strlen (search); +/* should not be called from interrupt context */ spin_lock (&ftrace_lock);if (enable) ftrace_filtered =1 ;do_for_each_ftrace_rec (pg, rec) { -int matched =0 ; -char *ptr;if (rec->flags &FTRACE_FL_FAILED )continue ; -kallsyms_lookup (rec->ip,NULL ,NULL ,NULL , str); -switch (type) { -case MATCH_FULL : -if (strcmp (str, buff) ==0 ) - matched =1 ; -break ; -case MATCH_FRONT_ONLY : -if (memcmp (str, buff, match) ==0 ) - matched =1 ; -break ; -case MATCH_MIDDLE_ONLY : -if (strstr (str, search)) - matched =1 ; -break ; -case MATCH_END_ONLY : - ptr =strstr (str, search); -if (ptr && (ptr[search_len] ==0 )) - matched =1 ; -break ; - } -if (matched) { + +if (ftrace_match_record (rec, search, search_len, type)) {if (not) rec->flags &= ~flag;else rec->flags |= flag; } + }while_for_each_ftrace_rec ();spin_unlock (&ftrace_lock); }
Signature · 3 lines
1.5.6.5 --
Re: [PATCH 04/15] ftrace: break up ftrace_match_records into smaller components
kernel/trace/ftrace.c enum { · line 1142
Quoted 9 lines · Steven Rostedt
kernel/trace/ftrace.c +9
-0+static void ftrace_match_records (char *buff,int len,int enable) +{ +char *search; +struct ftrace_page *pg; +struct dyn_ftrace *rec; +int type; +unsigned long flag = enable ?FTRACE_FL_FILTER :FTRACE_FL_NOTRACE ; +unsigned search_len; +int not;
Small style nit, the above looks better as:
kernel/trace/ftrace.c enum {
Quoted 9 lines
kernel/trace/ftrace.c +9
-0+unsigned int search_len; +struct ftrace_page *pg; +struct dyn_ftrace *rec; +unsigned long flag; +char *search; +int type; +int not; + + flag = enable ?FTRACE_FL_FILTER :FTRACE_FL_NOTRACE ;
and this is what we do in other places of the tracing code.
(Also note the change of 'unsigned search_len' to 'unsigned int search_len'.)
Ingo
Steven Rostedt
maintainerPATCH 5/15 ftrace: add module command function filter selection
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
This patch adds a "command" syntax to the function filtering files:
/debugfs/tracing/set_ftrace_filter
/debugfs/tracing/set_ftrace_notrace
Of the format: <function>:<command>:<parameter>
The command is optional, and dependent on the command, so are the parameters.
echo do_fork > set_ftrace_filter
Will only trace 'do_fork'.
echo 'sched_*' > set_ftrace_filter
Will only trace functions starting with the letters 'sched_'.
echo '*:mod:ext3' > set_ftrace_filter
Will trace only the ext3 module functions.
echo '*write*:mod:ext3' > set_ftrace_notrace
Will prevent the ext3 functions with the letters 'write' in the name from being traced.
echo '!*_allocate:mod:ext3' > set_ftrace_filter
Will remove the functions in ext3 that end with the letters '_allocate' from the ftrace filter.
Although this patch implements the 'command' format, only the 'mod' command is supported. More commands to follow.
--- kernel/trace/ftrace.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++--- 1 files changed, 109 insertions(+), 6 deletions(-)
kernel/trace/ftrace.c +109 -6
@@ -1067,7 +1067,7 @@ enum {* 0 otherwise. */ static int -ftrace_setup_glob (unsigned char *buff,int len,char **search,int *not) +ftrace_setup_glob (char *buff,int len,char **search,int *not) {int type =MATCH_FULL ;int i;
@@ -1100,14 +1100,11 @@ ftrace_setup_glob(unsigned char *buff, int len, char **search, int *not)return type; } -static int -ftrace_match_record (struct dyn_ftrace *rec,char *regex,int len,int type) +static int ftrace_match (char *str,char *regex,int len,int type) { -char str[KSYM_SYMBOL_LEN ];int matched =0 ;char *ptr; -kallsyms_lookup (rec->ip,NULL ,NULL ,NULL , str);switch (type) {case MATCH_FULL :if (strcmp (str, regex) ==0 )
@@ -1131,6 +1128,15 @@ ftrace_match_record(struct dyn_ftrace *rec, char *regex, int len, int type)return matched; } +static int +ftrace_match_record (struct dyn_ftrace *rec,char *regex,int len,int type) +{ +char str[KSYM_SYMBOL_LEN ]; + +kallsyms_lookup (rec->ip,NULL ,NULL ,NULL , str); +return ftrace_match (str, regex, len, type); +} +static void ftrace_match_records (char *buff,int len,int enable) {char *search;
@@ -1165,6 +1171,100 @@ static void ftrace_match_records(char *buff, int len, int enable)spin_unlock (&ftrace_lock); } +static int +ftrace_match_module_record (struct dyn_ftrace *rec,char *mod, +char *regex,int len,int type) +{ +char str[KSYM_SYMBOL_LEN ]; +char *modname; + +kallsyms_lookup (rec->ip,NULL ,NULL , &modname, str); + +if (!modname ||strcmp (modname, mod)) +return 0 ; + +/* blank search means to match all funcs in the mod */ +if (len) +return ftrace_match (str, regex, len, type); +else +return 1 ; +} + +static void ftrace_match_module_records (char *buff,char *mod,int enable) +{ +char *search = buff; +struct ftrace_page *pg; +struct dyn_ftrace *rec; +int type =MATCH_FULL ; +unsigned long flag = enable ?FTRACE_FL_FILTER :FTRACE_FL_NOTRACE ; +unsigned search_len =0 ; +int not =0 ; + +/* blank or '*' mean the same */ +if (strcmp (buff,"*" ) ==0 ) + buff[0 ] =0 ; + +/* handle the case of 'dont filter this module' */ +if (strcmp (buff,"!" ) ==0 ||strcmp (buff,"!*" ) ==0 ) { + buff[0 ] =0 ; + not =1 ; + } + +if (strlen (buff)) { + type =ftrace_setup_glob (buff,strlen (buff), &search, ¬); + search_len =strlen (search); + } + +/* should not be called from interrupt context */ +spin_lock (&ftrace_lock); +if (enable) + ftrace_filtered =1 ; + +do_for_each_ftrace_rec (pg, rec) { + +if (rec->flags &FTRACE_FL_FAILED ) +continue ; + +if (ftrace_match_module_record (rec, mod, + search, search_len, type)) { +if (not) + rec->flags &= ~flag; +else + rec->flags |= flag; + } + + }while_for_each_ftrace_rec (); +spin_unlock (&ftrace_lock); +} + +static int ftrace_process_regex (char *buff,int len,int enable) +{ +char *func, *mod, *command, *next = buff; + + func =strsep (&next,":" ); + +if (!next) { +ftrace_match_records (func, len, enable); +return 0 ; + } + +/* command fonud */ + + command =strsep (&next,":" ); + +if (strcmp (command,"mod" ) ==0 ) { +/* only match modules */ +if (!next) +return -EINVAL ; + + mod =strsep (&next,":" ); +ftrace_match_module_records (func, mod, enable); +return 0 ; + } + +return -EINVAL ; +} +static ssize_t ftrace_regex_write (struct file *file,const char __user *ubuf,size_t cnt,loff_t *ppos,int enable)
@@ -1232,7 +1332,10 @@ ftrace_regex_write(struct file *file, const char __user *ubuf,if (isspace (ch)) { iter->filtered++; iter->buffer[iter->buffer_idx] =0 ; -ftrace_match_records (iter->buffer, iter->buffer_idx, enable); + ret =ftrace_process_regex (iter->buffer, + iter->buffer_idx, enable); +if (ret) + goto out; iter->buffer_idx =0 ; }else iter->flags |=FTRACE_ITER_CONT ;
Signature · 3 lines
1.5.6.5 --
Steven Rostedt
maintainerPATCH 6/15 ftrace: enable filtering only when a function is filtered on
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Impact: fix to prevent empty set_ftrace_filter and no ftrace output
The function filter is used to only trace a given set of functions. The filter is enabled when a function name is echoed into the set_ftrace_filter file. But if the name has a typo and the function is not found, the filter is enabled, but no function is listed.
This makes a confusing situation where set_ftrace_filter is empty but no functions ever get enabled for tracing.
For example:
# cat /debug/tracing/set_ftrace_filter
#### all functions enabled ####
# echo bad_name > set_ftrace_filter # cat /debug/tracing/set_ftrace_filter
# echo function > current_tracer # cat trace
# tracer: nop
#
# TASK-PID CPU# TIMESTAMP FUNCTION
# | | | | |
This patch changes that to only enable filtering if a function is set to be filtered on. Now, the filter is not enabled if a bad name is echoed into set_ftrace_filter.
--- kernel/trace/ftrace.c | 14 ++++++++------ 1 files changed, 8 insertions(+), 6 deletions(-)
kernel/trace/ftrace.c +8 -6
@@ -1153,8 +1153,6 @@ static void ftrace_match_records(char *buff, int len, int enable)/* should not be called from interrupt context */ spin_lock (&ftrace_lock); -if (enable) - ftrace_filtered =1 ;do_for_each_ftrace_rec (pg, rec) {if (rec->flags &FTRACE_FL_FAILED )
@@ -1166,7 +1164,12 @@ static void ftrace_match_records(char *buff, int len, int enable)else rec->flags |= flag; } - +/* +* Only enable filtering if we have a function that +* is filtered on. +*/ +if (enable && (rec->flags &FTRACE_FL_FILTER )) + ftrace_filtered =1 ; }while_for_each_ftrace_rec ();spin_unlock (&ftrace_lock); }
@@ -1217,9 +1220,6 @@ static void ftrace_match_module_records(char *buff, char *mod, int enable)/* should not be called from interrupt context */ spin_lock (&ftrace_lock); -if (enable) - ftrace_filtered =1 ; -do_for_each_ftrace_rec (pg, rec) {if (rec->flags &FTRACE_FL_FAILED )
@@ -1232,6 +1232,8 @@ static void ftrace_match_module_records(char *buff, char *mod, int enable)else rec->flags |= flag; } +if (enable && (rec->flags &FTRACE_FL_FILTER )) + ftrace_filtered =1 ; }while_for_each_ftrace_rec ();spin_unlock (&ftrace_lock);
Signature · 3 lines
1.5.6.5 --
Steven Rostedt
maintainerPATCH 7/15 ftrace: add command interface for function selection
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Allow for other tracers to add their own commands for function selection. This interface gives a trace the ability to name a command for function selection. Right now it is pretty limited in what it offers, but this is a building step for more features.
The :mod: command is converted to this interface and also serves as a template for other implementations.
--- include/linux/ftrace.h | 16 +++++++ kernel/trace/ftrace.c | 106 +++++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 111 insertions(+), 11 deletions(-)
include/linux/ftrace.h +16 -0
@@ -95,6 +95,13 @@ stack_trace_sysctl(struct ctl_table *table, int write, loff_t *ppos);#endif +struct ftrace_func_command { +struct list_head list; +char *name; +int (*func)(char *func,char *cmd, +char *params,int enable); +}; +#ifdef CONFIG_DYNAMIC_FTRACE /* asm/ftrace.h must be defined for archs supporting dynamic ftrace */ #include <asm/ftrace.h>
@@ -119,6 +126,9 @@ struct dyn_ftrace {int ftrace_force_update (void );void ftrace_set_filter (unsigned char *buf,int len,int reset); +int register_ftrace_command (struct ftrace_func_command *cmd); +int unregister_ftrace_command (struct ftrace_func_command *cmd); +/* defined in arch */ extern int ftrace_ip_converted (unsigned long ip);extern int ftrace_dyn_arch_init (void *data);
@@ -202,6 +212,12 @@ extern void ftrace_enable_daemon(void);# define ftrace_disable_daemon () do { } while (0)# define ftrace_enable_daemon () do { } while (0)static inline void ftrace_release (void *start,unsigned long size) { } +static inline int register_ftrace_command (struct ftrace_func_command *cmd) +{ +} +static inline int unregister_ftrace_command (char *cmd_name) +{ +}#endif /* CONFIG_DYNAMIC_FTRACE */ /* totally disable ftrace - can not re-enable after this */
kernel/trace/ftrace.c +95 -11
125 lines not on this page · raw message
Signature · 3 lines
1.5.6.5 --
Steven Rostedt
maintainerPATCH 8/15 ftrace: convert ftrace_lock from a spinlock to mutex
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Impact: clean up
The older versions of ftrace required doing the ftrace list search under atomic context. Now all the calls are in non-atomic context. There is no reason to keep the ftrace_lock as a spinlock.
This patch converts it to a mutex.
--- kernel/trace/ftrace.c | 51 ++++++++++++++++++------------------------------ 1 files changed, 19 insertions(+), 32 deletions(-)
kernel/trace/ftrace.c +19 -32
@@ -61,7 +61,7 @@ int function_trace_stop;*/ static int ftrace_disabled__read_mostly ; -static DEFINE_SPINLOCK (ftrace_lock); +static DEFINE_MUTEX (ftrace_lock);static DEFINE_MUTEX (ftrace_sysctl_lock);static DEFINE_MUTEX (ftrace_start_lock);
@@ -134,8 +134,7 @@ static void ftrace_test_stop_func(unsigned long ip, unsigned long parent_ip)static int __register_ftrace_function (struct ftrace_ops *ops) { -/* should not be called from interrupt context */ -spin_lock (&ftrace_lock); +mutex_lock (&ftrace_lock); ops->next = ftrace_list;/*
@@ -172,7 +171,7 @@ static int __register_ftrace_function(struct ftrace_ops *ops)#endif } -spin_unlock (&ftrace_lock); +mutex_unlock (&ftrace_lock);return 0 ; }
@@ -182,8 +181,7 @@ static int __unregister_ftrace_function(struct ftrace_ops *ops)struct ftrace_ops **p;int ret =0 ; -/* should not be called from interrupt context */ -spin_lock (&ftrace_lock); +mutex_lock (&ftrace_lock);/* * If we are removing the last function, then simply point
@@ -224,7 +222,7 @@ static int __unregister_ftrace_function(struct ftrace_ops *ops) } out: -spin_unlock (&ftrace_lock); +mutex_unlock (&ftrace_lock);return ret; }
@@ -233,8 +231,7 @@ static void ftrace_update_pid_func(void) {ftrace_func_t func; -/* should not be called from interrupt context */ -spin_lock (&ftrace_lock); +mutex_lock (&ftrace_lock);if (ftrace_trace_function == ftrace_stub) goto out;
@@ -256,7 +253,7 @@ static void ftrace_update_pid_func(void)#endif out: -spin_unlock (&ftrace_lock); +mutex_unlock (&ftrace_lock); }#ifdef CONFIG_DYNAMIC_FTRACE
@@ -358,15 +355,12 @@ void ftrace_release(void *start, unsigned long size)if (ftrace_disabled || !start)return ; -/* should not be called from interrupt context */ -spin_lock (&ftrace_lock); - +mutex_lock (&ftrace_lock);do_for_each_ftrace_rec (pg, rec) {if ((rec->ip >= s) && (rec->ip < e))ftrace_free_rec (rec); }while_for_each_ftrace_rec (); - -spin_unlock (&ftrace_lock); +mutex_unlock (&ftrace_lock); }static struct dyn_ftrace *ftrace_alloc_dyn_node (unsigned long ip)
@@ -803,8 +797,7 @@ t_next(struct seq_file *m, void *v, loff_t *pos)if (iter->flags &FTRACE_ITER_PRINTALL )return NULL ; -/* should not be called from interrupt context */ -spin_lock (&ftrace_lock); +mutex_lock (&ftrace_lock); retry:if (iter->idx >= iter->pg->index) {if (iter->pg->next) {
@@ -833,7 +826,7 @@ t_next(struct seq_file *m, void *v, loff_t *pos) goto retry; } } -spin_unlock (&ftrace_lock); +mutex_unlock (&ftrace_lock);return rec; }
@@ -962,8 +955,7 @@ static void ftrace_filter_reset(int enable)struct dyn_ftrace *rec;unsigned long type = enable ?FTRACE_FL_FILTER :FTRACE_FL_NOTRACE ; -/* should not be called from interrupt context */ -spin_lock (&ftrace_lock); +mutex_lock (&ftrace_lock);if (enable) ftrace_filtered =0 ;do_for_each_ftrace_rec (pg, rec) {
@@ -971,8 +963,7 @@ static void ftrace_filter_reset(int enable)continue ; rec->flags &= ~type; }while_for_each_ftrace_rec (); - -spin_unlock (&ftrace_lock); +mutex_unlock (&ftrace_lock); } static int
@@ -1151,8 +1142,7 @@ static void ftrace_match_records(char *buff, int len, int enable) search_len =strlen (search); -/* should not be called from interrupt context */ -spin_lock (&ftrace_lock); +mutex_lock (&ftrace_lock);do_for_each_ftrace_rec (pg, rec) {if (rec->flags &FTRACE_FL_FAILED )
@@ -1171,7 +1161,7 @@ static void ftrace_match_records(char *buff, int len, int enable)if (enable && (rec->flags &FTRACE_FL_FILTER )) ftrace_filtered =1 ; }while_for_each_ftrace_rec (); -spin_unlock (&ftrace_lock); +mutex_unlock (&ftrace_lock); } static int
@@ -1218,8 +1208,7 @@ static void ftrace_match_module_records(char *buff, char *mod, int enable) search_len =strlen (search); } -/* should not be called from interrupt context */ -spin_lock (&ftrace_lock); +mutex_lock (&ftrace_lock);do_for_each_ftrace_rec (pg, rec) {if (rec->flags &FTRACE_FL_FAILED )
@@ -1236,7 +1225,7 @@ static void ftrace_match_module_records(char *buff, char *mod, int enable) ftrace_filtered =1 ; }while_for_each_ftrace_rec (); -spin_unlock (&ftrace_lock); +mutex_unlock (&ftrace_lock); }/*
@@ -1676,9 +1665,7 @@ ftrace_set_func(unsigned long *array, int idx, char *buffer)if (ftrace_disabled)return -ENODEV ; -/* should not be called from interrupt context */ -spin_lock (&ftrace_lock); - +mutex_lock (&ftrace_lock);do_for_each_ftrace_rec (pg, rec) {if (rec->flags & (FTRACE_FL_FAILED |FTRACE_FL_FREE ))
@@ -1699,7 +1686,7 @@ ftrace_set_func(unsigned long *array, int idx, char *buffer) } }while_for_each_ftrace_rec (); out: -spin_unlock (&ftrace_lock); +mutex_unlock (&ftrace_lock);return found ?0 : -EINVAL ; }
Signature · 3 lines
1.5.6.5 --
Steven Rostedt
maintainerPATCH 9/15 ftrace: consolidate mutexes
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Impact: clean up
Now that ftrace_lock is a mutex, there is no reason to have three different mutexes protecting similar data. All the mutex paths are not in hot paths, so having a mutex to cover more data is not a problem.
This patch removes the ftrace_sysctl_lock and ftrace_start_lock and uses the ftrace_lock to protect the locations that were protected by these locks. By doing so, this change also removes some of the lock nesting that was taking place.
There are still more mutexes in ftrace.c that can probably be consolidated, but they can be dealt with later. We need to be careful about the way the locks are nested, and by consolidating, we can cause a recursive deadlock.
--- kernel/trace/ftrace.c | 68 +++++++++++++++--------------------------------- 1 files changed, 21 insertions(+), 47 deletions(-)
kernel/trace/ftrace.c +21 -47
@@ -62,8 +62,6 @@ int function_trace_stop;static int ftrace_disabled__read_mostly ;static DEFINE_MUTEX (ftrace_lock); -static DEFINE_MUTEX (ftrace_sysctl_lock); -static DEFINE_MUTEX (ftrace_start_lock);static struct ftrace_ops ftrace_list_end__read_mostly = {
@@ -134,8 +132,6 @@ static void ftrace_test_stop_func(unsigned long ip, unsigned long parent_ip)static int __register_ftrace_function (struct ftrace_ops *ops) { -mutex_lock (&ftrace_lock); - ops->next = ftrace_list;/* * We are entering ops into the ftrace_list but another
@@ -171,17 +167,12 @@ static int __register_ftrace_function(struct ftrace_ops *ops)#endif } -mutex_unlock (&ftrace_lock); -return 0 ; }static int __unregister_ftrace_function (struct ftrace_ops *ops) {struct ftrace_ops **p; -int ret =0 ; - -mutex_lock (&ftrace_lock);/* * If we are removing the last function, then simply point
@@ -190,17 +181,15 @@ static int __unregister_ftrace_function(struct ftrace_ops *ops)if (ftrace_list == ops && ops->next == &ftrace_list_end) { ftrace_trace_function = ftrace_stub; ftrace_list = &ftrace_list_end; - goto out; +return 0 ; }for (p = &ftrace_list; *p != &ftrace_list_end; p = &(*p)->next)if (*p == ops)break ; -if (*p != ops) { - ret =-1 ; - goto out; - } +if (*p != ops) +return -1 ; *p = (*p)->next;
@@ -221,10 +210,7 @@ static int __unregister_ftrace_function(struct ftrace_ops *ops) } } - out: -mutex_unlock (&ftrace_lock); - -return ret; +return 0 ; }static void ftrace_update_pid_func (void )
@@ -622,13 +608,10 @@ static void ftrace_startup(int command)if (unlikely (ftrace_disabled))return ; -mutex_lock (&ftrace_start_lock); ftrace_start_up++; command |=FTRACE_ENABLE_CALLS ;ftrace_startup_enable (command); - -mutex_unlock (&ftrace_start_lock); }static void ftrace_shutdown (int command)
@@ -636,7 +619,6 @@ static void ftrace_shutdown(int command)if (unlikely (ftrace_disabled))return ; -mutex_lock (&ftrace_start_lock); ftrace_start_up--;if (!ftrace_start_up) command |=FTRACE_DISABLE_CALLS ;
@@ -647,11 +629,9 @@ static void ftrace_shutdown(int command) }if (!command || !ftrace_enabled) - goto out; +return ;ftrace_run_update_code (command); - out: -mutex_unlock (&ftrace_start_lock); }static void ftrace_startup_sysctl (void )
@@ -661,7 +641,6 @@ static void ftrace_startup_sysctl(void)if (unlikely (ftrace_disabled))return ; -mutex_lock (&ftrace_start_lock);/* Force update next time */ saved_ftrace_func =NULL ;/* ftrace_start_up is true if we want ftrace running */
@@ -669,7 +648,6 @@ static void ftrace_startup_sysctl(void) command |=FTRACE_ENABLE_CALLS ;ftrace_run_update_code (command); -mutex_unlock (&ftrace_start_lock); }static void ftrace_shutdown_sysctl (void )
@@ -679,13 +657,11 @@ static void ftrace_shutdown_sysctl(void)if (unlikely (ftrace_disabled))return ; -mutex_lock (&ftrace_start_lock);/* ftrace_start_up is true if ftrace is running */ if (ftrace_start_up) command |=FTRACE_DISABLE_CALLS ;ftrace_run_update_code (command); -mutex_unlock (&ftrace_start_lock); }static cycle_t ftrace_update_time;
@@ -1502,12 +1478,10 @@ ftrace_regex_release(struct inode *inode, struct file *file, int enable)ftrace_match_records (iter->buffer, iter->buffer_idx, enable); } -mutex_lock (&ftrace_sysctl_lock); -mutex_lock (&ftrace_start_lock); +mutex_lock (&ftrace_lock);if (ftrace_start_up && ftrace_enabled)ftrace_run_update_code (FTRACE_ENABLE_CALLS ); -mutex_unlock (&ftrace_start_lock); -mutex_unlock (&ftrace_sysctl_lock); +mutex_unlock (&ftrace_lock);kfree (iter);mutex_unlock (&ftrace_regex_lock);
@@ -1824,7 +1798,7 @@ static int ftrace_convert_nops(struct module *mod,unsigned long addr;unsigned long flags; -mutex_lock (&ftrace_start_lock); +mutex_lock (&ftrace_lock); p = start;while (p < end) { addr =ftrace_call_adjust (*p++);
@@ -1843,7 +1817,7 @@ static int ftrace_convert_nops(struct module *mod,local_irq_save (flags);ftrace_update_code (mod);local_irq_restore (flags); -mutex_unlock (&ftrace_start_lock); +mutex_unlock (&ftrace_lock);return 0 ; }
@@ -2016,7 +1990,7 @@ ftrace_pid_write(struct file *filp, const char __user *ubuf,if (ret <0 )return ret; -mutex_lock (&ftrace_start_lock); +mutex_lock (&ftrace_lock);if (val <0 ) {/* disable pid tracing */ if (!ftrace_pid_trace)
@@ -2055,7 +2029,7 @@ ftrace_pid_write(struct file *filp, const char __user *ubuf,ftrace_startup_enable (0 ); out: -mutex_unlock (&ftrace_start_lock); +mutex_unlock (&ftrace_lock);return cnt; }
@@ -2118,12 +2092,12 @@ int register_ftrace_function(struct ftrace_ops *ops)if (unlikely (ftrace_disabled))return -1 ; -mutex_lock (&ftrace_sysctl_lock); +mutex_lock (&ftrace_lock); ret =__register_ftrace_function (ops);ftrace_startup (0 ); -mutex_unlock (&ftrace_sysctl_lock); +mutex_unlock (&ftrace_lock);return ret; }
@@ -2137,10 +2111,10 @@ int unregister_ftrace_function(struct ftrace_ops *ops) {int ret; -mutex_lock (&ftrace_sysctl_lock); +mutex_lock (&ftrace_lock); ret =__unregister_ftrace_function (ops);ftrace_shutdown (0 ); -mutex_unlock (&ftrace_sysctl_lock); +mutex_unlock (&ftrace_lock);return ret; }
@@ -2155,7 +2129,7 @@ ftrace_enable_sysctl(struct ctl_table *table, int write,if (unlikely (ftrace_disabled))return -ENODEV ; -mutex_lock (&ftrace_sysctl_lock); +mutex_lock (&ftrace_lock); ret =proc_dointvec (table, write, file, buffer, lenp, ppos);
@@ -2184,7 +2158,7 @@ ftrace_enable_sysctl(struct ctl_table *table, int write, } out: -mutex_unlock (&ftrace_sysctl_lock); +mutex_unlock (&ftrace_lock);return ret; }
@@ -2296,7 +2270,7 @@ int register_ftrace_graph(trace_func_graph_ret_t retfunc, {int ret =0 ; -mutex_lock (&ftrace_sysctl_lock); +mutex_lock (&ftrace_lock); ftrace_suspend_notifier.notifier_call = ftrace_suspend_notifier_call;register_pm_notifier (&ftrace_suspend_notifier);
@@ -2314,13 +2288,13 @@ int register_ftrace_graph(trace_func_graph_ret_t retfunc,ftrace_startup (FTRACE_START_FUNC_RET ); out: -mutex_unlock (&ftrace_sysctl_lock); +mutex_unlock (&ftrace_lock);return ret; }void unregister_ftrace_graph (void ) { -mutex_lock (&ftrace_sysctl_lock); +mutex_lock (&ftrace_lock);atomic_dec (&ftrace_graph_active); ftrace_graph_return = (trace_func_graph_ret_t )ftrace_stub;
@@ -2328,7 +2302,7 @@ void unregister_ftrace_graph(void)ftrace_shutdown (FTRACE_STOP_FUNC_RET );unregister_pm_notifier (&ftrace_suspend_notifier); -mutex_unlock (&ftrace_sysctl_lock); +mutex_unlock (&ftrace_lock); }/* Allocate a return stack for newly created task */
Signature · 3 lines
1.5.6.5 --
Steven Rostedt
maintainerPATCH 10/15 ftrace: trace different functions with a different tracer
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Impact: new feature
Currently, the function tracer only gives you an ability to hook a tracer to all functions being traced. The dynamic function trace allows you to pick and choose which of those functions will be traced, but all functions being traced will call all tracers that registered with the function tracer.
This patch adds a new feature that allows a tracer to hook to specific functions, even when all functions are being traced. It allows for different functions to call different tracer hooks.
The way this is accomplished is by a special function that will hook to the function tracer and will set up a hash table knowing which tracer hook to call with which function. This is the most general and easiest method to accomplish this. Later, an arch may choose to supply their own method in changing the mcount call of a function to call a different tracer. But that will be an exercise for the future.
To register a function:
struct ftrace_hook_ops {
void (*func)(unsigned long ip,
unsigned long parent_ip,
void **data);
int (*callback)(unsigned long ip, void **data);
void (*free)(void **data);};
int register_ftrace_function_hook(char *glob, struct ftrace_hook_ops *ops,
void *data);
glob is a simple glob to search for the functions to hook. ops is a pointer to the operations (listed below)
data is the default data to be passed to the hook functions when traced
ops:
func is the hook function to call when the functions are traced callback is a callback function that is called when setting up the hash.
That is, if the tracer needs to do something special for each function, that is being traced, and wants to give each function its own data. The address of the entry data is passed to this callback, so that the callback may wish to update the entry to whatever it would like.
free is a callback for when the entry is freed. In case the tracer
allocated any data, it is give the chance to free it.
To unregister we have three functions:
void
unregister_ftrace_function_hook(char *glob, struct ftrace_hook_ops *ops,
void *data)
This will unregister all hooks that match glob, point to ops, and have its data matching data. (note, if glob is NULL, blank or '*', all functions will be tested).
void
unregister_ftrace_function_hook_func(char *glob,
struct ftrace_hook_ops *ops)
This will unregister all functions matching glob that has an entry pointing to ops.
void unregister_ftrace_function_hook_all(char *glob)
This simply unregisters all funcs.
--- include/linux/ftrace.h | 18 ++++ kernel/trace/ftrace.c | 247 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 265 insertions(+), 0 deletions(-)
include/linux/ftrace.h +18 -0
@@ -106,6 +106,24 @@ struct ftrace_func_command {/* asm/ftrace.h must be defined for archs supporting dynamic ftrace */ #include <asm/ftrace.h> +struct ftrace_hook_ops { +void (*func)(unsigned long ip, +unsigned long parent_ip, +void **data); +int (*callback)(unsigned long ip,void **data); +void (*free)(void **data); +}; + +extern int +register_ftrace_function_hook (char *glob,struct ftrace_hook_ops *ops, +void *data); +extern void +unregister_ftrace_function_hook (char *glob,struct ftrace_hook_ops *ops, +void *data); +extern void +unregister_ftrace_function_hook_func (char *glob,struct ftrace_hook_ops *ops); +extern void unregister_ftrace_function_hook_all (char *glob); +enum {FTRACE_FL_FREE = (1 <<0 ),FTRACE_FL_FAILED = (1 <<1 ),
kernel/trace/ftrace.c +247 -0
259 lines not on this page · raw message
Signature · 3 lines
1.5.6.5 --
Re: [PATCH 10/15] ftrace: trace different functions with a different tracer
Quoted 64 lines · Steven Rostedt
From: Steven Rostedt <[email protected]>
Impact: new feature
Currently, the function tracer only gives you an ability to hook a tracer to all functions being traced. The dynamic function trace allows you to pick and choose which of those functions will be traced, but all functions being traced will call all tracers that registered with the function tracer.
This patch adds a new feature that allows a tracer to hook to specific functions, even when all functions are being traced. It allows for different functions to call different tracer hooks.
The way this is accomplished is by a special function that will hook to the function tracer and will set up a hash table knowing which tracer hook to call with which function. This is the most general and easiest method to accomplish this. Later, an arch may choose to supply their own method in changing the mcount call of a function to call a different tracer. But that will be an exercise for the future.
To register a function:
struct ftrace_hook_ops { void (*func)(unsigned long ip, unsigned long parent_ip, void **data); int (*callback)(unsigned long ip, void **data); void (*free)(void **data);};
int register_ftrace_function_hook(char *glob, struct
ftrace_hook_ops*ops,void *data);glob is a simple glob to search for the functions to hook. ops is a pointer to the operations (listed below)
data is the default data to be passed to the hook functions when tracedops:
func is the hook function to call when the functions are traced callback is a callback function that is called when setting up the hash.
That is, if the tracer needs to do something special for each function, that is being traced, and wants to give each function its own data. The address of the entry data is passed to this callback, so that the callback may wish to update the entry to whatever it would like.
free is a callback for when the entry is freed. In case the tracer
allocated any data, it is give the chance to free it.To unregister we have three functions:
void
unregister_ftrace_function_hook(char *glob, structftrace_hook_ops*ops,
void *data)This will unregister all hooks that match glob, point to ops, and have its data matching data. (note, if glob is NULL, blank or '*', all functions will be tested).
void
unregister_ftrace_function_hook_func(char *glob,
structftrace_hook_ops*ops)
Quoted 6 lines
This will unregister all functions matching glob that has an entry pointing to ops.
void
unregister_ftrace_function_hook_all(char *glob)This simply unregisters all funcs.
Looks like a nice addition!
One RCU type mismatch called out below.
include/linux/ftrace.h struct ftrace_func_command { · line 107
Quoted 13 lines
Signed-off-by Steven Rostedt <[email protected]>--- include/linux/ftrace.h | 18 ++++ kernel/trace/ftrace.c | 247 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 265 insertions(+), 0 deletions(-)include/linux/ftrace.h +0
-0@@ -106,6 +106,24 @@ struct ftrace_func_command {/* asm/ftrace.h must be defined for archs supporting dynamic ftrace */ #include <asm/ftrace.h>
[ . . . ]
Quoted 38 lines
include/linux/ftrace.h +38
-0+ +static void +__unregister_ftrace_function_hook (char *glob,struct ftrace_hook_ops *ops, +void *data,int flags) +{ +struct ftrace_func_hook *entry; +struct hlist_node *n, *tmp; +char str[KSYM_SYMBOL_LEN ]; +int type =MATCH_FULL ; +int i, len =0 ; +char *search; + +if (glob && (strcmp (glob,"*" ) || !strlen (glob))) + glob =NULL ; +else { +int not; + + type =ftrace_setup_glob (glob,strlen (glob), &search, ¬); + len =strlen (search); + +/* we do not support '!' for function hooks */ +if (WARN_ON (not)) +return ; + } + +mutex_lock (&ftrace_lock); +for (i =0 ; i <FTRACE_FUNC_HASHSIZE ; i++) { +struct hlist_head *hhd = &ftrace_func_hash[i]; + +hlist_for_each_entry_safe (entry, n, tmp, hhd, node) { + +/* break up if statements for readability */ +if ((flags &HOOK_TEST_FUNC ) && entry->ops != ops) +continue ; + +if ((flags &HOOK_TEST_DATA ) && entry->data != data) +continue ; +
include/linux/ftrace.h struct ftrace_func_command {
Quoted 10 lines
include/linux/ftrace.h +10
-0+/* do this last, since it is the most expensive */ +if (glob) { +kallsyms_lookup (entry->ip,NULL ,NULL , +NULL , str); +if (!ftrace_match (str, glob, len, type)) +continue ; + } + +hlist_del (&entry->node); +call_rcu (&entry->rcu, ftrace_free_entry_rcu);
This should be call_rcu_sched() to match the preemption disabling.
Quoted 5 lines
include/linux/ftrace.h +5
-0+ } + } +__disable_ftrace_function_hook (); +mutex_unlock (&ftrace_lock); +}
Steven Rostedt
maintainerRe: [PATCH 10/15] ftrace: trace different functions with a different tracer
Quoted 69 lines · Paul E. McKenney
Quoted 6 lines
This will unregister all functions matching glob that has an entry pointing to ops.
void
unregister_ftrace_function_hook_all(char *glob)This simply unregisters all funcs.
Looks like a nice addition!
One RCU type mismatch called out below.
include/linux/ftrace.h struct ftrace_func_command { · line 107
Quoted 13 lines
Signed-off-by Steven Rostedt <[email protected]>--- include/linux/ftrace.h | 18 ++++ kernel/trace/ftrace.c | 247 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 265 insertions(+), 0 deletions(-)include/linux/ftrace.h +0
-0@@ -106,6 +106,24 @@ struct ftrace_func_command {/* asm/ftrace.h must be defined for archs supporting dynamic ftrace */ #include <asm/ftrace.h> [ . . . ]
Quoted 42 lines
include/linux/ftrace.h +42
-0+ +static void +__unregister_ftrace_function_hook (char *glob,struct ftrace_hook_ops *ops, +void *data,int flags) +{ +struct ftrace_func_hook *entry; +struct hlist_node *n, *tmp; +char str[KSYM_SYMBOL_LEN ]; +int type =MATCH_FULL ; +int i, len =0 ; +char *search; + +if (glob && (strcmp (glob,"*" ) || !strlen (glob))) + glob =NULL ; +else { +int not; + + type =ftrace_setup_glob (glob,strlen (glob), &search, ¬); + len =strlen (search); + +/* we do not support '!' for function hooks */ +if (WARN_ON (not)) +return ; + } + +mutex_lock (&ftrace_lock); +for (i =0 ; i <FTRACE_FUNC_HASHSIZE ; i++) { +struct hlist_head *hhd = &ftrace_func_hash[i]; + +hlist_for_each_entry_safe (entry, n, tmp, hhd, node) { + +/* break up if statements for readability */ +if ((flags &HOOK_TEST_FUNC ) && entry->ops != ops) +continue ; + +if ((flags &HOOK_TEST_DATA ) && entry->data != data) +continue ; + +/* do this last, since it is the most expensive */ +if (glob) { +kallsyms_lookup (entry->ip,NULL ,NULL , +NULL , str);
Quoted 8 lines
include/linux/ftrace.h
Quoted 6 lines
include/linux/ftrace.h +6
-0+if (!ftrace_match (str, glob, len, type)) +continue ; + } + +hlist_del (&entry->node); +call_rcu (&entry->rcu, ftrace_free_entry_rcu);This should be
call_rcu_sched()to match the preemption disabling.
Thanks! I should have CC'd you on that. I thought that was not right. Will fix.
-- Steve
Quoted 8 lines
Quoted 6 lines
include/linux/ftrace.h +5
-0+ } + } +__disable_ftrace_function_hook (); +mutex_unlock (&ftrace_lock); +}
Steven Rostedt
maintainerPATCH 11/15 ring-buffer: add tracing_is_on to test if ring buffer is enabled
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
This patch adds the tracing_is_on() interface to tell if the ring buffer is turned on or not.
--- include/linux/ring_buffer.h | 2 ++ kernel/trace/ring_buffer.c | 9 +++++++++ 2 files changed, 11 insertions(+), 0 deletions(-)
include/linux/ring_buffer.h +2 -0
@@ -128,10 +128,12 @@ void ring_buffer_normalize_time_stamp(int cpu, u64 *ts);void tracing_on (void );void tracing_off (void );void tracing_off_permanent (void ); +int tracing_is_on (void );#else static inline void tracing_on (void ) { }static inline void tracing_off (void ) { }static inline void tracing_off_permanent (void ) { } +static inline int tracing_is_on (void ) {return 0 ; }#endif void *ring_buffer_alloc_read_page (struct ring_buffer *buffer);
kernel/trace/ring_buffer.c +9 -0
@@ -98,6 +98,15 @@ void tracing_off_permanent(void)set_bit (RB_BUFFERS_DISABLED_BIT , &ring_buffer_flags); } +/** +* tracing_is_on - show state of ring buffers enabled +*/ +int tracing_is_on (void ) +{ +return ring_buffer_flags ==RB_BUFFERS_ON ; +} +EXPORT_SYMBOL_GPL (tracing_is_on); +#include "trace.h" /* Up this if you want to test the TIME_EXTENTS and normalization */
Signature · 3 lines
1.5.6.5 --
Steven Rostedt
maintainerPATCH 12/15 ftrace: add traceon traceoff commands to enable/disable the buffers
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
This patch adds the new function selection commands traceon and traceoff. traceon sets the function to enable the ring buffers while traceoff disables the ring buffers. You can pass in the number of times you want the command to be executed when the function is hit. It will only execute if the state of the buffers are not already in that state.
Example:
# echo do_fork:traceon:4
Will enable the ring buffers if they are disabled every time it hits do_fork, up to 4 times.
# echo sys_close:traceoff
This will disable the ring buffers every time (unlimited) when sys_close is called.
--- kernel/trace/trace_functions.c | 135 ++++++++++++++++++++++++++++++++++++++++ 1 files changed, 135 insertions(+), 0 deletions(-)
kernel/trace/trace_functions.c +135 -0
@@ -9,6 +9,7 @@* Copyright (C) 2004-2006 Ingo Molnar * Copyright (C) 2004 William Lee Irwin III */ +#include <linux/ring_buffer.h> #include <linux/debugfs.h> #include <linux/uaccess.h> #include <linux/ftrace.h>
@@ -231,9 +232,143 @@ static struct tracer function_trace __read_mostly =#endif }; +#ifdef CONFIG_DYNAMIC_FTRACE +static void +ftrace_traceon (unsigned long ip,unsigned long parent_ip,void **data) +{ +long *count = (long *)data; + +if (tracing_is_on ()) +return ; + +if (!*count) +return ; + +if (*count !=-1 ) + (*count)--; + +tracing_on (); +} + +static void +ftrace_traceoff (unsigned long ip,unsigned long parent_ip,void **data) +{ +long *count = (long *)data; + +if (!tracing_is_on ()) +return ; + +if (!*count) +return ; + +if (*count !=-1 ) + (*count)--; + +tracing_off (); +} + +static struct ftrace_hook_ops traceon_hook_ops = { + .func = ftrace_traceon, +}; + +static struct ftrace_hook_ops traceoff_hook_ops = { + .func = ftrace_traceoff, +}; + +static int +ftrace_trace_onoff_unreg (char *glob,char *cmd,char *param) +{ +struct ftrace_hook_ops *ops; + +/* we register both traceon and traceoff to this callback */ +if (strcmp (cmd,"traceon" ) ==0 ) + ops = &traceon_hook_ops; +else + ops = &traceoff_hook_ops; + +unregister_ftrace_function_hook_func (glob, ops); + +return 0 ; +} + +static int +ftrace_trace_onoff_callback (char *glob,char *cmd,char *param,int enable) +{ +struct ftrace_hook_ops *ops; +void *count = (void *)-1 ; +char *number; +int ret; + +/* hash funcs only work with set_ftrace_filter */ +if (!enable) +return -EINVAL ; + +if (glob[0 ] =='!' ) +return ftrace_trace_onoff_unreg (glob+1 , cmd, param); + +/* we register both traceon and traceoff to this callback */ +if (strcmp (cmd,"traceon" ) ==0 ) + ops = &traceon_hook_ops; +else + ops = &traceoff_hook_ops; + +if (!param) + goto out_reg; + + number =strsep (¶m,":" ); + +if (!strlen (number)) + goto out_reg; + +/* +* We use the callback data field (which is a pointer) +* as our counter. +*/ + ret =strict_strtoul (number,0 , (unsigned long *)&count); +if (ret) +return ret; + + out_reg: + ret =register_ftrace_function_hook (glob, ops, count); + +return ret; +} + +static struct ftrace_func_command ftrace_traceon_cmd = { + .name ="traceon" , + .func = ftrace_trace_onoff_callback, +}; + +static struct ftrace_func_command ftrace_traceoff_cmd = { + .name ="traceoff" , + .func = ftrace_trace_onoff_callback, +}; + +static int __init init_func_cmd_traceon (void ) +{ +int ret; + + ret =register_ftrace_command (&ftrace_traceoff_cmd); +if (ret) +return ret; + + ret =register_ftrace_command (&ftrace_traceon_cmd); +if (ret) +unregister_ftrace_command (&ftrace_traceoff_cmd); +return ret; +} +#else +static inline int init_func_cmd_traceon (void ) +{ +return 0 ; +} +#endif /* CONFIG_DYNAMIC_FTRACE */ +static __init int init_function_trace (void ) { +init_func_cmd_traceon ();return register_tracer (&function_trace); }device_initcall (init_function_trace); +
Signature · 3 lines
1.5.6.5 --
Re: [PATCH 12/15] ftrace: add traceon traceoff commands to enable/disable the buffers
Review
Quoted 2 lines · Steven Rostedt
(quoted diff) +2
-0+ if (*count != -1) + (*count)--;
minor nit: it's a tiny bit more compact as:
Review
Quoted 2 lines
(quoted diff) +2
-0+ if (*count != -1) + --*count;
and this:
kernel/trace/trace_functions.c static struct tracer function_trace __read_mostly = · line 374
Quoted 5 lines
kernel/trace/trace_functions.c +1
-0return register_tracer (&function_trace); }device_initcall (init_function_trace); +
there should be no newline between device_initcall() and the preceding function. (we can think of it as a late attribute of the function, attached to the function. We do the same for EXPORT_SYMBOL's too, etc.)
Ingo
Steven Rostedt
maintainerRe: [PATCH 12/15] ftrace: add traceon traceoff commands to enable/disable the buffers
Quoted 10 lines · Ingo Molnar
Review
Quoted 2 lines · Steven Rostedt
(quoted diff) +2
-0+ if (*count != -1) + (*count)--;minor nit: it's a tiny bit more compact as:
Review
Quoted 2 lines
(quoted diff) +2
-0+ if (*count != -1) + --*count;
Doesn't that move the pointer? Cases like these, I prefer the parenthesis, because I get confused about C ordering.
Quoted 6 lines
and this:
Quoted 3 lines
return
register_tracer(&function_trace);
}
Quoted 7 lines
Quoted 2 lines
device_initcall(init_function_trace);
+there should be no newline between
device_initcall()and the preceding function. (we can think of it as a late attribute of the function, attached to the function. We do the same for EXPORT_SYMBOL's too, etc.)
OK,
Do you want me to rebase, or just send out new "clean up" patches?
-- Steve
Re: [PATCH 12/15] ftrace: add traceon traceoff commands to enable/disable the buffers
Quoted 26 lines · Steven Rostedt
Quoted 10 lines · Ingo Molnar
Review
Quoted 2 lines · Steven Rostedt
(quoted diff) +2
-0+ if (*count != -1) + (*count)--;minor nit: it's a tiny bit more compact as:
Review
Quoted 2 lines
(quoted diff) +2
-0+ if (*count != -1) + --*count;Doesn't that move the pointer? Cases like these, I prefer the parenthesis, because I get confused about C ordering.
Quoted 9 lines
and this:
Quoted 5 lines
kernel/trace/trace_functions.c +1
-0return register_tracer (&function_trace); }device_initcall (init_function_trace); +
Quoted 8 lines
Quoted 4 lines
there should be no newline between
device_initcall()and the preceding function. (we can think of it as a late attribute of the function, attached to the function. We do the same for EXPORT_SYMBOL's too, etc.)OK,
Do you want me to rebase, or just send out new "clean up" patches?
I already pulled (these are small details, not showstoppers), please do a delta cleanup of those files.
( there's a few other instances of the same kind of problems -
please look at the whole file and try to figure out how to make them look nicer. )
Ingo
Steven Rostedt
maintainerPATCH 13/15 ftrace: show selected functions in set_ftrace_filter
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
This patch adds output to show what functions have tracer hooks attached to them.
# echo 'sys_open:traceon:4' > /debug/tracing/set_ftrace_filter # cat set_ftrace_filter
#### all functions enabled ####
sys_open:ftrace_traceon:0000000000000004
# echo 'do_fork:traceoff:' > set_ftrace_filter # cat set_ftrace_filter
#### all functions enabled ####
sys_open:ftrace_traceon:0000000000000002
do_fork:ftrace_traceoff:ffffffffffffffff
Note the 4 changed to a 2. This is because The code was executed twice since the traceoff was added. If a cat is done again:
#### all functions enabled ####
sys_open:ftrace_traceon
do_fork:ftrace_traceoff:ffffffffffffffff
The number disappears. That is because it will not print a NULL.
Callbacks to allow the tracer to pretty print will be implemented soon.
--- kernel/trace/ftrace.c | 123 +++++++++++++++++++++++++++++++++++++++++-------- 1 files changed, 103 insertions(+), 20 deletions(-)
kernel/trace/ftrace.c +103 -20
@@ -45,14 +45,14 @@ftrace_kill (); \ }while (0 ) +/* hash bits for specific function selection */ +#define FTRACE_HASH_BITS 7 +#define FTRACE_FUNC_HASHSIZE (1 << FTRACE_HASH_BITS) +/* ftrace_enabled is a method to turn ftrace on or off */ int ftrace_enabled__read_mostly ;static int last_ftrace_enabled; -/* set when tracing only a pid */ -struct pid *ftrace_pid_trace; -static struct pid *const ftrace_swapper_pid = &init_struct_pid; -/* Quick disabling of function tracer. */ int function_trace_stop;
@@ -248,6 +248,21 @@ static void ftrace_update_pid_func(void)# error Dynamic ftrace depends on MCOUNT_RECORD#endif +/* set when tracing only a pid */ +struct pid *ftrace_pid_trace; +static struct pid *const ftrace_swapper_pid = &init_struct_pid; +static struct hlist_head ftrace_func_hash[FTRACE_FUNC_HASHSIZE ]__read_mostly ; + +struct ftrace_func_hook { +struct hlist_node node; +struct ftrace_hook_ops *ops; +unsigned long flags; +unsigned long ip; +void *data; +struct rcu_head rcu; +}; + +enum {FTRACE_ENABLE_CALLS = (1 <<0 ),FTRACE_DISABLE_CALLS = (1 <<1 ),
@@ -750,12 +765,14 @@ enum {FTRACE_ITER_NOTRACE = (1 <<2 ),FTRACE_ITER_FAILURES = (1 <<3 ),FTRACE_ITER_PRINTALL = (1 <<4 ), + FTRACE_ITER_HASH = (1 << 5), };#define FTRACE_BUFF_MAX (KSYM_SYMBOL_LEN+4)/* room for wildcards */ struct ftrace_iterator {struct ftrace_page *pg; +int hidx;int idx;unsigned flags;unsigned char buffer[FTRACE_BUFF_MAX +1 ];
@@ -764,17 +781,86 @@ struct ftrace_iterator { };static void * +t_hash_next (struct seq_file *m,void *v,loff_t *pos) +{ +struct ftrace_iterator *iter = m->private; +struct hlist_node *hnd = v; +struct hlist_head *hhd; + +WARN_ON (!(iter->flags &FTRACE_ITER_HASH )); + + (*pos)++; + + retry: +if (iter->hidx >=FTRACE_FUNC_HASHSIZE ) +return NULL ; + + hhd = &ftrace_func_hash[iter->hidx]; + +if (hlist_empty (hhd)) { + iter->hidx++; + hnd =NULL ; + goto retry; + } + +if (!hnd) + hnd = hhd->first; +else { + hnd = hnd->next; +if (!hnd) { + iter->hidx++; + goto retry; + } + } + +return hnd; +} + +static void *t_hash_start (struct seq_file *m,loff_t *pos) +{ +struct ftrace_iterator *iter = m->private; +void *p =NULL ; + + iter->flags |=FTRACE_ITER_HASH ; + +return t_hash_next (m, p, pos); +} + +static int t_hash_show (struct seq_file *m,void *v) +{ +struct ftrace_func_hook *rec; +struct hlist_node *hnd = v; +char str[KSYM_SYMBOL_LEN ]; + + rec = hlist_entry(hnd, struct ftrace_func_hook, node); + +kallsyms_lookup (rec->ip,NULL ,NULL ,NULL , str); +seq_printf (m,"%s:" , str); + +kallsyms_lookup ((unsigned long )rec->ops->func,NULL ,NULL ,NULL , str); +seq_printf (m,"%s" , str); + +if (rec->data) +seq_printf (m,":%p" , rec->data); +seq_putc (m,'\n' ); + +return 0 ; +} + +static void *t_next (struct seq_file *m,void *v,loff_t *pos) {struct ftrace_iterator *iter = m->private;struct dyn_ftrace *rec =NULL ; +if (iter->flags &FTRACE_ITER_HASH ) +return t_hash_next (m, v, pos); + (*pos)++;if (iter->flags &FTRACE_ITER_PRINTALL )return NULL ; -mutex_lock (&ftrace_lock); retry:if (iter->idx >= iter->pg->index) {if (iter->pg->next) {
@@ -803,7 +889,6 @@ t_next(struct seq_file *m, void *v, loff_t *pos) goto retry; } } -mutex_unlock (&ftrace_lock);return rec; }
@@ -813,6 +898,7 @@ static void *t_start(struct seq_file *m, loff_t *pos)struct ftrace_iterator *iter = m->private;void *p =NULL ; +mutex_lock (&ftrace_lock);/* * For set_ftrace_filter reading, if we have the filter * off, we can short cut and just print out that all
@@ -820,12 +906,15 @@ static void *t_start(struct seq_file *m, loff_t *pos)*/ if (iter->flags &FTRACE_ITER_FILTER && !ftrace_filtered) {if (*pos >0 ) -return NULL ; +return t_hash_start (m, pos); iter->flags |=FTRACE_ITER_PRINTALL ; (*pos)++;return iter; } +if (iter->flags &FTRACE_ITER_HASH ) +return t_hash_start (m, pos); +if (*pos >0 ) {if (iter->idx <0 )return p;
@@ -835,11 +924,15 @@ static void *t_start(struct seq_file *m, loff_t *pos) p =t_next (m, p, pos); +if (!p) +return t_hash_start (m, pos); +return p; }static void t_stop (struct seq_file *m,void *p) { +mutex_unlock (&ftrace_lock); }static int t_show (struct seq_file *m,void *v)
@@ -848,6 +941,9 @@ static int t_show(struct seq_file *m, void *v)struct dyn_ftrace *rec = v;char str[KSYM_SYMBOL_LEN ]; +if (iter->flags &FTRACE_ITER_HASH ) +return t_hash_show (m, v); +if (iter->flags &FTRACE_ITER_PRINTALL ) {seq_printf (m,"#### all functions enabled ####\n" );return 0 ;
@@ -1246,19 +1342,6 @@ static int __init ftrace_mod_cmd_init(void) }device_initcall (ftrace_mod_cmd_init); -#define FTRACE_HASH_BITS 7 -#define FTRACE_FUNC_HASHSIZE (1 << FTRACE_HASH_BITS) -static struct hlist_head ftrace_func_hash[FTRACE_FUNC_HASHSIZE ]__read_mostly ; - -struct ftrace_func_hook { -struct hlist_node node; -struct ftrace_hook_ops *ops; -unsigned long flags; -unsigned long ip; -void *data; -struct rcu_head rcu; -}; -static void function_trace_hook_call (unsigned long ip,unsigned long parent_ip) {
Signature · 3 lines
1.5.6.5 --
Steven Rostedt
maintainerPATCH 14/15 ftrace: add pretty print to selected fuction traces
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
This patch adds a call back for the tracers that have hooks to selected functions. This allows the tracer to show better output in the set_ftrace_filter file.
--- include/linux/ftrace.h | 6 ++++++ kernel/trace/ftrace.c | 3 +++ 2 files changed, 9 insertions(+), 0 deletions(-)
include/linux/ftrace.h +6 -0
@@ -106,12 +106,18 @@ struct ftrace_func_command {/* asm/ftrace.h must be defined for archs supporting dynamic ftrace */ #include <asm/ftrace.h> +struct seq_file ; +struct ftrace_hook_ops {void (*func)(unsigned long ip,unsigned long parent_ip,void **data);int (*callback)(unsigned long ip,void **data);void (*free)(void **data); +int (*print)(struct seq_file *m, +unsigned long ip, +struct ftrace_hook_ops *ops, +void *data); }; extern int
kernel/trace/ftrace.c +3 -0
@@ -834,6 +834,9 @@ static int t_hash_show(struct seq_file *m, void *v) rec = hlist_entry(hnd, struct ftrace_func_hook, node); +if (rec->ops->print) +return rec->ops->kallsyms_lookup (rec->ip,NULL ,NULL ,NULL , str);seq_printf (m,"%s:" , str);
Signature · 3 lines
1.5.6.5 --
Steven Rostedt
maintainerPATCH 15/15 ftrace: add pretty print function for traceon and traceoff hooks
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
This patch adds a pretty print version of traceon and traceoff output for set_ftrace_filter.
# echo 'sys_open:traceon:4' > set_ftrace_filter # cat set_ftrace_filter
#### all functions enabled ####
sys_open:traceon:count=4
--- kernel/trace/trace_functions.c | 28 ++++++++++++++++++++++++++++ 1 files changed, 28 insertions(+), 0 deletions(-)
kernel/trace/trace_functions.c +28 -0
@@ -267,15 +267,43 @@ ftrace_traceoff(unsigned long ip, unsigned long parent_ip, void **data)tracing_off (); } +static int +ftrace_trace_onoff_print (struct seq_file *m,unsigned long ip, +struct ftrace_hook_ops *ops,void *data); +static struct ftrace_hook_ops traceon_hook_ops = { .func = ftrace_traceon, + .print = ftrace_trace_onoff_print, };static struct ftrace_hook_ops traceoff_hook_ops = { .func = ftrace_traceoff, + .print = ftrace_trace_onoff_print, };static int +ftrace_trace_onoff_print (struct seq_file *m,unsigned long ip, +struct ftrace_hook_ops *ops,void *data) +{ +char str[KSYM_SYMBOL_LEN ]; +long count = (long )data; + +kallsyms_lookup (ip,NULL ,NULL ,NULL , str); +seq_printf (m,"%s:" , str); + +if (ops == &traceon_hook_ops) +seq_printf (m,"traceon" ); +else +seq_printf (m,"traceoff" ); + +if (count !=-1 ) +seq_printf (m,":count=%ld" , count); +seq_putc (m,'\n' ); + +return 0 ; +} + +static int ftrace_trace_onoff_unreg (char *glob,char *cmd,char *param) {struct ftrace_hook_ops *ops;
Signature · 3 lines
1.5.6.5 --
Quoted 13 lines · Steven Rostedt
Ingo,
This is a patch series that offers a variety of features.
The first is to add:
#### all functions enabled ####
to
set_ftrace_filterwhen all functions will be traced when function tracing is on.The next is to add a :command: interface, and the module command:
Quoted 8 lines
echo '*:mod:ext4' >
set_ftrace_filterWill select all functions in the ext4 module.
echo '!*write*:mod:ext4' >
set_ftrace_filterwill remove all functions with the characters 'write' and are in ext4.
Very nice feature!
I'm wondering, would it be possible to somehow extend this to built-in drivers too, so that we get symmetry of usage independently of whether something is built in or a module? I suspect it needs some extensions to kbuild though ...
Quoted 4 lines
Next we extend the :command: interface as well as add a way to hook tracers to specific functions. Instead of tracing all functions the same, this patch series adds a hash to be able to trace different functions differently.
That's a nice extension too!
Small naming detail: instead of calling it a 'hook' (which really has a bad sound to it for various reasons), how about calling it a 'trigger' and/or a 'callback'?
Quoted 6 lines
I implement a 'traceon' and 'traceoff' to have a function start or stop tracing when it is hit. It also includes a counter for the number of times to implement the trace on/off.
echo '*_lock:traceon:4' >
set_ftrace_filter
echo '*_unlock:traceoff:5' >set_ftrace_filter
if there's no counter specified, does it default to 1 [single-shot trigger] or infinity?
Quoted 28 lines
This will have all functions that end with
_lockturn on tracing and all that end with_unlockdisable it, 4 times each.The following patches are in:
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.gitbranch: tip/tracing/ftrace
Steven Rostedt (15):
ftrace: state that all functions are enabled inset_ftrace_filter
ftrace: adddo_for_each_ftrace_recandwhile_for_each_ftrace_rec
ftrace: renameftrace_matchtoftrace_match_records
ftrace: break upftrace_match_recordsinto smaller components
ftrace: add module command function filter selection
ftrace: enable filtering only when a function is filtered on
ftrace: add command interface for function selection
ftrace: convertftrace_lockfrom a spinlock to mutex
ftrace: consolidate mutexes
ftrace: trace different functions with a different tracer
ring-buffer: addtracing_is_onto test if ring buffer is enabled
ftrace: add traceon traceoff commands to enable/disable the buffers
ftrace: show selected functions inset_ftrace_filter
ftrace: add pretty print to selected fuction traces
ftrace: add pretty print function for traceon and traceoff hooks
Quoted 7 lines
----
include/linux/ftrace.h | 40 ++ include/linux/ring_buffer.h | 2 + kernel/trace/ftrace.c | 929 +++++++++++++++++++++++++++++++--------- kernel/trace/ring_buffer.c | 9 + kernel/trace/trace_functions.c | 163 +++++++ 5 files changed, 946 insertions(+), 197 deletions(-)
Pulled into tip:tracing/ftrace, thanks Steve!
Ingo
Steven Rostedt
maintainerQuoted 25 lines · Ingo Molnar
Quoted 13 lines · Steven Rostedt
Ingo,
This is a patch series that offers a variety of features.
The first is to add:
#### all functions enabled ####
to
set_ftrace_filterwhen all functions will be traced when function tracing is on.The next is to add a :command: interface, and the module command:
Quoted 8 lines
echo '*:mod:ext4' >
set_ftrace_filterWill select all functions in the ext4 module.
echo '!*write*:mod:ext4' >
set_ftrace_filterwill remove all functions with the characters 'write' and are in ext4.
Quoted 6 lines
Very nice feature!
I'm wondering, would it be possible to somehow extend this to built-in drivers too, so that we get symmetry of usage independently of whether something is built in or a module? I suspect it needs some extensions to kbuild though ...
Yeah, that may needs some kbuild magic.
Quoted 11 lines
Quoted 5 lines
Next we extend the :command: interface as well as add a way to hook tracers to specific functions. Instead of tracing all functions the same, this patch series adds a hash to be able to trace different functions differently.
That's a nice extension too!
Small naming detail: instead of calling it a 'hook' (which really has a bad sound to it for various reasons), how about calling it a 'trigger' and/or a 'callback'?
Hmm, I can switch it to trigger. I used callback in too many other places in this series, and that would lead to more confusion.
Quoted 10 lines
Quoted 7 lines
I implement a 'traceon' and 'traceoff' to have a function start or stop tracing when it is hit. It also includes a counter for the number of times to implement the trace on/off.
echo '*_lock:traceon:4' >
set_ftrace_filter
echo '*_unlock:traceoff:5' >set_ftrace_filterif there's no counter specified, does it default to 1 [single-shot trigger] or infinity?
Bad documentation on my part :-/
No number is infinity, since infinity is no number ;-)
Quoted 37 lines
Quoted 29 lines
This will have all functions that end with
_lockturn on tracing and all that end with_unlockdisable it, 4 times each.The following patches are in:
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.gitbranch: tip/tracing/ftrace
Steven Rostedt (15):
ftrace: state that all functions are enabled inset_ftrace_filter
ftrace: adddo_for_each_ftrace_recandwhile_for_each_ftrace_rec
ftrace: renameftrace_matchtoftrace_match_records
ftrace: break upftrace_match_recordsinto smaller components
ftrace: add module command function filter selection
ftrace: enable filtering only when a function is filtered on
ftrace: add command interface for function selection
ftrace: convertftrace_lockfrom a spinlock to mutex
ftrace: consolidate mutexes
ftrace: trace different functions with a different tracer
ring-buffer: addtracing_is_onto test if ring buffer is enabled
ftrace: add traceon traceoff commands to enable/disable the buffers
ftrace: show selected functions inset_ftrace_filter
ftrace: add pretty print to selected fuction traces
ftrace: add pretty print function for traceon and traceoff hooksQuoted 7 lines
----
include/linux/ftrace.h | 40 ++ include/linux/ring_buffer.h | 2 + kernel/trace/ftrace.c | 929 +++++++++++++++++++++++++++++++--------- kernel/trace/ring_buffer.c | 9 + kernel/trace/trace_functions.c | 163 +++++++ 5 files changed, 946 insertions(+), 197 deletions(-)
Quoted 1 line
Pulled into tip:tracing/ftrace, thanks Steve!
Thanks! This answers my other question. I'll be sending out clean up patches later today.
-- Steve
Quoted 7 lines · Steven Rostedt
Quoted 7 lines
Quoted 6 lines
I implement a 'traceon' and 'traceoff' to have a function start or stop tracing when it is hit. It also includes a counter for the number of times to implement the trace on/off.
echo '*_lock:traceon:4' >
set_ftrace_filter
echo '*_unlock:traceoff:5' >set_ftrace_filter
Quoted 6 lines
Quoted 2 lines
if there's no counter specified, does it default to 1 [single-shot trigger] or infinity?
Bad documentation on my part :-/
No number is infinity, since infinity is no number ;-)
Well infinity is a number too, in most schools of algebra, just one with unusual properties :)
Ingo
Quoted 28 lines · Steven Rostedt
Quoted 25 lines · Ingo Molnar
Quoted 21 lines · Steven Rostedt
Ingo,
This is a patch series that offers a variety of features.
The first is to add:
#### all functions enabled ####
to
set_ftrace_filterwhen all functions will be traced when function tracing is on.The next is to add a :command: interface, and the module command:
echo '*:mod:ext4' >
set_ftrace_filterWill select all functions in the ext4 module.
echo '!*write*:mod:ext4' >
set_ftrace_filterwill remove all functions with the characters 'write' and are in ext4.
Quoted 8 lines
Quoted 6 lines
Very nice feature!
I'm wondering, would it be possible to somehow extend this to built-in drivers too, so that we get symmetry of usage independently of whether something is built in or a module? I suspect it needs some extensions to kbuild though ...
Yeah, that may needs some kbuild magic.
I have not followed this thread.
What kind of magic do you have in mind?
Sam
Quoted 36 lines · Sam Ravnborg
Quoted 35 lines · Steven Rostedt
Quoted 31 lines · Ingo Molnar
Quoted 13 lines · Steven Rostedt
Ingo,
This is a patch series that offers a variety of features.
The first is to add:
#### all functions enabled ####
to
set_ftrace_filterwhen all functions will be traced when function tracing is on.The next is to add a :command: interface, and the module command:
Quoted 8 lines
echo '*:mod:ext4' >
set_ftrace_filterWill select all functions in the ext4 module.
echo '!*write*:mod:ext4' >
set_ftrace_filterwill remove all functions with the characters 'write' and are in ext4.
Very nice feature!
I'm wondering, would it be possible to somehow extend this to built-in drivers too, so that we get symmetry of usage independently of whether something is built in or a module? I suspect it needs some extensions to kbuild though ...
Quoted 4 lines
Quoted 1 line
Yeah, that may needs some kbuild magic.
I have not followed this thread.
What kind of magic do you have in mind?
It would be nice to have a notion of 'modules' even for built-in drivers - at least as far as symbol lookups go.
It would allow such a module-specific trigger:
echo '!*write*:mod:ext4' > set_ftrace_filter
to work even if ext4 is built-in.
Ingo
Steven Rostedt
maintainerQuoted 4 lines · Ingo Molnar
Small naming detail: instead of calling it a 'hook' (which really has a bad sound to it for various reasons), how about calling it a 'trigger' and/or a 'callback'?
I started converting the names from hook to trigger, and the name looks funny to me. I asked on IRC for opinions and Luis suggested "_probe". I think I like that idea.
Ingo, do you agree that "_probe" would be a good substitute for "_hook"?
-- Steve
Quoted 7 lines · Steven Rostedt
Quoted 4 lines · Ingo Molnar
Small naming detail: instead of calling it a 'hook' (which really has a bad sound to it for various reasons), how about calling it a 'trigger' and/or a 'callback'?
Quoted 5 lines
I started converting the names from hook to trigger, and the name looks funny to me. I asked on IRC for opinions and Luis suggested "
_probe". I think I like that idea.Ingo, do you agree that "
_probe" would be a good substitute for "_hook"?
yeah, that's OK too i think.
Ingo
Quoted 28 lines · Steven Rostedt
Ingo,
This is a patch series that offers a variety of features.
The first is to add:
#### all functions enabled ####
to
set_ftrace_filterwhen all functions will be traced when function tracing is on.The next is to add a :command: interface, and the module command:
echo '*:mod:ext4' >
set_ftrace_filterWill select all functions in the ext4 module.
echo '!*write*:mod:ext4' >
set_ftrace_filterwill remove all functions with the characters 'write' and are in ext4.
Next we extend the :command: interface as well as add a way to hook tracers to specific functions. Instead of tracing all functions the same, this patch series adds a hash to be able to trace different functions differently.
Quoted 6 lines
I implement a 'traceon' and 'traceoff' to have a function start or stop tracing when it is hit. It also includes a counter for the number of times to implement the trace on/off.
echo '*_lock:traceon:4' >
set_ftrace_filter
echo '*_unlock:traceoff:5' >set_ftrace_filter
That's great!
I guess it could be useful to be able to pass it as a kernel parameter. Something like that ftrace=function_tracer ftrace_filter='*_lock:traceon:4'
Quoted 35 lines
This will have all functions that end with
_lockturn on tracing and all that end with_unlockdisable it, 4 times each.The following patches are in:
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.gitbranch: tip/tracing/ftrace
Steven Rostedt (15):
ftrace: state that all functions are enabled inset_ftrace_filter
ftrace: adddo_for_each_ftrace_recandwhile_for_each_ftrace_rec
ftrace: renameftrace_matchtoftrace_match_records
ftrace: break upftrace_match_recordsinto smaller components
ftrace: add module command function filter selection
ftrace: enable filtering only when a function is filtered on
ftrace: add command interface for function selection
ftrace: convertftrace_lockfrom a spinlock to mutex
ftrace: consolidate mutexes
ftrace: trace different functions with a different tracer
ring-buffer: addtracing_is_onto test if ring buffer is enabled
ftrace: add traceon traceoff commands to enable/disable the buffers
ftrace: show selected functions inset_ftrace_filter
ftrace: add pretty print to selected fuction traces
ftrace: add pretty print function for traceon and traceoff hooks----
include/linux/ftrace.h | 40 ++ include/linux/ring_buffer.h | 2 + kernel/trace/ftrace.c | 929 +++++++++++++++++++++++++++++++--------- kernel/trace/ring_buffer.c | 9 + kernel/trace/trace_functions.c | 163 +++++++ 5 files changed, 946 insertions(+), 197 deletions(-)