Optimize pid filters and add --no-filter option
v3 cover Optimize pid filters and add --no-filter option
This patchset optimizes how pid filters are expressed and makes it less likely that we overflow ftrace filters' size limit of one page.
Changes since v2:
Append exclude rules with &&
Changes since v1:
Add missing tags
Fix append_filter_pid_range() callers to pass valid range as [min,max]
Slavomir Kaslev (2):
trace-cmd: Optimize how pid filters are expressed
trace-cmd: Add --no-filter option to not filter recording processes
tracecmd/trace-record.c | 131 ++++++++++++++++++++++++++++------------ tracecmd/trace-usage.c | 1 + 2 files changed, 92 insertions(+), 40 deletions(-)
Signature · 1 line
2.19.1
PATCH v3 1/2 trace-cmd: Optimize how pid filters are expressed
Express pid filters as allowed/disallowed filter ranges
(pid>=100&&pid<=103)
instead of specifying them per pid
(pid==100||pid==101||pid==102||pid==103)
This makes the size of the resulting filter smaller (and faster) and avoids overflowing the filter size limit of one page which we can hit on bigger machines (say >160 CPUs).
--- tracecmd/trace-record.c | 117 +++++++++++++++++++++++++++------------- 1 file changed, 80 insertions(+), 37 deletions(-)
tracecmd/trace-record.c +80 -37
@@ -950,10 +950,63 @@ static void update_ftrace_pids(int reset)static void update_event_filters (struct buffer_instance *instance);static void update_pid_event_filters (struct buffer_instance *instance); +static void append_filter_pid_range (char **filter,int *curr_len, +const char *field, +int start_pid,int end_pid,bool exclude) +{ +const char *op ="" , *op1, *op2, *op3; +int len; + +if (*filter && **filter) + op = exclude ?"&&" :"||" ; + +/* Handle thus case explicitly so that we get `pid==3` instead of +* `pid>=3&&pid<=3` for singleton ranges +*/ +if (start_pid == end_pid) { +#define FMT "%s(%s%s%d)" + len =snprintf (NULL ,0 ,FMT , op, + field, exclude ?"!=" :"==" , start_pid); + *filter =realloc (*filter, *curr_len + len +1 ); +if (!*filter) +die ("realloc" ); + + len =snprintf (*filter + *curr_len, len +1 ,FMT , op, + field, exclude ?"!=" :"==" , start_pid); + *curr_len += len; + +return ; +#undef FMT + } + +if (exclude) { + op1 ="<" ; + op2 ="||" ; + op3 =">" ; + }else { + op1 =">=" ; + op2 ="&&" ; + op3 ="<=" ; + } + +#define FMT "%s(%s%s%d%s%s%s%d)" + len =snprintf (NULL ,0 ,FMT , op, + field, op1, start_pid, op2, + field, op3, end_pid); + *filter =realloc (*filter, *curr_len + len +1 ); +if (!*filter) +die ("realloc" ); + + len =snprintf (*filter + *curr_len, len +1 ,FMT , op, + field, op1, start_pid, op2, + field, op3, end_pid); + *curr_len += len; +} +/** * make_pid_filter - create a filter string to all pids against @field * @curr_filter: Append to a previous filter (may realloc). Can be NULL -* @field: The fild to compare the pids against +* @field: The field to compare the pids against * * Creates a new string or appends to an existing one if @curr_filter * is not NULL. The new string will contain a filter with all pids
@@ -963,54 +1016,44 @@ static void update_pid_event_filters(struct buffer_instance *instance);*/ static char *make_pid_filter (char *curr_filter,const char *field) { +int curr_len =0 , last_exclude =-1 ; +int start_pid =-1 , last_pid =-1 ; +char *filter =NULL , *save;struct filter_pids *p; -char *filter; -char *orit; -char *match; -char *str; -int curr_len =0 ; -int len;/* Use the new method if possible */ if (have_set_event_pid)return NULL ; - len = len_filter_pids + (strlen (field) +strlen ("(==)||" )) * nr_filter_pids; - -if (curr_filter) { - curr_len =strlen (curr_filter); - filter =realloc (curr_filter, curr_len + len +strlen ("(&&())" )); -if (!filter) -die ("realloc" ); -memmove (filter+1 , curr_filter, curr_len); - filter[0 ] ='(' ; -strcat (filter,")&&(" ); - curr_len =strlen (filter); - }else - filter =malloc (len); -if (!filter) -die ("Failed to allocate pid filter" ); - -/* Last '||' that is not used will cover the \0 */ - str = filter + curr_len; +if (!filter_pids) +return curr_filter;for (p = filter_pids; p; p = p->next) { -if (p->exclude) { - match ="!=" ; - orit ="&&" ; - }else { - match ="==" ; - orit ="||" ; +/* PIDs are inserted in `filter_pids` from the front and that's +* why we expect them in descending order here. +*/ +if (p->pid == last_pid -1 && p->exclude == last_exclude) { + last_pid = p->pid; +continue ; } -if (p == filter_pids) - orit ="" ; - len =sprintf (str,"%s(%s%s%d)" , orit, field, match, p->pid); - str += len; +if (start_pid !=-1 ) +append_filter_pid_range (&filter, &curr_len, field, + last_pid, start_pid, + last_exclude); + + start_pid = last_pid = p->pid; + last_exclude = p->exclude; + } +append_filter_pid_range (&filter, &curr_len, field, + last_pid, start_pid, last_exclude); -if (curr_len) -sprintf (str,")" ); +if (curr_filter) { + save = filter; +asprintf (&filter,"(%s)&&(%s)" , curr_filter, filter); +free (save); + }return filter; }
Signature · 1 line
2.19.1
Re: [PATCH v3 1/2] trace-cmd: Optimize how pid filters are expressed
tracecmd/trace-record.c static void update_pid_event_filters(struct buffer_instance *instance); · line 1019
Quoted 3 lines · On Tue, 16 Apr 2019 02:00:15 +0300 Slavomir Kaslev <[email protected]> wrote:
static char *make_pid_filter(char
*curr_filter, const char *field) {tracecmd/trace-record.c +1
-0+int curr_len =0 , last_exclude =-1 ;
Small nit. Usually when adding multiple variables on one line like this, the variables should be related. Because curr_len and last_exclude are not related, it is best to keep them separate.
tracecmd/trace-record.c static void update_pid_event_filters(struct buffer_instance *instance); · line 1020
Quoted 1 line
tracecmd/trace-record.c +1
-0+int start_pid =-1 , last_pid =-1 ;
start_pid and last_pid are fine on the same line or separate.
tracecmd/trace-record.c static void update_pid_event_filters(struct buffer_instance *instance); · line 1021
Quoted 1 line
tracecmd/trace-record.c +1
-0+char *filter =NULL , *save;
Note, It's better to keep variables separate. It makes it easier to see them and also when code changes and you get the "variable x is not used" warnings from gcc, it's easier to remove them. In emacs it brings you to the warning and places the cursor on the problem line. Then a simple "delete line" works well.
save could also be moved below.
Quoted 31 lines
tracecmd/trace-record.c +1
-24struct filter_pids *p; -char *filter; -char *orit; -char *match; -char *str; -int curr_len =0 ; -int len;/* Use the new method if possible */ if (have_set_event_pid)return NULL ; - len = len_filter_pids + (strlen (field) +strlen ("(==)||" )) * nr_filter_pids; - -if (curr_filter) { - curr_len =strlen (curr_filter); - filter =realloc (curr_filter, curr_len + len +strlen ("(&&())" )); -if (!filter) -die ("realloc" ); -memmove (filter+1 , curr_filter, curr_len); - filter[0 ] ='(' ; -strcat (filter,")&&(" ); - curr_len =strlen (filter); - }else - filter =malloc (len); -if (!filter) -die ("Failed to allocate pid filter" ); - -/* Last '||' that is not used will cover the \0 */ - str = filter + curr_len; +if (!filter_pids)
tracecmd/trace-record.c static void update_pid_event_filters(struct buffer_instance *instance); · line 1032
Quoted 10 lines
tracecmd/trace-record.c +2
-6+return curr_filter;for (p = filter_pids; p; p = p->next) { -if (p->exclude) { - match ="!=" ; - orit ="&&" ; - }else { - match ="==" ; - orit ="||" ; +/* PIDs are inserted in `filter_pids` from the front and that's
Comment notation should be:
/* * PIDs are inserted in ...
Only the Networking folks are allowed to do it that way in the kernel ;-)
Quoted 16 lines
tracecmd/trace-record.c +10
-4+* why we expect them in descending order here. +*/ +if (p->pid == last_pid -1 && p->exclude == last_exclude) { + last_pid = p->pid; +continue ; } -if (p == filter_pids) - orit ="" ; - len =sprintf (str,"%s(%s%s%d)" , orit, field, match, p->pid); - str += len; +if (start_pid !=-1 ) +append_filter_pid_range (&filter, &curr_len, field, + last_pid, start_pid, + last_exclude); +
tracecmd/trace-record.c static void update_pid_event_filters(struct buffer_instance *instance); · line 1052
Quoted 10 lines
tracecmd/trace-record.c +6
-2+ start_pid = last_pid = p->pid; + last_exclude = p->exclude; + } +append_filter_pid_range (&filter, &curr_len, field, + last_pid, start_pid, last_exclude); -if (curr_len) -sprintf (str,")" ); +if (curr_filter) {
As save is only used here, we could have:
char *save = filter;
tracecmd/trace-record.c static void update_pid_event_filters(struct buffer_instance *instance); · line 1059
Quoted 7 lines
tracecmd/trace-record.c +4
-0+ save = filter; +asprintf (&filter,"(%s)&&(%s)" , curr_filter, filter); +free (save); + }return filter; }
-- Steve
Re: [PATCH v3 1/2] trace-cmd: Optimize how pid filters are expressed
Quoted 10 lines · Steven Rostedt
tracecmd/trace-record.c static void update_pid_event_filters(struct buffer_instance *instance); · line 1019
Quoted 3 lines · On Tue, 16 Apr 2019 02:00:15 +0300 Slavomir Kaslev <[email protected]> wrote:
static char *make_pid_filter(char
*curr_filter, const char *field) {tracecmd/trace-record.c +1
-0+int curr_len =0 , last_exclude =-1 ;Small nit. Usually when adding multiple variables on one line like this, the variables should be related. Because
curr_lenandlast_excludeare not related, it is best to keep them separate.
Sent v4 addressing those and added a section about --no-filter to trace-cmd record's man page.
Cheers,
-- Slavi
PATCH v3 2/2 trace-cmd: Add --no-filter option to not filter recording processes
In reply to Slavomir KaslevAdd --no-filter option which doesn't install filters for the trace-cmd recording processes pids.
--- tracecmd/trace-record.c | 14 +++++++++++--- tracecmd/trace-usage.c | 1 + 2 files changed, 12 insertions(+), 3 deletions(-)
tracecmd/trace-record.c +11 -3
@@ -86,6 +86,7 @@ static int do_ptrace;static int filter_task;static int filter_pid =-1 ; +static bool no_filter = false;static int local_cpu_count;
@@ -1063,6 +1064,9 @@ static void update_task_filter(void)struct buffer_instance *instance;int pid =getpid (); +if (no_filter) +return ; +if (filter_task)add_filter_pid (pid,0 );
@@ -4377,9 +4381,9 @@ void update_first_instance(struct buffer_instance *instance, int topt) }enum { - - OPT_quiet =246 , - OPT_debug =247 , + OPT_quiet =245 , + OPT_debug =246 , + OPT_no_filter =247 , OPT_max_graph_depth =248 , OPT_tsoffset =249 , OPT_bycomm =250 ,
@@ -4603,6 +4607,7 @@ static void parse_record_options(int argc, {"by-comm", no_argument, NULL, OPT_bycomm}, {"ts-offset", required_argument, NULL, OPT_tsoffset}, {"max-graph-depth", required_argument, NULL, OPT_max_graph_depth}, + {"no-filter", no_argument, NULL, OPT_no_filter}, {"debug", no_argument, NULL, OPT_debug}, {"quiet", no_argument, NULL, OPT_quiet}, {"help", no_argument, NULL, '?'},
@@ -4877,6 +4882,9 @@ static void parse_record_options(int argc,if (!ctx->max_graph_depth)die ("Could not allocate option" );break ; +case OPT_no_filter: + no_filter = true; +break ;case OPT_debug: debug =1 ;break ;
tracecmd/trace-usage.c +1 -0
@@ -56,6 +56,7 @@ static struct usage_help usage_help[] = {" --func-stack perform a stack trace for function tracer\n" " (use with caution)\n" " --max-graph-depth limit function_graph depth\n" +" --no-filter do not set any event filters\n" }, { "start",
Signature · 1 line
2.19.1
Re: [PATCH v3 2/2] trace-cmd: Add --no-filter option to not filter recording processes
Quoted 5 lines · On Tue, 16 Apr 2019 02:00:16 +0300 Slavomir Kaslev <[email protected]> wrote:
Add --no-filter option which doesn't install filters for the trace-cmd recording processes pids.
Signed-off-by Slavomir Kaslev <[email protected]>Suggested-by Steven Rostedt (VMware) <[email protected]>
I'll apply this, but we should also update the man page.
-- Steve
Quoted 70 lines
--- tracecmd/trace-record.c | 14 +++++++++++--- tracecmd/trace-usage.c | 1 + 2 files changed, 12 insertions(+), 3 deletions(-)tracecmd/trace-record.c +11
-3@@ -86,6 +86,7 @@ static int do_ptrace;static int filter_task;static int filter_pid =-1 ; +static bool no_filter = false;static int local_cpu_count;@@ -1063,6 +1064,9 @@ static void update_task_filter(void)struct buffer_instance *instance;int pid =getpid (); +if (no_filter) +return ; +if (filter_task)add_filter_pid (pid,0 );@@ -4377,9 +4381,9 @@ void update_first_instance(struct buffer_instance *instance, int topt) }enum { - - OPT_quiet =246 , - OPT_debug =247 , + OPT_quiet =245 , + OPT_debug =246 , + OPT_no_filter =247 , OPT_max_graph_depth =248 , OPT_tsoffset =249 , OPT_bycomm =250 ,@@ -4603,6 +4607,7 @@ static void parse_record_options(int argc, {"by-comm", no_argument, NULL, OPT_bycomm}, {"ts-offset", required_argument, NULL, OPT_tsoffset}, {"max-graph-depth", required_argument, NULL, OPT_max_graph_depth}, + {"no-filter", no_argument, NULL, OPT_no_filter}, {"debug", no_argument, NULL, OPT_debug}, {"quiet", no_argument, NULL, OPT_quiet}, {"help", no_argument, NULL, '?'},@@ -4877,6 +4882,9 @@ static void parse_record_options(int argc,if (!ctx->max_graph_depth)die ("Could not allocate option" );break ; +case OPT_no_filter: + no_filter = true; +break ;case OPT_debug: debug =1 ;break ;tracecmd/trace-usage.c +1
-0@@ -56,6 +56,7 @@ static struct usage_help usage_help[] = {" --func-stack perform a stack trace for function tracer\n" " (use with caution)\n" " --max-graph-depth limit function_graph depth\n" +" --no-filter do not set any event filters\n" }, { "start",
Quoted 13 lines · On Tue, Apr 16, 2019 at 02:00:14AM +0300 Slavomir Kaslev wrote:
This patchset optimizes how pid filters are expressed and makes it less likely that we overflow ftrace filters' size limit of one page.
Changes since v2:
Append exclude rules with &&
Changes since v1:
Add missing tags
Fixappend_filter_pid_range()callers to pass valid range as [min,max]
Quoted 11 lines
Slavomir Kaslev (2):
trace-cmd: Optimize how pid filters are expressed
trace-cmd: Add --no-filter option to not filter recording processestracecmd/trace-record.c | 131 ++++++++++++++++++++++++++++------------ tracecmd/trace-usage.c | 1 + 2 files changed, 92 insertions(+), 40 deletions(-)Signature · 1 line
2.19.1
This version is also working for my use case. The logic in the filter does show the recording thread pids in the sched switch event, at least when switching to/from a non-excluded process. But I think that's desired.
The --no-filter option works as expected.
Thanks,
Phil
Quoted 10 lines · On Tue, 16 Apr 2019 09:22:54 -0400 Phil Auld <[email protected]> wrote:
This version is also working for my use case. The logic in the filter does show the recording thread pids in the sched switch event, at least when switching to/from a non-excluded process. But I think that's desired.
The --no-filter option works as expected.
Acked-by Phil Auld <[email protected]>
Could we get a Tested-by: from you for this series?
Thanks!
-- Steve
Phil Auld
Tested-byQuoted 8 lines · On Tue, Apr 16, 2019 at 05:39:09PM -0400 Steven Rostedt wrote:
Quoted 5 lines · On Tue, 16 Apr 2019 09:22:54 -0400 Phil Auld <[email protected]> wrote:
This version is also working for my use case. The logic in the filter does show the recording thread pids in the sched switch event, at least when switching to/from a non-excluded process. But I think that's desired.
Quoted 7 lines
Quoted 5 lines
The --no-filter option works as expected.
Acked-by Phil Auld <[email protected]>Could we get a Tested-by: from you for this series?
The second one for sure. I did it both with and without and saw the difference.
I was not comfortable with that for the first because I really only tested it in my setup on one machine (160 cpus). I didn't do really thorough testing. If that's enough for you then sure :)
Cheers,
Phil
Quoted 4 lines
Thanks!
-- Steve
Quoted 10 lines · On Tue, 16 Apr 2019 20:29:10 -0400 Phil Auld <[email protected]> wrote:
Quoted 1 line
Could we get a Tested-by: from you for this series?
The second one for sure. I did it both with and without and saw the difference.
I was not comfortable with that for the first because I really only tested it in my setup on one machine (160 cpus). I didn't do really thorough testing. If that's enough for you then sure :)
Tested-by Phil Auld <[email protected]>
I'll just put it on the second one then.
Thanks!
-- Steve