tracing/filter: Remove 32 pred limit and add short circuits

RFC PATCH In review Steven Rostedt 1 participant 14 messages Jan 28, 2011

Subsystems (2)

Tracing ftrace

lore

Share & tools

lore

Steven Rostedt

author maintainer

2011-01-28 04:35 UTC #0 lore raw

cover tracing/filter: Remove 32 pred limit and add short circuits

While doing some intense tracing, I hit the 32 pred limit in the filters. This annoyed me a little, so I decided to remove this constraint. But as I saw that the current approach needs to store state in a stack, the algorithm needed to be changed such that we could do any arbitrary number of preds without needing to store state.

This patch series converts the stack version into a tree walk that can use the direction of the walk to store the state.

An added bonus is that this also allows short circuit of AND and OR logic when one value is found to be false or true respectively. That is, if the left child is false and the operation is an AND, we do not need to process the right child. We can just set the result to false and pass it up to the parent.

For an optimization, the tree is also folded so that multiple ANDs or ORs in a row are converted to a single pred that has all the opperations in an array:

a || b || c || d || e ...

a && b && c && d && e ...

Either of the above does not need a tree walk. If any of the ORs are found to be true, we can exit and move to the parent. If any of the ANDs are found to be false we can do the same. Otherwise we can use the result of the last compare.

I'm sending this out as an RFC first, but they are pretty much set. I'll start running them through more vigorous tests and then package them up for 2.6.39.

This email is to let people know what is happening.

The following patches are in:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git

branch: rfc/tracing/filter

Steven Rostedt (12):
tracing/filter: Have no filter return a match tracing/filter: Move OR and AND logic out of fn() method tracing/filter: Dynamically allocate preds tracing/filter: Call synchronize_sched() just once for system filters tracing/filter: Allocate the preds in an array tracing/filter: Free pred array on disabling of filter tracing/filter: Use a tree instead of stack for filter_match_preds() tracing/filter: Optimize short ciruit check tracing/filter: Check the created pred tree tracing/filter: Optimize filter by folding the tree tracing/filter: Move MAX_FILTER_PRED to local tracing directory tracing/filter: Increase the max preds to 2^14

----

 include/linux/ftrace_event.h       |    1 -
 kernel/trace/trace.h               |   38 ++-
 kernel/trace/trace_events_filter.c |  751 ++++++++++++++++++++++++++++++-----
 3 files changed, 675 insertions(+), 115 deletions(-)

Steven Rostedt

author maintainer

2011-01-28 04:33 UTC #1 lore raw

RFC PATCH 1/12 tracing/filter: Have no filter return a match

From: Steven Rostedt <[email protected]>

The n_preds field of a file can change at anytime, and even can become zero, just as the filter is about to be processed by an event. In the case that is zero on entering the filter, return 1, telling the caller the event matchs and should be trace.

Also use a variable and assign it with ACCESS_ONCE() such that the count stays consistent within the function.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 kernel/trace/trace_events_filter.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

kernel/trace/trace_events_filter.c +6 -1

index 36d4010..7275f03 100644
@@ -383,9 +383,14 @@ int filter_match_preds(struct event_filter *filter, void *rec)
 	int match, top = 0, val1 = 0, val2 = 0;
 	int stack[MAX_FILTER_PRED];
 	struct filter_pred *pred;
+	int n_preds = ACCESS_ONCE(filter->n_preds);
 	int i;
 
-	for (i = 0; i < filter->n_preds; i++) {
+	/* no filter is considered a match */
+	if (!n_preds)
+		return 1;
+
+	for (i = 0; i < n_preds; i++) {
 		pred = filter->preds[i];
 		if (!pred->pop_n) {
 			match = pred->fn(pred, rec, val1, val2);

Signature · 1 line

1.7.2.3

Steven Rostedt

author maintainer

2011-01-28 04:34 UTC #2 lore raw

RFC PATCH 2/12 tracing/filter: Move OR and AND logic out of fn() method

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

The ops OR and AND act different from the other ops, as they are the only ones to take other ops as their arguements. These ops als change the logic of the filter_match_preds.

By removing the OR and AND fn's we can also remove the val1 and val2 that is passed to all other fn's and are unused.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 kernel/trace/trace.h               |    3 +-
 kernel/trace/trace_events_filter.c |   51 +++++++++++++----------------------
 2 files changed, 20 insertions(+), 34 deletions(-)

kernel/trace/trace.h +1 -2

index 9021f8c..1597bc0 100644
@@ -677,8 +677,7 @@ struct event_subsystem {
 struct filter_pred;
 struct regex;
 
-typedef int (*filter_pred_fn_t) (struct filter_pred *pred, void *event,
-				 int val1, int val2);
+typedef int (*filter_pred_fn_t) (struct filter_pred *pred, void *event);
 
 typedef int (*regex_match_func)(char *str, struct regex *r, int len);

kernel/trace/trace_events_filter.c +19 ~~-32~~

118 lines not on this page · raw message

Signature · 1 line

1.7.2.3

Steven Rostedt

author maintainer

2011-01-28 04:35 UTC #3 lore raw

RFC PATCH 3/12 tracing/filter: Dynamically allocate preds

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

For every filter that is made, we create predicates to hold every operation within the filter. We have a max of 32 predicates that we can hold. Currently, we allocate all 32 even if we only need to use one.

Part of the reason we do this is that the filter can be used at any moment by any event. Fortunately, the filter is only used with preemption disabled. By reseting the count of preds used "n_preds" to zero, then performing a synchronize_sched(), we can safely free and reallocate a new array of preds.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 kernel/trace/trace.h               |    3 +-
 kernel/trace/trace_events_filter.c |  142 +++++++++++++++++++++++++++---------
 2 files changed, 109 insertions(+), 36 deletions(-)

kernel/trace/trace.h +2 -1

index 1597bc0..441fc1b 100644
@@ -661,7 +661,8 @@ struct ftrace_event_field {
 };
 
 struct event_filter {
-	int			n_preds;
+	int			n_preds;	/* Number assigned */
+	int			a_preds;	/* allocated */
 	struct filter_pred	**preds;
 	char			*filter_string;
 };

kernel/trace/trace_events_filter.c +107 ~~-35~~

273 lines not on this page · raw message

Signature · 1 line

1.7.2.3

Steven Rostedt

author maintainer

2011-01-28 04:33 UTC #4 lore raw

RFC PATCH 4/12 tracing/filter: Call synchronize_sched() just once for system filters

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

By separating out the reseting of the filter->n_preds to zero from the reallocation of preds for the filter, we can reset groups of filters first, call synchronize_sched() just once, and then reallocate each of the filters in the system group.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 kernel/trace/trace_events_filter.c |   80 ++++++++++++++++++++++++++++-------
 1 files changed, 64 insertions(+), 16 deletions(-)

kernel/trace/trace_events_filter.c +64 ~~-16~~

index 932f605..1c14cf2 100644
@@ -570,17 +570,28 @@ static void __free_preds(struct event_filter *filter)
 	filter->n_preds = 0;
 }
 
+static void reset_preds(struct event_filter *filter)
+{
+	struct filter_pred *pred;
+	int n_preds = filter->n_preds;
+	int i;
+
+	filter->n_preds = 0;
+	if (!filter->preds)
+		return;
+
+	for (i = 0; i < n_preds; i++) {
+		pred = filter->preds[i];
+		pred->fn = filter_pred_none;
+	}
+}
+
 static void filter_disable_preds(struct ftrace_event_call *call)
 {
 	struct event_filter *filter = call->filter;
-	int i;
 
 	call->flags &= ~TRACE_EVENT_FL_FILTERED;
-	if (filter->preds) {
-		for (i = 0; i < filter->n_preds; i++)
-			filter->preds[i]->fn = filter_pred_none;
-	}
-	filter->n_preds = 0;
+	reset_preds(filter);
 }
 
 static void __free_filter(struct event_filter *filter)

@@ -620,15 +631,18 @@ static int __alloc_preds(struct event_filter *filter, int n_preds)
 
 	if (filter->preds) {
 		if (filter->a_preds < n_preds) {
-			/* We need to reallocate */
-			filter->n_preds = 0;
 			/*
-			 * It is possible that the filter is currently
-			 * being used. We need to zero out the number
-			 * of preds, wait on preemption and then free
-			 * the preds.
+			 * We need to reallocate.
+			 * We should have already have zeroed out
+			 * the pred count and called synchronized_sched()
+			 * to make sure no one is using the preds.
 			 */
 			synchronize_sched();
+			if (WARN_ON_ONCE(filter->n_preds)) {
+				/* We need to reset it now */
+				filter->n_preds = 0;
+				synchronize_sched();
+			}
 			__free_preds(filter);
 		}
 	}

@@ -1267,6 +1281,7 @@ static int replace_preds(struct ftrace_event_call *call,
 			return err;
 	}
 
+	n_preds = 0;
 	list_for_each_entry(elt, &ps->postfix, list) {
 		if (elt->op == OP_NONE) {
 			if (!operand1)

@@ -1327,6 +1342,30 @@ static int replace_system_preds(struct event_subsystem *system,
 		/* try to see if the filter can be applied */
 		err = replace_preds(call, filter, ps, filter_string, true);
 		if (err)
+			goto fail;
+	}
+
+	/* set all filter pred counts to zero */
+	list_for_each_entry(call, &ftrace_events, list) {
+		struct event_filter *filter = call->filter;
+
+		if (strcmp(call->class->system, system->name) != 0)
+			continue;
+
+		reset_preds(filter);
+	}
+
+	/*
+	 * Since some of the preds may be used under preemption
+	 * we need to wait for them to finish before we may
+	 * reallocate them.
+	 */
+	synchronize_sched();
+
+	list_for_each_entry(call, &ftrace_events, list) {
+		struct event_filter *filter = call->filter;
+
+		if (strcmp(call->class->system, system->name) != 0)
 			continue;
 
 		/* really apply the filter */

@@ -1341,11 +1380,13 @@ static int replace_system_preds(struct event_subsystem *system,
 		fail = false;
 	}
 
-	if (fail) {
-		parse_error(ps, FILT_ERR_BAD_SUBSYS_FILTER, 0);
-		return -EINVAL;
-	}
+	if (fail)
+		goto fail;
+
 	return 0;
+ fail:
+	parse_error(ps, FILT_ERR_BAD_SUBSYS_FILTER, 0);
+	return -EINVAL;
 }
 
 int apply_event_filter(struct ftrace_event_call *call, char *filter_string)

@@ -1380,6 +1421,13 @@ int apply_event_filter(struct ftrace_event_call *call, char *filter_string)
 		goto out;
 	}
 
+	/*
+	 * Make sure all the pred counts are zero so that
+	 * no task is using it when we reallocate the preds array.
+	 */
+	reset_preds(call->filter);
+	synchronize_sched();
+
 	err = replace_preds(call, call->filter, ps, filter_string, false);
 	if (err)
 		append_filter_err(ps, call->filter);

Signature · 1 line

1.7.2.3

Steven Rostedt

author maintainer

2011-01-28 04:34 UTC #5 lore raw

RFC PATCH 5/12 tracing/filter: Allocate the preds in an array

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

Currently we allocate an array of pointers to filter_preds, and then allocate a separate filter_pred for each item in the array. This adds slight overhead in the filters as it needs to derefernce twice to get to the op condition.

Allocating the preds themselves in a single array removes a dereference as well as helps on the cache footprint.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 kernel/trace/trace.h               |    2 +-
 kernel/trace/trace_events_filter.c |   31 +++++++++----------------------
 2 files changed, 10 insertions(+), 23 deletions(-)

kernel/trace/trace.h +1 -1

index 441fc1b..254d04a 100644
@@ -663,7 +663,7 @@ struct ftrace_event_field {
 struct event_filter {
 	int			n_preds;	/* Number assigned */
 	int			a_preds;	/* allocated */
-	struct filter_pred	**preds;
+	struct filter_pred	*preds;
 	char			*filter_string;
 };

kernel/trace/trace_events_filter.c +9 ~~-22~~

index 1c14cf2..8c560eb 100644
@@ -362,7 +362,7 @@ int filter_match_preds(struct event_filter *filter, void *rec)
 {
 	int match = -1, top = 0, val1 = 0, val2 = 0;
 	int stack[MAX_FILTER_PRED];
-	struct filter_pred **preds;
+	struct filter_pred *preds;
 	struct filter_pred *pred;
 	int n_preds = ACCESS_ONCE(filter->n_preds);
 	int i;

@@ -377,7 +377,7 @@ int filter_match_preds(struct event_filter *filter, void *rec)
 	preds = rcu_dereference_sched(filter->preds);
 
 	for (i = 0; i < n_preds; i++) {
-		pred = preds[i];
+		pred = &preds[i];
 		if (!pred->pop_n) {
 			match = pred->fn(pred, rec);
 			stack[top++] = match;

@@ -559,10 +559,8 @@ static void __free_preds(struct event_filter *filter)
 	int i;
 
 	if (filter->preds) {
-		for (i = 0; i < filter->a_preds; i++) {
-			if (filter->preds[i])
-				filter_free_pred(filter->preds[i]);
-		}
+		for (i = 0; i < filter->a_preds; i++)
+			kfree(filter->preds[i].field_name);
 		kfree(filter->preds);
 		filter->preds = NULL;
 	}

@@ -572,7 +570,6 @@ static void __free_preds(struct event_filter *filter)
 
 static void reset_preds(struct event_filter *filter)
 {
-	struct filter_pred *pred;
 	int n_preds = filter->n_preds;
 	int i;

@@ -580,10 +577,8 @@ static void reset_preds(struct event_filter *filter)
 	if (!filter->preds)
 		return;
 
-	for (i = 0; i < n_preds; i++) {
-		pred = filter->preds[i];
-		pred->fn = filter_pred_none;
-	}
+	for (i = 0; i < n_preds; i++)
+		filter->preds[i].fn = filter_pred_none;
 }
 
 static void filter_disable_preds(struct ftrace_event_call *call)

@@ -659,19 +654,11 @@ static int __alloc_preds(struct event_filter *filter, int n_preds)
 		return -EINVAL;
 
 	for (i = 0; i < n_preds; i++) {
-		pred = filter->preds[i];
-		if (!pred)
-			pred = kzalloc(sizeof(*pred), GFP_KERNEL);
-		if (!pred)
-			goto oom;
+		pred = &filter->preds[i];
 		pred->fn = filter_pred_none;
-		filter->preds[i] = pred;
 	}
 
 	return 0;
- oom:
-	__free_preds(filter);
-	return -ENOMEM;
 }
 
 static int init_filter(struct ftrace_event_call *call)

@@ -731,8 +718,8 @@ static int filter_add_pred_fn(struct filter_parse_state *ps,
 	}
 
 	idx = filter->n_preds;
-	filter_clear_pred(filter->preds[idx]);
-	err = filter_set_pred(filter->preds[idx], pred, fn);
+	filter_clear_pred(&filter->preds[idx]);
+	err = filter_set_pred(&filter->preds[idx], pred, fn);
 	if (err)
 		return err;

Signature · 1 line

1.7.2.3

Steven Rostedt

author maintainer

2011-01-28 04:35 UTC #6 lore raw

RFC PATCH 6/12 tracing/filter: Free pred array on disabling of filter

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

When a filter is disabled, free the preds.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 kernel/trace/trace_events_filter.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

kernel/trace/trace_events_filter.c +4 -0

index 8c560eb..23753f6 100644
@@ -1389,6 +1389,10 @@ int apply_event_filter(struct ftrace_event_call *call, char *filter_string)
 
 	if (!strcmp(strstrip(filter_string), "0")) {
 		filter_disable_preds(call);
+		reset_preds(call->filter);
+		/* Make sure the filter is not being used */
+		synchronize_sched();
+		__free_preds(call->filter);
 		remove_filter_string(call->filter);
 		goto out_unlock;
 	}

Signature · 1 line

1.7.2.3

Steven Rostedt

author maintainer

2011-01-28 04:34 UTC #7 lore raw

RFC PATCH 7/12 tracing/filter: Use a tree instead of stack for filter_match_preds()

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

Currently the filter_match_preds() requires a stack to push and pop the preds to determine if the filter matches the record or not. This has two drawbacks:

1) It requires a stack to store state information. As this is done in fast paths we can't allocate the storage for this stack, and we can't use a global as it must be re-entrant. The stack is stored on the kernel stack and this greatly limits how many preds we may allow.

2) All conditions are calculated even when a short circuit exists. a || b will always calculate a and b even though a was determined to be true.

Using a tree we can walk a constant structure that will save the state as we go. The algorithm is simply:

pred = root;

  do {
	switch (move) {
	case MOVE_DOWN:
		if (OR or AND) {
			pred = left;
			continue;
		}
		if (pred == root)
			break;
		match = pred->fn();
		pred = pred->parent;
		move = left child ? MOVE_UP_FROM_LEFT : MOVE_UP_FROM_RIGHT;
		continue;

	case MOVE_UP_FROM_LEFT:
		/* Only OR or AND can be a parent */
		if (match && OR || !match && AND) {
			/* short circuit */
			if (pred == root)
				break;
			pred = pred->parent;
			move = left child ?
				MOVE_UP_FROM_LEFT :
				MOVE_UP_FROM_RIGHT;
			continue;
		}
		pred = pred->right;
		move = MOVE_DOWN;
		continue;

	case MOVE_UP_FROM_RIGHT:
		if (pred == root)
			break;
		pred = pred->parent;
		move = left child ? MOVE_UP_FROM_LEFT : MOVE_UP_FROM_RIGHT;
		continue;
	}
	done = 1;

} while (!done);

This way there's no strict limit to how many preds we allow and it also will short circuit the logical operations when possible.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 kernel/trace/trace.h               |    9 ++-
 kernel/trace/trace_events_filter.c |  231 +++++++++++++++++++++++++++++-------
 2 files changed, 194 insertions(+), 46 deletions(-)

kernel/trace/trace.h +8 -1

index 254d04a..bba34a7 100644
@@ -664,6 +664,7 @@ struct event_filter {
 	int			n_preds;	/* Number assigned */
 	int			a_preds;	/* allocated */
 	struct filter_pred	*preds;
+	struct filter_pred	*root;
 	char			*filter_string;
 };

@@ -675,6 +676,9 @@ struct event_subsystem {
 	int			nr_events;
 };
 
+#define FILTER_PRED_INVALID	((unsigned short)-1)
+#define FILTER_PRED_IS_RIGHT	(1 << 15)
+
 struct filter_pred;
 struct regex;

@@ -704,7 +708,10 @@ struct filter_pred {
 	int 			offset;
 	int 			not;
 	int 			op;
-	int 			pop_n;
+	unsigned short		index;
+	unsigned short		parent;
+	unsigned short		left;
+	unsigned short		right;
 };
 
 extern struct list_head ftrace_common_fields;

kernel/trace/trace_events_filter.c +186 ~~-45~~

358 lines not on this page · raw message

Signature · 1 line

1.7.2.3

Steven Rostedt

author maintainer

2011-01-28 04:34 UTC #8 lore raw

RFC PATCH 8/12 tracing/filter: Optimize short ciruit check

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

The test if we should break out early for OR and AND operations can be optimized by comparing the current result with
(pred->op == OP_OR)

That is if the result is true and the op is an OP_OR, or if the result is false and the op is not an OP_OR (thus an OP_AND) we can break out early in either case. Otherwise we continue processing.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 kernel/trace/trace_events_filter.c |   12 +++++++++---
 1 files changed, 9 insertions(+), 3 deletions(-)

kernel/trace/trace_events_filter.c +9 -3

index 2677924..afe59ab 100644
@@ -426,9 +426,15 @@ int filter_match_preds(struct event_filter *filter, void *rec)
 					       pred->parent, &move);
 			continue;
 		case MOVE_UP_FROM_LEFT:
-			/* Check for short circuits */
-			if ((match && pred->op == OP_OR) ||
-			    (!match && pred->op == OP_AND)) {
+			/*
+			 * Check for short circuits.
+			 *
+			 * Optimization: !!match == (pred->op == OP_OR)
+			 *   is the same as:
+			 * if ((match && pred->op == OP_OR) ||
+			 *     (!match && pred->op == OP_AND))
+			 */
+			if (!!match == (pred->op == OP_OR))
 				if (pred == root)
 					break;
 				pred = get_pred_parent(pred, preds,

Signature · 1 line

1.7.2.3

Steven Rostedt

author maintainer

2011-01-28 05:37 UTC #9 lore raw

Re: [RFC][PATCH 08/12] tracing/filter: Optimize short ciruit check

Quoted 10 lines · Steven Rostedt

kernel/trace/trace_events_filter.c +0 -2

index 2677924..afe59ab 100644
@@ -426,9 +426,15 @@ int filter_match_preds(struct event_filter *filter, void *rec)
 					       pred->parent, &move);
 			continue;
 		case MOVE_UP_FROM_LEFT:
-			/* Check for short circuits */
-			if ((match && pred->op == OP_OR) ||

kernel/trace/trace_events_filter.c int filter_match_preds(struct event_filter *filter, void *rec) · line 437

Quoted 10 lines

kernel/trace/trace_events_filter.c +9 -1

-			    (!match && pred->op == OP_AND)) {
+			/*
+			 * Check for short circuits.
+			 *
+			 * Optimization: !!match == (pred->op == OP_OR)
+			 *   is the same as:
+			 * if ((match && pred->op == OP_OR) ||
+			 *     (!match && pred->op == OP_AND))
+			 */
+			if (!!match == (pred->op == OP_OR))

+ {

Bah! I tested these in quilt and when I pulled them into git, I must have accidentally deleted the '{' in the patch :-p

Thanks goodness this was only a RFC ;)

/me goes to rebase

-- Steve

Quoted 3 lines

kernel/trace/trace_events_filter.c +0 -0

 				if (pred == root)
 					break;
 				pred = get_pred_parent(pred, preds,

Steven Rostedt

author maintainer

2011-01-28 04:35 UTC #10 lore raw

RFC PATCH 9/12 tracing/filter: Check the created pred tree

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

Since the filter walks a tree to determine if a match is made or not, if the tree was incorrectly created, it could cause an infinite loop.

Add a check to walk the entire tree before assigning it as a filter to make sure the tree is correct.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 kernel/trace/trace_events_filter.c |   72 +++++++++++++++++++++++++++++++++++-
 1 files changed, 71 insertions(+), 1 deletions(-)

kernel/trace/trace_events_filter.c +71 -1

index afe59ab..47e9430 100644
@@ -1359,6 +1359,68 @@ static int count_preds(struct filter_parse_state *ps)
 	return n_preds;
 }
 
+/*
+ * The tree is walked at filtering of an event. If the tree is not correctly
+ * built, it may cause an infinite loop. Check here that the tree does
+ * indeed terminate.
+ */
+static int check_pred_tree(struct event_filter *filter,
+			   struct filter_pred *root)
+{
+	struct filter_pred *preds;
+	struct filter_pred *pred;
+	enum move_type move = MOVE_DOWN;
+	int count = 0;
+	int done = 0;
+	int max;
+
+	/*
+	 * The max that we can hit a node is three times.
+	 * Once going down, once coming up from left, and
+	 * once coming up from right. This is more than enough
+	 * since leafs are only hit a single time.
+	 */
+	max = 3 * filter->n_preds;
+
+	preds = filter->preds;
+	if  (!preds)
+		return -EINVAL;
+	pred = root;
+
+	do {
+		if (WARN_ON(count++ > max))
+			return -EINVAL;
+
+		switch (move) {
+		case MOVE_DOWN:
+			if (pred->left != FILTER_PRED_INVALID) {
+				pred = &preds[pred->left];
+				continue;
+			}
+			/* A leaf at the root is just a leaf in the tree */
+			if (pred == root)
+				break;
+			pred = get_pred_parent(pred, preds,
+					       pred->parent, &move);
+			continue;
+		case MOVE_UP_FROM_LEFT:
+			pred = &preds[pred->right];
+			move = MOVE_DOWN;
+			continue;
+		case MOVE_UP_FROM_RIGHT:
+			if (pred == root)
+				break;
+			pred = get_pred_parent(pred, preds,
+					       pred->parent, &move);
+			continue;
+		}
+		done = 1;
+	} while (!done);
+
+	/* We are fine. */
+	return 0;
+}
+
 static int replace_preds(struct ftrace_event_call *call,
 			 struct event_filter *filter,
 			 struct filter_parse_state *ps,

@@ -1367,6 +1429,7 @@ static int replace_preds(struct ftrace_event_call *call,
 {
 	char *operand1 = NULL, *operand2 = NULL;
 	struct filter_pred *pred;
+	struct filter_pred *root;
 	struct postfix_elt *elt;
 	struct pred_stack stack = { }; /* init to NULL */
 	int err;

@@ -1443,7 +1506,7 @@ add_pred:
 		if (!pred)
 			return -EINVAL;
 		/* This item is where we start from in matching */
-		filter->root = pred;
+		root = pred;
 		/* Make sure the stack is empty */
 		pred = __pop_pred_stack(&stack);
 		if (WARN_ON(pred)) {

@@ -1451,6 +1514,13 @@ add_pred:
 			filter->root = NULL;
 			goto fail;
 		}
+		err = check_pred_tree(filter, root);
+		if (err)
+			goto fail;
+
+		/* We don't set root until we know it works */
+		barrier();
+		filter->root = root;
 	}
 
 	err = 0;

Signature · 1 line

1.7.2.3

Steven Rostedt

author maintainer

2011-01-28 04:34 UTC #11 lore raw

RFC PATCH 10/12 tracing/filter: Optimize filter by folding the tree

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

There are many cases that a filter will contain multiple ORs or ANDs together near the leafs. Walking up and down the tree to get to the next compare can be a waste.

If there are several ORs or ANDs together, fold them into a single pred and allocate an array of the conditions that they check. This will speed up the filter by linearly walking an array and can still break out if a short circuit condition is met.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 kernel/trace/trace.h               |   12 ++-
 kernel/trace/trace_events_filter.c |  233 ++++++++++++++++++++++++++++++++++--
 2 files changed, 235 insertions(+), 10 deletions(-)

kernel/trace/trace.h +11 -1

index bba34a7..d754330 100644
@@ -678,6 +678,7 @@ struct event_subsystem {
 
 #define FILTER_PRED_INVALID	((unsigned short)-1)
 #define FILTER_PRED_IS_RIGHT	(1 << 15)
+#define FILTER_PRED_FOLD	(1 << 15)
 
 struct filter_pred;
 struct regex;

@@ -704,7 +705,16 @@ struct filter_pred {
 	filter_pred_fn_t 	fn;
 	u64 			val;
 	struct regex		regex;
-	char 			*field_name;
+	/*
+	 * Leaf nodes use field_name, ops is used by AND and OR
+	 * nodes. The field_name is always freed when freeing a pred.
+	 * We can overload field_name for ops and have it freed
+	 * as well.
+	 */
+	union {
+		char		*field_name;
+		unsigned short	*ops;
+	};
 	int 			offset;
 	int 			not;
 	int 			op;

kernel/trace/trace_events_filter.c +224 -9

270 lines not on this page · raw message

Signature · 1 line

1.7.2.3

Steven Rostedt

author maintainer

2011-01-28 04:34 UTC #12 lore raw

RFC PATCH 11/12 tracing/filter: Move MAX_FILTER_PRED to local tracing directory

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

The MAX_FILTER_PRED is only needed by the kernel/trace/*.c files. Move it to kernel/trace/trace.h.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 include/linux/ftrace_event.h |    1 -
 kernel/trace/trace.h         |    2 ++
 2 files changed, 2 insertions(+), 1 deletions(-)

include/linux/ftrace_event.h +0 -1

index 47e3997..1a99e79 100644
@@ -208,7 +208,6 @@ struct ftrace_event_call {
 
 #define PERF_MAX_TRACE_SIZE	2048
 
-#define MAX_FILTER_PRED		32
 #define MAX_FILTER_STR_VAL	256	/* Should handle KSYM_SYMBOL_LEN */
 
 extern void destroy_preds(struct ftrace_event_call *call);

kernel/trace/trace.h +2 -0

index d754330..fbff872 100644
@@ -680,6 +680,8 @@ struct event_subsystem {
 #define FILTER_PRED_IS_RIGHT	(1 << 15)
 #define FILTER_PRED_FOLD	(1 << 15)
 
+#define MAX_FILTER_PRED		32
+
 struct filter_pred;
 struct regex;

Signature · 1 line

1.7.2.3

Steven Rostedt

author maintainer

2011-01-28 04:34 UTC #13 lore raw

RFC PATCH 12/12 tracing/filter: Increase the max preds to 2^14

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

Now that the filter logic does not require to save the pred results on the stack, we can increase the max number of preds we allow. As the preds are index by a short value, and we use the MSBs as flags we can increase the max preds to 2^14 (16384) which should be way more than enough.

Cc Tom Zanussi <[email protected]>

Signed-off-by Steven Rostedt <[email protected]>

---
 kernel/trace/trace.h |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

kernel/trace/trace.h +8 -1

index fbff872..856e73c 100644
@@ -680,7 +680,14 @@ struct event_subsystem {
 #define FILTER_PRED_IS_RIGHT	(1 << 15)
 #define FILTER_PRED_FOLD	(1 << 15)
 
-#define MAX_FILTER_PRED		32
+/*
+ * The max preds is the size of unsigned short with
+ * two flags at the MSBs. One bit is used for both the IS_RIGHT
+ * and FOLD flags. The other is reserved.
+ *
+ * 2^14 preds is way more than enough.
+ */
+#define MAX_FILTER_PRED		16384
 
 struct filter_pred;
 struct regex;

Signature · 1 line

1.7.2.3