Skip to contentSkip to conversation

tracing: Fix regression of filter waiting a long time on RCU synchronization

PATCH New Steven Rostedt 2 participants 4 messages Jun 5, 2025 – Jun 10, 2025 Tracing

lore
Share & tools
lore

Steven Rostedt

author maintainer
#0 lore raw

PATCH 1/1 tracing: Fix regression of filter waiting a long time on RCU synchronization

From: Steven Rostedt <[email protected]>

When faultable trace events were added, a trace event may no longer use normal RCU to synchronize but instead used synchronize_rcu_tasks_trace(). This synchronization takes a much longer time to synchronize.

The filter logic would free the filters by calling tracepoint_synchronize_unregister() after it unhooked the filter strings and before freeing them. With this function now calling synchronize_rcu_tasks_trace() this increased the time to free a filter tremendously. On a PREEMPT_RT system, it was even more noticeable.

 # time trace-cmd record -p function sleep 1
 [..]
 real	2m29.052s
 user	0m0.244s
 sys	0m20.136s

As trace-cmd would clear out all the filters before recording, it could take up to 2 minutes to do a recording of "sleep 1".

To find out where the issues was:

~# trace-cmd sqlhist -e -n sched_stack select start.prev_state as state, end.next_comm as comm, TIMESTAMP_DELTA_USECS as delta, start.STACKTRACE as stack from sched_switch as start join sched_switch as end on start.prev_pid = end.next_pid

Which will produce the following commands (and -e will also execute them):

echo 's:sched_stack s64 state; char comm[16]; u64 delta; unsigned long stack[];' >> /sys/kernel/tracing/dynamic_events
echo 'hist:keys=prev_pid:__arg_18057_2=prev_state,__arg_18057_4=common_timestamp.usecs,__arg_18057_7=common_stacktrace' >> /sys/kernel/tracing/events/sched/sched_switch/trigger
echo 'hist:keys=next_pid:__state_18057_1=$__arg_18057_2,__comm_18057_3=next_comm,__delta_18057_5=common_timestamp.usecs-$__arg_18057_4,__stack_18057_6=$__arg_18057_7:onmatch(sched.sched_switch).trace(sched_stack,$__state_18057_1,$__comm_18057_3,$__delta_18057_5,$__stack_18057_6)' >> /sys/kernel/tracing/events/sched/sched_switch/trigger

The above creates a synthetic event that creates a stack trace when a task schedules out and records it with the time it scheduled back in. Basically the time a task is off the CPU. It also records the state of the task when it left the CPU (running, blocked, sleeping, etc). It also saves the comm of the task as "comm" (needed for the next command).

~# echo 'hist:keys=state,stack.stacktrace:vals=delta:sort=state,delta if comm == "trace-cmd" && state & 3' > /sys/kernel/tracing/events/synthetic/sched_stack/trigger

The above creates a histogram with buckets per state, per stack, and the value of the total time it was off the CPU for that stack trace. It filters on tasks with "comm == trace-cmd" and only the sleeping and blocked states (1 - sleeping, 2 - blocked).

~# trace-cmd record -p function sleep 1

~# cat /sys/kernel/tracing/events/synthetic/sched_stack/hist | tail -18
{ state: 2, stack.stacktrace __schedule+0x1545/0x3700

         schedule+0xe2/0x390
         schedule_timeout+0x175/0x200
         wait_for_completion_state+0x294/0x440
         __wait_rcu_gp+0x247/0x4f0
         synchronize_rcu_tasks_generic+0x151/0x230
         apply_subsystem_event_filter+0xa2b/0x1300
         subsystem_filter_write+0x67/0xc0
         vfs_write+0x1e2/0xeb0
         ksys_write+0xff/0x1d0
         do_syscall_64+0x7b/0x420
         entry_SYSCALL_64_after_hwframe+0x76/0x7e

} hitcount: 237 delta: 99756288 <<--------------- Delta is 99 seconds!

Totals:

    Hits: 525
    Entries: 21
    Dropped: 0

This shows that this particular trace waited for 99 seconds on synchronize_rcu_tasks() in apply_subsystem_event_filter().

In fact, there's a lot of places in the filter code that spends a lot of time waiting for synchronize_rcu_tasks_trace() in order to free the filters.

Add helper functions that will use call_rcu*() variants to asynchronously free the filters. This brings the timings back to normal:

 # time trace-cmd record -p function sleep 1
 [..]
 real	0m14.681s
 user	0m0.335s
 sys	0m28.616s

And the histogram also shows this:

~# cat /sys/kernel/tracing/events/synthetic/sched_stack/hist | tail -21
{ state: 2, stack.stacktrace __schedule+0x1545/0x3700

         schedule+0xe2/0x390
         schedule_timeout+0x175/0x200
         wait_for_completion_state+0x294/0x440
         __wait_rcu_gp+0x247/0x4f0
         synchronize_rcu_normal+0x3db/0x5c0
         tracing_reset_online_cpus+0x8f/0x1e0
         tracing_open+0x335/0x440
         do_dentry_open+0x4c6/0x17a0
         vfs_open+0x82/0x360
         path_openat+0x1a36/0x2990
         do_filp_open+0x1c5/0x420
         do_sys_openat2+0xed/0x180
         __x64_sys_openat+0x108/0x1d0
         do_syscall_64+0x7b/0x420

} hitcount: 2 delta: 77044

Totals:

    Hits: 55
    Entries: 28
    Dropped: 0

Where the total waiting time of synchronize_rcu_tasks_trace() is 77 milliseconds.

Fixes a363d27cdbc2 ("tracing: Allow system call tracepoints to handle page faults")
---
 kernel/trace/trace_events_filter.c | 164 ++++++++++++++++++++++-------
 1 file changed, 127 insertions(+), 37 deletions(-)
kernel/trace/trace_events_filter.c +127 -37
index 2048560264bb..3ff782d6b522 100644
@@ -1335,6 +1335,74 @@ static void filter_free_subsystem_preds(struct trace_subsystem_dir *dir,
 	}
 }
 
+struct filter_list {
+	struct list_head	list;
+	struct event_filter	*filter;
+};
+
+struct filter_head {
+	struct list_head	list;
+	struct rcu_head		rcu;
+};
+
+
+static void free_filter_list(struct rcu_head *rhp)
+{
+	struct filter_head *filter_list = container_of(rhp, struct filter_head, rcu);
+	struct filter_list *filter_item, *tmp;
+
+	list_for_each_entry_safe(filter_item, tmp, &filter_list->list, list) {
+		__free_filter(filter_item->filter);
+		list_del(&filter_item->list);
+		kfree(filter_item);
+	}
+	kfree(filter_list);
+}
+
+static void free_filter_list_tasks(struct rcu_head *rhp)
+{
+	call_rcu(rhp, free_filter_list);
+}
+
+/*
+ * The tracepoint_synchronize_unregister() is a double rcu call.
+ * It calls synchronize_rcu_tasks_trace() followed by synchronize_rcu().
+ * Instead of waiting for it, simply call these via the call_rcu*()
+ * variants.
+ */
+static void delay_free_filter(struct filter_head *head)
+{
+	call_rcu_tasks_trace(&head->rcu, free_filter_list_tasks);
+}
+
+static void try_delay_free_filter(struct event_filter *filter)
+{
+	struct filter_head *head;
+	struct filter_list *item;
+
+	head = kmalloc(sizeof(*head), GFP_KERNEL);
+	if (!head)
+		goto free_now;
+
+	INIT_LIST_HEAD(&head->list);
+
+	item = kmalloc(sizeof(*item), GFP_KERNEL);
+	if (!item) {
+		kfree(head);
+		goto free_now;
+	}
+
+	item->filter = filter;
+	list_add_tail(&item->list, &head->list);
+	delay_free_filter(head);
+	return;
+
+ free_now:
+	/* Make sure the filter is not being used */
+	tracepoint_synchronize_unregister();
+	__free_filter(filter);
+}
+
 static inline void __free_subsystem_filter(struct trace_event_file *file)
 {
 	__free_filter(file->filter);
@@ -1342,15 +1410,53 @@ static inline void __free_subsystem_filter(struct trace_event_file *file)
 }
 
 static void filter_free_subsystem_filters(struct trace_subsystem_dir *dir,
-					  struct trace_array *tr)
+					  struct trace_array *tr,
+					  struct event_filter *filter)
 {
 	struct trace_event_file *file;
+	struct filter_head *head;
+	struct filter_list *item;
+
+	head = kmalloc(sizeof(*head), GFP_KERNEL);
+	if (!head)
+		goto free_now;
+
+	INIT_LIST_HEAD(&head->list);
+
+	item = kmalloc(sizeof(*item), GFP_KERNEL);
+	if (!item) {
+		kfree(head);
+		goto free_now;
+	}
+
+	item->filter = filter;
+	list_add_tail(&item->list, &head->list);
 
 	list_for_each_entry(file, &tr->events, list) {
 		if (file->system != dir)
 			continue;
-		__free_subsystem_filter(file);
+		item = kmalloc(sizeof(*item), GFP_KERNEL);
+		if (!item)
+			goto free_now;
+		item->filter = file->filter;
+		list_add_tail(&item->list, &head->list);
+		file->filter = NULL;
+	}
+
+	delay_free_filter(head);
+	return;
+ free_now:
+	tracepoint_synchronize_unregister();
+
+	if (head)
+		free_filter_list(&head->rcu);
+
+	list_for_each_entry(file, &tr->events, list) {
+		if (file->system != dir || !file->filter)
+			continue;
+		__free_filter(file->filter);
 	}
+	__free_filter(filter);
 }
 
 int filter_assign_type(const char *type)
@@ -2131,11 +2237,6 @@ static inline void event_clear_filter(struct trace_event_file *file)
 	RCU_INIT_POINTER(file->filter, NULL);
 }
 
-struct filter_list {
-	struct list_head	list;
-	struct event_filter	*filter;
-};
-
 static int process_system_preds(struct trace_subsystem_dir *dir,
 				struct trace_array *tr,
 				struct filter_parse_error *pe,
@@ -2144,11 +2245,16 @@ static int process_system_preds(struct trace_subsystem_dir *dir,
 	struct trace_event_file *file;
 	struct filter_list *filter_item;
 	struct event_filter *filter = NULL;
-	struct filter_list *tmp;
-	LIST_HEAD(filter_list);
+	struct filter_head *filter_list;
 	bool fail = true;
 	int err;
 
+	filter_list = kmalloc(sizeof(*filter_list), GFP_KERNEL);
+	if (!filter_list)
+		return -ENOMEM;
+
+	INIT_LIST_HEAD(&filter_list->list);
+
 	list_for_each_entry(file, &tr->events, list) {
 
 		if (file->system != dir)
@@ -2175,7 +2281,7 @@ static int process_system_preds(struct trace_subsystem_dir *dir,
 		if (!filter_item)
 			goto fail_mem;
 
-		list_add_tail(&filter_item->list, &filter_list);
+		list_add_tail(&filter_item->list, &filter_list->list);
 		/*
 		 * Regardless of if this returned an error, we still
 		 * replace the filter for the call.
@@ -2195,31 +2301,22 @@ static int process_system_preds(struct trace_subsystem_dir *dir,
 	 * Do a synchronize_rcu() and to ensure all calls are
 	 * done with them before we free them.
 	 */
-	tracepoint_synchronize_unregister();
-	list_for_each_entry_safe(filter_item, tmp, &filter_list, list) {
-		__free_filter(filter_item->filter);
-		list_del(&filter_item->list);
-		kfree(filter_item);
-	}
+	delay_free_filter(filter_list);
 	return 0;
  fail:
 	/* No call succeeded */
-	list_for_each_entry_safe(filter_item, tmp, &filter_list, list) {
-		list_del(&filter_item->list);
-		kfree(filter_item);
-	}
+	free_filter_list(&filter_list->rcu);
 	parse_error(pe, FILT_ERR_BAD_SUBSYS_FILTER, 0);
 	return -EINVAL;
  fail_mem:
 	__free_filter(filter);
+
 	/* If any call succeeded, we still need to sync */
 	if (!fail)
-		tracepoint_synchronize_unregister();
-	list_for_each_entry_safe(filter_item, tmp, &filter_list, list) {
-		__free_filter(filter_item->filter);
-		list_del(&filter_item->list);
-		kfree(filter_item);
-	}
+		delay_free_filter(filter_list);
+	else
+		free_filter_list(&filter_list->rcu);
+
 	return -ENOMEM;
 }
 
@@ -2361,9 +2458,7 @@ int apply_event_filter(struct trace_event_file *file, char *filter_string)
 
 		event_clear_filter(file);
 
-		/* Make sure the filter is not being used */
-		tracepoint_synchronize_unregister();
-		__free_filter(filter);
+		try_delay_free_filter(filter);
 
 		return 0;
 	}
@@ -2387,11 +2482,8 @@ int apply_event_filter(struct trace_event_file *file, char *filter_string)
 
 		event_set_filter(file, filter);
 
-		if (tmp) {
-			/* Make sure the call is done with the filter */
-			tracepoint_synchronize_unregister();
-			__free_filter(tmp);
-		}
+		if (tmp)
+			try_delay_free_filter(tmp);
 	}
 
 	return err;
@@ -2417,9 +2509,7 @@ int apply_subsystem_event_filter(struct trace_subsystem_dir *dir,
 		filter = system->filter;
 		system->filter = NULL;
 		/* Ensure all filters are no longer used */
-		tracepoint_synchronize_unregister();
-		filter_free_subsystem_filters(dir, tr);
-		__free_filter(filter);
+		filter_free_subsystem_filters(dir, tr, filter);
 		return 0;
 	}
 
Signature · 1 line
2.47.2
#1 lore raw
kernel test robot noticed the following build warnings:

Hi Steven,

kernel test robot noticed the following build warnings:

[auto build test WARNING on trace/for-next]
[also build test WARNING on linus/master v6.15 next-20250606] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Steven-Rostedt/tracing-Fix-regression-of-filter-waiting-a-long-time-on-RCU-synchronization/20250606-041722
base:   https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace for-next
patch link:    https://lore.kernel.org/r/20250605161701.35f7989a%40gandalf.local.home
patch subject: [PATCH] tracing: Fix regression of filter waiting a long time on RCU synchronization
config: i386-randconfig-061-20250606 (https://download.01.org/0day-ci/archive/20250607/[email protected]/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250607/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)

>> kernel/trace/trace_events_filter.c:1441:30: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct event_filter *filter @@     got struct event_filter [noderef] __rcu *filter @@

kernel/trace/trace_events_filter.c:1441:30: sparse: expected struct event_filter *filter
kernel/trace/trace_events_filter.c:1441:30: sparse: got struct event_filter [noderef] __rcu *filter
kernel/trace/trace_events_filter.c:1457:35: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct event_filter *filter @@ got struct event_filter [noderef] __rcu *filter @@
kernel/trace/trace_events_filter.c:1457:35: sparse: expected struct event_filter *filter
kernel/trace/trace_events_filter.c:1457:35: sparse: got struct event_filter [noderef] __rcu *filter
kernel/trace/trace_events_filter.c:1253:20: sparse: sparse: incorrect type in return expression (different address spaces) @@ expected struct event_filter * @@ got struct event_filter [noderef] __rcu *filter @@
kernel/trace/trace_events_filter.c:1253:20: sparse: expected struct event_filter *
kernel/trace/trace_events_filter.c:1253:20: sparse: got struct event_filter [noderef] __rcu *filter
kernel/trace/trace_events_filter.c:1323:34: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct event_filter *filter @@ got struct event_filter [noderef] __rcu *filter @@
kernel/trace/trace_events_filter.c:1323:34: sparse: expected struct event_filter *filter
kernel/trace/trace_events_filter.c:1323:34: sparse: got struct event_filter [noderef] __rcu *filter
kernel/trace/trace_events_filter.c:1253:20: sparse: sparse: incorrect type in return expression (different address spaces) @@ expected struct event_filter * @@ got struct event_filter [noderef] __rcu *filter @@
kernel/trace/trace_events_filter.c:1253:20: sparse: expected struct event_filter *
kernel/trace/trace_events_filter.c:1253:20: sparse: got struct event_filter [noderef] __rcu *filter
kernel/trace/trace_events_filter.c:1253:20: sparse: sparse: incorrect type in return expression (different address spaces) @@ expected struct event_filter * @@ got struct event_filter [noderef] __rcu *filter @@
kernel/trace/trace_events_filter.c:1253:20: sparse: expected struct event_filter *
kernel/trace/trace_events_filter.c:1253:20: sparse: got struct event_filter [noderef] __rcu *filter
kernel/trace/trace_events_filter.c:1253:20: sparse: sparse: incorrect type in return expression (different address spaces) @@ expected struct event_filter * @@ got struct event_filter [noderef] __rcu *filter @@
kernel/trace/trace_events_filter.c:1253:20: sparse: expected struct event_filter *
kernel/trace/trace_events_filter.c:1253:20: sparse: got struct event_filter [noderef] __rcu *filter

vim +1441 kernel/trace/trace_events_filter.c

1411
1412 static void filter_free_subsystem_filters(struct trace_subsystem_dir *dir,
1413 struct trace_array *tr,
1414 struct event_filter *filter)
1415 {
1416 struct trace_event_file *file;
1417 struct filter_head *head;
1418 struct filter_list *item;
1419
1420 head = kmalloc(sizeof(*head), GFP_KERNEL);
1421 if (!head)
1422 goto free_now;
1423
1424 INIT_LIST_HEAD(&head->list);
1425
1426 item = kmalloc(sizeof(*item), GFP_KERNEL);
1427 if (!item) {
1428 kfree(head);
1429 goto free_now;
1430 }
1431
1432 item->filter = filter;
1433 list_add_tail(&item->list, &head->list);
1434
1435 list_for_each_entry(file, &tr->events, list) {
1436 if (file->system != dir)
1437 continue;
1438 item = kmalloc(sizeof(*item), GFP_KERNEL);
1439 if (!item)
1440 goto free_now;

> 1441			item->filter = file->filter;

1442 list_add_tail(&item->list, &head->list);
1443 file->filter = NULL;
1444 }
1445
1446 delay_free_filter(head);
1447 return;
1448 free_now:
1449 tracepoint_synchronize_unregister();
1450
1451 if (head)
1452 free_filter_list(&head->rcu);
1453
1454 list_for_each_entry(file, &tr->events, list) {
1455 if (file->system != dir || !file->filter)
1456 continue;
1457 __free_filter(file->filter);
1458 }
1459 __free_filter(filter);
1460 }
1461

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#2 lore raw
BCC: [email protected] CC: [email protected] In-Reply-To: <[email protected]> References: <[email protected]> TO: Steven Rostedt…

BCC: [email protected]
CC: [email protected]
In-Reply-To: <[email protected]>
References: <[email protected]>
TO: Steven Rostedt <[email protected]>

Hi Steven,

kernel test robot noticed the following build warnings:

[auto build test WARNING on trace/for-next]
[also build test WARNING on linus/master v6.15 next-20250606] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Steven-Rostedt/tracing-Fix-regression-of-filter-waiting-a-long-time-on-RCU-synchronization/20250606-041722
base:   https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace for-next
patch link:    https://lore.kernel.org/r/20250605161701.35f7989a%40gandalf.local.home
patch subject: [PATCH] tracing: Fix regression of filter waiting a long time on RCU synchronization

:::::: branch date: 25 hours ago
:::::: commit date: 25 hours ago
config: riscv-randconfig-r072-20250606 (https://download.01.org/0day-ci/archive/20250607/[email protected]/config)
compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags

| Reported-by: kernel test robot <[email protected]>
| Reported-by: Dan Carpenter <[email protected]>
| Closes: https://lore.kernel.org/r/[email protected]/

smatch warnings:
kernel/trace/trace_events_filter.c:1452 filter_free_subsystem_filters() warn: 'head' double freed

vim +/head +1452 kernel/trace/trace_events_filter.c

f306cc82a93d6b Tom Zanussi              2013-10-24  1411  
7967b3e0c40ff7 Steven Rostedt (Red Hat  2015-05-13  1412) static void filter_free_subsystem_filters(struct trace_subsystem_dir *dir,
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1413  					  struct trace_array *tr,
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1414  					  struct event_filter *filter)
f306cc82a93d6b Tom Zanussi              2013-10-24  1415  {
7f1d2f8210195c Steven Rostedt (Red Hat  2015-05-05  1416) 	struct trace_event_file *file;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1417  	struct filter_head *head;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1418  	struct filter_list *item;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1419  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1420  	head = kmalloc(sizeof(*head), GFP_KERNEL);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1421  	if (!head)
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1422  		goto free_now;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1423  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1424  	INIT_LIST_HEAD(&head->list);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1425  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1426  	item = kmalloc(sizeof(*item), GFP_KERNEL);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1427  	if (!item) {
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1428  		kfree(head);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1429  		goto free_now;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1430  	}
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1431  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1432  	item->filter = filter;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1433  	list_add_tail(&item->list, &head->list);
cfb180f3e71b2a Tom Zanussi              2009-03-22  1434  
f306cc82a93d6b Tom Zanussi              2013-10-24  1435  	list_for_each_entry(file, &tr->events, list) {
bb9ef1cb7d8668 Oleg Nesterov            2014-07-15  1436  		if (file->system != dir)
8e254c1d183f02 Li Zefan                 2009-08-31  1437  			continue;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1438  		item = kmalloc(sizeof(*item), GFP_KERNEL);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1439  		if (!item)
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1440  			goto free_now;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1441  		item->filter = file->filter;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1442  		list_add_tail(&item->list, &head->list);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1443  		file->filter = NULL;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1444  	}
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1445  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1446  	delay_free_filter(head);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1447  	return;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1448   free_now:
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1449  	tracepoint_synchronize_unregister();
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1450  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1451  	if (head)
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05 @1452  		free_filter_list(&head->rcu);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1453  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1454  	list_for_each_entry(file, &tr->events, list) {
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1455  		if (file->system != dir || !file->filter)
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1456  			continue;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1457  		__free_filter(file->filter);
cfb180f3e71b2a Tom Zanussi              2009-03-22  1458  	}
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1459  	__free_filter(filter);
cfb180f3e71b2a Tom Zanussi              2009-03-22  1460  }
ac1adc55fc71c7 Tom Zanussi              2009-04-17  1461  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author maintainer
#3 lore raw
Quoted 20 lines · On Tue, 10 Jun 2025 12:24:34 +0300 Dan Carpenter <[email protected]>…
7967b3e0c40ff7 Steven Rostedt (Red Hat  2015-05-13  1412) static void filter_free_subsystem_filters(struct trace_subsystem_dir *dir,
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1413  					  struct trace_array *tr,
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1414  					  struct event_filter *filter)
f306cc82a93d6b Tom Zanussi              2013-10-24  1415  {
7f1d2f8210195c Steven Rostedt (Red Hat  2015-05-05  1416) 	struct trace_event_file *file;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1417  	struct filter_head *head;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1418  	struct filter_list *item;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1419  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1420  	head = kmalloc(sizeof(*head), GFP_KERNEL);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1421  	if (!head)
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1422  		goto free_now;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1423  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1424  	INIT_LIST_HEAD(&head->list);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1425  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1426  	item = kmalloc(sizeof(*item), GFP_KERNEL);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1427  	if (!item) {
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1428  		kfree(head);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1429  		goto free_now;
Quoted 1 line

head is freed.

Ah, that's a mistake. It shouldn't be freed.

Quoted 25 lines
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1430  	}
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1431  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1432  	item->filter = filter;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1433  	list_add_tail(&item->list, &head->list);
cfb180f3e71b2a Tom Zanussi              2009-03-22  1434  
f306cc82a93d6b Tom Zanussi              2013-10-24  1435  	list_for_each_entry(file, &tr->events, list) {
bb9ef1cb7d8668 Oleg Nesterov            2014-07-15  1436  		if (file->system != dir)
8e254c1d183f02 Li Zefan                 2009-08-31  1437  			continue;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1438  		item = kmalloc(sizeof(*item), GFP_KERNEL);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1439  		if (!item)
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1440  			goto free_now;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1441  		item->filter = file->filter;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1442  		list_add_tail(&item->list, &head->list);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1443  		file->filter = NULL;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1444  	}
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1445  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1446  	delay_free_filter(head);
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1447  	return;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1448   free_now:
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1449  	tracepoint_synchronize_unregister();
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1450  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1451  	if (head)
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05 @1452  		free_filter_list(&head->rcu);
Quoted 3 lines

This is a use after free. Smatch calls is a "double free" because free_filter_list() has "free" in the name, but Smatch is wrong. However, still a bug.

Right. I'll remove the freeing of head.

Thanks,

-- Steve

Quoted 9 lines
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1453  
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1454  	list_for_each_entry(file, &tr->events, list) {
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1455  		if (file->system != dir || !file->filter)
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1456  			continue;
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1457  		__free_filter(file->filter);
cfb180f3e71b2a Tom Zanussi              2009-03-22  1458  	}
b9c3a0aeabb4e5 Steven Rostedt           2025-06-05  1459  	__free_filter(filter);
cfb180f3e71b2a Tom Zanussi              2009-03-22  1460  }