ftrace: Protect RCU accesses to direct_functions

PATCH In review Leon Hwang 2 participants 6 messages Jul 30 – Aug 2

Subsystems (3)

Stable branch Tracing ftrace

lore

Share & tools

lore

Leon Hwang

author

2026-07-30 15:04 UTC #0 lore raw

cover ftrace: Protect RCU accesses to direct_functions

Sashiko reported a potential issue of directly accessing direct_functions [1], which will be addressed by patch #1.

The patch #2 and #3 protect accesses to the RCU-managed direct_functions hash in its delete, and modify paths.

The final patch removes an unnecessary empty argument from a scoped_guard() invocation.

[1] https://lore.kernel.org/all/[email protected]/

Leon Hwang (4):
ftrace: Protect direct_functions in ftrace_find_rec_direct
ftrace: Protect direct_functions in update_ftrace_direct_del
ftrace: Protect direct_functions in update_ftrace_direct_mod
ftrace: Drop extra comma in trace_buffered_event_enable

 kernel/trace/ftrace.c | 27 ++++++++++++++++++---------
 kernel/trace/trace.c  |  2 +-
 2 files changed, 19 insertions(+), 10 deletions(-)

Signature · 1 line

2.55.0

Leon Hwang

author

2026-07-30 15:04 UTC #1 lore raw

PATCH 1/4 ftrace: Protect direct_functions in ftrace_find_rec_direct

Fix accessing the __rcu pointer direct_functions with RCU protection.

Cc [email protected]

Fixes d05cb470663a ("ftrace: Fix modification of direct_function hash while in use")

Signed-off-by Leon Hwang <[email protected]>

---
 kernel/trace/ftrace.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

kernel/trace/ftrace.c +2 -1

index 6c47a94f5924..c5d1d0d42ccc 100644
@@ -2645,7 +2645,8 @@ unsigned long ftrace_find_rec_direct(unsigned long ip)
 {
 	struct ftrace_func_entry *entry;
 
-	entry = __ftrace_lookup_ip(direct_functions, ip);
+	guard(preempt_notrace)();
+	entry = __ftrace_lookup_ip(rcu_dereference_sched(direct_functions), ip);
 	if (!entry)
 		return 0;

Signature · 1 line

2.55.0

Leon Hwang

author

2026-07-30 15:04 UTC #2 lore raw

PATCH 2/4 ftrace: Protect direct_functions in update_ftrace_direct_del

In reply to Leon Hwang

Fix accessing the __rcu pointer direct_functions with RCU protection.

Cc [email protected]

Fixes 8d2c1233f371 ("ftrace: Add update_ftrace_direct_del function")

Signed-off-by Leon Hwang <[email protected]>

---
 kernel/trace/ftrace.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

kernel/trace/ftrace.c +8 -5

index c5d1d0d42ccc..9ea39110927f 100644
@@ -6512,6 +6512,7 @@ int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash)
 	struct ftrace_hash *new_direct_functions;
 	struct ftrace_hash *new_filter_hash = NULL;
 	struct ftrace_hash *old_filter_hash;
+	struct ftrace_hash *direct_hash;
 	struct ftrace_func_entry *entry;
 	struct ftrace_func_entry *del;
 	unsigned long size;

@@ -6523,11 +6524,13 @@ int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash)
 		return -EINVAL;
 	if (!(ops->flags & FTRACE_OPS_FL_ENABLED))
 		return -EINVAL;
-	if (direct_functions == EMPTY_HASH)
-		return -EINVAL;
 
 	mutex_lock(&direct_mutex);
 
+	direct_hash = rcu_dereference_protected(direct_functions, lockdep_is_held(&direct_mutex));
+	if (direct_hash == EMPTY_HASH)
+		goto out_unlock;
+
 	old_filter_hash = ops->func_hash ? ops->func_hash->filter_hash : NULL;
 
 	if (!hash_count(old_filter_hash))

@@ -6537,7 +6540,7 @@ int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash)
 	size = 1 << hash->size_bits;
 	for (int i = 0; i < size; i++) {
 		hlist_for_each_entry(entry, &hash->buckets[i], hlist) {
-			del = __ftrace_lookup_ip(direct_functions, entry->ip);
+			del = __ftrace_lookup_ip(direct_hash, entry->ip);
 			if (!del || del->direct != entry->direct)
 				goto out_unlock;
 		}

@@ -6548,7 +6551,7 @@ int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash)
 	if (!new_filter_hash)
 		goto out_unlock;
 
-	new_direct_functions = hash_sub(direct_functions, hash);
+	new_direct_functions = hash_sub(direct_hash, hash);
 	if (!new_direct_functions)
 		goto out_unlock;

@@ -6575,7 +6578,7 @@ int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash)
 		/* free the new_direct_functions */
 		old_direct_functions = new_direct_functions;
 	} else {
-		old_direct_functions = direct_functions;
+		old_direct_functions = direct_hash;
 		rcu_assign_pointer(direct_functions, new_direct_functions);
 	}

Signature · 1 line

2.55.0

Leon Hwang

author

2026-07-30 15:05 UTC #3 lore raw

PATCH 3/4 ftrace: Protect direct_functions in update_ftrace_direct_mod

In reply to Leon Hwang

Fix accessing the __rcu pointer direct_functions with RCU protection.

Cc [email protected]

Fixes e93672f770d7 ("ftrace: Add update_ftrace_direct_mod function")

Signed-off-by Leon Hwang <[email protected]>

---
 kernel/trace/ftrace.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

kernel/trace/ftrace.c +8 -3

index 9ea39110927f..414e425c2d80 100644
@@ -6617,6 +6617,7 @@ int update_ftrace_direct_mod(struct ftrace_ops *ops, struct ftrace_hash *hash, b
 		.func		= ftrace_stub,
 		.flags		= FTRACE_OPS_FL_STUB,
 	};
+	struct ftrace_hash *direct_hash;
 	struct ftrace_hash *orig_hash;
 	unsigned long size, i;
 	int err = -EINVAL;

@@ -6627,8 +6628,6 @@ int update_ftrace_direct_mod(struct ftrace_ops *ops, struct ftrace_hash *hash, b
 		return -EINVAL;
 	if (!(ops->flags & FTRACE_OPS_FL_ENABLED))
 		return -EINVAL;
-	if (direct_functions == EMPTY_HASH)
-		return -EINVAL;
 
 	/*
 	 * We can be called from within ops_func callback with direct_mutex

@@ -6636,6 +6635,12 @@ int update_ftrace_direct_mod(struct ftrace_ops *ops, struct ftrace_hash *hash, b
 	 */
 	if (do_direct_lock)
 		mutex_lock(&direct_mutex);
+	else
+		lockdep_assert_held_once(&direct_mutex);
+
+	direct_hash = rcu_dereference_protected(direct_functions, lockdep_is_held(&direct_mutex));
+	if (direct_hash == EMPTY_HASH)
+		goto unlock;
 
 	orig_hash = ops->func_hash ? ops->func_hash->filter_hash : NULL;
 	if (!orig_hash)

@@ -6667,7 +6672,7 @@ int update_ftrace_direct_mod(struct ftrace_ops *ops, struct ftrace_hash *hash, b
 	size = 1 << hash->size_bits;
 	for (i = 0; i < size; i++) {
 		hlist_for_each_entry(entry, &hash->buckets[i], hlist) {
-			tmp = __ftrace_lookup_ip(direct_functions, entry->ip);
+			tmp = __ftrace_lookup_ip(direct_hash, entry->ip);
 			if (!tmp)
 				continue;
 			tmp->direct = entry->direct;

Signature · 1 line

2.55.0

Leon Hwang

author

2026-07-30 15:05 UTC #4 lore raw

PATCH 4/4 ftrace: Drop extra comma in trace_buffered_event_enable

In reply to Leon Hwang

Drop the extra comma in "scoped_guard()" to cleanup the code.

Signed-off-by Leon Hwang <[email protected]>

---
 kernel/trace/trace.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

kernel/trace/trace.c +1 -1

index 19cc07360005..19b0d7a53c58 100644
@@ -1788,7 +1788,7 @@ void trace_buffered_event_enable(void)
 
 		per_cpu(trace_buffered_event, cpu) = event;
 
-		scoped_guard(preempt,) {
+		scoped_guard(preempt) {
 			if (cpu == smp_processor_id() &&
 			    __this_cpu_read(trace_buffered_event) !=
 			    per_cpu(trace_buffered_event, cpu))

Signature · 1 line

2.55.0

Jiri Olsa

Acked-by on the v1 series

2026-08-02 20:37 UTC #5 lore raw

In reply to Leon Hwang

Quoted 9 lines · Leon Hwang

Sashiko reported a potential issue of directly accessing direct_functions [1], which will be addressed by patch #1.
The patch #2 and #3 protect accesses to the RCU-managed direct_functions hash in its delete, and modify paths.
The final patch removes an unnecessary empty argument from a scoped_guard() invocation.

Quoted 7 lines

[1] https://lore.kernel.org/all/[email protected]/
Leon Hwang (4):
ftrace: Protect direct_functions in ftrace_find_rec_direct
ftrace: Protect direct_functions in update_ftrace_direct_del
ftrace: Protect direct_functions in update_ftrace_direct_mod
ftrace: Drop extra comma in trace_buffered_event_enable

Acked-by Jiri Olsa <[email protected]>

thanks,
jirka

Quoted 8 lines

 kernel/trace/ftrace.c | 27 ++++++++++++++++++---------
 kernel/trace/trace.c  |  2 +-
 2 files changed, 19 insertions(+), 10 deletions(-)

Signature · 1 line

2.55.0