Skip to contentSkip to conversation

rcu: Return early if callback is not specified

PATCH In review Uladzislau Rezki (Sony) 4 participants 12 messages Jun 10, 2025 – Jun 17, 2025

Subsystems (2)
lore
Share & tools
lore

Uladzislau Rezki (Sony)

author maintainer
#0 lore raw

PATCH 1/3 rcu: Return early if callback is not specified

Currently the call_rcu() API does not check whether a callback pointer is NULL. If NULL is passed, rcu_core() will try to invoke it, resulting in NULL pointer dereference and a kernel crash.

To prevent this and improve debuggability, this patch adds a check for NULL and emits a kernel stack trace to help identify a faulty caller.

---
 kernel/rcu/tree.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/rcu/tree.c +4 -0
index e8a4b720d7d2..14d4499c6fc3 100644
@@ -3072,6 +3072,10 @@ __call_rcu_common(struct rcu_head *head, rcu_callback_t func, bool lazy_in)
 	/* Misaligned rcu_head! */
 	WARN_ON_ONCE((unsigned long)head & (sizeof(void *) - 1));
 
+	/* Avoid NULL dereference if callback is NULL. */
+	if (WARN_ON_ONCE(!func))
+		return;
+
 	if (debug_rcu_head_queue(head)) {
 		/*
 		 * Probable double call_rcu(), so leak the callback.
Signature · 1 line
2.39.5
author maintainer
#1 lore raw

PATCH 2/3 rcu: Enable rcu_normal_wake_from_gp on small systems

Automatically enable the rcu_normal_wake_from_gp parameter on systems with a small number of CPUs. The activation threshold is set to 16 CPUs.

This helps to reduce a latency of normal synchronize_rcu() API by waking up GP-waiters earlier and decoupling synchronize_rcu() callers from regular callback handling.

A benchmark running 64 parallel jobs invoking synchronize_rcu() demonstrates a notable latency reduction with the setting enabled.

Latency distribution (microseconds):

<default>

 0      - 9999   : 1
 10000  - 19999  : 4
 20000  - 29999  : 399
 30000  - 39999  : 3197
 40000  - 49999  : 10428
 50000  - 59999  : 17363
 60000  - 69999  : 15529
 70000  - 79999  : 9287
 80000  - 89999  : 4249
 90000  - 99999  : 1915

100000 - 109999 : 922
110000 - 119999 : 390
120000 - 129999 : 187
...
<default>

<rcu_normal_wake_from_gp>

 0      - 9999  : 1
 10000  - 19999 : 234
 20000  - 29999 : 6678
 30000  - 39999 : 33463
 40000  - 49999 : 20669
 50000  - 59999 : 2766
 60000  - 69999 : 183

...
<rcu_normal_wake_from_gp>

---
 kernel/rcu/tree.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +6 -1
index 14d4499c6fc3..c0e0b38a08dc 100644
@@ -1625,7 +1625,9 @@ static void rcu_sr_put_wait_head(struct llist_node *node)
 	atomic_set_release(&sr_wn->inuse, 0);
 }
 
-/* Disabled by default. */
+/* Enable rcu_normal_wake_from_gp automatically on small systems. */
+#define WAKE_FROM_GP_CPU_THRESHOLD 16
+
 static int rcu_normal_wake_from_gp;
 module_param(rcu_normal_wake_from_gp, int, 0644);
 static struct workqueue_struct *sync_wq;
@@ -4847,6 +4849,9 @@ void __init rcu_init(void)
 	sync_wq = alloc_workqueue("sync_wq", WQ_MEM_RECLAIM, 0);
 	WARN_ON(!sync_wq);
 
+	if (num_possible_cpus() <= WAKE_FROM_GP_CPU_THRESHOLD)
+		WRITE_ONCE(rcu_normal_wake_from_gp, 1);
+
 	/* Fill in default value for rcutree.qovld boot parameter. */
 	/* -After- the rcu_node ->lock fields are initialized! */
 	if (qovld < 0)
Signature · 1 line
2.39.5
maintainer
#2 lore raw

Re: [PATCH 2/3] rcu: Enable rcu_normal_wake_from_gp on small systems

Quoted 62 lines · Uladzislau Rezki (Sony)

Automatically enable the rcu_normal_wake_from_gp parameter on systems with a small number of CPUs. The activation threshold is set to 16 CPUs.

This helps to reduce a latency of normal synchronize_rcu() API by waking up GP-waiters earlier and decoupling synchronize_rcu() callers from regular callback handling.

A benchmark running 64 parallel jobs invoking synchronize_rcu() demonstrates a notable latency reduction with the setting enabled.

Latency distribution (microseconds):

<default>

 0      - 9999   : 1
 10000  - 19999  : 4
 20000  - 29999  : 399
 30000  - 39999  : 3197
 40000  - 49999  : 10428
 50000  - 59999  : 17363
 60000  - 69999  : 15529
 70000  - 79999  : 9287
 80000  - 89999  : 4249
 90000  - 99999  : 1915

100000 - 109999 : 922
110000 - 119999 : 390
120000 - 129999 : 187
...
<default>

<rcu_normal_wake_from_gp>

 0      - 9999  : 1
 10000  - 19999 : 234
 20000  - 29999 : 6678
 30000  - 39999 : 33463
 40000  - 49999 : 20669
 50000  - 59999 : 2766
 60000  - 69999 : 183

...
<rcu_normal_wake_from_gp>

---
 kernel/rcu/tree.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +3 -1
index 14d4499c6fc3..c0e0b38a08dc 100644
@@ -1625,7 +1625,9 @@ static void rcu_sr_put_wait_head(struct llist_node *node)
 	atomic_set_release(&sr_wn->inuse, 0);
 }
 
-/* Disabled by default. */
+/* Enable rcu_normal_wake_from_gp automatically on small systems. */
+#define WAKE_FROM_GP_CPU_THRESHOLD 16
+
 static int rcu_normal_wake_from_gp;
 module_param(rcu_normal_wake_from_gp, int, 0644);
 static struct workqueue_struct *sync_wq;

kernel/rcu/tree.c void __init rcu_init(void) · line 4854

Quoted 6 lines
kernel/rcu/tree.c +3 -0
@@ -4847,6 +4849,9 @@ void __init rcu_init(void)
 	sync_wq = alloc_workqueue("sync_wq", WQ_MEM_RECLAIM, 0);
 	WARN_ON(!sync_wq);
 
+	if (num_possible_cpus() <= WAKE_FROM_GP_CPU_THRESHOLD)
+		WRITE_ONCE(rcu_normal_wake_from_gp, 1);
+

I think this will get weird if user explictly specifies rcutree.rcu_normal_wake_from_gp=0 ? Then we're silently overriding the param. Maybe, initialize it to -1, and then if it was set 0 by user, don't override it. But otherwise, set it to 1. Per your third patch, '1' is a default, not a forced value.

thanks,

- Joel

author maintainer
#3 lore raw

Re: [PATCH 2/3] rcu: Enable rcu_normal_wake_from_gp on small systems

Quoted 68 lines · Joel Fernandes
Quoted 65 lines · Uladzislau Rezki (Sony)

Automatically enable the rcu_normal_wake_from_gp parameter on systems with a small number of CPUs. The activation threshold is set to 16 CPUs.

This helps to reduce a latency of normal synchronize_rcu() API by waking up GP-waiters earlier and decoupling synchronize_rcu() callers from regular callback handling.

A benchmark running 64 parallel jobs invoking synchronize_rcu() demonstrates a notable latency reduction with the setting enabled.

Latency distribution (microseconds):

<default>

 0      - 9999   : 1
 10000  - 19999  : 4
 20000  - 29999  : 399
 30000  - 39999  : 3197
 40000  - 49999  : 10428
 50000  - 59999  : 17363
 60000  - 69999  : 15529
 70000  - 79999  : 9287
 80000  - 89999  : 4249
 90000  - 99999  : 1915

100000 - 109999 : 922
110000 - 119999 : 390
120000 - 129999 : 187
...
<default>

<rcu_normal_wake_from_gp>

 0      - 9999  : 1
 10000  - 19999 : 234
 20000  - 29999 : 6678
 30000  - 39999 : 33463
 40000  - 49999 : 20669
 50000  - 59999 : 2766
 60000  - 69999 : 183

...
<rcu_normal_wake_from_gp>

---
 kernel/rcu/tree.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +3 -1
index 14d4499c6fc3..c0e0b38a08dc 100644
@@ -1625,7 +1625,9 @@ static void rcu_sr_put_wait_head(struct llist_node *node)
 	atomic_set_release(&sr_wn->inuse, 0);
 }
 
-/* Disabled by default. */
+/* Enable rcu_normal_wake_from_gp automatically on small systems. */
+#define WAKE_FROM_GP_CPU_THRESHOLD 16
+
 static int rcu_normal_wake_from_gp;
 module_param(rcu_normal_wake_from_gp, int, 0644);
 static struct workqueue_struct *sync_wq;
@@ -4847,6 +4849,9 @@ void __init rcu_init(void)
 	sync_wq = alloc_workqueue("sync_wq", WQ_MEM_RECLAIM, 0);
 	WARN_ON(!sync_wq);
 
Quoted 9 lines

kernel/rcu/tree.c void __init rcu_init(void) · line 4854

Quoted 3 lines
kernel/rcu/tree.c +3 -0
+	if (num_possible_cpus() <= WAKE_FROM_GP_CPU_THRESHOLD)
+		WRITE_ONCE(rcu_normal_wake_from_gp, 1);
+

I think this will get weird if user explictly specifies rcutree.rcu_normal_wake_from_gp=0 ? Then we're silently overriding the param. Maybe, initialize it to -1, and then if it was set 0 by user, don't override it. But otherwise, set it to 1. Per your third patch, '1' is a default, not a forced value.

Right. That case should be fixed.

--
Uladzislau Rezki

author maintainer
#4 lore raw

PATCH 3/3 Documentation/kernel-parameters: Update rcu_normal_wake_from_gp doc

In reply to Uladzislau Rezki (Sony)

Update the documentation about rcu_normal_wake_from_gp parameter.

---
 Documentation/admin-guide/kernel-parameters.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/admin-guide/kernel-parameters.txt +1 -1
index f1f2c0874da9..a2c47bdf75cf 100644
@@ -5485,7 +5485,7 @@
 			echo 1 > /sys/module/rcutree/parameters/rcu_normal_wake_from_gp
 			or pass a boot parameter "rcutree.rcu_normal_wake_from_gp=1"
 
-			Default is 0.
+			Default is 1 if num_possible_cpus() <= 16 otherwise 0.
 
 	rcuscale.gp_async= [KNL]
 			Measure performance of asynchronous
Signature · 1 line
2.39.5
maintainer Reviewed-by on v1 1/3
#5 lore raw
In reply to Uladzislau Rezki (Sony)
Quoted 9 lines · Uladzislau Rezki (Sony)

Currently the call_rcu() API does not check whether a callback pointer is NULL. If NULL is passed, rcu_core() will try to invoke it, resulting in NULL pointer dereference and a kernel crash.

To prevent this and improve debuggability, this patch adds a check for NULL and emits a kernel stack trace to help identify a faulty caller.

I will add this first one (only this one since we're discussing the others) to a new rcu/fixes-for-6.16 branch, but let me know if any objections.

Will push that branch out during -rc2 or -rc3 after sufficient testing.

thanks,

- Joel

Quoted 19 lines
---
 kernel/rcu/tree.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/rcu/tree.c +4 -0
index e8a4b720d7d2..14d4499c6fc3 100644
@@ -3072,6 +3072,10 @@ __call_rcu_common(struct rcu_head *head, rcu_callback_t func, bool lazy_in)
 	/* Misaligned rcu_head! */
 	WARN_ON_ONCE((unsigned long)head & (sizeof(void *) - 1));
 
+	/* Avoid NULL dereference if callback is NULL. */
+	if (WARN_ON_ONCE(!func))
+		return;
+
 	if (debug_rcu_head_queue(head)) {
 		/*
 		 * Probable double call_rcu(), so leak the callback.
author maintainer
#6 lore raw
Quoted 11 lines · Joel Fernandes
Quoted 8 lines · Uladzislau Rezki (Sony)

Currently the call_rcu() API does not check whether a callback pointer is NULL. If NULL is passed, rcu_core() will try to invoke it, resulting in NULL pointer dereference and a kernel crash.

To prevent this and improve debuggability, this patch adds a check for NULL and emits a kernel stack trace to help identify a faulty caller.

Quoted 9 lines
Quoted 1 line

I will add this first one (only this one since we're discussing the others) to a new rcu/fixes-for-6.16 branch, but let me know if any objections.

Will push that branch out during -rc2 or -rc3 after sufficient testing.

Yep, that sounds good to me about rc-2/3 releases.

--
Uladzislau Rezki

Reviewed-by
#7 lore raw
In reply to Joel Fernandes
Quoted 5 lines · Joel Fernandes
Quoted 4 lines · Uladzislau Rezki (Sony)

Currently the call_rcu() API does not check whether a callback pointer is NULL. If NULL is passed, rcu_core() will try to invoke it, resulting in NULL pointer dereference and a kernel crash.

Quoted 8 lines
Quoted 5 lines

To prevent this and improve debuggability, this patch adds a check for NULL and emits a kernel stack trace to help identify a faulty caller.

Quoted 3 lines

I will add this first one (only this one since we're discussing the others) to a new rcu/fixes-for-6.16 branch, but let me know if any objections.

Not sure it’s urgent enough given the current evidence.

Regards,
Boqun

Quoted 25 lines

Will push that branch out during -rc2 or -rc3 after sufficient testing.

thanks,

- Joel

Quoted 19 lines
---
 kernel/rcu/tree.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/rcu/tree.c +4 -0
index e8a4b720d7d2..14d4499c6fc3 100644
@@ -3072,6 +3072,10 @@ __call_rcu_common(struct rcu_head *head, rcu_callback_t func, bool lazy_in)
 	/* Misaligned rcu_head! */
 	WARN_ON_ONCE((unsigned long)head & (sizeof(void *) - 1));
 
+	/* Avoid NULL dereference if callback is NULL. */
+	if (WARN_ON_ONCE(!func))
+		return;
+
 	if (debug_rcu_head_queue(head)) {
 		/*
 		 * Probable double call_rcu(), so leak the callback.
#8 lore raw
Quoted 10 lines · Boqun Feng
Quoted 9 lines · Joel Fernandes
Quoted 8 lines · Uladzislau Rezki (Sony)

Currently the call_rcu() API does not check whether a callback pointer is NULL. If NULL is passed, rcu_core() will try to invoke it, resulting in NULL pointer dereference and a kernel crash.

To prevent this and improve debuggability, this patch adds a check for NULL and emits a kernel stack trace to help identify a faulty caller.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

(Accidentally sent from another email account, FWIW, this is Boqun Feng and I approve this message, sorry couldn’t resist :))

Regards,
Boqun

Quoted 34 lines
Quoted 3 lines

I will add this first one (only this one since we're discussing the others) to a new rcu/fixes-for-6.16 branch, but let me know if any objections.

Not sure it’s urgent enough given the current evidence.

Regards,
Boqun

Quoted 25 lines

Will push that branch out during -rc2 or -rc3 after sufficient testing.

thanks,

- Joel

Quoted 19 lines
---
 kernel/rcu/tree.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/rcu/tree.c +4 -0
index e8a4b720d7d2..14d4499c6fc3 100644
@@ -3072,6 +3072,10 @@ __call_rcu_common(struct rcu_head *head, rcu_callback_t func, bool lazy_in)
 	/* Misaligned rcu_head! */
 	WARN_ON_ONCE((unsigned long)head & (sizeof(void *) - 1));
 
+	/* Avoid NULL dereference if callback is NULL. */
+	if (WARN_ON_ONCE(!func))
+		return;
+
 	if (debug_rcu_head_queue(head)) {
 		/*
 		 * Probable double call_rcu(), so leak the callback.
author maintainer
#9 lore raw
In reply to Boqun Feng
Quoted 12 lines · Boqun Feng
Quoted 9 lines · Joel Fernandes
Quoted 8 lines · Uladzislau Rezki (Sony)

Currently the call_rcu() API does not check whether a callback pointer is NULL. If NULL is passed, rcu_core() will try to invoke it, resulting in NULL pointer dereference and a kernel crash.

To prevent this and improve debuggability, this patch adds a check for NULL and emits a kernel stack trace to help identify a faulty caller.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Thank you for review, Boqun!

Quoted 6 lines
Quoted 3 lines

I will add this first one (only this one since we're discussing the others) to a new rcu/fixes-for-6.16 branch, but let me know if any objections.

Not sure it’s urgent enough given the current evidence.

Let me clarify it a bit. My point is that, we get a kernel crash in a subsystem we are responsible for, i.e. no matter if there are faulty users of it(third party applications), the point is users can crash it.

The kernel robot reports it and it is already a strong indication that the subsystem is not hardened against invalid inputs:

"BUG: unable to handle kernel NULL pointer dereference in rcu_core (3)"

so this in the rcu_core() which is part of RCU.

But, anyway Joel should decide. I shared my opinion :)

--
Uladzislau Rezki

#10 lore raw
Quoted 26 lines · Uladzislau Rezki
Quoted 12 lines · Boqun Feng
Quoted 9 lines · Joel Fernandes
Quoted 8 lines · Uladzislau Rezki (Sony)

Currently the call_rcu() API does not check whether a callback pointer is NULL. If NULL is passed, rcu_core() will try to invoke it, resulting in NULL pointer dereference and a kernel crash.

To prevent this and improve debuggability, this patch adds a check for NULL and emits a kernel stack trace to help identify a faulty caller.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Thank you for review, Boqun!

Quoted 4 lines
Quoted 3 lines

I will add this first one (only this one since we're discussing the others) to a new rcu/fixes-for-6.16 branch, but let me know if any objections.

Quoted 6 lines
Quoted 2 lines

Not sure it´s urgent enough given the current evidence.

Let me clarify it a bit. My point is that, we get a kernel crash in a subsystem we are responsible for, i.e. no matter if there are faulty users of it(third party applications), the point is users can crash it.

Fair enough.

Quoted 9 lines

The kernel robot reports it and it is already a strong indication that the subsystem is not hardened against invalid inputs:

"BUG: unable to handle kernel NULL pointer dereference in rcu_core (3)"

so this in the rcu_core() which is part of RCU.

But, anyway Joel should decide. I shared my opinion :)

Of course, my point is that the urgency is not high enough so we have to put it in rcu/fixes, but it's a fix, and if Joel had the time to do it, feel free. Joel's decision.

Regards,
Boqun

Quoted 2 lines

--
Uladzislau Rezki

maintainer
#11 lore raw

On 6/12/2025 5:03 PM, Boqun Feng wrote:
[..]

Quoted 6 lines
Quoted 6 lines

The kernel robot reports it and it is already a strong indication that the subsystem is not hardened against invalid inputs:

"BUG: unable to handle kernel NULL pointer dereference in rcu_core (3)"

Quoted 9 lines
Quoted 4 lines

so this in the rcu_core() which is part of RCU.

But, anyway Joel should decide. I shared my opinion :)

Of course, my point is that the urgency is not high enough so we have to put it in rcu/fixes, but it's a fix, and if Joel had the time to do it, feel free. Joel's decision.

Yeah I feel Vlad's fix for a crash is important so I'll send this up to Linus for 6.16 after some testing this week. For the other 2 patches, since that is triggered by a trace point, I'll just let Neeraj take them for 6.17.

thanks,

- Joel