Skip to contentSkip to conversation

rcutorture: Synchronously wait for all rcu_torture_irq() callbacks to complete

PATCH In review Zqiang 2 participants 2 messages Aug 24 – Aug 27

Subsystems (2)
lore
Share & tools
lore

Zqiang

author reviewer
#0 lore raw

PATCH 1/1 rcutorture: Synchronously wait for all rcu_torture_irq() callbacks to complete

The rcu_torture_reader() drives RCU readers from interrupt context via smp_call_function_single(cpu, rcu_torture_irq, NULL, 0) with wait=0, to runs rcu_torture_irq() on a remote CPU. this is async, nothing waits for the remote handler to run.

On shutdown, torture_stop_kthread() only waits for each reader kthread to return, and the reader's timer_delete_sync() only drains its timer. Neither waits for a rcu_torture_irq() which still pending or executing on a remote CPU, so it can run after all readers have exited and rcu_torture_cleanup() has already advanced.

1. rcu_torture_irq() may issue cur_ops->call(rhp, rcu_torture_timer_cb) after cur_ops->cb_barrier() has been waiting for all outstanding callbacks complete. once the module is unloaded, fires into freed module text, a use-after-free happen.

2. rcu_torture_irq() may still be inside rcu_torture_one_read(), holding a read-side critical section, when cur_ops->cleanup() tears the flavor down (e.g. cleanup_srcu_struct()), triggering an active-reader warning or use-after-free of the torn-down structure.

This commit therefore issue a kick_all_cpus_sync() after all readers kthread have returned and before cur_ops->cb_barrier(), synchronous IPI round trip to every CPU guarantees that every rcu_torture_irq() which previously issued by any reader has completed.

---
 kernel/rcu/rcutorture.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/rcu/rcutorture.c +2 -0
index 4b4d9c70e827..f177ba9cb604 100644
@@ -4476,6 +4476,8 @@ rcu_torture_cleanup(void)
 		for (i = 0; i < nrealreaders; i++)
 			torture_stop_kthread(rcu_torture_reader,
 					     reader_tasks[i]);
+		if (irqreader && cur_ops->irq_capable)
+			kick_all_cpus_sync();
 		kfree(reader_tasks);
 		reader_tasks = NULL;
 	}
Signature · 1 line
2.17.1
maintainer
#1 lore raw
Quoted 21 lines · Zqiang

The rcu_torture_reader() drives RCU readers from interrupt context via smp_call_function_single(cpu, rcu_torture_irq, NULL, 0) with wait=0, to runs rcu_torture_irq() on a remote CPU. this is async, nothing waits for the remote handler to run.

On shutdown, torture_stop_kthread() only waits for each reader kthread to return, and the reader's timer_delete_sync() only drains its timer. Neither waits for a rcu_torture_irq() which still pending or executing on a remote CPU, so it can run after all readers have exited and rcu_torture_cleanup() has already advanced.

1. rcu_torture_irq() may issue cur_ops->call(rhp, rcu_torture_timer_cb) after cur_ops->cb_barrier() has been waiting for all outstanding callbacks complete. once the module is unloaded, fires into freed module text, a use-after-free happen.

2. rcu_torture_irq() may still be inside rcu_torture_one_read(), holding a read-side critical section, when cur_ops->cleanup() tears the flavor down (e.g. cleanup_srcu_struct()), triggering an active-reader warning or use-after-free of the torn-down structure.

Quoted 6 lines

This commit therefore issue a kick_all_cpus_sync() after all readers kthread have returned and before cur_ops->cb_barrier(), synchronous IPI round trip to every CPU guarantees that every rcu_torture_irq() which previously issued by any reader has completed.

Good catch! Queued for further review and testing, thank you!

Thanx, Paul

Quoted 20 lines
---
 kernel/rcu/rcutorture.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/rcu/rcutorture.c +2 -0
index 4b4d9c70e827..f177ba9cb604 100644
@@ -4476,6 +4476,8 @@ rcu_torture_cleanup(void)
 		for (i = 0; i < nrealreaders; i++)
 			torture_stop_kthread(rcu_torture_reader,
 					     reader_tasks[i]);
+		if (irqreader && cur_ops->irq_capable)
+			kick_all_cpus_sync();
 		kfree(reader_tasks);
 		reader_tasks = NULL;
 	}
Signature · 1 line
2.17.1