Skip to contentSkip to conversation

irq_work: Provide a soft-irq based queue

PATCH Applied Jan Kiszka 3 participants 3 messages Apr 23, 2015 – May 15, 2015

lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

PATCH v2 1/1 irq_work: Provide a soft-irq based queue

Instead of turning all irq_work requests into lazy ones on -rt, just move their execution from hard into soft-irq context.

This resolves deadlocks of ftrace which will queue work from arbitrary contexts, including those that have locks held that are needed for raising a soft-irq.

---

Changes in v2:
- fix execution of raised list (discovered by Mike Galbraith)
- added comment of irq_work_run (derived from Mike's suggestion)

 kernel/irq_work.c | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)
kernel/irq_work.c +17 -17
index 9dda38a..171dfac 100644
@@ -85,12 +85,9 @@ bool irq_work_queue_on(struct irq_work *work, int cpu)
 		raise_irqwork = llist_add(&work->llnode,
 					  &per_cpu(hirq_work_list, cpu));
 	else
-		raise_irqwork = llist_add(&work->llnode,
-					  &per_cpu(lazy_list, cpu));
-#else
+#endif
 		raise_irqwork = llist_add(&work->llnode,
 					  &per_cpu(raised_list, cpu));
-#endif
 
 	if (raise_irqwork)
 		arch_send_call_function_single_ipi(cpu);
@@ -114,21 +111,20 @@ bool irq_work_queue(struct irq_work *work)
 	if (work->flags & IRQ_WORK_HARD_IRQ) {
 		if (llist_add(&work->llnode, this_cpu_ptr(&hirq_work_list)))
 			arch_irq_work_raise();
-	} else {
+	} else
+#endif
+	if (work->flags & IRQ_WORK_LAZY) {
 		if (llist_add(&work->llnode, this_cpu_ptr(&lazy_list)) &&
 		    tick_nohz_tick_stopped())
+#ifdef CONFIG_PREEMPT_RT_FULL
 			raise_softirq(TIMER_SOFTIRQ);
-	}
 #else
-	if (work->flags & IRQ_WORK_LAZY) {
-		if (llist_add(&work->llnode, this_cpu_ptr(&lazy_list)) &&
-		    tick_nohz_tick_stopped())
 			arch_irq_work_raise();
+#endif
 	} else {
 		if (llist_add(&work->llnode, this_cpu_ptr(&raised_list)))
 			arch_irq_work_raise();
 	}
-#endif
 
 	preempt_enable();
 
@@ -202,6 +198,13 @@ void irq_work_run(void)
 {
 #ifdef CONFIG_PREEMPT_RT_FULL
 	irq_work_run_list(this_cpu_ptr(&hirq_work_list));
+	/*
+	 * NOTE: we raise softirq via IPI for safety (caller may hold locks
+	 * that raise_softirq needs) and execute in irq_work_tick() to move
+	 * the overhead from hard to soft irq context.
+	 */
+	if (!llist_empty(this_cpu_ptr(&raised_list)))
+		raise_softirq(TIMER_SOFTIRQ);
 #else
 	irq_work_run_list(this_cpu_ptr(&raised_list));
 	irq_work_run_list(this_cpu_ptr(&lazy_list));
@@ -211,15 +214,12 @@ EXPORT_SYMBOL_GPL(irq_work_run);
 
 void irq_work_tick(void)
 {
-#ifdef CONFIG_PREEMPT_RT_FULL
-	irq_work_run_list(this_cpu_ptr(&lazy_list));
-#else
-	struct llist_head *raised = &__get_cpu_var(raised_list);
+	struct llist_head *raised = this_cpu_ptr(&raised_list);
 
-	if (!llist_empty(raised) && !arch_irq_work_has_interrupt())
+	if (!llist_empty(raised) && (!arch_irq_work_has_interrupt() ||
+				     IS_ENABLED(CONFIG_PREEMPT_RT_FULL)))
 		irq_work_run_list(raised);
-	irq_work_run_list(&__get_cpu_var(lazy_list));
-#endif
+	irq_work_run_list(this_cpu_ptr(&lazy_list));
 }
 
 /*
Signature · 1 line
2.1.4
#1 lore raw
Quoted 8 lines · * Jan Kiszka | 2015-04-23 09:35:59 [+0200]:

Instead of turning all irq_work requests into lazy ones on -rt, just move their execution from hard into soft-irq context.

This resolves deadlocks of ftrace which will queue work from arbitrary contexts, including those that have locks held that are needed for raising a soft-irq.

Applied

Sebastian

#2 lore raw
Quoted 11 lines · Sebastian Andrzej Siewior
Quoted 8 lines · * Jan Kiszka | 2015-04-23 09:35:59 [+0200]:

Instead of turning all irq_work requests into lazy ones on -rt, just move their execution from hard into soft-irq context.

This resolves deadlocks of ftrace which will queue work from arbitrary contexts, including those that have locks held that are needed for raising a soft-irq.

Applied

The thread went on, and Mike suggested an alternative implementation [1] that works fine and is even cleaner. Let's pick his.

Jan

[1] http://thread.gmane.org/gmane.linux.kernel/1937960