Skip to contentSkip to conversation

full no_hz: how works arch_irq_work_raise() with tick off on generic implementation

DISCUSSION Sebastian Andrzej Siewior 2 participants 2 messages Mar 9, 2015

lore
Share & tools
lore
#0 lore raw

I just stumbled over this in -RT. The code:

87 bool irq_work_queue(struct irq_work *work)
88 {
…
96 /* If the work is "lazy", handle it from next tick if any */
97 if (work->flags & IRQ_WORK_LAZY) {
98 if (llist_add(&work->llnode, this_cpu_ptr(&lazy_list)) &&
99 tick_nohz_tick_stopped())
100 arch_irq_work_raise();
101 } else {
102 if (llist_add(&work->llnode, this_cpu_ptr(&raised_list)))
103 arch_irq_work_raise();
104 }
…
109 }

so what I asked myself: What happens if the CPU is in NO_HZ mode with the tick off and you invoke arch_irq_work_raise() on an architecture without a special IRQ_WORK interrupt?
And why you don't this tick_nohz_tick_stopped() check for the raised_list?
Wouldn't it be a easier for your full NO_HZ code to simply run in the hardirq exit path instead of using irq_work?

Sebastian

#1 lore raw
Quoted 17 lines · Sebastian Andrzej Siewior

I just stumbled over this in -RT. The code:

87 bool irq_work_queue(struct irq_work *work)
88 {
…
96 /* If the work is "lazy", handle it from next tick if any */
97 if (work->flags & IRQ_WORK_LAZY) {
98 if (llist_add(&work->llnode, this_cpu_ptr(&lazy_list)) &&
99 tick_nohz_tick_stopped())
100 arch_irq_work_raise();
101 } else {
102 if (llist_add(&work->llnode, this_cpu_ptr(&raised_list)))
103 arch_irq_work_raise();
104 }
…
109 }

Quoted 3 lines

so what I asked myself: What happens if the CPU is in NO_HZ mode with the tick off and you invoke arch_irq_work_raise() on an architecture without a special IRQ_WORK interrupt?

NO_HZ full can't run on such architecture. See the arch_irq_work_has_interrupt() test on tick_nohz_init().

Quoted 2 lines

And why you don't this tick_nohz_tick_stopped() check for the raised_list?

Because if the IRQ_WORK_LAZY flag isn't set, we want the interrupt to always fire quickly. We don't want to wait for the next tick unless we have IRQ_WORK_LAZY. Lazy works can wait a bit.

Now if the work is IRQ_WORK_LAZY and the tick is stopped, we need to trigger an IPI because the next tick maybe too far ahead.

Quoted 2 lines

Wouldn't it be a easier for your full NO_HZ code to simply run in the hardirq exit path instead of using irq_work?

Note only the tick wakeup happens from the IPI callback. The tick shutdown always happens from irq exit.

But I'm working on doing the wakeup as well from irq exit.

I plan to piggyback on scheduler events instead of playing with IPIs, although we'll still need them for perf and posix timers at least when those subsystems need periodic tick.

Quoted 1 line

Sebastian