tick/nohz: Requeue kicks flushed outside hardirq context
Subsystems (2)
PATCH 1/1 tick/nohz: Requeue kicks flushed outside hardirq context
A remote nohz kick can be flushed from task context after idle exit has checked tick dependencies. If polling suppressed the IPI, the empty callback is consumed without an IRQ exit to reevaluate the tick. A newly set dependency can then leave the non-idle tick stopped until another interrupt arrives.
Requeue the work locally when it runs outside hardirq context on an online CPU. This raises the IRQ needed for the existing tick reevaluation path. Skip offline CPUs whose pending work is being drained during teardown.
--- kernel/time/tick-sched.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
kernel/time/tick-sched.c +6 -1
@@ -402,7 +402,12 @@ static bool can_stop_full_tick(int cpu, struct tick_sched *ts)static void nohz_full_kick_func (struct irq_work *work) { -/* Empty, the tick restart happens on tick_nohz_irq_exit() */ +/* +* Remote irq_work may be flushed from task context. Force an IRQ +* exit to reevaluate the tick unless this CPU is going offline. +*/ +if (!in_hardirq () &&cpu_online (smp_processor_id ())) +irq_work_queue (work); } static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) =
base-commit: 70dd4ee72eb2d5793c62e71c14e3aba78e5a4e56
Signature · 1 line
2.47.0
Frederic Weisbecker
maintainerQuoted 25 lines · Peng Wang
A remote nohz kick can be flushed from task context after idle exit has checked tick dependencies. If polling suppressed the IPI, the empty callback is consumed without an IRQ exit to reevaluate the tick. A newly set dependency can then leave the non-idle tick stopped until another interrupt arrives.
Requeue the work locally when it runs outside hardirq context on an online CPU. This raises the IRQ needed for the existing tick reevaluation path. Skip offline CPUs whose pending work is being drained during teardown.
Fixes 4b44a21dd640 ("irq_work, smp: Allow irq_work on call_single_queue")Fixes a5183862e76f ("tick/nohz: Conditionally restart tick on idle exit")Co-developed-by He Rongguang <[email protected]>Signed-off-by He Rongguang <[email protected]>Signed-off-by Peng Wang <[email protected]>--- kernel/time/tick-sched.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/time/tick-sched.c +0
-0
kernel/time/tick-sched.c static bool can_stop_full_tick(int cpu, struct tick_sched *ts) · line 410
Quoted 10 lines
kernel/time/tick-sched.c +6
-1@@ -402,7 +402,12 @@ static bool can_stop_full_tick(int cpu, struct tick_sched *ts)static void nohz_full_kick_func (struct irq_work *work) { -/* Empty, the tick restart happens on tick_nohz_irq_exit() */ +/* +* Remote irq_work may be flushed from task context. Force an IRQ +* exit to reevaluate the tick unless this CPU is going offline. +*/ +if (!in_hardirq () &&cpu_online (smp_processor_id ())) +irq_work_queue (work);
Good catch but instead of requeuing, why not restarting the tick directly from here instead of from IRQ exit?
Thanks.
Quoted 8 lines
kernel/time/tick-sched.c +0
-0} static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) =base-commit:
70dd4ee72eb2d5793c62e71c14e3aba78e5a4e56Signature · 1 line
2.47.0
Signature · 2 lines
Frederic Weisbecker SUSE Labs
PATCH v2 1/1 tick/nohz: Reevaluate the tick for kicks flushed outside hardirq
The nohz full kick relies on IRQ exit to reevaluate the tick. However, a remote kick can also be flushed from task context without an IRQ exit guaranteed to follow. If a tick dependency is set or an earlier timer is queued after the idle-exit reevaluation, the required tick update can be delayed until another interrupt.
Reevaluate the tick directly from the callback outside hardirq context. Keep the existing IRQ-exit handling for hardirq callbacks. The offline flush remains harmless because tick_sched_timer_dying() has already cleared TS_FLAG_NOHZ.
---
Changes in v2:
- Reevaluate the tick directly outside hardirq instead of requeueing, as suggested by Frederic.
- Use the full update for both dependencies and timer deadlines. Keep IRQ-exit handling for hardirq callbacks and the existing NOHZ guard for CPU teardown.
kernel/time/tick-sched.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/time/tick-sched.c +5 -1
@@ -400,9 +400,13 @@ static bool can_stop_full_tick(int cpu, struct tick_sched *ts)return true; } +static void tick_nohz_full_update_tick (struct tick_sched *ts); +static void nohz_full_kick_func (struct irq_work *work) { -/* Empty, the tick restart happens on tick_nohz_irq_exit() */ +/* Task-context flushes may not be followed by an IRQ exit. */ +if (!in_hardirq ()) +tick_nohz_full_update_tick (this_cpu_ptr (&tick_cpu_sched)); } static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) =
base-commit: 70dd4ee72eb2d5793c62e71c14e3aba78e5a4e56
Signature · 1 line
2.47.3
Frederic Weisbecker
maintainerRe: [PATCH v2] tick/nohz: Reevaluate the tick for kicks flushed outside hardirq
Quoted 36 lines · Peng Wang
The nohz full kick relies on IRQ exit to reevaluate the tick. However, a remote kick can also be flushed from task context without an IRQ exit guaranteed to follow. If a tick dependency is set or an earlier timer is queued after the idle-exit reevaluation, the required tick update can be delayed until another interrupt.
Reevaluate the tick directly from the callback outside hardirq context. Keep the existing IRQ-exit handling for hardirq callbacks. The offline flush remains harmless because
tick_sched_timer_dying()has already clearedTS_FLAG_NOHZ.Fixes 4b44a21dd640 ("irq_work, smp: Allow irq_work on call_single_queue")Fixes a5183862e76f ("tick/nohz: Conditionally restart tick on idle exit")Suggested-by Frederic Weisbecker <[email protected]>Co-developed-by He Rongguang <[email protected]>Signed-off-by He Rongguang <[email protected]>Signed-off-by Peng Wang <[email protected]>---
Changes in v2:
- Reevaluate the tick directly outside hardirq instead of requeueing, as suggested by Frederic.
- Use the full update for both dependencies and timer deadlines. Keep IRQ-exit handling for hardirq callbacks and the existing NOHZ guard for CPU teardown.kernel/time/tick-sched.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)kernel/time/tick-sched.c +0
-0@@ -400,9 +400,13 @@ static bool can_stop_full_tick(int cpu, struct tick_sched *ts)return true;
kernel/time/tick-sched.c static bool can_stop_full_tick(int cpu, struct tick_sched *ts) · line 409
Quoted 10 lines
kernel/time/tick-sched.c +5
-1} +static void tick_nohz_full_update_tick (struct tick_sched *ts); +static void nohz_full_kick_func (struct irq_work *work) { -/* Empty, the tick restart happens on tick_nohz_irq_exit() */ +/* Task-context flushes may not be followed by an IRQ exit. */ +if (!in_hardirq ()) +tick_nohz_full_update_tick (this_cpu_ptr (&tick_cpu_sched));
Just in case, even though it shouldn't happen, just have the following sanity checks to return early:
if (WARN_ON_ONCE(!tick_nohz_full_cpu(smp_processor_id())))
return;
if (!in_hardirq()) {
if (WARN_ON_ONCE(tick_sched_flag_test(ts, TS_FLAG_INIDLE)))
return;
tick_nohz_full_update_tick(this_cpu_ptr(&tick_cpu_sched));
}Thanks!
Quoted 8 lines
kernel/time/tick-sched.c +0
-0} static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) =base-commit:
70dd4ee72eb2d5793c62e71c14e3aba78e5a4e56Signature · 1 line
2.47.3
Signature · 2 lines
Frederic Weisbecker SUSE Labs
PATCH v3 1/1 tick/nohz: Reevaluate the tick for kicks flushed outside hardirq
The nohz full kick relies on IRQ exit to reevaluate the tick. However, a remote kick can also be flushed from task context without an IRQ exit guaranteed to follow. If a tick dependency is set or an earlier timer is queued after the idle-exit reevaluation, the required tick update can be delayed until another interrupt.
Reevaluate the tick directly from the callback outside hardirq context. Keep the existing IRQ-exit handling for hardirq callbacks. The offline flush remains harmless because tick_sched_timer_dying() has already cleared TS_FLAG_NOHZ.
---
Changes in v3:
- Add WARN_ON_ONCE() checks for a non-nohz_full CPU and for a task-context callback still marked idle, as requested by Frederic.
v2: https://lore.kernel.org/all/[email protected]/
kernel/time/tick-sched.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
kernel/time/tick-sched.c +13 -1
@@ -400,9 +400,21 @@ static bool can_stop_full_tick(int cpu, struct tick_sched *ts)return true; } +static void tick_nohz_full_update_tick (struct tick_sched *ts); +static void nohz_full_kick_func (struct irq_work *work) { -/* Empty, the tick restart happens on tick_nohz_irq_exit() */ +struct tick_sched *ts =this_cpu_ptr (&tick_cpu_sched); + +if (WARN_ON_ONCE (!tick_nohz_full_cpu (smp_processor_id ()))) +return ; + +/* Task-context flushes may not be followed by an IRQ exit. */ +if (!in_hardirq ()) { +if (WARN_ON_ONCE (tick_sched_flag_test (ts,TS_FLAG_INIDLE ))) +return ; +tick_nohz_full_update_tick (ts); + } } static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) =
base-commit: 70dd4ee72eb2d5793c62e71c14e3aba78e5a4e56
Signature · 1 line
2.47.3
Re: [PATCH v3] tick/nohz: Reevaluate the tick for kicks flushed outside hardirq
Quoted 11 lines · Peng Wang
The nohz full kick relies on IRQ exit to reevaluate the tick. However, a remote kick can also be flushed from task context without an IRQ exit guaranteed to follow. If a tick dependency is set or an earlier timer is queued after the idle-exit reevaluation, the required tick update can be delayed until another interrupt.
Reevaluate the tick directly from the callback outside hardirq context. Keep the existing IRQ-exit handling for hardirq callbacks. The offline flush remains harmless because
tick_sched_timer_dying()has already clearedTS_FLAG_NOHZ.
Quoted 6 lines
Fixes 4b44a21dd640 ("irq_work, smp: Allow irq_work on call_single_queue")Fixes a5183862e76f ("tick/nohz: Conditionally restart tick on idle exit")Suggested-by Frederic Weisbecker <[email protected]>Co-developed-by He Rongguang <[email protected]>Signed-off-by He Rongguang <[email protected]>Signed-off-by Peng Wang <[email protected]>
Thanks a lot!
Signature · 2 lines
Frederic Weisbecker SUSE Labs
[tip: timers/urgent] tick/nohz: Reevaluate the tick for kicks flushed outside hardirq
In reply to Peng WangThe following commit has been merged into the timers/urgent branch of tip:
The following commit has been merged into the timers/urgent branch of tip:
Commit-ID: e389ae428847a2e6a3b698b5967df68ccf90361c Gitweb: https://git.kernel.org/tip/e389ae428847a2e6a3b698b5967df68ccf90361c Author: Peng Wang <[email protected]> AuthorDate: Fri, 23 Oct 2026 22:51:00 +08:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Sun, 25 Oct 2026 22:43:19 +01:00
tick/nohz: Reevaluate the tick for kicks flushed outside hardirq
The nohz full kick relies on interrupt exit to reevaluate the tick. However, a remote kick can also be flushed from task context without an interrupt exit guaranteed to follow. If a tick dependency is set or an earlier timer is queued after the idle-exit reevaluation, the required tick update can be delayed until another interrupt.
Reevaluate the tick directly from the callback outside hardirq context. Keep the existing interrupt exit handling for hardirq callbacks. The offline flush remains harmless because tick_sched_timer_dying() has already cleared TS_FLAG_NOHZ.
--- kernel/time/tick-sched.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
kernel/time/tick-sched.c +13 -1
@@ -400,9 +400,21 @@ static bool can_stop_full_tick(int cpu, struct tick_sched *ts)return true; } +static void tick_nohz_full_update_tick (struct tick_sched *ts); +static void nohz_full_kick_func (struct irq_work *work) { -/* Empty, the tick restart happens on tick_nohz_irq_exit() */ +struct tick_sched *ts =this_cpu_ptr (&tick_cpu_sched); + +if (WARN_ON_ONCE (!tick_nohz_full_cpu (smp_processor_id ()))) +return ; + +/* Task-context flushes may not be followed by an IRQ exit. */ +if (!in_hardirq ()) { +if (WARN_ON_ONCE (tick_sched_flag_test (ts,TS_FLAG_INIDLE ))) +return ; +tick_nohz_full_update_tick (ts); + } } static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) =