timers/migration: Fix livelock in tmigr_handle_remote_up()
PATCH 1/1 timers/migration: Fix livelock in tmigr_handle_remote_up()
tmigr_handle_remote_cpu() skips timer_expire_remote() when cpu == smp_processor_id(), assuming the local softirq path already handled this CPU's timers.
This assumption breaks when jiffies advances between run_timer_base(BASE_GLOBAL) and tmigr_handle_remote() in the same softirq invocation - a timer expires after the wheel ran but before the hierarchy snapshot is taken.
The stranded timer is never collected, fetch_next_timer_interrupt_remote() keeps reporting it as expired, and the event is re-queued with expires == now on each iteration. The goto-again loop spins indefinitely.
Fix by calling timer_expire_remote() unconditionally. __run_timer_base() already returns early when there is nothing to expire, making this a no-op in the common case.
---
Questions for maintainers:
1. What was the original rationale for the cpu != smp_processor_id() check? There is no code comment, commit message explanation or anything in the original patch's email discussion as to why timer_expire_remote() is skipped for the local CPU.
2. There seems to be a design tension where a CPU can have timers visible in the migration hierarchy while simultaneously running its own local softirq. Is the expectation that run_timer_base() always drains everything before tmigr_handle_remote() sees it, or should the remote path handle local-CPU timers as a fallback?
kernel/time/timer_migration.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
kernel/time/timer_migration.c +1 -2
@@ -978,8 +978,7 @@ static void tmigr_handle_remote_cpu(unsigned int cpu, u64 now,/* Drop the lock to allow the remote CPU to exit idle */ raw_spin_unlock_irq (&tmc->lock); -if (cpu !=smp_processor_id ()) -timer_expire_remote (cpu); +timer_expire_remote (cpu);/* * Lock ordering needs to be preserved - timer_base locks before tmigr
base-commit: e43ffb69e0438cddd72aaa30898b4dc446f664f8
Signature · 1 line
2.47.3
Quoted 18 lines · Amit Matityahu
tmigr_handle_remote_cpu()skipstimer_expire_remote()when cpu ==smp_processor_id(), assuming the local softirq path already handled this CPU's timers.This assumption breaks when jiffies advances between
run_timer_base(BASE_GLOBAL)andtmigr_handle_remote()in the same softirq invocation - a timer expires after the wheel ran but before the hierarchy snapshot is taken.The stranded timer is never collected,
fetch_next_timer_interrupt_remote()keeps reporting it as expired, and the event is re-queued with expires == now on each iteration. The goto-again loop spins indefinitely.Fix by calling
timer_expire_remote()unconditionally.__run_timer_base()already returns early when there is nothing to expire, making this a no-op in the common case.
Quoted 7 lines
Fixes 7ee988770326 ("timers: Implement the hierarchical pull model")Reported-by Alon Kariv <[email protected]>Cc Jonathan Chocron <[email protected]>Cc Akram Baransi <[email protected]>Cc David Woodhouse <[email protected]>Signed-off-by Amit Matityahu <[email protected]>
That's quite serious indeed!
Quoted 8 lines
---
Questions for maintainers:
1. What was the original rationale for the cpu !=
smp_processor_id()check? There is no code comment, commit message explanation or anything in the original patch's email discussion as to whytimer_expire_remote()is skipped for the local CPU.
The rationale was about assuming that such an expired timerqueue actually reflected a timer that was handled locally already and so it could be safely discarded. So we could spare some locking.
Quoted 6 lines
2. There seems to be a design tension where a CPU can have timers visible in the migration hierarchy while simultaneously running its own local softirq. Is the expectation that
run_timer_base()always drains everything beforetmigr_handle_remote()sees it, or should the remote path handle local-CPU timers as a fallback?
That's not easy to defer all global timers handling to remote expiration because the current CPU may or may not be the migrator.
Quoted 9 lines
kernel/time/timer_migration.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)kernel/time/timer_migration.c +0
-0
kernel/time/timer_migration.c static void tmigr_handle_remote_cpu(unsigned int cpu, u64 now, · line 981
Quoted 6 lines
kernel/time/timer_migration.c +1
-2@@ -978,8 +978,7 @@ static void tmigr_handle_remote_cpu(unsigned int cpu, u64 now,/* Drop the lock to allow the remote CPU to exit idle */ raw_spin_unlock_irq (&tmc->lock); -if (cpu !=smp_processor_id ()) -timer_expire_remote (cpu); +timer_expire_remote (cpu);
Thanks!
Quoted 8 lines
kernel/time/timer_migration.c +0
-0/* * Lock ordering needs to be preserved - timer_base locks before tmigr base-commit:
e43ffb69e0438cddd72aaa30898b4dc446f664f8Signature · 1 line
2.47.3
Signature · 2 lines
Frederic Weisbecker SUSE Labs
Thomas Gleixner
maintainerQuoted 11 lines · Frederic Weisbecker
Quoted 6 lines · Amit Matityahu
Questions for maintainers:
1. What was the original rationale for the cpu !=
smp_processor_id()check? There is no code comment, commit message explanation or anything in the original patch's email discussion as to whytimer_expire_remote()is skipped for the local CPU.The rationale was about assuming that such an expired timerqueue actually reflected a timer that was handled locally already and so it could be safely discarded. So we could spare some locking.
Right, but the assumption would only be valid _if_ the jiffies value which was used in run_timers(GLOBAL) is propagated into the remote handling.
Quoted 5 lines
kernel/time/timer_migration.c static void tmigr_handle_remote_cpu(unsigned int cpu, u64 now, · line 981
Quoted 3 lines
kernel/time/timer_migration.c +1
-2-if (cpu !=smp_processor_id ()) -timer_expire_remote (cpu); +timer_expire_remote (cpu);Reviewed-by Frederic Weisbecker <[email protected]>
I'll add a comment to that for posterity.
The following commit has been merged into the timers/urgent branch of tip:
The following commit has been merged into the timers/urgent branch of tip:
Commit-ID: d486b4934a8e504376b85cdb3766f306d57aff5b Gitweb: https://git.kernel.org/tip/d486b4934a8e504376b85cdb3766f306d57aff5b Author: Amit Matityahu <[email protected]> AuthorDate: Wed, 03 Jun 2026 17:01:39 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 04 Jun 2026 14:35:33 +02:00
timers/migration: Fix livelock in tmigr_handle_remote_up()
tmigr_handle_remote_cpu() skips timer_expire_remote() when cpu == smp_processor_id(), assuming the local softirq path already handled this CPU's timers.
This assumption is wrong because jiffies can advance after the handling of the CPU's global timers in run_timer_base(BASE_GLOBAL) and before tmigr_handle_remote() evaluates the expiry times.
As a consequence a timer which expires after the CPU local timer wheel advanced and becomes expired in the remote handling is ignored and the callback is never invoked and removed from the timer wheel.
What's worse is that fetch_next_timer_interrupt_remote() keeps reporting it as expired, and the event is re-queued with expires == now on each iteration. The goto-again loop spins indefinitely.
Fix this by calling timer_expire_remote() unconditionally. That's minimal overhead for the common case as __run_timer_base() returns immediately if there is nothing to expire in the local wheel.
[ tglx: Amend change log and add a comment ]
--- kernel/time/timer_migration.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
kernel/time/timer_migration.c +6 -2
@@ -978,8 +978,12 @@ static void tmigr_handle_remote_cpu(unsigned int cpu, u64 now,/* Drop the lock to allow the remote CPU to exit idle */ raw_spin_unlock_irq (&tmc->lock); -if (cpu !=smp_processor_id ()) -timer_expire_remote (cpu); +/* +* This can't exclude the local CPU because jiffies might have advanced +* after the timer softirq invoked run_timer_base(BASE_GLOBAL) and the +* point where the jiffies snapshot @jif was taken in tmigr_handle_remote(). +*/ +timer_expire_remote (cpu);/* * Lock ordering needs to be preserved - timer_base locks before tmigr