torture: Fix hang during kthread shutdown phase
PATCH 1/3 torture: Fix hang during kthread shutdown phase
From: Joel Fernandes <[email protected]>
[ Upstream commit d52d3a2bf408ff86f3a79560b5cce80efb340239 ]
During shutdown of rcutorture, the shutdown thread in rcu_torture_cleanup() calls torture_cleanup_begin() which sets fullstop to FULLSTOP_RMMOD. This is enough to cause the rcutorture threads for readers and fakewriters to breakout of their main while loop and start shutting down.
Once out of their main loop, they then call torture_kthread_stopping() which in turn waits for kthread_stop() to be called, however rcu_torture_cleanup() has not even called kthread_stop() on those threads yet, it does that a bit later. However, before it gets a chance to do so, torture_kthread_stopping() calls schedule_timeout_interruptible(1) in a tight loop. Tracing confirmed this makes the timer softirq constantly execute timer callbacks, while never returning back to the softirq exit path and is essentially "locked up" because of that. If the softirq preempts the shutdown thread, kthread_stop() may never be called.
This commit improves the situation dramatically, by increasing timeout passed to schedule_timeout_interruptible() 1/20th of a second. This causes the timer softirq to not lock up a CPU and everything works fine. Testing has shown 100 runs of TREE07 passing reliably, which was not the case before because of RCU stalls.
--- kernel/torture.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/torture.c +1 -1
@@ -788,7 +788,7 @@ void torture_kthread_stopping(char *title)VERBOSE_TOROUT_STRING (buf);while (!kthread_should_stop ()) {torture_shutdown_absorb (title); -schedule_timeout_uninterruptible (1 ); +schedule_timeout_uninterruptible (HZ /20 ); } }EXPORT_SYMBOL_GPL (torture_kthread_stopping);
Signature · 1 line
2.41.0.640.ga95def55d0-goog
PATCH 2/3 tick: Detect and fix jiffies update stall
From: Frederic Weisbecker <[email protected]>
[ Upstream commit a1ff03cd6fb9c501fff63a4a2bface9adcfa81cd ]
On some rare cases, the timekeeper CPU may be delaying its jiffies update duty for a while. Known causes include:
* The timekeeper is waiting on stop_machine in a MULTI_STOP_DISABLE_IRQ or MULTI_STOP_RUN state. Disabled interrupts prevent from timekeeping updates while waiting for the target CPU to complete its stop_machine() callback.
* The timekeeper vcpu has VMEXIT'ed for a long while due to some overload on the host.
Detect and fix these situations with emergency timekeeping catchups.
--- kernel/time/tick-sched.c | 17 +++++++++++++++++ kernel/time/tick-sched.h | 4 ++++ 2 files changed, 21 insertions(+)
kernel/time/tick-sched.c +17 -0
@@ -148,6 +148,8 @@ static ktime_t tick_init_jiffy_update(void)return period; } +#define MAX_STALLED_JIFFIES 5 +static void tick_sched_do_timer (struct tick_sched *ts,ktime_t now) {int cpu =smp_processor_id ();
@@ -175,6 +177,21 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now)if (tick_do_timer_cpu == cpu)tick_do_update_jiffies64 (now); +/* +* If jiffies update stalled for too long (timekeeper in stop_machine() +* or VMEXIT'ed for several msecs), force an update. +*/ +if (ts->last_tick_jiffies != jiffies) { + ts->stalled_jiffies =0 ; + ts->last_tick_jiffies =READ_ONCE (jiffies); + }else { +if (++ts->stalled_jiffies ==MAX_STALLED_JIFFIES ) { +tick_do_update_jiffies64 (now); + ts->stalled_jiffies =0 ; + ts->last_tick_jiffies =READ_ONCE (jiffies); + } + } +if (ts->inidle) ts->got_idle_tick =1 ; }
kernel/time/tick-sched.h +4 -0
@@ -49,6 +49,8 @@ enum tick_nohz_mode {* @timer_expires_base: Base time clock monotonic for @timer_expires * @next_timer: Expiry time of next expiring timer for debugging purpose only * @tick_dep_mask: Tick dependency mask - is set, if someone needs the tick +* @last_tick_jiffies: Value of jiffies seen on last tick +* @stalled_jiffies: Number of stalled jiffies detected across ticks */ struct tick_sched {struct hrtimer sched_timer;
@@ -77,6 +79,8 @@ struct tick_sched {u64 next_timer;ktime_t idle_expires;atomic_t tick_dep_mask; +unsigned long last_tick_jiffies; +unsigned int stalled_jiffies; };extern struct tick_sched *tick_get_tick_sched (int cpu);
Signature · 1 line
2.41.0.640.ga95def55d0-goog
PATCH 3/3 timers/nohz: Switch to ONESHOT_STOPPED in the low-res handler when the tick is stopped
In reply to Joel Fernandes (Google)From: Nicholas Piggin <[email protected]>
[ Upstream commit 62c1256d544747b38e77ca9b5bfe3a26f9592576 ]
When tick_nohz_stop_tick() stops the tick and high resolution timers are disabled, then the clock event device is not put into ONESHOT_STOPPED mode. This can lead to spurious timer interrupts with some clock event device drivers that don't shut down entirely after firing.
Eliminate these by putting the device into ONESHOT_STOPPED mode at points where it is not being reprogrammed. When there are no timers active, then tick_program_event() with KTIME_MAX can be used to stop the device. When there is a timer active, the device can be stopped at the next tick (any new timer added by timers will reprogram the tick).
--- kernel/time/tick-sched.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
kernel/time/tick-sched.c +10 -2
@@ -884,6 +884,8 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu)if (unlikely (expires ==KTIME_MAX )) {if (ts->nohz_mode ==NOHZ_MODE_HIGHRES )hrtimer_cancel (&ts->sched_timer); +else +tick_program_event (KTIME_MAX ,1 );return ; }
@@ -1274,9 +1276,15 @@ static void tick_nohz_handler(struct clock_event_device *dev)tick_sched_do_timer (ts, now);tick_sched_handle (ts, regs); -/* No need to reprogram if we are running tickless */ -if (unlikely (ts->tick_stopped)) +if (unlikely (ts->tick_stopped)) { +/* +* The clockevent device is not reprogrammed, so change the +* clock event device to ONESHOT_STOPPED to avoid spurious +* interrupts on devices which might not be truly one shot. +*/ +tick_program_event (KTIME_MAX ,1 );return ; + }hrtimer_forward (&ts->sched_timer, now,TICK_NSEC );tick_program_event (hrtimer_get_expires (&ts->sched_timer),1 );
Signature · 1 line
2.41.0.640.ga95def55d0-goog
Greg KH
maintainerQuoted 54 lines · Joel Fernandes (Google)
From: Joel Fernandes <[email protected]>
[ Upstream commit
d52d3a2bf408ff86f3a79560b5cce80efb340239]During shutdown of rcutorture, the shutdown thread in
rcu_torture_cleanup()callstorture_cleanup_begin()which sets fullstop toFULLSTOP_RMMOD. This is enough to cause the rcutorture threads for readers and fakewriters to breakout of their main while loop and start shutting down.Once out of their main loop, they then call
torture_kthread_stopping()which in turn waits forkthread_stop()to be called, howeverrcu_torture_cleanup()has not even calledkthread_stop()on those threads yet, it does that a bit later. However, before it gets a chance to do so,torture_kthread_stopping()callsschedule_timeout_interruptible(1)in a tight loop. Tracing confirmed this makes the timer softirq constantly execute timer callbacks, while never returning back to the softirq exit path and is essentially "locked up" because of that. If the softirq preempts the shutdown thread,kthread_stop()may never be called.This commit improves the situation dramatically, by increasing timeout passed to
schedule_timeout_interruptible()1/20th of a second. This causes the timer softirq to not lock up a CPU and everything works fine. Testing has shown 100 runs of TREE07 passing reliably, which was not the case before because of RCU stalls.Cc Paul McKenney <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Zhouyi Zhou <[email protected]>Cc <[email protected]> # 6.0.xSigned-off-by Joel Fernandes (Google) <[email protected]>Reviewed-by Davidlohr Bueso <[email protected]>Tested-by Zhouyi Zhou <[email protected]>--- kernel/torture.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/torture.c +1
-1@@ -788,7 +788,7 @@ void torture_kthread_stopping(char *title)VERBOSE_TOROUT_STRING (buf);while (!kthread_should_stop ()) {torture_shutdown_absorb (title); -schedule_timeout_uninterruptible (1 ); +schedule_timeout_uninterruptible (HZ /20 ); } }EXPORT_SYMBOL_GPL (torture_kthread_stopping);Signature · 1 line
2.41.0.640.ga95def55d0-goog
All now queued up, thanks.
greg k-h