qemu-timer/mc146818rtc: fix timer issues related to clock changes
cover qemu-timer/mc146818rtc: fix timer issues related to clock changes
If the host clock jumps forward or backward without resetting the mc146818rtc's periodic timer, the periodic timer interrupts will either fire rapidly (consuming host CPU and blocking operation of the VM) until the timer catches up to the new host clock, or will stall until the host clock catches up with the prior date used by the periodic timer.
A "reset notifier" mechanism was introduced a while back that resets the mc146818rtc's periodic timer if the host clock jumps backward while the VM is running. However, the timer is not reset if the host clock jumps forward while the VM is running, or if the clock jumps either forward or backward when loading a snapshot or migration.
In one test case, when loading a 3 week old snapshot, the timer loop consumed CPU and blocked the VM for nearly 90 seconds.
The following patches fix this issue by resetting the timer in these cases.
Paul Donohue (2):
qemu-timer: Call clock reset notifiers on large forward jumps
mc146818rtc: Reset the periodic timer on resume from snapshot
hw/timer/mc146818rtc.c | 6 ++++++ include/qemu/timer.h | 9 +++++++++ qemu-timer.c | 2 +- 3 files changed, 16 insertions(+), 1 deletion(-)
PATCH 1/2 qemu-timer: Call clock reset notifiers on forward jumps
Commit 691a0c9c introduced a mechanism by which QEMU_CLOCK_HOST can notify other parts of the emulator when the host clock has jumped backward. This is used to avoid stalling timers that were scheduled based on the host clock.
However, if the host clock jumps forward, then timers that were scheduled based on the host clock may fire rapidly and cause other problems. For example, the mc146818rtc periodic timer will block execution of the VM and consume host CPU while firing every interrupt for the time period that was skipped by the host clock.
To correct that problem, this commit fires the reset notification if the host clock jumps forward by more than a hard-coded limit. The limit is currently set to a value of 60 seconds, which should be small enough to prevent excessive timer loops, but large enough to avoid frequent resets in idle VMs.
--- include/qemu/timer.h | 9 +++++++++ qemu-timer.c | 2 +- 2 files changed, 10 insertions(+), 1 deletion(-)
include/qemu/timer.h +8 -0
@@ -787,6 +787,14 @@ static inline int64_t get_ticks_per_sec(void)return 1000000000LL ; } +static inline int64_t get_max_clock_jump (void ) +{ +// This should be small enough to prevent excessive interrupts from being +// generated by the RTC on clock jumps, but large enough to avoid frequent +// unnecessary resets in idle VMs. +return 60 *get_ticks_per_sec (); +} +/* * * Low level clock functions * */
qemu-timer.c +1 -1
@@ -573,7 +573,7 @@ int64_t qemu_clock_get_ns(QEMUClockType type) now =get_clock_realtime (); last = clock->last; clock->last = now; -if (now < last) { +if (now < last || now > (last +get_max_clock_jump ())) {notifier_list_notify (&clock->reset_notifiers, &now); }return now;
Signature · 1 line
2.1.4
PATCH 2/2 mc146818rtc: Reset the periodic timer on load
In reply to Paul DonohueWhen loading a VM from a snapshot or migration, clock changes can cause the periodic timer to stall or loop rapidly.
qemu-timer has a reset notifier mechanism that is used to avoid timer stalls or loops if the host clock changes while the VM is running when using QEMU_CLOCK_HOST. However, when loading a snapshot or migration, qemu-timer is initialized and fires the reset notifier before mc146818rtc is initialized and has registered its reset handler. In addition, this mechanism isn't used when using QEMU_CLOCK_REALTIME, which might also change when loading a snapshot or migration.
To correct that problem, this commit resets the periodic timer after loading from a snapshot or migration if the clock has either jumped backward or has jumped forward by more than the clock jump limit that is used by the reset notifier code in qemu-timer.
--- hw/timer/mc146818rtc.c | 6 ++++++ 1 file changed, 6 insertions(+)
hw/timer/mc146818rtc.c +6 -0
@@ -723,6 +723,12 @@ static int rtc_post_load(void *opaque, int version_id)check_update_timer (s); } +uint64_t now =qemu_clock_get_ns (rtc_clock); +if (now < (s->next_periodic_time -get_ticks_per_sec ()) || + now > (s->next_periodic_time +get_max_clock_jump ())) { +periodic_timer_update (s,qemu_clock_get_ns (rtc_clock)); + } +#ifdef TARGET_I386 if (version_id >=2 ) {if (s->lost_tick_policy ==LOST_TICK_POLICY_SLEW ) {
Signature · 1 line
2.1.4
Re: [Qemu-devel] [PATCH 1/2] qemu-timer: Call clock reset notifiers on forward jumps
In reply to Paul Donohueinclude/qemu/timer.h static inline int64_t get_ticks_per_sec(void) · line 794
Quoted 5 lines · Paul Donohue
include/qemu/timer.h +5
-0+static inline int64_t get_max_clock_jump (void ) +{ +// This should be small enough to prevent excessive interrupts from being +// generated by the RTC on clock jumps, but large enough to avoid frequent +// unnecessary resets in idle VMs.
This is not how comments are layed out in QEMU...
include/qemu/timer.h static inline int64_t get_ticks_per_sec(void) · line 800
Quoted 6 lines
include/qemu/timer.h +3
-0+return 60 *get_ticks_per_sec (); +} +/* * * Low level clock functions * */
... and it also looks like your editor has mangled this patch. Nevertheless, I've applied it to my local branch and will send it out for inclusion after some more testing.
Paolo
Quoted 4 lines
qemu-timer.c +0
-0
Re: [Qemu-devel] [PATCH 2/2] mc146818rtc: Reset the periodic timer on load
In reply to Paul DonohueQuoted 15 lines · Paul Donohue
To correct that problem, this commit resets the periodic timer after loading from a snapshot or migration if the clock has either jumped backward or has jumped forward by more than the clock jump limit that is used by the reset notifier code in qemu-timer.
Signed-off-by Paul Donohue <[email protected]>--- hw/timer/mc146818rtc.c | 6 ++++++ 1 file changed, 6 insertions(+)hw/timer/mc146818rtc.c +0
-0
hw/timer/mc146818rtc.c static int rtc_post_load(void *opaque, int version_id) · line 727
Quoted 5 lines
hw/timer/mc146818rtc.c +2
-0@@ -723,6 +723,12 @@ static int rtc_post_load(void *opaque, int version_id)check_update_timer (s); } +uint64_t now =qemu_clock_get_ns (rtc_clock); + if (now < (s->next_periodic_time - get_ticks_per_sec()) ||
What is the reason for the "- get_ticks_per_sec()" adjustment? Can I just remove it?
Paolo
Quoted 1 line
hw/timer/mc146818rtc.c +1
-0+ now > (s->next_periodic_time + get_max_clock_jump())) {
Re: [Qemu-devel] [PATCH 1/2] qemu-timer: Call clock reset notifiers on forward jumps
In reply to Paolo BonziniQuoted 7 lines · Paolo Bonzini
include/qemu/timer.h static inline int64_t get_ticks_per_sec(void) · line 794
Quoted 5 lines · Paul Donohue
include/qemu/timer.h +5
-0+static inline int64_t get_max_clock_jump (void ) +{ +// This should be small enough to prevent excessive interrupts from being +// generated by the RTC on clock jumps, but large enough to avoid frequent +// unnecessary resets in idle VMs. This is not how comments are layed out in QEMU...
Oops. Sorry, I'm not sure how I missed that. Force of habit from other projects, I guess...
Quoted 7 lines
include/qemu/timer.h static inline int64_t get_ticks_per_sec(void) · line 800
Quoted 6 lines
include/qemu/timer.h +3
-0+return 60 *get_ticks_per_sec (); +} +/* * * Low level clock functions * */ ... and it also looks like your editor has mangled this patch.
Ugh, yes. I do remember noticing that, and I thought I fixed it, but apparently not.
I will correct and send an updated patch.
Re: [Qemu-devel] [PATCH 1/2] qemu-timer: Call clock reset notifiers on forward jumps
Quoted 18 lines · Paul Donohue
Quoted 7 lines · Paolo Bonzini
include/qemu/timer.h static inline int64_t get_ticks_per_sec(void) · line 794
Quoted 5 lines · Paul Donohue
include/qemu/timer.h +5
-0+static inline int64_t get_max_clock_jump (void ) +{ +// This should be small enough to prevent excessive interrupts from being +// generated by the RTC on clock jumps, but large enough to avoid frequent +// unnecessary resets in idle VMs. This is not how comments are layed out in QEMU...
Oops. Sorry, I'm not sure how I missed that. Force of habit from other projects, I guess...
Quoted 7 lines
include/qemu/timer.h static inline int64_t get_ticks_per_sec(void) · line 800
Quoted 6 lines
include/qemu/timer.h +3
-0+return 60 *get_ticks_per_sec (); +} +/* * * Low level clock functions * */ ... and it also looks like your editor has mangled this patch.
Quoted 4 lines
Ugh, yes. I do remember noticing that, and I thought I fixed it, but apparently not.
I will correct and send an updated patch.
No problem, I fixed this already.
Paolo
Re: [Qemu-devel] [PATCH 2/2] mc146818rtc: Reset the periodic timer on load
In reply to Paolo BonziniQuoted 11 lines · Paolo Bonzini
Quoted 10 lines · Paul Donohue
To correct that problem, this commit resets the periodic timer after loading from a snapshot or migration if the clock has either jumped backward or has jumped forward by more than the clock jump limit that is used by the reset notifier code in qemu-timer.
hw/timer/mc146818rtc.c +0
-0@@ -723,6 +723,12 @@ static int rtc_post_load(void *opaque, int version_id)check_update_timer (s); }
Quoted 5 lines
hw/timer/mc146818rtc.c static int rtc_post_load(void *opaque, int version_id) · line 727
Quoted 2 lines
hw/timer/mc146818rtc.c +2
-0+uint64_t now =qemu_clock_get_ns (rtc_clock); + if (now < (s->next_periodic_time - get_ticks_per_sec()) ||What is the reason for the "-
get_ticks_per_sec()" adjustment? Can I just remove it?
Short answer: Yes, you can remove it.
Long answer:
In the other (qemu-timer) patch, I thought it was a good idea to try to avoid calling the clock reset notifiers in cases where the clock "jumped" because the VM was idle, and err on the side of not calling the notifiers for real but small host clock changes. At that layer of the abstraction, it is not clear how any particular RTC implementation might use the reset notification, so I didn't want to assume it was safe to trigger the notification too frequently.
That thought process carried over when I patched mc146818rtc.c, and I was trying to avoid calling periodic_timer_update() in any cases where it wasn't strictly needed. At the time when a snapshot is taken, "now" should generally be less than "next_periodic_time" by up to one period worth of time. In the case where the snapshot is loaded when "now" is still less than "next_periodic_time", "now < s->next_periodic_time" would not necessarily indicate that the clock has jumped backward. Calculating the period so we could evaluate
"now < s->next_periodic_time - period" would involve duplicating a lot of the code from periodic_timer_update(), which I thought was a bad idea. So, I simply assumed that the period would always be less than 1 second, and used "- get_ticks_per_sec()" to err on the side of skipping the call to periodic_timer_update().
However, now that I'm looking at it again, calling periodic_timer_update() when "now" is less than "next_periodic_time" by less than one period would have no effect anyway, so the
"- get_ticks_per_sec()" adjustment is unnecessary, and it was silly for me to try to avoid a call to to periodic_timer_update() in that case.
Thanks!