x86/hvm/rtc: preserved guest RTC offset during suspend/resume/migrate
PATCH 1/1 x86/hvm/rtc: preserved guest RTC offset during suspend/resume/migrate
The emulated RTC is synchronized with the PV wallclock; any write to the RTC will update struct domain's 'time_offset_seconds' field and call update_domain_wallclock().
However, the value of 'time_offset_seconds' is not preserved in any save record and indeed, when the RTC save record is loaded, the CMOS values will be updated based on an offset value which may or may not have been set by the toolstack [1]. This may result in making bogus values available to the guest and messing up any calculations done in the call to alarm_timer_update() at the end of rtc_load().
This patch extends the RTC save record to contain an offset value, which will be zero filled on load of an older record. The 'time_offset_secoonds' field in struct domain is also modified into a 'time_offset' struct, containing a 'seconds' field and a boolean 'set' field.
The code in rtc_load() then uses the new value in the save record to update the value of struct domain's 'time_offset.seconds' unless 'time_offset.set' is true, which will only be the case if the toolstack has already performed a XEN_DOMCTL_settimeoffset.
[1] There is currently no way for a toolstack to read the value of 'time_offset_seconds' from struct domain. In the past, any hope of preservation of the value across a guest life-cycle operation was based on relying on qemu-dm to write a value into xenstore whenever the RTC was updated, in response to an IOREQ with type IOREQ_TYPE_TIMEOFFSET being sent by Xen; see:
https://xenbits.xen.org/gitweb/?p=qemu-xen-traditional.git;a=blob;f=i386-dm/helper2.c#l457
but this behaviour was never forward-ported into upstream QEMU, which completely ignores that IOREQ type.
In either case, nothing in xl or libxl ever samples the value of RTC offset from xenstore so any offset adjustment to a non-zero value performed by the guest (which in the case of Windows is highly likely as it normally writes RTC in local time, whereas Xen maintains time in UTC) is completely lost with the de-facto toolstack, and always has been. Instead, PV drivers are relied upon to paper over this gaping hole.
---
Cc: Stefano Stabellini <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Volodymyr Babchuk <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
--- xen/arch/arm/platform_hypercall.c | 2 +- xen/arch/arm/time.c | 3 ++- xen/arch/arm/vtimer.c | 4 ++-- xen/arch/x86/hvm/rtc.c | 12 ++++++++++-- xen/arch/x86/time.c | 3 ++- xen/common/time.c | 6 +++--- xen/include/public/arch-x86/hvm/save.h | 2 ++ xen/include/xen/sched.h | 5 ++++- 8 files changed, 26 insertions(+), 11 deletions(-)
xen/arch/arm/platform_hypercall.c +1 -1
@@ -53,7 +53,7 @@ long do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op)if (likely (!op->u.settime64.mbz) )do_settime (op->u.settime64.secs, op->u.settime64.nsecs, - op->u.settime64.system_time +SECONDS (d->time_offset_seconds)); + op->u.settime64.system_time +SECONDS (d->time_offset.seconds));else ret = -EINVAL ;break ;
xen/arch/arm/time.c +2 -1
@@ -353,7 +353,8 @@ void update_vcpu_system_time(struct vcpu *v)void domain_set_time_offset (struct domain *d,int64_t time_offset_seconds) { - d->time_offset_seconds = time_offset_seconds; + d->time_offset.seconds = time_offset_seconds; + d->time_offset.set = true;/* XXX update guest visible wallclock time */ }
xen/arch/arm/vtimer.c +2 -2
@@ -64,8 +64,8 @@ int domain_vtimer_init(struct domain *d, struct xen_arch_domainconfig *config) { d->arch.phys_timer_base.offset =NOW (); d->arch.virt_timer_base.offset =READ_SYSREG64 (CNTPCT_EL0 ); - d->time_offset_seconds =ticks_to_ns (d->arch.virt_timer_base.offset - boot_count); -do_div (d->time_offset_seconds,1000000000 ); + d->time_offset.seconds =ticks_to_ns (d->arch.virt_timer_base.offset - boot_count); +do_div (d->time_offset.seconds,1000000000 ); config->clock_frequency = timer_dt_clock_frequency;
xen/arch/x86/hvm/rtc.c +10 -2
@@ -594,7 +594,7 @@ static void rtc_set_time(RTCState *s)/* We use the guest's setting of the RTC to define the local-time * offset for this domain. */ - d->time_offset_seconds += (after - before); + d->time_offset.seconds += (after - before);update_domain_wallclock_time (d);/* Also tell qemu-dm about it so it will be remembered for next boot. */ send_timeoffset_req (after - before);
@@ -747,8 +747,10 @@ static int rtc_save(struct vcpu *v, hvm_domain_context_t *h)return 0 ;spin_lock (&s->lock); + s->hw.rtc_offset = d->time_offset.seconds; rc =hvm_save_entry (RTC ,0 , h, &s->hw);spin_unlock (&s->lock); +return rc; }
@@ -763,7 +765,7 @@ static int rtc_load(struct domain *d, hvm_domain_context_t *h)spin_lock (&s->lock);/* Restore the registers */ -if (hvm_load_entry (RTC , h, &s->hw) !=0 ) +if (hvm_load_entry_zeroextend (RTC , h, &s->hw) !=0 ) {spin_unlock (&s->lock);return -EINVAL ;
@@ -771,6 +773,12 @@ static int rtc_load(struct domain *d, hvm_domain_context_t *h)/* Reset the wall-clock time. In normal running, this runs with host * time, so let's keep doing that. */ +if ( !d->time_offset.set ) + { + d->time_offset.seconds = s->hw.rtc_offset; +update_domain_wallclock_time (d); + } + s->current_tm =gmtime (get_localtime (d));rtc_copy_date (s);
xen/arch/x86/time.c +2 -1
@@ -1190,7 +1190,8 @@ static void update_domain_rtc(void)void domain_set_time_offset (struct domain *d,int64_t time_offset_seconds) { - d->time_offset_seconds = time_offset_seconds; + d->time_offset.seconds = time_offset_seconds; + d->time_offset.set = true;if (is_hvm_domain (d) )rtc_update_clock (d);update_domain_wallclock_time (d);
xen/common/time.c +3 -3
@@ -105,7 +105,7 @@ void update_domain_wallclock_time(struct domain *d) *wc_version =version_update_begin (*wc_version);smp_wmb (); - sec = wc_sec + d->time_offset_seconds; + sec = wc_sec + d->time_offset.seconds;shared_info (d, wc_sec) = sec;shared_info (d, wc_nsec) = wc_nsec;#ifdef CONFIG_X86
@@ -148,13 +148,13 @@ void do_settime(u64 secs, unsigned int nsecs, u64 system_time_base)unsigned long get_localtime (struct domain *d) {return wc_sec + (wc_nsec +NOW ()) /1000000000ULL - + d->time_offset_seconds; + + d->time_offset.seconds; }/* Return microsecs after 00:00:00 localtime, 1 January, 1970. */ uint64_t get_localtime_us (struct domain *d) { -return (SECONDS (wc_sec + d->time_offset_seconds) + wc_nsec +NOW ()) +return (SECONDS (wc_sec + d->time_offset.seconds) + wc_nsec +NOW ()) /1000UL ; }
xen/include/public/arch-x86/hvm/save.h +2 -0
@@ -500,6 +500,8 @@ struct hvm_hw_rtc {/* Index register for 2-part operations */ uint8_t cmos_index;uint8_t pad0; +/* RTC offset from host time */ +int64_t rtc_offset; }; DECLARE_HVM_SAVE_TYPE(RTC, 11, struct hvm_hw_rtc);
xen/include/xen/sched.h +4 -1
@@ -406,7 +406,10 @@ struct domain/* Domain is paused by controller software? */ int controller_pause_count; -int64_t time_offset_seconds; +struct { +int64_t seconds; +bool set; + } time_offset;#ifdef CONFIG_HAS_PCI struct list_head pdev_list;
Signature · 7 lines
2.20.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 40 lines · Paul Durrant
The emulated RTC is synchronized with the PV wallclock; any write to the RTC will update struct domain's '
time_offset_seconds' field and callupdate_domain_wallclock().However, the value of '
time_offset_seconds' is not preserved in any save record and indeed, when the RTC save record is loaded, the CMOS values will be updated based on an offset value which may or may not have been set by the toolstack [1]. This may result in making bogus values available to the guest and messing up any calculations done in the call toalarm_timer_update()at the end ofrtc_load().This patch extends the RTC save record to contain an offset value, which will be zero filled on load of an older record. The '
time_offset_secoonds' field in struct domain is also modified into a 'time_offset' struct, containing a 'seconds' field and a boolean 'set' field.The code in
rtc_load()then uses the new value in the save record to update the value of struct domain's 'time_offset.seconds' unless 'time_offset.set' is true, which will only be the case if the toolstack has already performed aXEN_DOMCTL_settimeoffset.[1] There is currently no way for a toolstack to read the value of '
time_offset_seconds' from struct domain. In the past, any hope of preservation of the value across a guest life-cycle operation was based on relying on qemu-dm to write a value into xenstore whenever the RTC was updated, in response to an IOREQ with typeIOREQ_TYPE_TIMEOFFSETbeing sent by Xen; see:https://xenbits.xen.org/gitweb/?p=qemu-xen-traditional.git;a=blob;f=i386-dm/helper2.c#l457
but this behaviour was never forward-ported into upstream QEMU, which completely ignores that IOREQ type.
In either case, nothing in xl or libxl ever samples the value of RTC offset from xenstore so any offset adjustment to a non-zero value performed by the guest (which in the case of Windows is highly likely as it normally writes RTC in local time, whereas Xen maintains time in UTC) is completely lost with the de-facto toolstack, and always has been. Instead, PV drivers are relied upon to paper over this gaping hole.
Quoted 1 line
Signed-off-by Paul Durrant <[email protected]>
with one remark:
xen/arch/x86/hvm/rtc.c static int rtc_load(struct domain *d, hvm_domain_context_t *h) · line 780
Quoted 9 lines
xen/arch/x86/hvm/rtc.c +5
-0@@ -771,6 +773,12 @@ static int rtc_load(struct domain *d, hvm_domain_context_t *h)/* Reset the wall-clock time. In normal running, this runs with host * time, so let's keep doing that. */ +if ( !d->time_offset.set ) + { + d->time_offset.seconds = s->hw.rtc_offset; +update_domain_wallclock_time (d); + }
It's not really clear to me which of the possible behaviors is the better one - overriding a tool stack set value with what the save record says, or (like you do) the other way around.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 60 lines
-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 19 December 2019 11:30
To: Durrant, Paul <[email protected]>
Cc: [email protected]; Stefano Stabellini <[email protected]>; Julien Grall <[email protected]>; Volodymyr Babchuk <[email protected]>; Andrew Cooper <[email protected]>; George Dunlap <[email protected]>; Ian Jackson <[email protected]>; Konrad Rzeszutek Wilk <[email protected]>; Wei Liu <[email protected]>; Roger Pau Monné <[email protected]>
Subject: Re: [PATCH] x86/hvm/rtc: preserved guest RTC offset during suspend/resume/migrateQuoted 8 lines · Paul Durrant
The emulated RTC is synchronized with the PV wallclock; any write to the RTC will update struct domain's '
time_offset_seconds' field and callupdate_domain_wallclock().However, the value of '
time_offset_seconds' is not preserved in any save record and indeed, when the RTC save record is loaded, the CMOS values will be updated based on an offset value which may or may not have been set by the toolstack [1]. This may result in making bogus valuesavailable
Quoted 5 lines
to the guest and messing up any calculations done in the call to
alarm_timer_update()at the end ofrtc_load().This patch extends the RTC save record to contain an offset value, which will be zero filled on load of an older record. The
'
time_offset_secoonds'Quoted 6 lines
field in struct domain is also modified into a '
time_offset' struct, containing a 'seconds' field and a boolean 'set' field.The code in
rtc_load()then uses the new value in the save record to update the value of struct domain's 'time_offset.seconds' unless 'time_offset.set' is true, which will only be the case if the toolstackhas
Quoted 5 lines
already performed a
XEN_DOMCTL_settimeoffset.[1] There is currently no way for a toolstack to read the value of '
time_offset_seconds' from struct domain. In the past, any hope of preservation of the value across a guest life-cycle operation wasbased
Quoted 1 line
on relying on qemu-dm to write a value into xenstore whenever the
RTC
Quoted 4 lines
was updated, in response to an IOREQ with type
IOREQ_TYPE_TIMEOFFSETbeing sent by Xen; see:traditional.git;a=blob;f=i386-dm/helper2.c#l457
Quoted 2 lines
but this behaviour was never forward-ported into upstream QEMU,
which
Quoted 3 lines
completely ignores that IOREQ type.
In either case, nothing in xl or libxl ever samples the value of RTC offset from xenstore so any offset adjustment to a non-zerovalue
Quoted 1 line
performed by the guest (which in the case of Windows is highly
likely
Quoted 1 line
as it normally writes RTC in local time, whereas Xen maintains time
Quoted 8 lines
in
Quoted 5 lines
UTC) is completely lost with the de-facto toolstack, and always has been. Instead, PV drivers are relied upon to paper over this gaping hole.
Signed-off-by Paul Durrant <[email protected]>Reviewed-by Jan Beulich <[email protected]>
Thanks.
Quoted 14 lines
with one remark:
Review static int rtc_load(struct domain *d,
Quoted 1 line
(quoted diff) +0
-0@@ -771,6 +773,12 @@ static int rtc_load(struct domain *d,
hvm_domain_context_t*h)Review static int rtc_load(struct domain *d, · line 773
Quoted 2 lines
(quoted diff) +0
-0/* Reset the wall-clock time. In normal running, this runs withhost
xen/arch/x86/hvm/rtc.c static int rtc_load(struct domain *d, hvm_domain_context_t *h) · line 780
Quoted 6 lines
xen/arch/x86/hvm/rtc.c +5
-0* time, so let's keep doing that. */ +if ( !d->time_offset.set ) + { + d->time_offset.seconds = s->hw.rtc_offset; +update_domain_wallclock_time (d); + }
Quoted 3 lines
It's not really clear to me which of the possible behaviors is the better one - overriding a tool stack set value with what the save record says, or (like you do) the other way around.
It's not clear to me either, which is why I erred on the side of caution. I didn't want to break any out-of-tree toolstacks that might be overriding the offset early in the restore phase from a value acquired via QEMU hackery. I guess the boolean could be dispensed with if we retire the IOREQ and silently ignore the DOMCTL for HVM guests (so overrides from the aforementioned toolstacks simply have no effect).
Paul
Quoted 2 lines
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Hi Paul,
Quoted 52 lines · Paul Durrant
The emulated RTC is synchronized with the PV wallclock; any write to the RTC will update struct domain's '
time_offset_seconds' field and callupdate_domain_wallclock().However, the value of '
time_offset_seconds' is not preserved in any save record and indeed, when the RTC save record is loaded, the CMOS values will be updated based on an offset value which may or may not have been set by the toolstack [1]. This may result in making bogus values available to the guest and messing up any calculations done in the call toalarm_timer_update()at the end ofrtc_load().This patch extends the RTC save record to contain an offset value, which will be zero filled on load of an older record. The '
time_offset_secoonds' field in struct domain is also modified into a 'time_offset' struct, containing a 'seconds' field and a boolean 'set' field.The code in
rtc_load()then uses the new value in the save record to update the value of struct domain's 'time_offset.seconds' unless 'time_offset.set' is true, which will only be the case if the toolstack has already performed aXEN_DOMCTL_settimeoffset.[1] There is currently no way for a toolstack to read the value of '
time_offset_seconds' from struct domain. In the past, any hope of preservation of the value across a guest life-cycle operation was based on relying on qemu-dm to write a value into xenstore whenever the RTC was updated, in response to an IOREQ with typeIOREQ_TYPE_TIMEOFFSETbeing sent by Xen; see:https://xenbits.xen.org/gitweb/?p=qemu-xen-traditional.git;a=blob;f=i386-dm/helper2.c#l457
but this behaviour was never forward-ported into upstream QEMU, which completely ignores that IOREQ type.
In either case, nothing in xl or libxl ever samples the value of RTC offset from xenstore so any offset adjustment to a non-zero value performed by the guest (which in the case of Windows is highly likely as it normally writes RTC in local time, whereas Xen maintains time in UTC) is completely lost with the de-facto toolstack, and always has been. Instead, PV drivers are relied upon to paper over this gaping hole.Signed-off-by Paul Durrant <[email protected]>---
Cc: Stefano Stabellini <[email protected]>
Cc: Julien Grall <[email protected]>
Cc: Volodymyr Babchuk <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: George Dunlap <[email protected]>
Cc: Ian Jackson <[email protected]>
Cc: Jan Beulich <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Quoted 4 lines
--- xen/arch/arm/platform_hypercall.c | 2 +- xen/arch/arm/time.c | 3 ++- xen/arch/arm/vtimer.c | 4 ++--
The Arm changes looks mostly mechanical:
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel