Provide HVM guest RTC change notification

This patch restores the capabilities initially provided in changeset 10010. When the RTC code was moved into the hypervisor (a good move), the control plane lost the ability to read the current time offset as well as to receive notification of changes to the RTC time base by a guest. This patch reintroduces these.

Additionally, there is a small window at initialization time in which the time offset may be set but not noticed or used. If xc_domain_set_time_offset is called before the domain is started, the offset won't be noticed until the next second update. An HVM guest could read the RTC in this window and get an unintended result. This patch closes the window.

The patch builds upon the existing change notification mechanism provided by VIRQ_DOM_EXC. I stopped short of renaming the releaseDomain watch to something like domainEvent as it wasn't clear who might be relying upon the existing name. A patch for that would be easy to create, though.

Last, the code introduced into qemu by 10010 is no longer really connected to anything and could be removed. It's found in tools/ioemu/patches/domain-timeoffset and can probably be excised at some suitable point. Currently, it's relatively harmless.

------------------------------------------------------------------------
Ben Thomas                                         Virtual Iron Software
[email protected]                            Tower 1, Floor 2
978-849-1214                                       900 Chelmsford Street
                                                   Lowell, MA 01851

Provide notification and reading of RTC change by HVM guest

If the HVM guest changes the current date/time in the RTC, signal this to the control plan via the DOM_EXC virq. Allow the control plane to obtain the new time via a a new call to read the time offset for a specific domain.

Close initial condition window. If the offset is set before the domain starts, it may not be used until after the first second update. This leaves a small hole in which the wrong values could be read.

diff -r 9646036c745d tools/libxc/xc_domain.c
@@ -180,6 +180,7 @@ int xc_domain_getinfo(int xc_handle,
         info->blocked  = !!(domctl.u.getdomaininfo.flags&XEN_DOMINF_blocked);
         info->running  = !!(domctl.u.getdomaininfo.flags&XEN_DOMINF_running);
         info->hvm      = !!(domctl.u.getdomaininfo.flags&XEN_DOMINF_hvm_guest);
+        info->time_chg = !!(domctl.u.getdomaininfo.flags&XEN_DOMINF_time_chg);
 
         info->shutdown_reason =
             (domctl.u.getdomaininfo.flags>>XEN_DOMINF_shutdownshift) &

@@ -425,6 +426,26 @@ int xc_domain_set_time_offset(int xc_han
     domctl.domain = (domid_t)domid;
     domctl.u.settimeoffset.time_offset_seconds = time_offset_seconds;
     return do_domctl(xc_handle, &domctl);
+}
+
+int xc_domain_get_time_offset(int xc_handle,
+                              uint32_t domid,
+                              int32_t *time_offset_seconds)
+{
+    int rc;
+    DECLARE_DOMCTL;
+
+    if (time_offset_seconds == NULL)
+        return -EINVAL;
+
+    domctl.cmd = XEN_DOMCTL_gettimeoffset;
+    domctl.domain = (domid_t)domid;
+
+    rc = do_domctl(xc_handle, &domctl);
+
+    *time_offset_seconds = domctl.u.gettimeoffset.time_offset_seconds;
+
+    return rc;
 }
 
 int xc_domain_memory_increase_reservation(int xc_handle,

diff -r 9646036c745d tools/libxc/xenctrl.h
@@ -150,7 +150,7 @@ typedef struct xc_dominfo {
     uint32_t      ssidref;
     unsigned int  dying:1, crashed:1, shutdown:1,
                   paused:1, blocked:1, running:1,
-                  hvm:1;
+                  hvm:1, time_chg:1;
     unsigned int  shutdown_reason; /* only meaningful if shutdown==1 */
     unsigned long nr_pages;
     unsigned long shared_info_frame;

@@ -487,6 +487,10 @@ int xc_domain_set_time_offset(int xc_han
 int xc_domain_set_time_offset(int xc_handle,
                               uint32_t domid,
                               int32_t time_offset_seconds);
+
+int xc_domain_get_time_offset(int xc_handle,
+                              uint32_t domid,
+                              int32_t *time_offset_seconds);
 
 int xc_domain_memory_increase_reservation(int xc_handle,
                                           uint32_t domid,

diff -r 9646036c745d tools/xenstore/xenstored_domain.c
@@ -195,6 +195,8 @@ static void domain_cleanup(void)
 				domain->shutdown = 1;
 				notify = 1;
 			}
+			if (dominfo.time_chg)
+				notify = 1;
 			if (!dominfo.dying)
 				continue;
 		}

diff -r 9646036c745d xen/common/domctl.c
@@ -117,6 +117,7 @@ void getdomaininfo(struct domain *d, str
         ((d->domain_flags & DOMF_dying)      ? XEN_DOMINF_dying    : 0) |
         ((d->domain_flags & DOMF_shutdown)   ? XEN_DOMINF_shutdown : 0) |
         ((d->domain_flags & DOMF_ctrl_pause) ? XEN_DOMINF_paused   : 0) |
+        ((d->domain_flags & DOMF_time_chg)   ? XEN_DOMINF_time_chg : 0) |
         d->shutdown_code << XEN_DOMINF_shutdownshift;
 
     if ( is_hvm_domain(d) )

@@ -711,6 +712,25 @@ long do_domctl(XEN_GUEST_HANDLE(xen_domc
     }
     break;
 
+    case XEN_DOMCTL_gettimeoffset:
+    {
+        struct domain *d;
+
+        ret = -ESRCH;
+        d = rcu_lock_domain_by_id(op->domain);
+        if ( d != NULL )
+        {
+            op->u.gettimeoffset.time_offset_seconds = d->time_offset_seconds;
+            ret = 0;
+            if ( copy_to_guest(u_domctl, op, 1) )
+                ret = -EFAULT;
+
+            clear_bit(_DOMF_time_chg, &d->domain_flags);
+            rcu_unlock_domain(d);
+        }
+    }
+    break;
+
     default:
         ret = arch_do_domctl(op, u_domctl);
         break;

diff -r 9646036c745d xen/include/asm-x86/hvm/vpt.h
@@ -93,6 +93,7 @@ typedef struct RTCState {
     struct timer second_timer2;
     struct periodic_time pt;
     int32_t time_offset_seconds;
+    unsigned int accessed:1;
 } RTCState;
 
 #define FREQUENCE_PMTIMER  3579545  /* Timer should run at 3.579545 MHz */

diff -r 9646036c745d xen/include/public/domctl.h
@@ -85,6 +85,10 @@ struct xen_domctl_getdomaininfo {
  /* Domain is currently running.            */
 #define _XEN_DOMINF_running   5
 #define XEN_DOMINF_running    (1U<<_XEN_DOMINF_running)
+/* Domain has a time offset change         */
+#define _XEN_DOMINF_time_chg  6
+#define XEN_DOMINF_time_chg   (1U<<_XEN_DOMINF_time_chg)
+
  /* CPU to which this domain is bound.      */
 #define XEN_DOMINF_cpumask      255
 #define XEN_DOMINF_cpushift       8

@@ -388,6 +392,13 @@ struct xen_domctl_settimeoffset {
 };
 typedef struct xen_domctl_settimeoffset xen_domctl_settimeoffset_t;
 DEFINE_XEN_GUEST_HANDLE(xen_domctl_settimeoffset_t);
+
+#define XEN_DOMCTL_gettimeoffset     37
+struct xen_domctl_gettimeoffset {
+    int32_t  time_offset_seconds; /* applied to domain wallclock time */
+};
+typedef struct xen_domctl_gettimeoffset xen_domctl_gettimeoffset_t;
+DEFINE_XEN_GUEST_HANDLE(xen_domctl_gettimeoffset_t);
 
  
 #define XEN_DOMCTL_gethvmcontext     33

@@ -453,6 +464,7 @@ struct xen_domctl {
         struct xen_domctl_hypercall_init    hypercall_init;
         struct xen_domctl_arch_setup        arch_setup;
         struct xen_domctl_settimeoffset     settimeoffset;
+        struct xen_domctl_gettimeoffset     gettimeoffset;
         struct xen_domctl_real_mode_area    real_mode_area;
         struct xen_domctl_hvmcontext        hvmcontext;
         struct xen_domctl_address_size      address_size;

diff -r 9646036c745d xen/include/xen/sched.h
@@ -474,6 +474,9 @@ extern struct domain *domain_list;
  /* Domain is a compatibility one? */
 #define _DOMF_compat           6
 #define DOMF_compat            (1UL<<_DOMF_compat)
+ /* Domain time offset has changed */
+#define _DOMF_time_chg         7
+#define DOMF_time_chg          (1UL<<_DOMF_time_chg)
 
 static inline int vcpu_runnable(struct vcpu *v)
 {

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

Isn't the "s->time_offset_seconds != s->pt.vcpu->domain->time_offset_seconds" condition an appropriate test whether you need to call rtc_copy_date?

The domain exception virq is already quite overloaded as is and it causes xend to do a quite expensive scan of all the domain information.

I think using an ioreq to qemu would be a more appropriate mechanism to signal that the guest has changed the timeoffset. The ioreq would include the delta by which the offset was changed. qemu could then update the timeoffset stored in xenstore, since we would want to make the changed timeoffset persist across reboots.

christian

It depends upon where you want this test. I didn't want it for every access to the RTC registers, and something is needed to set up the initial conditions. A different approach would have handled it when the RTC is set up, but that appears to be at domain building time and it wasn't clear to me that there was a way to pass the information without changing the domain builder API. Given an existing mechanism to set the RTC time, it seemed an interesting approach to use that and simply handle the initial condition.

Changing the register access routines to unconditionally check for a time offset change seemed too intrusive and at conflict with the per-second change mechanism. But, as long as it all works, I'm open to anything.

The unlikely check for the accessed flag seemed a simple, low-cost way to handle the initialization without unduly influencing other portions of the mechanism.

I completely agree about the persistence, and that's the reason I sent in the patch that resulted in c/s 10010. I'm just trying to get back to that level of capability.

The domain exception seemed like a reasonable approach. It may be overloaded in the "number of things it can signal", but it seems like a low-occurrence set of events. And, the code checking for that type of event easily accesses the additional flag. Last, but not least, using this particular virq completely removes all qemu involvement. Cutting back on qemu changes doesn't seem like a bad side-effect.

Thanks,
-b

------------------------------------------------------------------------
Ben Thomas                                         Virtual Iron Software
[email protected]                            Tower 1, Floor 2
978-849-1214                                       900 Chelmsford Street
                                                   Lowell, MA 01851

Hi,

What's the status of this discussion/patch ? It would be nice to get some closure on this. I know everyone's busy.

My understanding of the current concerns from Christian are (and these are my interpretation of his comments):

- simply use a inequality test in time offsets to trigger setting the new time offset.

My rationale for the as-submitted patch, was that there is an existing per-second mechanism that tries to emulate the set and uip bits and timing. Time changes appear to happen during this per-second mechanism. While it looks simple to just modify the time if the offset is detected to have changed, it would appear that this would negatively impact readers of the RTC and circumvent the work done by the per-second mechanism. More simply, it wasn't clear to me that modifying the time wouldn't break guests unless the per-second mechanism was utilized.

- don't use the VIRQ mechanism for time-offset (RTC write) notifications. Use an ioreq to qemu.

The as-submitted patch used the VIRQ mechanism for a few reasons. By building upon the domain event mechanism/virq, this low frequency event is handled in a fashion quite like other low-frequency domain events such as shutdown and crash. I was looking for a low-cost, low-impact, fire-and-forget means to signal an RTC change.

The ioreq approach is interesting. I'm uncertain about it for the following reasons, which some small additional explanation would help me with. Adding an ioreq for time change seems quite out of character with the other ioreq events. Additionally, qemu processing (and writing a store entry) are likely to be rather expensive. Wouldn't this also cause undue delay in the guest awaiting the completion of the ioreq ?

It is quite possible that I have misunderstood Christian's comments. If so, I'd love to better understand the issues. I really don't care what the final implementation is, other than the usual expectations of robustness, reasonable performance and clarity. Perhaps with a bit more information, this can be cleared up and closed up. I'd like to get back to the capabilities that existed before the RTC code was moved into the hypervisor.

Thanks !
-b

------------------------------------------------------------------------
Ben Thomas                                         Virtual Iron Software
[email protected]                            Tower 1, Floor 2
978-849-1214                                       900 Chelmsford Street
                                                   Lowell, MA 01851