Allow guest to register a secondary vcpu_time_info structure
PATCH 1/1 Allow guest to register a secondary vcpu_time_info structure
Allow a guest to register a second location for the VCPU time info structure for each vcpu. This is intended to allow the guest kernel to map this information into a usermode accessible page, so that usermode can efficiently calculate system time from the TSC without having to make a syscall.
The second vcpu_time_info structure is updated by copy, rather than being a shared page between the guest and Xen. It is not directly updated by copy; instead, Xen preserves and increments the existing version number in place. This allows the guest to also update the version number (useful to indicate vcpu context switches to usermode). This assumes that the guest will only ever update the structure for a given vcpu on that vcpu (as Xen does, so there are never any cross-cpu accesses).
xen/arch/x86/domain.c +18 -0
@@ -964,6 +964,24 @@break ; } +case VCPUOP_register_vcpu_time_memory_area: + { +struct vcpu_register_time_memory_area area; + rc = -EFAULT ; +if (copy_from_guest (&area, arg,1 ) ) +break ; + +if ( !guest_handle_okay (area.addr.h,1 ) ) +break ; + + rc =0 ; + v->time_info_guest = area.addr.h; + +__update_vcpu_system_time (v,1 ); + +break ; + } +case VCPUOP_get_physid: {struct vcpu_get_physid cpu_id;
xen/arch/x86/time.c +34 -2
@@ -22,6 +22,7 @@#include <xen/irq.h> #include <xen/softirq.h> #include <xen/keyhandler.h> +#include <xen/guest_access.h> #include <asm/io.h> #include <asm/msr.h> #include <asm/mpspec.h>
@@ -826,7 +827,31 @@ (*version)++; } -void update_vcpu_system_time (struct vcpu *v) +static void update_guest_time_info (struct vcpu *v,struct vcpu_time_info *u) +{ +struct vcpu_time_info info; +u32 version; + +if (guest_handle_is_null (v->time_info_guest) ) +return ; + +if (__copy_from_guest (&info, v->time_info_guest,1 ) ) +return ; + +/* +* Update the guest copy of the time info. We need to make sure +* we update the guest's version of the version number rather than +* use a verbtim copy of the master one, because the guest may +* update the version for its own purposes. +*/ + version = info.version; + info = *u; + info.version = (version +2 ) & ~1 ; + +__copy_to_guest (v->time_info_guest, &info,1 ); +} + +void __update_vcpu_system_time (struct vcpu *v,int force) {struct cpu_time *t;struct vcpu_time_info *u;
@@ -837,7 +862,7 @@ t = &this_cpu (cpu_time); u = &vcpu_info (v, time); -if ( u->tsc_timestamp == t->local_tsc_stamp ) +if ( !force && u->tsc_timestamp == t->local_tsc_stamp )return ;version_update_begin (&u->version);
@@ -848,6 +873,13 @@ u->tsc_shift = (s8 )t->tsc_scale.shift;version_update_end (&u->version); + +update_guest_time_info (v, u); +} + +void update_vcpu_system_time (struct vcpu *v) +{ +__update_vcpu_system_time (v,0 ); }void update_domain_wallclock_time (struct domain *d)
xen/include/public/vcpu.h +43 -0
49 lines not on this page · raw message
xen/include/xen/sched.h +3 -0
@@ -102,6 +102,9 @@ } runstate_guest; /* guest address */#endif +/* A secondary copy of the vcpu time info */ +XEN_GUEST_HANDLE (vcpu_time_info_t) time_info_guest; +/* last time when vCPU is scheduled out */ uint64_t last_run_time;
xen/include/xen/time.h +1 -0
@@ -54,6 +54,7 @@#define STIME_MAX ((s_time_t)((uint64_t)~0ull>>1))extern void update_vcpu_system_time (struct vcpu *v); +extern void __update_vcpu_system_time (struct vcpu *v,int force);extern void update_domain_wallclock_time (struct domain *d); extern void do_settime(
Quoted 8 lines · Jeremy Fitzhardinge
The second
vcpu_time_infostructure is updated by copy, rather than being a shared page between the guest and Xen. It is not directly updated by copy; instead, Xen preserves and increments the existing version number in place. This allows the guest to also update the version number (useful to indicate vcpu context switches to usermode). This assumes that the guest will only ever update the structure for a given vcpu on that vcpu (as Xen does, so there are never any cross-cpu accesses).
How do you feel about the 32-bit version number? We have space to easily expand it to 64 bits, and now would be the time to do it while we're extending the ABI.
-- Keir
xen/arch/x86/time.c line 839
Quoted 12 lines
Quoted 1 line
Quoted 1 line
Jeremy Fitzhardinge <[email protected]> 06.10.09 02:10 >>>
xen/arch/x86/time.c +10
-1-void update_vcpu_system_time (struct vcpu *v) +static void update_guest_time_info (struct vcpu *v,struct vcpu_time_info *u) +{ +struct vcpu_time_info info; +u32 version; + +if (guest_handle_is_null (v->time_info_guest) ) +return ; + +if (__copy_from_guest (&info, v->time_info_guest,1 ) ) +return ;
I'd suggest using __copy_field_from_guest() here to get just the version member, after copying *u into info.
xen/arch/x86/time.c line 852
Quoted 13 lines
xen/arch/x86/time.c +13
-0+ +/* +* Update the guest copy of the time info. We need to make sure +* we update the guest's version of the version number rather than +* use a verbtim copy of the master one, because the guest may +* update the version for its own purposes. +*/ + version = info.version; + info = *u; + info.version = (version +2 ) & ~1 ; + +__copy_to_guest (v->time_info_guest, &info,1 ); +}
Jan
Quoted 12 lines · Keir Fraser
Quoted 9 lines · Jeremy Fitzhardinge
The second
vcpu_time_infostructure is updated by copy, rather than being a shared page between the guest and Xen. It is not directly updated by copy; instead, Xen preserves and increments the existing version number in place. This allows the guest to also update the version number (useful to indicate vcpu context switches to usermode). This assumes that the guest will only ever update the structure for a given vcpu on that vcpu (as Xen does, so there are never any cross-cpu accesses).
Quoted 4 lines
How do you feel about the 32-bit version number? We have space to easily expand it to 64 bits, and now would be the time to do it while we're extending the ABI.
I think 32 bit (well, 31) is fine:
The version number doesn't increment that often, so I think it would take a long time to wrap. The only failure mode is if the version wraps and gets an aliased value between the "get version" and "re-test version" phases of the algorithm, which is only a few tens of instructions at most. I would think it unlikely the version increments once in that period, let alone 2^31 times. As far as I know, nobody is keeping long-term copies of the time info.
(One unlikely scenario is that you have usermode in the middle of a vsyscall gettimeofday when the domain gets suspended/migrated and it is suspended for some long period of time. But this is OK because the version gets saved along with the rest of the domain and incremented from there. I'm not sure if the same applies to the shared_info version, but that's less of a concern because the kernel will never preempt itself doing a clocksource_read to suspend.)
Aside from that, this isn't really an ABI extension. The registration of the secondary vcpu_time_info is a new ABI, but from there it is identical to the existing pvclock ABI; I use the same code for both the kernel and in the vsyscall page, so making them different would be awkward (of course I could change the kernel to use the "secondary" time values too, but it would still need to support old hypervisors so it wouldn't save anything). KVM is also using this ABI, so I wouldn't want to introduce an arbitrary ABI change.
Also, a >32 bit version field would be hard for 32 bit guests to deal with; it's not easy to either read or update its value in a single instruction. (32 bit Xen too, but I guess we're less concerned about that.)
J
Quoted 17 lines · Jan Beulich
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Jeremy Fitzhardinge <[email protected]> 06.10.09 02:10 >>>
xen/arch/x86/time.c +0
-1-void update_vcpu_system_time (struct vcpu *v)xen/arch/x86/time.c line 839
Quoted 10 lines
xen/arch/x86/time.c +10
-0+static void update_guest_time_info (struct vcpu *v,struct vcpu_time_info *u) +{ +struct vcpu_time_info info; +u32 version; + +if (guest_handle_is_null (v->time_info_guest) ) +return ; + +if (__copy_from_guest (&info, v->time_info_guest,1 ) ) +return ;I'd suggest using
__copy_field_from_guest()here to get just the version member, after copying *u into info.
Good idea. I'll update the patch.
J