allow userspace to adjust kvmclock offset
PATCH 1/1 allow userspace to adjust kvmclock offset
When we migrate a kvm guest that uses pvclock between two hosts, we may suffer a large skew. This is because there can be significant differences between the monotonic clock of the hosts involved. When a new host with a much larger monotonic time starts running the guest, the view of time will be significantly impacted.
Situation is much worse when we do the opposite, and migrate to a host with a smaller monotonic clock.
This new proposed ioctl will allow userspace to inform us what is the monotonic clock value in the source host, so we can keep the time skew short, and more importantly, never goes backwards.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 19 ++++++++++++++++++- include/linux/kvm.h | 1 + 3 files changed, 20 insertions(+), 1 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -410,6 +410,7 @@ struct kvm_arch{unsigned long irq_sources_bitmap;u64 vm_init_tsc; +s64 kvmclock_offset; };struct kvm_vm_stat {
arch/x86/kvm/x86.c +18 -1
@@ -699,7 +699,8 @@ static void kvm_write_guest_time(struct kvm_vcpu *v)/* With all the info we got, fill in the values */ vcpu->hv_clock.system_time = ts.tv_nsec + - (NSEC_PER_SEC * (u64 )ts.tv_sec); + (NSEC_PER_SEC * (u64 )ts.tv_sec) + v->kvm->arch.kvmclock_offset; +/* * The interface expects us to write an even number signaling that the * update is finished. Since the guest won't see the intermediate
@@ -2437,6 +2438,22 @@ long kvm_arch_vm_ioctl(struct file *filp, r =0 ;break ; } +case KVM_ADJUST_CLOCK : { +struct timespec now; +u64 now_ns, user_ns; +long delta; + + r = -EFAULT ; +if (copy_from_user (&user_ns, argp,sizeof (user_ns))) + goto out; + + r =0 ; +ktime_get_ts (&now); + now_ns =timespec_to_ns (&now) + kvm->arch.kvmclock_offset; + delta = user_ns - now_ns; + kvm->arch.kvmclock_offset = delta; +break ; + }default : ; }
include/linux/kvm.h +1 -0
@@ -546,6 +546,7 @@ struct kvm_irqfd {#define KVM_CREATE_PIT2 _IOW(KVMIO, 0x77, struct kvm_pit_config)#define KVM_SET_BOOT_CPU_ID _IO(KVMIO, 0x78)#define KVM_IOEVENTFD _IOW(KVMIO, 0x79, struct kvm_ioeventfd) +#define KVM_ADJUST_CLOCK _IOW(KVMIO, 0x7a, __u64)/* * ioctls for vcpu fds
Signature · 1 line
1.6.2.2
Quoted 20 lines · Glauber Costa
When we migrate a kvm guest that uses pvclock between two hosts, we may suffer a large skew. This is because there can be significant differences between the monotonic clock of the hosts involved. When a new host with a much larger monotonic time starts running the guest, the view of time will be significantly impacted.
Situation is much worse when we do the opposite, and migrate to a host with a smaller monotonic clock.
This new proposed ioctl will allow userspace to inform us what is the monotonic clock value in the source host, so we can keep the time skew short, and more importantly, never goes backwards.
include/linux/kvm.h +0
-0
include/linux/kvm.h struct kvm_irqfd { · line 550
Quoted 5 lines
include/linux/kvm.h +1
-0@@ -546,6 +546,7 @@ struct kvm_irqfd {#define KVM_CREATE_PIT2 _IOW(KVMIO, 0x77, struct kvm_pit_config)#define KVM_SET_BOOT_CPU_ID _IO(KVMIO, 0x78)#define KVM_IOEVENTFD _IOW(KVMIO, 0x79, struct kvm_ioeventfd) +#define KVM_ADJUST_CLOCK _IOW(KVMIO, 0x7a, __u64)
Please change to a struct with some reserved space.
Do we want an absolute or relative adjustment?
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 16 lines · Avi Kivity
Quoted 14 lines · Glauber Costa
When we migrate a kvm guest that uses pvclock between two hosts, we may suffer a large skew. This is because there can be significant differences between the monotonic clock of the hosts involved. When a new host with a much larger monotonic time starts running the guest, the view of time will be significantly impacted.
Situation is much worse when we do the opposite, and migrate to a host with a smaller monotonic clock.
This new proposed ioctl will allow userspace to inform us what is the monotonic clock value in the source host, so we can keep the time skew short, and more importantly, never goes backwards.
Quoted 12 lines
include/linux/kvm.h struct kvm_irqfd { · line 550
Quoted 10 lines
include/linux/kvm.h +1
-0@@ -546,6 +546,7 @@ struct kvm_irqfd {#define KVM_CREATE_PIT2 _IOW(KVMIO, 0x77, struct kvm_pit_config)#define KVM_SET_BOOT_CPU_ID _IO(KVMIO, 0x78)#define KVM_IOEVENTFD _IOW(KVMIO, 0x79, struct kvm_ioeventfd) +#define KVM_ADJUST_CLOCK _IOW(KVMIO, 0x7a, __u64)Please change to a struct with some reserved space.
Ok, can do it.
Quoted 2 lines
Do we want an absolute or relative adjustment?
What exactly do you mean?
Quoted 5 lines · Glauber Costa
Quoted 3 lines
Do we want an absolute or relative adjustment?
What exactly do you mean?
Absolute adjustment: clock = t
Relative adjustment: clock += t
Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.
Quoted 9 lines · Avi Kivity
Quoted 5 lines · Glauber Costa
Quoted 3 lines
Do we want an absolute or relative adjustment?
What exactly do you mean?
Absolute adjustment: clock = t
Relative adjustment: clock += t
The delta is absolute, but the adjustment in the clock is relative.
So we pick the difference between what userspace is passing us and what we currently have, then relatively adds up so we can make sure we won't go back or suffer a too big skew.
Quoted 11 lines · Glauber Costa
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Glauber Costa
Quoted 4 lines
Do we want an absolute or relative adjustment?
What exactly do you mean?
Quoted 9 lines
Quoted 3 lines
Absolute adjustment: clock = t
Relative adjustment: clock += tThe delta is absolute, but the adjustment in the clock is relative.
So we pick the difference between what userspace is passing us and what we currently have, then relatively adds up so we can make sure we won't go back or suffer a too big skew.
The motivation for relative adjustment is when you have a jitter resistant place to gather timing information (like the kernel, which can disable interrupts and preemption), then pass it on to kvm without losing information due to scheduling. For migration there is no such place since it involves two hosts, but it makes sense to support relative adjustments.
Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.
Quoted 22 lines · Avi Kivity
Quoted 11 lines · Glauber Costa
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Glauber Costa
Quoted 4 lines
Do we want an absolute or relative adjustment?
What exactly do you mean?
Quoted 9 lines
Quoted 3 lines
Absolute adjustment: clock = t
Relative adjustment: clock += tThe delta is absolute, but the adjustment in the clock is relative.
So we pick the difference between what userspace is passing us and what we currently have, then relatively adds up so we can make sure we won't go back or suffer a too big skew.
Quoted 6 lines
The motivation for relative adjustment is when you have a jitter resistant place to gather timing information (like the kernel, which can disable interrupts and preemption), then pass it on to kvm without losing information due to scheduling. For migration there is no such place since it involves two hosts, but it makes sense to support relative adjustments.
Since we added the padding you asked for, we could use that bit of information to define whether it will be a relative or absolute adjustment, then. Right now, I don't see the point of implementing a code path that will be completely untested.
I'd leave it this way until someone comes up with a need.
Quoted 8 lines · Glauber Costa
Quoted 8 lines
The motivation for relative adjustment is when you have a jitter resistant place to gather timing information (like the kernel, which can disable interrupts and preemption), then pass it on to kvm without losing information due to scheduling. For migration there is no such place since it involves two hosts, but it makes sense to support relative adjustments.
Quoted 6 lines
Since we added the padding you asked for, we could use that bit of information to define whether it will be a relative or absolute adjustment, then. Right now, I don't see the point of implementing a code path that will be completely untested.
I'd leave it this way until someone comes up with a need.
I agree with that, but padding by itself is insufficient. You also need a flags field.
Signature · 1 line
error compiling committee.c: too many arguments to function