KVM: x86: hyper-v: Fix TSC page update after KVM_SET_CLOCK(0) call
Vitaly Kuznetsov
maintainercover KVM: x86: hyper-v: Fix TSC page update after KVM_SET_CLOCK(0) call
I discovered that after KVM_SET_CLOCK(0) TSC page value in the guest can go through the roof and apparently we have a signedness issue when the update is performed. Fix the issue and add a selftest.
Vitaly Kuznetsov (2):
KVM: x86: hyper-v: Forbid unsigned hv_clock->system_time to go
negative after KVM_REQ_CLOCK_UPDATE
selftests: kvm: Check that TSC page value is small after
KVM_SET_CLOCK(0)
arch/x86/kvm/x86.c | 10 +++++++++- tools/testing/selftests/kvm/x86_64/hyperv_clock.c | 13 +++++++++++-- 2 files changed, 20 insertions(+), 3 deletions(-)
Signature · 1 line
2.30.2
Vitaly Kuznetsov
maintainerPATCH 1/2 KVM: x86: hyper-v: Forbid unsigned hv_clock->system_time to go negative after KVM_REQ_CLOCK_UPDATE
When guest time is reset with KVM_SET_CLOCK(0), it is possible for hv_clock->system_time to become a small negative number. This happens because in KVM_SET_CLOCK handling we set kvm->arch.kvmclock_offset based on get_kvmclock_ns(kvm) but when KVM_REQ_CLOCK_UPDATE is handled, kvm_guest_time_update() does
hv_clock.system_time = ka->master_kernel_ns + v->kvm->arch.kvmclock_offset;
And 'master_kernel_ns' represents the last time when masterclock got updated, it can precede KVM_SET_CLOCK() call. Normally, this is not a problem, the difference is very small, e.g. I'm observing hv_clock.system_time = -70 ns. The issue comes from the fact that 'hv_clock.system_time' is stored as unsigned and 'system_time / 100' in compute_tsc_page_parameters() becomes a very big number.
Forbid 'hv_clock.system_time' to go negative in kvm_guest_time_update(). A similar computation in get_kvmclock_ns() seems fine and doesn't require the quirk.
Alternatively, we could've used 'master_kernel_ns' when computing 'arch.kvmclock_offset' but that would reduce the precision for normal cases a bit. Another solution is to cast 'hv_clock.system_time' to 's64' in compute_tsc_page_parameters() but it seems we also use 'hv_clock.system_time' in trace_kvm_pvclock_update() as unsigned.
--- arch/x86/kvm/x86.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +9 -1
@@ -2742,7 +2742,15 @@ static int kvm_guest_time_update(struct kvm_vcpu *v) } vcpu->hv_clock.tsc_timestamp = tsc_timestamp; - vcpu->hv_clock.system_time = kernel_ns + v->kvm->arch.kvmclock_offset; + +/* +* 'kvmclock_offset' can be negative and its absolute value can be +* slightly greater than 'kernel_ns' because when KVM_SET_CLOCK is +* handled, we use more precise get_kvmclock_ns() there. Make sure +* unsigned 'system_time' doesn't go negative. +*/ + vcpu->hv_clock.system_time =max (kernel_ns + v->kvm->arch.kvmclock_offset, + (s64 )0 ); vcpu->last_guest_tsc = tsc_timestamp;/* If the host uses TSC clocksource, then it is stable */
Signature · 1 line
2.30.2
Paolo Bonzini
maintainerRe: [PATCH 1/2] KVM: x86: hyper-v: Forbid unsigned hv_clock->system_time to go negative after KVM_REQ_CLOCK_UPDATE
Quoted 3 lines · Vitaly Kuznetsov
Another solution is to cast 'hv_clock.system_time' to 's64' in
compute_tsc_page_parameters()but it seems we also use 'hv_clock.system_time' intrace_kvm_pvclock_update()as unsigned.
I think that is better. There is no reason really to clamp the value to to 0, while we know already that tsc_ref->tsc_offset can be either positive or negative. So treating hv_clock->system_time as signed before the division would make sense.
It should be just
arch/x86/kvm/hyperv.c +1 -3
@@ -1070,9 +1070,7 @@ static bool compute_tsc_page_parameters(struct pvclock_vcpu_time_info *hv_clock, hv_clock->tsc_to_system_mul, 100); - tsc_ref->tsc_offset = hv_clock->system_time; -do_div (tsc_ref->tsc_offset,100 ); - tsc_ref->tsc_offset -= + tsc_ref->tsc_offset =div_s64 (hv_clock->system_time,100 ) -mul_u64_u64_shr (hv_clock->tsc_timestamp, tsc_ref->tsc_scale,64 );return true; }
right? The test passes for me with this change.
Paolo
Vitaly Kuznetsov
maintainerRe: [PATCH 1/2] KVM: x86: hyper-v: Forbid unsigned hv_clock->system_time to go negative after KVM_REQ_CLOCK_UPDATE
Quoted 28 lines · Paolo Bonzini
Quoted 3 lines · Vitaly Kuznetsov
Another solution is to cast 'hv_clock.system_time' to 's64' in
compute_tsc_page_parameters()but it seems we also use 'hv_clock.system_time' intrace_kvm_pvclock_update()as unsigned.I think that is better. There is no reason really to clamp the value to to 0, while we know already that
tsc_ref->tsc_offsetcan be either positive or negative. So treatinghv_clock->system_timeas signed before the division would make sense.It should be just
arch/x86/kvm/hyperv.c +1
-3@@ -1070,9 +1070,7 @@ static bool compute_tsc_page_parameters(struct pvclock_vcpu_time_info *hv_clock, hv_clock->tsc_to_system_mul, 100); - tsc_ref->tsc_offset = hv_clock->system_time; -do_div (tsc_ref->tsc_offset,100 ); - tsc_ref->tsc_offset -= + tsc_ref->tsc_offset =div_s64 (hv_clock->system_time,100 ) -mul_u64_u64_shr (hv_clock->tsc_timestamp, tsc_ref->tsc_scale,64 );return true; }
Quoted 1 line
right? The test passes for me with this change.
Right,
in fact that's how v0 (which I've never sent out) of the patch looked like but then I relalized that the fact that unsigned 'hv_clock->system_time' can sometimes keep a negative value is a 'gotcha' which may cause issues in the future.
I'll re-test and send v2, thanks!
Signature · 1 line
Vitaly
Vitaly Kuznetsov
maintainerPATCH 2/2 selftests: kvm: Check that TSC page value is small after KVM_SET_CLOCK(0)
In reply to Vitaly KuznetsovAdd a test for the issue when KVM_SET_CLOCK(0) call could cause TSC page value to go very big because of a signedness issue around hv_clock->system_time.
--- tools/testing/selftests/kvm/x86_64/hyperv_clock.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/x86_64/hyperv_clock.c +11 -2
@@ -80,19 +80,24 @@ static inline void check_tsc_msr_rdtsc(void)GUEST_ASSERT (delta_ns *100 < (t2 - t1) *100 ); } +static inline u64 get_tscpage_ts (struct ms_hyperv_tsc_page *tsc_page) +{ +return mul_u64_u64_shr64 (rdtsc (), tsc_page->tsc_scale) + tsc_page->tsc_offset; +} +static inline void check_tsc_msr_tsc_page (struct ms_hyperv_tsc_page *tsc_page) {u64 r1, r2, t1, t2;/* Compare TSC page clocksource with HV_X64_MSR_TIME_REF_COUNT */ - t1 =mul_u64_u64_shr64 (rdtsc (), tsc_page->tsc_scale) + tsc_page->tsc_offset; + t1 =get_tscpage_ts (tsc_page); r1 =rdmsr (HV_X64_MSR_TIME_REF_COUNT );/* 10 ms tolerance */ GUEST_ASSERT (r1 >= t1 && r1 - t1 <100000 );nop_loop (); - t2 =mul_u64_u64_shr64 (rdtsc (), tsc_page->tsc_scale) + tsc_page->tsc_offset; + t2 =get_tscpage_ts (tsc_page); r2 =rdmsr (HV_X64_MSR_TIME_REF_COUNT );GUEST_ASSERT (r2 >= t1 && r2 - t2 <100000 ); }
@@ -130,7 +135,11 @@ static void guest_main(struct ms_hyperv_tsc_page *tsc_page, vm_paddr_t tsc_page_ tsc_offset = tsc_page->tsc_offset;/* Call KVM_SET_CLOCK from userspace, check that TSC page was updated */ +GUEST_SYNC (7 ); +/* Sanity check TSC page timestamp, it should be close to 0 */ +GUEST_ASSERT (get_tscpage_ts (tsc_page) <100000 ); +GUEST_ASSERT (tsc_page->tsc_offset != tsc_offset);nop_loop ();
Signature · 1 line
2.30.2