Skip to contentSkip to conversation

New: TSC synchronization issue in VM restore

Bug 217423 [email protected] 2 participants 7 messages May 9, 2023 – Jun 23, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

https://bugzilla.kernel.org/show_bug.cgi?id=217423

Bug ID: 217423
Summary: TSC synchronization issue in VM restore
Product: Virtualization
Version: unspecified
Hardware: All
OS: Linux
Status: NEW
Severity: normal
Priority: P3
Component: kvm
Assignee: [email protected]
Reporter: [email protected]
Regression: No

Hi

We are using lightweight VM with snapshot feature, the VM will be saved with 100ms+, and we found restore such VM will not get correct TSC, which will make the VM world stop about 100ms+ after restore (the stop time is same as time when VM saved).

After Investigation, we found the issue caused by TSC synchronization in setting MSR_IA32_TSC. In VM save, VMM (cloud-hypervisor) will record TSC of each
VCPU, then restore the TSC of VCPU in VM restore (about 100ms+ in guest time). But in KVM, setting a TSC within 1 second is identified as TSC synchronization, and the TSC offset will not be updated in stable TSC environment, this will cause the lapic set up a hrtimer expires after 100ms+, the restored VM now will in stop state about 100ms+, if no other event to wake guest kernel in NO_HZ mode.

More investigation show, the MSR_IA32_TSC set from guest side has disabled TSC synchronization in commit 0c899c25d754 (KVM: x86: do not attempt TSC synchronization on guest writes), now host side will do TSC synchronization when
setting MSR_IA32_TSC.

I think setting MSR_IA32_TSC within 1 second from host side should not be identified as TSC synchronization, like above case, VMM set TSC from host side always should be updated as user want.

The MSR_IA32_TSC set code is complicated and with a long history, so I come here
to try to get help about whether my thought is correct. Here is my fix to solve the issue, any comments are welcomed:

arch/x86/kvm/x86.c +0 -0
index ceb7c5e9cf9e..9380a88b9c1f 100644
@@ -2722,17 +2722,6 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu,

u64 data)

                         * kvm_clock stable after CPU hotplug
                         */
                        synchronizing = true;

- } else {
- u64 tsc_exp = kvm->arch.last_tsc_write +
- nsec_to_cycles(vcpu, elapsed);
- u64 tsc_hz = vcpu->arch.virtual_tsc_khz * 1000LL;
- /*
- * Special case: TSC write with a small delta (1
second)

-                        * of virtual cycle time against real time is
-                        * interpreted as an attempt to synchronize the CPU.
-                        */
-                       synchronizing = data < tsc_exp + tsc_hz &&
-                                       data + tsc_hz > tsc_exp;
                }
        }
Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.
#2 lore raw
In reply to [email protected]
Quoted 7 lines · On 5/9/2023 10:01 PM, [email protected] wrote:

Hi

We are using lightweight VM with snapshot feature, the VM will be saved with 100ms+, and we found restore such VM will not get correct TSC, which will make the VM world stop about 100ms+ after restore (the stop time is same as time when VM saved).

Quoted 7 lines

After Investigation, we found the issue caused by TSC synchronization in setting MSR_IA32_TSC. In VM save, VMM (cloud-hypervisor) will record TSC of each
VCPU, then restore the TSC of VCPU in VM restore (about 100ms+ in guest time). But in KVM, setting a TSC within 1 second is identified as TSC synchronization, and the TSC offset will not be updated in stable TSC environment, this will cause the lapic set up a hrtimer expires after 100ms+,

Can elaborate more on this hrtimer issue/code path?

Quoted 10 lines

the restored VM now will
in stop state about 100ms+, if no other event to wake guest kernel in NO_HZ mode.

More investigation show, the MSR_IA32_TSC set from guest side has disabled TSC synchronization in commit 0c899c25d754 (KVM: x86: do not attempt TSC synchronization on guest writes), now host side will do TSC synchronization when
setting MSR_IA32_TSC.

Quoted 3 lines

I think setting MSR_IA32_TSC within 1 second from host side should not be identified as TSC synchronization, like above case, VMM set TSC from host side always should be updated as user want.

This is heuristics, I think; at the very beginning, it was 5 seconds.
Perhaps nowadays, can we have some deterministic approach to identify a synchronization? e.g. add a new VM ioctl?

Quoted 29 lines

The MSR_IA32_TSC set code is complicated and with a long history, so I come here
to try to get help about whether my thought is correct. Here is my fix to solve the issue, any comments are welcomed:

arch/x86/kvm/x86.c +0 -11
index ceb7c5e9cf9e..9380a88b9c1f 100644
@@ -2722,17 +2722,6 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu,
 u64 data)
                          * kvm_clock stable after CPU hotplug
                          */
                         synchronizing = true;
-               } else {
-                       u64 tsc_exp = kvm->arch.last_tsc_write +
-                                               nsec_to_cycles(vcpu, elapsed);
-                       u64 tsc_hz = vcpu->arch.virtual_tsc_khz * 1000LL;
-                       /*
-                        * Special case: TSC write with a small delta (1
 second)
-                        * of virtual cycle time against real time is
-                        * interpreted as an attempt to synchronize the CPU.
-                        */
-                       synchronizing = data < tsc_exp + tsc_hz &&
-                                       data + tsc_hz > tsc_exp;
                 }
Quoted 2 lines
         }

This hunk of code is indeed historic and heuristic. But simply removing it isn't the way. Is the interval between your "save VM" and "restore VM" less than 1s?

An alternative, I think, is to bypass this directly write IA32_MSR_TSC way to set/sync TSC offsets, but follow new approach introduced in your VMM by

commit 828ca89628bfcb1b8f27535025f69dd00eb55207

Author: Oliver Upton <[email protected]>
Date:   Thu Sep 16 18:15:38 2021 +0000

KVM: x86: Expose TSC offset controls to userspace

...

Documentation/virt/kvm/devices/vcpu.rst:

4.1 ATTRIBUTE: KVM_VCPU_TSC_OFFSET

:Parameters: 64-bit unsigned TSC offset

...

Specifies the guest's TSC offset relative to the host's TSC. The guest's TSC is then derived by the following equation:

guest_tsc = host_tsc + KVM_VCPU_TSC_OFFSET

The following describes a possible algorithm to use for this purpose
...

#3 lore raw

[Bug 217423] TSC synchronization issue in VM restore

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=217423

--- Comment #2 from [email protected] ---

Quoted 8 lines · On 5/9/2023 10:01 PM, [email protected] wrote:

Hi

We are using lightweight VM with snapshot feature, the VM will be saved with 100ms+, and we found restore such VM will not get correct TSC, which will make
the VM world stop about 100ms+ after restore (the stop time is same as time when VM saved).

Quoted 9 lines

After Investigation, we found the issue caused by TSC synchronization in setting MSR_IA32_TSC. In VM save, VMM (cloud-hypervisor) will record TSC of each
VCPU, then restore the TSC of VCPU in VM restore (about 100ms+ in guest time).
But in KVM, setting a TSC within 1 second is identified as TSC synchronization,
and the TSC offset will not be updated in stable TSC environment, this will cause the lapic set up a hrtimer expires after 100ms+,

Can elaborate more on this hrtimer issue/code path?

Quoted 11 lines

the restored VM now will
in stop state about 100ms+, if no other event to wake guest kernel in NO_HZ mode.

More investigation show, the MSR_IA32_TSC set from guest side has disabled TSC
synchronization in commit 0c899c25d754 (KVM: x86: do not attempt TSC synchronization on guest writes), now host side will do TSC synchronization when
setting MSR_IA32_TSC.

Quoted 4 lines

I think setting MSR_IA32_TSC within 1 second from host side should not be identified as TSC synchronization, like above case, VMM set TSC from host side
always should be updated as user want.

This is heuristics, I think; at the very beginning, it was 5 seconds. Perhaps nowadays, can we have some deterministic approach to identify a synchronization? e.g. add a new VM ioctl?

Quoted 30 lines

The MSR_IA32_TSC set code is complicated and with a long history, so I come here
to try to get help about whether my thought is correct. Here is my fix to solve
the issue, any comments are welcomed:

arch/x86/kvm/x86.c +0 -11
index ceb7c5e9cf9e..9380a88b9c1f 100644
@@ -2722,17 +2722,6 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu,
 u64 data)
                          * kvm_clock stable after CPU hotplug
                          */
                         synchronizing = true;
-               } else {
-                       u64 tsc_exp = kvm->arch.last_tsc_write +
-                                               nsec_to_cycles(vcpu,
 elapsed);
-                       u64 tsc_hz = vcpu->arch.virtual_tsc_khz * 1000LL;
-                       /*
-                        * Special case: TSC write with a small delta (1
 second)
-                        * of virtual cycle time against real time is
-                        * interpreted as an attempt to synchronize the CPU.
-                        */
-                       synchronizing = data < tsc_exp + tsc_hz &&
-                                       data + tsc_hz > tsc_exp;
Quoted 3 lines
                 }
         }

This hunk of code is indeed historic and heuristic. But simply removing it isn't the way.
Is the interval between your "save VM" and "restore VM" less than 1s?

An alternative, I think, is to bypass this directly write IA32_MSR_TSC way to set/sync TSC offsets, but follow new approach introduced in your VMM by

commit 828ca89628bfcb1b8f27535025f69dd00eb55207

Author: Oliver Upton <[email protected]>
Date:   Thu Sep 16 18:15:38 2021 +0000

KVM: x86: Expose TSC offset controls to userspace

...

Documentation/virt/kvm/devices/vcpu.rst:

4.1 ATTRIBUTE: KVM_VCPU_TSC_OFFSET

:Parameters: 64-bit unsigned TSC offset

...

Specifies the guest's TSC offset relative to the host's TSC. The guest's TSC is then derived by the following equation:

guest_tsc = host_tsc + KVM_VCPU_TSC_OFFSET

The following describes a possible algorithm to use for this purpose
...

Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.
#4 lore raw

[Bug 217423] TSC synchronization issue in VM restore

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=217423

--- Comment #3 from zhuangel ([email protected]) --- (In reply to robert.hoo.linux from comment #2)

Quoted 20 lines · On 5/9/2023 10:01 PM, [email protected] wrote:
Quoted 3 lines

Hi

We are using lightweight VM with snapshot feature, the VM will be saved

with

Quoted 5 lines

100ms+, and we found restore such VM will not get correct TSC, which will make
the VM world stop about 100ms+ after restore (the stop time is same as time when VM saved).

Quoted 9 lines

After Investigation, we found the issue caused by TSC synchronization in setting MSR_IA32_TSC. In VM save, VMM (cloud-hypervisor) will record TSC of each
VCPU, then restore the TSC of VCPU in VM restore (about 100ms+ in guest time).
But in KVM, setting a TSC within 1 second is identified as TSC synchronization,
and the TSC offset will not be updated in stable TSC environment, this will cause the lapic set up a hrtimer expires after 100ms+,

Quoted 1 line

Can elaborate more on this hrtimer issue/code path?

Below are the steps in detail, I traced them via bpftrace, to simplify the analysis, the preemption timer on host is disabled, guest is running with TSC timer deadline mode.

TSC changes before save VM:
1 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc0 = 0 + offset0
2 pause VM after guest start finished (about 200ms)
host_tsc1 = guest_tsc1 + offset0 guest_tsc1_deadline = guest_tsc1 + expire1
3 save VM state
save guest_tsc1 by reading MSR_IA32_TSC
save guest_tsc1_deadline by reading MSR_IA32_TSC_DEADLINE

TSC changes in restore VM (to simplify the analysis, step 4 and step 5 ignore the host TSC changes in restore process):
4 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc3 = 0 + offset1
5 restore VM state
set MSR_IA32_TSC by guest_tsc1
set MSR_IA32_TSC_DEADLINE by guest_tsc1_deadline
6 start VM
VCPU_RUN

In step 5 setting MSR_IA32_TSC, because the guest_tsc1 is within 1 second, KVM will take this update as TSC synchronize, then skip update offset1. This means the guest TSC is still at 0 (initialize value).

In step 5 setting MSR_IA32_TSC_DEADLINE, VMM just want setup hrtimer after expire1, but KVM will get the current guest TSC is 0, and then calculate the expire time as (guest_tsc1_deadline - 0), then the hrtimer introduce guest_tsc1 latency (the guest kernel stopped if it will only kick by loapic timer).

Quoted 12 lines
Quoted 12 lines

the restored VM now will
in stop state about 100ms+, if no other event to wake guest kernel in NO_HZ mode.

More investigation show, the MSR_IA32_TSC set from guest side has disabled TSC
synchronization in commit 0c899c25d754 (KVM: x86: do not attempt TSC synchronization on guest writes), now host side will do TSC synchronization when
setting MSR_IA32_TSC.

Quoted 8 lines
Quoted 4 lines

I think setting MSR_IA32_TSC within 1 second from host side should not be identified as TSC synchronization, like above case, VMM set TSC from host side
always should be updated as user want.

This is heuristics, I think; at the very beginning, it was 5 seconds. Perhaps nowadays, can we have some deterministic approach to identify a synchronization? e.g. add a new VM ioctl?

The 5 seconds was original introduced in commit f38e098ff3a3 ("KVM: x86:
TSC reset compensation"), when setting TSC into same value in 5 second. And in 46543ba45fc49 ("KVM: x86: Robust TSC compensation"), remove the same value check.

From changes in these 2 commits, I think the TSC synchronize first introduced to handle synchronize from guest side (kernel boot), then changed to migration setting from host side.

In commit 0c899c25d754 ("KVM: x86: do not attempt TSC synchronization on guest writes"), Paolo has split the guest TSC write from this complicated synchronization.

Now just migration scenarios left, in migration the TSC restore should be set before VCPU start to run, one hack is to skip checking whether the VCPU starts running, such as check last_vmentry_cpu is -1.

Quoted 58 lines

arch/x86/kvm/x86.c static void kvm_synchronize_tsc(struct kvm_vcpu

Quoted 13 lines

The MSR_IA32_TSC set code is complicated and with a long history, so I come here
to try to get help about whether my thought is correct. Here is my fix to solve
the issue, any comments are welcomed:

arch/x86/kvm/x86.c +0 -0
index ceb7c5e9cf9e..9380a88b9c1f 100644
@@ -2722,17 +2722,6 @@ static void kvm_synchronize_tsc(struct kvm_vcpu

*vcpu,

Quoted 4 lines
arch/x86/kvm/x86.c +0 -0
 u64 data)
                          * kvm_clock stable after CPU hotplug
                          */
                         synchronizing = true;

arch/x86/kvm/x86.c static void kvm_synchronize_tsc(struct kvm_vcpu · line 2735

Quoted 10 lines
arch/x86/kvm/x86.c +0 -8
-               } else {
-                       u64 tsc_exp = kvm->arch.last_tsc_write +
-                                               nsec_to_cycles(vcpu,
 elapsed);
-                       u64 tsc_hz = vcpu->arch.virtual_tsc_khz * 1000LL;
-                       /*
-                        * Special case: TSC write with a small delta (1
 second)
-                        * of virtual cycle time against real time is
-                        * interpreted as an attempt to synchronize the

CPU.

Quoted 6 lines
arch/x86/kvm/x86.c +0 -3
-                        */
-                       synchronizing = data < tsc_exp + tsc_hz &&
-                                       data + tsc_hz > tsc_exp;
                 }
         }

This hunk of code is indeed historic and heuristic. But simply removing it isn't the way.
Is the interval between your "save VM" and "restore VM" less than 1s?

An alternative, I think, is to bypass this directly write IA32_MSR_TSC way to set/sync TSC offsets, but follow new approach introduced in your VMM by

commit 828ca89628bfcb1b8f27535025f69dd00eb55207

Author: Oliver Upton <[email protected]>
Date:   Thu Sep 16 18:15:38 2021 +0000

KVM: x86: Expose TSC offset controls to userspace

...

Documentation/virt/kvm/devices/vcpu.rst:

4.1 ATTRIBUTE: KVM_VCPU_TSC_OFFSET

:Parameters: 64-bit unsigned TSC offset

...

Quoted 7 lines

Specifies the guest's TSC offset relative to the host's TSC. The guest's TSC is then derived by the following equation:

guest_tsc = host_tsc + KVM_VCPU_TSC_OFFSET

The following describes a possible algorithm to use for this purpose
...

"TSC counts the time during which the VM was paused.", This new feature works for live migration. But if we save/restore VM with snapshot, the TSC should be paused either?

Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.
#5 lore raw

Re: [Bug 217423] TSC synchronization issue in VM restore

On 5/18/2023 8:48 PM, [email protected] wrote:
[...]
(Sorry for late response)

Quoted 26 lines
Quoted 1 line

Can elaborate more on this hrtimer issue/code path?

Below are the steps in detail, I traced them via bpftrace, to simplify the analysis, the preemption timer on host is disabled, guest is running with TSC timer deadline mode.

TSC changes before save VM:
1 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc0 = 0 + offset0
2 pause VM after guest start finished (about 200ms)
host_tsc1 = guest_tsc1 + offset0 guest_tsc1_deadline = guest_tsc1 + expire1
3 save VM state
save guest_tsc1 by reading MSR_IA32_TSC
save guest_tsc1_deadline by reading MSR_IA32_TSC_DEADLINE

TSC changes in restore VM (to simplify the analysis, step 4 and step 5 ignore the host TSC changes in restore process):
4 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc3 = 0 + offset1
5 restore VM state
set MSR_IA32_TSC by guest_tsc1
set MSR_IA32_TSC_DEADLINE by guest_tsc1_deadline
6 start VM
VCPU_RUN

Quoted 3 lines

In step 5 setting MSR_IA32_TSC, because the guest_tsc1 is within 1 second, KVM will take this update as TSC synchronize, then skip update offset1. This means the guest TSC is still at 0 (initialize value).

IIUC, here no matter synchronizing = true or false, offset will always be updated, i.e. __kvm_synchronize_tsc() will be called. But the offset value will differ.

I guess your environment is tsc_stable, then offset = kvm->arch.cur_tsc_offset, which is 0. That is to say, the elapsed time isn't counted in by the heuristics method in current code, that's the culprit.

static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)

{
	...
	offset = kvm_compute_l1_tsc_offset(vcpu, data);
	...

	/*
	 * For a reliable TSC, we can match TSC offsets, and for an unstable
	 * TSC, we add elapsed time in this computation.  We could let the
	 * compensation code attempt to catch up if we fall behind, but
	 * it's better to try to match offsets from the beginning.
         */
	if (synchronizing &&
	    vcpu->arch.virtual_tsc_khz == kvm->arch.last_tsc_khz) {
		if (!kvm_check_tsc_unstable()) {
			offset = kvm->arch.cur_tsc_offset;
		} else {
			u64 delta = nsec_to_cycles(vcpu, elapsed);
			data += delta;
			offset = kvm_compute_l1_tsc_offset(vcpu, data);
		}
		matched = true;
	}

	__kvm_synchronize_tsc(vcpu, offset, data, ns, matched);
	raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
}
Quoted 22 lines
Quoted 22 lines

An alternative, I think, is to bypass this directly write IA32_MSR_TSC way to set/sync TSC offsets, but follow new approach introduced in your VMM by

commit 828ca89628bfcb1b8f27535025f69dd00eb55207

Author: Oliver Upton <[email protected]>
Date:   Thu Sep 16 18:15:38 2021 +0000

KVM: x86: Expose TSC offset controls to userspace

...

Documentation/virt/kvm/devices/vcpu.rst:

4.1 ATTRIBUTE: KVM_VCPU_TSC_OFFSET

:Parameters: 64-bit unsigned TSC offset

...

Specifies the guest's TSC offset relative to the host's TSC. The guest's TSC is then derived by the following equation:

Quoted 9 lines
Quoted 4 lines

guest_tsc = host_tsc + KVM_VCPU_TSC_OFFSET

The following describes a possible algorithm to use for this purpose
...

"TSC counts the time during which the VM was paused.", This new feature works for live migration. But if we save/restore VM with snapshot, the TSC should be paused either?

Not sure what's host's TSC situation when host is, say, suspended/hibernated. VM Save/Restore can refer to that.
But, the key point of this new approach is to use OFFSET rather than direct TSC value, this is like x86 TSC_ADJUST was introduced, and is preferred.
Via this new interface,
"... Ensure that the KVM_CLOCK_REALTIME flag is set in the provided structure.
KVM will advance the VM's kvmclock to account for elapsed time since recording the clock values.", therefore I think it can solve your problem, rather than modify the ancient and heuristics code at high risk.

#6 lore raw

[Bug 217423] TSC synchronization issue in VM restore

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=217423

--- Comment #4 from [email protected] --- On 5/18/2023 8:48 PM, [email protected] wrote:
[...]
(Sorry for late response)

Quoted 26 lines
Quoted 1 line

Can elaborate more on this hrtimer issue/code path?

Below are the steps in detail, I traced them via bpftrace, to simplify the analysis, the preemption timer on host is disabled, guest is running with TSC timer deadline mode.

TSC changes before save VM:
1 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc0 = 0 + offset0
2 pause VM after guest start finished (about 200ms)
host_tsc1 = guest_tsc1 + offset0 guest_tsc1_deadline = guest_tsc1 + expire1
3 save VM state
save guest_tsc1 by reading MSR_IA32_TSC
save guest_tsc1_deadline by reading MSR_IA32_TSC_DEADLINE

TSC changes in restore VM (to simplify the analysis, step 4 and step 5 ignore the host TSC changes in restore process):
4 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc3 = 0 + offset1
5 restore VM state
set MSR_IA32_TSC by guest_tsc1
set MSR_IA32_TSC_DEADLINE by guest_tsc1_deadline
6 start VM
VCPU_RUN

Quoted 3 lines

In step 5 setting MSR_IA32_TSC, because the guest_tsc1 is within 1 second, KVM will take this update as TSC synchronize, then skip update offset1. This means the guest TSC is still at 0 (initialize value).

IIUC, here no matter synchronizing = true or false, offset will always be updated, i.e. __kvm_synchronize_tsc() will be called. But the offset value will differ.

I guess your environment is tsc_stable, then offset = kvm->arch.cur_tsc_offset, which is 0. That is to say, the elapsed time isn't counted in by the heuristics method in current code, that's the culprit.

static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)

{
        ...
        offset = kvm_compute_l1_tsc_offset(vcpu, data);
        ...

        /*
         * For a reliable TSC, we can match TSC offsets, and for an unstable
         * TSC, we add elapsed time in this computation.  We could let the
         * compensation code attempt to catch up if we fall behind, but
         * it's better to try to match offsets from the beginning.
          */
        if (synchronizing &&
            vcpu->arch.virtual_tsc_khz == kvm->arch.last_tsc_khz) {
                if (!kvm_check_tsc_unstable()) {
                        offset = kvm->arch.cur_tsc_offset;
                } else {
                        u64 delta = nsec_to_cycles(vcpu, elapsed);
                        data += delta;
                        offset = kvm_compute_l1_tsc_offset(vcpu, data);
                }
                matched = true;
        }

        __kvm_synchronize_tsc(vcpu, offset, data, ns, matched);
        raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
}
Quoted 24 lines
Quoted 24 lines

An alternative, I think, is to bypass this directly write IA32_MSR_TSC way to set/sync TSC offsets, but follow new approach introduced in your VMM by

commit 828ca89628bfcb1b8f27535025f69dd00eb55207

Author: Oliver Upton <[email protected]>
Date:   Thu Sep 16 18:15:38 2021 +0000

KVM: x86: Expose TSC offset controls to userspace

...

Documentation/virt/kvm/devices/vcpu.rst:

4.1 ATTRIBUTE: KVM_VCPU_TSC_OFFSET

:Parameters: 64-bit unsigned TSC offset

...

Specifies the guest's TSC offset relative to the host's TSC. The guest's TSC is then derived by the following equation:

guest_tsc = host_tsc + KVM_VCPU_TSC_OFFSET

Quoted 8 lines
Quoted 2 lines

The following describes a possible algorithm to use for this purpose
...

"TSC counts the time during which the VM was paused.", This new feature works for live migration. But if we save/restore VM with snapshot, the TSC should be
paused either?

Not sure what's host's TSC situation when host is, say, suspended/hibernated. VM
Save/Restore can refer to that.
But, the key point of this new approach is to use OFFSET rather than direct TSC value, this is like x86 TSC_ADJUST was introduced, and is preferred. Via this new interface,
"... Ensure that the KVM_CLOCK_REALTIME flag is set in the provided structure. KVM will advance the VM's kvmclock to account for elapsed time since recording the clock values.", therefore I think it can solve your problem, rather than modify the ancient and heuristics code at high risk.

Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.