New: TSC synchronization issue in VM restore
https://bugzilla.kernel.org/show_bug.cgi?id=217423
Bug ID: 217423
Summary: TSC synchronization issue in VM restore
Product: Virtualization
Version: unspecified
Hardware: All
OS: Linux
Status: NEW
Severity: normal
Priority: P3
Component: kvm
Assignee: [email protected]
Reporter: [email protected]
Regression: No
Hi
We are using lightweight VM with snapshot feature, the VM will be saved with 100ms+, and we found restore such VM will not get correct TSC, which will make the VM world stop about 100ms+ after restore (the stop time is same as time when VM saved).
After Investigation, we found the issue caused by TSC synchronization in setting MSR_IA32_TSC. In VM save, VMM (cloud-hypervisor) will record TSC of each
VCPU, then restore the TSC of VCPU in VM restore (about 100ms+ in guest time). But in KVM, setting a TSC within 1 second is identified as TSC synchronization, and the TSC offset will not be updated in stable TSC environment, this will cause the lapic set up a hrtimer expires after 100ms+, the restored VM now will in stop state about 100ms+, if no other event to wake guest kernel in NO_HZ mode.
More investigation show, the MSR_IA32_TSC set from guest side has disabled TSC synchronization in commit 0c899c25d754 (KVM: x86: do not attempt TSC synchronization on guest writes), now host side will do TSC synchronization when
setting MSR_IA32_TSC.
I think setting MSR_IA32_TSC within 1 second from host side should not be identified as TSC synchronization, like above case, VMM set TSC from host side always should be updated as user want.
The MSR_IA32_TSC set code is complicated and with a long history, so I come here
to try to get help about whether my thought is correct. Here is my fix to solve the issue, any comments are welcomed:
arch/x86/kvm/x86.c +0 -0
@@ -2722,17 +2722,6 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu,
u64 data)
* kvm_clock stable after CPU hotplug
*/
synchronizing = true;- } else {
- u64 tsc_exp = kvm->arch.last_tsc_write +
- nsec_to_cycles(vcpu, elapsed);
- u64 tsc_hz = vcpu->arch.virtual_tsc_khz * 1000LL;
- /*
- * Special case: TSC write with a small delta (1
second)
- * of virtual cycle time against real time is
- * interpreted as an attempt to synchronize the CPU.
- */
- synchronizing = data < tsc_exp + tsc_hz &&
- data + tsc_hz > tsc_exp;
}
}Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 217423] TSC synchronization issue in VM restore
https://bugzilla.kernel.org/show_bug.cgi?id=217423
--- Comment #1 from Artem S. Tashkinov ([email protected]) --- *** Bug 217424 has been marked as a duplicate of this bug. ***
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
Quoted 7 lines · On 5/9/2023 10:01 PM, [email protected] wrote:
Hi
We are using lightweight VM with snapshot feature, the VM will be saved with 100ms+, and we found restore such VM will not get correct TSC, which will make the VM world stop about 100ms+ after restore (the stop time is same as time when VM saved).
Quoted 7 lines
After Investigation, we found the issue caused by TSC synchronization in setting
MSR_IA32_TSC. In VM save, VMM (cloud-hypervisor) will record TSC of each
VCPU, then restore the TSC of VCPU in VM restore (about 100ms+ in guest time). But in KVM, setting a TSC within 1 second is identified as TSC synchronization, and the TSC offset will not be updated in stable TSC environment, this will cause the lapic set up a hrtimer expires after 100ms+,
Can elaborate more on this hrtimer issue/code path?
Quoted 10 lines
the restored VM now will
in stop state about 100ms+, if no other event to wake guest kernel inNO_HZmode.More investigation show, the
MSR_IA32_TSCset from guest side has disabled TSC synchronization in commit0c899c25d754(KVM: x86: do not attempt TSC synchronization on guest writes), now host side will do TSC synchronization when
settingMSR_IA32_TSC.
Quoted 3 lines
I think setting
MSR_IA32_TSCwithin 1 second from host side should not be identified as TSC synchronization, like above case, VMM set TSC from host side always should be updated as user want.
This is heuristics, I think; at the very beginning, it was 5 seconds.
Perhaps nowadays, can we have some deterministic approach to identify a synchronization? e.g. add a new VM ioctl?
Quoted 29 lines
The
MSR_IA32_TSCset code is complicated and with a long history, so I come here
to try to get help about whether my thought is correct. Here is my fix to solve the issue, any comments are welcomed:arch/x86/kvm/x86.c +0
-11@@ -2722,17 +2722,6 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu,u64 data) * kvm_clock stable after CPU hotplug */ synchronizing = true; - }else { -u64 tsc_exp = kvm->arch.last_tsc_write + -nsec_to_cycles (vcpu, elapsed); -u64 tsc_hz = vcpu->arch.virtual_tsc_khz *1000LL ; -/* -* Special case: TSC write with a small delta (1 second) -* of virtual cycle time against real time is -* interpreted as an attempt to synchronize the CPU. -*/ - synchronizing = data < tsc_exp + tsc_hz && - data + tsc_hz > tsc_exp; }
Quoted 2 lines
}
This hunk of code is indeed historic and heuristic. But simply removing it isn't the way. Is the interval between your "save VM" and "restore VM" less than 1s?
An alternative, I think, is to bypass this directly write IA32_MSR_TSC way to set/sync TSC offsets, but follow new approach introduced in your VMM by
commit 828ca89628bfcb1b8f27535025f69dd00eb55207
Author: Oliver Upton <[email protected]> Date: Thu Sep 16 18:15:38 2021 +0000
KVM: x86: Expose TSC offset controls to userspace
...
Documentation/virt/kvm/devices/vcpu.rst:
4.1 ATTRIBUTE: KVM_VCPU_TSC_OFFSET
:Parameters: 64-bit unsigned TSC offset
...
Specifies the guest's TSC offset relative to the host's TSC. The guest's TSC is then derived by the following equation:
guest_tsc = host_tsc + KVM_VCPU_TSC_OFFSET
The following describes a possible algorithm to use for this purpose
...
[Bug 217423] TSC synchronization issue in VM restore
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=217423
--- Comment #2 from [email protected] ---
Quoted 8 lines · On 5/9/2023 10:01 PM, [email protected] wrote:
Hi
We are using lightweight VM with snapshot feature, the VM will be saved with 100ms+, and we found restore such VM will not get correct TSC, which will make
the VM world stop about 100ms+ after restore (the stop time is same as time when VM saved).
Quoted 9 lines
After Investigation, we found the issue caused by TSC synchronization in setting
MSR_IA32_TSC. In VM save, VMM (cloud-hypervisor) will record TSC of each
VCPU, then restore the TSC of VCPU in VM restore (about 100ms+ in guest time).
But in KVM, setting a TSC within 1 second is identified as TSC synchronization,
and the TSC offset will not be updated in stable TSC environment, this will cause the lapic set up a hrtimer expires after 100ms+,
Can elaborate more on this hrtimer issue/code path?
Quoted 11 lines
the restored VM now will
in stop state about 100ms+, if no other event to wake guest kernel inNO_HZmode.More investigation show, the
MSR_IA32_TSCset from guest side has disabled TSC
synchronization in commit0c899c25d754(KVM: x86: do not attempt TSC synchronization on guest writes), now host side will do TSC synchronization when
settingMSR_IA32_TSC.
Quoted 4 lines
I think setting
MSR_IA32_TSCwithin 1 second from host side should not be identified as TSC synchronization, like above case, VMM set TSC from host side
always should be updated as user want.
This is heuristics, I think; at the very beginning, it was 5 seconds. Perhaps nowadays, can we have some deterministic approach to identify a synchronization? e.g. add a new VM ioctl?
Quoted 30 lines
The
MSR_IA32_TSCset code is complicated and with a long history, so I come here
to try to get help about whether my thought is correct. Here is my fix to solve
the issue, any comments are welcomed:arch/x86/kvm/x86.c +0
-11@@ -2722,17 +2722,6 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu,u64 data) * kvm_clock stable after CPU hotplug */ synchronizing = true; - }else { -u64 tsc_exp = kvm->arch.last_tsc_write + -nsec_to_cycles (vcpu, elapsed); -u64 tsc_hz = vcpu->arch.virtual_tsc_khz *1000LL ; -/* -* Special case: TSC write with a small delta (1 second) -* of virtual cycle time against real time is -* interpreted as an attempt to synchronize the CPU. -*/ - synchronizing = data < tsc_exp + tsc_hz && - data + tsc_hz > tsc_exp;
Quoted 3 lines
}
}This hunk of code is indeed historic and heuristic. But simply removing it isn't the way.
Is the interval between your "save VM" and "restore VM" less than 1s?
An alternative, I think, is to bypass this directly write IA32_MSR_TSC way to set/sync TSC offsets, but follow new approach introduced in your VMM by
commit 828ca89628bfcb1b8f27535025f69dd00eb55207
Author: Oliver Upton <[email protected]> Date: Thu Sep 16 18:15:38 2021 +0000
KVM: x86: Expose TSC offset controls to userspace
...
Documentation/virt/kvm/devices/vcpu.rst:
4.1 ATTRIBUTE: KVM_VCPU_TSC_OFFSET
:Parameters: 64-bit unsigned TSC offset
...
Specifies the guest's TSC offset relative to the host's TSC. The guest's TSC is then derived by the following equation:
guest_tsc = host_tsc + KVM_VCPU_TSC_OFFSET
The following describes a possible algorithm to use for this purpose
...
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 217423] TSC synchronization issue in VM restore
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=217423
--- Comment #3 from zhuangel ([email protected]) --- (In reply to robert.hoo.linux from comment #2)
Quoted 20 lines · On 5/9/2023 10:01 PM, [email protected] wrote:
Quoted 3 lines
Hi
We are using lightweight VM with snapshot feature, the VM will be saved
with
Quoted 5 lines
100ms+, and we found restore such VM will not get correct TSC, which will make
the VM world stop about 100ms+ after restore (the stop time is same as time when VM saved).Quoted 9 lines
After Investigation, we found the issue caused by TSC synchronization in setting
MSR_IA32_TSC. In VM save, VMM (cloud-hypervisor) will record TSC of each
VCPU, then restore the TSC of VCPU in VM restore (about 100ms+ in guest time).
But in KVM, setting a TSC within 1 second is identified as TSC synchronization,
and the TSC offset will not be updated in stable TSC environment, this will cause the lapic set up a hrtimer expires after 100ms+,
Quoted 1 line
Can elaborate more on this hrtimer issue/code path?
Below are the steps in detail, I traced them via bpftrace, to simplify the analysis, the preemption timer on host is disabled, guest is running with TSC timer deadline mode.
TSC changes before save VM:
1 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc0 = 0 + offset0
2 pause VM after guest start finished (about 200ms)
host_tsc1 = guest_tsc1 + offset0 guest_tsc1_deadline = guest_tsc1 + expire1
3 save VM state
save guest_tsc1 by reading MSR_IA32_TSC
save guest_tsc1_deadline by reading MSR_IA32_TSC_DEADLINE
TSC changes in restore VM (to simplify the analysis, step 4 and step 5 ignore the host TSC changes in restore process):
4 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc3 = 0 + offset1
5 restore VM state
set MSR_IA32_TSC by guest_tsc1
set MSR_IA32_TSC_DEADLINE by guest_tsc1_deadline
6 start VM
VCPU_RUN
In step 5 setting MSR_IA32_TSC, because the guest_tsc1 is within 1 second, KVM will take this update as TSC synchronize, then skip update offset1. This means the guest TSC is still at 0 (initialize value).
In step 5 setting MSR_IA32_TSC_DEADLINE, VMM just want setup hrtimer after expire1, but KVM will get the current guest TSC is 0, and then calculate the expire time as (guest_tsc1_deadline - 0), then the hrtimer introduce guest_tsc1 latency (the guest kernel stopped if it will only kick by loapic timer).
Quoted 12 lines
Quoted 12 lines
the restored VM now will
in stop state about 100ms+, if no other event to wake guest kernel inNO_HZmode.More investigation show, the
MSR_IA32_TSCset from guest side has disabled TSC
synchronization in commit0c899c25d754(KVM: x86: do not attempt TSC synchronization on guest writes), now host side will do TSC synchronization when
settingMSR_IA32_TSC.
Quoted 8 lines
Quoted 4 lines
I think setting
MSR_IA32_TSCwithin 1 second from host side should not be identified as TSC synchronization, like above case, VMM set TSC from host side
always should be updated as user want.This is heuristics, I think; at the very beginning, it was 5 seconds. Perhaps nowadays, can we have some deterministic approach to identify a synchronization? e.g. add a new VM ioctl?
The 5 seconds was original introduced in commit f38e098ff3a3 ("KVM: x86:
TSC reset compensation"), when setting TSC into same value in 5 second. And in 46543ba45fc49 ("KVM: x86: Robust TSC compensation"), remove the same value check.
From changes in these 2 commits, I think the TSC synchronize first introduced to handle synchronize from guest side (kernel boot), then changed to migration setting from host side.
In commit 0c899c25d754 ("KVM: x86: do not attempt TSC synchronization on guest writes"), Paolo has split the guest TSC write from this complicated synchronization.
Now just migration scenarios left, in migration the TSC restore should be set before VCPU start to run, one hack is to skip checking whether the VCPU starts running, such as check last_vmentry_cpu is -1.
Quoted 58 lines
arch/x86/kvm/x86.c static void kvm_synchronize_tsc(struct kvm_vcpu
Quoted 13 lines
The
MSR_IA32_TSCset code is complicated and with a long history, so I come here
to try to get help about whether my thought is correct. Here is my fix to solve
the issue, any comments are welcomed:arch/x86/kvm/x86.c +0
-0@@ -2722,17 +2722,6 @@ static void kvm_synchronize_tsc(struct kvm_vcpu*vcpu,
Quoted 4 lines
arch/x86/kvm/x86.c +0
-0u64 data) * kvm_clock stable after CPU hotplug */ synchronizing = true;arch/x86/kvm/x86.c static void kvm_synchronize_tsc(struct kvm_vcpu · line 2735
Quoted 10 lines
arch/x86/kvm/x86.c +0
-8- }else { -u64 tsc_exp = kvm->arch.last_tsc_write + -nsec_to_cycles (vcpu, elapsed); -u64 tsc_hz = vcpu->arch.virtual_tsc_khz *1000LL ; -/* -* Special case: TSC write with a small delta (1 second) -* of virtual cycle time against real time is -* interpreted as an attempt to synchronize the CPU.
Quoted 6 lines
arch/x86/kvm/x86.c +0
-3-*/ - synchronizing = data < tsc_exp + tsc_hz && - data + tsc_hz > tsc_exp;} }This hunk of code is indeed historic and heuristic. But simply removing it isn't the way.
Is the interval between your "save VM" and "restore VM" less than 1s?An alternative, I think, is to bypass this directly write
IA32_MSR_TSCway to set/sync TSC offsets, but follow new approach introduced in your VMM bycommit
828ca89628bfcb1b8f27535025f69dd00eb55207Author: Oliver Upton <[email protected]> Date: Thu Sep 16 18:15:38 2021 +0000KVM: x86: Expose TSC offset controls to userspace
...
Documentation/virt/kvm/devices/vcpu.rst:4.1 ATTRIBUTE:
KVM_VCPU_TSC_OFFSET:Parameters: 64-bit unsigned TSC offset
...
Quoted 7 lines
Specifies the guest's TSC offset relative to the host's TSC. The guest's TSC is then derived by the following equation:
guest_tsc=host_tsc+KVM_VCPU_TSC_OFFSETThe following describes a possible algorithm to use for this purpose
...
"TSC counts the time during which the VM was paused.", This new feature works for live migration. But if we save/restore VM with snapshot, the TSC should be paused either?
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
Re: [Bug 217423] TSC synchronization issue in VM restore
On 5/18/2023 8:48 PM, [email protected] wrote:
[...]
(Sorry for late response)
Quoted 26 lines
Quoted 1 line
Can elaborate more on this hrtimer issue/code path?
Below are the steps in detail, I traced them via bpftrace, to simplify the analysis, the preemption timer on host is disabled, guest is running with TSC timer deadline mode.
TSC changes before save VM:
1 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc0= 0 + offset0
2 pause VM after guest start finished (about 200ms)
host_tsc1=guest_tsc1+ offset0guest_tsc1_deadline=guest_tsc1+ expire1
3 save VM state
saveguest_tsc1by readingMSR_IA32_TSC
saveguest_tsc1_deadlineby readingMSR_IA32_TSC_DEADLINETSC changes in restore VM (to simplify the analysis, step 4 and step 5 ignore the host TSC changes in restore process):
4 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc3= 0 + offset1
5 restore VM state
setMSR_IA32_TSCbyguest_tsc1
setMSR_IA32_TSC_DEADLINEbyguest_tsc1_deadline
6 start VM
VCPU_RUN
Quoted 3 lines
In step 5 setting
MSR_IA32_TSC, because theguest_tsc1is within 1 second, KVM will take this update as TSC synchronize, then skip update offset1. This means the guest TSC is still at 0 (initialize value).
IIUC, here no matter synchronizing = true or false, offset will always be updated, i.e. __kvm_synchronize_tsc() will be called. But the offset value will differ.
I guess your environment is tsc_stable, then offset = kvm->arch.cur_tsc_offset, which is 0. That is to say, the elapsed time isn't counted in by the heuristics method in current code, that's the culprit.
static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)
{
...
offset = kvm_compute_l1_tsc_offset(vcpu, data);
...
/*
* For a reliable TSC, we can match TSC offsets, and for an unstable
* TSC, we add elapsed time in this computation. We could let the
* compensation code attempt to catch up if we fall behind, but
* it's better to try to match offsets from the beginning.
*/
if (synchronizing &&
vcpu->arch.virtual_tsc_khz == kvm->arch.last_tsc_khz) {
if (!kvm_check_tsc_unstable()) {
offset = kvm->arch.cur_tsc_offset;
} else {
u64 delta = nsec_to_cycles(vcpu, elapsed);
data += delta;
offset = kvm_compute_l1_tsc_offset(vcpu, data);
}
matched = true;
}
__kvm_synchronize_tsc(vcpu, offset, data, ns, matched);
raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
}Quoted 22 lines
Quoted 22 lines
An alternative, I think, is to bypass this directly write
IA32_MSR_TSCway to set/sync TSC offsets, but follow new approach introduced in your VMM bycommit
828ca89628bfcb1b8f27535025f69dd00eb55207Author: Oliver Upton <[email protected]> Date: Thu Sep 16 18:15:38 2021 +0000KVM: x86: Expose TSC offset controls to userspace
...
Documentation/virt/kvm/devices/vcpu.rst:4.1 ATTRIBUTE:
KVM_VCPU_TSC_OFFSET:Parameters: 64-bit unsigned TSC offset
...
Specifies the guest's TSC offset relative to the host's TSC. The guest's TSC is then derived by the following equation:
Quoted 9 lines
Quoted 4 lines
guest_tsc=host_tsc+KVM_VCPU_TSC_OFFSETThe following describes a possible algorithm to use for this purpose
..."TSC counts the time during which the VM was paused.", This new feature works for live migration. But if we save/restore VM with snapshot, the TSC should be paused either?
Not sure what's host's TSC situation when host is, say, suspended/hibernated. VM Save/Restore can refer to that.
But, the key point of this new approach is to use OFFSET rather than direct TSC value, this is like x86 TSC_ADJUST was introduced, and is preferred.
Via this new interface,
"... Ensure that the KVM_CLOCK_REALTIME flag is set in the provided structure.
KVM will advance the VM's kvmclock to account for elapsed time since recording the clock values.", therefore I think it can solve your problem, rather than modify the ancient and heuristics code at high risk.
[Bug 217423] TSC synchronization issue in VM restore
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=217423
--- Comment #4 from [email protected] --- On 5/18/2023 8:48 PM, [email protected] wrote:
[...]
(Sorry for late response)
Quoted 26 lines
Quoted 1 line
Can elaborate more on this hrtimer issue/code path?
Below are the steps in detail, I traced them via bpftrace, to simplify the analysis, the preemption timer on host is disabled, guest is running with TSC timer deadline mode.
TSC changes before save VM:
1 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc0= 0 + offset0
2 pause VM after guest start finished (about 200ms)
host_tsc1=guest_tsc1+ offset0guest_tsc1_deadline=guest_tsc1+ expire1
3 save VM state
saveguest_tsc1by readingMSR_IA32_TSC
saveguest_tsc1_deadlineby readingMSR_IA32_TSC_DEADLINETSC changes in restore VM (to simplify the analysis, step 4 and step 5 ignore the host TSC changes in restore process):
4 create VM/VCPU, guest TSC start from 0 (VCPU initial value)
host_tsc3= 0 + offset1
5 restore VM state
setMSR_IA32_TSCbyguest_tsc1
setMSR_IA32_TSC_DEADLINEbyguest_tsc1_deadline
6 start VM
VCPU_RUN
Quoted 3 lines
In step 5 setting
MSR_IA32_TSC, because theguest_tsc1is within 1 second, KVM will take this update as TSC synchronize, then skip update offset1. This means the guest TSC is still at 0 (initialize value).
IIUC, here no matter synchronizing = true or false, offset will always be updated, i.e. __kvm_synchronize_tsc() will be called. But the offset value will differ.
I guess your environment is tsc_stable, then offset = kvm->arch.cur_tsc_offset, which is 0. That is to say, the elapsed time isn't counted in by the heuristics method in current code, that's the culprit.
static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)
{
...
offset = kvm_compute_l1_tsc_offset(vcpu, data);
...
/*
* For a reliable TSC, we can match TSC offsets, and for an unstable
* TSC, we add elapsed time in this computation. We could let the
* compensation code attempt to catch up if we fall behind, but
* it's better to try to match offsets from the beginning.
*/
if (synchronizing &&
vcpu->arch.virtual_tsc_khz == kvm->arch.last_tsc_khz) {
if (!kvm_check_tsc_unstable()) {
offset = kvm->arch.cur_tsc_offset;
} else {
u64 delta = nsec_to_cycles(vcpu, elapsed);
data += delta;
offset = kvm_compute_l1_tsc_offset(vcpu, data);
}
matched = true;
}
__kvm_synchronize_tsc(vcpu, offset, data, ns, matched);
raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
}Quoted 24 lines
Quoted 24 lines
An alternative, I think, is to bypass this directly write
IA32_MSR_TSCway to set/sync TSC offsets, but follow new approach introduced in your VMM bycommit
828ca89628bfcb1b8f27535025f69dd00eb55207Author: Oliver Upton <[email protected]> Date: Thu Sep 16 18:15:38 2021 +0000KVM: x86: Expose TSC offset controls to userspace
...
Documentation/virt/kvm/devices/vcpu.rst:4.1 ATTRIBUTE:
KVM_VCPU_TSC_OFFSET:Parameters: 64-bit unsigned TSC offset
...
Specifies the guest's TSC offset relative to the host's TSC. The guest's TSC is then derived by the following equation:
guest_tsc=host_tsc+KVM_VCPU_TSC_OFFSET
Quoted 8 lines
Quoted 2 lines
The following describes a possible algorithm to use for this purpose
..."TSC counts the time during which the VM was paused.", This new feature works for live migration. But if we save/restore VM with snapshot, the TSC should be
paused either?
Not sure what's host's TSC situation when host is, say, suspended/hibernated. VM
Save/Restore can refer to that.
But, the key point of this new approach is to use OFFSET rather than direct TSC value, this is like x86 TSC_ADJUST was introduced, and is preferred. Via this new interface,
"... Ensure that the KVM_CLOCK_REALTIME flag is set in the provided structure. KVM will advance the VM's kvmclock to account for elapsed time since recording the clock values.", therefore I think it can solve your problem, rather than modify the ancient and heuristics code at high risk.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.