Skip to contentSkip to conversation

target/i386: always create kvmclock device

PATCH Applied Vitaly Kuznetsov 5 participants 14 messages Sep 17, 2020 – Sep 22, 2020

v1 of 2 Applied Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 target/i386: always create kvmclock device

QEMU's kvmclock device is only created when KVM PV feature bits for kvmclock (KVM_FEATURE_CLOCKSOURCE/KVM_FEATURE_CLOCKSOURCE2) are exposed to the guest. With 'kvm=off' cpu flag the device is not created and we don't call KVM_GET_CLOCK/KVM_SET_CLOCK upon migration. It was reported that without these call at least Hyper-V TSC page clocksouce (which can be enabled independently) gets broken after migration.

Switch to creating kvmclock QEMU device unconditionally, it seems to always make sense to call KVM_GET_CLOCK/KVM_SET_CLOCK on migration. Use KVM_CAP_ADJUST_CLOCK check instead of CPUID feature bits.

---
 hw/i386/kvm/clock.c    | 6 +-----
 target/i386/kvm.c      | 5 +++++
 target/i386/kvm_i386.h | 1 +
 3 files changed, 7 insertions(+), 5 deletions(-)
hw/i386/kvm/clock.c +1 -5
index 64283358f91d..526c9ea5172b 100644
@@ -330,11 +330,7 @@ static const TypeInfo kvmclock_info = {
 /* Note: Must be called after VCPU initialization. */
 void kvmclock_create(void)
 {
-    X86CPU *cpu = X86_CPU(first_cpu);
-
-    if (kvm_enabled() &&
-        cpu->env.features[FEAT_KVM] & ((1ULL << KVM_FEATURE_CLOCKSOURCE) |
-                                       (1ULL << KVM_FEATURE_CLOCKSOURCE2))) {
+    if (kvm_enabled() && kvm_has_adjust_clock()) {
         sysbus_create_simple(TYPE_KVM_CLOCK, -1, NULL);
     }
 }
target/i386/kvm.c +5 -0
index 4a8b3a41c1bc..20b31b65307b 100644
@@ -143,6 +143,11 @@ bool kvm_has_adjust_clock_stable(void)
     return (ret == KVM_CLOCK_TSC_STABLE);
 }
 
+bool kvm_has_adjust_clock(void)
+{
+    return kvm_check_extension(kvm_state, KVM_CAP_ADJUST_CLOCK);
+}
+
 bool kvm_has_exception_payload(void)
 {
     return has_exception_payload;
target/i386/kvm_i386.h +1 -0
index 064b8798a26c..0fce4e51d2d6 100644
@@ -34,6 +34,7 @@
 
 bool kvm_allows_irq0_override(void);
 bool kvm_has_smm(void);
+bool kvm_has_adjust_clock(void);
 bool kvm_has_adjust_clock_stable(void);
 bool kvm_has_exception_payload(void);
 void kvm_synchronize_all_tsc(void);
Signature · 1 line
2.25.4
#1 lore raw
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series seems to have some coding style problems. See output below for more information:

N/A. Internal error while reading log file

The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

#2 lore raw
Quoted 10 lines · [email protected] writes:

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series seems to have some coding style problems. See output below for more information:

N/A. Internal error while reading log file

error: copy-fd: write returned No space left on device
fatal: failed to copy file to '/var/tmp/patchew-tester-tmp-h4pn40by/src/.git/objects/pack/pack-614a2d933aa3d31ccd9f7bddf2f28bf8fae5e07c.pack': No space left on device
fatal: The remote end hung up unexpectedly

does not look like something I can fix. No idea why 'no space left' is called a 'coding style problem' btw :-)

Signature · 1 line
Vitaly
#3 lore raw
In reply to Vitaly Kuznetsov
Quoted 26 lines · Vitaly Kuznetsov

QEMU's kvmclock device is only created when KVM PV feature bits for kvmclock (KVM_FEATURE_CLOCKSOURCE/KVM_FEATURE_CLOCKSOURCE2) are exposed to the guest. With 'kvm=off' cpu flag the device is not created and we don't call KVM_GET_CLOCK/KVM_SET_CLOCK upon migration. It was reported that without these call at least Hyper-V TSC page clocksouce (which can be enabled independently) gets broken after migration.

Switch to creating kvmclock QEMU device unconditionally, it seems to always make sense to call KVM_GET_CLOCK/KVM_SET_CLOCK on migration. Use KVM_CAP_ADJUST_CLOCK check instead of CPUID feature bits.

---
 hw/i386/kvm/clock.c    | 6 +-----
 target/i386/kvm.c      | 5 +++++
 target/i386/kvm_i386.h | 1 +
 3 files changed, 7 insertions(+), 5 deletions(-)
hw/i386/kvm/clock.c +0 -0
index 64283358f91d..526c9ea5172b 100644

hw/i386/kvm/clock.c static const TypeInfo kvmclock_info = { · line 333

Quoted 9 lines
hw/i386/kvm/clock.c +1 -5
@@ -330,11 +330,7 @@ static const TypeInfo kvmclock_info = {
 /* Note: Must be called after VCPU initialization. */
 void kvmclock_create(void)
 {
-    X86CPU *cpu = X86_CPU(first_cpu);
-
-    if (kvm_enabled() &&
-        cpu->env.features[FEAT_KVM] & ((1ULL << KVM_FEATURE_CLOCKSOURCE) |
-                                       (1ULL << KVM_FEATURE_CLOCKSOURCE2))) {
+    if (kvm_enabled() && kvm_has_adjust_clock()) {

Shouldn't the old check used when machine type <= 5.1 in order to avoid migration incompatibility ?

Quoted 35 lines
hw/i386/kvm/clock.c +0 -0
         sysbus_create_simple(TYPE_KVM_CLOCK, -1, NULL);
     }
 }
target/i386/kvm.c +5 -0
index 4a8b3a41c1bc..20b31b65307b 100644
@@ -143,6 +143,11 @@ bool kvm_has_adjust_clock_stable(void)
     return (ret == KVM_CLOCK_TSC_STABLE);
 }
 
+bool kvm_has_adjust_clock(void)
+{
+    return kvm_check_extension(kvm_state, KVM_CAP_ADJUST_CLOCK);
+}
+
 bool kvm_has_exception_payload(void)
 {
     return has_exception_payload;
target/i386/kvm_i386.h +1 -0
index 064b8798a26c..0fce4e51d2d6 100644
@@ -34,6 +34,7 @@
 
 bool kvm_allows_irq0_override(void);
 bool kvm_has_smm(void);
+bool kvm_has_adjust_clock(void);
 bool kvm_has_adjust_clock_stable(void);
 bool kvm_has_exception_payload(void);
 void kvm_synchronize_all_tsc(void);
Signature · 1 line
2.25.4
Signature · 1 line
Antoine 'xdbob' Damhet
#4 lore raw
Quoted 31 lines · Antoine Damhet
Quoted 30 lines · Vitaly Kuznetsov

QEMU's kvmclock device is only created when KVM PV feature bits for kvmclock (KVM_FEATURE_CLOCKSOURCE/KVM_FEATURE_CLOCKSOURCE2) are exposed to the guest. With 'kvm=off' cpu flag the device is not created and we don't call KVM_GET_CLOCK/KVM_SET_CLOCK upon migration. It was reported that without these call at least Hyper-V TSC page clocksouce (which can be enabled independently) gets broken after migration.

Switch to creating kvmclock QEMU device unconditionally, it seems to always make sense to call KVM_GET_CLOCK/KVM_SET_CLOCK on migration. Use KVM_CAP_ADJUST_CLOCK check instead of CPUID feature bits.

---
 hw/i386/kvm/clock.c    | 6 +-----
 target/i386/kvm.c      | 5 +++++
 target/i386/kvm_i386.h | 1 +
 3 files changed, 7 insertions(+), 5 deletions(-)
hw/i386/kvm/clock.c +0 -1
index 64283358f91d..526c9ea5172b 100644
@@ -330,11 +330,7 @@ static const TypeInfo kvmclock_info = {
 /* Note: Must be called after VCPU initialization. */
 void kvmclock_create(void)
 {
-    X86CPU *cpu = X86_CPU(first_cpu);
Quoted 8 lines

hw/i386/kvm/clock.c static const TypeInfo kvmclock_info = { · line 333

Quoted 5 lines
hw/i386/kvm/clock.c +1 -4
-
-    if (kvm_enabled() &&
-        cpu->env.features[FEAT_KVM] & ((1ULL << KVM_FEATURE_CLOCKSOURCE) |
-                                       (1ULL << KVM_FEATURE_CLOCKSOURCE2))) {
+    if (kvm_enabled() && kvm_has_adjust_clock()) {

Shouldn't the old check used when machine type <= 5.1 in order to avoid migration incompatibility ?

Hm, when the check fails we just don't create the device and no error is reported, so even if we have kvmclock data in the migration stream but fail to create it migration will still succeed, right? (not a migration expert here :-)

Quoted 36 lines
Quoted 36 lines
hw/i386/kvm/clock.c +0 -0
         sysbus_create_simple(TYPE_KVM_CLOCK, -1, NULL);
     }
 }
target/i386/kvm.c +5 -0
index 4a8b3a41c1bc..20b31b65307b 100644
@@ -143,6 +143,11 @@ bool kvm_has_adjust_clock_stable(void)
     return (ret == KVM_CLOCK_TSC_STABLE);
 }
 
+bool kvm_has_adjust_clock(void)
+{
+    return kvm_check_extension(kvm_state, KVM_CAP_ADJUST_CLOCK);
+}
+
 bool kvm_has_exception_payload(void)
 {
     return has_exception_payload;
target/i386/kvm_i386.h +1 -0
index 064b8798a26c..0fce4e51d2d6 100644
@@ -34,6 +34,7 @@
 
 bool kvm_allows_irq0_override(void);
 bool kvm_has_smm(void);
+bool kvm_has_adjust_clock(void);
 bool kvm_has_adjust_clock_stable(void);
 bool kvm_has_exception_payload(void);
 void kvm_synchronize_all_tsc(void);
Signature · 1 line
2.25.4
Signature · 1 line
Vitaly
#5 lore raw
Quoted 42 lines · * Vitaly Kuznetsov ([email protected]) wrote:
Quoted 31 lines · Antoine Damhet
Quoted 30 lines · Vitaly Kuznetsov

QEMU's kvmclock device is only created when KVM PV feature bits for kvmclock (KVM_FEATURE_CLOCKSOURCE/KVM_FEATURE_CLOCKSOURCE2) are exposed to the guest. With 'kvm=off' cpu flag the device is not created and we don't call KVM_GET_CLOCK/KVM_SET_CLOCK upon migration. It was reported that without these call at least Hyper-V TSC page clocksouce (which can be enabled independently) gets broken after migration.

Switch to creating kvmclock QEMU device unconditionally, it seems to always make sense to call KVM_GET_CLOCK/KVM_SET_CLOCK on migration. Use KVM_CAP_ADJUST_CLOCK check instead of CPUID feature bits.

---
 hw/i386/kvm/clock.c    | 6 +-----
 target/i386/kvm.c      | 5 +++++
 target/i386/kvm_i386.h | 1 +
 3 files changed, 7 insertions(+), 5 deletions(-)
hw/i386/kvm/clock.c +0 -1
index 64283358f91d..526c9ea5172b 100644
@@ -330,11 +330,7 @@ static const TypeInfo kvmclock_info = {
 /* Note: Must be called after VCPU initialization. */
 void kvmclock_create(void)
 {
-    X86CPU *cpu = X86_CPU(first_cpu);
Quoted 8 lines

hw/i386/kvm/clock.c static const TypeInfo kvmclock_info = { · line 333

Quoted 5 lines
hw/i386/kvm/clock.c +1 -4
-
-    if (kvm_enabled() &&
-        cpu->env.features[FEAT_KVM] & ((1ULL << KVM_FEATURE_CLOCKSOURCE) |
-                                       (1ULL << KVM_FEATURE_CLOCKSOURCE2))) {
+    if (kvm_enabled() && kvm_has_adjust_clock()) {

Shouldn't the old check used when machine type <= 5.1 in order to avoid migration incompatibility ?

Quoted 4 lines

Hm, when the check fails we just don't create the device and no error is reported, so even if we have kvmclock data in the migration stream but fail to create it migration will still succeed, right? (not a migration expert here :-)

When the migration stream is parsed, it'll try and find a "kvmclock" device to pass the data it's reading to; if one doesn't exist it'll fail.

The other question is in the incoming direction from an older VM; you'll have a kvm clock created here, but you won't load the kvm clock state from the migration stream - what is this clock going to do?

Dave

Quoted 40 lines
Quoted 36 lines
Quoted 36 lines
hw/i386/kvm/clock.c +0 -0
         sysbus_create_simple(TYPE_KVM_CLOCK, -1, NULL);
     }
 }
target/i386/kvm.c +5 -0
index 4a8b3a41c1bc..20b31b65307b 100644
@@ -143,6 +143,11 @@ bool kvm_has_adjust_clock_stable(void)
     return (ret == KVM_CLOCK_TSC_STABLE);
 }
 
+bool kvm_has_adjust_clock(void)
+{
+    return kvm_check_extension(kvm_state, KVM_CAP_ADJUST_CLOCK);
+}
+
 bool kvm_has_exception_payload(void)
 {
     return has_exception_payload;
target/i386/kvm_i386.h +1 -0
index 064b8798a26c..0fce4e51d2d6 100644
@@ -34,6 +34,7 @@
 
 bool kvm_allows_irq0_override(void);
 bool kvm_has_smm(void);
+bool kvm_has_adjust_clock(void);
 bool kvm_has_adjust_clock_stable(void);
 bool kvm_has_exception_payload(void);
 void kvm_synchronize_all_tsc(void);
Signature · 1 line
2.25.4
Signature · 1 line
Vitaly
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#6 lore raw
Quoted 43 lines · Dr. David Alan Gilbert
Quoted 42 lines · * Vitaly Kuznetsov ([email protected]) wrote:
Quoted 39 lines · Antoine Damhet
Quoted 26 lines · Vitaly Kuznetsov

QEMU's kvmclock device is only created when KVM PV feature bits for kvmclock (KVM_FEATURE_CLOCKSOURCE/KVM_FEATURE_CLOCKSOURCE2) are exposed to the guest. With 'kvm=off' cpu flag the device is not created and we don't call KVM_GET_CLOCK/KVM_SET_CLOCK upon migration. It was reported that without these call at least Hyper-V TSC page clocksouce (which can be enabled independently) gets broken after migration.

Switch to creating kvmclock QEMU device unconditionally, it seems to always make sense to call KVM_GET_CLOCK/KVM_SET_CLOCK on migration. Use KVM_CAP_ADJUST_CLOCK check instead of CPUID feature bits.

---
 hw/i386/kvm/clock.c    | 6 +-----
 target/i386/kvm.c      | 5 +++++
 target/i386/kvm_i386.h | 1 +
 3 files changed, 7 insertions(+), 5 deletions(-)
hw/i386/kvm/clock.c +0 -0
index 64283358f91d..526c9ea5172b 100644

hw/i386/kvm/clock.c static const TypeInfo kvmclock_info = { · line 333

Quoted 9 lines
hw/i386/kvm/clock.c +1 -5
@@ -330,11 +330,7 @@ static const TypeInfo kvmclock_info = {
 /* Note: Must be called after VCPU initialization. */
 void kvmclock_create(void)
 {
-    X86CPU *cpu = X86_CPU(first_cpu);
-
-    if (kvm_enabled() &&
-        cpu->env.features[FEAT_KVM] & ((1ULL << KVM_FEATURE_CLOCKSOURCE) |
-                                       (1ULL << KVM_FEATURE_CLOCKSOURCE2))) {
+    if (kvm_enabled() && kvm_has_adjust_clock()) {

Shouldn't the old check used when machine type <= 5.1 in order to avoid migration incompatibility ?

Quoted 8 lines
Quoted 4 lines

Hm, when the check fails we just don't create the device and no error is reported, so even if we have kvmclock data in the migration stream but fail to create it migration will still succeed, right? (not a migration expert here :-)

When the migration stream is parsed, it'll try and find a "kvmclock" device to pass the data it's reading to; if one doesn't exist it'll fail.

This may happen with an older machine type when the destination is running an unfixed QEMU and the source has the fix, right? The solution would be to introduce a flag for older machine types (or for new ones) like 'kvmclock_always'.

Quoted 4 lines

The other question is in the incoming direction from an older VM; you'll have a kvm clock created here, but you won't load the kvm clock state from the migration stream - what is this clock going to do?

This is not really a problem I believe: the clock was absent on the source and things somehow worked for the guest so even if we don't initialize kvmclock properly on the destination nothing bad is expected.

Quoted 43 lines

Dave

Quoted 40 lines
Quoted 36 lines
Quoted 36 lines
hw/i386/kvm/clock.c +0 -0
         sysbus_create_simple(TYPE_KVM_CLOCK, -1, NULL);
     }
 }
target/i386/kvm.c +5 -0
index 4a8b3a41c1bc..20b31b65307b 100644
@@ -143,6 +143,11 @@ bool kvm_has_adjust_clock_stable(void)
     return (ret == KVM_CLOCK_TSC_STABLE);
 }
 
+bool kvm_has_adjust_clock(void)
+{
+    return kvm_check_extension(kvm_state, KVM_CAP_ADJUST_CLOCK);
+}
+
 bool kvm_has_exception_payload(void)
 {
     return has_exception_payload;
target/i386/kvm_i386.h +1 -0
index 064b8798a26c..0fce4e51d2d6 100644
@@ -34,6 +34,7 @@
 
 bool kvm_allows_irq0_override(void);
 bool kvm_has_smm(void);
+bool kvm_has_adjust_clock(void);
 bool kvm_has_adjust_clock_stable(void);
 bool kvm_has_exception_payload(void);
 void kvm_synchronize_all_tsc(void);
Signature · 1 line
2.25.4
Signature · 1 line
Vitaly
Signature · 1 line
Vitaly
#7 lore raw
Quoted 50 lines · * Vitaly Kuznetsov ([email protected]) wrote:
Quoted 48 lines · Dr. David Alan Gilbert
Quoted 46 lines · * Vitaly Kuznetsov ([email protected]) wrote:
Quoted 31 lines · Antoine Damhet
Quoted 30 lines · Vitaly Kuznetsov

QEMU's kvmclock device is only created when KVM PV feature bits for kvmclock (KVM_FEATURE_CLOCKSOURCE/KVM_FEATURE_CLOCKSOURCE2) are exposed to the guest. With 'kvm=off' cpu flag the device is not created and we don't call KVM_GET_CLOCK/KVM_SET_CLOCK upon migration. It was reported that without these call at least Hyper-V TSC page clocksouce (which can be enabled independently) gets broken after migration.

Switch to creating kvmclock QEMU device unconditionally, it seems to always make sense to call KVM_GET_CLOCK/KVM_SET_CLOCK on migration. Use KVM_CAP_ADJUST_CLOCK check instead of CPUID feature bits.

---
 hw/i386/kvm/clock.c    | 6 +-----
 target/i386/kvm.c      | 5 +++++
 target/i386/kvm_i386.h | 1 +
 3 files changed, 7 insertions(+), 5 deletions(-)
hw/i386/kvm/clock.c +0 -1
index 64283358f91d..526c9ea5172b 100644
@@ -330,11 +330,7 @@ static const TypeInfo kvmclock_info = {
 /* Note: Must be called after VCPU initialization. */
 void kvmclock_create(void)
 {
-    X86CPU *cpu = X86_CPU(first_cpu);
Quoted 8 lines

hw/i386/kvm/clock.c static const TypeInfo kvmclock_info = { · line 333

Quoted 5 lines
hw/i386/kvm/clock.c +1 -4
-
-    if (kvm_enabled() &&
-        cpu->env.features[FEAT_KVM] & ((1ULL << KVM_FEATURE_CLOCKSOURCE) |
-                                       (1ULL << KVM_FEATURE_CLOCKSOURCE2))) {
+    if (kvm_enabled() && kvm_has_adjust_clock()) {

Shouldn't the old check used when machine type <= 5.1 in order to avoid migration incompatibility ?

Hm, when the check fails we just don't create the device and no error is reported, so even if we have kvmclock data in the migration stream but fail to create it migration will still succeed, right? (not a migration expert here :-)

Quoted 6 lines
Quoted 3 lines

When the migration stream is parsed, it'll try and find a "kvmclock" device to pass the data it's reading to; if one doesn't exist it'll fail.

This may happen with an older machine type when the destination is running an unfixed QEMU and the source has the fix, right?

Yes I think so.

Quoted 3 lines

The solution
would be to introduce a flag for older machine types (or for new ones) like 'kvmclock_always'.

Yep sounds the normal answer.
(You might want to try it first to trigger the bug)

Quoted 7 lines
Quoted 3 lines

The other question is in the incoming direction from an older VM; you'll have a kvm clock created here, but you won't load the kvm clock state from the migration stream - what is this clock going to do?

This is not really a problem I believe: the clock was absent on the source and things somehow worked for the guest so even if we don't initialize kvmclock properly on the destination nothing bad is expected.

OK.

Dave

Quoted 47 lines
Quoted 43 lines

Dave

Quoted 40 lines
Quoted 36 lines
Quoted 36 lines
hw/i386/kvm/clock.c +0 -0
         sysbus_create_simple(TYPE_KVM_CLOCK, -1, NULL);
     }
 }
target/i386/kvm.c +5 -0
index 4a8b3a41c1bc..20b31b65307b 100644
@@ -143,6 +143,11 @@ bool kvm_has_adjust_clock_stable(void)
     return (ret == KVM_CLOCK_TSC_STABLE);
 }
 
+bool kvm_has_adjust_clock(void)
+{
+    return kvm_check_extension(kvm_state, KVM_CAP_ADJUST_CLOCK);
+}
+
 bool kvm_has_exception_payload(void)
 {
     return has_exception_payload;
target/i386/kvm_i386.h +1 -0
index 064b8798a26c..0fce4e51d2d6 100644
@@ -34,6 +34,7 @@
 
 bool kvm_allows_irq0_override(void);
 bool kvm_has_smm(void);
+bool kvm_has_adjust_clock(void);
 bool kvm_has_adjust_clock_stable(void);
 bool kvm_has_exception_payload(void);
 void kvm_synchronize_all_tsc(void);
Signature · 1 line
2.25.4
Signature · 1 line
Vitaly
Signature · 1 line
Vitaly
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#8 lore raw
Quoted 10 lines · Dr. David Alan Gilbert
Quoted 2 lines

This may happen with an older machine type when the destination is running an unfixed QEMU and the source has the fix, right?

Yes I think so.

Quoted 3 lines

The solution
would be to introduce a flag for older machine types (or for new ones) like 'kvmclock_always'.

Yep sounds the normal answer.
(You might want to try it first to trigger the bug)

Right -- unqueued it :)

Paolo

#9 lore raw
In reply to Dr. David Alan Gilbert

On Thu, Sep 17, 2020 at 06:44:10PM +0100, Dr. David Alan Gilbert wrote:

[...]

Quoted 16 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 3 lines

Shouldn't the old check used when machine type <= 5.1 in order to avoid migration incompatibility ?

Hm, when the check fails we just don't create the device and no error is reported, so even if we have kvmclock data in the migration stream but fail to create it migration will still succeed, right? (not a migration expert here :-)

Quoted 6 lines
Quoted 3 lines

When the migration stream is parsed, it'll try and find a "kvmclock" device to pass the data it's reading to; if one doesn't exist it'll fail.

This may happen with an older machine type when the destination is running an unfixed QEMU and the source has the fix, right?

Quoted 8 lines

Yes I think so.

Quoted 3 lines

The solution
would be to introduce a flag for older machine types (or for new ones) like 'kvmclock_always'.

Yep sounds the normal answer.
(You might want to try it first to trigger the bug)

So, I tried the patch and:

# patched -> patched

Everything working as expected

# patched -> unpatched

Migration failure with:

```
Unknown savevm section or instance 'kvmclock' 0. Make sure that your current VM setup matches your saved VM setup, including any hotplugged devices
load of migration failed: Invalid argument
```
# unpatched -> patched

The guest hangs upon arrival, I don't know which value is restored but something is restored (and far enough from 0 to confuse Windows).

Quoted 12 lines
Quoted 8 lines
Quoted 4 lines

The other question is in the incoming direction from an older VM; you'll have a kvm clock created here, but you won't load the kvm clock state from the migration stream - what is this clock going to do?

This is not really a problem I believe: the clock was absent on the source and things somehow worked for the guest so even if we don't initialize kvmclock properly on the destination nothing bad is expected.

OK.

Dave

[...]

Signature · 1 line
Antoine 'xdbob' Damhet
#10 lore raw
Quoted 35 lines · * Antoine Damhet ([email protected]) wrote:

On Thu, Sep 17, 2020 at 06:44:10PM +0100, Dr. David Alan Gilbert wrote:

[...]

Quoted 16 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 3 lines

Shouldn't the old check used when machine type <= 5.1 in order to avoid migration incompatibility ?

Hm, when the check fails we just don't create the device and no error is reported, so even if we have kvmclock data in the migration stream but fail to create it migration will still succeed, right? (not a migration expert here :-)

Quoted 6 lines
Quoted 3 lines

When the migration stream is parsed, it'll try and find a "kvmclock" device to pass the data it's reading to; if one doesn't exist it'll fail.

This may happen with an older machine type when the destination is running an unfixed QEMU and the source has the fix, right?

Quoted 8 lines

Yes I think so.

Quoted 3 lines

The solution
would be to introduce a flag for older machine types (or for new ones) like 'kvmclock_always'.

Yep sounds the normal answer.
(You might want to try it first to trigger the bug)

So, I tried the patch and:

# patched -> patched

Everything working as expected

Quoted 8 lines
# patched -> unpatched

Migration failure with:

```
Unknown savevm section or instance 'kvmclock' 0. Make sure that your current VM setup matches your saved VM setup, including any hotplugged devices
load of migration failed: Invalid argument
```

Right, that's what I expected and said we need to wire this fix to the machine type.

Dave

Quoted 23 lines
# unpatched -> patched

The guest hangs upon arrival, I don't know which value is restored but something is restored (and far enough from 0 to confuse Windows).

Quoted 12 lines
Quoted 8 lines
Quoted 4 lines

The other question is in the incoming direction from an older VM; you'll have a kvm clock created here, but you won't load the kvm clock state from the migration stream - what is this clock going to do?

This is not really a problem I believe: the clock was absent on the source and things somehow worked for the guest so even if we don't initialize kvmclock properly on the destination nothing bad is expected.

OK.

Dave

[...]

Signature · 1 line
Antoine 'xdbob' Damhet
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#11 lore raw
Quoted 40 lines · Dr. David Alan Gilbert
Quoted 39 lines · * Antoine Damhet ([email protected]) wrote:

On Thu, Sep 17, 2020 at 06:44:10PM +0100, Dr. David Alan Gilbert wrote:

[...]

Quoted 16 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 3 lines

Shouldn't the old check used when machine type <= 5.1 in order to avoid migration incompatibility ?

Hm, when the check fails we just don't create the device and no error is reported, so even if we have kvmclock data in the migration stream but fail to create it migration will still succeed, right? (not a migration expert here :-)

Quoted 6 lines
Quoted 3 lines

When the migration stream is parsed, it'll try and find a "kvmclock" device to pass the data it's reading to; if one doesn't exist it'll fail.

This may happen with an older machine type when the destination is running an unfixed QEMU and the source has the fix, right?

Quoted 8 lines

Yes I think so.

Quoted 3 lines

The solution
would be to introduce a flag for older machine types (or for new ones) like 'kvmclock_always'.

Yep sounds the normal answer.
(You might want to try it first to trigger the bug)

So, I tried the patch and:

# patched -> patched

Everything working as expected

# patched -> unpatched

Migration failure with:

Quoted 8 lines
Quoted 4 lines
```
Unknown savevm section or instance 'kvmclock' 0. Make sure that your current VM setup matches your saved VM setup, including any hotplugged devices
load of migration failed: Invalid argument
```

Right, that's what I expected and said we need to wire this fix to the machine type.

v2 with the idea implemented is coming. As I'm not a regular contributor to machine types, please review thoroughly :-)

Signature · 1 line
Vitaly
#12 lore raw
In reply to Dr. David Alan Gilbert

On Thu, Sep 17, 2020 at 03:42:37PM +0100, Dr. David Alan Gilbert wrote:

[...]

Quoted 9 lines
Quoted 8 lines
Quoted 3 lines

Shouldn't the old check used when machine type <= 5.1 in order to avoid migration incompatibility ?

Hm, when the check fails we just don't create the device and no error is reported, so even if we have kvmclock data in the migration stream but fail to create it migration will still succeed, right? (not a migration expert here :-)

Quoted 7 lines

When the migration stream is parsed, it'll try and find a "kvmclock" device to pass the data it's reading to; if one doesn't exist it'll fail.

The other question is in the incoming direction from an older VM; you'll have a kvm clock created here, but you won't load the kvm clock state from the migration stream - what is this clock going to do?

I guess that if the migration succeed (and the VMState keeps it's initial value) the timestamp will be restored to 0 which is the current behavior.

Quoted 2 lines

Dave

[...]

Signature · 1 line
Antoine 'xdbob' Damhet
#13 lore raw
In reply to Vitaly Kuznetsov
Quoted 60 lines · Vitaly Kuznetsov

QEMU's kvmclock device is only created when KVM PV feature bits for kvmclock (KVM_FEATURE_CLOCKSOURCE/KVM_FEATURE_CLOCKSOURCE2) are exposed to the guest. With 'kvm=off' cpu flag the device is not created and we don't call KVM_GET_CLOCK/KVM_SET_CLOCK upon migration. It was reported that without these call at least Hyper-V TSC page clocksouce (which can be enabled independently) gets broken after migration.

Switch to creating kvmclock QEMU device unconditionally, it seems to always make sense to call KVM_GET_CLOCK/KVM_SET_CLOCK on migration. Use KVM_CAP_ADJUST_CLOCK check instead of CPUID feature bits.

---
 hw/i386/kvm/clock.c    | 6 +-----
 target/i386/kvm.c      | 5 +++++
 target/i386/kvm_i386.h | 1 +
 3 files changed, 7 insertions(+), 5 deletions(-)
hw/i386/kvm/clock.c +1 -5
index 64283358f91d..526c9ea5172b 100644
@@ -330,11 +330,7 @@ static const TypeInfo kvmclock_info = {
 /* Note: Must be called after VCPU initialization. */
 void kvmclock_create(void)
 {
-    X86CPU *cpu = X86_CPU(first_cpu);
-
-    if (kvm_enabled() &&
-        cpu->env.features[FEAT_KVM] & ((1ULL << KVM_FEATURE_CLOCKSOURCE) |
-                                       (1ULL << KVM_FEATURE_CLOCKSOURCE2))) {
+    if (kvm_enabled() && kvm_has_adjust_clock()) {
         sysbus_create_simple(TYPE_KVM_CLOCK, -1, NULL);
     }
 }
target/i386/kvm.c +5 -0
index 4a8b3a41c1bc..20b31b65307b 100644
@@ -143,6 +143,11 @@ bool kvm_has_adjust_clock_stable(void)
     return (ret == KVM_CLOCK_TSC_STABLE);
 }
 
+bool kvm_has_adjust_clock(void)
+{
+    return kvm_check_extension(kvm_state, KVM_CAP_ADJUST_CLOCK);
+}
+
 bool kvm_has_exception_payload(void)
 {
     return has_exception_payload;
target/i386/kvm_i386.h +0 -0
index 064b8798a26c..0fce4e51d2d6 100644

target/i386/kvm_i386.h line 40

Quoted 7 lines
target/i386/kvm_i386.h +1 -0
@@ -34,6 +34,7 @@
 
 bool kvm_allows_irq0_override(void);
 bool kvm_has_smm(void);
+bool kvm_has_adjust_clock(void);
 bool kvm_has_adjust_clock_stable(void);
 bool kvm_has_exception_payload(void);
 void kvm_synchronize_all_tsc(void);

Queued, thanks.

Paolo