Skip to contentSkip to conversation

properly save kvm system time msr registers

PATCH In review Glauber Costa 3 participants 12 messages Oct 20, 2009 – Oct 22, 2009

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 properly save kvm system time msr registers

Currently, the msrs involved in setting up pvclock are not saved over migration and/or save/restore. This patch puts their value in special fields in our CPUState, and deal with them using vmstate.

kvm also has to account for it, by including them in the msr list for the ioctls.

This is a backport from qemu-kvm.git

---
 target-i386/cpu.h     |    2 ++
 target-i386/kvm.c     |   13 +++++++++++++
 target-i386/machine.c |    3 +++
 3 files changed, 18 insertions(+), 0 deletions(-)
target-i386/cpu.h +2 -0
index 5929d28..f589044 100644
@@ -651,6 +651,8 @@ typedef struct CPUX86State {
     target_ulong fmask;
     target_ulong kernelgsbase;
 #endif
+    uint64_t system_time_msr;
+    uint64_t wall_clock_msr;
 
     uint64_t tsc;
 
target-i386/kvm.c +13 -0
index 0e69b57..8a6fa29 100644
@@ -18,6 +18,7 @@
 #include <sys/utsname.h>
 
 #include <linux/kvm.h>
+#include <linux/kvm_para.h>
 
 #include "qemu-common.h"
 #include "sysemu.h"
@@ -500,6 +501,9 @@ static int kvm_put_msrs(CPUState *env)
         kvm_msr_entry_set(&msrs[n++], MSR_LSTAR, env->lstar);
     }
 #endif
+    kvm_msr_entry_set(&msrs[n++], MSR_KVM_SYSTEM_TIME,  env->system_time_msr);
+    kvm_msr_entry_set(&msrs[n++], MSR_KVM_WALL_CLOCK,  env->wall_clock_msr);
+
     msr_data.info.nmsrs = n;
 
     return kvm_vcpu_ioctl(env, KVM_SET_MSRS, &msr_data);
@@ -634,6 +638,9 @@ static int kvm_get_msrs(CPUState *env)
         msrs[n++].index = MSR_LSTAR;
     }
 #endif
+    msrs[n++].index = MSR_KVM_SYSTEM_TIME;
+    msrs[n++].index = MSR_KVM_WALL_CLOCK;
+
     msr_data.info.nmsrs = n;
     ret = kvm_vcpu_ioctl(env, KVM_GET_MSRS, &msr_data);
     if (ret < 0)
@@ -670,6 +677,12 @@ static int kvm_get_msrs(CPUState *env)
         case MSR_IA32_TSC:
             env->tsc = msrs[i].data;
             break;
+        case MSR_KVM_SYSTEM_TIME:
+            env->system_time_msr = msrs[i].data;
+            break;
+        case MSR_KVM_WALL_CLOCK:
+            env->wall_clock_msr = msrs[i].data;
+            break;
         }
     }
 
target-i386/machine.c +3 -0
index b13eff4..f6fe216 100644
@@ -475,6 +475,9 @@ const VMStateDescription vmstate_cpu = {
         VMSTATE_UINT64_ARRAY_V(mce_banks, CPUState, MCE_BANKS_DEF *4, 10),
         /* rdtscp */
         VMSTATE_UINT64_V(tsc_aux, CPUState, 11),
+        /* KVM pvclock msr */
+        VMSTATE_UINT64_V(system_time_msr, CPUState, 11),
+        VMSTATE_UINT64_V(wall_clock_msr, CPUState, 11),
         VMSTATE_END_OF_LIST()
     }
 };
Signature · 1 line
1.6.2.5
#1 lore raw
Quoted 6 lines · Glauber Costa

Currently, the msrs involved in setting up pvclock are not saved over migration and/or save/restore. This patch puts their value in special fields in our CPUState, and deal with them using vmstate.

kvm also has to account for it, by including them in the msr list for the ioctls.

Fails to build. Fedora 11 machine with distro kernel.

http://repo.or.cz/w/qemu/pdb.git?a=blob;f=build.log;h=5b8a2e83116dc6a74e5045e30c3efe71fa7397ef;hb=0b4f913c89b89b45d3844dc1cf321e3d94871ecb

cheers,
Gerd

#2 lore raw
Quoted 11 lines · Gerd Hoffmann
Quoted 6 lines · Glauber Costa

Currently, the msrs involved in setting up pvclock are not saved over migration and/or save/restore. This patch puts their value in special fields in our CPUState, and deal with them using vmstate.

kvm also has to account for it, by including them in the msr list for the ioctls.

Fails to build. Fedora 11 machine with distro kernel.

http://repo.or.cz/w/qemu/pdb.git?a=blob;f=build.log;h=5b8a2e83116dc6a74e5045e30c3efe71fa7397ef;hb=0b4f913c89b89b45d3844dc1cf321e3d94871ecb

strange message. Do we need --kerneldir for that? kvm_para.h should be available...

#3 lore raw
Quoted 6 lines · Glauber Costa
Quoted 5 lines · Gerd Hoffmann
Quoted 4 lines · Glauber Costa

Currently, the msrs involved in setting up pvclock are not saved over migration and/or save/restore. This patch puts their value in special fields in our CPUState, and deal with them using vmstate.

Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

kvm also has to account for it, by including them in the msr list for the ioctls.

Fails to build. Fedora 11 machine with distro kernel.

http://repo.or.cz/w/qemu/pdb.git?a=blob;f=build.log;h=5b8a2e83116dc6a74e5045e30c3efe71fa7397ef;hb=0b4f913c89b89b45d3844dc1cf321e3d94871ecb

strange message. Do we need --kerneldir for that?

Requiring --kerneldir for a successful build isn't going to fly.

Quoted 1 line

kvm_para.h should be available...

Nope:

   $ rpm -ql kernel-headers | grep kvm
   /usr/include/asm/kvm.h
   /usr/include/linux/kvm.h

Did you ever build this?
You should improve your patch qa ...

cheers,
Gerd

#4 lore raw
Quoted 19 lines · Gerd Hoffmann
Quoted 6 lines · Glauber Costa
Quoted 5 lines · Gerd Hoffmann
Quoted 4 lines · Glauber Costa

Currently, the msrs involved in setting up pvclock are not saved over migration and/or save/restore. This patch puts their value in special fields in our CPUState, and deal with them using vmstate.

Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

kvm also has to account for it, by including them in the msr list for the ioctls.

Fails to build. Fedora 11 machine with distro kernel.

http://repo.or.cz/w/qemu/pdb.git?a=blob;f=build.log;h=5b8a2e83116dc6a74e5045e30c3efe71fa7397ef;hb=0b4f913c89b89b45d3844dc1cf321e3d94871ecb

strange message. Do we need --kerneldir for that?

Requiring --kerneldir for a successful build isn't going to fly.

Quoted 1 line

kvm_para.h should be available...

Quoted 8 lines

Nope:

   $ rpm -ql kernel-headers | grep kvm
   /usr/include/asm/kvm.h
   /usr/include/linux/kvm.h

Did you ever build this?
You should improve your patch qa ...

built and tested. I was using --kerneldir all along, however.

One option I see is to redefine the values of msrs in this file. It seems it already happens for tsc anyway.

#5 lore raw
In reply to Gerd Hoffmann
Quoted 19 lines · Gerd Hoffmann
Quoted 6 lines · Glauber Costa
Quoted 5 lines · Gerd Hoffmann
Quoted 4 lines · Glauber Costa

Currently, the msrs involved in setting up pvclock are not saved over migration and/or save/restore. This patch puts their value in special fields in our CPUState, and deal with them using vmstate.

Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

kvm also has to account for it, by including them in the msr list for the ioctls.

Fails to build. Fedora 11 machine with distro kernel.

http://repo.or.cz/w/qemu/pdb.git?a=blob;f=build.log;h=5b8a2e83116dc6a74e5045e30c3efe71fa7397ef;hb=0b4f913c89b89b45d3844dc1cf321e3d94871ecb

strange message. Do we need --kerneldir for that?

Requiring --kerneldir for a successful build isn't going to fly.

Quoted 1 line

kvm_para.h should be available...

Quoted 8 lines

Nope:

   $ rpm -ql kernel-headers | grep kvm
   /usr/include/asm/kvm.h
   /usr/include/linux/kvm.h

Did you ever build this?
You should improve your patch qa ...

btw, it builds fine for me even without --kerneldir, just tested.

and the <linux/kvm_para.h> include is there since Feb08, a while now.

/me wonders what the heck is going on...

#6 lore raw
Quoted 9 lines · Glauber Costa
Quoted 6 lines
    $ rpm -ql kernel-headers | grep kvm
    /usr/include/asm/kvm.h
    /usr/include/linux/kvm.h

Did you ever build this?
You should improve your patch qa ...

btw, it builds fine for me even without --kerneldir, just tested.

and the<linux/kvm_para.h> include is there since Feb08, a while now.

Which doesn't imply "make headers" actually includes that file. Even rawhide aka soon-to-be-F12 ships no kvm_para.h header file in kernel-headers.

Just defining the msrs instead of using kvm_para.h should be fine, the numbers will not change ...

cheers,
Gerd

#7 lore raw
Quoted 9 lines · Gerd Hoffmann
Quoted 8 lines · Glauber Costa
Quoted 6 lines
    $ rpm -ql kernel-headers | grep kvm
    /usr/include/asm/kvm.h
    /usr/include/linux/kvm.h

Did you ever build this?
You should improve your patch qa ...

btw, it builds fine for me even without --kerneldir, just tested.

Quoted 8 lines
Quoted 1 line

and the<linux/kvm_para.h> include is there since Feb08, a while now.

Which doesn't imply "make headers" actually includes that file. Even rawhide aka soon-to-be-F12 ships no kvm_para.h header file in kernel-headers.

Just defining the msrs instead of using kvm_para.h should be fine, the numbers will not change ...

That's not the problem. If one removes kvm_para.h, a lot of other things will not build. For example, the capabilities list.
And it has been like this for almost a year now!

Without that header, your system should have never successfully built a qemu-kvm instance. This is what happens here, if I try to remove it:

error: ‘KVM_FEATURE_CLOCKSOURCE’ undeclared here (not in a function)
error: ‘KVM_FEATURE_NOP_IO_DELAY’ undeclared here (not in a function)
error: ‘KVM_FEATURE_MMU_OP’ undeclared here (not in a function)

So again, wth?

#8 lore raw
Quoted 8 lines
Quoted 2 lines

Just defining the msrs instead of using kvm_para.h should be fine, the numbers will not change ...

That's not the problem. If one removes kvm_para.h, a lot of other things will not build. For example, the capabilities list.
And it has been like this for almost a year now!

Without that header, your system should have never successfully built a qemu-kvm instance. This is what happens here, if I try to remove it:

qemu != qemu-kvm.

qemu-kvm ships with a copy of the kvm kernel header files. upstream qemu doesn't.

cheers,
Gerd

#9 lore raw
Quoted 6 lines · Gerd Hoffmann
Quoted 6 lines
Quoted 2 lines

Just defining the msrs instead of using kvm_para.h should be fine, the numbers will not change ...

That's not the problem. If one removes kvm_para.h, a lot of other things will not build. For example, the capabilities list.
And it has been like this for almost a year now!

Quoted 7 lines
Quoted 2 lines

Without that header, your system should have never successfully built a qemu-kvm instance. This is what happens here, if I try to remove it:

qemu != qemu-kvm.

qemu-kvm ships with a copy of the kvm kernel header files. upstream qemu doesn't.

ouch.

so this is the source of the confusion. My bad!

#10 lore raw
In reply to Gerd Hoffmann
Quoted 6 lines · Gerd Hoffmann
Quoted 6 lines
Quoted 2 lines

Just defining the msrs instead of using kvm_para.h should be fine, the numbers will not change ...

That's not the problem. If one removes kvm_para.h, a lot of other things will not build. For example, the capabilities list.
And it has been like this for almost a year now!

Quoted 7 lines
Quoted 2 lines

Without that header, your system should have never successfully built a qemu-kvm instance. This is what happens here, if I try to remove it:

qemu != qemu-kvm.

qemu-kvm ships with a copy of the kvm kernel header files. upstream qemu doesn't.

btw: I did tested qemu.git itself, but then I was using --kerneldir.

#11 lore raw
In reply to Glauber Costa
Quoted 4 lines · Glauber Costa

That's not the problem. If one removes kvm_para.h, a lot of other things will not build. For example, the capabilities list.
And it has been like this for almost a year now!

Quoted 9 lines

Without that header, your system should have never successfully built a qemu-kvm instance. This is what happens here, if I try to remove it:

error: ‘KVM_FEATURE_CLOCKSOURCE’ undeclared here (not in a function)
error: ‘KVM_FEATURE_NOP_IO_DELAY’ undeclared here (not in a function)
error: ‘KVM_FEATURE_MMU_OP’ undeclared here (not in a function)

So again, wth?

FWIW, I always build with --kerneldir. I'd like to pull in KVM headers but every time we discuss it it seems to rat hole.

Signature · 3 lines
Regards,

Anthony Liguori