porting fixes regarding kvm-clock and lost irqs to stable qemu-kvm 0.12.4

DISCUSSION Peter Lieven 3 participants 8 messages Jun 2, 2010 – Jun 6, 2010 KVM

lore

Share & tools

lore

Peter Lieven

author

2010-06-02 15:13 UTC #0 lore raw

Hi,

I would like to get latest stable qemu-kvm (0.12.4) to a usable state regarding live-migration.

Problems are fixed in git, but there is so much new stuff that has not extensively tested and therefore I would like to stay at 0.12.4 at the moment.

Therefore I would appreciate your help regarding 2 bugs:

a) -cpu xxx,-kvmclock doesn't work in 0.12.4. It started working first after applying a patchset from Avi that is not even in git yet. What needs to be done to get it working in 0.12.4 ?

b) There is a bug in 0.12.4 which leads to lost irqs. It was reported appearing in virtio_blk (#584131) and e1000 (#585113). It is finally fixed in GIT. Can someone give me a hint with commit fixed this?

Thanks,
Peter

Marcelo Tosatti

2010-06-04 18:20 UTC #1 lore raw

Quoted 9 lines · Peter Lieven

Hi,
I would like to get latest stable qemu-kvm (0.12.4) to a usable state regarding live-migration.
Problems are fixed in git, but there is so much new stuff that has not extensively tested and therefore I would like to stay at 0.12.4 at the moment.

Quoted 5 lines

Therefore I would appreciate your help regarding 2 bugs:
a) -cpu xxx,-kvmclock doesn't work in 0.12.4. It started working first after applying a patchset from Avi that is not even in git yet. What needs to be done to get it working in 0.12.4 ?

You want to disable kvmclock? -no-kvmclock kernel option.

Note migration should be working fine since kvm.git commit afbcf7ab8d1bc8c2d04792f6d9e786e0adeb328d.

Quoted 3 lines

b) There is a bug in 0.12.4 which leads to lost irqs. It was reported appearing in virtio_blk (#584131) and e1000 (#585113). It is finally fixed in GIT. Can someone give me a hint with commit fixed this?

c3f8f61157625d0bb5bfc135047573de48fdc675.

Peter Lieven

author

2010-06-04 20:25 UTC #2 lore raw

Am 04.06.2010 um 17:31 schrieb Marcelo Tosatti:

Quoted 10 lines · Peter Lieven

Quoted 9 lines
Hi,
I would like to get latest stable qemu-kvm (0.12.4) to a usable state regarding live-migration.
Problems are fixed in git, but there is so much new stuff that has not extensively tested and therefore I would like to stay at 0.12.4 at the moment.

Quoted 8 lines

Quoted 5 lines
Therefore I would appreciate your help regarding 2 bugs:
a) -cpu xxx,-kvmclock doesn't work in 0.12.4. It started working first after applying a patchset from Avi that is not even in git yet. What needs to be done to get it working in 0.12.4 ?
You want to disable kvmclock? -no-kvmclock kernel option.

i am afraid it is not, at least the last git i tested some days ago. it's still crashing with kvm-clock.

i was looking for a generic way to disable it in the hypervisor without the need to touch every guests kernel commandline. this would be easy revertible once its all working as expected.

Quoted 2 lines

Note migration should be working fine since kvm.git commit afbcf7ab8d1bc8c2d04792f6d9e786e0adeb328d.

would it be easy to backport this to stable? hw/msix.c is looking quite different than that of 0.12.4.

Quoted 7 lines

Quoted 4 lines
b) There is a bug in 0.12.4 which leads to lost irqs. It was reported appearing in virtio_blk (#584131) and e1000 (#585113). It is finally fixed in GIT. Can someone give me a hint with commit fixed this?
c3f8f61157625d0bb5bfc135047573de48fdc675.

thanks

Quoted 1 line

Faidon Liambotis

2010-06-05 01:44 UTC #3 lore raw

Hi,

Quoted 3 lines · Peter Lieven

i was looking for a generic way to disable it in the hypervisor without the need to touch every guests kernel commandline. this would be easy revertible once its all working as expected.

This is a huge hack but for the same reason I've made an LD_PRELOAD wrapper that wraps ioctl masking the hsot kernel's kvmclock feature from userspace kvm. I'll be happy to share it if you're interested.

Works like a charm, migration has worken reliably with qemu-kvm 0.11.3 + the wrapper.

Regards,
Faidon

Peter Lieven

author

2010-06-05 14:59 UTC #4 lore raw

Am 05.06.2010 um 03:34 schrieb Faidon Liambotis:

Quoted 9 lines

Hi,
Quoted 3 lines · Peter Lieven
i was looking for a generic way to disable it in the hypervisor without the need to touch every guests kernel commandline. this would be easy revertible once its all working as expected.
This is a huge hack but for the same reason I've made an LD_PRELOAD wrapper that wraps ioctl masking the hsot kernel's kvmclock feature from userspace kvm. I'll be happy to share it if you're interested.

i would be grateful if you can sent it to me offlist. however, i came up with this hack, that seems to work also:

qemu-kvm-x86.c +1 -1

diff -ur qemu-kvm-0.12.4.orig/qemu-kvm-x86.c qemu-kvm-0.12.4/qemu-kvm-x86.c
@@ -1240,7 +1240,7 @@
 	int feature;
 } para_features[] = {
 #ifdef KVM_CAP_CLOCKSOURCE
-	{ KVM_CAP_CLOCKSOURCE, KVM_FEATURE_CLOCKSOURCE },
+//	{ KVM_CAP_CLOCKSOURCE, KVM_FEATURE_CLOCKSOURCE },
 #endif
 #ifdef KVM_CAP_NOP_IO_DELAY
 	{ KVM_CAP_NOP_IO_DELAY, KVM_FEATURE_NOP_IO_DELAY },

Quoted 11 lines

Works like a charm, migration has worken reliably with qemu-kvm 0.11.3 + the wrapper.
Regards,
Faidon
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Faidon Liambotis

2010-06-06 15:34 UTC #5 lore raw

Quoted 8 lines · Peter Lieven

Quoted 6 lines
Quoted 3 lines
i was looking for a generic way to disable it in the hypervisor without the need to touch every guests kernel commandline. this would be easy revertible once its all working as expected.
This is a huge hack but for the same reason I've made an LD_PRELOAD wrapper that wraps ioctl masking the hsot kernel's kvmclock feature from userspace kvm. I'll be happy to share it if you're interested.
i would be grateful if you can sent it to me offlist.

http://people.debian.org/~paravoid/kvm-noclock-3.tar.gz

Quoted 2 lines

however, i came up with this hack,
that seems to work also:

<snip>

Yes, that's equivalent. We didn't use that since we wanted to use our distribution's binaries and be able to upgrade without modifying the source.

Regards,
Faidon

Peter Lieven

author

2010-06-06 16:51 UTC #6 lore raw

Am 06.06.2010 um 16:53 schrieb Faidon Liambotis:

Quoted 9 lines

Quoted 8 lines · Peter Lieven
Quoted 6 lines
Quoted 3 lines
i was looking for a generic way to disable it in the hypervisor without the need to touch every guests kernel commandline. this would be easy revertible once its all working as expected.
This is a huge hack but for the same reason I've made an LD_PRELOAD wrapper that wraps ioctl masking the hsot kernel's kvmclock feature from userspace kvm. I'll be happy to share it if you're interested.
i would be grateful if you can sent it to me offlist.

Quoted 8 lines

http://people.debian.org/~paravoid/kvm-noclock-3.tar.gz
Quoted 2 lines
however, i came up with this hack,
that seems to work also:
<snip>
Yes, that's equivalent. We didn't use that since we wanted to use our distribution's binaries and be able to upgrade without modifying the source.

thank you.

i want to share this patch for stable until -cpu xxx,-kvmclock works in the stable tree.

qemu-kvm.c +1 -1

diff -ur qemu-kvm-0.12.4.orig/qemu-kvm.c qemu-kvm-0.12.4/qemu-kvm.c
@@ -54,7 +54,7 @@
 int kvm_pit = 1;
 int kvm_pit_reinject = 1;
 int kvm_nested = 0;
-
+int kvm_clock = 1;
 
 KVMState *kvm_state;
 kvm_context_t kvm_context;

qemu-kvm.h +1 -0

diff -ur qemu-kvm-0.12.4.orig/qemu-kvm.h qemu-kvm-0.12.4/qemu-kvm.h
@@ -1035,6 +1035,7 @@
 extern int kvm_pit;
 extern int kvm_pit_reinject;
 extern int kvm_nested;
+extern int kvm_clock;
 extern kvm_context_t kvm_context;
 
 struct ioperm_data {

qemu-kvm-x86.c +1 -0

diff -ur qemu-kvm-0.12.4.orig/qemu-kvm-x86.c qemu-kvm-0.12.4/qemu-kvm-x86.c
@@ -1259,6 +1259,7 @@
 	int i, features = 0;
 
 	for (i = 0; i < ARRAY_SIZE(para_features)-1; i++) {
+                if (para_features[i].cap == KVM_CAP_CLOCKSOURCE && !kvm_clock) continue;
 		if (kvm_check_extension(kvm_state, para_features[i].cap))
 			features |= (1 << para_features[i].feature);
 	}

qemu-options.hx +3 -0

diff -ur qemu-kvm-0.12.4.orig/qemu-options.hx qemu-kvm-0.12.4/qemu-options.hx
@@ -1959,6 +1959,9 @@
     "-no-kvm-pit     disable KVM kernel mode PIT\n")
 DEF("no-kvm-pit-reinjection", 0, QEMU_OPTION_no_kvm_pit_reinjection,
     "-no-kvm-pit-reinjection disable KVM kernel mode PIT interrupt reinjection\n")
+DEF("no-kvm-clock", 0, QEMU_OPTION_no_kvm_clock,
+    "-no-kvm-clock   disable KVM pvclock\n")
+
 #if defined(TARGET_I386) || defined(TARGET_X86_64) || defined(TARGET_IA64) || defined(__linux__)
 DEF("pcidevice", HAS_ARG, QEMU_OPTION_pcidevice,
     "-pcidevice host=bus:dev.func[,dma=none][,name=string]\n"

vl.c +4 -0

diff -ur qemu-kvm-0.12.4.orig/vl.c qemu-kvm-0.12.4/vl.c
@@ -5530,6 +5530,10 @@
                 kvm_pit_reinject = 0;
                 break;
             }
+            case QEMU_OPTION_no_kvm_clock: {
+                kvm_clock = 0;
+                break;
+            }
 	    case QEMU_OPTION_enable_nesting: {
 		kvm_nested = 1;
 		break;

Quoted 4 lines

Regards,
Faidon

Peter Lieven

author

2010-06-04 20:33 UTC #7 lore raw

In reply to Marcelo Tosatti

Am 04.06.2010 um 17:31 schrieb Marcelo Tosatti:

Quoted 12 lines · Peter Lieven

Quoted 11 lines
Hi,
I would like to get latest stable qemu-kvm (0.12.4) to a usable state regarding live-migration.
Problems are fixed in git, but there is so much new stuff that has not extensively tested and therefore I would like to stay at 0.12.4 at the moment.
Therefore I would appreciate your help regarding 2 bugs:

Quoted 8 lines

Quoted 3 lines
a) -cpu xxx,-kvmclock doesn't work in 0.12.4. It started working first after applying a patchset from Avi that is not even in git yet. What needs to be done to get it working in 0.12.4 ?
You want to disable kvmclock? -no-kvmclock kernel option.
Note migration should be working fine since kvm.git commit afbcf7ab8d1bc8c2d04792f6d9e786e0adeb328d.

is this patch needed in the gust or on the host?
i have 2.6.33.3 on the host system and still observed crash conditions.

Quoted 8 lines

Quoted 4 lines
b) There is a bug in 0.12.4 which leads to lost irqs. It was reported appearing in virtio_blk (#584131) and e1000 (#585113). It is finally fixed in GIT. Can someone give me a hint with commit fixed this?
c3f8f61157625d0bb5bfc135047573de48fdc675.