KVM: x86: use smp_send_reschedule in kvm_vcpu_kick

include/linux/kvm_host.h struct kvm_vcpu { · line 78

Quoted 8 lines · Marcelo Tosatti

include/linux/kvm_host.h +0 -1

@@ -72,7 +73,6 @@ struct kvm_vcpu {
 	struct mutex mutex;
 	int   cpu;
 	struct kvm_run *run;
-	int guest_mode;
 	unsigned long requests;
 	unsigned long guest_debug;
 	int fpu_active;

The deletion will break other archs which depend on the filed.

Quoted 11 lines · Xiantao

include/linux/kvm_host.h struct kvm_vcpu { · line 78

Quoted 8 lines · Marcelo Tosatti

include/linux/kvm_host.h +0 -1

@@ -72,7 +73,6 @@ struct kvm_vcpu {
 	struct mutex mutex;
 	int   cpu;
 	struct kvm_run *run;
-	int guest_mode;
 	unsigned long requests;
 	unsigned long guest_debug;
 	int fpu_active;

The deletion will break other archs which depend on the filed.

Indeed. Will keep it, thanks.

Quoted 13 lines · Marcelo Tosatti

Quoted 11 lines · Xiantao

include/linux/kvm_host.h struct kvm_vcpu { · line 78

Quoted 8 lines · Marcelo Tosatti

include/linux/kvm_host.h +0 -1

@@ -72,7 +73,6 @@ struct kvm_vcpu {
 	struct mutex mutex;
 	int   cpu;
 	struct kvm_run *run;
-	int guest_mode;
 	unsigned long requests;
 	unsigned long guest_debug;
 	int fpu_active;

The deletion will break other archs which depend on the filed.

Quoted 1 line

Indeed. Will keep it, thanks.

The only other arch is ia64 and its kvm_vcpu_kick() should be fixed the same way. No?

KVM uses a function call IPI to cause the exit of a guest running on a physical cpu. For virtual interrupt notification there is no need to wait on IPI receival, or to execute any function.

This is exactly what the reschedule IPI does, without the overhead of function IPI. So use it instead of smp_call_function_single in kvm_vcpu_kick.

Also change the "guest_mode" variable to a bit in vcpu->requests, and use that to collapse multiple IPI's that would be issued between the first one and zeroing of guest mode.

This allows kvm_vcpu_kick to called from interrupt context.

Quoted 12 lines · Marcelo Tosatti

KVM uses a function call IPI to cause the exit of a guest running on a physical cpu. For virtual interrupt notification there is no need to wait on IPI receival, or to execute any function.

This is exactly what the reschedule IPI does, without the overhead of function IPI. So use it instead of smp_call_function_single in kvm_vcpu_kick.

Also change the "guest_mode" variable to a bit in vcpu->requests, and use that to collapse multiple IPI's that would be issued between the first one and zeroing of guest mode.

Quoted 2 lines

This allows kvm_vcpu_kick to called from interrupt context.

Looks good. The only worry I have is that we depend on smp_reschedule_interrupt() being a no-op. I guess that's a reasonable assumption though.

Quoted 16 lines · Avi Kivity

Quoted 14 lines · Marcelo Tosatti

KVM uses a function call IPI to cause the exit of a guest running on a physical cpu. For virtual interrupt notification there is no need to wait on IPI receival, or to execute any function.

This is exactly what the reschedule IPI does, without the overhead of function IPI. So use it instead of smp_call_function_single in kvm_vcpu_kick.

Also change the "guest_mode" variable to a bit in vcpu->requests, and use that to collapse multiple IPI's that would be issued between the first one and zeroing of guest mode.

This allows kvm_vcpu_kick to called from interrupt context.

Quoted 3 lines

Looks good. The only worry I have is that we depend on smp_reschedule_interrupt() being a no-op. I guess that's a reasonable assumption though.

It's a reasonable current assumption - but it might change in the future - so please also put it into the changelog that KVM will revert it or fix it differently if the scheduler grows some functionality there.

Quoted 10 lines · Ingo Molnar

Quoted 3 lines · Avi Kivity

Looks good. The only worry I have is that we depend on smp_reschedule_interrupt() being a no-op. I guess that's a reasonable assumption though.

It's a reasonable current assumption - but it might change in the future - so please also put it into the changelog that KVM will revert it or fix it differently if the scheduler grows some functionality there.

Alternatively, do something like the below, then anybody poking at that code will know to prod the KVM folks when they change anything.

[ I suspect other kvm arches will need to a similar comment ]

OK, reworked patch:
- change ia64 in addition to x86
- add comment on smp send reschedule handlers about KVM's usage

Untested on IA64.

KVM: use smp_send_reschedule in kvm_vcpu_kick

KVM uses a function call IPI to cause the exit of a guest running on a physical cpu. For virtual interrupt notification there is no need to wait on IPI receival, or to execute any function.

This is exactly what the reschedule IPI does, without the overhead of function IPI. So use it instead of smp_call_function_single in kvm_vcpu_kick.

Also change the "guest_mode" variable to a bit in vcpu->requests, and use that to collapse multiple IPI's that would be issued between the first one and zeroing of guest mode.

This allows kvm_vcpu_kick to called with interrupts disabled.

Quoted 16 lines · Marcelo Tosatti

OK, reworked patch:
- change ia64 in addition to x86
- add comment on smp send reschedule handlers about KVM's usage

Untested on IA64.

KVM: use smp_send_reschedule in kvm_vcpu_kick

KVM uses a function call IPI to cause the exit of a guest running on a physical cpu. For virtual interrupt notification there is no need to wait on IPI receival, or to execute any function.

This is exactly what the reschedule IPI does, without the overhead of function IPI. So use it instead of smp_call_function_single in kvm_vcpu_kick.

Quoted 6 lines

Also change the "guest_mode" variable to a bit in vcpu->requests, and use that to collapse multiple IPI's that would be issued between the first one and zeroing of guest mode.

This allows kvm_vcpu_kick to called with interrupts disabled.

Looks good. Will wait for Xiantao's test-n-ack before applying.

Quoted 3 lines

Looks good. Will wait for Xiantao's test-n-ack before applying.

kvm-ia64 is broken due to recent check-ins about irq-bits, and we are trying to fix it.

For this patch, ia64 has to export the symbol smp_send_reschedule before applying the patch.

Quoted 21 lines · Avi Kivity

Quoted 20 lines · Marcelo Tosatti

OK, reworked patch:
- change ia64 in addition to x86
- add comment on smp send reschedule handlers about KVM's usage

Untested on IA64.

KVM: use smp_send_reschedule in kvm_vcpu_kick

KVM uses a function call IPI to cause the exit of a guest running on a physical cpu. For virtual interrupt notification there is no need to wait on IPI receival, or to execute any function.

This is exactly what the reschedule IPI does, without the overhead of function IPI. So use it instead of smp_call_function_single in kvm_vcpu_kick.

Also change the "guest_mode" variable to a bit in vcpu->requests, and use that to collapse multiple IPI's that would be issued between the first one and zeroing of guest mode.

Quoted 4 lines

Quoted 2 lines

This allows kvm_vcpu_kick to called with interrupts disabled.

Looks good. Will wait for Xiantao's test-n-ack before applying.

kvm-ia64 is broken due to recent check-ins about irq-bits, and we are trying to fix it. For this patch, ia64 has to export the symbol smp_send_reschedule before applying the patch.

Quoted 21 lines · Avi Kivity

Quoted 20 lines · Marcelo Tosatti

OK, reworked patch:
- change ia64 in addition to x86
- add comment on smp send reschedule handlers about KVM's usage

Untested on IA64.

KVM: use smp_send_reschedule in kvm_vcpu_kick

KVM uses a function call IPI to cause the exit of a guest running on a physical cpu. For virtual interrupt notification there is no need to wait on IPI receival, or to execute any function.

This is exactly what the reschedule IPI does, without the overhead of function IPI. So use it instead of smp_call_function_single in kvm_vcpu_kick.

Also change the "guest_mode" variable to a bit in vcpu->requests, and use that to collapse multiple IPI's that would be issued between the first one and zeroing of guest mode.

Quoted 4 lines

Quoted 2 lines

This allows kvm_vcpu_kick to called with interrupts disabled.

Looks good. Will wait for Xiantao's test-n-ack before applying.

kvm-ia64 is broken due to recent check-ins about irq-bits, and we are trying to fix it. For this patch, ia64 has to export the symbol smp_send_reschedule before applying the patch.

We also hacked the source like the patch. But the issue is not caused by it. We are still trying to figure the reason out. Thanks! Xiantao

Quoted 4 lines · Xiantao

We also hacked the source like the patch. But the issue is not caused by it. We are still trying to figure the reason out. Thanks! Xiantao

With the patch below I am able to compile kvm-userspace on IA64 and run linux guest.
Network doesn't work since there is some kind of
problem with PCI interrupts. They are not delivered to a guest.

qemu/target-ia64/cpu.h +2 -1

 #define ELF_MACHINE	EM_IA_64
 

@@ -40,13 +40,14 @@
 #include "cpu-defs.h"
 
 #include "softfloat.h"
+#define CPUState struct CPUIA64State
+
 typedef struct CPUIA64State {
     CPU_COMMON;
     uint32_t hflags;
     int mp_state;
 } CPUIA64State;
 
-#define CPUState CPUIA64State
 #define cpu_gen_code cpu_ia64_gen_code
 #define cpu_init cpu_ia64_init
 #define cpu_signal_handler cpu_ia64_signal_handler

Quoted 6 lines · Gleb Natapov

Quoted 4 lines · Xiantao

We also hacked the source like the patch. But the issue is not caused by it. We are still trying to figure the reason out. Thanks! Xiantao

Quoted 4 lines

With the patch below I am able to compile kvm-userspace on IA64 and run linux guest.
Network doesn't work since there is some kind of
problem with PCI interrupts. They are not delivered to a guest.

Network should work before kvm-82, and not sure latest irq-bits changes has some side-impact for ipf side. You can reference ioapic_map_irq in ipf.c, and the mapping should match the PRT of bios.

qemu/target-ia64/cpu.h line 32

Quoted 6 lines

qemu/target-ia64/cpu.h +1 -1

@@ -29,7 +29,7 @@
 
 #define TARGET_LONG_BITS 64
 
-#define TARGET_PAGE_BITS 16
+#define TARGET_PAGE_BITS 14

If the host kernel's page size is 64k ,bits should be 16, and 16k with 14. We may reference kernel's page size here.