Skip to contentSkip to conversation

x86 linux: prevent halted VCPUs from eating up CPU bandwidth

PATCH In review Jan Beulich 2 participants 6 messages May 29, 2006 – May 30, 2006

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86 linux: prevent halted VCPUs from eating up CPU bandwidth

Since there was no reply to an earlier inquiry regarding this, here's a patch.

include/asm-i386/bugs.h +2 -0
Index: head-2006-05-26/include/asm-i386/bugs.h
===================================================================
@@ -92,6 +92,7 @@ static void __init check_fpu(void)
 
 static void __init check_hlt(void)
 {
+#ifndef CONFIG_XEN
 	printk(KERN_INFO "Checking 'hlt' instruction... ");
 	if (!boot_cpu_data.hlt_works_ok) {
 		printk("disabled\n");
@@ -102,6 +103,7 @@ static void __init check_hlt(void)
 	halt();
 	halt();
 	printk("OK.\n");
+#endif
 }
 
 /*
include/asm-i386/mach-xen/asm/system.h +2 -2
Index: head-2006-05-26/include/asm-i386/mach-xen/asm/system.h
===================================================================
@@ -625,8 +625,8 @@ do {									\
 		preempt_enable_no_resched();				\
 } while (0)
 
-#define safe_halt()		((void)0)
-#define halt()			((void)0)
+#define safe_halt()		({ __sti(); halt(); })
+#define halt()			((void)HYPERVISOR_block())
 
 #define __save_and_cli(x)						\
 do {									\
include/asm-x86_64/mach-xen/asm/system.h +2 -2
Index: head-2006-05-26/include/asm-x86_64/mach-xen/asm/system.h
===================================================================
@@ -424,8 +424,8 @@ do {									\
 	preempt_enable_no_resched();					\
 	___x; })
 
-#define safe_halt()		((void)0)
-#define halt()			((void)0)
+#define safe_halt()		({ __sti(); halt(); })
+#define halt()			((void)HYPERVISOR_block())
 
 void cpu_idle_wait(void);
 

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw
Quoted 3 lines · Jan Beulich

Since there was no reply to an earlier inquiry regarding this, here's a patch.

safe_halt() should simply be HYPERVISOR_block() -- the hypercall implicitly reenables interrupt delivery for the calling VCPU, so no need to call __sti() explicitly.

halt() is a bit trickier -- given that it is executed (as far as I can tell) only in contexts where the CPU has reached "end of life" (it's crashed, or offlined, or shut down for some other reason) it might make sense to define halt() as VCPUOP_down. Or, just in case it is used in places with interrupts still enabled, where the CPU may have a future:
#define halt() (irqs_disabled() ? HYPERVISOR_vcpu_op(VCPUOP_down) : HYPERVISOR_block())

-- Keir

#2 lore raw
Quoted 13 lines
Quoted 1 line
Quoted 1 line

Keir Fraser <[email protected]> 29.05.06 16:59 >>>

Quoted 4 lines · Jan Beulich

Since there was no reply to an earlier inquiry regarding this, here's a patch.

safe_halt() should simply be HYPERVISOR_block() -- the hypercall implicitly reenables interrupt delivery for the calling VCPU, so no need to call __sti() explicitly.

Quoted 7 lines

halt() is a bit trickier -- given that it is executed (as far as I can tell) only in contexts where the CPU has reached "end of life" (it's crashed, or offlined, or shut down for some other reason) it might make sense to define halt() as VCPUOP_down. Or, just in case it is used in places with interrupts still enabled, where the CPU may have a future:
#define halt() (irqs_disabled() ? HYPERVISOR_vcpu_op(VCPUOP_down) :
HYPERVISOR_block())

Indeed, this was the behavior I intended, but somehow I failed to find the VCPUOP_down operation that I was looking for. Here's an updated patch.

Jan

include/asm-i386/bugs.h +2 -0
Index: head-2006-05-26/include/asm-i386/bugs.h
===================================================================
@@ -92,6 +92,7 @@ static void __init check_fpu(void)
 
 static void __init check_hlt(void)
 {
+#ifndef CONFIG_XEN
 	printk(KERN_INFO "Checking 'hlt' instruction... ");
 	if (!boot_cpu_data.hlt_works_ok) {
 		printk("disabled\n");
@@ -102,6 +103,7 @@ static void __init check_hlt(void)
 	halt();
 	halt();
 	printk("OK.\n");
+#endif
 }
 
 /*
include/asm-i386/mach-xen/asm/hypervisor.h +1 -0
Index: head-2006-05-26/include/asm-i386/mach-xen/asm/hypervisor.h
===================================================================
@@ -43,6 +43,7 @@
 #include <xen/interface/event_channel.h>
 #include <xen/interface/physdev.h>
 #include <xen/interface/sched.h>
+#include <xen/interface/vcpu.h>
 #include <xen/interface/nmi.h>
 #include <asm/ptrace.h>
 #include <asm/page.h>
include/asm-i386/mach-xen/asm/system.h +5 -2
Index: head-2006-05-26/include/asm-i386/mach-xen/asm/system.h
===================================================================
@@ -625,8 +625,11 @@ do {									\
 		preempt_enable_no_resched();				\
 } while (0)
 
-#define safe_halt()		((void)0)
-#define halt()			((void)0)
+#define safe_halt()		((void)HYPERVISOR_block())
+#define halt()			((void)					\
+	(HYPERVISOR_shared_info->vcpu_info[__vcpu_id].evtchn_upcall_mask \
+	 ? HYPERVISOR_vcpu_op(VCPUOP_down, __vcpu_id, NULL)		\
+	 : HYPERVISOR_block()))
 
 #define __save_and_cli(x)						\
 do {									\
include/asm-x86_64/mach-xen/asm/system.h +5 -2
Index: head-2006-05-26/include/asm-x86_64/mach-xen/asm/system.h
===================================================================
@@ -424,8 +424,11 @@ do {									\
 	preempt_enable_no_resched();					\
 	___x; })
 
-#define safe_halt()		((void)0)
-#define halt()			((void)0)
+#define safe_halt()		((void)HYPERVISOR_block())
+#define halt()			((void)					\
+	(HYPERVISOR_shared_info->vcpu_info[__vcpu_id].evtchn_upcall_mask \
+	 ? HYPERVISOR_vcpu_op(VCPUOP_down, __vcpu_id, NULL)		\
+	 : HYPERVISOR_block()))
 
 void cpu_idle_wait(void);
 

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#3 lore raw
Quoted 2 lines · Jan Beulich

Indeed, this was the behavior I intended, but somehow I failed to find the VCPUOP_down operation that I was looking for. Here's an updated patch.

Given that you went for the more complicated definition of halt() that calls block when event delivery is enabled, is it actually necessary to not compile the halt-checking routine? If event delivery is enabled in that function, shouldn't we return from each invocation of halt() on (at least) each timer interrupt? If not, I wonder why our behaviour there differs from native?

-- Keir

#4 lore raw
Quoted 11 lines
Quoted 1 line
Quoted 1 line

Keir Fraser <[email protected]> 30.05.06 10:49 >>>

Quoted 3 lines · Jan Beulich

Indeed, this was the behavior I intended, but somehow I failed to find the VCPUOP_down operation that I was looking
for. Here's an updated patch.

Given that you went for the more complicated definition of halt() that calls block when event delivery is enabled, is it actually necessary to not compile the halt-checking routine? If event delivery is enabled in

What halt-checking routine do you mean here?

Quoted 3 lines

that function, shouldn't we return from each invocation of halt() on (at least) each timer interrupt? If not, I wonder why our behaviour there differs from native?

Yes, I think the net effect should be similar behavior to native.

Jan

#5 lore raw
Quoted 5 lines · Jan Beulich
Quoted 3 lines

Given that you went for the more complicated definition of halt() that calls block when event delivery is enabled, is it actually necessary to not compile the halt-checking routine? If event delivery is enabled in

What halt-checking routine do you mean here?

The one called check_hlt() that you modify in your patch to not build on Xen! I don't see why that part of the patch is necessary.

-- Keir