SCHEDOP_block_masked
PATCH 1/1 SCHEDOP_block_masked
Introduce SCHEDOP_block_masked, which blocks a VCPU without enabling interrupts.
We need this as there are circumstances where we cannot take interrupts, and are polling in a tight loop. In particular in Solaris's kernel debugger console input. We don't want to burn CPU in a domU, but we're not in a situation where we can safely use SCHEDOP_block.
regards,
john
# HG changeset patch
# User [email protected]
# Node ID 55b48bec29e44c93d14c3f51f5ef58bc216a3e4b # Parent fbeb0a5b7219630839986cf4cdb1b813618cbdce Introduce SCHEDOP_block_masked.
docs/src/interface.tex +9 -7
@@ -1721,13 +1721,15 @@ \hypercall{sched\_op(unsigned long op)} Request scheduling operation from hypervisor. The options are: {\it -SCHEDOP\_yield}, {\it SCHEDOP\_block}, and {\it SCHEDOP\_shutdown}. -{\it yield} keeps the calling domain runnable but may cause a -reschedule if other domains are runnable. {\it block} removes the -calling domain from the run queue and cause is to sleeps until an -event is delivered to it. {\it shutdown} is used to end the domain's -execution; the caller can additionally specify whether the domain -should reboot, halt or suspend. +SCHEDOP\_yield}, {\it SCHEDOP\_block}, {\it SCHEDOP\_block\_masked} +and {\it SCHEDOP\_shutdown}. {\it yield} keeps the calling vcpu +runnable but may cause a reschedule if other vcpus are runnable. +{\it block} removes the calling vcpu from the run queue, enables +events, and causes it to sleep until an event is delivered to it. +{\it block\_masked} does the same, without enabling events. +{\it shutdown} is used to end the domain's execution; the caller +can additionally specify whether the domain should reboot, halt +or suspend. \end{quote} To aid the implementation of a process scheduler within a guest OS,
xen/common/schedule.c +11 -5
@@ -249,11 +249,13 @@ }/* Block the currently-executing domain until a pertinent event occurs. */ -static long do_block (void ) +static void do_block (int clear_event_mask) {struct vcpu *v = current; - v->vcpu_info->evtchn_upcall_mask =0 ; +if ( clear_event_mask ) + v->vcpu_info->evtchn_upcall_mask =0 ; +set_bit (_VCPUF_blocked, &v->vcpu_flags);/* Check for events /after/ blocking: avoids wakeup waiting race. */
@@ -266,8 +268,6 @@TRACE_2D (TRC_SCHED_BLOCK , v->domain->domain_id, v->vcpu_id);__enter_scheduler (); } - -return 0 ; }/* Voluntarily yield the processor for this allocation. */
@@ -292,7 +292,13 @@case SCHEDOP_block: { - ret =do_block (); +do_block (1 ); +break ; + } + +case SCHEDOP_block_masked: + { +do_block (0 );break ; }
xen/include/public/sched.h +12 -3
@@ -20,7 +20,7 @@* Voluntarily yield the CPU. * @arg == 0. */ -#define SCHEDOP_yield 0 +#define SCHEDOP_yield 0/* * Block execution of this VCPU until an event is received for processing.
@@ -29,13 +29,22 @@* VCPU. This avoids a "wakeup waiting" race. * @arg == 0. */ -#define SCHEDOP_block 1 +#define SCHEDOP_block 1/* * Halt execution of this domain (all VCPUs) and notify the system controller. * @arg == SHUTDOWN_??? (reason for shutdown). */ -#define SCHEDOP_shutdown 2 +#define SCHEDOP_shutdown 2 + +/* +* Block execution of this VCPU until an event is received for processing, +* without clearing the upcall mask; a subsequent pending event will not be +* delivered, but will still wake up the domain. This is intended for use in +* polling loops when "interrupts" are disabled. +* @arg == 0. +*/ +#define SCHEDOP_block_masked 3/* * Reason codes for SCHEDOP_shutdown. These may be interpreted by controller
xen/include/xen/event.h +14 -0
@@ -14,6 +14,16 @@#include <xen/smp.h> #include <asm/bitops.h> #include <asm/event.h> + +/* +* Is the domain under SCHEDOP_block_masked? If so, then we should unblock +* it even if the event channel is already pending. +*/ +static inline void evtchn_check_unblock (struct vcpu *v) +{ +if ( v->vcpu_info->evtchn_upcall_mask && (v->vcpu_flags & VCPUF_blocked) ) +vcpu_unblock (v); +}/* * EVENT-CHANNEL NOTIFICATIONS
@@ -35,6 +45,10 @@ !test_and_set_bit(0, &v->vcpu_info->evtchn_upcall_pending) ) { evtchn_notify(v); + } + else + { +evtchn_check_unblock (v); } }
Quoted 7 lines · John Levon
Introduce
SCHEDOP_block_masked, which blocks a VCPU without enabling interrupts.We need this as there are circumstances where we cannot take interrupts, and are polling in a tight loop. In particular in Solaris's kernel debugger console input. We don't want to burn CPU in a domU, but we're not in a situation where we can safely use
SCHEDOP_block.
Modified do_block() still checks event_pending, even if you do not clear the event_mask. Presumably you don't really want to do that?
-- Keir
Quoted 11 lines · Keir Fraser
Quoted 8 lines
Introduce
SCHEDOP_block_masked, which blocks a VCPU without enabling interrupts.We need this as there are circumstances where we cannot take interrupts,
and are polling in a tight loop. In particular in Solaris's kernel debugger console input. We don't want to burn CPU in a domU, but we're not in a situation where we can safely useSCHEDOP_block.Modified
do_block()still checksevent_pending, even if you do not clear theevent_mask. Presumably you don't really want to do that?
It uses event_pending() which will be zero:
76 #define event_pending(v) \
77 (!!(v)->vcpu_info->evtchn_upcall_pending & \
78 !(v)->vcpu_info->evtchn_upcall_mask)But, now you mention it, I wonder if this wouldn't be better taking an event channel to check on before sleeping (though we'd still wake on any). This would avoid the small race (which doesn't really matter in the context we've been using this).
regards
john
Quoted 10 lines · John Levon
It uses
event_pending()which will be zero:76 #define event_pending(v) \ 77 (!!(v)->vcpu_info->evtchn_upcall_pending & \ 78 !(v)->vcpu_info->evtchn_upcall_mask)But, now you mention it, I wonder if this wouldn't be better taking an event channel to check on before sleeping (though we'd still wake on any). This would avoid the small race (which doesn't really matter in the context we've been using this).
Can you give more details about the usage scenario? What ultimately allows the VCPU to continue running (presumably by clearing its evtchn_upcall_mask)?
-- Keir
Quoted 6 lines · Keir Fraser
Quoted 4 lines
But, now you mention it, I wonder if this wouldn't be better taking an event channel to check on before sleeping (though we'd still wake on any). This would avoid the small race (which doesn't really matter in the context we've been using this).
Can you give more details about the usage scenario?
The debugger is calling out to the xen console driver getchar() routine. Currently it's doing something like:
if (!check_for_xencons_char) HYPERVISOR_yield();
Thus as soon as we see something on the console ring buffer we can continue on (still inside the kernel debugger). Events must remain disabled during this entire operation. With HYPERVISOR_block_on(), this would look something like:
clear_evtchn_pending(xencons_evtchn); if (check_for_xencons_char) break; HYPERVISOR_block_on(xencons_evtchn);
Xen would check for that particular evtchn before blocking the vcpu; any event delivery would wake up the vcpu again and return to the guest domain.
The clearing of the pending bit each time is obviously needed to stop block_on() from waking immediately again (and this implies that the domain needs some other way of checking for the event delivery, as above).
Does that clear things up a little?
regards,
john
Quoted 6 lines · John Levon
The clearing of the pending bit each time is obviously needed to stop
block_on()from waking immediately again (and this implies that the domain needs some other way of checking for the event delivery, as above).Does that clear things up a little?
Yeah, if you make an updated patch I think it'll be fine for inclusion.
-- Keir