vm_event: fix race-condition when disabling monitor events
cover vm_event: fix race-condition when disabling monitor events
For the last couple years we have received numerous reports from users of monitor vm_events of spurious guest crashes when using events. In particular, it has observed that the problem occurs when vm_events are being disabled. The nature of the guest crash varied widely and has only occured occasionally. This made debugging the issue particularly hard. We had discussions about this issue even here on the xen-devel mailinglist with no luck figuring it out.
The bug has now been identified as a race-condition between register event handling and disabling the vm_event interface.
Patch 96760e2fba100d694300a81baddb5740e0f8c0ee, "vm_event: deny register writes if refused by vm_event reply" is the patch that introduced the error. In this patch emulation of register write events can be postponed until the corresponding vm_event handler decides whether to allow such write to take place. Unfortunately this can only be implemented by performing the deny/allow step when the vCPU gets scheduled. Due to that postponed emulation of the event if the user decides to pause the VM in the vm_event handler and then disable events, the entire emulation step is skipped the next time the vCPU is resumed. Even if the user doesn't pause during the vm_event handling but exits immediately and disables vm_event, the situation becomes racey as disabling vm_event may succeed before the guest's vCPUs get scheduled with the pending emulation task. This has been particularly the case with VMs that have several vCPUs as after the VM is unpaused it may actually take a long time before all vCPUs get scheduled.
The only solution currently is to poll each vCPU before vm_events are disabled to verify they had been scheduled. The following patches resolve this issue in a much nicer way.
Patch 1 adds an option to the monitor_op domctl that needs to be specified if
the user wants to actually use the postponed register-write handling mechanism. If that option is not specified then handling is performed the same way as before patch 96760e2fba100d694300a81baddb5740e0f8c0ee.
Patch 2 performs sanity checking when disabling vm_events to determine whether
its safe to free all vm_event structures. The vCPUs still get unpaused to allow them to get scheduled and perform any of their pending operations, but otherwise an -EAGAIN error is returned signaling to the user that they need to wait and try again disabling the interface.
Patch 3 adds a vm_event specifically to signal to the user when it is safe to
continue disabling the interface.
Shout out to our friends at CERT.pl for stumbling upon a crucial piece of information that lead to finally squashing this nasty bug.
Tamas K Lengyel (3):
xen/monitor: Control register values
xen/vm_event: add vm_event_check_pending_op
xen/vm_event: Add safe to disable vm_event
xen/arch/x86/hvm/hvm.c | 69 +++++++++++++++++++++++-------- xen/arch/x86/hvm/monitor.c | 14 +++++++ xen/arch/x86/monitor.c | 23 ++++++++++- xen/arch/x86/vm_event.c | 23 +++++++++++ xen/common/vm_event.c | 17 ++++++-- xen/include/asm-arm/vm_event.h | 7 ++++ xen/include/asm-x86/domain.h | 2 + xen/include/asm-x86/hvm/monitor.h | 1 + xen/include/asm-x86/vm_event.h | 2 + xen/include/public/domctl.h | 3 ++ xen/include/public/vm_event.h | 8 ++++ 11 files changed, 147 insertions(+), 22 deletions(-)
Signature · 1 line
2.26.1
PATCH 1/3 xen/monitor: Control register values
Extend the monitor_op domctl to include option that enables controlling what values certain registers are permitted to hold by a monitor subscriber.
--- xen/arch/x86/hvm/hvm.c | 31 +++++++++++++++++++------------ xen/arch/x86/monitor.c | 10 +++++++++- xen/include/asm-x86/domain.h | 1 + xen/include/public/domctl.h | 1 + 4 files changed, 30 insertions(+), 13 deletions(-)
xen/arch/x86/hvm/hvm.c +19 -12
@@ -2263,9 +2263,10 @@ int hvm_set_cr0(unsigned long value, bool may_defer) {ASSERT (v->arch.vm_event); -if (hvm_monitor_crX (CR0 , value, old_value) ) +if (hvm_monitor_crX (CR0 , value, old_value) && + v->domain->arch.monitor.control_register_values ) { -/* The actual write will occur in hvm_do_resume(), if permitted. */ +/* The actual write will occur in hvm_do_resume, if permitted. */ v->arch.vm_event->write_data.do_write.cr0 =1 ; v->arch.vm_event->write_data.cr0 = value;
@@ -2362,9 +2363,10 @@ int hvm_set_cr3(unsigned long value, bool may_defer) {ASSERT (v->arch.vm_event); -if (hvm_monitor_crX (CR3 , value, old) ) +if (hvm_monitor_crX (CR3 , value, old) && + v->domain->arch.monitor.control_register_values ) { -/* The actual write will occur in hvm_do_resume(), if permitted. */ +/* The actual write will occur in hvm_do_resume, if permitted. */ v->arch.vm_event->write_data.do_write.cr3 =1 ; v->arch.vm_event->write_data.cr3 = value;
@@ -2443,9 +2445,10 @@ int hvm_set_cr4(unsigned long value, bool may_defer) {ASSERT (v->arch.vm_event); -if (hvm_monitor_crX (CR4 , value, old_cr) ) +if (hvm_monitor_crX (CR4 , value, old_cr) && + v->domain->arch.monitor.control_register_values ) { -/* The actual write will occur in hvm_do_resume(), if permitted. */ +/* The actual write will occur in hvm_do_resume, if permitted. */ v->arch.vm_event->write_data.do_write.cr4 =1 ; v->arch.vm_event->write_data.cr4 = value;
@@ -3587,13 +3590,17 @@ int hvm_msr_write_intercept(unsigned int msr, uint64_t msr_content,ASSERT (v->arch.vm_event); -/* The actual write will occur in hvm_do_resume() (if permitted). */ - v->arch.vm_event->write_data.do_write.msr =1 ; - v->arch.vm_event->write_data.msr = msr; - v->arch.vm_event->write_data.value = msr_content; -hvm_monitor_msr (msr, msr_content, msr_old_content); -return X86EMUL_OKAY ; + +if ( v->domain->arch.monitor.control_register_values ) + { +/* The actual write will occur in hvm_do_resume, if permitted. */ + v->arch.vm_event->write_data.do_write.msr =1 ; + v->arch.vm_event->write_data.msr = msr; + v->arch.vm_event->write_data.value = msr_content; + +return X86EMUL_OKAY ; + } }if ( (ret =guest_wrmsr (v, msr, msr_content)) !=X86EMUL_UNHANDLEABLE )
xen/arch/x86/monitor.c +9 -1
@@ -144,7 +144,15 @@ int arch_monitor_domctl_event(struct domain *d, struct xen_domctl_monitor_op *mop) {struct arch_domain *ad = &d->arch; -bool requested_status = (XEN_DOMCTL_MONITOR_OP_ENABLE == mop->op); +bool requested_status; + +if (XEN_DOMCTL_MONITOR_OP_CONTROL_REGISTERS == mop->op ) + { + ad->monitor.control_register_values = true; +return 0 ; + } + + requested_status = (XEN_DOMCTL_MONITOR_OP_ENABLE == mop->op);switch ( mop->event ) {
xen/include/asm-x86/domain.h +1 -0
@@ -416,6 +416,7 @@ struct arch_domain* This is used to filter out pagefaults. */ unsigned int inguest_pagefault_disabled : 1; + unsigned int control_register_values : 1;struct monitor_msr_bitmap *msr_bitmap;uint64_t write_ctrlreg_mask[4 ]; } monitor;
xen/include/public/domctl.h +1 -0
@@ -1025,6 +1025,7 @@ struct xen_domctl_psr_cmt_op {#define XEN_DOMCTL_MONITOR_OP_DISABLE 1#define XEN_DOMCTL_MONITOR_OP_GET_CAPABILITIES 2#define XEN_DOMCTL_MONITOR_OP_EMULATE_EACH_REP 3 +#define XEN_DOMCTL_MONITOR_OP_CONTROL_REGISTERS 4#define XEN_DOMCTL_MONITOR_EVENT_WRITE_CTRLREG 0#define XEN_DOMCTL_MONITOR_EVENT_MOV_TO_MSR 1
Signature · 1 line
2.26.1
Re: [PATCH 1/3] xen/monitor: Control register values
Quoted 3 lines · Tamas K Lengyel
Extend the
monitor_opdomctl to include option that enables controlling what values certain registers are permitted to hold by a monitor subscriber.
This needs a bit more explanation, especially for those of us who aren't that introspection savvy. For example, from the text here I didn't expect a simple bool control, but something where actual (register) values get passed back and forth.
xen/arch/x86/hvm/hvm.c int hvm_set_cr0(unsigned long value, bool may_defer) · line 2269
Quoted 12 lines
xen/arch/x86/hvm/hvm.c +3
-2@@ -2263,9 +2263,10 @@ int hvm_set_cr0(unsigned long value, bool may_defer) {ASSERT (v->arch.vm_event); -if (hvm_monitor_crX (CR0 , value, old_value) ) +if (hvm_monitor_crX (CR0 , value, old_value) && + v->domain->arch.monitor.control_register_values ) { -/* The actual write will occur in hvm_do_resume(), if permitted. */ +/* The actual write will occur in hvm_do_resume, if permitted. */
Please can you leave alone this and the similar comments below. And for consistency _add_ parentheses to the one new instance you add?
xen/arch/x86/monitor.c int arch_monitor_domctl_event(struct domain *d, · line 151
Quoted 12 lines
xen/arch/x86/monitor.c +5
-1@@ -144,7 +144,15 @@ int arch_monitor_domctl_event(struct domain *d, struct xen_domctl_monitor_op *mop) {struct arch_domain *ad = &d->arch; -bool requested_status = (XEN_DOMCTL_MONITOR_OP_ENABLE == mop->op); +bool requested_status; + +if (XEN_DOMCTL_MONITOR_OP_CONTROL_REGISTERS == mop->op ) + { + ad->monitor.control_register_values = true;
And there's no way to clear this flag again?
Jan
Re: [PATCH 1/3] xen/monitor: Control register values
Quoted 17 lines · Jan Beulich
Quoted 3 lines · Tamas K Lengyel
Extend the
monitor_opdomctl to include option that enables controlling what values certain registers are permitted to hold by a monitor subscriber.This needs a bit more explanation, especially for those of us who aren't that introspection savvy. For example, from the text here I didn't expect a simple bool control, but something where actual (register) values get passed back and forth.
Quoted 6 lines
xen/arch/x86/hvm/hvm.c +0
-0@@ -2263,9 +2263,10 @@ int hvm_set_cr0(unsigned long value, bool may_defer) {ASSERT (v->arch.vm_event);
Quoted 10 lines
xen/arch/x86/hvm/hvm.c int hvm_set_cr0(unsigned long value, bool may_defer) · line 2269
Quoted 6 lines
xen/arch/x86/hvm/hvm.c +3
-2-if (hvm_monitor_crX (CR0 , value, old_value) ) +if (hvm_monitor_crX (CR0 , value, old_value) && + v->domain->arch.monitor.control_register_values ) { -/* The actual write will occur in hvm_do_resume(), if permitted. */ +/* The actual write will occur in hvm_do_resume, if permitted. */ Please can you leave alone this and the similar comments below. And for consistency _add_ parentheses to the one new instance you add?
I changed to because now it doesn't fit into the 80-line limit below, and then changed it everywhere _for_ consistency.
Quoted 15 lines
xen/arch/x86/monitor.c int arch_monitor_domctl_event(struct domain *d, · line 151
Quoted 13 lines
xen/arch/x86/monitor.c +5
-1@@ -144,7 +144,15 @@ int arch_monitor_domctl_event(struct domain *d, struct xen_domctl_monitor_op *mop) {struct arch_domain *ad = &d->arch; -bool requested_status = (XEN_DOMCTL_MONITOR_OP_ENABLE == mop->op); +bool requested_status; + +if (XEN_DOMCTL_MONITOR_OP_CONTROL_REGISTERS == mop->op ) + { + ad->monitor.control_register_values = true;And there's no way to clear this flag again?
There is. Disable the monitor vm_event interface and reinitialize.
Tamas
Re: [PATCH 1/3] xen/monitor: Control register values
Quoted 29 lines · Tamas K Lengyel
Quoted 17 lines · Jan Beulich
Quoted 3 lines · Tamas K Lengyel
Extend the
monitor_opdomctl to include option that enables controlling what values certain registers are permitted to hold by a monitor subscriber.This needs a bit more explanation, especially for those of us who aren't that introspection savvy. For example, from the text here I didn't expect a simple bool control, but something where actual (register) values get passed back and forth.
Quoted 6 lines
xen/arch/x86/hvm/hvm.c +0
-0@@ -2263,9 +2263,10 @@ int hvm_set_cr0(unsigned long value, bool may_defer) {ASSERT (v->arch.vm_event);Quoted 10 lines
xen/arch/x86/hvm/hvm.c int hvm_set_cr0(unsigned long value, bool may_defer) · line 2269
Quoted 6 lines
xen/arch/x86/hvm/hvm.c +3
-2-if (hvm_monitor_crX (CR0 , value, old_value) ) +if (hvm_monitor_crX (CR0 , value, old_value) && + v->domain->arch.monitor.control_register_values ) { -/* The actual write will occur in hvm_do_resume(), if permitted. */ +/* The actual write will occur in hvm_do_resume, if permitted. */ Please can you leave alone this and the similar comments below. And for consistency _add_ parentheses to the one new instance you add?
Quoted 2 lines
I changed to because now it doesn't fit into the 80-line limit below, and then changed it everywhere _for_ consistency.
The 80-char limit is easy to deal with - wrap the line.
Quoted 15 lines
Quoted 6 lines
Quoted 6 lines
xen/arch/x86/monitor.c +0
-0@@ -144,7 +144,15 @@ int arch_monitor_domctl_event(struct domain *d, struct xen_domctl_monitor_op *mop) {struct arch_domain *ad = &d->arch;Quoted 8 lines
xen/arch/x86/monitor.c int arch_monitor_domctl_event(struct domain *d, · line 151
Quoted 6 lines
xen/arch/x86/monitor.c +5
-1-bool requested_status = (XEN_DOMCTL_MONITOR_OP_ENABLE == mop->op); +bool requested_status; + +if (XEN_DOMCTL_MONITOR_OP_CONTROL_REGISTERS == mop->op ) + { + ad->monitor.control_register_values = true;And there's no way to clear this flag again?
Quoted 1 line
There is. Disable the monitor
vm_eventinterface and reinitialize.
Quite heavy handed, isn't it?
Jan
Re: [PATCH 1/3] xen/monitor: Control register values
Quoted 51 lines · Jan Beulich
Quoted 29 lines · Tamas K Lengyel
Quoted 17 lines · Jan Beulich
Quoted 3 lines · Tamas K Lengyel
Extend the
monitor_opdomctl to include option that enables controlling what values certain registers are permitted to hold by a monitor subscriber.This needs a bit more explanation, especially for those of us who aren't that introspection savvy. For example, from the text here I didn't expect a simple bool control, but something where actual (register) values get passed back and forth.
Quoted 6 lines
xen/arch/x86/hvm/hvm.c +0
-0@@ -2263,9 +2263,10 @@ int hvm_set_cr0(unsigned long value, bool may_defer) {ASSERT (v->arch.vm_event);Quoted 10 lines
xen/arch/x86/hvm/hvm.c int hvm_set_cr0(unsigned long value, bool may_defer) · line 2269
Quoted 6 lines
xen/arch/x86/hvm/hvm.c +3
-2-if (hvm_monitor_crX (CR0 , value, old_value) ) +if (hvm_monitor_crX (CR0 , value, old_value) && + v->domain->arch.monitor.control_register_values ) { -/* The actual write will occur in hvm_do_resume(), if permitted. */ +/* The actual write will occur in hvm_do_resume, if permitted. */ Please can you leave alone this and the similar comments below. And for consistency _add_ parentheses to the one new instance you add?
Quoted 2 lines
I changed to because now it doesn't fit into the 80-line limit below, and then changed it everywhere _for_ consistency.
The 80-char limit is easy to deal with - wrap the line.
Quoted 15 lines
Quoted 14 lines
xen/arch/x86/monitor.c int arch_monitor_domctl_event(struct domain *d, · line 151
Quoted 12 lines
xen/arch/x86/monitor.c +5
-1@@ -144,7 +144,15 @@ int arch_monitor_domctl_event(struct domain *d, struct xen_domctl_monitor_op *mop) {struct arch_domain *ad = &d->arch; -bool requested_status = (XEN_DOMCTL_MONITOR_OP_ENABLE == mop->op); +bool requested_status; + +if (XEN_DOMCTL_MONITOR_OP_CONTROL_REGISTERS == mop->op ) + { + ad->monitor.control_register_values = true;And there's no way to clear this flag again?
Quoted 3 lines
Quoted 1 line
There is. Disable the monitor
vm_eventinterface and reinitialize.Quite heavy handed, isn't it?
Not really. It's perfectly suitable for what its used for. You either need this feature for the duration of your monitoring or you don't. There is no in-between.
Tamas
PATCH 2/3 xen/vm_event: add vm_event_check_pending_op
In reply to Tamas K LengyelPerform sanity checking when shutting vm_event down to determine whether it is safe to do so. Error out with -EAGAIN in case pending operations have been found for the domain.
--- xen/arch/x86/vm_event.c | 23 +++++++++++++++++++++++ xen/common/vm_event.c | 17 ++++++++++++++--- xen/include/asm-arm/vm_event.h | 7 +++++++ xen/include/asm-x86/vm_event.h | 2 ++ 4 files changed, 46 insertions(+), 3 deletions(-)
xen/arch/x86/vm_event.c +23 -0
@@ -297,6 +297,29 @@ void vm_event_emulate_check(struct vcpu *v, vm_event_response_t *rsp) }; } +bool vm_event_check_pending_op (struct vcpu *v) +{ +struct monitor_write_data *w = &v->arch.vm_event->write_data; + +if ( !v->arch.vm_event->sync_event ) +return false; + +if ( w->do_write.cr0 ) +return true; +if ( w->do_write.cr3 ) +return true; +if ( w->do_write.cr4 ) +return true; +if ( w->do_write.msr ) +return true; +if ( v->arch.vm_event->set_gprs ) +return true; +if ( v->arch.vm_event->emulate_flags ) +return true; + +return false; +} +/* * Local variables: * mode: C
xen/common/vm_event.c +14 -3
@@ -183,6 +183,7 @@ static int vm_event_disable(struct domain *d, struct vm_event_domain **p_ved)if (vm_event_check_ring (ved) ) {struct vcpu *v; +bool pending_op = false;spin_lock (&ved->lock);
@@ -192,9 +193,6 @@ static int vm_event_disable(struct domain *d, struct vm_event_domain **p_ved)return -EBUSY ; } -/* Free domU's event channel and leave the other one unbound */ -free_xen_event_channel (d, ved->xen_port); -/* Unblock all vCPUs */ for_each_vcpu ( d, v ) {
@@ -203,8 +201,21 @@ static int vm_event_disable(struct domain *d, struct vm_event_domain **p_ved)vcpu_unpause (v); ved->blocked--; } + +if (vm_event_check_pending_op (v) ) + pending_op = true; } +/* vm_event ops are still pending until vCPUs get scheduled */ +if ( pending_op ) + { +spin_unlock (&ved->lock); +return -EAGAIN ; + } + +/* Free domU's event channel and leave the other one unbound */ +free_xen_event_channel (d, ved->xen_port); +destroy_ring_for_helper (&ved->ring_page, ved->ring_pg_struct);vm_event_cleanup_domain (d);
xen/include/asm-arm/vm_event.h +7 -0
@@ -58,4 +58,11 @@ void vm_event_sync_event(struct vcpu *v, bool value) /* Not supported on ARM. */ } +static inline +bool vm_event_check_pending_op (struct vcpu *v) +{ +/* Not supported on ARM. */ +return false; +} +#endif /* __ASM_ARM_VM_EVENT_H__ */
xen/include/asm-x86/vm_event.h +2 -0
@@ -54,4 +54,6 @@ void vm_event_emulate_check(struct vcpu *v, vm_event_response_t *rsp);void vm_event_sync_event (struct vcpu *v,bool value); +bool vm_event_check_pending_op (struct vcpu *v); +#endif /* __ASM_X86_VM_EVENT_H__ */
Signature · 1 line
2.26.1
PATCH 3/3 xen/vm_event: Add safe to disable vm_event
In reply to Tamas K LengyelInstead of having to repeatedly try to disable vm_events, request a specific vm_event to be sent when the domain is safe to continue with shutting down the vm_event interface.
--- xen/arch/x86/hvm/hvm.c | 38 ++++++++++++++++++++++++++----- xen/arch/x86/hvm/monitor.c | 14 ++++++++++++ xen/arch/x86/monitor.c | 13 +++++++++++ xen/include/asm-x86/domain.h | 1 + xen/include/asm-x86/hvm/monitor.h | 1 + xen/include/public/domctl.h | 2 ++ xen/include/public/vm_event.h | 8 +++++++ 7 files changed, 71 insertions(+), 6 deletions(-)
xen/arch/x86/hvm/hvm.c +32 -6
@@ -563,15 +563,41 @@ void hvm_do_resume(struct vcpu *v) v->arch.hvm.inject_event.vector =HVM_EVENT_VECTOR_UNSET ; } -if (unlikely (v->arch.vm_event) && v->arch.monitor.next_interrupt_enabled ) +if (unlikely (v->arch.vm_event) ) { -struct x86_event info; +struct domain *d = v->domain; + +if ( v->arch.monitor.next_interrupt_enabled ) + { +struct x86_event info; + +if (hvm_get_pending_event (v, &info) ) + { +hvm_monitor_interrupt (info.vector, info.type, info.error_code, + info.cr2); + v->arch.monitor.next_interrupt_enabled = false; + } + } -if (hvm_get_pending_event (v, &info) ) +if ( d->arch.monitor.safe_to_disable ) { -hvm_monitor_interrupt (info.vector, info.type, info.error_code, - info.cr2); - v->arch.monitor.next_interrupt_enabled = false; +struct vcpu *check_vcpu; +bool pending_op = false; + + for_each_vcpu ( d, check_vcpu ) + { +if (vm_event_check_pending_op (check_vcpu) ) + { + pending_op = true; +break ; + } + } + +if ( !pending_op ) + { +hvm_monitor_safe_to_disable (); + d->arch.monitor.safe_to_disable = false; + } } } }
xen/arch/x86/hvm/monitor.c +14 -0
@@ -300,6 +300,20 @@ bool hvm_monitor_check_p2m(unsigned long gla, gfn_t gfn, uint32_t pfec,return monitor_traps (curr, true, &req) >=0 ; } +bool hvm_monitor_safe_to_disable (void ) +{ +struct vcpu *curr = current; +struct arch_domain *ad = &curr->domain->arch; +vm_event_request_t req = {}; + +if ( !ad->monitor.safe_to_disable ) +return 0 ; + + req.reason =VM_EVENT_REASON_SAFE_TO_DISABLE ; + +return monitor_traps (curr,0 , &req); +} +/* * Local variables: * mode: C
xen/arch/x86/monitor.c +13 -0
@@ -339,6 +339,19 @@ int arch_monitor_domctl_event(struct domain *d,break ; } +case XEN_DOMCTL_MONITOR_EVENT_SAFE_TO_DISABLE : + { +bool old_status = ad->monitor.safe_to_disable; + +if (unlikely (old_status == requested_status) ) +return -EEXIST ; + +domain_pause (d); + ad->monitor.safe_to_disable = requested_status; +domain_unpause (d); +break ; + } +default :/* * Should not be reached unless arch_monitor_get_capabilities() is
xen/include/asm-x86/domain.h +1 -0
@@ -417,6 +417,7 @@ struct arch_domain*/ unsigned int inguest_pagefault_disabled : 1; unsigned int control_register_values : 1; + unsigned int safe_to_disable : 1;struct monitor_msr_bitmap *msr_bitmap;uint64_t write_ctrlreg_mask[4 ]; } monitor;
xen/include/asm-x86/hvm/monitor.h +1 -0
@@ -52,6 +52,7 @@ bool hvm_monitor_emul_unimplemented(void);bool hvm_monitor_check_p2m (unsigned long gla,gfn_t gfn,uint32_t pfec,uint16_t kind); +bool hvm_monitor_safe_to_disable (void );#endif /* __ASM_X86_HVM_MONITOR_H__ */
xen/include/public/domctl.h +2 -0
@@ -1040,6 +1040,8 @@ struct xen_domctl_psr_cmt_op {#define XEN_DOMCTL_MONITOR_EVENT_EMUL_UNIMPLEMENTED 10/* Enabled by default */ #define XEN_DOMCTL_MONITOR_EVENT_INGUEST_PAGEFAULT 11 +/* Always async, disables automaticaly on first event */ +#define XEN_DOMCTL_MONITOR_EVENT_SAFE_TO_DISABLE 12struct xen_domctl_monitor_op {uint32_t op;/* XEN_DOMCTL_MONITOR_OP_* */
xen/include/public/vm_event.h +8 -0
@@ -159,6 +159,14 @@#define VM_EVENT_REASON_DESCRIPTOR_ACCESS 13/* Current instruction is not implemented by the emulator */ #define VM_EVENT_REASON_EMUL_UNIMPLEMENTED 14 +/* +* When shutting down vm_event it may not be immediately safe to complete the +* process as some vCPUs may be pending synchronization. This async event +* type can be used to receive a notification when its safe to finish disabling +* the vm_event interface. All other event types need to be disabled before +* registering to this one. +*/ +#define VM_EVENT_REASON_SAFE_TO_DISABLE 15/* Supported values for the vm_event_write_ctrlreg index. */ #define VM_EVENT_X86_CR0 0
Signature · 1 line
2.26.1
Re: [PATCH 3/3] xen/vm_event: Add safe to disable vm_event
xen/arch/x86/hvm/hvm.c void hvm_do_resume(struct vcpu *v) · line 568
Quoted 11 lines · Tamas K Lengyel
xen/arch/x86/hvm/hvm.c +2
-2@@ -563,15 +563,41 @@ void hvm_do_resume(struct vcpu *v) v->arch.hvm.inject_event.vector =HVM_EVENT_VECTOR_UNSET ; } -if (unlikely (v->arch.vm_event) && v->arch.monitor.next_interrupt_enabled ) +if (unlikely (v->arch.vm_event) ) { -struct x86_event info; +struct domain *d = v->domain;
const
Quoted 9 lines
xen/arch/x86/hvm/hvm.c +9
-0+if ( v->arch.monitor.next_interrupt_enabled ) + { +struct x86_event info; + +if (hvm_get_pending_event (v, &info) ) + { +hvm_monitor_interrupt (info.vector, info.type, info.error_code, + info.cr2); + v->arch.monitor.next_interrupt_enabled = false;
xen/arch/x86/hvm/hvm.c void hvm_do_resume(struct vcpu *v) · line 584
Quoted 10 lines
xen/arch/x86/hvm/hvm.c +4
-4+ } + } -if (hvm_get_pending_event (v, &info) ) +if ( d->arch.monitor.safe_to_disable ) { -hvm_monitor_interrupt (info.vector, info.type, info.error_code, - info.cr2); - v->arch.monitor.next_interrupt_enabled = false; +struct vcpu *check_vcpu;
const again, requiring a respective adjustment to patch 2.
Quoted 4 lines
xen/arch/x86/hvm/hvm.c +4
-0+bool pending_op = false; + + for_each_vcpu ( d, check_vcpu ) + {
xen/arch/x86/hvm/hvm.c void hvm_do_resume(struct vcpu *v) · line 598
Quoted 10 lines
xen/arch/x86/hvm/hvm.c +10
-0+if (vm_event_check_pending_op (check_vcpu) ) + { + pending_op = true; +break ; + } + } + +if ( !pending_op ) + { +hvm_monitor_safe_to_disable ();
This new function returns bool without the caller caring about the return value.
Jan
Re: [PATCH 3/3] xen/vm_event: Add safe to disable vm_event
Quoted 47 lines · Jan Beulich
xen/arch/x86/hvm/hvm.c void hvm_do_resume(struct vcpu *v) · line 568
Quoted 11 lines · Tamas K Lengyel
xen/arch/x86/hvm/hvm.c +2
-2@@ -563,15 +563,41 @@ void hvm_do_resume(struct vcpu *v) v->arch.hvm.inject_event.vector =HVM_EVENT_VECTOR_UNSET ; } -if (unlikely (v->arch.vm_event) && v->arch.monitor.next_interrupt_enabled ) +if (unlikely (v->arch.vm_event) ) { -struct x86_event info; +struct domain *d = v->domain;const
Quoted 9 lines
xen/arch/x86/hvm/hvm.c +9
-0+if ( v->arch.monitor.next_interrupt_enabled ) + { +struct x86_event info; + +if (hvm_get_pending_event (v, &info) ) + { +hvm_monitor_interrupt (info.vector, info.type, info.error_code, + info.cr2); + v->arch.monitor.next_interrupt_enabled = false;xen/arch/x86/hvm/hvm.c void hvm_do_resume(struct vcpu *v) · line 584
Quoted 10 lines
xen/arch/x86/hvm/hvm.c +4
-4+ } + } -if (hvm_get_pending_event (v, &info) ) +if ( d->arch.monitor.safe_to_disable ) { -hvm_monitor_interrupt (info.vector, info.type, info.error_code, - info.cr2); - v->arch.monitor.next_interrupt_enabled = false; +struct vcpu *check_vcpu;const again, requiring a respective adjustment to patch 2.
Quoted 9 lines
xen/arch/x86/hvm/hvm.c +9
-0+bool pending_op = false; + + for_each_vcpu ( d, check_vcpu ) + { +if (vm_event_check_pending_op (check_vcpu) ) + { + pending_op = true; +break ; + }
Quoted 8 lines
xen/arch/x86/hvm/hvm.c void hvm_do_resume(struct vcpu *v) · line 598
Quoted 5 lines
xen/arch/x86/hvm/hvm.c +5
-0+ } + +if ( !pending_op ) + { +hvm_monitor_safe_to_disable ();This new function returns bool without the caller caring about the return value.
Yea, there is actually nothing to be done if the event can't be sent for whatever reason, I guess I'll just turn it to void.
Tamas
Re: [PATCH 3/3] xen/vm_event: Add safe to disable vm_event
In reply to Jan BeulichQuoted 8 lines · Jan Beulich
Quoted 6 lines · Tamas K Lengyel
xen/arch/x86/hvm/hvm.c +0
-0@@ -563,15 +563,41 @@ void hvm_do_resume(struct vcpu *v) v->arch.hvm.inject_event.vector =HVM_EVENT_VECTOR_UNSET ; }
Quoted 7 lines
xen/arch/x86/hvm/hvm.c void hvm_do_resume(struct vcpu *v) · line 568
Quoted 5 lines
xen/arch/x86/hvm/hvm.c +2
-2-if (unlikely (v->arch.vm_event) && v->arch.monitor.next_interrupt_enabled ) +if (unlikely (v->arch.vm_event) ) { -struct x86_event info; +struct domain *d = v->domain;const
This can't be const, we disable the safe_to_disable option below after sending the one-shot async event.
Tamas