x86/irq: Skip unmap_domain_pirq XSM during destruction
PATCH 1/1 x86/irq: Skip unmap_domain_pirq XSM during destruction
xsm_unmap_domain_irq was seen denying unmap_domain_pirq when called from complete_domain_destroy as an RCU callback. The source context was an unexpected, random domain. Since this is a xen-internal operation, we don't want the XSM hook denying the operation.
Check d->is_dying and skip the check when the domain is dead. The RCU callback runs when a domain is in that state.
---
Dan wants to change current to point at DOMID_IDLE when the RCU callback runs. I think Juergen's commit 53594c7bd197 "rcu: don't use stop_machine_run() for rcu_barrier()" may have changed this since it mentions stop_machine_run scheduled the idle vcpus to run the callbacks for the old code.
Would that be as easy as changing rcu_do_batch() to do:
+ /* Run as "Xen" not a random domain's vcpu. */
+ vcpu = get_current();
+ set_current(idle_vcpu[smp_processor_id()]);
list->func(list);+ set_current(vcpu);
or is using set_current() only acceptable as part of context_switch?
xen/arch/x86/irq.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
xen/arch/x86/irq.c +8 -4
@@ -2340,10 +2340,14 @@ int unmap_domain_pirq(struct domain *d, int pirq) nr = msi_desc->msi.nvec; } - ret =xsm_unmap_domain_irq (XSM_HOOK , d, irq, - msi_desc ? msi_desc->dev :NULL ); -if ( ret ) - goto done; +/* When called by complete_domain_destroy via RCU, current is a random +* domain. Skip the XSM check since this is a Xen-initiated action. */ +if ( d->is_dying != DOMDYING_dead ) { + ret =xsm_unmap_domain_irq (XSM_HOOK , d, irq, + msi_desc ? msi_desc->dev :NULL ); +if ( ret ) + goto done; + } forced_unbind =pirq_guest_force_unbind (d, info);if ( forced_unbind )
Signature · 1 line
2.35.1
Quoted 43 lines · Jason Andryuk
xsm_unmap_domain_irqwas seen denyingunmap_domain_pirqwhen called fromcomplete_domain_destroyas an RCU callback. The source context was an unexpected, random domain. Since this is a xen-internal operation, we don't want the XSM hook denying the operation.Check
d->is_dyingand skip the check when the domain is dead. The RCU callback runs when a domain is in that state.Suggested-by Roger Pau Monné <[email protected]>Signed-off-by Jason Andryuk <[email protected]>---
Dan wants to change current to point atDOMID_IDLEwhen the RCU callback runs. I think Juergen's commit53594c7bd197"rcu: don't usestop_machine_run()forrcu_barrier()" may have changed this since it mentionsstop_machine_runscheduled the idle vcpus to run the callbacks for the old code.Would that be as easy as changing
rcu_do_batch()to do:(quoted diff) +4
-0+ /* Run as "Xen" not a random domain's vcpu. */ + vcpu = get_current(); + set_current(idle_vcpu[smp_processor_id()]); list->func(list); + set_current(vcpu);or is using
set_current()only acceptable as part ofcontext_switch?xen/arch/x86/irq.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)xen/arch/x86/irq.c +1
-4@@ -2340,10 +2340,14 @@ int unmap_domain_pirq(struct domain *d, int pirq) nr = msi_desc->msi.nvec; } - ret =xsm_unmap_domain_irq (XSM_HOOK , d, irq, - msi_desc ? msi_desc->dev :NULL ); -if ( ret ) - goto done; +/* When called by complete_domain_destroy via RCU, current is a random
xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2353
Quoted 10 lines
xen/arch/x86/irq.c +7
-0+* domain. Skip the XSM check since this is a Xen-initiated action. */ +if ( d->is_dying != DOMDYING_dead ) { + ret =xsm_unmap_domain_irq (XSM_HOOK , d, irq, + msi_desc ? msi_desc->dev :NULL ); +if ( ret ) + goto done; + } forced_unbind =pirq_guest_force_unbind (d, info);if ( forced_unbind )
I think this is sufficient though IMHO it would make it stronger if current accurately reflected the idle domain and the condition was added to the if statement check this fact.
Quoted 7 lines · Jason Andryuk
xsm_unmap_domain_irqwas seen denyingunmap_domain_pirqwhen called fromcomplete_domain_destroyas an RCU callback. The source context was an unexpected, random domain. Since this is a xen-internal operation, we don't want the XSM hook denying the operation.Check
d->is_dyingand skip the check when the domain is dead. The RCU callback runs when a domain is in that state.
One question which has always been puzzling me (perhaps to Daniel): While I can see why mapping of an IRQ needs to be subject to an XSM check, it's not really clear to me why unmapping would need to be, at least as long as it's the domain itself which requests the unmap (and which I would view to extend to the domain being cleaned up). But maybe that's why it's XSM_HOOK ...
Quoted 15 lines
---
Dan wants to change current to point atDOMID_IDLEwhen the RCU callback runs. I think Juergen's commit53594c7bd197"rcu: don't usestop_machine_run()forrcu_barrier()" may have changed this since it mentionsstop_machine_runscheduled the idle vcpus to run the callbacks for the old code.Would that be as easy as changing
rcu_do_batch()to do:(quoted diff) +4
-0+ /* Run as "Xen" not a random domain's vcpu. */ + vcpu = get_current(); + set_current(idle_vcpu[smp_processor_id()]); list->func(list); + set_current(vcpu);
Quoted 1 line
or is using
set_current()only acceptable as part ofcontext_switch?
Indeed I would question any uses outside of context_switch() (and system bringup).
xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2344
Quoted 12 lines
xen/arch/x86/irq.c +2
-4@@ -2340,10 +2340,14 @@ int unmap_domain_pirq(struct domain *d, int pirq) nr = msi_desc->msi.nvec; } - ret =xsm_unmap_domain_irq (XSM_HOOK , d, irq, - msi_desc ? msi_desc->dev :NULL ); -if ( ret ) - goto done; +/* When called by complete_domain_destroy via RCU, current is a random +* domain. Skip the XSM check since this is a Xen-initiated action. */
Comment style.
xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2345
Quoted 1 line
xen/arch/x86/irq.c +1
-0+if ( d->is_dying != DOMDYING_dead ) {
Please use !d->is_dying. Also please correct the placement of the brace. Or you could avoid the need for a brace by leveraging that ret is zero ahead of this if(), i.e. ...
xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2350
Quoted 5 lines
xen/arch/x86/irq.c +5
-0+ ret =xsm_unmap_domain_irq (XSM_HOOK , d, irq, + msi_desc ? msi_desc->dev :NULL ); +if ( ret ) + goto done; + }
if ( !d->is_dying )
ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
msi_desc ? msi_desc->dev : NULL);
if ( ret )
goto done;Jan
Quoted 9 lines · Jan Beulich
Quoted 7 lines · Jason Andryuk
xsm_unmap_domain_irqwas seen denyingunmap_domain_pirqwhen called fromcomplete_domain_destroyas an RCU callback. The source context was an unexpected, random domain. Since this is a xen-internal operation, we don't want the XSM hook denying the operation.Check
d->is_dyingand skip the check when the domain is dead. The RCU callback runs when a domain is in that state.
Quoted 6 lines
One question which has always been puzzling me (perhaps to Daniel): While I can see why mapping of an IRQ needs to be subject to an XSM check, it's not really clear to me why unmapping would need to be, at least as long as it's the domain itself which requests the unmap (and which I would view to extend to the domain being cleaned up). But maybe that's why it's
XSM_HOOK...
There are situations for instance where there is a flask-based system with one or more domains (v-platform-mgr) that are each responsible for the management of a subset of domains and are responsible for hotplugging in and out a device, i.e. granting the privilege to a v-platform-mgr to call PHYSDEVOP_map_pirq/PHYSDEVOP_unmap_pirq, for the domains each one is managing.
Quoted 15 lines
Quoted 15 lines
---
Dan wants to change current to point atDOMID_IDLEwhen the RCU callback runs. I think Juergen's commit53594c7bd197"rcu: don't usestop_machine_run()forrcu_barrier()" may have changed this since it mentionsstop_machine_runscheduled the idle vcpus to run the callbacks for the old code.Would that be as easy as changing
rcu_do_batch()to do:(quoted diff) +4
-0+ /* Run as "Xen" not a random domain's vcpu. */ + vcpu = get_current(); + set_current(idle_vcpu[smp_processor_id()]); list->func(list); + set_current(vcpu);
Quoted 4 lines
Quoted 1 line
or is using
set_current()only acceptable as part ofcontext_switch?Indeed I would question any uses outside of
context_switch()(and system bringup).
I am not familiar with the details of the scheduler, but from a higher level, conceptual perspective, I do not understand why an idle domain task is being executed without an explicit context switch to the idle domain to ensure the current world view is consistent with the task execution scope. Just seems to me like this is creating a situation where things have the potential to go sideways/wrong.
v/r,
dps
Quoted 43 lines · Jan Beulich
Quoted 7 lines · Jason Andryuk
xsm_unmap_domain_irqwas seen denyingunmap_domain_pirqwhen called fromcomplete_domain_destroyas an RCU callback. The source context was an unexpected, random domain. Since this is a xen-internal operation, we don't want the XSM hook denying the operation.Check
d->is_dyingand skip the check when the domain is dead. The RCU callback runs when a domain is in that state.One question which has always been puzzling me (perhaps to Daniel): While I can see why mapping of an IRQ needs to be subject to an XSM check, it's not really clear to me why unmapping would need to be, at least as long as it's the domain itself which requests the unmap (and which I would view to extend to the domain being cleaned up). But maybe that's why it's
XSM_HOOK...Quoted 15 lines
---
Dan wants to change current to point atDOMID_IDLEwhen the RCU callback runs. I think Juergen's commit53594c7bd197"rcu: don't usestop_machine_run()forrcu_barrier()" may have changed this since it mentionsstop_machine_runscheduled the idle vcpus to run the callbacks for the old code.Would that be as easy as changing
rcu_do_batch()to do:(quoted diff) +4
-0+ /* Run as "Xen" not a random domain's vcpu. */ + vcpu = get_current(); + set_current(idle_vcpu[smp_processor_id()]); list->func(list); + set_current(vcpu);Quoted 1 line
or is using
set_current()only acceptable as part ofcontext_switch?Indeed I would question any uses outside of
context_switch()(and system bringup).Quoted 6 lines
xen/arch/x86/irq.c +0
-0@@ -2340,10 +2340,14 @@ int unmap_domain_pirq(struct domain *d, int pirq) nr = msi_desc->msi.nvec; }
Quoted 8 lines
xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2344
Quoted 6 lines
xen/arch/x86/irq.c +2
-4- ret =xsm_unmap_domain_irq (XSM_HOOK , d, irq, - msi_desc ? msi_desc->dev :NULL ); -if ( ret ) - goto done; +/* When called by complete_domain_destroy via RCU, current is a random +* domain. Skip the XSM check since this is a Xen-initiated action. */ Comment style.
Yes. Sorry about that.
Quoted 5 lines
xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2345
Quoted 1 line
xen/arch/x86/irq.c +1
-0+if ( d->is_dying != DOMDYING_dead ) {Please use !d->is_dying. Also please correct the placement of the brace. Or you could avoid the need for a brace by leveraging that ret is zero ahead of this
if(), i.e. ...
Here I was patting myself on the back for remembering the spaces inside the parens, and I screwed up the brace... Sorry.
I intentionally chose DOMDYING_dead because, from my reading of the code, complete_domain_destroy should only reach here when dead (and not dying). If this function is reached when DOMDYING_dying, then that is unexpected. That would be a guest-initiated action and therefore the XSM check should apply.
Just checking is_dying is fine, but I want to explain and highlight this aspect.
Quoted 7 lines
xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2350
Quoted 5 lines
xen/arch/x86/irq.c +5
-0+ ret =xsm_unmap_domain_irq (XSM_HOOK , d, irq, + msi_desc ? msi_desc->dev :NULL ); +if ( ret ) + goto done; + }
Quoted 5 lines
if ( !d->is_dying )
ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
msi_desc ? msi_desc->dev : NULL);
if ( ret )
goto done;I'm planning to just do it this way.
Thank you for reviewing.
-Jason