Skip to contentSkip to conversation

x86/irq: Skip unmap_domain_pirq XSM during destruction

PATCH Superseded Jason Andryuk 3 participants 5 messages Mar 30, 2022 – Apr 5, 2022

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/irq: Skip unmap_domain_pirq XSM during destruction

xsm_unmap_domain_irq was seen denying unmap_domain_pirq when called from complete_domain_destroy as an RCU callback. The source context was an unexpected, random domain. Since this is a xen-internal operation, we don't want the XSM hook denying the operation.

Check d->is_dying and skip the check when the domain is dead. The RCU callback runs when a domain is in that state.

---
Dan wants to change current to point at DOMID_IDLE when the RCU callback runs. I think Juergen's commit 53594c7bd197 "rcu: don't use stop_machine_run() for rcu_barrier()" may have changed this since it mentions stop_machine_run scheduled the idle vcpus to run the callbacks for the old code.

Would that be as easy as changing rcu_do_batch() to do:

+        /* Run as "Xen" not a random domain's vcpu. */
+        vcpu = get_current();
+        set_current(idle_vcpu[smp_processor_id()]);
         list->func(list);

+ set_current(vcpu);

or is using set_current() only acceptable as part of context_switch?

 xen/arch/x86/irq.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
xen/arch/x86/irq.c +8 -4
index 285ac399fb..16488d287c 100644
@@ -2340,10 +2340,14 @@ int unmap_domain_pirq(struct domain *d, int pirq)
         nr = msi_desc->msi.nvec;
     }
 
-    ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
-                               msi_desc ? msi_desc->dev : NULL);
-    if ( ret )
-        goto done;
+    /* When called by complete_domain_destroy via RCU, current is a random
+     * domain.  Skip the XSM check since this is a Xen-initiated action. */
+    if ( d->is_dying != DOMDYING_dead ) {
+        ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
+                                   msi_desc ? msi_desc->dev : NULL);
+        if ( ret )
+            goto done;
+    }
 
     forced_unbind = pirq_guest_force_unbind(d, info);
     if ( forced_unbind )
Signature · 1 line
2.35.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 43 lines · Jason Andryuk

xsm_unmap_domain_irq was seen denying unmap_domain_pirq when called from complete_domain_destroy as an RCU callback. The source context was an unexpected, random domain. Since this is a xen-internal operation, we don't want the XSM hook denying the operation.

Check d->is_dying and skip the check when the domain is dead. The RCU callback runs when a domain is in that state.

---
Dan wants to change current to point at DOMID_IDLE when the RCU callback runs. I think Juergen's commit 53594c7bd197 "rcu: don't use stop_machine_run() for rcu_barrier()" may have changed this since it mentions stop_machine_run scheduled the idle vcpus to run the callbacks for the old code.

Would that be as easy as changing rcu_do_batch() to do:

(quoted diff) +4 -0
+        /* Run as "Xen" not a random domain's vcpu. */
+        vcpu = get_current();
+        set_current(idle_vcpu[smp_processor_id()]);
         list->func(list);
+        set_current(vcpu);

or is using set_current() only acceptable as part of context_switch?

 xen/arch/x86/irq.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
xen/arch/x86/irq.c +1 -4
index 285ac399fb..16488d287c 100644
@@ -2340,10 +2340,14 @@ int unmap_domain_pirq(struct domain *d, int pirq)
         nr = msi_desc->msi.nvec;
     }
 
-    ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
-                               msi_desc ? msi_desc->dev : NULL);
-    if ( ret )
-        goto done;
+    /* When called by complete_domain_destroy via RCU, current is a random

xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2353

Quoted 10 lines
xen/arch/x86/irq.c +7 -0
+     * domain.  Skip the XSM check since this is a Xen-initiated action. */
+    if ( d->is_dying != DOMDYING_dead ) {
+        ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
+                                   msi_desc ? msi_desc->dev : NULL);
+        if ( ret )
+            goto done;
+    }
 
     forced_unbind = pirq_guest_force_unbind(d, info);
     if ( forced_unbind )

I think this is sufficient though IMHO it would make it stronger if current accurately reflected the idle domain and the condition was added to the if statement check this fact.

#2 lore raw
In reply to Jason Andryuk
Quoted 7 lines · Jason Andryuk

xsm_unmap_domain_irq was seen denying unmap_domain_pirq when called from complete_domain_destroy as an RCU callback. The source context was an unexpected, random domain. Since this is a xen-internal operation, we don't want the XSM hook denying the operation.

Check d->is_dying and skip the check when the domain is dead. The RCU callback runs when a domain is in that state.

One question which has always been puzzling me (perhaps to Daniel): While I can see why mapping of an IRQ needs to be subject to an XSM check, it's not really clear to me why unmapping would need to be, at least as long as it's the domain itself which requests the unmap (and which I would view to extend to the domain being cleaned up). But maybe that's why it's XSM_HOOK ...

Quoted 15 lines

---
Dan wants to change current to point at DOMID_IDLE when the RCU callback runs. I think Juergen's commit 53594c7bd197 "rcu: don't use stop_machine_run() for rcu_barrier()" may have changed this since it mentions stop_machine_run scheduled the idle vcpus to run the callbacks for the old code.

Would that be as easy as changing rcu_do_batch() to do:

(quoted diff) +4 -0
+        /* Run as "Xen" not a random domain's vcpu. */
+        vcpu = get_current();
+        set_current(idle_vcpu[smp_processor_id()]);
         list->func(list);
+        set_current(vcpu);
Quoted 1 line

or is using set_current() only acceptable as part of context_switch?

Indeed I would question any uses outside of context_switch() (and system bringup).

xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2344

Quoted 12 lines
xen/arch/x86/irq.c +2 -4
@@ -2340,10 +2340,14 @@ int unmap_domain_pirq(struct domain *d, int pirq)
         nr = msi_desc->msi.nvec;
     }
 
-    ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
-                               msi_desc ? msi_desc->dev : NULL);
-    if ( ret )
-        goto done;
+    /* When called by complete_domain_destroy via RCU, current is a random
+     * domain.  Skip the XSM check since this is a Xen-initiated action. */

Comment style.

xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2345

Quoted 1 line
xen/arch/x86/irq.c +1 -0
+    if ( d->is_dying != DOMDYING_dead ) {

Please use !d->is_dying. Also please correct the placement of the brace. Or you could avoid the need for a brace by leveraging that ret is zero ahead of this if(), i.e. ...

xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2350

Quoted 5 lines
xen/arch/x86/irq.c +5 -0
+        ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
+                                   msi_desc ? msi_desc->dev : NULL);
+        if ( ret )
+            goto done;
+    }
    if ( !d->is_dying )
        ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
                                   msi_desc ? msi_desc->dev : NULL);
    if ( ret )
        goto done;

Jan

#3 lore raw
Quoted 9 lines · Jan Beulich
Quoted 7 lines · Jason Andryuk

xsm_unmap_domain_irq was seen denying unmap_domain_pirq when called from complete_domain_destroy as an RCU callback. The source context was an unexpected, random domain. Since this is a xen-internal operation, we don't want the XSM hook denying the operation.

Check d->is_dying and skip the check when the domain is dead. The RCU callback runs when a domain is in that state.

Quoted 6 lines

One question which has always been puzzling me (perhaps to Daniel): While I can see why mapping of an IRQ needs to be subject to an XSM check, it's not really clear to me why unmapping would need to be, at least as long as it's the domain itself which requests the unmap (and which I would view to extend to the domain being cleaned up). But maybe that's why it's XSM_HOOK ...

There are situations for instance where there is a flask-based system with one or more domains (v-platform-mgr) that are each responsible for the management of a subset of domains and are responsible for hotplugging in and out a device, i.e. granting the privilege to a v-platform-mgr to call PHYSDEVOP_map_pirq/PHYSDEVOP_unmap_pirq, for the domains each one is managing.

Quoted 15 lines
Quoted 15 lines

---
Dan wants to change current to point at DOMID_IDLE when the RCU callback runs. I think Juergen's commit 53594c7bd197 "rcu: don't use stop_machine_run() for rcu_barrier()" may have changed this since it mentions stop_machine_run scheduled the idle vcpus to run the callbacks for the old code.

Would that be as easy as changing rcu_do_batch() to do:

(quoted diff) +4 -0
+        /* Run as "Xen" not a random domain's vcpu. */
+        vcpu = get_current();
+        set_current(idle_vcpu[smp_processor_id()]);
         list->func(list);
+        set_current(vcpu);
Quoted 4 lines
Quoted 1 line

or is using set_current() only acceptable as part of context_switch?

Indeed I would question any uses outside of context_switch() (and system bringup).

I am not familiar with the details of the scheduler, but from a higher level, conceptual perspective, I do not understand why an idle domain task is being executed without an explicit context switch to the idle domain to ensure the current world view is consistent with the task execution scope. Just seems to me like this is creating a situation where things have the potential to go sideways/wrong.

v/r,
dps

#4 lore raw
In reply to Jan Beulich
Quoted 43 lines · Jan Beulich
Quoted 7 lines · Jason Andryuk

xsm_unmap_domain_irq was seen denying unmap_domain_pirq when called from complete_domain_destroy as an RCU callback. The source context was an unexpected, random domain. Since this is a xen-internal operation, we don't want the XSM hook denying the operation.

Check d->is_dying and skip the check when the domain is dead. The RCU callback runs when a domain is in that state.

One question which has always been puzzling me (perhaps to Daniel): While I can see why mapping of an IRQ needs to be subject to an XSM check, it's not really clear to me why unmapping would need to be, at least as long as it's the domain itself which requests the unmap (and which I would view to extend to the domain being cleaned up). But maybe that's why it's XSM_HOOK ...

Quoted 15 lines

---
Dan wants to change current to point at DOMID_IDLE when the RCU callback runs. I think Juergen's commit 53594c7bd197 "rcu: don't use stop_machine_run() for rcu_barrier()" may have changed this since it mentions stop_machine_run scheduled the idle vcpus to run the callbacks for the old code.

Would that be as easy as changing rcu_do_batch() to do:

(quoted diff) +4 -0
+        /* Run as "Xen" not a random domain's vcpu. */
+        vcpu = get_current();
+        set_current(idle_vcpu[smp_processor_id()]);
         list->func(list);
+        set_current(vcpu);
Quoted 1 line

or is using set_current() only acceptable as part of context_switch?

Indeed I would question any uses outside of context_switch() (and system bringup).

Quoted 6 lines
xen/arch/x86/irq.c +0 -0
@@ -2340,10 +2340,14 @@ int unmap_domain_pirq(struct domain *d, int pirq)
         nr = msi_desc->msi.nvec;
     }
 
Quoted 8 lines

xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2344

Quoted 6 lines
xen/arch/x86/irq.c +2 -4
-    ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
-                               msi_desc ? msi_desc->dev : NULL);
-    if ( ret )
-        goto done;
+    /* When called by complete_domain_destroy via RCU, current is a random
+     * domain.  Skip the XSM check since this is a Xen-initiated action. */

Comment style.

Yes. Sorry about that.

Quoted 5 lines

xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2345

Quoted 1 line
xen/arch/x86/irq.c +1 -0
+    if ( d->is_dying != DOMDYING_dead ) {

Please use !d->is_dying. Also please correct the placement of the brace. Or you could avoid the need for a brace by leveraging that ret is zero ahead of this if(), i.e. ...

Here I was patting myself on the back for remembering the spaces inside the parens, and I screwed up the brace... Sorry.

I intentionally chose DOMDYING_dead because, from my reading of the code, complete_domain_destroy should only reach here when dead (and not dying). If this function is reached when DOMDYING_dying, then that is unexpected. That would be a guest-initiated action and therefore the XSM check should apply.

Just checking is_dying is fine, but I want to explain and highlight this aspect.

Quoted 7 lines

xen/arch/x86/irq.c int unmap_domain_pirq(struct domain *d, int pirq) · line 2350

Quoted 5 lines
xen/arch/x86/irq.c +5 -0
+        ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
+                                   msi_desc ? msi_desc->dev : NULL);
+        if ( ret )
+            goto done;
+    }
Quoted 5 lines
    if ( !d->is_dying )
        ret = xsm_unmap_domain_irq(XSM_HOOK, d, irq,
                                   msi_desc ? msi_desc->dev : NULL);
    if ( ret )
        goto done;

I'm planning to just do it this way.

Thank you for reviewing.

-Jason