Skip to contentSkip to conversation

x86: irq: Do not BUG_ON multiple unbind calls for shared pirqs

PATCH Superseded Varad Gautam 6 participants 10 messages Jul 3, 2019 – Jan 29, 2020

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86: irq: Do not BUG_ON multiple unbind calls for shared pirqs

It is possible to receive multiple __pirq_guest_unbind calls for the same pirq if the pirq has not yet been removed from the domain's pirq_tree. For a shared pirq (nr_guests > 1), the first call zaps the current domain from the pirq's guests[] list, but the action handler is never freed as there are other guests using this pirq. As a result, on the second call, __pirq_guest_unbind tries search for the current domain which has been removed from the guests[] list, and hits a BUG_ON.

Allow xen to continue if a shared pirq has already been unbound from this guest. It will be cleaned up from the domain's pirq_tree during the destruction in complete_domain_destroy.

---

To apply stable-4.11 onwards.

 xen/arch/x86/irq.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
xen/arch/x86/irq.c +9 -1
index 094c3c5..256f503 100644
@@ -1711,7 +1711,15 @@ static irq_guest_action_t *__pirq_guest_unbind(
 
     for ( i = 0; (i < action->nr_guests) && (action->guest[i] != d); i++ )
         continue;
-    BUG_ON(i == action->nr_guests);
+    if ( i == action->nr_guests ) {
+        /* In case the pirq was shared, unbound for this domain in an earlier call, but still
+         * existed on the domain's pirq_tree, we still reach here if there are any later
+         * unbind calls on the same pirq. Return if such an unbind happens. */
+        if ( action->nr_guests > 0 && action->shareable )
+            return NULL;
+        BUG();
+    }
+
     memmove(&action->guest[i], &action->guest[i+1],
             (action->nr_guests-i-1) * sizeof(action->guest[0]));
     action->nr_guests--;
Signature · 20 lines
2.7.4




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879




_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw
Quoted 2 lines · Varad Gautam

It is possible to receive multiple __pirq_guest_unbind calls for the same pirq if the pirq has not yet been removed from the domain's pirq_tree.

I'd appreciate if you would make clear under what conditions this can happen, as I'm getting the impression that it's not the BUG_ON() that wants removing here, but that instead some caller may need fixing, or that instead the pirq tree removal needs to happen earlier. Afaict the call from evtchn_close() can't happen more than once, for example, and I wouldn't be surprised at all if one of the callers from xen/drivers/passthrough/ wasn't sufficiently gated.

Quoted 1 line

To apply stable-4.11 onwards.

That's based on you having found a broken commit in the 4.11 development window (if so, please name the commit), or simply because 4.10 is about to go out of general support?

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1717

Quoted 11 lines
xen/arch/x86/irq.c +4 -1
@@ -1711,7 +1711,15 @@ static irq_guest_action_t *__pirq_guest_unbind(
 
      for ( i = 0; (i < action->nr_guests) && (action->guest[i] != d); i++ )
          continue;
-    BUG_ON(i == action->nr_guests);
+    if ( i == action->nr_guests ) {
+        /* In case the pirq was shared, unbound for this domain in an earlier call, but still
+         * existed on the domain's pirq_tree, we still reach here if there are any later
+         * unbind calls on the same pirq. Return if such an unbind happens. */

Style: The opening brace is misplaced, plus the comment is not properly formatted and has overly long lines.

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1719

Quoted 2 lines
xen/arch/x86/irq.c +2 -0
+        if ( action->nr_guests > 0 && action->shareable )
+            return NULL;

Why does the sharable aspect matter here? Or asked differently, why can the same situation (multiple unbind requests) not arise for non-sharable IRQs?

Similary, why would this same situation not arise for the last guest getting unbound from the IRQ? There is an "action == NULL" check earlier on, but if multiple calls were legitimate, then the dprintk() there should go away (or be gated suitably) as well.

Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#2 lore raw
In reply to Varad Gautam
Quoted 7 lines · Varad Gautam

It is possible to receive multiple __pirq_guest_unbind calls for the same pirq if the pirq has not yet been removed from the domain's pirq_tree. For a shared pirq (nr_guests > 1), the first call zaps the current domain from the pirq's guests[] list, but the action handler is never freed as there are other guests using this pirq. As a result, on the second call, __pirq_guest_unbind tries search for the current domain which has been removed from the guests[] list, and hits a BUG_ON.

Thanks for digging into this, the passthrough code is quite complex and not easy to debug.

As raised by Jan, I'm not sure I see how the above can happen. For PIRQs bound to event channels the freeing can only happen once, as the event channel is afterwards closed and further calls won't make progress (regardless of whether the underlying PIRQ is shared or not).

So I assume this is something that you have seen with HVM guests and dpci?

Getting the full trace that leads to the BUG_ON would be very helpful, because as pointed out this is likely something that wants to be fixed in the caller of pirq_guest_unbind.

Do you have a reliable way to reproduce?

Quoted 9 lines

Allow xen to continue if a shared pirq has already been unbound from this guest. It will be cleaned up from the domain's pirq_tree during the destruction in complete_domain_destroy.

---

To apply stable-4.11 onwards.

Do you also have a bisection of when the issue was introduced?

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw

PATCH v2 1/1 x86: irq: Do not BUG_ON multiple unbind calls for shared pirqs

In reply to Varad Gautam

XEN_DOMCTL_destroydomain creates a continuation if domain_kill -ERESTARTS. In that scenario, it is possible to receive multiple _pirq_guest_unbind calls for the same pirq from domain_kill, if the pirq has not yet been removed from the domain's pirq_tree, as:
domain_kill()

    -> domain_relinquish_resources()
      -> pci_release_devices()
        -> pci_clean_dpci_irq()
          -> pirq_guest_unbind()
            -> __pirq_guest_unbind()

For a shared pirq (nr_guests > 1), the first call would zap the current domain from the pirq's guests[] list, but the action handler is never freed as there are other guests using this pirq. As a result, on the second call, __pirq_guest_unbind searches for the current domain which has been removed from the guests[] list, and hits a BUG_ON.

Make __pirq_guest_unbind safe to be called multiple times by letting xen continue if a shared pirq has already been unbound from this guest. The PIRQ will be cleaned up from the domain's pirq_tree during the destruction in complete_domain_destroy anyways.

CC Jan Beulich <[email protected]>
CC Roger Pau Monné <[email protected]>
CC Andrew Cooper <[email protected]>

v2: Split the check on action->nr_guests > 0 and make it an ASSERT, reword.

---
 xen/arch/x86/irq.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
xen/arch/x86/irq.c +10 -1
index 5d0d94c..3eb7b22 100644
@@ -1863,7 +1863,16 @@ static irq_guest_action_t *__pirq_guest_unbind(
 
     for ( i = 0; (i < action->nr_guests) && (action->guest[i] != d); i++ )
         continue;
-    BUG_ON(i == action->nr_guests);
+    if ( i == action->nr_guests ) {
+        ASSERT(action->nr_guests > 0) ;
+        /* In case the pirq was shared, unbound for this domain in an earlier call, but still
+         * existed on the domain's pirq_tree, we still reach here if there are any later
+         * unbind calls on the same pirq. Return if such an unbind happens. */
+        if ( action->shareable )
+            return NULL;
+        BUG();
+    }
+
     memmove(&action->guest[i], &action->guest[i+1],
             (action->nr_guests-i-1) * sizeof(action->guest[0]));
     action->nr_guests--;
Signature · 18 lines
2.7.4




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Reviewed-by on v2 1/1
#4 lore raw
Quoted 35 lines

-----Original Message-----
From: Xen-devel <[email protected]> On Behalf Of Varad Gautam
Sent: 18 December 2019 10:54
To: [email protected]
Cc: Andrew Cooper <[email protected]>; Gautam, Varad <[email protected]>; Jan Beulich <[email protected]>; Roger Pau Monné <[email protected]>
Subject: [Xen-devel] [PATCH v2] x86: irq: Do not BUG_ON multiple unbind calls for shared pirqs

XEN_DOMCTL_destroydomain creates a continuation if domain_kill -ERESTARTS. In that scenario, it is possible to receive multiple _pirq_guest_unbind calls for the same pirq from domain_kill, if the pirq has not yet been removed from the domain's pirq_tree, as:
domain_kill()

    -> domain_relinquish_resources()
      -> pci_release_devices()
        -> pci_clean_dpci_irq()
          -> pirq_guest_unbind()
            -> __pirq_guest_unbind()

For a shared pirq (nr_guests > 1), the first call would zap the current domain from the pirq's guests[] list, but the action handler is never freed
as there are other guests using this pirq. As a result, on the second call,
__pirq_guest_unbind searches for the current domain which has been removed from the guests[] list, and hits a BUG_ON.

Make __pirq_guest_unbind safe to be called multiple times by letting xen continue if a shared pirq has already been unbound from this guest. The PIRQ will be cleaned up from the domain's pirq_tree during the destruction in complete_domain_destroy anyways.

Quoted 4 lines
CC Jan Beulich <[email protected]>
CC Roger Pau Monné <[email protected]>
CC Andrew Cooper <[email protected]>
Quoted 48 lines

v2: Split the check on action->nr_guests > 0 and make it an ASSERT, reword.

---
 xen/arch/x86/irq.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
xen/arch/x86/irq.c +3 -1
index 5d0d94c..3eb7b22 100644
@@ -1863,7 +1863,16 @@ static irq_guest_action_t *__pirq_guest_unbind(
 
     for ( i = 0; (i < action->nr_guests) && (action->guest[i] != d); i++
 )
         continue;
-    BUG_ON(i == action->nr_guests);
+    if ( i == action->nr_guests ) {
+        ASSERT(action->nr_guests > 0) ;
+        /* In case the pirq was shared, unbound for this domain in an

earlier call, but still
+ * existed on the domain's pirq_tree, we still reach here if
there are any later
+ * unbind calls on the same pirq. Return if such an unbind
happens. */

xen/arch/x86/irq.c +5 -1
+        if ( action->shareable )
+            return NULL;
+        BUG();
+    }
+
     memmove(&action->guest[i], &action->guest[i+1],
             (action->nr_guests-i-1) * sizeof(action->guest[0]));
     action->nr_guests--;
--

2.7.4

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

Quoted 4 lines

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw
In reply to Varad Gautam
Quoted 11 lines · Varad Gautam

XEN_DOMCTL_destroydomain creates a continuation if domain_kill -ERESTARTS. In that scenario, it is possible to receive multiple _pirq_guest_unbind calls for the same pirq from domain_kill, if the pirq has not yet been removed from the domain's pirq_tree, as:
domain_kill()

    -> domain_relinquish_resources()
      -> pci_release_devices()
        -> pci_clean_dpci_irq()
          -> pirq_guest_unbind()
            -> __pirq_guest_unbind()
Quoted 5 lines

For a shared pirq (nr_guests > 1), the first call would zap the current domain from the pirq's guests[] list, but the action handler is never freed as there are other guests using this pirq. As a result, on the second call, __pirq_guest_unbind searches for the current domain which has been removed from the guests[] list, and hits a BUG_ON.

There must be more to this, seeing the cleanup_domain_irq_pirq() invocation at the end of pirq_guest_unbind(), which ought to be reached in the case you describe.

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1866

Quoted 8 lines
xen/arch/x86/irq.c +1 -1
@@ -1863,7 +1863,16 @@ static irq_guest_action_t *__pirq_guest_unbind(
 
     for ( i = 0; (i < action->nr_guests) && (action->guest[i] != d); i++ )
         continue;
-    BUG_ON(i == action->nr_guests);
+    if ( i == action->nr_guests ) {

Brace on its own line please.

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1867

Quoted 1 line
xen/arch/x86/irq.c +1 -0
+        ASSERT(action->nr_guests > 0) ;

Stray blank.

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1871

Quoted 4 lines
xen/arch/x86/irq.c +4 -0
+        /* In case the pirq was shared, unbound for this domain in an earlier call, but still
+         * existed on the domain's pirq_tree, we still reach here if there are any later
+         * unbind calls on the same pirq. Return if such an unbind happens. */
+        if ( action->shareable )

Long lines and malformed comment.

Do you perhaps also want to check that you take this path only for dying guests?

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#6 lore raw

Hey Jan,

Quoted 12 lines · Jan Beulich
Quoted 11 lines · Varad Gautam

XEN_DOMCTL_destroydomain creates a continuation if domain_kill -ERESTARTS. In that scenario, it is possible to receive multiple _pirq_guest_unbind calls for the same pirq from domain_kill, if the pirq has not yet been removed from the domain's pirq_tree, as:

   domain_kill()
     -> domain_relinquish_resources()
       -> pci_release_devices()
         -> pci_clean_dpci_irq()
           -> pirq_guest_unbind()
             -> __pirq_guest_unbind()
Quoted 8 lines
Quoted 5 lines

For a shared pirq (nr_guests > 1), the first call would zap the current domain from the pirq's guests[] list, but the action handler is never freed as there are other guests using this pirq. As a result, on the second call, __pirq_guest_unbind searches for the current domain which has been removed from the guests[] list, and hits a BUG_ON.

There must be more to this, seeing the cleanup_domain_irq_pirq() invocation at the end of pirq_guest_unbind(), which ought to be reached in the case you describe.

The calls to __pirq_guest_unbind and cleanup_domain_irq_pirq from pirq_guest_unbind are going to be mutually exclusive, since irq defaults to 0.

Quoted 10 lines

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1866

Quoted 8 lines
xen/arch/x86/irq.c +1 -1
@@ -1863,7 +1863,16 @@ static irq_guest_action_t *__pirq_guest_unbind(
 
      for ( i = 0; (i < action->nr_guests) && (action->guest[i] != d); i++ )
          continue;
-    BUG_ON(i == action->nr_guests);
+    if ( i == action->nr_guests ) {

Brace on its own line please.

Quoted 8 lines

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1867

Quoted 1 line
xen/arch/x86/irq.c +1 -0
+        ASSERT(action->nr_guests > 0) ;

Stray blank.

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1871

Quoted 4 lines
xen/arch/x86/irq.c +4 -0
+        /* In case the pirq was shared, unbound for this domain in an earlier call, but still
+         * existed on the domain's pirq_tree, we still reach here if there are any later
+         * unbind calls on the same pirq. Return if such an unbind happens. */
+        if ( action->shareable )

Long lines and malformed comment.

Handled all style fixups in v3.

Quoted 2 lines

Do you perhaps also want to check that you take this path only for dying guests?

The patch is about making pirq_guest_unbind() resilient/safe to multiple calls. At present, this happens when domain_kill -ERESTARTs for dying guests. It might also happen in other cases - and shouldn't cause xen to crash.

Quoted 6 lines

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#7 lore raw
In reply to Varad Gautam

Hi Varad,

Please send new version of a patch in a new thread rather than in-reply to the first version.

Quoted 37 lines · Varad Gautam

XEN_DOMCTL_destroydomain creates a continuation if domain_kill -ERESTARTS. In that scenario, it is possible to receive multiple _pirq_guest_unbind calls for the same pirq from domain_kill, if the pirq has not yet been removed from the domain's pirq_tree, as:

   domain_kill()
     -> domain_relinquish_resources()
       -> pci_release_devices()
         -> pci_clean_dpci_irq()
           -> pirq_guest_unbind()
             -> __pirq_guest_unbind()

For a shared pirq (nr_guests > 1), the first call would zap the current domain from the pirq's guests[] list, but the action handler is never freed as there are other guests using this pirq. As a result, on the second call, __pirq_guest_unbind searches for the current domain which has been removed from the guests[] list, and hits a BUG_ON.

Make __pirq_guest_unbind safe to be called multiple times by letting xen continue if a shared pirq has already been unbound from this guest. The PIRQ will be cleaned up from the domain's pirq_tree during the destruction in complete_domain_destroy anyways.

CC Jan Beulich <[email protected]>
CC Roger Pau Monné <[email protected]>
CC Andrew Cooper <[email protected]>

v2: Split the check on action->nr_guests > 0 and make it an ASSERT, reword.

---
  xen/arch/x86/irq.c | 11 ++++++++++-
  1 file changed, 10 insertions(+), 1 deletion(-)
xen/arch/x86/irq.c +0 -0
index 5d0d94c..3eb7b22 100644

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1866

Quoted 5 lines
xen/arch/x86/irq.c +1 -1
@@ -1863,7 +1863,16 @@ static irq_guest_action_t *__pirq_guest_unbind(
 
      for ( i = 0; (i < action->nr_guests) && (action->guest[i] != d); i++ )
          continue;
-    BUG_ON(i == action->nr_guests);
+    if ( i == action->nr_guests ) {

The { should be a new line.

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1867

Quoted 1 line
xen/arch/x86/irq.c +1 -0
+        ASSERT(action->nr_guests > 0) ;

The space before ; is not necessary.

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1870

Quoted 3 lines
xen/arch/x86/irq.c +3 -0
+        /* In case the pirq was shared, unbound for this domain in an earlier call, but still
+         * existed on the domain's pirq_tree, we still reach here if there are any later
+         * unbind calls on the same pirq. Return if such an unbind happens. */

The coding style for comment is:

/*
* Foo
* Bar
*/

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1873

Quoted 3 lines
xen/arch/x86/irq.c +3 -0
+        if ( action->shareable )
+            return NULL;
+        BUG();

Given that the previous BUG_ON() was hit, would it make sense to try to avoid a new BUG().

So why not just returning NULL as you do for action->shareable?

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1878

Quoted 6 lines
xen/arch/x86/irq.c +2 -0
+    }
+
      memmove(&action->guest[i], &action->guest[i+1],
              (action->nr_guests-i-1) * sizeof(action->guest[0]));
      action->nr_guests--;

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#8 lore raw

Hey Julien,

Quoted 68 lines · Julien Grall

Hi Varad,

Please send new version of a patch in a new thread rather than in-reply to the first version.

Quoted 38 lines · Varad Gautam

XEN_DOMCTL_destroydomain creates a continuation if domain_kill -ERESTARTS.
In that scenario, it is possible to receive multiple _pirq_guest_unbind calls for the same pirq from domain_kill, if the pirq has not yet been removed from the domain's pirq_tree, as:
   domain_kill()
     -> domain_relinquish_resources()
       -> pci_release_devices()
         -> pci_clean_dpci_irq()
           -> pirq_guest_unbind()
             -> __pirq_guest_unbind()

For a shared pirq (nr_guests > 1), the first call would zap the current domain from the pirq's guests[] list, but the action handler is never freed as there are other guests using this pirq. As a result, on the second call, __pirq_guest_unbind searches for the current domain which has been removed from the guests[] list, and hits a BUG_ON.

Make __pirq_guest_unbind safe to be called multiple times by letting xen continue if a shared pirq has already been unbound from this guest. The PIRQ will be cleaned up from the domain's pirq_tree during the destruction in complete_domain_destroy anyways.

CC Jan Beulich <[email protected]>
CC Roger Pau Monné <[email protected]>
CC Andrew Cooper <[email protected]>

v2: Split the check on action->nr_guests > 0 and make it an ASSERT, reword.
---
  xen/arch/x86/irq.c | 11 ++++++++++-

  1 file changed, 10 insertions(+), 1 deletion(-)
xen/arch/x86/irq.c +0 -0
index 5d0d94c..3eb7b22 100644

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1865

Quoted 4 lines
xen/arch/x86/irq.c +1 -1
@@ -1863,7 +1863,16 @@ static irq_guest_action_t *__pirq_guest_unbind(
         for ( i = 0; (i < action->nr_guests) && (action->guest[i] != d); i++ )
           continue;
-    BUG_ON(i == action->nr_guests);
+    if ( i == action->nr_guests ) {

The { should be a new line.

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1867

Quoted 1 line
xen/arch/x86/irq.c +1 -0
+        ASSERT(action->nr_guests > 0) ;

The space before ; is not necessary.

Quoted 6 lines
xen/arch/x86/irq.c +1 -0
+        /* In case the pirq was shared, unbound for this domain in 

an earlier call, but still
+         * existed on the domain's pirq_tree, we still reach here if there are any later
+         * unbind calls on the same pirq. Return if such an unbind happens. */

The coding style for comment is:

/*
 * Foo
 * Bar
 */

Quoted 8 lines
Quoted 3 lines

+        if ( action->shareable )
+            return NULL;
+        BUG();

Given that the previous BUG_ON() was hit, would it make sense to try to avoid a new BUG().

So why not just returning NULL as you do for action->shareable?

Thanks, I've done the style fixups in v3.

I'd argue that is indeed a BUG, if the pirq was _not_ shareable and the loop above couldn't find a matching domain for it - that implies the pirq shouldn't have existed in the first place.

Quoted 9 lines
Quoted 6 lines

+    }
+
      memmove(&action->guest[i], &action->guest[i+1], (action->nr_guests-i-1) * sizeof(action->guest[0])); action->nr_guests--;

Cheers,

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#9 lore raw
Quoted 1 line · On 29/01/2020 09:27, [email protected] wrote:

Hey Julien,

Hi,

Quoted 74 lines · Julien Grall
Quoted 72 lines

Hi Varad,

Please send new version of a patch in a new thread rather than in-reply to the first version.

Quoted 38 lines · Varad Gautam

XEN_DOMCTL_destroydomain creates a continuation if domain_kill -ERESTARTS.
In that scenario, it is possible to receive multiple _pirq_guest_unbind calls for the same pirq from domain_kill, if the pirq has not yet been removed from the domain's pirq_tree, as:
   domain_kill()
     -> domain_relinquish_resources()
       -> pci_release_devices()
         -> pci_clean_dpci_irq()
           -> pirq_guest_unbind()
             -> __pirq_guest_unbind()

For a shared pirq (nr_guests > 1), the first call would zap the current domain from the pirq's guests[] list, but the action handler is never freed as there are other guests using this pirq. As a result, on the second call, __pirq_guest_unbind searches for the current domain which has been removed from the guests[] list, and hits a BUG_ON.

Make __pirq_guest_unbind safe to be called multiple times by letting xen continue if a shared pirq has already been unbound from this guest. The PIRQ will be cleaned up from the domain's pirq_tree during the destruction in complete_domain_destroy anyways.

CC Jan Beulich <[email protected]>
CC Roger Pau Monné <[email protected]>
CC Andrew Cooper <[email protected]>

v2: Split the check on action->nr_guests > 0 and make it an ASSERT, reword.
---
  xen/arch/x86/irq.c | 11 ++++++++++-

  1 file changed, 10 insertions(+), 1 deletion(-)
xen/arch/x86/irq.c +0 -0
index 5d0d94c..3eb7b22 100644

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1865

Quoted 4 lines
xen/arch/x86/irq.c +1 -1
@@ -1863,7 +1863,16 @@ static irq_guest_action_t *__pirq_guest_unbind(
         for ( i = 0; (i < action->nr_guests) && (action->guest[i] != d); i++ )
           continue;
-    BUG_ON(i == action->nr_guests);
+    if ( i == action->nr_guests ) {

The { should be a new line.

xen/arch/x86/irq.c static irq_guest_action_t *__pirq_guest_unbind( · line 1867

Quoted 1 line
xen/arch/x86/irq.c +1 -0
+        ASSERT(action->nr_guests > 0) ;

The space before ; is not necessary.

Quoted 6 lines
xen/arch/x86/irq.c +1 -0
+        /* In case the pirq was shared, unbound for this domain in 

an earlier call, but still
+         * existed on the domain's pirq_tree, we still reach here if there are any later
+         * unbind calls on the same pirq. Return if such an unbind happens. */

The coding style for comment is:

/*
 * Foo
 * Bar
 */

Quoted 3 lines

+        if ( action->shareable )
+            return NULL;
+        BUG();

Quoted 8 lines
Quoted 4 lines

Given that the previous BUG_ON() was hit, would it make sense to try to avoid a new BUG().

So why not just returning NULL as you do for action->shareable?

Thanks, I've done the style fixups in v3.

I'd argue that is indeed a BUG, if the pirq was _not_ shareable and the loop above couldn't find a matching domain for it - that implies the pirq shouldn't have existed in the first place.

I am afraid this is only telling me how the BUG() could be triggered and not why a BUG() is more warrant than an ASSERT().

AFAIU, the BUG() can only be triggered if there is a programatic error. This is no different than your ASSERT(action->nr_guest > 0) you just added.

Reading the section "Handling unexpected conditions" in CODING_STYLE, it feels to me the BUG() is not the correct handling as you can return an error here and it would continue fine.

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel