Skip to contentSkip to conversation

x86/passthrough: fixes for PVH dom0 edge triggered interrupts

PATCH In review Roger Pau Monne 5 participants 16 messages Jun 10, 2020 – Jun 26, 2020

lore
Share & tools
lore
#0 lore raw

v2 cover x86/passthrough: fixes for PVH dom0 edge triggered interrupts

Hello,

Small series with two bugfixes to correctly handle edge triggered interrupts on PVH dom0.

for-4.14 reasoning: fixes are isolated to PVH dom0 specific passthrough code (IDENTITY_GSI kind of bindings), and hence shouldn't affect passthrough to HVM domUs. Without these fixes the RTC timer won't work correctly on a PVH dom0 because it's edge triggered (GSI 8).

Roger Pau Monne (2):
x86/passthrough: do not assert edge triggered GSIs for PVH dom0 x86/passthrough: introduce a flag for GSIs not requiring an EOI or
unmask

 xen/arch/x86/hvm/irq.c        | 13 ++++++++-----
 xen/drivers/passthrough/io.c  | 24 +++++++++++++++---------
 xen/include/asm-x86/hvm/irq.h |  2 ++
 3 files changed, 25 insertions(+), 14 deletions(-)
Signature · 1 line
2.26.2
#1 lore raw

PATCH v2 1/2 x86/passthrough: do not assert edge triggered GSIs for PVH dom0

Edge triggered interrupts do not assert the line, so the handling done in Xen should also avoid asserting it. Asserting the line prevents further edge triggered interrupts on the same vIO-APIC pin from being delivered, since the line is not de-asserted.

One case of such kind of interrupt is the RTC timer, which is edge triggered and available to a PVH dom0. Note this should not affect domUs, as it only modifies the behavior of IDENTITY_GSI kind of passed through interrupts.

---
Changes since v1:
- Compare the triggering against VIOAPIC_{EDGE/LEVEL}_TRIG.

---
 xen/arch/x86/hvm/irq.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
xen/arch/x86/hvm/irq.c +8 -5
index 9c8adbc495..fd02cf2e8d 100644
@@ -169,9 +169,10 @@ void hvm_pci_intx_deassert(
 
 void hvm_gsi_assert(struct domain *d, unsigned int gsi)
 {
+    int trig = vioapic_get_trigger_mode(d, gsi);
     struct hvm_irq *hvm_irq = hvm_domain_irq(d);
 
-    if ( gsi >= hvm_irq->nr_gsis )
+    if ( gsi >= hvm_irq->nr_gsis || trig < 0 )
     {
         ASSERT_UNREACHABLE();
         return;
@@ -186,9 +187,10 @@ void hvm_gsi_assert(struct domain *d, unsigned int gsi)
      * to know if the GSI is pending or not.
      */
     spin_lock(&d->arch.hvm.irq_lock);
-    if ( !hvm_irq->gsi_assert_count[gsi] )
+    if ( trig == VIOAPIC_EDGE_TRIG || !hvm_irq->gsi_assert_count[gsi] )
     {
-        hvm_irq->gsi_assert_count[gsi] = 1;
+        if ( trig == VIOAPIC_LEVEL_TRIG )
+            hvm_irq->gsi_assert_count[gsi] = 1;
         assert_gsi(d, gsi);
     }
     spin_unlock(&d->arch.hvm.irq_lock);
@@ -196,11 +198,12 @@ void hvm_gsi_assert(struct domain *d, unsigned int gsi)
 
 void hvm_gsi_deassert(struct domain *d, unsigned int gsi)
 {
+    int trig = vioapic_get_trigger_mode(d, gsi);
     struct hvm_irq *hvm_irq = hvm_domain_irq(d);
 
-    if ( gsi >= hvm_irq->nr_gsis )
+    if ( trig <= VIOAPIC_EDGE_TRIG || gsi >= hvm_irq->nr_gsis )
     {
-        ASSERT_UNREACHABLE();
+        ASSERT(trig == VIOAPIC_EDGE_TRIG && gsi < hvm_irq->nr_gsis);
         return;
     }
 
Signature · 1 line
2.26.2
Reviewed-by on v2 1/2
#2 lore raw

RE: [PATCH for-4.14 v2 1/2] x86/passthrough: do not assert edge triggered GSIs for PVH dom0

Quoted 13 lines

-----Original Message-----
From: Roger Pau Monne <[email protected]>
Sent: 10 June 2020 15:29
To: [email protected]
Cc: [email protected]; Roger Pau Monne <[email protected]>; Jan Beulich <[email protected]>; Andrew Cooper <[email protected]>; Wei Liu <[email protected]>
Subject: [PATCH for-4.14 v2 1/2] x86/passthrough: do not assert edge triggered GSIs for PVH dom0

Edge triggered interrupts do not assert the line, so the handling done in Xen should also avoid asserting it. Asserting the line prevents further edge triggered interrupts on the same vIO-APIC pin from being delivered, since the line is not de-asserted.

Quoted 7 lines

One case of such kind of interrupt is the RTC timer, which is edge triggered and available to a PVH dom0. Note this should not affect domUs, as it only modifies the behavior of IDENTITY_GSI kind of passed through interrupts.

Quoted 53 lines

---
Changes since v1:
- Compare the triggering against VIOAPIC_{EDGE/LEVEL}_TRIG.

---
 xen/arch/x86/hvm/irq.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
xen/arch/x86/hvm/irq.c +8 -6
index 9c8adbc495..fd02cf2e8d 100644
@@ -169,9 +169,10 @@ void hvm_pci_intx_deassert(
 
 void hvm_gsi_assert(struct domain *d, unsigned int gsi)
 {
+    int trig = vioapic_get_trigger_mode(d, gsi);
     struct hvm_irq *hvm_irq = hvm_domain_irq(d);
 
-    if ( gsi >= hvm_irq->nr_gsis )
+    if ( gsi >= hvm_irq->nr_gsis || trig < 0 )
     {
         ASSERT_UNREACHABLE();
         return;
@@ -186,9 +187,10 @@ void hvm_gsi_assert(struct domain *d, unsigned int gsi)
      * to know if the GSI is pending or not.
      */
     spin_lock(&d->arch.hvm.irq_lock);
-    if ( !hvm_irq->gsi_assert_count[gsi] )
+    if ( trig == VIOAPIC_EDGE_TRIG || !hvm_irq->gsi_assert_count[gsi] )
     {
-        hvm_irq->gsi_assert_count[gsi] = 1;
+        if ( trig == VIOAPIC_LEVEL_TRIG )
+            hvm_irq->gsi_assert_count[gsi] = 1;
         assert_gsi(d, gsi);
     }
     spin_unlock(&d->arch.hvm.irq_lock);
@@ -196,11 +198,12 @@ void hvm_gsi_assert(struct domain *d, unsigned int gsi)
 
 void hvm_gsi_deassert(struct domain *d, unsigned int gsi)
 {
+    int trig = vioapic_get_trigger_mode(d, gsi);
     struct hvm_irq *hvm_irq = hvm_domain_irq(d);
 
-    if ( gsi >= hvm_irq->nr_gsis )
+    if ( trig <= VIOAPIC_EDGE_TRIG || gsi >= hvm_irq->nr_gsis )
     {
-        ASSERT_UNREACHABLE();
+        ASSERT(trig == VIOAPIC_EDGE_TRIG && gsi < hvm_irq->nr_gsis);
         return;
     }
 
--

2.26.2

#3 lore raw

Re: [PATCH for-4.14 v2 1/2] x86/passthrough: do not assert edge triggered GSIs for PVH dom0

In reply to Roger Pau Monne

xen/arch/x86/hvm/irq.c void hvm_gsi_assert(struct domain *d, unsigned int gsi) · line 193

Quoted 10 lines · Roger Pau Monne
xen/arch/x86/hvm/irq.c +3 -2
@@ -186,9 +187,10 @@ void hvm_gsi_assert(struct domain *d, unsigned int gsi)
      * to know if the GSI is pending or not.
      */
     spin_lock(&d->arch.hvm.irq_lock);
-    if ( !hvm_irq->gsi_assert_count[gsi] )
+    if ( trig == VIOAPIC_EDGE_TRIG || !hvm_irq->gsi_assert_count[gsi] )
     {
-        hvm_irq->gsi_assert_count[gsi] = 1;
+        if ( trig == VIOAPIC_LEVEL_TRIG )
+            hvm_irq->gsi_assert_count[gsi] = 1;

Btw, along the lines of how you do things here, I think ...

xen/arch/x86/hvm/irq.c void hvm_gsi_assert(struct domain *d, unsigned int gsi) · line 204

Quoted 9 lines
xen/arch/x86/hvm/irq.c +2 -1
@@ -196,11 +198,12 @@ void hvm_gsi_assert(struct domain *d, unsigned int gsi)
 
 void hvm_gsi_deassert(struct domain *d, unsigned int gsi)
 {
+    int trig = vioapic_get_trigger_mode(d, gsi);
     struct hvm_irq *hvm_irq = hvm_domain_irq(d);
 
-    if ( gsi >= hvm_irq->nr_gsis )
+    if ( trig <= VIOAPIC_EDGE_TRIG || gsi >= hvm_irq->nr_gsis )

... this would better have been "trig != VIOAPIC_LEVEL_TRIG", to avoid the code being dependent upon the actual values of both VIOAPIC_*_TRIG constants.

Jan

Quoted 6 lines
xen/arch/x86/hvm/irq.c +1 -1
-        ASSERT_UNREACHABLE();
+        ASSERT(trig == VIOAPIC_EDGE_TRIG && gsi < hvm_irq->nr_gsis);
         return;
     }
 
 
#4 lore raw

Re: [PATCH for-4.14 v2 1/2] x86/passthrough: do not assert edge triggered GSIs for PVH dom0

Quoted 21 lines · Jan Beulich

xen/arch/x86/hvm/irq.c void hvm_gsi_assert(struct domain *d, unsigned int gsi) · line 193

Quoted 10 lines · Roger Pau Monne
xen/arch/x86/hvm/irq.c +3 -2
@@ -186,9 +187,10 @@ void hvm_gsi_assert(struct domain *d, unsigned int gsi)
      * to know if the GSI is pending or not.
      */
     spin_lock(&d->arch.hvm.irq_lock);
-    if ( !hvm_irq->gsi_assert_count[gsi] )
+    if ( trig == VIOAPIC_EDGE_TRIG || !hvm_irq->gsi_assert_count[gsi] )
     {
-        hvm_irq->gsi_assert_count[gsi] = 1;
+        if ( trig == VIOAPIC_LEVEL_TRIG )
+            hvm_irq->gsi_assert_count[gsi] = 1;

Btw, along the lines of how you do things here, I think ...

Quoted 7 lines
xen/arch/x86/hvm/irq.c +1 -0
@@ -196,11 +198,12 @@ void hvm_gsi_assert(struct domain *d, unsigned int gsi)
 
 void hvm_gsi_deassert(struct domain *d, unsigned int gsi)
 {
+    int trig = vioapic_get_trigger_mode(d, gsi);
     struct hvm_irq *hvm_irq = hvm_domain_irq(d);
 
Quoted 6 lines

xen/arch/x86/hvm/irq.c void hvm_gsi_assert(struct domain *d, unsigned int gsi) · line 204

Quoted 2 lines
xen/arch/x86/hvm/irq.c +1 -1
-    if ( gsi >= hvm_irq->nr_gsis )
+    if ( trig <= VIOAPIC_EDGE_TRIG || gsi >= hvm_irq->nr_gsis )

... this would better have been "trig != VIOAPIC_LEVEL_TRIG", to avoid the code being dependent upon the actual values of both VIOAPIC_*_TRIG constants.

Sure, let me send a follow up patch, it's trivial to fix.

Thanks, Roger.

#5 lore raw

PATCH v2 2/2 x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

In reply to Roger Pau Monne

There's no need to setup a timer for GSIs that are edge triggered, since those don't require any EIO or unmask, and hence couldn't block other interrupts.

Note this is only used by PVH dom0, that can setup the passthrough of edge triggered interrupts from the vIO-APIC. One example of such kind of interrupt that can be used by a PVH dom0 would be the RTC timer.

While there introduce an out label to do the unlock and reduce code duplication.

---
Changes since v1:
- Introduce an out label that does the unlock.

---
 xen/drivers/passthrough/io.c  | 24 +++++++++++++++---------
 xen/include/asm-x86/hvm/irq.h |  2 ++
 2 files changed, 17 insertions(+), 9 deletions(-)
xen/drivers/passthrough/io.c +15 -9
index b292e79382..6b1305a3e5 100644
@@ -138,7 +138,8 @@ static void pt_pirq_softirq_reset(struct hvm_pirq_dpci *pirq_dpci)
 
 bool pt_irq_need_timer(uint32_t flags)
 {
-    return !(flags & (HVM_IRQ_DPCI_GUEST_MSI | HVM_IRQ_DPCI_TRANSLATE));
+    return !(flags & (HVM_IRQ_DPCI_GUEST_MSI | HVM_IRQ_DPCI_TRANSLATE |
+                      HVM_IRQ_DPCI_NO_EOI));
 }
 
 static int pt_irq_guest_eoi(struct domain *d, struct hvm_pirq_dpci *pirq_dpci,
@@ -558,6 +559,12 @@ int pt_irq_create_bind(
                      */
                     ASSERT(!mask);
                     share = trigger_mode;
+                    if ( trigger_mode == VIOAPIC_EDGE_TRIG )
+                        /*
+                         * Edge IO-APIC interrupt, no EOI or unmask to perform
+                         * and hence no timer needed.
+                         */
+                        pirq_dpci->flags |= HVM_IRQ_DPCI_NO_EOI;
                 }
             }
 
@@ -897,17 +904,13 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
             send_guest_pirq(d, pirq);
 
             if ( pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI )
-            {
-                spin_unlock(&d->event_lock);
-                return;
-            }
+                goto out;
         }
 
         if ( pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI )
         {
             vmsi_deliver_pirq(d, pirq_dpci);
-            spin_unlock(&d->event_lock);
-            return;
+            goto out;
         }
 
         list_for_each_entry ( digl, &pirq_dpci->digl_list, list )
@@ -920,6 +923,8 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
         if ( pirq_dpci->flags & HVM_IRQ_DPCI_IDENTITY_GSI )
         {
             hvm_gsi_assert(d, pirq->pirq);
+            if ( pirq_dpci->flags & HVM_IRQ_DPCI_NO_EOI )
+                goto out;
             pirq_dpci->pending++;
         }
 
@@ -927,8 +932,7 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
         {
             /* for translated MSI to INTx interrupt, eoi as early as possible */
             __msi_pirq_eoi(pirq_dpci);
-            spin_unlock(&d->event_lock);
-            return;
+            goto out;
         }
 
         /*
@@ -941,6 +945,8 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
         ASSERT(pt_irq_need_timer(pirq_dpci->flags));
         set_timer(&pirq_dpci->timer, NOW() + PT_IRQ_TIME_OUT);
     }
+
+ out:
     spin_unlock(&d->event_lock);
 }
 
xen/include/asm-x86/hvm/irq.h +2 -0
index d306cfeade..532880d497 100644
@@ -121,6 +121,7 @@ struct dev_intx_gsi_link {
 #define _HVM_IRQ_DPCI_GUEST_PCI_SHIFT           4
 #define _HVM_IRQ_DPCI_GUEST_MSI_SHIFT           5
 #define _HVM_IRQ_DPCI_IDENTITY_GSI_SHIFT        6
+#define _HVM_IRQ_DPCI_NO_EOI_SHIFT              7
 #define _HVM_IRQ_DPCI_TRANSLATE_SHIFT          15
 #define HVM_IRQ_DPCI_MACH_PCI        (1u << _HVM_IRQ_DPCI_MACH_PCI_SHIFT)
 #define HVM_IRQ_DPCI_MACH_MSI        (1u << _HVM_IRQ_DPCI_MACH_MSI_SHIFT)
@@ -129,6 +130,7 @@ struct dev_intx_gsi_link {
 #define HVM_IRQ_DPCI_GUEST_PCI       (1u << _HVM_IRQ_DPCI_GUEST_PCI_SHIFT)
 #define HVM_IRQ_DPCI_GUEST_MSI       (1u << _HVM_IRQ_DPCI_GUEST_MSI_SHIFT)
 #define HVM_IRQ_DPCI_IDENTITY_GSI    (1u << _HVM_IRQ_DPCI_IDENTITY_GSI_SHIFT)
+#define HVM_IRQ_DPCI_NO_EOI          (1u << _HVM_IRQ_DPCI_NO_EOI_SHIFT)
 #define HVM_IRQ_DPCI_TRANSLATE       (1u << _HVM_IRQ_DPCI_TRANSLATE_SHIFT)
 
 struct hvm_gmsi_info {
Signature · 1 line
2.26.2
Acked-by on v2 2/2
#6 lore raw

Re: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

Quoted 4 lines · Roger Pau Monne

There's no need to setup a timer for GSIs that are edge triggered, since those don't require any EIO or unmask, and hence couldn't block other interrupts.

Quoted 8 lines

Note this is only used by PVH dom0, that can setup the passthrough of edge triggered interrupts from the vIO-APIC. One example of such kind of interrupt that can be used by a PVH dom0 would be the RTC timer.

While there introduce an out label to do the unlock and reduce code duplication.

#7 lore raw

RE: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

Quoted 17 lines

-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 11 June 2020 17:26
To: Roger Pau Monne <[email protected]>; [email protected]
Cc: [email protected]; Jan Beulich <[email protected]>; Wei Liu <[email protected]>
Subject: Re: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

Quoted 8 lines · Roger Pau Monne

There's no need to setup a timer for GSIs that are edge triggered, since those don't require any EIO or unmask, and hence couldn't block other interrupts.

Note this is only used by PVH dom0, that can setup the passthrough of edge triggered interrupts from the vIO-APIC. One example of such kind of interrupt that can be used by a PVH dom0 would be the RTC timer.

Quoted 6 lines
Quoted 4 lines

While there introduce an out label to do the unlock and reduce code duplication.

#8 lore raw

Re: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

Quoted 17 lines · Paul Durrant
Quoted 17 lines

-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 11 June 2020 17:26
To: Roger Pau Monne <[email protected]>; [email protected]
Cc: [email protected]; Jan Beulich <[email protected]>; Wei Liu <[email protected]>
Subject: Re: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

Quoted 8 lines · Roger Pau Monne

There's no need to setup a timer for GSIs that are edge triggered, since those don't require any EIO or unmask, and hence couldn't block other interrupts.

Note this is only used by PVH dom0, that can setup the passthrough of edge triggered interrupts from the vIO-APIC. One example of such kind of interrupt that can be used by a PVH dom0 would be the RTC timer.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

While there introduce an out label to do the unlock and reduce code duplication.

Strictly speaking these tags were too little for the patch to go in - the change to drivers/passthrough/io.c would also have required Paul's (or my) R-b. I take it that this was sort of implied by the R-a-b.

Jan

#9 lore raw

RE: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

Quoted 32 lines

-----Original Message-----
From: Jan Beulich <[email protected]>
Sent: 15 June 2020 17:17
To: [email protected]; 'Andrew Cooper' <[email protected]>
Cc: 'Roger Pau Monne' <[email protected]>; [email protected]; 'Wei Liu' <[email protected]>
Subject: Re: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

Quoted 6 lines · Paul Durrant
Quoted 6 lines

-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 11 June 2020 17:26
To: Roger Pau Monne <[email protected]>; [email protected]
Cc: [email protected]; Jan Beulich <[email protected]>; Wei Liu <[email protected]>
Subject: Re: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an

EOI

Quoted 16 lines
Quoted 16 lines

or unmask

Quoted 12 lines · Roger Pau Monne

There's no need to setup a timer for GSIs that are edge triggered, since those don't require any EIO or unmask, and hence couldn't block other interrupts.

Note this is only used by PVH dom0, that can setup the passthrough of edge triggered interrupts from the vIO-APIC. One example of such kind of interrupt that can be used by a PVH dom0 would be the RTC timer.

While there introduce an out label to do the unlock and reduce code duplication.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Strictly speaking these tags were too little for the patch to go in - the change to drivers/passthrough/io.c would also have required Paul's (or my) R-b. I take it that this was sort of implied by the R-a-b.

FAOD I have added my R-b.

Paul

Quoted 2 lines

Jan

Reviewed-by on v2 2/2
#10 lore raw

RE: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

In reply to Roger Pau Monne
Quoted 13 lines

-----Original Message-----
From: Roger Pau Monne <[email protected]>
Sent: 10 June 2020 15:29
To: [email protected]
Cc: [email protected]; Roger Pau Monne <[email protected]>; Jan Beulich <[email protected]>; Andrew Cooper <[email protected]>; Wei Liu <[email protected]>
Subject: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

There's no need to setup a timer for GSIs that are edge triggered, since those don't require any EIO or unmask, and hence couldn't block other interrupts.

Quoted 8 lines

Note this is only used by PVH dom0, that can setup the passthrough of edge triggered interrupts from the vIO-APIC. One example of such kind of interrupt that can be used by a PVH dom0 would be the RTC timer.

While there introduce an out label to do the unlock and reduce code duplication.

Quoted 105 lines

---
Changes since v1:
- Introduce an out label that does the unlock.

---
 xen/drivers/passthrough/io.c  | 24 +++++++++++++++---------
 xen/include/asm-x86/hvm/irq.h |  2 ++
 2 files changed, 17 insertions(+), 9 deletions(-)
xen/drivers/passthrough/io.c +15 -9
index b292e79382..6b1305a3e5 100644
@@ -138,7 +138,8 @@ static void pt_pirq_softirq_reset(struct hvm_pirq_dpci *pirq_dpci)
 
 bool pt_irq_need_timer(uint32_t flags)
 {
-    return !(flags & (HVM_IRQ_DPCI_GUEST_MSI | HVM_IRQ_DPCI_TRANSLATE));
+    return !(flags & (HVM_IRQ_DPCI_GUEST_MSI | HVM_IRQ_DPCI_TRANSLATE |
+                      HVM_IRQ_DPCI_NO_EOI));
 }
 
 static int pt_irq_guest_eoi(struct domain *d, struct hvm_pirq_dpci *pirq_dpci,
@@ -558,6 +559,12 @@ int pt_irq_create_bind(
                      */
                     ASSERT(!mask);
                     share = trigger_mode;
+                    if ( trigger_mode == VIOAPIC_EDGE_TRIG )
+                        /*
+                         * Edge IO-APIC interrupt, no EOI or unmask to perform
+                         * and hence no timer needed.
+                         */
+                        pirq_dpci->flags |= HVM_IRQ_DPCI_NO_EOI;
                 }
             }
 
@@ -897,17 +904,13 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
             send_guest_pirq(d, pirq);
 
             if ( pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI )
-            {
-                spin_unlock(&d->event_lock);
-                return;
-            }
+                goto out;
         }
 
         if ( pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI )
         {
             vmsi_deliver_pirq(d, pirq_dpci);
-            spin_unlock(&d->event_lock);
-            return;
+            goto out;
         }
 
         list_for_each_entry ( digl, &pirq_dpci->digl_list, list )
@@ -920,6 +923,8 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
         if ( pirq_dpci->flags & HVM_IRQ_DPCI_IDENTITY_GSI )
         {
             hvm_gsi_assert(d, pirq->pirq);
+            if ( pirq_dpci->flags & HVM_IRQ_DPCI_NO_EOI )
+                goto out;
             pirq_dpci->pending++;
         }
 
@@ -927,8 +932,7 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
         {
             /* for translated MSI to INTx interrupt, eoi as early as possible */
             __msi_pirq_eoi(pirq_dpci);
-            spin_unlock(&d->event_lock);
-            return;
+            goto out;
         }
 
         /*
@@ -941,6 +945,8 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
         ASSERT(pt_irq_need_timer(pirq_dpci->flags));
         set_timer(&pirq_dpci->timer, NOW() + PT_IRQ_TIME_OUT);
     }
+
+ out:
     spin_unlock(&d->event_lock);
 }
 
xen/include/asm-x86/hvm/irq.h +2 -1
index d306cfeade..532880d497 100644
@@ -121,6 +121,7 @@ struct dev_intx_gsi_link {
 #define _HVM_IRQ_DPCI_GUEST_PCI_SHIFT           4
 #define _HVM_IRQ_DPCI_GUEST_MSI_SHIFT           5
 #define _HVM_IRQ_DPCI_IDENTITY_GSI_SHIFT        6
+#define _HVM_IRQ_DPCI_NO_EOI_SHIFT              7
 #define _HVM_IRQ_DPCI_TRANSLATE_SHIFT          15
 #define HVM_IRQ_DPCI_MACH_PCI        (1u << _HVM_IRQ_DPCI_MACH_PCI_SHIFT)
 #define HVM_IRQ_DPCI_MACH_MSI        (1u << _HVM_IRQ_DPCI_MACH_MSI_SHIFT)
@@ -129,6 +130,7 @@ struct dev_intx_gsi_link {
 #define HVM_IRQ_DPCI_GUEST_PCI       (1u << _HVM_IRQ_DPCI_GUEST_PCI_SHIFT)
 #define HVM_IRQ_DPCI_GUEST_MSI       (1u << _HVM_IRQ_DPCI_GUEST_MSI_SHIFT)
 #define HVM_IRQ_DPCI_IDENTITY_GSI    (1u << _HVM_IRQ_DPCI_IDENTITY_GSI_SHIFT)
+#define HVM_IRQ_DPCI_NO_EOI          (1u << _HVM_IRQ_DPCI_NO_EOI_SHIFT)
 #define HVM_IRQ_DPCI_TRANSLATE       (1u << _HVM_IRQ_DPCI_TRANSLATE_SHIFT)
 
 struct hvm_gmsi_info {
--

2.26.2

#11 lore raw

Re: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

In reply to Roger Pau Monne

xen/drivers/passthrough/io.c int pt_irq_create_bind( · line 567

Quoted 10 lines · Roger Pau Monne
xen/drivers/passthrough/io.c +6 -0
@@ -558,6 +559,12 @@ int pt_irq_create_bind(
                      */
                     ASSERT(!mask);
                     share = trigger_mode;
+                    if ( trigger_mode == VIOAPIC_EDGE_TRIG )
+                        /*
+                         * Edge IO-APIC interrupt, no EOI or unmask to perform
+                         * and hence no timer needed.
+                         */
+                        pirq_dpci->flags |= HVM_IRQ_DPCI_NO_EOI;

Is this really limited to edge triggered IO-APIC interrupts? MSI ones are effectively edge triggered too, aren't they? Along the lines of irq_acktype() maskable MSI may then also not need any such arrangements? The pirq_guest_eoi() -> desc_guest_eoi() path looks to confirm this.

xen/drivers/passthrough/io.c static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci) · line 927

Quoted 6 lines
xen/drivers/passthrough/io.c +2 -0
@@ -920,6 +923,8 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
         if ( pirq_dpci->flags & HVM_IRQ_DPCI_IDENTITY_GSI )
         {
             hvm_gsi_assert(d, pirq->pirq);
+            if ( pirq_dpci->flags & HVM_IRQ_DPCI_NO_EOI )
+                goto out;

Immediately ahead of this there's a similar piece of code dealing with PCI INTx. They're commonly level triggered, but I don't think there's a strict need for this to be the case. At least hvm_pci_intx_assert() -> assert_gsi() -> vioapic_irq_positive_edge() also cover the edge triggered one.

Jan

#12 lore raw

Re: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

Quoted 14 lines · Jan Beulich

xen/drivers/passthrough/io.c int pt_irq_create_bind( · line 567

Quoted 10 lines · Roger Pau Monne
xen/drivers/passthrough/io.c +6 -0
@@ -558,6 +559,12 @@ int pt_irq_create_bind(
                      */
                     ASSERT(!mask);
                     share = trigger_mode;
+                    if ( trigger_mode == VIOAPIC_EDGE_TRIG )
+                        /*
+                         * Edge IO-APIC interrupt, no EOI or unmask to perform
+                         * and hence no timer needed.
+                         */
+                        pirq_dpci->flags |= HVM_IRQ_DPCI_NO_EOI;

Is this really limited to edge triggered IO-APIC interrupts? MSI ones are effectively edge triggered too, aren't they?

MSIs do sometimes require an EOI, depending on whether they can be masked, see irq_acktype.

Quoted 3 lines

Along the lines of irq_acktype() maskable MSI may then also not need any such arrangements? The pirq_guest_eoi() -> desc_guest_eoi() path looks to confirm this.

Yes, that's correct. AFAICT MSI interrupts won't get the EOI timer, since pt_irq_need_timer will return false because the HVM_IRQ_DPCI_GUEST_MSI flag will be set.

Quoted 4 lines
Quoted 4 lines
xen/drivers/passthrough/io.c +0 -0
@@ -920,6 +923,8 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
         if ( pirq_dpci->flags & HVM_IRQ_DPCI_IDENTITY_GSI )
         {
             hvm_gsi_assert(d, pirq->pirq);
Quoted 8 lines

xen/drivers/passthrough/io.c static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci) · line 927

Quoted 2 lines
xen/drivers/passthrough/io.c +2 -0
+            if ( pirq_dpci->flags & HVM_IRQ_DPCI_NO_EOI )
+                goto out;

Immediately ahead of this there's a similar piece of code dealing with PCI INTx. They're commonly level triggered, but I don't think there's a strict need for this to be the case. At least hvm_pci_intx_assert() -> assert_gsi() -> vioapic_irq_positive_edge() also cover the edge triggered one.

Hm, I'm not sure it's safe to passthrough edge triggered IO-APIC interrupts, as Xen will mark those as 'shared' always, and sharing edge interrupts cannot reliably work. In any case the EOI timer is definitely set for those, and needs to be disabled before exiting hvm_dirq_assist.

Thanks, Roger.

#13 lore raw

Re: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

Quoted 15 lines · Roger Pau Monné
Quoted 13 lines · Jan Beulich

xen/drivers/passthrough/io.c static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci) · line 927

Quoted 6 lines · Roger Pau Monne
xen/drivers/passthrough/io.c +2 -0
@@ -920,6 +923,8 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
         if ( pirq_dpci->flags & HVM_IRQ_DPCI_IDENTITY_GSI )
         {
             hvm_gsi_assert(d, pirq->pirq);
+            if ( pirq_dpci->flags & HVM_IRQ_DPCI_NO_EOI )
+                goto out;

Immediately ahead of this there's a similar piece of code dealing with PCI INTx. They're commonly level triggered, but I don't think there's a strict need for this to be the case. At least hvm_pci_intx_assert() -> assert_gsi() -> vioapic_irq_positive_edge() also cover the edge triggered one.

Quoted 5 lines

Hm, I'm not sure it's safe to passthrough edge triggered IO-APIC interrupts, as Xen will mark those as 'shared' always, and sharing edge interrupts cannot reliably work. In any case the EOI timer is definitely set for those, and needs to be disabled before exiting hvm_dirq_assist.

That's the

if ( !is_hardware_domain(d) )
share = BIND_PIRQ__WILL_SHARE;

in pt_irq_create_bind() aiui? I wonder why we have that ... At a guess it's to accommodate pciback in Dom0 also registering a handler. But wasn't it XenoLinux'es pciback only that does so, and upstream's doesn't?

Jan

#14 lore raw

Re: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

Quoted 24 lines · Jan Beulich
Quoted 15 lines · Roger Pau Monné
Quoted 13 lines · Jan Beulich

xen/drivers/passthrough/io.c static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci) · line 927

Quoted 6 lines · Roger Pau Monne
xen/drivers/passthrough/io.c +2 -0
@@ -920,6 +923,8 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
         if ( pirq_dpci->flags & HVM_IRQ_DPCI_IDENTITY_GSI )
         {
             hvm_gsi_assert(d, pirq->pirq);
+            if ( pirq_dpci->flags & HVM_IRQ_DPCI_NO_EOI )
+                goto out;

Immediately ahead of this there's a similar piece of code dealing with PCI INTx. They're commonly level triggered, but I don't think there's a strict need for this to be the case. At least hvm_pci_intx_assert() -> assert_gsi() -> vioapic_irq_positive_edge() also cover the edge triggered one.

Quoted 5 lines

Hm, I'm not sure it's safe to passthrough edge triggered IO-APIC interrupts, as Xen will mark those as 'shared' always, and sharing edge interrupts cannot reliably work. In any case the EOI timer is definitely set for those, and needs to be disabled before exiting hvm_dirq_assist.

That's the

Quoted 7 lines

if ( !is_hardware_domain(d) )
share = BIND_PIRQ__WILL_SHARE;

in pt_irq_create_bind() aiui? I wonder why we have that ... At a guess it's to accommodate pciback in Dom0 also registering a handler. But wasn't it XenoLinux'es pciback only that does so, and upstream's doesn't?

I'm not that familiar with pciback in Linux. I've taken a look and AFAICT modern Linux kernels will register a handler for the PCI interrupts before doing a device reset and when dealing with PV guests that use the pciif protocol (note that such IRQ is also unconditionally marked as shared in Linux).

It might be safe for HVM domU passed through interrupts to set the share bit based on the triggering mode.

Roger.

#15 lore raw

Re: [PATCH for-4.14 v2 2/2] x86/passthrough: introduce a flag for GSIs not requiring an EOI or unmask

In reply to Roger Pau Monne
Quoted 2 lines · Roger Pau Monne

There's no need to setup a timer for GSIs that are edge triggered, since those don't require any EIO or unmask, and hence couldn't block

One small nit. I think you meant "EOI" here.