usb: ohci: Prevent missed ohci interrupts
Subsystems (2)
PATCH v2 1/1 usb: ohci: Prevent missed ohci interrupts
Testing ohci functionality with qemu's pci-ohci emulation often results in ohci interface stalls, resulting in hung task timeouts.
The problem is caused by lost interrupts between the emulation and the Linux kernel code. Additional interrupts raised while the ohci interrupt handler in Linux is running and before the handler clears the interrupt status are not handled. The fix for a similar problem in ehci suggests that the problem is likely caused by edge-triggered MSI interrupts. See commit 0b60557230ad ("usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI") for details.
Ensure that the ohci interrupt code handles all pending interrupts before returning to solve the problem.
---
v2: Only repeat if the interface is still active
drivers/usb/host/ohci-hcd.c | 8 ++++++++ 1 file changed, 8 insertions(+)
drivers/usb/host/ohci-hcd.c +8 -0
@@ -888,6 +888,7 @@ static irqreturn_t ohci_irq (struct usb_hcd *hcd)/* Check for an all 1's result which is a typical consequence * of dead, unclocked, or unplugged (CardBus...) devices */ +again:if (ints == ~(u32 )0 ) { ohci->rh_state =OHCI_RH_HALTED ;ohci_dbg (ohci,"device removed!\n" );
@@ -982,6 +983,13 @@ static irqreturn_t ohci_irq (struct usb_hcd *hcd) }spin_unlock (&ohci->lock); +/* repeat until all enabled interrupts are handled */ +if (ohci->rh_state !=OHCI_RH_HALTED ) { + ints =ohci_readl (ohci, ®s->intrstatus); +if (ints &ohci_readl (ohci, ®s->intrenable)) + goto again; + } +return IRQ_HANDLED ; }
Signature · 1 line
2.39.2
Quoted 11 lines · Guenter Roeck
Testing ohci functionality with qemu's pci-ohci emulation often results in ohci interface stalls, resulting in hung task timeouts.
The problem is caused by lost interrupts between the emulation and the Linux kernel code. Additional interrupts raised while the ohci interrupt handler in Linux is running and before the handler clears the interrupt status are not handled. The fix for a similar problem in ehci suggests that the problem is likely caused by edge-triggered MSI interrupts. See commit
0b60557230ad("usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI") for details.
Quoted 8 lines
Ensure that the ohci interrupt code handles all pending interrupts before returning to solve the problem.
Cc Gerd Hoffmann <[email protected]>Fixes 306c54d0edb6 ("usb: hcd: Try MSI interrupts on PCI devices")Signed-off-by Guenter Roeck <[email protected]>---
v2: Only repeat if the interface is still active
Greg might insist that the patch be CC'ed to [email protected] since it contains a Fixes: tag.
Alan Stern
Quoted 32 lines
drivers/usb/host/ohci-hcd.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/usb/host/ohci-hcd.c +8
-0@@ -888,6 +888,7 @@ static irqreturn_t ohci_irq (struct usb_hcd *hcd)/* Check for an all 1's result which is a typical consequence * of dead, unclocked, or unplugged (CardBus...) devices */ +again:if (ints == ~(u32 )0 ) { ohci->rh_state =OHCI_RH_HALTED ;ohci_dbg (ohci,"device removed!\n" );@@ -982,6 +983,13 @@ static irqreturn_t ohci_irq (struct usb_hcd *hcd) }spin_unlock (&ohci->lock); +/* repeat until all enabled interrupts are handled */ +if (ohci->rh_state !=OHCI_RH_HALTED ) { + ints =ohci_readl (ohci, ®s->intrstatus); +if (ints &ohci_readl (ohci, ®s->intrenable)) + goto again; + } +return IRQ_HANDLED ; }Signature · 1 line
2.39.2
Greg Kroah-Hartman
maintainerQuoted 21 lines · Alan Stern
Quoted 11 lines · Guenter Roeck
Testing ohci functionality with qemu's pci-ohci emulation often results in ohci interface stalls, resulting in hung task timeouts.
The problem is caused by lost interrupts between the emulation and the Linux kernel code. Additional interrupts raised while the ohci interrupt handler in Linux is running and before the handler clears the interrupt status are not handled. The fix for a similar problem in ehci suggests that the problem is likely caused by edge-triggered MSI interrupts. See commit
0b60557230ad("usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI") for details.Quoted 8 lines
Ensure that the ohci interrupt code handles all pending interrupts before returning to solve the problem.
Cc Gerd Hoffmann <[email protected]>Fixes 306c54d0edb6 ("usb: hcd: Try MSI interrupts on PCI devices")Signed-off-by Guenter Roeck <[email protected]>---
v2: Only repeat if the interface is still active
Quoted 4 lines
Reviewed-by Alan Stern <[email protected]>Greg might insist that the patch be CC'ed to [email protected] since it contains a Fixes: tag.
I'll add that by hand, no worries.
Quoted 22 lines · Greg Kroah-Hartman
Quoted 21 lines · Alan Stern
Quoted 19 lines · Guenter Roeck
Testing ohci functionality with qemu's pci-ohci emulation often results in ohci interface stalls, resulting in hung task timeouts.
The problem is caused by lost interrupts between the emulation and the Linux kernel code. Additional interrupts raised while the ohci interrupt handler in Linux is running and before the handler clears the interrupt status are not handled. The fix for a similar problem in ehci suggests that the problem is likely caused by edge-triggered MSI interrupts. See commit
0b60557230ad("usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI") for details.Ensure that the ohci interrupt code handles all pending interrupts before returning to solve the problem.
Cc Gerd Hoffmann <[email protected]>Fixes 306c54d0edb6 ("usb: hcd: Try MSI interrupts on PCI devices")Signed-off-by Guenter Roeck <[email protected]>---
v2: Only repeat if the interface is still active
Quoted 6 lines
Quoted 4 lines
Reviewed-by Alan Stern <[email protected]>Greg might insist that the patch be CC'ed to [email protected] since it contains a Fixes: tag.
I'll add that by hand, no worries.
Thanks!
Guenter
Quoted 21 lines · Alan Stern
Quoted 11 lines · Guenter Roeck
Testing ohci functionality with qemu's pci-ohci emulation often results in ohci interface stalls, resulting in hung task timeouts.
The problem is caused by lost interrupts between the emulation and the Linux kernel code. Additional interrupts raised while the ohci interrupt handler in Linux is running and before the handler clears the interrupt status are not handled. The fix for a similar problem in ehci suggests that the problem is likely caused by edge-triggered MSI interrupts. See commit
0b60557230ad("usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI") for details.Quoted 8 lines
Ensure that the ohci interrupt code handles all pending interrupts before returning to solve the problem.
Cc Gerd Hoffmann <[email protected]>Fixes 306c54d0edb6 ("usb: hcd: Try MSI interrupts on PCI devices")Signed-off-by Guenter Roeck <[email protected]>---
v2: Only repeat if the interface is still active
Quoted 5 lines
Reviewed-by Alan Stern <[email protected]>Greg might insist that the patch be CC'ed to [email protected] since it contains a Fixes: tag.
Sure, I'll be happy to send v3 and add it if he wants me to do that. I never know if I should add it myself or if this is supposed to be a maintainer privilege.
Thanks,
Guenter
From: Guenter Roeck
Quoted 38 lines
Sent: 24 April 2024 21:00
Testing ohci functionality with qemu's pci-ohci emulation often results in ohci interface stalls, resulting in hung task timeouts.
The problem is caused by lost interrupts between the emulation and the Linux kernel code. Additional interrupts raised while the ohci interrupt handler in Linux is running and before the handler clears the interrupt status are not handled. The fix for a similar problem in ehci suggests that the problem is likely caused by edge-triggered MSI interrupts. See commit
0b60557230ad("usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI") for details.Ensure that the ohci interrupt code handles all pending interrupts before returning to solve the problem.
Cc Gerd Hoffmann <[email protected]>Fixes 306c54d0edb6 ("usb: hcd: Try MSI interrupts on PCI devices")Signed-off-by Guenter Roeck <[email protected]>---
v2: Only repeat if the interface is still activedrivers/usb/host/ohci-hcd.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/usb/host/ohci-hcd.c +1
-0@@ -888,6 +888,7 @@ static irqreturn_t ohci_irq (struct usb_hcd *hcd)/* Check for an all 1's result which is a typical consequence * of dead, unclocked, or unplugged (CardBus...) devices */ +again:if (ints == ~(u32 )0 ) { ohci->rh_state =OHCI_RH_HALTED ;ohci_dbg (ohci,"device removed!\n" );
drivers/usb/host/ohci-hcd.c static irqreturn_t ohci_irq (struct usb_hcd *hcd) · line 989
Quoted 7 lines
drivers/usb/host/ohci-hcd.c +4
-0@@ -982,6 +983,13 @@ static irqreturn_t ohci_irq (struct usb_hcd *hcd) }spin_unlock (&ohci->lock); +/* repeat until all enabled interrupts are handled */ +if (ohci->rh_state !=OHCI_RH_HALTED ) { + ints =ohci_readl (ohci, ®s->intrstatus); +if (ints &ohci_readl (ohci, ®s->intrenable))
Doesn't the driver know which interrupts are enabled? So it should be able to avoid doing two (likely) slow io reads? (PCIe reads are pretty much guaranteed to be high latency.)
David
Quoted 9 lines
drivers/usb/host/ohci-hcd.c +3
-1+ goto again; + } +return IRQ_HANDLED ; } --2.39.2
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Quoted 42 lines · David Laight
From: Guenter Roeck
Quoted 41 lines
Sent: 24 April 2024 21:00
Testing ohci functionality with qemu's pci-ohci emulation often results in ohci interface stalls, resulting in hung task timeouts.
The problem is caused by lost interrupts between the emulation and the Linux kernel code. Additional interrupts raised while the ohci interrupt handler in Linux is running and before the handler clears the interrupt status are not handled. The fix for a similar problem in ehci suggests that the problem is likely caused by edge-triggered MSI interrupts. See commit
0b60557230ad("usb: ehci: Prevent missed ehci interrupts with edge-triggered MSI") for details.Ensure that the ohci interrupt code handles all pending interrupts before returning to solve the problem.
Cc Gerd Hoffmann <[email protected]>Fixes 306c54d0edb6 ("usb: hcd: Try MSI interrupts on PCI devices")Signed-off-by Guenter Roeck <[email protected]>---
v2: Only repeat if the interface is still activedrivers/usb/host/ohci-hcd.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/usb/host/ohci-hcd.c +1
-0@@ -888,6 +888,7 @@ static irqreturn_t ohci_irq (struct usb_hcd *hcd)/* Check for an all 1's result which is a typical consequence * of dead, unclocked, or unplugged (CardBus...) devices */ +again:if (ints == ~(u32 )0 ) { ohci->rh_state =OHCI_RH_HALTED ;ohci_dbg (ohci,"device removed!\n" );@@ -982,6 +983,13 @@ static irqreturn_t ohci_irq (struct usb_hcd *hcd) }spin_unlock (&ohci->lock);
Quoted 9 lines
drivers/usb/host/ohci-hcd.c static irqreturn_t ohci_irq (struct usb_hcd *hcd) · line 989
Quoted 4 lines
drivers/usb/host/ohci-hcd.c +4
-0+/* repeat until all enabled interrupts are handled */ +if (ohci->rh_state !=OHCI_RH_HALTED ) { + ints =ohci_readl (ohci, ®s->intrstatus); +if (ints &ohci_readl (ohci, ®s->intrenable))Doesn't the driver know which interrupts are enabled? So it should be able to avoid doing two (likely) slow io reads? (PCIe reads are pretty much guaranteed to be high latency.)
No, the driver does not cache intrenable.
Guenter
Hi,
Quoted 10 lines
Quoted 8 lines
drivers/usb/host/ohci-hcd.c static irqreturn_t ohci_irq (struct usb_hcd *hcd) · line 989
Quoted 4 lines
drivers/usb/host/ohci-hcd.c +4
-0+/* repeat until all enabled interrupts are handled */ +if (ohci->rh_state !=OHCI_RH_HALTED ) { + ints =ohci_readl (ohci, ®s->intrstatus); +if (ints &ohci_readl (ohci, ®s->intrenable))Doesn't the driver know which interrupts are enabled? So it should be able to avoid doing two (likely) slow io reads? (PCIe reads are pretty much guaranteed to be high latency.)
No, the driver does not cache intrenable.
Does the driver ever change intrenable after initialization?
PCIe reads are expensive, especially in virtual machines where this goes vmexit to qemu, so doing that for a piece of information the driver should have (or is able to calculate) should indeed better be avoided.
take care,
Gerd
Quoted 11 lines · Gerd Hoffmann
Hi,Quoted 9 lines
Quoted 8 lines
drivers/usb/host/ohci-hcd.c static irqreturn_t ohci_irq (struct usb_hcd *hcd) · line 989
Quoted 4 lines
drivers/usb/host/ohci-hcd.c +4
-0+/* repeat until all enabled interrupts are handled */ +if (ohci->rh_state !=OHCI_RH_HALTED ) { + ints =ohci_readl (ohci, ®s->intrstatus); +if (ints &ohci_readl (ohci, ®s->intrenable))Doesn't the driver know which interrupts are enabled? So it should be able to avoid doing two (likely) slow io reads? (PCIe reads are pretty much guaranteed to be high latency.)
Quoted 4 lines
Quoted 1 line
No, the driver does not cache intrenable.
Does the driver ever change intrenable after initialization?
$ git grep -e intrenable -e intrdisable drivers/usb/host/*ohci*c | grep ohci_writel drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, (u32) ~0, &ohci->regs->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_OC, &ohci->regs->intrenable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_MIE, &ohci->regs->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, mask, &ohci->regs->intrenable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_UE, ®s->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, OHCI_INTR_RHSC, ®s->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_SF, ®s->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_MIE, ®s->intrenable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_MIE, &ohci->regs->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, OHCI_INTR_MIE, &ohci->regs->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, OHCI_INTR_MIE, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel(ohci, OHCI_INTR_SF, &ohci->regs->intrdisable); drivers/usb/host/ohci-hub.c: ohci_writel (ohci, OHCI_INTR_INIT, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel (ohci, OHCI_INTR_SF, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel(ohci, rhsc_enable, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel(ohci, OHCI_INTR_RHSC, &ohci->regs->intrenable); drivers/usb/host/ohci-q.c: ohci_writel (ohci, OHCI_INTR_SF, &ohci->regs->intrenable);
Quoted 5 lines
PCIe reads are expensive, especially in virtual machines where this goes vmexit to qemu, so doing that for a piece of information the driver should have (or is able to calculate) should indeed better be avoided.
I would agree, but I really think that should be a separate patch if implemented.
Guenter
From: Guenter Roeck
Quoted 29 lines
Sent: 29 April 2024 14:34
Quoted 11 lines · Gerd Hoffmann
Hi,Quoted 9 lines
Quoted 8 lines
drivers/usb/host/ohci-hcd.c static irqreturn_t ohci_irq (struct usb_hcd *hcd) · line 989
Quoted 4 lines
drivers/usb/host/ohci-hcd.c +4
-0+/* repeat until all enabled interrupts are handled */ +if (ohci->rh_state !=OHCI_RH_HALTED ) { + ints =ohci_readl (ohci, ®s->intrstatus); +if (ints &ohci_readl (ohci, ®s->intrenable))Doesn't the driver know which interrupts are enabled? So it should be able to avoid doing two (likely) slow io reads? (PCIe reads are pretty much guaranteed to be high latency.)
Quoted 4 lines
Quoted 1 line
No, the driver does not cache intrenable.
Does the driver ever change intrenable after initialization?
$ git grep -e intrenable -e intrdisable drivers/usb/host/*ohci*c | grep ohci_writel drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, (u32) ~0, &ohci->regs->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_OC, &ohci->regs->intrenable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_MIE, &ohci->regs->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, mask, &ohci->regs->intrenable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_UE, ®s->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, OHCI_INTR_RHSC, ®s->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_SF, ®s->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_MIE, ®s->intrenable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_MIE, &ohci->regs->intrdisable);
Quoted 8 lines
drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, OHCI_INTR_MIE, &ohci->regs->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, OHCI_INTR_MIE, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel(ohci, OHCI_INTR_SF, &ohci->regs->intrdisable); drivers/usb/host/ohci-hub.c: ohci_writel (ohci, OHCI_INTR_INIT, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel (ohci, OHCI_INTR_SF, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel(ohci, rhsc_enable, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel(ohci, OHCI_INTR_RHSC, &ohci->regs->intrenable); drivers/usb/host/ohci-q.c: ohci_writel (ohci, OHCI_INTR_SF, &ohci->regs->intrenable);
At least the hardware has separate enable/disable registers so the driver isn't doing RMW cycles.
I'd guess that the normal condition is that no interrupts are pending. So it can be held to one (slow) read by checking:
if (ints && (ints & ohci_readl(ohci, ®s->intrenable)))
Although maybe there are some 'never enabled' interrupts that might need masking?
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Quoted 39 lines · David Laight
From: Guenter Roeck
Quoted 29 lines
Sent: 29 April 2024 14:34
Quoted 11 lines · Gerd Hoffmann
Hi,Quoted 9 lines
Quoted 8 lines
drivers/usb/host/ohci-hcd.c static irqreturn_t ohci_irq (struct usb_hcd *hcd) · line 989
Quoted 4 lines
drivers/usb/host/ohci-hcd.c +4
-0+/* repeat until all enabled interrupts are handled */ +if (ohci->rh_state !=OHCI_RH_HALTED ) { + ints =ohci_readl (ohci, ®s->intrstatus); +if (ints &ohci_readl (ohci, ®s->intrenable))Doesn't the driver know which interrupts are enabled? So it should be able to avoid doing two (likely) slow io reads? (PCIe reads are pretty much guaranteed to be high latency.)
Quoted 4 lines
Quoted 1 line
No, the driver does not cache intrenable.
Does the driver ever change intrenable after initialization?
$ git grep -e intrenable -e intrdisable drivers/usb/host/*ohci*c | grep ohci_writel drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, (u32) ~0, &ohci->regs->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_OC, &ohci->regs->intrenable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_MIE, &ohci->regs->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, mask, &ohci->regs->intrenable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_UE, ®s->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, OHCI_INTR_RHSC, ®s->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_SF, ®s->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_MIE, ®s->intrenable); drivers/usb/host/ohci-hcd.c: ohci_writel (ohci, OHCI_INTR_MIE, &ohci->regs->intrdisable);Quoted 8 lines
drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, OHCI_INTR_MIE, &ohci->regs->intrdisable); drivers/usb/host/ohci-hcd.c: ohci_writel(ohci, OHCI_INTR_MIE, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel(ohci, OHCI_INTR_SF, &ohci->regs->intrdisable); drivers/usb/host/ohci-hub.c: ohci_writel (ohci, OHCI_INTR_INIT, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel (ohci, OHCI_INTR_SF, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel(ohci, rhsc_enable, &ohci->regs->intrenable); drivers/usb/host/ohci-hub.c: ohci_writel(ohci, OHCI_INTR_RHSC, &ohci->regs->intrenable); drivers/usb/host/ohci-q.c: ohci_writel (ohci, OHCI_INTR_SF, &ohci->regs->intrenable);
Quoted 6 lines
At least the hardware has separate enable/disable registers so the driver isn't doing RMW cycles.
I'd guess that the normal condition is that no interrupts are pending. So it can be held to one (slow) read by checking:
if (ints && (ints & ohci_readl(ohci, ®s->intrenable)))
Guess that can't hurt. I'll send v3.
Thanks,
Guenter