x86 PCI: Do not use interrupt links for devices using MSI-X
PATCH 1/1 x86 PCI: Do not use interrupt links for devices using MSI-X
From: Rafael J. Wysocki <[email protected]>
pcibios_enable_device() and pcibios_disable_device() don't handle IRQs for devices that have MSI enabled and it should tread the devices with MSI-X enabled in the same way.
--- arch/x86/pci/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/pci/common.c +2 -2
@@ -551,14 +551,14 @@ int pcibios_enable_device(struct pci_devif ((err =pci_enable_resources (dev, mask)) <0 )return err; -if (!dev->msi_enabled) +if (!dev->msi_enabled && !dev->msix_enabled)return pcibios_enable_irq (dev);return 0 ; }void pcibios_disable_device (struct pci_dev *dev) { -if (!dev->msi_enabled && pcibios_disable_irq) +if (!dev->msi_enabled && !dev->msix_enabled && pcibios_disable_irq)pcibios_disable_irq (dev); }
Quoted 4 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
pcibios_enable_device()andpcibios_disable_device()don't handle IRQs for devices that have MSI enabled and it should tread the
s/tread/treat
Quoted 6 lines
devices with MSI-X enabled in the same way.
Signed-off-by Rafael J. Wysocki <[email protected]>--- arch/x86/pci/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
looks good - Jesse, what do you think?
Rafael, i'm curious is this in response to some regression/bug? Did some box or driver get confused by us enabling/disabling the GSI? Some IRQ flood perhaps?
btw., there's a small observation:
Quoted 1 line
+ if (!dev->msi_enabled && !dev->msix_enabled)
maybe a "pci_has_gsi_irq()" wrapper would make these checks cleaner and would make things more robust, should there be any new IRQ delivery method be introduced in the future?
Ingo
Quoted 9 lines · Ingo Molnar
Quoted 4 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
pcibios_enable_device()andpcibios_disable_device()don't handle IRQs for devices that have MSI enabled and it should tread thes/tread/treat
Ah, thanks.
Quoted 7 lines
Quoted 6 lines
devices with MSI-X enabled in the same way.
Signed-off-by Rafael J. Wysocki <[email protected]>--- arch/x86/pci/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Quoted 5 lines
looks good - Jesse, what do you think?
Rafael, i'm curious is this in response to some regression/bug? Did some box or driver get confused by us enabling/disabling the GSI? Some IRQ flood perhaps?
Well, I don't have any MSI-X capable boxes around. :-)
I was just reviewing the code and spotted this.
Quoted 7 lines
btw., there's a small observation:
Quoted 1 line
+ if (!dev->msi_enabled && !dev->msix_enabled)
maybe a "
pci_has_gsi_irq()" wrapper would make these checks cleaner and would make things more robust, should there be any new IRQ delivery method be introduced in the future?
Well, perhaps something like the patch below?
Thanks,
Rafael
---
Subject: x86 PCI: Do not use interrupt links for devices using MSI-X (rev. 2)
From: Rafael J. Wysocki <[email protected]>
pcibios_enable_device() and pcibios_disable_device() don't handle IRQs for devices that have MSI enabled and it should treat the devices with MSI-X enabled in the same way.
--- arch/x86/pci/common.c | 4 ++-- include/linux/pci.h | 9 +++++++++ 2 files changed, 11 insertions(+), 2 deletions(-)
arch/x86/pci/common.c +2 -2
@@ -551,14 +551,14 @@ int pcibios_enable_device(struct pci_devif ((err =pci_enable_resources (dev, mask)) <0 )return err; -if (!dev->msi_enabled) +if (!pci_msi_enabled (dev))return pcibios_enable_irq (dev);return 0 ; }void pcibios_disable_device (struct pci_dev *dev) { -if (!dev->msi_enabled && pcibios_disable_irq) +if (!pci_msi_enabled (dev) && pcibios_disable_irq)pcibios_disable_irq (dev); }
include/linux/pci.h +9 -0
@@ -274,6 +274,15 @@ static inline void pci_add_saved_cap(strhlist_add_head (&new_cap->next, &pci_dev->saved_cap_space); } +#ifdef CONFIG_PCI_MSI +static inline bool pci_msi_enabled (struct pci_dev *pci_dev) +{ +return dev->msi_enabled || dev->msix_enabled; +} +#else +static inline bool pci_msi_enabled (struct pci_dev *pci_dev) {return false; } +#endif +/* * For PCI devices, the region numbers are assigned this way: *
Quoted 44 lines · Rafael J. Wysocki
Quoted 7 lines · Ingo Molnar
Quoted 4 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
pcibios_enable_device()andpcibios_disable_device()don't handle IRQs for devices that have MSI enabled and it should tread thes/tread/treat
Ah, thanks.
Quoted 7 lines
Quoted 6 lines
devices with MSI-X enabled in the same way.
Signed-off-by Rafael J. Wysocki <[email protected]>--- arch/x86/pci/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)Quoted 5 lines
looks good - Jesse, what do you think?
Rafael, i'm curious is this in response to some regression/bug? Did some box or driver get confused by us enabling/disabling the GSI? Some IRQ flood perhaps?
Well, I don't have any MSI-X capable boxes around. :-)
I was just reviewing the code and spotted this.
Quoted 6 lines
btw., there's a small observation:
Quoted 1 line
+ if (!dev->msi_enabled && !dev->msix_enabled)
maybe a "
pci_has_gsi_irq()" wrapper would make these checks cleaner and would make things more robust, should there be any new IRQ delivery method be introduced in the future?Well, perhaps something like the patch below?
Thanks,
Rafael---
Subject: x86 PCI: Do not use interrupt links for devices using MSI-X (rev.
2) From: Rafael J. Wysocki <[email protected]>
Quoted 5 lines
pcibios_enable_device()andpcibios_disable_device()don't handle IRQs for devices that have MSI enabled and it should treat the devices with MSI-X enabled in the same way.Signed-off-by Rafael J. Wysocki <[email protected]>
Looks good, applied this one.
Thanks,
Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
Quoted 8 lines · Ingo Molnar
Quoted 4 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
pcibios_enable_device()andpcibios_disable_device()don't handle IRQs for devices that have MSI enabled and it should tread thes/tread/treat
Quoted 8 lines
Quoted 6 lines
devices with MSI-X enabled in the same way.
Signed-off-by Rafael J. Wysocki <[email protected]>--- arch/x86/pci/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)looks good - Jesse, what do you think?
Yeah, seems obviously correct, I'll queue it up.
Quoted 10 lines
Rafael, i'm curious is this in response to some regression/bug? Did some box or driver get confused by us enabling/disabling the GSI? Some IRQ flood perhaps?
btw., there's a small observation:
Quoted 1 line
+ if (!dev->msi_enabled && !dev->msix_enabled)
maybe a "
pci_has_gsi_irq()" wrapper would make these checks cleaner and would make things more robust, should there be any new IRQ delivery method be introduced in the future?
pci_has_msi_irq surely? Otherwise we'll catch pretty much everything? Or did you mean !pci_has_gsi_irq() here instead?
Thanks,
Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
Quoted 25 lines · Jesse Barnes
Quoted 8 lines · Ingo Molnar
Quoted 4 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
pcibios_enable_device()andpcibios_disable_device()don't handle IRQs for devices that have MSI enabled and it should tread thes/tread/treat
Quoted 8 lines
Quoted 6 lines
devices with MSI-X enabled in the same way.
Signed-off-by Rafael J. Wysocki <[email protected]>--- arch/x86/pci/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)looks good - Jesse, what do you think?
Yeah, seems obviously correct, I'll queue it up.
Quoted 5 lines
Rafael, i'm curious is this in response to some regression/bug? Did some box or driver get confused by us enabling/disabling the GSI? Some IRQ flood perhaps?
btw., there's a small observation:
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
+ if (!dev->msi_enabled && !dev->msix_enabled)
maybe a "
pci_has_gsi_irq()" wrapper would make these checks cleaner and would make things more robust, should there be any new IRQ delivery method be introduced in the future?
pci_has_msi_irqsurely? Otherwise we'll catch pretty much everything? Or did you mean !pci_has_gsi_irq() here instead?
Well - here the check is: "if (not MSI or MSIX)" in essence. I thought that it might be confusing to call it _msi() as well, so we could approach it via the inverse space: general system interrupts (GSIs) - which are device irqs that are neither MSI nor MSIX.
But if pci_has_msi_irq() can cleanly include the MSIX portion too, that's fine too. (MSI-X is really MSI with wider eventing capabilities but otherwise non-GSI just as much - and we dont want to enable (or even touch) the legacy IRQ line registers for any of them, even if they happen to be enumerated)
Right?
Ingo
Quoted 40 lines · Ingo Molnar
Quoted 25 lines · Jesse Barnes
Quoted 8 lines · Ingo Molnar
Quoted 4 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
pcibios_enable_device()andpcibios_disable_device()don't handle IRQs for devices that have MSI enabled and it should tread thes/tread/treat
Quoted 8 lines
Quoted 6 lines
devices with MSI-X enabled in the same way.
Signed-off-by Rafael J. Wysocki <[email protected]>--- arch/x86/pci/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)looks good - Jesse, what do you think?
Yeah, seems obviously correct, I'll queue it up.
Quoted 5 lines
Rafael, i'm curious is this in response to some regression/bug? Did some box or driver get confused by us enabling/disabling the GSI? Some IRQ flood perhaps?
btw., there's a small observation:
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
+ if (!dev->msi_enabled && !dev->msix_enabled)
maybe a "
pci_has_gsi_irq()" wrapper would make these checks cleaner and would make things more robust, should there be any new IRQ delivery method be introduced in the future?
pci_has_msi_irqsurely? Otherwise we'll catch pretty much everything? Or did you mean !pci_has_gsi_irq() here instead?Well - here the check is: "if (not MSI or MSIX)" in essence. I thought that it might be confusing to call it
_msi()as well, so we could approach it via the inverse space: general system interrupts (GSIs) - which are device irqs that are neither MSI nor MSIX.
Quoted 7 lines
But if
pci_has_msi_irq()can cleanly include the MSIX portion too, that's fine too. (MSI-X is really MSI with wider eventing capabilities but otherwise non-GSI just as much - and we dont want to enable (or even touch) the legacy IRQ line registers for any of them, even if they happen to be enumerated)Right?
Right, I see where you're coming from. However, I've queued up Rafael's last patch with some fixes for dev vs. pci_dev and a name collision (pci_msi_enabled -> pci_dev_msi_enabled). Bjorn caught the fact that some other arches may want similar treatment too, I think Rafael is checking that out.
Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
Quoted 47 lines · Jesse Barnes
Quoted 46 lines · Ingo Molnar
Quoted 25 lines · Jesse Barnes
Quoted 8 lines · Ingo Molnar
Quoted 4 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
pcibios_enable_device()andpcibios_disable_device()don't handle IRQs for devices that have MSI enabled and it should tread thes/tread/treat
Quoted 8 lines
Quoted 6 lines
devices with MSI-X enabled in the same way.
Signed-off-by Rafael J. Wysocki <[email protected]>--- arch/x86/pci/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)looks good - Jesse, what do you think?
Yeah, seems obviously correct, I'll queue it up.
Quoted 5 lines
Rafael, i'm curious is this in response to some regression/bug? Did some box or driver get confused by us enabling/disabling the GSI? Some IRQ flood perhaps?
btw., there's a small observation:
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
+ if (!dev->msi_enabled && !dev->msix_enabled)
maybe a "
pci_has_gsi_irq()" wrapper would make these checks cleaner and would make things more robust, should there be any new IRQ delivery method be introduced in the future?
pci_has_msi_irqsurely? Otherwise we'll catch pretty much everything? Or did you mean !pci_has_gsi_irq() here instead?Well - here the check is: "if (not MSI or MSIX)" in essence. I thought that it might be confusing to call it
_msi()as well, so we could approach it via the inverse space: general system interrupts (GSIs) - which are device irqs that are neither MSI nor MSIX.But if
pci_has_msi_irq()can cleanly include the MSIX portion too, that's fine too. (MSI-X is really MSI with wider eventing capabilities but otherwise non-GSI just as much - and we dont want to enable (or even touch) the legacy IRQ line registers for any of them, even if they happen to be enumerated)
Quoted 7 lines
Quoted 1 line
Right?
Right, I see where you're coming from. However, I've queued up Rafael's last patch with some fixes for dev vs.
pci_devand a name collision (pci_msi_enabled->pci_dev_msi_enabled). Bjorn caught the fact that some other arches may want similar treatment too, I think Rafael is checking that out.
Sure - that sounds good too!
Ingo
Quoted 47 lines · Jesse Barnes
Quoted 46 lines · Ingo Molnar
Quoted 25 lines · Jesse Barnes
Quoted 8 lines · Ingo Molnar
Quoted 4 lines · Rafael J. Wysocki
From: Rafael J. Wysocki <[email protected]>
pcibios_enable_device()andpcibios_disable_device()don't handle IRQs for devices that have MSI enabled and it should tread thes/tread/treat
Quoted 8 lines
Quoted 6 lines
devices with MSI-X enabled in the same way.
Signed-off-by Rafael J. Wysocki <[email protected]>--- arch/x86/pci/common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)looks good - Jesse, what do you think?
Yeah, seems obviously correct, I'll queue it up.
Quoted 5 lines
Rafael, i'm curious is this in response to some regression/bug? Did some box or driver get confused by us enabling/disabling the GSI? Some IRQ flood perhaps?
btw., there's a small observation:
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
+ if (!dev->msi_enabled && !dev->msix_enabled)
maybe a "
pci_has_gsi_irq()" wrapper would make these checks cleaner and would make things more robust, should there be any new IRQ delivery method be introduced in the future?
pci_has_msi_irqsurely? Otherwise we'll catch pretty much everything? Or did you mean !pci_has_gsi_irq() here instead?Well - here the check is: "if (not MSI or MSIX)" in essence. I thought that it might be confusing to call it
_msi()as well, so we could approach it via the inverse space: general system interrupts (GSIs) - which are device irqs that are neither MSI nor MSIX.But if
pci_has_msi_irq()can cleanly include the MSIX portion too, that's fine too. (MSI-X is really MSI with wider eventing capabilities but otherwise non-GSI just as much - and we dont want to enable (or even touch) the legacy IRQ line registers for any of them, even if they happen to be enumerated)
Quoted 7 lines
Quoted 1 line
Right?
Right, I see where you're coming from. However, I've queued up Rafael's last patch with some fixes for dev vs.
pci_devand a name collision (pci_msi_enabled->pci_dev_msi_enabled). Bjorn caught the fact that some other arches may want similar treatment too, I think Rafael is checking that out.
Yes, I am.
Thanks,
Rafael
Quoted 3 lines · Rafael J. Wysocki
pcibios_enable_device()andpcibios_disable_device()don't handle IRQs for devices that have MSI enabled and it should tread the devices with MSI-X enabled in the same way.
There are other places that need similar fixes, too, aren't there? I see cris, frv, ia64, and a driver or two testing dev->msi_enabled.
Quoted 18 lines
arch/x86/pci/common.c +2
-2@@ -551,14 +551,14 @@ int pcibios_enable_device(struct pci_devif ((err =pci_enable_resources (dev, mask)) <0 )return err; -if (!dev->msi_enabled) +if (!dev->msi_enabled && !dev->msix_enabled)return pcibios_enable_irq (dev);return 0 ; }void pcibios_disable_device (struct pci_dev *dev) { -if (!dev->msi_enabled && pcibios_disable_irq) +if (!dev->msi_enabled && !dev->msix_enabled && pcibios_disable_irq)pcibios_disable_irq (dev); }
Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Rafael J. Wysocki
pcibios_enable_device()andpcibios_disable_device()don't handle IRQs for devices that have MSI enabled and it should tread the devices with MSI-X enabled in the same way.There are other places that need similar fixes, too, aren't there? I see cris, frv, ia64, and a driver or two testing
dev->msi_enabled.
Well, I didn't look at the other places, just found this one while reviewing the code.
I'll check them later this week.
Quoted 18 lines
Quoted 18 lines
arch/x86/pci/common.c +2
-2@@ -551,14 +551,14 @@ int pcibios_enable_device(struct pci_devif ((err =pci_enable_resources (dev, mask)) <0 )return err; -if (!dev->msi_enabled) +if (!dev->msi_enabled && !dev->msix_enabled)return pcibios_enable_irq (dev);return 0 ; }void pcibios_disable_device (struct pci_dev *dev) { -if (!dev->msi_enabled && pcibios_disable_irq) +if (!dev->msi_enabled && !dev->msix_enabled && pcibios_disable_irq)pcibios_disable_irq (dev); }