PCI/MSI: Set device flag indicating only 32-bit MSI support
PATCH 1/1 PCI/MSI: Set device flag indicating only 32-bit MSI support
There are devices (Ex:- Marvell SATA controller) that don't support 64-bit MSIs and the same is advertised through their MSI capability register. Set no_64bit_msi flag explicitly for such devices in the MSI setup code so that the msi_verify_entries() API would catch if the MSI arch code tries to use 64-bit MSI.
--- drivers/pci/msi.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/pci/msi.c +4 -2
@@ -581,10 +581,12 @@ msi_setup_entry(struct pci_dev *dev, int nvec, struct irq_affinity *affd) entry->msi_attrib.multi_cap = (control &PCI_MSI_FLAGS_QMASK ) >>1 ; entry->msi_attrib.multiple =ilog2 (__roundup_pow_of_two (nvec)); -if (control &PCI_MSI_FLAGS_64BIT ) +if (control &PCI_MSI_FLAGS_64BIT ) { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_64 ; -else + }else { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_32 ; + dev->no_64bit_msi =1 ; + }/* Save the initial mask status */ if (entry->msi_attrib.maskbit)
Signature · 1 line
2.17.1
Quoted 3 lines · Vidya Sagar
There are devices (Ex:- Marvell SATA controller) that don't support 64-bit MSIs and the same is advertised through their MSI capability register.
I *think* you're saying these devices behave correctly per spec: they don't support 64-bit MSI, and they don't advertise support for 64-bit MSI. Right?
Quoted 3 lines
Set
no_64bit_msiflag explicitly for such devices in the MSI setup code so that themsi_verify_entries()API would catch if the MSI arch code tries to use 64-bit MSI.
And you want msi_verify_entries() to catch attempts by the arch code to assign a 64-bit MSI address?
That sounds OK, but the error message ("Device has broken 64-bit MSI") is not appropriate if the device is actually *not* broken.
Quoted 27 lines
Signed-off-by Vidya Sagar <[email protected]>--- drivers/pci/msi.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)drivers/pci/msi.c +4
-2@@ -581,10 +581,12 @@ msi_setup_entry(struct pci_dev *dev, int nvec, struct irq_affinity *affd) entry->msi_attrib.multi_cap = (control &PCI_MSI_FLAGS_QMASK ) >>1 ; entry->msi_attrib.multiple =ilog2 (__roundup_pow_of_two (nvec)); -if (control &PCI_MSI_FLAGS_64BIT ) +if (control &PCI_MSI_FLAGS_64BIT ) { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_64 ; -else + }else { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_32 ; + dev->no_64bit_msi =1 ; + }/* Save the initial mask status */ if (entry->msi_attrib.maskbit)Signature · 1 line
2.17.1
Quoted 11 lines · Bjorn Helgaas
External email: Use caution opening links or attachments
Quoted 3 lines · Vidya Sagar
There are devices (Ex:- Marvell SATA controller) that don't support 64-bit MSIs and the same is advertised through their MSI capability register.
I *think* you're saying these devices behave correctly per spec: they don't support 64-bit MSI, and they don't advertise support for 64-bit MSI. Right?
Yes. That is what I intended to say.
Quoted 7 lines
Quoted 4 lines
Set
no_64bit_msiflag explicitly for such devices in the MSI setup code so that themsi_verify_entries()API would catch if the MSI arch code tries to use 64-bit MSI.And you want
msi_verify_entries()to catch attempts by the arch code to assign a 64-bit MSI address?
Yes.
Quoted 3 lines
That sounds OK, but the error message ("Device has broken 64-bit MSI") is not appropriate if the device is actually *not* broken.
Ok. I didn't change the existing error message. I'll change it to cover both the scenarios i.e. either the device is broken or the device doesn't really support 64-bit MSI.
Quoted 28 lines
Quoted 28 lines
Signed-off-by Vidya Sagar <[email protected]>--- drivers/pci/msi.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)drivers/pci/msi.c +4
-3@@ -581,10 +581,12 @@ msi_setup_entry(struct pci_dev *dev, int nvec, struct irq_affinity *affd) entry->msi_attrib.multi_cap = (control &PCI_MSI_FLAGS_QMASK ) >>1 ; entry->msi_attrib.multiple =ilog2 (__roundup_pow_of_two (nvec)); -if (control &PCI_MSI_FLAGS_64BIT ) +if (control &PCI_MSI_FLAGS_64BIT ) { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_64 ; -else + }else { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_32 ; + dev->no_64bit_msi =1 ; + }/* Save the initial mask status */ if (entry->msi_attrib.maskbit) --2.17.1
PATCH v2 1/1 PCI/MSI: Set device flag indicating only 32-bit MSI support
In reply to Vidya SagarThere are devices (Ex:- Marvell SATA controller) that don't support 64-bit MSIs and the same is advertised through their MSI capability register. Set no_64bit_msi flag explicitly for such devices in the MSI setup code so that the msi_verify_entries() API would catch if the MSI arch code tries to use 64-bit MSI.
---
V2:
* Addressed Bjorn's comment and changed the error message
drivers/pci/msi.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
drivers/pci/msi.c +7 -4
@@ -581,10 +581,12 @@ msi_setup_entry(struct pci_dev *dev, int nvec, struct irq_affinity *affd) entry->msi_attrib.multi_cap = (control &PCI_MSI_FLAGS_QMASK ) >>1 ; entry->msi_attrib.multiple =ilog2 (__roundup_pow_of_two (nvec)); -if (control &PCI_MSI_FLAGS_64BIT ) +if (control &PCI_MSI_FLAGS_64BIT ) { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_64 ; -else + }else { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_32 ; + dev->no_64bit_msi =1 ; + }/* Save the initial mask status */ if (entry->msi_attrib.maskbit)
@@ -602,8 +604,9 @@ static int msi_verify_entries(struct pci_dev *dev)for_each_pci_msi_entry (entry, dev) {if (!dev->no_64bit_msi || !entry->msg.address_hi)continue ; -pci_err (dev,"Device has broken 64-bit MSI but arch" -" tried to assign one above 4G\n" ); +pci_err (dev,"Device has either broken 64-bit MSI or " +"only 32-bit MSI support but " +"arch tried to assign one above 4G\n" );return -EIO ; }return 0 ;
Signature · 1 line
2.17.1
Hi Bjorn,
Do you have any further comments for this patch?
Thanks,
Vidya Sagar
Quoted 46 lines · Vidya Sagar
There are devices (Ex:- Marvell SATA controller) that don't support 64-bit MSIs and the same is advertised through their MSI capability register. Set
no_64bit_msiflag explicitly for such devices in the MSI setup code so that themsi_verify_entries()API would catch if the MSI arch code tries to use 64-bit MSI.Signed-off-by Vidya Sagar <[email protected]>---
V2:
* Addressed Bjorn's comment and changed the error messagedrivers/pci/msi.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)drivers/pci/msi.c +7
-4@@ -581,10 +581,12 @@ msi_setup_entry(struct pci_dev *dev, int nvec, struct irq_affinity *affd) entry->msi_attrib.multi_cap = (control &PCI_MSI_FLAGS_QMASK ) >>1 ; entry->msi_attrib.multiple =ilog2 (__roundup_pow_of_two (nvec)); -if (control &PCI_MSI_FLAGS_64BIT ) +if (control &PCI_MSI_FLAGS_64BIT ) { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_64 ; -else + }else { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_32 ; + dev->no_64bit_msi =1 ; + }/* Save the initial mask status */ if (entry->msi_attrib.maskbit)@@ -602,8 +604,9 @@ static int msi_verify_entries(struct pci_dev *dev)for_each_pci_msi_entry (entry, dev) {if (!dev->no_64bit_msi || !entry->msg.address_hi)continue ; -pci_err (dev,"Device has broken 64-bit MSI but arch" -" tried to assign one above 4G\n" ); +pci_err (dev,"Device has either broken 64-bit MSI or " +"only 32-bit MSI support but " +"arch tried to assign one above 4G\n" );return -EIO ; }return 0 ;
Quoted 5 lines · Vidya Sagar
There are devices (Ex:- Marvell SATA controller) that don't support 64-bit MSIs and the same is advertised through their MSI capability register. Set
no_64bit_msiflag explicitly for such devices in the MSI setup code so that themsi_verify_entries()API would catch if the MSI arch code tries to use 64-bit MSI.
This seems good to me. I'll post a possible revision to set dev->no_64bit_msi in the device enumeration path instead of in the IRQ allocation path, since it's really a property of the device, not of the msi_desc.
I like the extra checking this gives us. Was this prompted by tripping over something, or is it something you noticed by code reading? If the former, a hint about what was wrong and how it's being fixed would be useful.
Quoted 42 lines
Signed-off-by Vidya Sagar <[email protected]>---
V2:
* Addressed Bjorn's comment and changed the error messagedrivers/pci/msi.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)drivers/pci/msi.c +7
-4@@ -581,10 +581,12 @@ msi_setup_entry(struct pci_dev *dev, int nvec, struct irq_affinity *affd) entry->msi_attrib.multi_cap = (control &PCI_MSI_FLAGS_QMASK ) >>1 ; entry->msi_attrib.multiple =ilog2 (__roundup_pow_of_two (nvec)); -if (control &PCI_MSI_FLAGS_64BIT ) +if (control &PCI_MSI_FLAGS_64BIT ) { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_64 ; -else + }else { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_32 ; + dev->no_64bit_msi =1 ; + }/* Save the initial mask status */ if (entry->msi_attrib.maskbit)@@ -602,8 +604,9 @@ static int msi_verify_entries(struct pci_dev *dev)for_each_pci_msi_entry (entry, dev) {if (!dev->no_64bit_msi || !entry->msg.address_hi)continue ; -pci_err (dev,"Device has broken 64-bit MSI but arch" -" tried to assign one above 4G\n" ); +pci_err (dev,"Device has either broken 64-bit MSI or " +"only 32-bit MSI support but " +"arch tried to assign one above 4G\n" );return -EIO ; }return 0 ;Signature · 1 line
2.17.1
Quoted 15 lines · Bjorn Helgaas
External email: Use caution opening links or attachments
Quoted 5 lines · Vidya Sagar
There are devices (Ex:- Marvell SATA controller) that don't support 64-bit MSIs and the same is advertised through their MSI capability register. Set
no_64bit_msiflag explicitly for such devices in the MSI setup code so that themsi_verify_entries()API would catch if the MSI arch code tries to use 64-bit MSI.This seems good to me. I'll post a possible revision to set
dev->no_64bit_msiin the device enumeration path instead of in the IRQ allocation path, since it's really a property of the device, not of themsi_desc.
Quoted 4 lines
I like the extra checking this gives us. Was this prompted by tripping over something, or is it something you noticed by code reading? If the former, a hint about what was wrong and how it's being fixed would be useful.
I observed functionality issue with Marvell SATA controller (1b4b:9171) when the allocated MSI target address was a 64-bit address. I mentioned the Marvell SATA controller as an example in the commit message.
Thanks,
Vidya Sagar
Quoted 43 lines
Quoted 43 lines
Signed-off-by Vidya Sagar <[email protected]>---
V2:
* Addressed Bjorn's comment and changed the error messagedrivers/pci/msi.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)drivers/pci/msi.c +7
-5@@ -581,10 +581,12 @@ msi_setup_entry(struct pci_dev *dev, int nvec, struct irq_affinity *affd) entry->msi_attrib.multi_cap = (control &PCI_MSI_FLAGS_QMASK ) >>1 ; entry->msi_attrib.multiple =ilog2 (__roundup_pow_of_two (nvec)); -if (control &PCI_MSI_FLAGS_64BIT ) +if (control &PCI_MSI_FLAGS_64BIT ) { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_64 ; -else + }else { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_32 ; + dev->no_64bit_msi =1 ; + }/* Save the initial mask status */ if (entry->msi_attrib.maskbit)@@ -602,8 +604,9 @@ static int msi_verify_entries(struct pci_dev *dev)for_each_pci_msi_entry (entry, dev) {if (!dev->no_64bit_msi || !entry->msg.address_hi)continue ; -pci_err (dev,"Device has broken 64-bit MSI but arch" -" tried to assign one above 4G\n" ); +pci_err (dev,"Device has either broken 64-bit MSI or " +"only 32-bit MSI support but " +"arch tried to assign one above 4G\n" );return -EIO ; }return 0 ; --2.17.1
Quoted 13 lines · Vidya Sagar
Quoted 12 lines · Bjorn Helgaas
Quoted 5 lines · Vidya Sagar
There are devices (Ex:- Marvell SATA controller) that don't support 64-bit MSIs and the same is advertised through their MSI capability register. Set
no_64bit_msiflag explicitly for such devices in the MSI setup code so that themsi_verify_entries()API would catch if the MSI arch code tries to use 64-bit MSI.This seems good to me. I'll post a possible revision to set
dev->no_64bit_msiin the device enumeration path instead of in the IRQ allocation path, since it's really a property of the device, not of themsi_desc.
Quoted 7 lines
Quoted 4 lines
I like the extra checking this gives us. Was this prompted by tripping over something, or is it something you noticed by code reading? If the former, a hint about what was wrong and how it's being fixed would be useful.
I observed functionality issue with Marvell SATA controller (1b4b:9171) when the allocated MSI target address was a 64-bit address. I mentioned the Marvell SATA controller as an example in the commit message.
I know you mentioned the Marvell controller, but apparently that device is working perfectly correctly: it does not support 64-bit MSI, and it does not advertise support for 64-bit MSI.
So if there's a functionality issue, that means something is wrong in Linux that caused us to assign a 64-bit MSI address to it. *That* issue is what I want to know about. Your patch only warns about the issue; it doesn't fix it.
I don't think there's any point in specifically mentioning the Marvell device if it is working correctly, because the same issue should affect *any* device that doesn't support 64-bit MSI. But if there's some arch code that incorrectly assigns a 64-bit address, it would definitely be useful to specify the arch. And hopefully there's a fix for that arch code, too.
Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
Signed-off-by Vidya Sagar <[email protected]>---
V2:
* Addressed Bjorn's comment and changed the error messagedrivers/pci/msi.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)drivers/pci/msi.c +7
-5@@ -581,10 +581,12 @@ msi_setup_entry(struct pci_dev *dev, int nvec, struct irq_affinity *affd) entry->msi_attrib.multi_cap = (control &PCI_MSI_FLAGS_QMASK ) >>1 ; entry->msi_attrib.multiple =ilog2 (__roundup_pow_of_two (nvec)); -if (control &PCI_MSI_FLAGS_64BIT ) +if (control &PCI_MSI_FLAGS_64BIT ) { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_64 ; -else + }else { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_32 ; + dev->no_64bit_msi =1 ; + }/* Save the initial mask status */ if (entry->msi_attrib.maskbit)@@ -602,8 +604,9 @@ static int msi_verify_entries(struct pci_dev *dev)for_each_pci_msi_entry (entry, dev) {if (!dev->no_64bit_msi || !entry->msg.address_hi)continue ; -pci_err (dev,"Device has broken 64-bit MSI but arch" -" tried to assign one above 4G\n" ); +pci_err (dev,"Device has either broken 64-bit MSI or " +"only 32-bit MSI support but " +"arch tried to assign one above 4G\n" );return -EIO ; }return 0 ; --2.17.1
Quoted 25 lines · Bjorn Helgaas
External email: Use caution opening links or attachments
Quoted 13 lines · Vidya Sagar
Quoted 12 lines · Bjorn Helgaas
Quoted 5 lines · Vidya Sagar
There are devices (Ex:- Marvell SATA controller) that don't support 64-bit MSIs and the same is advertised through their MSI capability register. Set
no_64bit_msiflag explicitly for such devices in the MSI setup code so that themsi_verify_entries()API would catch if the MSI arch code tries to use 64-bit MSI.This seems good to me. I'll post a possible revision to set
dev->no_64bit_msiin the device enumeration path instead of in the IRQ allocation path, since it's really a property of the device, not of themsi_desc.Quoted 7 lines
Quoted 4 lines
I like the extra checking this gives us. Was this prompted by tripping over something, or is it something you noticed by code reading? If the former, a hint about what was wrong and how it's being fixed would be useful.
I observed functionality issue with Marvell SATA controller (1b4b:9171) when the allocated MSI target address was a 64-bit address. I mentioned the Marvell SATA controller as an example in the commit message.
Quoted 8 lines
I know you mentioned the Marvell controller, but apparently that device is working perfectly correctly: it does not support 64-bit MSI, and it does not advertise support for 64-bit MSI.
So if there's a functionality issue, that means something is wrong in Linux that caused us to assign a 64-bit MSI address to it. *That* issue is what I want to know about. Your patch only warns about the issue; it doesn't fix it.
The issue is in the DWC code. I pushed a generic patch to fix that issue by specifying the limit of MSI target address to 32-bit region. Patch is up for review at http://patchwork.ozlabs.org/project/linux-pci/patch/[email protected]/
Quoted 50 lines
I don't think there's any point in specifically mentioning the Marvell device if it is working correctly, because the same issue should affect *any* device that doesn't support 64-bit MSI. But if there's some arch code that incorrectly assigns a 64-bit address, it would definitely be useful to specify the arch. And hopefully there's a fix for that arch code, too.
Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
Signed-off-by Vidya Sagar <[email protected]>---
V2:
* Addressed Bjorn's comment and changed the error messagedrivers/pci/msi.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)drivers/pci/msi.c +7
-5@@ -581,10 +581,12 @@ msi_setup_entry(struct pci_dev *dev, int nvec, struct irq_affinity *affd) entry->msi_attrib.multi_cap = (control &PCI_MSI_FLAGS_QMASK ) >>1 ; entry->msi_attrib.multiple =ilog2 (__roundup_pow_of_two (nvec)); -if (control &PCI_MSI_FLAGS_64BIT ) +if (control &PCI_MSI_FLAGS_64BIT ) { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_64 ; -else + }else { entry->mask_pos = dev->msi_cap +PCI_MSI_MASK_32 ; + dev->no_64bit_msi =1 ; + }/* Save the initial mask status */ if (entry->msi_attrib.maskbit)@@ -602,8 +604,9 @@ static int msi_verify_entries(struct pci_dev *dev)for_each_pci_msi_entry (entry, dev) {if (!dev->no_64bit_msi || !entry->msg.address_hi)continue ; -pci_err (dev,"Device has broken 64-bit MSI but arch" -" tried to assign one above 4G\n" ); +pci_err (dev,"Device has either broken 64-bit MSI or " +"only 32-bit MSI support but " +"arch tried to assign one above 4G\n" );return -EIO ; }return 0 ; --2.17.1