PCI: vmd: Add AHCI to fast interrupt list
Subsystems (3)
PATCH 1/1 PCI: vmd: Add AHCI to fast interrupt list
Some platforms have an AHCI controller behind VMD. These platforms are working correctly except for a case when the AHCI MSI is programmed with VMD IRQ vector 0 (0xfee00000). When programmed with any other interrupt (0xfeeNN000), the MSI is routed correctly and is handled by VMD. Placing the AHCI MSI(s) in the fast-interrupt allow list solves the issue.
This also requires that VMD allocate more than one MSI/X vector and changes the minimum MSI/X vectors allocated to two.
--- drivers/pci/controller/vmd.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
drivers/pci/controller/vmd.c +3 -5
@@ -202,15 +202,13 @@ static struct vmd_irq_list *vmd_next_irq(struct vmd_dev *vmd, struct msi_desc *dint i, best =1 ;unsigned long flags; -if (vmd->msix_count ==1 ) -return &vmd->irqs[0 ]; -/* -* White list for fast-interrupt handlers. All others will share the +* Allow list for fast-interrupt handlers. All others will share the * "slow" interrupt vector. */ switch (msi_desc_to_pci_dev (desc)->class) {case PCI_CLASS_STORAGE_EXPRESS : +case PCI_CLASS_STORAGE_SATA_AHCI :break ;default :return &vmd->irqs[0 ];
@@ -657,7 +655,7 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)if (vmd->msix_count <0 )return -ENODEV ; - vmd->msix_count =pci_alloc_irq_vectors (dev,1 , vmd->msix_count, + vmd->msix_count =pci_alloc_irq_vectors (dev,2 , vmd->msix_count,PCI_IRQ_MSIX );if (vmd->msix_count <0 )return vmd->msix_count;
Signature · 1 line
2.17.1
Quoted 10 lines · Jon Derrick
Some platforms have an AHCI controller behind VMD. These platforms are working correctly except for a case when the AHCI MSI is programmed with VMD IRQ vector 0 (
0xfee00000). When programmed with any other interrupt (0xfeeNN000), the MSI is routed correctly and is handled by VMD. Placing the AHCI MSI(s) in the fast-interrupt allow list solves the issue.This also requires that VMD allocate more than one MSI/X vector and changes the minimum MSI/X vectors allocated to two.
Signed-off-by Jon Derrick <[email protected]>
Verified two platforms with such configuration. Thank you.
You-Sheng Yang
Quoted 36 lines
--- drivers/pci/controller/vmd.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)drivers/pci/controller/vmd.c +3
-5@@ -202,15 +202,13 @@ static struct vmd_irq_list *vmd_next_irq(struct vmd_dev *vmd, struct msi_desc *dint i, best =1 ;unsigned long flags; -if (vmd->msix_count ==1 ) -return &vmd->irqs[0 ]; -/* -* White list for fast-interrupt handlers. All others will share the +* Allow list for fast-interrupt handlers. All others will share the * "slow" interrupt vector. */ switch (msi_desc_to_pci_dev (desc)->class) {case PCI_CLASS_STORAGE_EXPRESS : +case PCI_CLASS_STORAGE_SATA_AHCI :break ;default :return &vmd->irqs[0 ];@@ -657,7 +655,7 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)if (vmd->msix_count <0 )return -ENODEV ; - vmd->msix_count =pci_alloc_irq_vectors (dev,1 , vmd->msix_count, + vmd->msix_count =pci_alloc_irq_vectors (dev,2 , vmd->msix_count,PCI_IRQ_MSIX );if (vmd->msix_count <0 )return vmd->msix_count;
Quoted 5 lines · Jon Derrick
Some platforms have an AHCI controller behind VMD. These platforms are working correctly except for a case when the AHCI MSI is programmed with VMD IRQ vector 0 (
0xfee00000). When programmed with any other interrupt (0xfeeNN000), the MSI is routed correctly and is handled by VMD. Placing the AHCI MSI(s) in the fast-interrupt allow list solves the issue.
The only reason we have the fast vs. slow is because of the non-posted transactions from slow driver's interrupt handlers tanking performance of the nvme devices sharing the same vector. The microsemi switchtec was one of the first such devices identified that led to the current split in the VMD domain. AHCI's driver also has non-posted transactions in their interrupt handling, so you probably don't want those devices sharing vectors with your fast nvme devices.
Quoted 7 lines · Keith Busch
Quoted 5 lines · Jon Derrick
Some platforms have an AHCI controller behind VMD. These platforms are working correctly except for a case when the AHCI MSI is programmed with VMD IRQ vector 0 (
0xfee00000). When programmed with any other interrupt (0xfeeNN000), the MSI is routed correctly and is handled by VMD. Placing the AHCI MSI(s) in the fast-interrupt allow list solves the issue.
Quoted 7 lines
The only reason we have the fast vs. slow is because of the non-posted transactions from slow driver's interrupt handlers tanking performance of the nvme devices sharing the same vector. The microsemi switchtec was one of the first such devices identified that led to the current split in the VMD domain. AHCI's driver also has non-posted transactions in their interrupt handling, so you probably don't want those devices sharing vectors with your fast nvme devices.
Good points and I think this would be simply resolved by offsetting the starting vector and avoiding vector 0 for all vectors.
Thanks
Quoted 7 lines · Jon Derrick
Quoted 6 lines
Some platforms have an AHCI controller behind VMD. These platforms are working correctly except for a case when the AHCI MSI is programmed with VMD IRQ vector 0 (
0xfee00000). When programmed with any other interrupt (0xfeeNN000), the MSI is routed correctly and is handled by VMD. Placing the AHCI MSI(s) in the fast-interrupt allow list solves the issue.
Quoted 8 lines
Quoted 4 lines
This also requires that VMD allocate more than one MSI/X vector and changes the minimum MSI/X vectors allocated to two.
Signed-off-by Jon Derrick <[email protected]>Verified two platforms with such configuration. Thank you.
You-Sheng Yang
We have some laptops equipped with Tiger Lake chips and such configuration that hit the same issue, too. They all could be fixed by this patch. Thank you
Quoted 12 lines · Jian-Hong Pan
Quoted 12 lines · Jon Derrick
Quoted 10 lines
Some platforms have an AHCI controller behind VMD. These platforms are working correctly except for a case when the AHCI MSI is programmed with VMD IRQ vector 0 (
0xfee00000). When programmed with any other interrupt (0xfeeNN000), the MSI is routed correctly and is handled by VMD. Placing the AHCI MSI(s) in the fast-interrupt allow list solves the issue.This also requires that VMD allocate more than one MSI/X vector and changes the minimum MSI/X vectors allocated to two.
Signed-off-by Jon Derrick <[email protected]>
Quoted 8 lines
Quoted 3 lines
Verified two platforms with such configuration. Thank you.
You-Sheng Yang
We have some laptops equipped with Tiger Lake chips and such configuration that hit the same issue, too. They all could be fixed by this patch. Thank you
Tested-by Jian-Hong Pan <[email protected]>
Please use the v3 instead:
https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/
I'm not sure if there's anything remaining that's gating inclusion, though.