RFC Re-design MSI/MSI-X interrupts enablement pattern
Status from this message: Nack, this check is unnecessary.
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest77 patches 72 repliesNAK
Patches in v1
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/77 | PCI/MSI: Fix return value when populate_msi_sysfs() faileddrivers/ | +5 | 3 replies | |
| 2/77 | PCI/MSI/PPC: Fix wrong RTAS error code reportingarch/ | +1 | 0 replies | |
| 3/77 | PCI/MSI/s390: Fix single MSI only checkarch/ | +2 | 1 reply | |
| 4/77 | PCI/MSI/s390: Remove superfluous check of MSI typearch/ | +0 | 2 replies | |
| 5/77 | PCI/MSI: Convert pci_msix_table_size() to a public interfaceDocumentation/ drivers/ drivers/ | +19 | 2 replies | |
| 6/77 | PCI/MSI: Factor out pci_get_msi_cap() interfaceDocumentation/ drivers/ include/ | +42 | 2 replies | |
| 7/77 | PCI/MSI: Re-design MSI/MSI-X interrupts enablement patternDocumentation/ arch/ arch/ +4 more | +58 | 3 replies | |
| 8/77 | PCI/MSI: Get rid of pci_enable_msi_block_auto() interfaceDocumentation/ drivers/ include/ | +2 | 0 replies | |
| 9/77 | ahci: Update MSI/MSI-X interrupts enablement codedrivers/ | +35 | 0 replies | |
| 10/77 | ahci: Check MRSM bit when multiple MSIs enableddrivers/ drivers/ | +18 | 0 replies | |
| 11/77 | benet: Return -ENOSPC when not enough MSI-Xs availabledrivers/ | +2 | 0 replies | |
| 12/77 | benet: Update MSI/MSI-X interrupts enablement codedrivers/ | +20 | 0 replies | |
| 13/77 | bna: Update MSI/MSI-X interrupts enablement codedrivers/ | +14 | 0 replies | |
| 14/77 | bnx2x: Update MSI/MSI-X interrupts enablement codedrivers/ | +23 | 0 replies | |
| 15/77 | bnx2: Update MSI/MSI-X interrupts enablement codedrivers/ | +14 | 0 replies | |
| 16/77 | cciss: Update MSI/MSI-X interrupts enablement codedrivers/ | +7 | 0 replies | |
| 17/77 | cciss: Update a misleading comment on interrupt usagedrivers/ | +4 | 0 replies | |
| 18/77 | cciss: Fallback to single MSI mode in case MSI-X faileddrivers/ | +2 | 0 replies | |
| 19/77 | csiostor: Do not call pci_disable_msix() if pci_enable_msix() faileddrivers/ | +1 | 0 replies | |
| 20/77 | csiostor: Return -ENOSPC when not enough MSI-X vectors availabledrivers/ | +1 | 0 replies | |
| 21/77 | csiostor: Update MSI/MSI-X interrupts enablement codedrivers/ | +12 | 0 replies | |
| 22/77 | cxgb3: Do not call pci_disable_msix() if pci_enable_msix() faileddrivers/ | +0 | 0 replies | |
| 23/77 | cxgb3: Return -ENOSPC when not enough MSI-X vectors availabledrivers/ | +1 | 0 replies | |
| 24/77 | cxgb3: Update MSI/MSI-X interrupts enablement codedrivers/ | +15 | 0 replies | |
| 25/77 | cxgb4: Return -ENOSPC when not enough MSI-X vectors availabledrivers/ | +3 | 0 replies | |
| 26/77 | cxgb4: Update MSI/MSI-X interrupts enablement codedrivers/ | +35 | 0 replies | |
| 27/77 | cxgb4vf: Do not call pci_disable_msix() if pci_enable_msix() faileddrivers/ | +0 | 0 replies | |
| 28/77 | cxgb4vf: Return -ENOSPC when not enough MSI-X vectors availabledrivers/ | +1 | 0 replies | |
| 29/77 | cxgb4vf: Update MSI/MSI-X interrupts enablement codedrivers/ | +29 | 0 replies | |
| 30/77 | hpsa: Update a misleading comment on interrupt usagedrivers/ | +5 | 0 replies | |
| 31/77 | hpsa: Update MSI/MSI-X interrupts enablement codedrivers/ | +13 | 0 replies | |
| 32/77 | hpsa: Fallback to single MSI mode in case MSI-X faileddrivers/ | +3 | 0 replies | |
| 33/77 | ioat: Disable MSI-X in case request of IRQ faileddrivers/ | +1 | 0 replies | |
| 34/77 | ioat: Update MSI/MSI-X interrupts enablement codedrivers/ | +7 | 4 replies | |
| 35/77 | ipr: Do not call pci_disable_msi/msix() if pci_enable_msi/msix() faileddrivers/ | +2 | 0 replies | |
| 36/77 | ipr: Enable MSI-X when IPR_USE_MSIX type is set, not IPR_USE_MSIdrivers/ | +1 | 1 reply | |
| 37/77 | ipr: Update MSI/MSI-X interrupts enablement codedrivers/ | +23 | 0 replies | |
| 38/77 | ixgbe: Update MSI/MSI-X interrupts enablement codedrivers/ | +32 | 0 replies | |
| 39/77 | ixgbevf: Return -ENOSPC when not enough MSI-X vectors availabledrivers/ | +1 | 0 replies | |
| 40/77 | ixgbevf: Update MSI/MSI-X interrupts enablement codedrivers/ | +7 | 1 reply | |
| 41/77 | lpfc: Do not call pci_disable_msix() if pci_enable_msix() faileddrivers/ | +6 | 0 replies | |
| 42/77 | lpfc: Update MSI/MSI-X interrupts enablement codedrivers/ | +11 | 0 replies | |
| 43/77 | lpfc: Return -ENOSPC when not enough MSI-X vectors availabledrivers/ | +7 | 0 replies | |
| 44/77 | lpfc: Make MSI-X initialization routine more readabledrivers/ | +11 | 0 replies | |
| 45/77 | megaraid: Update MSI/MSI-X interrupts enablement codedrivers/ | +9 | 0 replies | |
| 46/77 | mlx4: Update MSI/MSI-X interrupts enablement codedrivers/ | +8 | 3 replies | |
| 47/77 | mlx5: Fix memory leak in case not enough MSI-X vectors availabledrivers/ | +1 | 0 replies | |
| 48/77 | mlx5: Return -ENOSPC when not enough MSI-X vectors availabledrivers/ | +2 | 0 replies | |
| 49/77 | mlx5: Fix minimum number of MSI-Xsdrivers/ | +1 | 0 replies | |
| 50/77 | mlx5: Update MSI/MSI-X interrupts enablement codedrivers/ | +9 | 3 replies | |
| 51/77 | mthca: Update MSI/MSI-X interrupts enablement codedrivers/ | +11 | 1 reply | |
| 52/77 | niu: Update MSI/MSI-X interrupts enablement codedrivers/ | +11 | 0 replies | |
| 53/77 | ntb: Fix missed call to pci_enable_msix()drivers/ | +4 | 1 reply | |
| 54/77 | ntb: Ensure number of MSIs on SNB is enough for the link interruptdrivers/ | +1 | 5 replies | |
| 55/77 | ntb: Update MSI/MSI-X interrupts enablement codedrivers/ drivers/ | +13 | 1 reply | |
| 56/77 | nvme: Update MSI/MSI-X interrupts enablement codedrivers/ | +25 | 0 replies | |
| 57/77 | pmcraid: Update MSI/MSI-X interrupts enablement codedrivers/ | +11 | 0 replies | |
| 58/77 | qib: Update MSI/MSI-X interrupts enablement codedrivers/ | +0 | 0 replies | |
| 59/77 | qla2xxx: Update MSI/MSI-X interrupts enablement codedrivers/ | +11 | 1 reply | |
| 60/77 | qlcnic: Return -ENOSPC when not enough MSI-X vectors availabledrivers/ | +2 | 0 replies | |
| 61/77 | qlogic: Return -EINVAL in case MSI-X is not supporteddrivers/ | +1 | 0 replies | |
| 62/77 | qlcnic: Remove redundant return operatordrivers/ | +0 | 0 replies | |
| 63/77 | qlcnic: Update MSI/MSI-X interrupts enablement codedrivers/ | +32 | 1 reply | |
| 64/77 | qlcnic: Make MSI-X initialization routine bit more readabledrivers/ | +56 | 0 replies | |
| 65/77 | qlge: Remove a redundant assignmentdrivers/ | +0 | 0 replies | |
| 66/77 | qlge: Update MSI/MSI-X interrupts enablement codedrivers/ | +17 | 0 replies | |
| 67/77 | rapidio: Update MSI/MSI-X interrupts enablement codedrivers/ | +17 | 0 replies | |
| 68/77 | sfc: Update MSI/MSI-X interrupts enablement codedrivers/ | +11 | 0 replies | |
| 69/77 | tg3: Update MSI/MSI-X interrupts enablement codedrivers/ | +13 | 0 replies | |
| 70/77 | vmci: Update MSI/MSI-X interrupts enablement codedrivers/ | +15 | 0 replies | |
| 71/77 | vmxnet3: Return -EINVAL if number of requested MSI-Xs is not enoughdrivers/ | +1 | 0 replies | |
| 72/77 | vmxnet3: Fixup a weird loop exitdrivers/ | +1 | 0 replies | |
| 73/77 | vmxnet3: Return -ENOSPC when not enough MSI-X vectors availabledrivers/ | +4 | 0 replies | |
| 74/77 | vmxnet3: Limit number of rx queues to 1 if per-queue MSI-Xs faileddrivers/ | +8 | 0 replies | |
| 75/77 | vmxnet3: Update MSI/MSI-X interrupts enablement codedrivers/ | +33 | 0 replies | |
| 76/77 | vxge: Sanitize MSI-X allocation routine error codesdrivers/ | +2 | 0 replies | |
| 77/77 | vxge: Update MSI/MSI-X interrupts enablement codedrivers/ | +16 | 1 reply |
Reviews trailers per patch
3 of 77 patches have a Reviewed-by or Acked-by.
| Patch | Saurav Kashyap | Martin Schwidefsky |
|---|---|---|
| 1/77 | ||
| 2/77 | ||
| 3/77 | A-b | |
| 4/77 | A-b | |
| 5/77 | ||
| 6/77 | ||
| 7/77 | ||
| 8/77 | ||
| 9/77 | ||
| 10/77 | ||
| 11/77 | ||
| 12/77 | ||
| 13/77 | ||
| 14/77 | ||
| 15/77 | ||
| 16/77 | ||
| 17/77 | ||
| 18/77 | ||
| 19/77 | ||
| 20/77 | ||
| 21/77 | ||
| 22/77 | ||
| 23/77 | ||
| 24/77 | ||
| 25/77 | ||
| 26/77 | ||
| 27/77 | ||
| 28/77 | ||
| 29/77 | ||
| 30/77 | ||
| 31/77 | ||
| 32/77 | ||
| 33/77 | ||
| 34/77 | ||
| 35/77 | ||
| 36/77 | ||
| 37/77 | ||
| 38/77 | ||
| 39/77 | ||
| 40/77 | ||
| 41/77 | ||
| 42/77 | ||
| 43/77 | ||
| 44/77 | ||
| 45/77 | ||
| 46/77 | ||
| 47/77 | ||
| 48/77 | ||
| 49/77 | ||
| 50/77 | ||
| 51/77 | ||
| 52/77 | ||
| 53/77 | ||
| 54/77 | ||
| 55/77 | ||
| 56/77 | ||
| 57/77 | ||
| 58/77 | ||
| 59/77 | A-b | |
| 60/77 | ||
| 61/77 | ||
| 62/77 | ||
| 63/77 | ||
| 64/77 | ||
| 65/77 | ||
| 66/77 | ||
| 67/77 | ||
| 68/77 | ||
| 69/77 | ||
| 70/77 | ||
| 71/77 | ||
| 72/77 | ||
| 73/77 | ||
| 74/77 | ||
| 75/77 | ||
| 76/77 | ||
| 77/77 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH RFC 00/77] Re-design MSI/MSI-X interrupts enablement pattern
This series is against "next" branch in Bjorn's repo:git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git
Currently pci_enable_msi_block() and pci_enable_msix() interfaces return a error code in case of failure, 0 in case of success and a positive value which indicates the number of MSI-X/MSI interrupts that could have been allocated. The latter value should be passed to a repeated call to the interfaces until a failure or success:
for (i = 0; i < FOO_DRIVER_MAXIMUM_NVEC; i++)
adapter->msix_entries[i].entry = i;
while (nvec >= FOO_DRIVER_MINIMUM_NVEC) {
rc = pci_enable_msix(adapter->pdev,
adapter->msix_entries, nvec);
if (rc > 0)
nvec = rc;
else
return rc;
}
return -ENOSPC;This technique proved to be confusing and error-prone. Vast share of device drivers simply fail to follow the described guidelines.
This update converts pci_enable_msix() and pci_enable_msi_block() interfaces to canonical kernel functions and makes them return a error code in case of failure or 0 in case of success.
As result, device drivers will cease to use the overcomplicated repeated fallbacks technique and resort to a straightforward pattern - determine the number of MSI/MSI-X interrupts required before calling pci_enable_msi_block() and pci_enable_msix() interfaces:
rc = pci_msix_table_size(adapter->pdev);
if (rc < 0)
return rc;
nvec = min(nvec, rc);
if (nvec < FOO_DRIVER_MINIMUM_NVEC) {
return -ENOSPC;
for (i = 0; i < nvec; i++)
adapter->msix_entries[i].entry = i;
rc = pci_enable_msix(adapter->pdev,
adapter->msix_entries, nvec);
return rc;Device drivers will use their knowledge of underlying hardware to determine the number of MSI/MSI-X interrupts required.
The simplest case would be requesting all available interrupts - to obtain that value device drivers will use pci_get_msi_cap() interface for MSI and pci_msix_table_size() for MSI-X.
More complex cases would entail matching device capabilities to the system environment, i.e. limiting number of hardware queues (and hence associated MSI/MSI-X interrupts) to the number of online CPUs.
Device drivers using MSI/MSI-X could be divided in three groups:
- drivers that request a hardcoded number of interrupts;
- drivers that request a number of interrupts using one call to pci_enable_msix() and then enable MSI/MSI-X using a follow-up to pci_enable_msix();
- drivers that fully follow the guidelines and repeatedly call pci_enable_msix() in a loop;
This series converts to the new technique second and third groups.
To simplify device drivers code review I tried to make as little changes as possible - the scope of this series is an introduction of the new technique rather than clean-up effort for all drivers affected.
The testing was very limited - I ensured successful booting on all affected architectures except MIPS and operation of few devices with and without pci=nomsi kernel parameter.
There is a ongoing discussion about impact of this update on PowerPC pSeries platform. I am going to incorporate the outcome of this discussion into the next version. Yet, the rest of the platforms and the vast majority of device drivers already can start getting initial reviews.
Patches 5,6,8 - update of the generic MSI code
Patch 7 - update of architectures affected
Patches 9-77 - bugfixes and updates of device drivers affected
The tree could be found in "pci-next-msi-v1" branch in repo:
https://github.com/a-gordeev/linux.git
Alexander Gordeev (77):
PCI/MSI: Fix return value when populate_msi_sysfs() failed PCI/MSI/PPC: Fix wrong RTAS error code reporting PCI/MSI/s390: Fix single MSI only check
PCI/MSI/s390: Remove superfluous check of MSI type PCI/MSI: Convert pci_msix_table_size() to a public interface PCI/MSI: Factor out pci_get_msi_cap() interface
PCI/MSI: Re-design MSI/MSI-X interrupts enablement pattern PCI/MSI: Get rid of pci_enable_msi_block_auto() interface
ahci: Update MSI/MSI-X interrupts enablement code
ahci: Check MRSM bit when multiple MSIs enabled
benet: Return -ENOSPC when not enough MSI-Xs available
benet: Update MSI/MSI-X interrupts enablement code
bna: Update MSI/MSI-X interrupts enablement code
bnx2x: Update MSI/MSI-X interrupts enablement code
bnx2: Update MSI/MSI-X interrupts enablement code
cciss: Update MSI/MSI-X interrupts enablement code
cciss: Update a misleading comment on interrupt usage
cciss: Fallback to single MSI mode in case MSI-X failed
csiostor: Do not call pci_disable_msix() if pci_enable_msix() failed
csiostor: Return -ENOSPC when not enough MSI-X vectors available
csiostor: Update MSI/MSI-X interrupts enablement code
cxgb3: Do not call pci_disable_msix() if pci_enable_msix() failed
cxgb3: Return -ENOSPC when not enough MSI-X vectors available
cxgb3: Update MSI/MSI-X interrupts enablement code
cxgb4: Return -ENOSPC when not enough MSI-X vectors available
cxgb4: Update MSI/MSI-X interrupts enablement code
cxgb4vf: Do not call pci_disable_msix() if pci_enable_msix() failed
cxgb4vf: Return -ENOSPC when not enough MSI-X vectors available
cxgb4vf: Update MSI/MSI-X interrupts enablement code
hpsa: Update a misleading comment on interrupt usage
hpsa: Update MSI/MSI-X interrupts enablement code
hpsa: Fallback to single MSI mode in case MSI-X failed
ioat: Disable MSI-X in case request of IRQ failed
ioat: Update MSI/MSI-X interrupts enablement code
ipr: Do not call pci_disable_msi/msix() if pci_enable_msi/msix()
failed
ipr: Enable MSI-X when IPR_USE_MSIX type is set, not IPR_USE_MSI
ipr: Update MSI/MSI-X interrupts enablement code
ixgbe: Update MSI/MSI-X interrupts enablement code
ixgbevf: Return -ENOSPC when not enough MSI-X vectors available
ixgbevf: Update MSI/MSI-X interrupts enablement code
lpfc: Do not call pci_disable_msix() if pci_enable_msix() failed
lpfc: Update MSI/MSI-X interrupts enablement code
lpfc: Return -ENOSPC when not enough MSI-X vectors available
lpfc: Make MSI-X initialization routine more readable
megaraid: Update MSI/MSI-X interrupts enablement code
mlx4: Update MSI/MSI-X interrupts enablement code
mlx5: Fix memory leak in case not enough MSI-X vectors available
mlx5: Return -ENOSPC when not enough MSI-X vectors available
mlx5: Fix minimum number of MSI-Xs
mlx5: Update MSI/MSI-X interrupts enablement code
mthca: Update MSI/MSI-X interrupts enablement code
niu: Update MSI/MSI-X interrupts enablement code
ntb: Fix missed call to pci_enable_msix()
ntb: Ensure number of MSIs on SNB is enough for the link interrupt
ntb: Update MSI/MSI-X interrupts enablement code
nvme: Update MSI/MSI-X interrupts enablement code
pmcraid: Update MSI/MSI-X interrupts enablement code
qib: Update MSI/MSI-X interrupts enablement code
qla2xxx: Update MSI/MSI-X interrupts enablement code
qlcnic: Return -ENOSPC when not enough MSI-X vectors available
qlogic: Return -EINVAL in case MSI-X is not supported
qlcnic: Remove redundant return operator
qlcnic: Update MSI/MSI-X interrupts enablement code
qlcnic: Make MSI-X initialization routine bit more readable
qlge: Remove a redundant assignment
qlge: Update MSI/MSI-X interrupts enablement code
rapidio: Update MSI/MSI-X interrupts enablement code
sfc: Update MSI/MSI-X interrupts enablement code
tg3: Update MSI/MSI-X interrupts enablement code
vmci: Update MSI/MSI-X interrupts enablement code
vmxnet3: Return -EINVAL if number of requested MSI-Xs is not enough
vmxnet3: Fixup a weird loop exit
vmxnet3: Return -ENOSPC when not enough MSI-X vectors available
vmxnet3: Limit number of rx queues to 1 if per-queue MSI-Xs failed
vmxnet3: Update MSI/MSI-X interrupts enablement code
vxge: Sanitize MSI-X allocation routine error codes
vxge: Update MSI/MSI-X interrupts enablement code
Documentation/PCI/MSI-HOWTO.txt | 123 +++++++++++--------- arch/mips/pci/msi-octeon.c | 2 +- arch/powerpc/kernel/msi.c | 2 +- arch/powerpc/platforms/pseries/msi.c | 4 +- arch/s390/pci/pci.c | 2 +- arch/x86/kernel/apic/io_apic.c | 2 +- drivers/ata/ahci.c | 71 ++++++++--- drivers/ata/ahci.h | 1 + drivers/block/cciss.c | 22 ++-- drivers/block/nvme-core.c | 48 ++++---- drivers/dma/ioat/dma.c | 11 ++- drivers/infiniband/hw/mthca/mthca_main.c | 16 ++- drivers/infiniband/hw/qib/qib_pcie.c | 4 - drivers/misc/vmw_vmci/vmci_guest.c | 22 +++- drivers/net/ethernet/broadcom/bnx2.c | 27 +++-- drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c | 54 ++++----- drivers/net/ethernet/broadcom/tg3.c | 24 ++-- drivers/net/ethernet/brocade/bna/bnad.c | 34 +++--- drivers/net/ethernet/chelsio/cxgb3/cxgb3_main.c | 32 +++--- drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 62 ++++++---- .../net/ethernet/chelsio/cxgb4vf/cxgb4vf_main.c | 49 +++++--- drivers/net/ethernet/emulex/benet/be_main.c | 36 +++--- drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c | 62 +++++----- drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 18 +-- drivers/net/ethernet/mellanox/mlx4/main.c | 17 ++-- drivers/net/ethernet/mellanox/mlx5/core/main.c | 17 ++-- drivers/net/ethernet/neterion/vxge/vxge-main.c | 38 +++---- drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c | 108 +++++++++-------- drivers/net/ethernet/qlogic/qlge/qlge_main.c | 40 +++---- drivers/net/ethernet/sfc/efx.c | 18 ++- drivers/net/ethernet/sun/niu.c | 20 ++-- drivers/net/vmxnet3/vmxnet3_drv.c | 84 +++++++------- drivers/ntb/ntb_hw.c | 37 ++---- drivers/ntb/ntb_hw.h | 2 - drivers/pci/msi.c | 93 +++++---------- drivers/pci/pcie/portdrv_core.c | 2 + drivers/rapidio/devices/tsi721.c | 27 +++-- drivers/scsi/csiostor/csio_isr.c | 20 ++-- drivers/scsi/hpsa.c | 35 +++--- drivers/scsi/ipr.c | 52 ++++----- drivers/scsi/lpfc/lpfc_init.c | 40 ++++--- drivers/scsi/megaraid/megaraid_sas_base.c | 20 ++-- drivers/scsi/pmcraid.c | 23 ++-- drivers/scsi/qla2xxx/qla_isr.c | 18 ++- include/linux/pci.h | 7 +- 45 files changed, 744 insertions(+), 702 deletions(-)
Signature · 6 lines
1.7.7.6 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html