RDMA/efa: Limit EQs to available MSI-X vectors
Subsystems (2)
PATCH v2 1/1 RDMA/efa: Limit EQs to available MSI-X vectors
From: Yonatan Nachum <[email protected]>
When creating EQs we take into consideration the max number of EQs the device reported it can support and the number of available CPUs. There are situations where the number of EQs the device reported it can support and the PCI configuration of MSI-X is different, take it in account as well when creating EQs.
Also request at least 1 MSI-X vector for the management queue and allow the kernel to return a number of vectors in a range between 1 and the max supported MSI-X vectors according to the PCI config.
--- drivers/infiniband/hw/efa/efa.h | 1 + drivers/infiniband/hw/efa/efa_main.c | 32 +++++++++++++--------------- 2 files changed, 16 insertions(+), 17 deletions(-)
drivers/infiniband/hw/efa/efa.h +1 -0
@@ -57,6 +57,7 @@ struct efa_dev {u64 db_bar_addr;u64 db_bar_len; +unsigned int num_irq_vectors;int admin_msix_vector_idx;struct efa_irq admin_irq;
drivers/infiniband/hw/efa/efa_main.c +15 -17
@@ -322,7 +322,9 @@ static int efa_create_eqs(struct efa_dev *dev)int err;int i; - neqs = min_t(unsigned int, neqs, num_online_cpus()); + neqs = min_t(unsigned int, neqs, + dev->num_irq_vectors -EFA_COMP_EQS_VEC_BASE ); + dev->neqs = neqs; dev->eqs =kcalloc (neqs,sizeof (*dev->eqs),GFP_KERNEL );if (!dev->eqs)
@@ -468,34 +470,30 @@ static void efa_disable_msix(struct efa_dev *dev)static int efa_enable_msix (struct efa_dev *dev) { -int msix_vecs, irq_num; +int max_vecs, num_vecs;/* * Reserve the max msix vectors we might need, one vector is reserved * for admin. */ - msix_vecs =min_t (int,pci_msix_vec_count (dev->pdev), -num_online_cpus () +1 ); + max_vecs =min_t (int,pci_msix_vec_count (dev->pdev), +num_online_cpus () +1 );dev_dbg (&dev->pdev->dev,"Trying to enable MSI-X, vectors %d\n" , - msix_vecs); + max_vecs); dev->admin_msix_vector_idx =EFA_MGMNT_MSIX_VEC_IDX ; - irq_num =pci_alloc_irq_vectors (dev->pdev, msix_vecs, - msix_vecs,PCI_IRQ_MSIX ); + num_vecs =pci_alloc_irq_vectors (dev->pdev,1 , + max_vecs,PCI_IRQ_MSIX ); -if (irq_num <0 ) { -dev_err (&dev->pdev->dev,"Failed to enable MSI-X. irq_num %d\n" , - irq_num); +if (num_vecs <0 ) { +dev_err (&dev->pdev->dev,"Failed to enable MSI-X. error %d\n" , + num_vecs);return -ENOSPC ; } -if (irq_num != msix_vecs) { -efa_disable_msix (dev); -dev_err (&dev->pdev->dev, -"Allocated %d MSI-X (out of %d requested)\n" , - irq_num, msix_vecs); -return -ENOSPC ; - } +dev_dbg (&dev->pdev->dev,"Allocated %d MSI-X vectors\n" , num_vecs); + + dev->num_irq_vectors = num_vecs;return 0 ; }
Signature · 1 line
2.40.1
Quoted 11 lines · On Wed, Jan 31, 2024 at 09:34:03AM +0000, [email protected] wrote:
From: Yonatan Nachum <[email protected]>
When creating EQs we take into consideration the max number of EQs the device reported it can support and the number of available CPUs. There are situations where the number of EQs the device reported it can support and the PCI configuration of MSI-X is different, take it in account as well when creating EQs.
Also request at least 1 MSI-X vector for the management queue and allow the kernel to return a number of vectors in a range between 1 and the max supported MSI-X vectors according to the PCI config.
Quoted 7 lines
Reviewed-by Michael Margolin <[email protected]>Reviewed-by Yonatan Goldhirsh <[email protected]>Signed-off-by Yonatan Nachum <[email protected]>--- drivers/infiniband/hw/efa/efa.h | 1 + drivers/infiniband/hw/efa/efa_main.c | 32 +++++++++++++--------------- 2 files changed, 16 insertions(+), 17 deletions(-)
I applied this patch to -next, because it lacks Fixes line and previous version was targeted to -next too.
Please add changelog next time.
Thanks
Leon Romanovsky
AppliedQuoted 10 lines · On Wed, 31 Jan 2024 09:34:03 +0000, [email protected] wrote:
When creating EQs we take into consideration the max number of EQs the device reported it can support and the number of available CPUs. There are situations where the number of EQs the device reported it can support and the PCI configuration of MSI-X is different, take it in account as well when creating EQs.
Also request at least 1 MSI-X vector for the management queue and allow the kernel to return a number of vectors in a range between 1 and the max supported MSI-X vectors according to the PCI config.[...]
Applied, thanks!
[1/1] RDMA/efa: Limit EQs to available MSI-X vectors
https://git.kernel.org/rdma/rdma/c/809c9c3bd6997e
Best regards,
Signature · 1 line
Leon Romanovsky <[email protected]>