RDMA/bnxt_re: Bug fixes
Subsystems (2)
Selvin Xavier
maintainercover RDMA/bnxt_re: Bug fixes
Few bugfixes for bnxt_re driver. Please review and apply.
Thanks,
Selvin Xavier
Abhishek Mohapatra (1):
RDMA/bnxt_re: Fix the max CQ WQEs for older adapters
Bhargava Chenna Marreddy (1):
RDMA/bnxt_re: Fix a bug while setting up Level-2 PBL pages
Chandramohan Akula (1):
RDMA/bnxt_re: Change the sequence of updating the CQ toggle value
Kalesh AP (5):
RDMA/bnxt_re: Fix out of bound check
RDMA/ bnxt_re: Return more meaningful error RDMA/bnxt_re: Fix a possible NULL pointer dereference RDMA/bnxt_re: Fix an error path in bnxt_re_add_device RDMA/bnxt_re: Fix the GID table length
Kashyap Desai (1):
RDMA/bnxt_re: Fix incorrect dereference of srq in async event
Selvin Xavier (1):
RDMA/bnxt_re: Avoid CPU lockups due fifo occupancy check loop
drivers/infiniband/hw/bnxt_re/hw_counters.c | 2 +- drivers/infiniband/hw/bnxt_re/main.c | 42 ++++++++++++++--------------- drivers/infiniband/hw/bnxt_re/qplib_fp.c | 5 ++++ drivers/infiniband/hw/bnxt_re/qplib_rcfw.c | 2 +- drivers/infiniband/hw/bnxt_re/qplib_res.c | 19 +++---------- drivers/infiniband/hw/bnxt_re/qplib_sp.c | 11 +++++++- drivers/infiniband/hw/bnxt_re/qplib_sp.h | 1 + 7 files changed, 42 insertions(+), 40 deletions(-)
Signature · 1 line
2.5.5
Selvin Xavier
maintainerPATCH 1/10 RDMA/bnxt_re: Fix the max CQ WQEs for older adapters
From: Abhishek Mohapatra <[email protected]>
Older adapters doesn't support the MAX CQ WQEs reported by older FW. So restrict the value reported to 1M always for older adapters.
--- drivers/infiniband/hw/bnxt_re/qplib_sp.c | 2 ++ drivers/infiniband/hw/bnxt_re/qplib_sp.h | 1 + 2 files changed, 3 insertions(+)
drivers/infiniband/hw/bnxt_re/qplib_sp.c +2 -0
@@ -140,6 +140,8 @@ int bnxt_qplib_get_dev_attr(struct bnxt_qplib_rcfw *rcfw, min_t(u32, sb->max_sge_var_wqe, BNXT_VAR_MAX_SGE) : 6; attr->max_cq =le32_to_cpu (sb->max_cq); attr->max_cq_wqes =le32_to_cpu (sb->max_cqe); +if (!bnxt_qplib_is_chip_gen_p7 (rcfw->res->cctx)) + attr->max_cq_wqes =min_t (u32,BNXT_QPLIB_MAX_CQ_WQES , attr->max_cq_wqes); attr->max_cq_sges = attr->max_qp_sges; attr->max_mr =le32_to_cpu (sb->max_mr); attr->max_mw =le32_to_cpu (sb->max_mw);
drivers/infiniband/hw/bnxt_re/qplib_sp.h +1 -0
@@ -56,6 +56,7 @@ struct bnxt_qplib_dev_attr {u32 max_qp_wqes;u32 max_qp_sges;u32 max_cq; +#define BNXT_QPLIB_MAX_CQ_WQES 0xfffffu32 max_cq_wqes;u32 max_cq_sges;u32 max_mr;
Signature · 1 line
2.5.5
Selvin Xavier
maintainerPATCH 2/10 RDMA/bnxt_re: Fix out of bound check
In reply to Selvin XavierFrom: Kalesh AP <[email protected]>
Driver exports pacing stats only on GenP5 and P7 adapters. But while parsing the pacing stats, driver has a check for "rdev->dbr_pacing". This caused a trace when KASAN is enabled.
BUG: KASAN: slab-out-of-bounds in bnxt_re_get_hw_stats+0x2b6a/0x2e00 [bnxt_re] Write of size 8 at addr ffff8885942a6340 by task modprobe/4809
--- drivers/infiniband/hw/bnxt_re/hw_counters.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/hw/bnxt_re/hw_counters.c +1 -1
@@ -366,7 +366,7 @@ int bnxt_re_ib_get_hw_stats(struct ib_device *ibdev, goto done; } } -if (rdev->pacing.dbr_pacing) +if (rdev->pacing.dbr_pacing &&bnxt_qplib_is_chip_gen_p5_p7 (rdev->chip_ctx))bnxt_re_copy_db_pacing_stats (rdev, stats); }
Signature · 1 line
2.5.5
Selvin Xavier
maintainerPATCH 3/10 RDMA/bnxt_re: Fix incorrect dereference of srq in async event
In reply to Selvin XavierFrom: Kashyap Desai <[email protected]>
Currently driver is not getting correct srq. Dereference only if qplib has a valid srq.
--- drivers/infiniband/hw/bnxt_re/main.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/infiniband/hw/bnxt_re/main.c +5 -2
@@ -1028,12 +1028,15 @@ static int bnxt_re_handle_unaffi_async_event(struct creq_func_eventstatic int bnxt_re_handle_qp_async_event (struct creq_qp_event *qp_event,struct bnxt_re_qp *qp) { - struct bnxt_re_srq *srq = container_of(qp->qplib_qp.srq, struct bnxt_re_srq, - qplib_srq);struct creq_qp_error_notification *err_event; +struct bnxt_re_srq *srq =NULL ;struct ib_event event = {};unsigned int flags; +if (qp->qplib_qp.srq) + srq = container_of(qp->qplib_qp.srq, struct bnxt_re_srq, + qplib_srq); +if (qp->qplib_qp.state ==CMDQ_MODIFY_QP_NEW_STATE_ERR &&rdma_is_kernel_res (&qp->ib_qp.res)) { flags =bnxt_re_lock_cqs (qp);
Signature · 1 line
2.5.5
Selvin Xavier
maintainerPATCH 4/10 RDMA/ bnxt_re: Return more meaningful error
In reply to Selvin XavierFrom: Kalesh AP <[email protected]>
When the HWRM command fails, driver currently returns -EFAULT(Bad address). This does not look correct.
Modified to return -EIO(I/O error).
--- drivers/infiniband/hw/bnxt_re/qplib_rcfw.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/infiniband/hw/bnxt_re/qplib_rcfw.c +1 -1
@@ -525,7 +525,7 @@ static int __bnxt_qplib_rcfw_send_message(struct bnxt_qplib_rcfw *rcfw,/* failed with status */ dev_err (&rcfw->pdev->dev,"cmdq[%#x]=%#x status %#x\n" , cookie, opcode, evnt->status); - rc = -EFAULT ; + rc = -EIO ; }return rc;
Signature · 1 line
2.5.5
Selvin Xavier
maintainerPATCH 5/10 RDMA/bnxt_re: Fix a possible NULL pointer dereference
In reply to Selvin XavierFrom: Kalesh AP <[email protected]>
There is a possibility of a NULL pointer dereference in the failure path of bnxt_re_add_device().
To address that, moved the update of "rdev->adev" to bnxt_re_dev_add().
--- drivers/infiniband/hw/bnxt_re/main.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/infiniband/hw/bnxt_re/main.c +3 -3
@@ -960,7 +960,7 @@ static int bnxt_re_register_ib(struct bnxt_re_dev *rdev)return ib_register_device (ibdev,"bnxt_re%d" , &rdev->en_dev->pdev->dev); } -static struct bnxt_re_dev *bnxt_re_dev_add (struct bnxt_aux_priv *aux_priv, +static struct bnxt_re_dev *bnxt_re_dev_add (struct auxiliary_device *adev,struct bnxt_en_dev *en_dev) {struct bnxt_re_dev *rdev;
@@ -976,6 +976,7 @@ static struct bnxt_re_dev *bnxt_re_dev_add(struct bnxt_aux_priv *aux_priv, rdev->nb.notifier_call =NULL ; rdev->netdev = en_dev->net; rdev->en_dev = en_dev; + rdev->adev = adev; rdev->id = rdev->en_dev->pdev->devfn;INIT_LIST_HEAD (&rdev->qp_list);mutex_init (&rdev->qp_lock);
@@ -1829,7 +1830,6 @@ static void bnxt_re_update_en_info_rdev(struct bnxt_re_dev *rdev,*/ rtnl_lock (); en_info->rdev = rdev; - rdev->adev = adev;rtnl_unlock (); }
@@ -1846,7 +1846,7 @@ static int bnxt_re_add_device(struct auxiliary_device *adev, u8 op_type) en_dev = en_info->en_dev; - rdev =bnxt_re_dev_add (aux_priv, en_dev); + rdev =bnxt_re_dev_add (adev, en_dev);if (!rdev || !rdev_to_dev (rdev)) { rc = -ENOMEM ; goto exit;
Signature · 1 line
2.5.5
Selvin Xavier
maintainerPATCH 6/10 RDMA/bnxt_re: Avoid CPU lockups due fifo occupancy check loop
In reply to Selvin XavierDriver waits indefinitely for the fifo occupancy to go below
a threshold as soon as the pacing interrupt is received. This can cause soft lockup on one of the processors, if the rate of DB is very high.
Add a loop count for FPGA and exit the __wait_for_fifo_occupancy_below_th if the loop is taking more time. Pacing will be continuing until the occupancy is below the threshold. This is ensured by the checks in bnxt_re_pacing_timer_exp and further scheduling the work for pacing based on the fifo occupancy.
--- drivers/infiniband/hw/bnxt_re/main.c | 9 +++++++++ 1 file changed, 9 insertions(+)
drivers/infiniband/hw/bnxt_re/main.c +9 -0
@@ -534,6 +534,7 @@ static bool is_dbr_fifo_full(struct bnxt_re_dev *rdev)static void __wait_for_fifo_occupancy_below_th (struct bnxt_re_dev *rdev) {struct bnxt_qplib_db_pacing_data *pacing_data = rdev->qplib_res.pacing_data; +u32 retry_fifo_check =1000 ;u32 fifo_occup;/* loop shouldn't run infintely as the occupancy usually goes
@@ -547,6 +548,14 @@ static void __wait_for_fifo_occupancy_below_th(struct bnxt_re_dev *rdev)if (fifo_occup < pacing_data->pacing_th)break ; +if (!retry_fifo_check--) { +dev_info_once (rdev_to_dev (rdev), +"%s: fifo_occup = 0x%xfifo_max_depth = 0x%x pacing_th = 0x%x\n" , + __func__, fifo_occup, pacing_data->fifo_max_depth, + pacing_data->pacing_th); +break ; + } + } }
Signature · 1 line
2.5.5
Selvin Xavier
maintainerPATCH 7/10 RDMA/bnxt_re: Fix an error path in bnxt_re_add_device
In reply to Selvin XavierFrom: Kalesh AP <[email protected]>
In bnxt_re_add_device(), when register netdev notifier fails, driver is not unregistering the IB device in the error cleanup path. Also, removed the duplicate cleanup in error path of bnxt_re_probe.
--- drivers/infiniband/hw/bnxt_re/main.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-)
drivers/infiniband/hw/bnxt_re/main.c +3 -9
@@ -1880,12 +1880,14 @@ static int bnxt_re_add_device(struct auxiliary_device *adev, u8 op_type) rdev->nb.notifier_call =NULL ;pr_err ("%s: Cannot register to netdevice_notifier" ,ROCE_DRV_MODULE_NAME ); -return rc; + goto re_dev_unreg; }bnxt_re_setup_cc (rdev, true);return 0 ; +re_dev_unreg: +ib_unregister_device (&rdev->ibdev); re_dev_uninit:bnxt_re_update_en_info_rdev (NULL , en_info, adev);bnxt_re_dev_uninit (rdev,BNXT_RE_COMPLETE_REMOVE );
@@ -2029,15 +2031,7 @@ static int bnxt_re_probe(struct auxiliary_device *adev,auxiliary_set_drvdata (adev, en_info); rc =bnxt_re_add_device (adev,BNXT_RE_COMPLETE_INIT ); -if (rc) - goto err;mutex_unlock (&bnxt_re_mutex); -return 0 ; - -err: -mutex_unlock (&bnxt_re_mutex); -bnxt_re_remove (adev); -return rc; }
Signature · 1 line
2.5.5
Selvin Xavier
maintainerPATCH 8/10 RDMA/bnxt_re: Change the sequence of updating the CQ toggle value
In reply to Selvin XavierFrom: Chandramohan Akula <[email protected]>
Currently the CQ toggle value in the shared page (read by the userlib) is updated as part of the cqn_handler. There is a potential race of application calling the CQ ARM doorbell immediately and using the old toggle value.
Change the sequence of updating CQ toggle value to update in the bnxt_qplib_service_nq function immediately after reading the toggle value to be in sync with the HW updated value.
--- drivers/infiniband/hw/bnxt_re/main.c | 8 +------- drivers/infiniband/hw/bnxt_re/qplib_fp.c | 5 +++++ 2 files changed, 6 insertions(+), 7 deletions(-)
drivers/infiniband/hw/bnxt_re/main.c +1 -7
@@ -1274,15 +1274,9 @@ static int bnxt_re_cqn_handler(struct bnxt_qplib_nq *nq, { struct bnxt_re_cq *cq = container_of(handle, struct bnxt_re_cq, qplib_cq); -u32 *cq_ptr; -if (cq->ib_cq.comp_handler) { -if (cq->uctx_cq_page) { - cq_ptr = (u32 *)cq->uctx_cq_page; - *cq_ptr = cq->qplib_cq.toggle; - } +if (cq->ib_cq.comp_handler) (*cq->ib_cq.comp_handler)(&cq->ib_cq, cq->ib_cq.cq_context); - }return 0 ; }
drivers/infiniband/hw/bnxt_re/qplib_fp.c +5 -0
@@ -327,6 +327,7 @@ static void bnxt_qplib_service_nq(struct tasklet_struct *t)case NQ_BASE_TYPE_CQ_NOTIFICATION : {struct nq_cn *nqcne = (struct nq_cn *)nqe; +struct bnxt_re_cq *cq_p; q_handle =le32_to_cpu (nqcne->cq_handle_low); q_handle |= (u64 )le32_to_cpu (nqcne->cq_handle_high)
@@ -337,6 +338,10 @@ static void bnxt_qplib_service_nq(struct tasklet_struct *t) cq->toggle = (le16_to_cpu (nqe->info10_type) &NQ_CN_TOGGLE_MASK ) >>NQ_CN_TOGGLE_SFT ; cq->dbinfo.toggle = cq->toggle; + cq_p = container_of(cq, struct bnxt_re_cq, qplib_cq); +if (cq_p->uctx_cq_page) + *((u32 *)cq_p->uctx_cq_page) = cq->toggle; +bnxt_qplib_armen_db (&cq->dbinfo,DBC_DBC_TYPE_CQ_ARMENA );spin_lock_bh (&cq->compl_lock);
Signature · 1 line
2.5.5
Selvin Xavier
maintainerPATCH 9/10 RDMA/bnxt_re: Fix a bug while setting up Level-2 PBL pages
In reply to Selvin XavierFrom: Bhargava Chenna Marreddy <[email protected]>
Avoid memory corruption while setting up Level-2 PBL pages for the non MR resources when num_pages > 256K.
There will be a single PDE page address (contiguous pages in the case of > PAGE_SIZE), but, current logic assumes multiple pages, leading to invalid memory access after 256K PBL entries in the PDE.
--- drivers/infiniband/hw/bnxt_re/qplib_res.c | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-)
drivers/infiniband/hw/bnxt_re/qplib_res.c +3 -16
@@ -257,22 +257,9 @@ int bnxt_qplib_alloc_init_hwq(struct bnxt_qplib_hwq *hwq, dst_virt_ptr = (dma_addr_t **)hwq->pbl[PBL_LVL_0 ].pg_arr; src_phys_ptr = hwq->pbl[PBL_LVL_1 ].pg_map_arr; -if (hwq_attr->type ==HWQ_TYPE_MR ) { -/* For MR it is expected that we supply only 1 contigous -* page i.e only 1 entry in the PDL that will contain -* all the PBLs for the user supplied memory region -*/ -for (i =0 ; i < hwq->pbl[PBL_LVL_1 ].pg_count; - i++) - dst_virt_ptr[0 ][i] = src_phys_ptr[i] | - flag; - }else { -for (i =0 ; i < hwq->pbl[PBL_LVL_1 ].pg_count; - i++) - dst_virt_ptr[PTR_PG (i)][PTR_IDX (i)] = - src_phys_ptr[i] | -PTU_PDE_VALID ; - } +for (i =0 ; i < hwq->pbl[PBL_LVL_1 ].pg_count; i++) + dst_virt_ptr[0 ][i] = src_phys_ptr[i] | flag; +/* Alloc or init PTEs */ rc =__alloc_pbl (res, &hwq->pbl[PBL_LVL_2 ], hwq_attr->sginfo);
Signature · 1 line
2.5.5
Selvin Xavier
maintainerPATCH 10/10 RDMA/bnxt_re: Fix the GID table length
In reply to Selvin XavierFrom: Kalesh AP <[email protected]>
GID table length is reported by FW. The gid index which is passed to the driver during modify_qp/create_ah is restricted by the sgid_index field of struct ib_global_route. sgid_index is u8 and the max sgid possible is 256.
Each GID entry in HW will have 2 GID entries in the kernel gid table. So we can support twice the gid table size reported by FW. Also, restrict the max GID to 256 also.
--- drivers/infiniband/hw/bnxt_re/qplib_sp.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/infiniband/hw/bnxt_re/qplib_sp.c +8 -1
@@ -159,7 +159,14 @@ int bnxt_qplib_get_dev_attr(struct bnxt_qplib_rcfw *rcfw,if (!bnxt_qplib_is_chip_gen_p7 (rcfw->res->cctx)) attr->l2_db_size = (sb->l2_db_space_size +1 ) * (0x01 <<RCFW_DBR_BASE_PAGE_SHIFT ); - attr->max_sgid =BNXT_QPLIB_NUM_GIDS_SUPPORTED ; +/* +* Read the max gid supported by HW. +* For each entry in HW GID in HW table, we consume 2 +* GID entries in the kernel GID table. So max_gid reported +* to stack can be up to twice the value reported by the HW, up to 256 gids. +*/ + attr->max_sgid =le32_to_cpu (sb->max_gid); + attr->max_sgid =min_t (u32,BNXT_QPLIB_NUM_GIDS_SUPPORTED ,2 * attr->max_sgid); attr->dev_cap_flags =le16_to_cpu (sb->dev_cap_flags); attr->dev_cap_flags2 =le16_to_cpu (sb->dev_cap_ext_flags_2);
Signature · 1 line
2.5.5
Quoted 21 lines · Selvin Xavier
Few bugfixes for
bnxt_redriver. Please review and apply.Thanks,
Selvin XavierAbhishek Mohapatra (1):
RDMA/bnxt_re: Fix the max CQ WQEs for older adaptersBhargava Chenna Marreddy (1):
RDMA/bnxt_re: Fix a bug while setting up Level-2 PBL pagesChandramohan Akula (1):
RDMA/bnxt_re: Change the sequence of updating the CQ toggle valueKalesh AP (5):
RDMA/bnxt_re: Fix out of bound check
RDMA/bnxt_re: Return more meaningful error RDMA/bnxt_re: Fix a possible NULL pointer dereference RDMA/bnxt_re: Fix an error path inbnxt_re_add_deviceRDMA/bnxt_re: Fix the GID table length
Quoted 5 lines
Kashyap Desai (1):
RDMA/bnxt_re: Fix incorrect dereference of srq in async eventSelvin Xavier (1):
RDMA/bnxt_re: Avoid CPU lockups due fifo occupancy check loop
Applied to for-rc, thanks
Jason