Skip to contentSkip to conversation

bnxt_en: Bug fixes.

PATCH In review Michael Chan 2 participants 7 messages Sep 3, 2018 – Sep 4, 2018

Subsystems (3)
lore
Share & tools
lore

Michael Chan

author maintainer
#0 lore raw

cover bnxt_en: Bug fixes.

This short series fixes resource related logic in the driver, mostly affecting the RDMA driver under corner cases.

Michael Chan (3):
bnxt_en: Fix firmware signaled resource change logic in open.
bnxt_en: Clean up unused functions.
bnxt_en: Do not adjust max_cp_rings by the ones used by RDMA.

 drivers/net/ethernet/broadcom/bnxt/bnxt.c       | 22 +++++++++++++++-------
 drivers/net/ethernet/broadcom/bnxt/bnxt.h       |  3 +--
 drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c |  7 ++++---
 drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c   | 20 --------------------
 drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h   |  1 -
 5 files changed, 20 insertions(+), 33 deletions(-)
Signature · 1 line
2.5.1
author maintainer
#1 lore raw

PATCH 1/3 bnxt_en: Fix firmware signaled resource change logic in open.

When the driver detects that resources have changed during open, it should reset the rx and tx rings to 0. This will properly setup the init sequence to initialize the default rings again. We also need to signal the RDMA driver to stop and clear its interrupts. We then call the RoCE driver to restart if a new set of default rings is successfully reserved.

Fixes 25e1acd6b92b ("bnxt_en: Notify firmware about IF state changes.")
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +10 -3
index 8bb1e38..6a1baf3 100644
@@ -6684,6 +6684,8 @@ static int bnxt_hwrm_if_change(struct bnxt *bp, bool up)
 		hw_resc->resv_rx_rings = 0;
 		hw_resc->resv_hw_ring_grps = 0;
 		hw_resc->resv_vnics = 0;
+		bp->tx_nr_rings = 0;
+		bp->rx_nr_rings = 0;
 	}
 	return rc;
 }
@@ -8769,20 +8771,25 @@ static int bnxt_init_dflt_ring_mode(struct bnxt *bp)
 	if (bp->tx_nr_rings)
 		return 0;
 
+	bnxt_ulp_irq_stop(bp);
+	bnxt_clear_int_mode(bp);
 	rc = bnxt_set_dflt_rings(bp, true);
 	if (rc) {
 		netdev_err(bp->dev, "Not enough rings available.\n");
-		return rc;
+		goto init_dflt_ring_err;
 	}
 	rc = bnxt_init_int_mode(bp);
 	if (rc)
-		return rc;
+		goto init_dflt_ring_err;
+
 	bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
 	if (bnxt_rfs_supported(bp) && bnxt_rfs_capable(bp)) {
 		bp->flags |= BNXT_FLAG_RFS;
 		bp->dev->features |= NETIF_F_NTUPLE;
 	}
-	return 0;
+init_dflt_ring_err:
+	bnxt_ulp_irq_restart(bp, rc);
+	return rc;
 }
 
 int bnxt_restore_pf_fw_resources(struct bnxt *bp)
Signature · 1 line
2.5.1
author maintainer
#2 lore raw

PATCH 2/3 bnxt_en: Clean up unused functions.

In reply to Michael Chan

Remove unused bnxt_subtract_ulp_resources(). Change bnxt_get_max_func_irqs() to static since it is only locally used.

---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c     |  2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt.h     |  1 -
 drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 15 ---------------
 drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h |  1 -
 4 files changed, 1 insertion(+), 18 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +1 -1
index 6a1baf3..6472ce4 100644
@@ -5918,7 +5918,7 @@ void bnxt_set_max_func_cp_rings(struct bnxt *bp, unsigned int max)
 	bp->hw_resc.max_cp_rings = max;
 }
 
-unsigned int bnxt_get_max_func_irqs(struct bnxt *bp)
+static unsigned int bnxt_get_max_func_irqs(struct bnxt *bp)
 {
 	struct bnxt_hw_resc *hw_resc = &bp->hw_resc;
 
drivers/net/ethernet/broadcom/bnxt/bnxt.h +0 -1
index fefa011..c4c77b9 100644
@@ -1482,7 +1482,6 @@ unsigned int bnxt_get_max_func_stat_ctxs(struct bnxt *bp);
 void bnxt_set_max_func_stat_ctxs(struct bnxt *bp, unsigned int max);
 unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp);
 void bnxt_set_max_func_cp_rings(struct bnxt *bp, unsigned int max);
-unsigned int bnxt_get_max_func_irqs(struct bnxt *bp);
 int bnxt_get_avail_msix(struct bnxt *bp, int num);
 int bnxt_reserve_rings(struct bnxt *bp);
 void bnxt_tx_disable(struct bnxt *bp);
drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +0 -15
index c37b284..deac73e 100644
@@ -220,21 +220,6 @@ int bnxt_get_ulp_msix_base(struct bnxt *bp)
 	return 0;
 }
 
-void bnxt_subtract_ulp_resources(struct bnxt *bp, int ulp_id)
-{
-	ASSERT_RTNL();
-	if (bnxt_ulp_registered(bp->edev, ulp_id)) {
-		struct bnxt_en_dev *edev = bp->edev;
-		unsigned int msix_req, max;
-
-		msix_req = edev->ulp_tbl[ulp_id].msix_requested;
-		max = bnxt_get_max_func_cp_rings(bp);
-		bnxt_set_max_func_cp_rings(bp, max - msix_req);
-		max = bnxt_get_max_func_stat_ctxs(bp);
-		bnxt_set_max_func_stat_ctxs(bp, max - 1);
-	}
-}
-
 static int bnxt_send_msg(struct bnxt_en_dev *edev, int ulp_id,
 			 struct bnxt_fw_msg *fw_msg)
 {
drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +0 -1
index df48ac7..d9bea37 100644
@@ -90,7 +90,6 @@ static inline bool bnxt_ulp_registered(struct bnxt_en_dev *edev, int ulp_id)
 
 int bnxt_get_ulp_msix_num(struct bnxt *bp);
 int bnxt_get_ulp_msix_base(struct bnxt *bp);
-void bnxt_subtract_ulp_resources(struct bnxt *bp, int ulp_id);
 void bnxt_ulp_stop(struct bnxt *bp);
 void bnxt_ulp_start(struct bnxt *bp);
 void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs);
Signature · 1 line
2.5.1
author maintainer
#3 lore raw

PATCH 3/3 bnxt_en: Do not adjust max_cp_rings by the ones used by RDMA.

In reply to Michael Chan

Currently, the driver adjusts the bp->hw_resc.max_cp_rings by the number of MSIX vectors used by RDMA. There is one code path in open that needs to check the true max_cp_rings including any used by RDMA. This code is now checking for the reduced max_cp_rings which will fail when the number of cp rings is very small.

To fix this in a clean way, we don't adjust max_cp_rings anymore. Instead, we add a helper bnxt_get_max_func_cp_rings_for_en() to get the reduced max_cp_rings when appropriate.

Fixes ec86f14ea506 ("bnxt_en: Add ULP calls to stop and restart IRQs.")
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c       | 7 ++++---
 drivers/net/ethernet/broadcom/bnxt/bnxt.h       | 2 +-
 drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c | 7 ++++---
 drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c   | 5 -----
 4 files changed, 9 insertions(+), 12 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.c +4 -3
index 6472ce4..cecbb1d 100644
@@ -5913,9 +5913,9 @@ unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp)
 	return bp->hw_resc.max_cp_rings;
 }
 
-void bnxt_set_max_func_cp_rings(struct bnxt *bp, unsigned int max)
+unsigned int bnxt_get_max_func_cp_rings_for_en(struct bnxt *bp)
 {
-	bp->hw_resc.max_cp_rings = max;
+	return bp->hw_resc.max_cp_rings - bnxt_get_ulp_msix_num(bp);
 }
 
 static unsigned int bnxt_get_max_func_irqs(struct bnxt *bp)
@@ -8631,7 +8631,8 @@ static void _bnxt_get_max_rings(struct bnxt *bp, int *max_rx, int *max_tx,
 
 	*max_tx = hw_resc->max_tx_rings;
 	*max_rx = hw_resc->max_rx_rings;
-	*max_cp = min_t(int, hw_resc->max_irqs, hw_resc->max_cp_rings);
+	*max_cp = min_t(int, bnxt_get_max_func_cp_rings_for_en(bp),
+			hw_resc->max_irqs);
 	*max_cp = min_t(int, *max_cp, hw_resc->max_stat_ctxs);
 	max_ring_grps = hw_resc->max_hw_ring_grps;
 	if (BNXT_CHIP_TYPE_NITRO_A0(bp) && BNXT_PF(bp)) {
drivers/net/ethernet/broadcom/bnxt/bnxt.h +1 -1
index c4c77b9..bde3846 100644
@@ -1481,7 +1481,7 @@ int bnxt_hwrm_set_coal(struct bnxt *);
 unsigned int bnxt_get_max_func_stat_ctxs(struct bnxt *bp);
 void bnxt_set_max_func_stat_ctxs(struct bnxt *bp, unsigned int max);
 unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp);
-void bnxt_set_max_func_cp_rings(struct bnxt *bp, unsigned int max);
+unsigned int bnxt_get_max_func_cp_rings_for_en(struct bnxt *bp);
 int bnxt_get_avail_msix(struct bnxt *bp, int num);
 int bnxt_reserve_rings(struct bnxt *bp);
 void bnxt_tx_disable(struct bnxt *bp);
drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c +4 -3
index 6d583bc..fcd085a 100644
@@ -451,7 +451,7 @@ static int bnxt_hwrm_func_vf_resc_cfg(struct bnxt *bp, int num_vfs)
 
 	bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_FUNC_VF_RESOURCE_CFG, -1, -1);
 
-	vf_cp_rings = hw_resc->max_cp_rings - bp->cp_nr_rings;
+	vf_cp_rings = bnxt_get_max_func_cp_rings_for_en(bp) - bp->cp_nr_rings;
 	vf_stat_ctx = hw_resc->max_stat_ctxs - bp->num_stat_ctxs;
 	if (bp->flags & BNXT_FLAG_AGG_RINGS)
 		vf_rx_rings = hw_resc->max_rx_rings - bp->rx_nr_rings * 2;
@@ -549,7 +549,8 @@ static int bnxt_hwrm_func_cfg(struct bnxt *bp, int num_vfs)
 	max_stat_ctxs = hw_resc->max_stat_ctxs;
 
 	/* Remaining rings are distributed equally amongs VF's for now */
-	vf_cp_rings = (hw_resc->max_cp_rings - bp->cp_nr_rings) / num_vfs;
+	vf_cp_rings = (bnxt_get_max_func_cp_rings_for_en(bp) -
+		       bp->cp_nr_rings) / num_vfs;
 	vf_stat_ctx = (max_stat_ctxs - bp->num_stat_ctxs) / num_vfs;
 	if (bp->flags & BNXT_FLAG_AGG_RINGS)
 		vf_rx_rings = (hw_resc->max_rx_rings - bp->rx_nr_rings * 2) /
@@ -643,7 +644,7 @@ static int bnxt_sriov_enable(struct bnxt *bp, int *num_vfs)
 	 */
 	vfs_supported = *num_vfs;
 
-	avail_cp = hw_resc->max_cp_rings - bp->cp_nr_rings;
+	avail_cp = bnxt_get_max_func_cp_rings_for_en(bp) - bp->cp_nr_rings;
 	avail_stat = hw_resc->max_stat_ctxs - bp->num_stat_ctxs;
 	avail_cp = min_t(int, avail_cp, avail_stat);
 
drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +0 -5
index deac73e..beee612 100644
@@ -169,7 +169,6 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, int ulp_id,
 		edev->ulp_tbl[ulp_id].msix_requested = avail_msix;
 	}
 	bnxt_fill_msix_vecs(bp, ent);
-	bnxt_set_max_func_cp_rings(bp, max_cp_rings - avail_msix);
 	edev->flags |= BNXT_EN_FLAG_MSIX_REQUESTED;
 	return avail_msix;
 }
@@ -178,7 +177,6 @@ static int bnxt_free_msix_vecs(struct bnxt_en_dev *edev, int ulp_id)
 {
 	struct net_device *dev = edev->net;
 	struct bnxt *bp = netdev_priv(dev);
-	int max_cp_rings, msix_requested;
 
 	ASSERT_RTNL();
 	if (ulp_id != BNXT_ROCE_ULP)
@@ -187,9 +185,6 @@ static int bnxt_free_msix_vecs(struct bnxt_en_dev *edev, int ulp_id)
 	if (!(edev->flags & BNXT_EN_FLAG_MSIX_REQUESTED))
 		return 0;
 
-	max_cp_rings = bnxt_get_max_func_cp_rings(bp);
-	msix_requested = edev->ulp_tbl[ulp_id].msix_requested;
-	bnxt_set_max_func_cp_rings(bp, max_cp_rings + msix_requested);
 	edev->ulp_tbl[ulp_id].msix_requested = 0;
 	edev->flags &= ~BNXT_EN_FLAG_MSIX_REQUESTED;
 	if (netif_running(dev)) {
Signature · 1 line
2.5.1
maintainer
#4 lore raw
In reply to Michael Chan

From: Michael Chan <[email protected]>
Date: Mon, 3 Sep 2018 04:23:16 -0400

Quoted 2 lines

This short series fixes resource related logic in the driver, mostly affecting the RDMA driver under corner cases.

Series applied, thanks Michael.

Do you want patch #3 queued up for -stable?

author maintainer
#5 lore raw
Quoted 10 lines · David Miller

From: Michael Chan <[email protected]>
Date: Mon, 3 Sep 2018 04:23:16 -0400

Quoted 2 lines

This short series fixes resource related logic in the driver, mostly affecting the RDMA driver under corner cases.

Series applied, thanks Michael.

Do you want patch #3 queued up for -stable?

Yes, please go ahead. Thanks.

author maintainer
#6 lore raw
Quoted 5 lines · Michael Chan
Quoted 4 lines · David Miller

From: Michael Chan <[email protected]>
Date: Mon, 3 Sep 2018 04:23:16 -0400

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

This short series fixes resource related logic in the driver, mostly affecting the RDMA driver under corner cases.

Series applied, thanks Michael.

Do you want patch #3 queued up for -stable?

Yes, please go ahead. Thanks.

But there is a dependency on patch #2 though. So #2 needs to be queued as well.