Skip to contentSkip to conversation

bnx2x: fixes

PATCH In review Dmitry Kravkov 6 participants 16 messages Aug 12, 2013 – Aug 13, 2013

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v5 cover bnx2x: fixes

Hello Dave

Please consider applying the series of bnx2x fixes to net:
* statistics may cause FW assert
* missing fairness configuration in DCB flow
* memory leak in sriov related part
* Illegal PTE access
* Pagefault crash in shutdown flow with cnic
v1->v2
* fixed sparse error pointed by Joe Perches
* added missing signed-off from Sergei Shtylyov
v2->v3
* added missing signed-off from Sergei Shtylyov
* fixed formatting from Sergei Shtylyov
v3->v4
* patch 1/6: fixed declaration order
* patch 2/6 replaced with: protect flows using set_bit constraints
v4->v5
* patch 2/6: replace proprietary locking with semaphore
* droped 1/6: since adds redundant code from Benjamin Poirier

Thanks
Dmitry

#1 lore raw

PATCH v5 1/5 bnx2x: protect different statistics flows

Add locking to protect different statistics flows from running simultaneously.
This in order to serialize statistics requests sent to FW, otherwise two outstanding queries may cause FW assert.

---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x.h       |  2 +
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c  |  1 +
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c | 66 ++++++++++++++++++-----
 3 files changed, 57 insertions(+), 12 deletions(-)
drivers/net/ethernet/broadcom/bnx2x/bnx2x.h +2 -0
index d80e34b..98be67f 100644
@@ -1830,6 +1830,8 @@ struct bnx2x {
 
 	int fp_array_size;
 	u32 dump_preset_idx;
+	bool					stats_started;
+	struct semaphore			stats_sema;
 };
 
 /* Tx queues may be less or equal to Rx queues */
drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +1 -0
index e06186c..6e1e9e7 100644
@@ -11524,6 +11524,7 @@ static int bnx2x_init_bp(struct bnx2x *bp)
 	mutex_init(&bp->port.phy_mutex);
 	mutex_init(&bp->fw_mb_mutex);
 	spin_lock_init(&bp->stats_lock);
+	sema_init(&bp->stats_sema, 1);
 
 	INIT_DELAYED_WORK(&bp->sp_task, bnx2x_sp_task);
 	INIT_DELAYED_WORK(&bp->sp_rtnl_task, bnx2x_sp_rtnl_task);
drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c +54 -12

157 lines not on this page · raw message

Signature · 1 line
1.8.1.4
#2 lore raw

Re: [PATCH net v5 1/5] bnx2x: protect different statistics flows

drivers/net/ethernet/broadcom/bnx2x/bnx2x.h struct bnx2x { · line 1834

Quoted 2 lines · On Tue, 13 Aug 2013 02:24:59 +0300 "Dmitry Kravkov" <[email protected]> wrote:
drivers/net/ethernet/broadcom/bnx2x/bnx2x.h +2 -0
+	bool					stats_started;
+	struct semaphore			stats_sema;

Is there a reason to use a counting semaphore? Do you expect it to be held across user process boundary? or want count > 1?

Use of semaphores as a locking primitive is discouraged, instead us a mutex.

maintainer
#3 lore raw

Re: [PATCH net v5 1/5] bnx2x: protect different statistics flows

From: Stephen Hemminger <[email protected]>
Date: Mon, 12 Aug 2013 17:12:08 -0700

Quoted 11 lines

drivers/net/ethernet/broadcom/bnx2x/bnx2x.h struct bnx2x { · line 1834

Quoted 2 lines · On Tue, 13 Aug 2013 02:24:59 +0300 "Dmitry Kravkov" <[email protected]> wrote:
drivers/net/ethernet/broadcom/bnx2x/bnx2x.h +2 -0
+	bool					stats_started;
+	struct semaphore			stats_sema;

Is there a reason to use a counting semaphore? Do you expect it to be held across user process boundary? or want count > 1?

Use of semaphores as a locking primitive is discouraged, instead us a mutex.

Would you please look at the feedback I gave these guys to the previous iteration of these changes?

They were using custom locking primitives and semaphores gave the best approximation to the semantics they were looking for.

Thanks.

#4 lore raw

Re: [PATCH net v5 1/5] bnx2x: protect different statistics flows

Quoted 12 lines · On Mon, 12 Aug 2013 21:24:02 -0700 (PDT) David Miller <[email protected]>…

From: Stephen Hemminger <[email protected]>
Date: Mon, 12 Aug 2013 17:12:08 -0700

Quoted 9 lines

drivers/net/ethernet/broadcom/bnx2x/bnx2x.h struct bnx2x { · line 1834

Quoted 2 lines · On Tue, 13 Aug 2013 02:24:59 +0300 "Dmitry Kravkov" <[email protected]> wrote:
drivers/net/ethernet/broadcom/bnx2x/bnx2x.h +2 -0
+	bool					stats_started;
+	struct semaphore			stats_sema;

Is there a reason to use a counting semaphore? Do you expect it to be held across user process boundary? or want count > 1?

Quoted 8 lines
Quoted 2 lines

Use of semaphores as a locking primitive is discouraged, instead us a mutex.

Would you please look at the feedback I gave these guys to the previous iteration of these changes?

They were using custom locking primitives and semaphores gave the best approximation to the semantics they were looking for.

Your right in this case sempahore makes sense because it is used to hold off process while hardware responds.

Acked-by on v5 1/5
#5 lore raw

Re: [PATCH net v5 1/5] bnx2x: protect different statistics flows

In reply to Dmitry Kravkov

drivers/net/ethernet/broadcom/bnx2x/bnx2x.h

Quoted 16 lines · Dmitry Kravkov

Add locking to protect different statistics flows from running simultaneously.
This in order to serialize statistics requests sent to FW, otherwise two outstanding queries may cause FW assert.

---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x.h       |  2 +
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c  |  1 +
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_stats.c | 66 ++++++++++++++++++-----
 3 files changed, 57 insertions(+), 12 deletions(-)
drivers/net/ethernet/broadcom/bnx2x/bnx2x.h +0 -0
index d80e34b..98be67f 100644

Thanks!

neal

#6 lore raw

PATCH v5 2/5 bnx2x: update fairness parameters following DCB negotiation

In reply to Dmitry Kravkov

ETS can be enabled as a result of DCB negotiation, then fairness must be recalculated after each negotiation.

---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x.h      |  2 ++
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_dcb.c  |  4 ++++
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 30 ++++++++++++++----------
 3 files changed, 24 insertions(+), 12 deletions(-)
drivers/net/ethernet/broadcom/bnx2x/bnx2x.h +2 -0
index 98be67f..f07a7ff 100644
@@ -2453,4 +2453,6 @@ enum bnx2x_pci_bus_speed {
 	BNX2X_PCI_LINK_SPEED_5000 = 5000,
 	BNX2X_PCI_LINK_SPEED_8000 = 8000
 };
+
+void bnx2x_set_local_cmng(struct bnx2x *bp);
 #endif /* bnx2x.h */
drivers/net/ethernet/broadcom/bnx2x/bnx2x_dcb.c +4 -0
index 0c94df4..f9122f2 100644
@@ -753,6 +753,10 @@ void bnx2x_dcbx_set_params(struct bnx2x *bp, u32 state)
 		bnx2x_pfc_set_pfc(bp);
 
 		bnx2x_dcbx_update_ets_params(bp);
+
+		/* ets may affect cmng configuration: reinit it in hw */
+		bnx2x_set_local_cmng(bp);
+
 		bnx2x_dcbx_resume_hw_tx(bp);
 
 		return;
drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +18 -12
index 6e1e9e7..78b7195 100644
@@ -2476,7 +2476,7 @@ static void bnx2x_cmng_fns_init(struct bnx2x *bp, u8 read_cfg, u8 cmng_type)
 
 	input.port_rate = bp->link_vars.line_speed;
 
-	if (cmng_type == CMNG_FNS_MINMAX) {
+	if (cmng_type == CMNG_FNS_MINMAX && input.port_rate) {
 		int vn;
 
 		/* read mf conf from shmem */
@@ -2533,6 +2533,21 @@ static void storm_memset_cmng(struct bnx2x *bp,
 	}
 }
 
+/* init cmng mode in HW according to local configuration */
+void bnx2x_set_local_cmng(struct bnx2x *bp)
+{
+	int cmng_fns = bnx2x_get_cmng_fns_mode(bp);
+
+	if (cmng_fns != CMNG_FNS_NONE) {
+		bnx2x_cmng_fns_init(bp, false, cmng_fns);
+		storm_memset_cmng(bp, &bp->cmng, BP_PORT(bp));
+	} else {
+		/* rate shaping and fairness are disabled */
+		DP(NETIF_MSG_IFUP,
+		   "single function mode without fairness\n");
+	}
+}
+
 /* This function is called upon link interrupt */
 static void bnx2x_link_attn(struct bnx2x *bp)
 {
@@ -2568,17 +2583,8 @@ static void bnx2x_link_attn(struct bnx2x *bp)
 			bnx2x_stats_handle(bp, STATS_EVENT_LINK_UP);
 	}
 
-	if (bp->link_vars.link_up && bp->link_vars.line_speed) {
-		int cmng_fns = bnx2x_get_cmng_fns_mode(bp);
-
-		if (cmng_fns != CMNG_FNS_NONE) {
-			bnx2x_cmng_fns_init(bp, false, cmng_fns);
-			storm_memset_cmng(bp, &bp->cmng, BP_PORT(bp));
-		} else
-			/* rate shaping and fairness are disabled */
-			DP(NETIF_MSG_IFUP,
-			   "single function mode without fairness\n");
-	}
+	if (bp->link_vars.link_up && bp->link_vars.line_speed)
+		bnx2x_set_local_cmng(bp);
 
 	__bnx2x_link_report(bp);
 
Signature · 1 line
1.8.1.4
#7 lore raw

PATCH v5 3/5 bnx2x: fix memory leak in VF

In reply to Dmitry Kravkov

From: Ariel Elior <[email protected]>

---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c +1 -1
index 95861ef..44104fb 100644
@@ -3463,7 +3463,7 @@ int bnx2x_vf_pci_alloc(struct bnx2x *bp)
 alloc_mem_err:
 	BNX2X_PCI_FREE(bp->vf2pf_mbox, bp->vf2pf_mbox_mapping,
 		       sizeof(struct bnx2x_vf_mbx_msg));
-	BNX2X_PCI_FREE(bp->vf2pf_mbox, bp->vf2pf_mbox_mapping,
+	BNX2X_PCI_FREE(bp->vf2pf_mbox, bp->pf2vf_bulletin_mapping,
 		       sizeof(union pf_vf_bulletin));
 	return -ENOMEM;
 }
Signature · 1 line
1.8.1.4
#8 lore raw

PATCH v5 4/5 bnx2x: fix PTE write access error

In reply to Dmitry Kravkov

From: Barak Witkowsky <[email protected]>

PTE write access error might occur in MF_ALLOWED mode when IOMMU is active. The patch adds rmmod HSI indicating to MFW to stop running queries which might trigger this failure.

---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x.h      | 1 +
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_hsi.h  | 5 +++++
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 9 +++++++++
 3 files changed, 15 insertions(+)
drivers/net/ethernet/broadcom/bnx2x/bnx2x.h +1 -0
index f07a7ff..ce9b387 100644
@@ -1502,6 +1502,7 @@ struct bnx2x {
 #define BC_SUPPORTS_DCBX_MSG_NON_PMF	(1 << 21)
 #define IS_VF_FLAG			(1 << 22)
 #define INTERRUPTS_ENABLED_FLAG		(1 << 23)
+#define BC_SUPPORTS_RMMOD_CMD		(1 << 24)
 
 #define BP_NOMCP(bp)			((bp)->flags & NO_MCP_FLAG)
 
drivers/net/ethernet/broadcom/bnx2x/bnx2x_hsi.h +5 -0
index 5018e52..32767f6 100644
@@ -1300,6 +1300,9 @@ struct drv_func_mb {
 
 	#define DRV_MSG_CODE_EEE_RESULTS_ACK            0xda000000
 
+	#define DRV_MSG_CODE_RMMOD                      0xdb000000
+	#define REQ_BC_VER_4_RMMOD_CMD                  0x0007080f
+
 	#define DRV_MSG_CODE_SET_MF_BW                  0xe0000000
 	#define REQ_BC_VER_4_SET_MF_BW                  0x00060202
 	#define DRV_MSG_CODE_SET_MF_BW_ACK              0xe1000000
@@ -1372,6 +1375,8 @@ struct drv_func_mb {
 
 	#define FW_MSG_CODE_EEE_RESULS_ACK              0xda100000
 
+	#define FW_MSG_CODE_RMMOD_ACK                   0xdb100000
+
 	#define FW_MSG_CODE_SET_MF_BW_SENT              0xe0000000
 	#define FW_MSG_CODE_SET_MF_BW_DONE              0xe1000000
 
drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +9 -0
index 78b7195..339c388 100644
@@ -10368,6 +10368,10 @@ static void bnx2x_get_common_hwinfo(struct bnx2x *bp)
 
 	bp->flags |= (val >= REQ_BC_VER_4_DCBX_ADMIN_MSG_NON_PMF) ?
 			BC_SUPPORTS_DCBX_MSG_NON_PMF : 0;
+
+	bp->flags |= (val >= REQ_BC_VER_4_RMMOD_CMD) ?
+			BC_SUPPORTS_RMMOD_CMD : 0;
+
 	boot_mode = SHMEM_RD(bp,
 			dev_info.port_feature_config[BP_PORT(bp)].mba_config) &
 			PORT_FEATURE_MBA_BOOT_AGENT_TYPE_MASK;
@@ -12824,6 +12828,11 @@ static void __bnx2x_remove(struct pci_dev *pdev,
 	bnx2x_dcbnl_update_applist(bp, true);
 #endif
 
+	if (IS_PF(bp) &&
+	    !BP_NOMCP(bp) &&
+	    (bp->flags & BC_SUPPORTS_RMMOD_CMD))
+		bnx2x_fw_command(bp, DRV_MSG_CODE_RMMOD, 0);
+
 	/* Close the interface - either directly or implicitly */
 	if (remove_netdev) {
 		unregister_netdev(dev);
Signature · 1 line
1.8.1.4
#9 lore raw

PATCH v5 5/5 bnx2x: prevent crash in shutdown flow with CNIC

In reply to Dmitry Kravkov

From: Yuval Mintz <[email protected]>

There might be a crash as during shutdown flow CNIC might try to access resources already freed by bnx2x.
Change bnx2x_close() into dev_close() in __bnx2x_remove (shutdown flow) to guarantee CNIC is notified of the device's change of status.

---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +1 -2
index 339c388..955d6cf 100644
@@ -12838,8 +12838,7 @@ static void __bnx2x_remove(struct pci_dev *pdev,
 		unregister_netdev(dev);
 	} else {
 		rtnl_lock();
-		if (netif_running(dev))
-			bnx2x_close(dev);
+		dev_close(dev);
 		rtnl_unlock();
 	}
 
Signature · 1 line
1.8.1.4
#10 lore raw
In reply to Dmitry Kravkov
Quoted 12 lines · Dmitry Kravkov

Hello Dave

Please consider applying the series of bnx2x fixes to net:
* statistics may cause FW assert
* missing fairness configuration in DCB flow
* memory leak in sriov related part
* Illegal PTE access
* Pagefault crash in shutdown flow with cnic
v1->v2
* fixed sparse error pointed by Joe Perches
* added missing signed-off from Sergei Shtylyov
v2->v3

Quoted 8 lines

* added missing signed-off from Sergei Shtylyov
* fixed formatting from Sergei Shtylyov
v3->v4
* patch 1/6: fixed declaration order
* patch 2/6 replaced with: protect flows using set_bit constraints
v4->v5
* patch 2/6: replace proprietary locking with semaphore
* droped 1/6: since adds redundant code from Benjamin Poirier

I'm confused. Wasn't "[PATCH net v4 1/6] bnx2x: properly initialize statistic counters" supposed to fix a race condition? According to earlier communication with Ariel:
In this issue a race condition at driver startup causes a second statistics query to be sent before the first one completes, resulting in a firmware assert and a stuck chip. A patch was sent upstream fixing this:
http://patchwork.ozlabs.org/patch/264810/

#11 lore raw
Quoted 8 lines

I'm confused. Wasn't "[PATCH net v4 1/6] bnx2x: properly initialize statistic counters" supposed to fix a race condition? According to earlier communication with Ariel:
In this issue a race condition at driver startup causes a second statistics query to be sent before the first one completes, resulting in a firmware assert and a stuck chip. A patch was sent upstream fixing this:
http://patchwork.ozlabs.org/patch/264810/

As I explicitly mentioned in the communique which you quoted above, the patch was sent but not yet accepted. This is precisely the upstream process - patches are being sent, reviewed and sometimes rejected and revised. If you have further questions please address them to me - the technical forum is no place for this kind of discussion. Thanks,
Ariel

#12 lore raw
Quoted 9 lines · Ariel Elior
Quoted 8 lines

I'm confused. Wasn't "[PATCH net v4 1/6] bnx2x: properly initialize statistic counters" supposed to fix a race condition? According to earlier communication with Ariel:
In this issue a race condition at driver startup causes a second statistics query to be sent before the first one completes, resulting in a firmware assert and a stuck chip. A patch was sent upstream fixing this:
http://patchwork.ozlabs.org/patch/264810/

Quoted 6 lines

As I explicitly mentioned in the communique which you quoted above, the patch was sent but not yet accepted. This is precisely the upstream process - patches are being sent, reviewed and sometimes rejected and revised. If you have further questions please address them to me - the technical forum is no place for this kind of discussion. Thanks,
Ariel

It's a technical question about a patch which was sent upstream. Where should the discussion happen if not upstream?

Let me rephrase my question:

I'm confused. I thought that "[PATCH net v4 1/6] bnx2x: properly initialize statistic counters" was meant to fix a race condition at driver startup which causes a second statistics query to be sent before the first one completes, resulting in a firmware assert and a stuck chip. Am I mistaken and there is no such race condition, or is it addressed by the other patches in this series?

#13 lore raw
Quoted 32 lines

-----Original Message-----
From: [email protected] [mailto:netdev- [email protected]] On Behalf Of Benjamin Poirier
Sent: Tuesday, August 13, 2013 5:07 PM
To: Ariel Elior
Cc: Dmitry Kravkov; [email protected]; [email protected]; Eilon Greenstein
Subject: Re: [PATCH net v5 0/5] bnx2x: fixes

Quoted 10 lines · Ariel Elior
Quoted 8 lines

I'm confused. Wasn't "[PATCH net v4 1/6] bnx2x: properly initialize statistic counters" supposed to fix a race condition? According to earlier communication with Ariel:
In this issue a race condition at driver startup causes a second statistics query to be sent before the first one completes, resulting in a firmware assert and a stuck chip. A patch was sent upstream fixing this:
http://patchwork.ozlabs.org/patch/264810/

As I explicitly mentioned in the communique which you quoted above,

the patch was sent but not yet accepted.

Quoted 1 line

This is precisely the upstream process - patches are being sent, reviewed

and sometimes rejected and revised.

Quoted 1 line

If you have further questions please address them to me - the technical

forum is no place for this kind of discussion.

Quoted 3 lines

Thanks,
Ariel

It's a technical question about a patch which was sent upstream. Where should the discussion happen if not upstream?

Quoted 8 lines

Let me rephrase my question:

I'm confused. I thought that "[PATCH net v4 1/6] bnx2x: properly initialize statistic counters" was meant to fix a race condition at driver startup which causes a second statistics query to be sent before the first one completes, resulting in a firmware assert and a stuck chip. Am I mistaken and there is no such race condition, or is it addressed by the other patches in this series?

There is such condition, but it's not caused by wrongly initialized counted - the initialization done in a correct way. Patch "[PATCH net v5 1/5] bnx2x: protect different statistics flows" - prevents from two outstanding queries to be sent simultaneously.

Thanks
Dmitry

#14 lore raw

On 2013/08/13 14:16, Dmitry Kravkov wrote:
[...]

Quoted 9 lines
Quoted 9 lines

Let me rephrase my question:

I'm confused. I thought that "[PATCH net v4 1/6] bnx2x: properly initialize statistic counters" was meant to fix a race condition at driver startup which causes a second statistics query to be sent before the first one completes, resulting in a firmware assert and a stuck chip. Am I mistaken and there is no such race condition, or is it addressed by the other patches in this series?

Quoted 4 lines

There is such condition, but it's not caused by wrongly initialized counted - the initialization done in a correct way. Patch "[PATCH net v5 1/5] bnx2x: protect different statistics flows" - prevents from two outstanding queries to be sent simultaneously.

Thank you for the clarification.

maintainer
#15 lore raw
In reply to Dmitry Kravkov

From: "Dmitry Kravkov" <[email protected]>
Date: Tue, 13 Aug 2013 02:24:58 +0300

Quoted 6 lines

Please consider applying the series of bnx2x fixes to net:
* statistics may cause FW assert
* missing fairness configuration in DCB flow
* memory leak in sriov related part
* Illegal PTE access
* Pagefault crash in shutdown flow with cnic

Series applied, thanks.