Skip to contentSkip to conversation

hinic: BugFixes

PATCH In review Luo bin 3 participants 8 messages Sep 2, 2020 – Sep 3, 2020

Subsystems (3)
lore
Share & tools
lore

Luo bin

author
#0 lore raw

cover hinic: BugFixes

The bugs fixed in this patchset have been present since the following commits:
patch #1: Fixes: 00e57a6d4ad3 ("net-next/hinic: Add Tx operation") patch #2: Fixes: 5e126e7c4e52 ("hinic: add firmware update support") patch #3: Fixes: 2eed5a8b614b ("hinic: add set_channels ethtool_ops support")

Luo bin (3):
hinic: bump up the timeout of SET_FUNC_STATE cmd
hinic: bump up the timeout of UPDATE_FW cmd
hinic: fix bug of send pkts while setting channels

 .../net/ethernet/huawei/hinic/hinic_hw_mgmt.c | 20 ++++++++++++++-----
 drivers/net/ethernet/huawei/hinic/hinic_tx.c  |  5 +++++
 2 files changed, 20 insertions(+), 5 deletions(-)
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH 1/3 hinic: bump up the timeout of SET_FUNC_STATE cmd

We free memory regardless of the return value of SET_FUNC_STATE cmd in hinic_close function to avoid memory leak and this cmd may timeout when fw is busy with handling other cmds, so we bump up the timeout of this cmd to ensure it won't return failure.

Fixes 00e57a6d4ad3 ("net-next/hinic: Add Tx operation")
---
 .../net/ethernet/huawei/hinic/hinic_hw_mgmt.c    | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)
drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c +11 -5
index c6ce5966284c..0d56c6ceccd9 100644
@@ -47,6 +47,8 @@
 
 #define MGMT_MSG_TIMEOUT                5000
 
+#define SET_FUNC_PORT_MBOX_TIMEOUT	30000
+
 #define SET_FUNC_PORT_MGMT_TIMEOUT	25000
 
 #define mgmt_to_pfhwdev(pf_mgmt)        \
@@ -361,16 +363,20 @@ int hinic_msg_to_mgmt(struct hinic_pf_to_mgmt *pf_to_mgmt,
 		return -EINVAL;
 	}
 
-	if (cmd == HINIC_PORT_CMD_SET_FUNC_STATE)
-		timeout = SET_FUNC_PORT_MGMT_TIMEOUT;
+	if (HINIC_IS_VF(hwif)) {
+		if (cmd == HINIC_PORT_CMD_SET_FUNC_STATE)
+			timeout = SET_FUNC_PORT_MBOX_TIMEOUT;
 
-	if (HINIC_IS_VF(hwif))
 		return hinic_mbox_to_pf(pf_to_mgmt->hwdev, mod, cmd, buf_in,
-					in_size, buf_out, out_size, 0);
-	else
+					in_size, buf_out, out_size, timeout);
+	} else {
+		if (cmd == HINIC_PORT_CMD_SET_FUNC_STATE)
+			timeout = SET_FUNC_PORT_MGMT_TIMEOUT;
+
 		return msg_to_mgmt_sync(pf_to_mgmt, mod, cmd, buf_in, in_size,
 				buf_out, out_size, MGMT_DIRECT_SEND,
 				MSG_NOT_RESP, timeout);
+	}
 }
 
 static void recv_mgmt_msg_work_handler(struct work_struct *work)
Signature · 1 line
2.17.1
author
#2 lore raw

PATCH 2/3 hinic: bump up the timeout of UPDATE_FW cmd

In reply to Luo bin

Firmware erases the entire flash region which may take several seconds before flashing, so we bump up the timeout to ensure this cmd won't return failure.

Fixes 5e126e7c4e52 ("hinic: add firmware update support")
---
 drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c +4 -0
index 0d56c6ceccd9..2ebae6cb5db5 100644
@@ -51,6 +51,8 @@
 
 #define SET_FUNC_PORT_MGMT_TIMEOUT	25000
 
+#define UPDATE_FW_MGMT_TIMEOUT		20000
+
 #define mgmt_to_pfhwdev(pf_mgmt)        \
 		container_of(pf_mgmt, struct hinic_pfhwdev, pf_to_mgmt)
 
@@ -372,6 +374,8 @@ int hinic_msg_to_mgmt(struct hinic_pf_to_mgmt *pf_to_mgmt,
 	} else {
 		if (cmd == HINIC_PORT_CMD_SET_FUNC_STATE)
 			timeout = SET_FUNC_PORT_MGMT_TIMEOUT;
+		else if (cmd == HINIC_PORT_CMD_UPDATE_FW)
+			timeout = UPDATE_FW_MGMT_TIMEOUT;
 
 		return msg_to_mgmt_sync(pf_to_mgmt, mod, cmd, buf_in, in_size,
 				buf_out, out_size, MGMT_DIRECT_SEND,
Signature · 1 line
2.17.1
author
#3 lore raw

PATCH 3/3 hinic: fix bug of send pkts while setting channels

In reply to Luo bin

When calling hinic_close in hinic_set_channels, netif_carrier_off and netif_tx_disable are excuted, and TX host resources are freed after that. Core may call hinic_xmit_frame to send pkt after netif_tx_disable within a short time, so we should judge whether carrier is on before sending pkt otherwise the resources that have already been freed in hinic_close may be accessed.

Fixes 2eed5a8b614b ("hinic: add set_channels ethtool_ops support")
---
 drivers/net/ethernet/huawei/hinic/hinic_tx.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/net/ethernet/huawei/hinic/hinic_tx.c +5 -0
index a97498ee6914..a0662552a39c 100644
@@ -531,6 +531,11 @@ netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev)
 	struct hinic_txq *txq;
 	struct hinic_qp *qp;
 
+	if (unlikely(!netif_carrier_ok(netdev))) {
+		dev_kfree_skb_any(skb);
+		return NETDEV_TX_OK;
+	}
+
 	txq = &nic_dev->txqs[q_id];
 	qp = container_of(txq->sq, struct hinic_qp, sq);
 
Signature · 1 line
2.17.1
#4 lore raw

Re: [PATCH net 3/3] hinic: fix bug of send pkts while setting channels

Quoted 20 lines · Luo bin

When calling hinic_close in hinic_set_channels, netif_carrier_off and netif_tx_disable are excuted, and TX host resources are freed after that. Core may call hinic_xmit_frame to send pkt after netif_tx_disable within a short time, so we should judge whether carrier is on before sending pkt otherwise the resources that have already been freed in hinic_close may be accessed.

Fixes 2eed5a8b614b ("hinic: add set_channels ethtool_ops support")
---
 drivers/net/ethernet/huawei/hinic/hinic_tx.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/net/ethernet/huawei/hinic/hinic_tx.c +0 -0
index a97498ee6914..a0662552a39c 100644
@@ -531,6 +531,11 @@ netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev)
 	struct hinic_txq *txq;

drivers/net/ethernet/huawei/hinic/hinic_tx.c netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev) · line 541

Quoted 10 lines
drivers/net/ethernet/huawei/hinic/hinic_tx.c +5 -0
 	struct hinic_qp *qp;
 
+	if (unlikely(!netif_carrier_ok(netdev))) {
+		dev_kfree_skb_any(skb);
+		return NETDEV_TX_OK;
+	}
+
 	txq = &nic_dev->txqs[q_id];
 	qp = container_of(txq->sq, struct hinic_qp, sq);
 

Adding this kind of tests in fast path seems a big hammer to me.

See https://marc.info/?l=linux-netdev&m=159903844423389&w=2 for a similar problem.

Normally, after hinic_close() operation, no packet should be sent by core networking stack.

Trying to work around some core networking issue in each driver is a dead end.

author
#5 lore raw

Re: [PATCH net 3/3] hinic: fix bug of send pkts while setting channels

Quoted 36 lines · Eric Dumazet
Quoted 20 lines · Luo bin

When calling hinic_close in hinic_set_channels, netif_carrier_off and netif_tx_disable are excuted, and TX host resources are freed after that. Core may call hinic_xmit_frame to send pkt after netif_tx_disable within a short time, so we should judge whether carrier is on before sending pkt otherwise the resources that have already been freed in hinic_close may be accessed.

Fixes 2eed5a8b614b ("hinic: add set_channels ethtool_ops support")
---
 drivers/net/ethernet/huawei/hinic/hinic_tx.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/net/ethernet/huawei/hinic/hinic_tx.c +0 -0
index a97498ee6914..a0662552a39c 100644
@@ -531,6 +531,11 @@ netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev)
 	struct hinic_txq *txq;

drivers/net/ethernet/huawei/hinic/hinic_tx.c netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev) · line 541

Quoted 10 lines
drivers/net/ethernet/huawei/hinic/hinic_tx.c +5 -0
 	struct hinic_qp *qp;
 
+	if (unlikely(!netif_carrier_ok(netdev))) {
+		dev_kfree_skb_any(skb);
+		return NETDEV_TX_OK;
+	}
+
 	txq = &nic_dev->txqs[q_id];
 	qp = container_of(txq->sq, struct hinic_qp, sq);
 

Adding this kind of tests in fast path seems a big hammer to me.

Quoted 5 lines

See https://marc.info/?l=linux-netdev&m=159903844423389&w=2 for a similar problem.

Normally, after hinic_close() operation, no packet should be sent by core networking stack.

Trying to work around some core networking issue in each driver is a dead end.

Thanks for your review. I agree with what you said. Theoretically, core can't call ndo_start_xmit to send packet after netif_tx_disable called by hinic_close because __QUEUE_STATE_DRV_XOFF bit is set and this bit is protected by __netif_tx_lock but it does call hinic_xmit_frame after netif_tx_disable in my debug message. I'll try to figure out why and fix it. It seems like that the patch from
https://marc.info/?l=linux-netdev&m=159903844423389&w=2 can't fix this problem.

Quoted 8 lines

.

maintainer
#6 lore raw

Re: [PATCH net 3/3] hinic: fix bug of send pkts while setting channels

In reply to Luo bin

From: Luo bin <[email protected]>
Date: Wed, 2 Sep 2020 17:41:45 +0800

drivers/net/ethernet/huawei/hinic/hinic_tx.c netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev) · line 537

Quoted 8 lines
drivers/net/ethernet/huawei/hinic/hinic_tx.c +4 -0
@@ -531,6 +531,11 @@ netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev)
 	struct hinic_txq *txq;
 	struct hinic_qp *qp;
 
+	if (unlikely(!netif_carrier_ok(netdev))) {
+		dev_kfree_skb_any(skb);
+		return NETDEV_TX_OK;
+	}

As Eric said, these kinds of tests should not be placed in the fast path of the driver.

If you invoke close and the core networking still sends packets to the driver, that's a bug that needs to be fixed in the core networking.

author
#7 lore raw

Re: [PATCH net 3/3] hinic: fix bug of send pkts while setting channels

Quoted 15 lines · David Miller

From: Luo bin <[email protected]>
Date: Wed, 2 Sep 2020 17:41:45 +0800

drivers/net/ethernet/huawei/hinic/hinic_tx.c netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev) · line 537

Quoted 8 lines
drivers/net/ethernet/huawei/hinic/hinic_tx.c +4 -0
@@ -531,6 +531,11 @@ netdev_tx_t hinic_xmit_frame(struct sk_buff *skb, struct net_device *netdev)
 	struct hinic_txq *txq;
 	struct hinic_qp *qp;
 
+	if (unlikely(!netif_carrier_ok(netdev))) {
+		dev_kfree_skb_any(skb);
+		return NETDEV_TX_OK;
+	}

As Eric said, these kinds of tests should not be placed in the fast path of the driver.

Quoted 4 lines

If you invoke close and the core networking still sends packets to the driver, that's a bug that needs to be fixed in the core networking. .

Okay, I'm trying to figure out why the core networking can still call ndo_start_xmit after netif_tx_disable and solve the problem fundamentally. And I'll undo this patch temporarily.