Skip to contentSkip to conversation

ionic support for firmware upgrade

PATCH In review Shannon Nelson 2 participants 10 messages Mar 28, 2020 – Mar 30, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v2 cover ionic support for firmware upgrade

The Pensando Distributed Services Card can get firmware upgrades from the off-host centralized management suite, and can be upgraded without a host reboot or driver reload. This patchset sets up the support for fw upgrade in the Linux driver.

When the upgrade begins, the DSC first brings the link down, then stops the firmware. The driver will notice this and quiesce itself by stopping the queues and releasing DMA resources, then monitoring for firmware to start back up. When the upgrade is finished the firmware is restarted and link is brought up, and the driver rebuilds the queues and restarts traffic flow.

First we separate the Link state from the netdev state, then reorganize a few things to prepare for partial tear-down of the queues. Next we fix up the state machine so that we take the Tx and Rx queues down and back up when we get LINK_DOWN and LINK_UP events. Lastly, we add handling of the FW reset itself by tearing down the lif internals and rebuilding them with the new FW setup.

v2: This changes the design from (ab)using the full .ndo_stop and
.ndo_open routines to getting a better separation between the alloc and the init functions so that we can keep our resource allocations as long as possible.

Shannon Nelson (8):
ionic: decouple link message from netdev state
ionic: check for linkup in watchdog
ionic: move debugfs add/delete to match alloc/free
ionic: move irq request to qcq alloc
ionic: clean tx queue of unfinished requests
ionic: check for queues before deleting
ionic: disable the queues on link down
ionic: remove lifs on fw reset

 .../net/ethernet/pensando/ionic/ionic_dev.c   |  46 ++-
 .../net/ethernet/pensando/ionic/ionic_dev.h   |   1 +
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 390 +++++++++++-------
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   5 +
 .../net/ethernet/pensando/ionic/ionic_main.c  |   8 +-
 .../net/ethernet/pensando/ionic/ionic_txrx.c  |  16 +
 .../net/ethernet/pensando/ionic/ionic_txrx.h  |   1 +
 7 files changed, 322 insertions(+), 145 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH v2 1/8 ionic: decouple link message from netdev state

Rearrange the link_up/link_down messages so that we announce link up when we first notice that the link is up when the driver loads, and decouple the link_up/link_down messages from the UP and DOWN netdev state.

---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 32 +++++++++++--------
 1 file changed, 18 insertions(+), 14 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +18 -14
index 8b442eb010a2..3e9c0e9bcad2 100644
@@ -73,31 +73,35 @@ static void ionic_link_status_check(struct ionic_lif *lif)
 	u16 link_status;
 	bool link_up;
 
+	if (lif->ionic->is_mgmt_nic)
+		return;
+
 	link_status = le16_to_cpu(lif->info->status.link_status);
 	link_up = link_status == IONIC_PORT_OPER_STATUS_UP;
 
-	/* filter out the no-change cases */
-	if (link_up == netif_carrier_ok(netdev))
-		goto link_out;
-
 	if (link_up) {
-		netdev_info(netdev, "Link up - %d Gbps\n",
-			    le32_to_cpu(lif->info->status.link_speed) / 1000);
+		if (!netif_carrier_ok(netdev)) {
+			u32 link_speed;
 
-		if (test_bit(IONIC_LIF_F_UP, lif->state)) {
-			netif_tx_wake_all_queues(lif->netdev);
+			ionic_port_identify(lif->ionic);
+			link_speed = le32_to_cpu(lif->info->status.link_speed);
+			netdev_info(netdev, "Link up - %d Gbps\n",
+				    link_speed / 1000);
 			netif_carrier_on(netdev);
 		}
+
+		if (test_bit(IONIC_LIF_F_UP, lif->state))
+			netif_tx_wake_all_queues(lif->netdev);
 	} else {
-		netdev_info(netdev, "Link down\n");
+		if (netif_carrier_ok(netdev)) {
+			netdev_info(netdev, "Link down\n");
+			netif_carrier_off(netdev);
+		}
 
-		/* carrier off first to avoid watchdog timeout */
-		netif_carrier_off(netdev);
 		if (test_bit(IONIC_LIF_F_UP, lif->state))
 			netif_tx_stop_all_queues(netdev);
 	}
 
-link_out:
 	clear_bit(IONIC_LIF_F_LINK_CHECK_REQUESTED, lif->state);
 }
 
@@ -1587,8 +1591,6 @@ int ionic_open(struct net_device *netdev)
 	struct ionic_lif *lif = netdev_priv(netdev);
 	int err;
 
-	netif_carrier_off(netdev);
-
 	err = ionic_txrx_alloc(lif);
 	if (err)
 		return err;
@@ -1936,6 +1938,8 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index
 	ionic_ethtool_set_ops(netdev);
 
 	netdev->watchdog_timeo = 2 * HZ;
+	netif_carrier_off(netdev);
+
 	netdev->min_mtu = IONIC_MIN_MTU;
 	netdev->max_mtu = IONIC_MAX_MTU;
 
Signature · 1 line
2.17.1
#2 lore raw

PATCH v2 2/8 ionic: check for linkup in watchdog

In reply to Shannon Nelson

Add a link_status_check to the heartbeat watchdog.

---
 drivers/net/ethernet/pensando/ionic/ionic_dev.c | 6 +++++-
 drivers/net/ethernet/pensando/ionic/ionic_lif.c | 2 +-
 drivers/net/ethernet/pensando/ionic/ionic_lif.h | 1 +
 3 files changed, 7 insertions(+), 2 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_dev.c +5 -1
index 46107de5e6c3..f03a092f370f 100644
@@ -14,11 +14,15 @@
 static void ionic_watchdog_cb(struct timer_list *t)
 {
 	struct ionic *ionic = from_timer(ionic, t, watchdog_timer);
+	int hb;
 
 	mod_timer(&ionic->watchdog_timer,
 		  round_jiffies(jiffies + ionic->watchdog_period));
 
-	ionic_heartbeat_check(ionic);
+	hb = ionic_heartbeat_check(ionic);
+
+	if (hb >= 0 && ionic->master_lif)
+		ionic_link_status_check_request(ionic->master_lif);
 }
 
 void ionic_init_devinfo(struct ionic *ionic)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +1 -1
index 3e9c0e9bcad2..ddbff44cda89 100644
@@ -105,7 +105,7 @@ static void ionic_link_status_check(struct ionic_lif *lif)
 	clear_bit(IONIC_LIF_F_LINK_CHECK_REQUESTED, lif->state);
 }
 
-static void ionic_link_status_check_request(struct ionic_lif *lif)
+void ionic_link_status_check_request(struct ionic_lif *lif)
 {
 	struct ionic_deferred_work *work;
 
drivers/net/ethernet/pensando/ionic/ionic_lif.h +1 -0
index 7c0c6fef8c0b..8aaa7daf3842 100644
@@ -224,6 +224,7 @@ static inline u32 ionic_coal_hw_to_usec(struct ionic *ionic, u32 units)
 	return (units * div) / mult;
 }
 
+void ionic_link_status_check_request(struct ionic_lif *lif);
 int ionic_lifs_alloc(struct ionic *ionic);
 void ionic_lifs_free(struct ionic *ionic);
 void ionic_lifs_deinit(struct ionic *ionic);
Signature · 1 line
2.17.1
#3 lore raw

PATCH v2 3/8 ionic: move debugfs add/delete to match alloc/free

In reply to Shannon Nelson

Move the qcq debugfs add to the queue alloc, and likewise move the debugfs delete to the queue free. The LIF debugfs add also needs to be moved, but the del is already in the LIF free.

---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 20 ++++++++-----------
 1 file changed, 8 insertions(+), 12 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +8 -12
index ddbff44cda89..4ade43219256 100644
@@ -271,8 +271,6 @@ static void ionic_lif_qcq_deinit(struct ionic_lif *lif, struct ionic_qcq *qcq)
 	if (!qcq)
 		return;
 
-	ionic_debugfs_del_qcq(qcq);
-
 	if (!(qcq->flags & IONIC_QCQ_F_INITED))
 		return;
 
@@ -295,6 +293,8 @@ static void ionic_qcq_free(struct ionic_lif *lif, struct ionic_qcq *qcq)
 	if (!qcq)
 		return;
 
+	ionic_debugfs_del_qcq(qcq);
+
 	dma_free_coherent(dev, qcq->total_size, qcq->base, qcq->base_pa);
 	qcq->base = NULL;
 	qcq->base_pa = 0;
@@ -509,6 +509,7 @@ static int ionic_qcqs_alloc(struct ionic_lif *lif)
 			      0, lif->kern_pid, &lif->adminqcq);
 	if (err)
 		return err;
+	ionic_debugfs_add_qcq(lif, lif->adminqcq);
 
 	if (lif->ionic->nnqs_per_lif) {
 		flags = IONIC_QCQ_F_NOTIFYQ;
@@ -519,6 +520,7 @@ static int ionic_qcqs_alloc(struct ionic_lif *lif)
 				      0, lif->kern_pid, &lif->notifyqcq);
 		if (err)
 			goto err_out_free_adminqcq;
+		ionic_debugfs_add_qcq(lif, lif->notifyqcq);
 
 		/* Let the notifyq ride on the adminq interrupt */
 		ionic_link_qcq_interrupts(lif->adminqcq, lif->notifyqcq);
@@ -616,8 +618,6 @@ static int ionic_lif_txq_init(struct ionic_lif *lif, struct ionic_qcq *qcq)
 
 	qcq->flags |= IONIC_QCQ_F_INITED;
 
-	ionic_debugfs_add_qcq(lif, qcq);
-
 	return 0;
 }
 
@@ -672,8 +672,6 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq)
 
 	qcq->flags |= IONIC_QCQ_F_INITED;
 
-	ionic_debugfs_add_qcq(lif, qcq);
-
 	return 0;
 }
 
@@ -1490,6 +1488,7 @@ static int ionic_txrx_alloc(struct ionic_lif *lif)
 			goto err_out;
 
 		lif->txqcqs[i].qcq->stats = lif->txqcqs[i].stats;
+		ionic_debugfs_add_qcq(lif, lif->txqcqs[i].qcq);
 	}
 
 	flags = IONIC_QCQ_F_RX_STATS | IONIC_QCQ_F_SG | IONIC_QCQ_F_INTR;
@@ -1510,6 +1509,7 @@ static int ionic_txrx_alloc(struct ionic_lif *lif)
 				     lif->rx_coalesce_hw);
 		ionic_link_qcq_interrupts(lif->rxqcqs[i].qcq,
 					  lif->txqcqs[i].qcq);
+		ionic_debugfs_add_qcq(lif, lif->rxqcqs[i].qcq);
 	}
 
 	return 0;
@@ -1974,6 +1974,8 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index
 		goto err_out_free_netdev;
 	}
 
+	ionic_debugfs_add_lif(lif);
+
 	/* allocate queues */
 	err = ionic_qcqs_alloc(lif);
 	if (err)
@@ -2154,8 +2156,6 @@ static int ionic_lif_adminq_init(struct ionic_lif *lif)
 
 	qcq->flags |= IONIC_QCQ_F_INITED;
 
-	ionic_debugfs_add_qcq(lif, qcq);
-
 	return 0;
 }
 
@@ -2203,8 +2203,6 @@ static int ionic_lif_notifyq_init(struct ionic_lif *lif)
 
 	qcq->flags |= IONIC_QCQ_F_INITED;
 
-	ionic_debugfs_add_qcq(lif, qcq);
-
 	return 0;
 }
 
@@ -2258,8 +2256,6 @@ static int ionic_lif_init(struct ionic_lif *lif)
 	int dbpage_num;
 	int err;
 
-	ionic_debugfs_add_lif(lif);
-
 	mutex_lock(&lif->ionic->dev_cmd_lock);
 	ionic_dev_cmd_lif_init(idev, lif->index, lif->info_pa);
 	err = ionic_dev_cmd_wait(lif->ionic, DEVCMD_TIMEOUT);
Signature · 1 line
2.17.1
#4 lore raw

PATCH v2 4/8 ionic: move irq request to qcq alloc

In reply to Shannon Nelson

Move the irq request and free out of the qcq_init and deinit and into the alloc and free routines where they belong for better resource management.

---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 41 +++++++++----------
 1 file changed, 19 insertions(+), 22 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +19 -22
index 4ade43219256..b3f568356824 100644
@@ -266,7 +266,6 @@ static void ionic_lif_quiesce(struct ionic_lif *lif)
 static void ionic_lif_qcq_deinit(struct ionic_lif *lif, struct ionic_qcq *qcq)
 {
 	struct ionic_dev *idev = &lif->ionic->idev;
-	struct device *dev = lif->ionic->dev;
 
 	if (!qcq)
 		return;
@@ -277,10 +276,7 @@ static void ionic_lif_qcq_deinit(struct ionic_lif *lif, struct ionic_qcq *qcq)
 	if (qcq->flags & IONIC_QCQ_F_INTR) {
 		ionic_intr_mask(idev->intr_ctrl, qcq->intr.index,
 				IONIC_INTR_MASK_SET);
-		irq_set_affinity_hint(qcq->intr.vector, NULL);
-		devm_free_irq(dev, qcq->intr.vector, &qcq->napi);
 		netif_napi_del(&qcq->napi);
-		qcq->intr.vector = 0;
 	}
 
 	qcq->flags &= ~IONIC_QCQ_F_INITED;
@@ -299,8 +295,12 @@ static void ionic_qcq_free(struct ionic_lif *lif, struct ionic_qcq *qcq)
 	qcq->base = NULL;
 	qcq->base_pa = 0;
 
-	if (qcq->flags & IONIC_QCQ_F_INTR)
+	if (qcq->flags & IONIC_QCQ_F_INTR) {
+		irq_set_affinity_hint(qcq->intr.vector, NULL);
+		devm_free_irq(dev, qcq->intr.vector, &qcq->napi);
+		qcq->intr.vector = 0;
 		ionic_intr_free(lif, qcq->intr.index);
+	}
 
 	devm_kfree(dev, qcq->cq.info);
 	qcq->cq.info = NULL;
@@ -432,6 +432,12 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type,
 		ionic_intr_mask_assert(idev->intr_ctrl, new->intr.index,
 				       IONIC_INTR_MASK_SET);
 
+		err = ionic_request_irq(lif, new);
+		if (err) {
+			netdev_warn(lif->netdev, "irq request failed %d\n", err);
+			goto err_out_free_intr;
+		}
+
 		new->intr.cpu = cpumask_local_spread(new->intr.index,
 						     dev_to_node(dev));
 		if (new->intr.cpu != -1)
@@ -446,13 +452,13 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type,
 	if (!new->cq.info) {
 		netdev_err(lif->netdev, "Cannot allocate completion queue info\n");
 		err = -ENOMEM;
-		goto err_out_free_intr;
+		goto err_out_free_irq;
 	}
 
 	err = ionic_cq_init(lif, &new->cq, &new->intr, num_descs, cq_desc_size);
 	if (err) {
 		netdev_err(lif->netdev, "Cannot initialize completion queue\n");
-		goto err_out_free_intr;
+		goto err_out_free_irq;
 	}
 
 	new->base = dma_alloc_coherent(dev, total_size, &new->base_pa,
@@ -460,7 +466,7 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type,
 	if (!new->base) {
 		netdev_err(lif->netdev, "Cannot allocate queue DMA memory\n");
 		err = -ENOMEM;
-		goto err_out_free_intr;
+		goto err_out_free_irq;
 	}
 
 	new->total_size = total_size;
@@ -486,8 +492,12 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type,
 
 	return 0;
 
+err_out_free_irq:
+	if (flags & IONIC_QCQ_F_INTR)
+		devm_free_irq(dev, new->intr.vector, &new->napi);
 err_out_free_intr:
-	ionic_intr_free(lif, new->intr.index);
+	if (flags & IONIC_QCQ_F_INTR)
+		ionic_intr_free(lif, new->intr.index);
 err_out:
 	dev_err(dev, "qcq alloc of %s%d failed %d\n", name, index, err);
 	return err;
@@ -664,12 +674,6 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq)
 	netif_napi_add(lif->netdev, &qcq->napi, ionic_rx_napi,
 		       NAPI_POLL_WEIGHT);
 
-	err = ionic_request_irq(lif, qcq);
-	if (err) {
-		netif_napi_del(&qcq->napi);
-		return err;
-	}
-
 	qcq->flags |= IONIC_QCQ_F_INITED;
 
 	return 0;
@@ -2141,13 +2145,6 @@ static int ionic_lif_adminq_init(struct ionic_lif *lif)
 	netif_napi_add(lif->netdev, &qcq->napi, ionic_adminq_napi,
 		       NAPI_POLL_WEIGHT);
 
-	err = ionic_request_irq(lif, qcq);
-	if (err) {
-		netdev_warn(lif->netdev, "adminq irq request failed %d\n", err);
-		netif_napi_del(&qcq->napi);
-		return err;
-	}
-
 	napi_enable(&qcq->napi);
 
 	if (qcq->flags & IONIC_QCQ_F_INTR)
Signature · 1 line
2.17.1
#5 lore raw

PATCH v2 5/8 ionic: clean tx queue of unfinished requests

In reply to Shannon Nelson

Clean out tx requests that didn't get finished before shutting down the queue.

---
 drivers/net/ethernet/pensando/ionic/ionic_lif.c  |  1 +
 drivers/net/ethernet/pensando/ionic/ionic_txrx.c | 16 ++++++++++++++++
 drivers/net/ethernet/pensando/ionic/ionic_txrx.h |  1 +
 3 files changed, 18 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +1 -0
index b3f568356824..2804690657fd 100644
@@ -1454,6 +1454,7 @@ static void ionic_txrx_deinit(struct ionic_lif *lif)
 	for (i = 0; i < lif->nxqs; i++) {
 		ionic_lif_qcq_deinit(lif, lif->txqcqs[i].qcq);
 		ionic_tx_flush(&lif->txqcqs[i].qcq->cq);
+		ionic_tx_empty(&lif->txqcqs[i].qcq->q);
 
 		ionic_lif_qcq_deinit(lif, lif->rxqcqs[i].qcq);
 		ionic_rx_flush(&lif->rxqcqs[i].qcq->cq);
drivers/net/ethernet/pensando/ionic/ionic_txrx.c +16 -0
index 15ff633e81ba..d233b6e77b1e 100644
@@ -593,6 +593,22 @@ void ionic_tx_flush(struct ionic_cq *cq)
 				   work_done, 0);
 }
 
+void ionic_tx_empty(struct ionic_queue *q)
+{
+	struct ionic_desc_info *desc_info;
+	int done = 0;
+
+	/* walk the not completed tx entries, if any */
+	while (q->head != q->tail) {
+		desc_info = q->tail;
+		q->tail = desc_info->next;
+		ionic_tx_clean(q, desc_info, NULL, desc_info->cb_arg);
+		desc_info->cb = NULL;
+		desc_info->cb_arg = NULL;
+		done++;
+	}
+}
+
 static int ionic_tx_tcp_inner_pseudo_csum(struct sk_buff *skb)
 {
 	int err;
drivers/net/ethernet/pensando/ionic/ionic_txrx.h +1 -0
index 53775c62c85a..71973e3c35a6 100644
@@ -9,6 +9,7 @@ void ionic_tx_flush(struct ionic_cq *cq);
 
 void ionic_rx_fill(struct ionic_queue *q);
 void ionic_rx_empty(struct ionic_queue *q);
+void ionic_tx_empty(struct ionic_queue *q);
 int ionic_rx_napi(struct napi_struct *napi, int budget);
 netdev_tx_t ionic_start_xmit(struct sk_buff *skb, struct net_device *netdev);
 
Signature · 1 line
2.17.1
#6 lore raw

PATCH v2 6/8 ionic: check for queues before deleting

In reply to Shannon Nelson

Make sure the queue structures exist before trying to delete them. This addresses a couple of error recovery issues.

---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 57 ++++++++++++-------
 1 file changed, 38 insertions(+), 19 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +38 -19
index 2804690657fd..e2281542644b 100644
@@ -1437,13 +1437,20 @@ static void ionic_txrx_disable(struct ionic_lif *lif)
 	unsigned int i;
 	int err;
 
-	for (i = 0; i < lif->nxqs; i++) {
-		err = ionic_qcq_disable(lif->txqcqs[i].qcq);
-		if (err == -ETIMEDOUT)
-			break;
-		err = ionic_qcq_disable(lif->rxqcqs[i].qcq);
-		if (err == -ETIMEDOUT)
-			break;
+	if (lif->txqcqs) {
+		for (i = 0; i < lif->nxqs; i++) {
+			err = ionic_qcq_disable(lif->txqcqs[i].qcq);
+			if (err == -ETIMEDOUT)
+				break;
+		}
+	}
+
+	if (lif->rxqcqs) {
+		for (i = 0; i < lif->nxqs; i++) {
+			err = ionic_qcq_disable(lif->rxqcqs[i].qcq);
+			if (err == -ETIMEDOUT)
+				break;
+		}
 	}
 }
 
@@ -1451,14 +1458,20 @@ static void ionic_txrx_deinit(struct ionic_lif *lif)
 {
 	unsigned int i;
 
-	for (i = 0; i < lif->nxqs; i++) {
-		ionic_lif_qcq_deinit(lif, lif->txqcqs[i].qcq);
-		ionic_tx_flush(&lif->txqcqs[i].qcq->cq);
-		ionic_tx_empty(&lif->txqcqs[i].qcq->q);
+	if (lif->txqcqs) {
+		for (i = 0; i < lif->nxqs; i++) {
+			ionic_lif_qcq_deinit(lif, lif->txqcqs[i].qcq);
+			ionic_tx_flush(&lif->txqcqs[i].qcq->cq);
+			ionic_tx_empty(&lif->txqcqs[i].qcq->q);
+		}
+	}
 
-		ionic_lif_qcq_deinit(lif, lif->rxqcqs[i].qcq);
-		ionic_rx_flush(&lif->rxqcqs[i].qcq->cq);
-		ionic_rx_empty(&lif->rxqcqs[i].qcq->q);
+	if (lif->rxqcqs) {
+		for (i = 0; i < lif->nxqs; i++) {
+			ionic_lif_qcq_deinit(lif, lif->rxqcqs[i].qcq);
+			ionic_rx_flush(&lif->rxqcqs[i].qcq->cq);
+			ionic_rx_empty(&lif->rxqcqs[i].qcq->q);
+		}
 	}
 }
 
@@ -1466,12 +1479,18 @@ static void ionic_txrx_free(struct ionic_lif *lif)
 {
 	unsigned int i;
 
-	for (i = 0; i < lif->nxqs; i++) {
-		ionic_qcq_free(lif, lif->txqcqs[i].qcq);
-		lif->txqcqs[i].qcq = NULL;
+	if (lif->txqcqs) {
+		for (i = 0; i < lif->nxqs; i++) {
+			ionic_qcq_free(lif, lif->txqcqs[i].qcq);
+			lif->txqcqs[i].qcq = NULL;
+		}
+	}
 
-		ionic_qcq_free(lif, lif->rxqcqs[i].qcq);
-		lif->rxqcqs[i].qcq = NULL;
+	if (lif->rxqcqs) {
+		for (i = 0; i < lif->nxqs; i++) {
+			ionic_qcq_free(lif, lif->rxqcqs[i].qcq);
+			lif->rxqcqs[i].qcq = NULL;
+		}
 	}
 }
 
Signature · 1 line
2.17.1
#7 lore raw

PATCH v2 7/8 ionic: disable the queues on link down

In reply to Shannon Nelson

When the link goes down, we need to disable the queues on the NIC in addition to stopping the netdev stack. This lets the FW know that the driver has stopped queue activity, and then the FW can do internal reconfiguration work, whether actually Link related, or for other internal FW needs. To do this, we pull out the queue enable and disable from ionic_open() and ionic_stop() so they can be used by other routines.

To help keep things sane, we swap the queue enables so that the rx queue and its napi are enabled before the tx queue which rides on the rx queues napi.

We also drop the ionic_lif_quiesce() as it doesn't do anything more than what the queue disable has already taken care of.

---
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 117 +++++++++---------
 1 file changed, 61 insertions(+), 56 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +61 -56
index e2281542644b..8584a7c51446 100644
@@ -22,6 +22,9 @@ static int ionic_lif_addr_add(struct ionic_lif *lif, const u8 *addr);
 static int ionic_lif_addr_del(struct ionic_lif *lif, const u8 *addr);
 static void ionic_link_status_check(struct ionic_lif *lif);
 
+static int ionic_start_queues(struct ionic_lif *lif);
+static void ionic_stop_queues(struct ionic_lif *lif);
+
 static void ionic_lif_deferred_work(struct work_struct *work)
 {
 	struct ionic_lif *lif = container_of(work, struct ionic_lif, deferred.work);
@@ -73,6 +76,9 @@ static void ionic_link_status_check(struct ionic_lif *lif)
 	u16 link_status;
 	bool link_up;
 
+	if (!test_bit(IONIC_LIF_F_LINK_CHECK_REQUESTED, lif->state))
+		return;
+
 	if (lif->ionic->is_mgmt_nic)
 		return;
 
@@ -90,16 +96,16 @@ static void ionic_link_status_check(struct ionic_lif *lif)
 			netif_carrier_on(netdev);
 		}
 
-		if (test_bit(IONIC_LIF_F_UP, lif->state))
-			netif_tx_wake_all_queues(lif->netdev);
+		if (netif_running(lif->netdev))
+			ionic_start_queues(lif);
 	} else {
 		if (netif_carrier_ok(netdev)) {
 			netdev_info(netdev, "Link down\n");
 			netif_carrier_off(netdev);
 		}
 
-		if (test_bit(IONIC_LIF_F_UP, lif->state))
-			netif_tx_stop_all_queues(netdev);
+		if (netif_running(lif->netdev))
+			ionic_stop_queues(lif);
 	}
 
 	clear_bit(IONIC_LIF_F_LINK_CHECK_REQUESTED, lif->state);
@@ -248,21 +254,6 @@ static int ionic_qcq_disable(struct ionic_qcq *qcq)
 	return ionic_adminq_post_wait(lif, &ctx);
 }
 
-static void ionic_lif_quiesce(struct ionic_lif *lif)
-{
-	struct ionic_admin_ctx ctx = {
-		.work = COMPLETION_INITIALIZER_ONSTACK(ctx.work),
-		.cmd.lif_setattr = {
-			.opcode = IONIC_CMD_LIF_SETATTR,
-			.attr = IONIC_LIF_ATTR_STATE,
-			.index = lif->index,
-			.state = IONIC_LIF_DISABLE
-		},
-	};
-
-	ionic_adminq_post_wait(lif, &ctx);
-}
-
 static void ionic_lif_qcq_deinit(struct ionic_lif *lif, struct ionic_qcq *qcq)
 {
 	struct ionic_dev *idev = &lif->ionic->idev;
@@ -615,6 +606,10 @@ static int ionic_lif_txq_init(struct ionic_lif *lif, struct ionic_qcq *qcq)
 	dev_dbg(dev, "txq_init.ring_base 0x%llx\n", ctx.cmd.q_init.ring_base);
 	dev_dbg(dev, "txq_init.ring_size %d\n", ctx.cmd.q_init.ring_size);
 
+	q->tail = q->info;
+	q->head = q->tail;
+	cq->tail = cq->info;
+
 	err = ionic_adminq_post_wait(lif, &ctx);
 	if (err)
 		return err;
@@ -660,6 +655,10 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq)
 	dev_dbg(dev, "rxq_init.ring_base 0x%llx\n", ctx.cmd.q_init.ring_base);
 	dev_dbg(dev, "rxq_init.ring_size %d\n", ctx.cmd.q_init.ring_size);
 
+	q->tail = q->info;
+	q->head = q->tail;
+	cq->tail = cq->info;
+
 	err = ionic_adminq_post_wait(lif, &ctx);
 	if (err)
 		return err;
@@ -1473,6 +1472,7 @@ static void ionic_txrx_deinit(struct ionic_lif *lif)
 			ionic_rx_empty(&lif->rxqcqs[i].qcq->q);
 		}
 	}
+	lif->rx_mode = 0;
 }
 
 static void ionic_txrx_free(struct ionic_lif *lif)
@@ -1582,15 +1582,15 @@ static int ionic_txrx_enable(struct ionic_lif *lif)
 	int i, err;
 
 	for (i = 0; i < lif->nxqs; i++) {
-		err = ionic_qcq_enable(lif->txqcqs[i].qcq);
+		ionic_rx_fill(&lif->rxqcqs[i].qcq->q);
+		err = ionic_qcq_enable(lif->rxqcqs[i].qcq);
 		if (err)
 			goto err_out;
 
-		ionic_rx_fill(&lif->rxqcqs[i].qcq->q);
-		err = ionic_qcq_enable(lif->rxqcqs[i].qcq);
+		err = ionic_qcq_enable(lif->txqcqs[i].qcq);
 		if (err) {
 			if (err != -ETIMEDOUT)
-				ionic_qcq_disable(lif->txqcqs[i].qcq);
+				ionic_qcq_disable(lif->rxqcqs[i].qcq);
 			goto err_out;
 		}
 	}
@@ -1599,10 +1599,10 @@ static int ionic_txrx_enable(struct ionic_lif *lif)
 
 err_out:
 	while (i--) {
-		err = ionic_qcq_disable(lif->rxqcqs[i].qcq);
+		err = ionic_qcq_disable(lif->txqcqs[i].qcq);
 		if (err == -ETIMEDOUT)
 			break;
-		err = ionic_qcq_disable(lif->txqcqs[i].qcq);
+		err = ionic_qcq_disable(lif->rxqcqs[i].qcq);
 		if (err == -ETIMEDOUT)
 			break;
 	}
@@ -1610,6 +1610,23 @@ static int ionic_txrx_enable(struct ionic_lif *lif)
 	return err;
 }
 
+static int ionic_start_queues(struct ionic_lif *lif)
+{
+	int err;
+
+	if (test_and_set_bit(IONIC_LIF_F_UP, lif->state))
+		return 0;
+
+	err = ionic_txrx_enable(lif);
+	if (err) {
+		clear_bit(IONIC_LIF_F_UP, lif->state);
+		return err;
+	}
+	netif_tx_wake_all_queues(lif->netdev);
+
+	return 0;
+}
+
 int ionic_open(struct net_device *netdev)
 {
 	struct ionic_lif *lif = netdev_priv(netdev);
@@ -1621,54 +1638,42 @@ int ionic_open(struct net_device *netdev)
 
 	err = ionic_txrx_init(lif);
 	if (err)
-		goto err_txrx_free;
-
-	err = ionic_txrx_enable(lif);
-	if (err)
-		goto err_txrx_deinit;
-
-	netif_set_real_num_tx_queues(netdev, lif->nxqs);
-	netif_set_real_num_rx_queues(netdev, lif->nxqs);
-
-	set_bit(IONIC_LIF_F_UP, lif->state);
+		goto err_out;
 
-	ionic_link_status_check_request(lif);
-	if (netif_carrier_ok(netdev))
-		netif_tx_wake_all_queues(netdev);
+	/* don't start the queues until we have link */
+	if (netif_carrier_ok(netdev)) {
+		err = ionic_start_queues(lif);
+		if (err)
+			goto err_txrx_deinit;
+	}
 
 	return 0;
 
 err_txrx_deinit:
 	ionic_txrx_deinit(lif);
-err_txrx_free:
+err_out:
 	ionic_txrx_free(lif);
 	return err;
 }
 
-int ionic_stop(struct net_device *netdev)
+static void ionic_stop_queues(struct ionic_lif *lif)
 {
-	struct ionic_lif *lif = netdev_priv(netdev);
-	int err = 0;
+	if (!test_and_clear_bit(IONIC_LIF_F_UP, lif->state))
+		return;
 
-	if (!test_bit(IONIC_LIF_F_UP, lif->state)) {
-		dev_dbg(lif->ionic->dev, "%s: %s state=DOWN\n",
-			__func__, lif->name);
-		return 0;
-	}
-	dev_dbg(lif->ionic->dev, "%s: %s state=UP\n", __func__, lif->name);
-	clear_bit(IONIC_LIF_F_UP, lif->state);
+	ionic_txrx_disable(lif);
+	netif_tx_disable(lif->netdev);
+}
 
-	/* carrier off before disabling queues to avoid watchdog timeout */
-	netif_carrier_off(netdev);
-	netif_tx_stop_all_queues(netdev);
-	netif_tx_disable(netdev);
+int ionic_stop(struct net_device *netdev)
+{
+	struct ionic_lif *lif = netdev_priv(netdev);
 
-	ionic_txrx_disable(lif);
-	ionic_lif_quiesce(lif);
+	ionic_stop_queues(lif);
 	ionic_txrx_deinit(lif);
 	ionic_txrx_free(lif);
 
-	return err;
+	return 0;
 }
 
 static int ionic_get_vf_config(struct net_device *netdev,
Signature · 1 line
2.17.1
#8 lore raw

PATCH v2 8/8 ionic: remove lifs on fw reset

In reply to Shannon Nelson

When the FW RESET event comes to the driver from the firmware, or the fw_status goes to 0 (stopped) or to 0xff (no PCI connection), then shut down the driver activity. This event signals a FW upgrade where we need to quiesce all operations and wait for the FW to restart. The FW will continue the update process once it sees all the LIFs are reset. When the update process is done it will set the fw_status back to RUNNING. Meanwhile, the heartbeat check continues and when the fw_status is seen as set to running we can restart the driver operations.

---
 .../net/ethernet/pensando/ionic/ionic_dev.c   |  40 +++++-
 .../net/ethernet/pensando/ionic/ionic_dev.h   |   1 +
 .../net/ethernet/pensando/ionic/ionic_lif.c   | 130 +++++++++++++++---
 .../net/ethernet/pensando/ionic/ionic_lif.h   |   4 +
 .../net/ethernet/pensando/ionic/ionic_main.c  |   8 +-
 5 files changed, 158 insertions(+), 25 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_dev.c +38 -2
index f03a092f370f..f4ae40ae1e53 100644
@@ -86,6 +86,7 @@ int ionic_dev_setup(struct ionic *ionic)
 		return -EFAULT;
 	}
 
+	idev->last_fw_status = 0xff;
 	timer_setup(&ionic->watchdog_timer, ionic_watchdog_cb, 0);
 	ionic->watchdog_period = IONIC_WATCHDOG_SECS * HZ;
 	mod_timer(&ionic->watchdog_timer,
@@ -119,8 +120,43 @@ int ionic_heartbeat_check(struct ionic *ionic)
 	 * fw_status != 0xff (bad PCI read)
 	 */
 	fw_status = ioread8(&idev->dev_info_regs->fw_status);
-	if (fw_status == 0xff ||
-	    !(fw_status & IONIC_FW_STS_F_RUNNING))
+	if (fw_status != 0xff)
+		fw_status &= IONIC_FW_STS_F_RUNNING;  /* use only the run bit */
+
+	/* is this a transition? */
+	if (fw_status != idev->last_fw_status &&
+	    idev->last_fw_status != 0xff) {
+		struct ionic_lif *lif = ionic->master_lif;
+		bool trigger = false;
+
+		if (!fw_status || fw_status == 0xff) {
+			dev_info(ionic->dev, "FW stopped %u\n", fw_status);
+			if (lif && !test_bit(IONIC_LIF_F_FW_RESET, lif->state))
+				trigger = true;
+		} else {
+			dev_info(ionic->dev, "FW running %u\n", fw_status);
+			if (lif && test_bit(IONIC_LIF_F_FW_RESET, lif->state))
+				trigger = true;
+		}
+
+		if (trigger) {
+			struct ionic_deferred_work *work;
+
+			work = kzalloc(sizeof(*work), GFP_ATOMIC);
+			if (!work) {
+				dev_err(ionic->dev, "%s OOM\n", __func__);
+			} else {
+				work->type = IONIC_DW_TYPE_LIF_RESET;
+				if (fw_status & IONIC_FW_STS_F_RUNNING &&
+				    fw_status != 0xff)
+					work->fw_status = 1;
+				ionic_lif_deferred_enqueue(&lif->deferred, work);
+			}
+		}
+	}
+	idev->last_fw_status = fw_status;
+
+	if (!fw_status || fw_status == 0xff)
 		return -ENXIO;
 
 	/* early FW has no heartbeat, else FW will return non-zero */
drivers/net/ethernet/pensando/ionic/ionic_dev.h +1 -0
index 7838e342c4fd..587398b01997 100644
@@ -132,6 +132,7 @@ struct ionic_dev {
 
 	unsigned long last_hb_time;
 	u32 last_hb;
+	u8 last_fw_status;
 
 	u64 __iomem *db_pages;
 	dma_addr_t phy_db_pages;
drivers/net/ethernet/pensando/ionic/ionic_lif.c +110 -20

216 lines not on this page · raw message

drivers/net/ethernet/pensando/ionic/ionic_lif.h +4 -0
index 8aaa7daf3842..5d4ffda5c05f 100644
@@ -98,6 +98,7 @@ struct ionic_deferred_work {
 	union {
 		unsigned int rx_mode;
 		u8 addr[ETH_ALEN];
+		u8 fw_status;
 	};
 };
 
@@ -126,6 +127,7 @@ enum ionic_lif_state_flags {
 	IONIC_LIF_F_UP,
 	IONIC_LIF_F_LINK_CHECK_REQUESTED,
 	IONIC_LIF_F_QUEUE_RESET,
+	IONIC_LIF_F_FW_RESET,
 
 	/* leave this as last */
 	IONIC_LIF_F_STATE_SIZE
@@ -225,6 +227,8 @@ static inline u32 ionic_coal_hw_to_usec(struct ionic *ionic, u32 units)
 }
 
 void ionic_link_status_check_request(struct ionic_lif *lif);
+void ionic_lif_deferred_enqueue(struct ionic_deferred *def,
+				struct ionic_deferred_work *work);
 int ionic_lifs_alloc(struct ionic *ionic);
 void ionic_lifs_free(struct ionic *ionic);
 void ionic_lifs_deinit(struct ionic *ionic);
drivers/net/ethernet/pensando/ionic/ionic_main.c +5 -3
index c16dbbe54bf7..588c62e9add7 100644
@@ -286,9 +286,11 @@ int ionic_adminq_post_wait(struct ionic_lif *lif, struct ionic_admin_ctx *ctx)
 
 	err = ionic_adminq_post(lif, ctx);
 	if (err) {
-		name = ionic_opcode_to_str(ctx->cmd.cmd.opcode);
-		netdev_err(netdev, "Posting of %s (%d) failed: %d\n",
-			   name, ctx->cmd.cmd.opcode, err);
+		if (!test_bit(IONIC_LIF_F_FW_RESET, lif->state)) {
+			name = ionic_opcode_to_str(ctx->cmd.cmd.opcode);
+			netdev_err(netdev, "Posting of %s (%d) failed: %d\n",
+				   name, ctx->cmd.cmd.opcode, err);
+		}
 		return err;
 	}
 
Signature · 1 line
2.17.1
maintainer
#9 lore raw
In reply to Shannon Nelson

From: Shannon Nelson <[email protected]>
Date: Fri, 27 Mar 2020 20:14:40 -0700

Quoted 19 lines

The Pensando Distributed Services Card can get firmware upgrades from the off-host centralized management suite, and can be upgraded without a host reboot or driver reload. This patchset sets up the support for fw upgrade in the Linux driver.

When the upgrade begins, the DSC first brings the link down, then stops the firmware. The driver will notice this and quiesce itself by stopping the queues and releasing DMA resources, then monitoring for firmware to start back up. When the upgrade is finished the firmware is restarted and link is brought up, and the driver rebuilds the queues and restarts traffic flow.

First we separate the Link state from the netdev state, then reorganize a few things to prepare for partial tear-down of the queues. Next we fix up the state machine so that we take the Tx and Rx queues down and back up when we get LINK_DOWN and LINK_UP events. Lastly, we add handling of the FW reset itself by tearing down the lif internals and rebuilding them with the new FW setup.

Quoted 4 lines

v2: This changes the design from (ab)using the full .ndo_stop and
.ndo_open routines to getting a better separation between the alloc and the init functions so that we can keep our resource allocations as long as possible.

Series applied, thank you.