ionic error recovery fixes
Subsystems (3)
cover ionic error recovery fixes
These are a few little patches to make error recovery a little more safe and successful.
Shannon Nelson (6):
ionic: add timeout error checking for queue disable
ionic: leave dev cmd request contents alone on FW timeout
ionic: only save good lif dentry
ionic: ignore eexist on rx filter add
ionic: clean irq affinity on queue deinit
ionic: check for NULL structs on teardown
.../ethernet/pensando/ionic/ionic_debugfs.c | 8 ++- .../net/ethernet/pensando/ionic/ionic_lif.c | 52 ++++++++++++------- .../net/ethernet/pensando/ionic/ionic_main.c | 12 ++++- 3 files changed, 50 insertions(+), 22 deletions(-)
Signature · 1 line
2.17.1
PATCH 1/6 ionic: add timeout error checking for queue disable
Short circuit the cleanup if we get a timeout error from ionic_qcq_disable() so as to not have to wait too long on shutdown when we already know the FW is not responding.
--- .../net/ethernet/pensando/ionic/ionic_lif.c | 20 ++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +15 -5
@@ -1425,10 +1425,15 @@ static void ionic_lif_rss_deinit(struct ionic_lif *lif)static void ionic_txrx_disable (struct ionic_lif *lif) {unsigned int i; +int err;for (i =0 ; i < lif->nxqs; i++) { -ionic_qcq_disable (lif->txqcqs[i].qcq); -ionic_qcq_disable (lif->rxqcqs[i].qcq); + err =ionic_qcq_disable (lif->txqcqs[i].qcq); +if (err == -ETIMEDOUT ) +break ; + err =ionic_qcq_disable (lif->rxqcqs[i].qcq); +if (err == -ETIMEDOUT ) +break ; } }
@@ -1552,7 +1557,8 @@ static int ionic_txrx_enable(struct ionic_lif *lif)ionic_rx_fill (&lif->rxqcqs[i].qcq->q); err =ionic_qcq_enable (lif->rxqcqs[i].qcq);if (err) { -ionic_qcq_disable (lif->txqcqs[i].qcq); +if (err != -ETIMEDOUT ) +ionic_qcq_disable (lif->txqcqs[i].qcq); goto err_out; } }
@@ -1561,8 +1567,12 @@ static int ionic_txrx_enable(struct ionic_lif *lif) err_out:while (i--) { -ionic_qcq_disable (lif->rxqcqs[i].qcq); -ionic_qcq_disable (lif->txqcqs[i].qcq); + err =ionic_qcq_disable (lif->rxqcqs[i].qcq); +if (err == -ETIMEDOUT ) +break ; + err =ionic_qcq_disable (lif->txqcqs[i].qcq); +if (err == -ETIMEDOUT ) +break ; }return err;
Signature · 1 line
2.17.1
PATCH 2/6 ionic: leave dev cmd request contents alone on FW timeout
In reply to Shannon NelsonIt is possible (but unlikely) that FW was busy and missed a heartbeat check but is still alive and will process the pending request, so don't clean the dev_cmd in this case. This occasionally occurs when working with a card that is supporting many devices and is trying to shut them all down at once, but still wants to see that last LIF disable request.
--- drivers/net/ethernet/pensando/ionic/ionic_main.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/net/ethernet/pensando/ionic/ionic_main.c +4 -1
@@ -360,7 +360,10 @@ int ionic_dev_cmd_wait(struct ionic *ionic, unsigned long max_seconds) done, duration / HZ, duration);if (!done && hb) { -ionic_dev_cmd_clean (ionic); +/* It is possible (but unlikely) that FW was busy and missed a +* heartbeat check but is still alive and will process this +* request, so don't clean the dev_cmd in this case. +*/ dev_warn (ionic->dev,"DEVCMD %s (%d) failed - FW halted\n" ,ionic_opcode_to_str (opcode), opcode);return -ENXIO ;
Signature · 1 line
2.17.1
PATCH 3/6 ionic: only save good lif dentry
In reply to Shannon NelsonDon't save the lif->dentry until we know we have
a good value.
--- drivers/net/ethernet/pensando/ionic/ionic_debugfs.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/net/ethernet/pensando/ionic/ionic_debugfs.c +7 -1
@@ -228,7 +228,13 @@ DEFINE_SHOW_ATTRIBUTE(netdev);void ionic_debugfs_add_lif (struct ionic_lif *lif) { - lif->dentry =debugfs_create_dir (lif->name, lif->ionic->dentry); +struct dentry *lif_dentry; + + lif_dentry =debugfs_create_dir (lif->name, lif->ionic->dentry); +if (IS_ERR_OR_NULL (lif_dentry)) +return ; + lif->dentry = lif_dentry; +debugfs_create_file ("netdev" ,0400 , lif->dentry, lif->netdev, &netdev_fops); }
Signature · 1 line
2.17.1
PATCH 4/6 ionic: ignore eexist on rx filter add
In reply to Shannon NelsonDon't worry if the rx filter add firmware request fails on EEXIST, at least we know the filter is there. Same for the delete request, at least we know it isn't there.
--- drivers/net/ethernet/pensando/ionic/ionic_lif.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +2 -2
@@ -832,7 +832,7 @@ static int ionic_lif_addr_add(struct ionic_lif *lif, const u8 *addr)memcpy (ctx.cmd.rx_filter_add.mac.addr, addr,ETH_ALEN ); err =ionic_adminq_post_wait (lif, &ctx); -if (err) +if (err && err != -EEXIST )return err;return ionic_rx_filter_save (lif,0 ,IONIC_RXQ_INDEX_ANY ,0 , &ctx);
@@ -862,7 +862,7 @@ static int ionic_lif_addr_del(struct ionic_lif *lif, const u8 *addr)spin_unlock_bh (&lif->rx_filters.lock); err =ionic_adminq_post_wait (lif, &ctx); -if (err) +if (err && err != -EEXIST )return err; netdev_dbg(lif->netdev, "rx_filter del ADDR %pM (id %d)\n", addr,
Signature · 1 line
2.17.1
Jakub Kicinski
maintainerRe: [PATCH net-next 4/6] ionic: ignore eexist on rx filter add
Quoted 6 lines · On Thu, 19 Mar 2020 19:31:51 -0700 Shannon Nelson wrote:
Don't worry if the rx filter add firmware request fails on EEXIST, at least we know the filter is there. Same for the delete request, at least we know it isn't there.
Fixes 2a654540be10 ("ionic: Add Rx filter and rx_mode ndo support")Signed-off-by Shannon Nelson <[email protected]>
Why could the filter be there? Seems like the FW shouldn't have filters the driver didn't add, could a flush/reset command help to start from clean state?
Just curious.
Re: [PATCH net-next 4/6] ionic: ignore eexist on rx filter add
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines · On Thu, 19 Mar 2020 19:31:51 -0700 Shannon Nelson wrote:
Don't worry if the rx filter add firmware request fails on EEXIST, at least we know the filter is there. Same for the delete request, at least we know it isn't there.
Quoted 7 lines
Quoted 2 lines
Fixes 2a654540be10 ("ionic: Add Rx filter and rx_mode ndo support")Signed-off-by Shannon Nelson <[email protected]>Why could the filter be there? Seems like the FW shouldn't have filters the driver didn't add, could a flush/reset command help to start from clean state?
Just curious.
Because there are use cases where the device is configured by an external centralized agent and may have already stuck the appropriate filters into the its list.
sln
Jakub Kicinski
maintainerRe: [PATCH net-next 4/6] ionic: ignore eexist on rx filter add
Quoted 8 lines · On Thu, 19 Mar 2020 22:21:35 -0700 Shannon Nelson wrote:
Quoted 7 lines · Jakub Kicinski
Quoted 6 lines · On Thu, 19 Mar 2020 19:31:51 -0700 Shannon Nelson wrote:
Don't worry if the rx filter add firmware request fails on EEXIST, at least we know the filter is there. Same for the delete request, at least we know it isn't there.
Fixes 2a654540be10 ("ionic: Add Rx filter and rx_mode ndo support")Signed-off-by Shannon Nelson <[email protected]>
Quoted 8 lines
Quoted 5 lines
Why could the filter be there? Seems like the FW shouldn't have filters the driver didn't add, could a flush/reset command help to start from clean state?
Just curious.
Because there are use cases where the device is configured by an external centralized agent and may have already stuck the appropriate filters into the its list.
Thanks, seems a little leaky for the host to be able to probe the state installed by the agent, but it does explain the need.
PATCH 5/6 ionic: clean irq affinity on queue deinit
In reply to Shannon NelsonAdd a little more cleanup when tearing down the queues.
--- drivers/net/ethernet/pensando/ionic/ionic_lif.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +2 -0
@@ -275,8 +275,10 @@ static void ionic_lif_qcq_deinit(struct ionic_lif *lif, struct ionic_qcq *qcq)if (qcq->flags &IONIC_QCQ_F_INTR ) {ionic_intr_mask (idev->intr_ctrl, qcq->intr.index,IONIC_INTR_MASK_SET ); +irq_set_affinity_hint (qcq->intr.vector,NULL );devm_free_irq (dev, qcq->intr.vector, &qcq->napi);netif_napi_del (&qcq->napi); + qcq->intr.vector =0 ; } qcq->flags &= ~IONIC_QCQ_F_INITED ;
Signature · 1 line
2.17.1
PATCH 6/6 ionic: check for NULL structs on teardown
In reply to Shannon NelsonMake sure the queue structs exist before trying to tear them down to make for safer error recovery.
--- .../net/ethernet/pensando/ionic/ionic_lif.c | 26 ++++++++++--------- .../net/ethernet/pensando/ionic/ionic_main.c | 7 ++++- 2 files changed, 20 insertions(+), 13 deletions(-)
drivers/net/ethernet/pensando/ionic/ionic_lif.c +14 -12
@@ -320,19 +320,21 @@ static void ionic_qcqs_free(struct ionic_lif *lif) lif->adminqcq =NULL ; } -for (i =0 ; i < lif->nxqs; i++) -if (lif->rxqcqs[i].stats) -devm_kfree (dev, lif->rxqcqs[i].stats); - -devm_kfree (dev, lif->rxqcqs); - lif->rxqcqs =NULL ; - -for (i =0 ; i < lif->nxqs; i++) -if (lif->txqcqs[i].stats) -devm_kfree (dev, lif->txqcqs[i].stats); +if (lif->rxqcqs) { +for (i =0 ; i < lif->nxqs; i++) +if (lif->rxqcqs[i].stats) +devm_kfree (dev, lif->rxqcqs[i].stats); +devm_kfree (dev, lif->rxqcqs); + lif->rxqcqs =NULL ; + } -devm_kfree (dev, lif->txqcqs); - lif->txqcqs =NULL ; +if (lif->txqcqs) { +for (i =0 ; i < lif->nxqs; i++) +if (lif->txqcqs[i].stats) +devm_kfree (dev, lif->txqcqs[i].stats); +devm_kfree (dev, lif->txqcqs); + lif->txqcqs =NULL ; + } } static void ionic_link_qcq_interrupts(struct ionic_qcq *src_qcq,
drivers/net/ethernet/pensando/ionic/ionic_main.c +6 -1
@@ -243,11 +243,16 @@ static void ionic_adminq_cb(struct ionic_queue *q,static int ionic_adminq_post (struct ionic_lif *lif,struct ionic_admin_ctx *ctx) { -struct ionic_queue *adminq = &lif->adminqcq->q; +struct ionic_queue *adminq;int err =0 ;WARN_ON (in_interrupt ()); +if (!lif->adminqcq) +return -EIO ; + + adminq = &lif->adminqcq->q; +spin_lock (&lif->adminq_lock);if (!ionic_q_has_space (adminq,1 )) { err = -ENOSPC ;
Signature · 1 line
2.17.1
Jakub Kicinski
maintainerQuoted 2 lines · On Thu, 19 Mar 2020 19:31:47 -0700 Shannon Nelson wrote:
These are a few little patches to make error recovery a little more safe and successful.
Patches looks good to me, FWIW. Thanks for dropping the controversial one. I think this should have been v2 since we seen most if not all of these.
I'm not sure why most of them have a Fixes tag, though. The AUTOSEL bot is quite likely to pull them into the stable trees once they land. Is this some intentional strategy on your part?
Quoted 6 lines · Jakub Kicinski
Quoted 2 lines · On Thu, 19 Mar 2020 19:31:47 -0700 Shannon Nelson wrote:
These are a few little patches to make error recovery a little more safe and successful.
Patches looks good to me, FWIW. Thanks for dropping the controversial one. I think this should have been v2 since we seen most if not all of these.
I suppose v2 would have been okay, but that patchset was specifically labeled as for firmware upgrade, and these are "merely" supporting fixes. It didn't seem right to keep the firmware upgrade in the name, yet it didn't seem right to call something with a new name 'v2'. I suppose it would have been good to call out in the cover letter that these were refugee patches from a previous patchset.
Quoted 3 lines
I'm not sure why most of them have a Fixes tag, though. The AUTOSEL bot is quite likely to pull them into the stable trees once they land. Is this some intentional strategy on your part?
Because in my last patchset I had similar types of fixes without the Fixes tag and got reminded that I should have the tag. These do fix "issues in a previous commit" as suggested in the submitting-patches.rst, so it seemed appropriate. They aren't high priority issues, else I would have targeted them at net rather than net-next. If accepted, they'll get there soon enough.
Meanwhile, thanks for the review time :-)
Cheers,
sln
David Miller
maintainerFrom: Shannon Nelson <[email protected]>
Date: Thu, 19 Mar 2020 19:31:47 -0700
Quoted 2 lines
These are a few little patches to make error recovery a little more safe and successful.
Series applied, thank you.