Skip to contentSkip to conversation

iavf: correctly track whether the interface is running during resets

PATCH New Tony Nguyen 3 participants 24 messages Jun 4, 2021 – Nov 17, 2021

Subsystems (2)
lore
Share & tools
lore

Tony Nguyen

author maintainer
#0 lore raw

PATCH 1/15 iavf: correctly track whether the interface is running during resets

From: Nicholas Nunley <[email protected]>

During a hardware reset the driver needs to know if the interface is running so it can appropriately shut down and restore itself to the previous state. However, as described in commit 44b034b40621 ("i40evf:
don't rely on netif_running() outside rtnl_lock()") the driver can't simply grab rtnl_lock() in the reset path when it needs to check netif_running().

The previous fix for this was to have the driver use the __IAVF_RUNNING state to stand in for netif_running(). This turns out to be incorrect, since although __IAVF_RUNNING does tell is if the interface is running, there are other states the driver could be in, and they don't accurately indicate whether the interface is actually running or not.

Although adapter->state can't be used to reliably determine if the interface is running, adapter->vsi.state can, so use this instead.

This patch also replaces the use of netif_running() in iavf_reinit_interrupt_scheme() which was presumably overlooked earlier.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +3 -5
index 1323778f461d..bf96a9dab962 100644
@@ -1486,7 +1486,7 @@ static int iavf_reinit_interrupt_scheme(struct iavf_adapter *adapter)
 	struct net_device *netdev = adapter->netdev;
 	int err;
 
-	if (netif_running(netdev))
+	if (!test_bit(__IAVF_VSI_DOWN, adapter->vsi.state))
 		iavf_free_traffic_irqs(adapter);
 	iavf_free_misc_irq(adapter);
 	iavf_reset_interrupt_capability(adapter);
@@ -2029,7 +2029,7 @@ static void iavf_disable_vf(struct iavf_adapter *adapter)
 	 * ndo_open() returning, so we can't assume it means all our open
 	 * tasks have finished, since we're not holding the rtnl_lock here.
 	 */
-	if (adapter->state == __IAVF_RUNNING) {
+	if (!test_bit(__IAVF_VSI_DOWN, adapter->vsi.state)) {
 		set_bit(__IAVF_VSI_DOWN, adapter->vsi.state);
 		netif_carrier_off(adapter->netdev);
 		netif_tx_disable(adapter->netdev);
@@ -2169,9 +2169,7 @@ static void iavf_reset_task(struct work_struct *work)
 	 * ndo_open() returning, so we can't assume it means all our open
 	 * tasks have finished, since we're not holding the rtnl_lock here.
 	 */
-	running = ((adapter->state == __IAVF_RUNNING) ||
-		   (adapter->state == __IAVF_RESETTING));
-
+	running = !test_bit(__IAVF_VSI_DOWN, adapter->vsi.state);
 	if (running) {
 		netif_carrier_off(netdev);
 		netif_tx_stop_all_queues(netdev);
Signature · 1 line
2.20.1
author maintainer
#1 lore raw

PATCH 2/15 iavf: obtain the crit_section lock in iavf_open() immediately

From: Nicholas Nunley <[email protected]>

iavf_open() checks for IAVF_FLAG_PF_COMMS_FAILED outside of the crit_section lock so that it can return early if possible, without needing to acquire the lock. This is perfectly fine, but once the lock is actually obtained the code assumes the value hasn't changed. This is not correct, since the IAVF_FLAG_PF_COMMS_FAILED field can certainly change by the time the lock is obtained, especially if iavf_open() has to wait for it while iavf_reset_task() runs on another thread.

To avoid this, simply grab the lock before checking the value.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +6 -5
index bf96a9dab962..4c55773c6ee1 100644
@@ -3196,15 +3196,16 @@ static int iavf_open(struct net_device *netdev)
 	struct iavf_adapter *adapter = netdev_priv(netdev);
 	int err;
 
-	if (adapter->flags & IAVF_FLAG_PF_COMMS_FAILED) {
-		dev_err(&adapter->pdev->dev, "Unable to open device due to PF driver failure.\n");
-		return -EIO;
-	}
-
 	while (test_and_set_bit(__IAVF_IN_CRITICAL_TASK,
 				&adapter->crit_section))
 		usleep_range(500, 1000);
 
+	if (adapter->flags & IAVF_FLAG_PF_COMMS_FAILED) {
+		dev_err(&adapter->pdev->dev, "Unable to open device due to PF driver failure.\n");
+		err = -EIO;
+		goto err_unlock;
+	}
+
 	if (adapter->state != __IAVF_DOWN) {
 		err = -EBUSY;
 		goto err_unlock;
Signature · 1 line
2.20.1
author maintainer
#2 lore raw

PATCH 3/15 iavf: obtain crit_section lock in iavf_close() immediately

In reply to Tony Nguyen

From: Nicholas Nunley <[email protected]>

iavf_close() checks the adapter state outside of the crit_section lock so that it can quickly bail out of the function if the interface is already down. This doesn't work as intended, though, since it may be seeing a transient state if iavf_reset_task() is running on a separate thread. If this happens the driver will hit a panic later in iavf_remove() since the active resources were never unconfigured correctly.

To fix this the adapter state is checked after grabbing the crit_section lock.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +5 -3
index 4c55773c6ee1..f97d04b47292 100644
@@ -3271,13 +3271,15 @@ static int iavf_close(struct net_device *netdev)
 	struct iavf_adapter *adapter = netdev_priv(netdev);
 	int status;
 
-	if (adapter->state <= __IAVF_DOWN_PENDING)
-		return 0;
-
 	while (test_and_set_bit(__IAVF_IN_CRITICAL_TASK,
 				&adapter->crit_section))
 		usleep_range(500, 1000);
 
+	if (adapter->state <= __IAVF_DOWN_PENDING) {
+		clear_bit(__IAVF_IN_CRITICAL_TASK, &adapter->crit_section);
+		return 0;
+	}
+
 	set_bit(__IAVF_VSI_DOWN, adapter->vsi.state);
 	if (CLIENT_ENABLED(adapter))
 		adapter->flags |= IAVF_FLAG_CLIENT_NEEDS_CLOSE;
Signature · 1 line
2.20.1
author maintainer
#3 lore raw

PATCH 4/15 iavf: wrap driver state change in crit_section lock

In reply to Tony Nguyen

From: Nicholas Nunley <[email protected]>

The changes to driver state in iavf_disable_vf() should be made inside the crit_section lock, not outside of it, since the complete configuration flow needs to be seen as one atomic operation.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +1 -1
index f97d04b47292..91818ba7c8a3 100644
@@ -2071,9 +2071,9 @@ static void iavf_disable_vf(struct iavf_adapter *adapter)
 	memset(adapter->vf_res, 0, IAVF_VIRTCHNL_VF_RESOURCE_SIZE);
 	iavf_shutdown_adminq(&adapter->hw);
 	adapter->netdev->flags &= ~IFF_UP;
-	clear_bit(__IAVF_IN_CRITICAL_TASK, &adapter->crit_section);
 	adapter->flags &= ~IAVF_FLAG_RESET_PENDING;
 	adapter->state = __IAVF_DOWN;
+	clear_bit(__IAVF_IN_CRITICAL_TASK, &adapter->crit_section);
 	wake_up(&adapter->down_waitqueue);
 	dev_info(&adapter->pdev->dev, "Reset task did not complete, VF disabled\n");
 }
Signature · 1 line
2.20.1
author maintainer
#4 lore raw

PATCH 5/15 iavf: untangle any pending iavf_open() operations from iavf_close()

In reply to Tony Nguyen

From: Nicholas Nunley <[email protected]>

When the iavf interface is opened some of the steps necessary to configure the hardware require communication with the PF driver. Since these operations involve waiting for a response from the PF driver they can be time-consuming, so the iavf driver schedules them for later and proceeds with the remaining configuration. If the interface is closed immediately after it is opened then some of these operations may still be pending, although the iavf_close() routine assumes they have all completed. In rare cases this can lead to iavf_open() configuration operations completing after iavf_close(), which can mean the device interrupts and/or DMA engine are active on a disabled interface.

To fix this:
1. In iavf_close() any pending unsent operations from iavf_open() are canceled.
2. If the operation was already sent by the time iavf_close() is called, but the driver is still awaiting the response back from the PF driver, then ignore the response if it is received when the interface is down instead of handling it in the usual manner.
3. Place a lock around the handling of all PF driver responses to ensure that these can't conflict with concurrent processing of iavf_open() and iavf_close(), or the other configuration tasks. In essence change (2) above protects against unexpected responses received after iavf_close(), and this change protects against responses received during iavf_close().

---
 drivers/net/ethernet/intel/iavf/iavf_main.c     | 10 ++++++++++
 drivers/net/ethernet/intel/iavf/iavf_virtchnl.c |  3 ++-
 2 files changed, 12 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +10 -0
index 91818ba7c8a3..eda8ebb8e7b8 100644
@@ -1025,6 +1025,12 @@ void iavf_down(struct iavf_adapter *adapter)
 		adapter->aq_required |= IAVF_FLAG_AQ_DEL_FDIR_FILTER;
 		adapter->aq_required |= IAVF_FLAG_AQ_DEL_ADV_RSS_CFG;
 		adapter->aq_required |= IAVF_FLAG_AQ_DISABLE_QUEUES;
+		/* In case the queue configure or enable operations are still
+		 * pending from when the interface was opened, make sure
+		 * they're canceled here.
+		 */
+		adapter->aq_required &= ~IAVF_FLAG_AQ_ENABLE_QUEUES;
+		adapter->aq_required &= ~IAVF_FLAG_AQ_CONFIGURE_QUEUES;
 	}
 
 	mod_delayed_work(iavf_wq, &adapter->watchdog_task, 0);
@@ -2325,8 +2331,12 @@ static void iavf_adminq_task(struct work_struct *work)
 		if (ret || !v_op)
 			break; /* No event to process or error cleaning ARQ */
 
+		while (test_and_set_bit(__IAVF_IN_CRITICAL_TASK,
+					&adapter->crit_section))
+			usleep_range(500, 1000);
 		iavf_virtchnl_completion(adapter, v_op, v_ret, event.msg_buf,
 					 event.msg_len);
+		clear_bit(__IAVF_IN_CRITICAL_TASK, &adapter->crit_section);
 		if (pending != 0)
 			memset(event.msg_buf, 0, IAVF_MAX_AQ_BUF_SIZE);
 	} while (pending);
drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +2 -1
index 0eab3c43bdc5..69e479eb5534 100644
@@ -1685,7 +1685,8 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
 		break;
 	case VIRTCHNL_OP_ENABLE_QUEUES:
 		/* enable transmits */
-		iavf_irq_enable(adapter, true);
+		if (adapter->state == __IAVF_RUNNING)
+			iavf_irq_enable(adapter, true);
 		adapter->flags &= ~IAVF_FLAG_QUEUES_DISABLED;
 		break;
 	case VIRTCHNL_OP_DISABLE_QUEUES:
Signature · 1 line
2.20.1
#5 lore raw

PATCH 5/15 iavf: untangle any pending iavf_open() operations from iavf_close()

Quoted 51 lines · Tony Nguyen

From: Nicholas Nunley <[email protected]>

When the iavf interface is opened some of the steps necessary to configure the hardware require communication with the PF driver. Since these operations involve waiting for a response from the PF driver they can be time-consuming, so the iavf driver schedules them for later and proceeds with the remaining configuration. If the interface is closed immediately after it is opened then some of these operations may still be pending, although the iavf_close() routine assumes they have all completed. In rare cases this can lead to iavf_open() configuration operations completing after iavf_close(), which can mean the device interrupts and/or DMA engine are active on a disabled interface.

To fix this:
1. In iavf_close() any pending unsent operations from iavf_open() are canceled.
2. If the operation was already sent by the time iavf_close() is called, but the driver is still awaiting the response back from the PF driver, then ignore the response if it is received when the interface is down instead of handling it in the usual manner.
3. Place a lock around the handling of all PF driver responses to ensure that these can't conflict with concurrent processing of iavf_open() and iavf_close(), or the other configuration tasks. In essence change (2) above protects against unexpected responses received after iavf_close(), and this change protects against responses received during iavf_close().

---
 drivers/net/ethernet/intel/iavf/iavf_main.c     | 10 ++++++++++
 drivers/net/ethernet/intel/iavf/iavf_virtchnl.c |  3 ++-
 2 files changed, 12 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +6 -0
index 91818ba7c8a3..eda8ebb8e7b8 100644
@@ -1025,6 +1025,12 @@ void iavf_down(struct iavf_adapter *adapter)
 		adapter->aq_required |= IAVF_FLAG_AQ_DEL_FDIR_FILTER;
 		adapter->aq_required |= IAVF_FLAG_AQ_DEL_ADV_RSS_CFG;
 		adapter->aq_required |= IAVF_FLAG_AQ_DISABLE_QUEUES;
+		/* In case the queue configure or enable operations are still
+		 * pending from when the interface was opened, make sure
+		 * they're canceled here.
+		 */
+		adapter->aq_required &= ~IAVF_FLAG_AQ_ENABLE_QUEUES;
+		adapter->aq_required &= ~IAVF_FLAG_AQ_CONFIGURE_QUEUES;
 	}
 
 	mod_delayed_work(iavf_wq, &adapter->watchdog_task, 0);

drivers/net/ethernet/intel/iavf/iavf_main.c static void iavf_adminq_task(struct work_struct *work) · line 2339

Quoted 9 lines
drivers/net/ethernet/intel/iavf/iavf_main.c +4 -0
@@ -2325,8 +2331,12 @@ static void iavf_adminq_task(struct work_struct *work)
 		if (ret || !v_op)
 			break; /* No event to process or error cleaning ARQ */
 
+		while (test_and_set_bit(__IAVF_IN_CRITICAL_TASK,
+					&adapter->crit_section))
+			usleep_range(500, 1000);
 		iavf_virtchnl_completion(adapter, v_op, v_ret, event.msg_buf,
 					 event.msg_len);
+		clear_bit(__IAVF_IN_CRITICAL_TASK, &adapter->crit_section);

This has potential to cause a deadlock. Please see my patch from about 3 months ago [1] which fixes the locking here and elsewhere. Also adds a timeout and prints a warning just in case.
I'm glad you recognized the problem so I'd say it's finally time to give that 3 months old patch the attention it deserves.

Stefan

[1] https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20210316100141.53551-1-sassmann at kpanic.de/

author maintainer
#6 lore raw

PATCH 6/15 iavf: disable interrupts before disabling napi

In reply to Tony Nguyen

From: Nicholas Nunley <[email protected]>

Interrupts should be turned off first before disabling napi, not the other way around, since the interrupt handler can schedule napi.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +4 -3
index eda8ebb8e7b8..1904000943dc 100644
@@ -970,8 +970,8 @@ void iavf_down(struct iavf_adapter *adapter)
 	netif_carrier_off(netdev);
 	netif_tx_disable(netdev);
 	adapter->link_up = false;
-	iavf_napi_disable_all(adapter);
 	iavf_irq_disable(adapter);
+	iavf_napi_disable_all(adapter);
 
 	spin_lock_bh(&adapter->mac_vlan_list_lock);
 
@@ -2040,8 +2040,8 @@ static void iavf_disable_vf(struct iavf_adapter *adapter)
 		netif_carrier_off(adapter->netdev);
 		netif_tx_disable(adapter->netdev);
 		adapter->link_up = false;
-		iavf_napi_disable_all(adapter);
 		iavf_irq_disable(adapter);
+		iavf_napi_disable_all(adapter);
 		iavf_free_traffic_irqs(adapter);
 		iavf_free_all_tx_resources(adapter);
 		iavf_free_all_rx_resources(adapter);
@@ -2171,6 +2171,8 @@ static void iavf_reset_task(struct work_struct *work)
 	}
 
 continue_reset:
+	iavf_irq_disable(adapter);
+
 	/* We don't use netif_running() because it may be true prior to
 	 * ndo_open() returning, so we can't assume it means all our open
 	 * tasks have finished, since we're not holding the rtnl_lock here.
@@ -2182,7 +2184,6 @@ static void iavf_reset_task(struct work_struct *work)
 		adapter->link_up = false;
 		iavf_napi_disable_all(adapter);
 	}
-	iavf_irq_disable(adapter);
 
 	adapter->state = __IAVF_RESETTING;
 	adapter->flags &= ~IAVF_FLAG_RESET_PENDING;
Signature · 1 line
2.20.1
author maintainer
#7 lore raw

PATCH 7/15 iavf: Restore non MAC filters after link down

In reply to Tony Nguyen

From: Akeem G Abodunrin <[email protected]>

Restore VLAN filters after the link is brought down, and up - since all filters are deleted from HW during the netdev link down routine.

---
 drivers/net/ethernet/intel/iavf/iavf.h      |  1 +
 drivers/net/ethernet/intel/iavf/iavf_main.c | 35 ++++++++++++++++++---
 2 files changed, 31 insertions(+), 5 deletions(-)
drivers/net/ethernet/intel/iavf/iavf.h +1 -0
index 1e996d9c1351..9ccb88e7fc65 100644
@@ -39,6 +39,7 @@
 #include "iavf_txrx.h"
 #include "iavf_fdir.h"
 #include "iavf_adv_rss.h"
+#include <linux/bitmap.h>
 
 #define DEFAULT_DEBUG_LEVEL_SHIFT 3
 #define PFX "iavf: "
drivers/net/ethernet/intel/iavf/iavf_main.c +30 -5
index 1904000943dc..06db563a6190 100644
@@ -666,6 +666,23 @@ static void iavf_del_vlan(struct iavf_adapter *adapter, u16 vlan)
 	spin_unlock_bh(&adapter->mac_vlan_list_lock);
 }
 
+/**
+ * iavf_restore_filters
+ * @adapter: board private structure
+ *
+ * Restore existing non MAC filters when VF netdev comes back up
+ **/
+static void iavf_restore_filters(struct iavf_adapter *adapter)
+{
+	/* re-add all VLAN filters */
+	if (VLAN_ALLOWED(adapter)) {
+		u16 vid;
+
+		for_each_set_bit(vid, adapter->vsi.active_vlans, VLAN_N_VID)
+			iavf_add_vlan(adapter, vid);
+	}
+}
+
 /**
  * iavf_vlan_rx_add_vid - Add a VLAN filter to a device
  * @netdev: network device struct
@@ -679,8 +696,11 @@ static int iavf_vlan_rx_add_vid(struct net_device *netdev,
 
 	if (!VLAN_ALLOWED(adapter))
 		return -EIO;
+
 	if (iavf_add_vlan(adapter, vid) == NULL)
 		return -ENOMEM;
+
+	set_bit(vid, adapter->vsi.active_vlans);
 	return 0;
 }
 
@@ -695,11 +715,13 @@ static int iavf_vlan_rx_kill_vid(struct net_device *netdev,
 {
 	struct iavf_adapter *adapter = netdev_priv(netdev);
 
-	if (VLAN_ALLOWED(adapter)) {
-		iavf_del_vlan(adapter, vid);
-		return 0;
-	}
-	return -EIO;
+	if (!VLAN_ALLOWED(adapter))
+		return -EIO;
+
+	iavf_del_vlan(adapter, vid);
+	clear_bit(vid, adapter->vsi.active_vlans);
+
+	return 0;
 }
 
 /**
@@ -3243,6 +3265,9 @@ static int iavf_open(struct net_device *netdev)
 
 	spin_unlock_bh(&adapter->mac_vlan_list_lock);
 
+	/* Restore VLAN filters that were removed with IFF_DOWN */
+	iavf_restore_filters(adapter);
+
 	iavf_configure(adapter);
 
 	iavf_up_complete(adapter);
Signature · 1 line
2.20.1
#8 lore raw

PATCH 7/15 iavf: Restore non MAC filters after link down

Quoted 13 lines

-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Nguyen, Anthony L
Sent: Friday, June 4, 2021 9:53 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH net-next 07/15] iavf: Restore non MAC filters after link down

From: Akeem G Abodunrin <[email protected]>

Restore VLAN filters after the link is brought down, and up - since all filters are deleted from HW during the netdev link down routine.

Quoted 7 lines
---
 drivers/net/ethernet/intel/iavf/iavf.h      |  1 +
 drivers/net/ethernet/intel/iavf/iavf_main.c | 35 ++++++++++++++++++---
 2 files changed, 31 insertions(+), 5 deletions(-)
author maintainer
#9 lore raw

PATCH 8/15 iavf: restore MSI state on reset

In reply to Tony Nguyen

From: Mitch Williams <[email protected]>

If the PF experiences an FLR, the VF's MSI and MSI-X configuration will be conveniently and silently removed in the process. When this happens, reset recovery will appear to complete normally but no traffic will pass. The netdev watchdog will helpfully notify everyone of this issue.

To prevent such public embarrassment, restore MSI configuration at every reset. For normal resets, this will do no harm, but for VF resets resulting from a PF FLR, this will keep the VF working.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 1 +
 1 file changed, 1 insertion(+)
drivers/net/ethernet/intel/iavf/iavf_main.c +1 -0
index 06db563a6190..2c639c7ebd1b 100644
@@ -2183,6 +2183,7 @@ static void iavf_reset_task(struct work_struct *work)
 	}
 
 	pci_set_master(adapter->pdev);
+	pci_restore_msi_state(adapter->pdev);
 
 	if (i == IAVF_RESET_WAIT_COMPLETE_COUNT) {
 		dev_err(&adapter->pdev->dev, "Reset never finished (%x)\n",
Signature · 1 line
2.20.1
#10 lore raw

PATCH 8/15 iavf: restore MSI state on reset

Quoted 18 lines

-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Nguyen, Anthony L
Sent: Friday, June 4, 2021 9:53 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH net-next 08/15] iavf: restore MSI state on reset

From: Mitch Williams <[email protected]>

If the PF experiences an FLR, the VF's MSI and MSI-X configuration will be conveniently and silently removed in the process. When this happens, reset recovery will appear to complete normally but no traffic will pass. The netdev watchdog will helpfully notify everyone of this issue.

To prevent such public embarrassment, restore MSI configuration at every reset. For normal resets, this will do no harm, but for VF resets resulting from a PF FLR, this will keep the VF working.

Quoted 6 lines
---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 1 +
 1 file changed, 1 insertion(+)
author maintainer
#11 lore raw

PATCH 9/15 iavf: Fix carrier on state

In reply to Tony Nguyen

From: Paul Greenwalt <[email protected]>

Carrier on is not set if the link up event (VIRTCHNL_EVENT_LINK_CHANGE) occurs before the queues are enabled (VIRTCHNL_OP_ENABLE_QUEUES) since setting carrier on during a link up event when queues are not enabled can result in a Tx hang.

Therefore set carrier on when queues are enabled if link is up and carrier is off.

---
 drivers/net/ethernet/intel/iavf/iavf_virtchnl.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +10 -1
index 69e479eb5534..3c3fd604218e 100644
@@ -1685,8 +1685,17 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
 		break;
 	case VIRTCHNL_OP_ENABLE_QUEUES:
 		/* enable transmits */
-		if (adapter->state == __IAVF_RUNNING)
+		if (adapter->state == __IAVF_RUNNING) {
 			iavf_irq_enable(adapter, true);
+
+			/* If queues not enabled when handling link event,
+			 * then set carrier on now
+			 */
+			if (adapter->link_up && !netif_carrier_ok(netdev)) {
+				netif_tx_start_all_queues(netdev);
+				netif_carrier_on(netdev);
+			}
+		}
 		adapter->flags &= ~IAVF_FLAG_QUEUES_DISABLED;
 		break;
 	case VIRTCHNL_OP_DISABLE_QUEUES:
Signature · 1 line
2.20.1
author maintainer
#12 lore raw

PATCH 10/15 iavf: Add change MTU message

In reply to Tony Nguyen

From: Patryk Ma?ek <[email protected]>

Add a netdev_info log entry in case of a change of MTU so that user is notified about this change in the same manner as in case of pf driver.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/net/ethernet/intel/iavf/iavf_main.c +2 -0
index 2c639c7ebd1b..e7b9c64db3b5 100644
@@ -3364,6 +3364,8 @@ static int iavf_change_mtu(struct net_device *netdev, int new_mtu)
 {
 	struct iavf_adapter *adapter = netdev_priv(netdev);
 
+	netdev_info(netdev, "changing MTU from %d to %d\n",
+		    netdev->mtu, new_mtu);
 	netdev->mtu = new_mtu;
 	if (CLIENT_ENABLED(adapter)) {
 		iavf_notify_client_l2_params(&adapter->vsi);
Signature · 1 line
2.20.1
#13 lore raw

PATCH 10/15 iavf: Add change MTU message

Quoted 8 lines

From: Intel-wired-lan <[email protected]> On Behalf Of Nguyen, Anthony L
Sent: Friday, June 4, 2021 9:54 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH net-next 10/15] iavf: Add change MTU message

From: Patryk Ma?ek <[email protected]>

Quoted 9 lines

Add a netdev_info log entry in case of a change of MTU so that user is notified about this change in the same manner as in case of pf driver.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 2 ++
 1 file changed, 2 insertions(+)
author maintainer
#14 lore raw

PATCH 11/15 iavf: Prevent changing static ITR values if adaptive moderation is on

In reply to Tony Nguyen

From: Nitesh B Venkatesh <[email protected]>

Resolve being able to change static values on VF when adaptive interrupt moderation is enabled.

This problem is fixed by checking the interrupt settings is not a combination of change of static value while adaptive interrupt moderation is turned on.

Without this fix, the user would be able to change static values on VF with adaptive moderation enabled.

---
 .../net/ethernet/intel/iavf/iavf_ethtool.c    | 30 ++++++++++++++++---
 1 file changed, 26 insertions(+), 4 deletions(-)
drivers/net/ethernet/intel/iavf/iavf_ethtool.c +26 -4
index 0d22a5275218..73901aedafdb 100644
@@ -719,12 +719,31 @@ static int iavf_get_per_queue_coalesce(struct net_device *netdev, u32 queue,
  *
  * Change the ITR settings for a specific queue.
  **/
-static void iavf_set_itr_per_queue(struct iavf_adapter *adapter,
-				   struct ethtool_coalesce *ec, int queue)
+static int iavf_set_itr_per_queue(struct iavf_adapter *adapter,
+				  struct ethtool_coalesce *ec, int queue)
 {
 	struct iavf_ring *rx_ring = &adapter->rx_rings[queue];
 	struct iavf_ring *tx_ring = &adapter->tx_rings[queue];
 	struct iavf_q_vector *q_vector;
+	u16 itr_setting;
+
+	itr_setting = rx_ring->itr_setting & ~IAVF_ITR_DYNAMIC;
+
+	if (ec->rx_coalesce_usecs != itr_setting &&
+	    ec->use_adaptive_rx_coalesce) {
+		netif_info(adapter, drv, adapter->netdev,
+			   "Rx interrupt throttling cannot be changed if adaptive-rx is enabled\n");
+		return -EINVAL;
+	}
+
+	itr_setting = tx_ring->itr_setting & ~IAVF_ITR_DYNAMIC;
+
+	if (ec->tx_coalesce_usecs != itr_setting &&
+	    ec->use_adaptive_tx_coalesce) {
+		netif_info(adapter, drv, adapter->netdev,
+			   "Tx interrupt throttling cannot be changed if adaptive-tx is enabled\n");
+		return -EINVAL;
+	}
 
 	rx_ring->itr_setting = ITR_REG_ALIGN(ec->rx_coalesce_usecs);
 	tx_ring->itr_setting = ITR_REG_ALIGN(ec->tx_coalesce_usecs);
@@ -747,6 +766,7 @@ static void iavf_set_itr_per_queue(struct iavf_adapter *adapter,
 	 * the Tx and Rx ITR values based on the values we have entered
 	 * into the q_vector, no need to write the values now.
 	 */
+	return 0;
 }
 
 /**
@@ -788,9 +808,11 @@ static int __iavf_set_coalesce(struct net_device *netdev,
 	 */
 	if (queue < 0) {
 		for (i = 0; i < adapter->num_active_queues; i++)
-			iavf_set_itr_per_queue(adapter, ec, i);
+			if (iavf_set_itr_per_queue(adapter, ec, i))
+				return -EINVAL;
 	} else if (queue < adapter->num_active_queues) {
-		iavf_set_itr_per_queue(adapter, ec, queue);
+		if (iavf_set_itr_per_queue(adapter, ec, queue))
+			return -EINVAL;
 	} else {
 		netif_info(adapter, drv, netdev, "Invalid queue value, queue range is 0 - %d\n",
 			   adapter->num_active_queues - 1);
Signature · 1 line
2.20.1
#15 lore raw

PATCH 11/15 iavf: Prevent changing static ITR values if adaptive moderation is on

Quoted 16 lines

-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Nguyen, Anthony L
Sent: Friday, June 4, 2021 9:54 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH net-next 11/15] iavf: Prevent changing static ITR values if adaptive moderation is on

From: Nitesh B Venkatesh <[email protected]>

Resolve being able to change static values on VF when adaptive interrupt moderation is enabled.

This problem is fixed by checking the interrupt settings is not a combination of change of static value while adaptive interrupt moderation is turned on.

Quoted 8 lines

Without this fix, the user would be able to change static values on VF with adaptive moderation enabled.

---
 .../net/ethernet/intel/iavf/iavf_ethtool.c    | 30 ++++++++++++++++---
 1 file changed, 26 insertions(+), 4 deletions(-)
author maintainer
#16 lore raw

PATCH 12/15 iavf: Log info when VF is entering and leaving Allmulti mode

In reply to Tony Nguyen

From: Grzegorz Szczurek <[email protected]>

Add log when VF is entering and leaving Allmulti mode. The change of VF state is visible in dmesg now. Without this commit, entering and leaving Allmulti mode is not logged in dmesg.

---
 .../net/ethernet/intel/iavf/iavf_virtchnl.c   | 20 +++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +14 -6
index 3c3fd604218e..d78206e168bc 100644
@@ -711,15 +711,23 @@ void iavf_set_promiscuous(struct iavf_adapter *adapter, int flags)
 	if (flags & FLAG_VF_MULTICAST_PROMISC) {
 		adapter->flags |= IAVF_FLAG_ALLMULTI_ON;
 		adapter->aq_required &= ~IAVF_FLAG_AQ_REQUEST_ALLMULTI;
-		dev_info(&adapter->pdev->dev, "Entering multicast promiscuous mode\n");
+		dev_info(&adapter->pdev->dev, "%s is entering multicast promiscuous mode\n",
+			 adapter->netdev->name);
 	}
 
 	if (!flags) {
-		adapter->flags &= ~(IAVF_FLAG_PROMISC_ON |
-				    IAVF_FLAG_ALLMULTI_ON);
-		adapter->aq_required &= ~(IAVF_FLAG_AQ_RELEASE_PROMISC |
-					  IAVF_FLAG_AQ_RELEASE_ALLMULTI);
-		dev_info(&adapter->pdev->dev, "Leaving promiscuous mode\n");
+		if (adapter->flags & IAVF_FLAG_PROMISC_ON) {
+			adapter->flags &= ~IAVF_FLAG_PROMISC_ON;
+			adapter->aq_required &= ~IAVF_FLAG_AQ_RELEASE_PROMISC;
+			dev_info(&adapter->pdev->dev, "Leaving promiscuous mode\n");
+		}
+
+		if (adapter->flags & IAVF_FLAG_ALLMULTI_ON) {
+			adapter->flags &= ~IAVF_FLAG_ALLMULTI_ON;
+			adapter->aq_required &= ~IAVF_FLAG_AQ_RELEASE_ALLMULTI;
+			dev_info(&adapter->pdev->dev, "%s is leaving multicast promiscuous mode\n",
+				 adapter->netdev->name);
+		}
 	}
 
 	adapter->current_op = VIRTCHNL_OP_CONFIG_PROMISCUOUS_MODE;
Signature · 1 line
2.20.1
#17 lore raw

PATCH 12/15 iavf: Log info when VF is entering and leaving Allmulti mode

Quoted 14 lines

-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Nguyen, Anthony L
Sent: Friday, June 4, 2021 9:54 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH net-next 12/15] iavf: Log info when VF is entering and leaving Allmulti mode

From: Grzegorz Szczurek <[email protected]>

Add log when VF is entering and leaving Allmulti mode.
The change of VF state is visible in dmesg now.
Without this commit, entering and leaving Allmulti mode is not logged in dmesg.

Quoted 6 lines
---
 .../net/ethernet/intel/iavf/iavf_virtchnl.c   | 20 +++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
author maintainer
#18 lore raw

PATCH 13/15 iavf: Set RSS LUT and key in reset handle path

In reply to Tony Nguyen

From: Md Fahad Iqbal Polash <[email protected]>

iavf driver should set RSS LUT and key unconditionally in reset path. Currently, the driver does not do that. This patch fixes this issue.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +8 -5
index e7b9c64db3b5..892aa22b39da 100644
@@ -1534,11 +1534,6 @@ static int iavf_reinit_interrupt_scheme(struct iavf_adapter *adapter)
 	set_bit(__IAVF_VSI_DOWN, adapter->vsi.state);
 
 	iavf_map_rings_to_vectors(adapter);
-
-	if (RSS_AQ(adapter))
-		adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_RSS;
-	else
-		err = iavf_init_rss(adapter);
 err:
 	return err;
 }
@@ -2233,6 +2228,14 @@ static void iavf_reset_task(struct work_struct *work)
 			goto reset_err;
 	}
 
+	if (RSS_AQ(adapter)) {
+		adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_RSS;
+	} else {
+		err = iavf_init_rss(adapter);
+		if (err)
+			goto reset_err;
+	}
+
 	adapter->aq_required |= IAVF_FLAG_AQ_GET_CONFIG;
 	adapter->aq_required |= IAVF_FLAG_AQ_MAP_VECTORS;
 
Signature · 1 line
2.20.1
#19 lore raw

PATCH 13/15 iavf: Set RSS LUT and key in reset handle path

Quoted 47 lines

From: Md Fahad Iqbal Polash <[email protected]>

iavf driver should set RSS LUT and key unconditionally in reset path. Currently, the driver does not do that. This patch fixes this issue.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +0 -5
index e7b9c64db3b5..892aa22b39da 100644
@@ -1534,11 +1534,6 @@ static int iavf_reinit_interrupt_scheme(struct
 iavf_adapter *adapter)
 	set_bit(__IAVF_VSI_DOWN, adapter->vsi.state);
 
 	iavf_map_rings_to_vectors(adapter);
-
-	if (RSS_AQ(adapter))
-		adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_RSS;
-	else
-		err = iavf_init_rss(adapter);
 err:
 	return err;

}

drivers/net/ethernet/intel/iavf/iavf_main.c +8 -1
@@ -2233,6 +2228,14 @@ static void iavf_reset_task(struct work_struct
 *work)
 			goto reset_err;
 	}
 
+	if (RSS_AQ(adapter)) {
+		adapter->aq_required |= IAVF_FLAG_AQ_CONFIGURE_RSS;
+	} else {
+		err = iavf_init_rss(adapter);
+		if (err)
+			goto reset_err;
+	}
+
 	adapter->aq_required |= IAVF_FLAG_AQ_GET_CONFIG;
 	adapter->aq_required |= IAVF_FLAG_AQ_MAP_VECTORS;
 
--
Quoted 1 line

2.20.1

author maintainer
#20 lore raw

PATCH 14/15 iavf: return errno code instead of status code

In reply to Tony Nguyen

From: Jacob Keller <[email protected]>

The iavf_parse_cls_flower function returns an integer error code, and not an iavf_status enumeration.

Fix the function to use the standard errno value EINVAL as its return instead of using IAVF_ERR_CONFIG.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +10 -10
index 892aa22b39da..7730de0ef236 100644
@@ -2845,7 +2845,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
 			} else {
 				dev_err(&adapter->pdev->dev, "Bad ether dest mask %pM\n",
 					match.mask->dst);
-				return IAVF_ERR_CONFIG;
+				return -EINVAL;
 			}
 		}
 
@@ -2855,7 +2855,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
 			} else {
 				dev_err(&adapter->pdev->dev, "Bad ether src mask %pM\n",
 					match.mask->src);
-				return IAVF_ERR_CONFIG;
+				return -EINVAL;
 			}
 		}
 
@@ -2890,7 +2890,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
 			} else {
 				dev_err(&adapter->pdev->dev, "Bad vlan mask %u\n",
 					match.mask->vlan_id);
-				return IAVF_ERR_CONFIG;
+				return -EINVAL;
 			}
 		}
 		vf->mask.tcp_spec.vlan_id |= cpu_to_be16(0xffff);
@@ -2914,7 +2914,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
 			} else {
 				dev_err(&adapter->pdev->dev, "Bad ip dst mask 0x%08x\n",
 					be32_to_cpu(match.mask->dst));
-				return IAVF_ERR_CONFIG;
+				return -EINVAL;
 			}
 		}
 
@@ -2924,13 +2924,13 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
 			} else {
 				dev_err(&adapter->pdev->dev, "Bad ip src mask 0x%08x\n",
 					be32_to_cpu(match.mask->dst));
-				return IAVF_ERR_CONFIG;
+				return -EINVAL;
 			}
 		}
 
 		if (field_flags & IAVF_CLOUD_FIELD_TEN_ID) {
 			dev_info(&adapter->pdev->dev, "Tenant id not allowed for ip filter\n");
-			return IAVF_ERR_CONFIG;
+			return -EINVAL;
 		}
 		if (match.key->dst) {
 			vf->mask.tcp_spec.dst_ip[0] |= cpu_to_be32(0xffffffff);
@@ -2951,7 +2951,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
 		if (ipv6_addr_any(&match.mask->dst)) {
 			dev_err(&adapter->pdev->dev, "Bad ipv6 dst mask 0x%02x\n",
 				IPV6_ADDR_ANY);
-			return IAVF_ERR_CONFIG;
+			return -EINVAL;
 		}
 
 		/* src and dest IPv6 address should not be LOOPBACK
@@ -2961,7 +2961,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
 		    ipv6_addr_loopback(&match.key->src)) {
 			dev_err(&adapter->pdev->dev,
 				"ipv6 addr should not be loopback\n");
-			return IAVF_ERR_CONFIG;
+			return -EINVAL;
 		}
 		if (!ipv6_addr_any(&match.mask->dst) ||
 		    !ipv6_addr_any(&match.mask->src))
@@ -2986,7 +2986,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
 			} else {
 				dev_err(&adapter->pdev->dev, "Bad src port mask %u\n",
 					be16_to_cpu(match.mask->src));
-				return IAVF_ERR_CONFIG;
+				return -EINVAL;
 			}
 		}
 
@@ -2996,7 +2996,7 @@ static int iavf_parse_cls_flower(struct iavf_adapter *adapter,
 			} else {
 				dev_err(&adapter->pdev->dev, "Bad dst port mask %u\n",
 					be16_to_cpu(match.mask->dst));
-				return IAVF_ERR_CONFIG;
+				return -EINVAL;
 			}
 		}
 		if (match.key->dst) {
Signature · 1 line
2.20.1
#21 lore raw

PATCH 14/15 iavf: return errno code instead of status code

drivers/net/ethernet/intel/iavf/iavf_main.c

Quoted 29 lines

-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Tony Nguyen
Sent: pi?tek, 4 czerwca 2021 18:54
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH net-next 14/15] iavf: return errno code instead of status code

From: Jacob Keller <[email protected]>

The iavf_parse_cls_flower function returns an integer error code, and not an iavf_status enumeration.

Fix the function to use the standard errno value EINVAL as its return instead of using IAVF_ERR_CONFIG.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +0 -0
index 892aa22b39da..7730de0ef236 100644
author maintainer
#22 lore raw

PATCH 15/15 iavf: don't be so alarming

In reply to Tony Nguyen

From: Mitch Williams <[email protected]>

Reduce the log level of a couple of messages. These can appear during normal reset and rmmod processing, and the driver recovers just fine. Debug level is fine for these.

---
 drivers/net/ethernet/intel/iavf/iavf_main.c     | 2 +-
 drivers/net/ethernet/intel/iavf/iavf_virtchnl.c | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/iavf/iavf_main.c +1 -1
index 7730de0ef236..06be785c0d82 100644
@@ -3352,7 +3352,7 @@ static int iavf_close(struct net_device *netdev)
 				    adapter->state == __IAVF_DOWN,
 				    msecs_to_jiffies(500));
 	if (!status)
-		netdev_warn(netdev, "Device resources not yet released\n");
+		netdev_dbg(netdev, "Device resources not yet released\n");
 	return 0;
 }
 
drivers/net/ethernet/intel/iavf/iavf_virtchnl.c +2 -2
index d78206e168bc..ba27b2168fd3 100644
@@ -1866,8 +1866,8 @@ void iavf_virtchnl_completion(struct iavf_adapter *adapter,
 		break;
 	default:
 		if (adapter->current_op && (v_opcode != adapter->current_op))
-			dev_warn(&adapter->pdev->dev, "Expected response %d from PF, received %d\n",
-				 adapter->current_op, v_opcode);
+			dev_dbg(&adapter->pdev->dev, "Expected response %d from PF, received %d\n",
+				adapter->current_op, v_opcode);
 		break;
 	} /* switch v_opcode */
 	adapter->current_op = VIRTCHNL_OP_UNKNOWN;
Signature · 1 line
2.20.1
#23 lore raw

PATCH 15/15 iavf: don't be so alarming

Quoted 13 lines

-----Original Message-----
From: Intel-wired-lan <[email protected]> On Behalf Of Nguyen, Anthony L
Sent: Friday, June 4, 2021 9:54 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH net-next 15/15] iavf: don't be so alarming

From: Mitch Williams <[email protected]>

Reduce the log level of a couple of messages. These can appear during normal reset and rmmod processing, and the driver recovers just fine. Debug level is fine for these.

Quoted 7 lines
---
 drivers/net/ethernet/intel/iavf/iavf_main.c     | 2 +-
 drivers/net/ethernet/intel/iavf/iavf_virtchnl.c | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)