Skip to contentSkip to conversation

ixgbe: Set the correct pool when VLANs are added in SR-IOV mode.

PATCH Applied Jeff Kirsher 2 participants 14 messages Jan 23, 2010

Subsystems (2)
Networking Network drivers
lore
Share & tools
lore

Jeff Kirsher

author
#0 lore raw

PATCH 1/7 ixgbe: Set the correct pool when VLANs are added in SR-IOV mode.

From: Greg Rose <[email protected]>

When VFs are allocated (as indicated by adapter->num_vfs is non-zero) then the PF pool is no longer zero. Instead it will be the same as the number of VFs allocated. When setting the VLVF entry for the PF we need to use the correct pool otherwise the PF will get VLAN packets from the wire because the packet will pass VFTA filtering and the PF has the default pool, but it will not get VLAN packets from the VFs because it has not set the correct pool bit in the VLVF entry.

Signed-off-by Greg Rose <[email protected]>
Signed-off-by Jeff Kirsher <[email protected]>

---

 drivers/net/ixgbe/ixgbe_main.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
drivers/net/ixgbe/ixgbe_main.c +4 -2
index 81971ed..2091658 100644
@@ -2432,15 +2432,17 @@ static void ixgbe_vlan_rx_add_vid(struct net_device *netdev, u16 vid)
 {
 	struct ixgbe_adapter *adapter = netdev_priv(netdev);
 	struct ixgbe_hw *hw = &adapter->hw;
+	int pool_ndx = adapter->num_vfs;
 
 	/* add VID to filter table */
-	hw->mac.ops.set_vfta(&adapter->hw, vid, 0, true);
+	hw->mac.ops.set_vfta(&adapter->hw, vid, pool_ndx, true);
 }
 
 static void ixgbe_vlan_rx_kill_vid(struct net_device *netdev, u16 vid)
 {
 	struct ixgbe_adapter *adapter = netdev_priv(netdev);
 	struct ixgbe_hw *hw = &adapter->hw;
+	int pool_ndx = adapter->num_vfs;
 
 	if (!test_bit(__IXGBE_DOWN, &adapter->state))
 		ixgbe_irq_disable(adapter);

@@ -2451,7 +2453,7 @@ static void ixgbe_vlan_rx_kill_vid(struct net_device *netdev, u16 vid)
 		ixgbe_irq_enable(adapter);
 
 	/* remove VID from filter table */
-	hw->mac.ops.set_vfta(&adapter->hw, vid, 0, false);
+	hw->mac.ops.set_vfta(&adapter->hw, vid, pool_ndx, false);
 }
 
 static void ixgbe_vlan_rx_register(struct net_device *netdev,
author
#1 lore raw

PATCH 2/7 ixgbe: Remove unused emulation MAC storage from the per VF data structure.

From: Greg Rose <[email protected]>

This data storage for SW emulated MAC addresses is unlikely to ever be used so pull it.

Signed-off-by Greg Rose <[email protected]>
Signed-off-by Jeff Kirsher <[email protected]>

---

 drivers/net/ixgbe/ixgbe.h |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)
drivers/net/ixgbe/ixgbe.h +0 -1
index ed73585..e3909d3 100644
@@ -110,7 +110,6 @@ struct vf_data_storage {
 	u16 num_vf_mc_hashes;
 	u16 default_vf_vlan_id;
 	u16 vlans_enabled;
-	unsigned char em_mac_addresses[MAX_EMULATION_MAC_ADDRS * ETH_ALEN];
 	bool clear_to_send;
 	int rar;
 };
author
#3 lore raw

PATCH 3/7 ixgbe: Allow the VF driver to be loaded before the PF driver

In reply to Jeff Kirsher

From: Greg Rose <[email protected]>

The PF Reset Done bit should not be set in the extended control register until the PF has actually completed the bring up process. It is a mis- interpretation of the purpose of this bit to assume it should be set when the physical reset of the device is done. Instead it should be used to indicate to the VFs when the PF is ready to provide them with required services. This is not until after the PF is finished coming up and ready to process mailbox events.

Signed-off-by Greg Rose <[email protected]>
Signed-off-by Jeff Kirsher <[email protected]>

---

 drivers/net/ixgbe/ixgbe_82599.c |    6 +-----
 drivers/net/ixgbe/ixgbe_main.c  |    7 +++++++
 2 files changed, 8 insertions(+), 5 deletions(-)
drivers/net/ixgbe/ixgbe_82599.c +1 -5
index 9ec296c..591c7b9 100644
@@ -890,7 +890,7 @@ static s32 ixgbe_setup_copper_link_82599(struct ixgbe_hw *hw,
 static s32 ixgbe_reset_hw_82599(struct ixgbe_hw *hw)
 {
 	s32 status = 0;
-	u32 ctrl, ctrl_ext;
+	u32 ctrl;
 	u32 i;
 	u32 autoc;
 	u32 autoc2;

@@ -945,10 +945,6 @@ static s32 ixgbe_reset_hw_82599(struct ixgbe_hw *hw)
 		status = IXGBE_ERR_RESET_FAILED;
 		hw_dbg(hw, "Reset polling failed to complete.\n");
 	}
-	/* Clear PF Reset Done bit so PF/VF Mail Ops can work */
-	ctrl_ext = IXGBE_READ_REG(hw, IXGBE_CTRL_EXT);
-	ctrl_ext |= IXGBE_CTRL_EXT_PFRSTD;
-	IXGBE_WRITE_REG(hw, IXGBE_CTRL_EXT, ctrl_ext);
 
 	msleep(50);
drivers/net/ixgbe/ixgbe_main.c +7 -0
index 2091658..a986f1d 100644
@@ -2826,6 +2826,7 @@ static int ixgbe_up_complete(struct ixgbe_adapter *adapter)
 	u32 txdctl, rxdctl, mhadd;
 	u32 dmatxctl;
 	u32 gpie;
+	u32 ctrl_ext;
 
 	ixgbe_get_hw_control(adapter);
 

@@ -3015,6 +3016,12 @@ static int ixgbe_up_complete(struct ixgbe_adapter *adapter)
 	adapter->flags |= IXGBE_FLAG_NEED_LINK_UPDATE;
 	adapter->link_check_timeout = jiffies;
 	mod_timer(&adapter->watchdog_timer, jiffies);
+
+	/* Set PF Reset Done bit so PF/VF Mail Ops can work */
+	ctrl_ext = IXGBE_READ_REG(hw, IXGBE_CTRL_EXT);
+	ctrl_ext |= IXGBE_CTRL_EXT_PFRSTD;
+	IXGBE_WRITE_REG(hw, IXGBE_CTRL_EXT, ctrl_ext);
+
 	return 0;
 }
maintainer Applied
#4 lore raw

Re: [net-next-2.6 PATCH 3/7] ixgbe: Allow the VF driver to be loaded before the PF driver

From: Jeff Kirsher <[email protected]>
Date: Sat, 23 Jan 2010 00:46:22 -0800

Quoted 10 lines

From: Greg Rose <[email protected]>

The PF Reset Done bit should not be set in the extended control register until the PF has actually completed the bring up process. It is a mis- interpretation of the purpose of this bit to assume it should be set when the physical reset of the device is done. Instead it should be used to indicate to the VFs when the PF is ready to provide them with required services. This is not until after the PF is finished coming up and ready to process mailbox events.

Quoted 2 lines
Signed-off-by Greg Rose <[email protected]>
Signed-off-by Jeff Kirsher <[email protected]>

Applied.

author
#5 lore raw

PATCH 4/7 ixgbe: Improve reset coordination between the PF and the VF

In reply to Jeff Kirsher

From: Greg Rose <[email protected]>

Inadequate coordination between the PF driver and the VF driver results in tx hangs in the VF driver when you perform certain actions that will lead to a re-init of the PF. Add feature to notify active VFs when the PF is about to re-initialize so that the VFs can take appropriate action.

Signed-off-by Greg Rose <[email protected]>
Signed-off-by Jeff Kirsher <[email protected]>

---

 drivers/net/ixgbe/ixgbe_main.c  |   11 +++++++++++
 drivers/net/ixgbe/ixgbe_sriov.c |   26 ++++++++++++++++++++++++++
 drivers/net/ixgbe/ixgbe_sriov.h |    2 ++
 3 files changed, 39 insertions(+), 0 deletions(-)
drivers/net/ixgbe/ixgbe_main.c +11 -0
index a986f1d..3d9c075 100644
@@ -3203,6 +3203,17 @@ void ixgbe_down(struct ixgbe_adapter *adapter)
 	/* signal that we are down to the interrupt handler */
 	set_bit(__IXGBE_DOWN, &adapter->state);
 
+	/* disable receive for all VFs and wait one second */
+	if (adapter->num_vfs) {
+		for (i = 0 ; i < adapter->num_vfs; i++)
+			adapter->vfinfo[i].clear_to_send = 0;
+
+		/* ping all the active vfs to let them know we are going down */
+		ixgbe_ping_all_vfs(adapter);
+		/* Disable all VFTE/VFRE TX/RX */
+		ixgbe_disable_tx_rx(adapter);
+	}
+
 	/* disable receives */
 	rxctrl = IXGBE_READ_REG(hw, IXGBE_RXCTRL);
 	IXGBE_WRITE_REG(hw, IXGBE_RXCTRL, rxctrl & ~IXGBE_RXCTRL_RXEN);
drivers/net/ixgbe/ixgbe_sriov.c +26 -0
index 74bca74..d4cd20f 100644
@@ -334,3 +334,29 @@ void ixgbe_msg_task(struct ixgbe_adapter *adapter)
 	}
 }
 
+void ixgbe_disable_tx_rx(struct ixgbe_adapter *adapter)
+{
+	struct ixgbe_hw *hw = &adapter->hw;
+
+	/* disable transmit and receive for all vfs */
+	IXGBE_WRITE_REG(hw, IXGBE_VFTE(0), 0);
+	IXGBE_WRITE_REG(hw, IXGBE_VFTE(1), 0);
+
+	IXGBE_WRITE_REG(hw, IXGBE_VFRE(0), 0);
+	IXGBE_WRITE_REG(hw, IXGBE_VFRE(1), 0);
+}
+
+void ixgbe_ping_all_vfs(struct ixgbe_adapter *adapter)
+{
+	struct ixgbe_hw *hw = &adapter->hw;
+	u32 ping;
+	int i;
+
+	for (i = 0 ; i < adapter->num_vfs; i++) {
+		ping = IXGBE_PF_CONTROL_MSG;
+		if (adapter->vfinfo[i].clear_to_send)
+			ping |= IXGBE_VT_MSGTYPE_CTS;
+		ixgbe_write_mbx(hw, &ping, 1, i);
+	}
+}
+
drivers/net/ixgbe/ixgbe_sriov.h +2 -0
index 664b237..51d1106 100644
@@ -39,6 +39,8 @@ void ixgbe_msg_task(struct ixgbe_adapter *adapter);
 int ixgbe_set_vf_mac(struct ixgbe_adapter *adapter,
                      int vf, unsigned char *mac_addr);
 int ixgbe_vf_configuration(struct pci_dev *pdev, unsigned int event_mask);
+void ixgbe_disable_tx_rx(struct ixgbe_adapter *adapter);
+void ixgbe_ping_all_vfs(struct ixgbe_adapter *adapter);
 void ixgbe_dump_registers(struct ixgbe_adapter *adapter);
 
 #endif /* _IXGBE_SRIOV_H_ */
maintainer Applied
#6 lore raw

Re: [net-next-2.6 PATCH 4/7] ixgbe: Improve reset coordination between the PF and the VF

From: Jeff Kirsher <[email protected]>
Date: Sat, 23 Jan 2010 00:46:40 -0800

Quoted 9 lines

From: Greg Rose <[email protected]>

Inadequate coordination between the PF driver and the VF driver results in tx hangs in the VF driver when you perform certain actions that will lead to a re-init of the PF. Add feature to notify active VFs when the PF is about to re-initialize so that the VFs can take appropriate action.

Signed-off-by Greg Rose <[email protected]>
Signed-off-by Jeff Kirsher <[email protected]>

Applied.

author
#7 lore raw

PATCH 5/7 ixgbevf: Take action when the PF notifies the VF it is resetting.

In reply to Jeff Kirsher

From: Greg Rose <[email protected]>

When the VF driver gets a control message from the PF that indicates the PF is about to reset or go down we schedule the watchdog timer so that it will detect the PF has gone offline and take appropriate action.

Signed-off-by Greg Rose <[email protected]>
Signed-off-by Jeff Kirsher <[email protected]>

---

 drivers/net/ixgbevf/ixgbevf_main.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)
drivers/net/ixgbevf/ixgbevf_main.c +7 -0
index 39544af..bd2fd46 100644
@@ -956,10 +956,17 @@ static irqreturn_t ixgbevf_msix_mbx(int irq, void *data)
 	struct ixgbevf_adapter *adapter = netdev_priv(netdev);
 	struct ixgbe_hw *hw = &adapter->hw;
 	u32 eicr;
+	u32 msg;
 
 	eicr = IXGBE_READ_REG(hw, IXGBE_VTEICS);
 	IXGBE_WRITE_REG(hw, IXGBE_VTEICR, eicr);
 
+	hw->mbx.ops.read(hw, &msg, 1);
+
+	if ((msg & IXGBE_MBVFICR_VFREQ_MASK) == IXGBE_PF_CONTROL_MSG)
+		mod_timer(&adapter->watchdog_timer,
+			  round_jiffies(jiffies + 10));
+
 	return IRQ_HANDLED;
 }
maintainer Applied
#8 lore raw

Re: [net-next-2.6 PATCH 5/7] ixgbevf: Take action when the PF notifies the VF it is resetting.

From: Jeff Kirsher <[email protected]>
Date: Sat, 23 Jan 2010 00:47:00 -0800

Quoted 8 lines

From: Greg Rose <[email protected]>

When the VF driver gets a control message from the PF that indicates the PF is about to reset or go down we schedule the watchdog timer so that it will detect the PF has gone offline and take appropriate action.

Signed-off-by Greg Rose <[email protected]>
Signed-off-by Jeff Kirsher <[email protected]>

Applied.

author
#9 lore raw

PATCH 6/7 ixgbevf: Fix panics in the VF driver

In reply to Jeff Kirsher

From: Greg Rose <[email protected]>

Fix panics in the VF driver that occur when you bring it down after having already brought the PF down.

Signed-off-by Greg Rose <[email protected]>
Signed-off-by Jeff Kirsher <[email protected]>

---

 drivers/net/ixgbevf/ixgbevf_main.c |   24 ++++++++++++++++++++----
 1 files changed, 20 insertions(+), 4 deletions(-)
drivers/net/ixgbevf/ixgbevf_main.c +20 -4
index bd2fd46..0a27fa1 100644
@@ -1693,8 +1693,10 @@ static void ixgbevf_clean_rx_ring(struct ixgbevf_adapter *adapter,
 	unsigned long size;
 	unsigned int i;
 
-	/* Free all the Rx ring sk_buffs */
+	if (!rx_ring->rx_buffer_info)
+		return;
 
+	/* Free all the Rx ring sk_buffs */
 	for (i = 0; i < rx_ring->count; i++) {
 		struct ixgbevf_rx_buffer *rx_buffer_info;
 

@@ -1751,6 +1753,9 @@ static void ixgbevf_clean_tx_ring(struct ixgbevf_adapter *adapter,
 	unsigned long size;
 	unsigned int i;
 
+	if (!tx_ring->tx_buffer_info)
+		return;
+
 	/* Free all the Tx ring sk_buffs */
 
 	for (i = 0; i < tx_ring->count; i++) {

@@ -1843,12 +1848,24 @@ void ixgbevf_down(struct ixgbevf_adapter *adapter)
 
 void ixgbevf_reinit_locked(struct ixgbevf_adapter *adapter)
 {
+	struct ixgbe_hw *hw = &adapter->hw;
+
 	WARN_ON(in_interrupt());
+
 	while (test_and_set_bit(__IXGBEVF_RESETTING, &adapter->state))
 		msleep(1);
 
-	ixgbevf_down(adapter);
-	ixgbevf_up(adapter);
+	/*
+	 * Check if PF is up before re-init.  If not then skip until
+	 * later when the PF is up and ready to service requests from
+	 * the VF via mailbox.  If the VF is up and running then the
+	 * watchdog task will continue to schedule reset tasks until
+	 * the PF is up and running.
+	 */
+	if (!hw->mac.ops.reset_hw(hw)) {
+		ixgbevf_down(adapter);
+		ixgbevf_up(adapter);
+	}
 
 	clear_bit(__IXGBEVF_RESETTING, &adapter->state);
 }

@@ -2423,7 +2440,6 @@ void ixgbevf_free_tx_resources(struct ixgbevf_adapter *adapter,
 {
 	struct pci_dev *pdev = adapter->pdev;
 
-
 	ixgbevf_clean_tx_ring(adapter, tx_ring);
 
 	vfree(tx_ring->tx_buffer_info);
author
#11 lore raw

PATCH 7/7 ixgbevf: Tell network stack to stop tx when the VF detects PF reset

In reply to Jeff Kirsher

From: Greg Rose <[email protected]>

When the VF detects that the PF has reset turn off carrier and stop all tx queues.

Signed-off-by Greg Rose <[email protected]>
Signed-off-by Jeff Kirsher <[email protected]>

---

 drivers/net/ixgbevf/ixgbevf_main.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
drivers/net/ixgbevf/ixgbevf_main.c +2 -0
index 0a27fa1..623353d 100644
@@ -2381,6 +2381,8 @@ static void ixgbevf_watchdog_task(struct work_struct *work)
 					    &link_up, false)) != 0) {
 			adapter->link_up = link_up;
 			adapter->link_speed = link_speed;
+			netif_carrier_off(netdev);
+			netif_tx_stop_all_queues(netdev);
 			schedule_work(&adapter->reset_task);
 			goto pf_has_reset;
 		}
maintainer Applied
#13 lore raw
In reply to Jeff Kirsher

From: Jeff Kirsher <[email protected]>
Date: Sat, 23 Jan 2010 00:45:43 -0800

Quoted 10 lines

From: Greg Rose <[email protected]>

When VFs are allocated (as indicated by adapter->num_vfs is non-zero) then the PF pool is no longer zero. Instead it will be the same as the number of VFs allocated. When setting the VLVF entry for the PF we need to use the correct pool otherwise the PF will get VLAN packets from the wire because the packet will pass VFTA filtering and the PF has the default pool, but it will not get VLAN packets from the VFs because it has not set the correct pool bit in the VLVF entry.

Quoted 2 lines
Signed-off-by Greg Rose <[email protected]>
Signed-off-by Jeff Kirsher <[email protected]>

Applied.