Skip to contentSkip to conversation

ptp cornercase cleanups

PATCH New Jacob Keller 5 participants 34 messages May 3, 2017 – Jun 22, 2017

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v4 cover ptp cornercase cleanups

This series contains fixes for similar corner cases in the e1000e, igb, i40e, and ixgbe drivers reported by David Mirabito on the ptp4l mailing list.

The first few patches fix a race condition with skb_tstamp_tx(). The next few patches ensure that we cleanup the state bit when exiting the _xmit_frame_ring() function with a DMA or TSO failure. After this, we add new statistic to count how many times the Tx timestamp is skipped due to already outstanding requests. The final few patches add some handling in the potential case where we somehow never finish a timestamp request, ensuring that we don't permanently lock the Tx timestamps forever in this rare case.

Changes in v4
- Split the driver patches at request of Shannon Nelson
- Updated commit messages, but code content should be identical

I apologize in advance, the patchworks will not track this properly because I necessarily changed the commit subjects when splitting the patches apart by driver.

Jacob Keller (15):
e1000e: fix race condition around skb_tstamp_tx()
i40e: fix race condition with PTP_TX_IN_PROGRESS bits
igb: fix race condition with PTP_TX_IN_PROGRESS bits
ixgbe: fix race condition with PTP_TX_IN_PROGRESS bits
i40e: avoid permanent lock of *_PTP_TX_IN_PROGRESS
igb: avoid permanent lock of *_PTP_TX_IN_PROGRESS
ixgbe: avoid permanent lock of *_PTP_TX_IN_PROGRESS
e1000e: add statistic indicating number of skipped Tx timestamps
i40e: add statistic indicating number of skipped Tx timestamps
igb: add statistic indicating number of skipped Tx timestamps
ixgbe: add statistic indicating number of skipped Tx timestamps
i40e: use pf data structure directly in i40e_ptp_rx_hang
i40e: check for Tx timestamp timeouts during watchdog
igb: check for Tx timestamp timeouts during watchdog
ixgbe: check for Tx timestamp timeouts during watchdog

 drivers/net/ethernet/intel/e1000e/e1000.h        |  1 +
 drivers/net/ethernet/intel/e1000e/ethtool.c      |  1 +
 drivers/net/ethernet/intel/e1000e/netdev.c       | 27 ++++++++-----
 drivers/net/ethernet/intel/i40e/i40e.h           |  5 ++-
 drivers/net/ethernet/intel/i40e/i40e_ethtool.c   |  1 +
 drivers/net/ethernet/intel/i40e/i40e_main.c      |  3 +-
 drivers/net/ethernet/intel/i40e/i40e_ptp.c       | 48 +++++++++++++++++++++---
 drivers/net/ethernet/intel/i40e/i40e_txrx.c      | 28 +++++++++++---
 drivers/net/ethernet/intel/igb/igb.h             |  2 +
 drivers/net/ethernet/intel/igb/igb_ethtool.c     |  1 +
 drivers/net/ethernet/intel/igb/igb_main.c        | 26 ++++++++++---
 drivers/net/ethernet/intel/igb/igb_ptp.c         | 41 +++++++++++++++++++-
 drivers/net/ethernet/intel/ixgbe/ixgbe.h         |  2 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c |  3 ++
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c    | 42 ++++++++++++++-------
 drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c     | 42 +++++++++++++++++++--
 16 files changed, 227 insertions(+), 46 deletions(-)
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#1 lore raw

PATCH v4 1/15 e1000e: fix race condition around skb_tstamp_tx()

The e1000e driver and related hardware has a limitation on Tx PTP packets which requires we limit to timestamping a single packet at once. We do this by verifying that we never request a new Tx timestamp while we still have a tx_hwtstamp_skb pointer.

Unfortunately the driver suffers from a race condition around this. The tx_hwtstamp_skb pointer is not set to NULL until after skb_tstamp_tx() is called. This function notifies the stack and applications of a new timestamp. Even a well behaved application that only sends a new request when the first one is finished might be woken up and possibly send a packet before we can free the timestamp in the driver again. The result is that we needlessly ignore some Tx timestamp requests in this corner case.

Fix this by assigning the tx_hwtstamp_skb pointer prior to calling skb_tstamp_tx() and use a temporary pointer to hold the timestamped skb until that function finishes. This ensures that the application is not woken up until the driver is ready to begin timestamping a new packet.

This ensures that well behaved applications do not accidentally race with condition to skip Tx timestamps. Obviously an application which sends multiple Tx timestamp requests at once will still only timestamp one packet at a time. Unfortunately there is nothing we can do about this.

---
 drivers/net/ethernet/intel/e1000e/netdev.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/net/ethernet/intel/e1000e/netdev.c +8 -2
index b3679728caac..ec9a50a72550 100644
@@ -1183,6 +1183,7 @@ static void e1000e_tx_hwtstamp_work(struct work_struct *work)
 	struct e1000_hw *hw = &adapter->hw;
 
 	if (er32(TSYNCTXCTL) & E1000_TSYNCTXCTL_VALID) {
+		struct sk_buff *skb = adapter->tx_hwtstamp_skb;
 		struct skb_shared_hwtstamps shhwtstamps;
 		u64 txstmp;
 
@@ -1191,9 +1192,14 @@ static void e1000e_tx_hwtstamp_work(struct work_struct *work)
 
 		e1000e_systim_to_hwtstamp(adapter, &shhwtstamps, txstmp);
 
-		skb_tstamp_tx(adapter->tx_hwtstamp_skb, &shhwtstamps);
-		dev_kfree_skb_any(adapter->tx_hwtstamp_skb);
+		/* Clear the global tx_hwtstamp_skb pointer and force writes
+		 * prior to notifying the stack of a Tx timestamp.
+		 */
 		adapter->tx_hwtstamp_skb = NULL;
+		wmb(); /* force write prior to skb_tstamp_tx */
+
+		skb_tstamp_tx(skb, &shhwtstamps);
+		dev_kfree_skb_any(skb);
 	} else if (time_after(jiffies, adapter->tx_hwtstamp_start
 			      + adapter->tx_timeout_factor * HZ)) {
 		dev_kfree_skb_any(adapter->tx_hwtstamp_skb);
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#2 lore raw

PATCH v4 1/15 e1000e: fix race condition around skb_tstamp_tx()

Quoted 33 lines

From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 01/15] e1000e: fix race condition around skb_tstamp_tx()

The e1000e driver and related hardware has a limitation on Tx PTP packets which requires we limit to timestamping a single packet at once. We do this by verifying that we never request a new Tx timestamp while we still have a tx_hwtstamp_skb pointer.

Unfortunately the driver suffers from a race condition around this. The tx_hwtstamp_skb pointer is not set to NULL until after skb_tstamp_tx() is called. This function notifies the stack and applications of a new timestamp. Even a well behaved application that only sends a new request when the first one is finished might be woken up and possibly send a packet before we can free the timestamp in the driver again. The result is that we needlessly ignore some Tx timestamp requests in this corner case.

Fix this by assigning the tx_hwtstamp_skb pointer prior to calling skb_tstamp_tx() and use a temporary pointer to hold the timestamped skb until that function finishes. This ensures that the application is not woken up until the driver is ready to begin timestamping a new packet.

This ensures that well behaved applications do not accidentally race with condition to skip Tx timestamps. Obviously an application which sends multiple Tx timestamp requests at once will still only timestamp one packet at a time. Unfortunately there is nothing we can do about this.

Quoted 6 lines
---
 drivers/net/ethernet/intel/e1000e/netdev.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
#3 lore raw

PATCH v4 2/15 i40e: fix race condition with PTP_TX_IN_PROGRESS bits

In reply to Jacob Keller

Hardware related to the i40e driver has a limitation on Tx PTP packets. This requires us to limit the driver to timestamping a single packet at once. This is done using a state bitlock which enforces that only one timestamp request is honored at a time.

Unfortunately this suffers from a race condition. The bit lock is not cleared until after skb_tstamp_tx() is called notifying applications of a new Tx timestamp. Even a well behaved application sending only one packet at a time and waiting for a response can wake up and send a new timestamped packet request before the bit lock is cleared. This results in needlessly dropping some Tx timestamp requests.

We can fix this by unlocking the state bit as soon as we read the Timestamp register, as this is the first point at which it is safe to timestamp another packet.

To avoid issues with the skb pointer, we'll use a copy of the pointer and set the global variable in the driver structure to NULL first. This ensures that the next timestamp request does not modify our local copy of the skb pointer.

Now, a well behaved application which has at most one outstanding timestamp request will not accidentally race with the driver unlock bit. Obviously an application attempting to timestamp faster than one request at a time will have some timestamp requests skipped. Unfortunately there is nothing we can do about that.

---
 drivers/net/ethernet/intel/i40e/i40e_ptp.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/i40e/i40e_ptp.c +11 -3
index 18c1cc08da97..5a4854e9fd29 100644
@@ -338,6 +338,7 @@ void i40e_ptp_rx_hang(struct i40e_vsi *vsi)
 void i40e_ptp_tx_hwtstamp(struct i40e_pf *pf)
 {
 	struct skb_shared_hwtstamps shhwtstamps;
+	struct sk_buff *skb = pf->ptp_tx_skb;
 	struct i40e_hw *hw = &pf->hw;
 	u32 hi, lo;
 	u64 ns;
@@ -353,12 +354,19 @@ void i40e_ptp_tx_hwtstamp(struct i40e_pf *pf)
 	hi = rd32(hw, I40E_PRTTSYN_TXTIME_H);
 
 	ns = (((u64)hi) << 32) | lo;
-
 	i40e_ptp_convert_to_hwtstamp(&shhwtstamps, ns);
-	skb_tstamp_tx(pf->ptp_tx_skb, &shhwtstamps);
-	dev_kfree_skb_any(pf->ptp_tx_skb);
+
+	/* Clear the bit lock as soon as possible after reading the register,
+	 * and prior to notifying the stack via skb_tstamp_tx(). Otherwise
+	 * applications might wake up and attempt to request another transmit
+	 * timestamp prior to the bit lock being cleared.
+	 */
 	pf->ptp_tx_skb = NULL;
 	clear_bit_unlock(__I40E_PTP_TX_IN_PROGRESS, pf->state);
+
+	/* Notify the stack and free the skb after we've unlocked */
+	skb_tstamp_tx(skb, &shhwtstamps);
+	dev_kfree_skb_any(skb);
 }
 
 /**
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#4 lore raw

PATCH v4 2/15 i40e: fix race condition with PTP_TX_IN_PROGRESS bits

Quoted 35 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 02/15] i40e: fix race condition with PTP_TX_IN_PROGRESS bits

Hardware related to the i40e driver has a limitation on Tx PTP packets. This requires us to limit the driver to timestamping a single packet at once. This is done using a state bitlock which enforces that only one timestamp request is honored at a time.

Unfortunately this suffers from a race condition. The bit lock is not cleared until after skb_tstamp_tx() is called notifying applications of a new Tx timestamp. Even a well behaved application sending only one packet at a time and waiting for a response can wake up and send a new timestamped packet request before the bit lock is cleared. This results in needlessly dropping some Tx timestamp requests.

We can fix this by unlocking the state bit as soon as we read the Timestamp register, as this is the first point at which it is safe to timestamp another packet.

To avoid issues with the skb pointer, we'll use a copy of the pointer and set the global variable in the driver structure to NULL first. This ensures that the next timestamp request does not modify our local copy of the skb pointer.

Now, a well behaved application which has at most one outstanding timestamp request will not accidentally race with the driver unlock bit. Obviously an application attempting to timestamp faster than one request at a time will have some timestamp requests skipped. Unfortunately there is nothing we can do about that.

Quoted 5 lines
---
 drivers/net/ethernet/intel/i40e/i40e_ptp.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)
#5 lore raw

PATCH v4 3/15 igb: fix race condition with PTP_TX_IN_PROGRESS bits

In reply to Jacob Keller

Hardware related to the igb driver has a limiation of only handling one Tx timestamp at a time. Thus, the driver uses a state bit lock to enforce that only one timestamp request is honored at a time.

Unfortunately this suffers from a simple race condition. The bit lock is not cleared until after skb_tstamp_tx() is called notifying the stack of a new Tx timestamp. Even a well behaved application which sends only one timestamp request at once and waits for a response might wake up and send a new packet before the bit lock is cleared. This results in needlessly dropping some Tx timestamp requests.

We can fix this by unlocking the state bit as soon as we read the Timestamp register, as this is the first point at which it is safe to unlock.

To avoid issues with the skb pointer, we'll use a copy of the pointer and set the global variable in the driver structure to NULL first. This ensures that the next timestamp request does not modify our local copy of the skb pointer.

This ensures that well behaved applications do not accidentally race with the unlock bit. Obviously an application which sends multiple Tx timestamp requests at once will still only timestamp one packet at a time. Unfortunately there is nothing we can do about this.

---
 drivers/net/ethernet/intel/igb/igb_ptp.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
drivers/net/ethernet/intel/igb/igb_ptp.c +10 -2
index 7a3fd4d74592..a2da5738bfb5 100644
@@ -721,6 +721,7 @@ void igb_ptp_rx_hang(struct igb_adapter *adapter)
  **/
 static void igb_ptp_tx_hwtstamp(struct igb_adapter *adapter)
 {
+	struct sk_buff *skb = adapter->ptp_tx_skb;
 	struct e1000_hw *hw = &adapter->hw;
 	struct skb_shared_hwtstamps shhwtstamps;
 	u64 regval;
@@ -748,10 +749,17 @@ static void igb_ptp_tx_hwtstamp(struct igb_adapter *adapter)
 	shhwtstamps.hwtstamp =
 		ktime_add_ns(shhwtstamps.hwtstamp, adjust);
 
-	skb_tstamp_tx(adapter->ptp_tx_skb, &shhwtstamps);
-	dev_kfree_skb_any(adapter->ptp_tx_skb);
+	/* Clear the lock early before calling skb_tstamp_tx so that
+	 * applications are not woken up before the lock bit is clear. We use
+	 * a copy of the skb pointer to ensure other threads can't change it
+	 * while we're notifying the stack.
+	 */
 	adapter->ptp_tx_skb = NULL;
 	clear_bit_unlock(__IGB_PTP_TX_IN_PROGRESS, &adapter->state);
+
+	/* Notify the stack and free the skb after we've unlocked */
+	skb_tstamp_tx(skb, &shhwtstamps);
+	dev_kfree_skb_any(skb);
 }
 
 /**
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#6 lore raw

PATCH v4 3/15 igb: fix race condition with PTP_TX_IN_PROGRESS bits

Quoted 33 lines

From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 03/15] igb: fix race condition with PTP_TX_IN_PROGRESS bits

Hardware related to the igb driver has a limiation of only handling one Tx timestamp at a time. Thus, the driver uses a state bit lock to enforce that only one timestamp request is honored at a time.

Unfortunately this suffers from a simple race condition. The bit lock is not cleared until after skb_tstamp_tx() is called notifying the stack of a new Tx timestamp. Even a well behaved application which sends only one timestamp request at once and waits for a response might wake up and send a new packet before the bit lock is cleared. This results in needlessly dropping some Tx timestamp requests.

We can fix this by unlocking the state bit as soon as we read the Timestamp register, as this is the first point at which it is safe to unlock.

To avoid issues with the skb pointer, we'll use a copy of the pointer and set the global variable in the driver structure to NULL first. This ensures that the next timestamp request does not modify our local copy of the skb pointer.

This ensures that well behaved applications do not accidentally race with the unlock bit. Obviously an application which sends multiple Tx timestamp requests at once will still only timestamp one packet at a time. Unfortunately there is nothing we can do about this.

Quoted 6 lines
---
 drivers/net/ethernet/intel/igb/igb_ptp.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
#7 lore raw

PATCH v4 4/15 ixgbe: fix race condition with PTP_TX_IN_PROGRESS bits

In reply to Jacob Keller

Hardware related to the ixgbe driver is limited to handling a single Tx timestamp request at a time. Thus, the driver ignores requests for Tx timestamp while waiting for the current request to finish. It usess a state bit lock which enforces that only one timestamp request is honored at a time.

Unfortunately this suffers from a simple race condition. The bit lock is not cleared until after skb_tstamp_tx() is called notifying applications of a new Tx timestamp. Even a well behaved application sending only one packet at a time and waiting for a response can wake up and send a new packet before the bit lock is cleared. This results in needlessly dropping some Tx timestamp requests.

We can fix this by unlocking the state bit as soon as we read the Timestamp register, as this is the first point at which it is safe to unlock.

To avoid issues with the skb pointer, we'll use a copy of the pointer and set the global variable in the driver structure to NULL first. This ensures that the next timestamp request does not modify our local copy of the skb pointer.

This ensures that well behaved applications do not accidentally race with the unlock bit. Obviously an application which sends multiple Tx timestamp requests at once will still only timestamp one packet at a time. Unfortunately there is nothing we can do about this.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c +12 -3
index ef0635e0918c..9270e6f4fcff 100644
@@ -672,17 +672,26 @@ static void ixgbe_ptp_clear_tx_timestamp(struct ixgbe_adapter *adapter)
  */
 static void ixgbe_ptp_tx_hwtstamp(struct ixgbe_adapter *adapter)
 {
+	struct sk_buff *skb = adapter->ptp_tx_skb;
 	struct ixgbe_hw *hw = &adapter->hw;
 	struct skb_shared_hwtstamps shhwtstamps;
 	u64 regval = 0;
 
 	regval |= (u64)IXGBE_READ_REG(hw, IXGBE_TXSTMPL);
 	regval |= (u64)IXGBE_READ_REG(hw, IXGBE_TXSTMPH) << 32;
-
 	ixgbe_ptp_convert_to_hwtstamp(adapter, &shhwtstamps, regval);
-	skb_tstamp_tx(adapter->ptp_tx_skb, &shhwtstamps);
 
-	ixgbe_ptp_clear_tx_timestamp(adapter);
+	/* Handle cleanup of the ptp_tx_skb ourselves, and unlock the state
+	 * bit prior to notifying the stack via skb_tstamp_tx(). This prevents
+	 * well behaved applications from attempting to timestamp again prior
+	 * to the lock bit being clear.
+	 */
+	adapter->ptp_tx_skb = NULL;
+	clear_bit_unlock(__IXGBE_PTP_TX_IN_PROGRESS, &adapter->state);
+
+	/* Notify the stack and then free the skb after we've unlocked */
+	skb_tstamp_tx(skb, &shhwtstamps);
+	dev_kfree_skb_any(skb);
 }
 
 /**
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#8 lore raw

PATCH v4 4/15 ixgbe: fix race condition with PTP_TX_IN_PROGRESS bits

Quoted 33 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 04/15] ixgbe: fix race condition with PTP_TX_IN_PROGRESS bits

Hardware related to the ixgbe driver is limited to handling a single Tx timestamp request at a time. Thus, the driver ignores requests for Tx timestamp while waiting for the current request to finish. It usess a state bit lock which enforces that only one timestamp request is honored at a time.

Unfortunately this suffers from a simple race condition. The bit lock is not cleared until after skb_tstamp_tx() is called notifying applications of a new Tx timestamp. Even a well behaved application sending only one packet at a time and waiting for a response can wake up and send a new packet before the bit lock is cleared. This results in needlessly dropping some Tx timestamp requests.

We can fix this by unlocking the state bit as soon as we read the Timestamp register, as this is the first point at which it is safe to unlock.

To avoid issues with the skb pointer, we'll use a copy of the pointer and set the global variable in the driver structure to NULL first. This ensures that the next timestamp request does not modify our local copy of the skb pointer.

This ensures that well behaved applications do not accidentally race with the unlock bit. Obviously an application which sends multiple Tx timestamp requests at once will still only timestamp one packet at a time. Unfortunately there is nothing we can do about this.

Quoted 5 lines
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
#9 lore raw

PATCH v4 5/15 i40e: avoid permanent lock of *_PTP_TX_IN_PROGRESS

In reply to Jacob Keller

The i40e driver uses a bit lock to indicate when a Tx timestamp is in progress to avoid attempting to timestamp multiple packets at once. This is required because hardware only has registers to handle one request at a time.

There is a corner case where we failed to cleanup the bit lock after a failed transmit. This can potentially result in a state bit being locked forever.

Add some cleanup code to i40e_xmit_frame_ring to check and make sure we cleanup incase of these failures. We also modify i40e_tx_map to return an error code indication DMA failure.

---
 drivers/net/ethernet/intel/i40e/i40e_txrx.c | 26 ++++++++++++++++++++------
 1 file changed, 20 insertions(+), 6 deletions(-)
drivers/net/ethernet/intel/i40e/i40e_txrx.c +20 -6
index 29321a6167a6..19984be0f70c 100644
@@ -2932,10 +2932,12 @@ bool __i40e_chk_linearize(struct sk_buff *skb)
  * @hdr_len:  size of the packet header
  * @td_cmd:   the command field in the descriptor
  * @td_offset: offset for checksum or crc
+ *
+ * Returns 0 on success, -1 on failure to DMA
  **/
-static inline void i40e_tx_map(struct i40e_ring *tx_ring, struct sk_buff *skb,
-			       struct i40e_tx_buffer *first, u32 tx_flags,
-			       const u8 hdr_len, u32 td_cmd, u32 td_offset)
+static inline int i40e_tx_map(struct i40e_ring *tx_ring, struct sk_buff *skb,
+			      struct i40e_tx_buffer *first, u32 tx_flags,
+			      const u8 hdr_len, u32 td_cmd, u32 td_offset)
 {
 	unsigned int data_len = skb->data_len;
 	unsigned int size = skb_headlen(skb);
@@ -3093,7 +3095,7 @@ static inline void i40e_tx_map(struct i40e_ring *tx_ring, struct sk_buff *skb,
 		mmiowb();
 	}
 
-	return;
+	return 0;
 
 dma_error:
 	dev_info(tx_ring->dev, "TX DMA map failed\n");
@@ -3110,6 +3112,8 @@ static inline void i40e_tx_map(struct i40e_ring *tx_ring, struct sk_buff *skb,
 	}
 
 	tx_ring->next_to_use = i;
+
+	return -1;
 }
 
 /**
@@ -3210,8 +3214,9 @@ static netdev_tx_t i40e_xmit_frame_ring(struct sk_buff *skb,
 	 */
 	i40e_atr(tx_ring, skb, tx_flags);
 
-	i40e_tx_map(tx_ring, skb, first, tx_flags, hdr_len,
-		    td_cmd, td_offset);
+	if (i40e_tx_map(tx_ring, skb, first, tx_flags, hdr_len,
+			td_cmd, td_offset))
+		goto cleanup_tx_tstamp;
 
 	return NETDEV_TX_OK;
 
@@ -3219,6 +3224,15 @@ static netdev_tx_t i40e_xmit_frame_ring(struct sk_buff *skb,
 	i40e_trace(xmit_frame_ring_drop, first->skb, tx_ring);
 	dev_kfree_skb_any(first->skb);
 	first->skb = NULL;
+cleanup_tx_tstamp:
+	if (unlikely(tx_flags & I40E_TX_FLAGS_TSYN)) {
+		struct i40e_pf *pf = i40e_netdev_to_pf(tx_ring->netdev);
+
+		dev_kfree_skb_any(pf->ptp_tx_skb);
+		pf->ptp_tx_skb = NULL;
+		clear_bit_unlock(__I40E_PTP_TX_IN_PROGRESS, pf->state);
+	}
+
 	return NETDEV_TX_OK;
 }
 
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#10 lore raw

PATCH v4 5/15 i40e: avoid permanent lock of *_PTP_TX_IN_PROGRESS

Quoted 20 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 05/15] i40e: avoid permanent lock of *_PTP_TX_IN_PROGRESS

The i40e driver uses a bit lock to indicate when a Tx timestamp is in progress to avoid attempting to timestamp multiple packets at once. This is required because hardware only has registers to handle one request at a time.

There is a corner case where we failed to cleanup the bit lock after a failed transmit. This can potentially result in a state bit being locked forever.

Add some cleanup code to i40e_xmit_frame_ring to check and make sure we cleanup incase of these failures. We also modify i40e_tx_map to return an error code indication DMA failure.

Quoted 6 lines
---
 drivers/net/ethernet/intel/i40e/i40e_txrx.c | 26 ++++++++++++++++++++--
----
 1 file changed, 20 insertions(+), 6 deletions(-)
#11 lore raw

PATCH v4 6/15 igb: avoid permanent lock of *_PTP_TX_IN_PROGRESS

In reply to Jacob Keller

The igb driver uses a state bit lock to avoid handling more than one Tx timestamp request at once. This is required because hardware is limited to a single set of registers for Tx timestamps.

The state bit lock is not properly cleaned up during igb_xmit_frame_ring() if the transmit fails such as due to DMA or TSO failure. In some hardware this results in blocking timestamps until the service task times out. In other hardware this results in a permanent lock of the timestamp bit because we never receive an interrupt indicating the timestamp occurred, since indeed the packet was never transmitted.

Fix this by checking for DMA and TSO errors in igb_xmit_frame_ring() and properly cleaning up after ourselves when these occur.

---
 drivers/net/ethernet/intel/igb/igb_main.c | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)
drivers/net/ethernet/intel/igb/igb_main.c +18 -5
index dbb4e3cf7dbf..978908f426a7 100644
@@ -5201,9 +5201,9 @@ static inline int igb_maybe_stop_tx(struct igb_ring *tx_ring, const u16 size)
 	return __igb_maybe_stop_tx(tx_ring, size);
 }
 
-static void igb_tx_map(struct igb_ring *tx_ring,
-		       struct igb_tx_buffer *first,
-		       const u8 hdr_len)
+static int igb_tx_map(struct igb_ring *tx_ring,
+		      struct igb_tx_buffer *first,
+		      const u8 hdr_len)
 {
 	struct sk_buff *skb = first->skb;
 	struct igb_tx_buffer *tx_buffer;
@@ -5314,7 +5314,7 @@ static void igb_tx_map(struct igb_ring *tx_ring,
 		 */
 		mmiowb();
 	}
-	return;
+	return 0;
 
 dma_error:
 	dev_err(tx_ring->dev, "TX DMA map failed\n");
@@ -5345,6 +5345,8 @@ static void igb_tx_map(struct igb_ring *tx_ring,
 	tx_buffer->skb = NULL;
 
 	tx_ring->next_to_use = i;
+
+	return -1;
 }
 
 netdev_tx_t igb_xmit_frame_ring(struct sk_buff *skb,
@@ -5410,13 +5412,24 @@ netdev_tx_t igb_xmit_frame_ring(struct sk_buff *skb,
 	else if (!tso)
 		igb_tx_csum(tx_ring, first);
 
-	igb_tx_map(tx_ring, first, hdr_len);
+	if (igb_tx_map(tx_ring, first, hdr_len))
+		goto cleanup_tx_tstamp;
 
 	return NETDEV_TX_OK;
 
 out_drop:
 	dev_kfree_skb_any(first->skb);
 	first->skb = NULL;
+cleanup_tx_tstamp:
+	if (unlikely(tx_flags & IGB_TX_FLAGS_TSTAMP)) {
+		struct igb_adapter *adapter = netdev_priv(tx_ring->netdev);
+
+		dev_kfree_skb_any(adapter->ptp_tx_skb);
+		adapter->ptp_tx_skb = NULL;
+		if (adapter->hw.mac.type == e1000_82576)
+			cancel_work_sync(&adapter->ptp_tx_work);
+		clear_bit_unlock(__IGB_PTP_TX_IN_PROGRESS, &adapter->state);
+	}
 
 	return NETDEV_TX_OK;
 }
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#12 lore raw

PATCH v4 6/15 igb: avoid permanent lock of *_PTP_TX_IN_PROGRESS

Quoted 20 lines

From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 06/15] igb: avoid permanent lock of *_PTP_TX_IN_PROGRESS

The igb driver uses a state bit lock to avoid handling more than one Tx timestamp request at once. This is required because hardware is limited to a single set of registers for Tx timestamps.

The state bit lock is not properly cleaned up during igb_xmit_frame_ring() if the transmit fails such as due to DMA or TSO failure. In some hardware this results in blocking timestamps until the service task times out. In other hardware this results in a permanent lock of the timestamp bit because we never receive an interrupt indicating the timestamp occurred, since indeed the packet was never transmitted.

Quoted 9 lines

Fix this by checking for DMA and TSO errors in igb_xmit_frame_ring() and properly cleaning up after ourselves when these occur.

---
 drivers/net/ethernet/intel/igb/igb_main.c | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)
#13 lore raw

PATCH v4 7/15 ixgbe: avoid permanent lock of *_PTP_TX_IN_PROGRESS

In reply to Jacob Keller

The ixgbe driver uses a state bit lock to avoid handling more than one Tx timestamp request at once. This is required because hardware is limited to a single set of registers for Tx timestamps.

The state bit lock is not properly cleaned up during ixgbe_xmit_frame_ring() if the transmit fails such as due to DMA or TSO failure. In some hardware this results in blocking timestamps until the service task times out. In other hardware this results in a permanent lock of the timestamp bit because we never receive an interrupt indicating the timestamp occurred, since indeed the packet was never transmitted.

Fix this by checking for DMA and TSO errors in ixgbe_xmit_frame_ring() and properly cleaning up after ourselves when these occur.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -5
index a3b7d18924a5..f93330bc7b97 100644
@@ -7885,9 +7885,9 @@ static inline int ixgbe_maybe_stop_tx(struct ixgbe_ring *tx_ring, u16 size)
 #define IXGBE_TXD_CMD (IXGBE_TXD_CMD_EOP | \
 		       IXGBE_TXD_CMD_RS)
 
-static void ixgbe_tx_map(struct ixgbe_ring *tx_ring,
-			 struct ixgbe_tx_buffer *first,
-			 const u8 hdr_len)
+static int ixgbe_tx_map(struct ixgbe_ring *tx_ring,
+			struct ixgbe_tx_buffer *first,
+			const u8 hdr_len)
 {
 	struct sk_buff *skb = first->skb;
 	struct ixgbe_tx_buffer *tx_buffer;
@@ -8014,7 +8014,7 @@ static void ixgbe_tx_map(struct ixgbe_ring *tx_ring,
 		mmiowb();
 	}
 
-	return;
+	return 0;
 dma_error:
 	dev_err(tx_ring->dev, "TX DMA map failed\n");
 	tx_buffer = &tx_ring->tx_buffer_info[i];
@@ -8044,6 +8044,8 @@ static void ixgbe_tx_map(struct ixgbe_ring *tx_ring,
 	first->skb = NULL;
 
 	tx_ring->next_to_use = i;
+
+	return -1;
 }
 
 static void ixgbe_atr(struct ixgbe_ring *ring,
@@ -8416,13 +8418,21 @@ netdev_tx_t ixgbe_xmit_frame_ring(struct sk_buff *skb,
 #ifdef IXGBE_FCOE
 xmit_fcoe:
 #endif /* IXGBE_FCOE */
-	ixgbe_tx_map(tx_ring, first, hdr_len);
+	if (ixgbe_tx_map(tx_ring, first, hdr_len))
+		goto cleanup_tx_timestamp;
 
 	return NETDEV_TX_OK;
 
 out_drop:
 	dev_kfree_skb_any(first->skb);
 	first->skb = NULL;
+cleanup_tx_timestamp:
+	if (unlikely(tx_flags & IXGBE_TX_FLAGS_TSTAMP)) {
+		dev_kfree_skb_any(adapter->ptp_tx_skb);
+		adapter->ptp_tx_skb = NULL;
+		cancel_work_sync(&adapter->ptp_tx_work);
+		clear_bit_unlock(__IXGBE_PTP_TX_IN_PROGRESS, &adapter->state);
+	}
 
 	return NETDEV_TX_OK;
 }
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#14 lore raw

PATCH v4 7/15 ixgbe: avoid permanent lock of *_PTP_TX_IN_PROGRESS

Quoted 20 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 07/15] ixgbe: avoid permanent lock of *_PTP_TX_IN_PROGRESS

The ixgbe driver uses a state bit lock to avoid handling more than one Tx timestamp request at once. This is required because hardware is limited to a single set of registers for Tx timestamps.

The state bit lock is not properly cleaned up during ixgbe_xmit_frame_ring() if the transmit fails such as due to DMA or TSO failure. In some hardware this results in blocking timestamps until the service task times out. In other hardware this results in a permanent lock of the timestamp bit because we never receive an interrupt indicating the timestamp occurred, since indeed the packet was never transmitted.

Quoted 8 lines

Fix this by checking for DMA and TSO errors in ixgbe_xmit_frame_ring() and properly cleaning up after ourselves when these occur.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)
#15 lore raw

PATCH v4 8/15 e1000e: add statistic indicating number of skipped Tx timestamps

In reply to Jacob Keller

The e1000e driver can only handle one Tx timestamp request at a time. This means it is possible for an application timestamp request to be ignored.

There is no easy way for an administrator to determine if this occurred. Add a new statistic which tracks this, tx_hwtstamp_skipped.

---
 drivers/net/ethernet/intel/e1000e/e1000.h   |  1 +
 drivers/net/ethernet/intel/e1000e/ethtool.c |  1 +
 drivers/net/ethernet/intel/e1000e/netdev.c  | 17 ++++++++++-------
 3 files changed, 12 insertions(+), 7 deletions(-)
drivers/net/ethernet/intel/e1000e/e1000.h +1 -0
index c7c994eb410e..98e68888abb1 100644
@@ -268,6 +268,7 @@ struct e1000_adapter {
 	u32 tx_fifo_size;
 	u32 tx_dma_failed;
 	u32 tx_hwtstamp_timeouts;
+	u32 tx_hwtstamp_skipped;
 
 	/* Rx */
 	bool (*clean_rx)(struct e1000_ring *ring, int *work_done,
drivers/net/ethernet/intel/e1000e/ethtool.c +1 -0
index e23dbd9190d6..c658f6ebf7cb 100644
@@ -105,6 +105,7 @@ static const struct e1000_stats e1000_gstrings_stats[] = {
 	E1000_STAT("uncorr_ecc_errors", uncorr_errors),
 	E1000_STAT("corr_ecc_errors", corr_errors),
 	E1000_STAT("tx_hwtstamp_timeouts", tx_hwtstamp_timeouts),
+	E1000_STAT("tx_hwtstamp_skipped", tx_hwtstamp_skipped),
 };
 
 #define E1000_GLOBAL_STATS_LEN	ARRAY_SIZE(e1000_gstrings_stats)
drivers/net/ethernet/intel/e1000e/netdev.c +10 -7
index ec9a50a72550..42424274853c 100644
@@ -5866,13 +5866,16 @@ static netdev_tx_t e1000_xmit_frame(struct sk_buff *skb,
 			     nr_frags);
 	if (count) {
 		if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) &&
-		    (adapter->flags & FLAG_HAS_HW_TIMESTAMP) &&
-		    !adapter->tx_hwtstamp_skb) {
-			skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
-			tx_flags |= E1000_TX_FLAGS_HWTSTAMP;
-			adapter->tx_hwtstamp_skb = skb_get(skb);
-			adapter->tx_hwtstamp_start = jiffies;
-			schedule_work(&adapter->tx_hwtstamp_work);
+		    (adapter->flags & FLAG_HAS_HW_TIMESTAMP)) {
+			if (!adapter->tx_hwtstamp_skb) {
+				skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
+				tx_flags |= E1000_TX_FLAGS_HWTSTAMP;
+				adapter->tx_hwtstamp_skb = skb_get(skb);
+				adapter->tx_hwtstamp_start = jiffies;
+				schedule_work(&adapter->tx_hwtstamp_work);
+			} else {
+				adapter->tx_hwtstamp_skipped++;
+			}
 		} else {
 			skb_tx_timestamp(skb);
 		}
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#16 lore raw

PATCH v4 8/15 e1000e: add statistic indicating number of skipped Tx timestamps

Quoted 15 lines

From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 08/15] e1000e: add statistic indicating number of skipped Tx timestamps

The e1000e driver can only handle one Tx timestamp request at a time. This means it is possible for an application timestamp request to be ignored.

There is no easy way for an administrator to determine if this occurred. Add a new statistic which tracks this, tx_hwtstamp_skipped.

Quoted 7 lines
---
 drivers/net/ethernet/intel/e1000e/e1000.h   |  1 +
 drivers/net/ethernet/intel/e1000e/ethtool.c |  1 +
 drivers/net/ethernet/intel/e1000e/netdev.c  | 17 ++++++++++-------
 3 files changed, 12 insertions(+), 7 deletions(-)
#17 lore raw

PATCH v4 9/15 i40e: add statistic indicating number of skipped Tx timestamps

In reply to Jacob Keller

The i40e driver can only handle one Tx timestamp request at a time. This means it is possible for an application timestamp request to be ignored.

There is no easy way for an administrator to determine if this occurred. Add a new statistic which tracks this, tx_hwtstamp_skipped.

---
 drivers/net/ethernet/intel/i40e/i40e.h         | 1 +
 drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 1 +
 drivers/net/ethernet/intel/i40e/i40e_txrx.c    | 1 +
 3 files changed, 3 insertions(+)
drivers/net/ethernet/intel/i40e/i40e.h +1 -0
index cdde3cc28fb5..aa46ae016539 100644
@@ -506,6 +506,7 @@ struct i40e_pf {
 	struct mutex tmreg_lock; /* Used to protect the SYSTIME registers. */
 	u64 ptp_base_adj;
 	u32 tx_hwtstamp_timeouts;
+	u32 tx_hwtstamp_skipped;
 	u32 rx_hwtstamp_cleared;
 	u32 latch_event_flags;
 	spinlock_t ptp_rx_lock; /* Used to protect Rx timestamp registers. */
drivers/net/ethernet/intel/i40e/i40e_ethtool.c +1 -0
index 7a8eb486b9ea..35a246f05520 100644
@@ -147,6 +147,7 @@ static const struct i40e_stats i40e_gstrings_stats[] = {
 	I40E_PF_STAT("VF_admin_queue_requests", vf_aq_requests),
 	I40E_PF_STAT("arq_overflows", arq_overflows),
 	I40E_PF_STAT("rx_hwtstamp_cleared", rx_hwtstamp_cleared),
+	I40E_PF_STAT("tx_hwtstamp_skipped", tx_hwtstamp_skipped),
 	I40E_PF_STAT("fdir_flush_cnt", fd_flush_cnt),
 	I40E_PF_STAT("fdir_atr_match", stats.fd_atr_match),
 	I40E_PF_STAT("fdir_atr_tunnel_match", stats.fd_atr_tunnel_match),
drivers/net/ethernet/intel/i40e/i40e_txrx.c +1 -0
index 19984be0f70c..c69ee4b0cfe2 100644
@@ -2630,6 +2630,7 @@ static int i40e_tsyn(struct i40e_ring *tx_ring, struct sk_buff *skb,
 		skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
 		pf->ptp_tx_skb = skb_get(skb);
 	} else {
+		pf->tx_hwtstamp_skipped++;
 		return 0;
 	}
 
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#18 lore raw

PATCH v4 9/15 i40e: add statistic indicating number of skipped Tx timestamps

Quoted 15 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 09/15] i40e: add statistic indicating number of skipped Tx timestamps

The i40e driver can only handle one Tx timestamp request at a time. This means it is possible for an application timestamp request to be ignored.

There is no easy way for an administrator to determine if this occurred. Add a new statistic which tracks this, tx_hwtstamp_skipped.

Quoted 6 lines
---
 drivers/net/ethernet/intel/i40e/i40e.h         | 1 +
 drivers/net/ethernet/intel/i40e/i40e_ethtool.c | 1 +
 drivers/net/ethernet/intel/i40e/i40e_txrx.c    | 1 +
 3 files changed, 3 insertions(+)
#19 lore raw

PATCH v4 10/15 igb: add statistic indicating number of skipped Tx timestamps

In reply to Jacob Keller

The igb driver can only handle one Tx timestamp request at a time. This means it is possible for an application timestamp request to be ignored.

There is no easy way for an administrator to determine if this occurred. Add a new statistic which tracks this, tx_hwtstamp_skipped.

---
 drivers/net/ethernet/intel/igb/igb.h         | 1 +
 drivers/net/ethernet/intel/igb/igb_ethtool.c | 1 +
 drivers/net/ethernet/intel/igb/igb_main.c    | 2 ++
 3 files changed, 4 insertions(+)
drivers/net/ethernet/intel/igb/igb.h +1 -0
index bf9bf9056d0c..be35edcf6b08 100644
@@ -563,6 +563,7 @@ struct igb_adapter {
 	struct cyclecounter cc;
 	struct timecounter tc;
 	u32 tx_hwtstamp_timeouts;
+	u32 tx_hwtstamp_skipped;
 	u32 rx_hwtstamp_cleared;
 	bool pps_sys_wrap_on;
 
drivers/net/ethernet/intel/igb/igb_ethtool.c +1 -0
index 0efb62db6efd..8730f7cbce68 100644
@@ -90,6 +90,7 @@ static const struct igb_stats igb_gstrings_stats[] = {
 	IGB_STAT("os2bmc_tx_by_host", stats.o2bspc),
 	IGB_STAT("os2bmc_rx_by_host", stats.b2ogprc),
 	IGB_STAT("tx_hwtstamp_timeouts", tx_hwtstamp_timeouts),
+	IGB_STAT("tx_hwtstamp_skipped", tx_hwtstamp_skipped),
 	IGB_STAT("rx_hwtstamp_cleared", rx_hwtstamp_cleared),
 };
 
drivers/net/ethernet/intel/igb/igb_main.c +2 -0
index 978908f426a7..21b455bfb4ca 100644
@@ -5392,6 +5392,8 @@ netdev_tx_t igb_xmit_frame_ring(struct sk_buff *skb,
 			adapter->ptp_tx_start = jiffies;
 			if (adapter->hw.mac.type == e1000_82576)
 				schedule_work(&adapter->ptp_tx_work);
+		} else {
+			adapter->tx_hwtstamp_skipped++;
 		}
 	}
 
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#20 lore raw

PATCH v4 10/15 igb: add statistic indicating number of skipped Tx timestamps

Quoted 15 lines

From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 10/15] igb: add statistic indicating number of skipped Tx timestamps

The igb driver can only handle one Tx timestamp request at a time. This means it is possible for an application timestamp request to be ignored.

There is no easy way for an administrator to determine if this occurred. Add a new statistic which tracks this, tx_hwtstamp_skipped.

Quoted 7 lines
---
 drivers/net/ethernet/intel/igb/igb.h         | 1 +
 drivers/net/ethernet/intel/igb/igb_ethtool.c | 1 +
 drivers/net/ethernet/intel/igb/igb_main.c    | 2 ++
 3 files changed, 4 insertions(+)
#21 lore raw

PATCH v4 11/15 ixgbe: add statistic indicating number of skipped Tx timestamps

In reply to Jacob Keller

The ixgbe driver can only handle one Tx timestamp request at a time. This means it is possible for an application timestamp request to be ignored.

There is no easy way for an administrator to determine if this occurred. Add a new statistic which tracks this, tx_hwtstamp_skipped.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe.h         |  1 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c |  3 +++
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c    | 21 ++++++++++++---------
 3 files changed, 16 insertions(+), 9 deletions(-)
drivers/net/ethernet/intel/ixgbe/ixgbe.h +1 -0
index 76263762bea1..eb36106218ad 100644
@@ -733,6 +733,7 @@ struct ixgbe_adapter {
 	struct timecounter hw_tc;
 	u32 base_incval;
 	u32 tx_hwtstamp_timeouts;
+	u32 tx_hwtstamp_skipped;
 	u32 rx_hwtstamp_cleared;
 	void (*ptp_setup_sdp)(struct ixgbe_adapter *);
 
drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c +3 -0
index cced74dd5a63..0b75d304b009 100644
@@ -111,6 +111,9 @@ static const struct ixgbe_stats ixgbe_gstrings_stats[] = {
 	{"os2bmc_tx_by_bmc", IXGBE_STAT(stats.b2ospc)},
 	{"os2bmc_tx_by_host", IXGBE_STAT(stats.o2bspc)},
 	{"os2bmc_rx_by_host", IXGBE_STAT(stats.b2ogprc)},
+	{"tx_hwtstamp_timeouts", IXGBE_STAT(tx_hwtstamp_timeouts)},
+	{"tx_hwtstamp_skipped", IXGBE_STAT(tx_hwtstamp_skipped)},
+	{"rx_hwtstamp_cleared", IXGBE_STAT(rx_hwtstamp_cleared)},
 #ifdef IXGBE_FCOE
 	{"fcoe_bad_fccrc", IXGBE_STAT(stats.fccrc)},
 	{"rx_fcoe_dropped", IXGBE_STAT(stats.fcoerpdc)},
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +12 -9
index f93330bc7b97..85a10458ab59 100644
@@ -8346,16 +8346,19 @@ netdev_tx_t ixgbe_xmit_frame_ring(struct sk_buff *skb,
 	protocol = vlan_get_protocol(skb);
 
 	if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) &&
-	    adapter->ptp_clock &&
-	    !test_and_set_bit_lock(__IXGBE_PTP_TX_IN_PROGRESS,
-				   &adapter->state)) {
-		skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
-		tx_flags |= IXGBE_TX_FLAGS_TSTAMP;
+	    adapter->ptp_clock) {
+		if (!test_and_set_bit_lock(__IXGBE_PTP_TX_IN_PROGRESS,
+					   &adapter->state)) {
+			skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
+			tx_flags |= IXGBE_TX_FLAGS_TSTAMP;
 
-		/* schedule check for Tx timestamp */
-		adapter->ptp_tx_skb = skb_get(skb);
-		adapter->ptp_tx_start = jiffies;
-		schedule_work(&adapter->ptp_tx_work);
+			/* schedule check for Tx timestamp */
+			adapter->ptp_tx_skb = skb_get(skb);
+			adapter->ptp_tx_start = jiffies;
+			schedule_work(&adapter->ptp_tx_work);
+		} else {
+			adapter->tx_hwtstamp_skipped++;
+		}
 	}
 
 	skb_tx_timestamp(skb);
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#22 lore raw

PATCH v4 12/15 i40e: use pf data structure directly in i40e_ptp_rx_hang

In reply to Jacob Keller

There's no reason to pass a *vsi pointer if we already have the *pf pointer in the only location where we call this function. Lets update the signature and directly pass the *pf data structure pointer.

---
 drivers/net/ethernet/intel/i40e/i40e.h      | 2 +-
 drivers/net/ethernet/intel/i40e/i40e_main.c | 2 +-
 drivers/net/ethernet/intel/i40e/i40e_ptp.c  | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)
drivers/net/ethernet/intel/i40e/i40e.h +1 -1
index aa46ae016539..f4465afe1fe1 100644
@@ -956,7 +956,7 @@ bool i40e_dcb_need_reconfig(struct i40e_pf *pf,
 			    struct i40e_dcbx_config *old_cfg,
 			    struct i40e_dcbx_config *new_cfg);
 #endif /* CONFIG_I40E_DCB */
-void i40e_ptp_rx_hang(struct i40e_vsi *vsi);
+void i40e_ptp_rx_hang(struct i40e_pf *pf);
 void i40e_ptp_tx_hwtstamp(struct i40e_pf *pf);
 void i40e_ptp_rx_hwtstamp(struct i40e_pf *pf, struct sk_buff *skb, u8 index);
 void i40e_ptp_set_increment(struct i40e_pf *pf);
drivers/net/ethernet/intel/i40e/i40e_main.c +1 -1
index d5c9c9e06ff5..c019dec988e3 100644
@@ -6372,7 +6372,7 @@ static void i40e_watchdog_subtask(struct i40e_pf *pf)
 				i40e_update_veb_stats(pf->veb[i]);
 	}
 
-	i40e_ptp_rx_hang(pf->vsi[pf->lan_vsi]);
+	i40e_ptp_rx_hang(pf);
 }
 
 /**
drivers/net/ethernet/intel/i40e/i40e_ptp.c +2 -2
index 5a4854e9fd29..876ea169816a 100644
@@ -269,6 +269,7 @@ static u32 i40e_ptp_get_rx_events(struct i40e_pf *pf)
 
 /**
  * i40e_ptp_rx_hang - Detect error case when Rx timestamp registers are hung
+ * @pf: The PF private data structure
  * @vsi: The VSI with the rings relevant to 1588
  *
  * This watchdog task is scheduled to detect error case where hardware has
@@ -276,9 +277,8 @@ static u32 i40e_ptp_get_rx_events(struct i40e_pf *pf)
  * particular error is rare but leaves the device in a state unable to timestamp
  * any future packets.
  **/
-void i40e_ptp_rx_hang(struct i40e_vsi *vsi)
+void i40e_ptp_rx_hang(struct i40e_pf *pf)
 {
-	struct i40e_pf *pf = vsi->back;
 	struct i40e_hw *hw = &pf->hw;
 	unsigned int i, cleared = 0;
 
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#23 lore raw

PATCH v4 12/15 i40e: use pf data structure directly in i40e_ptp_rx_hang

Quoted 13 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 12/15] i40e: use pf data structure directly in i40e_ptp_rx_hang

There's no reason to pass a *vsi pointer if we already have the *pf pointer in the only location where we call this function. Lets update the signature and directly pass the *pf data structure pointer.

Quoted 6 lines
---
 drivers/net/ethernet/intel/i40e/i40e.h      | 2 +-
 drivers/net/ethernet/intel/i40e/i40e_main.c | 2 +-
drivers/net/ethernet/intel/i40e/i40e_ptp.c  | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)
#24 lore raw

PATCH v4 13/15 i40e: check for Tx timestamp timeouts during watchdog

In reply to Jacob Keller

The i40e driver has logic to handle only one Tx timestamp at a time, using a state bit lock to avoid multiple requests at once.

It may be possible, if incredibly unlikely, that a Tx timestamp event is requested but never completes. Since we use an interrupt scheme to determine when the Tx timestamp occurred we would never clear the state bit in this case.

Add an i40e_ptp_tx_hang() function similar to the already existing i40e_ptp_rx_hang() function. This function runs in the watchdog routine and makes sure we eventually recover from this case instead of permanently disabling Tx timestamps.

Note: there is no currently known way to cause this without hacking the driver code to force it.

---
 drivers/net/ethernet/intel/i40e/i40e.h      |  2 ++
 drivers/net/ethernet/intel/i40e/i40e_main.c |  1 +
 drivers/net/ethernet/intel/i40e/i40e_ptp.c  | 30 +++++++++++++++++++++++++++++
 drivers/net/ethernet/intel/i40e/i40e_txrx.c |  1 +
 4 files changed, 34 insertions(+)
drivers/net/ethernet/intel/i40e/i40e.h +2 -0
index f4465afe1fe1..25bf336c5f38 100644
@@ -502,6 +502,7 @@ struct i40e_pf {
 	struct ptp_clock *ptp_clock;
 	struct ptp_clock_info ptp_caps;
 	struct sk_buff *ptp_tx_skb;
+	unsigned long ptp_tx_start;
 	struct hwtstamp_config tstamp_config;
 	struct mutex tmreg_lock; /* Used to protect the SYSTIME registers. */
 	u64 ptp_base_adj;
@@ -957,6 +958,7 @@ bool i40e_dcb_need_reconfig(struct i40e_pf *pf,
 			    struct i40e_dcbx_config *new_cfg);
 #endif /* CONFIG_I40E_DCB */
 void i40e_ptp_rx_hang(struct i40e_pf *pf);
+void i40e_ptp_tx_hang(struct i40e_pf *pf);
 void i40e_ptp_tx_hwtstamp(struct i40e_pf *pf);
 void i40e_ptp_rx_hwtstamp(struct i40e_pf *pf, struct sk_buff *skb, u8 index);
 void i40e_ptp_set_increment(struct i40e_pf *pf);
drivers/net/ethernet/intel/i40e/i40e_main.c +1 -0
index c019dec988e3..e4eb97832413 100644
@@ -6373,6 +6373,7 @@ static void i40e_watchdog_subtask(struct i40e_pf *pf)
 	}
 
 	i40e_ptp_rx_hang(pf);
+	i40e_ptp_tx_hang(pf);
 }
 
 /**
drivers/net/ethernet/intel/i40e/i40e_ptp.c +30 -0
index 876ea169816a..cd35c4b9a8b0 100644
@@ -327,6 +327,36 @@ void i40e_ptp_rx_hang(struct i40e_pf *pf)
 	pf->rx_hwtstamp_cleared += cleared;
 }
 
+/**
+ * i40e_ptp_tx_hang - Detect error case when Tx timestamp register is hung
+ * @pf: The PF private data structure
+ *
+ * This watchdog task is run periodically to make sure that we clear the Tx
+ * timestamp logic if we don't obtain a timestamp in a reasonable amount of
+ * time. It is unexpected in the normal case but if it occurs it results in
+ * permanently prevent timestamps of future packets
+ **/
+void i40e_ptp_tx_hang(struct i40e_pf *pf)
+{
+	if (!(pf->flags & I40E_FLAG_PTP) || !pf->ptp_tx)
+		return;
+
+	/* Nothing to do if we're not already waiting for a timestamp */
+	if (!test_bit(__I40E_PTP_TX_IN_PROGRESS, pf->state))
+		return;
+
+	/* We already have a handler routine which is run when we are notified
+	 * of a Tx timestamp in the hardware. If we don't get an interrupt
+	 * within a second it is reasonable to assume that we never will.
+	 */
+	if (time_is_before_jiffies(pf->ptp_tx_start + HZ)) {
+		dev_kfree_skb_any(pf->ptp_tx_skb);
+		pf->ptp_tx_skb = NULL;
+		clear_bit_unlock(__I40E_PTP_TX_IN_PROGRESS, pf->state);
+		pf->tx_hwtstamp_timeouts++;
+	}
+}
+
 /**
  * i40e_ptp_tx_hwtstamp - Utility function which returns the Tx timestamp
  * @pf: Board private structure
drivers/net/ethernet/intel/i40e/i40e_txrx.c +1 -0
index c69ee4b0cfe2..c2e9013d05eb 100644
@@ -2628,6 +2628,7 @@ static int i40e_tsyn(struct i40e_ring *tx_ring, struct sk_buff *skb,
 	if (pf->ptp_tx &&
 	    !test_and_set_bit_lock(__I40E_PTP_TX_IN_PROGRESS, pf->state)) {
 		skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
+		pf->ptp_tx_start = jiffies;
 		pf->ptp_tx_skb = skb_get(skb);
 	} else {
 		pf->tx_hwtstamp_skipped++;
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#25 lore raw

PATCH v4 13/15 i40e: check for Tx timestamp timeouts during watchdog

Quoted 25 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 13/15] i40e: check for Tx timestamp timeouts during watchdog

The i40e driver has logic to handle only one Tx timestamp at a time, using a state bit lock to avoid multiple requests at once.

It may be possible, if incredibly unlikely, that a Tx timestamp event is requested but never completes. Since we use an interrupt scheme to determine when the Tx timestamp occurred we would never clear the state bit in this case.

Add an i40e_ptp_tx_hang() function similar to the already existing i40e_ptp_rx_hang() function. This function runs in the watchdog routine and makes sure we eventually recover from this case instead of permanently disabling Tx timestamps.

Note: there is no currently known way to cause this without hacking the driver code to force it.

Quoted 8 lines
---
 drivers/net/ethernet/intel/i40e/i40e.h      |  2 ++
 drivers/net/ethernet/intel/i40e/i40e_main.c |  1 +
drivers/net/ethernet/intel/i40e/i40e_ptp.c  | 30
+++++++++++++++++++++++++++++
drivers/net/ethernet/intel/i40e/i40e_txrx.c |  1 +
 4 files changed, 34 insertions(+)
#26 lore raw

PATCH v4 14/15 igb: check for Tx timestamp timeouts during watchdog

In reply to Jacob Keller

The igb driver has logic to handle only one Tx timestamp at a time, using a state bit lock to avoid multiple requests at once.

It may be possible, if incredibly unlikely, that a Tx timestamp event is requested but never completes. Since we use an interrupt scheme to determine when the Tx timestamp occurred we would never clear the state bit in this case.

Add an igb_ptp_tx_hang() function similar to the already existing igb_ptp_rx_hang() function. This function runs in the watchdog routine and makes sure we eventually recover from this case instead of permanently disabling Tx timestamps.

Note: there is no currently known way to cause this without hacking the driver code to force it.

---
 drivers/net/ethernet/intel/igb/igb.h      |  1 +
 drivers/net/ethernet/intel/igb/igb_main.c |  1 +
 drivers/net/ethernet/intel/igb/igb_ptp.c  | 29 +++++++++++++++++++++++++++++
 3 files changed, 31 insertions(+)
drivers/net/ethernet/intel/igb/igb.h +1 -0
index be35edcf6b08..ff4d9073781a 100644
@@ -677,6 +677,7 @@ void igb_ptp_stop(struct igb_adapter *adapter);
 void igb_ptp_reset(struct igb_adapter *adapter);
 void igb_ptp_suspend(struct igb_adapter *adapter);
 void igb_ptp_rx_hang(struct igb_adapter *adapter);
+void igb_ptp_tx_hang(struct igb_adapter *adapter);
 void igb_ptp_rx_rgtstamp(struct igb_q_vector *q_vector, struct sk_buff *skb);
 void igb_ptp_rx_pktstamp(struct igb_q_vector *q_vector, void *va,
 			 struct sk_buff *skb);
drivers/net/ethernet/intel/igb/igb_main.c +1 -0
index 21b455bfb4ca..eec54d9df06b 100644
@@ -4726,6 +4726,7 @@ static void igb_watchdog_task(struct work_struct *work)
 
 	igb_spoof_check(adapter);
 	igb_ptp_rx_hang(adapter);
+	igb_ptp_tx_hang(adapter);
 
 	/* Check LVMMC register on i350/i354 only */
 	if ((adapter->hw.mac.type == e1000_i350) ||
drivers/net/ethernet/intel/igb/igb_ptp.c +29 -0
index a2da5738bfb5..b2f67c169bd2 100644
@@ -711,6 +711,35 @@ void igb_ptp_rx_hang(struct igb_adapter *adapter)
 	}
 }
 
+/**
+ * igb_ptp_tx_hang - detect error case where Tx timestamp never finishes
+ * @adapter: private network adapter structure
+ */
+void igb_ptp_tx_hang(struct igb_adapter *adapter)
+{
+	bool timeout = time_is_before_jiffies(adapter->ptp_tx_start +
+					      IGB_PTP_TX_TIMEOUT);
+
+	if (!adapter->ptp_tx_skb)
+		return;
+
+	if (!test_bit(__IGB_PTP_TX_IN_PROGRESS, &adapter->state))
+		return;
+
+	/* If we haven't received a timestamp within the timeout, it is
+	 * reasonable to assume that it will never occur, so we can unlock the
+	 * timestamp bit when this occurs.
+	 */
+	if (timeout) {
+		cancel_work_sync(&adapter->ptp_tx_work);
+		dev_kfree_skb_any(adapter->ptp_tx_skb);
+		adapter->ptp_tx_skb = NULL;
+		clear_bit_unlock(__IGB_PTP_TX_IN_PROGRESS, &adapter->state);
+		adapter->tx_hwtstamp_timeouts++;
+		dev_warn(&adapter->pdev->dev, "clearing Tx timestamp hang\n");
+	}
+}
+
 /**
  * igb_ptp_tx_hwtstamp - utility function which checks for TX time stamp
  * @adapter: Board private structure.
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#27 lore raw

PATCH v4 14/15 igb: check for Tx timestamp timeouts during watchdog

Quoted 24 lines

From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 14/15] igb: check for Tx timestamp timeouts during watchdog

The igb driver has logic to handle only one Tx timestamp at a time, using a state bit lock to avoid multiple requests at once.

It may be possible, if incredibly unlikely, that a Tx timestamp event is requested but never completes. Since we use an interrupt scheme to determine when the Tx timestamp occurred we would never clear the state bit in this case.

Add an igb_ptp_tx_hang() function similar to the already existing igb_ptp_rx_hang() function. This function runs in the watchdog routine and makes sure we eventually recover from this case instead of permanently disabling Tx timestamps.

Note: there is no currently known way to cause this without hacking the driver code to force it.

Quoted 7 lines
---
 drivers/net/ethernet/intel/igb/igb.h      |  1 +
 drivers/net/ethernet/intel/igb/igb_main.c |  1 +
 drivers/net/ethernet/intel/igb/igb_ptp.c  | 29
+++++++++++++++++++++++++++++
 3 files changed, 31 insertions(+)
#28 lore raw

PATCH v4 15/15 ixgbe: check for Tx timestamp timeouts during watchdog

In reply to Jacob Keller

The ixgbe driver has logic to handle only one Tx timestamp at a time, using a state bit lock to avoid multiple requests at once.

It may be possible, if incredibly unlikely, that a Tx timestamp event is requested but never completes. Since we use an interrupt scheme to determine when the Tx timestamp occurred we would never clear the state bit in this case.

Add an ixgbe_ptp_tx_hang() function similar to the already existing ixgbe_ptp_rx_hang() function. This function runs in the watchdog routine and makes sure we eventually recover from this case instead of permanently disabling Tx timestamps.

Note: there is no currently known way to cause this without hacking the driver code to force it.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe.h      |  1 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |  1 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c  | 27 +++++++++++++++++++++++++++
 3 files changed, 29 insertions(+)
drivers/net/ethernet/intel/ixgbe/ixgbe.h +1 -0
index eb36106218ad..dd5578756ae0 100644
@@ -961,6 +961,7 @@ void ixgbe_ptp_suspend(struct ixgbe_adapter *adapter);
 void ixgbe_ptp_stop(struct ixgbe_adapter *adapter);
 void ixgbe_ptp_overflow_check(struct ixgbe_adapter *adapter);
 void ixgbe_ptp_rx_hang(struct ixgbe_adapter *adapter);
+void ixgbe_ptp_tx_hang(struct ixgbe_adapter *adapter);
 void ixgbe_ptp_rx_pktstamp(struct ixgbe_q_vector *, struct sk_buff *);
 void ixgbe_ptp_rx_rgtstamp(struct ixgbe_q_vector *, struct sk_buff *skb);
 static inline void ixgbe_ptp_rx_hwtstamp(struct ixgbe_ring *rx_ring,
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -0
index 85a10458ab59..ee20a2bf5514 100644
@@ -7645,6 +7645,7 @@ static void ixgbe_service_task(struct work_struct *work)
 	if (test_bit(__IXGBE_PTP_RUNNING, &adapter->state)) {
 		ixgbe_ptp_overflow_check(adapter);
 		ixgbe_ptp_rx_hang(adapter);
+		ixgbe_ptp_tx_hang(adapter);
 	}
 
 	ixgbe_service_event_complete(adapter);
drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c +27 -0
index 9270e6f4fcff..24f4eaf37727 100644
@@ -662,6 +662,33 @@ static void ixgbe_ptp_clear_tx_timestamp(struct ixgbe_adapter *adapter)
 	clear_bit_unlock(__IXGBE_PTP_TX_IN_PROGRESS, &adapter->state);
 }
 
+/**
+ * ixgbe_ptp_tx_hang - detect error case where Tx timestamp never finishes
+ * @adapter: private network adapter structure
+ */
+void ixgbe_ptp_tx_hang(struct ixgbe_adapter *adapter)
+{
+	bool timeout = time_is_before_jiffies(adapter->ptp_tx_start +
+					      IXGBE_PTP_TX_TIMEOUT);
+
+	if (!adapter->ptp_tx_skb)
+		return;
+
+	if (!test_bit(__IXGBE_PTP_TX_IN_PROGRESS, &adapter->state))
+		return;
+
+	/* If we haven't received a timestamp within the timeout, it is
+	 * reasonable to assume that it will never occur, so we can unlock the
+	 * timestamp bit when this occurs.
+	 */
+	if (timeout) {
+		cancel_work_sync(&adapter->ptp_tx_work);
+		ixgbe_ptp_clear_tx_timestamp(adapter);
+		adapter->tx_hwtstamp_timeouts++;
+		e_warn(drv, "clearing Tx timestamp hang\n");
+	}
+}
+
 /**
  * ixgbe_ptp_tx_hwtstamp - utility function which checks for TX time stamp
  * @adapter: the private adapter struct
Signature · 1 line
2.13.0.rc0.317.gcc792a6cad5a
#29 lore raw

PATCH v4 15/15 ixgbe: check for Tx timestamp timeouts during watchdog

Quoted 25 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of Jacob Keller
Sent: Wednesday, May 3, 2017 10:29 AM
To: Intel Wired LAN <[email protected]>
Cc: Richard Cochran <[email protected]>
Subject: [Intel-wired-lan] [PATCH v4 15/15] ixgbe: check for Tx timestamp timeouts during watchdog

The ixgbe driver has logic to handle only one Tx timestamp at a time, using a state bit lock to avoid multiple requests at once.

It may be possible, if incredibly unlikely, that a Tx timestamp event is requested but never completes. Since we use an interrupt scheme to determine when the Tx timestamp occurred we would never clear the state bit in this case.

Add an ixgbe_ptp_tx_hang() function similar to the already existing ixgbe_ptp_rx_hang() function. This function runs in the watchdog routine and makes sure we eventually recover from this case instead of permanently disabling Tx timestamps.

Note: there is no currently known way to cause this without hacking the driver code to force it.

Quoted 7 lines
---
 drivers/net/ethernet/intel/ixgbe/ixgbe.h      |  1 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |  1 +
drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c  | 27
+++++++++++++++++++++++++++
 3 files changed, 29 insertions(+)
#30 lore raw

v4 cover ptp cornercase cleanups

In reply to Jacob Keller
Quoted 13 lines · Jacob Keller

This series contains fixes for similar corner cases in the e1000e, igb, i40e, and ixgbe drivers reported by David Mirabito on the ptp4l mailing list.

The first few patches fix a race condition with skb_tstamp_tx(). The next few patches ensure that we cleanup the state bit when exiting the _xmit_frame_ring() function with a DMA or TSO failure. After this, we add new statistic to count how many times the Tx timestamp is skipped due to already outstanding requests. The final few patches add some handling in the potential case where we somehow never finish a timestamp request, ensuring that we don't permanently lock the Tx timestamps forever in this rare case.

Quoted 2 lines

Changes in v4
- Split the driver patches at request of Shannon Nelson

Thanks, looks good to me.

Quoted 41 lines

- Updated commit messages, but code content should be identical

I apologize in advance, the patchworks will not track this properly because I necessarily changed the commit subjects when splitting the patches apart by driver.

Jacob Keller (15):
e1000e: fix race condition around skb_tstamp_tx()
i40e: fix race condition with PTP_TX_IN_PROGRESS bits
igb: fix race condition with PTP_TX_IN_PROGRESS bits
ixgbe: fix race condition with PTP_TX_IN_PROGRESS bits
i40e: avoid permanent lock of *_PTP_TX_IN_PROGRESS
igb: avoid permanent lock of *_PTP_TX_IN_PROGRESS
ixgbe: avoid permanent lock of *_PTP_TX_IN_PROGRESS
e1000e: add statistic indicating number of skipped Tx timestamps
i40e: add statistic indicating number of skipped Tx timestamps
igb: add statistic indicating number of skipped Tx timestamps
ixgbe: add statistic indicating number of skipped Tx timestamps
i40e: use pf data structure directly in i40e_ptp_rx_hang
i40e: check for Tx timestamp timeouts during watchdog
igb: check for Tx timestamp timeouts during watchdog
ixgbe: check for Tx timestamp timeouts during watchdog

 drivers/net/ethernet/intel/e1000e/e1000.h        |  1 +
 drivers/net/ethernet/intel/e1000e/ethtool.c      |  1 +
 drivers/net/ethernet/intel/e1000e/netdev.c       | 27 ++++++++-----
 drivers/net/ethernet/intel/i40e/i40e.h           |  5 ++-
 drivers/net/ethernet/intel/i40e/i40e_ethtool.c   |  1 +
 drivers/net/ethernet/intel/i40e/i40e_main.c      |  3 +-
 drivers/net/ethernet/intel/i40e/i40e_ptp.c       | 48 +++++++++++++++++++++---
 drivers/net/ethernet/intel/i40e/i40e_txrx.c      | 28 +++++++++++---
 drivers/net/ethernet/intel/igb/igb.h             |  2 +
 drivers/net/ethernet/intel/igb/igb_ethtool.c     |  1 +
 drivers/net/ethernet/intel/igb/igb_main.c        | 26 ++++++++++---
 drivers/net/ethernet/intel/igb/igb_ptp.c         | 41 +++++++++++++++++++-
 drivers/net/ethernet/intel/ixgbe/ixgbe.h         |  2 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c |  3 ++
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c    | 42 ++++++++++++++-------
 drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c     | 42 +++++++++++++++++++--
 16 files changed, 227 insertions(+), 46 deletions(-)
#31 lore raw

v4 cover ptp cornercase cleanups

In reply to Jacob Keller

Hi Jacob,

I'm just testing this patchset on IGB as applied to the upstream net-next repo (4.12.0-rc5+), and the problem persists. It seems to occur more seldom than without the patch, but I'm still seeing occasional messages along the lines of

ptp4l [1122.159]: timed out while polling for tx timestamp ptp4l [1122.159]: increasing tx_timestamp_timeout may correct this issue, but it is likely caused by a driver bug

roughly every 10 minutes. Other than ptp4l running, the machine is basically idle. Is there something missing in net-next?

Thanks,
Corinna

Quoted 47 lines · Jacob Keller

This series contains fixes for similar corner cases in the e1000e, igb, i40e, and ixgbe drivers reported by David Mirabito on the ptp4l mailing list.

The first few patches fix a race condition with skb_tstamp_tx(). The next few patches ensure that we cleanup the state bit when exiting the _xmit_frame_ring() function with a DMA or TSO failure. After this, we add new statistic to count how many times the Tx timestamp is skipped due to already outstanding requests. The final few patches add some handling in the potential case where we somehow never finish a timestamp request, ensuring that we don't permanently lock the Tx timestamps forever in this rare case.

Changes in v4
- Split the driver patches at request of Shannon Nelson
- Updated commit messages, but code content should be identical

I apologize in advance, the patchworks will not track this properly because I necessarily changed the commit subjects when splitting the patches apart by driver.

Jacob Keller (15):
e1000e: fix race condition around skb_tstamp_tx()
i40e: fix race condition with PTP_TX_IN_PROGRESS bits
igb: fix race condition with PTP_TX_IN_PROGRESS bits
ixgbe: fix race condition with PTP_TX_IN_PROGRESS bits
i40e: avoid permanent lock of *_PTP_TX_IN_PROGRESS
igb: avoid permanent lock of *_PTP_TX_IN_PROGRESS
ixgbe: avoid permanent lock of *_PTP_TX_IN_PROGRESS
e1000e: add statistic indicating number of skipped Tx timestamps
i40e: add statistic indicating number of skipped Tx timestamps
igb: add statistic indicating number of skipped Tx timestamps
ixgbe: add statistic indicating number of skipped Tx timestamps
i40e: use pf data structure directly in i40e_ptp_rx_hang
i40e: check for Tx timestamp timeouts during watchdog
igb: check for Tx timestamp timeouts during watchdog
ixgbe: check for Tx timestamp timeouts during watchdog

 drivers/net/ethernet/intel/e1000e/e1000.h        |  1 +
 drivers/net/ethernet/intel/e1000e/ethtool.c      |  1 +
 drivers/net/ethernet/intel/e1000e/netdev.c       | 27 ++++++++-----
 drivers/net/ethernet/intel/i40e/i40e.h           |  5 ++-
 drivers/net/ethernet/intel/i40e/i40e_ethtool.c   |  1 +
 drivers/net/ethernet/intel/i40e/i40e_main.c      |  3 +-
 drivers/net/ethernet/intel/i40e/i40e_ptp.c       | 48 +++++++++++++++++++++---
 drivers/net/ethernet/intel/i40e/i40e_txrx.c      | 28 +++++++++++---
 drivers/net/ethernet/intel/igb/igb.h             |  2 +
Quoted 16 lines
 drivers/net/ethernet/intel/igb/igb_ethtool.c     |  1 +
 drivers/net/ethernet/intel/igb/igb_main.c        | 26 ++++++++++---
 drivers/net/ethernet/intel/igb/igb_ptp.c         | 41 +++++++++++++++++++-
 drivers/net/ethernet/intel/ixgbe/ixgbe.h         |  2 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c |  3 ++
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c    | 42 ++++++++++++++-------
 drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c     | 42 +++++++++++++++++++--
 16 files changed, 227 insertions(+), 46 deletions(-)
Signature · 6 lines
2.13.0.rc0.317.gcc792a6cad5a

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan at lists.osuosl.org
http://lists.osuosl.org/mailman/listinfo/intel-wired-lan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20170621/9aba6ac5/attachment.asc>

#32 lore raw

v4 cover ptp cornercase cleanups

For what it's worth, I found that although the patches solved the case where a tx timestamp would *never* come (on an I354/igb), we still had to set "tx_timestamp_timeout 750" in ptp4l.conf as there were still the occasional case where it'd come, just a bit later than ptp4l seems to expect. YMMV of course, but we needed both the patch and this config change at once in order to be stable.

Note 750 was just a number that worked. It may be that a smaller number would suffice but it seems OK for us.

Cheers,
- DavidM

Quoted 88 lines · Corinna Vinschen

Hi Jacob,

I'm just testing this patchset on IGB as applied to the upstream net-next repo (4.12.0-rc5+), and the problem persists. It seems to occur more seldom than without the patch, but I'm still seeing occasional messages along the lines of

ptp4l [1122.159]: timed out while polling for tx timestamp ptp4l [1122.159]: increasing tx_timestamp_timeout may correct this issue, but it is likely caused by a driver bug

roughly every 10 minutes. Other than ptp4l running, the machine is basically idle. Is there something missing in net-next?

Thanks,
Corinna

Quoted 38 lines · Jacob Keller

This series contains fixes for similar corner cases in the e1000e, igb, i40e, and ixgbe drivers reported by David Mirabito on the ptp4l mailing list.

The first few patches fix a race condition with skb_tstamp_tx(). The next few patches ensure that we cleanup the state bit when exiting the _xmit_frame_ring() function with a DMA or TSO failure. After this, we add new statistic to count how many times the Tx timestamp is skipped due to already outstanding requests. The final few patches add some handling in the potential case where we somehow never finish a timestamp request, ensuring that we don't permanently lock the Tx timestamps forever in this rare case.

Changes in v4
- Split the driver patches at request of Shannon Nelson
- Updated commit messages, but code content should be identical

I apologize in advance, the patchworks will not track this properly because I necessarily changed the commit subjects when splitting the patches apart by driver.

Jacob Keller (15):
e1000e: fix race condition around skb_tstamp_tx()
i40e: fix race condition with PTP_TX_IN_PROGRESS bits
igb: fix race condition with PTP_TX_IN_PROGRESS bits
ixgbe: fix race condition with PTP_TX_IN_PROGRESS bits
i40e: avoid permanent lock of *_PTP_TX_IN_PROGRESS
igb: avoid permanent lock of *_PTP_TX_IN_PROGRESS
ixgbe: avoid permanent lock of *_PTP_TX_IN_PROGRESS
e1000e: add statistic indicating number of skipped Tx timestamps
i40e: add statistic indicating number of skipped Tx timestamps
igb: add statistic indicating number of skipped Tx timestamps
ixgbe: add statistic indicating number of skipped Tx timestamps
i40e: use pf data structure directly in i40e_ptp_rx_hang
i40e: check for Tx timestamp timeouts during watchdog
igb: check for Tx timestamp timeouts during watchdog
ixgbe: check for Tx timestamp timeouts during watchdog

Quoted 7 lines
 drivers/net/ethernet/intel/e1000e/e1000.h        |  1 +
 drivers/net/ethernet/intel/e1000e/ethtool.c      |  1 +
 drivers/net/ethernet/intel/e1000e/netdev.c       | 27 ++++++++-----
 drivers/net/ethernet/intel/i40e/i40e.h           |  5 ++-
 drivers/net/ethernet/intel/i40e/i40e_ethtool.c   |  1 +
 drivers/net/ethernet/intel/i40e/i40e_main.c      |  3 +-
 drivers/net/ethernet/intel/i40e/i40e_ptp.c       | 48

+++++++++++++++++++++---

Quoted 5 lines
 drivers/net/ethernet/intel/i40e/i40e_txrx.c      | 28 +++++++++++---
 drivers/net/ethernet/intel/igb/igb.h             |  2 +
 drivers/net/ethernet/intel/igb/igb_ethtool.c     |  1 +
 drivers/net/ethernet/intel/igb/igb_main.c        | 26 ++++++++++---
 drivers/net/ethernet/intel/igb/igb_ptp.c         | 41

+++++++++++++++++++-

Quoted 3 lines
 drivers/net/ethernet/intel/ixgbe/ixgbe.h         |  2 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c |  3 ++
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c    | 42

++++++++++++++-------

Quoted 1 line
 drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c     | 42

+++++++++++++++++++--

Quoted 9 lines
 16 files changed, 227 insertions(+), 46 deletions(-)

--
2.13.0.rc0.317.gcc792a6cad5a

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan at lists.osuosl.org
http://lists.osuosl.org/mailman/listinfo/intel-wired-lan

Quoted 6 lines

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan at osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20170622/75da2d2f/attachment-0001.html>

#33 lore raw

v4 cover ptp cornercase cleanups

Hi David,

Quoted 9 lines · David Mirabito

For what it's worth, I found that although the patches solved the case where a tx timestamp would *never* come (on an I354/igb), we still had to set "tx_timestamp_timeout 750" in ptp4l.conf as there were still the occasional case where it'd come, just a bit later than ptp4l seems to expect. YMMV of course, but we needed both the patch and this config change at once in order to be stable.

Note 750 was just a number that worked. It may be that a smaller number would suffice but it seems OK for us.

Thanks for the info. I'm just giving it a try. A value of 100 appears to work fine in my environment. Interestingly, I only have to raise tx_timestamp_timeout on i350, not on an i210 in the same machine.

Thanks,
Corinna

Quoted 101 lines

Cheers,
- DavidM

Quoted 88 lines · Corinna Vinschen

Hi Jacob,

I'm just testing this patchset on IGB as applied to the upstream net-next repo (4.12.0-rc5+), and the problem persists. It seems to occur more seldom than without the patch, but I'm still seeing occasional messages along the lines of

ptp4l [1122.159]: timed out while polling for tx timestamp ptp4l [1122.159]: increasing tx_timestamp_timeout may correct this issue, but it is likely caused by a driver bug

roughly every 10 minutes. Other than ptp4l running, the machine is basically idle. Is there something missing in net-next?

Thanks,
Corinna

Quoted 38 lines · Jacob Keller

This series contains fixes for similar corner cases in the e1000e, igb, i40e, and ixgbe drivers reported by David Mirabito on the ptp4l mailing list.

The first few patches fix a race condition with skb_tstamp_tx(). The next few patches ensure that we cleanup the state bit when exiting the _xmit_frame_ring() function with a DMA or TSO failure. After this, we add new statistic to count how many times the Tx timestamp is skipped due to already outstanding requests. The final few patches add some handling in the potential case where we somehow never finish a timestamp request, ensuring that we don't permanently lock the Tx timestamps forever in this rare case.

Changes in v4
- Split the driver patches at request of Shannon Nelson
- Updated commit messages, but code content should be identical

I apologize in advance, the patchworks will not track this properly because I necessarily changed the commit subjects when splitting the patches apart by driver.

Jacob Keller (15):
e1000e: fix race condition around skb_tstamp_tx()
i40e: fix race condition with PTP_TX_IN_PROGRESS bits
igb: fix race condition with PTP_TX_IN_PROGRESS bits
ixgbe: fix race condition with PTP_TX_IN_PROGRESS bits
i40e: avoid permanent lock of *_PTP_TX_IN_PROGRESS
igb: avoid permanent lock of *_PTP_TX_IN_PROGRESS
ixgbe: avoid permanent lock of *_PTP_TX_IN_PROGRESS
e1000e: add statistic indicating number of skipped Tx timestamps
i40e: add statistic indicating number of skipped Tx timestamps
igb: add statistic indicating number of skipped Tx timestamps
ixgbe: add statistic indicating number of skipped Tx timestamps
i40e: use pf data structure directly in i40e_ptp_rx_hang
i40e: check for Tx timestamp timeouts during watchdog
igb: check for Tx timestamp timeouts during watchdog
ixgbe: check for Tx timestamp timeouts during watchdog

Quoted 7 lines
 drivers/net/ethernet/intel/e1000e/e1000.h        |  1 +
 drivers/net/ethernet/intel/e1000e/ethtool.c      |  1 +
 drivers/net/ethernet/intel/e1000e/netdev.c       | 27 ++++++++-----
 drivers/net/ethernet/intel/i40e/i40e.h           |  5 ++-
 drivers/net/ethernet/intel/i40e/i40e_ethtool.c   |  1 +
 drivers/net/ethernet/intel/i40e/i40e_main.c      |  3 +-
 drivers/net/ethernet/intel/i40e/i40e_ptp.c       | 48

+++++++++++++++++++++---

Quoted 5 lines
 drivers/net/ethernet/intel/i40e/i40e_txrx.c      | 28 +++++++++++---
 drivers/net/ethernet/intel/igb/igb.h             |  2 +
 drivers/net/ethernet/intel/igb/igb_ethtool.c     |  1 +
 drivers/net/ethernet/intel/igb/igb_main.c        | 26 ++++++++++---
 drivers/net/ethernet/intel/igb/igb_ptp.c         | 41

+++++++++++++++++++-

Quoted 3 lines
 drivers/net/ethernet/intel/ixgbe/ixgbe.h         |  2 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c |  3 ++
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c    | 42

++++++++++++++-------

Quoted 1 line
 drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c     | 42

+++++++++++++++++++--

Quoted 9 lines
 16 files changed, 227 insertions(+), 46 deletions(-)

--
2.13.0.rc0.317.gcc792a6cad5a

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan at lists.osuosl.org
http://lists.osuosl.org/mailman/listinfo/intel-wired-lan

Quoted 6 lines

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan at osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

Quoted 4 lines

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan at osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20170622/c8c79e5d/attachment.asc>