Skip to contentSkip to conversation

tsnep: Throttle irq, rotten pkts, RX buffer alloc and ethtool_get_channels()

PATCH In review Gerhard Engleder 3 participants 13 messages Nov 17, 2022 – Nov 19, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover tsnep: Throttle irq, rotten pkts, RX buffer alloc and ethtool_get_channels()

Collection of improvements found during development of XDP support. Hopefully the last patch series before the XDP support.

Fix of rotten packets increased CPU load and caused slight drop of iperf performance, because CPU load was already at 100% before. This performance drop is compensated with interrupt throttling, which makes sense anyway.

ethtool_get_channels() is needed for automatic TAPRIO configuration in combination with multiple queues.

Rework of RX buffer allocation is prework of XDP. It ensures that packets are only dropped if RX queue would otherwise run empty because of failed allocations. So it should reduce the number of dropped packets under low memory conditions.

Gerhard Engleder (4):
tsnep: Throttle interrupts
tsnep: Fix rotten packets
tsnep: Add ethtool get_channels support
tsnep: Rework RX buffer allocation

 drivers/net/ethernet/engleder/tsnep.h         |   4 +
 drivers/net/ethernet/engleder/tsnep_ethtool.c |  19 ++
 drivers/net/ethernet/engleder/tsnep_hw.h      |   7 +
 drivers/net/ethernet/engleder/tsnep_main.c    | 252 +++++++++++++-----
 4 files changed, 214 insertions(+), 68 deletions(-)
Signature · 1 line
2.30.2
#1 lore raw

PATCH 1/4 tsnep: Throttle interrupts

Without interrupt throttling, iperf server mode generates a CPU load of 100% (A53 1.2GHz). Also the throughput suffers with less than 900Mbit/s on a 1Gbit/s link. The reason is a high interrupt load with interrupts every ~20us.

Reduce interrupt load by throttling of interrupts. Interrupt delay is configured statically to 64us. For iperf server mode the CPU load is significantly reduced to ~20% and the throughput reaches the maximum of 941MBit/s. Interrupts are generated every ~140us.

---
 drivers/net/ethernet/engleder/tsnep_hw.h   | 7 +++++++
 drivers/net/ethernet/engleder/tsnep_main.c | 6 ++++++
 2 files changed, 13 insertions(+)
drivers/net/ethernet/engleder/tsnep_hw.h +7 -0
index 315dada75323..55e1caf193a6 100644
@@ -48,6 +48,13 @@
 #define ECM_COUNTER_LOW 0x0028
 #define ECM_COUNTER_HIGH 0x002C
 
+/* interrupt delay */
+#define ECM_INT_DELAY 0x0030
+#define ECM_INT_DELAY_MASK 0xF0
+#define ECM_INT_DELAY_SHIFT 4
+#define ECM_INT_DELAY_BASE_US 16
+#define ECM_INT_DELAY_OFFSET 1
+
 /* control and status */
 #define ECM_STATUS 0x0080
 #define ECM_LINK_MODE_OFF 0x01000000
drivers/net/ethernet/engleder/tsnep_main.c +6 -0
index 48fb391951dd..a99320e03279 100644
@@ -39,6 +39,8 @@
 #endif
 #define DMA_ADDR_LOW(dma_addr) ((u32)((dma_addr) & 0xFFFFFFFF))
 
+#define INT_DELAY (64 / ECM_INT_DELAY_BASE_US)
+
 static void tsnep_enable_irq(struct tsnep_adapter *adapter, u32 mask)
 {
 	iowrite32(mask, adapter->addr + ECM_INT_ENABLE);
@@ -1298,6 +1300,7 @@ static int tsnep_phy_init(struct tsnep_adapter *adapter)
 static int tsnep_queue_init(struct tsnep_adapter *adapter, int queue_count)
 {
 	u32 irq_mask = ECM_INT_TX_0 | ECM_INT_RX_0;
+	u8 irq_delay = (INT_DELAY << ECM_INT_DELAY_SHIFT) & ECM_INT_DELAY_MASK;
 	char name[8];
 	int i;
 	int retval;
@@ -1316,6 +1319,7 @@ static int tsnep_queue_init(struct tsnep_adapter *adapter, int queue_count)
 	adapter->queue[0].tx = &adapter->tx[0];
 	adapter->queue[0].rx = &adapter->rx[0];
 	adapter->queue[0].irq_mask = irq_mask;
+	iowrite8(irq_delay, adapter->addr + ECM_INT_DELAY);
 
 	adapter->netdev->irq = adapter->queue[0].irq;
 
@@ -1336,6 +1340,8 @@ static int tsnep_queue_init(struct tsnep_adapter *adapter, int queue_count)
 		adapter->queue[i].rx = &adapter->rx[i];
 		adapter->queue[i].irq_mask =
 			irq_mask << (ECM_INT_TXRX_SHIFT * i);
+		iowrite8(irq_delay, adapter->addr + ECM_INT_DELAY +
+				    ECM_INT_DELAY_OFFSET * i);
 	}
 
 	return 0;
Signature · 1 line
2.30.2
#2 lore raw

Re: [PATCH net-next 1/4] tsnep: Throttle interrupts

Quoted 4 lines · Gerhard Engleder

Without interrupt throttling, iperf server mode generates a CPU load of 100% (A53 1.2GHz). Also the throughput suffers with less than 900Mbit/s on a 1Gbit/s link. The reason is a high interrupt load with interrupts every ~20us.

Not my area of expertise, but is NAPI working correctly? It should be that you disable interrupts while NAPI is polling, and only re-enable interrupts when polling has stopped. If you are receiving at near line rate at 100% load, i would of thought that NAPI would be polling most of the time and interrupts would be mostly disabled?

Interrupt coalescence makes a lot of sense thought, so the patch itself is useful.

Andrew

#3 lore raw

Re: [PATCH net-next 1/4] tsnep: Throttle interrupts

Quoted 11 lines · Andrew Lunn
Quoted 4 lines · Gerhard Engleder

Without interrupt throttling, iperf server mode generates a CPU load of 100% (A53 1.2GHz). Also the throughput suffers with less than 900Mbit/s on a 1Gbit/s link. The reason is a high interrupt load with interrupts every ~20us.

Not my area of expertise, but is NAPI working correctly? It should be that you disable interrupts while NAPI is polling, and only re-enable interrupts when polling has stopped. If you are receiving at near line rate at 100% load, i would of thought that NAPI would be polling most of the time and interrupts would be mostly disabled?

I disable and re-enable interrupts in the driver, but I don't know exactly under which conditions interrupts will be disabled mostly. At least during my XDP tests with minimum size packets the interrupt rate dropped significantly. Here the CPU had also 100% load but packet rate was much higher with ~880000 pkt/s.

I compared the performance and the interrupt rate with the macb driver on the same CPU and the results were similar.

Quoted 2 lines

Interrupt coalescence makes a lot of sense thought, so the patch itself is useful.

Thanks!

Gerhard

maintainer
#4 lore raw

Re: [PATCH net-next 1/4] tsnep: Throttle interrupts

In reply to Gerhard Engleder
Quoted 9 lines · On Thu, 17 Nov 2022 21:14:37 +0100 Gerhard Engleder wrote:

Without interrupt throttling, iperf server mode generates a CPU load of 100% (A53 1.2GHz). Also the throughput suffers with less than 900Mbit/s on a 1Gbit/s link. The reason is a high interrupt load with interrupts every ~20us.

Reduce interrupt load by throttling of interrupts. Interrupt delay is configured statically to 64us. For iperf server mode the CPU load is significantly reduced to ~20% and the throughput reaches the maximum of 941MBit/s. Interrupts are generated every ~140us.

User should be able to control these settings, please implement ethtool::get_coalesce / set_coalesce.

#5 lore raw

Re: [PATCH net-next 1/4] tsnep: Throttle interrupts

Quoted 6 lines · Jakub Kicinski
Quoted 5 lines · On Thu, 17 Nov 2022 21:14:37 +0100 Gerhard Engleder wrote:

Without interrupt throttling, iperf server mode generates a CPU load of 100% (A53 1.2GHz). Also the throughput suffers with less than 900Mbit/s on a 1Gbit/s link. The reason is a high interrupt load with interrupts every ~20us.

Quoted 7 lines
Quoted 4 lines

Reduce interrupt load by throttling of interrupts. Interrupt delay is configured statically to 64us. For iperf server mode the CPU load is significantly reduced to ~20% and the throughput reaches the maximum of 941MBit/s. Interrupts are generated every ~140us.

User should be able to control these settings, please implement ethtool::get_coalesce / set_coalesce.

I will implement it.

#6 lore raw

PATCH 2/4 tsnep: Fix rotten packets

In reply to Gerhard Engleder

If PTP synchronisation is done every second, then sporadic the interval is higher than one second:

ptp4l[696.582]: master offset        -17 s2 freq   -1891 path delay 573
ptp4l[697.582]: master offset        -22 s2 freq   -1901 path delay 573
ptp4l[699.368]: master offset         -1 s2 freq   -1887 path delay 573
      ^^^^^^^ Should be 698.582!

This problem is caused by rotten packets, which are received after polling but before interrupts are enabled again. This can be fixed by checking for pending work and rescheduling if necessary after interrupts has been enabled again.

Fixes 403f69bbdbad ("tsnep: Add TSN endpoint Ethernet MAC driver")
---
 drivers/net/ethernet/engleder/tsnep_main.c | 59 +++++++++++++++++++++-
 1 file changed, 58 insertions(+), 1 deletion(-)
drivers/net/ethernet/engleder/tsnep_main.c +58 -1
index a99320e03279..0aca2ba97757 100644
@@ -544,6 +544,27 @@ static bool tsnep_tx_poll(struct tsnep_tx *tx, int napi_budget)
 	return (budget != 0);
 }
 
+static bool tsnep_tx_pending(struct tsnep_tx *tx)
+{
+	unsigned long flags;
+	struct tsnep_tx_entry *entry;
+	bool pending = false;
+
+	spin_lock_irqsave(&tx->lock, flags);
+
+	if (tx->read != tx->write) {
+		entry = &tx->entry[tx->read];
+		if ((__le32_to_cpu(entry->desc_wb->properties) &
+		     TSNEP_TX_DESC_OWNER_MASK) ==
+		    (entry->properties & TSNEP_TX_DESC_OWNER_MASK))
+			pending = true;
+	}
+
+	spin_unlock_irqrestore(&tx->lock, flags);
+
+	return pending;
+}
+
 static int tsnep_tx_open(struct tsnep_adapter *adapter, void __iomem *addr,
 			 int queue_index, struct tsnep_tx *tx)
 {
@@ -823,6 +844,21 @@ static int tsnep_rx_poll(struct tsnep_rx *rx, struct napi_struct *napi,
 	return done;
 }
 
+static bool tsnep_rx_pending(struct tsnep_rx *rx)
+{
+	struct tsnep_rx_entry *entry;
+
+	if (rx->read != rx->write) {
+		entry = &rx->entry[rx->read];
+		if ((__le32_to_cpu(entry->desc_wb->properties) &
+		     TSNEP_DESC_OWNER_COUNTER_MASK) ==
+		    (entry->properties & TSNEP_DESC_OWNER_COUNTER_MASK))
+			return true;
+	}
+
+	return false;
+}
+
 static int tsnep_rx_open(struct tsnep_adapter *adapter, void __iomem *addr,
 			 int queue_index, struct tsnep_rx *rx)
 {
@@ -868,6 +904,17 @@ static void tsnep_rx_close(struct tsnep_rx *rx)
 	tsnep_rx_ring_cleanup(rx);
 }
 
+static bool tsnep_pending(struct tsnep_queue *queue)
+{
+	if (queue->tx && tsnep_tx_pending(queue->tx))
+		return true;
+
+	if (queue->rx && tsnep_rx_pending(queue->rx))
+		return true;
+
+	return false;
+}
+
 static int tsnep_poll(struct napi_struct *napi, int budget)
 {
 	struct tsnep_queue *queue = container_of(napi, struct tsnep_queue,
@@ -888,9 +935,19 @@ static int tsnep_poll(struct napi_struct *napi, int budget)
 	if (!complete)
 		return budget;
 
-	if (likely(napi_complete_done(napi, done)))
+	if (likely(napi_complete_done(napi, done))) {
 		tsnep_enable_irq(queue->adapter, queue->irq_mask);
 
+		/* reschedule if work is already pending, prevent rotten packets
+		 * which are transmitted or received after polling but before
+		 * interrupt enable
+		 */
+		if (tsnep_pending(queue)) {
+			tsnep_disable_irq(queue->adapter, queue->irq_mask);
+			napi_schedule(napi);
+		}
+	}
+
 	return min(done, budget - 1);
 }
 
Signature · 1 line
2.30.2
#7 lore raw

Re: [PATCH net-next 2/4] tsnep: Fix rotten packets

Quoted 10 lines · Gerhard Engleder

If PTP synchronisation is done every second, then sporadic the interval is higher than one second:

ptp4l[696.582]: master offset        -17 s2 freq   -1891 path delay 573
ptp4l[697.582]: master offset        -22 s2 freq   -1901 path delay 573
ptp4l[699.368]: master offset         -1 s2 freq   -1887 path delay 573
      ^^^^^^^ Should be 698.582!

This problem is caused by rotten packets, which are received after polling but before interrupts are enabled again.

Is this a hardware bug? At the end of the interrupt coalescence period, should it not check the queue and fire an interrupt?

Andrew

#8 lore raw

Re: [PATCH net-next 2/4] tsnep: Fix rotten packets

Quoted 9 lines · Andrew Lunn
Quoted 8 lines · Gerhard Engleder

If PTP synchronisation is done every second, then sporadic the interval is higher than one second:

ptp4l[696.582]: master offset        -17 s2 freq   -1891 path delay 573
ptp4l[697.582]: master offset        -22 s2 freq   -1901 path delay 573
ptp4l[699.368]: master offset         -1 s2 freq   -1887 path delay 573
       ^^^^^^^ Should be 698.582!
Quoted 5 lines
Quoted 2 lines

This problem is caused by rotten packets, which are received after polling but before interrupts are enabled again.

Is this a hardware bug? At the end of the interrupt coalescence period, should it not check the queue and fire an interrupt?

In my case, the hardware is not signaled if a descriptor is processed by the software. The hardware is only signaled if it gets new descriptors assigned. So the hardware does not know if there are still descriptors in the RX queue which need to be processed by the software. As a result, it would only be possible to trigger an interrupt for descriptors which may has been processed already anyway.

In the end I made the hardware stupid. If interrupts are disabled for NAPI polling, then interrupts events in the hardware are ignored. If interrupts are enabled again, then only new interrupt events will trigger the interrupt. I was afraid that too intelligent hardware will lead to hardware bugs in this case.

Gerhard

maintainer
#9 lore raw

Re: [PATCH net-next 2/4] tsnep: Fix rotten packets

In reply to Gerhard Engleder
Quoted 8 lines · On Thu, 17 Nov 2022 21:14:38 +0100 Gerhard Engleder wrote:

If PTP synchronisation is done every second, then sporadic the interval is higher than one second:

ptp4l[696.582]: master offset        -17 s2 freq   -1891 path delay 573
ptp4l[697.582]: master offset        -22 s2 freq   -1901 path delay 573
ptp4l[699.368]: master offset         -1 s2 freq   -1887 path delay 573
      ^^^^^^^ Should be 698.582!
Quoted 7 lines

This problem is caused by rotten packets, which are received after polling but before interrupts are enabled again. This can be fixed by checking for pending work and rescheduling if necessary after interrupts has been enabled again.

Fixes 403f69bbdbad ("tsnep: Add TSN endpoint Ethernet MAC driver")

This patch needs to go to net separately :(
Packets getting stuck in a queue can cause real issues to users.

#10 lore raw

Re: [PATCH net-next 2/4] tsnep: Fix rotten packets

Quoted 14 lines · Jakub Kicinski
Quoted 13 lines · On Thu, 17 Nov 2022 21:14:38 +0100 Gerhard Engleder wrote:

If PTP synchronisation is done every second, then sporadic the interval is higher than one second:

ptp4l[696.582]: master offset        -17 s2 freq   -1891 path delay 573
ptp4l[697.582]: master offset        -22 s2 freq   -1901 path delay 573
ptp4l[699.368]: master offset         -1 s2 freq   -1887 path delay 573
       ^^^^^^^ Should be 698.582!

This problem is caused by rotten packets, which are received after polling but before interrupts are enabled again. This can be fixed by checking for pending work and rescheduling if necessary after interrupts has been enabled again.

Quoted 5 lines
Quoted 2 lines
Fixes 403f69bbdbad ("tsnep: Add TSN endpoint Ethernet MAC driver")

This patch needs to go to net separately :(
Packets getting stuck in a queue can cause real issues to users.

I will post it separately.

#11 lore raw

PATCH 3/4 tsnep: Add ethtool get_channels support

In reply to Gerhard Engleder

Allow user space to read number of TX and RX queue. This is useful for device dependent qdisc configurations like TAPRIO with hardware offload.

---
 drivers/net/ethernet/engleder/tsnep_ethtool.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
drivers/net/ethernet/engleder/tsnep_ethtool.c +12 -0
index a713a126b227..3da5cb75aa55 100644
@@ -288,6 +288,17 @@ static int tsnep_ethtool_set_rxnfc(struct net_device *dev,
 	}
 }
 
+static void tsnep_ethtool_get_channels(struct net_device *dev,
+				       struct ethtool_channels *ch)
+{
+	struct tsnep_adapter *adapter = netdev_priv(dev);
+
+	ch->max_rx = adapter->num_rx_queues;
+	ch->max_tx = adapter->num_tx_queues;
+	ch->rx_count = adapter->num_rx_queues;
+	ch->tx_count = adapter->num_tx_queues;
+}
+
 static int tsnep_ethtool_get_ts_info(struct net_device *dev,
 				     struct ethtool_ts_info *info)
 {
@@ -327,6 +338,7 @@ const struct ethtool_ops tsnep_ethtool_ops = {
 	.get_sset_count = tsnep_ethtool_get_sset_count,
 	.get_rxnfc = tsnep_ethtool_get_rxnfc,
 	.set_rxnfc = tsnep_ethtool_set_rxnfc,
+	.get_channels = tsnep_ethtool_get_channels,
 	.get_ts_info = tsnep_ethtool_get_ts_info,
 	.get_link_ksettings = phy_ethtool_get_link_ksettings,
 	.set_link_ksettings = phy_ethtool_set_link_ksettings,
Signature · 1 line
2.30.2
#12 lore raw

PATCH 4/4 tsnep: Rework RX buffer allocation

In reply to Gerhard Engleder

Refill RX queue in batches of descriptors to improve performance. Refill is allowed to fail as long as a minimum number of descriptors is active. Thus, a limited number of failed RX buffer allocations is now allowed for normal operation. Previously every failed allocation resulted in a dropped frame.

If the minimum number of active descriptors is reached, then RX buffers are still reused and frames are dropped. This ensures that the RX queue never runs empty and always continues to operate.

Prework for future XDP support.

---
 drivers/net/ethernet/engleder/tsnep.h         |   4 +
 drivers/net/ethernet/engleder/tsnep_ethtool.c |   7 +
 drivers/net/ethernet/engleder/tsnep_main.c    | 191 +++++++++++-------
 3 files changed, 133 insertions(+), 69 deletions(-)
drivers/net/ethernet/engleder/tsnep.h +4 -0
index 09a723b827c7..9a840d145727 100644
@@ -18,6 +18,8 @@
 #define TSNEP "tsnep"
 
 #define TSNEP_RING_SIZE 256
+#define TSNEP_RING_RX_REFILL 16
+#define TSNEP_RING_RX_REUSE (TSNEP_RING_SIZE - TSNEP_RING_SIZE / 4)
 #define TSNEP_RING_ENTRIES_PER_PAGE (PAGE_SIZE / TSNEP_DESC_SIZE)
 #define TSNEP_RING_PAGE_COUNT (TSNEP_RING_SIZE / TSNEP_RING_ENTRIES_PER_PAGE)
 
@@ -110,6 +112,7 @@ struct tsnep_rx {
 	dma_addr_t page_dma[TSNEP_RING_PAGE_COUNT];
 
 	struct tsnep_rx_entry entry[TSNEP_RING_SIZE];
+	int write;
 	int read;
 	u32 owner_counter;
 	int increment_owner_counter;
@@ -119,6 +122,7 @@ struct tsnep_rx {
 	u32 bytes;
 	u32 dropped;
 	u32 multicast;
+	u32 alloc_failed;
 };
 
 struct tsnep_queue {
drivers/net/ethernet/engleder/tsnep_ethtool.c +7 -0
index 3da5cb75aa55..b74eba512755 100644
@@ -8,6 +8,7 @@ static const char tsnep_stats_strings[][ETH_GSTRING_LEN] = {
 	"rx_bytes",
 	"rx_dropped",
 	"rx_multicast",
+	"rx_alloc_failed",
 	"rx_phy_errors",
 	"rx_forwarded_phy_errors",
 	"rx_invalid_frame_errors",
@@ -21,6 +22,7 @@ struct tsnep_stats {
 	u64 rx_bytes;
 	u64 rx_dropped;
 	u64 rx_multicast;
+	u64 rx_alloc_failed;
 	u64 rx_phy_errors;
 	u64 rx_forwarded_phy_errors;
 	u64 rx_invalid_frame_errors;
@@ -36,6 +38,7 @@ static const char tsnep_rx_queue_stats_strings[][ETH_GSTRING_LEN] = {
 	"rx_%d_bytes",
 	"rx_%d_dropped",
 	"rx_%d_multicast",
+	"rx_%d_alloc_failed",
 	"rx_%d_no_descriptor_errors",
 	"rx_%d_buffer_too_small_errors",
 	"rx_%d_fifo_overflow_errors",
@@ -47,6 +50,7 @@ struct tsnep_rx_queue_stats {
 	u64 rx_bytes;
 	u64 rx_dropped;
 	u64 rx_multicast;
+	u64 rx_alloc_failed;
 	u64 rx_no_descriptor_errors;
 	u64 rx_buffer_too_small_errors;
 	u64 rx_fifo_overflow_errors;
@@ -178,6 +182,7 @@ static void tsnep_ethtool_get_ethtool_stats(struct net_device *netdev,
 		tsnep_stats.rx_bytes += adapter->rx[i].bytes;
 		tsnep_stats.rx_dropped += adapter->rx[i].dropped;
 		tsnep_stats.rx_multicast += adapter->rx[i].multicast;
+		tsnep_stats.rx_alloc_failed += adapter->rx[i].alloc_failed;
 	}
 	reg = ioread32(adapter->addr + ECM_STAT);
 	tsnep_stats.rx_phy_errors =
@@ -200,6 +205,8 @@ static void tsnep_ethtool_get_ethtool_stats(struct net_device *netdev,
 		tsnep_rx_queue_stats.rx_bytes = adapter->rx[i].bytes;
 		tsnep_rx_queue_stats.rx_dropped = adapter->rx[i].dropped;
 		tsnep_rx_queue_stats.rx_multicast = adapter->rx[i].multicast;
+		tsnep_rx_queue_stats.rx_alloc_failed =
+			adapter->rx[i].alloc_failed;
 		reg = ioread32(adapter->addr + TSNEP_QUEUE(i) +
 			       TSNEP_RX_STATISTIC);
 		tsnep_rx_queue_stats.rx_no_descriptor_errors =
drivers/net/ethernet/engleder/tsnep_main.c +122 -69

259 lines not on this page · raw message

Signature · 1 line
2.30.2