Skip to contentSkip to conversation

netdevice: Add netif_tx_lock_q

RFC PATCH In review David Wu 2 participants 4 messages Mar 24, 2020 – Apr 16, 2020

Subsystems (4)
lore
Share & tools
lore

David Wu

author
#0 lore raw

RFC PATCH 1/2 netdevice: Add netif_tx_lock_q

It is similar to netif_tx_lock, but only locks one queue, so it seems to be suitable for multiple queues.

If the current queue status is frozen, it will requeue skb, which will not block the current thread, it improve performance when tested.

---
 include/linux/netdevice.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
include/linux/netdevice.h +16 -0
index b6fedd54cd8e..30054a94210e 100644
@@ -4085,6 +4085,16 @@ static inline void netif_tx_lock(struct net_device *dev)
 	}
 }
 
+static inline void netif_tx_lock_q(struct netdev_queue *txq)
+{
+	int cpu;
+
+	cpu = smp_processor_id();
+	__netif_tx_lock(txq, cpu);
+	set_bit(__QUEUE_STATE_FROZEN, &txq->state);
+	__netif_tx_unlock(txq);
+}
+
 static inline void netif_tx_lock_bh(struct net_device *dev)
 {
 	local_bh_disable();
@@ -4108,6 +4118,12 @@ static inline void netif_tx_unlock(struct net_device *dev)
 	spin_unlock(&dev->tx_global_lock);
 }
 
+static inline void netif_tx_unlock_q(struct netdev_queue *txq)
+{
+	clear_bit(__QUEUE_STATE_FROZEN, &txq->state);
+	netif_schedule_queue(txq);
+}
+
 static inline void netif_tx_unlock_bh(struct net_device *dev)
 {
 	netif_tx_unlock(dev);
Signature · 1 line
2.19.1
author
#1 lore raw

RFC PATCH 2/2 net: stmmac: Change the tx clean lock

At tx clean, use a frozen queue instead of blocking the current queue, could still queue skb, which improve performance.

---
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +2 -2
index cb7a5bad4cfe..946058bcc9ed 100644
@@ -1897,7 +1897,7 @@ static int stmmac_tx_clean(struct stmmac_priv *priv, int budget, u32 queue)
 	unsigned int bytes_compl = 0, pkts_compl = 0;
 	unsigned int entry, count = 0;
 
-	__netif_tx_lock_bh(netdev_get_tx_queue(priv->dev, queue));
+	netif_tx_lock_q(netdev_get_tx_queue(priv->dev, queue));
 
 	priv->xstats.tx_clean++;
 
@@ -1994,7 +1994,7 @@ static int stmmac_tx_clean(struct stmmac_priv *priv, int budget, u32 queue)
 	if (tx_q->dirty_tx != tx_q->cur_tx)
 		mod_timer(&tx_q->txtimer, STMMAC_COAL_TIMER(priv->tx_coal_timer));
 
-	__netif_tx_unlock_bh(netdev_get_tx_queue(priv->dev, queue));
+	netif_tx_unlock_q(netdev_get_tx_queue(priv->dev, queue));
 
 	return count;
 }
Signature · 1 line
2.19.1
#2 lore raw

RE: [RFC,PATCH 2/2] net: stmmac: Change the tx clean lock

From: David Wu <[email protected]>
Date: Mar/24/2020, 09:38:28 (UTC+00:00)

Quoted 3 lines

At tx clean, use a frozen queue instead of blocking the current queue, could still queue skb, which improve performance.

Please provide performance improvement numbers.

---
Thanks,
Jose Miguel Abreu

author
#3 lore raw

Re: [RFC,PATCH 2/2] net: stmmac: Change the tx clean lock

Hi Jose,

From the test I did, there will be some improvement, an improvement of tens Mbits/sec.

Before patch:

# iperf -c 192.168.1.102 -i 1 -t 10 -w 300K -u -b 1000M
------------------------------------------------------------
Client connecting to 192.168.1.102, UDP port 5001
Sending 1470 byte datagrams, IPG target: 11.76 us (kalman adjust)
UDP buffer size:  600 KByte (WARNING: requested  300 KByte)
------------------------------------------------------------
[  3] local 192.168.1.103 port 45018 connected with 192.168.1.102 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec   103 MBytes   862 Mbits/sec
[  3]  1.0- 2.0 sec   104 MBytes   868 Mbits/sec
[  3]  2.0- 3.0 sec   104 MBytes   869 Mbits/sec
[  3]  3.0- 4.0 sec   104 MBytes   869 Mbits/sec
[  3]  4.0- 5.0 sec   104 MBytes   870 Mbits/sec
[  3]  5.0- 6.0 sec   104 MBytes   869 Mbits/sec
[  3]  6.0- 7.0 sec   104 MBytes   869 Mbits/sec
[  3]  7.0- 8.0 sec   104 MBytes   870 Mbits/sec
[  3]  8.0- 9.0 sec   104 MBytes   871 Mbits/sec
[  3]  0.0-10.0 sec  1.01 GBytes   869 Mbits/sec
[  3] Sent 738834 datagrams

After patch:

# iperf -c 192.168.1.102 -i 1 -t 10 -w 300K -u -b 1000M
------------------------------------------------------------
Client connecting to 192.168.1.102, UDP port 5001
Sending 1470 byte datagrams, IPG target: 11.76 us (kalman adjust)
UDP buffer size:  600 KByte (WARNING: requested  300 KByte)
------------------------------------------------------------
[  3] local 192.168.1.103 port 35654 connected with 192.168.1.102 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec   114 MBytes   953 Mbits/sec
[  3]  1.0- 2.0 sec   114 MBytes   956 Mbits/sec
[  3]  2.0- 3.0 sec   115 MBytes   962 Mbits/sec
[  3]  3.0- 4.0 sec   114 MBytes   955 Mbits/sec
[  3]  4.0- 5.0 sec   114 MBytes   957 Mbits/sec
[  3]  5.0- 6.0 sec   114 MBytes   957 Mbits/sec
[  3]  6.0- 7.0 sec   114 MBytes   955 Mbits/sec
[  3]  7.0- 8.0 sec   114 MBytes   956 Mbits/sec
[  3]  8.0- 9.0 sec   114 MBytes   955 Mbits/sec
[  3]  0.0- 9.9 sec  1.10 GBytes   957 Mbits/sec
[  3] Sent 804442 datagrams
Quoted 16 lines · Jose Abreu

From: David Wu <[email protected]>
Date: Mar/24/2020, 09:38:28 (UTC+00:00)

Quoted 3 lines

At tx clean, use a frozen queue instead of blocking the current queue, could still queue skb, which improve performance.

Please provide performance improvement numbers.

---
Thanks,
Jose Miguel Abreu