bgmac: Remove all offloading features, including GRO.

PATCH In review Rosen Penev 3 participants 6 messages Sep 16, 2017 – Sep 20, 2017

Subsystems (3)

Networking Network drivers Broadcom iproc gbit Ethernet driver

lore

Share & tools

lore

Rosen Penev

author

2017-09-16 00:23 UTC #0 lore raw

PATCH 1/1 bgmac: Remove all offloading features, including GRO.

On a linksys E1200v1 (actually a crossflashed E1000v2), the offloading features give no measurable benefit to speed or latency. Furthermore, disabling GRO actually improves iperf performance by a whoppimg 3mbps. Results:

Currently:

[ 4] local 192.168.1.1 port 5001 connected with 192.168.1.100 port 52410

[ ID] Interval        Transfer    Bandwidth       Reads   Dist(bin=16.0K)
[  4] 0.00-10.02 sec  52.4 MBytes  43.8 Mbits/sec  641    75:181:12:1:1:0:0:371
[  5] local 192.168.1.1 port 5001 connected with 192.168.1.100 port 52412
[  5] 0.00-10.02 sec  52.4 MBytes  43.8 Mbits/sec  629    51:194:13:1:0:1:0:369
[  4] local 192.168.1.1 port 5001 connected with 192.168.1.100 port 52414
[  4] 0.00-10.02 sec  51.9 MBytes  43.4 Mbits/sec  695    126:203:1:0:0:0:2:363
[  5] local 192.168.1.1 port 5001 connected with 192.168.1.100 port 52416
[  5] 0.00-10.01 sec  52.4 MBytes  43.9 Mbits/sec  626    57:186:10:0:0:0:0:373
[  4] local 192.168.1.1 port 5001 connected with 192.168.1.100 port 52420
[  4] 0.00-10.02 sec  52.4 MBytes  43.8 Mbits/sec  605    36:179:16:1:0:1:0:372
[  5] local 192.168.1.1 port 5001 connected with 192.168.1.100 port 52422

After disabling everything - including GRO:

[ 4] local 192.168.1.1 port 5001 connected with 192.168.1.100 port 52440

[ ID] Interval        Transfer    Bandwidth       Reads   Dist(bin=16.0K)
[  4] 0.00-10.01 sec  55.1 MBytes  46.2 Mbits/sec  672    180:82:0:0:1:0:0:409
[  5] local 192.168.1.1 port 5001 connected with 192.168.1.100 port 52442
[  5] 0.00-10.01 sec  56.0 MBytes  46.9 Mbits/sec  636    117:96:0:0:1:0:0:422
[  4] local 192.168.1.1 port 5001 connected with 192.168.1.100 port 52444
[  4] 0.00-10.01 sec  55.4 MBytes  46.4 Mbits/sec  675    172:92:0:0:1:0:0:410
[  5] local 192.168.1.1 port 5001 connected with 192.168.1.100 port 52446
[  5] 0.00-10.01 sec  56.0 MBytes  46.9 Mbits/sec  633    119:90:0:1:1:0:0:422
[  4] local 192.168.1.1 port 5001 connected with 192.168.1.100 port 52448
[  4] 0.00-10.01 sec  55.2 MBytes  46.3 Mbits/sec  688    157:123:0:0:2:0:0:406

v2: Changed napi_gro_receive to netif_receive_skb. Seems to have an identical result.

Signed-off-by Rosen Penev <[email protected]>

---
 drivers/net/ethernet/broadcom/bgmac.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

drivers/net/ethernet/broadcom/bgmac.c +1 -1

index 48d672b204a4..1fb0053aeee7 100644
@@ -483,7 +483,7 @@ static int bgmac_dma_rx_read(struct bgmac *bgmac, struct bgmac_dma_ring *ring,
 			skb->protocol = eth_type_trans(skb, bgmac->net_dev);
 			bgmac->net_dev->stats.rx_bytes += len;
 			bgmac->net_dev->stats.rx_packets++;
-			napi_gro_receive(&bgmac->napi, skb);
+			netif_receive_skb(skb);
 			handled++;
 		} while (0);

Signature · 1 line

2.13.5

Eric Dumazet

2017-09-16 00:34 UTC #1 lore raw

Quoted 17 lines · Rosen Penev

On a linksys E1200v1 (actually a crossflashed E1000v2), the offloading features give no measurable benefit to speed or latency. Furthermore, disabling GRO actually improves iperf performance by a whoppimg 3mbps. Results:
Currently:
v2: Changed napi_gro_receive to netif_receive_skb. Seems to have an identical result.
Signed-off-by Rosen Penev <[email protected]>
---
 drivers/net/ethernet/broadcom/bgmac.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bgmac.c +0 -0
index 48d672b204a4..1fb0053aeee7 100644

drivers/net/ethernet/broadcom/bgmac.c static int bgmac_dma_rx_read(struct bgmac *bgmac, struct bgmac_dma_ring *ring, · line 489

Quoted 8 lines

drivers/net/ethernet/broadcom/bgmac.c +1 -1

@@ -483,7 +483,7 @@ static int bgmac_dma_rx_read(struct bgmac *bgmac, struct bgmac_dma_ring *ring,
 			skb->protocol = eth_type_trans(skb, bgmac->net_dev);
 			bgmac->net_dev->stats.rx_bytes += len;
 			bgmac->net_dev->stats.rx_packets++;
-			napi_gro_receive(&bgmac->napi, skb);
+			netif_receive_skb(skb);
 			handled++;
 		} while (0);

And have you tested 1Gbit link speed ?
( Or 2.5 Gbit link speed )

If you want to disable GRO on your host, fine : you can use ethtool -K

[email protected]

author

2017-09-16 00:38 UTC #2 lore raw

I have not. Unfortunately I own no gigabit hardware to test this on. The MIPS CPU runs at 300MHz on my unit.

Quoted 41 lines · Eric Dumazet

Quoted 29 lines · Rosen Penev
On a linksys E1200v1 (actually a crossflashed E1000v2), the offloading features give no measurable benefit to speed or latency. Furthermore, disabling GRO actually improves iperf performance by a whoppimg 3mbps. Results:
Currently:
v2: Changed napi_gro_receive to netif_receive_skb. Seems to have an identical result.
Signed-off-by Rosen Penev <[email protected]>
---
 drivers/net/ethernet/broadcom/bgmac.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bgmac.c +1 -1
index 48d672b204a4..1fb0053aeee7 100644
@@ -483,7 +483,7 @@ static int bgmac_dma_rx_read(struct bgmac
 *bgmac, struct bgmac_dma_ring *ring,
 			skb->protocol = eth_type_trans(skb, bgmac-
 >net_dev);
 			bgmac->net_dev->stats.rx_bytes += len;
 			bgmac->net_dev->stats.rx_packets++;
-			napi_gro_receive(&bgmac->napi, skb);
+			netif_receive_skb(skb);
 			handled++;
Quoted 2 lines
} while (0);
And have you tested 1Gbit link speed ?
( Or 2.5 Gbit link speed )
If you want to disable GRO on your host, fine : you can use ethtool -K

Florian Fainelli

2017-09-16 06:04 UTC #3 lore raw

Quoted 3 lines · On September 15, 2017 5:38:42 PM PDT, [email protected] wrote:

I have not. Unfortunately I own no gigabit hardware to test this on. The MIPS CPU runs at 300MHz on my unit.

bgmac is used on Gigabit capable hardware, like Northstar and Northstar Plus, and others too, so unless you can get access to such HW or get confirmation from someone that your patches changes something, I would just drop this change and not bother. This is already not 100mbits/sec linerate...

Quoted 37 lines · Eric Dumazet

Quoted 36 lines · Rosen Penev
Quoted 29 lines
On a linksys E1200v1 (actually a crossflashed E1000v2), the offloading features give no measurable benefit to speed or latency. Furthermore, disabling GRO actually improves iperf performance by a whoppimg 3mbps. Results:
Currently:
v2: Changed napi_gro_receive to netif_receive_skb. Seems to have an identical result.
Signed-off-by Rosen Penev <[email protected]>
---
 drivers/net/ethernet/broadcom/bgmac.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bgmac.c +1 -1
index 48d672b204a4..1fb0053aeee7 100644
@@ -483,7 +483,7 @@ static int bgmac_dma_rx_read(struct bgmac
 *bgmac, struct bgmac_dma_ring *ring,
 			skb->protocol = eth_type_trans(skb, bgmac-
 >net_dev);
 			bgmac->net_dev->stats.rx_bytes += len;
 			bgmac->net_dev->stats.rx_packets++;
-			napi_gro_receive(&bgmac->napi, skb);
+			netif_receive_skb(skb);
 			handled++;
Quoted 2 lines
} while (0);
And have you tested 1Gbit link speed ?
( Or 2.5 Gbit link speed )

Quoted 5 lines

Quoted 5 lines
If you want to disable GRO on your host, fine : you can use ethtool -K

(please don't top-post)

Signature · 1 line

Florian

[email protected]

author

2017-09-20 21:27 UTC #4 lore raw

Sorry for the noise. After more testing I've found out that the cause was that I had BBR enabled on my laptop. Switching back to CUBIC fixed the issue.

In other words, this patch is detrimental.

~67mbps - gro off
~87mbps - gro on

Quoted 62 lines · Florian Fainelli

Quoted 4 lines · On September 15, 2017 5:38:42 PM PDT, [email protected] wrote:
I have not. Unfortunately I own no gigabit hardware to test this on.
The MIPS CPU runs at 300MHz on my unit.
bgmac is used on Gigabit capable hardware, like Northstar and Northstar Plus, and others too, so unless you can get access to such HW or get confirmation from someone that your patches changes something, I would just drop this change and not bother. This is already not 100mbits/sec linerate...
Quoted 42 lines · Eric Dumazet
Quoted 41 lines · Rosen Penev
Quoted 30 lines
On a linksys E1200v1 (actually a crossflashed E1000v2), the offloading features give no measurable benefit to speed or latency.
Furthermore, disabling GRO actually improves iperf performance by a
whoppimg 3mbps. Results:
Currently:
v2: Changed napi_gro_receive to netif_receive_skb. Seems to have an
identical result.
Signed-off-by Rosen Penev <[email protected]>
---
 drivers/net/ethernet/broadcom/bgmac.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bgmac.c +0 -0
index 48d672b204a4..1fb0053aeee7 100644
@@ -483,7 +483,7 @@ static int bgmac_dma_rx_read(struct bgmac
 *bgmac, struct bgmac_dma_ring *ring,
 			skb->protocol = eth_type_trans(skb,
bgmac-
Quoted 1 line
net_dev);
drivers/net/ethernet/broadcom/bgmac.c static int bgmac_dma_rx_read(struct bgmac *bgmac, struct bgmac_dma_ring *ring, · line 488
Quoted 6 lines
drivers/net/ethernet/broadcom/bgmac.c +1 -1
 			bgmac->net_dev->stats.rx_bytes += len;
 			bgmac->net_dev->stats.rx_packets++;
-			napi_gro_receive(&bgmac->napi, skb);
+			netif_receive_skb(skb);
 			handled++;
 		} while (0);
And have you tested 1Gbit link speed ?
( Or 2.5 Gbit link speed )
Quoted 6 lines
Quoted 6 lines
If you want to disable GRO on your host, fine : you can use ethtool
-K
(please don't top-post)

Florian Fainelli

2017-09-20 21:32 UTC #5 lore raw

Quoted 5 lines · On 09/20/2017 02:27 PM, [email protected] wrote:

Sorry for the noise. After more testing I've found out that the cause was that I had BBR enabled on my laptop. Switching back to CUBIC fixed the issue.
In other words, this patch is detrimental.

Quite unsurprisingly, thanks for coming back, and please don't top-post on netdev.

Quoted 67 lines

~67mbps - gro off
~87mbps - gro on
Quoted 62 lines · Florian Fainelli
Quoted 4 lines · On September 15, 2017 5:38:42 PM PDT, [email protected] wrote:
I have not. Unfortunately I own no gigabit hardware to test this on.
The MIPS CPU runs at 300MHz on my unit.
bgmac is used on Gigabit capable hardware, like Northstar and Northstar Plus, and others too, so unless you can get access to such HW or get confirmation from someone that your patches changes something, I would just drop this change and not bother. This is already not 100mbits/sec linerate...
Quoted 42 lines · Eric Dumazet
Quoted 41 lines · Rosen Penev
Quoted 30 lines
On a linksys E1200v1 (actually a crossflashed E1000v2), the offloading features give no measurable benefit to speed or latency.
Furthermore, disabling GRO actually improves iperf performance by a
whoppimg 3mbps. Results:
Currently:
v2: Changed napi_gro_receive to netif_receive_skb. Seems to have an
identical result.
Signed-off-by Rosen Penev <[email protected]>
---
 drivers/net/ethernet/broadcom/bgmac.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/broadcom/bgmac.c +0 -0
index 48d672b204a4..1fb0053aeee7 100644
@@ -483,7 +483,7 @@ static int bgmac_dma_rx_read(struct bgmac
 *bgmac, struct bgmac_dma_ring *ring,
 			skb->protocol = eth_type_trans(skb,
bgmac-
Quoted 1 line
net_dev);
drivers/net/ethernet/broadcom/bgmac.c static int bgmac_dma_rx_read(struct bgmac *bgmac, struct bgmac_dma_ring *ring, · line 488
Quoted 6 lines
drivers/net/ethernet/broadcom/bgmac.c +1 -1
 			bgmac->net_dev->stats.rx_bytes += len;
 			bgmac->net_dev->stats.rx_packets++;
-			napi_gro_receive(&bgmac->napi, skb);
+			netif_receive_skb(skb);
 			handled++;
 		} while (0);
And have you tested 1Gbit link speed ?
( Or 2.5 Gbit link speed )
Quoted 6 lines
Quoted 6 lines
If you want to disable GRO on your host, fine : you can use ethtool
-K
(please don't top-post)

Signature · 1 line

Florian