Skip to contentSkip to conversation

r8169: disable TSO on a single version of RTL8168c to fix performance

PATCH In review Corinna Vinschen 3 participants 7 messages Nov 18, 2019 – Nov 20, 2019

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 r8169: disable TSO on a single version of RTL8168c to fix performance

During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.

Running netperf's TCP_STREAM test didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running the TCP_MAERTS and UDP_STREAM tests all returned with throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).

Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.

I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.

Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")
---
 drivers/net/ethernet/realtek/r8169_main.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/net/ethernet/realtek/r8169_main.c +6 -2
index d8fcdb9db8d1..1de11ac05bd6 100644
@@ -6952,8 +6952,12 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 		dev->gso_max_segs = RTL_GSO_MAX_SEGS_V1;
 	}
 
-	/* RTL8168e-vl has a HW issue with TSO */
-	if (tp->mac_version == RTL_GIGA_MAC_VER_34) {
+	/*
+	 * RTL8168e-vl and one RTL8168c variant are known to have a
+	 * HW issue with TSO.
+	 */
+	if (tp->mac_version == RTL_GIGA_MAC_VER_34 ||
+	    tp->mac_version == RTL_GIGA_MAC_VER_22) {
 		dev->vlan_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
 		dev->hw_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
 		dev->features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
Signature · 1 line
2.20.1
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 12 lines · Corinna Vinschen

During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.

Running netperf's TCP_STREAM test didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running the TCP_MAERTS and UDP_STREAM tests all returned with throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).

Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.

Quoted 6 lines

I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.

Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")

Thanks for reporting and the fix. Just two small nits:
- fix should be annotated "net", not "net-next"
- comment blocks in net subsystem don't have /* on a separate line

Apart from that:

Out of curiosity: Did you test also with iperf3? If yes, do you see the same issue?

Quoted 9 lines
---
 drivers/net/ethernet/realtek/r8169_main.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/net/ethernet/realtek/r8169_main.c +0 -0
index d8fcdb9db8d1..1de11ac05bd6 100644

drivers/net/ethernet/realtek/r8169_main.c static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) · line 6955

Quoted 6 lines
drivers/net/ethernet/realtek/r8169_main.c +1 -2
@@ -6952,8 +6952,12 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 		dev->gso_max_segs = RTL_GSO_MAX_SEGS_V1;
 	}
 
-	/* RTL8168e-vl has a HW issue with TSO */
-	if (tp->mac_version == RTL_GIGA_MAC_VER_34) {
+	/*

not net style comment

Quoted 9 lines
drivers/net/ethernet/realtek/r8169_main.c +5 -0
+	 * RTL8168e-vl and one RTL8168c variant are known to have a
+	 * HW issue with TSO.
+	 */
+	if (tp->mac_version == RTL_GIGA_MAC_VER_34 ||
+	    tp->mac_version == RTL_GIGA_MAC_VER_22) {
 		dev->vlan_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
 		dev->hw_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
 		dev->features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
#2 lore raw

PATCH 1/1 r8169: disable TSO on a single version of RTL8168c to fix performance

During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.

Running netperf's TCP_STREAM test didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running the TCP_MAERTS and UDP_STREAM tests all returned with throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).

Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.

I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.

Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")
---
 drivers/net/ethernet/realtek/r8169_main.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/net/ethernet/realtek/r8169_main.c +5 -2
index d8fcdb9db8d1..1de11ac05bd6 100644
@@ -6952,8 +6952,11 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 		dev->gso_max_segs = RTL_GSO_MAX_SEGS_V1;
 	}
 
-	/* RTL8168e-vl has a HW issue with TSO */
-	if (tp->mac_version == RTL_GIGA_MAC_VER_34) {
+	/* RTL8168e-vl and one RTL8168c variant are known to have a
+	 * HW issue with TSO.
+	 */
+	if (tp->mac_version == RTL_GIGA_MAC_VER_34 ||
+	    tp->mac_version == RTL_GIGA_MAC_VER_22) {
 		dev->vlan_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
 		dev->hw_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
 		dev->features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
Signature · 1 line
2.20.1
maintainer Reviewed-by on v1 1/1
#3 lore raw
Quoted 12 lines · Corinna Vinschen

During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.

Running netperf's TCP_STREAM test didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running the TCP_MAERTS and UDP_STREAM tests all returned with throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).

Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.

Quoted 7 lines

I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.

Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")

---

maintainer Applied
#4 lore raw
In reply to Corinna Vinschen

From: Corinna Vinschen <[email protected]>
Date: Tue, 19 Nov 2019 10:09:39 +0100

Quoted 12 lines

During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.

Running netperf's TCP_STREAM test didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running the TCP_MAERTS and UDP_STREAM tests all returned with throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).

Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.

Quoted 6 lines

I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.

Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")

Applied, but it would be really nice to know why this is happening instead of just turning it off completely.

maintainer
#5 lore raw
Quoted 19 lines · David Miller

From: Corinna Vinschen <[email protected]>
Date: Tue, 19 Nov 2019 10:09:39 +0100

Quoted 16 lines

During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.

Running netperf's TCP_STREAM test didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running the TCP_MAERTS and UDP_STREAM tests all returned with throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).

Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.

I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.

Quoted 6 lines
Quoted 2 lines
Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")

Applied, but it would be really nice to know why this is happening instead of just turning it off completely.

For RTL8168e-vl Realtek confirmed a HW issue, for this RTL8168c version supposedly it's the same as other chip versions are working fine with TSO.

#6 lore raw
In reply to Heiner Kallweit
Quoted 17 lines · Heiner Kallweit
Quoted 16 lines · Corinna Vinschen

During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.

Running netperf's TCP_STREAM test didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running the TCP_MAERTS and UDP_STREAM tests all returned with throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).

Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.

I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.

Quoted 6 lines
Quoted 2 lines
Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")

Thanks for reporting and the fix. Just two small nits:
- fix should be annotated "net", not "net-next"
- comment blocks in net subsystem don't have /* on a separate line

See my v2 patch.

Quoted 5 lines

Apart from that:

Out of curiosity: Did you test also with iperf3? If yes, do you see the same issue?

I didn't test with iperf3 originally, but I did so now. The results are the same. 941 Mbits/sec vs. 23.3 Mbits/sec.

Thanks,
Corinna