r8169: disable TSO on a single version of RTL8168c to fix performance
PATCH 1/1 r8169: disable TSO on a single version of RTL8168c to fix performance
During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.
Running netperf's TCP_STREAM test didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running the TCP_MAERTS and UDP_STREAM tests all returned with throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).
Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.
I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.
--- drivers/net/ethernet/realtek/r8169_main.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/net/ethernet/realtek/r8169_main.c +6 -2
@@ -6952,8 +6952,12 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) dev->gso_max_segs =RTL_GSO_MAX_SEGS_V1 ; } -/* RTL8168e-vl has a HW issue with TSO */ -if (tp->mac_version ==RTL_GIGA_MAC_VER_34 ) { +/* +* RTL8168e-vl and one RTL8168c variant are known to have a +* HW issue with TSO. +*/ +if (tp->mac_version ==RTL_GIGA_MAC_VER_34 || + tp->mac_version ==RTL_GIGA_MAC_VER_22 ) { dev->vlan_features &= ~(NETIF_F_ALL_TSO |NETIF_F_SG ); dev->hw_features &= ~(NETIF_F_ALL_TSO |NETIF_F_SG ); dev->features &= ~(NETIF_F_ALL_TSO |NETIF_F_SG );
Signature · 1 line
2.20.1
Quoted 12 lines · Corinna Vinschen
During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.
Running netperf's
TCP_STREAMtest didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running theTCP_MAERTSandUDP_STREAMtests all returned with throughput > 900 Mb/s, as didTCP_STREAMwith the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).Bisecting turned up commit
93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.
Quoted 6 lines
I added my 8168c version,
RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")Signed-off-by Corinna Vinschen <[email protected]>
Thanks for reporting and the fix. Just two small nits:
- fix should be annotated "net", not "net-next"
- comment blocks in net subsystem don't have /* on a separate line
Apart from that:
Out of curiosity: Did you test also with iperf3? If yes, do you see the same issue?
Quoted 9 lines
--- drivers/net/ethernet/realtek/r8169_main.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)drivers/net/ethernet/realtek/r8169_main.c +0
-0
drivers/net/ethernet/realtek/r8169_main.c static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) · line 6955
Quoted 6 lines
drivers/net/ethernet/realtek/r8169_main.c +1
-2@@ -6952,8 +6952,12 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) dev->gso_max_segs =RTL_GSO_MAX_SEGS_V1 ; } -/* RTL8168e-vl has a HW issue with TSO */ -if (tp->mac_version ==RTL_GIGA_MAC_VER_34 ) { +/*
not net style comment
Quoted 9 lines
drivers/net/ethernet/realtek/r8169_main.c +5
-0+* RTL8168e-vl and one RTL8168c variant are known to have a +* HW issue with TSO. +*/ +if (tp->mac_version ==RTL_GIGA_MAC_VER_34 || + tp->mac_version ==RTL_GIGA_MAC_VER_22 ) { dev->vlan_features &= ~(NETIF_F_ALL_TSO |NETIF_F_SG ); dev->hw_features &= ~(NETIF_F_ALL_TSO |NETIF_F_SG ); dev->features &= ~(NETIF_F_ALL_TSO |NETIF_F_SG );
PATCH 1/1 r8169: disable TSO on a single version of RTL8168c to fix performance
During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.
Running netperf's TCP_STREAM test didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running the TCP_MAERTS and UDP_STREAM tests all returned with throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).
Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.
I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.
--- drivers/net/ethernet/realtek/r8169_main.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/net/ethernet/realtek/r8169_main.c +5 -2
@@ -6952,8 +6952,11 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) dev->gso_max_segs =RTL_GSO_MAX_SEGS_V1 ; } -/* RTL8168e-vl has a HW issue with TSO */ -if (tp->mac_version ==RTL_GIGA_MAC_VER_34 ) { +/* RTL8168e-vl and one RTL8168c variant are known to have a +* HW issue with TSO. +*/ +if (tp->mac_version ==RTL_GIGA_MAC_VER_34 || + tp->mac_version ==RTL_GIGA_MAC_VER_22 ) { dev->vlan_features &= ~(NETIF_F_ALL_TSO |NETIF_F_SG ); dev->hw_features &= ~(NETIF_F_ALL_TSO |NETIF_F_SG ); dev->features &= ~(NETIF_F_ALL_TSO |NETIF_F_SG );
Signature · 1 line
2.20.1
Quoted 12 lines · Corinna Vinschen
During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.
Running netperf's
TCP_STREAMtest didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running theTCP_MAERTSandUDP_STREAMtests all returned with throughput > 900 Mb/s, as didTCP_STREAMwith the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).Bisecting turned up commit
93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.
Quoted 7 lines
I added my 8168c version,
RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")Signed-off-by Corinna Vinschen <[email protected]>---
From: Corinna Vinschen <[email protected]>
Date: Tue, 19 Nov 2019 10:09:39 +0100
Quoted 12 lines
During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.
Running netperf's
TCP_STREAMtest didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running theTCP_MAERTSandUDP_STREAMtests all returned with throughput > 900 Mb/s, as didTCP_STREAMwith the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).Bisecting turned up commit
93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.
Quoted 6 lines
I added my 8168c version,
RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")Signed-off-by Corinna Vinschen <[email protected]>
Applied, but it would be really nice to know why this is happening instead of just turning it off completely.
Heiner Kallweit
maintainerQuoted 19 lines · David Miller
From: Corinna Vinschen <[email protected]>
Date: Tue, 19 Nov 2019 10:09:39 +0100Quoted 16 lines
During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.
Running netperf's
TCP_STREAMtest didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running theTCP_MAERTSandUDP_STREAMtests all returned with throughput > 900 Mb/s, as didTCP_STREAMwith the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).Bisecting turned up commit
93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.I added my 8168c version,
RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.
Quoted 6 lines
Quoted 2 lines
Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")Signed-off-by Corinna Vinschen <[email protected]>Applied, but it would be really nice to know why this is happening instead of just turning it off completely.
For RTL8168e-vl Realtek confirmed a HW issue, for this RTL8168c version supposedly it's the same as other chip versions are working fine with TSO.
Quoted 17 lines · Heiner Kallweit
Quoted 16 lines · Corinna Vinschen
During performance testing, I found that one of my r8169 NICs suffered a major performance loss, a 8168c model.
Running netperf's
TCP_STREAMtest didn't return the expected throughput of > 900 Mb/s, but rather only about 22 Mb/s. Strange enough, running theTCP_MAERTSandUDP_STREAMtests all returned with throughput > 900 Mb/s, as didTCP_STREAMwith the other r8169 NICs I can test (either one of 8169s, 8168e, 8168f).Bisecting turned up commit
93681cd7d94f83903cb3f0f95433d10c28a7e9a5, "r8169: enable HW csum and TSO" as the culprit.I added my 8168c version,
RTL_GIGA_MAC_VER_22, to the code special-casing the 8168evl as per the patch below. This fixed the performance problem for me.
Quoted 6 lines
Quoted 2 lines
Fixes 93681cd7d94f ("r8169: enable HW csum and TSO")Signed-off-by Corinna Vinschen <[email protected]>Thanks for reporting and the fix. Just two small nits:
- fix should be annotated "net", not "net-next"
- comment blocks in net subsystem don't have /* on a separate line
See my v2 patch.
Quoted 5 lines
Apart from that:
Reviewed-by Heiner Kallweit <[email protected]>Out of curiosity: Did you test also with iperf3? If yes, do you see the same issue?
I didn't test with iperf3 originally, but I did so now. The results are the same. 941 Mbits/sec vs. 23.3 Mbits/sec.
Thanks,
Corinna