Skip to contentSkip to conversation

high RTT in 40 MHz channel width

DISCUSSION Iñaki Pascual 4 participants 6 messages Apr 29, 2016 – May 5, 2016 Qualcomm Atheros ATH10K wireless driver

lore
Share & tools
lore
#0 lore raw

Hi,

I am measuring TCP and UDP latency (actually RTT) and I am getting too high values when working with channels with 40MHz (also with 80MHz) width.

I am using hping3 for testing and these are the RTT avg values:

BW 20 MHz: TCP 0.9 UDP 1.1
BW 40 MHz: TCP 7.7 UDP 1.3
BW 80 MHz: TCP 3.5 UDP 1.3

I have tried different channels with similar results. According hping there are no packet loss and with wireshark I don't see any retransmission.

I am using:
driver: ath10k_pci
version: 4.2.0+
firmware-version: 10.1.467-ct-com-full-014-96d543

Any ideas?

Bests,

Iñaki

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

#1 lore raw

Hi all,
just to clarify, results are ms. And stations are in IBSS mode.

BW 20 MHz: TCP 0.9ms UDP 1.1ms
BW 40 MHz: TCP 7.7ms UDP 1.3ms
BW 80 MHz: TCP 3.5ms UDP 1.3ms

Behavior with 40MHz and 80MHz channel bandwidth is really weird taking into account there are no retransmissions. Anyone facing this kind of problem?

Thanks,
Jose

Quoted 27 lines · Iñaki Pascual

Hi,

I am measuring TCP and UDP latency (actually RTT) and I am getting too high values when working with channels with 40MHz (also with 80MHz) width.

I am using hping3 for testing and these are the RTT avg values:

BW 20 MHz: TCP 0.9 UDP 1.1
BW 40 MHz: TCP 7.7 UDP 1.3
BW 80 MHz: TCP 3.5 UDP 1.3

I have tried different channels with similar results. According hping there are no packet loss and with wireshark I don't see any retransmission.

I am using:
driver: ath10k_pci
version: 4.2.0+
firmware-version: 10.1.467-ct-com-full-014-96d543

Any ideas?

Bests,

Iñaki

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

Signature · 14 lines
Jose Núñez-Martínez, PhD
Senior Researcher   email: [email protected]
COMNET Division     Web: http://networks.cttc.cat

Centre Tecnològic de Telecomunicacions de Catalunya (CTTC)
Parc Mediterrani de la Tecnologia
Av. Carl Friedrich Gauss, 7                Ph.:+34 936452927
08860 Castelldefels - Barcelona            Fax: +34 936452901


_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#2 lore raw

Since UDP seems to mostly not be effected, maybe it has something to do with your tcp stack and/or congestion control?

You could sniff on the wlan0 interface, as well as on-air, and compare timestamps to see if the firmware is being slow about putting the TCP frames on the air?

What TCP congestion control are you using?

And also, if there is any interference on the secondary channels, the NIC will hold off transmitting. So, you would not even see retransmits in this case.

Have you tried in regular managed mode (AP/STA) to see if it is the same with CT firmware?

You could try stock firmware in AP/STA mode. If CT firmware shows a regression, I will be happy to look at it.

Finally, you might try the latest CT firmware in case it acts better. I ripped out some queue control in the non-full builds somewhat recently, and possibly that would decrease latency.

Thanks,
Ben

Quoted 41 lines · Jose Núñez-Martínez (CTTC)

Hi all,
just to clarify, results are ms. And stations are in IBSS mode.

BW 20 MHz: TCP 0.9ms UDP 1.1ms
BW 40 MHz: TCP 7.7ms UDP 1.3ms
BW 80 MHz: TCP 3.5ms UDP 1.3ms

Behavior with 40MHz and 80MHz channel bandwidth is really weird taking into account there are no retransmissions. Anyone facing this kind of problem?

Thanks,
Jose

Quoted 27 lines · Iñaki Pascual

Hi,

I am measuring TCP and UDP latency (actually RTT) and I am getting too high values when working with channels with 40MHz (also with 80MHz) width.

I am using hping3 for testing and these are the RTT avg values:

BW 20 MHz: TCP 0.9 UDP 1.1
BW 40 MHz: TCP 7.7 UDP 1.3
BW 80 MHz: TCP 3.5 UDP 1.3

I have tried different channels with similar results. According hping there are no packet loss and with wireshark I don't see any retransmission.

I am using:
driver: ath10k_pci
version: 4.2.0+
firmware-version: 10.1.467-ct-com-full-014-96d543

Any ideas?

Bests,

Iñaki

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

Signature · 7 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
#3 lore raw

Hi Ben,

thanks for your comments. Please see my in-line answers in your email below.

Best regards,

Iñaki

Quoted 2 lines · Ben Greear

Since UDP seems to mostly not be effected, maybe it has something to do with your tcp stack and/or congestion control?

Yes, we agree the problem is TCP-related so we have performed some tests (described below) using hping3 TCP

Quoted 5 lines

You could sniff on the wlan0 interface, as well as on-air, and compare timestamps to see if the firmware is being slow about putting the TCP frames on the air?

we did as you say and the results are
TCP hping request on-air timeStamp: 15:11:19,194250000
TCP hping request on-interface timeStamp: 15:11:19,194253000
TCP hping reply on-interface timeStamp: 15:11:19,194280000
TCP hping reply on-air timeStamp: 15:11:19,197582000

So it looks like the firmware takes 3.3 ms to process/send the reply frame. This number is in line with the aprox. 7.7 ms RTT we are measuring (3,3 ms *2 + aprox 1 ms delay) We repeated the experiment with other machines and channels and the results are consistent.

Quoted 2 lines

What TCP congestion control are you using?

reno

Quoted 3 lines

And also, if there is any interference on the secondary channels, the NIC will hold off transmitting. So, you would not even see retransmits in this case.

for these tests, in order to to avoid interferences, we put down all other wireless interfaces and equipments in our testbed

Quoted 2 lines

Have you tried in regular managed mode (AP/STA) to see if it is the same with CT firmware?

we tried this test, but we were not able to configure the client with BW 40MHz nor 20MHz, only 80MHz. The average measured RTT for 80 MHz was 3.5 ms, same as ad-hoc mode.

Quoted 2 lines

You could try stock firmware in AP/STA mode. If CT firmware shows a regression, I will be happy to look at it.

we tried with an Ubuntu 14.04 fresh install, kernel and firmware are:
driver: ath10k_pci
version: 4.2.0-34-generic
firmware-version: 1.0.0.636
we run the test in ad-hoc mode, we could not configure BW 80 MHz (not supported) results:
40MHz
round-trip min/avg/max = 0.5/2.8/5.5 ms
20MHz
round-trip min/avg/max = 0.4/2.5/4.4 ms

Quoted 4 lines

Finally, you might try the latest CT firmware in case it acts better. I ripped out some queue control in the non-full builds somewhat recently, and possibly that would decrease latency.

we tried with
firmware-version: 10.1.467-ct-_fH-016-b797ef5
results show almost same RTT values

Quoted 4 lines

Thanks,
Ben

Quoted 43 lines
Quoted 41 lines · Jose Núñez-Martínez (CTTC)

Hi all,
just to clarify, results are ms. And stations are in IBSS mode.

BW 20 MHz: TCP 0.9ms UDP 1.1ms
BW 40 MHz: TCP 7.7ms UDP 1.3ms
BW 80 MHz: TCP 3.5ms UDP 1.3ms

Behavior with 40MHz and 80MHz channel bandwidth is really weird taking into account there are no retransmissions. Anyone facing this kind of problem?

Thanks,
Jose

Quoted 27 lines · Iñaki Pascual

Hi,

I am measuring TCP and UDP latency (actually RTT) and I am getting too high values when working with channels with 40MHz (also with 80MHz) width.

I am using hping3 for testing and these are the RTT avg values:

BW 20 MHz: TCP 0.9 UDP 1.1
BW 40 MHz: TCP 7.7 UDP 1.3
BW 80 MHz: TCP 3.5 UDP 1.3

I have tried different channels with similar results. According hping there are no packet loss and with wireshark I don't see any retransmission.

I am using:
driver: ath10k_pci
version: 4.2.0+
firmware-version: 10.1.467-ct-com-full-014-96d543

Any ideas?

Bests,

Iñaki

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

#4 lore raw
In reply to Iñaki Pascual
Quoted 10 lines · Iñaki Pascual

Hi,

I am measuring TCP and UDP latency (actually RTT) and I am getting too high values when working with channels with 40MHz (also with 80MHz) width.

I am using hping3 for testing and these are the RTT avg values:

BW 20 MHz: TCP 0.9 UDP 1.1
BW 40 MHz: TCP 7.7 UDP 1.3
BW 80 MHz: TCP 3.5 UDP 1.3

Just 2cc. Did you try changing the a-msdu/a-mpdu limits in firmware? You can tune it via debugfs file:

echo 3 64 > /sys/kernel/debug/ieee80211/phy0/ath10k/htt_max_amsdu_ampdu

The "3" stands for A-MSDU limit, "64" stands for A-MPDU limit. "3 64" is the default. You can test, e.g.

- "1 64"
- "3 8"
- "1 1"
- "1 8"

And see if this changes TCP RTT in any way.

You didn't really tell your setup (or I missed it). Are you using ath10k as AP, client, or both (i.e. have two ath10k supported devices)?

Are you bridging traffic or is to locally generated? Perhaps there's a problem with hw offloaded ip/tcp checksumming. You might want to remove it from the driver (manually) to verify that or check tcpdump to confirm whether OTA frames have correct checksums.

Michał

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

#5 lore raw

Hi Michael,

no, we did not try changing a-msdu/a-mpdu limits. We will take a look at this, thanks.

Regarding the setup, these tests were run with two machines with three ath10k devices each (QCA988x 802.11ac Wireless Network Adapter). Kernel and firmware versions:
driver: ath10k_pci
version: 4.2.0+
firmware-version: 10.1.467-ct-com-full-014-96d543

We just enabled one interface on each machine and configured them in ibss mode. Then we run hping3 (TCP and UDP) from one to another. The equipments are at 4 meters distance with good line view.

Bests,

Iñaki

Quoted 36 lines · Michal Kazior
Quoted 10 lines · Iñaki Pascual

Hi,

I am measuring TCP and UDP latency (actually RTT) and I am getting too high values when working with channels with 40MHz (also with 80MHz) width.

I am using hping3 for testing and these are the RTT avg values:

BW 20 MHz: TCP 0.9 UDP 1.1
BW 40 MHz: TCP 7.7 UDP 1.3
BW 80 MHz: TCP 3.5 UDP 1.3

Just 2cc. Did you try changing the a-msdu/a-mpdu limits in firmware? You can tune it via debugfs file:

   echo 3 64 > /sys/kernel/debug/ieee80211/phy0/ath10k/htt_max_amsdu_ampdu

The "3" stands for A-MSDU limit, "64" stands for A-MPDU limit. "3 64" is the default. You can test, e.g.

- "1 64"
- "3 8"
- "1 1"
- "1 8"

And see if this changes TCP RTT in any way.

You didn't really tell your setup (or I missed it). Are you using ath10k as AP, client, or both (i.e. have two ath10k supported devices)?

Are you bridging traffic or is to locally generated? Perhaps there's a problem with hw offloaded ip/tcp checksumming. You might want to remove it from the driver (manually) to verify that or check tcpdump to confirm whether OTA frames have correct checksums.

Quoted 6 lines

Michał

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k

_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k