Skip to contentSkip to conversation

Low latency communication over wifi

DISCUSSION Pavel Machek 3 participants 12 messages Sep 23, 2015 – Sep 30, 2015

lore
Share & tools
lore

Pavel Machek

author
#0 lore raw

Hi!

I'm trying to transfer some audio data over wifi. Latency constrains are such that if it would took more than 20msec to wait for transmission, the packet should be dropped instead, and new one should be transmitted.

My final hardware contains CSR6026 chip, but drivers are way too ugly, I decided it would be easier to prototype on ath5k-based card.

I hacked qcu.c to limit latencies:

  	  qi->tqi_cw_min = ath5k_cw_validate(0);
          qi->tqi_cw_max = ath5k_cw_validate(20);

...but I don't think it did what I wanted. What am I missing?

(I could use Intel 3945ABG-based card instead, but I figured out ath5k might be easier to hack?)

Is there way to manipulate type of service from userland to get similar behaviour without patching kernel?

Thanks,
Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#1 lore raw
Quoted 6 lines · Pavel Machek

Hi!

I'm trying to transfer some audio data over wifi. Latency constrains are such that if it would took more than 20msec to wait for transmission, the packet should be dropped instead, and new one should be transmitted.

This is typically not supported, at least not by the drivers right now. Some hardware might support it, not sure.

Quoted 4 lines
  	  qi->tqi_cw_min = ath5k_cw_validate(0);
          qi->tqi_cw_max = ath5k_cw_validate(20);

...but I don't think it did what I wanted. What am I missing?

That just changes the content window settings, which changes the probability of getting medium access.

Quoted 5 lines

(I could use Intel 3945ABG-based card instead, but I figured out ath5k might be easier to hack?)

Is there way to manipulate type of service from userland to get similar behaviour without patching kernel?

If your AP has WMM/QoS then you can use setsockopt(SO_PRIORITY) to change the TID and thus the AC and get your packets classified as voice (VO) on wifi, which makes them much more likely to get access to the medium.

johannes

author
#2 lore raw

ath5k filling logs (was Re: Low latency communication over wifi)

In reply to Pavel Machek

Hi!

Quoted 9 lines

I hacked qcu.c to limit latencies:

  	  qi->tqi_cw_min = ath5k_cw_validate(0);
          qi->tqi_cw_max = ath5k_cw_validate(20);

...but I don't think it did what I wanted. What am I missing?

(I could use Intel 3945ABG-based card instead, but I figured out ath5k might be easier to hack?)

Speaking of ath5k... it fills logs with lots of messages:

[   53.486222] ath5k: ath5k_hw_get_isr: ISR: 0x00000400 IMR: 0x80081035
[   61.217713] ath5k: ath5k_hw_get_isr: ISR: 0x00000400 IMR: 0x80081035

Should I just delete the print from kernel, or is there something I should debug?

Thanks,
Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#3 lore raw
In reply to Pavel Machek
Quoted 2 lines · Pavel Machek

Is there way to manipulate type of service from userland to get similar behaviour without patching kernel?

yes, iptables DSCP / DiffServ / Differentiated Services Field

# you have to add some magic for matching your data
e.g. $IPT -t mangle -I OUTPUT -j DSCP --set-dscp-class CS7

also you should consider force the ACK-timing to 450m / Class1 and forbid retransmission in minstrel

bye, bastian

author
#4 lore raw

Hi!

Quoted 10 lines
Quoted 5 lines

(I could use Intel 3945ABG-based card instead, but I figured out ath5k might be easier to hack?)

Is there way to manipulate type of service from userland to get similar behaviour without patching kernel?

If your AP has WMM/QoS then you can use setsockopt(SO_PRIORITY) to change the TID and thus the AC and get your packets classified as voice (VO) on wifi, which makes them much more likely to get access to the medium.

That would be equivalent to ping -Q, right? It does not seem to have any effect :-(. I'd expect at least local machine to use shorter waits for medium, and thus drop packets instead of waiting.

Quoted 8 lines · Bastian Bittorf
Quoted 2 lines · Pavel Machek

Is there way to manipulate type of service from userland to get similar behaviour without patching kernel?

yes, iptables DSCP / DiffServ / Differentiated Services Field

# you have to add some magic for matching your data
e.g. $IPT -t mangle -I OUTPUT -j DSCP --set-dscp-class CS7

Again, this is ping -Q equivalent, right? I was doing

ping -c 300 -i .2 -Q $[56*4] -s 500 amd
300 packets transmitted, 300 received, 0% packet loss, time 60989ms rtt min/avg/max/mdev = 2.155/8.599/44.475/5.677 ms
300 packets transmitted, 300 received, +1 duplicates, 0% packet loss, time 61030ms rtt min/avg/max/mdev = 2.158/23.809/300.956/49.969 ms, pipe 2

I would expect packet loss, but got long delays instead.

Quoted 2 lines

also you should consider force the ACK-timing to 450m / Class1 and forbid retransmission in minstrel

Yes, disabling retransmission would be useful. How would I do that?

Thanks and best regards,

Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#5 lore raw
Quoted 4 lines · Pavel Machek

That would be equivalent to ping -Q, right? It does not seem to have any effect :-(. I'd expect at least local machine to use shorter waits for medium, and thus drop packets instead of waiting.

Correct. But it won't *drop* packets, it just increases the chances of getting medium access.

Quoted 4 lines
Quoted 2 lines
# you have to add some magic for matching your data
e.g. $IPT -t mangle -I OUTPUT -j DSCP --set-dscp-class CS7

Again, this is ping -Q equivalent, right? I was doing

Yes.

Quoted 7 lines

ping -c 300 -i .2 -Q $[56*4] -s 500 amd
300 packets transmitted, 300 received, 0% packet loss, time 60989ms rtt min/avg/max/mdev = 2.155/8.599/44.475/5.677 ms
300 packets transmitted, 300 received, +1 duplicates, 0% packet loss, time 61030ms rtt min/avg/max/mdev = 2.158/23.809/300.956/49.969 ms, pipe 2

I would expect packet loss, but got long delays instead.

See above :)

Quoted 5 lines
Quoted 2 lines

also you should consider force the ACK-timing to 450m / Class1 and forbid retransmission in minstrel

Yes, disabling retransmission would be useful. How would I do that?

It won't work on Intel devices though since they don't use minstrel(_ht)

johannes

#6 lore raw
Quoted 6 lines · Johannes Berg
Quoted 4 lines
Quoted 2 lines
# you have to add some magic for matching your data
e.g. $IPT -t mangle -I OUTPUT -j DSCP --set-dscp-class CS7

Again, this is ping -Q equivalent, right? I was doing

Yes.

IMHO this is not the same: you have to do it on both ends. (also the iptables thingy)

bye, bastian

#7 lore raw
Quoted 11 lines · Bastian Bittorf
Quoted 6 lines · Johannes Berg
Quoted 4 lines
Quoted 2 lines
# you have to add some magic for matching your data
e.g. $IPT -t mangle -I OUTPUT -j DSCP --set-dscp-class CS7

Again, this is ping -Q equivalent, right? I was doing

Yes.

IMHO this is not the same: you have to do it on both ends. (also the iptables thingy)

Depends what you're measuring, but ping should apply the same DSCP tag on the response I believe.

johannes

#8 lore raw
Quoted 2 lines · Johannes Berg

Depends what you're measuring, but ping should apply the same DSCP tag on the response I believe.

your are right, I did not know this till yet 8-) (i applied 0xb8 with 'ping -c1 -Q' just for testing)

14:49:52.491925 IP (tos 0xb8, ttl 64, id 51117, offset 0, flags [+], proto ICMP (1),

	length 1500) 192.168.2.1 > 192.168.2.103: ICMP echo reply, id 14925, seq 1, length 1480
14:49:52.491993 IP (tos 0xb8, ttl 64, id 51117, offset 1480, flags [none], proto ICMP (1),
	length 48) 192.168.2.1 > 192.168.2.103: icmp

bye, bastian

author
#9 lore raw
In reply to Johannes Berg
Quoted 8 lines · Johannes Berg
Quoted 4 lines · Pavel Machek

That would be equivalent to ping -Q, right? It does not seem to have any effect :-(. I'd expect at least local machine to use shorter waits for medium, and thus drop packets instead of waiting.

Correct. But it won't *drop* packets, it just increases the chances of getting medium access.

Increases chances of medium access, but limits number of retries, so it should drop sooner, no?

Quoted 17 lines
Quoted 4 lines
Quoted 2 lines
# you have to add some magic for matching your data
e.g. $IPT -t mangle -I OUTPUT -j DSCP --set-dscp-class CS7

Again, this is ping -Q equivalent, right? I was doing

Yes.

Quoted 7 lines

ping -c 300 -i .2 -Q $[56*4] -s 500 amd
300 packets transmitted, 300 received, 0% packet loss, time 60989ms rtt min/avg/max/mdev = 2.155/8.599/44.475/5.677 ms
300 packets transmitted, 300 received, +1 duplicates, 0% packet loss, time 61030ms rtt min/avg/max/mdev = 2.158/23.809/300.956/49.969 ms, pipe 2

I would expect packet loss, but got long delays instead.

See above :)

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

also you should consider force the ACK-timing to 450m / Class1 and forbid retransmission in minstrel

Yes, disabling retransmission would be useful. How would I do that?

It won't work on Intel devices though since they don't use minstrel(_ht)

I'm using ath5k for testing... that should use minstrel AFAICT.

02:02.0 Ethernet controller: Qualcomm Atheros AR5211 Wireless Network Adapter [AR5001X 802.11ab] (rev 01)

I'm now trying:

# Background load
sudo ping -f amd -s 8000
# Test
ping -c 30 -i .2 -Q $[0*4] -s 500 amd
ping -c 30 -i .2 -Q $[40*4] -s 500 amd
ping -c 30 -i .2 -Q $[56*4] -s 500 amd

This should send the second ping to the priority queue based on -Q, but I don't see an effect against one access point.... and it seems to work somehow against second one. Good!

avg/maximum latency goes from 8/24 (-Q $[40*4]) to 80/258 (same settings, every second frame is slow?!) to 8/26 to 7/32 to 6/18. But this jumps to 92/123, 95/139, 91/128 and 135/678 with (-Q 0). -Q $[56*4] produces 6/10, 8/23, 6/14, 5/9, 5/14. Good.

Thanks!

Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#10 lore raw
Quoted 3 lines · Pavel Machek

This should send the second ping to the priority queue based on -Q, but I don't see an effect against one access point.... and it seems to work somehow against second one. Good!

you still need to hack/tune minstrel, because a lot of magic is happen there including rts/cts on/off switching. a starting point is e.g. this:
https://github.com/steinwurf/openwrt-patches/blob/master/openwrt-r41097/mac80211/910-minstrel-retry-limit.patch

bye, bastian

#11 lore raw
In reply to Pavel Machek
Quoted 7 lines · Pavel Machek
Quoted 6 lines · Johannes Berg
Quoted 4 lines · Pavel Machek

That would be equivalent to ping -Q, right? It does not seem to have any effect :-(. I'd expect at least local machine to use shorter waits for medium, and thus drop packets instead of waiting.

Quoted 5 lines
Quoted 2 lines

Correct. But it won't *drop* packets, it just increases the chances of getting medium access.

Increases chances of medium access, but limits number of retries, so it should drop sooner, no?

No, assigning VO AC by itself won't (typically) limit the number of retries.

Quoted 4 lines

This should send the second ping to the priority queue based on -Q, but I don't see an effect against one access point.... and it seems to work somehow against second one. Good!

Well it depends on whether or not your AP supports WMM/QoS to start with :)

johannes