is the multicast performance throttled?
Hello,
I am trying to build an application using UDP multicast sockets based on a network mesh built with devices connected in IBSS mode.
The kernel documentation says that this was the place to ask questions or anything :).
Testing the application in the mentioned environment, it suffers some kind of throttling or buffering limiting the throughput. Testing with my application, without the batman interface, just 2 IBSS connected devices, the multicast rate goes around 100kb/s (not saying that the wifi link actually tops at that speed), but then just to add the batman-adv interface and use that instead, the multicast rate becomes unstable between 0-20kb/s.
Disabling multicast optimization, it changes a bit the behaviour but still the same speed.
If I change the multicast packet to a unicast packet, both setups achieve the same speed (~100kb/s), so the batman interface is working but the multicast packets are being handled in a different way. Is it supposed to be like that?
The packets are sent right away according to the iptables counter of packets as well as tcpdump, but the receiver only gets them in a very slow way (counting but not calculating, I think it is around 8 packets per second).
The MTU is set as advised, 1532 and the payload for the UDP packets is as big as it can be so the packet does not get fragmented.
Any way to increase the rate or disabling the throttling?
Signature · 2 lines
MVH Ignacio Quezada
Hi Ignacio,
Could you describe your setup a little more, just two devices? What hardware are you using, which wifi cards? Which operating system and version?
Which version of batman-adv are you using? What is your kernel version?
What is a typical bitrate your wifi cards are agreeing on (iw station dump)? What multicast rate have you configured for your wifi cards?
What software are you using for testing? What is the bitrate you have configured in there? Is the unicast test UDP as well? What bitrates are you able to achieve via TCP? Does a lower payload length make any difference?
Regards, Linus
Quoted 35 lines · Ignacio Quezada
Hello,
I am trying to build an application using UDP multicast sockets based on a network mesh built with devices connected in IBSS mode.The kernel documentation says that this was the place to ask questions or anything :).
Testing the application in the mentioned environment, it suffers some kind of throttling or buffering limiting the throughput. Testing with my application, without the batman interface, just 2 IBSS connected devices, the multicast rate goes around 100kb/s (not saying that the wifi link actually tops at that speed), but then just to add the batman-adv interface and use that instead, the multicast rate becomes unstable between 0-20kb/s.
Disabling multicast optimization, it changes a bit the behaviour but still the same speed.
If I change the multicast packet to a unicast packet, both setups achieve the same speed (~100kb/s), so the batman interface is working but the multicast packets are being handled in a different way. Is it supposed to be like that?
The packets are sent right away according to the iptables counter of packets as well as tcpdump, but the receiver only gets them in a very slow way (counting but not calculating, I think it is around 8 packets per second).
The MTU is set as advised, 1532 and the payload for the UDP packets is as big as it can be so the packet does not get fragmented.Any way to increase the rate or disabling the throttling?
Signature · 2 lines
MVH Ignacio Quezada
Hi Linus,
Quoted 8 lines · Linus Lüssing
Hi Ignacio,
Could you describe your setup a little more, just two devices? What hardware are you using, which wifi cards? Which operating system and version?
Which version of batman-adv are you using? What is your kernel version?
The test environment is 2 android phones with Android 5.1 running kernel 3.4 where I backported batman 2016.2, same wifi chip (same soc) BCM4339.
Quoted 3 lines
What is a typical bitrate your wifi cards are agreeing on (iw station dump)? What multicast rate have you configured for your wifi cards?
Sadly, the support of iw for these kind of chipsets is very limited and iw dev wlan0 station dump returns nothing. I don't know how to get you that information.
Quoted 5 lines
What software are you using for testing? What is the bitrate you have configured in there? Is the unicast test UDP as well? What bitrates are you able to achieve via TCP? Does a lower payload length make any difference?
The software used is a java program with 1 socket each where one sends and the other receives. The unicast is UDP as well, yes. I haven't tried with TCP but I can do it if it is necessary, my application only relies on UDP.
I just tried a lower payload (504 bytes) and the amount of packets per second becomes considerably higher but the amount of data is still almost the same.
Quoted 53 lines
Regards, Linus
Quoted 49 lines · Ignacio Quezada
Hello,
I am trying to build an application using UDP multicast sockets based
on a network mesh built with devices connected in IBSS mode.The kernel documentation says that this was the place to ask questions
or anything :).Testing the application in the mentioned environment, it suffers some
kind of throttling or buffering limiting the throughput. Testing with
my application, without the batman interface, just 2 IBSS connected devices, the multicast rate goes around 100kb/s (not saying that the
wifi link actually tops at that speed), but then just to add the batman-adv interface and use that instead, the multicast rate becomes
unstable between 0-20kb/s.Disabling multicast optimization, it changes a bit the behaviour but
still the same speed.If I change the multicast packet to a unicast packet, both setups achieve the same speed (~100kb/s), so the batman interface is working
but the multicast packets are being handled in a different way. Is it
supposed to be like that?The packets are sent right away according to the iptables counter of
packets as well as tcpdump, but the receiver only gets them in a very
slow way (counting but not calculating, I think it is around 8 packets
per second).
The MTU is set as advised, 1532 and the payload for the UDP packets is
as big as it can be so the packet does not get fragmented.Any way to increase the rate or disabling the throttling? --
MVH
Ignacio Quezada
--
MVH
Ignacio Quezada
Quoted 9 lines · Ignacio Quezada
Hi Linus,
Quoted 6 lines · Linus Lüssing
Hi Ignacio,
Could you describe your setup a little more, just two devices? What hardware are you using, which wifi cards? Which operating system and version?
Quoted 6 lines
Quoted 2 lines
Which version of batman-adv are you using? What is your kernel version?
The test environment is 2 android phones with Android 5.1 running kernel 3.4 where I backported batman 2016.2, same wifi chip (same soc) BCM4339.
Note sure how usable IBSS on Android is. Last time I heard someone talking about it, it was supposedly unusable. But maybe someone else has some more current information :-).
Quoted 8 lines
Quoted 4 lines
What is a typical bitrate your wifi cards are agreeing on (iw station dump)? What multicast rate have you configured for your wifi cards?
Sadly, the support of iw for these kind of chipsets is very limited and iw dev wlan0 station dump returns nothing. I don't know how to get you that information.
And the multicast rate is still left at the 1MBit/s default? Unfortunately, usually multicast packets are sent at the lowest rate available as there are no wifi ACKs. To maximize the probability of reception. There are some tweaks in OpenWRT though.
Also, often people change the bitrate from 1MBit/s to more like 12MBit/s or even above. That gives more free airtime and enhances the batman-adv estimations (BATMAN IV).
Quoted 11 lines
Quoted 6 lines
What software are you using for testing? What is the bitrate you have configured in there? Is the unicast test UDP as well? What bitrates are you able to achieve via TCP? Does a lower payload length make any difference?
The software used is a java program with 1 socket each where one sends and the other receives. The unicast is UDP as well, yes. I haven't tried with TCP but I can do it if it is necessary, my application only relies on UDP.
UDP has no flow control. That's why I was asking about the bitrate you have configured in the application. Sending with "as fast as possible" will basically jam your (and your neighbors) wifi. And will create trouble for batman-adv to find usable routes too without any airtime available.
Regards, Linus
Signature · 4 lines
-- MVH Ignacio Quezada
Quoted 15 lines · Linus Lüssing
Quoted 14 lines · Ignacio Quezada
Hi Linus,
Quoted 9 lines · Linus Lüssing
Hi Ignacio,
Could you describe your setup a little more, just two devices? What hardware are you using, which wifi cards? Which operating system and version?
Which version of batman-adv are you using? What is your kernel version?
Quoted 9 lines
Quoted 4 lines
The test environment is 2 android phones with Android 5.1 running kernel 3.4 where I backported batman 2016.2, same wifi chip (same soc)
BCM4339.Note sure how usable IBSS on Android is. Last time I heard someone talking about it, it was supposedly unusable. But maybe someone else has some more current information :-).
After some tweaking, it works good enough.
Quoted 19 lines
Quoted 7 lines
Quoted 6 lines
What is a typical bitrate your wifi cards are agreeing on (iw station dump)? What multicast rate have you configured for your wifi cards?
Quoted 5 lines
Sadly, the support of iw for these kind of chipsets is very limited and
iw dev wlan0 station dump returns nothing. I don't know how to get you
that information.And the multicast rate is still left at the 1MBit/s default? Unfortunately, usually multicast packets are sent at the lowest rate available as there are no wifi ACKs. To maximize the probability of
reception. There are some tweaks in OpenWRT though.
Quoted 4 lines
Also, often people change the bitrate from 1MBit/s to more like 12MBit/s or even above. That gives more free airtime and enhances the batman-adv estimations (BATMAN IV).
Do you mean that with 1MBit/s for multicast, batman protocol needs 80kb/s to set up the connections? 1MBit/s for my application is more than enough.
Quoted 16 lines
Quoted 9 lines
Quoted 8 lines
What software are you using for testing? What is the bitrate you have configured in there? Is the unicast test UDP as well? What bitrates are you able to achieve via TCP? Does a lower payload length make any difference?
Quoted 6 lines
The software used is a java program with 1 socket each where one sends
and the other receives. The unicast is UDP as well, yes. I haven't tried with TCP but I can do it if it is necessary, my application only
relies on UDP.
Quoted 7 lines
UDP has no flow control. That's why I was asking about the bitrate you have configured in the application. Sending with "as fast as possible" will basically jam your (and your neighbors) wifi. And will create trouble for batman-adv to find usable routes too without any airtime available.
Regards, Linus
What I don't understand is that you are explaining standard behavior of WiFi networks, but using plain WiFi without batman works ok and the multicast goes up to 100kb/s (I think I wrote that in my first email). The tests have been like this so far:
- wlan0 (no batman), multicast: 100kb/s
- wlan0 (no batman), unicast: 100kb/s
- bat0, unicast: 100kb/s
- bat0, multicast: 20kb/s
PS: Emails from the Gmail Android App get rejected by the mailinglist, I guess it is because of the HTML format? I wasn't able to disable it, back at the desktop.
MVH
Ignacio Quezada
Quoted 8 lines · Ignacio Quezada
Quoted 7 lines
UDP has no flow control. That's why I was asking about the bitrate you have configured in the application. Sending with "as fast as possible" will basically jam your (and your neighbors) wifi. And will create trouble for batman-adv to find usable routes too without any airtime available.
Regards, Linus
Quoted 8 lines
What I don't understand is that you are explaining standard behavior of WiFi networks, but using plain WiFi without batman works ok and the multicast goes up to 100kb/s (I think I wrote that in my first email). The tests have been like this so far:
- wlan0 (no batman), multicast: 100kb/s
- wlan0 (no batman), unicast: 100kb/s
- bat0, unicast: 100kb/s
- bat0, multicast: 20kb/s
Yes, read your first email and that's the oddity I'm trying to figure out :-).
Let me try to rephrase. One guess I'm having (which might be wrong), that the multicast traffic on bat0 is interfering with the control packets ("Originator Messages" in batman speak). And that this breaks even one-hop routes. Hm, ok, on the other hand, even with no routes, flooded multicast would "work" because it doesn't use the routes discovered by OGMs. Next guess. (although the output of the originator table, "batctl o", before and during your application transmitting would be interesting to check whether there is an effect/interference)
Second guess (which is probably it :-) ): In general, batman-adv (re)broadcasts a multicast packet three times on an interface. To ensure proper reception even over multiple hops.
That might even add up with your numbers: Initial sender broadcasts three times, neighbor rebroadcasts three times, so 6*16kb/s ~ 100kb/s. 100kb/s => 0.800Mbit/s, which is already oversaturating the gross 1MBit/s wifi multicast rate (@1MBit/s gross it's usually more like 0.5MBit/s net at least for unicast, multicast might be something between 1MBit/s and 0.5MBit/s net).
Quoted 5 lines
PS: Emails from the Gmail Android App get rejected by the mailinglist, I guess it is because of the HTML format? I wasn't able to disable it, back at the desktop.
Yes, this mailinglist rejects HTML mail. To cope with spam, I think.
Regards, Linus
Quoted 3 lines · Linus Lüssing
Second guess (which is probably it :-) ): In general, batman-adv (re)broadcasts a multicast packet three times on an interface. To ensure proper reception even over multiple hops.
PS: As you are compiling batman-adv yourself, you might want to try different values for BATADV_NUM_BCASTS_WIRELESS in main.h to (in)validate this hypothesis.
Hi,
Quoted 7 lines
Quoted 3 lines
Second guess (which is probably it :-) ): In general, batman-adv (re)broadcasts a multicast packet three times on an interface. To ensure proper reception even over multiple hops.
PS: As you are compiling batman-adv yourself, you might want to try different values for
BATADV_NUM_BCASTS_WIRELESSinmain.hto (in)validate this hypothesis.
This was indeed "the problem". Setting that value to 1 allowed much more throughput. Is the value 3 just an arbitrary number to "ensure" reliability? Because that would mean that if dropping some multicast packets now and then is not really an issue, I could just set this to 1 in my case.
--
MVH
Ignacio Quezada