batmand still doesn't start the right way

Hi all,

still having problem with batman. Mostly it starts without any problems, but sometimes it seems not to do what it should. In the following example, for instance, batmand sends OG-messages only through ath0:1 (192.168.41.186) and ath1:1 (192.168.42.186), but not through eth0:1 (which is the only interface where another batman-node, 192.168.40.184 is connected, OGs from this node are received). Any Ideas?

Regards,
Rene

--------------------------------------------------------------------

root@OpenWrt:~# /etc/init.d/batmand start
Starting batmand...
Using interface eth0:1 with address 192.168.40.186 and broadcast address
192.168.43.255
Using interface ath0:1 with address 192.168.41.186 and broadcast address
192.168.43.255
Using interface ath1:1 with address 192.168.42.186 and broadcast address
192.168.43.255
root@OpenWrt:~# tcpdump -i ath1:1 port 4305
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ath1:1, link-type EN10MB (Ethernet), capture size 96 bytes
05:51:36.614733 IP 192.168.42.186.4305 > 192.168.43.255.4305: UDP, length 25
05:51:36.832907 IP 192.168.42.186.4305 > 192.168.43.255.4305: UDP, length 15
05:51:37.224704 IP 192.168.42.186.4305 > 192.168.43.255.4305: UDP, length 25
05:51:37.683458 IP 192.168.42.186.4305 > 192.168.43.255.4305: UDP, length 25
05:51:37.843369 IP 192.168.42.186.4305 > 192.168.43.255.4305: UDP, length 15

5 packets captured
10 packets received by filter
0 packets dropped by kernel

root@OpenWrt:~# tcpdump -i ath0:1 port 4305
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ath0:1, link-type EN10MB (Ethernet), capture size 96 bytes
05:51:44.454408 IP 192.168.41.186.4305 > 192.168.43.255.4305: UDP, length 25
05:51:44.702985 IP 192.168.41.186.4305 > 192.168.43.255.4305: UDP, length 15
05:51:44.732650 IP 192.168.41.186.4305 > 192.168.43.255.4305: UDP, length 25
05:51:45.492874 IP 192.168.41.186.4305 > 192.168.43.255.4305: UDP, length 25
05:51:45.682386 IP 192.168.41.186.4305 > 192.168.43.255.4305: UDP, length 15
05:51:45.693262 IP 192.168.41.186.4305 > 192.168.43.255.4305: UDP, length 25
05:51:46.582916 IP 192.168.41.186.4305 > 192.168.43.255.4305: UDP, length 25
05:51:46.622357 IP 192.168.41.186.4305 > 192.168.43.255.4305: UDP, length 15
05:51:46.712604 IP 192.168.41.186.4305 > 192.168.43.255.4305: UDP, length 25

9 packets captured
18 packets received by filter
0 packets dropped by kernel

root@OpenWrt:~# tcpdump -i eth0:1 port 4305
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0:1, link-type EN10MB (Ethernet), capture size 96 bytes
05:51:52.051159 IP 192.168.40.184.4305 > 192.168.43.255.4305: UDP, length 25
05:51:52.543207 IP 192.168.40.184.4305 > 192.168.43.255.4305: UDP, length 25
05:51:53.130721 IP 192.168.40.184.4305 > 192.168.43.255.4305: UDP, length 25
05:51:53.622574 IP 192.168.40.184.4305 > 192.168.43.255.4305: UDP, length 25
05:51:54.200698 IP 192.168.40.184.4305 > 192.168.43.255.4305: UDP, length 25
05:51:54.661832 IP 192.168.40.184.4305 > 192.168.43.255.4305: UDP, length 25
05:51:55.180689 IP 192.168.40.184.4305 > 192.168.43.255.4305: UDP, length 25
05:51:55.581781 IP 192.168.40.184.4305 > 192.168.43.255.4305: UDP, length 25
05:51:56.150711 IP 192.168.40.184.4305 > 192.168.43.255.4305: UDP, length 25

9 packets captured
18 packets received by filter
0 packets dropped by kernel
root@OpenWrt:~#

Hi

can you give the exact IP configuration (network, netmask end bcast IP) for every iface?

-- Antonio

Hi,

-------------------------------------------------------------

root@OpenWrt:~# ifconfig eth0:1
eth0:1    Link encap:Ethernet  HWaddr 00:0D:B9:04:C3:74
          inet addr:192.168.40.186  Bcast:192.168.43.255  Mask:255.255.252.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:10 Base address:0xc000

root@OpenWrt:~# ifconfig ath0:1
ath0:1    Link encap:Ethernet  HWaddr 00:0B:6B:57:01:46
          inet addr:192.168.41.186  Bcast:192.168.43.255  Mask:255.255.252.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

root@OpenWrt:~# ifconfig ath1:1
ath1:1    Link encap:Ethernet  HWaddr 00:0B:6B:57:04:6C
          inet addr:192.168.42.186  Bcast:192.168.43.255  Mask:255.255.252.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

root@OpenWrt:~#
-----------------------------------------------------------------

BATMAN was just started with 'batmand eth0:1 ath0:1 ath1:1'

Regards,
Rene

Hi rene,

your interface configuration looks correct, the following may sound weird but I've once observed a similar effect. The following happened:

run batman on my notebook on a alias ethernet interface, connected to a small
batman testbed - everything fine. Then send the notebook to deep sleep (suspend to disk) and resumed it when coming back the next day. Batman just continued but without any neighbors. Tcpdump showed exactly your observations. It traced the OGMs from the neighbors but not the own ones. I opened debug-level 4 to see whats going on and recognized that the batman on my notebook did broadcast OGM (at least the debug output claimed so) and it also revealed that the daemon received its own OGMs back (and then dropped then). The debug-level-4 then prints something like:

[ 4570480] Received BATMAN packet via NB: 103.130.30.218 , IF: eth3:bmx
103.130.30.218 (from OG: 103.130.30.218, seqno 46443, TTL 50, V 9, UDF 0, IDF 0, DPF 1)
[ 4570480] Drop packet: received my own broadcast (sender: 103.130.30.218)

So from the notebooks' batmand point of view the OGM is send and received, but its not even shown with tcpdump. I dont remember if the notebooks' batmand received the OGMs from the neighbor (but anyway, a neighbor is not fully accepted until there is a prooven bidirectional communication).
I think even restarting the daemon did not help, but what finally helped was to pull the ethernet cable out of the NIC connector and plug it back again.

For me it seemed like there was a problem with the (re-)initialization of the interface or just its alias interfaces. Maybe for some reason you are locked in a similar trap.

If this is the case the you might validate (e.g. using ping -I 192.168.40.184 192.168.40.186 from your neighboring device) or settingup another alias network only on the interfaces of this problematic link and see if the problem is also there.

happy debugging,

/axel

Hi,

just installed batmand on around ten nodes with multiple interfaces today. At least four of them faced the described problem (maybe more and I didn't realized yet), batmand didn't used all recognized interfaces to send OGMs.
Solution? Killed batman, started batman, killed batman, started batman, for an incredible amount of times and at some point it started running the way it should.

But this seems really buggy, nobody else facing this problem?

Regards,
Rene

Hi,

no, I have never seen this kind of problem apart from the wake up problems Axel described.
But you could log debug level 4 on the nodes in question and send it to me. May be that way we come closer to the source of your problem.

Greetings,
Marek

that sounds quite familiar, but for me it looks more like a hardware-problem, or combination of hardware/firmware/arch. i see problems like that only on atheros, especially atheros on i386.

cheers

--Jan

Hi,

at least ...

no, sure it's not hardware-specific. The problems documented here had been on WRAP-boards(x86) with atheros, this is true, cause they had the majority in the testbed and we have more tools available on them. But had the problem on mipsel/broadcom (1 Buffalo, 1 unknown) too.

hope to post more Information soon,
Rene

Hi,

just had a look around over the installations and found the problem again. This time between two mipsel's. The setup: AP144 has a wireless connection to AP43. Just like that (192.168.1.X are olsr-routes):

-- AP 144 --------------------------------------------------------

root@144:~# route -n | grep ^192.168.1.43
192.168.1.43    0.0.0.0         255.255.255.255 UH    1      0        0 eth1
root@144:~# ping 192.168.1.43
PING 192.168.1.43 (192.168.1.43): 56 data bytes
64 bytes from 192.168.1.43: icmp_seq=0 ttl=64 time=2.8 ms
64 bytes from 192.168.1.43: icmp_seq=1 ttl=64 time=5.5 ms

--- 192.168.1.43 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 2.8/4.1/5.5 ms

root@144:~# ps | grep batman
  905 root        436 S   /usr/sbin/batmand eth1:1 vlan1:1
  907 root        436 S   /usr/sbin/batmand eth1:1 vlan1:1
  909 root        436 S   /usr/sbin/batmand eth1:1 vlan1:1
 9661 root        280 S   grep batman
root@144:~# ifconfig eth1:1
eth1:1    Link encap:Ethernet  HWaddr 00:13:10:27:D6:A6
          inet addr:192.168.40.144  Bcast:192.168.43.255  Mask:255.255.252.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:4 Base address:0x1000

root@144:~# ifconfig vlan1:1
vlan1:1   Link encap:Ethernet  HWaddr 00:13:10:27:D6:A4
          inet addr:192.168.41.144  Bcast:192.168.43.255  Mask:255.255.252.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

-- AP 43 ---------------------------------------------------------

root@43:~#  route -n | grep ^192.168.1.144
192.168.1.144   0.0.0.0         255.255.255.255 UH    1      0        0 eth1
root@43:~# ping 192.168.1.144
PING 192.168.1.144 (192.168.1.144): 56 data bytes
64 bytes from 192.168.1.144: icmp_seq=0 ttl=64 time=2.3 ms
64 bytes from 192.168.1.144: icmp_seq=1 ttl=64 time=2.0 ms

--- 192.168.1.144 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 2.0/2.1/2.3 ms

root@43:~# ps | grep batman
  903 root        436 R   /usr/sbin/batmand eth1:1 vlan1:1
  907 root        436 S   /usr/sbin/batmand eth1:1 vlan1:1
  908 root        436 S   /usr/sbin/batmand eth1:1 vlan1:1
 5230 root        280 S   grep batman
root@43:~# ifconfig eth1:1
eth1:1    Link encap:Ethernet  HWaddr 00:18:39:6B:E0:95
          inet addr:192.168.40.43  Bcast:192.168.43.255  Mask:255.255.252.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:2 Base address:0x5000

root@43:~# ifconfig vlan1:1
vlan1:1   Link encap:Ethernet  HWaddr 00:18:39:6B:E0:93
          inet addr:192.168.41.43  Bcast:192.168.43.255  Mask:255.255.252.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

root@43:~#

------------------------------------------------------------------

And now lets see what batman is doing:

-- AP 144 --------------------------------------------------------

root@144:~# tcpdump -i eth1:1 port 4305 and src 192.168.40.144
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1:1, link-type EN10MB (Ethernet), capture size 96 bytes

0 packets captured
0 packets received by filter
0 packets dropped by kernel

root@144:~# tcpdump -i vlan1:1 port 4305 and src 192.168.41.144
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vlan1:1, link-type EN10MB (Ethernet), capture size 96 bytes
06:41:17.865478 IP 192.168.41.144.4305 > 192.168.43.255.4305: UDP, length 15
06:41:17.904019 IP 192.168.41.144.4305 > 192.168.43.255.4305: UDP, length 25
...

-- AP 43 ---------------------------------------------------------

root@43:~# tcpdump -i eth1:1 port 4305 and src 192.168.40.43
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1:1, link-type EN10MB (Ethernet), capture size 96 bytes
06:41:02.984846 IP 192.168.40.43.4305 > 192.168.43.255.4305: UDP, length 20
06:41:03.065075 IP 192.168.40.43.4305 > 192.168.43.255.4305: UDP, length 20
...
19 packets captured
38 packets received by filter
0 packets dropped by kernel
root@43:~# tcpdump -i vlan1:1 port 4305 and src 192.168.41.43
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vlan1:1, link-type EN10MB (Ethernet), capture size 96 bytes
06:41:31.255300 IP 192.168.41.43.4305 > 192.168.43.255.4305: UDP, length 20
06:41:31.406760 IP 192.168.41.43.4305 > 192.168.43.255.4305: UDP, length 25
...
18 packets captured

----------------------------------------------------------------

So batman does not send any OGMs over eth1:1 from AP144. As a result there is no wireless batman-link between the two Accesspoints.

You can find the output of batmand -c -d4 at:
http://www.opennet-initiative.de/firmware/misc/log_AP43.log.gz
http://www.opennet-initiative.de/firmware/misc/log_AP144.log.gz

If you require more information, just ask.
Regards,
Rene

Hi,
According to your log files the 144 batman does send OGM and also receives them. For me that does not look like a problem with batman.

If it is not HW related, it might still be a problem with the alias interfaces.

I guess your olsr is operating on the real interface names (not alias interfaces). So if the alias interface implementation is broken only batman is affected.

ciao,
axel

Hi,

Maybe, but this is happening on a range of different hardware. At least on WRAP(x86), Buffalo and Linksys, all running OpenWrt, the WRAPs a kamikaze(half-year-old) and the BuffaloS and LinkSysS running Whiterussian0.9.
Any Problems about the usage of alias-interfaces known on these platforms?

Thanks for your reply, the one APs wasn't changed and still has no OGMs sended - so further debugging is possible, if somebody suggests how...

Regards,
Rene

Hi,

try to let another application use the suspect alias interface. for example ping from AP144:
ping -I 192.168.41.144 192.168.41.43

-I specifies the outgoing interface. what does tcpdump report then?

ciao,
axel

PS.:
if you need a non-busybox version of ping for buffalo/linksys:
http://downloads.open-mesh.net/misc/handy-tools/wrt-freifunk/ping.tgz
and i386/wrap
http://downloads.open-mesh.net/misc/handy-tools/sources/iputils-current/ping

Hi,

tested the ping, interestingly it used mostly the wrong IP for the interface (eth1:1 is 192.168.40.144, not 192.168.41(!).144)

CASE1 - broken
----------------------------------------------------------------------

root@144:~# ./ping -I eth1:1 192.168.40.43
PING 192.168.40.43 (192.168.40.43) from 192.168.41.144 eth1:1: 56(84)
bytes of data.
64 bytes from 192.168.40.43: icmp_seq=1 ttl=64 time=3.46 ms
64 bytes from 192.168.40.43: icmp_seq=2 ttl=64 time=1.47 ms
64 bytes from 192.168.40.43: icmp_seq=3 ttl=64 time=5.90 ms
64 bytes from 192.168.40.43: icmp_seq=4 ttl=64 time=1.40 ms
64 bytes from 192.168.40.43: icmp_seq=5 ttl=64 time=1.40 ms
...

a tcpdump looks like:

root@144:~# tcpdump -i eth1:1 proto \\icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1:1, link-type EN10MB (Ethernet), capture size 96 bytes
18:33:29.868686 IP 192.168.1.144 > 192.168.40.43: ICMP echo request, id
48764, seq 1, length 64
18:33:29.869863 IP 192.168.40.43 > 192.168.1.144: ICMP echo reply, id
48764, seq 1, length 64

CASE2 - ok
----------------------------------------------------------------------

root@144:~# ./ping -I eth1:1 192.168.40.43
PING 192.168.40.43 (192.168.40.43) from 192.168.40.144 eth1:1: 56(84)
bytes of data.
64 bytes from 192.168.40.43: icmp_seq=1 ttl=57 time=7.91 ms
64 bytes from 192.168.40.43: icmp_seq=2 ttl=57 time=7.77 ms

root@144:~# tcpdump -i eth1:1 proto \\icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1:1, link-type EN10MB (Ethernet), capture size 96 bytes
18:29:51.015329 IP 192.168.40.144 > 192.168.40.43: ICMP echo request, id
13432, seq 23, length 64
18:29:52.025343 IP 192.168.40.144 > 192.168.40.43: ICMP echo request, id
13432, seq 24, length 64

CASE3 - ?? (the IP is right but packages using the wrong interface)
----------------------------------------------------------------------

root@144:~# ./ping -I 192.168.40.144 192.168.40.43
PING 192.168.40.43 (192.168.40.43) from 192.168.40.144 : 56(84) bytes of
data.
64 bytes from 192.168.40.43: icmp_seq=5 ttl=57 time=2116 ms
64 bytes from 192.168.40.43: icmp_seq=6 ttl=57 time=1117 ms
64 bytes from 192.168.40.43: icmp_seq=7 ttl=57 time=119 ms

and tcpdump:

root@144:~# tcpdump -i eth1:1 proto \\icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1:1, link-type EN10MB (Ethernet), capture size 96 bytes

0 packets captured
0 packets received by filter
0 packets dropped by kernel

root@144:~# tcpdump -i vlan1:1 proto \\icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vlan1:1, link-type EN10MB (Ethernet), capture size 96 bytes
18:43:21.685189 IP 192.168.40.144 > 192.168.40.43: ICMP echo request, id
62214, seq 30, length 64
18:43:21.735541 IP 192.168.40.43 > 192.168.40.144: ICMP echo reply, id
62214, seq 30, length 64
18:43:22.695179 IP 192.168.40.144 > 192.168.40.43: ICMP echo request, id
62214, seq 31, length 64
1

------------------------------------------------------------------------

Wow, seems a little disturbed the whole thing... Another Router showed the same transmission problem with BATMAN, this time one that had only one interface. Solving the problem seems somehow possible by killing batmand (not removing the interfaces), waiting a few seconds and starting batmand. after a few trials it works :)

Regards,
Rene

On Sonntag 25 November 2007, rene wrote:
...

indeed

Did you try to run batmand on the real-interface name (not alias) and if yes, did it also show such problems

this does not sound like an acceptable solution :-)

I am runing kamikaze from svn (currently rv 9118) on foneras(mips), netgears(mipsel), wrap(i386) and always bind batman to an alias interface. However, Ive never recognized that with kamikaze.

My wrap board usually does not mesh on a lan device (only on the atheros wlan devices), but I'll test that sometime.

ciao,
axel