Mesh with access from wifi and lan. Which is the Only True Way?
Hi!
I am struggling with setting up a mesh network. It should be available for users without batmand both on the lan interfaces and wan interfaces.
Surely I do have some misconceptions in my setup, but don't know where.
The nodes are wistron ca8 ones, with official OpenWrt Backfire 10.03.1-rc3 r22796 atheros.
I try to set up the following network topology:
- there is a /c subnet for batman nodes, 10.42.0.N/24
- there is a /c subnet on each node for non-batman users, 10.42.N.1/24 (dhcp runs there in force mode)
- there is a /c subnet on each node for lan (non-batman) 10.43.N.1/24 (dhcp runs there)
with only two nodes I have managed to handle all the traffic with the following rules in firewall.user:
iptables -A forwarding_rule -s 10.42.N.0/24 -i ath0 -j ACCEPT
iptables -A forwarding_rule -d 10.42.N.0/24 -o ath0 -j ACCEPT
But with three nodes in a linear topology user-A-B-C, the reject firewall chain (basically the FORWARD chain) eats up the packets user->A
This is the same rule which rejects batman packets A->C which go through B, and there are the gate interface which I do not understand, so while the following rules seem to solve the problem at least for three nodes, I have a feeling that I am not on the right path, and maybe on a way to cause packet storms. iptables -A forwarding_rule -d 10.42.0.0/24 ! -s 10.42.0.0/24 -i ath0 -o ath0 -j ACCEPT
iptables -A forwarding_rule -s 10.42.0.0/24 ! -d 10.42.0.0/24 -o ath0 -i ath0 -j ACCEPT
Can someone tell me what I have missed (or is it the right solution indeed?)
Quoted 11 lines · On Tuesday 28 September 2010 21:08:53 Magosányi Árpád wrote:
But with three nodes in a linear topology user-A-B-C, the reject firewall chain (basically the FORWARD chain) eats up the packets
user->AThis is the same rule which rejects batman packetsA->Cwhich go through B, and there are the gate interface which I do not understand, so while the following rules seem to solve the problem at least for three nodes, I have a feeling that I am not on the right path, and maybe on a way to cause packet storms.
iptables -Aforwarding_rule-d 10.42.0.0/24 ! -s 10.42.0.0/24 -i ath0 -o ath0 -j ACCEPT
iptables -Aforwarding_rule-s 10.42.0.0/24 ! -d 10.42.0.0/24 -o ath0 -i ath0 -j ACCEPT
Could you explain what kind of traffic you actually want to block ? I also don't understand what packet storm you are afraid of.
Regards,
Marek
Quoted 12 lines · Marek Lindner
Quoted 11 lines · On Tuesday 28 September 2010 21:08:53 Magosányi Árpád wrote:
But with three nodes in a linear topology user-A-B-C, the reject firewall chain (basically the FORWARD chain) eats up the packets
user->AThis is the same rule which rejects batman packetsA->Cwhich go through B, and there are the gate interface which I do not understand, so while the following rules seem to solve the problem at least for three nodes, I have a feeling that I am not on the right path, and maybe on a way to cause packet storms.
iptables -Aforwarding_rule-d 10.42.0.0/24 ! -s 10.42.0.0/24 -i ath0 -o ath0 -j ACCEPT
iptables -Aforwarding_rule-s 10.42.0.0/24 ! -d 10.42.0.0/24 -o ath0 -i ath0 -j ACCEPT
Quoted 3 lines
Could you explain what kind of traffic you actually want to block ? I also don't understand what packet storm you are afraid of.
I actually want to _enable_ traffic. Any traffic from any node in the mesh.
Regarding packet storm: I thought that the reject in the default iptables config might be there to stop propagation of some packets which would otherwise propagate and thus multiply in multiple routes. I am seeing batman packets rejected there. As you can see, working of batmand is somewhat a black magic for me, esp. wrt. the role of the gate interface.
From the fact that I had to touch the default firewall config to make it work has suggested that I either do some nonstandard thing, or I am doing it in the wrong way.
This is why I try to figure out whether my config is sound enough before I give it to my village (some 8 thousand people).
Quoted 10 lines · On Wednesday 29 September 2010 21:41:29 Magosányi Árpád wrote:
I actually want to enable traffic. Any traffic from any node in the mesh. Regarding packet storm: I thought that the reject in the default iptables config might be there to stop propagation of some packets which would otherwise propagate and thus multiply in multiple routes. I am seeing batman packets rejected there.
As you can see, working of batmand is somewhat a black magic for me, esp. wrt. the role of the gate interface.
From the fact that I had to touch the default firewall config to make
it work has suggested that I either do some nonstandard thing, or I am doing it in the wrong way.
Normally, you don't any specific rules to make it work - it works as-is out of the box unless your system configures a firewall. If you tell your system to disable the firewall entirely everything will work.
What distribution are you using ? OpenWRT ?
The gateway handling (i.e. gate0) is documented:
http://www.open-mesh.org/wiki/InternetTuning
Feel free to ask questions that the documentation does not answer. :-)
Quoted 2 lines
This is why I try to figure out whether my config is sound enough before I give it to my village (some 8 thousand people).
Wow, sounds interesting. Where will that be ?
Regards,
Marek
Quoted 6 lines · Marek Lindner
Normally, you don't any specific rules to make it work - it works as-is out of the box unless your system configures a firewall. If you tell your system to disable the firewall entirely everything will work.
What distribution are you using ? OpenWRT ?
I am using OpenWrt Backfire.
One sign that I am using nonstandard setting is that a non-batman node will end up in a throw route of the batman nodes. But if it should work without firewall, then I will be happy with my current setup.
Quoted 4 lines
The gateway handling (i.e. gate0) is documented:
http://www.open-mesh.org/wiki/InternetTuning
Feel free to ask questions that the documentation does not answer. :-)
Thank you for the link. What I am seeing is that traffic from local wifi net does not "go down" to the tunnels, but goes through the batman nodes untunneled. Maybe I have to set up some routes from the local wifi net? For your reference here is the network setup again (N is the node number, ip of the node/netmask len):
backbone batman network: 10.42.0.N/24 (for batman nodes)
local wifi net: 10.42.N.1/24 (for non-batman users, dhcp served from the node in force mode)
local lan: 10.43.N.1/24 (for wired users, DHCP served from the node)
Quoted 3 lines
Quoted 2 lines
This is why I try to figure out whether my config is sound enough before I give it to my village (some 8 thousand people).
Wow, sounds interesting. Where will that be ?
Páty, Hungary. We are experimenting with participatory democracy, and this is one of the side effects:)
Quoted 1 line · On Wednesday 29 September 2010 22:44:36 Magosányi Árpád wrote:
I am using OpenWrt Backfire.
Ok.
Quoted 3 lines
One sign that I am using nonstandard setting is that a non-batman node will end up in a throw route of the batman nodes. But if it should work without firewall, then I will be happy with my current setup.
A throw route is no problem by itself. It only means that the Linux kernel will leave the current table to check the next one for a suitable routing entry.
If you are interested in understanding the routing techniques used by batman:
http://www.open-mesh.org/wiki/RoutingVodoo
Quoted 8 lines
What I am seeing is that traffic from local wifi net does not "go down" to the tunnels, but goes through the batman nodes untunneled. Maybe I have to set up some routes from the local wifi net? For your reference here is the network setup again (N is the node number, ip of the node/netmask len):
backbone batman network: 10.42.0.N/24 (for batman nodes)
local wifi net: 10.42.N.1/24 (for non-batman users, dhcp served from the node in force mode)
local lan: 10.43.N.1/24 (for wired users, DHCP served from the node)
Does each node announce the "local wifi net" via HNA ? You will need these announcements to make the routing towards this addresses work. As soon as batmand knows that it is responsible for a certain IP address space it will add the appropriate routing entries for you.
Again, we have a document describing the process:
http://www.open-mesh.org/wiki/AnnouncingNetworks
Quoted 2 lines
Páty, Hungary. We are experimenting with participatory democracy, and this is one of the side effects:)
Cool! Good luck with your experiments. :-)
Regards,
Marek
Quoted 24 lines · Marek Lindner
Quoted 1 line · On Wednesday 29 September 2010 22:44:36 Magosányi Árpád wrote:
I am using OpenWrt Backfire.
Ok.
Quoted 3 lines
One sign that I am using nonstandard setting is that a non-batman node will end up in a throw route of the batman nodes. But if it should work without firewall, then I will be happy with my current setup.
A throw route is no problem by itself. It only means that the Linux kernel will leave the current table to check the next one for a suitable routing entry.
If you are interested in understanding the routing techniques used by batman:
http://www.open-mesh.org/wiki/RoutingVodooQuoted 8 lines
What I am seeing is that traffic from local wifi net does not "go down" to the tunnels, but goes through the batman nodes untunneled. Maybe I have to set up some routes from the local wifi net? For your reference here is the network setup again (N is the node number, ip of the node/netmask len):
backbone batman network: 10.42.0.N/24 (for batman nodes)
local wifi net: 10.42.N.1/24 (for non-batman users, dhcp served from the node in force mode)
local lan: 10.43.N.1/24 (for wired users, DHCP served from the node)
Quoted 8 lines
Does each node announce the "local wifi net" via HNA ? You will need these announcements to make the routing towards this addresses work. As soon as batmand knows that it is responsible for a certain IP address space it will add the appropriate routing entries for you.
Again, we have a document describing the process:
http://www.open-mesh.org/wiki/AnnouncingNetworks
I do announce local wifi net through HNA.
In the meantime my config started to not work. I saw that the node in the middle does REJECT tunnel traffic from packet filter, so added a firewall rule to accept everything in the FORWARD chain in all nodes. Then as packets started to come out from the system with tunnel source IP, I have added a MASQUERADE on the node which is connected to the internet gateway.
Now it works, but uses the tunnel in an assymetric way: packets out go through the tunnel, packets in go in the plain route.
13:38:53.464186 00:22:fa:95:1d:c4 > 00:0b:6b:3c:74:86, ethertype IPv4 (0x0800), length 74: 10.42.3.178.51723 > 1.2.3.4.80: Flags [S], seq 2151644104, win 5840, options [mss 1460,sackOK,TS val 6321427 ecr 0,nop,wscale 6], length 0 13:38:53.468526 00:0b:6b:3c:74:86 > 00:0b:6b:3c:73:56, ethertype IPv4 (0x0800), length 103: 10.42.0.3.4306 > 10.42.0.4.4306: UDP, length 61
13:38:53.469091 00:0b:6b:3c:73:56 > 00:0b:6b:3c:74:8c, ethertype IPv4 (0x0800), length 103: 10.42.0.3.4306 > 10.42.0.4.4306: UDP, length 61
13:38:53.474019 00:0b:6b:3c:74:8c > 00:0b:6b:3c:73:56, ethertype IPv4 (0x0800), length 74: 1.2.3.4.80 > 10.42.3.178.51723: Flags [S.], seq 1125027318, ack 2151644105, win 5792, options [mss 1460,sackOK,TS val 19849628 ecr 6321427,nop,wscale 5], length 0 13:38:53.474497 00:0b:6b:3c:73:56 > 00:0b:6b:3c:74:86, ethertype IPv4 (0x0800), length 74: 1.2.3.4.80 > 10.42.3.178.51723: Flags [S.], seq 1125027318, ack 2151644105, win 5792, options [mss 1460,sackOK,TS val 19849628 ecr 6321427,nop,wscale 5], length 0 13:38:53.475385 00:0b:6b:3c:74:86 > 00:22:fa:95:1d:c4, ethertype IPv4 (0x0800), length 74: 1.2.3.4.80 > 10.42.3.178.51723: Flags [S.], seq 1125027318, ack 2151644105, win 5792, options [mss 1460,sackOK,TS val 19849628 ecr 6321427,nop,wscale 5], length 0
Quoted 7 lines
Quoted 2 lines
Páty, Hungary. We are experimenting with participatory democracy, and this is one of the side effects:)
Cool! Good luck with your experiments. :-)
Regards,
Marek
Quoted 10 lines · On Thursday 30 September 2010 13:50:52 Magosányi Árpád wrote:
I do announce local wifi net through HNA.
In the meantime my config started to not work. I saw that the node in the middle does REJECT tunnel traffic from packet filter, so added a firewall rule to accept everything in the FORWARD chain in all nodes. Then as packets started to come out from the system with tunnel source IP, I have added a MASQUERADE on the node which is connected to the internet gateway.Now it works, but uses the tunnel in an assymetric way: packets out go through the tunnel, packets in go in the plain route.
Batmand offers 2 types of tunnel: the "half tunnel" and the "full tunnel" which allows you to choose what suits you best.
The full tunnel has the advantage that batmand can automatically detect whether the gateway has a working internet connection and switch to another gateway if necessary. As tunneling in user space is painfully CPU intense the full tunnel comes with a performance penalty with many simultaneous users. If that is an issue you might want to have a look at the batgat kernel module which does the tunneling in kernel space.
The half tunnel does not run into the performance issue that easily but can't detect internet availability. As the user traffic is not natted in the tunnel this mode is more suitable for SIP and similar protocols.
Half tunnel mode is the default, full tunneling can be activated by natting on the client's gate0 interface:
iptables -t nat -I POSTROUTING -o gate0 -j MASQUERADE
Regards,
Marek