What's required for a stateful firewall + ipvs in 2.6 kernel?
I'm trying to get a handle on whether or not it's possible to set up the following on a redundant pair of boxes:
1. Stateful iptables firewall
2. LVS director (keepalived)
3. DNAT, SNAT and fwmarks
4. Connection synchronization for failover
I currently have CentOS/RHEL 5 running 1, 2 and 3 above but the RHEL 2.6.18-* kernels don't export LVS connections to netfilter resulting in lots of INVALID packets on return traffic from real servers. It also prevents connection synchronization to the backup fw/director for failover. Google has been giving me conflicting results on the following questions:
* Do the antefacto patches allow netfilter to access connections managed by ipvs and support DNAT, SNAT and fwmarks used in the LVS configuration?
* Has anyone gotten this to work on RHEL/CentOS via a kernel recompile with the antefacto patches?
If so, is there anything needed beyond the following?:
1. Recompile CentOS kernel (2.6.18 ok?) with Antefacto patches (http://www.ssi.bg/~ja/nfct/)
2. Setup conntrackd - will mirror the connection information synchronized by keepalived at the netfilter level. Will conntrackd work on RHEL/CentOS 5.2?
Are libntnetlink or libnetfilter_conntrack required? I have been reading all day but don't yet follow how all of the pieces go together.
Many thanks for any advice here...
Brian
Quoted 6 lines · Brian Ghidinelli
I'm trying to get a handle on whether or not it's possible to set up the following on a redundant pair of boxes:
1. Stateful iptables firewall
2. LVS director (keepalived)
3. DNAT, SNAT and fwmarks
4. Connection synchronization for failover
You should easily be able to get SPI (1), NAT (3), and failover (4) between multiple systems. However I'm not sure if you will get LVS (2) to play properly in this or not. Traditionally LVS worked independently / completely out side of IPTables (1 and 3) and thus was not able to be synchronized / failed over (4) between multiple boxen. This does not mean that it can not be done, just that it is not going to be documented in the usual locations if it is possible.
Quoted 1 line
* Do the antefacto patches allow netfilter to access connections managed by ipvs and support DNAT, SNAT and fwmarks used in the LVS configuration?
Based on the (below) referenced web page from Julian, yes to some extent it does..
Quoted 1 line
2. Setup conntrackd - will mirror the connection information synchronized by keepalived at the netfilter level. Will conntrackd work on RHEL/CentOS 5.2?
It is my (mis)understanding that keepalived does not do the synchronization, rather just the monitoring of things. Conntrackd will do the synchronization for NetFilter.
As far as whether or not conntrackd will work on RHEL/CentOS, it should.
I don't know of any reason you can't compile it and get it to work.
You may have to change some underlying libraries if versions are not correct (I don't know b/c I run different distro(s)).
Quoted 1 line
Are libntnetlink or
libnetfilter_conntrackrequired? I have been reading all day but don't yet follow how all of the pieces go together.
I don't know. If you read the documentation with conntrackd you should be able to find out if libnetlink / libnetfilter are needed or not. I would not be surprised if you need libnetfilter.
Grant. . . .
Quoted 7 lines · Grant Taylor
Quoted 4 lines
1. Stateful iptables firewall
2. LVS director (keepalived)
3. DNAT, SNAT and fwmarks
4. Connection synchronization for failover...
synchronized / failed over (4) between multiple boxen. This does not mean that it can not be done, just that it is not going to be documented in the usual locations if it is possible.
That's the issue... there are a lot of posts about LVS and netfilter on Austintek.com and other sites but the dates range from 2000 to 2006 or so making it hard to figure out what's current.
In sysadmining, I don't really like to be the pioneer. :) No one else has turned an RHEL box into a Firewall + LVS Director?
Quoted 1 line
It is my (mis)understanding that keepalived does not do the synchronization, rather just the monitoring of things. Conntrackd will do the synchronization for NetFilter.
I believe keepalived synchronizes the LVS connections between ipvs on the two boxes. There is a config option "lvs_sync_daemon_inteface" for this (as I understand it).
This is only half the picture though, and conntrackd appears to solve the other half by also keeping netfilter in sync about which connections are already established or related so iptables rules don't kill valid sessions.
So in the end I suppose the real question is whether or not anyone has successfully used the Antefacto patches on RHEL? I will try the lvs-users mailing list for that one...
Thanks for the help Grant,
Brian
Quoted 1 line · Brian Ghidinelli
That's the issue... there are a lot of posts about LVS and netfilter on Austintek.com and other sites but the dates range from 2000 to 2006 or so making it hard to figure out what's current.
*nod* This is the case with a lot of things, not just LVS.
Quoted 1 line
In sysadmining, I don't really like to be the pioneer. :) No one else has turned an RHEL box into a Firewall + LVS Director?
I doubt that you are the first, but I don't know that others have documented things for people to find.
Quoted 1 line
I believe keepalived synchronizes the LVS connections between ipvs on the two boxes. There is a config option "
lvs_sync_daemon_inteface" for this (as I understand it).
Ok...
Quoted 1 line
This is only half the picture though, and conntrackd appears to solve the other half by also keeping netfilter in sync about which connections are already established or related so iptables rules don't kill valid sessions.
*nod*
Quoted 1 line
So in the end I suppose the real question is whether or not anyone has successfully used the Antefacto patches on RHEL? I will try the lvs-users mailing list for that one...
Please follow up with what you find so others searching this archive in the future will have some information.
Quoted 1 line
Thanks for the help Grant,
You are welcome.
Grant. . . .
Quoted 18 lines · Brian Ghidinelli
I'm trying to get a handle on whether or not it's possible to set up the following on a redundant pair of boxes:
1. Stateful iptables firewall
2. LVS director (keepalived)
3. DNAT, SNAT and fwmarks
4. Connection synchronization for failoverI currently have CentOS/RHEL 5 running 1, 2 and 3 above but the RHEL 2.6.18-* kernels don't export LVS connections to netfilter resulting in lots of INVALID packets on return traffic from real servers. It also prevents connection synchronization to the backup fw/director for failover. Google has been giving me conflicting results on the following questions:
* Do the antefacto patches allow netfilter to access connections managed by ipvs and support DNAT, SNAT and fwmarks used in the LVS configuration?
Quoted 7 lines
* Has anyone gotten this to work on RHEL/CentOS via a kernel recompile with the antefacto patches?
If so, is there anything needed beyond the following?:
1. Recompile CentOS kernel (2.6.18 ok?) with Antefacto patches
(http://www.ssi.bg/~ja/nfct/)
The last time that I had a look at the antefacto patch it look to me like a hack. IIRC, the problem is the LVS design (at least time ago when I had a look at it) as it bypasses the network stack. This screws up the possibility of having stateful firewalling and LVS.
Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
Quoted 1 line · Pablo Neira Ayuso
The last time that I had a look at the antefacto patch it look to me like a hack. IIRC, the problem is the LVS design (at least time ago when I had a look at it) as it bypasses the network stack. This screws up the possibility of having stateful firewalling and LVS.
I can offer a recent confirmation (with in the last three months) that LVS does indeed still interfere with firewalling.
Grant. . . .