netfilter: xtables: add cluster match
Subsystems (2)
Pablo Neira Ayuso
maintainerPATCH 1/1 netfilter: xtables: add cluster match
This patch adds the iptables cluster match. This match can be used to deploy gateway and back-end load-sharing clusters. The cluster can be composed of 32 nodes maximum (although I have only tested this with two nodes, so I cannot tell what is the real scalability limit of this solution in terms of cluster nodes).
Assuming that all the nodes see all packets (see below for an example on how to do that if your switch does not allow this), the cluster match decides if this node has to handle a packet given:
(jhash(source IP) % total_nodes) & node_mask
For related connections, the master conntrack is used. The following is an example of its use to deploy a gateway cluster composed of two nodes (where this is the node 1):
iptables -I PREROUTING -t mangle -i eth1 -m cluster \ --cluster-total-nodes 2 --cluster-local-node 1 \ --cluster-proc-name eth1 -j MARK --set-mark 0xffff iptables -A PREROUTING -t mangle -i eth1 \ -m mark ! --mark 0xffff -j DROP iptables -A PREROUTING -t mangle -i eth2 -m cluster \ --cluster-total-nodes 2 --cluster-local-node 1 \ --cluster-proc-name eth2 -j MARK --set-mark 0xffff iptables -A PREROUTING -t mangle -i eth2 \ -m mark ! --mark 0xffff -j DROP
And the following commands to make all nodes see the same packets:
ip maddr add 01:00:5e:00:01:01 dev eth1
ip maddr add 01:00:5e:00:01:02 dev eth2
arptables -I OUTPUT -o eth1 --h-length 6 \ -j mangle --mangle-mac-s 01:00:5e:00:01:01 arptables -I INPUT -i eth1 --h-length 6 \ --destination-mac 01:00:5e:00:01:01 \ -j mangle --mangle-mac-d 00:zz:yy:xx:5a:27 arptables -I OUTPUT -o eth2 --h-length 6 \ -j mangle --mangle-mac-s 01:00:5e:00:01:02 arptables -I INPUT -i eth2 --h-length 6 \ --destination-mac 01:00:5e:00:01:02 \ -j mangle --mangle-mac-d 00:zz:yy:xx:5a:27
In the case of TCP connections, pickup facility has to be disabled to avoid marking TCP ACK packets coming in the reply direction as valid.
echo 0 > /proc/sys/net/netfilter/nf_conntrack_tcp_loose
BTW, some final notes:
* This match mangles the skbuff pkt_type in case that it detectsPACKET_MULTICAST for a non-multicast address. This may be done in a PKTTYPE target for this sole purpose.
* This match supersedes the CLUSTERIP target.
---
include/linux/netfilter/xt_cluster.h | 15 +++ net/netfilter/Kconfig | 16 +++ net/netfilter/Makefile | 1 net/netfilter/xt_cluster.c | 164 ++++++++++++++++++++++++++++++++++ 4 files changed, 196 insertions(+), 0 deletions(-) create mode 100644 include/linux/netfilter/xt_cluster.h create mode 100644 net/netfilter/xt_cluster.c
include/linux/netfilter/xt_cluster.h new file +15 -0
@@ -0,0 +1,15 @@ +#ifndef _XT_CLUSTER_MATCH_H +#define _XT_CLUSTER_MATCH_H + +enum xt_cluster_flags { +XT_CLUSTER_F_INV = (1 <<0 ) +}; + +struct xt_cluster_match_info { +u_int32_t total_nodes; +u_int32_t node_mask; +u_int32_t hash_seed; +u_int32_t flags; +}; + +#endif /* _XT_CLUSTER_MATCH_H */
net/netfilter/Kconfig +16 -0
@@ -488,6 +488,22 @@ config NETFILTER_XT_TARGET_TCPOPTSTRIP This option adds a "TCPOPTSTRIP" target, which allows you to strip TCP options from TCP packets. +config NETFILTER_XT_MATCH_CLUSTER + tristate '"cluster" match support' + depends on NF_CONNTRACK + depends on NETFILTER_ADVANCED + ---help--- + This option allows you to build work-load-sharing clusters of + network servers/stateful firewalls without having a dedicated + load-balancing router/server/switch. Basically, this match returns + true when the packet must be handled by this cluster node. Thus, + all nodes see all packets and this match decides which node handles + what packets. The work-load sharing algorithm is based on source + address hashing. + + If you say Y or M here, try `iptables -m cluster --help` for + more information. + config NETFILTER_XT_MATCH_COMMENT tristate '"comment" match support' depends on NETFILTER_ADVANCED
net/netfilter/Makefile +1 -0
@@ -57,6 +57,7 @@ obj-$(CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP) += xt_TCPOPTSTRIP.o obj-$(CONFIG_NETFILTER_XT_TARGET_TRACE) += xt_TRACE.o # matches +obj-$(CONFIG_NETFILTER_XT_MATCH_CLUSTER) += xt_cluster.o obj-$(CONFIG_NETFILTER_XT_MATCH_COMMENT) += xt_comment.o obj-$(CONFIG_NETFILTER_XT_MATCH_CONNBYTES) += xt_connbytes.o obj-$(CONFIG_NETFILTER_XT_MATCH_CONNLIMIT) += xt_connlimit.o
net/netfilter/xt_cluster.c new file +164 -0
164 lines not on this page · raw message
include/linux/netfilter/xt_cluster.h line 13
Quoted 10 lines · Pablo Neira Ayuso
include/linux/netfilter/xt_cluster.h new file +10
-0+enum xt_cluster_flags { +XT_CLUSTER_F_INV = (1 <<0 ) +}; + +struct xt_cluster_match_info { +u_int32_t total_nodes; +u_int32_t node_mask; +u_int32_t hash_seed; +u_int32_t flags; +};
This doesn't seem like such a hot idea. I haven't seen the new userspace patch, but assuming you're interested in the flags and not ignoring them in userspace, the user has to specify the hash seed for rule deletions.
You also have to chose the same seed for all nodes in a cluster. This seems needlessly complicated, I'd suggest to simply use zero.
Pablo Neira Ayuso
maintainerQuoted 16 lines · Patrick McHardy
include/linux/netfilter/xt_cluster.h line 13
Quoted 10 lines · Pablo Neira Ayuso
include/linux/netfilter/xt_cluster.h new file +10
-0+enum xt_cluster_flags { +XT_CLUSTER_F_INV = (1 <<0 ) +}; + +struct xt_cluster_match_info { +u_int32_t total_nodes; +u_int32_t node_mask; +u_int32_t hash_seed; +u_int32_t flags; +};This doesn't seem like such a hot idea. I haven't seen the new userspace patch, but assuming you're interested in the flags and not ignoring them in userspace, the user has to specify the hash seed for rule deletions.
The user has to specify the hash seed to delete the rule if it's non-zero, otherwise it must be specified. The hash seed is optional. I don't quite see the problem.
Quoted 2 lines
You also have to chose the same seed for all nodes in a cluster. This seems needlessly complicated, I'd suggest to simply use zero.
One may want to forge traffic to flood a single node? The hash seed avoids this.
Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
Quoted 18 lines · Pablo Neira Ayuso
Quoted 16 lines · Patrick McHardy
include/linux/netfilter/xt_cluster.h line 13
Quoted 10 lines · Pablo Neira Ayuso
include/linux/netfilter/xt_cluster.h new file +10
-0+enum xt_cluster_flags { +XT_CLUSTER_F_INV = (1 <<0 ) +}; + +struct xt_cluster_match_info { +u_int32_t total_nodes; +u_int32_t node_mask; +u_int32_t hash_seed; +u_int32_t flags; +};This doesn't seem like such a hot idea. I haven't seen the new userspace patch, but assuming you're interested in the flags and not ignoring them in userspace, the user has to specify the hash seed for rule deletions.
Quoted 1 line
The user has to specify the hash seed to delete the rule if it's non-zero, otherwise it must be specified. The hash seed is optional. I don't quite see the problem.
Its a parameter without a meaning, the user is needlessly bothered with this.
Quoted 4 lines
Quoted 2 lines
You also have to chose the same seed for all nodes in a cluster. This seems needlessly complicated, I'd suggest to simply use zero.
One may want to forge traffic to flood a single node? The hash seed avoids this.
No, it only makes it easier to shut off since I have to use the same source address to be sure I hit the same node. This seems like a valid argument though.
The fact that you have to specify it for deletion still seems unnecesary though. You would never have two rules differing only in the seed value since that would mean the node is part of two clusters. So we might as well move it to the end and ignore it in userspace. What do you think? In case you agree, I also think "secret" would be a more fitting name.
Pablo Neira Ayuso
maintainerQuoted 19 lines · Patrick McHardy
Quoted 18 lines · Pablo Neira Ayuso
Quoted 16 lines · Patrick McHardy
include/linux/netfilter/xt_cluster.h line 13
Quoted 10 lines · Pablo Neira Ayuso
include/linux/netfilter/xt_cluster.h new file +10
-0+enum xt_cluster_flags { +XT_CLUSTER_F_INV = (1 <<0 ) +}; + +struct xt_cluster_match_info { +u_int32_t total_nodes; +u_int32_t node_mask; +u_int32_t hash_seed; +u_int32_t flags; +};This doesn't seem like such a hot idea. I haven't seen the new userspace patch, but assuming you're interested in the flags and not ignoring them in userspace, the user has to specify the hash seed for rule deletions.
Quoted 8 lines
Quoted 3 lines
The user has to specify the hash seed to delete the rule if it's non-zero, otherwise it must be specified. The hash seed is optional. I don't quite see the problem.
Its a parameter without a meaning, the user is needlessly bothered with this.
From the user view, yes. No matter what value you set as long as it is
the same in all the cluster nodes.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
You also have to chose the same seed for all nodes in a cluster. This seems needlessly complicated, I'd suggest to simply use zero.
One may want to forge traffic to flood a single node? The hash seed avoids this.
Quoted 8 lines
No, it only makes it easier to shut off since I have to use the same source address to be sure I hit the same node. This seems like a valid argument though.
The fact that you have to specify it for deletion still seems unnecesary though. You would never have two rules differing only in the seed value since that would mean the node is part of two clusters. So we might as well move it to the end and ignore it in userspace. What do you think?
But the value has to be the same in all the cluster nodes, so how can it be set to ensure that it is the same value?
Quoted 1 line
In case you agree, I also think "secret" would be a more fitting name.
I can rename the field to "secret" in the structure or change the iptables cluster match option to be "--cluster-secret" instead of "--cluster-hash-seed" if you like.
Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
Quoted 8 lines · Pablo Neira Ayuso
Quoted 4 lines · Patrick McHardy
The fact that you have to specify it for deletion still seems unnecesary though. You would never have two rules differing only in the seed value since that would mean the node is part of two clusters. So we might as well move it to the end and ignore it in userspace. What do you think?
But the value has to be the same in all the cluster nodes, so how can it be set to ensure that it is the same value?
I only meant ignoring it on comparisons of course, just as we do with all the private pointer stuff. Anyways, its not that important and it in fact would be slightly different behaviour from what we do in other cases, where we only ignore state. So perhaps not a good idea after all.
Quoted 5 lines
Quoted 1 line
In case you agree, I also think "secret" would be a more fitting name.
I can rename the field to "secret" in the structure or change the iptables cluster match option to be "--cluster-secret" instead of "--cluster-hash-seed" if you like.
Its more fitting in my opinion, but I don't really care.
Pablo Neira Ayuso
maintainerQuoted 8 lines · Patrick McHardy
Quoted 5 lines · Pablo Neira Ayuso
Quoted 1 line
In case you agree, I also think "secret" would be a more fitting name.
I can rename the field to "secret" in the structure or change the iptables cluster match option to be "--cluster-secret" instead of "--cluster-hash-seed" if you like.
Its more fitting in my opinion, but I don't really care.
I don't either, would you apply the patch as is now?
Signature · 1 line
"Los honestos son inadaptados sociales" -- Les Luthiers
Quoted 6 lines · Pablo Neira Ayuso
Quoted 4 lines · Patrick McHardy
Quoted 2 lines · Pablo Neira Ayuso
Quoted 2 lines
In case you agree, I also think "secret" would be a more fitting name.
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
I can rename the field to "secret" in the structure or change the iptables cluster match option to be "--cluster-secret" instead of "--cluster-hash-seed" if you like.
Its more fitting in my opinion, but I don't really care.
I don't either, would you apply the patch as is now?
I will :)
Quoted 58 lines · Pablo Neira Ayuso
This patch adds the iptables cluster match. This match can be used to deploy gateway and back-end load-sharing clusters. The cluster can be composed of 32 nodes maximum (although I have only tested this with two nodes, so I cannot tell what is the real scalability limit of this solution in terms of cluster nodes).
Assuming that all the nodes see all packets (see below for an example on how to do that if your switch does not allow this), the cluster match decides if this node has to handle a packet given:
(jhash(source IP) %
total_nodes) &node_maskFor related connections, the master conntrack is used. The following is an example of its use to deploy a gateway cluster composed of two nodes (where this is the node 1):
iptables -I PREROUTING -t mangle -i eth1 -m cluster \ --cluster-total-nodes 2 --cluster-local-node 1 \ --cluster-proc-name eth1 -j MARK --set-mark 0xffff iptables -A PREROUTING -t mangle -i eth1 \ -m mark ! --mark 0xffff -j DROP iptables -A PREROUTING -t mangle -i eth2 -m cluster \ --cluster-total-nodes 2 --cluster-local-node 1 \ --cluster-proc-name eth2 -j MARK --set-mark 0xffff iptables -A PREROUTING -t mangle -i eth2 \ -m mark ! --mark 0xffff -j DROPAnd the following commands to make all nodes see the same packets:
ip maddr add 01:00:5e:00:01:01 dev eth1
ip maddr add 01:00:5e:00:01:02 dev eth2arptables -I OUTPUT -o eth1 --h-length 6 \ -j mangle --mangle-mac-s 01:00:5e:00:01:01 arptables -I INPUT -i eth1 --h-length 6 \ --destination-mac 01:00:5e:00:01:01 \ -j mangle --mangle-mac-d 00:zz:yy:xx:5a:27 arptables -I OUTPUT -o eth2 --h-length 6 \ -j mangle --mangle-mac-s 01:00:5e:00:01:02 arptables -I INPUT -i eth2 --h-length 6 \ --destination-mac 01:00:5e:00:01:02 \ -j mangle --mangle-mac-d 00:zz:yy:xx:5a:27In the case of TCP connections, pickup facility has to be disabled to avoid marking TCP ACK packets coming in the reply direction as valid.
echo 0 >
/proc/sys/net/netfilter/nf_conntrack_tcp_looseBTW, some final notes:
* This match mangles the skbuff
pkt_typein case that it detectsPACKET_MULTICASTfor a non-multicast address. This may be done in a PKTTYPE target for this sole purpose.
* This match supersedes the CLUSTERIP target.Signed-off-by Pablo Neira Ayuso <[email protected]>---
Quoted 7 lines
include/linux/netfilter/xt_cluster.h | 15 +++ net/netfilter/Kconfig | 16 +++ net/netfilter/Makefile | 1 net/netfilter/xt_cluster.c | 164 ++++++++++++++++++++++++++++++++++ 4 files changed, 196 insertions(+), 0 deletions(-) create mode 100644 include/linux/netfilter/xt_cluster.h create mode 100644 net/netfilter/xt_cluster.c
Applied, thanks. I've also added xt_cluster.h to the Kbuild file so the header will be installed.