transparent proxying
Subsystems (2)
PATCH 1/1 transparent proxying
This seems to be a pretty clean solution to a real problem.
Ultimately I would like to see IPVS move into the forward chain. This seems to be a nice way to explore that, without breaking any existing setups.
Signature · 115 lines
Horms H: http://www.vergenet.net/~horms/ W: http://www.valinux.co.jp/en/ [IPVS] transparent proxying Patch from Jinhua Luo <[email protected]> to allow a web cluseter using transparent proxying. It works by simply grabing packets that have the fwmark set and have not already been processed by ipvs (ip_vs_out) and throwing them into ip_vs_in. See: http://archive.linuxvirtualserver.org/html/lvs-users/2006-11/msg00261.html Normally LVS packets are processed by ip_vs_in fron on the INPUT chain, and packets that are processed in this way never show up on the FORWARD chain, so they won't hit this rule. This patch seems like a good precursor to moving LVS permanantly to the FORWARD chain. As I'm struggling to think how it could break things. The changes to the original patch are: * Reformated to use tabs for indentation (instead of 4 spaces) * Reformated to be < 80 columns wide * Added some comments * Rewrote description (this text) Signed-off-by: Simon Horman <[email protected]> Signed-off-by: Jinhua Luo <[email protected]> Index: linux-2.6/net/ipv4/ipvs/ip_vs_core.c =================================================================== --- linux-2.6.orig/net/ipv4/ipvs/ip_vs_core.c 2006-11-28 15:30:00.000000000 +0900 +++ linux-2.6/net/ipv4/ipvs/ip_vs_core.c 2006-11-29 10:27:49.000000000 +0900 @@ -23,7 +23,9 @@ * Changes: * Paul `Rusty' Russell properly handle non-linear skbs * Harald Welte don't use nfcache - * + * Jinhua Luo redirect packets with fwmark on + * NF_IP_FORWARD chain to ip_vs_in(), + * mainly for transparent cache cluster */ #include <linux/module.h> @@ -1070,6 +1072,26 @@ return ip_vs_in_icmp(pskb, &r, hooknum); } +/* + * This is hooked into the NF_IP_FORWARD. It catches + * packets that have not already been handled by ipvs (out) + * and have a fwmark set. This is to allow transparent proxying + * of fwmark virtual services. + * + * It will not process packets that are handled by ipvs (in) + * as they never traverse the NF_IP_FORWARD. + */ +static unsigned int +ip_vs_forward_with_fwmark(unsigned int hooknum, struct sk_buff **pskb, + const struct net_device *in, + const struct net_device *out, + int (*okfn)(struct sk_buff *)) +{ + if ((*pskb)->ipvs_property || ! (*pskb)->nfmark) + return NF_ACCEPT; + + return ip_vs_in(hooknum, pskb, in, out, okfn); +} /* After packet filtering, forward packet through VS/DR, VS/TUN, or VS/NAT(change destination), so that filtering rules can be @@ -1082,6 +1104,16 @@ .priority = 100, }; +/* Allow transparent proxying by fishing packets + * out of the forward chain. */ +static struct nf_hook_ops ip_vs_forward_with_fwmark_ops = { + .hook = ip_vs_forward_with_fwmark, + .owner = THIS_MODULE, + .pf = PF_INET, + .hooknum = NF_IP_FORWARD, + .priority = 101, +}; + /* After packet filtering, change source only for VS/NAT */ static struct nf_hook_ops ip_vs_out_ops = { .hook = ip_vs_out, @@ -1160,9 +1192,17 @@ goto cleanup_postroutingops; } + ret = nf_register_hook(&ip_vs_forward_with_fwmark_ops); + if (ret < 0) { + IP_VS_ERR("can't register forward_with_fwmark hook.\n"); + goto cleanup_forwardicmpops; + } + IP_VS_INFO("ipvs loaded.\n"); return ret; + cleanup_forwardicmpops: + nf_unregister_hook(&ip_vs_forward_icmp_ops); cleanup_postroutingops: nf_unregister_hook(&ip_vs_post_routing_ops); cleanup_outops: @@ -1182,6 +1222,7 @@ static void __exit ip_vs_cleanup(void) { + nf_unregister_hook(&ip_vs_forward_with_fwmark_ops); nf_unregister_hook(&ip_vs_forward_icmp_ops); nf_unregister_hook(&ip_vs_post_routing_ops); nf_unregister_hook(&ip_vs_out_ops);
* Horms <[email protected]> 2006-11-29 15:21
Quoted 63 lines
This seems to be a pretty clean solution to a real problem.
Ultimately I would like to see IPVS move into the forward chain. This seems to be a nice way to explore that, without breaking any existing setups.
Signature · 32 lines
Horms H: http://www.vergenet.net/~horms/ W: http://www.valinux.co.jp/en/ [IPVS] transparent proxying Patch from Jinhua Luo <[email protected]> to allow a web cluseter using transparent proxying. It works by simply grabing packets that have the fwmark set and have not already been processed by ipvs (ip_vs_out) and throwing them into ip_vs_in. See: http://archive.linuxvirtualserver.org/html/lvs-users/2006-11/msg00261.html Normally LVS packets are processed by ip_vs_in fron on the INPUT chain, and packets that are processed in this way never show up on the FORWARD chain, so they won't hit this rule. This patch seems like a good precursor to moving LVS permanantly to the FORWARD chain. As I'm struggling to think how it could break things. The changes to the original patch are: * Reformated to use tabs for indentation (instead of 4 spaces) * Reformated to be < 80 columns wide * Added some comments * Rewrote description (this text) Signed-off-by: Simon Horman <[email protected]> Signed-off-by: Jinhua Luo <[email protected]> Index: linux-2.6/net/ipv4/ipvs/ip_vs_core.c ===================================================================net/ipv4/ipvs/ip_vs_core.c +10
-1@@ -23,7 +23,9 @@* Changes: * Paul `Rusty' Russell properly handle non-linear skbs * Harald Welte don't use nfcache -* +* Jinhua Luo redirect packets with fwmark on +* NF_IP_FORWARD chain to ip_vs_in(), +* mainly for transparent cache cluster */ #include <linux/module.h> @@ -1070,6 +1072,26 @@return ip_vs_in_icmp (pskb, &r, hooknum); } +/* +* This is hooked into the NF_IP_FORWARD. It catches +* packets that have not already been handled by ipvs (out) +* and have a fwmark set. This is to allow transparent proxying +* of fwmark virtual services. +* +* It will not process packets that are handled by ipvs (in)
net/ipv4/ipvs/ip_vs_core.c line 1091
Quoted 10 lines
net/ipv4/ipvs/ip_vs_core.c +10
-0+* as they never traverse the NF_IP_FORWARD. +*/ +static unsigned int +ip_vs_forward_with_fwmark (unsigned int hooknum,struct sk_buff **pskb, +const struct net_device *in, +const struct net_device *out, +int (*okfn)(struct sk_buff *)) +{ +if ((*pskb)->ipvs_property || ! (*pskb)->nfmark) +return NF_ACCEPT ;
This patch seems to be based on an old tree, I've renamed nfmark to mark in net-2.6.20. The term fwmark and nfmark shouldn't be used anymore.
Quoted 70 lines · Thomas Graf
* Horms <[email protected]> 2006-11-29 15:21
Quoted 69 lines
This seems to be a pretty clean solution to a real problem.
Ultimately I would like to see IPVS move into the forward chain. This seems to be a nice way to explore that, without breaking any existing setups.
Signature · 32 lines
Horms H: http://www.vergenet.net/~horms/ W: http://www.valinux.co.jp/en/ [IPVS] transparent proxying Patch from Jinhua Luo <[email protected]> to allow a web cluseter using transparent proxying. It works by simply grabing packets that have the fwmark set and have not already been processed by ipvs (ip_vs_out) and throwing them into ip_vs_in. See: http://archive.linuxvirtualserver.org/html/lvs-users/2006-11/msg00261.html Normally LVS packets are processed by ip_vs_in fron on the INPUT chain, and packets that are processed in this way never show up on the FORWARD chain, so they won't hit this rule. This patch seems like a good precursor to moving LVS permanantly to the FORWARD chain. As I'm struggling to think how it could break things. The changes to the original patch are: * Reformated to use tabs for indentation (instead of 4 spaces) * Reformated to be < 80 columns wide * Added some comments * Rewrote description (this text) Signed-off-by: Simon Horman <[email protected]> Signed-off-by: Jinhua Luo <[email protected]> Index: linux-2.6/net/ipv4/ipvs/ip_vs_core.c ===================================================================net/ipv4/ipvs/ip_vs_core.c +16
-1@@ -23,7 +23,9 @@* Changes: * Paul `Rusty' Russell properly handle non-linear skbs * Harald Welte don't use nfcache -* +* Jinhua Luo redirect packets with fwmark on +* NF_IP_FORWARD chain to ip_vs_in(), +* mainly for transparent cache cluster */ #include <linux/module.h> @@ -1070,6 +1072,26 @@return ip_vs_in_icmp (pskb, &r, hooknum); } +/* +* This is hooked into the NF_IP_FORWARD. It catches +* packets that have not already been handled by ipvs (out) +* and have a fwmark set. This is to allow transparent proxying +* of fwmark virtual services. +* +* It will not process packets that are handled by ipvs (in) +* as they never traverse the NF_IP_FORWARD. +*/ +static unsigned int +ip_vs_forward_with_fwmark(unsigned int hooknum, struct sk_buff **pskb, + const struct net_device *in, + const struct net_device *out,
Quoted 8 lines
net/ipv4/ipvs/ip_vs_core.c line 1091
Quoted 4 lines
net/ipv4/ipvs/ip_vs_core.c +4
-0+ int (*okfn)(struct sk_buff *)) +{ +if ((*pskb)->ipvs_property || ! (*pskb)->nfmark) +return NF_ACCEPT ;This patch seems to be based on an old tree, I've renamed nfmark to mark in net-2.6.20. The term fwmark and nfmark shouldn't be used anymore.
Sorry, I based this patch on Linus's tree. I'll port it to net-2.6.20.
Signature · 3 lines
Horms H: http://www.vergenet.net/~horms/ W: http://www.valinux.co.jp/en/
Quoted 1 line · Horms
On Wed, Nov 29, 2006 at 03:15:23PM +0100, Thomas Graf wrote:
[split]
Quoted 5 lines
Quoted 3 lines
This patch seems to be based on an old tree, I've renamed nfmark to mark in net-2.6.20. The term fwmark and nfmark shouldn't be used anymore.
Sorry, I based this patch on Linus's tree. I'll port it to net-2.6.20.
This took too long for me to get around to :(
Am I correct in thinking I just need to replace fwmark with mark? If so, the updated version is below.
Signature · 111 lines
Horms H: http://www.vergenet.net/~horms/ W: http://www.valinux.co.jp/en/ [IPVS] transparent proxying Patch from home_king <[email protected]> to allow a web cluseter using transparent proxying. It works by simply grabing packets that have the fwmark set and have not already been processed by ipvs (ip_vs_out) and throwing them into ip_vs_in. See: http://archive.linuxvirtualserver.org/html/lvs-users/2006-11/msg00261.html Normally LVS packets are processed by ip_vs_in fron on the INPUT chain, and packets that are processed in this way never show up on the FORWARD chain, so they won't hit this rule. This patch seems like a good precursor to moving LVS permanantly to the FORWARD chain. As I'm struggling to think how it could break things. Reformated to use tabs for indentation (instead of 4 spaces) Reformated to be < 80 columns wide Updated fwmark to mark Cc: Jinhua Luo <[email protected]> Signed-off-by: Simon Horman <[email protected]> Index: net-2.6/net/ipv4/ipvs/ip_vs_core.c =================================================================== --- net-2.6.orig/net/ipv4/ipvs/ip_vs_core.c 2006-12-18 11:46:10.000000000 +0900 +++ net-2.6/net/ipv4/ipvs/ip_vs_core.c 2006-12-18 12:13:32.000000000 +0900 @@ -23,7 +23,9 @@ * Changes: * Paul `Rusty' Russell properly handle non-linear skbs * Harald Welte don't use nfcache - * + * Jinhua Luo redirect packets with fwmark on + * NF_IP_FORWARD chain to ip_vs_in(), + * mainly for transparent cache cluster */ #include <linux/module.h> @@ -1070,6 +1072,26 @@ return ip_vs_in_icmp(pskb, &r, hooknum); } +/* + * This is hooked into the NF_IP_FORWARD. It catches + * packets that have not already been handled by ipvs (out) + * and have a fwmark set. This is to allow transparent proxying + * of fwmark virtual services. + * + * It will not process packets that are handled by ipvs (in) + * as they never traverse the NF_IP_FORWARD. + */ +static unsigned int +ip_vs_forward_with_fwmark(unsigned int hooknum, struct sk_buff **pskb, + const struct net_device *in, + const struct net_device *out, + int (*okfn)(struct sk_buff *)) +{ + if ((*pskb)->ipvs_property || ! (*pskb)->mark) + return NF_ACCEPT; + + return ip_vs_in(hooknum, pskb, in, out, okfn); +} /* After packet filtering, forward packet through VS/DR, VS/TUN, or VS/NAT(change destination), so that filtering rules can be @@ -1082,6 +1104,16 @@ .priority = 100, }; +/* Allow transparent proxying by fishing packets + * out of the forward chain. */ +static struct nf_hook_ops ip_vs_forward_with_fwmark_ops = { + .hook = ip_vs_forward_with_fwmark, + .owner = THIS_MODULE, + .pf = PF_INET, + .hooknum = NF_IP_FORWARD, + .priority = 101, +}; + /* After packet filtering, change source only for VS/NAT */ static struct nf_hook_ops ip_vs_out_ops = { .hook = ip_vs_out, @@ -1160,9 +1192,17 @@ goto cleanup_postroutingops; } + ret = nf_register_hook(&ip_vs_forward_with_fwmark_ops); + if (ret < 0) { + IP_VS_ERR("can't register forward_with_fwmark hook.\n"); + goto cleanup_forwardicmpops; + } + IP_VS_INFO("ipvs loaded.\n"); return ret; + cleanup_forwardicmpops: + nf_unregister_hook(&ip_vs_forward_icmp_ops); cleanup_postroutingops: nf_unregister_hook(&ip_vs_post_routing_ops); cleanup_outops: @@ -1182,6 +1222,7 @@ static void __exit ip_vs_cleanup(void) { + nf_unregister_hook(&ip_vs_forward_with_fwmark_ops); nf_unregister_hook(&ip_vs_forward_icmp_ops); nf_unregister_hook(&ip_vs_post_routing_ops); nf_unregister_hook(&ip_vs_out_ops);
* Horms <[email protected]> 2006-12-18 12:19
Quoted 5 lines · Horms
Quoted 1 line · Thomas Graf
[split]
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
This patch seems to be based on an old tree, I've renamed nfmark to mark in net-2.6.20. The term fwmark and nfmark shouldn't be used anymore.
Sorry, I based this patch on Linus's tree. I'll port it to net-2.6.20.
This took too long for me to get around to :(
Am I correct in thinking I just need to replace fwmark with mark?
Yes, that's all there is to it.
Hi Horms,
I see that this patch probably makes IPVS code a bit complicated and packet traversing less efficiently.
If I remember correctly, policy-based routing can work with IPVS in kernel 2.2 and 2.4 for transparent cache cluster for a long time. It should work in kernel 2.6 too.
For example, we can use iptables/ipchains to mark all web traffic with fwmark 1, then use policy-based routing to route all web traffic through NF_IP_LOCAL_IN, so that ip_vs_in can capture the packets and load balance packets to cache servers. ip rule add prio 100 fwmark 1 table 100
ip route add local 0/0 dev lo table 100
ipvsadm -A -f 1 -s wlc
ipvsadm -a -f 1 -w 100 -r cache1
ipvsadm -a -f 1 -w 100 -r cache2
ipvsadm -a -f 1 -w 100 -r cache2
...
Cheers,
Wensong
Quoted 7 lines · Horms
This seems to be a pretty clean solution to a real problem.
Ultimately I would like to see IPVS move into the forward chain. This seems to be a nice way to explore that, without breaking any existing setups.