Skip to contentSkip to conversation

IPVS: SIP Persistence Engine

PATCH In review Simon Horman 2 participants 18 messages Aug 22, 2010 – Sep 18, 2010

Subsystems (5)
Networking Network drivers netfilter IPv4/IPv6 +1 more subsystems
lore
Share & tools
lore

Simon Horman

author maintainer
#0 lore raw

cover IPVS: SIP Persistence Engine

This patch series adds load-balancing of UDP SIP based on Call-ID to IPVS as well as a frame-work for extending IPVS to handle alternate persistence requirements.

REVISIONS

This is "patch v1" of this series, which addresses a few minor problems, as annotated on a per-patch basis, since the initial "rfc" posting. Internally there were 4 rfc versions, 0.1, 0.2, 0.3 and 0.4, some of the notes for some of the patches reflect those versions.

OVERVIEW

The approach that I have taken is what I call persistence engines. The basic idea being that you can provide a module to LVS that alters the way that it handles connection templates, which are at the core of persistence. In particular, an additional key can be added, and any of the normal IP address, port and protocol information can either be used or ignored.

In the case of the SIP persistence engine, the only persistence engine, all the keys used by the default persistence behaviour are used and the callid is added as an extra key. I originally intended to ignore the cip, but this can optionally be done by setting the persistence mask (-M) to 0.0.0.0 while allowing the flexibility of other mask values.

It is envisaged that the SIP persistence engine will be used in conjunction with one-packet scheduling. I'm interested to hear if that doesn't fit your needs.

CONFIGURATION

A persistence engine is associated with a virtual service (as are schedulers). I have added the --pe option to the ivpsadm -A and -E commands to allow the persistence engine of a virtual service to be added, changed, or deleted.

e.g. ipvsadm -A -u 10.4.3.192:5060 -p 60 -M 0.0.0.0 -o --pe sip

There are no other configuration parameters at this time.

RUNNING

When a connection template is created, if its virtual service has a persistence engine, then the persistence engine can add an extra key to the connection template. For the SIP module this is the callid. More generically, it is known as "pe data". And both the name of the persistence engine, "pe name", and "pe data" can be viewed in /proc/net/ip_vs_conn and by passing the --persistent-conn option to ipvsadm -Lc.

e.g.

# ipvsadm -Lcn --persistent-conn
UDP 00:38  UDP         10.4.3.0:0         10.4.3.192:5060    127.0.0.1:5060 sip 193373839

Here we see a single persistence template (cport is 0), which has been handled by the sip persistence engine. The pe data (callid) is 193373839.

In the case where the persistence engine can't match a packet for some reason, the connection will fall back to the normal persistence handling. This seems reasonable, as that if the packet ought to be dropped, iptables could be used.

A limited amount of debugging information has been added which can be enabled using a value of 9 or greater in /proc/sys/net/ipv4/vs/debug_level

CODE AVAILABILITY

The kernel patches (13) are available in git as the pe-0.4 branch of
git://git.kernel.org/pub/scm/linux/kernel/git/horms/lvs-test-2.6.git

The ipvsadm patches (2) are available in git as the pe-0.4 branch of
git://github.com/horms/ipvsadm-test.git

I will post the ipvsadm patches separately

--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author maintainer
#1 lore raw

PATCH 1/12 netfilter: nf_conntrack_sip: Allow ct_sip_get_header() to be called with a null ct argument

Signed-off-by Simon Horman <[email protected]>

---

The motivation for this is to allow ct_sip_get_header() to be used by LVS without connection tracking as per subsequent patches.

net/netfilter/nf_conntrack_sip.c +3 -0
Index: nf-next-2.6/net/netfilter/nf_conntrack_sip.c
===================================================================
@@ -152,6 +152,9 @@ static int parse_addr(const struct nf_co
 	const char *end;
 	int ret = 0;
 
+	if (!ct)
+		return 0;
+
 	memset(addr, 0, sizeof(*addr));
 	switch (nf_ct_l3num(ct)) {
 	case AF_INET:
author maintainer
#2 lore raw

PATCH 2/12 netfilter: nf_conntrack_sip: Add callid parser

In reply to Simon Horman
Signed-off-by Simon Horman <[email protected]>

---

The motivation for this is for it to be used by LVS as per subsequent patches.

* Patrick McHardy suggested changing word_len to check for the next newline or whitespace. But I believe this is incorrect. For example '#' would be permitted but it is invalid in a word according to RFC3261.

net/netfilter/nf_conntrack_sip.c +39 -0
Index: nf-next-2.6/net/netfilter/nf_conntrack_sip.c
===================================================================
@@ -130,6 +130,44 @@ static int digits_len(const struct nf_co
 	return len;
 }
 
+static int iswordc(const char c)
+{
+	if (isalnum(c) || c == '!' || c == '"' || c == '%' ||
+	    (c >= '(' && c <= '/') || c == ':' || c == '<' || c == '>' ||
+	    c == '?' || (c >= '[' && c <= ']') || c == '_' || c == '`' ||
+	    c == '{' || c == '}' || c == '~')
+		return 1;
+	return 0;
+}
+
+static int word_len(const char *dptr, const char *limit)
+{
+	int len = 0;
+	while (dptr < limit && iswordc(*dptr)) {
+		dptr++;
+		len++;
+	}
+	return len;
+}
+
+static int callid_len(const struct nf_conn *ct, const char *dptr,
+		      const char *limit, int *shift)
+{
+	int len, domain_len;
+
+	len = word_len(dptr, limit);
+	dptr += len;
+	if (!len || dptr == limit || *dptr != '@')
+		return len;
+	dptr++;
+	len++;
+
+	domain_len = word_len(dptr, limit);
+	if (!domain_len)
+		return 0;
+	return len + domain_len;
+}
+
 /* get media type + port length */
 static int media_len(const struct nf_conn *ct, const char *dptr,
 		     const char *limit, int *shift)

@@ -299,6 +337,7 @@ static const struct sip_header ct_sip_hd
 	[SIP_HDR_VIA_TCP]		= SIP_HDR("Via", "v", "TCP ", epaddr_len),
 	[SIP_HDR_EXPIRES]		= SIP_HDR("Expires", NULL, NULL, digits_len),
 	[SIP_HDR_CONTENT_LENGTH]	= SIP_HDR("Content-Length", "l", NULL, digits_len),
+	[SIP_HDR_CALL_ID]		= SIP_HDR("Call-Id", "i", NULL, callid_len),
 };
 
 static const char *sip_follow_continuation(const char *dptr, const char *limit)
include/linux/netfilter/nf_conntrack_sip.h +1 -0
Index: nf-next-2.6/include/linux/netfilter/nf_conntrack_sip.h
===================================================================
@@ -89,6 +89,7 @@ enum sip_header_types {
 	SIP_HDR_VIA_TCP,
 	SIP_HDR_EXPIRES,
 	SIP_HDR_CONTENT_LENGTH,
+	SIP_HDR_CALL_ID,
 };
 
 enum sdp_header_types {
author maintainer
#3 lore raw

PATCH 3/12 IPVS: compact ip_vs_sched_persist()

In reply to Simon Horman

Compact ip_vs_sched_persist() by setting up parameters and calling functions once.

Signed-off-by Simon Horman <[email protected]>
net/netfilter/ipvs/ip_vs_core.c +50 -106
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_core.c
===================================================================
@@ -192,10 +192,15 @@ ip_vs_sched_persist(struct ip_vs_service
 	struct ip_vs_iphdr iph;
 	struct ip_vs_dest *dest;
 	struct ip_vs_conn *ct;
-	__be16  dport;			/* destination port to forward */
+	__be16  dport = 0;
 	__be16  flags;
 	union nf_inet_addr snet;	/* source network of the client,
 					   after masking */
+	int protocol = iph.protocol;
+	const union nf_inet_addr *vaddr = &iph.daddr;
+	union nf_inet_addr fwmark = { .ip = htonl(svc->fwmark) };
+	__be16 vport = 0;
+
 
 	ip_vs_fill_iphdr(svc->af, skb_network_header(skb), &iph);
 

@@ -226,119 +231,58 @@ ip_vs_sched_persist(struct ip_vs_service
 	 * service, and a template like <caddr, 0, vaddr, vport, daddr, dport>
 	 * is created for other persistent services.
 	 */
-	if (ports[1] == svc->port) {
-		/* Check if a template already exists */
-		if (svc->port != FTPPORT)
-			ct = ip_vs_ct_in_get(svc->af, iph.protocol, &snet, 0,
-					     &iph.daddr, ports[1]);
-		else
-			ct = ip_vs_ct_in_get(svc->af, iph.protocol, &snet, 0,
-					     &iph.daddr, 0);
-
-		if (!ct || !ip_vs_check_template(ct)) {
-			/*
-			 * No template found or the dest of the connection
-			 * template is not available.
-			 */
-			dest = svc->scheduler->schedule(svc, skb);
-			if (dest == NULL) {
-				IP_VS_DBG(1, "p-schedule: no dest found.\n");
-				return NULL;
-			}
-
-			/*
-			 * Create a template like <protocol,caddr,0,
-			 * vaddr,vport,daddr,dport> for non-ftp service,
-			 * and <protocol,caddr,0,vaddr,0,daddr,0>
-			 * for ftp service.
+	{
+		if (ports[1] == svc->port) {
+			/* non-FTP template:
+			 * <protocol, caddr, 0, vaddr, vport, daddr, dport>
+			 * FTP template:
+			 * <protocol, caddr, 0, vaddr, 0, daddr, 0>
 			 */
 			if (svc->port != FTPPORT)
-				ct = ip_vs_conn_new(svc->af, iph.protocol,
-						    &snet, 0,
-						    &iph.daddr,
-						    ports[1],
-						    &dest->addr, dest->port,
-						    IP_VS_CONN_F_TEMPLATE,
-						    dest);
-			else
-				ct = ip_vs_conn_new(svc->af, iph.protocol,
-						    &snet, 0,
-						    &iph.daddr, 0,
-						    &dest->addr, 0,
-						    IP_VS_CONN_F_TEMPLATE,
-						    dest);
-			if (ct == NULL)
-				return NULL;
-
-			ct->timeout = svc->timeout;
+				vport = ports[1];
 		} else {
-			/* set destination with the found template */
-			dest = ct->dest;
-		}
-		dport = dest->port;
-	} else {
-		/*
-		 * Note: persistent fwmark-based services and persistent
-		 * port zero service are handled here.
-		 * fwmark template: <IPPROTO_IP,caddr,0,fwmark,0,daddr,0>
-		 * port zero template: <protocol,caddr,0,vaddr,0,daddr,0>
-		 */
-		if (svc->fwmark) {
-			union nf_inet_addr fwmark = {
-				.ip = htonl(svc->fwmark)
-			};
-
-			ct = ip_vs_ct_in_get(svc->af, IPPROTO_IP, &snet, 0,
-					     &fwmark, 0);
-		} else
-			ct = ip_vs_ct_in_get(svc->af, iph.protocol, &snet, 0,
-					     &iph.daddr, 0);
-
-		if (!ct || !ip_vs_check_template(ct)) {
-			/*
-			 * If it is not persistent port zero, return NULL,
-			 * otherwise create a connection template.
+			/* Note: persistent fwmark-based services and
+			 * persistent port zero service are handled here.
+			 * fwmark template:
+			 * <IPPROTO_IP,caddr,0,fwmark,0,daddr,0>
+			 * port zero template:
+			 * <protocol,caddr,0,vaddr,0,daddr,0>
 			 */
-			if (svc->port)
-				return NULL;
-
-			dest = svc->scheduler->schedule(svc, skb);
-			if (dest == NULL) {
-				IP_VS_DBG(1, "p-schedule: no dest found.\n");
-				return NULL;
+			if (svc->fwmark) {
+				protocol = IPPROTO_IP;
+				vaddr = &fwmark;
 			}
+		}
+	}
 
-			/*
-			 * Create a template according to the service
-			 */
-			if (svc->fwmark) {
-				union nf_inet_addr fwmark = {
-					.ip = htonl(svc->fwmark)
-				};
-
-				ct = ip_vs_conn_new(svc->af, IPPROTO_IP,
-						    &snet, 0,
-						    &fwmark, 0,
-						    &dest->addr, 0,
-						    IP_VS_CONN_F_TEMPLATE,
-						    dest);
-			} else
-				ct = ip_vs_conn_new(svc->af, iph.protocol,
-						    &snet, 0,
-						    &iph.daddr, 0,
-						    &dest->addr, 0,
-						    IP_VS_CONN_F_TEMPLATE,
-						    dest);
-			if (ct == NULL)
-				return NULL;
+	/* Check if a template already exists */
+	ct = ip_vs_ct_in_get(svc->af, protocol, &snet, 0, vaddr, vport);
 
-			ct->timeout = svc->timeout;
-		} else {
-			/* set destination with the found template */
-			dest = ct->dest;
+	if (!ct || !ip_vs_check_template(ct)) {
+		/* No template found or the dest of the connection
+		 * template is not available.
+		 */
+		dest = svc->scheduler->schedule(svc, skb);
+		if (!dest) {
+			IP_VS_DBG(1, "p-schedule: no dest found.\n");
+			return NULL;
 		}
-		dport = ports[1];
-	}
+
+		if (ports[1] == svc->port && svc->port != FTPPORT)
+			dport = dest->port;
+
+		/* Create a template */
+		ct = ip_vs_conn_new(svc->af, protocol, &snet, 0,vaddr, vport,
+				    &dest->addr, dport,
+				    IP_VS_CONN_F_TEMPLATE, dest);
+		if (ct == NULL)
+			return NULL;
+
+		ct->timeout = svc->timeout;
+	} else
+		/* set destination with the found template */
+		dest = ct->dest;
+	dport = dest->port;
 
 	flags = (svc->flags & IP_VS_SVC_F_ONEPACKET
 		 && iph.protocol == IPPROTO_UDP)?
author maintainer
#4 lore raw

PATCH 4/12 IPVS: Add struct ip_vs_conn_param

In reply to Simon Horman
Signed-off-by Simon Horman <[email protected]>

---

The motivation for this is to allow persistence engine modules to fill in the parameters.

v0.3: Add missing changes to ip_vs_ftp.c

v0.1: Initial release

include/net/ip_vs.h +31 -15
Index: nf-next-2.6/include/net/ip_vs.h
===================================================================
@@ -355,6 +355,15 @@ struct ip_vs_protocol {
 
 extern struct ip_vs_protocol * ip_vs_proto_get(unsigned short proto);
 
+struct ip_vs_conn_param {
+	const union nf_inet_addr	*caddr;
+	const union nf_inet_addr	*vaddr;
+	__be16				cport;
+	__be16				vport;
+	__u16				protocol;
+	u16				af;
+};
+
 /*
  *	IP_VS structure allocated for each dynamically scheduled connection
  */

@@ -624,13 +633,23 @@ enum {
 	IP_VS_DIR_LAST,
 };
 
-extern struct ip_vs_conn *ip_vs_conn_in_get
-(int af, int protocol, const union nf_inet_addr *s_addr, __be16 s_port,
- const union nf_inet_addr *d_addr, __be16 d_port);
-
-extern struct ip_vs_conn *ip_vs_ct_in_get
-(int af, int protocol, const union nf_inet_addr *s_addr, __be16 s_port,
- const union nf_inet_addr *d_addr, __be16 d_port);
+static inline void ip_vs_conn_fill_param(int af, int protocol,
+					 const union nf_inet_addr *caddr,
+					 __be16 cport,
+					 const union nf_inet_addr *vaddr,
+					 __be16 vport,
+					 struct ip_vs_conn_param *p)
+{
+	p->af = af;
+	p->protocol = protocol;
+	p->caddr = caddr;
+	p->cport = cport;
+	p->vaddr = vaddr;
+	p->vport = vport;
+}
+
+struct ip_vs_conn *ip_vs_conn_in_get(const struct ip_vs_conn_param *p);
+struct ip_vs_conn *ip_vs_ct_in_get(const struct ip_vs_conn_param *p);
 
 struct ip_vs_conn * ip_vs_conn_in_get_proto(int af, const struct sk_buff *skb,
 					    struct ip_vs_protocol *pp,

@@ -638,9 +657,7 @@ struct ip_vs_conn * ip_vs_conn_in_get_pr
 					    unsigned int proto_off,
 					    int inverse);
 
-extern struct ip_vs_conn *ip_vs_conn_out_get
-(int af, int protocol, const union nf_inet_addr *s_addr, __be16 s_port,
- const union nf_inet_addr *d_addr, __be16 d_port);
+struct ip_vs_conn *ip_vs_conn_out_get(const struct ip_vs_conn_param *p);
 
 struct ip_vs_conn * ip_vs_conn_out_get_proto(int af, const struct sk_buff *skb,
 					     struct ip_vs_protocol *pp,

@@ -656,11 +673,10 @@ static inline void __ip_vs_conn_put(stru
 extern void ip_vs_conn_put(struct ip_vs_conn *cp);
 extern void ip_vs_conn_fill_cport(struct ip_vs_conn *cp, __be16 cport);
 
-extern struct ip_vs_conn *
-ip_vs_conn_new(int af, int proto, const union nf_inet_addr *caddr, __be16 cport,
-	       const union nf_inet_addr *vaddr, __be16 vport,
-	       const union nf_inet_addr *daddr, __be16 dport, unsigned flags,
-	       struct ip_vs_dest *dest);
+struct ip_vs_conn *ip_vs_conn_new(const struct ip_vs_conn_param *p,
+				  const union nf_inet_addr *daddr,
+				  __be16 dport, unsigned flags,
+				  struct ip_vs_dest *dest);
 extern void ip_vs_conn_expire_now(struct ip_vs_conn *cp);
 
 extern const char * ip_vs_state_name(__u16 proto, int state);
net/netfilter/ipvs/ip_vs_core.c +32 -31

108 lines not on this page · raw message

net/netfilter/ipvs/ip_vs_conn.c +80 -81

275 lines not on this page · raw message

net/netfilter/ipvs/ip_vs_sync.c +14 -19
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_sync.c
===================================================================
@@ -301,6 +301,7 @@ static void ip_vs_process_message(const
 	struct ip_vs_conn *cp;
 	struct ip_vs_protocol *pp;
 	struct ip_vs_dest *dest;
+	struct ip_vs_conn_param param;
 	char *p;
 	int i;
 

@@ -370,18 +371,17 @@ static void ip_vs_process_message(const
 			}
 		}
 
-		if (!(flags & IP_VS_CONN_F_TEMPLATE))
-			cp = ip_vs_conn_in_get(AF_INET, s->protocol,
-					       (union nf_inet_addr *)&s->caddr,
-					       s->cport,
-					       (union nf_inet_addr *)&s->vaddr,
-					       s->vport);
-		else
-			cp = ip_vs_ct_in_get(AF_INET, s->protocol,
-					     (union nf_inet_addr *)&s->caddr,
-					     s->cport,
-					     (union nf_inet_addr *)&s->vaddr,
-					     s->vport);
+		{
+			ip_vs_conn_fill_param(AF_INET, s->protocol,
+					      (union nf_inet_addr *)&s->caddr,
+					      s->cport,
+					      (union nf_inet_addr *)&s->vaddr,
+					      s->vport, &param);
+			if (!(flags & IP_VS_CONN_F_TEMPLATE))
+				cp = ip_vs_conn_in_get(&param);
+			else
+				cp = ip_vs_ct_in_get(&param);
+		}
 		if (!cp) {
 			/*
 			 * Find the appropriate destination for the connection.

@@ -406,14 +406,9 @@ static void ip_vs_process_message(const
 				else
 					flags &= ~IP_VS_CONN_F_INACTIVE;
 			}
-			cp = ip_vs_conn_new(AF_INET, s->protocol,
-					    (union nf_inet_addr *)&s->caddr,
-					    s->cport,
-					    (union nf_inet_addr *)&s->vaddr,
-					    s->vport,
+			cp = ip_vs_conn_new(&param,
 					    (union nf_inet_addr *)&s->daddr,
-					    s->dport,
-					    flags, dest);
+					    s->dport, flags, dest);
 			if (dest)
 				atomic_dec(&dest->refcnt);
 			if (!cp) {
net/netfilter/ipvs/ip_vs_proto_ah_esp.c +19 -28

67 lines not on this page · raw message

net/netfilter/ipvs/ip_vs_ftp.c +33 -25

91 lines not on this page · raw message

author maintainer
#5 lore raw

PATCH 5/12 IPVS: Allow null argument to ip_vs_scheduler_put()

In reply to Simon Horman

This simplifies caller logic sightly.

Signed-off-by Simon Horman <[email protected]>
net/netfilter/ipvs/ip_vs_ctl.c +4 -9
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_ctl.c
===================================================================
@@ -1167,7 +1167,7 @@ ip_vs_add_service(struct ip_vs_service_u
 	if (sched == NULL) {
 		pr_info("Scheduler module ip_vs_%s not found\n", u->sched_name);
 		ret = -ENOENT;
-		goto out_mod_dec;
+		goto out_err;
 	}
 
 #ifdef CONFIG_IP_VS_IPV6

@@ -1227,7 +1227,7 @@ ip_vs_add_service(struct ip_vs_service_u
 	*svc_p = svc;
 	return 0;
 
-  out_err:
+ out_err:
 	if (svc != NULL) {
 		if (svc->scheduler)
 			ip_vs_unbind_scheduler(svc);

@@ -1240,7 +1240,6 @@ ip_vs_add_service(struct ip_vs_service_u
 	}
 	ip_vs_scheduler_put(sched);
 
-  out_mod_dec:
 	/* decrease the module use count */
 	ip_vs_use_count_dec();
 

@@ -1323,10 +1322,7 @@ ip_vs_edit_service(struct ip_vs_service
 #ifdef CONFIG_IP_VS_IPV6
   out:
 #endif
-
-	if (old_sched)
-		ip_vs_scheduler_put(old_sched);
-
+	ip_vs_scheduler_put(old_sched);
 	return ret;
 }
 

@@ -1350,8 +1346,7 @@ static void __ip_vs_del_service(struct i
 	/* Unbind scheduler */
 	old_sched = svc->scheduler;
 	ip_vs_unbind_scheduler(svc);
-	if (old_sched)
-		ip_vs_scheduler_put(old_sched);
+	ip_vs_scheduler_put(old_sched);
 
 	/* Unbind app inc */
 	if (svc->inc) {
net/netfilter/ipvs/ip_vs_sched.c +1 -1
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_sched.c
===================================================================
@@ -159,7 +159,7 @@ struct ip_vs_scheduler *ip_vs_scheduler_
 
 void ip_vs_scheduler_put(struct ip_vs_scheduler *scheduler)
 {
-	if (scheduler->module)
+	if (scheduler && scheduler->module)
 		module_put(scheduler->module);
 }
author maintainer
#6 lore raw

PATCH 6/12 IPVS: ip_vs_{un,}bind_scheduler NULL arguments

In reply to Simon Horman

In general NULL arguments aren't passed by the few callers that exist, so don't test for them.

The exception is to make passing NULL to ip_vs_unbind_scheduler() a noop.

Signed-off-by Simon Horman <[email protected]>
net/netfilter/ipvs/ip_vs_ctl.c +1 -2
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_ctl.c
===================================================================
@@ -1229,8 +1229,7 @@ ip_vs_add_service(struct ip_vs_service_u
 
  out_err:
 	if (svc != NULL) {
-		if (svc->scheduler)
-			ip_vs_unbind_scheduler(svc);
+		ip_vs_unbind_scheduler(svc);
 		if (svc->inc) {
 			local_bh_disable();
 			ip_vs_app_inc_put(svc->inc);
net/netfilter/ipvs/ip_vs_sched.c +3 -20
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_sched.c
===================================================================
@@ -46,15 +46,6 @@ int ip_vs_bind_scheduler(struct ip_vs_se
 {
 	int ret;
 
-	if (svc == NULL) {
-		pr_err("%s(): svc arg NULL\n", __func__);
-		return -EINVAL;
-	}
-	if (scheduler == NULL) {
-		pr_err("%s(): scheduler arg NULL\n", __func__);
-		return -EINVAL;
-	}
-
 	svc->scheduler = scheduler;
 
 	if (scheduler->init_service) {

@@ -74,18 +65,10 @@ int ip_vs_bind_scheduler(struct ip_vs_se
  */
 int ip_vs_unbind_scheduler(struct ip_vs_service *svc)
 {
-	struct ip_vs_scheduler *sched;
+	struct ip_vs_scheduler *sched = svc->scheduler;
 
-	if (svc == NULL) {
-		pr_err("%s(): svc arg NULL\n", __func__);
-		return -EINVAL;
-	}
-
-	sched = svc->scheduler;
-	if (sched == NULL) {
-		pr_err("%s(): svc isn't bound\n", __func__);
-		return -EINVAL;
-	}
+	if (!sched)
+		return 0;
 
 	if (sched->done_service) {
 		if (sched->done_service(svc) != 0) {
author maintainer
#7 lore raw

PATCH 7/12 IPVS: Add struct ip_vs_pe

In reply to Simon Horman
Signed-off-by Simon Horman <[email protected]>

---

This the first of several patches to add persistence engines.

net/netfilter/ipvs/ip_vs_conn.c +42 -11
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_conn.c
===================================================================
@@ -148,6 +148,29 @@ static unsigned int ip_vs_conn_hashkey(i
 		& ip_vs_conn_tab_mask;
 }
 
+static unsigned int ip_vs_conn_hashkey_param(const struct ip_vs_conn_param *p)
+{
+	if (p->pe && p->pe->hashkey_raw)
+		return p->pe->hashkey_raw(p, ip_vs_conn_rnd) &
+			ip_vs_conn_tab_mask;
+	return ip_vs_conn_hashkey(p->af, p->protocol, p->caddr, p->cport);
+}
+
+static unsigned int ip_vs_conn_hashkey_conn(const struct ip_vs_conn *cp)
+{
+	struct ip_vs_conn_param p;
+
+	ip_vs_conn_fill_param(cp->af, cp->protocol, &cp->caddr, cp->cport,
+			      NULL, 0, &p);
+
+	if (cp->dest->svc->pe) {
+		p.pe = cp->dest->svc->pe;
+		p.pe_data = cp->pe_data;
+		p.pe_data_len = cp->pe_data_len;
+	}
+
+	return ip_vs_conn_hashkey_param(&p);
+}
 
 /*
  *	Hashes ip_vs_conn in ip_vs_conn_tab by proto,addr,port.

@@ -162,7 +185,7 @@ static inline int ip_vs_conn_hash(struct
 		return 0;
 
 	/* Hash by protocol, client address and port */
-	hash = ip_vs_conn_hashkey(cp->af, cp->protocol, &cp->caddr, cp->cport);
+	hash = ip_vs_conn_hashkey_conn(cp);
 
 	ct_write_lock(hash);
 	spin_lock(&cp->lock);

@@ -195,7 +218,7 @@ static inline int ip_vs_conn_unhash(stru
 	int ret;
 
 	/* unhash it and decrease its reference counter */
-	hash = ip_vs_conn_hashkey(cp->af, cp->protocol, &cp->caddr, cp->cport);
+	hash = ip_vs_conn_hashkey_conn(cp);
 
 	ct_write_lock(hash);
 	spin_lock(&cp->lock);

@@ -227,7 +250,7 @@ __ip_vs_conn_in_get(const struct ip_vs_c
 	unsigned hash;
 	struct ip_vs_conn *cp;
 
-	hash = ip_vs_conn_hashkey(p->af, p->protocol, p->caddr, p->cport);
+	hash = ip_vs_conn_hashkey_param(p);
 
 	ct_read_lock(hash);
 

@@ -312,11 +335,17 @@ struct ip_vs_conn *ip_vs_ct_in_get(const
 	unsigned hash;
 	struct ip_vs_conn *cp;
 
-	hash = ip_vs_conn_hashkey(p->af, p->protocol, p->caddr, p->cport);
+	hash = ip_vs_conn_hashkey_param(p);
 
 	ct_read_lock(hash);
 
 	list_for_each_entry(cp, &ip_vs_conn_tab[hash], c_list) {
+		if (p->pe && p->pe->ct_match) {
+			if (p->pe->ct_match(p, cp))
+				goto out;
+			continue;
+		}
+
 		if (cp->af == p->af &&
 		    ip_vs_addr_equal(p->af, p->caddr, &cp->caddr) &&
 		    /* protocol should only be IPPROTO_IP if

@@ -325,15 +354,14 @@ struct ip_vs_conn *ip_vs_ct_in_get(const
 				     p->af, p->vaddr, &cp->vaddr) &&
 		    p->cport == cp->cport && p->vport == cp->vport &&
 		    cp->flags & IP_VS_CONN_F_TEMPLATE &&
-		    p->protocol == cp->protocol) {
-			/* HIT */
-			atomic_inc(&cp->refcnt);
+		    p->protocol == cp->protocol)
 			goto out;
-		}
 	}
 	cp = NULL;
 
   out:
+	if (cp)
+		atomic_inc(&cp->refcnt);
 	ct_read_unlock(hash);
 
 	IP_VS_DBG_BUF(9, "template lookup/in %s %s:%d->%s:%d %s\n",

@@ -359,7 +387,7 @@ struct ip_vs_conn *ip_vs_conn_out_get(co
 	/*
 	 *	Check for "full" addressed entries
 	 */
-	hash = ip_vs_conn_hashkey(p->af, p->protocol, p->vaddr, p->vport);
+	hash = ip_vs_conn_hashkey_param(p);
 
 	ct_read_lock(hash);
 

@@ -777,6 +805,10 @@ ip_vs_conn_new(const struct ip_vs_conn_p
 			&cp->daddr, daddr);
 	cp->dport          = dport;
 	cp->flags	   = flags;
+	if (p->pe_data) {
+		cp->pe_data = p->pe_data;
+		cp->pe_data_len = p->pe_data_len;
+	}
 	spin_lock_init(&cp->lock);
 
 	/*

@@ -817,7 +849,6 @@ ip_vs_conn_new(const struct ip_vs_conn_p
 	return cp;
 }
 
-
 /*
  *	/proc/net/ip_vs_conn entries
  */

@@ -833,7 +864,7 @@ static void *ip_vs_conn_array(struct seq
 		list_for_each_entry(cp, &ip_vs_conn_tab[idx], c_list) {
 			if (pos-- == 0) {
 				seq->private = &ip_vs_conn_tab[idx];
-				return cp;
+			return cp;
 			}
 		}
 		ct_read_unlock_bh(idx);
net/netfilter/ipvs/ip_vs_core.c +18 -4
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_core.c
===================================================================
@@ -175,6 +175,19 @@ ip_vs_set_state(struct ip_vs_conn *cp, i
 	return pp->state_transition(cp, direction, skb, pp);
 }
 
+static inline int
+ip_vs_conn_fill_param_persist(const struct ip_vs_service *svc,
+			      struct sk_buff *skb, int protocol,
+			      const union nf_inet_addr *caddr, __be16 cport,
+			      const union nf_inet_addr *vaddr, __be16 vport,
+			      struct ip_vs_conn_param *p)
+{
+	ip_vs_conn_fill_param(svc->af, protocol, caddr, cport, vaddr, vport, p);
+	p->pe = svc->pe;
+	if (p->pe && p->pe->fill_param)
+		return p->pe->fill_param(p, skb);
+	return 0;
+}
 
 /*
  *  IPVS persistent scheduling function

@@ -185,7 +198,7 @@ ip_vs_set_state(struct ip_vs_conn *cp, i
  */
 static struct ip_vs_conn *
 ip_vs_sched_persist(struct ip_vs_service *svc,
-		    const struct sk_buff *skb,
+		    struct sk_buff *skb,
 		    __be16 ports[2])
 {
 	struct ip_vs_conn *cp = NULL;

@@ -254,8 +267,9 @@ ip_vs_sched_persist(struct ip_vs_service
 				vaddr = &fwmark;
 			}
 		}
-		ip_vs_conn_fill_param(svc->af, protocol, &snet, 0,
-				      vaddr, vport, &param);
+		if (ip_vs_conn_fill_param_persist(svc, skb, protocol, &snet, 0,
+						  vaddr, vport, &param))
+			return NULL;
 	}
 
 	/* Check if a template already exists */

@@ -316,7 +330,7 @@ ip_vs_sched_persist(struct ip_vs_service
  *  Protocols supported: TCP, UDP
  */
 struct ip_vs_conn *
-ip_vs_schedule(struct ip_vs_service *svc, const struct sk_buff *skb)
+ip_vs_schedule(struct ip_vs_service *svc, struct sk_buff *skb)
 {
 	struct ip_vs_conn *cp = NULL;
 	struct ip_vs_iphdr iph;
net/netfilter/ipvs/ip_vs_sync.c +15 -2
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_sync.c
===================================================================
@@ -288,6 +288,16 @@ void ip_vs_sync_conn(struct ip_vs_conn *
 		ip_vs_sync_conn(cp->control);
 }
 
+static inline int
+ip_vs_conn_fill_param_sync(int af, int protocol,
+			   const union nf_inet_addr *caddr, __be16 cport,
+			   const union nf_inet_addr *vaddr, __be16 vport,
+			   struct ip_vs_conn_param *p)
+{
+	/* XXX: Need to take into account persistence engine */
+	ip_vs_conn_fill_param(af, protocol, caddr, cport, vaddr, vport, p);
+	return 0;
+}
 
 /*
  *      Process received multicast message and create the corresponding

@@ -372,11 +382,14 @@ static void ip_vs_process_message(const
 		}
 
 		{
-			ip_vs_conn_fill_param(AF_INET, s->protocol,
+			if (ip_vs_conn_fill_param_sync(AF_INET, s->protocol,
 					      (union nf_inet_addr *)&s->caddr,
 					      s->cport,
 					      (union nf_inet_addr *)&s->vaddr,
-					      s->vport, &param);
+					      s->vport, &param)) {
+				pr_err("ip_vs_conn_fill_param_sync failed");
+				return;
+			}
 			if (!(flags & IP_VS_CONN_F_TEMPLATE))
 				cp = ip_vs_conn_in_get(&param);
 			else
include/net/ip_vs.h +26 -1
Index: nf-next-2.6/include/net/ip_vs.h
===================================================================
@@ -362,6 +362,10 @@ struct ip_vs_conn_param {
 	__be16				vport;
 	__u16				protocol;
 	u16				af;
+
+	const struct ip_vs_pe		*pe;
+	char				*pe_data;
+	__u8				pe_data_len;
 };
 
 /*

@@ -414,6 +418,9 @@ struct ip_vs_conn {
 	void                    *app_data;      /* Application private data */
 	struct ip_vs_seq        in_seq;         /* incoming seq. struct */
 	struct ip_vs_seq        out_seq;        /* outgoing seq. struct */
+
+	char			*pe_data;
+	__u8			pe_data_len;
 };
 
 

@@ -484,6 +491,9 @@ struct ip_vs_service {
 	struct ip_vs_scheduler	*scheduler;    /* bound scheduler object */
 	rwlock_t		sched_lock;    /* lock sched_data */
 	void			*sched_data;   /* scheduler application data */
+
+	/* alternate persistence engine */
+	struct ip_vs_pe		*pe;
 };
 
 

@@ -547,6 +557,19 @@ struct ip_vs_scheduler {
 				       const struct sk_buff *skb);
 };
 
+/* The persistence engine object */
+struct ip_vs_pe {
+	struct list_head	n_list;		/* d-linked list head */
+	char			*name;		/* scheduler name */
+	atomic_t		refcnt;		/* reference counter */
+	struct module		*module;	/* THIS_MODULE/NULL */
+
+	/* get the connection template, if any */
+	int (*fill_param)(struct ip_vs_conn_param *p, struct sk_buff *skb);
+	bool (*ct_match)(const struct ip_vs_conn_param *p,
+			 struct ip_vs_conn *ct);
+	u32 (*hashkey_raw)(const struct ip_vs_conn_param *p, u32 initval);
+};
 
 /*
  *	The application module object (a.k.a. app incarnation)

@@ -646,6 +669,8 @@ static inline void ip_vs_conn_fill_param
 	p->cport = cport;
 	p->vaddr = vaddr;
 	p->vport = vport;
+	p->pe = NULL;
+	p->pe_data = NULL;
 }
 
 struct ip_vs_conn *ip_vs_conn_in_get(const struct ip_vs_conn_param *p);

@@ -801,7 +826,7 @@ extern int ip_vs_unbind_scheduler(struct
 extern struct ip_vs_scheduler *ip_vs_scheduler_get(const char *sched_name);
 extern void ip_vs_scheduler_put(struct ip_vs_scheduler *scheduler);
 extern struct ip_vs_conn *
-ip_vs_schedule(struct ip_vs_service *svc, const struct sk_buff *skb);
+ip_vs_schedule(struct ip_vs_service *svc, struct sk_buff *skb);
 extern int ip_vs_leave(struct ip_vs_service *svc, struct sk_buff *skb,
 			struct ip_vs_protocol *pp);
include/linux/ip_vs.h +2 -0
Index: nf-next-2.6/include/linux/ip_vs.h
===================================================================
@@ -89,8 +89,10 @@
 #define IP_VS_CONN_F_ONE_PACKET	0x2000		/* forward only one packet */
 
 #define IP_VS_SCHEDNAME_MAXLEN	16
+#define IP_VS_PENAME_MAXLEN	16
 #define IP_VS_IFNAME_MAXLEN	16
 
+#define IP_VS_PEDATA_MAXLEN     255
 
 /*
  *	The struct ip_vs_service_user and struct ip_vs_dest_user are
author maintainer
#8 lore raw

PATCH 8/12 IPVS: Add persistence engine data to /proc/net/ip_vs_conn

In reply to Simon Horman

This shouldn't break compatibility with userspace as the new data is at the end of the line.

I have confirmed that this doesn't break ipvsadm, the main (only?) user-space user of this data.

Signed-off-by Simon Horman <[email protected]>

* Jan Engelhardt suggested using netlink to do this, but it seems like overkill to me. I'm willing to be convinced otherwise.

include/net/ip_vs.h +1 -0
Index: nf-next-2.6/include/net/ip_vs.h
===================================================================
@@ -569,6 +569,7 @@ struct ip_vs_pe {
 	bool (*ct_match)(const struct ip_vs_conn_param *p,
 			 struct ip_vs_conn *ct);
 	u32 (*hashkey_raw)(const struct ip_vs_conn_param *p, u32 initval);
+	int (*show_pe_data)(const struct ip_vs_conn *cp, char *buf);
 };
 
 /*
net/netfilter/ipvs/ip_vs_conn.c +19 -5
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_conn.c
===================================================================
@@ -921,30 +921,44 @@ static int ip_vs_conn_seq_show(struct se
 
 	if (v == SEQ_START_TOKEN)
 		seq_puts(seq,
-   "Pro FromIP   FPrt ToIP     TPrt DestIP   DPrt State       Expires\n");
+   "Pro FromIP   FPrt ToIP     TPrt DestIP   DPrt State       Expires PEName PEData\n");
 	else {
 		const struct ip_vs_conn *cp = v;
+		char pe_data[IP_VS_PENAME_MAXLEN + IP_VS_PEDATA_MAXLEN + 3];
+		size_t len = 0;
+
+		if (cp->dest->svc->pe && cp->dest->svc->pe->show_pe_data) {
+			pe_data[0] = ' ';
+			len = strlen(cp->dest->svc->pe->name);
+			memcpy(pe_data + 1, cp->dest->svc->pe->name, len);
+			pe_data[len + 1] = ' ';
+			len += 2;
+			len += cp->dest->svc->pe->show_pe_data(cp,
+							       pe_data + len);
+		}
+		pe_data[len] = '\0';
 
 #ifdef CONFIG_IP_VS_IPV6
 		if (cp->af == AF_INET6)
-			seq_printf(seq, "%-3s %pI6 %04X %pI6 %04X %pI6 %04X %-11s %7lu\n",
+			seq_printf(seq, "%-3s %pI6 %04X %pI6 %04X "
+				"%pI6 %04X %-11s %7lu%s\n",
 				ip_vs_proto_name(cp->protocol),
 				&cp->caddr.in6, ntohs(cp->cport),
 				&cp->vaddr.in6, ntohs(cp->vport),
 				&cp->daddr.in6, ntohs(cp->dport),
 				ip_vs_state_name(cp->protocol, cp->state),
-				(cp->timer.expires-jiffies)/HZ);
+				(cp->timer.expires-jiffies)/HZ, pe_data);
 		else
 #endif
 			seq_printf(seq,
 				"%-3s %08X %04X %08X %04X"
-				" %08X %04X %-11s %7lu\n",
+				" %08X %04X %-11s %7lu%s\n",
 				ip_vs_proto_name(cp->protocol),
 				ntohl(cp->caddr.ip), ntohs(cp->cport),
 				ntohl(cp->vaddr.ip), ntohs(cp->vport),
 				ntohl(cp->daddr.ip), ntohs(cp->dport),
 				ip_vs_state_name(cp->protocol, cp->state),
-				(cp->timer.expires-jiffies)/HZ);
+				(cp->timer.expires-jiffies)/HZ, pe_data);
 	}
 	return 0;
 }
author maintainer
#9 lore raw

PATCH 9/12 IPVS: management of persistence engine modules

In reply to Simon Horman

This is based heavily on the scheduler management code

Signed-off-by Simon Horman <[email protected]>

v1
* As suggested by Stephen Hemminger
- Convert __ip_vs_pe_lock from a rwlock to a spinlock.
This code isn't performance-critical, so there is no need for RCU.
- Rename __ip_vs_pe_lock as ip_vs_pe_lock
* Stephen also suggested open-coding ip_vs_{un,}bind_pe()
as they are very short. But I would prefer to keep them as they are used along side ip_vs_{un,}bind_scheduler which are too long to be open-coded.

v0.4
* Export register_ip_vs_pe and unregister_ip_vs_pe
* Use one line comment format for one line comments.
* Only use at most one blank line consecutively

include/net/ip_vs.h +6 -0
Index: lvs-test-2.6/include/net/ip_vs.h
===================================================================
@@ -793,6 +793,12 @@ extern int ip_vs_app_pkt_in(struct ip_vs
 extern int ip_vs_app_init(void);
 extern void ip_vs_app_cleanup(void);
 
+void ip_vs_bind_pe(struct ip_vs_service *svc, struct ip_vs_pe *pe);
+void ip_vs_unbind_pe(struct ip_vs_service *svc);
+int register_ip_vs_pe(struct ip_vs_pe *pe);
+int unregister_ip_vs_pe(struct ip_vs_pe *pe);
+extern struct ip_vs_pe *ip_vs_pe_get(const char *name);
+extern void ip_vs_pe_put(struct ip_vs_pe *pe);
 
 /*
  *	IPVS protocol functions (from ip_vs_proto.c)
net/netfilter/ipvs/Makefile +1 -1
Index: lvs-test-2.6/net/netfilter/ipvs/Makefile
===================================================================
@@ -11,7 +11,7 @@ ip_vs_proto-objs-$(CONFIG_IP_VS_PROTO_SC
 
 ip_vs-objs :=	ip_vs_conn.o ip_vs_core.o ip_vs_ctl.o ip_vs_sched.o	   \
 		ip_vs_xmit.o ip_vs_app.o ip_vs_sync.o	   		   \
-		ip_vs_est.o ip_vs_proto.o 				   \
+		ip_vs_est.o ip_vs_proto.o ip_vs_pe.o		   \
 		$(ip_vs_proto-objs-y)
net/netfilter/ipvs/ip_vs_pe.c new file +147 -0

147 lines not on this page · raw message

author maintainer
#10 lore raw

PATCH 10/12 IPVS: Allow configuration of persistence engines

In reply to Simon Horman

Allow the persistence engine of a virtual service to be set, edited and unset.

This feature only works with the netlink user-space interface.

Signed-off-by Simon Horman <[email protected]>
net/netfilter/ipvs/ip_vs_ctl.c +42 -3
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_ctl.c
===================================================================
@@ -1157,6 +1157,7 @@ ip_vs_add_service(struct ip_vs_service_u
 {
 	int ret = 0;
 	struct ip_vs_scheduler *sched = NULL;
+	struct ip_vs_pe *pe = NULL;
 	struct ip_vs_service *svc = NULL;
 
 	/* increase the module use count */

@@ -1170,6 +1171,16 @@ ip_vs_add_service(struct ip_vs_service_u
 		goto out_err;
 	}
 
+	if (u->pe_name && *u->pe_name) {
+		pe = ip_vs_pe_get(u->pe_name);
+		if (pe == NULL) {
+			pr_info("persistence engine module ip_vs_pe_%s "
+				"not found\n", u->pe_name);
+			ret = -ENOENT;
+			goto out_err;
+		}
+	}
+
 #ifdef CONFIG_IP_VS_IPV6
 	if (u->af == AF_INET6 && (u->netmask < 1 || u->netmask > 128)) {
 		ret = -EINVAL;

@@ -1207,6 +1218,10 @@ ip_vs_add_service(struct ip_vs_service_u
 		goto out_err;
 	sched = NULL;
 
+	/* Bind the ct retriever */
+	ip_vs_bind_pe(svc, pe);
+	pe = NULL;
+
 	/* Update the virtual service counters */
 	if (svc->port == FTPPORT)
 		atomic_inc(&ip_vs_ftpsvc_counter);

@@ -1238,6 +1253,7 @@ ip_vs_add_service(struct ip_vs_service_u
 		kfree(svc);
 	}
 	ip_vs_scheduler_put(sched);
+	ip_vs_pe_put(pe);
 
 	/* decrease the module use count */
 	ip_vs_use_count_dec();

@@ -1253,6 +1269,7 @@ static int
 ip_vs_edit_service(struct ip_vs_service *svc, struct ip_vs_service_user_kern *u)
 {
 	struct ip_vs_scheduler *sched, *old_sched;
+	struct ip_vs_pe *pe = NULL, *old_pe = NULL;
 	int ret = 0;
 
 	/*

@@ -1265,6 +1282,17 @@ ip_vs_edit_service(struct ip_vs_service
 	}
 	old_sched = sched;
 
+	if (u->pe_name && *u->pe_name) {
+		pe = ip_vs_pe_get(u->pe_name);
+		if (pe == NULL) {
+			pr_info("persistence engine module ip_vs_pe_%s "
+				"not found\n", u->pe_name);
+			ret = -ENOENT;
+			goto out;
+		}
+		old_pe = pe;
+	}
+
 #ifdef CONFIG_IP_VS_IPV6
 	if (u->af == AF_INET6 && (u->netmask < 1 || u->netmask > 128)) {
 		ret = -EINVAL;

@@ -1316,12 +1344,17 @@ ip_vs_edit_service(struct ip_vs_service
 		}
 	}
 
+	old_pe = svc->pe;
+	if (pe != old_pe) {
+		ip_vs_unbind_pe(svc);
+		ip_vs_bind_pe(svc, pe);
+	}
+
   out_unlock:
 	write_unlock_bh(&__ip_vs_svc_lock);
-#ifdef CONFIG_IP_VS_IPV6
   out:
-#endif
 	ip_vs_scheduler_put(old_sched);
+	ip_vs_pe_put(old_pe);
 	return ret;
 }
 

@@ -2553,6 +2586,8 @@ static const struct nla_policy ip_vs_svc
 	[IPVS_SVC_ATTR_FWMARK]		= { .type = NLA_U32 },
 	[IPVS_SVC_ATTR_SCHED_NAME]	= { .type = NLA_NUL_STRING,
 					    .len = IP_VS_SCHEDNAME_MAXLEN },
+	[IPVS_SVC_ATTR_PE_NAME]		= { .type = NLA_NUL_STRING,
+					    .len = IP_VS_PENAME_MAXLEN },
 	[IPVS_SVC_ATTR_FLAGS]		= { .type = NLA_BINARY,
 					    .len = sizeof(struct ip_vs_flags) },
 	[IPVS_SVC_ATTR_TIMEOUT]		= { .type = NLA_U32 },

@@ -2629,6 +2664,8 @@ static int ip_vs_genl_fill_service(struc
 	}
 
 	NLA_PUT_STRING(skb, IPVS_SVC_ATTR_SCHED_NAME, svc->scheduler->name);
+	if (svc->pe)
+		NLA_PUT_STRING(skb, IPVS_SVC_ATTR_PE_NAME, svc->pe->name);
 	NLA_PUT(skb, IPVS_SVC_ATTR_FLAGS, sizeof(flags), &flags);
 	NLA_PUT_U32(skb, IPVS_SVC_ATTR_TIMEOUT, svc->timeout / HZ);
 	NLA_PUT_U32(skb, IPVS_SVC_ATTR_NETMASK, svc->netmask);

@@ -2746,12 +2783,13 @@ static int ip_vs_genl_parse_service(stru
 
 	/* If a full entry was requested, check for the additional fields */
 	if (full_entry) {
-		struct nlattr *nla_sched, *nla_flags, *nla_timeout,
+		struct nlattr *nla_sched, *nla_pe, *nla_flags, *nla_timeout,
 			      *nla_netmask;
 		struct ip_vs_flags flags;
 		struct ip_vs_service *svc;
 
 		nla_sched = attrs[IPVS_SVC_ATTR_SCHED_NAME];
+		nla_pe = attrs[IPVS_SVC_ATTR_PE_NAME];
 		nla_flags = attrs[IPVS_SVC_ATTR_FLAGS];
 		nla_timeout = attrs[IPVS_SVC_ATTR_TIMEOUT];
 		nla_netmask = attrs[IPVS_SVC_ATTR_NETMASK];

@@ -2777,6 +2815,7 @@ static int ip_vs_genl_parse_service(stru
 		usvc->flags = (usvc->flags & ~flags.mask) |
 			      (flags.flags & flags.mask);
 		usvc->sched_name = nla_data(nla_sched);
+		usvc->pe_name = nla_pe ? nla_data(nla_pe) : NULL;
 		usvc->timeout = nla_get_u32(nla_timeout);
 		usvc->netmask = nla_get_u32(nla_netmask);
 	}
include/linux/ip_vs.h +3 -0
Index: nf-next-2.6/include/linux/ip_vs.h
===================================================================
@@ -326,6 +326,9 @@ enum {
 	IPVS_SVC_ATTR_NETMASK,		/* persistent netmask */
 
 	IPVS_SVC_ATTR_STATS,		/* nested attribute for service stats */
+
+	IPVS_SVC_ATTR_PE_NAME,		/* name of ct retriever */
+
 	__IPVS_SVC_ATTR_MAX,
 };
include/net/ip_vs.h +1 -0
Index: nf-next-2.6/include/net/ip_vs.h
===================================================================
@@ -442,6 +442,7 @@ struct ip_vs_service_user_kern {
 
 	/* virtual service options */
 	char			*sched_name;
+	char			*pe_name;
 	unsigned		flags;		/* virtual service flags */
 	unsigned		timeout;	/* persistent timeout in sec */
 	u32			netmask;	/* persistent netmask */
author maintainer
#11 lore raw

PATCH 11/12 IPVS: Fallback if persistence engine fails

In reply to Simon Horman

Fall back to normal persistence handling if the persistence engine fails to recognise a packet.

This way, at least the packet will go somewhere.

It is envisaged that iptables could be used to block packets such if this is not desired although nf_conntrack_sip would likely need to be enhanced first.

Signed-off-by Simon Horman <[email protected]>
net/netfilter/ipvs/ip_vs_conn.c +3 -3
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_conn.c
===================================================================
@@ -150,7 +150,7 @@ static unsigned int ip_vs_conn_hashkey(i
 
 static unsigned int ip_vs_conn_hashkey_param(const struct ip_vs_conn_param *p)
 {
-	if (p->pe && p->pe->hashkey_raw)
+	if (p->pe_data && p->pe->hashkey_raw)
 		return p->pe->hashkey_raw(p, ip_vs_conn_rnd) &
 			ip_vs_conn_tab_mask;
 	return ip_vs_conn_hashkey(p->af, p->protocol, p->caddr, p->cport);

@@ -340,7 +340,7 @@ struct ip_vs_conn *ip_vs_ct_in_get(const
 	ct_read_lock(hash);
 
 	list_for_each_entry(cp, &ip_vs_conn_tab[hash], c_list) {
-		if (p->pe && p->pe->ct_match) {
+		if (p->pe_data && p->pe->ct_match) {
 			if (p->pe->ct_match(p, cp))
 				goto out;
 			continue;

@@ -927,7 +927,7 @@ static int ip_vs_conn_seq_show(struct se
 		char pe_data[IP_VS_PENAME_MAXLEN + IP_VS_PEDATA_MAXLEN + 3];
 		size_t len = 0;
 
-		if (cp->dest->svc->pe && cp->dest->svc->pe->show_pe_data) {
+		if (cp->pe_data && cp->dest->svc->pe->show_pe_data) {
 			pe_data[0] = ' ';
 			len = strlen(cp->dest->svc->pe->name);
 			memcpy(pe_data + 1, cp->dest->svc->pe->name, len);
net/netfilter/ipvs/ip_vs_core.c +4 -6
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_core.c
===================================================================
@@ -175,7 +175,7 @@ ip_vs_set_state(struct ip_vs_conn *cp, i
 	return pp->state_transition(cp, direction, skb, pp);
 }
 
-static inline int
+static inline void
 ip_vs_conn_fill_param_persist(const struct ip_vs_service *svc,
 			      struct sk_buff *skb, int protocol,
 			      const union nf_inet_addr *caddr, __be16 cport,

@@ -185,8 +185,7 @@ ip_vs_conn_fill_param_persist(const stru
 	ip_vs_conn_fill_param(svc->af, protocol, caddr, cport, vaddr, vport, p);
 	p->pe = svc->pe;
 	if (p->pe && p->pe->fill_param)
-		return p->pe->fill_param(p, skb);
-	return 0;
+		p->pe->fill_param(p, skb);
 }
 
 /*

@@ -267,9 +266,8 @@ ip_vs_sched_persist(struct ip_vs_service
 				vaddr = &fwmark;
 			}
 		}
-		if (ip_vs_conn_fill_param_persist(svc, skb, protocol, &snet, 0,
-						  vaddr, vport, &param))
-			return NULL;
+		ip_vs_conn_fill_param_persist(svc, skb, protocol, &snet, 0,
+					      vaddr, vport, &param);
 	}
 
 	/* Check if a template already exists */
author maintainer
#12 lore raw

PATCH 12/12 IPVS: sip persistence engine

In reply to Simon Horman

Add the SIP callid as a key for persistence.

This allows multiple connections from the same IP address to be differentiated on the basis of the callid.

When used in conjunction with the persistence mask, it allows connections from different IP addresses to be aggregated on the basis of the callid.

It is envisaged that a persistence mask of 0.0.0.0 will be a useful setting. That is, ignore the source IP address when checking for persistence.

It is envisaged that this option will be used in conjunction with one-packet scheduling.

This only works with UDP and cannot be made to work with TCP within the current framework.

Signed-off-by Simon Horman <[email protected]>

v1
* Use buf[] instead of poiter arithmetic in ip_vs_dbg_callid() As suggested by Jan Engelhardt

net/netfilter/ipvs/Kconfig +7 -0
Index: nf-next-2.6/net/netfilter/ipvs/Kconfig
===================================================================
@@ -247,4 +247,11 @@ config	IP_VS_FTP
 	  If you want to compile it in kernel, say Y. To compile it as a
 	  module, choose M here. If unsure, say N.
 
+config	IP_VS_PE_SIP
+	tristate "SIP persistence engine"
+        depends on IP_VS_PROTO_UDP
+	depends on NF_CONNTRACK_SIP
+	---help---
+	  Allow persistence based on the SIP Call-ID
+
 endif # IP_VS
net/netfilter/ipvs/Makefile +3 -0
Index: nf-next-2.6/net/netfilter/ipvs/Makefile
===================================================================
@@ -32,3 +32,6 @@ obj-$(CONFIG_IP_VS_NQ) += ip_vs_nq.o
 
 # IPVS application helpers
 obj-$(CONFIG_IP_VS_FTP) += ip_vs_ftp.o
+
+# IPVS connection template retrievers
+obj-$(CONFIG_IP_VS_PE_SIP) += ip_vs_pe_sip.o
net/netfilter/ipvs/ip_vs_pe_sip.c new file +167 -0

167 lines not on this page · raw message

author maintainer
#13 lore raw
In reply to Simon Horman
Quoted 69 lines · Simon Horman

This patch series adds load-balancing of UDP SIP based on Call-ID to IPVS as well as a frame-work for extending IPVS to handle alternate persistence requirements.

REVISIONS

This is "patch v1" of this series, which addresses a few minor problems, as annotated on a per-patch basis, since the initial "rfc" posting. Internally there were 4 rfc versions, 0.1, 0.2, 0.3 and 0.4, some of the notes for some of the patches reflect those versions.

OVERVIEW

The approach that I have taken is what I call persistence engines. The basic idea being that you can provide a module to LVS that alters the way that it handles connection templates, which are at the core of persistence. In particular, an additional key can be added, and any of the normal IP address, port and protocol information can either be used or ignored.

In the case of the SIP persistence engine, the only persistence engine, all the keys used by the default persistence behaviour are used and the callid is added as an extra key. I originally intended to ignore the cip, but this can optionally be done by setting the persistence mask (-M) to 0.0.0.0 while allowing the flexibility of other mask values.

It is envisaged that the SIP persistence engine will be used in conjunction with one-packet scheduling. I'm interested to hear if that doesn't fit your needs.

CONFIGURATION

A persistence engine is associated with a virtual service (as are schedulers). I have added the --pe option to the ivpsadm -A and -E commands to allow the persistence engine of a virtual service to be added, changed, or deleted.

e.g. ipvsadm -A -u 10.4.3.192:5060 -p 60 -M 0.0.0.0 -o --pe sip

There are no other configuration parameters at this time.

RUNNING

When a connection template is created, if its virtual service has a persistence engine, then the persistence engine can add an extra key to the connection template. For the SIP module this is the callid. More generically, it is known as "pe data". And both the name of the persistence engine, "pe name", and "pe data" can be viewed in /proc/net/ip_vs_conn and by passing the --persistent-conn option to ipvsadm -Lc.

e.g.

# ipvsadm -Lcn --persistent-conn
UDP 00:38  UDP         10.4.3.0:0         10.4.3.192:5060    127.0.0.1:5060 sip 193373839

Here we see a single persistence template (cport is 0), which has been handled by the sip persistence engine. The pe data (callid) is 193373839.

In the case where the persistence engine can't match a packet for some reason, the connection will fall back to the normal persistence handling. This seems reasonable, as that if the packet ought to be dropped, iptables could be used.

A limited amount of debugging information has been added which can be enabled using a value of 9 or greater in /proc/sys/net/ipv4/vs/debug_level

Quoted 7 lines

CODE AVAILABILITY

The kernel patches (13) are available in git as the pe-0.4 branch of
git://git.kernel.org/pub/scm/linux/kernel/git/horms/lvs-test-2.6.git

The ipvsadm patches (2) are available in git as the pe-0.4 branch of
git://github.com/horms/ipvsadm-test.git

pe-0.4 should read pe-1 above (x2).
pe-0.4 is an older revision.

Quoted 1 line

I will post the ipvsadm patches separately

#14 lore raw
In reply to Simon Horman

Am 22.08.2010 14:44, schrieb Simon Horman:

Quoted 10 lines

This patch series adds load-balancing of UDP SIP based on Call-ID to IPVS as well as a frame-work for extending IPVS to handle alternate persistence requirements.

REVISIONS

This is "patch v1" of this series, which addresses a few minor problems, as annotated on a per-patch basis, since the initial "rfc" posting. Internally there were 4 rfc versions, 0.1, 0.2, 0.3 and 0.4, some of the notes for some of the patches reflect those versions.

This looks fine to me as far as I can judge.

Quoted 4 lines

CODE AVAILABILITY

The kernel patches (13) are available in git as the pe-0.4 branch of
git://git.kernel.org/pub/scm/linux/kernel/git/horms/lvs-test-2.6.git

Just to clarify, do you want me to merge this version?

author maintainer
#15 lore raw
Quoted 12 lines · Patrick McHardy

Am 22.08.2010 14:44, schrieb Simon Horman:

Quoted 10 lines

This patch series adds load-balancing of UDP SIP based on Call-ID to IPVS as well as a frame-work for extending IPVS to handle alternate persistence requirements.

REVISIONS

This is "patch v1" of this series, which addresses a few minor problems, as annotated on a per-patch basis, since the initial "rfc" posting. Internally there were 4 rfc versions, 0.1, 0.2, 0.3 and 0.4, some of the notes for some of the patches reflect those versions.

Quoted 8 lines

This looks fine to me as far as I can judge.

Quoted 4 lines

CODE AVAILABILITY

The kernel patches (13) are available in git as the pe-0.4 branch of
git://git.kernel.org/pub/scm/linux/kernel/git/horms/lvs-test-2.6.git

Just to clarify, do you want me to merge this version?

Assuming that it applies cleanly, yes.
Otherwise I can re-base an re-post.

#16 lore raw

Am 17.09.2010 04:52, schrieb Simon Horman:

Quoted 17 lines · Patrick McHardy
Quoted 16 lines

Am 22.08.2010 14:44, schrieb Simon Horman:

Quoted 10 lines

This patch series adds load-balancing of UDP SIP based on Call-ID to IPVS as well as a frame-work for extending IPVS to handle alternate persistence requirements.

REVISIONS

This is "patch v1" of this series, which addresses a few minor problems, as annotated on a per-patch basis, since the initial "rfc" posting. Internally there were 4 rfc versions, 0.1, 0.2, 0.3 and 0.4, some of the notes for some of the patches reflect those versions.

This looks fine to me as far as I can judge.

Quoted 2 lines

CODE AVAILABILITY

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

The kernel patches (13) are available in git as the pe-0.4 branch of
git://git.kernel.org/pub/scm/linux/kernel/git/horms/lvs-test-2.6.git

Just to clarify, do you want me to merge this version?

Assuming that it applies cleanly, yes.
Otherwise I can re-base an re-post.

There are some smaller conflicts, please rebase to the current nf-next tree. Thanks.

author maintainer
#17 lore raw
Quoted 21 lines · Patrick McHardy

Am 17.09.2010 04:52, schrieb Simon Horman:

Quoted 20 lines · Patrick McHardy
Quoted 19 lines

Am 22.08.2010 14:44, schrieb Simon Horman:

Quoted 10 lines

This patch series adds load-balancing of UDP SIP based on Call-ID to IPVS as well as a frame-work for extending IPVS to handle alternate persistence requirements.

REVISIONS

This is "patch v1" of this series, which addresses a few minor problems, as annotated on a per-patch basis, since the initial "rfc" posting. Internally there were 4 rfc versions, 0.1, 0.2, 0.3 and 0.4, some of the notes for some of the patches reflect those versions.

This looks fine to me as far as I can judge.

Quoted 4 lines

CODE AVAILABILITY

The kernel patches (13) are available in git as the pe-0.4 branch of
git://git.kernel.org/pub/scm/linux/kernel/git/horms/lvs-test-2.6.git

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Just to clarify, do you want me to merge this version?

Assuming that it applies cleanly, yes.
Otherwise I can re-base an re-post.

There are some smaller conflicts, please rebase to the current nf-next tree. Thanks.

Unfortunately a few bugs have shown up.
I'll get them ironed out before reposting.