Skip to contentSkip to conversation

We all need more expectations

PATCH Applied Pablo Neira Ayuso 2 participants 16 messages Sep 21, 2010 – Sep 28, 2010

Subsystems (2)
lore
Share & tools
lore

Pablo Neira Ayuso

author maintainer
#0 lore raw

cover We all need more expectations

Hi Patrick,

The following patches are focuses on conntrack expectations. The first one is an improvement for the situation in which the expectation table is full for conntrack NAT helpers. Then, another quite simple to include a missing attribute validation. To conclude, a couple of patches oriented to support user-space conntrack helpers.

Hope that you like them.

---

Pablo Neira Ayuso (4):
netfilter: nf_nat: better error handling of nf_ct_expect_related() in helpers
netfilter: ctnetlink: missing validation of CTA_EXPECT_ZONE attribute
netfilter: ctnetlink: allow to specify the expectation flags
netfilter: ctnetlink: add support for user-space expectation helpers

 include/linux/netfilter/nf_conntrack_common.h |    5 ++
 include/linux/netfilter/nfnetlink_conntrack.h |    1 
 include/net/netfilter/nf_conntrack_expect.h   |    3 -
 net/ipv4/netfilter/nf_nat_amanda.c            |    9 ++++
 net/ipv4/netfilter/nf_nat_ftp.c               |    9 ++++
 net/ipv4/netfilter/nf_nat_h323.c              |   53 ++++++++++++++++++++++---
 net/ipv4/netfilter/nf_nat_irc.c               |    9 ++++
 net/ipv4/netfilter/nf_nat_sip.c               |   27 +++++++++++--
 net/netfilter/nf_conntrack_expect.c           |   40 ++++++++++++-------
 net/netfilter/nf_conntrack_netlink.c          |   38 ++++++++++++------
 10 files changed, 149 insertions(+), 45 deletions(-)
author maintainer
#1 lore raw

PATCH 1/4 netfilter: nf_nat: better error handling of nf_ct_expect_related() in helpers

This patch improves the situation in which the expectation table is full for conntrack NAT helpers. Basically, we give up if we don't find a place in the table instead of looping over nf_ct_expect_related() with a different port (we should only do this if it returns -EBUSY, for -EMFILE or -ESHUTDOWN I think that it's better to skip this).

---
 net/ipv4/netfilter/nf_nat_amanda.c |    9 +++++-
 net/ipv4/netfilter/nf_nat_ftp.c    |    9 +++++-
 net/ipv4/netfilter/nf_nat_h323.c   |   53 +++++++++++++++++++++++++++++++-----
 net/ipv4/netfilter/nf_nat_irc.c    |    9 +++++-
 net/ipv4/netfilter/nf_nat_sip.c    |   27 ++++++++++++++++--
 5 files changed, 93 insertions(+), 14 deletions(-)
net/ipv4/netfilter/nf_nat_amanda.c +8 -1
index c31b876..0f23b3f 100644
@@ -44,9 +44,16 @@ static unsigned int help(struct sk_buff *skb,
 
 	/* Try to get same port: if not, try to change it. */
 	for (port = ntohs(exp->saved_proto.tcp.port); port != 0; port++) {
+		int ret;
+
 		exp->tuple.dst.u.tcp.port = htons(port);
-		if (nf_ct_expect_related(exp) == 0)
+		ret = nf_ct_expect_related(exp);
+		if (ret == 0)
+			break;
+		else if (ret != -EBUSY) {
+			port = 0;
 			break;
+		}
 	}
 
 	if (port == 0)
net/ipv4/netfilter/nf_nat_ftp.c +8 -1
index 86e0e84..dc73abb 100644
@@ -79,9 +79,16 @@ static unsigned int nf_nat_ftp(struct sk_buff *skb,
 
 	/* Try to get same port: if not, try to change it. */
 	for (port = ntohs(exp->saved_proto.tcp.port); port != 0; port++) {
+		int ret;
+
 		exp->tuple.dst.u.tcp.port = htons(port);
-		if (nf_ct_expect_related(exp) == 0)
+		ret = nf_ct_expect_related(exp);
+		if (ret == 0)
+			break;
+		else if (ret != -EBUSY) {
+			port = 0;
 			break;
+		}
 	}
 
 	if (port == 0)
net/ipv4/netfilter/nf_nat_h323.c +46 -7

95 lines not on this page · raw message

net/ipv4/netfilter/nf_nat_irc.c +8 -1
index ea83a88..535e1a8 100644
@@ -45,9 +45,16 @@ static unsigned int help(struct sk_buff *skb,
 
 	/* Try to get same port: if not, try to change it. */
 	for (port = ntohs(exp->saved_proto.tcp.port); port != 0; port++) {
+		int ret;
+
 		exp->tuple.dst.u.tcp.port = htons(port);
-		if (nf_ct_expect_related(exp) == 0)
+		ret = nf_ct_expect_related(exp);
+		if (ret == 0)
+			break;
+		else if (ret != -EBUSY) {
+			port = 0;
 			break;
+		}
 	}
 
 	if (port == 0)
net/ipv4/netfilter/nf_nat_sip.c +23 -4
index 11b538d..e40cf78 100644
@@ -307,9 +307,16 @@ static unsigned int ip_nat_sip_expect(struct sk_buff *skb, unsigned int dataoff,
 	exp->expectfn = ip_nat_sip_expected;
 
 	for (; port != 0; port++) {
+		int ret;
+
 		exp->tuple.dst.u.udp.port = htons(port);
-		if (nf_ct_expect_related(exp) == 0)
+		ret = nf_ct_expect_related(exp);
+		if (ret == 0)
+			break;
+		else if (ret != -EBUSY) {
+			port = 0;
 			break;
+		}
 	}
 
 	if (port == 0)
@@ -480,13 +487,25 @@ static unsigned int ip_nat_sdp_media(struct sk_buff *skb, unsigned int dataoff,
 	/* Try to get same pair of ports: if not, try to change them. */
 	for (port = ntohs(rtp_exp->tuple.dst.u.udp.port);
 	     port != 0; port += 2) {
+		int ret;
+
 		rtp_exp->tuple.dst.u.udp.port = htons(port);
-		if (nf_ct_expect_related(rtp_exp) != 0)
+		ret = nf_ct_expect_related(rtp_exp);
+		if (ret == -EBUSY)
 			continue;
+		else if (ret < 0) {
+			port = 0;
+			break;
+		}
 		rtcp_exp->tuple.dst.u.udp.port = htons(port + 1);
-		if (nf_ct_expect_related(rtcp_exp) == 0)
+		ret = nf_ct_expect_related(rtcp_exp);
+		if (ret == 0)
 			break;
-		nf_ct_unexpect_related(rtp_exp);
+		else if (ret != -EBUSY) {
+			nf_ct_unexpect_related(rtp_exp);
+			port = 0;
+			break;
+		}
 	}
 
 	if (port == 0)
#2 lore raw

Re: [PATCH 1/4] netfilter: nf_nat: better error handling of nf_ct_expect_related() in helpers

Am 21.09.2010 11:34, schrieb Pablo Neira Ayuso:

Quoted 5 lines

This patch improves the situation in which the expectation table is full for conntrack NAT helpers. Basically, we give up if we don't find a place in the table instead of looping over nf_ct_expect_related() with a different port (we should only do this if it returns -EBUSY, for -EMFILE or -ESHUTDOWN I think that it's better to skip this).

Indeed, this makes a lot more sense than our current behaviour.

#3 lore raw

Re: [PATCH 1/4] netfilter: nf_nat: better error handling of nf_ct_expect_related() in helpers

In reply to Pablo Neira Ayuso

Am 21.09.2010 11:34, schrieb Pablo Neira Ayuso:

Quoted 5 lines

This patch improves the situation in which the expectation table is full for conntrack NAT helpers. Basically, we give up if we don't find a place in the table instead of looping over nf_ct_expect_related() with a different port (we should only do this if it returns -EBUSY, for -EMFILE or -ESHUTDOWN I think that it's better to skip this).

Applied, thanks Pablo.

author maintainer
#4 lore raw

PATCH 2/4 netfilter: ctnetlink: missing validation of CTA_EXPECT_ZONE attribute

In reply to Pablo Neira Ayuso

This patch adds the missing validation of the CTA_EXPECT_ZONE attribute in the ctnetlink code.

---
 net/netfilter/nf_conntrack_netlink.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
net/netfilter/nf_conntrack_netlink.c +1 -0
index 5bae1cd..37533a3 100644
@@ -1733,6 +1733,7 @@ static const struct nla_policy exp_nla_policy[CTA_EXPECT_MAX+1] = {
 	[CTA_EXPECT_TIMEOUT]	= { .type = NLA_U32 },
 	[CTA_EXPECT_ID]		= { .type = NLA_U32 },
 	[CTA_EXPECT_HELP_NAME]	= { .type = NLA_NUL_STRING },
+	[CTA_EXPECT_ZONE]	= { .type = NLA_U16 },
 };
 
 static int
#5 lore raw

Re: [PATCH 2/4] netfilter: ctnetlink: missing validation of CTA_EXPECT_ZONE attribute

Am 21.09.2010 11:34, schrieb Pablo Neira Ayuso:

Quoted 2 lines

This patch adds the missing validation of the CTA_EXPECT_ZONE attribute in the ctnetlink code.

Applied, thanks.

author maintainer
#6 lore raw

PATCH 3/4 netfilter: ctnetlink: allow to specify the expectation flags

In reply to Pablo Neira Ayuso

With this patch, you can specify the expectation flags for user-space created expectations.

---
 include/linux/netfilter/nf_conntrack_common.h |    4 ++++
 include/linux/netfilter/nfnetlink_conntrack.h |    1 +
 include/net/netfilter/nf_conntrack_expect.h   |    3 ---
 net/netfilter/nf_conntrack_netlink.c          |    7 ++++++-
 4 files changed, 11 insertions(+), 4 deletions(-)
include/linux/netfilter/nf_conntrack_common.h +4 -0
index 1afd18c..fdc50ca 100644
@@ -100,6 +100,10 @@ enum ip_conntrack_expect_events {
 	IPEXP_NEW,		/* new expectation */
 };
 
+/* expectation flags */
+#define NF_CT_EXPECT_PERMANENT		0x1
+#define NF_CT_EXPECT_INACTIVE		0x2
+
 #ifdef __KERNEL__
 struct ip_conntrack_stat {
 	unsigned int searched;
include/linux/netfilter/nfnetlink_conntrack.h +1 -0
index 9ed534c..455f0ce 100644
@@ -161,6 +161,7 @@ enum ctattr_expect {
 	CTA_EXPECT_ID,
 	CTA_EXPECT_HELP_NAME,
 	CTA_EXPECT_ZONE,
+	CTA_EXPECT_FLAGS,
 	__CTA_EXPECT_MAX
 };
 #define CTA_EXPECT_MAX (__CTA_EXPECT_MAX - 1)
include/net/netfilter/nf_conntrack_expect.h +0 -3
index 11e8150..96bb42a 100644
@@ -67,9 +67,6 @@ struct nf_conntrack_expect_policy {
 
 #define NF_CT_EXPECT_CLASS_DEFAULT	0
 
-#define NF_CT_EXPECT_PERMANENT	0x1
-#define NF_CT_EXPECT_INACTIVE	0x2
-
 int nf_conntrack_expect_init(struct net *net);
 void nf_conntrack_expect_fini(struct net *net);
 
net/netfilter/nf_conntrack_netlink.c +6 -1
index 37533a3..82363f0 100644
@@ -1734,6 +1734,7 @@ static const struct nla_policy exp_nla_policy[CTA_EXPECT_MAX+1] = {
 	[CTA_EXPECT_ID]		= { .type = NLA_U32 },
 	[CTA_EXPECT_HELP_NAME]	= { .type = NLA_NUL_STRING },
 	[CTA_EXPECT_ZONE]	= { .type = NLA_U16 },
+	[CTA_EXPECT_FLAGS]	= { .type = NLA_U32 },
 };
 
 static int
@@ -1933,9 +1934,13 @@ ctnetlink_create_expect(struct net *net, u16 zone,
 		goto out;
 	}
 
+	if (cda[CTA_EXPECT_FLAGS])
+		exp->flags = ntohl(nla_get_be32(cda[CTA_EXPECT_FLAGS]));
+	else
+		exp->flags = 0;
+
 	exp->class = 0;
 	exp->expectfn = NULL;
-	exp->flags = 0;
 	exp->master = ct;
 	exp->helper = NULL;
 	memcpy(&exp->tuple, &tuple, sizeof(struct nf_conntrack_tuple));
#7 lore raw

Re: [PATCH 3/4] netfilter: ctnetlink: allow to specify the expectation flags

Am 21.09.2010 11:35, schrieb Pablo Neira Ayuso:

Quoted 2 lines

With this patch, you can specify the expectation flags for user-space created expectations.

We don't seem to be dumping the flags to userspace. I think this should be added for consistency.

author maintainer
#8 lore raw

Re: [PATCH 3/4] netfilter: ctnetlink: allow to specify the expectation flags

Quoted 6 lines · Patrick McHardy

Am 21.09.2010 11:35, schrieb Pablo Neira Ayuso:

Quoted 2 lines

With this patch, you can specify the expectation flags for user-space created expectations.

We don't seem to be dumping the flags to userspace. I think this should be added for consistency.

I have included this in a new version of the patch, find it attached.

netfilter: ctnetlink: allow to specify the expectation flags

With this patch, you can specify the expectation flags for user-space created expectations.

---
 include/linux/netfilter/nf_conntrack_common.h |    4 ++++
 include/linux/netfilter/nfnetlink_conntrack.h |    1 +
 include/net/netfilter/nf_conntrack_expect.h   |    3 ---
 net/netfilter/nf_conntrack_netlink.c          |    8 +++++++-
 4 files changed, 12 insertions(+), 4 deletions(-)
include/linux/netfilter/nf_conntrack_common.h +4 -0
index 1afd18c..fdc50ca 100644
@@ -100,6 +100,10 @@ enum ip_conntrack_expect_events {
 	IPEXP_NEW,		/* new expectation */
 };
 
+/* expectation flags */
+#define NF_CT_EXPECT_PERMANENT		0x1
+#define NF_CT_EXPECT_INACTIVE		0x2
+
 #ifdef __KERNEL__
 struct ip_conntrack_stat {
 	unsigned int searched;
include/linux/netfilter/nfnetlink_conntrack.h +1 -0
index 9ed534c..455f0ce 100644
@@ -161,6 +161,7 @@ enum ctattr_expect {
 	CTA_EXPECT_ID,
 	CTA_EXPECT_HELP_NAME,
 	CTA_EXPECT_ZONE,
+	CTA_EXPECT_FLAGS,
 	__CTA_EXPECT_MAX
 };
 #define CTA_EXPECT_MAX (__CTA_EXPECT_MAX - 1)
include/net/netfilter/nf_conntrack_expect.h +0 -3
index 11e8150..96bb42a 100644
@@ -67,9 +67,6 @@ struct nf_conntrack_expect_policy {
 
 #define NF_CT_EXPECT_CLASS_DEFAULT	0
 
-#define NF_CT_EXPECT_PERMANENT	0x1
-#define NF_CT_EXPECT_INACTIVE	0x2
-
 int nf_conntrack_expect_init(struct net *net);
 void nf_conntrack_expect_fini(struct net *net);
 
net/netfilter/nf_conntrack_netlink.c +7 -1
index 37533a3..0804e0e 100644
@@ -1577,6 +1577,7 @@ ctnetlink_exp_dump_expect(struct sk_buff *skb,
 
 	NLA_PUT_BE32(skb, CTA_EXPECT_TIMEOUT, htonl(timeout));
 	NLA_PUT_BE32(skb, CTA_EXPECT_ID, htonl((unsigned long)exp));
+	NLA_PUT_BE32(skb, CTA_EXPECT_FLAGS, htonl(exp->flags));
 	helper = rcu_dereference(nfct_help(master)->helper);
 	if (helper)
 		NLA_PUT_STRING(skb, CTA_EXPECT_HELP_NAME, helper->name);
@@ -1734,6 +1735,7 @@ static const struct nla_policy exp_nla_policy[CTA_EXPECT_MAX+1] = {
 	[CTA_EXPECT_ID]		= { .type = NLA_U32 },
 	[CTA_EXPECT_HELP_NAME]	= { .type = NLA_NUL_STRING },
 	[CTA_EXPECT_ZONE]	= { .type = NLA_U16 },
+	[CTA_EXPECT_FLAGS]	= { .type = NLA_U32 },
 };
 
 static int
@@ -1933,9 +1935,13 @@ ctnetlink_create_expect(struct net *net, u16 zone,
 		goto out;
 	}
 
+	if (cda[CTA_EXPECT_FLAGS])
+		exp->flags = ntohl(nla_get_be32(cda[CTA_EXPECT_FLAGS]));
+	else
+		exp->flags = 0;
+
 	exp->class = 0;
 	exp->expectfn = NULL;
-	exp->flags = 0;
 	exp->master = ct;
 	exp->helper = NULL;
 	memcpy(&exp->tuple, &tuple, sizeof(struct nf_conntrack_tuple));
#9 lore raw

Re: [PATCH 3/4] netfilter: ctnetlink: allow to specify the expectation flags

Am 22.09.2010 00:38, schrieb Pablo Neira Ayuso:

Quoted 9 lines · Patrick McHardy
Quoted 6 lines

Am 21.09.2010 11:35, schrieb Pablo Neira Ayuso:

Quoted 2 lines

With this patch, you can specify the expectation flags for user-space created expectations.

We don't seem to be dumping the flags to userspace. I think this should be added for consistency.

I have included this in a new version of the patch, find it attached.

Applied, thanks.

author maintainer
#10 lore raw

PATCH 4/4 netfilter: ctnetlink: add support for user-space expectation helpers

In reply to Pablo Neira Ayuso

This patch adds the basic infrastructure to support user-space expectation helpers via ctnetlink and the netfilter queuing infrastructure NFQUEUE. Basically, this patch:

* adds NF_CT_EXPECT_USERSPACE flag to identify user-space created expectations. I have also added a sanity check in __nf_ct_expect_check() to avoid that kernel-space helpers may create an expectation if the master conntrack has no helper assigned.
* adds some branches to check if the master conntrack helper exists, otherwise we skip the code that refers to kernel-space helper such as the local expectation list and the expectation policy.
* allows to set the timeout for user-space expectations with no helper assigned.

This patch also modifies ctnetlink to skip including the helper name in the Netlink messages if no kernel-space helper is set (since no user-space expectation has not kernel-space kernel assigned).

You can access an example user-space FTP conntrack helper at:
http://people.netfilter.org/pablo/nf-ftp-helper-userspace-POC.tar.bz

---
 include/linux/netfilter/nf_conntrack_common.h |    1 +
 net/netfilter/nf_conntrack_expect.c           |   40 ++++++++++++++++---------
 net/netfilter/nf_conntrack_netlink.c          |   38 ++++++++++++++----------
 3 files changed, 48 insertions(+), 31 deletions(-)
include/linux/netfilter/nf_conntrack_common.h +1 -0
index fdc50ca..23a1a08 100644
@@ -103,6 +103,7 @@ enum ip_conntrack_expect_events {
 /* expectation flags */
 #define NF_CT_EXPECT_PERMANENT		0x1
 #define NF_CT_EXPECT_INACTIVE		0x2
+#define NF_CT_EXPECT_USERSPACE		0x4
 
 #ifdef __KERNEL__
 struct ip_conntrack_stat {
net/netfilter/nf_conntrack_expect.c +25 -15
index acb29cc..b53e817 100644
@@ -44,14 +44,15 @@ void nf_ct_unlink_expect(struct nf_conntrack_expect *exp)
 	struct nf_conn_help *master_help = nfct_help(exp->master);
 	struct net *net = nf_ct_exp_net(exp);
 
-	NF_CT_ASSERT(master_help);
 	NF_CT_ASSERT(!timer_pending(&exp->timeout));
 
 	hlist_del_rcu(&exp->hnode);
 	net->ct.expect_count--;
 
-	hlist_del(&exp->lnode);
-	master_help->expecting[exp->class]--;
+	if (master_help) {
+		hlist_del(&exp->lnode);
+		master_help->expecting[exp->class]--;
+	}
 	nf_ct_expect_put(exp);
 
 	NF_CT_STAT_INC(net, expect_delete);
@@ -320,16 +321,20 @@ static void nf_ct_expect_insert(struct nf_conntrack_expect *exp)
 
 	atomic_inc(&exp->use);
 
-	hlist_add_head(&exp->lnode, &master_help->expectations);
-	master_help->expecting[exp->class]++;
+	if (master_help) {
+		hlist_add_head(&exp->lnode, &master_help->expectations);
+		master_help->expecting[exp->class]++;
+	}
 
 	hlist_add_head_rcu(&exp->hnode, &net->ct.expect_hash[h]);
 	net->ct.expect_count++;
 
 	setup_timer(&exp->timeout, nf_ct_expectation_timed_out,
 		    (unsigned long)exp);
-	p = &master_help->helper->expect_policy[exp->class];
-	exp->timeout.expires = jiffies + p->timeout * HZ;
+	if (master_help) {
+		p = &master_help->helper->expect_policy[exp->class];
+		exp->timeout.expires = jiffies + p->timeout * HZ;
+	}
 	add_timer(&exp->timeout);
 
 	atomic_inc(&exp->use);
@@ -380,7 +385,9 @@ static inline int __nf_ct_expect_check(struct nf_conntrack_expect *expect)
 	unsigned int h;
 	int ret = 1;
 
-	if (!master_help->helper) {
+	/* Don't allow expectations created from kernel-space with no helper */
+	if (!(expect->flags & NF_CT_EXPECT_USERSPACE) &&
+	    (!master_help || (master_help && !master_help->helper))) {
 		ret = -ESHUTDOWN;
 		goto out;
 	}
@@ -398,13 +405,16 @@ static inline int __nf_ct_expect_check(struct nf_conntrack_expect *expect)
 		}
 	}
 	/* Will be over limit? */
-	p = &master_help->helper->expect_policy[expect->class];
-	if (p->max_expected &&
-	    master_help->expecting[expect->class] >= p->max_expected) {
-		evict_oldest_expect(master, expect);
-		if (master_help->expecting[expect->class] >= p->max_expected) {
-			ret = -EMFILE;
-			goto out;
+	if (master_help) {
+		p = &master_help->helper->expect_policy[expect->class];
+		if (p->max_expected &&
+		    master_help->expecting[expect->class] >= p->max_expected) {
+			evict_oldest_expect(master, expect);
+			if (master_help->expecting[expect->class]
+						>= p->max_expected) {
+				ret = -EMFILE;
+				goto out;
+			}
 		}
 	}
 
net/netfilter/nf_conntrack_netlink.c +22 -16
index 82363f0..cddf4b6 100644
@@ -1560,8 +1560,8 @@ ctnetlink_exp_dump_expect(struct sk_buff *skb,
 			  const struct nf_conntrack_expect *exp)
 {
 	struct nf_conn *master = exp->master;
-	struct nf_conntrack_helper *helper;
 	long timeout = (exp->timeout.expires - jiffies) / HZ;
+	struct nf_conn_help *help;
 
 	if (timeout < 0)
 		timeout = 0;
@@ -1577,9 +1577,14 @@ ctnetlink_exp_dump_expect(struct sk_buff *skb,
 
 	NLA_PUT_BE32(skb, CTA_EXPECT_TIMEOUT, htonl(timeout));
 	NLA_PUT_BE32(skb, CTA_EXPECT_ID, htonl((unsigned long)exp));
-	helper = rcu_dereference(nfct_help(master)->helper);
-	if (helper)
-		NLA_PUT_STRING(skb, CTA_EXPECT_HELP_NAME, helper->name);
+	help = nfct_help(master);
+	if (help) {
+		struct nf_conntrack_helper *helper;
+
+		helper = rcu_dereference(help->helper);
+		if (helper)
+			NLA_PUT_STRING(skb, CTA_EXPECT_HELP_NAME, helper->name);
+	}
 
 	return 0;
 
@@ -1920,24 +1925,25 @@ ctnetlink_create_expect(struct net *net, u16 zone,
 	if (!h)
 		return -ENOENT;
 	ct = nf_ct_tuplehash_to_ctrack(h);
-	help = nfct_help(ct);
-
-	if (!help || !help->helper) {
-		/* such conntrack hasn't got any helper, abort */
-		err = -EOPNOTSUPP;
-		goto out;
-	}
-
 	exp = nf_ct_expect_alloc(ct);
 	if (!exp) {
 		err = -ENOMEM;
 		goto out;
 	}
-
-	if (cda[CTA_EXPECT_FLAGS])
+	help = nfct_help(ct);
+	if (!help) {
+		if (!cda[CTA_EXPECT_TIMEOUT]) {
+			err = -EINVAL;
+			goto out;
+		}
+		exp->timeout.expires =
+		  jiffies + ntohl(nla_get_be32(cda[CTA_EXPECT_TIMEOUT])) * HZ;
+	}
+	if (cda[CTA_EXPECT_FLAGS]) {
 		exp->flags = ntohl(nla_get_be32(cda[CTA_EXPECT_FLAGS]));
-	else
-		exp->flags = 0;
+		exp->flags |= NF_CT_EXPECT_USERSPACE;
+	} else
+		exp->flags = NF_CT_EXPECT_USERSPACE;
 
 	exp->class = 0;
 	exp->expectfn = NULL;
#11 lore raw

Re: [PATCH 4/4] netfilter: ctnetlink: add support for user-space expectation helpers

Am 21.09.2010 11:35, schrieb Pablo Neira Ayuso:

Quoted 16 lines

This patch adds the basic infrastructure to support user-space expectation helpers via ctnetlink and the netfilter queuing infrastructure NFQUEUE. Basically, this patch:

* adds NF_CT_EXPECT_USERSPACE flag to identify user-space created expectations. I have also added a sanity check in __nf_ct_expect_check() to avoid that kernel-space helpers may create an expectation if the master conntrack has no helper assigned.
* adds some branches to check if the master conntrack helper exists, otherwise we skip the code that refers to kernel-space helper such as the local expectation list and the expectation policy.
* allows to set the timeout for user-space expectations with no helper assigned.

Quoted 4 lines

This patch also modifies ctnetlink to skip including the helper name in the Netlink messages if no kernel-space helper is set (since no user-space expectation has not kernel-space kernel assigned).

My main question is - what will be cleaning up these expectations on module unload? Currently expectations are cleaned up on unload of the corresponding helper module, which obviously doesn't happen in this case.

author maintainer
#12 lore raw

Re: [PATCH 4/4] netfilter: ctnetlink: add support for user-space expectation helpers

Quoted 22 lines · Patrick McHardy

Am 21.09.2010 11:35, schrieb Pablo Neira Ayuso:

Quoted 16 lines

This patch adds the basic infrastructure to support user-space expectation helpers via ctnetlink and the netfilter queuing infrastructure NFQUEUE. Basically, this patch:

* adds NF_CT_EXPECT_USERSPACE flag to identify user-space created expectations. I have also added a sanity check in __nf_ct_expect_check() to avoid that kernel-space helpers may create an expectation if the master conntrack has no helper assigned.
* adds some branches to check if the master conntrack helper exists, otherwise we skip the code that refers to kernel-space helper such as the local expectation list and the expectation policy.
* allows to set the timeout for user-space expectations with no helper assigned.

Quoted 4 lines

This patch also modifies ctnetlink to skip including the helper name in the Netlink messages if no kernel-space helper is set (since no user-space expectation has not kernel-space kernel assigned).

Quoted 4 lines

My main question is - what will be cleaning up these expectations on module unload? Currently expectations are cleaned up on unload of the corresponding helper module, which obviously doesn't happen in this case.

Indeed. I have reworked the patch to add the nf_ct_userspace_expect_list that is used to delete all the user-space created expectations if ctnetlink is unloaded.

netfilter: ctnetlink: add support for user-space expectation helpers

This patch adds the basic infrastructure to support user-space expectation helpers via ctnetlink and the netfilter queuing infrastructure NFQUEUE. Basically, this patch:

* adds NF_CT_EXPECT_USERSPACE flag to identify user-space created expectations. I have also added a sanity check in __nf_ct_expect_check() to avoid that kernel-space helpers may create an expectation if the master conntrack has no helper assigned.
* adds some branches to check if the master conntrack helper exists, otherwise we skip the code that refers to kernel-space helper such as the local expectation list and the expectation policy.
* allows to set the timeout for user-space expectations with no helper assigned.
* a list of expectations created from user-space that depends on ctnetlink (if this module is removed, they are deleted).

This patch also modifies ctnetlink to skip including the helper name in the Netlink messages if no kernel-space helper is set (since no user-space expectation has not kernel-space kernel assigned).

You can access an example user-space FTP conntrack helper at:
http://people.netfilter.org/pablo/nf-ftp-helper-userspace-POC.tar.bz

---
 include/linux/netfilter/nf_conntrack_common.h |    1 
 include/net/netfilter/nf_conntrack_expect.h   |    1 
 net/netfilter/nf_conntrack_expect.c           |   61 +++++++++++++++++++------
 net/netfilter/nf_conntrack_netlink.c          |   39 +++++++++-------
 4 files changed, 72 insertions(+), 30 deletions(-)
include/linux/netfilter/nf_conntrack_common.h +1 -0
index fdc50ca..23a1a08 100644
@@ -103,6 +103,7 @@ enum ip_conntrack_expect_events {
 /* expectation flags */
 #define NF_CT_EXPECT_PERMANENT		0x1
 #define NF_CT_EXPECT_INACTIVE		0x2
+#define NF_CT_EXPECT_USERSPACE		0x4
 
 #ifdef __KERNEL__
 struct ip_conntrack_stat {
include/net/netfilter/nf_conntrack_expect.h +1 -0
index 96bb42a..416b838 100644
@@ -85,6 +85,7 @@ nf_ct_find_expectation(struct net *net, u16 zone,
 void nf_ct_unlink_expect(struct nf_conntrack_expect *exp);
 void nf_ct_remove_expectations(struct nf_conn *ct);
 void nf_ct_unexpect_related(struct nf_conntrack_expect *exp);
+void nf_ct_remove_userspace_expectations(void);
 
 /* Allocate space for an expectation: this is mandatory before calling
    nf_ct_expect_related.  You will have to call put afterwards. */
net/netfilter/nf_conntrack_expect.c +47 -14

109 lines not on this page · raw message

net/netfilter/nf_conntrack_netlink.c +23 -16
index 0804e0e..83b11e3 100644
@@ -1560,8 +1560,8 @@ ctnetlink_exp_dump_expect(struct sk_buff *skb,
 			  const struct nf_conntrack_expect *exp)
 {
 	struct nf_conn *master = exp->master;
-	struct nf_conntrack_helper *helper;
 	long timeout = (exp->timeout.expires - jiffies) / HZ;
+	struct nf_conn_help *help;
 
 	if (timeout < 0)
 		timeout = 0;
@@ -1578,9 +1578,14 @@ ctnetlink_exp_dump_expect(struct sk_buff *skb,
 	NLA_PUT_BE32(skb, CTA_EXPECT_TIMEOUT, htonl(timeout));
 	NLA_PUT_BE32(skb, CTA_EXPECT_ID, htonl((unsigned long)exp));
 	NLA_PUT_BE32(skb, CTA_EXPECT_FLAGS, htonl(exp->flags));
-	helper = rcu_dereference(nfct_help(master)->helper);
-	if (helper)
-		NLA_PUT_STRING(skb, CTA_EXPECT_HELP_NAME, helper->name);
+	help = nfct_help(master);
+	if (help) {
+		struct nf_conntrack_helper *helper;
+
+		helper = rcu_dereference(help->helper);
+		if (helper)
+			NLA_PUT_STRING(skb, CTA_EXPECT_HELP_NAME, helper->name);
+	}
 
 	return 0;
 
@@ -1921,24 +1926,25 @@ ctnetlink_create_expect(struct net *net, u16 zone,
 	if (!h)
 		return -ENOENT;
 	ct = nf_ct_tuplehash_to_ctrack(h);
-	help = nfct_help(ct);
-
-	if (!help || !help->helper) {
-		/* such conntrack hasn't got any helper, abort */
-		err = -EOPNOTSUPP;
-		goto out;
-	}
-
 	exp = nf_ct_expect_alloc(ct);
 	if (!exp) {
 		err = -ENOMEM;
 		goto out;
 	}
-
-	if (cda[CTA_EXPECT_FLAGS])
+	help = nfct_help(ct);
+	if (!help) {
+		if (!cda[CTA_EXPECT_TIMEOUT]) {
+			err = -EINVAL;
+			goto out;
+		}
+		exp->timeout.expires =
+		  jiffies + ntohl(nla_get_be32(cda[CTA_EXPECT_TIMEOUT])) * HZ;
+	}
+	if (cda[CTA_EXPECT_FLAGS]) {
 		exp->flags = ntohl(nla_get_be32(cda[CTA_EXPECT_FLAGS]));
-	else
-		exp->flags = 0;
+		exp->flags |= NF_CT_EXPECT_USERSPACE;
+	} else
+		exp->flags = NF_CT_EXPECT_USERSPACE;
 
 	exp->class = 0;
 	exp->expectfn = NULL;
@@ -2109,6 +2115,7 @@ static void __exit ctnetlink_exit(void)
 {
 	pr_info("ctnetlink: unregistering from nfnetlink.\n");
 
+	nf_ct_remove_userspace_expectations();
 #ifdef CONFIG_NF_CONNTRACK_EVENTS
 	nf_ct_expect_unregister_notifier(&ctnl_notifier_exp);
 	nf_conntrack_unregister_notifier(&ctnl_notifier);
#13 lore raw

Re: [PATCH 4/4] netfilter: ctnetlink: add support for user-space expectation helpers

Am 22.09.2010 00:38, schrieb Pablo Neira Ayuso:

Quoted 15 lines
Quoted 5 lines

My main question is - what will be cleaning up these expectations on module unload? Currently expectations are cleaned up on unload of the corresponding helper module, which obviously doesn't happen in this case.

Indeed. I have reworked the patch to add the nf_ct_userspace_expect_list that is used to delete all the user-space created expectations if ctnetlink is unloaded.

include/linux/netfilter/nf_conntrack_common.h +0 -0
index fdc50ca..23a1a08 100644

include/linux/netfilter/nf_conntrack_common.h enum ip_conntrack_expect_events { · line 106

Quoted 4 lines
include/linux/netfilter/nf_conntrack_common.h +1 -0
@@ -103,6 +103,7 @@ enum ip_conntrack_expect_events {
 /* expectation flags */
 #define NF_CT_EXPECT_PERMANENT		0x1
 #define NF_CT_EXPECT_INACTIVE		0x2
+#define NF_CT_EXPECT_USERSPACE		0x4

Does this flag need to be exposed to userspace? I also don't see anything preventing userspace incorrectly setting it on an expectation that actually does have a master, which will probably cause problems later on.

net/netfilter/nf_conntrack_expect.c static int nf_ct_expect_hash_rnd_initted __read_mostly; · line 42

Quoted 10 lines
net/netfilter/nf_conntrack_expect.c +2 -0
index acb29cc..361a8ba 100644
@@ -38,20 +38,26 @@ static int nf_ct_expect_hash_rnd_initted __read_mostly;
 
 static struct kmem_cache *nf_ct_expect_cachep __read_mostly;
 
+static HLIST_HEAD(nf_ct_userspace_expect_list);
+static int nf_ct_userspace_expect_list_counter;

This counter is write-only.

author maintainer
#14 lore raw

Re: [PATCH 4/4] netfilter: ctnetlink: add support for user-space expectation helpers

Hi Patrick,

Quoted 6 lines · Patrick McHardy
Quoted 6 lines
include/linux/netfilter/nf_conntrack_common.h +0 -0
index fdc50ca..23a1a08 100644
@@ -103,6 +103,7 @@ enum ip_conntrack_expect_events {
 /* expectation flags */
Quoted 8 lines

include/linux/netfilter/nf_conntrack_common.h enum ip_conntrack_expect_events { · line 106

Quoted 3 lines
include/linux/netfilter/nf_conntrack_common.h +1 -0
 #define NF_CT_EXPECT_PERMANENT		0x1
 #define NF_CT_EXPECT_INACTIVE		0x2
+#define NF_CT_EXPECT_USERSPACE		0x4

Does this flag need to be exposed to userspace? I also don't see anything preventing userspace incorrectly setting it on an expectation that actually does have a master, which will probably cause problems later on.

I have reworked the patch again: the flag is exposed to user-space but you cannot set/unset it.

Quoted 6 lines
Quoted 6 lines
net/netfilter/nf_conntrack_expect.c +0 -0
index acb29cc..361a8ba 100644
@@ -38,20 +38,26 @@ static int nf_ct_expect_hash_rnd_initted __read_mostly;
 
Quoted 6 lines

net/netfilter/nf_conntrack_expect.c static int nf_ct_expect_hash_rnd_initted __read_mostly; · line 42

Quoted 4 lines
net/netfilter/nf_conntrack_expect.c +2 -0
 static struct kmem_cache *nf_ct_expect_cachep __read_mostly;
 
+static HLIST_HEAD(nf_ct_userspace_expect_list);
+static int nf_ct_userspace_expect_list_counter;

This counter is write-only.

Removed this counter!

I have also modified the /proc output to display USERSPACE for expectation whose user-space flag is set (this was missing in the previous patch).

netfilter: ctnetlink: add support for user-space expectation helpers

This patch adds the basic infrastructure to support user-space expectation helpers via ctnetlink and the netfilter queuing infrastructure NFQUEUE. Basically, this patch:

* adds NF_CT_EXPECT_USERSPACE flag to identify user-space created expectations. I have also added a sanity check in __nf_ct_expect_check() to avoid that kernel-space helpers may create an expectation if the master conntrack has no helper assigned.
* adds some branches to check if the master conntrack helper exists, otherwise we skip the code that refers to kernel-space helper such as the local expectation list and the expectation policy.
* allows to set the timeout for user-space expectations with no helper assigned.
* a list of expectations created from user-space that depends on ctnetlink (if this module is removed, they are deleted).
* includes USERSPACE in the /proc output for expectations that have been created by a user-space helper.

This patch also modifies ctnetlink to skip including the helper name in the Netlink messages if no kernel-space helper is set (since no user-space expectation has not kernel-space kernel assigned).

You can access an example user-space FTP conntrack helper at:
http://people.netfilter.org/pablo/nf-ftp-helper-userspace-POC.tar.bz

---
 include/linux/netfilter/nf_conntrack_common.h |    1 
 include/net/netfilter/nf_conntrack_expect.h   |    1 
 net/netfilter/nf_conntrack_expect.c           |   62 +++++++++++++++++++------
 net/netfilter/nf_conntrack_netlink.c          |   46 ++++++++++++-------
 4 files changed, 79 insertions(+), 31 deletions(-)
include/linux/netfilter/nf_conntrack_common.h +1 -0
index fdc50ca..23a1a08 100644
@@ -103,6 +103,7 @@ enum ip_conntrack_expect_events {
 /* expectation flags */
 #define NF_CT_EXPECT_PERMANENT		0x1
 #define NF_CT_EXPECT_INACTIVE		0x2
+#define NF_CT_EXPECT_USERSPACE		0x4
 
 #ifdef __KERNEL__
 struct ip_conntrack_stat {
include/net/netfilter/nf_conntrack_expect.h +1 -0
index 96bb42a..416b838 100644
@@ -85,6 +85,7 @@ nf_ct_find_expectation(struct net *net, u16 zone,
 void nf_ct_unlink_expect(struct nf_conntrack_expect *exp);
 void nf_ct_remove_expectations(struct nf_conn *ct);
 void nf_ct_unexpect_related(struct nf_conntrack_expect *exp);
+void nf_ct_remove_userspace_expectations(void);
 
 /* Allocate space for an expectation: this is mandatory before calling
    nf_ct_expect_related.  You will have to call put afterwards. */
net/netfilter/nf_conntrack_expect.c +47 -15

117 lines not on this page · raw message

net/netfilter/nf_conntrack_netlink.c +30 -16

77 lines not on this page · raw message

#15 lore raw

Re: [PATCH 4/4] netfilter: ctnetlink: add support for user-space expectation helpers

In reply to Pablo Neira Ayuso

Am 21.09.2010 11:35, schrieb Pablo Neira Ayuso:

Quoted 16 lines

This patch adds the basic infrastructure to support user-space expectation helpers via ctnetlink and the netfilter queuing infrastructure NFQUEUE. Basically, this patch:

* adds NF_CT_EXPECT_USERSPACE flag to identify user-space created expectations. I have also added a sanity check in __nf_ct_expect_check() to avoid that kernel-space helpers may create an expectation if the master conntrack has no helper assigned.
* adds some branches to check if the master conntrack helper exists, otherwise we skip the code that refers to kernel-space helper such as the local expectation list and the expectation policy.
* allows to set the timeout for user-space expectations with no helper assigned.

Quoted 7 lines

This patch also modifies ctnetlink to skip including the helper name in the Netlink messages if no kernel-space helper is set (since no user-space expectation has not kernel-space kernel assigned).

You can access an example user-space FTP conntrack helper at:
http://people.netfilter.org/pablo/nf-ftp-helper-userspace-POC.tar.bz

Applied, thanks Pablo. I've also fixed up the URL to include userspace-conntrack-helpers/ in the path :)