Skip to contentSkip to conversation

net: sched: add helper support in act_ct for ovs offloading

PATCH In review Xin Long 3 participants 9 messages Sep 23, 2022 – Oct 4, 2022

Subsystems (3)
lore
Share & tools
lore

Xin Long

author
#0 lore raw

cover net: sched: add helper support in act_ct for ovs offloading

Ilya reported an issue that FTP traffic would be broken when the OVS flow with ct(commit,alg=ftp) was installed in the OVS kernel module, which was caused by that TC didn't support the ftp helper offloaded from OVS.

This patchset is to add the helper support for ovs offloading. Note that the 1st patch is fixing a memleak issue, and also making the 2nd patch easier to write for the feature support.

Xin Long (2):
net: sched: fix the err path of tcf_ct_init in act_ct
net: sched: add helper support in act_ct

 include/net/tc_act/tc_ct.h        |   1 +
 include/uapi/linux/tc_act/tc_ct.h |   3 +
 net/sched/act_ct.c                | 193 +++++++++++++++++++++++++++---
 3 files changed, 182 insertions(+), 15 deletions(-)
Signature · 1 line
2.31.1
author
#1 lore raw

PATCH 1/2 net: sched: fix the err path of tcf_ct_init in act_ct

When it returns err from tcf_ct_flow_table_get(), the param tmpl should have been freed in the cleanup. Otherwise a memory leak will occur.

While fixing this problem, this patch also makes the err path simple by calling tcf_ct_params_free(), so that it won't cause problems when more members are added into param and need freeing on the err path.

Fixes c34b961a2492 ("net/sched: act_ct: Create nf flow table per zone")
---
 net/sched/act_ct.c | 30 +++++++++++++++++-------------
 1 file changed, 17 insertions(+), 13 deletions(-)
net/sched/act_ct.c +17 -13
index 9d19710835b0..193a460a9d7f 100644
@@ -345,11 +345,9 @@ static void tcf_ct_flow_table_cleanup_work(struct work_struct *work)
 	module_put(THIS_MODULE);
 }
 
-static void tcf_ct_flow_table_put(struct tcf_ct_params *params)
+static void tcf_ct_flow_table_put(struct tcf_ct_flow_table *ct_ft)
 {
-	struct tcf_ct_flow_table *ct_ft = params->ct_ft;
-
-	if (refcount_dec_and_test(&params->ct_ft->ref)) {
+	if (refcount_dec_and_test(&ct_ft->ref)) {
 		rhashtable_remove_fast(&zones_ht, &ct_ft->node, zones_params);
 		INIT_RCU_WORK(&ct_ft->rwork, tcf_ct_flow_table_cleanup_work);
 		queue_rcu_work(act_ct_wq, &ct_ft->rwork);
@@ -832,18 +830,23 @@ static int tcf_ct_handle_fragments(struct net *net, struct sk_buff *skb,
 	return err;
 }
 
-static void tcf_ct_params_free(struct rcu_head *head)
+static void tcf_ct_params_free(struct tcf_ct_params *params)
 {
-	struct tcf_ct_params *params = container_of(head,
-						    struct tcf_ct_params, rcu);
-
-	tcf_ct_flow_table_put(params);
-
+	if (params->ct_ft)
+		tcf_ct_flow_table_put(params->ct_ft);
 	if (params->tmpl)
 		nf_ct_put(params->tmpl);
 	kfree(params);
 }
 
+static void tcf_ct_params_free_rcu(struct rcu_head *head)
+{
+	struct tcf_ct_params *params;
+
+	params = container_of(head, struct tcf_ct_params, rcu);
+	tcf_ct_params_free(params);
+}
+
 #if IS_ENABLED(CONFIG_NF_NAT)
 /* Modelled after nf_nat_ipv[46]_fn().
  * range is only used for new, uninitialized NAT state.
@@ -1401,14 +1404,15 @@ static int tcf_ct_init(struct net *net, struct nlattr *nla,
 	if (goto_ch)
 		tcf_chain_put_by_act(goto_ch);
 	if (params)
-		call_rcu(&params->rcu, tcf_ct_params_free);
+		call_rcu(&params->rcu, tcf_ct_params_free_rcu);
 
 	return res;
 
 cleanup:
 	if (goto_ch)
 		tcf_chain_put_by_act(goto_ch);
-	kfree(params);
+	if (params)
+		tcf_ct_params_free(params);
 	tcf_idr_release(*a, bind);
 	return err;
 }
@@ -1420,7 +1424,7 @@ static void tcf_ct_cleanup(struct tc_action *a)
 
 	params = rcu_dereference_protected(c->params, 1);
 	if (params)
-		call_rcu(&params->rcu, tcf_ct_params_free);
+		call_rcu(&params->rcu, tcf_ct_params_free_rcu);
 }
 
 static int tcf_ct_dump_key_val(struct sk_buff *skb,
Signature · 1 line
2.31.1
author
#5 lore raw

PATCH 2/2 net: sched: add helper support in act_ct

In reply to Xin Long

This patch is to add helper support in act_ct for OVS actions=ct(alg=xxx) offloading, which is corresponding to Commit cae3a2627520 ("openvswitch:
Allow attaching helpers to ct action") in OVS kernel part.

The difference is when adding TC actions family and proto cannot be got from the filter/match, other than helper name in tb[TCA_CT_HELPER_NAME], we also need to send the family in tb[TCA_CT_HELPER_FAMILY] and the proto in tb[TCA_CT_HELPER_PROTO] to kernel.

Note when calling helper->help() in tcf_ct_act(), the packet will be dropped if skb's family and proto do not match the helper's.

---
 include/net/tc_act/tc_ct.h        |   1 +
 include/uapi/linux/tc_act/tc_ct.h |   3 +
 net/sched/act_ct.c                | 163 +++++++++++++++++++++++++++++-
 3 files changed, 165 insertions(+), 2 deletions(-)
include/net/tc_act/tc_ct.h +1 -0
index 8250d6f0a462..b24ea2d9400b 100644
@@ -10,6 +10,7 @@
 #include <net/netfilter/nf_conntrack_labels.h>
 
 struct tcf_ct_params {
+	struct nf_conntrack_helper *helper;
 	struct nf_conn *tmpl;
 	u16 zone;
 
include/uapi/linux/tc_act/tc_ct.h +3 -0
index 5fb1d7ac1027..6c5200f0ed38 100644
@@ -22,6 +22,9 @@ enum {
 	TCA_CT_NAT_PORT_MIN,	/* be16 */
 	TCA_CT_NAT_PORT_MAX,	/* be16 */
 	TCA_CT_PAD,
+	TCA_CT_HELPER_NAME,	/* string */
+	TCA_CT_HELPER_FAMILY,	/* u8 */
+	TCA_CT_HELPER_PROTO,	/* u8 */
 	__TCA_CT_MAX
 };
 
net/sched/act_ct.c +161 -2

233 lines not on this page · raw message

Signature · 1 line
2.31.1
maintainer
#6 lore raw

Re: [PATCH net-next 2/2] net: sched: add helper support in act_ct

Quoted 9 lines · Xin Long

This patch is to add helper support in act_ct for OVS actions=ct(alg=xxx) offloading, which is corresponding to Commit cae3a2627520 ("openvswitch:
Allow attaching helpers to ct action") in OVS kernel part.

The difference is when adding TC actions family and proto cannot be got from the filter/match, other than helper name in tb[TCA_CT_HELPER_NAME], we also need to send the family in tb[TCA_CT_HELPER_FAMILY] and the proto in tb[TCA_CT_HELPER_PROTO] to kernel.

Quoted 4 lines

Note when calling helper->help() in tcf_ct_act(), the packet will be dropped if skb's family and proto do not match the helper's.

This tag is a bit out of place here, as it should belong to fixes. Do you mean 'Suggested-by' ?

Quoted 60 lines
---
 include/net/tc_act/tc_ct.h        |   1 +
 include/uapi/linux/tc_act/tc_ct.h |   3 +
 net/sched/act_ct.c                | 163 +++++++++++++++++++++++++++++-
 3 files changed, 165 insertions(+), 2 deletions(-)
include/net/tc_act/tc_ct.h +1 -0
index 8250d6f0a462..b24ea2d9400b 100644
@@ -10,6 +10,7 @@
 #include <net/netfilter/nf_conntrack_labels.h>
 
 struct tcf_ct_params {
+	struct nf_conntrack_helper *helper;
 	struct nf_conn *tmpl;
 	u16 zone;
 
include/uapi/linux/tc_act/tc_ct.h +3 -0
index 5fb1d7ac1027..6c5200f0ed38 100644
@@ -22,6 +22,9 @@ enum {
 	TCA_CT_NAT_PORT_MIN,	/* be16 */
 	TCA_CT_NAT_PORT_MAX,	/* be16 */
 	TCA_CT_PAD,
+	TCA_CT_HELPER_NAME,	/* string */
+	TCA_CT_HELPER_FAMILY,	/* u8 */
+	TCA_CT_HELPER_PROTO,	/* u8 */
 	__TCA_CT_MAX
 };
 
net/sched/act_ct.c +8 -0
index 193a460a9d7f..771cf72ee9e1 100644
@@ -33,6 +33,7 @@
 #include <net/netfilter/nf_conntrack_acct.h>
 #include <net/netfilter/ipv6/nf_defrag_ipv6.h>
 #include <net/netfilter/nf_conntrack_act_ct.h>
+#include <net/netfilter/nf_conntrack_seqadj.h>
 #include <uapi/linux/netfilter/nf_nat.h>
 
 static struct workqueue_struct *act_ct_wq;
@@ -832,6 +833,13 @@ static int tcf_ct_handle_fragments(struct net *net, struct sk_buff *skb,
 
 static void tcf_ct_params_free(struct tcf_ct_params *params)
 {
+	if (params->helper) {
+#if IS_ENABLED(CONFIG_NF_NAT)
+		if (params->ct_action & TCA_CT_ACT_NAT)
+			nf_nat_helper_put(params->helper);
+#endif
+		nf_conntrack_helper_put(params->helper);
+	}
 	if (params->ct_ft)
 		tcf_ct_flow_table_put(params->ct_ft);
 	if (params->tmpl)

net/sched/act_ct.c static int tcf_ct_act_nat(struct sk_buff *skb, · line 1034

Quoted 5 lines
net/sched/act_ct.c +2 -0
@@ -1022,6 +1030,69 @@ static int tcf_ct_act_nat(struct sk_buff *skb,
 #endif
 }
 
+static int tcf_ct_helper(struct sk_buff *skb, u8 family)
+{

This is very similar to ovs_ct_helper(), I'm wondering if a common helper could be factored out?

Quoted 28 lines
net/sched/act_ct.c +28 -0
+	const struct nf_conntrack_helper *helper;
+	const struct nf_conn_help *help;
+	enum ip_conntrack_info ctinfo;
+	unsigned int protoff;
+	struct nf_conn *ct;
+	u8 proto;
+	int err;
+
+	ct = nf_ct_get(skb, &ctinfo);
+	if (!ct || ctinfo == IP_CT_RELATED_REPLY)
+		return NF_ACCEPT;
+
+	help = nfct_help(ct);
+	if (!help)
+		return NF_ACCEPT;
+
+	helper = rcu_dereference(help->helper);
+	if (!helper)
+		return NF_ACCEPT;
+
+	if (helper->tuple.src.l3num != NFPROTO_UNSPEC &&
+	    helper->tuple.src.l3num != family)
+		return NF_DROP;
+
+	switch (family) {
+	case NFPROTO_IPV4:
+		protoff = ip_hdrlen(skb);
+		proto = ip_hdr(skb)->protocol;

net/sched/act_ct.c static int tcf_ct_act_nat(struct sk_buff *skb, · line 1072

Quoted 10 lines
net/sched/act_ct.c +10 -0
+		break;
+	case NFPROTO_IPV6: {
+		__be16 frag_off;
+		int ofs;
+
+		proto = ipv6_hdr(skb)->nexthdr;
+		ofs = ipv6_skip_exthdr(skb, sizeof(struct ipv6hdr), &proto, &frag_off);
+		if (ofs < 0 || (frag_off & htons(~0x7)) != 0) {
+			pr_debug("proto header not found\n");
+			return NF_DROP;

Why this is returning NF_DROP while ovs_ct_helper() returns NF_ACCEPT here?

net/sched/act_ct.c static int tcf_ct_act_nat(struct sk_buff *skb, · line 1083

Quoted 11 lines
net/sched/act_ct.c +11 -0
+		}
+		protoff = ofs;
+		break;
+	}
+	default:
+		WARN_ONCE(1, "helper invoked on non-IP family!");
+		return NF_DROP;
+	}
+
+	if (helper->tuple.dst.protonum != proto)
+		return NF_DROP;

I'm wondering if NF_DROP is appropriate here. This should be a situation similar to the above one: the current packet does not match the helper.

Thanks!

Paolo

maintainer
#2 lore raw

Re: [PATCH net-next 1/2] net: sched: fix the err path of tcf_ct_init in act_ct

In reply to Xin Long
Quoted 9 lines · Xin Long

When it returns err from tcf_ct_flow_table_get(), the param tmpl should have been freed in the cleanup. Otherwise a memory leak will occur.

While fixing this problem, this patch also makes the err path simple by calling tcf_ct_params_free(), so that it won't cause problems when more members are added into param and need freeing on the err path.

Fixes c34b961a2492 ("net/sched: act_ct: Create nf flow table per zone")

I think it's better if you re-submit this patch for -net explicitly, as it LGTM and makes sense to let it reach the affected kernel soon.

Thanks!

Paolo

author
#3 lore raw

Re: [PATCH net-next 1/2] net: sched: fix the err path of tcf_ct_init in act_ct

Quoted 9 lines · Paolo Abeni
Quoted 7 lines · Xin Long

When it returns err from tcf_ct_flow_table_get(), the param tmpl should have been freed in the cleanup. Otherwise a memory leak will occur.

While fixing this problem, this patch also makes the err path simple by calling tcf_ct_params_free(), so that it won't cause problems when more members are added into param and need freeing on the err path.

Quoted 5 lines
Quoted 2 lines
Fixes c34b961a2492 ("net/sched: act_ct: Create nf flow table per zone")

I think it's better if you re-submit this patch for -net explicitly, as it LGTM and makes sense to let it reach the affected kernel soon.

If so, I will have to wait until this patch is merged on net-next, then post the other one for net-next, right?

Quoted 6 lines

Thanks!

Paolo

author
#7 lore raw

Re: [PATCH net-next 2/2] net: sched: add helper support in act_ct

In reply to Paolo Abeni
Quoted 11 lines · Paolo Abeni
Quoted 9 lines · Xin Long

This patch is to add helper support in act_ct for OVS actions=ct(alg=xxx) offloading, which is corresponding to Commit cae3a2627520 ("openvswitch:
Allow attaching helpers to ct action") in OVS kernel part.

The difference is when adding TC actions family and proto cannot be got from the filter/match, other than helper name in tb[TCA_CT_HELPER_NAME], we also need to send the family in tb[TCA_CT_HELPER_FAMILY] and the proto in tb[TCA_CT_HELPER_PROTO] to kernel.

Quoted 7 lines
Quoted 4 lines

Note when calling helper->help() in tcf_ct_act(), the packet will be dropped if skb's family and proto do not match the helper's.

This tag is a bit out of place here, as it should belong to fixes. Do you mean 'Suggested-by' ?

This one was reported as an OVS bug, but from TC side, it's a feature. I will remove it.

Quoted 64 lines
Quoted 64 lines
---
 include/net/tc_act/tc_ct.h        |   1 +
 include/uapi/linux/tc_act/tc_ct.h |   3 +
 net/sched/act_ct.c                | 163 +++++++++++++++++++++++++++++-
 3 files changed, 165 insertions(+), 2 deletions(-)
include/net/tc_act/tc_ct.h +1 -0
index 8250d6f0a462..b24ea2d9400b 100644
@@ -10,6 +10,7 @@
 #include <net/netfilter/nf_conntrack_labels.h>
 
 struct tcf_ct_params {
+     struct nf_conntrack_helper *helper;
      struct nf_conn *tmpl;
      u16 zone;
 
include/uapi/linux/tc_act/tc_ct.h +3 -0
index 5fb1d7ac1027..6c5200f0ed38 100644
@@ -22,6 +22,9 @@ enum {
      TCA_CT_NAT_PORT_MIN,    /* be16 */
      TCA_CT_NAT_PORT_MAX,    /* be16 */
      TCA_CT_PAD,
+     TCA_CT_HELPER_NAME,     /* string */
+     TCA_CT_HELPER_FAMILY,   /* u8 */
+     TCA_CT_HELPER_PROTO,    /* u8 */
      __TCA_CT_MAX
 };
 
net/sched/act_ct.c +8 -0
index 193a460a9d7f..771cf72ee9e1 100644
@@ -33,6 +33,7 @@
 #include <net/netfilter/nf_conntrack_acct.h>
 #include <net/netfilter/ipv6/nf_defrag_ipv6.h>
 #include <net/netfilter/nf_conntrack_act_ct.h>
+#include <net/netfilter/nf_conntrack_seqadj.h>
 #include <uapi/linux/netfilter/nf_nat.h>
 
 static struct workqueue_struct *act_ct_wq;
@@ -832,6 +833,13 @@ static int tcf_ct_handle_fragments(struct net *net, struct sk_buff *skb,
 
 static void tcf_ct_params_free(struct tcf_ct_params *params)
 {
+     if (params->helper) {
+#if IS_ENABLED(CONFIG_NF_NAT)
+             if (params->ct_action & TCA_CT_ACT_NAT)
+                     nf_nat_helper_put(params->helper);
+#endif
+             nf_conntrack_helper_put(params->helper);
+     }
      if (params->ct_ft)
              tcf_ct_flow_table_put(params->ct_ft);
      if (params->tmpl)
@@ -1022,6 +1030,69 @@ static int tcf_ct_act_nat(struct sk_buff *skb,
 #endif
 }
 
Quoted 5 lines

net/sched/act_ct.c static int tcf_ct_act_nat(struct sk_buff *skb, · line 1034

Quoted 2 lines
net/sched/act_ct.c +2 -0
+static int tcf_ct_helper(struct sk_buff *skb, u8 family)
+{

This is very similar to ovs_ct_helper(), I'm wondering if a common helper could be factored out?

I wanted to, but these are two modules, I don't expect one depends another. Although this is for OVS offloading, but it can still be used independently. maybe I should move this function to nf_conntrack_helper.c in netfilter?

Quoted 55 lines
Quoted 29 lines
net/sched/act_ct.c +28 -0
+     const struct nf_conntrack_helper *helper;
+     const struct nf_conn_help *help;
+     enum ip_conntrack_info ctinfo;
+     unsigned int protoff;
+     struct nf_conn *ct;
+     u8 proto;
+     int err;
+
+     ct = nf_ct_get(skb, &ctinfo);
+     if (!ct || ctinfo == IP_CT_RELATED_REPLY)
+             return NF_ACCEPT;
+
+     help = nfct_help(ct);
+     if (!help)
+             return NF_ACCEPT;
+
+     helper = rcu_dereference(help->helper);
+     if (!helper)
+             return NF_ACCEPT;
+
+     if (helper->tuple.src.l3num != NFPROTO_UNSPEC &&
+         helper->tuple.src.l3num != family)
+             return NF_DROP;
+
+     switch (family) {
+     case NFPROTO_IPV4:
+             protoff = ip_hdrlen(skb);
+             proto = ip_hdr(skb)->protocol;

net/sched/act_ct.c static int tcf_ct_act_nat(struct sk_buff *skb, · line 1072

Quoted 10 lines
net/sched/act_ct.c +10 -0
+             break;
+     case NFPROTO_IPV6: {
+             __be16 frag_off;
+             int ofs;
+
+             proto = ipv6_hdr(skb)->nexthdr;
+             ofs = ipv6_skip_exthdr(skb, sizeof(struct ipv6hdr), &proto, &frag_off);
+             if (ofs < 0 || (frag_off & htons(~0x7)) != 0) {
+                     pr_debug("proto header not found\n");
+                     return NF_DROP;

Why this is returning NF_DROP while ovs_ct_helper() returns NF_ACCEPT here?

net/sched/act_ct.c static int tcf_ct_act_nat(struct sk_buff *skb, · line 1083

Quoted 11 lines
net/sched/act_ct.c +11 -0
+             }
+             protoff = ofs;
+             break;
+     }
+     default:
+             WARN_ONCE(1, "helper invoked on non-IP family!");
+             return NF_DROP;
+     }
+
+     if (helper->tuple.dst.protonum != proto)
+             return NF_DROP;
Quoted 3 lines

I'm wondering if NF_DROP is appropriate here. This should be a situation similar to the above one: the current packet does not match the helper.

I was thinking the packets arriving here should be matched with the helper's, this would force users do a right configuration on flower or other match. But this might be too harsh and will change to NF_ACCEPT.

Thanks!

Quoted 5 lines

Thanks!

Paolo

maintainer
#4 lore raw

Re: [PATCH net-next 1/2] net: sched: fix the err path of tcf_ct_init in act_ct

In reply to Xin Long
Quoted 10 lines · Xin Long
Quoted 9 lines · Paolo Abeni
Quoted 7 lines · Xin Long

When it returns err from tcf_ct_flow_table_get(), the param tmpl should have been freed in the cleanup. Otherwise a memory leak will occur.

While fixing this problem, this patch also makes the err path simple by calling tcf_ct_params_free(), so that it won't cause problems when more members are added into param and need freeing on the err path.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Fixes c34b961a2492 ("net/sched: act_ct: Create nf flow table per zone")

I think it's better if you re-submit this patch for -net explicitly, as it LGTM and makes sense to let it reach the affected kernel soon.

If so, I will have to wait until this patch is merged on net-next, then post the other one for net-next, right?

Yes. Note that such merge happens once per week...

Cheers,

Paolo

#8 lore raw

Re: [PATCH net-next 2/2] net: sched: add helper support in act_ct

In reply to Xin Long
Quoted 12 lines · Xin Long
Quoted 11 lines · Paolo Abeni
Quoted 9 lines · Xin Long

This patch is to add helper support in act_ct for OVS actions=ct(alg=xxx) offloading, which is corresponding to Commit cae3a2627520 ("openvswitch:
Allow attaching helpers to ct action") in OVS kernel part.

The difference is when adding TC actions family and proto cannot be got from the filter/match, other than helper name in tb[TCA_CT_HELPER_NAME], we also need to send the family in tb[TCA_CT_HELPER_FAMILY] and the proto in tb[TCA_CT_HELPER_PROTO] to kernel.

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

Note when calling helper->help() in tcf_ct_act(), the packet will be dropped if skb's family and proto do not match the helper's.

This tag is a bit out of place here, as it should belong to fixes. Do you mean 'Suggested-by' ?

This one was reported as an OVS bug, but from TC side, it's a feature.

My 2c:
- The fact that act_ct doesn't execute helpers attached to skb outside of TC (in OVS) can be considered as a bug.
- The ability to set helpers in act_ct itself is indeed a new feature.

Though it was decided to implement both things at the same time to avoid confusion around what is supported and what is not supported, especially since there will be no meaningful way to detect if the bug actually fixed in the kernel or not.

CC Eelco.

P.S. might also make sense to CC: ovs-dev on a next revision for visibility.

Best regards, Ilya Maximets.