Skip to contentSkip to conversation

Control nf flow table timeouts

PATCH In review Oz Shlomo 3 participants 7 messages Jun 3, 2021 – Jun 10, 2021

Subsystems (2)
lore
Share & tools
lore

Oz Shlomo

author
#0 lore raw

cover Control nf flow table timeouts

TCP and UDP connections may be offloaded from nf conntrack to nf flow table. Offloaded connections are aged after 30 seconds of inactivity. Once aged, ownership is returned to conntrack with a hard coded tcp/udp pickup time of 120/30 seconds, after which the connection may be deleted.

The current hard-coded pickup intervals may introduce a very aggressive aging policy. For example, offloaded tcp connections in established state will timeout from nf conntrack after just 150 seconds of inactivity, instead of 5 days. In addition, the hard-coded 30 second offload timeout period can significantly increase the hardware insertion rate requirements in some use cases.

This patchset provides the user with the ability to configure protocol specific offload timeout and pickup intervals via sysctl. The first and second patches introduce the sysctl configuration for tcp and udp protocols. The last patch modifies nf flow table aging mechanisms to use the configured time intervals.

Oz Shlomo (3):
netfilter: conntrack: Introduce tcp offload timeout configuration
netfilter: conntrack: Introduce udp offload timeout configuration
netfilter: flowtable: Set offload timeouts according to proto values

 include/net/netfilter/nf_flow_table.h   |  2 ++
 include/net/netns/conntrack.h           |  8 ++++++
 net/netfilter/nf_conntrack_proto_tcp.c  |  5 ++++
 net/netfilter/nf_conntrack_proto_udp.c  |  5 ++++
 net/netfilter/nf_conntrack_standalone.c | 46 ++++++++++++++++++++++++++++++++
 net/netfilter/nf_flow_table_core.c      | 47 ++++++++++++++++++++++++++-------
 net/netfilter/nf_flow_table_offload.c   |  4 +--
 7 files changed, 105 insertions(+), 12 deletions(-)
Signature · 1 line
1.8.3.1
author
#1 lore raw

PATCH 1/3 netfilter: conntrack: Introduce tcp offload timeout configuration

TCP connections may be offloaded from nf conntrack to nf flow table. Offloaded connections are aged after 30 seconds of inactivity. Once aged, ownership is returned to conntrack with a hard coded pickup time of 120 seconds, after which the connection may be deleted. eted. The current aging intervals may be too aggressive for some users.

Provide users with the ability to control the nf flow table offload aging and pickup time intervals via sysctl parameter as a pre-step for configuring the nf flow table GC timeout intervals.

---
 include/net/netns/conntrack.h           |  4 ++++
 net/netfilter/nf_conntrack_proto_tcp.c  |  5 +++++
 net/netfilter/nf_conntrack_standalone.c | 24 ++++++++++++++++++++++++
 3 files changed, 33 insertions(+)
include/net/netns/conntrack.h +4 -0
index ad0a95c2335e..3a391e27ec60 100644
@@ -27,6 +27,10 @@ struct nf_tcp_net {
 	u8 tcp_loose;
 	u8 tcp_be_liberal;
 	u8 tcp_max_retrans;
+#if IS_ENABLED(CONFIG_NF_FLOW_TABLE)
+	unsigned int offload_timeout;
+	unsigned int offload_pickup;
+#endif
 };
 
 enum udp_conntrack {
net/netfilter/nf_conntrack_proto_tcp.c +5 -0
index 34e22416a721..de840fc41a2e 100644
@@ -1441,6 +1441,11 @@ void nf_conntrack_tcp_init_net(struct net *net)
 	 * will be started.
 	 */
 	tn->tcp_max_retrans = 3;
+
+#if IS_ENABLED(CONFIG_NF_FLOW_TABLE)
+	tn->offload_timeout = 30 * HZ;
+	tn->offload_pickup = 120 * HZ;
+#endif
 }
 
 const struct nf_conntrack_l4proto nf_conntrack_l4proto_tcp =
net/netfilter/nf_conntrack_standalone.c +24 -0
index aaa55246d0ca..4d0c6c5e705a 100644
@@ -575,6 +575,10 @@ enum nf_ct_sysctl_index {
 	NF_SYSCTL_CT_PROTO_TIMEOUT_TCP_CLOSE,
 	NF_SYSCTL_CT_PROTO_TIMEOUT_TCP_RETRANS,
 	NF_SYSCTL_CT_PROTO_TIMEOUT_TCP_UNACK,
+#if IS_ENABLED(CONFIG_NF_FLOW_TABLE)
+	NF_SYSCTL_CT_PROTO_TIMEOUT_TCP_OFFLOAD,
+	NF_SYSCTL_CT_PROTO_TIMEOUT_TCP_OFFLOAD_PICKUP,
+#endif
 	NF_SYSCTL_CT_PROTO_TCP_LOOSE,
 	NF_SYSCTL_CT_PROTO_TCP_LIBERAL,
 	NF_SYSCTL_CT_PROTO_TCP_MAX_RETRANS,
@@ -762,6 +766,20 @@ enum nf_ct_sysctl_index {
 		.mode		= 0644,
 		.proc_handler	= proc_dointvec_jiffies,
 	},
+#if IS_ENABLED(CONFIG_NF_FLOW_TABLE)
+	[NF_SYSCTL_CT_PROTO_TIMEOUT_TCP_OFFLOAD] = {
+		.procname	= "nf_flowtable_tcp_timeout",
+		.maxlen		= sizeof(unsigned int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec_jiffies,
+	},
+	[NF_SYSCTL_CT_PROTO_TIMEOUT_TCP_OFFLOAD_PICKUP] = {
+		.procname	= "nf_flowtable_tcp_pickup",
+		.maxlen		= sizeof(unsigned int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec_jiffies,
+	},
+#endif
 	[NF_SYSCTL_CT_PROTO_TCP_LOOSE] = {
 		.procname	= "nf_conntrack_tcp_loose",
 		.maxlen		= sizeof(u8),
@@ -971,6 +989,12 @@ static void nf_conntrack_standalone_init_tcp_sysctl(struct net *net,
 	XASSIGN(LIBERAL, &tn->tcp_be_liberal);
 	XASSIGN(MAX_RETRANS, &tn->tcp_max_retrans);
 #undef XASSIGN
+
+#if IS_ENABLED(CONFIG_NF_FLOW_TABLE)
+	table[NF_SYSCTL_CT_PROTO_TIMEOUT_TCP_OFFLOAD].data = &tn->offload_timeout;
+	table[NF_SYSCTL_CT_PROTO_TIMEOUT_TCP_OFFLOAD_PICKUP].data = &tn->offload_pickup;
+#endif
+
 }
 
 static void nf_conntrack_standalone_init_sctp_sysctl(struct net *net,
Signature · 1 line
1.8.3.1
author
#2 lore raw

PATCH 2/3 netfilter: conntrack: Introduce udp offload timeout configuration

In reply to Oz Shlomo

UDP connections may be offloaded from nf conntrack to nf flow table. Offloaded connections are aged after 30 seconds of inactivity. Once aged, ownership is returned to conntrack with a hard coded pickup time of 30 seconds, after which the connection may be deleted. eted. The current aging intervals may be too aggressive for some users.

Provide users with the ability to control the nf flow table offload aging and pickup time intervals via sysctl parameter as a pre-step for configuring the nf flow table GC timeout intervals.

---
 include/net/netns/conntrack.h           |  4 ++++
 net/netfilter/nf_conntrack_proto_udp.c  |  5 +++++
 net/netfilter/nf_conntrack_standalone.c | 22 ++++++++++++++++++++++
 3 files changed, 31 insertions(+)
include/net/netns/conntrack.h +4 -0
index 3a391e27ec60..c3094b83a525 100644
@@ -41,6 +41,10 @@ enum udp_conntrack {
 
 struct nf_udp_net {
 	unsigned int timeouts[UDP_CT_MAX];
+#if IS_ENABLED(CONFIG_NF_FLOW_TABLE)
+	unsigned int offload_timeout;
+	unsigned int offload_pickup;
+#endif
 };
 
 struct nf_icmp_net {
net/netfilter/nf_conntrack_proto_udp.c +5 -0
index af402f458ee0..68911fcaa0f1 100644
@@ -270,6 +270,11 @@ void nf_conntrack_udp_init_net(struct net *net)
 
 	for (i = 0; i < UDP_CT_MAX; i++)
 		un->timeouts[i] = udp_timeouts[i];
+
+#if IS_ENABLED(CONFIG_NF_FLOW_TABLE)
+	un->offload_timeout = 30 * HZ;
+	un->offload_pickup = 30 * HZ;
+#endif
 }
 
 const struct nf_conntrack_l4proto nf_conntrack_l4proto_udp =
net/netfilter/nf_conntrack_standalone.c +22 -0
index 4d0c6c5e705a..7331d8316af1 100644
@@ -584,6 +584,10 @@ enum nf_ct_sysctl_index {
 	NF_SYSCTL_CT_PROTO_TCP_MAX_RETRANS,
 	NF_SYSCTL_CT_PROTO_TIMEOUT_UDP,
 	NF_SYSCTL_CT_PROTO_TIMEOUT_UDP_STREAM,
+#if IS_ENABLED(CONFIG_NF_FLOW_TABLE)
+	NF_SYSCTL_CT_PROTO_TIMEOUT_UDP_OFFLOAD,
+	NF_SYSCTL_CT_PROTO_TIMEOUT_UDP_OFFLOAD_PICKUP,
+#endif
 	NF_SYSCTL_CT_PROTO_TIMEOUT_ICMP,
 	NF_SYSCTL_CT_PROTO_TIMEOUT_ICMPV6,
 #ifdef CONFIG_NF_CT_PROTO_SCTP
@@ -814,6 +818,20 @@ enum nf_ct_sysctl_index {
 		.mode		= 0644,
 		.proc_handler	= proc_dointvec_jiffies,
 	},
+#if IS_ENABLED(CONFIG_NFT_FLOW_OFFLOAD)
+	[NF_SYSCTL_CT_PROTO_TIMEOUT_UDP_OFFLOAD] = {
+		.procname	= "nf_flowtable_udp_timeout",
+		.maxlen		= sizeof(unsigned int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec_jiffies,
+	},
+	[NF_SYSCTL_CT_PROTO_TIMEOUT_UDP_OFFLOAD_PICKUP] = {
+		.procname	= "nf_flowtable_udp_pickup",
+		.maxlen		= sizeof(unsigned int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec_jiffies,
+	},
+#endif
 	[NF_SYSCTL_CT_PROTO_TIMEOUT_ICMP] = {
 		.procname	= "nf_conntrack_icmp_timeout",
 		.maxlen		= sizeof(unsigned int),
@@ -1083,6 +1101,10 @@ static int nf_conntrack_standalone_init_sysctl(struct net *net)
 	table[NF_SYSCTL_CT_PROTO_TIMEOUT_ICMPV6].data = &nf_icmpv6_pernet(net)->timeout;
 	table[NF_SYSCTL_CT_PROTO_TIMEOUT_UDP].data = &un->timeouts[UDP_CT_UNREPLIED];
 	table[NF_SYSCTL_CT_PROTO_TIMEOUT_UDP_STREAM].data = &un->timeouts[UDP_CT_REPLIED];
+#if IS_ENABLED(CONFIG_NF_FLOW_TABLE)
+	table[NF_SYSCTL_CT_PROTO_TIMEOUT_UDP_OFFLOAD].data = &un->offload_timeout;
+	table[NF_SYSCTL_CT_PROTO_TIMEOUT_UDP_OFFLOAD_PICKUP].data = &un->offload_pickup;
+#endif
 
 	nf_conntrack_standalone_init_tcp_sysctl(net, table);
 	nf_conntrack_standalone_init_sctp_sysctl(net, table);
Signature · 1 line
1.8.3.1
author
#3 lore raw

PATCH 3/3 netfilter: flowtable: Set offload timeouts according to proto values

In reply to Oz Shlomo

Currently the aging period for tcp/udp connections is hard coded to 30 seconds. Aged tcp/udp connections configure a hard coded 120/30 seconds pickup timeout for conntrack.
This configuration may be too aggressive or permissive for some users.

Dynamically configure the nf flow table GC timeout intervals according to the user defined values.

---
 include/net/netfilter/nf_flow_table.h |  2 ++
 net/netfilter/nf_flow_table_core.c    | 47 +++++++++++++++++++++++++++--------
 net/netfilter/nf_flow_table_offload.c |  4 +--
 3 files changed, 41 insertions(+), 12 deletions(-)
include/net/netfilter/nf_flow_table.h +2 -0
index 48ef7460ff30..a3647fadf1cc 100644
@@ -177,6 +177,8 @@ struct flow_offload {
 #define NF_FLOW_TIMEOUT (30 * HZ)
 #define nf_flowtable_time_stamp	(u32)jiffies
 
+unsigned long flow_offload_get_timeout(struct flow_offload *flow);
+
 static inline __s32 nf_flow_timeout_delta(unsigned int timeout)
 {
 	return (__s32)(timeout - nf_flowtable_time_stamp);
net/netfilter/nf_flow_table_core.c +37 -10

79 lines not on this page · raw message

net/netfilter/nf_flow_table_offload.c +2 -2
index 528b2f172684..f92006cec94c 100644
@@ -937,7 +937,7 @@ static void flow_offload_work_stats(struct flow_offload_work *offload)
 
 	lastused = max_t(u64, stats[0].lastused, stats[1].lastused);
 	offload->flow->timeout = max_t(u64, offload->flow->timeout,
-				       lastused + NF_FLOW_TIMEOUT);
+				       lastused + flow_offload_get_timeout(offload->flow));
 
 	if (offload->flowtable->flags & NF_FLOWTABLE_COUNTER) {
 		if (stats[0].pkts)
@@ -1041,7 +1041,7 @@ void nf_flow_offload_stats(struct nf_flowtable *flowtable,
 	__s32 delta;
 
 	delta = nf_flow_timeout_delta(flow->timeout);
-	if ((delta >= (9 * NF_FLOW_TIMEOUT) / 10))
+	if ((delta >= (9 * flow_offload_get_timeout(flow)) / 10))
 		return;
 
 	offload = nf_flow_offload_work_alloc(flowtable, flow, FLOW_CLS_STATS);
Signature · 1 line
1.8.3.1
#4 lore raw
In reply to Oz Shlomo
Quoted 12 lines · Oz Shlomo

TCP and UDP connections may be offloaded from nf conntrack to nf flow table. Offloaded connections are aged after 30 seconds of inactivity. Once aged, ownership is returned to conntrack with a hard coded tcp/udp pickup time of 120/30 seconds, after which the connection may be deleted.

The current hard-coded pickup intervals may introduce a very aggressive aging policy. For example, offloaded tcp connections in established state will timeout from nf conntrack after just 150 seconds of inactivity, instead of 5 days. In addition, the hard-coded 30 second offload timeout period can significantly increase the hardware insertion rate requirements in some use cases.

Quoted 5 lines

This patchset provides the user with the ability to configure protocol specific offload timeout and pickup intervals via sysctl. The first and second patches introduce the sysctl configuration for tcp and udp protocols. The last patch modifies nf flow table aging mechanisms to use the configured time intervals.

Series applied, thanks.

#5 lore raw
Quoted 13 lines · Pablo Neira Ayuso
Quoted 12 lines · Oz Shlomo

TCP and UDP connections may be offloaded from nf conntrack to nf flow table. Offloaded connections are aged after 30 seconds of inactivity. Once aged, ownership is returned to conntrack with a hard coded tcp/udp pickup time of 120/30 seconds, after which the connection may be deleted.

The current hard-coded pickup intervals may introduce a very aggressive aging policy. For example, offloaded tcp connections in established state will timeout from nf conntrack after just 150 seconds of inactivity, instead of 5 days. In addition, the hard-coded 30 second offload timeout period can significantly increase the hardware insertion rate requirements in some use cases.

Quoted 7 lines
Quoted 5 lines

This patchset provides the user with the ability to configure protocol specific offload timeout and pickup intervals via sysctl. The first and second patches introduce the sysctl configuration for tcp and udp protocols. The last patch modifies nf flow table aging mechanisms to use the configured time intervals.

Series applied, thanks.

Patchset missed a description of the new sysctl entries in nf_conntrack-sysctl.rst, btw.

Marcelo

author
#6 lore raw
Quoted 20 lines · Marcelo Ricardo Leitner
Quoted 19 lines · Pablo Neira Ayuso
Quoted 17 lines · Oz Shlomo

TCP and UDP connections may be offloaded from nf conntrack to nf flow table. Offloaded connections are aged after 30 seconds of inactivity. Once aged, ownership is returned to conntrack with a hard coded tcp/udp pickup time of 120/30 seconds, after which the connection may be deleted.

The current hard-coded pickup intervals may introduce a very aggressive aging policy. For example, offloaded tcp connections in established state will timeout from nf conntrack after just 150 seconds of inactivity, instead of 5 days. In addition, the hard-coded 30 second offload timeout period can significantly increase the hardware insertion rate requirements in some use cases.

This patchset provides the user with the ability to configure protocol specific offload timeout and pickup intervals via sysctl. The first and second patches introduce the sysctl configuration for tcp and udp protocols. The last patch modifies nf flow table aging mechanisms to use the configured time intervals.

Quoted 4 lines
Quoted 1 line

Series applied, thanks.

Patchset missed a description of the new sysctl entries in nf_conntrack-sysctl.rst, btw.

I will update the documentation,
Thanks

Quoted 3 lines

Marcelo