Skip to contentSkip to conversation

net: ctnetlink: support filtering by zone

PATCH In review Felix Huettner 3 participants 9 messages Nov 27, 2023 – Feb 5, 2024

Subsystems (4)
Networking Network drivers Kselftest netfilter
lore
Share & tools
lore

Felix Huettner

author
#0 lore raw

PATCH v2 1/1 net: ctnetlink: support filtering by zone

conntrack zones are heavily used by tools like openvswitch to run multiple virtual "routers" on a single machine. In this context each conntrack zone matches to a single router, thereby preventing overlapping IPs from becoming issues.
In these systems it is common to operate on all conntrack entries of a given zone, e.g. to delete them when a router is deleted. Previously this required these tools to dump the full conntrack table and filter out the relevant entries in userspace potentially causing performance issues.

To do this we reuse the existing CTA_ZONE attribute. This was previous parsed but not used during dump and flush requests. Now if CTA_ZONE is set we filter these operations based on the provided zone. However this means that users that previously passed CTA_ZONE will experience a difference in functionality.

Alternatively CTA_FILTER could have been used for the same functionality. However it is not yet supported during flush requests and is only available when using AF_INET or AF_INET6.

Co-developed-by Luca Czesla <[email protected]>
Signed-off-by Luca Czesla <[email protected]>
Co-developed-by Max Lamprecht <[email protected]>
Signed-off-by Max Lamprecht <[email protected]>
Signed-off-by Felix Huettner <[email protected]>
---
 net/netfilter/nf_conntrack_netlink.c          |  15 +-
 tools/testing/selftests/netfilter/.gitignore  |   2 +
 tools/testing/selftests/netfilter/Makefile    |   3 +-
 .../netfilter/conntrack_dump_flush.c          | 430 ++++++++++++++++++
 4 files changed, 444 insertions(+), 6 deletions(-)
 create mode 100644 tools/testing/selftests/netfilter/conntrack_dump_flush.c
net/netfilter/nf_conntrack_netlink.c +10 -5
index fb0ae15e96df..4e9133f61251 100644
@@ -992,13 +992,13 @@ ctnetlink_alloc_filter(const struct nlattr * const cda[], u8 family)
 	if (err)
 		goto err_filter;
 
+	err = ctnetlink_parse_zone(cda[CTA_ZONE], &filter->zone);
+	if (err < 0)
+		goto err_filter;
+
 	if (!cda[CTA_FILTER])
 		return filter;
 
-	err = ctnetlink_parse_zone(cda[CTA_ZONE], &filter->zone);
-	if (err < 0)
-		goto err_filter;
-
 	err = ctnetlink_parse_filter(cda[CTA_FILTER], filter);
 	if (err < 0)
 		goto err_filter;

@@ -1043,7 +1043,8 @@ ctnetlink_alloc_filter(const struct nlattr * const cda[], u8 family)
 
 static bool ctnetlink_needs_filter(u8 family, const struct nlattr * const *cda)
 {
-	return family || cda[CTA_MARK] || cda[CTA_FILTER] || cda[CTA_STATUS];
+	return family || cda[CTA_MARK] || cda[CTA_FILTER] || cda[CTA_STATUS] ||
+		cda[CTA_ZONE];
 }
 
 static int ctnetlink_start(struct netlink_callback *cb)

@@ -1148,6 +1149,10 @@ static int ctnetlink_filter_match(struct nf_conn *ct, void *data)
 	if (filter->family && nf_ct_l3num(ct) != filter->family)
 		goto ignore_entry;
 
+	if (filter->zone.id != NF_CT_DEFAULT_ZONE_ID &&
+	    !nf_ct_zone_equal_any(ct, &filter->zone))
+		goto ignore_entry;
+
 	if (filter->orig_flags) {
 		tuple = nf_ct_tuple(ct, IP_CT_DIR_ORIGINAL);
 		if (!ctnetlink_filter_match_tuple(&filter->orig, tuple,
tools/testing/selftests/netfilter/.gitignore +2 -0
index 4b2928e1c19d..c2229b3e40d4 100644
@@ -2,3 +2,5 @@
 nf-queue
 connect_close
 audit_logread
+conntrack_dump_flush
+sctp_collision
tools/testing/selftests/netfilter/Makefile +2 -1
index bced422b78f7..db27153eb4a0 100644
@@ -14,6 +14,7 @@ HOSTPKG_CONFIG := pkg-config
 CFLAGS += $(shell $(HOSTPKG_CONFIG) --cflags libmnl 2>/dev/null)
 LDLIBS += $(shell $(HOSTPKG_CONFIG) --libs libmnl 2>/dev/null || echo -lmnl)
 
-TEST_GEN_FILES =  nf-queue connect_close audit_logread sctp_collision
+TEST_GEN_FILES =  nf-queue connect_close audit_logread sctp_collision \
+	conntrack_dump_flush
 
 include ../lib.mk
tools/testing/selftests/netfilter/conntrack_dump_flush.c new file +430 -0

430 lines not on this page · raw message

base-commit: aadbd27f9674d7f5457331fe0248b370d5c1f25d

Signature · 1 line
2.42.1
maintainer
#1 lore raw

Hi,

Quoted 15 lines · Felix Huettner

conntrack zones are heavily used by tools like openvswitch to run multiple virtual "routers" on a single machine. In this context each conntrack zone matches to a single router, thereby preventing overlapping IPs from becoming issues.
In these systems it is common to operate on all conntrack entries of a given zone, e.g. to delete them when a router is deleted. Previously this required these tools to dump the full conntrack table and filter out the relevant entries in userspace potentially causing performance issues.

To do this we reuse the existing CTA_ZONE attribute. This was previous parsed but not used during dump and flush requests. Now if CTA_ZONE is set we filter these operations based on the provided zone. However this means that users that previously passed CTA_ZONE will experience a difference in functionality.

Quoted 3 lines

Alternatively CTA_FILTER could have been used for the same functionality. However it is not yet supported during flush requests and is only available when using AF_INET or AF_INET6.

You mean, AF_UNSPEC cannot be specified in CTA_FILTER?

Please, extend libnetfilter_conntrack to support for this feature, there is a filter API that can be used for this purpose.

Thanks.

author
#2 lore raw

Hi,

Quoted 18 lines · Pablo Neira Ayuso

Hi,

Quoted 15 lines · Felix Huettner

conntrack zones are heavily used by tools like openvswitch to run multiple virtual "routers" on a single machine. In this context each conntrack zone matches to a single router, thereby preventing overlapping IPs from becoming issues.
In these systems it is common to operate on all conntrack entries of a given zone, e.g. to delete them when a router is deleted. Previously this required these tools to dump the full conntrack table and filter out the relevant entries in userspace potentially causing performance issues.

To do this we reuse the existing CTA_ZONE attribute. This was previous parsed but not used during dump and flush requests. Now if CTA_ZONE is set we filter these operations based on the provided zone. However this means that users that previously passed CTA_ZONE will experience a difference in functionality.

Quoted 5 lines
Quoted 3 lines

Alternatively CTA_FILTER could have been used for the same functionality. However it is not yet supported during flush requests and is only available when using AF_INET or AF_INET6.

You mean, AF_UNSPEC cannot be specified in CTA_FILTER?

Currently ctnetlink_parse_tuple_filter returns EOPNOTSUPP if e.g. AF_UNSPEC would be passed. This could probably be changed to only require this for the CTA_IP_SRC and CTA_IP_DST filter flags. However i am unsure about assumptions in other pieces of code that these fields exist.

Also CTA_FILTER seems to be currently not supported when deleting conntrack entries.

Quoted 3 lines

Please, extend libnetfilter_conntrack to support for this feature, there is a filter API that can be used for this purpose.

I will do that and post it here (or in the next version) once i am done.

Quoted 2 lines

Thanks.

Thanks for the fast feedback

maintainer
#4 lore raw
In reply to Felix Huettner
Quoted 15 lines · Felix Huettner

conntrack zones are heavily used by tools like openvswitch to run multiple virtual "routers" on a single machine. In this context each conntrack zone matches to a single router, thereby preventing overlapping IPs from becoming issues.
In these systems it is common to operate on all conntrack entries of a given zone, e.g. to delete them when a router is deleted. Previously this required these tools to dump the full conntrack table and filter out the relevant entries in userspace potentially causing performance issues.

To do this we reuse the existing CTA_ZONE attribute. This was previous parsed but not used during dump and flush requests. Now if CTA_ZONE is set we filter these operations based on the provided zone. However this means that users that previously passed CTA_ZONE will experience a difference in functionality.

Quoted 3 lines

Alternatively CTA_FILTER could have been used for the same functionality. However it is not yet supported during flush requests and is only available when using AF_INET or AF_INET6.

For the record, this is applied to nf-next.

#5 lore raw
Quoted 16 lines · Pablo Neira Ayuso
Quoted 15 lines · Felix Huettner

conntrack zones are heavily used by tools like openvswitch to run multiple virtual "routers" on a single machine. In this context each conntrack zone matches to a single router, thereby preventing overlapping IPs from becoming issues.
In these systems it is common to operate on all conntrack entries of a given zone, e.g. to delete them when a router is deleted. Previously this required these tools to dump the full conntrack table and filter out the relevant entries in userspace potentially causing performance issues.

To do this we reuse the existing CTA_ZONE attribute. This was previous parsed but not used during dump and flush requests. Now if CTA_ZONE is set we filter these operations based on the provided zone. However this means that users that previously passed CTA_ZONE will experience a difference in functionality.

Quoted 5 lines
Quoted 3 lines

Alternatively CTA_FILTER could have been used for the same functionality. However it is not yet supported during flush requests and is only available when using AF_INET or AF_INET6.

For the record, this is applied to nf-next.

Hi, Felix and Pablo.

I was looking through the code and the following part is bothering me:

diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c index fb0ae15e96df..4e9133f61251 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -1148,6 +1149,10 @@ static int ctnetlink_filter_match(struct nf_conn *ct, void *data)

         if (filter->family && nf_ct_l3num(ct) != filter->family)
                 goto ignore_entry;

+ if (filter->zone.id != NF_CT_DEFAULT_ZONE_ID &&
+ !nf_ct_zone_equal_any(ct, &filter->zone))
+ goto ignore_entry;
+

         if (filter->orig_flags) {
                 tuple = nf_ct_tuple(ct, IP_CT_DIR_ORIGINAL);
                 if (!ctnetlink_filter_match_tuple(&filter->orig, tuple,

If I'm reading that right, the default zone is always flushed, even if the user requested to flush a different zone. I.e. the entry is never ignored for a default zone. Is that correct or am I reading that wrong?

If my observation is correct, then I don't think this functionality can actually be used by applications as it does something unexpected.

Best regards, Ilya Maximets.

maintainer
#6 lore raw
Quoted 43 lines · Ilya Maximets
Quoted 16 lines · Pablo Neira Ayuso
Quoted 15 lines · Felix Huettner

conntrack zones are heavily used by tools like openvswitch to run multiple virtual "routers" on a single machine. In this context each conntrack zone matches to a single router, thereby preventing overlapping IPs from becoming issues.
In these systems it is common to operate on all conntrack entries of a given zone, e.g. to delete them when a router is deleted. Previously this required these tools to dump the full conntrack table and filter out the relevant entries in userspace potentially causing performance issues.

To do this we reuse the existing CTA_ZONE attribute. This was previous parsed but not used during dump and flush requests. Now if CTA_ZONE is set we filter these operations based on the provided zone. However this means that users that previously passed CTA_ZONE will experience a difference in functionality.

Quoted 5 lines
Quoted 3 lines

Alternatively CTA_FILTER could have been used for the same functionality. However it is not yet supported during flush requests and is only available when using AF_INET or AF_INET6.

For the record, this is applied to nf-next.

Hi, Felix and Pablo.

I was looking through the code and the following part is bothering me:

diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c index fb0ae15e96df..4e9133f61251 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -1148,6 +1149,10 @@ static int ctnetlink_filter_match(struct nf_conn *ct, void *data)

         if (filter->family && nf_ct_l3num(ct) != filter->family)
                 goto ignore_entry;

+ if (filter->zone.id != NF_CT_DEFAULT_ZONE_ID &&
+ !nf_ct_zone_equal_any(ct, &filter->zone))
+ goto ignore_entry;
+

         if (filter->orig_flags) {
                 tuple = nf_ct_tuple(ct, IP_CT_DIR_ORIGINAL);
                 if (!ctnetlink_filter_match_tuple(&filter->orig, tuple,
Quoted 6 lines

If I'm reading that right, the default zone is always flushed, even if the user requested to flush a different zone. I.e. the entry is never ignored for a default zone. Is that correct or am I reading that wrong?

If my observation is correct, then I don't think this functionality can actually be used by applications as it does something unexpected.

This needs a fix, the NF_CT_DEFAULT_ZONE_ID is used as a marker to indicate if the filtering by zone needs to happen or not.

I'd suggest to add a boolean flag that specifies that zone filtering is set on.

author
#7 lore raw
Quoted 48 lines · Pablo Neira Ayuso
Quoted 47 lines · Ilya Maximets
Quoted 16 lines · Pablo Neira Ayuso
Quoted 15 lines · Felix Huettner

conntrack zones are heavily used by tools like openvswitch to run multiple virtual "routers" on a single machine. In this context each conntrack zone matches to a single router, thereby preventing overlapping IPs from becoming issues.
In these systems it is common to operate on all conntrack entries of a given zone, e.g. to delete them when a router is deleted. Previously this required these tools to dump the full conntrack table and filter out the relevant entries in userspace potentially causing performance issues.

To do this we reuse the existing CTA_ZONE attribute. This was previous parsed but not used during dump and flush requests. Now if CTA_ZONE is set we filter these operations based on the provided zone. However this means that users that previously passed CTA_ZONE will experience a difference in functionality.

Quoted 5 lines
Quoted 3 lines

Alternatively CTA_FILTER could have been used for the same functionality. However it is not yet supported during flush requests and is only available when using AF_INET or AF_INET6.

For the record, this is applied to nf-next.

Hi, Felix and Pablo.

I was looking through the code and the following part is bothering me:

diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c index fb0ae15e96df..4e9133f61251 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -1148,6 +1149,10 @@ static int ctnetlink_filter_match(struct nf_conn *ct, void *data)

         if (filter->family && nf_ct_l3num(ct) != filter->family)
                 goto ignore_entry;

+ if (filter->zone.id != NF_CT_DEFAULT_ZONE_ID &&
+ !nf_ct_zone_equal_any(ct, &filter->zone))
+ goto ignore_entry;
+

         if (filter->orig_flags) {
                 tuple = nf_ct_tuple(ct, IP_CT_DIR_ORIGINAL);
                 if (!ctnetlink_filter_match_tuple(&filter->orig, tuple,

If I'm reading that right, the default zone is always flushed, even if the user requested to flush a different zone. I.e. the entry is never ignored for a default zone. Is that correct or am I reading that wrong?

Quoted 8 lines
Quoted 2 lines

If my observation is correct, then I don't think this functionality can actually be used by applications as it does something unexpected.

This needs a fix, the NF_CT_DEFAULT_ZONE_ID is used as a marker to indicate if the filtering by zone needs to happen or not.

I'd suggest to add a boolean flag that specifies that zone filtering is set on.

Hi Pablo and Ilya,

thanks for finding that.
i will build a fix for that.

author
#8 lore raw
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines

Hi, Felix and Pablo.

I was looking through the code and the following part is bothering me:

diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c index fb0ae15e96df..4e9133f61251 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -1148,6 +1149,10 @@ static int ctnetlink_filter_match(struct nf_conn *ct, void *data)

         if (filter->family && nf_ct_l3num(ct) != filter->family)
                 goto ignore_entry;

+ if (filter->zone.id != NF_CT_DEFAULT_ZONE_ID &&
+ !nf_ct_zone_equal_any(ct, &filter->zone))
+ goto ignore_entry;
+

         if (filter->orig_flags) {
                 tuple = nf_ct_tuple(ct, IP_CT_DIR_ORIGINAL);
                 if (!ctnetlink_filter_match_tuple(&filter->orig, tuple,

If I'm reading that right, the default zone is always flushed, even if the user requested to flush a different zone. I.e. the entry is never ignored for a default zone. Is that correct or am I reading that wrong?

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

If my observation is correct, then I don't think this functionality can actually be used by applications as it does something unexpected.

This needs a fix, the NF_CT_DEFAULT_ZONE_ID is used as a marker to indicate if the filtering by zone needs to happen or not.

I'd suggest to add a boolean flag that specifies that zone filtering is set on.

Hi everyone,

i build a patch along with tests for the mentioned issue. However the issue was rather that the filter would be skipped if we wanted to flush zone 0, which caused all zones to be flushed.

The fix is available here: https://lore.kernel.org/netdev/[email protected]/T/#u