Some fixes for nested sets
Phil Sutter
reviewercover Some fixes for nested sets
This series fixes a number of issues with nested anonymous sets.
Phil Sutter (2):
evaluate: set: Allow for set elems to be sets
evaluate: set: Fix nested set merge size adjustment
src/evaluate.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
Signature · 1 line
2.11.0
Phil Sutter
reviewerPATCH 2/2 evaluate: set: Fix nested set merge size adjustment
When merging a nested set into the parent one, we are actually replacing one item with the items of the nested set. Therefore we have to remove the replaced item from size.
The respective bug isn't as easy to trigger, since the size field seems to be relevant only when set elements are ranges which are checked for overlaps. Here's an example of how to trigger it:
| add rule ip saddr { { 1.1.1.0/24, 3.3.3.0/24 }, 2.2.2.0/24 }
--- src/evaluate.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
src/evaluate.c +1 -1
@@ -1149,7 +1149,7 @@ static int expr_evaluate_set(struct eval_ctx *ctx, struct expr **expr)/* Merge recursive set definitions */ list_splice_tail_init (&i->expressions, &i->list);list_del (&i->list); - set->size += i->size; + set->size += i->size -1 ; set->set_flags |= i->set_flags;expr_free (i); }else if (!expr_is_singleton (i))
Signature · 1 line
2.11.0
Phil Sutter
reviewerPATCH 1/2 evaluate: set: Allow for set elems to be sets
In reply to Phil SutterRecursive use of sets is handled in parts by parser_bison.y, which has a rule for inline unnamed sets in set_list_member_expr, e.g. like this:
| add rule ip saddr { { 1.1.1.0, 2.2.2.0 }, 3.3.3.0 }
Yet there is another way to have an unnamed set inline, which is via define:
| define myset = {
| 1.1.1.0,
| 2.2.2.0,
| }
| add rule ip saddr { $myset, 3.3.3.0 }This didn't work because the inline set comes in as EXPR_SET_ELEM with EXPR_SET as key. This patch handles that case by replacing the former by a copy of the latter, so the following set list merging can take place.
--- src/evaluate.c | 9 +++++++++ 1 file changed, 9 insertions(+)
src/evaluate.c +9 -0
@@ -1132,6 +1132,15 @@ static int expr_evaluate_set(struct eval_ctx *ctx, struct expr **expr)return expr_error (ctx->msgs, i,"Set reference cannot be part of another set" ); +if (i->ops->type ==EXPR_SET_ELEM && + i->key->ops->type ==EXPR_SET ) { +struct expr *new =expr_clone (i->key); + +list_replace (&i->list, &new->list); +expr_free (i); + i = new; + } +if (!expr_is_constant (i))return expr_error (ctx->msgs, i,"Set member is not constant" );
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerQuoted 1 line · Phil Sutter
This series fixes a number of issues with nested anonymous sets.
Series applied.
Phil, I'd appreciate if you can follow up with a patch that adds tests/shell to cover this fix. Thanks.
Phil Sutter
reviewerPATCH 1/3 tests: Add test cases for nested anonymous sets
This makes sure nesting of anonymous sets works regardless of whether defines are used or not. As a side-effect, it also checks that overlap checking when IP address prefixes are used, works.
--- tests/py/ip/sets.t | 4 +++ tests/py/ip/sets.t.payload.inet | 19 +++++++++++++ tests/py/ip/sets.t.payload.ip | 15 +++++++++++ tests/py/ip/sets.t.payload.netdev | 19 +++++++++++++ tests/shell/testcases/sets/0021nesting_0 | 46 ++++++++++++++++++++++++++++++++ 5 files changed, 103 insertions(+) create mode 100755 tests/shell/testcases/sets/0021nesting_0
tests/py/ip/sets.t +4 -0
@@ -46,3 +46,7 @@ ip saddr != @set33 drop;fail ?set4 192.168.2.0/24;ok ?set4 192.168.1.1;fail ?set4 192.168.3.0/24;ok + +# test nested anonymous sets +ip saddr { { 1.1.1.0, 3.3.3.0 }, 2.2.2.0 };ok;ip saddr { 1.1.1.0, 2.2.2.0, 3.3.3.0 } +ip saddr { { 1.1.1.0/24, 3.3.3.0/24 }, 2.2.2.0/24 };ok;ip saddr { 1.1.1.0/24, 2.2.2.0/24, 3.3.3.0/24 }
tests/py/ip/sets.t.payload.inet +19 -0
@@ -30,3 +30,22 @@ inet test-inet input [ lookup reg 1 set set2 0x1 ] [ immediate reg 0 drop ] +# ip saddr { { 1.1.1.0, 3.3.3.0 }, 2.2.2.0 } +__set%d t 3 +__set%d t 0 + element 00010101 : 0 [end] element 00030303 : 0 [end] element 00020202 : 0 [end] +inet test-inet input + [ meta load nfproto => reg 1 ] + [ cmp eq reg 1 0x00000002 ] + [ payload load 4b @ network header + 12 => reg 1 ] + [ lookup reg 1 set __set%d ] + +# ip saddr { { 1.1.1.0/24, 3.3.3.0/24 }, 2.2.2.0/24 } +__set%d t 7 +__set%d t 0 + element 00000000 : 1 [end] element 00010101 : 0 [end] element 00020101 : 1 [end] element 00020202 : 0 [end] element 00030202 : 1 [end] element 00030303 : 0 [end] element 00040303 : 1 [end] +inet test-inet input + [ meta load nfproto => reg 1 ] + [ cmp eq reg 1 0x00000002 ] + [ payload load 4b @ network header + 12 => reg 1 ] + [ lookup reg 1 set __set%d ]
tests/py/ip/sets.t.payload.ip +15 -0
@@ -22,3 +22,18 @@ ip test-ip4 input [ lookup reg 1 set set2 0x1 ] [ immediate reg 0 drop ] +# ip saddr { { 1.1.1.0, 3.3.3.0 }, 2.2.2.0 } +__set%d test-ip4 3 +__set%d test-ip4 0 + element 00010101 : 0 [end] element 00030303 : 0 [end] element 00020202 : 0 [end] +ip test-ip4 input + [ payload load 4b @ network header + 12 => reg 1 ] + [ lookup reg 1 set __set%d ] + +# ip saddr { { 1.1.1.0/24, 3.3.3.0/24 }, 2.2.2.0/24 } +__set%d test-ip4 7 +__set%d test-ip4 0 + element 00000000 : 1 [end] element 00010101 : 0 [end] element 00020101 : 1 [end] element 00020202 : 0 [end] element 00030202 : 1 [end] element 00030303 : 0 [end] element 00040303 : 1 [end] +ip test-ip4 input + [ payload load 4b @ network header + 12 => reg 1 ] + [ lookup reg 1 set __set%d ]
tests/py/ip/sets.t.payload.netdev +19 -0
@@ -30,3 +30,22 @@ netdev test-netdev ingress [ lookup reg 1 set set2 0x1 ] [ immediate reg 0 drop ] +# ip saddr { { 1.1.1.0, 3.3.3.0 }, 2.2.2.0 } +__set%d test-netdev 3 +__set%d test-netdev 0 + element 00010101 : 0 [end] element 00030303 : 0 [end] element 00020202 : 0 [end] +netdev test-netdev ingress + [ meta load protocol => reg 1 ] + [ cmp eq reg 1 0x00000008 ] + [ payload load 4b @ network header + 12 => reg 1 ] + [ lookup reg 1 set __set%d ] + +# ip saddr { { 1.1.1.0/24, 3.3.3.0/24 }, 2.2.2.0/24 } +__set%d test-netdev 7 +__set%d test-netdev 0 + element 00000000 : 1 [end] element 00010101 : 0 [end] element 00020101 : 1 [end] element 00020202 : 0 [end] element 00030202 : 1 [end] element 00030303 : 0 [end] element 00040303 : 1 [end] +netdev test-netdev ingress + [ meta load protocol => reg 1 ] + [ cmp eq reg 1 0x00000008 ] + [ payload load 4b @ network header + 12 => reg 1 ] + [ lookup reg 1 set __set%d ]
tests/shell/testcases/sets/0021nesting_0 new file +46 -0
46 lines not on this page · raw message
Signature · 1 line
2.11.0
Phil Sutter
reviewerPATCH 2/3 tests: shell: netns/0003many_0: Fix cleanup after error
If rule set applying failed, this would leave a stray netns in place.
Interestingly, this situation led to other, seemingly unrelated testcases to fail with spurious errors, e.g. sets/0015rulesetflush_0:
| $ ./run-tests.sh testcases/sets/0015rulesetflush_0
| I: using nft binary ../../src/nft
|
| W: [FAILED] testcases/sets/0015rulesetflush_0: expected 0 but got 1
| /tmp/tmp.BY7cuUYL8f:5:1-2: Error: Could not process rule: Operation not supported
| table inet filter {
| ^^
| /tmp/tmp.BY7cuUYL8f:9:1-2: Error: Could not process rule: No such file or directory
| add element inet filter blacklist_v4 {
| ^^
| /tmp/tmp.BY7cuUYL8f:5:1-2: Error: Could not process rule: Operation not supported
| table inet filter {
| ^^
| /tmp/tmp.BY7cuUYL8f:9:1-2: Error: Could not process rule: No such file or directory
| add element inet filter blacklist_v4 {
| ^^
|
| I: results: [OK] 0 [FAILED] 1 [TOTAL] 1
|
| $ ip netns list
| 1_0003many_0
| $ ip netns del 1_0003many_0
|
| $ ./run-tests.sh testcases/sets/0015rulesetflush_0
| I: using nft binary ../../src/nft
|
| I: [OK] testcases/sets/0015rulesetflush_0
|
| I: results: [OK] 1 [FAILED] 0 [TOTAL] 1--- tests/shell/testcases/netns/0003many_0 | 1 + 1 file changed, 1 insertion(+)
tests/shell/testcases/netns/0003many_0 +1 -0
@@ -109,6 +109,7 @@ function test_netns() echo "E: ruleset in netns $NETNS_NAME differs from the loaded" >&2 DIFF="$(which diff)" [ -x $DIFF ] && $DIFF -u <(echo "$RULESET") <(echo "$KERNEL_RULESET") + $IP netns del $NETNS_NAME exit 1 fi
Signature · 1 line
2.11.0
Phil Sutter
reviewerPATCH 3/3 sets: Fix for missing space after last element
In reply to Phil SutterNot having a space between the last element in a set and the closing curly brace looks ugly, so add it here.
This also adjusts all shell testcases as they match whitespace in nft output and therefore fail otherwise.
--- src/expression.c | 2 +- tests/shell/testcases/maps/0003map_add_many_elements_0 | 2 +- tests/shell/testcases/maps/0004interval_map_create_once_0 | 2 +- .../testcases/maps/0005interval_map_add_many_elements_0 | 2 +- tests/shell/testcases/maps/0006interval_map_overlap_0 | 2 +- tests/shell/testcases/netns/0001nft-f_0 | 12 ++++++------ tests/shell/testcases/netns/0002loosecommands_0 | 4 ++-- tests/shell/testcases/netns/0003many_0 | 12 ++++++------ tests/shell/testcases/nft-f/0002rollback_rule_0 | 4 ++-- tests/shell/testcases/nft-f/0003rollback_jump_0 | 4 ++-- tests/shell/testcases/nft-f/0004rollback_set_0 | 4 ++-- tests/shell/testcases/nft-f/0005rollback_map_0 | 6 +++--- tests/shell/testcases/sets/0021nesting_0 | 2 +- tests/shell/testcases/transactions/0035set_0 | 2 +- tests/shell/testcases/transactions/0038set_0 | 2 +- tests/shell/testcases/transactions/0039set_0 | 2 +- 16 files changed, 32 insertions(+), 32 deletions(-)
src/expression.c +1 -1
@@ -746,7 +746,7 @@ static void set_expr_print(const struct expr *expr) {printf ("{ " );compound_expr_print (expr,", " ); -printf ("}" ); +printf (" }" ); } static void set_expr_set_type(const struct expr *expr,
tests/shell/testcases/maps/0003map_add_many_elements_0 +1 -1
@@ -49,7 +49,7 @@ $NFT -f $tmpfile EXPECTED="table ip x { map y { type ipv4_addr : ipv4_addr - elements = { $(generate_test)} + elements = { $(generate_test) } } }" GET=$($NFT list ruleset)
tests/shell/testcases/maps/0004interval_map_create_once_0 +1 -1
@@ -48,7 +48,7 @@ EXPECTED="table ip x { map y { type ipv4_addr : ipv4_addr flags interval - elements = { $(generate_test)} + elements = { $(generate_test) } } }" GET=$($NFT list ruleset)
tests/shell/testcases/maps/0005interval_map_add_many_elements_0 +1 -1
@@ -54,7 +54,7 @@ EXPECTED="table ip x { map y { type ipv4_addr : ipv4_addr flags interval - elements = { $(generate_test)} + elements = { $(generate_test) } } }" GET=$($NFT list ruleset)
tests/shell/testcases/maps/0006interval_map_overlap_0 +1 -1
@@ -29,7 +29,7 @@ EXPECTED="table ip x { map y { type ipv4_addr : ipv4_addr flags interval - elements = { 10.0.1.0/24 : 10.0.0.1, 10.0.2.0/24 : 10.0.0.2} + elements = { 10.0.1.0/24 : 10.0.0.1, 10.0.2.0/24 : 10.0.0.2 } } }" GET=$($NFT list ruleset)
tests/shell/testcases/netns/0001nft-f_0 +6 -6
@@ -19,12 +19,12 @@ trap "rm -rf $tmpfile" EXIT # cleanup if aborted RULESET="table ip t { set s { type ipv4_addr - elements = { 1.1.0.0} + elements = { 1.1.0.0 } } chain c { ct state new - udp dport { 12345} + udp dport { 12345 } ip saddr @s drop jump other }
@@ -35,12 +35,12 @@ RULESET="table ip t { table ip6 t { set s { type ipv6_addr - elements = { fe00::1} + elements = { fe00::1 } } chain c { ct state new - udp dport { 12345} + udp dport { 12345 } ip6 saddr @s drop jump other }
@@ -51,12 +51,12 @@ table ip6 t { table inet t { set s { type ipv6_addr - elements = { fe00::1} + elements = { fe00::1 } } chain c { ct state new - udp dport { 12345} + udp dport { 12345 } ip6 saddr @s drop jump other }
tests/shell/testcases/netns/0002loosecommands_0 +2 -2
@@ -39,12 +39,12 @@ netns_exec $NETNS_NAME "$NFT add rule ip t c jump other" RULESET="table ip t { set s { type ipv4_addr - elements = { 1.1.0.0} + elements = { 1.1.0.0 } } chain c { ct state new - udp dport { 12345} + udp dport { 12345 } ip saddr @s drop jump other }
tests/shell/testcases/netns/0003many_0 +6 -6
@@ -22,12 +22,12 @@ trap "rm -rf $tmpfile" EXIT # cleanup if aborted RULESET="table ip t { set s { type ipv4_addr - elements = { 1.1.0.0} + elements = { 1.1.0.0 } } chain c { ct state new - udp dport { 12345} + udp dport { 12345 } ip saddr @s drop jump other }
@@ -38,12 +38,12 @@ RULESET="table ip t { table ip6 t { set s { type ipv6_addr - elements = { fe00::1} + elements = { fe00::1 } } chain c { ct state new - udp dport { 12345} + udp dport { 12345 } ip6 saddr @s drop jump other }
@@ -54,12 +54,12 @@ table ip6 t { table inet t { set s { type ipv6_addr - elements = { fe00::1} + elements = { fe00::1 } } chain c { ct state new - udp dport { 12345} + udp dport { 12345 } ip6 saddr @s drop jump other }
tests/shell/testcases/nft-f/0002rollback_rule_0 +2 -2
@@ -14,12 +14,12 @@ trap "rm -rf $tmpfile" EXIT # cleanup if aborted GOOD_RULESET="table ip t { set t { type ipv4_addr - elements = { 1.1.1.1} + elements = { 1.1.1.1 } } chain c { ct state new - tcp dport { 22222} + tcp dport { 22222 } ip saddr @t drop jump other }
tests/shell/testcases/nft-f/0003rollback_jump_0 +2 -2
@@ -14,12 +14,12 @@ trap "rm -rf $tmpfile" EXIT # cleanup if aborted GOOD_RULESET="table ip t { set t { type ipv4_addr - elements = { 1.1.1.1} + elements = { 1.1.1.1 } } chain c { ct state new - tcp dport { 22222} + tcp dport { 22222 } ip saddr @t drop jump other }
tests/shell/testcases/nft-f/0004rollback_set_0 +2 -2
@@ -14,12 +14,12 @@ trap "rm -rf $tmpfile" EXIT # cleanup if aborted GOOD_RULESET="table ip t { set t { type ipv4_addr - elements = { 1.1.1.1} + elements = { 1.1.1.1 } } chain c { ct state new - tcp dport { 22222} + tcp dport { 22222 } ip saddr @t drop jump other }
tests/shell/testcases/nft-f/0005rollback_map_0 +3 -3
@@ -14,12 +14,12 @@ trap "rm -rf $tmpfile" EXIT # cleanup if aborted GOOD_RULESET="table ip t { set t { type ipv4_addr - elements = { 1.1.1.1} + elements = { 1.1.1.1 } } chain c { ct state new - tcp dport { 22222} + tcp dport { 22222 } ip saddr @t drop jump other }
@@ -31,7 +31,7 @@ GOOD_RULESET="table ip t { BAD_RULESET="flush ruleset table ip t2 { chain c2 { - tcp dport map {22222: jump other, 11111: jump invalid} + tcp dport map { 22222: jump other, 11111: jump invalid } } chain other {
tests/shell/testcases/sets/0021nesting_0 +1 -1
@@ -33,7 +33,7 @@ fi EXPECTED="table ip x { chain y { - ip saddr { 1.1.1.0/24, 2.2.2.0/24, 3.3.3.0/24} + ip saddr { 1.1.1.0/24, 2.2.2.0/24, 3.3.3.0/24 } } }"
tests/shell/testcases/transactions/0035set_0 +1 -1
@@ -27,7 +27,7 @@ fi EXPECTED="table ip x { set y { type ipv4_addr - elements = { 3.3.3.3} + elements = { 3.3.3.3 } } }"
tests/shell/testcases/transactions/0038set_0 +1 -1
@@ -28,7 +28,7 @@ EXPECTED="table ip x { set y { type ipv4_addr flags interval - elements = { 192.168.4.0/24} + elements = { 192.168.4.0/24 } } }"
tests/shell/testcases/transactions/0039set_0 +1 -1
@@ -28,7 +28,7 @@ EXPECTED="table ip x { set y { type ipv4_addr flags interval - elements = { 192.168.4.0/24} + elements = { 192.168.4.0/24 } } }"
Signature · 1 line
2.11.0
Re: [nft PATCH 2/3] tests: shell: netns/0003many_0: Fix cleanup after error
In reply to Phil SutterQuoted 2 lines · Phil Sutter
If rule set applying failed, this would leave a stray netns in place.
Thanks Phil.
Re: [nft PATCH 1/3] tests: Add test cases for nested anonymous sets
In reply to Phil SutterQuoted 3 lines · Phil Sutter
This makes sure nesting of anonymous sets works regardless of whether defines are used or not. As a side-effect, it also checks that overlap checking when IP address prefixes are used, works.
Applied, thanks.
Pablo Neira Ayuso
maintainerRe: [nft PATCH 2/3] tests: shell: netns/0003many_0: Fix cleanup after error
In reply to Phil SutterQuoted 26 lines · Phil Sutter
If rule set applying failed, this would leave a stray netns in place.
Interestingly, this situation led to other, seemingly unrelated testcases to fail with spurious errors, e.g. sets/0015rulesetflush_0:
| $ ./run-tests.sh testcases/sets/0015rulesetflush_0 | I: using nft binary ../../src/nft | | W: [FAILED] testcases/sets/0015rulesetflush_0: expected 0 but got 1 | /tmp/tmp.BY7cuUYL8f:5:1-2: Error: Could not process rule: Operation not supported | table inet filter { | ^^ | /tmp/tmp.BY7cuUYL8f:9:1-2: Error: Could not process rule: No such file or directory | add element inet filter blacklist_v4 { | ^^ | /tmp/tmp.BY7cuUYL8f:5:1-2: Error: Could not process rule: Operation not supported | table inet filter { | ^^ | /tmp/tmp.BY7cuUYL8f:9:1-2: Error: Could not process rule: No such file or directory | add element inet filter blacklist_v4 { | ^^ | | I: results: [OK] 0 [FAILED] 1 [TOTAL] 1 | | $ ip netns list | 1_0003many_0
Quoted 8 lines
| $ ip netns del 1_0003many_0 | | $ ./run-tests.sh testcases/sets/0015rulesetflush_0 | I: using nft binary ../../src/nft | | I: [OK] testcases/sets/0015rulesetflush_0 | | I: results: [OK] 1 [FAILED] 0 [TOTAL] 1
Also applied, thanks.
Pablo Neira Ayuso
maintainerRe: [nft PATCH 3/3] sets: Fix for missing space after last element
In reply to Phil SutterQuoted 5 lines · Phil Sutter
Not having a space between the last element in a set and the closing curly brace looks ugly, so add it here.
This also adjusts all shell testcases as they match whitespace in nft output and therefore fail otherwise.
It's just one extra byte in the output. I don't mind, so applied this.