Skip to contentSkip to conversation

src: print count variable in normal set listings

PATCH New Florian Westphal 1 participant 5 messages Apr 8, 2025

Subsystems (2)
lore
Share & tools
lore

Florian Westphal

author maintainer
#0 lore raw

v2 cover src: print count variable in normal set listings

Print the number of allocated set elements if the set provided an upper size limit and there is at least one element.

Example:

table ip t {

   set s {
       type ipv4_addr
       size 65535      # count 1
       flags dynamic
       counter
       elements = { 1.1.1.1 counter packets 1 bytes 11 }

}
...

First patch prepares python tests to skip the new debug information coming with updated libnftnl.
This is needed to avoid a (large) update of all the existing / recorded dumpinfo.

Second patch dumps set debug info with --debug=netlink flag, i.e.:
nft --debug=netlink list ruleset
family 2 s t 0 backend nft_set_rhash_type
family 2 __set0 t 3 size 3 backend nft_set_hash_fast_type count 3 (actual ruleset omitted)

Third patch includes the new 'size <value> # count <value>' output.

Last patch fixes shell tests:
Dumps are updated with the new format (not too many changes, the change is only needed for sets that have both 'size' set and are not empty at the end of the test).

For old kernels (missing COUNT attribute), the dumps will be postprocessed after failure to strip '# count', then re-diff happens.

Florian Westphal (4):
tests/py: prepare for set debug change
debug: include kernel set information on cache fill
src: print count variable in normal set listings
tests: shell: add feature check for count output change

 include/rule.h                                |  2 ++
 src/mnl.c                                     | 15 +++++++--
 src/netlink.c                                 |  6 ++++
 src/rule.c                                    |  9 +++--
 tests/py/nft-test.py                          | 15 ++++++---
 tests/shell/features/setcount.sh              | 13 ++++++++
 tests/shell/helpers/test-wrapper.sh           | 33 +++++++++++++++----
 .../testcases/rule_management/0011reset_0     | 30 ++++++++++-------
 .../rule_management/dumps/0011reset_0.nft     |  2 +-
 .../sets/dumps/0016element_leak_0.nft         |  2 +-
 .../sets/dumps/0017add_after_flush_0.nft      |  2 +-
 .../sets/dumps/0018set_check_size_1.nft       |  2 +-
 .../sets/dumps/0019set_check_size_0.nft       |  2 +-
 .../sets/dumps/0045concat_ipv4_service.nft    |  2 +-
 .../sets/dumps/0057set_create_fails_0.nft     |  2 +-
 .../sets/dumps/0060set_multistmt_1.nft        |  2 +-
 16 files changed, 102 insertions(+), 37 deletions(-)
 create mode 100755 tests/shell/features/setcount.sh
Signature · 1 line
2.49.0
author maintainer
#1 lore raw

PATCH v2 1/4 tests/py: prepare for set debug change

Next patch will make initial set dump from kernel emit set debug information, so the obtained netlink debug file won't match what is recorded in tests/py.

Furthermore, as the python add rules for each of the family the test is for, subsequent dump will include debug information of the other/previous families.

Change the script to skip all unrelated information to only compare the relevant set element information and the generated expressions.

This change still finds changes in [ expr ... ] and set elem debug output.

---
 tests/py/nft-test.py | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
tests/py/nft-test.py +10 -5
index 7acdb77f2d0a..ea70f19b196d 100755
@@ -717,17 +717,20 @@ def payload_check(payload_buffer, file, cmd):
         return False
 
     for lineno, want_line in enumerate(payload_buffer):
+        # skip irreleant parts, such as "ip test-ipv4 output"
+        if want_line.find("[") < 0 or want_line.find("]") < 0:
+             continue
+
         line = file.readline()
+        while line.find("[") < 0 or line.find("]") < 0:
+            line = file.readline()
+            if line == "":
+                break
 
         if want_line == line:
             i += 1
             continue
 
-        if want_line.find('[') < 0 and line.find('[') < 0:
-            continue
-        if want_line.find(']') < 0 and line.find(']') < 0:
-            continue
-
         if payload_check_set_elems(want_line, line):
             continue
 
@@ -877,6 +880,8 @@ def rule_add(rule, filename, lineno, force_all_family_option, filename_path):
                     gotf.write("# %s\n" % rule[0])
                     while True:
                         line = payload_log.readline()
+                        if line.startswith("family "):
+                            continue
                         if line == "":
                             break
                         gotf.write(line)
Signature · 1 line
2.49.0
author maintainer
#2 lore raw

PATCH v2 2/4 debug: include kernel set information on cache fill

In reply to Florian Westphal

Honor --debug=netlink flag also when doing initial set dump from the kernel.

With recent libnftnl update this will include the chosen set backend name that is used by the kernel.

Because set names are scoped by table and protocol family, also include the family protocol number.

Dumping this information breaks tests/py as the recorded debug output no longer matches, this is fixed in previous change.

---
 src/mnl.c     | 15 +++++++++++++--
 src/netlink.c |  3 +++
 2 files changed, 16 insertions(+), 2 deletions(-)
src/mnl.c +13 -2
index 64b1aaedb84c..22041bc994fb 100644
@@ -1385,9 +1385,15 @@ int mnl_nft_set_del(struct netlink_ctx *ctx, struct cmd *cmd)
 	return 0;
 }
 
+struct set_cb_args {
+	struct netlink_ctx *ctx;
+	struct nftnl_set_list *list;
+};
+
 static int set_cb(const struct nlmsghdr *nlh, void *data)
 {
-	struct nftnl_set_list *nls_list = data;
+	struct set_cb_args *args = data;
+	struct nftnl_set_list *nls_list = args->list;
 	struct nftnl_set *s;
 
 	if (check_genid(nlh) < 0)
@@ -1400,6 +1406,8 @@ static int set_cb(const struct nlmsghdr *nlh, void *data)
 	if (nftnl_set_nlmsg_parse(nlh, s) < 0)
 		goto err_free;
 
+	netlink_dump_set(s, args->ctx);
+
 	nftnl_set_list_add_tail(s, nls_list);
 	return MNL_CB_OK;
 
@@ -1418,6 +1426,7 @@ mnl_nft_set_dump(struct netlink_ctx *ctx, int family,
 	struct nlmsghdr *nlh;
 	struct nftnl_set *s;
 	int ret;
+	struct set_cb_args args;
 
 	s = nftnl_set_alloc();
 	if (s == NULL)
@@ -1439,7 +1448,9 @@ mnl_nft_set_dump(struct netlink_ctx *ctx, int family,
 	if (nls_list == NULL)
 		memory_allocation_error();
 
-	ret = nft_mnl_talk(ctx, nlh, nlh->nlmsg_len, set_cb, nls_list);
+	args.list = nls_list;
+	args.ctx  = ctx;
+	ret = nft_mnl_talk(ctx, nlh, nlh->nlmsg_len, set_cb, &args);
 	if (ret < 0 && errno != ENOENT)
 		goto err;
 
src/netlink.c +3 -0
index dfb7f4d17147..98ec3cdba996 100644
@@ -847,10 +847,13 @@ static const struct datatype *dtype_map_from_kernel(enum nft_data_types type)
 void netlink_dump_set(const struct nftnl_set *nls, struct netlink_ctx *ctx)
 {
 	FILE *fp = ctx->nft->output.output_fp;
+	uint32_t family;
 
 	if (!(ctx->nft->debug_mask & NFT_DEBUG_NETLINK) || !fp)
 		return;
 
+	family = nftnl_set_get_u32(nls, NFTNL_SET_FAMILY);
+	fprintf(fp, "family %d ", family);
 	nftnl_set_fprintf(fp, nls, 0, 0);
 	fprintf(fp, "\n");
 }
Signature · 1 line
2.49.0
author maintainer
#3 lore raw

PATCH v2 3/4 src: print count variable in normal set listings

In reply to Florian Westphal

Also print the number of allocated set elements if the set provided an upper size limit and there is at least one element.

Example:

table ip t {

   set s {
       type ipv4_addr
       size 65535      # count 1
       flags dynamic
       counter
       elements = { 1.1.1.1 counter packets 1 bytes 11 }

}
...

JSON output is unchanged as this only has informational purposes.

This change breaks tests, followup patch addresses this.

---
 include/rule.h | 2 ++
 src/netlink.c  | 3 +++
 src/rule.c     | 9 ++++++---
 3 files changed, 11 insertions(+), 3 deletions(-)
include/rule.h +2 -0
index 85a0d9c0b524..5c8870032472 100644
@@ -321,6 +321,7 @@ void rule_stmt_insert_at(struct rule *rule, struct stmt *nstmt,
  * @refcnt:	reference count
  * @flags:	bitmask of set flags
  * @gc_int:	garbage collection interval
+ * @count:	count of kernel-allocated elements
  * @timeout:	default timeout value
  * @key:	key expression (data type, length))
  * @data:	mapping data expression
@@ -345,6 +346,7 @@ struct set {
 	unsigned int		refcnt;
 	uint32_t		flags;
 	uint32_t		gc_int;
+	uint32_t		count;
 	uint64_t		timeout;
 	struct expr		*key;
 	struct expr		*data;
src/netlink.c +3 -0
index 98ec3cdba996..9b197f089d40 100644
@@ -1129,6 +1129,9 @@ struct set *netlink_delinearize_set(struct netlink_ctx *ctx,
 	if (nftnl_set_is_set(nls, NFTNL_SET_DESC_SIZE))
 		set->desc.size = nftnl_set_get_u32(nls, NFTNL_SET_DESC_SIZE);
 
+	if (nftnl_set_is_set(nls, NFTNL_SET_COUNT))
+		set->count = nftnl_set_get_u32(nls, NFTNL_SET_COUNT);
+
 	if (nftnl_set_is_set(nls, NFTNL_SET_DESC_CONCAT)) {
 		uint32_t len = NFT_REG32_COUNT;
 		const uint8_t *data;
src/rule.c +6 -3
index 80315837baf0..6af8d57eddb6 100644
@@ -335,10 +335,13 @@ static void set_print_declaration(const struct set *set,
 		}
 
 		if (set->desc.size > 0) {
-			nft_print(octx, "%s%ssize %u%s",
+			nft_print(octx, "%s%ssize %u",
 				  opts->tab, opts->tab,
-				  set->desc.size,
-				  opts->stmt_separator);
+				  set->desc.size);
+			if (set->count > 0)
+				nft_print(octx, "%s# count %u", opts->tab,
+					  set->count);
+			nft_print(octx, "%s", opts->stmt_separator);
 		}
 	}
 
Signature · 1 line
2.49.0
author maintainer
#4 lore raw

PATCH v2 4/4 tests: shell: add feature check for count output change

In reply to Florian Westphal

New kernels with latest nft release will print the number of set elements allocated on the kernel side.

This causes shell test dump validation to fail in several places. We can't just update the affected dump files because the test cases are also supposed to pass on current -stable releases.

Add a feature check for this. Dump failure can then use sed to postprocess the stored dump file and can then call diff a second time.

---
 tests/shell/features/setcount.sh              | 13 ++++++++
 tests/shell/helpers/test-wrapper.sh           | 33 +++++++++++++++----
 .../testcases/rule_management/0011reset_0     | 30 ++++++++++-------
 .../rule_management/dumps/0011reset_0.nft     |  2 +-
 .../sets/dumps/0016element_leak_0.nft         |  2 +-
 .../sets/dumps/0017add_after_flush_0.nft      |  2 +-
 .../sets/dumps/0018set_check_size_1.nft       |  2 +-
 .../sets/dumps/0019set_check_size_0.nft       |  2 +-
 .../sets/dumps/0045concat_ipv4_service.nft    |  2 +-
 .../sets/dumps/0057set_create_fails_0.nft     |  2 +-
 .../sets/dumps/0060set_multistmt_1.nft        |  2 +-
 11 files changed, 65 insertions(+), 27 deletions(-)
 create mode 100755 tests/shell/features/setcount.sh
tests/shell/features/setcount.sh new file +13 -0
new file mode 100755
index 000000000000..9c2f75c3ae96
@@ -0,0 +1,13 @@
+#!/bin/bash
+
+$NFT -f - <<EOF
+table ip t {
+	set s {
+		type ipv4_addr
+		size 2
+		elements = { 1.2.3.4 }
+	}
+}
+EOF
+
+$NFT list set ip t s | grep -q 'size 2	# count 1'
tests/shell/helpers/test-wrapper.sh +26 -7
index c016e0ce1d39..8c0263febaef 100755
@@ -5,6 +5,8 @@
 #
 # For some printf debugging, you can also patch this file.
 
+rc_dump=0
+
 array_contains() {
 	local needle="$1"
 	local a
@@ -25,6 +27,29 @@ show_file() {
 	printf "<<<<\n"
 }
 
+diff_check_setcount() {
+	local dumpfile="$1"
+	local after="$2"
+
+	if $DIFF -u "$dumpfile" "$after" &> "$NFT_TEST_TESTTMPDIR/ruleset-diff" ; then
+		rm -f "$NFT_TEST_TESTTMPDIR/ruleset-diff"
+		return
+	fi
+
+	if [ $NFT_TEST_HAVE_setcount = n ];then
+		# old kernel or nft binary, expect "size 42", not "size 42	# count 1".
+		sed s/.\#\ count\ .\*//g "$dumpfile" > "$NFT_TEST_TESTTMPDIR/ruleset-diff-postprocess"
+
+		if $DIFF -u "$NFT_TEST_TESTTMPDIR/ruleset-diff-postprocess" "$after" > /dev/null ; then
+			rm -f "$NFT_TEST_TESTTMPDIR/ruleset-diff" "$NFT_TEST_TESTTMPDIR/ruleset-diff-postprocess"
+			return
+		fi
+	fi
+
+	show_file "$NFT_TEST_TESTTMPDIR/ruleset-diff" "Failed \`$DIFF -u \"$dumpfile\" \"$after\"\`" >> "$NFT_TEST_TESTTMPDIR/rc-failed-dump"
+	rc_dump=1
+}
+
 json_pretty() {
 	"$NFT_TEST_BASEDIR/helpers/json-pretty.sh" "$@" 2>&1 || :
 }
@@ -195,15 +220,9 @@ if [ "$rc_test" -eq 0 -a '(' "$DUMPGEN" = all -o "$DUMPGEN" = y ')' ] ; then
 	fi
 fi
 
-rc_dump=0
 if [ "$rc_test" -ne 77 -a "$dump_written" != y ] ; then
 	if [ -f "$DUMPFILE" ] ; then
-		if ! $DIFF -u "$DUMPFILE" "$NFT_TEST_TESTTMPDIR/ruleset-after" &> "$NFT_TEST_TESTTMPDIR/ruleset-diff" ; then
-			show_file "$NFT_TEST_TESTTMPDIR/ruleset-diff" "Failed \`$DIFF -u \"$DUMPFILE\" \"$NFT_TEST_TESTTMPDIR/ruleset-after\"\`" >> "$NFT_TEST_TESTTMPDIR/rc-failed-dump"
-			rc_dump=1
-		else
-			rm -f "$NFT_TEST_TESTTMPDIR/ruleset-diff"
-		fi
+		diff_check_setcount "$DUMPFILE" "$NFT_TEST_TESTTMPDIR/ruleset-after"
 	fi
 	if [ "$NFT_TEST_HAVE_json" != n -a -f "$JDUMPFILE" ] ; then
 		if ! $DIFF -u "$JDUMPFILE" "$NFT_TEST_TESTTMPDIR/ruleset-after.json-pretty" &> "$NFT_TEST_TESTTMPDIR/ruleset-diff.json" ; then
tests/shell/testcases/rule_management/0011reset_0 +18 -12
index 2004b17d5822..5e65ced946e5 100755
@@ -4,6 +4,12 @@
 
 set -e
 
+if [ $NFT_TEST_HAVE_setcount = y ]; then
+	size="size 65535	# count 1"
+else
+	size="size 65535"
+fi
+
 echo "loading ruleset with anonymous set"
 $NFT -f - <<EOF
 table t {
@@ -60,10 +66,10 @@ EOF
 echo "resetting specific rule"
 handle=$($NFT -a list chain t c | sed -n 's/.*accept # handle \([0-9]*\)$/\1/p')
 $NFT reset rule t c handle $handle
-EXPECT='table ip t {
+EXPECT="table ip t {
 	set s {
 		type ipv4_addr
-		size 65535
+		$size
 		flags dynamic
 		counter
 		elements = { 1.1.1.1 counter packets 1 bytes 11 }
@@ -90,7 +96,7 @@ table ip t2 {
 		counter packets 7 bytes 17 accept
 		counter packets 8 bytes 18 drop
 	}
-}'
+}"
 $DIFF -u <(echo "$EXPECT") <($NFT list ruleset)
 
 echo "resetting specific chain"
@@ -103,10 +109,10 @@ EXPECT='table ip t {
 $DIFF -u <(echo "$EXPECT") <($NFT reset rules chain t c2)
 
 echo "resetting specific table"
-EXPECT='table ip t {
+EXPECT="table ip t {
 	set s {
 		type ipv4_addr
-		size 65535
+		$size
 		flags dynamic
 		counter
 		elements = { 1.1.1.1 counter packets 1 bytes 11 }
@@ -121,14 +127,14 @@ EXPECT='table ip t {
 		counter packets 0 bytes 0 accept
 		counter packets 0 bytes 0 drop
 	}
-}'
+}"
 $DIFF -u <(echo "$EXPECT") <($NFT reset rules table t)
 
 echo "resetting specific family"
-EXPECT='table ip t {
+EXPECT="table ip t {
 	set s {
 		type ipv4_addr
-		size 65535
+		$size
 		flags dynamic
 		counter
 		elements = { 1.1.1.1 counter packets 1 bytes 11 }
@@ -149,14 +155,14 @@ table ip t2 {
 		counter packets 7 bytes 17 accept
 		counter packets 8 bytes 18 drop
 	}
-}'
+}"
 $DIFF -u <(echo "$EXPECT") <($NFT reset rules ip)
 
 echo "resetting whole ruleset"
-EXPECT='table ip t {
+EXPECT="table ip t {
 	set s {
 		type ipv4_addr
-		size 65535
+		$size
 		flags dynamic
 		counter
 		elements = { 1.1.1.1 counter packets 1 bytes 11 }
@@ -183,5 +189,5 @@ table ip t2 {
 		counter packets 0 bytes 0 accept
 		counter packets 0 bytes 0 drop
 	}
-}'
+}"
 $DIFF -u <(echo "$EXPECT") <($NFT reset rules)
tests/shell/testcases/rule_management/dumps/0011reset_0.nft +1 -1
index 3b4f5a11a96e..3c29b582355d 100644
@@ -1,7 +1,7 @@
 table ip t {
 	set s {
 		type ipv4_addr
-		size 65535
+		size 65535	# count 1
 		flags dynamic
 		counter
 		elements = { 1.1.1.1 counter packets 1 bytes 11 }
tests/shell/testcases/sets/dumps/0016element_leak_0.nft +1 -1
index 9d2b0afed425..debd819dfc2b 100644
@@ -1,7 +1,7 @@
 table ip x {
 	set s {
 		type ipv4_addr
-		size 2
+		size 2	# count 1
 		elements = { 1.1.1.1 }
 	}
 }
tests/shell/testcases/sets/dumps/0017add_after_flush_0.nft +1 -1
index 9d2b0afed425..debd819dfc2b 100644
@@ -1,7 +1,7 @@
 table ip x {
 	set s {
 		type ipv4_addr
-		size 2
+		size 2	# count 1
 		elements = { 1.1.1.1 }
 	}
 }
tests/shell/testcases/sets/dumps/0018set_check_size_1.nft +1 -1
index 8cd3707607b3..c4b69ef8e409 100644
@@ -1,7 +1,7 @@
 table ip x {
 	set s {
 		type ipv4_addr
-		size 2
+		size 2	# count 2
 		elements = { 1.1.1.1, 1.1.1.2 }
 	}
 }
tests/shell/testcases/sets/dumps/0019set_check_size_0.nft +1 -1
index 8cd3707607b3..c4b69ef8e409 100644
@@ -1,7 +1,7 @@
 table ip x {
 	set s {
 		type ipv4_addr
-		size 2
+		size 2	# count 2
 		elements = { 1.1.1.1, 1.1.1.2 }
 	}
 }
tests/shell/testcases/sets/dumps/0045concat_ipv4_service.nft +1 -1
index e548a17a142d..fb9634e685d3 100644
@@ -1,7 +1,7 @@
 table inet t {
 	set s {
 		type ipv4_addr . inet_service
-		size 65536
+		size 65536	# count 1
 		flags dynamic,timeout
 		elements = { 192.168.7.1 . 22 }
 	}
tests/shell/testcases/sets/dumps/0057set_create_fails_0.nft +1 -1
index de43d565084b..443ca7110f4f 100644
@@ -1,7 +1,7 @@
 table inet filter {
 	set test {
 		type ipv4_addr
-		size 65535
+		size 65535	# count 1
 		elements = { 1.1.1.1 }
 	}
 }
tests/shell/testcases/sets/dumps/0060set_multistmt_1.nft +1 -1
index befc2f75bd42..0743453f62e0 100644
@@ -1,7 +1,7 @@
 table ip x {
 	set y {
 		type ipv4_addr
-		size 65535
+		size 65535	# count 3
 		flags dynamic
 		counter quota 500 bytes
 		elements = { 1.1.1.1 counter packets 0 bytes 0 quota 500 bytes,
Signature · 1 line
2.49.0