Skip to contentSkip to conversation

Speed up iptables-test.py

PATCH In review Phil Sutter 1 participant 14 messages Oct 12, 2022 – Oct 26, 2022 netfilter

lore
Share & tools
lore

Phil Sutter

author reviewer
#0 lore raw

v2 cover Speed up iptables-test.py

Changes since v1:
- Add fallout NFQUEUE patch 1
- Adjust iptables-test.py enhancement as suggested by Jan, details in patch 2
- Avoid changing what extensions print, this is a separate discussion - instead update expected output accordingly

Original cover letter text:

I had this in mind for a while now and finally got around to do it: When testing an extensions/*.t file with iptables-tests.py, act in a "batch" mode applying all rules at once and checking the expected output in one go, thereby reducing the overhead per test file to a single iptables-restore and iptables-save call each. This was a bit optimistic, but the result is still significant - on my rather slow testing VM, a full iptables-tests.py run completes in ~7min instead of ~30min (yes, it's slow).

See patch 1 for the implementation details. As a side-effect, rule existence checking became much stricter, so the remaining patches in this series deal with eliminating those differences:

* Patch 2 avoids having to add '-j CONTINUE' to almost all ebtables rules.
* Patches 3-10 adjust expected output to reality, mostly adding content the script didn't care about since the old 'output.find(<rule>)' worked fine as long as the output *started* like '<rule>'.
* Patch 11 Changes output by omitting an obvious default value, so a real functional change.
* Patch 12 drops another default value (from NFQUEUE target) I'm not sure we should keep.

So patches are roughly sorted by my confidence in correctness. Please have (at least) a close look at the last two, I don't want to break iptables for anyone just to keep test files small.

Phil Sutter (12):
extensions: NFQUEUE: Document queue-balance limitation
tests: iptables-test: Implement fast test mode
tests: iptables-test: Cover for obligatory -j CONTINUE in ebtables
tests: *.t: Fix expected output for simple calls
tests: *.t: Fix for hexadecimal output
tests: libebt_redirect.t: Plain redirect prints with trailing
whitespace
tests: libxt_length.t: Fix odd use-case output
tests: libxt_recent.t: Add missing default values
tests: libxt_tos.t, libxt_TOS.t: Add missing masks in output
tests: libebt_vlan.t: Drop trailing whitespace from rules
tests: libxt_connlimit.t: Add missing default values
tests: *.t: Add missing all-one's netmasks to expected output

 extensions/libebt_log.t      |   2 +-
 extensions/libebt_nflog.t    |   2 +-
 extensions/libebt_redirect.t |   2 +-
 extensions/libebt_vlan.t     |   4 +-
 extensions/libip6t_NETMAP.t  |   2 +-
 extensions/libip6t_REJECT.t  |   2 +-
 extensions/libipt_NETMAP.t   |   2 +-
 extensions/libipt_REJECT.t   |   2 +-
 extensions/libxt_CONNMARK.t  |   8 +-
 extensions/libxt_DSCP.t      |   2 +-
 extensions/libxt_MARK.t      |   4 +-
 extensions/libxt_NFQUEUE.c   |   2 +-
 extensions/libxt_NFQUEUE.man |   2 +
 extensions/libxt_NFQUEUE.t   |   7 +-
 extensions/libxt_TOS.t       |  12 +--
 extensions/libxt_connlimit.t |  12 +--
 extensions/libxt_connmark.t  |   4 +-
 extensions/libxt_dscp.t      |   2 +-
 extensions/libxt_length.t    |   2 +-
 extensions/libxt_mark.t      |   2 +-
 extensions/libxt_recent.t    |   4 +-
 extensions/libxt_tos.t       |   8 +-
 iptables-test.py             | 163 ++++++++++++++++++++++++++++++++++-
 23 files changed, 207 insertions(+), 45 deletions(-)
Signature · 1 line
2.34.1
author reviewer
#1 lore raw

PATCH v2 1/12 extensions: NFQUEUE: Document queue-balance limitation

The range is not communicated to the kernel as "min and max queue number", but "first queue number and count" instead. With 16bits for each value, it is not possible to balance between all 65536 possible queues. Although probably never used in practice, point this detail out in man page and make the parser complain instead of the cryptic "xt_NFQUEUE: number of total queues is 0" emitted by the kernel module.

---
 extensions/libxt_NFQUEUE.c   | 2 +-
 extensions/libxt_NFQUEUE.man | 2 ++
 extensions/libxt_NFQUEUE.t   | 5 ++---
 3 files changed, 5 insertions(+), 4 deletions(-)
extensions/libxt_NFQUEUE.c +1 -1
index fe5190789e306..ca6cdaf49703c 100644
@@ -64,7 +64,7 @@ static const struct xt_option_entry NFQUEUE_opts[] = {
 	{.name = "queue-num", .id = O_QUEUE_NUM, .type = XTTYPE_UINT16,
 	 .flags = XTOPT_PUT, XTOPT_POINTER(s, queuenum),
 	 .excl = F_QUEUE_BALANCE},
-	{.name = "queue-balance", .id = O_QUEUE_BALANCE,
+	{.name = "queue-balance", .id = O_QUEUE_BALANCE, .max = UINT16_MAX - 1,
 	 .type = XTTYPE_UINT16RC, .excl = F_QUEUE_NUM},
 	{.name = "queue-bypass", .id = O_QUEUE_BYPASS, .type = XTTYPE_NONE},
 	{.name = "queue-cpu-fanout", .id = O_QUEUE_CPU_FANOUT,
extensions/libxt_NFQUEUE.man +2 -0
index 1bfb7b843587f..950b0d2412e58 100644
@@ -18,6 +18,8 @@ This specifies a range of queues to use. Packets are then balanced across the gi
 This is useful for multicore systems: start multiple instances of the userspace program on
 queues x, x+1, .. x+n and use "\-\-queue\-balance \fIx\fP\fB:\fP\fIx+n\fP".
 Packets belonging to the same connection are put into the same nfqueue.
+Due to implementation details, a lower range value of 0 limits the higher range
+value to 65534, i.e. one can only balance between at most 65535 queues.
 .PP
 .TP
 \fB\-\-queue\-bypass\fP
extensions/libxt_NFQUEUE.t +2 -3
index b51b19fd435f7..5a2df6e7d00d0 100644
@@ -4,9 +4,8 @@
 -j NFQUEUE --queue-num 65535;=;OK
 -j NFQUEUE --queue-num 65536;;FAIL
 -j NFQUEUE --queue-num -1;;FAIL
-# it says "NFQUEUE: number of total queues is 0", overflow in NFQUEUE_parse_v1?
-# ERROR: cannot load: iptables -A INPUT -j NFQUEUE --queue-balance 0:65535
-# -j NFQUEUE --queue-balance 0:65535;=;OK
+-j NFQUEUE --queue-balance 0:65534;=;OK
+-j NFQUEUE --queue-balance 0:65535;;FAIL
 -j NFQUEUE --queue-balance 0:65536;;FAIL
 -j NFQUEUE --queue-balance -1:65535;;FAIL
 -j NFQUEUE --queue-num 10 --queue-bypass;=;OK
Signature · 1 line
2.34.1
author reviewer
#2 lore raw

PATCH v2 2/12 tests: iptables-test: Implement fast test mode

In reply to Phil Sutter

Implement a faster mode of operation for suitable test files:

1) Collect all rules to add and all expected output in lists
2) Any supposedly failing rules are checked immediately like in slow mode.
3) Create and load iptables-restore input from the list in (1)
5) Construct the expected iptables-save output from (1) and check it in a single search
5) If any of the steps above fail, fall back to slow mode for verification and detailed error analysis. Fast mode failures are not fatal, merely warn about them.

To keep things simple (and feasible), avoid complicated test files involving external commands, multiple tables or variant-specific results.

Aside from speeding up testsuite run-time, rule searching has become more strict since EOL char is practically part of the search string. This revealed many false positives where the expected string was actually a substring of the printed rule.

---
Changes since v1:
- Pass 'iptables' variable to run_test_file_fast() and construct -save and -restore commands from there
- Construct differing dumps log message before printing to simplify code a bit

---
 iptables-test.py | 160 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 159 insertions(+), 1 deletion(-)
iptables-test.py +159 -1
index b5a70e44b9e44..d9b3ee4e29464 100755
@@ -222,6 +222,153 @@ STDERR_IS_TTY = sys.stderr.isatty()
         return alt_res
     return res_inverse[res]
 
+def fast_run_possible(filename):
+    '''
+    Keep things simple, run only for simple test files:
+    - no external commands
+    - no multiple tables
+    - no variant-specific results
+    '''
+    table = None
+    rulecount = 0
+    for line in open(filename):
+        if line[0] in ["#", ":"] or len(line.strip()) == 0:
+            continue
+        if line[0] == "*":
+            if table or rulecount > 0:
+                return False
+            table = line.rstrip()[1:]
+        if line[0] in ["@", "%"]:
+            return False
+        if len(line.split(";")) > 3:
+            return False
+        rulecount += 1
+
+    return True
+
+def run_test_file_fast(iptables, filename, netns):
+    '''
+    Run a test file, but fast
+
+    :param filename: name of the file with the test rules
+    :param netns: network namespace to perform test run in
+    '''
+
+    f = open(filename)
+
+    rules = {}
+    table = "filter"
+    chain_array = []
+    tests = 0
+
+    for lineno, line in enumerate(f):
+        if line[0] == "#" or len(line.strip()) == 0:
+            continue
+
+        if line[0] == "*":
+            table = line.rstrip()[1:]
+            continue
+
+        if line[0] == ":":
+            chain_array = line.rstrip()[1:].split(",")
+            continue
+
+        if len(chain_array) == 0:
+            return -1
+
+        tests += 1
+
+        for chain in chain_array:
+            item = line.split(";")
+            rule = chain + " " + item[0]
+
+            if item[1] == "=":
+                rule_save = chain + " " + item[0]
+            else:
+                rule_save = chain + " " + item[1]
+
+            res = item[2].rstrip()
+            if res != "OK":
+                rule = chain + " -t " + table + " " + item[0]
+                ret = run_test(iptables, rule, rule_save,
+                               res, filename, lineno + 1, netns)
+
+                if ret < 0:
+                    return -1
+                continue
+
+            if not chain in rules.keys():
+                rules[chain] = []
+            rules[chain].append((rule, rule_save))
+
+    restore_data = ["*" + table]
+    out_expect = []
+    for chain in ["PREROUTING", "INPUT", "FORWARD", "OUTPUT", "POSTROUTING"]:
+        if not chain in rules.keys():
+            continue
+        for rule in rules[chain]:
+            restore_data.append("-A " + rule[0])
+            out_expect.append("-A " + rule[1])
+    restore_data.append("COMMIT")
+
+    out_expect = "\n".join(out_expect)
+
+    # load all rules via iptables_restore
+
+    command = EXECUTABLE + " " + iptables + "-restore"
+    if netns:
+        command = "ip netns exec " + netns + " " + command
+
+    for line in restore_data:
+        print(iptables + "-restore: " + line, file=log_file)
+
+    proc = subprocess.Popen(command, shell = True, text = True,
+                            stdin = subprocess.PIPE,
+                            stdout = subprocess.PIPE,
+                            stderr = subprocess.PIPE)
+    restore_data = "\n".join(restore_data) + "\n"
+    out, err = proc.communicate(input = restore_data)
+
+    if proc.returncode == -11:
+        reason = iptables + "-restore segfaults: " + cmd
+        print_error(reason, filename, lineno)
+        return -1
+
+    if proc.returncode != 0:
+        print("%s-restore returned %d: %s" % (iptables, proc.returncode, err),
+              file=log_file)
+        return -1
+
+    # find all rules in iptables_save output
+
+    command = EXECUTABLE + " " + iptables + "-save"
+    if netns:
+        command = "ip netns exec " + netns + " " + command
+
+    proc = subprocess.Popen(command, shell = True,
+                            stdin = subprocess.PIPE,
+                            stdout = subprocess.PIPE,
+                            stderr = subprocess.PIPE)
+    out, err = proc.communicate()
+
+    if proc.returncode == -11:
+        reason = iptables + "-save segfaults: " + cmd
+        print_error(reason, filename, lineno)
+        return -1
+
+    cmd = iptables + " -F -t " + table
+    execute_cmd(cmd, filename, 0, netns)
+
+    out = out.decode('utf-8').rstrip()
+    if out.find(out_expect) < 0:
+        msg = ["dumps differ!"]
+        msg.extend(["expect: " + l for l in out_expect.split("\n")])
+        msg.extend(["got: " + l for l in out.split("\n")
+                                if not l[0] in ['*', ':', '#']])
+        print("\n".join(msg), file=log_file)
+        return -1
+
+    return tests
 
 def run_test_file(filename, netns):
     '''
@@ -256,6 +403,14 @@ STDERR_IS_TTY = sys.stderr.isatty()
         # default to iptables if not known prefix
         iptables = IPTABLES
 
+    fast_failed = False
+    if fast_run_possible(filename):
+        tests = run_test_file_fast(iptables, filename, netns)
+        if tests > 0:
+            print(filename + ": " + maybe_colored('green', "OK", STDOUT_IS_TTY))
+            return tests, tests
+        fast_failed = True
+
     f = open(filename)
 
     tests = 0
@@ -330,7 +485,10 @@ STDERR_IS_TTY = sys.stderr.isatty()
     if netns:
         execute_cmd("ip netns del " + netns, filename)
     if total_test_passed:
-        print(filename + ": " + maybe_colored('green', "OK", STDOUT_IS_TTY))
+        suffix = ""
+        if fast_failed:
+            suffix = maybe_colored('red', " but fast mode failed!", STDOUT_IS_TTY)
+        print(filename + ": " + maybe_colored('green', "OK", STDOUT_IS_TTY) + suffix)
 
     f.close()
     return tests, passed
Signature · 1 line
2.34.1
author reviewer
#3 lore raw

PATCH v2 3/12 tests: iptables-test: Cover for obligatory -j CONTINUE in ebtables

In reply to Phil Sutter

Unlike iptables, ebtables includes the default rule target in output. Instead of adding it to every rule in ebtables tests, add special casing to the testscript checking if the expected rule output contains a target already and adding the default one if not.

---
 iptables-test.py | 3 +++
 1 file changed, 3 insertions(+)
iptables-test.py +3 -0
index d9b3ee4e29464..dc031c2b60450 100755
@@ -287,6 +287,9 @@ STDERR_IS_TTY = sys.stderr.isatty()
             else:
                 rule_save = chain + " " + item[1]
 
+            if iptables == EBTABLES and rule_save.find('-j') < 0:
+                rule_save += " -j CONTINUE"
+
             res = item[2].rstrip()
             if res != "OK":
                 rule = chain + " -t " + table + " " + item[0]
Signature · 1 line
2.34.1
author reviewer
#4 lore raw

PATCH v2 4/12 tests: *.t: Fix expected output for simple calls

In reply to Phil Sutter

These minimal extension uses print in more detailed form. Track this, the output is desired.

---
 extensions/libebt_log.t     | 2 +-
 extensions/libebt_nflog.t   | 2 +-
 extensions/libip6t_REJECT.t | 2 +-
 extensions/libipt_REJECT.t  | 2 +-
 extensions/libxt_NFQUEUE.t  | 2 +-
 5 files changed, 5 insertions(+), 5 deletions(-)
extensions/libebt_log.t +1 -1
index f7116c417b0ab..e6adbd3b44380 100644
@@ -1,5 +1,5 @@
 :INPUT,FORWARD,OUTPUT
---log;=;OK
+--log;--log-level notice;OK
 --log-level crit;=;OK
 --log-level 1;--log-level alert;OK
 --log-level emerg --log-ip --log-arp --log-ip6;=;OK
extensions/libebt_nflog.t +1 -1
index f867df303fa95..e98d8f5fdb9d6 100644
@@ -1,5 +1,5 @@
 :INPUT,FORWARD,OUTPUT
---nflog;=;OK
+--nflog;--nflog-group 1;OK
 --nflog-group 42;=;OK
 --nflog-range 42;--nflog-group 1 --nflog-range 42 -j CONTINUE;OK
 --nflog-threshold 100 --nflog-prefix foo;--nflog-prefix "foo" --nflog-group 1 --nflog-threshold 100 -j CONTINUE;OK
extensions/libip6t_REJECT.t +1 -1
index d2b337d7ebdeb..8294f0bb77b29 100644
@@ -1,5 +1,5 @@
 :INPUT,FORWARD,OUTPUT
--j REJECT;=;OK
+-j REJECT;-j REJECT --reject-with icmp6-port-unreachable;OK
 # manpage for IPv6 variant of REJECT does not show up for some reason?
 -j REJECT --reject-with icmp6-no-route;=;OK
 -j REJECT --reject-with icmp6-adm-prohibited;=;OK
extensions/libipt_REJECT.t +1 -1
index 5b26b1076b5b2..3f69a72994955 100644
@@ -1,5 +1,5 @@
 :INPUT,FORWARD,OUTPUT
--j REJECT;=;OK
+-j REJECT;-j REJECT --reject-with icmp-port-unreachable;OK
 -j REJECT --reject-with icmp-net-unreachable;=;OK
 -j REJECT --reject-with icmp-host-unreachable;=;OK
 -j REJECT --reject-with icmp-port-unreachable;=;OK
extensions/libxt_NFQUEUE.t +1 -1
index 5a2df6e7d00d0..8fb2b760a13bc 100644
@@ -1,5 +1,5 @@
 :INPUT,FORWARD,OUTPUT
--j NFQUEUE;=;OK
+-j NFQUEUE;-j NFQUEUE --queue-num 0;OK
 -j NFQUEUE --queue-num 0;=;OK
 -j NFQUEUE --queue-num 65535;=;OK
 -j NFQUEUE --queue-num 65536;;FAIL
Signature · 1 line
2.34.1
author reviewer
#5 lore raw

PATCH v2 5/12 tests: *.t: Fix for hexadecimal output

In reply to Phil Sutter

Use hex input to avoid having to specify an expected output in trivial cases.

---
 extensions/libxt_DSCP.t     | 2 +-
 extensions/libxt_MARK.t     | 2 +-
 extensions/libxt_connmark.t | 4 ++--
 extensions/libxt_dscp.t     | 2 +-
 extensions/libxt_mark.t     | 2 +-
 5 files changed, 6 insertions(+), 6 deletions(-)
extensions/libxt_DSCP.t +1 -1
index fcc55986dbcd7..762fcd31d4d28 100644
@@ -1,6 +1,6 @@
 :PREROUTING,INPUT,FORWARD,OUTPUT,POSTROUTING
 *mangle
--j DSCP --set-dscp 0;=;OK
+-j DSCP --set-dscp 0x00;=;OK
 -j DSCP --set-dscp 0x3f;=;OK
 -j DSCP --set-dscp -1;;FAIL
 -j DSCP --set-dscp 0x40;;FAIL
extensions/libxt_MARK.t +1 -1
index 9d1aa7d7d58a4..2902a14f06742 100644
@@ -1,6 +1,6 @@
 :INPUT,FORWARD,OUTPUT
 -j MARK --set-xmark 0xfeedcafe/0xfeedcafe;=;OK
--j MARK --set-xmark 0;=;OK
+-j MARK --set-xmark 0x0;=;OK
 -j MARK --set-xmark 4294967295;-j MARK --set-xmark 0xffffffff;OK
 -j MARK --set-xmark 4294967296;;FAIL
 -j MARK --set-xmark -1;;FAIL
extensions/libxt_connmark.t +2 -2
index 4dd7d9af265a5..353970a8995f6 100644
@@ -2,8 +2,8 @@
 *mangle
 -m connmark --mark 0xffffffff;=;OK
 -m connmark --mark 0xffffffff/0xffffffff;-m connmark --mark 0xffffffff;OK
--m connmark --mark 0xffffffff/0;=;OK
--m connmark --mark 0/0xffffffff;-m connmark --mark 0;OK
+-m connmark --mark 0xffffffff/0x0;=;OK
+-m connmark --mark 0/0xffffffff;-m connmark --mark 0x0;OK
 -m connmark --mark -1;;FAIL
 -m connmark --mark 0xfffffffff;;FAIL
 -m connmark;;FAIL
extensions/libxt_dscp.t +1 -1
index 38d7f04e18698..e010190661ae8 100644
@@ -1,5 +1,5 @@
 :INPUT,FORWARD,OUTPUT
--m dscp --dscp 0;=;OK
+-m dscp --dscp 0x00;=;OK
 -m dscp --dscp 0x3f;=;OK
 -m dscp --dscp -1;;FAIL
 -m dscp --dscp 0x40;;FAIL
extensions/libxt_mark.t +1 -1
index 7c005379f6d64..7aeb871588ca6 100644
@@ -1,6 +1,6 @@
 :INPUT,FORWARD,OUTPUT
 -m mark --mark 0xfeedcafe/0xfeedcafe;=;OK
--m mark --mark 0;=;OK
+-m mark --mark 0x0;=;OK
 -m mark --mark 4294967295;-m mark --mark 0xffffffff;OK
 -m mark --mark 4294967296;;FAIL
 -m mark --mark -1;;FAIL
Signature · 1 line
2.34.1
author reviewer
#6 lore raw

PATCH v2 6/12 tests: libebt_redirect.t: Plain redirect prints with trailing whitespace

In reply to Phil Sutter

Eliminating the trailing whitespace is non-trivial due to how ebtables extension printing works, so just update the test case instead.

---
 extensions/libebt_redirect.t | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
extensions/libebt_redirect.t +1 -1
index 23858afa3b588..58492b7968153 100644
@@ -1,4 +1,4 @@
 :PREROUTING
 *nat
--j redirect;=;OK
+-j redirect ;=;OK
 -j redirect --redirect-target RETURN;=;OK
Signature · 1 line
2.34.1
author reviewer
#7 lore raw

PATCH v2 7/12 tests: libxt_length.t: Fix odd use-case output

In reply to Phil Sutter

Specifying the lower boundary suffixed by colon is an undocumented feature. Explicitly printing the upper boundary in that case seems sane.

---
 extensions/libxt_length.t | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
extensions/libxt_length.t +1 -1
index 0b6624ee069f6..8b70fc317485c 100644
@@ -2,7 +2,7 @@
 -m length --length 1;=;OK
 -m length --length :2;-m length --length 0:2;OK
 -m length --length 0:3;=;OK
--m length --length 4:;=;OK
+-m length --length 4:;-m length --length 4:65535;OK
 -m length --length 0:65535;=;OK
 -m length ! --length 0:65535;=;OK
 -m length --length 0:65536;;FAIL
Signature · 1 line
2.34.1
author reviewer
#8 lore raw

PATCH v2 8/12 tests: libxt_recent.t: Add missing default values

In reply to Phil Sutter

The extensions always prints --name, --mask and (default) --rsource options.

---
 extensions/libxt_recent.t | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
extensions/libxt_recent.t +2 -2
index 9a83918ea5835..cf23aabc6e63b 100644
@@ -1,8 +1,8 @@
 :INPUT,FORWARD,OUTPUT
--m recent --set;=;OK
+-m recent --set;-m recent --set --name DEFAULT --mask 255.255.255.255 --rsource;OK
 -m recent --rcheck --hitcount 8 --name foo --mask 255.255.255.255 --rsource;=;OK
 -m recent --rcheck --hitcount 12 --name foo --mask 255.255.255.255 --rsource;=;OK
--m recent --update --rttl;=;OK
+-m recent --update --rttl;-m recent --update --rttl --name DEFAULT --mask 255.255.255.255 --rsource;OK
 -m recent --set --rttl;;FAIL
 -m recent --rcheck --hitcount 999 --name foo --mask 255.255.255.255 --rsource;;FAIL
 # nonsensical, but all should load successfully:
Signature · 1 line
2.34.1
author reviewer
#9 lore raw

PATCH v2 9/12 tests: libxt_tos.t, libxt_TOS.t: Add missing masks in output

In reply to Phil Sutter

Mask differs between numeric --set-tos values and symbolic ones, make sure it is covered by the tests.

---
 extensions/libxt_TOS.t | 12 ++++++------
 extensions/libxt_tos.t |  8 ++++----
 2 files changed, 10 insertions(+), 10 deletions(-)
extensions/libxt_TOS.t +6 -6
index ae8531cc582e2..9f8e33fda7116 100644
@@ -1,15 +1,15 @@
 :PREROUTING,INPUT,FORWARD,OUTPUT,POSTROUTING
 *mangle
--j TOS --set-tos 0x1f;=;OK
+-j TOS --set-tos 0x1f;-j TOS --set-tos 0x1f/0xff;OK
 -j TOS --set-tos 0x1f/0x1f;=;OK
 # maximum TOS is 0x1f (5 bits)
 # ERROR: should fail: iptables -A PREROUTING -t mangle -j TOS --set-tos 0xff
 # -j TOS --set-tos 0xff;;FAIL
--j TOS --set-tos Minimize-Delay;-j TOS --set-tos 0x10;OK
--j TOS --set-tos Maximize-Throughput;-j TOS --set-tos 0x08;OK
--j TOS --set-tos Maximize-Reliability;-j TOS --set-tos 0x04;OK
--j TOS --set-tos Minimize-Cost;-j TOS --set-tos 0x02;OK
--j TOS --set-tos Normal-Service;-j TOS --set-tos 0x00;OK
+-j TOS --set-tos Minimize-Delay;-j TOS --set-tos 0x10/0x3f;OK
+-j TOS --set-tos Maximize-Throughput;-j TOS --set-tos 0x08/0x3f;OK
+-j TOS --set-tos Maximize-Reliability;-j TOS --set-tos 0x04/0x3f;OK
+-j TOS --set-tos Minimize-Cost;-j TOS --set-tos 0x02/0x3f;OK
+-j TOS --set-tos Normal-Service;-j TOS --set-tos 0x00/0x3f;OK
 -j TOS --and-tos 0x12;-j TOS --set-tos 0x00/0xed;OK
 -j TOS --or-tos 0x12;-j TOS --set-tos 0x12/0x12;OK
 -j TOS --xor-tos 0x12;-j TOS --set-tos 0x12/0x00;OK
extensions/libxt_tos.t +4 -4
index ccbe80099bdd9..6cceeeb4adbd8 100644
@@ -4,10 +4,10 @@
 -m tos --tos Maximize-Reliability;-m tos --tos 0x04/0x3f;OK
 -m tos --tos Minimize-Cost;-m tos --tos 0x02/0x3f;OK
 -m tos --tos Normal-Service;-m tos --tos 0x00/0x3f;OK
--m tos --tos 0xff;=;OK
--m tos ! --tos 0xff;=;OK
--m tos --tos 0x00;=;OK
--m tos --tos 0x0f;=;OK
+-m tos --tos 0xff;-m tos --tos 0xff/0xff;OK
+-m tos ! --tos 0xff;-m tos ! --tos 0xff/0xff;OK
+-m tos --tos 0x00;-m tos --tos 0x00/0xff;OK
+-m tos --tos 0x0f;-m tos --tos 0x0f/0xff;OK
 -m tos --tos 0x0f/0x0f;=;OK
 -m tos --tos wrong;;FAIL
 -m tos;;FAIL
Signature · 1 line
2.34.1
author reviewer
#10 lore raw

PATCH v2 10/12 tests: libebt_vlan.t: Drop trailing whitespace from rules

In reply to Phil Sutter

Fast iptables-test.py mode is picky and it has to: Plain redirect target prints a trailing whitespace, generally stripping the rules in test cases won't work therefore.

---
 extensions/libebt_vlan.t | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
extensions/libebt_vlan.t +2 -2
index 81c795854fca0..3ec0559953331 100644
@@ -4,8 +4,8 @@
 -p 802_1Q --vlan-prio 1;=;OK
 -p 802_1Q --vlan-prio ! 1;=;OK
 -p 802_1Q --vlan-encap ip;-p 802_1Q --vlan-encap 0800 -j CONTINUE;OK
--p 802_1Q --vlan-encap 0800 ;=;OK
--p 802_1Q --vlan-encap ! 0800 ;=;OK
+-p 802_1Q --vlan-encap 0800;=;OK
+-p 802_1Q --vlan-encap ! 0800;=;OK
 -p 802_1Q --vlan-encap IPv6 ! --vlan-id 1;-p 802_1Q --vlan-id ! 1 --vlan-encap 86DD -j CONTINUE;OK
 -p 802_1Q --vlan-id ! 1 --vlan-encap 86DD;=;OK
 --vlan-encap ip;=;FAIL
Signature · 1 line
2.34.1
author reviewer
#11 lore raw

PATCH v2 11/12 tests: libxt_connlimit.t: Add missing default values

In reply to Phil Sutter

This extension always prints --connlimit-mask and (the default) --connlimit-saddr options.

---
 extensions/libxt_connlimit.t | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
extensions/libxt_connlimit.t +6 -6
index c7ea61e95fbc8..366cea745c653 100644
@@ -1,11 +1,11 @@
 :INPUT,FORWARD,OUTPUT
--m connlimit --connlimit-upto 0;=;OK
--m connlimit --connlimit-upto 4294967295;=;OK
--m connlimit --connlimit-upto 4294967296;;FAIL
+-m connlimit --connlimit-upto 0;-m connlimit --connlimit-upto 0 --connlimit-mask 32 --connlimit-saddr;OK
+-m connlimit --connlimit-upto 4294967295 --connlimit-mask 32 --connlimit-saddr;=;OK
+-m connlimit --connlimit-upto 4294967296 --connlimit-mask 32 --connlimit-saddr;;FAIL
 -m connlimit --connlimit-upto -1;;FAIL
--m connlimit --connlimit-above 0;=;OK
--m connlimit --connlimit-above 4294967295;=;OK
--m connlimit --connlimit-above 4294967296;;FAIL
+-m connlimit --connlimit-above 0;-m connlimit --connlimit-above 0 --connlimit-mask 32 --connlimit-saddr;OK
+-m connlimit --connlimit-above 4294967295 --connlimit-mask 32 --connlimit-saddr;=;OK
+-m connlimit --connlimit-above 4294967296 --connlimit-mask 32 --connlimit-saddr;;FAIL
 -m connlimit --connlimit-above -1;;FAIL
 -m connlimit --connlimit-upto 1 --conlimit-above 1;;FAIL
 -m connlimit --connlimit-above 10 --connlimit-saddr;-m connlimit --connlimit-above 10 --connlimit-mask 32 --connlimit-saddr;OK
Signature · 1 line
2.34.1
author reviewer
#12 lore raw

PATCH v2 12/12 tests: *.t: Add missing all-one's netmasks to expected output

In reply to Phil Sutter
---
 extensions/libip6t_NETMAP.t | 2 +-
 extensions/libipt_NETMAP.t  | 2 +-
 extensions/libxt_CONNMARK.t | 8 ++++----
 extensions/libxt_MARK.t     | 4 ++--
 4 files changed, 8 insertions(+), 8 deletions(-)
extensions/libip6t_NETMAP.t +1 -1
index 043562d261243..79d47bf649720 100644
@@ -1,4 +1,4 @@
 :PREROUTING,INPUT,OUTPUT,POSTROUTING
 *nat
 -j NETMAP --to dead::/64;=;OK
--j NETMAP --to dead::beef;=;OK
+-j NETMAP --to dead::beef;-j NETMAP --to dead::beef/128;OK
extensions/libipt_NETMAP.t +1 -1
index 31924b985cd6f..0de856f057d7d 100644
@@ -1,4 +1,4 @@
 :PREROUTING,INPUT,OUTPUT,POSTROUTING
 *nat
 -j NETMAP --to 1.2.3.0/24;=;OK
--j NETMAP --to 1.2.3.4;=;OK
+-j NETMAP --to 1.2.3.4;-j NETMAP --to 1.2.3.4/32;OK
extensions/libxt_CONNMARK.t +4 -4
index 79a838fefaa14..c9b2b4a504416 100644
@@ -1,7 +1,7 @@
 :PREROUTING,FORWARD,OUTPUT,POSTROUTING
 *mangle
--j CONNMARK --restore-mark;=;OK
--j CONNMARK --save-mark;=;OK
--j CONNMARK --save-mark --nfmask 0xfffffff --ctmask 0xffffffff;-j CONNMARK --save-mark;OK
--j CONNMARK --restore-mark --nfmask 0xfffffff --ctmask 0xffffffff;-j CONNMARK --restore-mark;OK
+-j CONNMARK --restore-mark;-j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff;OK
+-j CONNMARK --save-mark;-j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff;OK
+-j CONNMARK --save-mark --nfmask 0xfffffff --ctmask 0xffffffff;=;OK
+-j CONNMARK --restore-mark --nfmask 0xfffffff --ctmask 0xffffffff;=;OK
 -j CONNMARK;;FAIL
extensions/libxt_MARK.t +2 -2
index 2902a14f06742..ae026dbbce59f 100644
@@ -1,7 +1,7 @@
 :INPUT,FORWARD,OUTPUT
 -j MARK --set-xmark 0xfeedcafe/0xfeedcafe;=;OK
--j MARK --set-xmark 0x0;=;OK
--j MARK --set-xmark 4294967295;-j MARK --set-xmark 0xffffffff;OK
+-j MARK --set-xmark 0x0;-j MARK --set-xmark 0x0/0xffffffff;OK
+-j MARK --set-xmark 4294967295;-j MARK --set-xmark 0xffffffff/0xffffffff;OK
 -j MARK --set-xmark 4294967296;;FAIL
 -j MARK --set-xmark -1;;FAIL
 -j MARK;;FAIL
Signature · 1 line
2.34.1
author reviewer
#13 lore raw
In reply to Phil Sutter

On Wed, Oct 12, 2022 at 05:17:50PM +0200, Phil Sutter wrote:
[...]

Quoted 8 lines

I had this in mind for a while now and finally got around to do it: When testing an extensions/*.t file with iptables-tests.py, act in a "batch" mode applying all rules at once and checking the expected output in one go, thereby reducing the overhead per test file to a single iptables-restore and iptables-save call each. This was a bit optimistic, but the result is still significant - on my rather slow testing VM, a full iptables-tests.py run completes in ~7min instead of ~30min (yes, it's slow).

FTR: I tested this once more on a not entirely broken machine which managed to complete a full testrun in 3min (i.e. a tenth of my VM). With this patch applied, the same run took merely 40s - so still significant.

Series applied meanwhile.

Cheers, Phil