Speed up iptables-tests.py
Phil Sutter
reviewercover Speed up iptables-tests.py
I had this in mind for a while now and finally got around to do it: When testing an extensions/*.t file with iptables-tests.py, act in a "batch" mode applying all rules at once and checking the expected output in one go, thereby reducing the overhead per test file to a single iptables-restore and iptables-save call each. This was a bit optimistic, but the result is still significant - on my rather slow testing VM, a full iptables-tests.py run completes in ~7min instead of ~30min (yes, it's slow).
See patch 1 for the implementation details. As a side-effect, rule existence checking became much stricter, so the remaining patches in this series deal with eliminating those differences:
* Patch 2 avoids having to add '-j CONTINUE' to almost all ebtables rules.
* Patches 3-10 adjust expected output to reality, mostly adding content the script didn't care about since the old 'output.find(<rule>)' worked fine as long as the output *started* like '<rule>'.
* Patch 11 Changes output by omitting an obvious default value, so a real functional change.
* Patch 12 drops another default value (from NFQUEUE target) I'm not sure we should keep.
So patches are roughly sorted by my confidence in correctness. Please have (at least) a close look at the last two, I don't want to break iptables for anyone just to keep test files small.
Phil Sutter (12):
tests: iptables-test: Implement fast test mode
tests: iptables-test: Cover for obligatory -j CONTINUE in ebtables
tests: *.t: Fix expected output for simple calls
tests: *.t: Fix for hexadecimal output
tests: libebt_redirect.t: Plain redirect prints with trailing
whitespace
tests: libxt_length.t: Fix odd use-case output
tests: libxt_recent.t: Add missing default values
tests: libxt_tos.t, libxt_TOS.t: Add missing masks in output
tests: libebt_vlan.t: Drop trailing whitespace from rules
tests: libxt_connlimit.t: Add missing --connlimit-saddr
extensions: Do not print all-one's netmasks
extensions: NFQUEUE: Do not print default queue number 0
extensions/libebt_log.t | 2 +- extensions/libebt_nflog.t | 2 +- extensions/libebt_redirect.t | 2 +- extensions/libebt_vlan.t | 4 +- extensions/libip6t_NETMAP.c | 2 +- extensions/libip6t_REJECT.t | 2 +- extensions/libipt_NETMAP.c | 2 +- extensions/libipt_REJECT.t | 2 +- extensions/libxt_CONNMARK.c | 32 ++++-- extensions/libxt_CONNMARK.t | 4 +- extensions/libxt_DSCP.t | 2 +- extensions/libxt_MARK.c | 4 +- extensions/libxt_MARK.t | 2 +- extensions/libxt_NFQUEUE.c | 27 ++--- extensions/libxt_NFQUEUE.t | 4 +- extensions/libxt_TOS.t | 12 +-- extensions/libxt_connlimit.c | 8 +- extensions/libxt_connlimit.t | 20 ++-- extensions/libxt_connmark.t | 4 +- extensions/libxt_dscp.t | 2 +- extensions/libxt_length.t | 2 +- extensions/libxt_mark.t | 2 +- extensions/libxt_recent.c | 45 ++++---- extensions/libxt_recent.t | 14 +-- extensions/libxt_tos.t | 8 +- iptables-test.py | 200 ++++++++++++++++++++++++++++++++++- 26 files changed, 317 insertions(+), 93 deletions(-)
Signature · 1 line
2.34.1
Phil Sutter
reviewerPATCH 1/12 tests: iptables-test: Implement fast test mode
Implement a faster mode of operation for suitable test files:
1) Collect all rules to add and all expected output in lists
2) Any supposedly failing rules are checked immediately like in slow mode.
3) Create and load iptables-restore input from the list in (1)
5) Construct the expected iptables-save output from (1) and check it in a single search
5) If any of the steps above fail, fall back to slow mode for verification and detailed error analysis. Fast mode failures are not fatal, merely warn about them.
To keep things simple (and feasible), avoid complicated test files involving external commands, multiple tables or variant-specific results.
Aside from speeding up testsuite run-time, rule searching has become more strict since EOL char is practically part of the search string. This revealed many false positives where the expected string was actually a substring of the printed rule.
--- iptables-test.py | 197 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 196 insertions(+), 1 deletion(-)
iptables-test.py +196 -1
@@ -28,6 +28,11 @@ EBTABLES_SAVE = "ebtables-save" #IPTABLES_SAVE = ['xtables-save','-4'] #IP6TABLES_SAVE = ['xtables-save','-6'] +IPTABLES_RESTORE = "iptables-restore" +IP6TABLES_RESTORE = "ip6tables-restore" +ARPTABLES_RESTORE = "arptables-restore" +EBTABLES_RESTORE = "ebtables-restore" + EXTENSIONS_PATH = "extensions" LOGFILE="/tmp/iptables-test.log" log_file = None
@@ -222,6 +227,185 @@ STDERR_IS_TTY = sys.stderr.isatty() return alt_res return res_inverse[res] +def fast_run_possible(filename): + ''' + Keep things simple, run only for simple test files: + - no external commands + - no multiple tables + - no variant-specific results + ''' + table = None + rulecount = 0 + for line in open(filename): + if line[0] in ["#", ":"] or len(line.strip()) == 0: + continue + if line[0] == "*": + if table or rulecount > 0: + return False + table = line.rstrip()[1:] + if line[0] in ["@", "%"]: + return False + if len(line.split(";")) > 3: + return False + rulecount += 1 + + return True + +def run_test_file_fast(filename, netns): + ''' + Run a test file, but fast + + :param filename: name of the file with the test rules + :param netns: network namespace to perform test run in + ''' + if "libipt_" in filename: + iptables = IPTABLES + iptables_save = IPTABLES_SAVE + iptables_restore = IPTABLES_RESTORE + elif "libip6t_" in filename: + iptables = IP6TABLES + iptables_save = IP6TABLES_SAVE + iptables_restore = IP6TABLES_RESTORE + elif "libxt_" in filename: + iptables = IPTABLES + iptables_save = IPTABLES_SAVE + iptables_restore = IPTABLES_RESTORE + elif "libarpt_" in filename: + # only supported with nf_tables backend + if EXECUTABLE != "xtables-nft-multi": + return 0 + iptables = ARPTABLES + iptables_save = ARPTABLES_SAVE + iptables_restore = ARPTABLES_RESTORE + elif "libebt_" in filename: + # only supported with nf_tables backend + if EXECUTABLE != "xtables-nft-multi": + return 0 + iptables = EBTABLES + iptables_save = EBTABLES_SAVE + iptables_restore = EBTABLES_RESTORE + else: + # default to iptables if not known prefix + iptables = IPTABLES + iptables_save = IPTABLES_SAVE + iptables_restore = IPTABLES_RESTORE + + f = open(filename) + + rules = {} + table = "filter" + chain_array = [] + tests = 0 + + for lineno, line in enumerate(f): + if line[0] == "#" or len(line.strip()) == 0: + continue + + if line[0] == "*": + table = line.rstrip()[1:] + continue + + if line[0] == ":": + chain_array = line.rstrip()[1:].split(",") + continue + + if len(chain_array) == 0: + return -1 + + tests += 1 + + for chain in chain_array: + item = line.split(";") + rule = chain + " " + item[0] + + if item[1] == "=": + rule_save = chain + " " + item[0] + else: + rule_save = chain + " " + item[1] + + res = item[2].rstrip() + if res != "OK": + rule = chain + " -t " + table + " " + item[0] + ret = run_test(iptables, rule, rule_save, + res, filename, lineno + 1, netns) + + if ret < 0: + return -1 + continue + + if not chain in rules.keys(): + rules[chain] = [] + rules[chain].append((rule, rule_save)) + + restore_data = ["*" + table] + out_expect = [] + for chain in ["PREROUTING", "INPUT", "FORWARD", "OUTPUT", "POSTROUTING"]: + if not chain in rules.keys(): + continue + for rule in rules[chain]: + restore_data.append("-A " + rule[0]) + out_expect.append("-A " + rule[1]) + restore_data.append("COMMIT") + + out_expect = "\n".join(out_expect) + + # load all rules via iptables_restore + + command = EXECUTABLE + " " + iptables_restore + if netns: + command = "ip netns exec " + netns + " " + command + + for line in restore_data: + print(iptables_restore + ": " + line, file=log_file) + + proc = subprocess.Popen(command, shell = True, text = True, + stdin = subprocess.PIPE, + stdout = subprocess.PIPE, + stderr = subprocess.PIPE) + restore_data = "\n".join(restore_data) + "\n" + out, err = proc.communicate(input = restore_data) + + if proc.returncode == -11: + reason = "iptables-restore segfaults: " + cmd + print_error(reason, filename, lineno) + return -1 + + if proc.returncode != 0: + print("%s returned %d: %s" % (iptables_restore, proc.returncode, err), + file=log_file) + return -1 + + # find all rules in iptables_save output + + command = EXECUTABLE + " " + iptables_save + if netns: + command = "ip netns exec " + netns + " " + command + + proc = subprocess.Popen(command, shell = True, + stdin = subprocess.PIPE, + stdout = subprocess.PIPE, + stderr = subprocess.PIPE) + out, err = proc.communicate() + + if proc.returncode == -11: + reason = "iptables-save segfaults: " + cmd + print_error(reason, filename, lineno) + return -1 + + cmd = iptables + " -F -t " + table + execute_cmd(cmd, filename, 0, netns) + + out = out.decode('utf-8').rstrip() + if out.find(out_expect) < 0: + print("dumps differ!", file=log_file) + print("\n".join(["expect: " + l for l in out_expect.split("\n")]), file=log_file) + print("\n".join(["got: " + l + for l in out.split("\n") + if not l[0] in ['*', ':', '#']]), + file=log_file) + return -1 + + return tests def run_test_file(filename, netns): '''
@@ -236,6 +420,14 @@ STDERR_IS_TTY = sys.stderr.isatty() if not filename.endswith(".t"): return 0, 0 + fast_failed = False + if fast_run_possible(filename): + tests = run_test_file_fast(filename, netns) + if tests > 0: + print(filename + ": " + maybe_colored('green', "OK", STDOUT_IS_TTY)) + return tests, tests + fast_failed = True + if "libipt_" in filename: iptables = IPTABLES elif "libip6t_" in filename:
@@ -330,7 +522,10 @@ STDERR_IS_TTY = sys.stderr.isatty() if netns: execute_cmd("ip netns del " + netns, filename) if total_test_passed: - print(filename + ": " + maybe_colored('green', "OK", STDOUT_IS_TTY)) + suffix = "" + if fast_failed: + suffix = maybe_colored('red', " but fast mode failed!", STDOUT_IS_TTY) + print(filename + ": " + maybe_colored('green', "OK", STDOUT_IS_TTY) + suffix) f.close() return tests, passed
Signature · 1 line
2.34.1
Re: [iptables PATCH 01/12] tests: iptables-test: Implement fast test mode
iptables-test.py STDERR_IS_TTY = sys.stderr.isatty() · line 276
Quoted 6 lines · Phil Sutter
iptables-test.py +5
-0+def run_test_file_fast(filename, netns): + elif "libarpt_" in filename: + # only supported with nf_tables backend + if EXECUTABLE != "xtables-nft-multi": + return 0
Should this particular check for executable be part of fast_run_possible instead? (Or somewhere else completely - if not running under x-n-m, even slow mode is not possible ;)
Phil Sutter
reviewerRe: [iptables PATCH 01/12] tests: iptables-test: Implement fast test mode
Quoted 9 lines · Jan Engelhardt
iptables-test.py STDERR_IS_TTY = sys.stderr.isatty() · line 276
Quoted 6 lines · Phil Sutter
iptables-test.py +5
-0+def run_test_file_fast(filename, netns): + elif "libarpt_" in filename: + # only supported with nf_tables backend + if EXECUTABLE != "xtables-nft-multi": + return 0
Quoted 3 lines
Should this particular check for executable be part of
fast_run_possibleinstead? (Or somewhere else completely - if not running under x-n-m, even slow mode is not possible ;)
Ah, you caught me c'n'p programming. ;)
I'll move the run_test_file_fast() call to after the same code in run_test_file() and pass 'iptables' variable as parameter. The -save and -restore commands may be constructed by simply appending the suffix.
Thanks, Phil
Phil Sutter
reviewerPATCH 2/12 tests: iptables-test: Cover for obligatory -j CONTINUE in ebtables
In reply to Phil SutterUnlike iptables, ebtables includes the default rule target in output. Instead of adding it to every rule in ebtables tests, add special casing to the testscript checking if the expected rule output contains a target already and adding the default one if not.
--- iptables-test.py | 3 +++ 1 file changed, 3 insertions(+)
iptables-test.py +3 -0
@@ -323,6 +323,9 @@ STDERR_IS_TTY = sys.stderr.isatty() else: rule_save = chain + " " + item[1] + if iptables == EBTABLES and rule_save.find('-j') < 0: + rule_save += " -j CONTINUE" + res = item[2].rstrip() if res != "OK": rule = chain + " -t " + table + " " + item[0]
Signature · 1 line
2.34.1
Phil Sutter
reviewerPATCH 3/12 tests: *.t: Fix expected output for simple calls
In reply to Phil SutterThese minimal extension uses print in more detailed form. Track this, the output is desired.
--- extensions/libebt_log.t | 2 +- extensions/libebt_nflog.t | 2 +- extensions/libip6t_REJECT.t | 2 +- extensions/libipt_REJECT.t | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-)
extensions/libebt_log.t +1 -1
@@ -1,5 +1,5 @@ :INPUT,FORWARD,OUTPUT ---log;=;OK +--log;--log-level notice;OK --log-level crit;=;OK --log-level 1;--log-level alert;OK --log-level emerg --log-ip --log-arp --log-ip6;=;OK
extensions/libebt_nflog.t +1 -1
@@ -1,5 +1,5 @@ :INPUT,FORWARD,OUTPUT ---nflog;=;OK +--nflog;--nflog-group 1;OK --nflog-group 42;=;OK --nflog-range 42;--nflog-group 1 --nflog-range 42 -j CONTINUE;OK --nflog-threshold 100 --nflog-prefix foo;--nflog-prefix "foo" --nflog-group 1 --nflog-threshold 100 -j CONTINUE;OK
extensions/libip6t_REJECT.t +1 -1
@@ -1,5 +1,5 @@ :INPUT,FORWARD,OUTPUT --j REJECT;=;OK +-j REJECT;-j REJECT --reject-with icmp6-port-unreachable;OK # manpage for IPv6 variant of REJECT does not show up for some reason? -j REJECT --reject-with icmp6-no-route;=;OK -j REJECT --reject-with icmp6-adm-prohibited;=;OK
extensions/libipt_REJECT.t +1 -1
@@ -1,5 +1,5 @@ :INPUT,FORWARD,OUTPUT --j REJECT;=;OK +-j REJECT;-j REJECT --reject-with icmp-port-unreachable;OK -j REJECT --reject-with icmp-net-unreachable;=;OK -j REJECT --reject-with icmp-host-unreachable;=;OK -j REJECT --reject-with icmp-port-unreachable;=;OK
Signature · 1 line
2.34.1
Phil Sutter
reviewerPATCH 4/12 tests: *.t: Fix for hexadecimal output
In reply to Phil SutterUse hex input to avoid having to specify an expected output in trivial cases.
--- extensions/libxt_DSCP.t | 2 +- extensions/libxt_MARK.t | 2 +- extensions/libxt_connmark.t | 4 ++-- extensions/libxt_dscp.t | 2 +- extensions/libxt_mark.t | 2 +- 5 files changed, 6 insertions(+), 6 deletions(-)
extensions/libxt_DSCP.t +1 -1
@@ -1,6 +1,6 @@ :PREROUTING,INPUT,FORWARD,OUTPUT,POSTROUTING *mangle --j DSCP --set-dscp 0;=;OK +-j DSCP --set-dscp 0x00;=;OK -j DSCP --set-dscp 0x3f;=;OK -j DSCP --set-dscp -1;;FAIL -j DSCP --set-dscp 0x40;;FAIL
extensions/libxt_MARK.t +1 -1
@@ -1,6 +1,6 @@ :INPUT,FORWARD,OUTPUT -j MARK --set-xmark 0xfeedcafe/0xfeedcafe;=;OK --j MARK --set-xmark 0;=;OK +-j MARK --set-xmark 0x0;=;OK -j MARK --set-xmark 4294967295;-j MARK --set-xmark 0xffffffff;OK -j MARK --set-xmark 4294967296;;FAIL -j MARK --set-xmark -1;;FAIL
extensions/libxt_connmark.t +2 -2
@@ -2,8 +2,8 @@ *mangle -m connmark --mark 0xffffffff;=;OK -m connmark --mark 0xffffffff/0xffffffff;-m connmark --mark 0xffffffff;OK --m connmark --mark 0xffffffff/0;=;OK --m connmark --mark 0/0xffffffff;-m connmark --mark 0;OK +-m connmark --mark 0xffffffff/0x0;=;OK +-m connmark --mark 0/0xffffffff;-m connmark --mark 0x0;OK -m connmark --mark -1;;FAIL -m connmark --mark 0xfffffffff;;FAIL -m connmark;;FAIL
extensions/libxt_dscp.t +1 -1
@@ -1,5 +1,5 @@ :INPUT,FORWARD,OUTPUT --m dscp --dscp 0;=;OK +-m dscp --dscp 0x00;=;OK -m dscp --dscp 0x3f;=;OK -m dscp --dscp -1;;FAIL -m dscp --dscp 0x40;;FAIL
extensions/libxt_mark.t +1 -1
@@ -1,6 +1,6 @@ :INPUT,FORWARD,OUTPUT -m mark --mark 0xfeedcafe/0xfeedcafe;=;OK --m mark --mark 0;=;OK +-m mark --mark 0x0;=;OK -m mark --mark 4294967295;-m mark --mark 0xffffffff;OK -m mark --mark 4294967296;;FAIL -m mark --mark -1;;FAIL
Signature · 1 line
2.34.1
Phil Sutter
reviewerPATCH 5/12 tests: libebt_redirect.t: Plain redirect prints with trailing whitespace
In reply to Phil Sutter--- extensions/libebt_redirect.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
extensions/libebt_redirect.t +1 -1
@@ -1,4 +1,4 @@ :PREROUTING *nat --j redirect;=;OK +-j redirect ;=;OK -j redirect --redirect-target RETURN;=;OK
Signature · 1 line
2.34.1
Phil Sutter
reviewerPATCH 6/12 tests: libxt_length.t: Fix odd use-case output
In reply to Phil SutterSpecifying the lower boundary suffixed by colon is an undocumented feature. Explicitly printing the upper boundary in that case seems sane.
--- extensions/libxt_length.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
extensions/libxt_length.t +1 -1
@@ -2,7 +2,7 @@ -m length --length 1;=;OK -m length --length :2;-m length --length 0:2;OK -m length --length 0:3;=;OK --m length --length 4:;=;OK +-m length --length 4:;-m length --length 4:65535;OK -m length --length 0:65535;=;OK -m length ! --length 0:65535;=;OK -m length --length 0:65536;;FAIL
Signature · 1 line
2.34.1
Phil Sutter
reviewerPATCH 7/12 tests: libxt_recent.t: Add missing default values
In reply to Phil Sutter--- extensions/libxt_recent.t | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
extensions/libxt_recent.t +2 -2
@@ -1,8 +1,8 @@ :INPUT,FORWARD,OUTPUT --m recent --set;=;OK +-m recent --set;-m recent --set --name DEFAULT --rsource;OK -m recent --rcheck --hitcount 8 --name foo --mask 255.255.255.255 --rsource;=;OK -m recent --rcheck --hitcount 12 --name foo --mask 255.255.255.255 --rsource;=;OK --m recent --update --rttl;=;OK +-m recent --update --rttl;-m recent --update --rttl --name DEFAULT --mask 255.255.255.255 --rsource;OK -m recent --set --rttl;;FAIL -m recent --rcheck --hitcount 999 --name foo --mask 255.255.255.255 --rsource;;FAIL # nonsensical, but all should load successfully:
Signature · 1 line
2.34.1
Phil Sutter
reviewerPATCH 8/12 tests: libxt_tos.t, libxt_TOS.t: Add missing masks in output
In reply to Phil SutterMask differs between numeric --set-tos values and symbolic ones, make sure it is covered by the tests.
--- extensions/libxt_TOS.t | 12 ++++++------ extensions/libxt_tos.t | 8 ++++---- 2 files changed, 10 insertions(+), 10 deletions(-)
extensions/libxt_TOS.t +6 -6
@@ -1,15 +1,15 @@ :PREROUTING,INPUT,FORWARD,OUTPUT,POSTROUTING *mangle --j TOS --set-tos 0x1f;=;OK +-j TOS --set-tos 0x1f;-j TOS --set-tos 0x1f/0xff;OK -j TOS --set-tos 0x1f/0x1f;=;OK # maximum TOS is 0x1f (5 bits) # ERROR: should fail: iptables -A PREROUTING -t mangle -j TOS --set-tos 0xff # -j TOS --set-tos 0xff;;FAIL --j TOS --set-tos Minimize-Delay;-j TOS --set-tos 0x10;OK --j TOS --set-tos Maximize-Throughput;-j TOS --set-tos 0x08;OK --j TOS --set-tos Maximize-Reliability;-j TOS --set-tos 0x04;OK --j TOS --set-tos Minimize-Cost;-j TOS --set-tos 0x02;OK --j TOS --set-tos Normal-Service;-j TOS --set-tos 0x00;OK +-j TOS --set-tos Minimize-Delay;-j TOS --set-tos 0x10/0x3f;OK +-j TOS --set-tos Maximize-Throughput;-j TOS --set-tos 0x08/0x3f;OK +-j TOS --set-tos Maximize-Reliability;-j TOS --set-tos 0x04/0x3f;OK +-j TOS --set-tos Minimize-Cost;-j TOS --set-tos 0x02/0x3f;OK +-j TOS --set-tos Normal-Service;-j TOS --set-tos 0x00/0x3f;OK -j TOS --and-tos 0x12;-j TOS --set-tos 0x00/0xed;OK -j TOS --or-tos 0x12;-j TOS --set-tos 0x12/0x12;OK -j TOS --xor-tos 0x12;-j TOS --set-tos 0x12/0x00;OK
extensions/libxt_tos.t +4 -4
@@ -4,10 +4,10 @@ -m tos --tos Maximize-Reliability;-m tos --tos 0x04/0x3f;OK -m tos --tos Minimize-Cost;-m tos --tos 0x02/0x3f;OK -m tos --tos Normal-Service;-m tos --tos 0x00/0x3f;OK --m tos --tos 0xff;=;OK --m tos ! --tos 0xff;=;OK --m tos --tos 0x00;=;OK --m tos --tos 0x0f;=;OK +-m tos --tos 0xff;-m tos --tos 0xff/0xff;OK +-m tos ! --tos 0xff;-m tos ! --tos 0xff/0xff;OK +-m tos --tos 0x00;-m tos --tos 0x00/0xff;OK +-m tos --tos 0x0f;-m tos --tos 0x0f/0xff;OK -m tos --tos 0x0f/0x0f;=;OK -m tos --tos wrong;;FAIL -m tos;;FAIL
Signature · 1 line
2.34.1
Phil Sutter
reviewerPATCH 9/12 tests: libebt_vlan.t: Drop trailing whitespace from rules
In reply to Phil SutterFast iptables-test.py mode is picky and it has to: Plain redirect target prints a trailing whitespace, generally stripping the rules in test cases won't work therefore.
--- extensions/libebt_vlan.t | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
extensions/libebt_vlan.t +2 -2
@@ -4,8 +4,8 @@ -p 802_1Q --vlan-prio 1;=;OK -p 802_1Q --vlan-prio ! 1;=;OK -p 802_1Q --vlan-encap ip;-p 802_1Q --vlan-encap 0800 -j CONTINUE;OK --p 802_1Q --vlan-encap 0800 ;=;OK --p 802_1Q --vlan-encap ! 0800 ;=;OK +-p 802_1Q --vlan-encap 0800;=;OK +-p 802_1Q --vlan-encap ! 0800;=;OK -p 802_1Q --vlan-encap IPv6 ! --vlan-id 1;-p 802_1Q --vlan-id ! 1 --vlan-encap 86DD -j CONTINUE;OK -p 802_1Q --vlan-id ! 1 --vlan-encap 86DD;=;OK --vlan-encap ip;=;FAIL
Signature · 1 line
2.34.1
Phil Sutter
reviewerPATCH 10/12 tests: libxt_connlimit.t: Add missing --connlimit-saddr
In reply to Phil SutterThis default option is included in output and it is not intuitive enough to omit it. So extend the test cases to cover for it.
--- extensions/libxt_connlimit.t | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
extensions/libxt_connlimit.t +6 -6
@@ -1,11 +1,11 @@ :INPUT,FORWARD,OUTPUT --m connlimit --connlimit-upto 0;=;OK --m connlimit --connlimit-upto 4294967295;=;OK --m connlimit --connlimit-upto 4294967296;;FAIL +-m connlimit --connlimit-upto 0;-m connlimit --connlimit-upto 0 --connlimit-saddr;OK +-m connlimit --connlimit-upto 4294967295 --connlimit-saddr;=;OK +-m connlimit --connlimit-upto 4294967296 --connlimit-saddr;;FAIL -m connlimit --connlimit-upto -1;;FAIL --m connlimit --connlimit-above 0;=;OK --m connlimit --connlimit-above 4294967295;=;OK --m connlimit --connlimit-above 4294967296;;FAIL +-m connlimit --connlimit-above 0;-m connlimit --connlimit-above 0 --connlimit-saddr;OK +-m connlimit --connlimit-above 4294967295 --connlimit-saddr;=;OK +-m connlimit --connlimit-above 4294967296 --connlimit-saddr;;FAIL -m connlimit --connlimit-above -1;;FAIL -m connlimit --connlimit-upto 1 --conlimit-above 1;;FAIL -m connlimit --connlimit-above 10 --connlimit-saddr;-m connlimit --connlimit-above 10 --connlimit-mask 32 --connlimit-saddr;OK
Signature · 1 line
2.34.1
Phil Sutter
reviewerPATCH 11/12 extensions: Do not print all-one's netmasks
In reply to Phil SutterAll one's netmasks are a trivial default, no point in printing them.
--- extensions/libip6t_NETMAP.c | 2 +- extensions/libipt_NETMAP.c | 2 +- extensions/libxt_CONNMARK.c | 32 +++++++++++++++++-------- extensions/libxt_CONNMARK.t | 4 ++-- extensions/libxt_MARK.c | 4 +++- extensions/libxt_connlimit.c | 8 +++++-- extensions/libxt_connlimit.t | 8 +++---- extensions/libxt_recent.c | 45 ++++++++++++++++++++---------------- extensions/libxt_recent.t | 12 +++++----- 9 files changed, 70 insertions(+), 47 deletions(-)
extensions/libip6t_NETMAP.c +1 -1
@@ -64,7 +64,7 @@ static void __NETMAP_print(const void *ip, const struct xt_entry_target *target, bits =xtables_ip6mask_to_cidr (&a);if (bits <0 )printf ("/%s" ,xtables_ip6addr_to_numeric (&a)); -else +else if (bits <sizeof (a) *8 )printf ("/%d" , bits); }
extensions/libipt_NETMAP.c +1 -1
@@ -76,7 +76,7 @@ static void __NETMAP_print(const void *ip, const struct xt_entry_target *target, bits =netmask2bits (a.s_addr);if (bits <0 )printf ("/%s" ,xtables_ipaddr_to_numeric (&a)); -else +else if (bits <sizeof (a) *8 )printf ("/%d" , bits); }
extensions/libxt_CONNMARK.c +22 -10
@@ -496,6 +496,12 @@ static void CONNMARK_init(struct xt_entry_target *t) markinfo->mask =0xffffffffUL ; } +static void print_mask_v1 (const char *pfx,__u32 mask) +{ +if (mask !=UINT32_MAX ) +printf ("%s0x%x" , pfx, mask); +} +static void connmark_tg_save (const void *ip,const struct xt_entry_target *target) {
@@ -503,15 +509,18 @@ connmark_tg_save(const void *ip, const struct xt_entry_target *target)switch (info->mode) {case XT_CONNMARK_SET : -printf (" --set-xmark 0x%x/0x%x" , info->ctmark, info->ctmask); +printf (" --set-xmark 0x%x" , info->ctmark); +print_mask_v1 ("/" , info->ctmask);break ;case XT_CONNMARK_SAVE : -printf (" --save-mark --nfmask 0x%x --ctmask 0x%x" , - info->nfmask, info->ctmask); +printf (" --save-mark" ); +print_mask_v1 (" --nfmask " , info->nfmask); +print_mask_v1 (" --ctmask " , info->ctmask);break ;case XT_CONNMARK_RESTORE : -printf (" --restore-mark --nfmask 0x%x --ctmask 0x%x" , - info->nfmask, info->ctmask); +printf (" --restore-mark" ); +print_mask_v1 (" --nfmask " , info->nfmask); +print_mask_v1 (" --ctmask " , info->ctmask);break ;default :printf (" ERROR: UNKNOWN CONNMARK MODE" );
@@ -527,15 +536,18 @@ connmark_tg_save_v2(const void *ip, const struct xt_entry_target *target)switch (info->mode) {case XT_CONNMARK_SET : -printf (" --set-xmark 0x%x/0x%x" , info->ctmark, info->ctmask); +printf (" --set-xmark 0x%x" , info->ctmark); +print_mask_v1 ("/" , info->ctmask);break ;case XT_CONNMARK_SAVE : -printf (" --save-mark --nfmask 0x%x --ctmask 0x%x" , - info->nfmask, info->ctmask); +printf (" --save-mark" ); +print_mask_v1 (" --nfmask " , info->nfmask); +print_mask_v1 (" --ctmask " , info->ctmask);break ;case XT_CONNMARK_RESTORE : -printf (" --restore-mark --nfmask 0x%x --ctmask 0x%x" , - info->nfmask, info->ctmask); +printf (" --restore-mark" ); +print_mask_v1 (" --nfmask " , info->nfmask); +print_mask_v1 (" --ctmask " , info->ctmask);break ;default :printf (" ERROR: UNKNOWN CONNMARK MODE" );
extensions/libxt_CONNMARK.t +2 -2
@@ -2,6 +2,6 @@ *mangle -j CONNMARK --restore-mark;=;OK -j CONNMARK --save-mark;=;OK --j CONNMARK --save-mark --nfmask 0xfffffff --ctmask 0xffffffff;-j CONNMARK --save-mark;OK --j CONNMARK --restore-mark --nfmask 0xfffffff --ctmask 0xffffffff;-j CONNMARK --restore-mark;OK +-j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff;-j CONNMARK --save-mark;OK +-j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff;-j CONNMARK --restore-mark;OK -j CONNMARK;;FAIL
extensions/libxt_MARK.c +3 -1
@@ -242,7 +242,9 @@ static void mark_tg_save(const void *ip, const struct xt_entry_target *target) {const struct xt_mark_tginfo2 *info = (const void *)target->data; -printf (" --set-xmark 0x%x/0x%x" , info->mark, info->mask); +printf (" --set-xmark 0x%x" , info->mark); +if (info->mask !=0xffffffffU ) +printf ("/0x%x" , info->mask); }static void mark_tg_arp_save (const void *ip,const struct xt_entry_target *target)
extensions/libxt_connlimit.c +6 -2
@@ -152,13 +152,15 @@ static void connlimit_print6(const void *ip,static void connlimit_save4 (const void *ip,const struct xt_entry_match *match) {const struct xt_connlimit_info *info = (const void *)match->data; +unsigned int bits =count_bits4 (info->v4_mask);const int revision = match->u.user.revision;if (info->flags &XT_CONNLIMIT_INVERT )printf (" --connlimit-upto %u" , info->limit);else printf (" --connlimit-above %u" , info->limit); -printf (" --connlimit-mask %u" ,count_bits4 (info->v4_mask)); +if (bits !=32 ) +printf (" --connlimit-mask %u" , bits);if (revision >=1 ) {if (info->flags &XT_CONNLIMIT_DADDR )printf (" --connlimit-daddr" );
@@ -170,13 +172,15 @@ static void connlimit_save4(const void *ip, const struct xt_entry_match *match)static void connlimit_save6 (const void *ip,const struct xt_entry_match *match) {const struct xt_connlimit_info *info = (const void *)match->data; +unsigned int bits =count_bits6 (info->v6_mask);const int revision = match->u.user.revision;if (info->flags &XT_CONNLIMIT_INVERT )printf (" --connlimit-upto %u" , info->limit);else printf (" --connlimit-above %u" , info->limit); -printf (" --connlimit-mask %u" ,count_bits6 (info->v6_mask)); +if (bits !=128 ) +printf (" --connlimit-mask %u" , bits);if (revision >=1 ) {if (info->flags &XT_CONNLIMIT_DADDR )printf (" --connlimit-daddr" );
extensions/libxt_connlimit.t +4 -4
@@ -8,9 +8,9 @@ -m connlimit --connlimit-above 4294967296 --connlimit-saddr;;FAIL -m connlimit --connlimit-above -1;;FAIL -m connlimit --connlimit-upto 1 --conlimit-above 1;;FAIL --m connlimit --connlimit-above 10 --connlimit-saddr;-m connlimit --connlimit-above 10 --connlimit-mask 32 --connlimit-saddr;OK --m connlimit --connlimit-above 10 --connlimit-daddr;-m connlimit --connlimit-above 10 --connlimit-mask 32 --connlimit-daddr;OK +-m connlimit --connlimit-above 10 --connlimit-saddr;=;OK +-m connlimit --connlimit-above 10 --connlimit-daddr;=;OK -m connlimit --connlimit-above 10 --connlimit-saddr --connlimit-daddr;;FAIL --m connlimit --connlimit-above 10 --connlimit-mask 32 --connlimit-saddr;=;OK --m connlimit --connlimit-above 10 --connlimit-mask 32 --connlimit-daddr;=;OK +-m connlimit --connlimit-above 10 --connlimit-mask 32 --connlimit-saddr;-m connlimit --connlimit-above 10 --connlimit-saddr;OK +-m connlimit --connlimit-above 10 --connlimit-mask 32 --connlimit-daddr;-m connlimit --connlimit-above 10 --connlimit-daddr;OK -m connlimit;;FAIL
extensions/libxt_recent.c +25 -20
63 lines not on this page · raw message
extensions/libxt_recent.t +6 -6
@@ -1,11 +1,11 @@ :INPUT,FORWARD,OUTPUT -m recent --set;-m recent --set --name DEFAULT --rsource;OK --m recent --rcheck --hitcount 8 --name foo --mask 255.255.255.255 --rsource;=;OK --m recent --rcheck --hitcount 12 --name foo --mask 255.255.255.255 --rsource;=;OK --m recent --update --rttl;-m recent --update --rttl --name DEFAULT --mask 255.255.255.255 --rsource;OK +-m recent --rcheck --hitcount 8 --name foo --mask 255.255.255.255 --rsource;-m recent --rcheck --hitcount 8 --name foo --rsource;OK +-m recent --rcheck --hitcount 12 --name foo --mask 255.255.255.255 --rsource;-m recent --rcheck --hitcount 12 --name foo --rsource;OK +-m recent --update --rttl;-m recent --update --rttl --name DEFAULT --rsource;OK -m recent --set --rttl;;FAIL -m recent --rcheck --hitcount 999 --name foo --mask 255.255.255.255 --rsource;;FAIL # nonsensical, but all should load successfully: --m recent --rcheck --hitcount 3 --name foo --mask 255.255.255.255 --rsource -m recent --rcheck --hitcount 4 --name foo --mask 255.255.255.255 --rsource;=;OK --m recent --rcheck --hitcount 4 --name foo --mask 255.255.255.255 --rsource -m recent --rcheck --hitcount 4 --name foo --mask 255.255.255.255 --rsource;=;OK --m recent --rcheck --hitcount 8 --name foo --mask 255.255.255.255 --rsource -m recent --rcheck --hitcount 12 --name foo --mask 255.255.255.255 --rsource;=;OK +-m recent --rcheck --hitcount 3 --name foo --rsource -m recent --rcheck --hitcount 4 --name foo --rsource;=;OK +-m recent --rcheck --hitcount 4 --name foo --rsource -m recent --rcheck --hitcount 4 --name foo --rsource;=;OK +-m recent --rcheck --hitcount 8 --name foo --rsource -m recent --rcheck --hitcount 12 --name foo --rsource;=;OK
Signature · 1 line
2.34.1
Re: [iptables PATCH 11/12] extensions: Do not print all-one's netmasks
extensions/libip6t_NETMAP.c static void __NETMAP_print(const void *ip, const struct xt_entry_target *target, · line 68
Quoted 9 lines · Phil Sutter
All one's netmasks are a trivial default, no point in printing them.
extensions/libip6t_NETMAP.c +1
-1@@ -64,7 +64,7 @@ static void __NETMAP_print(const void *ip, const struct xt_entry_target *target, bits =xtables_ip6mask_to_cidr (&a);if (bits <0 )printf ("/%s" ,xtables_ip6addr_to_numeric (&a)); -else +else if (bits <sizeof (a) *8 )printf ("/%d" , bits);
I would rather see it stay.
- iproute2 also always prints the /128 suffix
- test parsers need not special case the absence of /128
extensions/libxt_MARK.c static void mark_tg_save(const void *ip, const struct xt_entry_target *target) · line 247
Quoted 9 lines
extensions/libxt_MARK.c +3
-2--- a/extensions/libxt_MARK.c@@ -242,7 +242,9 @@ static void mark_tg_save(const void *ip, const struct xt_entry_target *target) {const struct xt_mark_tginfo2 *info = (const void *)target->data; -printf (" --set-xmark 0x%x/0x%x" , info->mark, info->mask); +printf (" --set-xmark 0x%x" , info->mark); +if (info->mask !=0xffffffffU ) +printf ("/0x%x" , info->mask);
if (info->mask != UINT32_MAX)
Phil Sutter
reviewerRe: [iptables PATCH 11/12] extensions: Do not print all-one's netmasks
Hi,
Quoted 5 lines · Jan Engelhardt
Quoted 2 lines · Phil Sutter
All one's netmasks are a trivial default, no point in printing them.
Quoted 11 lines
extensions/libip6t_NETMAP.c static void __NETMAP_print(const void *ip, const struct xt_entry_target *target, · line 68
Quoted 7 lines
extensions/libip6t_NETMAP.c +1
-1@@ -64,7 +64,7 @@ static void __NETMAP_print(const void *ip, const struct xt_entry_target *target, bits =xtables_ip6mask_to_cidr (&a);if (bits <0 )printf ("/%s" ,xtables_ip6addr_to_numeric (&a)); -else +else if (bits <sizeof (a) *8 )printf ("/%d" , bits);I would rather see it stay.
- iproute2 also always prints the /128 suffix
- test parsers need not special case the absence of /128
I get your point. Screen-scraping is also not uncommon among iptables users, so care has to be taken when "optimizing" output.
OTOH we're a bit inconsistent: xtables_ip(6)mask_to_numeric() explicitly omits output if it would print "/32" or "/128".
Maybe I'll just leave the code as-is and adjust only the test cases instead?
Quoted 11 lines
extensions/libxt_MARK.c static void mark_tg_save(const void *ip, const struct xt_entry_target *target) · line 247
Quoted 9 lines
extensions/libxt_MARK.c +3
-2--- a/extensions/libxt_MARK.c@@ -242,7 +242,9 @@ static void mark_tg_save(const void *ip, const struct xt_entry_target *target) {const struct xt_mark_tginfo2 *info = (const void *)target->data; -printf (" --set-xmark 0x%x/0x%x" , info->mark, info->mask); +printf (" --set-xmark 0x%x" , info->mark); +if (info->mask !=0xffffffffU ) +printf ("/0x%x" , info->mask);if (
info->mask!=UINT32_MAX)
ACK. I copied from mark_tg_print(), so that's a useful fix unrelated to the discussion above.
Thanks, Phil
Re: [iptables PATCH 11/12] extensions: Do not print all-one's netmasks
Quoted 11 lines · Phil Sutter
Quoted 8 lines
Quoted 3 lines
(quoted diff) +1
-1- else + else if (bits < sizeof(a) * 8)printf("/%d", bits);I would rather see it stay.
- iproute2 also always prints the /128 suffix
- test parsers need not special case the absence of /128 [as in, when something reads e.g. iptables-save output][...] OTOH we're a bit inconsistent: xtables_ip(6)mask_to_numeric() explicitly omits output if it would print "/32" or "/128".
We could consider just changing those two functions.
Any third-party program that interprets our output already has to deal with a "/N" suffix in the same spot anyway.
Phil Sutter
reviewerPATCH 12/12 extensions: NFQUEUE: Do not print default queue number 0
In reply to Phil SutterInstead make sure it is mentioned in help output.
--- extensions/libxt_NFQUEUE.c | 27 +++++++++++++++------------ extensions/libxt_NFQUEUE.t | 4 ++-- 2 files changed, 17 insertions(+), 14 deletions(-)
extensions/libxt_NFQUEUE.c +15 -12
@@ -24,7 +24,7 @@ static void NFQUEUE_help(void)printf ("NFQUEUE target options\n" " --queue-num value Send packet to QUEUE number <value>.\n" -" Valid queue numbers are 0-65535\n" +" Valid queue numbers are 0-65535 (default 0)\n" ); }
@@ -33,7 +33,7 @@ static void NFQUEUE_help_v1(void)printf ("NFQUEUE target options\n" " --queue-num value Send packet to QUEUE number <value>.\n" -" Valid queue numbers are 0-65535\n" +" Valid queue numbers are 0-65535 (default 0)\n" " --queue-balance first:last Balance flows between queues <value> to <value>.\n" ); }
@@ -42,7 +42,7 @@ static void NFQUEUE_help_v2(void)printf ("NFQUEUE target options\n" " --queue-num value Send packet to QUEUE number <value>.\n" -" Valid queue numbers are 0-65535\n" +" Valid queue numbers are 0-65535 (default 0)\n" " --queue-balance first:last Balance flows between queues <value> to <value>.\n" " --queue-bypass Bypass Queueing if no queue instance exists.\n" " --queue-cpu-fanout Use current CPU (no hashing)\n" );
@@ -53,7 +53,7 @@ static void NFQUEUE_help_v3(void)printf ("NFQUEUE target options\n" " --queue-num value Send packet to QUEUE number <value>.\n" -" Valid queue numbers are 0-65535\n" +" Valid queue numbers are 0-65535 (default 0)\n" " --queue-balance first:last Balance flows between queues <value> to <value>.\n" " --queue-bypass Bypass Queueing if no queue instance exists.\n" " --queue-cpu-fanout Use current CPU (no hashing)\n" );
@@ -157,7 +157,9 @@ static void NFQUEUE_print(const void *ip, {const struct xt_NFQ_info *tinfo = (const struct xt_NFQ_info *)target->data; -printf (" NFQUEUE num %u" , tinfo->queuenum); + +if (tinfo->queuenum) +printf (" NFQUEUE num %u" , tinfo->queuenum); } static void NFQUEUE_print_v1(const void *ip,
@@ -169,7 +171,7 @@ static void NFQUEUE_print_v1(const void *ip,if (last >1 ) { last += tinfo->queuenum -1 ;printf (" NFQUEUE balance %u:%u" , tinfo->queuenum, last); - }else { + }else if (tinfo->queuenum) {printf (" NFQUEUE num %u" , tinfo->queuenum); } }
@@ -183,7 +185,7 @@ static void NFQUEUE_print_v2(const void *ip,if (last >1 ) { last += info->queuenum -1 ;printf (" NFQUEUE balance %u:%u" , info->queuenum, last); - }else + }else if (info->queuenum)printf (" NFQUEUE num %u" , info->queuenum);if (info->bypass &NFQ_FLAG_BYPASS )
@@ -199,7 +201,7 @@ static void NFQUEUE_print_v3(const void *ip,if (last >1 ) { last += info->queuenum -1 ;printf (" NFQUEUE balance %u:%u" , info->queuenum, last); - }else + }else if (info->queuenum)printf (" NFQUEUE num %u" , info->queuenum);if (info->flags &NFQ_FLAG_BYPASS )
@@ -214,7 +216,8 @@ static void NFQUEUE_save(const void *ip, const struct xt_entry_target *target)const struct xt_NFQ_info *tinfo = (const struct xt_NFQ_info *)target->data; -printf (" --queue-num %u" , tinfo->queuenum); +if (tinfo->queuenum) +printf (" --queue-num %u" , tinfo->queuenum); }static void NFQUEUE_save_v1 (const void *ip,const struct xt_entry_target *target)
@@ -225,7 +228,7 @@ static void NFQUEUE_save_v1(const void *ip, const struct xt_entry_target *targetif (last >1 ) { last += tinfo->queuenum -1 ;printf (" --queue-balance %u:%u" , tinfo->queuenum, last); - }else { + }else if (tinfo->queuenum) {printf (" --queue-num %u" , tinfo->queuenum); } }
@@ -238,7 +241,7 @@ static void NFQUEUE_save_v2(const void *ip, const struct xt_entry_target *targetif (last >1 ) { last += info->queuenum -1 ;printf (" --queue-balance %u:%u" , info->queuenum, last); - }else + }else if (info->queuenum)printf (" --queue-num %u" , info->queuenum);if (info->bypass &NFQ_FLAG_BYPASS )
@@ -254,7 +257,7 @@ static void NFQUEUE_save_v3(const void *ip,if (last >1 ) { last += info->queuenum -1 ;printf (" --queue-balance %u:%u" , info->queuenum, last); - }else + }else if (info->queuenum)printf (" --queue-num %u" , info->queuenum);if (info->flags &NFQ_FLAG_BYPASS )
extensions/libxt_NFQUEUE.t +2 -2
@@ -1,6 +1,6 @@ :INPUT,FORWARD,OUTPUT -j NFQUEUE;=;OK --j NFQUEUE --queue-num 0;=;OK +-j NFQUEUE --queue-num 0;-j NFQUEUE;OK -j NFQUEUE --queue-num 65535;=;OK -j NFQUEUE --queue-num 65536;;FAIL -j NFQUEUE --queue-num -1;;FAIL
@@ -13,4 +13,4 @@ -j NFQUEUE --queue-balance 0:6 --queue-cpu-fanout --queue-bypass;-j NFQUEUE --queue-balance 0:6 --queue-bypass --queue-cpu-fanout;OK -j NFQUEUE --queue-bypass --queue-balance 0:6 --queue-cpu-fanout;-j NFQUEUE --queue-balance 0:6 --queue-bypass --queue-cpu-fanout;OK -j NFQUEUE --queue-balance 0:6 --queue-bypass;=;OK --j NFQUEUE --queue-bypass;-j NFQUEUE --queue-num 0 --queue-bypass;OK +-j NFQUEUE --queue-bypass;=;OK
Signature · 1 line
2.34.1