selftests: drv-net: convert so_txtime to drv-net
Subsystems (4)
Willem de Bruijn
maintainerv7 cover selftests: drv-net: convert so_txtime to drv-net
From: Willem de Bruijn <[email protected]>
In preparation for extending to pacing hardware offload, convert the so_txtime.sh test to a drv-net test that can be run against netdevsim and real hardware.
Two preparatory patches
1. support negative tests, where tests are expected to fail
2. add a tc helper
See individual patches for details and detailed changelog
Previous versions:
v6: https://lore.kernel.org/netdev/[email protected]/
v5: https://lore.kernel.org/netdev/[email protected]/
v4: https://lore.kernel.org/netdev/[email protected]/
v3: https://lore.kernel.org/netdev/[email protected]/
v2: https://lore.kernel.org/netdev/[email protected]/
v1: https://lore.kernel.org/netdev/[email protected]/
Willem de Bruijn (3):
selftests: net: py: support cmd verifying expected failure
selftests: net: py: add tc utility
selftests: drv-net: convert so_txtime to drv-net
.../testing/selftests/drivers/net/.gitignore | 1 +
tools/testing/selftests/drivers/net/Makefile | 2 +
tools/testing/selftests/drivers/net/config | 2 +
.../selftests/drivers/net/lib/py/__init__.py | 5 +-
.../selftests/{ => drivers}/net/so_txtime.c | 25 +++-
.../selftests/drivers/net/so_txtime.py | 96 +++++++++++++++
tools/testing/selftests/net/.gitignore | 1 -
tools/testing/selftests/net/Makefile | 2 -
.../testing/selftests/net/lib/py/__init__.py | 4 +-
tools/testing/selftests/net/lib/py/utils.py | 51 ++++++--
tools/testing/selftests/net/so_txtime.sh | 110 ------------------
11 files changed, 166 insertions(+), 133 deletions(-)
rename tools/testing/selftests/{ => drivers}/net/so_txtime.c (96%)
create mode 100755 tools/testing/selftests/drivers/net/so_txtime.py
delete mode 100755 tools/testing/selftests/net/so_txtime.shSignature · 1 line
2.54.0.545.g6539524ca2-goog
Willem de Bruijn
maintainerPATCH v7 1/3 selftests: net: py: support cmd verifying expected failure
From: Willem de Bruijn <[email protected]>
Support negative tests, where cmd raises an exception if the command succeeded.
Add optional argument expect_fail to cmd and bkg. Where fail fails the test on unexpected error, expect_fail fails it on unexpected success.
Both fail on negative return code. Python subprocess may set a negative return code on process crash or timeout. Those are never anticipated failures.
---
In this design fail and expect_fail are two complementary tests. If both are set to true, an exception would always be raised.
This is contrast to the previous approach, where fail enables fail_on_unexpected_returncode and expect_fail modifies what that unexpected_returncode value range is.
---
v6 -> v7
- convert from 'verify_failed' value for fail, to separate variable
v4 -> v5
- initial version of standalone patch
--- tools/testing/selftests/net/lib/py/utils.py | 39 +++++++++++++++------ 1 file changed, 29 insertions(+), 10 deletions(-)
tools/testing/selftests/net/lib/py/utils.py +29 -10
@@ -23,6 +23,10 @@ class CmdExitFailure(Exception): self.cmd = cmd_obj +class CmdExitZeroFailure(CmdExitFailure): + """ Command succeeded (returned zero exit code), but expected failure. """ + + def fd_read_timeout(fd, timeout): rlist, _, _ = select.select([fd], [], [], timeout) if rlist:
@@ -39,8 +43,9 @@ class cmd: Use bkg() instead to run a command in the background. """ - def __init__(self, comm, shell=None, fail=True, ns=None, background=False, - host=None, timeout=5, ksft_ready=None, ksft_wait=None): + def __init__(self, comm, shell=None, fail=True, expect_fail=False, ns=None, + background=False, host=None, timeout=5, ksft_ready=None, + ksft_wait=None): if ns: comm = f'ip netns exec {ns} ' + comm
@@ -88,7 +93,8 @@ class cmd: self._process_terminate(terminate=terminate, timeout=1) raise CmdInitFailure("Did not receive ready message", self) if not background: - self.process(terminate=False, fail=fail, timeout=timeout) + self.process(terminate=False, fail=fail, expect_fail=expect_fail, + timeout=timeout) def _process_terminate(self, terminate, timeout): if terminate:
@@ -102,7 +108,7 @@ class cmd: return stdout, stderr - def process(self, terminate=True, fail=None, timeout=5): + def process(self, terminate=True, fail=None, expect_fail=False, timeout=5): if fail is None: fail = not terminate
@@ -111,10 +117,19 @@ class cmd: stdout, stderr = self._process_terminate(terminate=terminate, timeout=timeout) - if self.proc.returncode != 0 and fail: + + # Fail on unexpected test failure if fail. + # Fail on unexpected test success if expect_fail. + # Fail on negative returncode if either: + # Set by subprocess on crash or signal, this is never expected failure. + if (self.proc.returncode != 0 and fail or + (self.proc.returncode < 0 and expect_fail)): if len(stderr) > 0 and stderr[-1] == "\n": stderr = stderr[:-1] raise CmdExitFailure("Command failed", self) + elif self.proc.returncode == 0 and expect_fail: + raise CmdExitZeroFailure("Command succeeded (expected fail)", self) + def __repr__(self): def str_fmt(name, s):
@@ -157,14 +172,17 @@ class bkg(cmd): with bkg("my_binary", ksft_wait=5): """ - def __init__(self, comm, shell=None, fail=None, ns=None, host=None, - exit_wait=False, ksft_ready=None, ksft_wait=None): + def __init__(self, comm, shell=None, fail=None, expect_fail=None, + ns=None, host=None, exit_wait=False, ksft_ready=None, + ksft_wait=None): super().__init__(comm, background=True, - shell=shell, fail=fail, ns=ns, host=host, - ksft_ready=ksft_ready, ksft_wait=ksft_wait) + shell=shell, fail=fail, expect_fail=expect_fail, + ns=ns, host=host, ksft_ready=ksft_ready, + ksft_wait=ksft_wait) self.terminate = not exit_wait and not ksft_wait self._exit_wait = exit_wait self.check_fail = fail + self.expect_fail = expect_fail if shell and self.terminate: print("# Warning: combining shell and terminate is risky!")
@@ -179,7 +197,8 @@ class bkg(cmd): # since forcing termination silences failures with fail=None if self.proc.poll() is None: terminate = terminate or (self._exit_wait and ex_type is not None) - return self.process(terminate=terminate, fail=self.check_fail) + return self.process(terminate=terminate, fail=self.check_fail, + expect_fail=self.expect_fail) GLOBAL_DEFER_QUEUE = []
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Willem de Bruijn
maintainerPATCH v7 2/3 selftests: net: py: add tc utility
In reply to Willem de BruijnFrom: Willem de Bruijn <[email protected]>
Add a wrapper similar to existing ip, ethtool, ... commands.
Tc takes a slightly different syntax. Account for that.
The first user is the next patch in this series, converting so_txtime to drv-net. Pacing offload is supported by selected qdiscs only.
--- .../testing/selftests/drivers/net/lib/py/__init__.py | 5 +++-- tools/testing/selftests/net/lib/py/__init__.py | 4 ++-- tools/testing/selftests/net/lib/py/utils.py | 12 +++++++++++- 3 files changed, 16 insertions(+), 5 deletions(-)
tools/testing/selftests/drivers/net/lib/py/__init__.py +3 -2
@@ -23,7 +23,8 @@ try: NlError, RtnlFamily, DevlinkFamily, PSPFamily, Netlink from net.lib.py import CmdExitFailure from net.lib.py import bkg, cmd, bpftool, bpftrace, defer, ethtool, \ - fd_read_timeout, ip, rand_port, rand_ports, wait_port_listen, wait_file + fd_read_timeout, ip, rand_port, rand_ports, tc, wait_port_listen, \ + wait_file from net.lib.py import bpf_map_set, bpf_map_dump, bpf_prog_map_ids from net.lib.py import KsftSkipEx, KsftFailEx, KsftXfailEx from net.lib.py import ksft_disruptive, ksft_exit, ksft_pr, ksft_run, \
@@ -36,7 +37,7 @@ try: "NlError", "RtnlFamily", "DevlinkFamily", "PSPFamily", "Netlink", "CmdExitFailure", "bkg", "cmd", "bpftool", "bpftrace", "defer", "ethtool", - "fd_read_timeout", "ip", "rand_port", "rand_ports", + "fd_read_timeout", "ip", "rand_port", "rand_ports", "tc", "wait_port_listen", "wait_file", "bpf_map_set", "bpf_map_dump", "bpf_prog_map_ids", "KsftSkipEx", "KsftFailEx", "KsftXfailEx",
tools/testing/selftests/net/lib/py/__init__.py +2 -2
@@ -14,7 +14,7 @@ from .netns import NetNS, NetNSEnter from .nsim import NetdevSim, NetdevSimDev from .utils import CmdExitFailure, fd_read_timeout, cmd, bkg, defer, \ bpftool, ip, ethtool, bpftrace, rand_port, rand_ports, wait_port_listen, \ - wait_file, tool + wait_file, tool, tc from .bpf import bpf_map_set, bpf_map_dump, bpf_prog_map_ids from .ynl import NlError, NlctrlFamily, YnlFamily, \ EthtoolFamily, NetdevFamily, RtnlFamily, RtnlAddrFamily
@@ -29,7 +29,7 @@ __all__ = ["KSRC", "NetNS", "NetNSEnter", "CmdExitFailure", "fd_read_timeout", "cmd", "bkg", "defer", "bpftool", "ip", "ethtool", "bpftrace", "rand_port", "rand_ports", - "wait_port_listen", "wait_file", "tool", + "wait_port_listen", "wait_file", "tool", "tc", "bpf_map_set", "bpf_map_dump", "bpf_prog_map_ids", "NetdevSim", "NetdevSimDev", "NetshaperFamily", "DevlinkFamily", "PSPFamily", "NlError",
tools/testing/selftests/net/lib/py/utils.py +11 -1
@@ -239,7 +239,10 @@ class defer: def tool(name, args, json=None, ns=None, host=None): cmd_str = name + ' ' if json: - cmd_str += '--json ' + if name == 'tc': + cmd_str += '-json ' + else: + cmd_str += '--json ' cmd_str += args cmd_obj = cmd(cmd_str, ns=ns, host=host) if json:
@@ -257,6 +260,13 @@ def ip(args, json=None, ns=None, host=None): return tool('ip', args, json=json, host=host) +def tc(args, json=None, ns=None, host=None): + """ Helper to call tc with standard set of optional args. """ + if ns: + args = f'-netns {ns} ' + args + return tool('tc', args, json=json, host=host) + + def ethtool(args, json=None, ns=None, host=None): return tool('ethtool', args, json=json, ns=ns, host=host)
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Willem de Bruijn
maintainerPATCH v7 3/3 selftests: drv-net: convert so_txtime to drv-net
In reply to Willem de BruijnFrom: Willem de Bruijn <[email protected]>
In preparation for extending to pacing hardware offload, convert the so_txtime.sh test to a drv-net test that can be run against netdevsim and real hardware.
Also update so_txtime.c to not exit on first failure, but run to completion and report exit code there. This helps with debugging unexpected results, especially when processing multiple packets, as happens in the "reverse_order" testcase.
----
v6 -> v7
- update test to use new argument expect_fail
- v6 received Reviewed-by, but dropped due to above (minor) change
v5 -> v6
- fix order in tools/testing/selftests/drivers/net/config
v4 -> v5
- move qdisc setup/restore into each test
- add tc to utils.py (separate patch)
- test expected failure (separate patch)
- fix pylint
- convert fail to pass for timing errors if KSFT_MACHINE_SLOW (cmd does not special case KSFT_SKIP process returncode yet)
Responses to sashiko review
- The test converts per packet failure to errors, to continue testing other packets, but other error() cases are not in scope.
- The test starts sender and receiver at an absolute future time, like the original test. This assumes ~msec scale sync'ed clocks.
- The tc qdisc replace command works fine with noqueue. Tested manually.
v3 -> v4
- restore original qdisc after test
- drop unnecessary underscore in tap test names
v2 -> v3
- Makefile: so_txtime from YNL_GEN_FILES to TEST_GEN_FILES (Sashiko, NIPA)
v1 -> v2
- move so_txtime.c for net/lib to drivers/net (Jakub)
- fix drivers/net/config order (Jakub)
- detect passing when failure is expected (Jakub, Sashiko)
- pass pylint --disable=R (Jakub)
- only call ksft_run once (Jakub)
- do not sleep if waiting time is negative (Sashiko)
- add \n when converting error() to fprintf() (Sashiko)
- 4 space indentation, instead of 2 space
- increase sync delay from 100 to 200ms, to fix rare vng flakes
---
.../testing/selftests/drivers/net/.gitignore | 1 +
tools/testing/selftests/drivers/net/Makefile | 2 +
tools/testing/selftests/drivers/net/config | 2 +
.../selftests/{ => drivers}/net/so_txtime.c | 25 +++-
.../selftests/drivers/net/so_txtime.py | 96 +++++++++++++++
tools/testing/selftests/net/.gitignore | 1 -
tools/testing/selftests/net/Makefile | 2 -
tools/testing/selftests/net/so_txtime.sh | 110 ------------------
8 files changed, 121 insertions(+), 118 deletions(-)
rename tools/testing/selftests/{ => drivers}/net/so_txtime.c (96%)
create mode 100755 tools/testing/selftests/drivers/net/so_txtime.py
delete mode 100755 tools/testing/selftests/net/so_txtime.shtools/testing/selftests/drivers/net/.gitignore +1 -0
@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only napi_id_helper psp_responder +so_txtime
tools/testing/selftests/drivers/net/Makefile +2 -0
@@ -7,6 +7,7 @@ TEST_INCLUDES := $(wildcard lib/py/*.py) \ TEST_GEN_FILES := \ napi_id_helper \ + so_txtime \ # end of TEST_GEN_FILES TEST_PROGS := \
@@ -21,6 +22,7 @@ TEST_PROGS := \ queues.py \ ring_reconfig.py \ shaper.py \ + so_txtime.py \ stats.py \ xdp.py \ # end of TEST_PROGS
tools/testing/selftests/drivers/net/config +2 -0
@@ -8,5 +8,7 @@ CONFIG_NETCONSOLE=m CONFIG_NETCONSOLE_DYNAMIC=y CONFIG_NETCONSOLE_EXTENDED_LOG=y CONFIG_NETDEVSIM=m +CONFIG_NET_SCH_ETF=m +CONFIG_NET_SCH_FQ=m CONFIG_VLAN_8021Q=m CONFIG_XDP_SOCKETS=y
tools/testing/selftests/drivers/net/so_txtime.c renamed from tools/testing/selftests/net/so_txtime.c +20 -5
@@ -33,6 +33,8 @@#include <unistd.h> #include <poll.h> +#include "kselftest.h" +static int cfg_clockid =CLOCK_TAI ;static uint16_t cfg_port =8000 ;static int cfg_variance_us =4000 ;
@@ -43,6 +45,8 @@ static bool cfg_rx;static uint64_t glob_tstart;static uint64_t tdeliver_max; +static int errors; +/* encode one timed transmission (of a 1B payload) */ struct timed_send {char data;
@@ -131,13 +135,15 @@ static void do_recv_one(int fdr, struct timed_send *ts)fprintf (stderr,"payload:%c delay:%lld expected:%lld (us)\n" , rbuf[0 ], (long long )tstop, (long long )texpect); -if (rbuf[0 ] != ts->data) -error (1 ,0 ,"payload mismatch. expected %c" , ts->data); +if (rbuf[0 ] != ts->data) { +fprintf (stderr,"payload mismatch. expected %c\n" , ts->data); + errors++; + }if (llabs (tstop - texpect) > cfg_variance_us) {fprintf (stderr,"exceeds variance (%d us)\n" , cfg_variance_us);if (!getenv ("KSFT_MACHINE_SLOW" )) -exit (1 ); + errors++; } }
@@ -255,8 +261,12 @@ static void start_time_wait(void)return ; now =gettime_ns (CLOCK_REALTIME ); -if (cfg_start_time_ns < now) +if (cfg_start_time_ns < now) { +fprintf (stderr,"FAIL: start time already passed\n" ); +if (!getenv ("KSFT_MACHINE_SLOW" )) + errors++;return ; + } err =usleep ((cfg_start_time_ns - now) /1000 );if (err)
@@ -513,5 +523,10 @@ int main(int argc, char **argv)else do_test_tx ((void *)&cfg_src_addr, cfg_alen); -return 0 ; +if (errors) { +fprintf (stderr,"FAIL: %d errors\n" , errors); +return KSFT_FAIL ; + } + +return KSFT_PASS ; }
tools/testing/selftests/drivers/net/so_txtime.py new file +96 -0
96 lines not on this page · raw message
tools/testing/selftests/net/.gitignore +0 -1
@@ -40,7 +40,6 @@ skf_net_off socket so_incoming_cpu so_netns_cookie -so_txtime so_rcv_listener stress_reuseport_listen tap
tools/testing/selftests/net/Makefile +0 -2
@@ -83,7 +83,6 @@ TEST_PROGS := \ rxtimestamp.sh \ sctp_vrf.sh \ skf_net_off.sh \ - so_txtime.sh \ srv6_end_dt46_l3vpn_test.sh \ srv6_end_dt4_l3vpn_test.sh \ srv6_end_dt6_l3vpn_test.sh \
@@ -157,7 +156,6 @@ TEST_GEN_FILES := \ skf_net_off \ so_netns_cookie \ so_rcv_listener \ - so_txtime \ socket \ stress_reuseport_listen \ tcp_fastopen_backup_key \
tools/testing/selftests/net/so_txtime.sh deleted +0 -110
110 lines not on this page · raw message
Signature · 1 line
2.54.0.545.g6539524ca2-goog
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Quoted 11 lines · On Mon, 4 May 2026 13:38:31 -0400 you wrote:
From: Willem de Bruijn <[email protected]>
In preparation for extending to pacing hardware offload, convert the
so_txtime.shtest to a drv-net test that can be run against netdevsim and real hardware.Two preparatory patches
1. support negative tests, where tests are expected to fail
2. add a tc helper[...]
Here is the summary with links:
- [net-next,v7,1/3] selftests: net: py: support cmd verifying expected failure
https://git.kernel.org/netdev/net-next/c/d9c1e7dc4ada - [net-next,v7,2/3] selftests: net: py: add tc utility https://git.kernel.org/netdev/net-next/c/49bf9e5e7c7c - [net-next,v7,3/3] selftests: drv-net: convert so_txtime to drv-net https://git.kernel.org/netdev/net-next/c/5c6baef3885c
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
Jakub Kicinski
maintainerQuoted 11 lines · On Mon, 4 May 2026 13:38:31 -0400 Willem de Bruijn wrote:
From: Willem de Bruijn <[email protected]>
In preparation for extending to pacing hardware offload, convert the
so_txtime.shtest to a drv-net test that can be run against netdevsim and real hardware.Two preparatory patches
1. support negative tests, where tests are expected to fail
2. add a tc helperSee individual patches for details and detailed changelog
Hi Willem! Looks like we have some flakiness here:
https://netdev.bots.linux.dev/contest.html?executor=vmksft-net-drv-dbg&test=so-txtime-py
Willem de Bruijn
maintainerQuoted 11 lines · Jakub Kicinski
Quoted 10 lines · On Mon, 4 May 2026 13:38:31 -0400 Willem de Bruijn wrote:
From: Willem de Bruijn <[email protected]>
In preparation for extending to pacing hardware offload, convert the
so_txtime.shtest to a drv-net test that can be run against netdevsim and real hardware.Two preparatory patches
1. support negative tests, where tests are expected to fail
2. add a tc helper
Quoted 5 lines
Quoted 1 line
See individual patches for details and detailed changelog
Hi Willem! Looks like we have some flakiness here:
https://netdev.bots.linux.dev/contest.html?executor=vmksft-net-drv-dbg&test=so-txtime-py
Argh sorry. I should have run dbg mode, but hadn't.
The previous version skips all receiver errors if KSFT_MACHINE_SLOW is set, as of commit a7ee79b9c455 ("selftests: net: cope with slow env in so_txtime.sh test").
This achieves the same in the new version and seems to indeed mask all the timing (and other) errors in debug mode.
@@ -24,8 +25,12 @@ def test_so_txtime(cfg, clockid, ipver, args_tx, args_rx, expect_success):
cmd_rx = f"{cmd_base} {args_rx} -r"
cmd_tx = f"{cmd_base} {args_tx}"
+ expect_fail = not expect_success
+ if os.environ.get('KSFT_MACHINE_SLOW'):
+ expect_success = False
+
with bkg(cmd_rx, host=cfg.remote, fail=expect_success,
- expect_fail=(not expect_success), exit_wait=True):
+ expect_fail=expect_fail, exit_wait=True):
cmd(cmd_tx)