Skip to contentSkip to conversation

selftests: drv-net: convert so_txtime to drv-net

PATCH Applied Willem de Bruijn 3 participants 7 messages May 4 – May 11

Subsystems (4)
lore
Share & tools
lore

Willem de Bruijn

author maintainer
#0 lore raw

v7 cover selftests: drv-net: convert so_txtime to drv-net

From: Willem de Bruijn <[email protected]>

In preparation for extending to pacing hardware offload, convert the so_txtime.sh test to a drv-net test that can be run against netdevsim and real hardware.

Two preparatory patches
1. support negative tests, where tests are expected to fail
2. add a tc helper

See individual patches for details and detailed changelog

Previous versions:

v6: https://lore.kernel.org/netdev/[email protected]/
v5: https://lore.kernel.org/netdev/[email protected]/
v4: https://lore.kernel.org/netdev/[email protected]/
v3: https://lore.kernel.org/netdev/[email protected]/
v2: https://lore.kernel.org/netdev/[email protected]/
v1: https://lore.kernel.org/netdev/[email protected]/

Willem de Bruijn (3):
selftests: net: py: support cmd verifying expected failure
selftests: net: py: add tc utility
selftests: drv-net: convert so_txtime to drv-net

 .../testing/selftests/drivers/net/.gitignore  |   1 +
 tools/testing/selftests/drivers/net/Makefile  |   2 +
 tools/testing/selftests/drivers/net/config    |   2 +
 .../selftests/drivers/net/lib/py/__init__.py  |   5 +-
 .../selftests/{ => drivers}/net/so_txtime.c   |  25 +++-
 .../selftests/drivers/net/so_txtime.py        |  96 +++++++++++++++
 tools/testing/selftests/net/.gitignore        |   1 -
 tools/testing/selftests/net/Makefile          |   2 -
 .../testing/selftests/net/lib/py/__init__.py  |   4 +-
 tools/testing/selftests/net/lib/py/utils.py   |  51 ++++++--
 tools/testing/selftests/net/so_txtime.sh      | 110 ------------------
 11 files changed, 166 insertions(+), 133 deletions(-)
 rename tools/testing/selftests/{ => drivers}/net/so_txtime.c (96%)
 create mode 100755 tools/testing/selftests/drivers/net/so_txtime.py
 delete mode 100755 tools/testing/selftests/net/so_txtime.sh
Signature · 1 line
2.54.0.545.g6539524ca2-goog
author maintainer
#1 lore raw

PATCH v7 1/3 selftests: net: py: support cmd verifying expected failure

From: Willem de Bruijn <[email protected]>

Support negative tests, where cmd raises an exception if the command succeeded.

Add optional argument expect_fail to cmd and bkg. Where fail fails the test on unexpected error, expect_fail fails it on unexpected success.

Both fail on negative return code. Python subprocess may set a negative return code on process crash or timeout. Those are never anticipated failures.

---

In this design fail and expect_fail are two complementary tests. If both are set to true, an exception would always be raised.

This is contrast to the previous approach, where fail enables fail_on_unexpected_returncode and expect_fail modifies what that unexpected_returncode value range is.

---

v6 -> v7
- convert from 'verify_failed' value for fail, to separate variable

v4 -> v5
- initial version of standalone patch

---
 tools/testing/selftests/net/lib/py/utils.py | 39 +++++++++++++++------
 1 file changed, 29 insertions(+), 10 deletions(-)
tools/testing/selftests/net/lib/py/utils.py +29 -10
index 6c44a3d2bbf7..b68d4607114d 100644
@@ -23,6 +23,10 @@ class CmdExitFailure(Exception):
         self.cmd = cmd_obj
 
 
+class CmdExitZeroFailure(CmdExitFailure):
+    """ Command succeeded (returned zero exit code), but expected failure. """
+
+
 def fd_read_timeout(fd, timeout):
     rlist, _, _ = select.select([fd], [], [], timeout)
     if rlist:
@@ -39,8 +43,9 @@ class cmd:
 
     Use bkg() instead to run a command in the background.
     """
-    def __init__(self, comm, shell=None, fail=True, ns=None, background=False,
-                 host=None, timeout=5, ksft_ready=None, ksft_wait=None):
+    def __init__(self, comm, shell=None, fail=True, expect_fail=False, ns=None,
+                 background=False, host=None, timeout=5, ksft_ready=None,
+                 ksft_wait=None):
         if ns:
             comm = f'ip netns exec {ns} ' + comm
 
@@ -88,7 +93,8 @@ class cmd:
                     self._process_terminate(terminate=terminate, timeout=1)
                     raise CmdInitFailure("Did not receive ready message", self)
         if not background:
-            self.process(terminate=False, fail=fail, timeout=timeout)
+            self.process(terminate=False, fail=fail, expect_fail=expect_fail,
+                         timeout=timeout)
 
     def _process_terminate(self, terminate, timeout):
         if terminate:
@@ -102,7 +108,7 @@ class cmd:
 
         return stdout, stderr
 
-    def process(self, terminate=True, fail=None, timeout=5):
+    def process(self, terminate=True, fail=None, expect_fail=False, timeout=5):
         if fail is None:
             fail = not terminate
 
@@ -111,10 +117,19 @@ class cmd:
 
         stdout, stderr = self._process_terminate(terminate=terminate,
                                                  timeout=timeout)
-        if self.proc.returncode != 0 and fail:
+
+        # Fail on unexpected test failure if fail.
+        # Fail on unexpected test success if expect_fail.
+        # Fail on negative returncode if either:
+        # Set by subprocess on crash or signal, this is never expected failure.
+        if (self.proc.returncode != 0 and fail or
+            (self.proc.returncode < 0 and expect_fail)):
             if len(stderr) > 0 and stderr[-1] == "\n":
                 stderr = stderr[:-1]
             raise CmdExitFailure("Command failed", self)
+        elif self.proc.returncode == 0 and expect_fail:
+            raise CmdExitZeroFailure("Command succeeded (expected fail)", self)
+
 
     def __repr__(self):
         def str_fmt(name, s):
@@ -157,14 +172,17 @@ class bkg(cmd):
 
         with bkg("my_binary", ksft_wait=5):
     """
-    def __init__(self, comm, shell=None, fail=None, ns=None, host=None,
-                 exit_wait=False, ksft_ready=None, ksft_wait=None):
+    def __init__(self, comm, shell=None, fail=None, expect_fail=None,
+                 ns=None, host=None, exit_wait=False, ksft_ready=None,
+                 ksft_wait=None):
         super().__init__(comm, background=True,
-                         shell=shell, fail=fail, ns=ns, host=host,
-                         ksft_ready=ksft_ready, ksft_wait=ksft_wait)
+                         shell=shell, fail=fail, expect_fail=expect_fail,
+                         ns=ns, host=host, ksft_ready=ksft_ready,
+                         ksft_wait=ksft_wait)
         self.terminate = not exit_wait and not ksft_wait
         self._exit_wait = exit_wait
         self.check_fail = fail
+        self.expect_fail = expect_fail
 
         if shell and self.terminate:
             print("# Warning: combining shell and terminate is risky!")
@@ -179,7 +197,8 @@ class bkg(cmd):
         # since forcing termination silences failures with fail=None
         if self.proc.poll() is None:
             terminate = terminate or (self._exit_wait and ex_type is not None)
-        return self.process(terminate=terminate, fail=self.check_fail)
+        return self.process(terminate=terminate, fail=self.check_fail,
+                            expect_fail=self.expect_fail)
 
 
 GLOBAL_DEFER_QUEUE = []
Signature · 1 line
2.54.0.545.g6539524ca2-goog
author maintainer
#2 lore raw

PATCH v7 2/3 selftests: net: py: add tc utility

In reply to Willem de Bruijn

From: Willem de Bruijn <[email protected]>

Add a wrapper similar to existing ip, ethtool, ... commands.

Tc takes a slightly different syntax. Account for that.

The first user is the next patch in this series, converting so_txtime to drv-net. Pacing offload is supported by selected qdiscs only.

---
 .../testing/selftests/drivers/net/lib/py/__init__.py |  5 +++--
 tools/testing/selftests/net/lib/py/__init__.py       |  4 ++--
 tools/testing/selftests/net/lib/py/utils.py          | 12 +++++++++++-
 3 files changed, 16 insertions(+), 5 deletions(-)
tools/testing/selftests/drivers/net/lib/py/__init__.py +3 -2
index 2b5ec0505672..09aac4ce67bc 100644
@@ -23,7 +23,8 @@ try:
         NlError, RtnlFamily, DevlinkFamily, PSPFamily, Netlink
     from net.lib.py import CmdExitFailure
     from net.lib.py import bkg, cmd, bpftool, bpftrace, defer, ethtool, \
-        fd_read_timeout, ip, rand_port, rand_ports, wait_port_listen, wait_file
+        fd_read_timeout, ip, rand_port, rand_ports, tc, wait_port_listen, \
+        wait_file
     from net.lib.py import bpf_map_set, bpf_map_dump, bpf_prog_map_ids
     from net.lib.py import KsftSkipEx, KsftFailEx, KsftXfailEx
     from net.lib.py import ksft_disruptive, ksft_exit, ksft_pr, ksft_run, \
@@ -36,7 +37,7 @@ try:
                "NlError", "RtnlFamily", "DevlinkFamily", "PSPFamily", "Netlink",
                "CmdExitFailure",
                "bkg", "cmd", "bpftool", "bpftrace", "defer", "ethtool",
-               "fd_read_timeout", "ip", "rand_port", "rand_ports",
+               "fd_read_timeout", "ip", "rand_port", "rand_ports", "tc",
                "wait_port_listen", "wait_file",
                "bpf_map_set", "bpf_map_dump", "bpf_prog_map_ids",
                "KsftSkipEx", "KsftFailEx", "KsftXfailEx",
tools/testing/selftests/net/lib/py/__init__.py +2 -2
index 7c81d86a7e97..64a8c1ed4950 100644
@@ -14,7 +14,7 @@ from .netns import NetNS, NetNSEnter
 from .nsim import NetdevSim, NetdevSimDev
 from .utils import CmdExitFailure, fd_read_timeout, cmd, bkg, defer, \
     bpftool, ip, ethtool, bpftrace, rand_port, rand_ports, wait_port_listen, \
-    wait_file, tool
+    wait_file, tool, tc
 from .bpf import bpf_map_set, bpf_map_dump, bpf_prog_map_ids
 from .ynl import NlError, NlctrlFamily, YnlFamily, \
     EthtoolFamily, NetdevFamily, RtnlFamily, RtnlAddrFamily
@@ -29,7 +29,7 @@ __all__ = ["KSRC",
            "NetNS", "NetNSEnter",
            "CmdExitFailure", "fd_read_timeout", "cmd", "bkg", "defer",
            "bpftool", "ip", "ethtool", "bpftrace", "rand_port", "rand_ports",
-           "wait_port_listen", "wait_file", "tool",
+           "wait_port_listen", "wait_file", "tool", "tc",
            "bpf_map_set", "bpf_map_dump", "bpf_prog_map_ids",
            "NetdevSim", "NetdevSimDev",
            "NetshaperFamily", "DevlinkFamily", "PSPFamily", "NlError",
tools/testing/selftests/net/lib/py/utils.py +11 -1
index b68d4607114d..be9408a77168 100644
@@ -239,7 +239,10 @@ class defer:
 def tool(name, args, json=None, ns=None, host=None):
     cmd_str = name + ' '
     if json:
-        cmd_str += '--json '
+        if name == 'tc':
+            cmd_str += '-json '
+        else:
+            cmd_str += '--json '
     cmd_str += args
     cmd_obj = cmd(cmd_str, ns=ns, host=host)
     if json:
@@ -257,6 +260,13 @@ def ip(args, json=None, ns=None, host=None):
     return tool('ip', args, json=json, host=host)
 
 
+def tc(args, json=None, ns=None, host=None):
+    """ Helper to call tc with standard set of optional args. """
+    if ns:
+        args = f'-netns {ns} ' + args
+    return tool('tc', args, json=json, host=host)
+
+
 def ethtool(args, json=None, ns=None, host=None):
     return tool('ethtool', args, json=json, ns=ns, host=host)
 
Signature · 1 line
2.54.0.545.g6539524ca2-goog
author maintainer
#3 lore raw

PATCH v7 3/3 selftests: drv-net: convert so_txtime to drv-net

In reply to Willem de Bruijn

From: Willem de Bruijn <[email protected]>

In preparation for extending to pacing hardware offload, convert the so_txtime.sh test to a drv-net test that can be run against netdevsim and real hardware.

Also update so_txtime.c to not exit on first failure, but run to completion and report exit code there. This helps with debugging unexpected results, especially when processing multiple packets, as happens in the "reverse_order" testcase.

----

v6 -> v7

- update test to use new argument expect_fail
- v6 received Reviewed-by, but dropped due to above (minor) change

v5 -> v6

- fix order in tools/testing/selftests/drivers/net/config

v4 -> v5

- move qdisc setup/restore into each test
- add tc to utils.py (separate patch)
- test expected failure (separate patch)
- fix pylint
- convert fail to pass for timing errors if KSFT_MACHINE_SLOW (cmd does not special case KSFT_SKIP process returncode yet)

Responses to sashiko review

- The test converts per packet failure to errors, to continue testing other packets, but other error() cases are not in scope.
- The test starts sender and receiver at an absolute future time, like the original test. This assumes ~msec scale sync'ed clocks.
- The tc qdisc replace command works fine with noqueue. Tested manually.

v3 -> v4

- restore original qdisc after test
- drop unnecessary underscore in tap test names

v2 -> v3

- Makefile: so_txtime from YNL_GEN_FILES to TEST_GEN_FILES (Sashiko, NIPA)

v1 -> v2
- move so_txtime.c for net/lib to drivers/net (Jakub)
- fix drivers/net/config order (Jakub)
- detect passing when failure is expected (Jakub, Sashiko)
- pass pylint --disable=R (Jakub)
- only call ksft_run once (Jakub)
- do not sleep if waiting time is negative (Sashiko)
- add \n when converting error() to fprintf() (Sashiko)
- 4 space indentation, instead of 2 space
- increase sync delay from 100 to 200ms, to fix rare vng flakes

---
 .../testing/selftests/drivers/net/.gitignore  |   1 +
 tools/testing/selftests/drivers/net/Makefile  |   2 +
 tools/testing/selftests/drivers/net/config    |   2 +
 .../selftests/{ => drivers}/net/so_txtime.c   |  25 +++-
 .../selftests/drivers/net/so_txtime.py        |  96 +++++++++++++++
 tools/testing/selftests/net/.gitignore        |   1 -
 tools/testing/selftests/net/Makefile          |   2 -
 tools/testing/selftests/net/so_txtime.sh      | 110 ------------------
 8 files changed, 121 insertions(+), 118 deletions(-)
 rename tools/testing/selftests/{ => drivers}/net/so_txtime.c (96%)
 create mode 100755 tools/testing/selftests/drivers/net/so_txtime.py
 delete mode 100755 tools/testing/selftests/net/so_txtime.sh
tools/testing/selftests/drivers/net/.gitignore +1 -0
index 585ecb4d5dc4..e5314ce4bb2d 100644
@@ -1,3 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0-only
 napi_id_helper
 psp_responder
+so_txtime
tools/testing/selftests/drivers/net/Makefile +2 -0
index b72080c6d06b..d5bf4cb638a8 100644
@@ -7,6 +7,7 @@ TEST_INCLUDES := $(wildcard lib/py/*.py) \
 
 TEST_GEN_FILES := \
 	napi_id_helper \
+	so_txtime \
 # end of TEST_GEN_FILES
 
 TEST_PROGS := \
@@ -21,6 +22,7 @@ TEST_PROGS := \
 	queues.py \
 	ring_reconfig.py \
 	shaper.py \
+	so_txtime.py \
 	stats.py \
 	xdp.py \
 # end of TEST_PROGS
tools/testing/selftests/drivers/net/config +2 -0
index fd16994366f4..2309109a94ec 100644
@@ -8,5 +8,7 @@ CONFIG_NETCONSOLE=m
 CONFIG_NETCONSOLE_DYNAMIC=y
 CONFIG_NETCONSOLE_EXTENDED_LOG=y
 CONFIG_NETDEVSIM=m
+CONFIG_NET_SCH_ETF=m
+CONFIG_NET_SCH_FQ=m
 CONFIG_VLAN_8021Q=m
 CONFIG_XDP_SOCKETS=y
tools/testing/selftests/drivers/net/so_txtime.c renamed from tools/testing/selftests/net/so_txtime.c +20 -5
similarity index 96%
rename from tools/testing/selftests/net/so_txtime.c
rename to tools/testing/selftests/drivers/net/so_txtime.c
index b76df1efc2ef..b6930883569b 100644
@@ -33,6 +33,8 @@
 #include <unistd.h>
 #include <poll.h>
 
+#include "kselftest.h"
+
 static int	cfg_clockid	= CLOCK_TAI;
 static uint16_t	cfg_port	= 8000;
 static int	cfg_variance_us	= 4000;
@@ -43,6 +45,8 @@ static bool	cfg_rx;
 static uint64_t glob_tstart;
 static uint64_t tdeliver_max;
 
+static int errors;
+
 /* encode one timed transmission (of a 1B payload) */
 struct timed_send {
 	char	data;
@@ -131,13 +135,15 @@ static void do_recv_one(int fdr, struct timed_send *ts)
 	fprintf(stderr, "payload:%c delay:%lld expected:%lld (us)\n",
 			rbuf[0], (long long)tstop, (long long)texpect);
 
-	if (rbuf[0] != ts->data)
-		error(1, 0, "payload mismatch. expected %c", ts->data);
+	if (rbuf[0] != ts->data) {
+		fprintf(stderr, "payload mismatch. expected %c\n", ts->data);
+		errors++;
+	}
 
 	if (llabs(tstop - texpect) > cfg_variance_us) {
 		fprintf(stderr, "exceeds variance (%d us)\n", cfg_variance_us);
 		if (!getenv("KSFT_MACHINE_SLOW"))
-			exit(1);
+			errors++;
 	}
 }
 
@@ -255,8 +261,12 @@ static void start_time_wait(void)
 		return;
 
 	now = gettime_ns(CLOCK_REALTIME);
-	if (cfg_start_time_ns < now)
+	if (cfg_start_time_ns < now) {
+		fprintf(stderr, "FAIL: start time already passed\n");
+		if (!getenv("KSFT_MACHINE_SLOW"))
+			errors++;
 		return;
+	}
 
 	err = usleep((cfg_start_time_ns - now) / 1000);
 	if (err)
@@ -513,5 +523,10 @@ int main(int argc, char **argv)
 	else
 		do_test_tx((void *)&cfg_src_addr, cfg_alen);
 
-	return 0;
+	if (errors) {
+		fprintf(stderr, "FAIL: %d errors\n", errors);
+		return KSFT_FAIL;
+	}
+
+	return KSFT_PASS;
 }
tools/testing/selftests/drivers/net/so_txtime.py new file +96 -0

96 lines not on this page · raw message

tools/testing/selftests/net/.gitignore +0 -1
index 97ad4d551d44..02ad4c99a2b4 100644
@@ -40,7 +40,6 @@ skf_net_off
 socket
 so_incoming_cpu
 so_netns_cookie
-so_txtime
 so_rcv_listener
 stress_reuseport_listen
 tap
tools/testing/selftests/net/Makefile +0 -2
index a275ed584026..d53670c0d7a2 100644
@@ -83,7 +83,6 @@ TEST_PROGS := \
 	rxtimestamp.sh \
 	sctp_vrf.sh \
 	skf_net_off.sh \
-	so_txtime.sh \
 	srv6_end_dt46_l3vpn_test.sh \
 	srv6_end_dt4_l3vpn_test.sh \
 	srv6_end_dt6_l3vpn_test.sh \
@@ -157,7 +156,6 @@ TEST_GEN_FILES := \
 	skf_net_off \
 	so_netns_cookie \
 	so_rcv_listener \
-	so_txtime \
 	socket \
 	stress_reuseport_listen \
 	tcp_fastopen_backup_key \
tools/testing/selftests/net/so_txtime.sh deleted +0 -110

110 lines not on this page · raw message

Signature · 1 line
2.54.0.545.g6539524ca2-goog
#4 lore raw
In reply to Willem de Bruijn
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Quoted 11 lines · On Mon, 4 May 2026 13:38:31 -0400 you wrote:

From: Willem de Bruijn <[email protected]>

In preparation for extending to pacing hardware offload, convert the so_txtime.sh test to a drv-net test that can be run against netdevsim and real hardware.

Two preparatory patches
1. support negative tests, where tests are expected to fail
2. add a tc helper

[...]

Here is the summary with links:
- [net-next,v7,1/3] selftests: net: py: support cmd verifying expected failure

    https://git.kernel.org/netdev/net-next/c/d9c1e7dc4ada
  - [net-next,v7,2/3] selftests: net: py: add tc utility
    https://git.kernel.org/netdev/net-next/c/49bf9e5e7c7c
  - [net-next,v7,3/3] selftests: drv-net: convert so_txtime to drv-net
    https://git.kernel.org/netdev/net-next/c/5c6baef3885c

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
maintainer
#5 lore raw
In reply to Willem de Bruijn
Quoted 11 lines · On Mon, 4 May 2026 13:38:31 -0400 Willem de Bruijn wrote:

From: Willem de Bruijn <[email protected]>

In preparation for extending to pacing hardware offload, convert the so_txtime.sh test to a drv-net test that can be run against netdevsim and real hardware.

Two preparatory patches
1. support negative tests, where tests are expected to fail
2. add a tc helper

See individual patches for details and detailed changelog

Hi Willem! Looks like we have some flakiness here:

https://netdev.bots.linux.dev/contest.html?executor=vmksft-net-drv-dbg&test=so-txtime-py

author maintainer
#6 lore raw
Quoted 11 lines · Jakub Kicinski
Quoted 10 lines · On Mon, 4 May 2026 13:38:31 -0400 Willem de Bruijn wrote:

From: Willem de Bruijn <[email protected]>

In preparation for extending to pacing hardware offload, convert the so_txtime.sh test to a drv-net test that can be run against netdevsim and real hardware.

Two preparatory patches
1. support negative tests, where tests are expected to fail
2. add a tc helper

Quoted 5 lines
Quoted 1 line

See individual patches for details and detailed changelog

Hi Willem! Looks like we have some flakiness here:

https://netdev.bots.linux.dev/contest.html?executor=vmksft-net-drv-dbg&test=so-txtime-py

Argh sorry. I should have run dbg mode, but hadn't.

The previous version skips all receiver errors if KSFT_MACHINE_SLOW is set, as of commit a7ee79b9c455 ("selftests: net: cope with slow env in so_txtime.sh test").

This achieves the same in the new version and seems to indeed mask all the timing (and other) errors in debug mode.

@@ -24,8 +25,12 @@ def test_so_txtime(cfg, clockid, ipver, args_tx, args_rx, expect_success):
cmd_rx = f"{cmd_base} {args_rx} -r"
cmd_tx = f"{cmd_base} {args_tx}"

+ expect_fail = not expect_success
+ if os.environ.get('KSFT_MACHINE_SLOW'):
+ expect_success = False
+
with bkg(cmd_rx, host=cfg.remote, fail=expect_success,
- expect_fail=(not expect_success), exit_wait=True):
+ expect_fail=expect_fail, exit_wait=True):
cmd(cmd_tx)