Skip to contentSkip to conversation

selftests: drv-net: support testing with a remote system

PATCH Superseded Jakub Kicinski 2 participants 9 messages Apr 18, 2024 – Apr 19, 2024

Subsystems (3)

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Jakub Kicinski

author maintainer
#0 lore raw

v4 cover selftests: drv-net: support testing with a remote system

Hi!

Implement support for tests which require access to a remote system / endpoint which can generate traffic.
This series concludes the "groundwork" for upstream driver tests.

I wanted to support the three models which came up in discussions:
- SW testing with netdevsim
- "local" testing with two ports on the same system in a loopback
- "remote" testing via SSH
so there is a tiny bit of an abstraction which wraps up how "remote" commands are executed. Otherwise hopefully there's nothing surprising.

I'm only adding a ping test. I had a bigger one written but I was worried we'll get into discussing the details of the test itself and how I chose to hack up netdevsim, instead of the test infra... So that test will be a follow up :)

v4:
- improve coding sytle of patch 5
- switch from netcat to socat (patch 6)
- support exit_wait for bkg() in context manager
- add require_XYZ() helpers (patch 7)
- increase timeouts a little (1,3 -> 5 sec)
v3: https://lore.kernel.org/all/[email protected]
- first two patches are new
- make Remote::cmd() return Popen() object (patch 3)
- always operate on absolute paths (patch 3)
- last two patches are new
v2: https://lore.kernel.org/all/[email protected]
- rename endpoint -> remote
- use 2001:db8:: v6 prefix
- add a note about persistent SSH connections
- add the kernel config
v1: https://lore.kernel.org/all/[email protected]

Jakub Kicinski (7):
selftests: drv-net: define endpoint structures
selftests: drv-net: factor out parsing of the env
selftests: drv-net: construct environment for running tests which
require an endpoint
selftests: drv-net: add a trivial ping test
selftests: net: support matching cases by name prefix
selftests: drv-net: add a TCP ping test case (and useful helpers)
selftests: drv-net: add require_XYZ() helpers for validating env

 tools/testing/selftests/drivers/net/Makefile  |   5 +-
 .../testing/selftests/drivers/net/README.rst  |  33 ++++
 .../selftests/drivers/net/lib/py/__init__.py  |   1 +
 .../selftests/drivers/net/lib/py/env.py       | 174 ++++++++++++++++--
 .../selftests/drivers/net/lib/py/remote.py    |  15 ++
 .../drivers/net/lib/py/remote_netns.py        |  21 +++
 .../drivers/net/lib/py/remote_ssh.py          |  39 ++++
 tools/testing/selftests/drivers/net/ping.py   |  54 ++++++
 .../testing/selftests/net/lib/py/__init__.py  |   1 +
 tools/testing/selftests/net/lib/py/ksft.py    |  13 +-
 tools/testing/selftests/net/lib/py/netns.py   |  31 ++++
 tools/testing/selftests/net/lib/py/utils.py   |  52 +++++-
 12 files changed, 410 insertions(+), 29 deletions(-)
 create mode 100644 tools/testing/selftests/drivers/net/lib/py/remote.py
 create mode 100644 tools/testing/selftests/drivers/net/lib/py/remote_netns.py
 create mode 100644 tools/testing/selftests/drivers/net/lib/py/remote_ssh.py
 create mode 100755 tools/testing/selftests/drivers/net/ping.py
 create mode 100644 tools/testing/selftests/net/lib/py/netns.py
Signature · 1 line
2.44.0
author maintainer
#1 lore raw

PATCH v4 1/7 selftests: drv-net: define endpoint structures

Define the remote endpoint "model". To execute most meaningful device driver tests we need to be able to communicate with a remote system, and have it send traffic to the device under test.

Various test environments will have different requirements.

0) "Local" netdevsim-based testing can simply use net namespaces. netdevsim supports connecting two devices now, to form a veth-like construct.

1) Similarly on hosts with multiple NICs, the NICs may be connected together with a loopback cable or internal device loopback. One interface may be placed into separate netns, and tests would proceed much like in the netdevsim case. Note that the loopback config or the moving of one interface
into a netns is not expected to be part of selftest code.

2) Some systems may need to communicate with the remote endpoint via SSH.

3) Last but not least environment may have its own custom communication method.

Fundamentally we only need two operations:
- run a command remotely
- deploy a binary (if some tool we need is built as part of kselftests)

Wrap these two in a class. Use dynamic loading to load the Remote class. This will allow very easy definition of other communication methods without bothering upstream code base.

Stick to the "simple" / "no unnecessary abstractions" model for referring to the remote endpoints. The host / remote object are passed as an argument to the usual cmd() or ip() invocation. For example:

ip("link show", json=True, host=remote)

---
v4:
- bump timeout to 5 sec, in case of really slow SSH
v3:
- make Remote() return Popen() object
- always operate on absolute paths
v2:
- rename Endpoint -> Remote

---
 .../selftests/drivers/net/lib/py/__init__.py  |  1 +
 .../selftests/drivers/net/lib/py/remote.py    | 15 +++++++
 .../drivers/net/lib/py/remote_netns.py        | 21 ++++++++++
 .../drivers/net/lib/py/remote_ssh.py          | 39 +++++++++++++++++++
 tools/testing/selftests/net/lib/py/utils.py   | 17 ++++----
 5 files changed, 85 insertions(+), 8 deletions(-)
 create mode 100644 tools/testing/selftests/drivers/net/lib/py/remote.py
 create mode 100644 tools/testing/selftests/drivers/net/lib/py/remote_netns.py
 create mode 100644 tools/testing/selftests/drivers/net/lib/py/remote_ssh.py
tools/testing/selftests/drivers/net/lib/py/__init__.py +1 -0
index 4653dffcd962..4789c1a4282d 100644
@@ -15,3 +15,4 @@ KSFT_DIR = (Path(__file__).parent / "../../../..").resolve()
     sys.exit(4)
 
 from .env import *
+from .remote import Remote
tools/testing/selftests/drivers/net/lib/py/remote.py new file +15 -0
new file mode 100644
index 000000000000..b1780b987722
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0
+
+import os
+import importlib
+
+_modules = {}
+
+def Remote(kind, args, src_path):
+    global _modules
+
+    if kind not in _modules:
+        _modules[kind] = importlib.import_module("..remote_" + kind, __name__)
+
+    dir_path = os.path.abspath(src_path + "/../")
+    return getattr(_modules[kind], "Remote")(args, dir_path)
tools/testing/selftests/drivers/net/lib/py/remote_netns.py new file +21 -0
new file mode 100644
index 000000000000..7d5eeb0271bc
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0
+
+import os
+import subprocess
+
+from lib.py import cmd
+
+
+class Remote:
+    def __init__(self, name, dir_path):
+        self.name = name
+        self.dir_path = dir_path
+
+    def cmd(self, comm):
+        return subprocess.Popen(["ip", "netns", "exec", self.name, "bash", "-c", comm],
+                                 stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+
+    def deploy(self, what):
+        if os.path.isabs(what):
+            return what
+        return os.path.abspath(self.dir_path + "/" + what)
tools/testing/selftests/drivers/net/lib/py/remote_ssh.py new file +39 -0
new file mode 100644
index 000000000000..924addde19a3
@@ -0,0 +1,39 @@
+# SPDX-License-Identifier: GPL-2.0
+
+import os
+import string
+import subprocess
+import random
+
+from lib.py import cmd
+
+
+class Remote:
+    def __init__(self, name, dir_path):
+        self.name = name
+        self.dir_path = dir_path
+        self._tmpdir = None
+
+    def __del__(self):
+        if self._tmpdir:
+            cmd("rm -rf " + self._tmpdir, host=self)
+            self._tmpdir = None
+
+    def cmd(self, comm):
+        return subprocess.Popen(["ssh", "-q", self.name, comm],
+                                stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+
+    def _mktmp(self):
+        return ''.join(random.choice(string.ascii_lowercase) for _ in range(8))
+
+    def deploy(self, what):
+        if not self._tmpdir:
+            self._tmpdir = "/tmp/" + self._mktmp()
+            cmd("mkdir " + self._tmpdir, host=self)
+        file_name = self._tmpdir + "/" + self._mktmp() + os.path.basename(what)
+
+        if not os.path.isabs(what):
+            what = os.path.abspath(self.dir_path + "/" + what)
+
+        cmd(f"scp {what} {self.name}:{file_name}")
+        return file_name
tools/testing/selftests/net/lib/py/utils.py +9 -8
index 19612348c30d..7347d0c0ff05 100644
@@ -4,10 +4,8 @@ import json as _json
 import subprocess
 
 class cmd:
-    def __init__(self, comm, shell=True, fail=True, ns=None, background=False):
+    def __init__(self, comm, shell=True, fail=True, ns=None, background=False, host=None):
         if ns:
-            if isinstance(ns, NetNS):
-                ns = ns.name
             comm = f'ip netns exec {ns} ' + comm
 
         self.stdout = None
@@ -15,15 +13,18 @@ import subprocess
         self.ret = None
 
         self.comm = comm
-        self.proc = subprocess.Popen(comm, shell=shell, stdout=subprocess.PIPE,
-                                     stderr=subprocess.PIPE)
+        if host:
+            self.proc = host.cmd(comm)
+        else:
+            self.proc = subprocess.Popen(comm, shell=shell, stdout=subprocess.PIPE,
+                                         stderr=subprocess.PIPE)
         if not background:
             self.process(terminate=False, fail=fail)
 
     def process(self, terminate=True, fail=None):
         if terminate:
             self.proc.terminate()
-        stdout, stderr = self.proc.communicate()
+        stdout, stderr = self.proc.communicate(timeout=5)
         self.stdout = stdout.decode("utf-8")
         self.stderr = stderr.decode("utf-8")
         self.proc.stdout.close()
@@ -37,12 +38,12 @@ import subprocess
                             (self.proc.args, stdout, stderr))
 
 
-def ip(args, json=None, ns=None):
+def ip(args, json=None, ns=None, host=None):
     cmd_str = "ip "
     if json:
         cmd_str += '-j '
     cmd_str += args
-    cmd_obj = cmd(cmd_str, ns=ns)
+    cmd_obj = cmd(cmd_str, ns=ns, host=host)
     if json:
         return _json.loads(cmd_obj.stdout)
     return cmd_obj
Signature · 1 line
2.44.0
author maintainer
#2 lore raw

PATCH v4 2/7 selftests: drv-net: factor out parsing of the env

In reply to Jakub Kicinski

The tests with a remote end will use a different class, for clarity, but will also need to parse the env. So factor parsing the env out to a function.

---
 .../selftests/drivers/net/lib/py/env.py       | 43 +++++++++++--------
 1 file changed, 26 insertions(+), 17 deletions(-)
tools/testing/selftests/drivers/net/lib/py/env.py +26 -17
index e1abe9491daf..a081e168f3db 100644
@@ -6,12 +6,36 @@ from pathlib import Path
 from lib.py import ip
 from lib.py import NetdevSimDev
 
+
+def _load_env_file(src_path):
+    env = os.environ.copy()
+
+    src_dir = Path(src_path).parent.resolve()
+    if not (src_dir / "net.config").exists():
+        return env
+
+    lexer = shlex.shlex(open((src_dir / "net.config").as_posix(), 'r').read())
+    k = None
+    for token in lexer:
+        if k is None:
+            k = token
+            env[k] = ""
+        elif token == "=":
+            pass
+        else:
+            env[k] = token
+            k = None
+    return env
+
+
 class NetDrvEnv:
+    """
+    Class for a single NIC / host env, with no remote end
+    """
     def __init__(self, src_path):
         self._ns = None
 
-        self.env = os.environ.copy()
-        self._load_env_file(src_path)
+        self.env = _load_env_file(src_path)
 
         if 'NETIF' in self.env:
             self.dev = ip("link show dev " + self.env['NETIF'], json=True)[0]
@@ -34,19 +58,4 @@ from lib.py import NetdevSimDev
             self._ns.remove()
             self._ns = None
 
-    def _load_env_file(self, src_path):
-        src_dir = Path(src_path).parent.resolve()
-        if not (src_dir / "net.config").exists():
-            return
 
-        lexer = shlex.shlex(open((src_dir / "net.config").as_posix(), 'r').read())
-        k = None
-        for token in lexer:
-            if k is None:
-                k = token
-                self.env[k] = ""
-            elif token == "=":
-                pass
-            else:
-                self.env[k] = token
-                k = None
Signature · 1 line
2.44.0
author maintainer
#3 lore raw

PATCH v4 3/7 selftests: drv-net: construct environment for running tests which require an endpoint

In reply to Jakub Kicinski

Nothing surprising here, hopefully. Wrap the variables from the environment into a class or spawn a netdevsim based env and pass it to the tests.

---
 .../testing/selftests/drivers/net/README.rst  | 33 +++++++
 .../selftests/drivers/net/lib/py/env.py       | 98 ++++++++++++++++++-
 .../testing/selftests/net/lib/py/__init__.py  |  1 +
 tools/testing/selftests/net/lib/py/netns.py   | 31 ++++++
 4 files changed, 162 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/net/lib/py/netns.py
tools/testing/selftests/drivers/net/README.rst +33 -0
index 5ef7c417d431..0cbab33dad1f 100644
@@ -23,8 +23,41 @@ Variables can be set in the environment or by creating a net.config
   # Variable set in a file
   NETIF=eth0
 
+Please note that the config parser is very simple, if there are
+any non-alphanumeric characters in the value it needs to be in
+double quotes.
+
 NETIF
 ~~~~~
 
 Name of the netdevice against which the test should be executed.
 When empty or not set software devices will be used.
+
+LOCAL_V4, LOCAL_V6, REMOTE_V4, REMOTE_V6
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Local and remote endpoint IP addresses.
+
+REMOTE_TYPE
+~~~~~~~~~~~
+
+Communication method used to run commands on the remote endpoint.
+Test framework has built-in support for ``netns`` and ``ssh`` channels.
+``netns`` assumes the "remote" interface is part of the same
+host, just moved to the specified netns.
+``ssh`` communicates with remote endpoint over ``ssh`` and ``scp``.
+Using persistent SSH connections is strongly encouraged to avoid
+the latency of SSH connection setup on every command.
+
+Communication methods are defined by classes in ``lib/py/remote_{name}.py``.
+It should be possible to add a new method without modifying any of
+the framework, by simply adding an appropriately named file to ``lib/py``.
+
+REMOTE_ARGS
+~~~~~~~~~~~
+
+Arguments used to construct the communication channel.
+Communication channel dependent::
+
+  for netns - name of the "remote" namespace
+  for ssh - name/address of the remote host
tools/testing/selftests/drivers/net/lib/py/env.py +97 -1

107 lines not on this page · raw message

tools/testing/selftests/net/lib/py/__init__.py +1 -0
index ded7102df18a..b6d498d125fe 100644
@@ -2,6 +2,7 @@
 
 from .consts import KSRC
 from .ksft import *
+from .netns import NetNS
 from .nsim import *
 from .utils import *
 from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily
tools/testing/selftests/net/lib/py/netns.py new file +31 -0
new file mode 100644
index 000000000000..ecff85f9074f
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: GPL-2.0
+
+from .utils import ip
+import random
+import string
+
+
+class NetNS:
+    def __init__(self, name=None):
+        if name:
+            self.name = name
+        else:
+            self.name = ''.join(random.choice(string.ascii_lowercase) for _ in range(8))
+        ip('netns add ' + self.name)
+
+    def __del__(self):
+        if self.name:
+            ip('netns del ' + self.name)
+            self.name = None
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, ex_type, ex_value, ex_tb):
+        self.__del__()
+
+    def __str__(self):
+        return self.name
+
+    def __repr__(self):
+        return f"NetNS({self.name})"
Signature · 1 line
2.44.0
author maintainer
#4 lore raw

PATCH v4 4/7 selftests: drv-net: add a trivial ping test

In reply to Jakub Kicinski

Add a very simple test for testing with a remote system. Both IPv4 and IPv6 connectivity is optional, later change will add checks to skip tests based on available addresses.

Using netdevsim:

 $ ./run_kselftest.sh -t drivers/net:ping.py
 TAP version 13
 1..1
 # timeout set to 45
 # selftests: drivers/net: ping.py
 # KTAP version 1
 # 1..2
 # ok 1 ping.test_v4
 # ok 2 ping.test_v6
 # # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0
 ok 1 selftests: drivers/net: ping.py

Command line SSH:

 $ NETIF=virbr0 REMOTE_TYPE=ssh [email protected] \
    LOCAL_V4=192.168.122.1 REMOTE_V4=192.168.122.123 \
    ./tools/testing/selftests/drivers/net/ping.py
 KTAP version 1
 1..2
 ok 1 ping.test_v4
 ok 2 ping.test_v6 # SKIP Test requires IPv6 connectivity
 # Totals: pass:1 fail:0 xfail:1 xpass:0 skip:0 error:0

Existing devices placed in netns (and using net.config):

 $ cat drivers/net/net.config
 NETIF=veth0
 REMOTE_TYPE=netns
 REMOTE_ARGS=red
 LOCAL_V4="192.168.1.1"
 REMOTE_V4="192.168.1.2"
 $ ./run_kselftest.sh -t drivers/net:ping.py
 TAP version 13
 1..1
 # timeout set to 45
 # selftests: drivers/net: ping.py
 # KTAP version 1
 # 1..2
 # ok 1 ping.test_v4
 # ok 2 ping.test_v6 # SKIP Test requires IPv6 connectivity
 # # Totals: pass:1 fail:0 xfail:1 xpass:0 skip:0 error:0

---
v4:
- move adding the "require_XYZ()" checks to last patch

---
 tools/testing/selftests/drivers/net/Makefile |  5 +++-
 tools/testing/selftests/drivers/net/ping.py  | 27 ++++++++++++++++++++
 2 files changed, 31 insertions(+), 1 deletion(-)
 create mode 100755 tools/testing/selftests/drivers/net/ping.py
tools/testing/selftests/drivers/net/Makefile +4 -1
index 379cdb1960a7..754ec643768a 100644
@@ -2,6 +2,9 @@
 
 TEST_INCLUDES := $(wildcard lib/py/*.py)
 
-TEST_PROGS := stats.py
+TEST_PROGS := \
+	ping.py \
+	stats.py \
+# end of TEST_PROGS
 
 include ../../lib.mk
tools/testing/selftests/drivers/net/ping.py new file +27 -0
new file mode 100755
index 000000000000..e75908d7c558
@@ -0,0 +1,27 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+
+from lib.py import ksft_run, ksft_exit
+from lib.py import NetDrvEpEnv
+from lib.py import cmd
+
+
+def test_v4(cfg) -> None:
+    cmd(f"ping -c 1 -W0.5 {cfg.remote_v4}")
+    cmd(f"ping -c 1 -W0.5 {cfg.v4}", host=cfg.remote)
+
+
+def test_v6(cfg) -> None:
+    cmd(f"ping -c 1 -W0.5 {cfg.remote_v6}")
+    cmd(f"ping -c 1 -W0.5 {cfg.v6}", host=cfg.remote)
+
+
+def main() -> None:
+    with NetDrvEpEnv(__file__) as cfg:
+        ksft_run([test_v4, test_v6],
+                 args=(cfg, ))
+    ksft_exit()
+
+
+if __name__ == "__main__":
+    main()
Signature · 1 line
2.44.0
author maintainer
#5 lore raw

PATCH v4 5/7 selftests: net: support matching cases by name prefix

In reply to Jakub Kicinski

While writing tests with a lot more cases I got tired of having to jump back and forth to add the name of the test to the ksft_run() list. Most unittest frameworks do some name matching, e.g. assume that functions with names starting with test_ are test cases.

Support similar flow in ksft_run(). Let the author list the desired prefixes. globals() need to be passed explicitly, IDK how to work around that.

---
v4:
- spell the code out a little to make it clearer

---
 tools/testing/selftests/drivers/net/ping.py |  3 +--
 tools/testing/selftests/net/lib/py/ksft.py  | 13 ++++++++++++-
 2 files changed, 13 insertions(+), 3 deletions(-)
tools/testing/selftests/drivers/net/ping.py +1 -2
index e75908d7c558..9f65a0764aab 100755
@@ -18,8 +18,7 @@ from lib.py import cmd
 
 def main() -> None:
     with NetDrvEpEnv(__file__) as cfg:
-        ksft_run([test_v4, test_v6],
-                 args=(cfg, ))
+        ksft_run(globs=globals(), case_pfx={"test_"}, args=(cfg, ))
     ksft_exit()
 
 
tools/testing/selftests/net/lib/py/ksft.py +12 -1
index 25f2572fa540..8018bf98a9d2 100644
@@ -81,7 +81,18 @@ KSFT_RESULT_ALL = True
     print(res)
 
 
-def ksft_run(cases, args=()):
+def ksft_run(cases=None, globs=None, case_pfx=None, args=()):
+    cases = cases or []
+
+    if globs and case_pfx:
+        for key, value in globs.items():
+            if not callable(value):
+                continue
+            for prefix in case_pfx:
+                if key.startswith(prefix):
+                    cases.append(value)
+                    break
+
     totals = {"pass": 0, "fail": 0, "skip": 0, "xfail": 0}
 
     print("KTAP version 1")
Signature · 1 line
2.44.0
author maintainer
#6 lore raw

PATCH v4 6/7 selftests: drv-net: add a TCP ping test case (and useful helpers)

In reply to Jakub Kicinski

More complex tests often have to spawn a background process, like a server which will respond to requests or tcpdump.

Add support for creating such processes using the with keyword:

with bkg("my-daemon", ..):

     # my-daemon is alive in this block

My initial thought was to add this support to cmd() directly but it runs the command in the constructor, so by the time we __enter__ it's too late to make sure we used "background=True".

Second useful helper transplanted from net_helper.sh is wait_port_listen().

The test itself uses socat, which insists on v6 addresses being wrapped in [], it's not the only command which requires this format, so add the wrapped address to env. The hope is to save test code from checking if address is v6.

---
 .../selftests/drivers/net/lib/py/env.py       |  4 +++
 tools/testing/selftests/drivers/net/ping.py   | 24 ++++++++++++-
 tools/testing/selftests/net/lib/py/utils.py   | 35 +++++++++++++++++++
 3 files changed, 62 insertions(+), 1 deletion(-)
tools/testing/selftests/drivers/net/lib/py/env.py +4 -0
index 579c5b34e6fd..2f62270d59fa 100644
@@ -110,6 +110,10 @@ from .remote import Remote
         self.addr = self.v6 if self.v6 else self.v4
         self.remote_addr = self.remote_v6 if self.remote_v6 else self.remote_v4
 
+        # Bracketed addresses, some commands need IPv6 to be inside []
+        self.baddr = f"[{self.v6}]" if self.v6 else self.v4
+        self.remote_baddr = f"[{self.remote_v6}]" if self.remote_v6 else self.remote_v4
+
         self.ifname = self.dev['ifname']
         self.ifindex = self.dev['ifindex']
 
tools/testing/selftests/drivers/net/ping.py +23 -1
index 9f65a0764aab..15a0bdcb46e2 100755
@@ -1,9 +1,12 @@
 #!/usr/bin/env python3
 # SPDX-License-Identifier: GPL-2.0
 
+import random
+
 from lib.py import ksft_run, ksft_exit
+from lib.py import ksft_eq
 from lib.py import NetDrvEpEnv
-from lib.py import cmd
+from lib.py import bkg, cmd, wait_port_listen
 
 
 def test_v4(cfg) -> None:
@@ -16,6 +19,25 @@ from lib.py import cmd
     cmd(f"ping -c 1 -W0.5 {cfg.v6}", host=cfg.remote)
 
 
+def test_tcp(cfg) -> None:
+    port = random.randrange(1024 + (1 << 15))
+
+    with bkg(f"socat -t 2 -u TCP-LISTEN:{port} STDOUT", exit_wait=True) as nc:
+        wait_port_listen(port)
+
+        cmd(f"echo ping | socat -t 2 -u STDIN TCP:{cfg.baddr}:{port}",
+            shell=True, host=cfg.remote)
+    ksft_eq(nc.stdout.strip(), "ping")
+
+    port = random.randrange(1024 + (1 << 15))
+    with bkg(f"socat -t 2 -u TCP-LISTEN:{port} STDOUT", host=cfg.remote,
+             exit_wait=True) as nc:
+        wait_port_listen(port, host=cfg.remote)
+
+        cmd(f"echo ping | socat -t 2 -u STDIN TCP:{cfg.remote_baddr}:{port}", shell=True)
+    ksft_eq(nc.stdout.strip(), "ping")
+
+
 def main() -> None:
     with NetDrvEpEnv(__file__) as cfg:
         ksft_run(globs=globals(), case_pfx={"test_"}, args=(cfg, ))
tools/testing/selftests/net/lib/py/utils.py +35 -0
index 7347d0c0ff05..92d9078c164a 100644
@@ -1,7 +1,10 @@
 # SPDX-License-Identifier: GPL-2.0
 
 import json as _json
+import re
 import subprocess
+import time
+
 
 class cmd:
     def __init__(self, comm, shell=True, fail=True, ns=None, background=False, host=None):
@@ -38,6 +41,20 @@ import subprocess
                             (self.proc.args, stdout, stderr))
 
 
+class bkg(cmd):
+    def __init__(self, comm, shell=True, fail=True, ns=None, host=None,
+                 exit_wait=False):
+        super().__init__(comm, background=True,
+                         shell=shell, fail=fail, ns=ns, host=host)
+        self.terminate = not exit_wait
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, ex_type, ex_value, ex_tb):
+        return self.process(terminate=self.terminate)
+
+
 def ip(args, json=None, ns=None, host=None):
     cmd_str = "ip "
     if json:
@@ -47,3 +64,21 @@ import subprocess
     if json:
         return _json.loads(cmd_obj.stdout)
     return cmd_obj
+
+
+def wait_port_listen(port, proto="tcp", ns=None, host=None, sleep=0.005, deadline=5):
+    end = time.monotonic() + deadline
+
+    pattern = f":{port:04X} .* "
+    if proto == "tcp": # for tcp protocol additionally check the socket state
+        pattern += "0A"
+    pattern = re.compile(pattern)
+
+    while True:
+        data = cmd(f'cat /proc/net/{proto}*', ns=ns, host=host, shell=True).stdout
+        for row in data.split("\n"):
+            if pattern.search(row):
+                return
+        if time.monotonic() > end:
+            raise Exception("Waiting for port listen timed out")
+        time.sleep(sleep)
Signature · 1 line
2.44.0
author maintainer
#8 lore raw

PATCH v4 7/7 selftests: drv-net: add require_XYZ() helpers for validating env

In reply to Jakub Kicinski

Wrap typical checks like whether given command used by the test is available in helpers.

---
 .../selftests/drivers/net/lib/py/env.py       | 29 ++++++++++++++++++-
 tools/testing/selftests/drivers/net/ping.py   |  6 ++++
 2 files changed, 34 insertions(+), 1 deletion(-)
tools/testing/selftests/drivers/net/lib/py/env.py +28 -1
index 2f62270d59fa..619dd214c095 100644
@@ -3,7 +3,8 @@
 import os
 import shlex
 from pathlib import Path
-from lib.py import ip
+from lib.py import KsftSkipEx
+from lib.py import cmd, ip
 from lib.py import NetNS, NetdevSimDev
 from .remote import Remote
 
@@ -117,6 +118,8 @@ from .remote import Remote
         self.ifname = self.dev['ifname']
         self.ifindex = self.dev['ifindex']
 
+        self._required_cmd = {}
+
     def create_local(self):
         self._netns = NetNS()
         self._ns = NetdevSimDev()
@@ -159,3 +162,27 @@ from .remote import Remote
         if self.remote:
             del self.remote
             self.remote = None
+
+    def require_v4(self):
+        if not self.v4 or not self.remote_v4:
+            raise KsftSkipEx("Test requires IPv4 connectivity")
+
+    def require_v6(self):
+        if not self.v6 or not self.remote_v6:
+            raise KsftSkipEx("Test requires IPv6 connectivity")
+
+    def _require_cmd(self, comm, key, host=None):
+        cached = self._required_cmd.get(comm, {})
+        if cached.get(key) is None:
+            cached[key] = cmd("command -v -- " + comm, fail=False,
+                              shell=True, host=host).ret == 0
+        self._required_cmd[comm] = cached
+        return cached[key]
+
+    def require_cmd(self, comm, local=True, remote=False):
+        if local:
+            if not self._require_cmd(comm, "local"):
+                raise KsftSkipEx("Test requires command: " + comm)
+        if remote:
+            if not self._require_cmd(comm, "remote"):
+                raise KsftSkipEx("Test requires (remote) command: " + comm)
tools/testing/selftests/drivers/net/ping.py +6 -0
index 15a0bdcb46e2..a933a154b341 100755
@@ -10,16 +10,22 @@ from lib.py import bkg, cmd, wait_port_listen
 
 
 def test_v4(cfg) -> None:
+    cfg.require_v4()
+
     cmd(f"ping -c 1 -W0.5 {cfg.remote_v4}")
     cmd(f"ping -c 1 -W0.5 {cfg.v4}", host=cfg.remote)
 
 
 def test_v6(cfg) -> None:
+    cfg.require_v6()
+
     cmd(f"ping -c 1 -W0.5 {cfg.remote_v6}")
     cmd(f"ping -c 1 -W0.5 {cfg.v6}", host=cfg.remote)
 
 
 def test_tcp(cfg) -> None:
+    cfg.require_cmd("socat", remote=True)
+
     port = random.randrange(1024 + (1 << 15))
 
     with bkg(f"socat -t 2 -u TCP-LISTEN:{port} STDOUT", exit_wait=True) as nc:
Signature · 1 line
2.44.0
#7 lore raw

Re: [PATCH net-next v4 6/7] selftests: drv-net: add a TCP ping test case (and useful helpers)

In reply to Jakub Kicinski
Quoted 61 lines · Jakub Kicinski

More complex tests often have to spawn a background process, like a server which will respond to requests or tcpdump.

Add support for creating such processes using the with keyword:

with bkg("my-daemon", ..):

     # my-daemon is alive in this block

My initial thought was to add this support to cmd() directly but it runs the command in the constructor, so by the time we __enter__ it's too late to make sure we used "background=True".

Second useful helper transplanted from net_helper.sh is wait_port_listen().

The test itself uses socat, which insists on v6 addresses being wrapped in [], it's not the only command which requires this format, so add the wrapped address to env. The hope is to save test code from checking if address is v6.

---
 .../selftests/drivers/net/lib/py/env.py       |  4 +++
 tools/testing/selftests/drivers/net/ping.py   | 24 ++++++++++++-
 tools/testing/selftests/net/lib/py/utils.py   | 35 +++++++++++++++++++
 3 files changed, 62 insertions(+), 1 deletion(-)
tools/testing/selftests/drivers/net/lib/py/env.py +4 -0
index 579c5b34e6fd..2f62270d59fa 100644
@@ -110,6 +110,10 @@ from .remote import Remote
         self.addr = self.v6 if self.v6 else self.v4
         self.remote_addr = self.remote_v6 if self.remote_v6 else self.remote_v4
 
+        # Bracketed addresses, some commands need IPv6 to be inside []
+        self.baddr = f"[{self.v6}]" if self.v6 else self.v4
+        self.remote_baddr = f"[{self.remote_v6}]" if self.remote_v6 else self.remote_v4
+
         self.ifname = self.dev['ifname']
         self.ifindex = self.dev['ifindex']
 
tools/testing/selftests/drivers/net/ping.py +4 -1
index 9f65a0764aab..15a0bdcb46e2 100755
@@ -1,9 +1,12 @@
 #!/usr/bin/env python3
 # SPDX-License-Identifier: GPL-2.0
 
+import random
+
 from lib.py import ksft_run, ksft_exit
+from lib.py import ksft_eq
 from lib.py import NetDrvEpEnv
-from lib.py import cmd
+from lib.py import bkg, cmd, wait_port_listen
 
 
 def test_v4(cfg) -> None:

tools/testing/selftests/drivers/net/ping.py from lib.py import cmd · line 23

Quoted 5 lines
tools/testing/selftests/drivers/net/ping.py +2 -0
@@ -16,6 +19,25 @@ from lib.py import cmd
     cmd(f"ping -c 1 -W0.5 {cfg.v6}", host=cfg.remote)
 
 
+def test_tcp(cfg) -> None:
+    port = random.randrange(1024 + (1 << 15))

Intended to be 1024 + random.randrange(1 << 15)?

tools/testing/selftests/drivers/net/ping.py from lib.py import cmd · line 25

Quoted 2 lines
tools/testing/selftests/drivers/net/ping.py +2 -0
+
+    with bkg(f"socat -t 2 -u TCP-LISTEN:{port} STDOUT", exit_wait=True) as nc:

Perhaps add reuseaddr to all tests with hand picked ports?

I guess that here the client initiates the close and hits the TIME_WAIT. But not sure.

The odds of consecutive tests picking the same random ports are low. But as we add tests and they are run in continuous testing, that can become an annoying source of flaky runs.

Quoted 15 lines
tools/testing/selftests/drivers/net/ping.py +15 -0
+        wait_port_listen(port)
+
+        cmd(f"echo ping | socat -t 2 -u STDIN TCP:{cfg.baddr}:{port}",
+            shell=True, host=cfg.remote)
+    ksft_eq(nc.stdout.strip(), "ping")
+
+    port = random.randrange(1024 + (1 << 15))
+    with bkg(f"socat -t 2 -u TCP-LISTEN:{port} STDOUT", host=cfg.remote,
+             exit_wait=True) as nc:
+        wait_port_listen(port, host=cfg.remote)
+
+        cmd(f"echo ping | socat -t 2 -u STDIN TCP:{cfg.remote_baddr}:{port}", shell=True)
+    ksft_eq(nc.stdout.strip(), "ping")
+
+