selftests: net: groundwork for YNL-based tests
Subsystems (6)
Jakub Kicinski
maintainerv2 cover selftests: net: groundwork for YNL-based tests
Currently the options for writing networking tests are C, bash or some mix of the two. YAML/Netlink gives us the ability to easily interface with Netlink in higher level laguages. In particular, there is a Python library already available in tree, under tools/net. Add the scaffolding which allows writing tests using this library.
The "scaffolding" is needed because the library lives under tools/net and uses YAML files from under Documentation/. So we need a small amount of glue code to find those things and add them to TEST_FILES.
This series adds both a basic SW sanity test and driver test which can be run against netdevsim or a real device. When I develop core code I usually test with netdevsim, then a real device, and then a backport to Meta's kernel. Because of the lack of integration, until now I had to throw away the (YNL-based) test script and netdevsim code.
Running tests in tree directly:
$ ./tools/testing/selftests/net/nl_netdev.py KTAP version 1 1..2 ok 1 nl_netdev.empty_check ok 2 nl_netdev.lo_check # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0
in tree via make:
$ make -C tools/testing/selftests/ TARGETS=net \ TEST_PROGS=nl_netdev.py TEST_GEN_PROGS="" run_tests [ ... ]
and installed externally, all seem to work:
$ make -C tools/testing/selftests/ TARGETS=net \ install INSTALL_PATH=/tmp/ksft-net $ /tmp/ksft-net/run_kselftest.sh -t net:nl_netdev.py [ ... ]
For driver tests I followed the lead of net/forwarding and get the device name from env and/or a config file.
v2: (see patches for minor changes)
- don't add to TARGETS, create a deperate variable with deps
- support and use with
- support and use passing arguments to tests
v1: https://lore.kernel.org/all/[email protected]/
Jakub Kicinski (7):
netlink: specs: define ethtool header flags
tools: ynl: copy netlink error to NlError
selftests: net: add scaffolding for Netlink tests in Python
selftests: nl_netdev: add a trivial Netlink netdev test
netdevsim: report stats by default, like a real device
selftests: drivers: add scaffolding for Netlink tests in Python
testing: net-drv: add a driver test for stats reporting
Documentation/netlink/specs/ethtool.yaml | 6 + drivers/net/netdevsim/ethtool.c | 11 ++ drivers/net/netdevsim/netdev.c | 45 +++++++ tools/net/ynl/lib/ynl.py | 3 +- tools/testing/selftests/Makefile | 10 +- tools/testing/selftests/drivers/net/Makefile | 7 ++ .../testing/selftests/drivers/net/README.rst | 30 +++++ .../selftests/drivers/net/lib/py/__init__.py | 17 +++ .../selftests/drivers/net/lib/py/env.py | 52 ++++++++ tools/testing/selftests/drivers/net/stats.py | 86 +++++++++++++ tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/lib/Makefile | 8 ++ .../testing/selftests/net/lib/py/__init__.py | 7 ++ tools/testing/selftests/net/lib/py/consts.py | 9 ++ tools/testing/selftests/net/lib/py/ksft.py | 96 ++++++++++++++ tools/testing/selftests/net/lib/py/nsim.py | 118 ++++++++++++++++++ tools/testing/selftests/net/lib/py/utils.py | 47 +++++++ tools/testing/selftests/net/lib/py/ynl.py | 49 ++++++++ tools/testing/selftests/net/nl_netdev.py | 24 ++++ 19 files changed, 624 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/drivers/net/Makefile create mode 100644 tools/testing/selftests/drivers/net/README.rst create mode 100644 tools/testing/selftests/drivers/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/drivers/net/lib/py/env.py create mode 100755 tools/testing/selftests/drivers/net/stats.py create mode 100644 tools/testing/selftests/net/lib/Makefile create mode 100644 tools/testing/selftests/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/net/lib/py/consts.py create mode 100644 tools/testing/selftests/net/lib/py/ksft.py create mode 100644 tools/testing/selftests/net/lib/py/nsim.py create mode 100644 tools/testing/selftests/net/lib/py/utils.py create mode 100644 tools/testing/selftests/net/lib/py/ynl.py create mode 100755 tools/testing/selftests/net/nl_netdev.py
Signature · 1 line
2.44.0
Jakub Kicinski
maintainerPATCH v2 1/7 netlink: specs: define ethtool header flags
When interfacing with the ethtool commands it's handy to be able to use the names of the flags. Example:
ethnl.pause_get({"header": {"dev-index": cfg.ifindex,
"flags": {'stats'}}})Note that not all commands accept all the flags, but the meaning of the bits does not change command to command.
---
v2:
- make sure we don't try to code gen enum (add enum-name:)
--- Documentation/netlink/specs/ethtool.yaml | 6 ++++++ 1 file changed, 6 insertions(+)
Documentation/netlink/specs/ethtool.yaml +6 -0
@@ -16,6 +16,11 @@ doc: Partial family for Ethtool Netlink. name: stringset type: enum entries: [] + - + name: header-flags + enum-name: + type: flags + entries: [ compact-bitsets, omit-reply, stats ] attribute-sets: -
@@ -30,6 +35,7 @@ doc: Partial family for Ethtool Netlink. - name: flags type: u32 + enum: header-flags - name: bitset-bit
Signature · 1 line
2.44.0
Jakub Kicinski
maintainerPATCH v2 2/7 tools: ynl: copy netlink error to NlError
In reply to Jakub KicinskiTyping e.nl_msg.error when processing exception is a bit tedious and counter-intuitive. Set a local .error member to the positive value of the netlink level error.
---
CC: [email protected]
CC: [email protected]
--- tools/net/ynl/lib/ynl.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/net/ynl/lib/ynl.py +2 -1
@@ -100,9 +100,10 @@ from .nlspec import SpecFamily class NlError(Exception): def __init__(self, nl_msg): self.nl_msg = nl_msg + self.error = -nl_msg.error def __str__(self): - return f"Netlink error: {os.strerror(-self.nl_msg.error)}\n{self.nl_msg}" + return f"Netlink error: {os.strerror(self.error)}\n{self.nl_msg}" class ConfigError(Exception):
Signature · 1 line
2.44.0
Jakub Kicinski
maintainerPATCH v2 3/7 selftests: net: add scaffolding for Netlink tests in Python
In reply to Jakub KicinskiAdd glue code for accessing the YNL library which lives under tools/net and YAML spec files from under Documentation/. Automatically figure out if tests are run in tree or not. Since we'll want to use this library both from net and drivers/net test targets make the library a target as well, and automatically include it when net or drivers/net are included. Making net/lib a target ensures that we end up with only one copy of it, and saves us some path guessing.
Add a tiny bit of formatting support to be able to output KTAP from the start.
---
v2:
- include the net/lib only in install
- support passing state to tests
- don't apply Path() on what's already a Path()
- fix spacing in Makefile's filter()
- sort imports
--- tools/testing/selftests/Makefile | 9 +- tools/testing/selftests/net/lib/Makefile | 8 ++ .../testing/selftests/net/lib/py/__init__.py | 6 ++ tools/testing/selftests/net/lib/py/consts.py | 9 ++ tools/testing/selftests/net/lib/py/ksft.py | 96 +++++++++++++++++++ tools/testing/selftests/net/lib/py/utils.py | 47 +++++++++ tools/testing/selftests/net/lib/py/ynl.py | 49 ++++++++++ 7 files changed, 223 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/net/lib/Makefile create mode 100644 tools/testing/selftests/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/net/lib/py/consts.py create mode 100644 tools/testing/selftests/net/lib/py/ksft.py create mode 100644 tools/testing/selftests/net/lib/py/utils.py create mode 100644 tools/testing/selftests/net/lib/py/ynl.py
tools/testing/selftests/Makefile +8 -1
@@ -116,6 +116,13 @@ TARGETS += zram TARGETS_HOTPLUG = cpu-hotplug TARGETS_HOTPLUG += memory-hotplug +# Networking tests want the net/lib target, include it automatically +ifneq ($(filter net,$(TARGETS)),) +ifeq ($(filter net/lib,$(TARGETS)),) + INSTALL_DEP_TARGETS := net/lib +endif +endif + # User can optionally provide a TARGETS skiplist. By default we skip # BPF since it has cutting edge build time dependencies which require # more effort to install.
@@ -245,7 +252,7 @@ ifdef INSTALL_PATH install -m 744 run_kselftest.sh $(INSTALL_PATH)/ rm -f $(TEST_LIST) @ret=1; \ - for TARGET in $(TARGETS); do \ + for TARGET in $(TARGETS) $(INSTALL_DEP_TARGETS); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET install \ INSTALL_PATH=$(INSTALL_PATH)/$$TARGET \
tools/testing/selftests/net/lib/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_FILES := ../../../../../Documentation/netlink/s* +TEST_FILES += ../../../../net/* + +TEST_INCLUDES := $(wildcard py/*.py) + +include ../../lib.mk
tools/testing/selftests/net/lib/py/__init__.py new file +6 -0
@@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0 + +from .consts import KSRC +from .ksft import * +from .utils import * +from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily
tools/testing/selftests/net/lib/py/consts.py new file +9 -0
@@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0 + +import sys +from pathlib import Path + +KSFT_DIR = (Path(__file__).parent / "../../..").resolve() +KSRC = (Path(__file__).parent / "../../../../../..").resolve() + +KSFT_MAIN_NAME = Path(sys.argv[0]).with_suffix("").name
tools/testing/selftests/net/lib/py/ksft.py new file +96 -0
96 lines not on this page · raw message
tools/testing/selftests/net/lib/py/utils.py new file +47 -0
47 lines not on this page · raw message
tools/testing/selftests/net/lib/py/ynl.py new file +49 -0
49 lines not on this page · raw message
Signature · 1 line
2.44.0
Jakub Kicinski
maintainerPATCH v2 4/7 selftests: nl_netdev: add a trivial Netlink netdev test
In reply to Jakub KicinskiAdd a trivial test using YNL.
$ ./tools/testing/selftests/net/nl_netdev.py KTAP version 1 1..2 ok 1 nl_netdev.empty_check ok 2 nl_netdev.lo_check
Instantiate the family once, it takes longer than the test itself.
---
v2:
- pass family as argument
--- tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/nl_netdev.py | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+) create mode 100755 tools/testing/selftests/net/nl_netdev.py
tools/testing/selftests/net/Makefile +1 -0
@@ -34,6 +34,7 @@ TEST_PROGS += gre_gso.sh TEST_PROGS += cmsg_so_mark.sh TEST_PROGS += cmsg_time.sh cmsg_ipv6.sh TEST_PROGS += netns-name.sh +TEST_PROGS += nl_netdev.py TEST_PROGS += srv6_end_dt46_l3vpn_test.sh TEST_PROGS += srv6_end_dt4_l3vpn_test.sh TEST_PROGS += srv6_end_dt6_l3vpn_test.sh
tools/testing/selftests/net/nl_netdev.py new file +24 -0
@@ -0,0 +1,24 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 + +from lib.py import ksft_run, ksft_pr, ksft_eq, ksft_ge, NetdevFamily + + +def empty_check(nf) -> None: + devs = nf.dev_get({}, dump=True) + ksft_ge(len(devs), 1) + + +def lo_check(nf) -> None: + lo_info = nf.dev_get({"ifindex": 1}) + ksft_eq(len(lo_info['xdp-features']), 0) + ksft_eq(len(lo_info['xdp-rx-metadata-features']), 0) + + +def main() -> None: + nf = NetdevFamily() + ksft_run([empty_check, lo_check], args=(nf, )) + + +if __name__ == "__main__": + main()
Signature · 1 line
2.44.0
Jakub Kicinski
maintainerPATCH v2 5/7 netdevsim: report stats by default, like a real device
In reply to Jakub KicinskiReal devices should implement qstats. Devices which support pause or FEC configuration should also report the relevant stats.
nsim was missing FEC stats completely, some of the qstats and pause stats required toggling a debugfs knob.
Note that the tests which used pause always initialize the setting so they shouldn't be affected by the different starting value.
--- drivers/net/netdevsim/ethtool.c | 11 ++++++++ drivers/net/netdevsim/netdev.c | 45 +++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+)
drivers/net/netdevsim/ethtool.c +11 -0
@@ -140,6 +140,13 @@ nsim_set_fecparam(struct net_device *dev, struct ethtool_fecparam *fecparam)return 0 ; } +static void +nsim_get_fec_stats (struct net_device *dev,struct ethtool_fec_stats *fec_stats) +{ + fec_stats->corrected_blocks.total =123 ; + fec_stats->uncorrectable_blocks.total =4 ; +} +static int nsim_get_ts_info (struct net_device *dev,struct ethtool_ts_info *info) {
@@ -163,6 +170,7 @@ static const struct ethtool_ops nsim_ethtool_ops = { .set_channels = nsim_set_channels, .get_fecparam = nsim_get_fecparam, .set_fecparam = nsim_set_fecparam, + .get_fec_stats = nsim_get_fec_stats, .get_ts_info = nsim_get_ts_info, };
@@ -182,6 +190,9 @@ void nsim_ethtool_init(struct netdevsim *ns)nsim_ethtool_ring_init (ns); + ns->ethtool.pauseparam.report_stats_rx = true; + ns->ethtool.pauseparam.report_stats_tx = true; + ns->ethtool.fec.fec =ETHTOOL_FEC_NONE ; ns->ethtool.fec.active_fec =ETHTOOL_FEC_NONE ;
drivers/net/netdevsim/netdev.c +45 -0
63 lines not on this page · raw message
Signature · 1 line
2.44.0
Jakub Kicinski
maintainerPATCH v2 6/7 selftests: drivers: add scaffolding for Netlink tests in Python
In reply to Jakub KicinskiAdd drivers/net as a target for mixed-use tests.
The setup is expected to work similarly to the forwarding tests. Since we only need one interface (unlike forwarding tests) read the target device name from NETIF. If not present we'll try to run the test against netdevsim.
---
v2:
- mark regex as r""
- add support of using env as with
--- tools/testing/selftests/Makefile | 3 +- tools/testing/selftests/drivers/net/Makefile | 7 ++ .../testing/selftests/drivers/net/README.rst | 30 +++++ .../selftests/drivers/net/lib/py/__init__.py | 17 +++ .../selftests/drivers/net/lib/py/env.py | 52 ++++++++ .../testing/selftests/net/lib/py/__init__.py | 1 + tools/testing/selftests/net/lib/py/nsim.py | 118 ++++++++++++++++++ 7 files changed, 227 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/drivers/net/Makefile create mode 100644 tools/testing/selftests/drivers/net/README.rst create mode 100644 tools/testing/selftests/drivers/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/drivers/net/lib/py/env.py create mode 100644 tools/testing/selftests/net/lib/py/nsim.py
tools/testing/selftests/Makefile +2 -1
@@ -17,6 +17,7 @@ TARGETS += devices TARGETS += dmabuf-heaps TARGETS += drivers/dma-buf TARGETS += drivers/s390x/uvdevice +TARGETS += drivers/net TARGETS += drivers/net/bonding TARGETS += drivers/net/team TARGETS += dt
@@ -117,7 +118,7 @@ TARGETS_HOTPLUG = cpu-hotplug TARGETS_HOTPLUG += memory-hotplug # Networking tests want the net/lib target, include it automatically -ifneq ($(filter net,$(TARGETS)),) +ifneq ($(filter net drivers/net,$(TARGETS)),) ifeq ($(filter net/lib,$(TARGETS)),) INSTALL_DEP_TARGETS := net/lib endif
tools/testing/selftests/drivers/net/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_INCLUDES := $(wildcard lib/py/*.py) + +TEST_PROGS := stats.py + +include ../../lib.mk
tools/testing/selftests/drivers/net/README.rst new file +30 -0
@@ -0,0 +1,30 @@ +Running tests +============= + +Tests are executed within kselftest framework like any other tests. +By default tests execute against software drivers such as netdevsim. +All tests must support running against a real device (SW-only tests +should instead be placed in net/ or drivers/net/netdevsim, HW-only +tests in drivers/net/hw). + +Set appropriate variables to point the tests at a real device. + +Variables +========= + +Variables can be set in the environment or by creating a net.config +file in the same directory as this README file. Example:: + + $ NETIF=eth0 ./some_test.sh + +or:: + + $ cat tools/testing/selftests/drivers/net/net.config + # Variable set in a file + NETIF=eth0 + +NETIF +~~~~~ + +Name of the netdevice against which the test should be executed. +When empty or not set software devices will be used.
tools/testing/selftests/drivers/net/lib/py/__init__.py new file +17 -0
@@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0 + +import sys +from pathlib import Path + +KSFT_DIR = (Path(__file__).parent / "../../../..").resolve() + +try: + sys.path.append(KSFT_DIR.as_posix()) + from net.lib.py import * +except ModuleNotFoundError as e: + ksft_pr("Failed importing `net` library from kernel sources") + ksft_pr(str(e)) + ktap_result(True, comment="SKIP") + sys.exit(4) + +from .env import *
tools/testing/selftests/drivers/net/lib/py/env.py new file +52 -0
52 lines not on this page · raw message
tools/testing/selftests/net/lib/py/__init__.py +1 -0
@@ -2,5 +2,6 @@ from .consts import KSRC from .ksft import * +from .nsim import * from .utils import * from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily
tools/testing/selftests/net/lib/py/nsim.py new file +118 -0
118 lines not on this page · raw message
Signature · 1 line
2.44.0
Jakub Kicinski
maintainerPATCH v2 7/7 testing: net-drv: add a driver test for stats reporting
In reply to Jakub KicinskiAdd a very simple test to make sure drivers report expected stats. Drivers which implement FEC or pause configuration should report relevant stats. Qstats must be reported, at least packet and byte counts, and they must match total device stats.
Tested with netdevsim, bnxt, in-tree and installed.
---
v2:
- pass cfg as argument
- use with
--- tools/testing/selftests/drivers/net/stats.py | 86 ++++++++++++++++++++ 1 file changed, 86 insertions(+) create mode 100755 tools/testing/selftests/drivers/net/stats.py
tools/testing/selftests/drivers/net/stats.py new file +86 -0
@@ -0,0 +1,86 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 + +from lib.py import ksft_run, ksft_in, ksft_true, KsftSkipEx, KsftXfailEx +from lib.py import EthtoolFamily, NetdevFamily, RtnlFamily, NlError +from lib.py import NetDrvEnv + +ethnl = EthtoolFamily() +netfam = NetdevFamily() +rtnl = RtnlFamily() + + +def check_pause(cfg) -> None: + global ethnl + + try: + ethnl.pause_get({"header": {"dev-index": cfg.ifindex}}) + except NlError as e: + if e.error == 95: + raise KsftXfailEx("pause not supported by the device") + raise + + data = ethnl.pause_get({"header": {"dev-index": cfg.ifindex, + "flags": {'stats'}}}) + ksft_true(data['stats'], "driver does not report stats") + + +def check_fec(cfg) -> None: + global ethnl + + try: + ethnl.fec_get({"header": {"dev-index": cfg.ifindex}}) + except NlError as e: + if e.error == 95: + raise KsftXfailEx("FEC not supported by the device") + raise + + data = ethnl.fec_get({"header": {"dev-index": cfg.ifindex, + "flags": {'stats'}}}) + ksft_true(data['stats'], "driver does not report stats") + + +def pkt_byte_sum(cfg) -> None: + global netfam, rtnl + + def get_qstat(test): + global netfam + stats = netfam.qstats_get({}, dump=True) + if stats: + for qs in stats: + if qs["ifindex"]== test.ifindex: + return qs + + qstat = get_qstat(cfg) + if qstat is None: + raise KsftSkipEx("qstats not supported by the device") + + for key in ['tx-packets', 'tx-bytes', 'rx-packets', 'rx-bytes']: + ksft_in(key, qstat, "Drivers should always report basic keys") + + # Compare stats, rtnl stats and qstats must match, + # but the interface may be up, so do a series of dumps + # each time the more "recent" stats must be higher or same. + def stat_cmp(rstat, qstat): + for key in ['tx-packets', 'tx-bytes', 'rx-packets', 'rx-bytes']: + if rstat[key] != qstat[key]: + return rstat[key] - qstat[key] + return 0 + + for _ in range(10): + rtstat = rtnl.getlink({"ifi-index": cfg.ifindex})['stats'] + if stat_cmp(rtstat, qstat) < 0: + raise Exception("RTNL stats are lower, fetched later") + qstat = get_qstat(cfg) + if stat_cmp(rtstat, qstat) > 0: + raise Exception("Qstats are lower, fetched later") + + +def main() -> None: + with NetDrvEnv(__file__) as cfg: + ksft_run([check_pause, check_fec, pkt_byte_sum], + args=(cfg, )) + + +if __name__ == "__main__": + main()
Signature · 1 line
2.44.0
Re: [PATCH net-next v2 1/7] netlink: specs: define ethtool header flags
In reply to Jakub KicinskiQuoted 11 lines · Jakub Kicinski
When interfacing with the ethtool commands it's handy to be able to use the names of the flags. Example:
ethnl.pause_get({"header": {"dev-index": cfg.ifindex, "flags": {'stats'}}})Note that not all commands accept all the flags, but the meaning of the bits does not change command to command.
Signed-off-by Jakub Kicinski <[email protected]>
Re: [PATCH net-next v2 2/7] tools: ynl: copy netlink error to NlError
In reply to Jakub KicinskiQuoted 5 lines · Jakub Kicinski
Typing e.nl_msg.error when processing exception is a bit tedious and counter-intuitive. Set a local .error member to the positive value of the netlink level error.
Signed-off-by Jakub Kicinski <[email protected]>
Re: [PATCH net-next v2 3/7] selftests: net: add scaffolding for Netlink tests in Python
In reply to Jakub KicinskiQuoted 72 lines · Jakub Kicinski
Add glue code for accessing the YNL library which lives under
tools/netand YAML spec files from underDocumentation/. Automatically figure out if tests are run in tree or not. Since we'll want to use this library both from net anddrivers/nettest targets make the library a target as well, and automatically include it when net ordrivers/netare included. Making net/lib a target ensures that we end up with only one copy of it, and saves us some path guessing.Add a tiny bit of formatting support to be able to output KTAP from the start.
Signed-off-by Jakub Kicinski <[email protected]>---
v2:
- include the net/lib only in install
- support passing state to tests
- don't applyPath()on what's already aPath()
- fix spacing in Makefile'sfilter()
- sort imports--- tools/testing/selftests/Makefile | 9 +- tools/testing/selftests/net/lib/Makefile | 8 ++ .../testing/selftests/net/lib/py/__init__.py | 6 ++ tools/testing/selftests/net/lib/py/consts.py | 9 ++ tools/testing/selftests/net/lib/py/ksft.py | 96 +++++++++++++++++++ tools/testing/selftests/net/lib/py/utils.py | 47 +++++++++ tools/testing/selftests/net/lib/py/ynl.py | 49 ++++++++++ 7 files changed, 223 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/net/lib/Makefile create mode 100644 tools/testing/selftests/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/net/lib/py/consts.py create mode 100644 tools/testing/selftests/net/lib/py/ksft.py create mode 100644 tools/testing/selftests/net/lib/py/utils.py create mode 100644 tools/testing/selftests/net/lib/py/ynl.pytools/testing/selftests/Makefile +8
-1@@ -116,6 +116,13 @@ TARGETS += zram TARGETS_HOTPLUG = cpu-hotplug TARGETS_HOTPLUG += memory-hotplug +# Networking tests want the net/lib target, include it automatically +ifneq ($(filter net,$(TARGETS)),) +ifeq ($(filter net/lib,$(TARGETS)),) + INSTALL_DEP_TARGETS := net/lib +endif +endif + # User can optionally provide a TARGETS skiplist. By default we skip # BPF since it has cutting edge build time dependencies which require # more effort to install.@@ -245,7 +252,7 @@ ifdef INSTALL_PATH install -m 744 run_kselftest.sh $(INSTALL_PATH)/ rm -f $(TEST_LIST) @ret=1; \ - for TARGET in $(TARGETS); do \ + for TARGET in $(TARGETS) $(INSTALL_DEP_TARGETS); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET install \ INSTALL_PATH=$(INSTALL_PATH)/$$TARGET \tools/testing/selftests/net/lib/Makefile new file +0
-0
tools/testing/selftests/net/lib/Makefile line 3
Quoted 3 lines
tools/testing/selftests/net/lib/Makefile new file +3
-0@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_FILES := ../../../../../Documentation/netlink/s*
Not sure I understand the motivation for the wildcard. Currently this matches Documentation/netlink/specs. Do you expect everything that starts with s to be a testfile?
tools/testing/selftests/net/lib/Makefile line 4
Quoted 1 line
tools/testing/selftests/net/lib/Makefile new file +1
-0+TEST_FILES += ../../../../net/*
Likewise this -- it's just tools/net/ynl? Will everything that's there be a testfile?
tools/testing/selftests/net/lib/Makefile line 8
Quoted 3 lines
tools/testing/selftests/net/lib/Makefile new file +3
-0+TEST_INCLUDES := $(wildcard py/*.py) + +include ../../lib.mk
Other than that it looks OK.
Re: [PATCH net-next v2 4/7] selftests: nl_netdev: add a trivial Netlink netdev test
In reply to Jakub KicinskiQuoted 11 lines · Jakub Kicinski
Add a trivial test using YNL.
$ ./tools/testing/selftests/net/nl_netdev.py KTAP version 1 1..2 ok 1 nl_netdev.empty_check ok 2 nl_netdev.lo_checkInstantiate the family once, it takes longer than the test itself.
Signed-off-by Jakub Kicinski <[email protected]>
Re: [PATCH net-next v2 5/7] netdevsim: report stats by default, like a real device
In reply to Jakub KicinskiQuoted 10 lines · Jakub Kicinski
Real devices should implement qstats. Devices which support pause or FEC configuration should also report the relevant stats.
nsim was missing FEC stats completely, some of the qstats and pause stats required toggling a debugfs knob.
Note that the tests which used pause always initialize the setting so they shouldn't be affected by the different starting value.
Signed-off-by Jakub Kicinski <[email protected]>
Just:
drivers/net/netdevsim/netdev.c static const struct net_device_ops nsim_vf_netdev_ops = { · line 334
Quoted 5 lines
drivers/net/netdevsim/netdev.c +1
-0@@ -330,6 +331,49 @@ static const struct net_device_ops nsim_vf_netdev_ops = { .ndo_set_features = nsim_set_features, }; +/* We don't have true par-queue stats, yet, so do some random fakery here. */
per
drivers/net/netdevsim/netdev.c static const struct net_device_ops nsim_vf_netdev_ops = { · line 342
Quoted 8 lines
drivers/net/netdevsim/netdev.c +8
-0+static void nsim_get_queue_stats_rx (struct net_device *dev,int idx, +struct netdev_queue_stats_rx *stats) +{ +struct rtnl_link_stats64 rtstats = {}; + +nsim_get_stats64 (dev, &rtstats); + + stats->packets = rtstats.rx_packets - !!rtstats.rx_packets;
This is just to make sure that per-queue stats are lower than the overall rtstats I presume?
Quoted 13 lines
drivers/net/netdevsim/netdev.c +13
-0+ stats->bytes = rtstats.rx_bytes; +} + +static void nsim_get_queue_stats_tx (struct net_device *dev,int idx, +struct netdev_queue_stats_tx *stats) +{ +struct rtnl_link_stats64 rtstats = {}; + +nsim_get_stats64 (dev, &rtstats); + + stats->packets = rtstats.tx_packets - !!rtstats.tx_packets; + stats->bytes = rtstats.tx_bytes; +}
Re: [PATCH net-next v2 6/7] selftests: drivers: add scaffolding for Netlink tests in Python
In reply to Jakub KicinskiQuoted 7 lines · Jakub Kicinski
Add
drivers/netas a target for mixed-use tests.
The setup is expected to work similarly to the forwarding tests. Since we only need one interface (unlike forwarding tests) read the target device name from NETIF. If not present we'll try to run the test against netdevsim.Signed-off-by Jakub Kicinski <[email protected]>
However:
Quoted 26 lines
tools/testing/selftests/net/lib/py/nsim.py new file +20
-0@@ -0,0 +1,118 @@ +# SPDX-License-Identifier: GPL-2.0 + +import json +import os +import random +import re +import time +from .utils import cmd, ip + + +class NetdevSim: + """ + Class for netdevsim netdevice and its attributes. + """ + + def __init__(self, nsimdev, port_index, ifname, ns=None): + # In case udev renamed the netdev to according to new schema, + # check if the name matches the port_index. + nsimnamere = re.compile(r"eni\d+np(\d+)") + match = nsimnamere.match(ifname)
tools/testing/selftests/net/lib/py/nsim.py line 30
Quoted 10 lines
tools/testing/selftests/net/lib/py/nsim.py new file +10
-0+ if match and int(match.groups()[0]) != port_index + 1: + raise Exception("netdevice name mismatches the expected one") + + self.ifname = ifname + self.nsimdev = nsimdev + self.port_index = port_index + self.ns = ns + self.dfs_dir = "%s/ports/%u/" % (nsimdev.dfs_dir, port_index) + ret = ip("-j link show dev %s" % ifname, ns=ns) + self.dev = json.loads(ret.stdout)[0]
I don't think self.ifname, .ns, .dfs_dir, .dev are actually used outside of this function.
Quoted 12 lines
tools/testing/selftests/net/lib/py/nsim.py new file +12
-0+ + def dfs_write(self, path, val): + self.nsimdev.dfs_write(f'ports/{self.port_index}/' + path, val) + + +class NetdevSimDev: + """ + Class for netdevsim bus device and its attributes. + """ + @staticmethod + def ctrl_write(path, val): + fullpath = os.path.join("/sys/bus/netdevsim/", path)
tools/testing/selftests/net/lib/py/nsim.py line 52
Quoted 10 lines
tools/testing/selftests/net/lib/py/nsim.py new file +10
-0+ with open(fullpath, "w") as f: + f.write(val) + + def dfs_write(self, path, val): + fullpath = os.path.join(f"/sys/kernel/debug/netdevsim/netdevsim{self.addr}/", path) + with open(fullpath, "w") as f: + f.write(val) + + def __init__(self, port_count=1, ns=None): + # nsim will spawn in init_net, we'll set to actual ns once we switch it the.sre
the.sre?
Paolo Abeni
maintainerRe: [PATCH net-next v2 5/7] netdevsim: report stats by default, like a real device
In reply to Jakub KicinskiQuoted 78 lines · Jakub Kicinski
Real devices should implement qstats. Devices which support pause or FEC configuration should also report the relevant stats.
nsim was missing FEC stats completely, some of the qstats and pause stats required toggling a debugfs knob.
Note that the tests which used pause always initialize the setting so they shouldn't be affected by the different starting value.
Signed-off-by Jakub Kicinski <[email protected]>--- drivers/net/netdevsim/ethtool.c | 11 ++++++++ drivers/net/netdevsim/netdev.c | 45 +++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+)drivers/net/netdevsim/ethtool.c +11
-0@@ -140,6 +140,13 @@ nsim_set_fecparam(struct net_device *dev, struct ethtool_fecparam *fecparam)return 0 ; } +static void +nsim_get_fec_stats (struct net_device *dev,struct ethtool_fec_stats *fec_stats) +{ + fec_stats->corrected_blocks.total =123 ; + fec_stats->uncorrectable_blocks.total =4 ; +} +static int nsim_get_ts_info (struct net_device *dev,struct ethtool_ts_info *info) {@@ -163,6 +170,7 @@ static const struct ethtool_ops nsim_ethtool_ops = { .set_channels = nsim_set_channels, .get_fecparam = nsim_get_fecparam, .set_fecparam = nsim_set_fecparam, + .get_fec_stats = nsim_get_fec_stats, .get_ts_info = nsim_get_ts_info, };@@ -182,6 +190,9 @@ void nsim_ethtool_init(struct netdevsim *ns)nsim_ethtool_ring_init (ns); + ns->ethtool.pauseparam.report_stats_rx = true; + ns->ethtool.pauseparam.report_stats_tx = true; + ns->ethtool.fec.fec =ETHTOOL_FEC_NONE ; ns->ethtool.fec.active_fec =ETHTOOL_FEC_NONE ;drivers/net/netdevsim/netdev.c +12
-0@@ -19,6 +19,7 @@#include <linux/module.h> #include <linux/netdevice.h> #include <linux/slab.h> +#include <net/netdev_queues.h> #include <net/netlink.h> #include <net/pkt_cls.h> #include <net/rtnetlink.h> @@ -330,6 +331,49 @@ static const struct net_device_ops nsim_vf_netdev_ops = { .ndo_set_features = nsim_set_features, }; +/* We don't have true par-queue stats, yet, so do some random fakery here. */ +static void nsim_get_queue_stats_rx (struct net_device *dev,int idx, +struct netdev_queue_stats_rx *stats) +{ +struct rtnl_link_stats64 rtstats = {}; + +nsim_get_stats64 (dev, &rtstats); + + stats->packets = rtstats.rx_packets - !!rtstats.rx_packets; + stats->bytes = rtstats.rx_bytes; +}
drivers/net/netdevsim/netdev.c static const struct net_device_ops nsim_vf_netdev_ops = { · line 354
Quoted 10 lines
drivers/net/netdevsim/netdev.c +10
-0+ +static void nsim_get_queue_stats_tx (struct net_device *dev,int idx, +struct netdev_queue_stats_tx *stats) +{ +struct rtnl_link_stats64 rtstats = {}; + +nsim_get_stats64 (dev, &rtstats); + + stats->packets = rtstats.tx_packets - !!rtstats.tx_packets; + stats->bytes = rtstats.tx_bytes;
It looks the stats will not be self-consistent with multiple queues enabled.
What about zeroing 'stats' when idx > 0 ?
/P
Jakub Kicinski
maintainerRe: [PATCH net-next v2 5/7] netdevsim: report stats by default, like a real device
In reply to Petr MachataQuoted 4 lines · On Thu, 4 Apr 2024 12:40:04 +0200 Petr Machata wrote:
drivers/net/netdevsim/netdev.c static const struct net_device_ops nsim_vf_netdev_ops = { · line 342
Quoted 1 line
drivers/net/netdevsim/netdev.c +1
-0+ stats->packets = rtstats.rx_packets - !!rtstats.rx_packets;This is just to make sure that per-queue stats are lower than the overall rtstats I presume?
Yes, to fake some "non-queue" stats.
Jakub Kicinski
maintainerRe: [PATCH net-next v2 5/7] netdevsim: report stats by default, like a real device
In reply to Paolo AbeniQuoted 9 lines · On Thu, 04 Apr 2024 15:40:33 +0200 Paolo Abeni wrote:
drivers/net/netdevsim/netdev.c static const struct net_device_ops nsim_vf_netdev_ops = { · line 354
Quoted 4 lines
drivers/net/netdevsim/netdev.c +4
-0+nsim_get_stats64 (dev, &rtstats); + + stats->packets = rtstats.tx_packets - !!rtstats.tx_packets; + stats->bytes = rtstats.tx_bytes;It looks the stats will not be self-consistent with multiple queues enabled.
What about zeroing 'stats' when idx > 0 ?
Ah, good catch. We do allow allocating multiple queues, even tho we don't do anything with them on the driver side, yet..
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Quoted 6 lines · On Tue, 2 Apr 2024 19:34:19 -0700 you wrote:
Currently the options for writing networking tests are C, bash or some mix of the two. YAML/Netlink gives us the ability to easily interface with Netlink in higher level laguages. In particular, there is a Python library already available in tree, under
tools/net. Add the scaffolding which allows writing tests using this library.
Quoted 6 lines
The "scaffolding" is needed because the library lives under
tools/netand uses YAML files from underDocumentation/. So we need a small amount of glue code to find those things and add them toTEST_FILES.[...]
Here is the summary with links:
- [net-next,v2,1/7] netlink: specs: define ethtool header flags
https://git.kernel.org/netdev/net-next/c/1d056bf9a4c1 - [net-next,v2,2/7] tools: ynl: copy netlink error to NlError https://git.kernel.org/netdev/net-next/c/b269d2b4a523
- [net-next,v2,3/7] selftests: net: add scaffolding for Netlink tests in Python (no matching commit)
- [net-next,v2,4/7] selftests: nl_netdev: add a trivial Netlink netdev test (no matching commit)
- [net-next,v2,5/7] netdevsim: report stats by default, like a real device (no matching commit)
- [net-next,v2,6/7] selftests: drivers: add scaffolding for Netlink tests in Python (no matching commit)
- [net-next,v2,7/7] testing: net-drv: add a driver test for stats reporting (no matching commit)
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
Jakub Kicinski
maintainerRe: [PATCH net-next v2 6/7] selftests: drivers: add scaffolding for Netlink tests in Python
In reply to Petr MachataQuoted 16 lines · On Thu, 4 Apr 2024 12:42:24 +0200 Petr Machata wrote:
Quoted 5 lines
tools/testing/selftests/net/lib/py/nsim.py new file +5
-0+ def __init__(self, nsimdev, port_index, ifname, ns=None): + # In case udev renamed the netdev to according to new schema, + # check if the name matches the port_index. + nsimnamere = re.compile(r"eni\d+np(\d+)") + match = nsimnamere.match(ifname)tools/testing/selftests/net/lib/py/nsim.py line 30
Quoted 10 lines
tools/testing/selftests/net/lib/py/nsim.py new file +10
-0+ if match and int(match.groups()[0]) != port_index + 1: + raise Exception("netdevice name mismatches the expected one") + + self.ifname = ifname + self.nsimdev = nsimdev + self.port_index = port_index + self.ns = ns + self.dfs_dir = "%s/ports/%u/" % (nsimdev.dfs_dir, port_index) + ret = ip("-j link show dev %s" % ifname, ns=ns) + self.dev = json.loads(ret.stdout)[0]
Quoted 2 lines
I don't think self.ifname, .ns, .dfs_dir, .dev are actually used outside of this function.
Right, one of the "further down the line" tests need these. I'll remove for now.
Quoted 15 lines
Quoted 15 lines
tools/testing/selftests/net/lib/py/nsim.py new file +15
-0+ def dfs_write(self, path, val): + self.nsimdev.dfs_write(f'ports/{self.port_index}/' + path, val) + + +class NetdevSimDev: + """ + Class for netdevsim bus device and its attributes. + """ + @staticmethod + def ctrl_write(path, val): + fullpath = os.path.join("/sys/bus/netdevsim/", path) + with open(fullpath, "w") as f: + f.write(val) + + def dfs_write(self, path, val):
Quoted 8 lines
tools/testing/selftests/net/lib/py/nsim.py line 52
Quoted 6 lines
tools/testing/selftests/net/lib/py/nsim.py new file +6
-0+ fullpath = os.path.join(f"/sys/kernel/debug/netdevsim/netdevsim{self.addr}/", path) + with open(fullpath, "w") as f: + f.write(val) + + def __init__(self, port_count=1, ns=None): + # nsim will spawn in init_net, we'll set to actual ns once we switch it the.srethe.sre?
Ha, must have started typing with focus on the wrong window. Good it wasn't my password :D