Skip to contentSkip to conversation

selftests: drv-net: Test queue stall upon reconfig

PATCH Applied Mohsin Bashir 2 participants 2 messages Jul 31 – Aug 4

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v5 1/1 selftests: drv-net: Test queue stall upon reconfig

From: Mohsin Bashir <[email protected]>

Add a reconfig_tx_stall test that detects the possibility of a TX stall after ring reconfiguration. The key observation is that drivers using netif_tx_start_all_queues() are prone to experiencing a stall when reconfiguration completes compared to drivers using netif_tx_wake_all_queues(). start_all_queues only clears DRV_XOFF, while wake_all_queues also calls __netif_schedule() to kick the qdisc. Without the kick, qdisc backlog present at reconfig time can stay stuck until a new trigger is issued.

The test caps the TX ring at 64 entries so it fills quickly, then installs FQ on a target TX queue and sends UDP packets with SO_TXTIME scheduled in the future. With napi_defer_hard_irqs slowing completions, the small ring can fill when FQ releases the burst, leaving requeued qdisc backlog with no FQ timer to rescue it. A subsequent ring reconfig must wake the queues to drain the backlog. Simply starting the queues can leave it stuck.

Some drivers lack backpressure on the TX path and may not be able to build up the qdisc backlog the test relies on. In that case report an expected failure (xfail) instead of a hard failure.

Testing on some of the existing drivers: Driver-A does not have the bug, Driver-B has the bug, Driver-C had the bug but it is fixed now.

Driver-A:
./drivers/net/ring_reconfig.py -t reconfig_tx_stall TAP version 13
1..1
Sent 1024 SO_TXTIME packets (+100ms)
Backlog before reconfig: 1176378 bytes
ok 1 ring_reconfig.reconfig_tx_stall
Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0

Driver-B:
TAP version 13
1..1
Sent 128 SO_TXTIME packets (+100ms)
Sent 128 SO_TXTIME packets (+200ms)
Backlog before reconfig: 148372 bytes
Check| At ./drivers/net/ring_reconfig.py, line 397, in reconfig_tx_stall:
Check| ksft_eq(0, backlog,
Check failed 0 != 148372 qdisc backlog stuck on queue 1 after ring ....
not ok 1 ring_reconfig.reconfig_tx_stall
Totals: pass:0 fail:1 xfail:0 xpass:0 skip:0 error:0

Driver-C:
TAP version 13
1..1
Sent 128 SO_TXTIME packets (+100ms)
Backlog before reconfig: 192278 bytes
ok 1 ring_reconfig.reconfig_tx_stall
Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0

---
Changelog:
V5:
- Probe for the smallest TX ring the driver supports starting with 32, doubling up to 1024
- Use a large socket send buffer (sized to the burst) so every in-flight skb stays charged to the send buffer
- Raise fq's per-flow limit. The default per-flow limit may not be enough
- Scale the burst to the probed ring since ring size is now determined at runtime

V4: https://lore.kernel.org/netdev/[email protected]

V3: https://lore.kernel.org/netdev/[email protected]

V2: https://lore.kernel.org/netdev/[email protected]

V1: https://lore.kernel.org/netdev/[email protected]

---
 tools/testing/selftests/drivers/net/config    |   3 +
 .../selftests/drivers/net/lib/py/env.py       |  11 +-
 .../selftests/drivers/net/ring_reconfig.py    | 255 +++++++++++++++++-
 3 files changed, 263 insertions(+), 6 deletions(-)
tools/testing/selftests/drivers/net/config +3 -0
index 2070e890e064..f3933cf3e6be 100644
@@ -4,8 +4,11 @@ CONFIG_DEBUG_INFO_BTF_MODULES=n
 CONFIG_INET_PSP=y
 CONFIG_IPV6=y
 CONFIG_MACSEC=m
+CONFIG_NET_ACT_SKBEDIT=m
 CONFIG_NET_CLS_ACT=y
 CONFIG_NET_CLS_BPF=y
+CONFIG_NET_CLS_FLOWER=m
+CONFIG_NET_CLS_MATCHALL=m
 CONFIG_NETCONSOLE=m
 CONFIG_NETCONSOLE_DYNAMIC=y
 CONFIG_NETCONSOLE_EXTENDED_LOG=y
tools/testing/selftests/drivers/net/lib/py/env.py +8 -3
index e4acf3d8333f..b1c6f6cef7dd 100644
@@ -114,10 +114,11 @@ class NetDrvEpEnv(NetDrvEnvBase):
     nsim_v4_pfx = "192.0.2."
     nsim_v6_pfx = "2001:db8::"
 
-    def __init__(self, src_path, nsim_test=None):
+    def __init__(self, src_path, nsim_test=None, queue_count=None):
         super().__init__(src_path)
 
         self._stats_settle_time = None
+        self._queue_count = queue_count
 
         # Things we try to destroy
         self.remote = None
@@ -173,9 +174,13 @@ class NetDrvEpEnv(NetDrvEnvBase):
         self._required_cmd = {}
 
     def create_local(self):
+        nsim_kwargs = {}
+        if self._queue_count:
+            nsim_kwargs["queue_count"] = self._queue_count
+
         self._netns = NetNS()
-        self._ns = NetdevSimDev()
-        self._ns_peer = NetdevSimDev(ns=self._netns)
+        self._ns = NetdevSimDev(**nsim_kwargs)
+        self._ns_peer = NetdevSimDev(ns=self._netns, **nsim_kwargs)
 
         with open("/proc/self/ns/net") as nsfd0, \
              open("/var/run/netns/" + self._netns.name) as nsfd1:
tools/testing/selftests/drivers/net/ring_reconfig.py +252 -3

276 lines not on this page · raw message

Signature · 1 line
2.53.0-Meta
#1 lore raw
This patch was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Hello:

This patch was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Quoted 11 lines · On Thu, 30 Jul 2026 19:15:42 -0700 you wrote:

From: Mohsin Bashir <[email protected]>

Add a reconfig_tx_stall test that detects the possibility of a TX stall after ring reconfiguration. The key observation is that drivers using netif_tx_start_all_queues() are prone to experiencing a stall when reconfiguration completes compared to drivers using netif_tx_wake_all_queues(). start_all_queues only clears DRV_XOFF, while wake_all_queues also calls __netif_schedule() to kick the qdisc. Without the kick, qdisc backlog present at reconfig time can stay stuck until a new trigger is issued.

Quoted 1 line

[...]

Here is the summary with links:
- [net-next,v5] selftests: drv-net: Test queue stall upon reconfig

    https://git.kernel.org/netdev/net-next/c/c98610c2eb70

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html