Skip to contentSkip to conversation

selftests: drv-net: gro: signal over-coalescing more reliably

PATCH Applied Jakub Kicinski 4 participants 5 messages Jun 7 – Jun 9

Subsystems (3)
lore
Share & tools
lore

Jakub Kicinski

author maintainer
#0 lore raw

PATCH 1/1 selftests: drv-net: gro: signal over-coalescing more reliably

GRO test is very timing-sensitive, packets may be delayed by the network or just sent slowly. Because of this we retry each test case up to 6 times.

This makes perfect sense for positive cases, in which we want to see coalescing. Negative test cases, which modify headers and expect no coalescing should have opposite treatment. We should really try 6 times and make sure that each time the test failed. This would, however, require that we annotate each test to indicate whether its positive or negative. Let's start with a simpler improvement. Do not allow retries if we detected over-coalescing. Previously the negative case would have to get lucky at least once in 6 tries to pass. Now the first failure breaks the retry loop.

For background - NICs tend to ignore the contents of the TCP timestamp option, so that test case commonly fails. In NIPA having 6 attempts, however, was enough for some NICs to get multiple successful runs in a row, getting the test cases auto-classified as expected to pass, even tho the NIC does not comply with the expectations.

---
CC: [email protected]
CC: [email protected]
CC: [email protected]

---
 tools/testing/selftests/net/lib/gro.c      | 16 +++++++++++++++-
 tools/testing/selftests/drivers/net/gro.py |  8 +++++++-
 2 files changed, 22 insertions(+), 2 deletions(-)
tools/testing/selftests/net/lib/gro.c +15 -1
index fa35dfc8e790..7a333155de1a 100644
@@ -108,6 +108,8 @@
 #define EXT_PAYLOAD_1 "\x00\x00\x00\x00\x00\x00"
 #define EXT_PAYLOAD_2 "\x11\x11\x11\x11\x11\x11"
 
+#define EXIT_OVER_COALESCE	42
+
 #define ipv6_optlen(p)  (((p)->hdrlen+1) << 3) /* calculate IPv6 extension header len */
 #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)]))
 
@@ -1165,6 +1167,8 @@ static void check_recv_pkts(int fd, int *correct_payload,
 	struct ipv6hdr *ip6h = (struct ipv6hdr *)(buffer + nhoff);
 	struct tcphdr *tcph;
 	bool bad_packet = false;
+	int bytes_expected = 0;
+	int bytes_received = 0;
 	int tcp_ext_len = 0;
 	int ip_ext_len = 0;
 	int pkt_size = -1;
@@ -1173,8 +1177,10 @@ static void check_recv_pkts(int fd, int *correct_payload,
 	int i;
 
 	vlog("Expected {");
-	for (i = 0; i < correct_num_pkts; i++)
+	for (i = 0; i < correct_num_pkts; i++) {
 		vlog("%d ", correct_payload[i]);
+		bytes_expected += correct_payload[i];
+	}
 	vlog("}, Total %d packets\nReceived {", correct_num_pkts);
 
 	while (1) {
@@ -1209,9 +1215,17 @@ static void check_recv_pkts(int fd, int *correct_payload,
 			vlog("[!=%d]", correct_payload[num_pkt]);
 			bad_packet = true;
 		}
+		bytes_received += data_len;
 		num_pkt++;
 	}
 	vlog("}, Total %d packets.\n", num_pkt);
+	/* Signal over-coalescing explicitly, it's a hard failure, unlike
+	 * under-coalescing which could be timing- or loss-related.
+	 */
+	if (num_pkt < correct_num_pkts && bytes_received == bytes_expected)
+		error(EXIT_OVER_COALESCE, 0,
+		      "over-coalesced: got %d pkts vs expected %d (%d B)",
+		      num_pkt, correct_num_pkts, bytes_received);
 	if (num_pkt != correct_num_pkts)
 		error(1, 0, "incorrect number of packets");
 	if (bad_packet)
tools/testing/selftests/drivers/net/gro.py +7 -1
index fd158c775b1c..6ab8c97880d1 100755
@@ -40,7 +40,7 @@ import glob
 import os
 import re
 from lib.py import ksft_run, ksft_exit, ksft_pr
-from lib.py import NetDrvEpEnv, KsftXfailEx
+from lib.py import NetDrvEpEnv, KsftFailEx, KsftXfailEx
 from lib.py import NetdevFamily, EthtoolFamily
 from lib.py import bkg, cmd, defer, ethtool, ip
 from lib.py import ksft_variants, KsftNamedVariant
@@ -370,6 +370,12 @@ def _run_gro_bin(cfg, test_name, protocol=None, num_flows=None,
 
         ksft_pr(rx_proc)
 
+        # ret==42 means the receiver detected over-coalescing.
+        # This is unambiguous proof of a bug, retries can only cause
+        # false negatives.
+        if rx_proc.ret == 42:
+            raise KsftFailEx(f"GRO over-coalesced in {protocol}/{test_name}")
+
         if test_name.startswith("large_") and os.environ.get("KSFT_MACHINE_SLOW"):
             ksft_pr(f"Ignoring {protocol}/{test_name} failure due to slow environment")
             return
Signature · 1 line
2.54.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 15 lines · Jakub Kicinski

GRO test is very timing-sensitive, packets may be delayed by the network or just sent slowly. Because of this we retry each test case up to 6 times.

This makes perfect sense for positive cases, in which we want to see coalescing. Negative test cases, which modify headers and expect no coalescing should have opposite treatment. We should really try 6 times and make sure that each time the test failed. This would, however, require that we annotate each test to indicate whether its positive or negative. Let's start with a simpler improvement. Do not allow retries if we detected over-coalescing. Previously the negative case would have to get lucky at least once in 6 tries to pass. Now the first failure breaks the retry loop.

Quoted 8 lines

For background - NICs tend to ignore the contents of the TCP timestamp option, so that test case commonly fails. In NIPA having 6 attempts, however, was enough for some NICs to get multiple successful runs in a row, getting the test cases auto-classified as expected to pass, even tho the NIC does not comply with the expectations.

author maintainer
#2 lore raw
In reply to Jakub Kicinski

tools/testing/selftests/net/lib/gro.c static void check_recv_pkts(int fd, int *correct_payload, · line 1228

Quoted 7 lines · On Sat, 6 Jun 2026 17:24:01 -0700 Jakub Kicinski wrote:
tools/testing/selftests/net/lib/gro.c +7 -0
+	/* Signal over-coalescing explicitly, it's a hard failure, unlike
+	 * under-coalescing which could be timing- or loss-related.
+	 */
+	if (num_pkt < correct_num_pkts && bytes_received == bytes_expected)
+		error(EXIT_OVER_COALESCE, 0,
+		      "over-coalesced: got %d pkts vs expected %d (%d B)",
+		      num_pkt, correct_num_pkts, bytes_received);

Sashiko complains that this condition may mis-fire if we lose a zero-len RST or ACK. This is true, tho, if the device drops pure acks ignoring failures where ack disappeared puts us back at risk of false negatives. Happy to respin but feels like we only have bad choices here..

maintainer
#3 lore raw
Quoted 13 lines · Jakub Kicinski

tools/testing/selftests/net/lib/gro.c static void check_recv_pkts(int fd, int *correct_payload, · line 1228

Quoted 7 lines · On Sat, 6 Jun 2026 17:24:01 -0700 Jakub Kicinski wrote:
tools/testing/selftests/net/lib/gro.c +7 -0
+	/* Signal over-coalescing explicitly, it's a hard failure, unlike
+	 * under-coalescing which could be timing- or loss-related.
+	 */
+	if (num_pkt < correct_num_pkts && bytes_received == bytes_expected)
+		error(EXIT_OVER_COALESCE, 0,
+		      "over-coalesced: got %d pkts vs expected %d (%d B)",
+		      num_pkt, correct_num_pkts, bytes_received);

Sashiko complains that this condition may mis-fire if we lose a zero-len RST or ACK. This is true, tho, if the device drops pure acks ignoring failures where ack disappeared puts us back at risk of false negatives. Happy to respin but feels like we only have bad choices here..

My personal take here is that packet loss at (very) low rate in a controlled environment should be threaded as fatal: I think the patch in the current form is the better option.

/P

#4 lore raw
In reply to Jakub Kicinski
This patch was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:

Hello:

This patch was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:

Quoted 15 lines · On Sat, 6 Jun 2026 17:24:01 -0700 you wrote:

GRO test is very timing-sensitive, packets may be delayed by the network or just sent slowly. Because of this we retry each test case up to 6 times.

This makes perfect sense for positive cases, in which we want to see coalescing. Negative test cases, which modify headers and expect no coalescing should have opposite treatment. We should really try 6 times and make sure that each time the test failed. This would, however, require that we annotate each test to indicate whether its positive or negative. Let's start with a simpler improvement. Do not allow retries if we detected over-coalescing. Previously the negative case would have to get lucky at least once in 6 tries to pass. Now the first failure breaks the retry loop.

Quoted 1 line

[...]

Here is the summary with links:
- [net-next] selftests: drv-net: gro: signal over-coalescing more reliably

    https://git.kernel.org/netdev/net-next/c/69a4e74e4cc4

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html