tcp: do not shrink window clamp when SO_RCVBUF is locked
Subsystems (4)
PATCH 1/1 tcp: do not shrink window clamp when SO_RCVBUF is locked
When an application explicitly sets SO_RCVBUF, the window clamp should not be dynamically recalculated based on the memory scaling_ratio.
Currently, tcp_measure_rcv_mss() aggressively crushes the window clamp down when it sees a poor skb->len to skb->truesize ratio. If the application explicitly locked the buffer via SO_RCVBUF, this recalculation causes the advertised window to drop severely.
If the window drops below the interface MSS, it triggers Silly Window Syndrome (SWS) avoidance on the sender. The sender defers transmission and drops the connection into a perpetual 200ms PROBE0 timer loop, drastically reducing throughput.
This is highly reproducible on loopback interfaces (MTU 65536) using Java-based workloads (like Tomcat/GemFire) where the JVM sets SO_RCVBUF to 32K or 64K. The bloated loopback truesize forces the scaling ratio to drop, crushing the window clamp to ~26K, instantly triggering SWS stalls and causing gigabyte transfers to take minutes instead of milliseconds.
Since the application locked the buffer, the kernel should respect the clamp boundary and not dynamically crush it based on runtime ratios.
---
Note to reviewers:
Testing Context:
- The SWS deadlock was successfully reproduced on the latest netdev/net tree (v7.1-rc1) using the actual enterprise Java workload.
- Applying this patch completely resolves the 504 Timeouts and restores loopback throughput.
- Baseline iperf3 auto-tuning remains unaffected by this patch.
For context, here is the exact sequence of events that triggers the recalculation flaw, illustrated in a packetdrill-style flow. Unpatched kernels aggressively crush the window at step 3, triggering SWS.
// 1. Tomcat creates socket and hardcodes the buffer to 32K 0 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3
+0 setsockopt(3, SOL_SOCKET, SO_RCVBUF, [32768]) = 0 +0 bind(3, ..., ...) = 0
+0 listen(3, 1) = 0
// 2. GemFire connects over loopback (simulating Jumbo MSS of 65496) +0 < S 0:0(0) win 65535 <mss 65496, sackOK, TS val 100 ecr 0, nop, wscale 8> +0 > S. 0:0(0) ack 1 <...>
+0 < . 1:1(0) ack 1 win 65535 <TS val 200 ecr 1>
+0 accept(3, ..., ...) = 4
// 3. GemFire sends a 20KB packet, dropping the scaling_ratio. // Without the patch, tcp_measure_rcv_mss() crushes the window_clamp here. +0.1 < . 1:20001(20000) ack 1 win 65535 <TS val 300 ecr 1> +0.1 read(4, ..., 20000) = 20000
// 4. Assert window did not crush
// WITH the patch, the kernel respects the SOCK_RCVBUF_LOCK. +0 > . 1:1(0) ack 20001 win 65535
--- net/ipv4/tcp_input.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
net/ipv4/tcp_input.c +2 -1
@@ -248,7 +248,8 @@ static void tcp_measure_rcv_mss(struct sock *sk, const struct sk_buff *skb)do_div (val, skb->truesize);tcp_sk (sk)->scaling_ratio = val ? val :1 ; -if (old_ratio !=tcp_sk (sk)->scaling_ratio) { +if (old_ratio !=tcp_sk (sk)->scaling_ratio && + !(sk->sk_userlocks &SOCK_RCVBUF_LOCK )) {struct tcp_sock *tp =tcp_sk (sk); val =tcp_win_from_space (sk, sk->sk_rcvbuf);
Signature · 1 line
2.53.0
Quoted 21 lines · Ankit Jain
When an application explicitly sets
SO_RCVBUF, the window clamp should not be dynamically recalculated based on the memoryscaling_ratio.Currently,
tcp_measure_rcv_mss()aggressively crushes the window clamp down when it sees a poorskb->lentoskb->truesizeratio. If the application explicitly locked the buffer viaSO_RCVBUF, this recalculation causes the advertised window to drop severely.If the window drops below the interface MSS, it triggers Silly Window Syndrome (SWS) avoidance on the sender. The sender defers transmission and drops the connection into a perpetual 200ms PROBE0 timer loop, drastically reducing throughput.
This is highly reproducible on loopback interfaces (MTU 65536) using Java-based workloads (like Tomcat/GemFire) where the JVM sets
SO_RCVBUFto 32K or 64K. The bloated loopback truesize forces the scaling ratio to drop, crushing the window clamp to ~26K, instantly triggering SWS stalls and causing gigabyte transfers to take minutes instead of milliseconds.
Quoted 7 lines
Since the application locked the buffer, the kernel should respect the clamp boundary and not dynamically crush it based on runtime ratios.
Fixes a2cbb1603943 ("tcp: Update window clamping condition")Reported-by Karen Badiryan <[email protected]>Signed-off-by Ankit Jain <[email protected]>
Make sure to add a selftests (in ./tools/testing/selftests/net/packetdrill/ )
Thanks.
Jakub Kicinski
maintainerQuoted 25 lines · On Mon, 27 Apr 2026 08:38:44 -0700 Eric Dumazet wrote:
Quoted 24 lines · Ankit Jain
When an application explicitly sets
SO_RCVBUF, the window clamp should not be dynamically recalculated based on the memoryscaling_ratio.Currently,
tcp_measure_rcv_mss()aggressively crushes the window clamp down when it sees a poorskb->lentoskb->truesizeratio. If the application explicitly locked the buffer viaSO_RCVBUF, this recalculation causes the advertised window to drop severely.If the window drops below the interface MSS, it triggers Silly Window Syndrome (SWS) avoidance on the sender. The sender defers transmission and drops the connection into a perpetual 200ms PROBE0 timer loop, drastically reducing throughput.
This is highly reproducible on loopback interfaces (MTU 65536) using Java-based workloads (like Tomcat/GemFire) where the JVM sets
SO_RCVBUFto 32K or 64K. The bloated loopback truesize forces the scaling ratio to drop, crushing the window clamp to ~26K, instantly triggering SWS stalls and causing gigabyte transfers to take minutes instead of milliseconds.Since the application locked the buffer, the kernel should respect the clamp boundary and not dynamically crush it based on runtime ratios.
Quoted 6 lines
Quoted 4 lines
Fixes a2cbb1603943 ("tcp: Update window clamping condition")Reported-by Karen Badiryan <[email protected]>Signed-off-by Ankit Jain <[email protected]>Make sure to add a selftests (in
./tools/testing/selftests/net/packetdrill/)
And I think it makes tcp_rcv_neg_window.pkt fail
reminder - please wait 24h before posting v2 on netdev, and when posting v2 start a new thread.
Hi Jakub,
Thank you for pointing this out.
I checked the logic again, I placed the SOCK_RCVBUF_LOCK check on the outer if-statement. This successfully prevented tcp_set_window_clamp() from crushing the window (fixing the Java SWS bug), but it accidentally bypassed the tp->rcvq_space.space bounds check as well. Because of this, the receive queue tracking got out of sync when the test hit a negative window.
For v2, I will move the lock check inside the block so it only bypasses tcp_set_window_clamp(), ensuring the rcvq_space.space limits still execute as expected.
I will test the refined logic against my reproducer and the tcp_rcv_neg_window.pkt script locally, and I will post the v2 series in a new thread after the 24-hour cooldown.
Thanks,
Ankit
Quoted 26 lines · Eric Dumazet
Quoted 24 lines · Ankit Jain
When an application explicitly sets
SO_RCVBUF, the window clamp should not be dynamically recalculated based on the memoryscaling_ratio.Currently,
tcp_measure_rcv_mss()aggressively crushes the window clamp down when it sees a poorskb->lentoskb->truesizeratio. If the application explicitly locked the buffer viaSO_RCVBUF, this recalculation causes the advertised window to drop severely.If the window drops below the interface MSS, it triggers Silly Window Syndrome (SWS) avoidance on the sender. The sender defers transmission and drops the connection into a perpetual 200ms PROBE0 timer loop, drastically reducing throughput.
This is highly reproducible on loopback interfaces (MTU 65536) using Java-based workloads (like Tomcat/GemFire) where the JVM sets
SO_RCVBUFto 32K or 64K. The bloated loopback truesize forces the scaling ratio to drop, crushing the window clamp to ~26K, instantly triggering SWS stalls and causing gigabyte transfers to take minutes instead of milliseconds.Since the application locked the buffer, the kernel should respect the clamp boundary and not dynamically crush it based on runtime ratios.
Quoted 8 lines
Quoted 4 lines
Fixes a2cbb1603943 ("tcp: Update window clamping condition")Reported-by Karen Badiryan <[email protected]>Signed-off-by Ankit Jain <[email protected]>Make sure to add a selftests (in
./tools/testing/selftests/net/packetdrill/)Thanks.
Hi Eric,
Thank you for the review.
I will formalize the packetdrill sequence from the commit notes into a proper runnable .pkt selftest. I will include it as the second patch in the upcoming v2 series.
Thanks,
Ankit