net: count last skb of each sk_backlog pass in __release_sock()
Subsystems (4)
PATCH 1/1 net: count last skb of each sk_backlog pass in __release_sock()
Running 500 concurrent TCPv6 streams (neper "tcp_stream -6 -T 1 -F 500") on a 2-core Cortex-A53 (CONFIG_PREEMPT_NONE=y, HZ=1000, LATENCY_WARN) triggers resched_latency_warn() in __release_sock():
[ 3655.311821] sched: CPU 0 need_resched set for > 100006892 ns (100 ticks) without schedule [ 3655.311889] Call trace: ... [ 3655.311944] resched_latency_warn+0x7c/0x88 [ 3655.311958] sched_tick+0x1c0/0x418 ... [ 3655.312578] net_rx_action+0x238/0x438 ... [ 3655.312641] do_softirq+0x4c/0x90 [ 3655.312651] __local_bh_enable_ip+0x84/0x90 [ 3655.312663] local_bh_enable+0x1c/0x28 [ 3655.312674] __dev_queue_xmit+0x4a8/0xb00 ... [ 3655.312782] tcp_v6_do_rcv+0x240/0x5f8 [ 3655.312793] __release_sock+0x9c/0x188 [ 3655.312806] release_sock+0x150/0x168 [ 3655.312817] tcp_sendmsg+0x4c/0x70
__release_sock() can loop past 100 ms without calling cond_resched():
- Each outer-loop pass drains sk_backlog, but "if (!skb) break;" runs before "++nb", so a pass of B skbs only adds B - 1 to nb (0 for 1 skb).
- With 500 TCPv6 streams and NAPI sharing two Cortex-A53 cores, runnable tasks keep TIF_NEED_RESCHED set on CPU 0.
- During spin_unlock_bh() and TX softirqs (__dev_queue_xmit() -> local_bh_enable()), tcp_add_backlog() coalesces arriving pure ACKs into a single tail skb.
- Repeated 1-skb passes never advance nb or call cond_resched(), triggering resched_latency_warn() after 100 ticks.
Restore the do-while loop and drop "if (!skb) break;" so every skb increments nb across passes and calls cond_resched() every 16 skbs.
Reproduced the issue and verified the fix on two 2-core Cortex-A53 hosts (CONFIG_PREEMPT_NONE=y, HZ=1000, LATENCY_WARN, latency_warn_once=0) with a userspace reproducer over /dev/net/tun:
- CPU 0 runs a TCPv6 sender and a second runnable task while CPU 1 writes pure ACKs to /dev/net/tun (~30k TCPBacklogCoalesce/run), sustaining 1-skb sk_backlog refills in __release_sock() for ~400 ms (ftrace).
- Before: CPU 0 stayed unscheduled in __release_sock(), hitting resched_latency_warn() 293 and 275 times on the two hosts (> 100.9 ms).
- After: cond_resched() yielded CPU 0 inside __release_sock(), with 0 resched_latency_warn() hits on both hosts.
- Also observed no regression with the workload from
commit 16c610162d1f ("net: call cond_resched() less often in __release_sock()"), "super_netperf 4 -H srv -T,10 -l 3000 &".
--- net/core/sock.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
net/core/sock.c +2 -4
@@ -3333,7 +3333,7 @@ void __release_sock(struct sock *sk)spin_unlock_bh (&sk->sk_lock.slock); -while (1 ) { + do { next = skb->next; prefetch(next); DEBUG_NET_WARN_ON_ONCE(skb_dst_is_noref(skb));
@@ -3341,12 +3341,10 @@ void __release_sock(struct sock *sk)sk_backlog_rcv (sk, skb); skb = next; -if (!skb) -break ;if (!(++nb &15 ))cond_resched (); - } + }while (skb);spin_lock_bh (&sk->sk_lock.slock); }
base-commit: cb284914e749388eef402938bdb3115fe1f6d337
Signature · 1 line
2.56.0.317.g454f8302e1-goog
Quoted 2 lines · Ray Zhang
Running 500 concurrent TCPv6 streams (neper "
tcp_stream-6 -T 1 -F 500") on a 2-core Cortex-A53 (CONFIG_PREEMPT_NONE=y, HZ=1000,LATENCY_WARN)
I guess you tested it on 6.18, but note that CONFIG_PREEMPT_NONE is no longer selectable on arm64 since 7dadeaa6e851.
Quoted 50 lines
triggers
resched_latency_warn()in__release_sock():[ 3655.311821] sched: CPU 0 need_resched set for > 100006892 ns (100 ticks) without schedule [ 3655.311889] Call trace: ... [ 3655.311944] resched_latency_warn+0x7c/0x88 [ 3655.311958] sched_tick+0x1c0/0x418 ... [ 3655.312578] net_rx_action+0x238/0x438 ... [ 3655.312641] do_softirq+0x4c/0x90 [ 3655.312651] __local_bh_enable_ip+0x84/0x90 [ 3655.312663] local_bh_enable+0x1c/0x28 [ 3655.312674] __dev_queue_xmit+0x4a8/0xb00 ... [ 3655.312782] tcp_v6_do_rcv+0x240/0x5f8 [ 3655.312793] __release_sock+0x9c/0x188 [ 3655.312806] release_sock+0x150/0x168 [ 3655.312817] tcp_sendmsg+0x4c/0x70
__release_sock()can loop past 100 ms without callingcond_resched():- Each outer-loop pass drains
sk_backlog, but "if (!skb) break;" runs before "++nb", so a pass of B skbs only adds B - 1 to nb (0 for 1 skb).
- With 500 TCPv6 streams and NAPI sharing two Cortex-A53 cores, runnable tasks keepTIF_NEED_RESCHEDset on CPU 0.
- Duringspin_unlock_bh()and TX softirqs (__dev_queue_xmit()->local_bh_enable()),tcp_add_backlog()coalesces arriving pure ACKs into a single tail skb.
- Repeated 1-skb passes never advance nb or callcond_resched(), triggeringresched_latency_warn()after 100 ticks.Restore the do-while loop and drop "if (!skb) break;" so every skb increments nb across passes and calls
cond_resched()every 16 skbs.Reproduced the issue and verified the fix on two 2-core Cortex-A53 hosts (
CONFIG_PREEMPT_NONE=y, HZ=1000,LATENCY_WARN, latency_warn_once=0) with a userspace reproducer over/dev/net/tun:- CPU 0 runs a TCPv6 sender and a second runnable task while CPU 1 writes pure ACKs to
/dev/net/tun(~30k TCPBacklogCoalesce/run), sustaining 1-skbsk_backlogrefills in__release_sock()for ~400 ms (ftrace).
- Before: CPU 0 stayed unscheduled in__release_sock(), hittingresched_latency_warn()293 and 275 times on the two hosts (> 100.9 ms).
- After:cond_resched()yielded CPU 0 inside__release_sock(), with 0resched_latency_warn()hits on both hosts.
- Also observed no regression with the workload from
commit16c610162d1f("net: callcond_resched()less often in__release_sock()"), "super_netperf4 -H srv -T,10 -l 3000 &".
Quoted 3 lines
Fixes 16c610162d1f ("net: call cond_resched() less often in __release_sock()")Signed-off-by Ray Zhang <[email protected]>
Quoted 7 lines · Kuniyuki Iwashima
Quoted 2 lines · Ray Zhang
Running 500 concurrent TCPv6 streams (neper "
tcp_stream-6 -T 1 -F 500") on a 2-core Cortex-A53 (CONFIG_PREEMPT_NONE=y, HZ=1000,LATENCY_WARN)I guess you tested it on 6.18, but note that
CONFIG_PREEMPT_NONEis no longer selectable on arm64 since7dadeaa6e851.
Right, we hit and tested this on 6.18 -- thanks for pointing out 7dadeaa6e851 and for the review!
On mainline, cond_resched() in __release_sock() still applies to !ARCH_HAS_PREEMPT_LAZY configs (e.g. CONFIG_PREEMPT_VOLUNTARY on 32-bit arm/mips/sparc, or CONFIG_PREEMPT_NONE with ARCH_NO_PREEMPT), and landing it in net lets 6.18.y stable backport it cleanly.
Quoted 10 lines · Ray Zhang
Quoted 7 lines · Kuniyuki Iwashima
Quoted 2 lines · Ray Zhang
Running 500 concurrent TCPv6 streams (neper "
tcp_stream-6 -T 1 -F 500") on a 2-core Cortex-A53 (CONFIG_PREEMPT_NONE=y, HZ=1000,LATENCY_WARN)I guess you tested it on 6.18, but note that
CONFIG_PREEMPT_NONEis no longer selectable on arm64 since7dadeaa6e851.
Quoted 7 lines
Right, we hit and tested this on 6.18 -- thanks for pointing out
7dadeaa6e851and for the review!On mainline,
cond_resched()in__release_sock()still applies to !ARCH_HAS_PREEMPT_LAZY configs (e.g.CONFIG_PREEMPT_VOLUNTARYon 32-bit arm/mips/sparc, orCONFIG_PREEMPT_NONEwithARCH_NO_PREEMPT), and landing it in net lets 6.18.y stable backport it cleanly.
SGTM, thanks!
This patch was applied to netdev/net.git (main) by Paolo Abeni <[email protected]>:
Hello:
This patch was applied to netdev/net.git (main)
by Paolo Abeni <[email protected]>:
Quoted 22 lines · On Sun, 1 Nov 2026 00:51:45 -0700 you wrote:
Running 500 concurrent TCPv6 streams (neper "
tcp_stream-6 -T 1 -F 500") on a 2-core Cortex-A53 (CONFIG_PREEMPT_NONE=y, HZ=1000,LATENCY_WARN) triggersresched_latency_warn()in__release_sock():[ 3655.311821] sched: CPU 0 need_resched set for > 100006892 ns (100 ticks) without schedule [ 3655.311889] Call trace: ... [ 3655.311944] resched_latency_warn+0x7c/0x88 [ 3655.311958] sched_tick+0x1c0/0x418 ... [ 3655.312578] net_rx_action+0x238/0x438 ... [ 3655.312641] do_softirq+0x4c/0x90 [ 3655.312651] __local_bh_enable_ip+0x84/0x90 [ 3655.312663] local_bh_enable+0x1c/0x28 [ 3655.312674] __dev_queue_xmit+0x4a8/0xb00 ... [ 3655.312782] tcp_v6_do_rcv+0x240/0x5f8 [ 3655.312793] __release_sock+0x9c/0x188 [ 3655.312806] release_sock+0x150/0x168 [ 3655.312817] tcp_sendmsg+0x4c/0x70
Quoted 1 line
[...]
Here is the summary with links:
- [net] net: count last skb of each sk_backlog pass in __release_sock()
https://git.kernel.org/netdev/net/c/c4e4056b2135
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html