tcp: add tcp_mstamp_refresh_inline()
Subsystems (3)
PATCH 1/1 tcp: add tcp_mstamp_refresh_inline()
We want to inline tcp_mstamp_refresh() in fast path only:
- tcp_rcv_established()
- tcp_write_xmit()
Add tcp_mstamp_refresh_inline() for this purpose.
Add noinline qualifier on tcp_mstamp_refresh() for the other paths, to reduce bloat.
$ scripts/bloat-o-meter -t vmlinux.old vmlinux.new add/remove: 0/0 grow/shrink: 1/4 up/down: 26/-123 (-97) Function old new delta tcp_rcv_established 2238 2264 +26 tcp_connect 4027 4003 -24 tcp_tsq_write 152 120 -32 tcp_send_active_reset 476 444 -32 tcp_send_window_probe 235 200 -35 Total: Before=25316710, After=25316613, chg -0.00%
--- include/net/tcp.h | 10 ++++++++++ net/ipv4/tcp_input.c | 2 +- net/ipv4/tcp_output.c | 12 +++--------- 3 files changed, 14 insertions(+), 10 deletions(-)
include/net/tcp.h +10 -0
@@ -1015,6 +1015,16 @@ static inline u32 tcp_time_stamp_ts(const struct tcp_sock *tp)return tcp_time_stamp_ms (tp); } +/* Refresh clocks of a TCP socket, +* ensuring monotically increasing values. +*/ +static inline void tcp_mstamp_refresh_inline (struct tcp_sock *tp) +{ +u64 val =tcp_clock_ns (); + + tp->tcp_clock_cache = val; + tp->tcp_mstamp =div_u64 (val,NSEC_PER_USEC ); +}void tcp_mstamp_refresh (struct tcp_sock *tp);static inline u32 tcp_stamp_us_delta (u64 t1,u64 t0)
net/ipv4/tcp_input.c +1 -1
@@ -6474,7 +6474,7 @@ void tcp_rcv_established(struct sock *sk, struct sk_buff *skb)/* TCP congestion window tracking */ trace_tcp_probe (sk, skb); -tcp_mstamp_refresh (tp); +tcp_mstamp_refresh_inline (tp);if (unlikely (!rcu_access_pointer (sk->sk_rx_dst)))inet_csk (sk)->icsk_af_ops->sk_rx_dst_set (sk, skb);/*
net/ipv4/tcp_output.c +3 -9
@@ -52,15 +52,9 @@#include <trace/events/tcp.h> -/* Refresh clocks of a TCP socket, -* ensuring monotically increasing values. -*/ -void tcp_mstamp_refresh (struct tcp_sock *tp) +void noinline tcp_mstamp_refresh(struct tcp_sock *tp) { -u64 val =tcp_clock_ns (); - - tp->tcp_clock_cache = val; - tp->tcp_mstamp =div_u64 (val,NSEC_PER_USEC ); +tcp_mstamp_refresh_inline (tp); } static bool tcp_write_xmit(struct sock *sk, unsigned int mss_now, int nonagle,
@@ -2971,7 +2965,7 @@ static bool tcp_write_xmit(struct sock *sk, unsigned int mss_now, int nonagle, sent_pkts =0 ; -tcp_mstamp_refresh (tp); +tcp_mstamp_refresh_inline (tp);/* AccECN option beacon depends on mstamp, it may change mss */ if (tcp_ecn_mode_accecn (tp) &&tcp_accecn_option_beacon_check (sk))
Signature · 1 line
2.54.0.545.g6539524ca2-goog
Quoted 21 lines · Eric Dumazet
We want to inline
tcp_mstamp_refresh()in fast path only:-
tcp_rcv_established()
-tcp_write_xmit()Add
tcp_mstamp_refresh_inline()for this purpose.Add noinline qualifier on
tcp_mstamp_refresh()for the other paths, to reduce bloat.$ scripts/bloat-o-meter -t vmlinux.old vmlinux.new add/remove: 0/0 grow/shrink: 1/4 up/down: 26/-123 (-97) Function old new delta tcp_rcv_established 2238 2264 +26 tcp_connect 4027 4003 -24 tcp_tsq_write 152 120 -32 tcp_send_active_reset 476 444 -32 tcp_send_window_probe 235 200 -35 Total: Before=25316710, After=25316613, chg -0.00%
Quoted 2 lines
Signed-off-by Eric Dumazet <[email protected]>---
Thanks, Eric!
neal
This patch was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This patch was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Quoted 8 lines · On Wed, 29 Apr 2026 01:08:09 +0000 you wrote:
We want to inline
tcp_mstamp_refresh()in fast path only:-
tcp_rcv_established()
-tcp_write_xmit()Add
tcp_mstamp_refresh_inline()for this purpose.[...]
Here is the summary with links:
- [net-next] tcp: add tcp_mstamp_refresh_inline()
https://git.kernel.org/netdev/net-next/c/28df22acc275
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html