net/tcp: move TCP hash fail messages out of line
Subsystems (3)
PATCH 1/1 net/tcp: move TCP hash fail messages out of line
tcp_hash_fail() is used multiple times on hotpath, including static inlines. It contains a couple branches and a lot of code, which all gets inlined into the call sites. For example, one call emits two calls to net_ratelimit() etc.
Move as much as we can out of line to a new global function. Use enum to determine the type of failure. Check for net_ratelimit() only once, format common fields only once as well to pass only unique strings to pr_info().
The result for vmlinux with Clang 19:
add/remove: 2/0 grow/shrink: 0/4 up/down: 773/-4908 (-4135)
Function old new delta__tcp_hash_fail - 757 +757__pfx___tcp_hash_fail - 16 +16
tcp_inbound_ao_hash 1819 1062 -757 tcp_ao_verify_hash 1217 451 -766 tcp_inbound_md5_hash 1591 374 -1217 tcp_inbound_hash 3566 1398 -2168
--- include/net/tcp.h | 15 ++--- include/net/tcp_ao.h | 58 ++++++++----------- net/ipv4/tcp.c | 129 +++++++++++++++++++++++++++++++++++++------ net/ipv4/tcp_ao.c | 13 ++--- 4 files changed, 148 insertions(+), 67 deletions(-)
include/net/tcp.h +6 -9
@@ -2809,17 +2809,14 @@ tcp_inbound_hash(struct sock *sk, const struct request_sock *req,/* Invalid option or two times meet any of auth options */ if (tcp_parse_auth_options (th, &md5_location, &aoh)) { -tcp_hash_fail ("TCP segment has incorrect auth options set" , - family, skb,"" ); +tcp_hash_fail (TCP_HASH_FAIL_OPTS , family, skb);return SKB_DROP_REASON_TCP_AUTH_HDR ; }if (req) {if (tcp_rsk_used_ao (req) != !!aoh) {NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPAOBAD ); -tcp_hash_fail ("TCP connection can't start/end using TCP-AO" , - family, skb,"%s" , - !aoh ?"missing AO" :"AO signed" ); +tcp_hash_fail (TCP_HASH_FAIL_CONN , family, skb, !aoh);return SKB_DROP_REASON_TCP_AOFAILURE ; } }
@@ -2837,14 +2834,14 @@ tcp_inbound_hash(struct sock *sk, const struct request_sock *req,* always at least one current_key. */ if (tcp_ao_required (sk, saddr, family, l3index, true)) { -tcp_hash_fail ("AO hash is required, but not found" , - family, skb,"L3 index %d" , l3index); +tcp_hash_fail (TCP_HASH_FAIL_NOAO , family, skb, + l3index);return SKB_DROP_REASON_TCP_AONOTFOUND ; }if (unlikely (tcp_md5_do_lookup (sk, l3index, saddr, family))) {NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPMD5NOTFOUND ); -tcp_hash_fail ("MD5 Hash not found" , - family, skb,"L3 index %d" , l3index); +tcp_hash_fail (TCP_HASH_FAIL_NOMD5 , family, skb, + l3index);return SKB_DROP_REASON_TCP_MD5NOTFOUND ; }return SKB_NOT_DROPPED_YET ;
include/net/tcp_ao.h +24 -34
66 lines not on this page · raw message
net/ipv4/tcp.c +113 -16
147 lines not on this page · raw message
net/ipv4/tcp_ao.c +5 -8
@@ -892,8 +892,7 @@ tcp_ao_verify_hash(const struct sock *sk, const struct sk_buff *skb,NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPAOBAD );atomic64_inc (&info->counters.pkt_bad);atomic64_inc (&key->pkt_bad); -tcp_hash_fail ("AO hash wrong length" , family, skb, -"%u != %d L3index: %d" , maclen, +tcp_hash_fail (TCP_HASH_FAIL_LEN , family, skb, maclen,tcp_ao_maclen (key), l3index);return SKB_DROP_REASON_TCP_AOFAILURE ; }
@@ -909,8 +908,7 @@ tcp_ao_verify_hash(const struct sock *sk, const struct sk_buff *skb,NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPAOBAD );atomic64_inc (&info->counters.pkt_bad);atomic64_inc (&key->pkt_bad); -tcp_hash_fail ("AO hash mismatch" , family, skb, -"L3index: %d" , l3index); +tcp_hash_fail (TCP_HASH_FAIL_MIS , family, skb, l3index);kfree (hash_buf);return SKB_DROP_REASON_TCP_AOFAILURE ; }
@@ -938,8 +936,8 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb, info =rcu_dereference (tcp_sk (sk)->ao_info);if (!info) {NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPAOKEYNOTFOUND ); -tcp_hash_fail ("AO key not found" , family, skb, -"keyid: %u L3index: %d" , aoh->keyid, l3index); +tcp_hash_fail (TCP_HASH_FAIL_NOKEY , family, skb, aoh->keyid, + l3index);return SKB_DROP_REASON_TCP_AOUNEXPECTED ; }
@@ -1041,8 +1039,7 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb, key_not_found:NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPAOKEYNOTFOUND );atomic64_inc (&info->counters.key_not_found); -tcp_hash_fail ("Requested by the peer AO key id not found" , - family, skb,"L3index: %d" , l3index); +tcp_hash_fail (TCP_HASH_FAIL_NOID , family, skb, l3index);return SKB_DROP_REASON_TCP_AOKEYNOTFOUND ; }
Signature · 1 line
2.44.0
Hi Alexander,
Quoted 11 lines · Alexander Lobakin
tcp_hash_fail()is used multiple times on hotpath, including static inlines. It contains a couple branches and a lot of code, which all gets inlined into the call sites. For example, one call emits two calls tonet_ratelimit()etc.
Move as much as we can out of line to a new global function. Use enum to determine the type of failure. Check fornet_ratelimit()only once, format common fields only once as well to pass only unique strings topr_info().
The result for vmlinux with Clang 19:
Quoted 8 lines
add/remove: 2/0 grow/shrink: 0/4 up/down: 773/-4908 (-4135)
Function old new delta__tcp_hash_fail- 757 +757__pfx___tcp_hash_fail- 16 +16tcp_inbound_ao_hash 1819 1062 -757 tcp_ao_verify_hash 1217 451 -766 tcp_inbound_md5_hash 1591 374 -1217 tcp_inbound_hash 3566 1398 -2168
I can see that as an improvement, albeit that enum and the resulting switch are quite gross, sorry.
I had patches to convert those messages to tracepoints (by Jakub's suggestion). That seems to work quite nice and will remove this macro entirely:
https://lore.kernel.org/all/[email protected]/
I need to send version 2 for that. Unfortunately, that got delayed by me migrating
from my previous work laptop. That was not my choice, resulting in little disruption.
I'm planning to send the new version optimistically by the end of this week, at worst the next week.
Thanks,
Dmitry
From: Dmitry Safonov <[email protected]>
Date: Wed, 3 Apr 2024 18:00:49 +0100
Quoted 24 lines
Hi Alexander,
Quoted 11 lines · Alexander Lobakin
tcp_hash_fail()is used multiple times on hotpath, including static inlines. It contains a couple branches and a lot of code, which all gets inlined into the call sites. For example, one call emits two calls tonet_ratelimit()etc.
Move as much as we can out of line to a new global function. Use enum to determine the type of failure. Check fornet_ratelimit()only once, format common fields only once as well to pass only unique strings topr_info().
The result for vmlinux with Clang 19:Quoted 8 lines
add/remove: 2/0 grow/shrink: 0/4 up/down: 773/-4908 (-4135)
Function old new delta__tcp_hash_fail- 757 +757__pfx___tcp_hash_fail- 16 +16tcp_inbound_ao_hash 1819 1062 -757 tcp_ao_verify_hash 1217 451 -766 tcp_inbound_md5_hash 1591 374 -1217 tcp_inbound_hash 3566 1398 -2168
Quoted 2 lines
I can see that as an improvement, albeit that enum and the resulting switch are quite gross, sorry.
I know, but that's the only way to move that print out of line.
Quoted 3 lines
I had patches to convert those messages to tracepoints (by Jakub's suggestion). That seems to work quite nice and will remove this macro entirely:
https://lore.kernel.org/all/[email protected]/
Oh sorry, I didn't search lore for the related patches before sending. Sounds good! My solution can be dropped then. Just make sure you have the same code size reduction in tcp_*_hash() :D
Quoted 10 lines
I need to send version 2 for that. Unfortunately, that got delayed by me migrating
from my previous work laptop. That was not my choice, resulting in little disruption.
I'm planning to send the new version optimistically by the end of this week, at worst the next week.Thanks,
Dmitry
Thanks,
Olek