Implement reset reason mechanism to detect
Subsystems (6)
v6 cover Implement reset reason mechanism to detect
From: Jason Xing <[email protected]>
In production, there are so many cases about why the RST skb is sent but we don't have a very convenient/fast method to detect the exact underlying reasons.
RST is implemented in two kinds: passive kind (like tcp_v4_send_reset()) and active kind (like tcp_send_active_reset()). The former can be traced carefully 1) in TCP, with the help of drop reasons, which is based on Eric's idea[1], 2) in MPTCP, with the help of reset options defined in RFC 8684. The latter is relatively independent, which should be implemented on our own.
In this series, I focus on the fundamental implement mostly about how the rstreason mechnism works and give the detailed passive part as an example, not including the active reset part. In future, we can go further and refine those NOT_SPECIFIED reasons.
Here are some examples when tracing:
<idle>-0 [002] ..s1. 1830.262425: tcp_send_reset: skbaddr=x
skaddr=x src=x dest=x state=x reason=NOT_SPECIFIED
<idle>-0 [002] ..s1. 1830.262425: tcp_send_reset: skbaddr=x
skaddr=x src=x dest=x state=x reason=NO_SOCKET
v6
1. add back casts, or else they are treated as error.
v5
1. address format issue (like reverse xmas tree) (Eric, Paolo)
2. remove unnecessary casts. (Eric)
3. introduce a helper used in mptcp active reset. See patch 6. (Paolo)
v4
1. passing 'enum sk_rst_reason' for readability when tracing (Antoine)
v3
1. rebase (mptcp part) and address what Mat suggested.
v2
1. rebase against the latest net-next tree
Jason Xing (7):
net: introduce rstreason to detect why the RST is sent
rstreason: prepare for passive reset
rstreason: prepare for active reset
tcp: support rstreason for passive reset
mptcp: support rstreason for passive reset
mptcp: introducing a helper into active reset logic
rstreason: make it work in trace world
include/net/request_sock.h | 4 +- include/net/rstreason.h | 93 ++++++++++++++++++++++++++++++++++++++ include/net/tcp.h | 3 +- include/trace/events/tcp.h | 37 +++++++++++++-- net/dccp/ipv4.c | 10 ++-- net/dccp/ipv6.c | 10 ++-- net/dccp/minisocks.c | 3 +- net/ipv4/tcp.c | 15 ++++-- net/ipv4/tcp_ipv4.c | 14 +++--- net/ipv4/tcp_minisocks.c | 3 +- net/ipv4/tcp_output.c | 5 +- net/ipv4/tcp_timer.c | 9 ++-- net/ipv6/tcp_ipv6.c | 17 ++++--- net/mptcp/protocol.c | 2 +- net/mptcp/protocol.h | 11 +++++ net/mptcp/subflow.c | 27 ++++++++--- 16 files changed, 216 insertions(+), 47 deletions(-) create mode 100644 include/net/rstreason.h
Signature · 1 line
2.37.3
PATCH v6 1/7 net: introduce rstreason to detect why the RST is sent
From: Jason Xing <[email protected]>
Add a new standalone file for the easy future extension to support both active reset and passive reset in the TCP/DCCP/MPTCP protocols.
This patch only does the preparations for reset reason mechanism, nothing else changes.
The reset reasons are divided into three parts:
1) reuse drop reasons for passive reset in TCP
2) reuse MP_TCPRST option for MPTCP
3) our own reasons
I will implement the basic codes of active/passive reset reason in those three protocols, which is not complete for this moment. But it provides a new chance to let other people add more reasons into it:)
--- include/net/rstreason.h | 93 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 include/net/rstreason.h
include/net/rstreason.h new file +93 -0
@@ -0,0 +1,93 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef _LINUX_RSTREASON_H +#define _LINUX_RSTREASON_H +#include <net/dropreason-core.h> + +#define DEFINE_RST_REASON (FN , FNe) \ + FN(MPTCP_RST_EUNSPEC) \ + FN(MPTCP_RST_EMPTCP) \ + FN(MPTCP_RST_ERESOURCE) \ + FN(MPTCP_RST_EPROHIBIT) \ + FN(MPTCP_RST_EWQ2BIG) \ + FN(MPTCP_RST_EBADPERF) \ + FN(MPTCP_RST_EMIDDLEBOX) \ + FN(NOT_SPECIFIED) \ + FNe(MAX) + +#define RST_REASON_START (SKB_DROP_REASON_MAX + 1) + +/* There are three parts in order: +* 1) 0 - SKB_DROP_REASON_MAX: rely on drop reasons for passive reset in TCP +* 2) SKB_DROP_REASON_MAX + 1 - MPTCP_RST_EMIDDLEBOX: for MPTCP use +* 3) MPTCP_RST_EMIDDLEBOX - SK_RST_REASON_MAX: independent reset reason +*/ +enum sk_rst_reason { +/* Leave this 'blank' part (0-SKB_DROP_REASON_MAX) for the reuse +* of skb drop reason because rst reason relies on what drop reason +* indicates exactly why it could happen. +*/ + +/* Copy from include/uapi/linux/mptcp.h. +* These reset fields will not be changed since they adhere to +* RFC 8684. So do not touch them. I'm going to list each definition +* of them respectively. +*/ +/* Unspecified error. +* This is the default error; it implies that the subflow is no +* longer available. The presence of this option shows that the +* RST was generated by an MPTCP-aware device. +*/ +SK_RST_REASON_MPTCP_RST_EUNSPEC =RST_REASON_START , +/* MPTCP-specific error. +* An error has been detected in the processing of MPTCP options. +* This is the usual reason code to return in the cases where a RST +* is being sent to close a subflow because of an invalid response. +*/ +SK_RST_REASON_MPTCP_RST_EMPTCP , +/* Lack of resources. +* This code indicates that the sending host does not have enough +* resources to support the terminated subflow. +*/ +SK_RST_REASON_MPTCP_RST_ERESOURCE , +/* Administratively prohibited. +* This code indicates that the requested subflow is prohibited by +* the policies of the sending host. +*/ +SK_RST_REASON_MPTCP_RST_EPROHIBIT , +/* Too much outstanding data. +* This code indicates that there is an excessive amount of data +* that needs to be transmitted over the terminated subflow while +* having already been acknowledged over one or more other subflows. +* This may occur if a path has been unavailable for a short period +* and it is more efficient to reset and start again than it is to +* retransmit the queued data. +*/ +SK_RST_REASON_MPTCP_RST_EWQ2BIG , +/* Unacceptable performance. +* This code indicates that the performance of this subflow was +* too low compared to the other subflows of this Multipath TCP +* connection. +*/ +SK_RST_REASON_MPTCP_RST_EBADPERF , +/* Middlebox interference. +* Middlebox interference has been detected over this subflow, +* making MPTCP signaling invalid. For example, this may be sent +* if the checksum does not validate. +*/ +SK_RST_REASON_MPTCP_RST_EMIDDLEBOX , + +/* For the real standalone socket reset reason, we start from here */ +SK_RST_REASON_NOT_SPECIFIED , + +/* Maximum of socket reset reasons. +* It shouldn't be used as a real 'reason'. +*/ +SK_RST_REASON_MAX , +}; + +static inline enum sk_rst_reason convert_mptcp_reason (u32 reason) +{ +return reason +=RST_REASON_START ; +} +#endif
Signature · 1 line
2.37.3
PATCH v6 2/7 rstreason: prepare for passive reset
In reply to Jason XingFrom: Jason Xing <[email protected]>
Adjust the parameter and support passing reason of reset which is for now NOT_SPECIFIED. No functional changes.
--- include/net/request_sock.h | 4 +++- net/dccp/ipv4.c | 10 ++++++---- net/dccp/ipv6.c | 10 ++++++---- net/dccp/minisocks.c | 3 ++- net/ipv4/tcp_ipv4.c | 12 +++++++----- net/ipv4/tcp_minisocks.c | 3 ++- net/ipv6/tcp_ipv6.c | 15 +++++++++------ net/mptcp/subflow.c | 8 +++++--- 8 files changed, 40 insertions(+), 25 deletions(-)
include/net/request_sock.h +3 -1
@@ -18,6 +18,7 @@#include <linux/refcount.h> #include <net/sock.h> +#include <net/rstreason.h> struct request_sock ;struct sk_buff ;
@@ -34,7 +35,8 @@ struct request_sock_ops {void (*send_ack)(const struct sock *sk,struct sk_buff *skb,struct request_sock *req);void (*send_reset)(const struct sock *sk, -struct sk_buff *skb); +struct sk_buff *skb, +enum sk_rst_reason reason);void (*destructor)(struct request_sock *req);void (*syn_ack_timeout)(const struct request_sock *req); };
net/dccp/ipv4.c +6 -4
@@ -24,6 +24,7 @@#include <net/xfrm.h> #include <net/secure_seq.h> #include <net/netns/generic.h> +#include <net/rstreason.h> #include "ackvec.h" #include "ccid.h"
@@ -521,7 +522,8 @@ static int dccp_v4_send_response(const struct sock *sk, struct request_sock *reqreturn err; } -static void dccp_v4_ctl_send_reset (const struct sock *sk,struct sk_buff *rxskb) +static void dccp_v4_ctl_send_reset (const struct sock *sk,struct sk_buff *rxskb, +enum sk_rst_reason reason) {int err;const struct iphdr *rxiph;
@@ -706,7 +708,7 @@ int dccp_v4_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -dccp_v4_ctl_send_reset (sk, skb); +dccp_v4_ctl_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED );kfree_skb (skb);return 0 ; }
@@ -869,7 +871,7 @@ static int dccp_v4_rcv(struct sk_buff *skb)if (nsk == sk) {reqsk_put (req); }else if (dccp_child_process (sk, nsk, skb)) { -dccp_v4_ctl_send_reset (sk, skb); +dccp_v4_ctl_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); goto discard_and_relse; }else {sock_put (sk);
@@ -909,7 +911,7 @@ static int dccp_v4_rcv(struct sk_buff *skb)if (dh->dccph_type !=DCCP_PKT_RESET ) {DCCP_SKB_CB (skb)->dccpd_reset_code =DCCP_RESET_CODE_NO_CONNECTION ; -dccp_v4_ctl_send_reset (sk, skb); +dccp_v4_ctl_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); } discard_it:
net/dccp/ipv6.c +6 -4
@@ -29,6 +29,7 @@#include <net/secure_seq.h> #include <net/netns/generic.h> #include <net/sock.h> +#include <net/rstreason.h> #include "dccp.h" #include "ipv6.h"
@@ -256,7 +257,8 @@ static void dccp_v6_reqsk_destructor(struct request_sock *req)kfree_skb (inet_rsk (req)->pktopts); } -static void dccp_v6_ctl_send_reset (const struct sock *sk,struct sk_buff *rxskb) +static void dccp_v6_ctl_send_reset (const struct sock *sk,struct sk_buff *rxskb, +enum sk_rst_reason reason) {const struct ipv6hdr *rxip6h;struct sk_buff *skb;
@@ -656,7 +658,7 @@ static int dccp_v6_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -dccp_v6_ctl_send_reset (sk, skb); +dccp_v6_ctl_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); discard:if (opt_skb !=NULL )__kfree_skb (opt_skb);
@@ -762,7 +764,7 @@ static int dccp_v6_rcv(struct sk_buff *skb)if (nsk == sk) {reqsk_put (req); }else if (dccp_child_process (sk, nsk, skb)) { -dccp_v6_ctl_send_reset (sk, skb); +dccp_v6_ctl_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); goto discard_and_relse; }else {sock_put (sk);
@@ -801,7 +803,7 @@ static int dccp_v6_rcv(struct sk_buff *skb)if (dh->dccph_type !=DCCP_PKT_RESET ) {DCCP_SKB_CB (skb)->dccpd_reset_code =DCCP_RESET_CODE_NO_CONNECTION ; -dccp_v6_ctl_send_reset (sk, skb); +dccp_v6_ctl_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); } discard_it:
net/dccp/minisocks.c +2 -1
@@ -15,6 +15,7 @@#include <net/sock.h> #include <net/xfrm.h> #include <net/inet_timewait_sock.h> +#include <net/rstreason.h> #include "ackvec.h" #include "ccid.h"
@@ -202,7 +203,7 @@ struct sock *dccp_check_req(struct sock *sk, struct sk_buff *skb,DCCP_SKB_CB (skb)->dccpd_reset_code =DCCP_RESET_CODE_TOO_BUSY ; drop:if (dccp_hdr (skb)->dccph_type !=DCCP_PKT_RESET ) - req->rsk_ops->send_reset (sk, skb); + req->rsk_ops->send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED );inet_csk_reqsk_queue_drop (sk, req); out:
net/ipv4/tcp_ipv4.c +7 -5
@@ -70,6 +70,7 @@#include <net/xfrm.h> #include <net/secure_seq.h> #include <net/busy_poll.h> +#include <net/rstreason.h> #include <linux/inet.h> #include <linux/ipv6.h>
@@ -723,7 +724,8 @@ static bool tcp_v4_ao_sign_reset(const struct sock *sk, struct sk_buff *skb,* Exception: precedence violation. We do not implement it in any case. */ -static void tcp_v4_send_reset (const struct sock *sk,struct sk_buff *skb) +static void tcp_v4_send_reset (const struct sock *sk,struct sk_buff *skb, +enum sk_rst_reason reason) {const struct tcphdr *th =tcp_hdr (skb);struct {
@@ -1934,7 +1936,7 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -tcp_v4_send_reset (rsk, skb); +tcp_v4_send_reset (rsk, skb,SK_RST_REASON_NOT_SPECIFIED ); discard:kfree_skb_reason (skb, reason);/* Be careful here. If this function gets more complicated and
@@ -2276,7 +2278,7 @@ int tcp_v4_rcv(struct sk_buff *skb) }else { drop_reason =tcp_child_process (sk, nsk, skb);if (drop_reason) { -tcp_v4_send_reset (nsk, skb); +tcp_v4_send_reset (nsk, skb,SK_RST_REASON_NOT_SPECIFIED ); goto discard_and_relse; }sock_put (sk);
@@ -2355,7 +2357,7 @@ int tcp_v4_rcv(struct sk_buff *skb) bad_packet:__TCP_INC_STATS (net,TCP_MIB_INERRS ); }else { -tcp_v4_send_reset (NULL , skb); +tcp_v4_send_reset (NULL , skb,SK_RST_REASON_NOT_SPECIFIED ); } discard_it:
@@ -2407,7 +2409,7 @@ int tcp_v4_rcv(struct sk_buff *skb)tcp_v4_timewait_ack (sk, skb);break ;case TCP_TW_RST : -tcp_v4_send_reset (sk, skb); +tcp_v4_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED );inet_twsk_deschedule_put (inet_twsk (sk)); goto discard_it;case TCP_TW_SUCCESS :;
net/ipv4/tcp_minisocks.c +2 -1
@@ -22,6 +22,7 @@#include <net/tcp.h> #include <net/xfrm.h> #include <net/busy_poll.h> +#include <net/rstreason.h> static bool tcp_in_window (u32 seq,u32 end_seq,u32 s_win,u32 e_win) {
@@ -879,7 +880,7 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,* avoid becoming vulnerable to outside attack aiming at * resetting legit local connections. */ - req->rsk_ops->send_reset (sk, skb); + req->rsk_ops->send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); }else if (fastopen) {/* received a valid RST pkt */ reqsk_fastopen_remove (sk, req, true);tcp_reset (sk, skb);
net/ipv6/tcp_ipv6.c +9 -6
@@ -60,6 +60,7 @@#include <net/secure_seq.h> #include <net/hotdata.h> #include <net/busy_poll.h> +#include <net/rstreason.h> #include <linux/proc_fs.h> #include <linux/seq_file.h>
@@ -69,7 +70,8 @@#include <trace/events/tcp.h> -static void tcp_v6_send_reset (const struct sock *sk,struct sk_buff *skb); +static void tcp_v6_send_reset (const struct sock *sk,struct sk_buff *skb, +enum sk_rst_reason reason);static void tcp_v6_reqsk_send_ack (const struct sock *sk,struct sk_buff *skb,struct request_sock *req);
@@ -1008,7 +1010,8 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32kfree_skb (buff); } -static void tcp_v6_send_reset (const struct sock *sk,struct sk_buff *skb) +static void tcp_v6_send_reset (const struct sock *sk,struct sk_buff *skb, +enum sk_rst_reason reason) {const struct tcphdr *th =tcp_hdr (skb);struct ipv6hdr *ipv6h =ipv6_hdr (skb);
@@ -1677,7 +1680,7 @@ int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -tcp_v6_send_reset (sk, skb); +tcp_v6_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); discard:if (opt_skb)__kfree_skb (opt_skb);
@@ -1862,7 +1865,7 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb) }else { drop_reason =tcp_child_process (sk, nsk, skb);if (drop_reason) { -tcp_v6_send_reset (nsk, skb); +tcp_v6_send_reset (nsk, skb,SK_RST_REASON_NOT_SPECIFIED ); goto discard_and_relse; }sock_put (sk);
@@ -1939,7 +1942,7 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb) bad_packet:__TCP_INC_STATS (net,TCP_MIB_INERRS ); }else { -tcp_v6_send_reset (NULL , skb); +tcp_v6_send_reset (NULL , skb,SK_RST_REASON_NOT_SPECIFIED ); } discard_it:
@@ -1995,7 +1998,7 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb)tcp_v6_timewait_ack (sk, skb);break ;case TCP_TW_RST : -tcp_v6_send_reset (sk, skb); +tcp_v6_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED );inet_twsk_deschedule_put (inet_twsk (sk)); goto discard_it;case TCP_TW_SUCCESS :
net/mptcp/subflow.c +5 -3
@@ -20,6 +20,8 @@#include <net/transp_v6.h> #endif #include <net/mptcp.h> +#include <net/rstreason.h> +#include "protocol.h" #include "mib.h"
@@ -308,7 +310,7 @@ static struct dst_entry *subflow_v4_route_req(const struct sock *sk,dst_release (dst);if (!req->syncookie) - tcp_request_sock_ops.send_reset (sk, skb); + tcp_request_sock_ops.send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED );return NULL ; }
@@ -376,7 +378,7 @@ static struct dst_entry *subflow_v6_route_req(const struct sock *sk,dst_release (dst);if (!req->syncookie) - tcp6_request_sock_ops.send_reset (sk, skb); + tcp6_request_sock_ops.send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); return NULL; }#endif
@@ -911,7 +913,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,tcp_rsk (req)->drop_req = true;inet_csk_prepare_for_destroy_sock (child);tcp_done (child); - req->rsk_ops->send_reset (sk, skb); + req->rsk_ops->send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED );/* The last child reference will be released by the caller */ return child;
Signature · 1 line
2.37.3
PATCH v6 3/7 rstreason: prepare for active reset
In reply to Jason XingFrom: Jason Xing <[email protected]>
Like what we did to passive reset:
only passing possible reset reason in each active reset path.
No functional changes.
--- include/net/tcp.h | 3 ++- net/ipv4/tcp.c | 15 ++++++++++----- net/ipv4/tcp_output.c | 3 ++- net/ipv4/tcp_timer.c | 9 ++++++--- net/mptcp/protocol.c | 4 +++- net/mptcp/subflow.c | 5 +++-- 6 files changed, 26 insertions(+), 13 deletions(-)
include/net/tcp.h +2 -1
@@ -670,7 +670,8 @@ int tcp_fragment(struct sock *sk, enum tcp_queue tcp_queue,void tcp_send_probe0 (struct sock *);int tcp_write_wakeup (struct sock *,int mib);void tcp_send_fin (struct sock *sk); -void tcp_send_active_reset (struct sock *sk,gfp_t priority); +void tcp_send_active_reset (struct sock *sk,gfp_t priority, +enum sk_rst_reason reason);int tcp_send_synack (struct sock *);void tcp_push_one (struct sock *,unsigned int mss_now);void __tcp_send_ack (struct sock *sk,u32 rcv_nxt);
net/ipv4/tcp.c +10 -5
@@ -275,6 +275,7 @@#include <net/xfrm.h> #include <net/ip.h> #include <net/sock.h> +#include <net/rstreason.h> #include <linux/uaccess.h> #include <asm/ioctls.h>
@@ -2811,7 +2812,8 @@ void __tcp_close(struct sock *sk, long timeout)/* Unread data was tossed, zap the connection. */ NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPABORTONCLOSE );tcp_set_state (sk,TCP_CLOSE ); -tcp_send_active_reset (sk, sk->sk_allocation); +tcp_send_active_reset (sk, sk->sk_allocation, +SK_RST_REASON_NOT_SPECIFIED ); }else if (sock_flag (sk,SOCK_LINGER ) && !sk->sk_lingertime) {/* Check zero linger _after_ checking for unread data. */ sk->sk_prot->disconnect (sk,0 );
@@ -2885,7 +2887,8 @@ void __tcp_close(struct sock *sk, long timeout)struct tcp_sock *tp =tcp_sk (sk);if (READ_ONCE (tp->linger2) <0 ) {tcp_set_state (sk,TCP_CLOSE ); -tcp_send_active_reset (sk,GFP_ATOMIC ); +tcp_send_active_reset (sk,GFP_ATOMIC , +SK_RST_REASON_NOT_SPECIFIED );__NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPABORTONLINGER ); }else {
@@ -2903,7 +2906,8 @@ void __tcp_close(struct sock *sk, long timeout)if (sk->sk_state !=TCP_CLOSE ) {if (tcp_check_oom (sk,0 )) {tcp_set_state (sk,TCP_CLOSE ); -tcp_send_active_reset (sk,GFP_ATOMIC ); +tcp_send_active_reset (sk,GFP_ATOMIC , +SK_RST_REASON_NOT_SPECIFIED );__NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPABORTONMEMORY ); }else if (!check_net (sock_net (sk))) {
@@ -3007,7 +3011,7 @@ int tcp_disconnect(struct sock *sk, int flags)/* The last check adjusts for discrepancy of Linux wrt. RFC * states */ -tcp_send_active_reset (sk,gfp_any ()); +tcp_send_active_reset (sk,gfp_any (),SK_RST_REASON_NOT_SPECIFIED );WRITE_ONCE (sk->sk_err,ECONNRESET ); }else if (old_state ==TCP_SYN_SENT )WRITE_ONCE (sk->sk_err,ECONNRESET );
@@ -4564,7 +4568,8 @@ int tcp_abort(struct sock *sk, int err)smp_wmb ();sk_error_report (sk);if (tcp_need_reset (sk->sk_state)) -tcp_send_active_reset (sk,GFP_ATOMIC ); +tcp_send_active_reset (sk,GFP_ATOMIC , +SK_RST_REASON_NOT_SPECIFIED );tcp_done (sk); }
net/ipv4/tcp_output.c +2 -1
@@ -3586,7 +3586,8 @@ void tcp_send_fin(struct sock *sk)* was unread data in the receive queue. This behavior is recommended * by RFC 2525, section 2.17. -DaveM */ -void tcp_send_active_reset (struct sock *sk,gfp_t priority) +void tcp_send_active_reset (struct sock *sk,gfp_t priority, +enum sk_rst_reason reason) {struct sk_buff *skb;
net/ipv4/tcp_timer.c +6 -3
@@ -22,6 +22,7 @@#include <linux/module.h> #include <linux/gfp.h> #include <net/tcp.h> +#include <net/rstreason.h> static u32 tcp_clamp_rto_to_user_timeout (const struct sock *sk) {
@@ -127,7 +128,8 @@ static int tcp_out_of_resources(struct sock *sk, bool do_reset) (!tp->snd_wnd && !tp->packets_out)) do_reset = true;if (do_reset) -tcp_send_active_reset (sk,GFP_ATOMIC ); +tcp_send_active_reset (sk,GFP_ATOMIC , +SK_RST_REASON_NOT_SPECIFIED );tcp_done (sk);__NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPABORTONMEMORY );return 1 ;
@@ -768,7 +770,7 @@ static void tcp_keepalive_timer (struct timer_list *t) goto out; } } -tcp_send_active_reset (sk,GFP_ATOMIC ); +tcp_send_active_reset (sk,GFP_ATOMIC ,SK_RST_REASON_NOT_SPECIFIED ); goto death; }
@@ -795,7 +797,8 @@ static void tcp_keepalive_timer (struct timer_list *t) icsk->icsk_probes_out > 0) || (user_timeout == 0 && icsk->icsk_probes_out >= keepalive_probes(tp))) { -tcp_send_active_reset (sk,GFP_ATOMIC ); +tcp_send_active_reset (sk,GFP_ATOMIC , +SK_RST_REASON_NOT_SPECIFIED );tcp_write_err (sk); goto out; }
net/mptcp/protocol.c +3 -1
@@ -21,6 +21,7 @@#endif #include <net/mptcp.h> #include <net/xfrm.h> +#include <net/rstreason.h> #include <asm/ioctls.h> #include "protocol.h" #include "mib.h"
@@ -2569,7 +2570,8 @@ static void mptcp_check_fastclose(struct mptcp_sock *msk) slow =lock_sock_fast (tcp_sk);if (tcp_sk->sk_state !=TCP_CLOSE ) { -tcp_send_active_reset (tcp_sk,GFP_ATOMIC ); +tcp_send_active_reset (tcp_sk,GFP_ATOMIC , +SK_RST_REASON_NOT_SPECIFIED );tcp_set_state (tcp_sk,TCP_CLOSE ); }unlock_sock_fast (tcp_sk, slow);
net/mptcp/subflow.c +3 -2
@@ -414,7 +414,7 @@ void mptcp_subflow_reset(struct sock *ssk)/* must hold: tcp_done() could drop last reference on parent */ sock_hold (sk); -tcp_send_active_reset (ssk,GFP_ATOMIC ); +tcp_send_active_reset (ssk,GFP_ATOMIC ,SK_RST_REASON_NOT_SPECIFIED );tcp_done (ssk);if (!test_and_set_bit (MPTCP_WORK_CLOSE_SUBFLOW , &mptcp_sk (sk)->flags))mptcp_schedule_work (sk);
@@ -1350,7 +1350,8 @@ static bool subflow_check_data_avail(struct sock *ssk)tcp_set_state (ssk,TCP_CLOSE );while ((skb =skb_peek (&ssk->sk_receive_queue)))sk_eat_skb (ssk, skb); -tcp_send_active_reset (ssk,GFP_ATOMIC ); +tcp_send_active_reset (ssk,GFP_ATOMIC , +SK_RST_REASON_NOT_SPECIFIED );WRITE_ONCE (subflow->data_avail, false);return false; }
Signature · 1 line
2.37.3
PATCH v6 4/7 tcp: support rstreason for passive reset
In reply to Jason XingFrom: Jason Xing <[email protected]>
Reuse the dropreason logic to show the exact reason of tcp reset, so we don't need to implement those duplicated reset reasons. This patch replaces all the prior NOT_SPECIFIED reasons.
--- net/ipv4/tcp_ipv4.c | 8 ++++---- net/ipv6/tcp_ipv6.c | 8 ++++---- 2 files changed, 8 insertions(+), 8 deletions(-)
net/ipv4/tcp_ipv4.c +4 -4
@@ -1936,7 +1936,7 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -tcp_v4_send_reset (rsk, skb,SK_RST_REASON_NOT_SPECIFIED ); +tcp_v4_send_reset (rsk, skb, (u32 )reason); discard:kfree_skb_reason (skb, reason);/* Be careful here. If this function gets more complicated and
@@ -2278,7 +2278,7 @@ int tcp_v4_rcv(struct sk_buff *skb) }else { drop_reason =tcp_child_process (sk, nsk, skb);if (drop_reason) { -tcp_v4_send_reset (nsk, skb,SK_RST_REASON_NOT_SPECIFIED ); +tcp_v4_send_reset (nsk, skb, (u32 )drop_reason); goto discard_and_relse; }sock_put (sk);
@@ -2357,7 +2357,7 @@ int tcp_v4_rcv(struct sk_buff *skb) bad_packet:__TCP_INC_STATS (net,TCP_MIB_INERRS ); }else { -tcp_v4_send_reset (NULL , skb,SK_RST_REASON_NOT_SPECIFIED ); +tcp_v4_send_reset (NULL , skb, (u32 )drop_reason); } discard_it:
@@ -2409,7 +2409,7 @@ int tcp_v4_rcv(struct sk_buff *skb)tcp_v4_timewait_ack (sk, skb);break ;case TCP_TW_RST : -tcp_v4_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); +tcp_v4_send_reset (sk, skb, (u32 )drop_reason);inet_twsk_deschedule_put (inet_twsk (sk)); goto discard_it;case TCP_TW_SUCCESS :;
net/ipv6/tcp_ipv6.c +4 -4
@@ -1680,7 +1680,7 @@ int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -tcp_v6_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); +tcp_v6_send_reset (sk, skb, (u32 )reason); discard:if (opt_skb)__kfree_skb (opt_skb);
@@ -1865,7 +1865,7 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb) }else { drop_reason =tcp_child_process (sk, nsk, skb);if (drop_reason) { -tcp_v6_send_reset (nsk, skb,SK_RST_REASON_NOT_SPECIFIED ); +tcp_v6_send_reset (nsk, skb, (u32 )drop_reason); goto discard_and_relse; }sock_put (sk);
@@ -1942,7 +1942,7 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb) bad_packet:__TCP_INC_STATS (net,TCP_MIB_INERRS ); }else { -tcp_v6_send_reset (NULL , skb,SK_RST_REASON_NOT_SPECIFIED ); +tcp_v6_send_reset (NULL , skb, (u32 )drop_reason); } discard_it:
@@ -1998,7 +1998,7 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb)tcp_v6_timewait_ack (sk, skb);break ;case TCP_TW_RST : -tcp_v6_send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); +tcp_v6_send_reset (sk, skb, (u32 )drop_reason);inet_twsk_deschedule_put (inet_twsk (sk)); goto discard_it;case TCP_TW_SUCCESS :
Signature · 1 line
2.37.3
PATCH v6 5/7 mptcp: support rstreason for passive reset
In reply to Jason XingFrom: Jason Xing <[email protected]>
It relys on what reset options in the skb are as rfc8684 says. Reusing this logic can save us much energy. This patch replaces most of the prior NOT_SPECIFIED reasons.
--- net/mptcp/subflow.c | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
net/mptcp/subflow.c +17 -5
@@ -309,8 +309,13 @@ static struct dst_entry *subflow_v4_route_req(const struct sock *sk,return dst;dst_release (dst); -if (!req->syncookie) - tcp_request_sock_ops.send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); +if (!req->syncookie) { +struct mptcp_ext *mpext =mptcp_get_ext (skb); +enum sk_rst_reason reason; + + reason =convert_mptcp_reason (mpext->reset_reason); + tcp_request_sock_ops.send_reset (sk, skb, reason); + }return NULL ; }
@@ -377,8 +382,13 @@ static struct dst_entry *subflow_v6_route_req(const struct sock *sk,return dst;dst_release (dst); -if (!req->syncookie) - tcp6_request_sock_ops.send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); +if (!req->syncookie) { +struct mptcp_ext *mpext =mptcp_get_ext (skb); +enum sk_rst_reason reason; + + reason =convert_mptcp_reason (mpext->reset_reason); + tcp6_request_sock_ops.send_reset (sk, skb, reason); + } return NULL; }#endif
@@ -783,6 +793,7 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,struct mptcp_subflow_request_sock *subflow_req;struct mptcp_options_received mp_opt;bool fallback, fallback_is_fatal; +enum sk_rst_reason reason;struct mptcp_sock *owner;struct sock *child;
@@ -913,7 +924,8 @@ static struct sock *subflow_syn_recv_sock(const struct sock *sk,tcp_rsk (req)->drop_req = true;inet_csk_prepare_for_destroy_sock (child);tcp_done (child); - req->rsk_ops->send_reset (sk, skb,SK_RST_REASON_NOT_SPECIFIED ); + reason =convert_mptcp_reason (mptcp_get_ext (skb)->reset_reason); + req->rsk_ops->send_reset (sk, skb, reason);/* The last child reference will be released by the caller */ return child;
Signature · 1 line
2.37.3
PATCH v6 6/7 mptcp: introducing a helper into active reset logic
In reply to Jason XingFrom: Jason Xing <[email protected]>
Since we have mapped every mptcp reset reason definition in enum sk_rst_reason, introducing a new helper can cover some missing places where we have already set the subflow->reset_reason.
Note: using SK_RST_REASON_NOT_SPECIFIED is the same as SK_RST_REASON_MPTCP_RST_EUNSPEC. They are both unknown. So we can convert it directly.
---
--- net/mptcp/protocol.c | 4 +--- net/mptcp/protocol.h | 11 +++++++++++ net/mptcp/subflow.c | 6 ++---- 3 files changed, 14 insertions(+), 7 deletions(-)
net/mptcp/protocol.c +1 -3
@@ -21,7 +21,6 @@#endif #include <net/mptcp.h> #include <net/xfrm.h> -#include <net/rstreason.h> #include <asm/ioctls.h> #include "protocol.h" #include "mib.h"
@@ -2570,8 +2569,7 @@ static void mptcp_check_fastclose(struct mptcp_sock *msk) slow =lock_sock_fast (tcp_sk);if (tcp_sk->sk_state !=TCP_CLOSE ) { -tcp_send_active_reset (tcp_sk,GFP_ATOMIC , -SK_RST_REASON_NOT_SPECIFIED ); +mptcp_send_active_reset_reason (tcp_sk);tcp_set_state (tcp_sk,TCP_CLOSE ); }unlock_sock_fast (tcp_sk, slow);
net/mptcp/protocol.h +11 -0
@@ -12,6 +12,7 @@#include <net/inet_connection_sock.h> #include <uapi/linux/mptcp.h> #include <net/genetlink.h> +#include <net/rstreason.h> #include "mptcp_pm_gen.h"
@@ -581,6 +582,16 @@ mptcp_subflow_ctx_reset(struct mptcp_subflow_context *subflow)WRITE_ONCE (subflow->local_id,-1 ); } +static inline void +mptcp_send_active_reset_reason (struct sock *sk) +{ +struct mptcp_subflow_context *subflow =mptcp_subflow_ctx (sk); +enum sk_rst_reason reason; + + reason =convert_mptcp_reason (subflow->reset_reason); +tcp_send_active_reset (sk,GFP_ATOMIC , reason); +} +static inline u64 mptcp_subflow_get_map_offset (const struct mptcp_subflow_context *subflow) {
net/mptcp/subflow.c +2 -4
@@ -20,7 +20,6 @@#include <net/transp_v6.h> #endif #include <net/mptcp.h> -#include <net/rstreason.h> #include "protocol.h" #include "mib.h"
@@ -424,7 +423,7 @@ void mptcp_subflow_reset(struct sock *ssk)/* must hold: tcp_done() could drop last reference on parent */ sock_hold (sk); -tcp_send_active_reset (ssk,GFP_ATOMIC ,SK_RST_REASON_NOT_SPECIFIED ); +mptcp_send_active_reset_reason (ssk);tcp_done (ssk);if (!test_and_set_bit (MPTCP_WORK_CLOSE_SUBFLOW , &mptcp_sk (sk)->flags))mptcp_schedule_work (sk);
@@ -1362,8 +1361,7 @@ static bool subflow_check_data_avail(struct sock *ssk)tcp_set_state (ssk,TCP_CLOSE );while ((skb =skb_peek (&ssk->sk_receive_queue)))sk_eat_skb (ssk, skb); -tcp_send_active_reset (ssk,GFP_ATOMIC , -SK_RST_REASON_NOT_SPECIFIED ); +mptcp_send_active_reset_reason (ssk);WRITE_ONCE (subflow->data_avail, false);return false; }
Signature · 1 line
2.37.3
PATCH v6 7/7 rstreason: make it work in trace world
In reply to Jason XingFrom: Jason Xing <[email protected]>
At last, we should let it work by introducing this reset reason in trace world.
One of the possible expected outputs is:
... tcp_send_reset: skbaddr=xxx skaddr=xxx src=xxx dest=xxx state=TCP_ESTABLISHED reason=NOT_SPECIFIED
--- include/trace/events/tcp.h | 37 +++++++++++++++++++++++++++++++++---- net/ipv4/tcp_ipv4.c | 2 +- net/ipv4/tcp_output.c | 2 +- net/ipv6/tcp_ipv6.c | 2 +- 4 files changed, 36 insertions(+), 7 deletions(-)
include/trace/events/tcp.h +33 -4
@@ -11,6 +11,7 @@#include <net/ipv6.h> #include <net/tcp.h> #include <linux/sock_diag.h> +#include <net/rstreason.h> /* * tcp event with arguments sk and skb
@@ -74,20 +75,38 @@ DEFINE_EVENT(tcp_event_sk_skb, tcp_retransmit_skb,TP_ARGS (sk, skb) ); +#undef FN1 +#define FN1 (reason) TRACE_DEFINE_ENUM(SK_RST_REASON_##reason); +#undef FN2 +#define FN2 (reason) TRACE_DEFINE_ENUM(SKB_DROP_REASON_##reason); +DEFINE_RST_REASON(FN1, FN1) + +#undef FN1 +#undef FNe1 +#define FN1 (reason) { SK_RST_REASON_##reason, #reason }, +#define FNe1 (reason) { SK_RST_REASON_##reason, #reason } + +#undef FN2 +#undef FNe2 +#define FN2 (reason) { SKB_DROP_REASON_##reason, #reason }, +#define FNe2 (reason) { SKB_DROP_REASON_##reason, #reason }/* * skb of trace_tcp_send_reset is the skb that caused RST. In case of * active reset, skb should be NULL */ TRACE_EVENT(tcp_send_reset, - TP_PROTO(const struct sock *sk, const struct sk_buff *skb), + TP_PROTO(const struct sock *sk, + const struct sk_buff *skb, + const enum sk_rst_reason reason), - TP_ARGS(sk, skb), + TP_ARGS(sk, skb, reason), TP_STRUCT__entry( __field(const void *, skbaddr) __field(const void *, skaddr) __field(int, state) + __field(enum sk_rst_reason, reason) __array(__u8, saddr, sizeof(struct sockaddr_in6)) __array(__u8, daddr, sizeof(struct sockaddr_in6)) ),
@@ -113,14 +132,24 @@ TRACE_EVENT(tcp_send_reset,*/ TP_STORE_ADDR_PORTS_SKB (skb, th, entry->daddr, entry->saddr); } + __entry->reason = reason; ), -TP_printk ("skbaddr=%p skaddr=%p src=%pISpc dest=%pISpc state=%s" , +TP_printk ("skbaddr=%p skaddr=%p src=%pISpc dest=%pISpc state=%s reason=%s" , __entry->skbaddr, __entry->skaddr, __entry->saddr, __entry->daddr, - __entry->state ?show_tcp_state_name (__entry->state) :"UNKNOWN" ) + __entry->state ?show_tcp_state_name (__entry->state) :"UNKNOWN" , + __entry->reason <RST_REASON_START ? +__print_symbolic (__entry->reason,DEFINE_DROP_REASON (FN2 , FNe2)) : +__print_symbolic (__entry->reason,DEFINE_RST_REASON (FN1 , FNe1))) ); +#undef FN1 +#undef FNe1 + +#undef FN2 +#undef FNe2 +/* * tcp event with arguments sk *
net/ipv4/tcp_ipv4.c +1 -1
@@ -871,7 +871,7 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb,if (sk) arg.bound_dev_if = sk->sk_bound_dev_if; -trace_tcp_send_reset (sk, skb); +trace_tcp_send_reset (sk, skb, reason);BUILD_BUG_ON (offsetof(struct sock , sk_bound_dev_if) != offsetof(struct inet_timewait_sock , tw_bound_dev_if));
net/ipv4/tcp_output.c +1 -1
@@ -3612,7 +3612,7 @@ void tcp_send_active_reset(struct sock *sk, gfp_t priority,/* skb of trace_tcp_send_reset() keeps the skb that caused RST, * skb here is different to the troublesome skb, so use NULL */ -trace_tcp_send_reset (sk,NULL ); +trace_tcp_send_reset (sk,NULL ,SK_RST_REASON_NOT_SPECIFIED ); }/* Send a crossed SYN-ACK during socket establishment.
net/ipv6/tcp_ipv6.c +1 -1
@@ -1133,7 +1133,7 @@ static void tcp_v6_send_reset(const struct sock *sk, struct sk_buff *skb, label =ip6_flowlabel (ipv6h); } -trace_tcp_send_reset (sk, skb); +trace_tcp_send_reset (sk, skb, reason); tcp_v6_send_response(sk, skb, seq, ack_seq, 0, 0, 0, oif, 1, ipv6_get_dsfield(ipv6h), label, priority, txhash,
Signature · 1 line
2.37.3
Re: [PATCH net-next v6 1/7] net: introduce rstreason to detect why the RST is sent
In reply to Jason XingQuoted 14 lines · Jason Xing
From: Jason Xing <[email protected]>
Add a new standalone file for the easy future extension to support both active reset and passive reset in the TCP/DCCP/MPTCP protocols.
This patch only does the preparations for reset reason mechanism, nothing else changes.
The reset reasons are divided into three parts:
1) reuse drop reasons for passive reset in TCP
2) reuseMP_TCPRSToption for MPTCP
3) our own reasons
Quoted 6 lines
I will implement the basic codes of active/passive reset reason in those three protocols, which is not complete for this moment. But it provides a new chance to let other people add more reasons into it:)
Signed-off-by Jason Xing <[email protected]>
My original suggestion was to use normal values in 'enum skb_drop_reason', even if there was not necessarily a 'drop' in the common sense.
https://lore.kernel.org/all/CANn89iJw8x-LqgsWOeJQQvgVg6DnL5aBRLi10QN2WBdr+X4k=w@mail.gmail.com/
This would avoid these ugly casts later, even casting an enum to other ones is not very logical.
Going through an u32 pivot is quite a hack.
If you feel the need to put them in a special group, this is fine by me.
Re: [PATCH net-next v6 1/7] net: introduce rstreason to detect why the RST is sent
Hello Eric,
Quoted 23 lines · Eric Dumazet
Quoted 14 lines · Jason Xing
From: Jason Xing <[email protected]>
Add a new standalone file for the easy future extension to support both active reset and passive reset in the TCP/DCCP/MPTCP protocols.
This patch only does the preparations for reset reason mechanism, nothing else changes.
The reset reasons are divided into three parts:
1) reuse drop reasons for passive reset in TCP
2) reuseMP_TCPRSToption for MPTCP
3) our own reasonsQuoted 6 lines
I will implement the basic codes of active/passive reset reason in those three protocols, which is not complete for this moment. But it provides a new chance to let other people add more reasons into it:)
Signed-off-by Jason Xing <[email protected]>
Quoted 8 lines
My original suggestion was to use normal values in 'enum
skb_drop_reason', even if there was not necessarily a 'drop' in the common sense.https://lore.kernel.org/all/CANn89iJw8x-LqgsWOeJQQvgVg6DnL5aBRLi10QN2WBdr+X4k=w@mail.gmail.com/
This would avoid these ugly casts later, even casting an enum to other ones is not very logical.
Thanks for your comment.
It's a little bit tricky. That's the reason I documented and commented on this in the rstreason.h file. I hope it's not that hard to understand.
Quoted 3 lines
Going through an u32 pivot is quite a hack.
If you feel the need to put them in a special group, this is fine by me.
Yes, rst reasons only partially rely on the drop reason mechanism to support passive rst for TCP well, but not supporting other cases. My final goal is to cover all the cases for the future, so I wish I can put it into a separate group, then people like me who find it useful can introduce more reasons into it.
Thanks,
Jason
Hi Jason,
Thank you for your modifications, that's great!
Our CI did some validations and here is its report:
- KVM Validation: normal: Success! ✅
- KVM Validation: debug: Unstable: 2 failed test(s): packetdrill_dss packetdrill_fastopen 🔴
- KVM Validation: btf (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/8719220506
Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/a79c83426c9f
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=845351
If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:
$ cd [kernel source code]
$ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
--pull always mptcp/mptcp-upstream-virtme-docker:latest \
auto-normalFor more details:
https://github.com/multipath-tcp/mptcp-upstream-virtme-docker
Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)
Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)
Quoted 79 lines · Jason Xing
From: Jason Xing <[email protected]>
In production, there are so many cases about why the RST skb is sent but we don't have a very convenient/fast method to detect the exact underlying reasons.
RST is implemented in two kinds: passive kind (like
tcp_v4_send_reset()) and active kind (liketcp_send_active_reset()). The former can be traced carefully 1) in TCP, with the help of drop reasons, which is based on Eric's idea[1], 2) in MPTCP, with the help of reset options defined in RFC 8684. The latter is relatively independent, which should be implemented on our own.In this series, I focus on the fundamental implement mostly about how the rstreason mechnism works and give the detailed passive part as an example, not including the active reset part. In future, we can go further and refine those
NOT_SPECIFIEDreasons.Here are some examples when tracing:
<idle>-0 [002] ..s1. 1830.262425:tcp_send_reset: skbaddr=x
skaddr=x src=x dest=x state=x reason=NOT_SPECIFIED
<idle>-0 [002] ..s1. 1830.262425:tcp_send_reset: skbaddr=x
skaddr=x src=x dest=x state=x reason=NO_SOCKET[1]v6
1. add back casts, or else they are treated as error.v5
1. address format issue (like reverse xmas tree) (Eric, Paolo)
2. remove unnecessary casts. (Eric)
3. introduce a helper used in mptcp active reset. See patch 6. (Paolo)v4
1. passing 'enum
sk_rst_reason' for readability when tracing (Antoine)v3
1. rebase (mptcp part) and address what Mat suggested.
v2
1. rebase against the latest net-next tree
Jason Xing (7):
net: introduce rstreason to detect why the RST is sent
rstreason: prepare for passive reset
rstreason: prepare for active reset
tcp: support rstreason for passive reset
mptcp: support rstreason for passive reset
mptcp: introducing a helper into active reset logic
rstreason: make it work in trace worldinclude/net/request_sock.h | 4 +- include/net/rstreason.h | 93 ++++++++++++++++++++++++++++++++++++++ include/net/tcp.h | 3 +- include/trace/events/tcp.h | 37 +++++++++++++-- net/dccp/ipv4.c | 10 ++-- net/dccp/ipv6.c | 10 ++-- net/dccp/minisocks.c | 3 +- net/ipv4/tcp.c | 15 ++++-- net/ipv4/tcp_ipv4.c | 14 +++--- net/ipv4/tcp_minisocks.c | 3 +- net/ipv4/tcp_output.c | 5 +- net/ipv4/tcp_timer.c | 9 ++-- net/ipv6/tcp_ipv6.c | 17 ++++--- net/mptcp/protocol.c | 2 +- net/mptcp/protocol.h | 11 +++++ net/mptcp/subflow.c | 27 ++++++++--- 16 files changed, 216 insertions(+), 47 deletions(-) create mode 100644 include/net/rstreason.h
Quoted 3 lines
--
2.37.3
Hello maintainers,
I'm not sure why the patch series has been changed to 'Changes Requested', until now I don't think I need to change something.
Should I repost this series (keeping the v6 tag) and then wait for more comments?
Thanks,
Jason
Jakub Kicinski
maintainerQuoted 5 lines · On Thu, 18 Apr 2024 11:30:02 +0800 Jason Xing wrote:
I'm not sure why the patch series has been changed to 'Changes Requested', until now I don't think I need to change something.
Should I repost this series (keeping the v6 tag) and then wait for more comments?
If Eric doesn't like it - it's not getting merged.
Quoted 9 lines · Jakub Kicinski
Quoted 5 lines · On Thu, 18 Apr 2024 11:30:02 +0800 Jason Xing wrote:
I'm not sure why the patch series has been changed to 'Changes Requested', until now I don't think I need to change something.
Should I repost this series (keeping the v6 tag) and then wait for more comments?
If Eric doesn't like it - it's not getting merged.
I'm not a English native speaker. If I understand correctly, it seems that Eric doesn't object to the patch series. Here is the quotation [1]:
"If you feel the need to put them in a special group, this is fine by me."
This rst reason mechanism can cover all the possible reasons for both TCP and MPTCP. We don't need to reinvent some definitions of reset reasons which are totally the same as drop reasons. Also, we don't need to reinvent something to cover MPTCP. If people are willing to contribute more rst reasons, they can find a good place.
Reset is one big complicated 'issue' in production. I spent a lot of time handling all kinds of reset reasons daily. I'm apparently not the only one. I just want to make admins' lives easier, including me. This special/separate reason group is important because we can extend it in the future, which will not get confused.
I hope it can have a chance to get merged :) Thank you.
[1]: https://lore.kernel.org/all/CANn89i+aLO_aGYC8dr8dkFyi+6wpzCGrogysvgR8FrfRvaa-Vg@mail.gmail.com/
Thanks,
Jason
Quoted 29 lines · Jason Xing
Quoted 9 lines · Jakub Kicinski
Quoted 5 lines · On Thu, 18 Apr 2024 11:30:02 +0800 Jason Xing wrote:
I'm not sure why the patch series has been changed to 'Changes Requested', until now I don't think I need to change something.
Should I repost this series (keeping the v6 tag) and then wait for more comments?
If Eric doesn't like it - it's not getting merged.
I'm not a English native speaker. If I understand correctly, it seems that Eric doesn't object to the patch series. Here is the quotation [1]:
"If you feel the need to put them in a special group, this is fine by me."This rst reason mechanism can cover all the possible reasons for both TCP and MPTCP. We don't need to reinvent some definitions of reset reasons which are totally the same as drop reasons. Also, we don't need to reinvent something to cover MPTCP. If people are willing to contribute more rst reasons, they can find a good place.
Reset is one big complicated 'issue' in production. I spent a lot of time handling all kinds of reset reasons daily. I'm apparently not the only one. I just want to make admins' lives easier, including me. This special/separate reason group is important because we can extend it in the future, which will not get confused.
Quoted 6 lines
I hope it can have a chance to get merged :) Thank you.
[1]: https://lore.kernel.org/all/CANn89i+aLO_aGYC8dr8dkFyi+6wpzCGrogysvgR8FrfRvaa-Vg@mail.gmail.com/
Thanks,
Jason
My objection was these casts between enums. Especially if hiding with (u32)
I see no reason for adding these casts in TCP stack.
When I said "If you feel the need to put them in a special group, this is fine by me.",
this was really about partitioning the existing enum into groups, if you prefer having a group of 'RES reasons'
Quoted 31 lines · Eric Dumazet
Quoted 29 lines · Jason Xing
Quoted 9 lines · Jakub Kicinski
Quoted 5 lines · On Thu, 18 Apr 2024 11:30:02 +0800 Jason Xing wrote:
I'm not sure why the patch series has been changed to 'Changes Requested', until now I don't think I need to change something.
Should I repost this series (keeping the v6 tag) and then wait for more comments?
If Eric doesn't like it - it's not getting merged.
I'm not a English native speaker. If I understand correctly, it seems that Eric doesn't object to the patch series. Here is the quotation [1]:
"If you feel the need to put them in a special group, this is fine by me."This rst reason mechanism can cover all the possible reasons for both TCP and MPTCP. We don't need to reinvent some definitions of reset reasons which are totally the same as drop reasons. Also, we don't need to reinvent something to cover MPTCP. If people are willing to contribute more rst reasons, they can find a good place.
Reset is one big complicated 'issue' in production. I spent a lot of time handling all kinds of reset reasons daily. I'm apparently not the only one. I just want to make admins' lives easier, including me. This special/separate reason group is important because we can extend it in the future, which will not get confused.
Quoted 8 lines
Quoted 6 lines
I hope it can have a chance to get merged :) Thank you.
[1]: https://lore.kernel.org/all/CANn89i+aLO_aGYC8dr8dkFyi+6wpzCGrogysvgR8FrfRvaa-Vg@mail.gmail.com/
Thanks,
JasonMy objection was these casts between enums. Especially if hiding with (u32)
So I should explicitly cast it like this:
tcp_v4_send_reset(rsk, skb, (enum sk_rst_reason)reason);
?
Quoted 2 lines
I see no reason for adding these casts in TCP stack.
Sorry, I don't know why the casts really make you so annoyed. But I still think it's not a bad way to handle all the cases for RST.
Supposing not to add a enum sk_rst_reason{}, passing drop reasons only works well in TCP for passive rests. For active reset cases (in the tcp_send_active_reset()), it's meaningless/confusing to insist on reusing the drop reason because I have to add some reset reasons (that are only used in RST cases) in the enum skb_drop_reason{}, which is really weird, in my view. The same problem exists in how to handle MPTCP. So I prefer putting them in a separate group like now. What do you think about it, right now?
Thanks,
Jason
Quoted 52 lines · Jason Xing
Quoted 31 lines · Eric Dumazet
Quoted 29 lines · Jason Xing
Quoted 9 lines · Jakub Kicinski
Quoted 5 lines · On Thu, 18 Apr 2024 11:30:02 +0800 Jason Xing wrote:
I'm not sure why the patch series has been changed to 'Changes Requested', until now I don't think I need to change something.
Should I repost this series (keeping the v6 tag) and then wait for more comments?
If Eric doesn't like it - it's not getting merged.
I'm not a English native speaker. If I understand correctly, it seems that Eric doesn't object to the patch series. Here is the quotation [1]:
"If you feel the need to put them in a special group, this is fine by me."This rst reason mechanism can cover all the possible reasons for both TCP and MPTCP. We don't need to reinvent some definitions of reset reasons which are totally the same as drop reasons. Also, we don't need to reinvent something to cover MPTCP. If people are willing to contribute more rst reasons, they can find a good place.
Reset is one big complicated 'issue' in production. I spent a lot of time handling all kinds of reset reasons daily. I'm apparently not the only one. I just want to make admins' lives easier, including me. This special/separate reason group is important because we can extend it in the future, which will not get confused.
Quoted 8 lines
Quoted 6 lines
I hope it can have a chance to get merged :) Thank you.
[1]: https://lore.kernel.org/all/CANn89i+aLO_aGYC8dr8dkFyi+6wpzCGrogysvgR8FrfRvaa-Vg@mail.gmail.com/
Thanks,
JasonMy objection was these casts between enums. Especially if hiding with (u32)
So I should explicitly cast it like this:
tcp_v4_send_reset(rsk, skb, (enum sk_rst_reason)reason);?
Quoted 2 lines
I see no reason for adding these casts in TCP stack.
Sorry, I don't know why the casts really make you so annoyed. But I still think it's not a bad way to handle all the cases for RST.
Quoted 8 lines
Supposing not to add a enum sk_rst_reason{}, passing drop reasons only works well in TCP for passive rests. For active reset cases (in the
tcp_send_active_reset()), it's meaningless/confusing to insist on reusing the drop reason because I have to add some reset reasons (that are only used in RST cases) in the enum skb_drop_reason{}, which is really weird, in my view. The same problem exists in how to handle MPTCP. So I prefer putting them in a separate group like now. What do you think about it, right now?
The description in the previous email may be too long. The key point is that we're not supporting only for passive resets, right?
Thanks,
Jason
Quoted 4 lines
When I said "If you feel the need to put them in a special group, this is fine by me.",
this was really about partitioning the existing enum into groups, if you prefer having a group of 'RES reasons'
Are you suggesting copying what we need from enum skb_drop_reason{} to enum sk_rst_reason{}? Why not reusing them directly. I have no idea what the side effect of cast conversion itself is?
If __not__ doing so (copying reasons one by one), for passive rests, we can totally rely on the drop reason, which means if we implement more reasons for skb drop happening in reset cases, we don't need to handle reset cases over and over again (like adding rst reasons just after newly added drop reasons if without cast conversions). It's easier to maintain the reset reason part if we can apply the current patch series.
Thank you.
Quoted 9 lines · Jason Xing
Quoted 5 lines
When I said "If you feel the need to put them in a special group, this is fine by me.",
this was really about partitioning the existing enum into groups, if you prefer having a group of 'RES reasons'Are you suggesting copying what we need from enum skb_drop_reason{} to enum sk_rst_reason{}? Why not reusing them directly. I have no idea what the side effect of cast conversion itself is?
Sorry that I'm writing this email. I'm worried my statement is not that clear, so I write one simple snippet which can help me explain well :)
Allow me give NO_SOCKET as an example:
net/ipv4/icmp.c +0 -0
@@ -767,6 +767,7 @@ void __icmp_send(struct sk_buff *skb_in, int type,
int code, __be32 info,
if (!fl4.saddr)
fl4.saddr = htonl(INADDR_DUMMY);+ trace_icmp_send(skb_in, type, code);
icmp_push_reply(sk, &icmp_param, &fl4, &ipc, &rt);
ende:
ip_rt_put(rt);net/ipv4/tcp_ipv4.c +3 -1
@@ -2160,6 +2160,7 @@ int tcp_v4_rcv(struct sk_buff *skb) {struct net *net =dev_net (skb->dev);enum skb_drop_reason drop_reason; +enum sk_rst_reason rst_reason;int sdif =inet_sdif (skb);int dif =inet_iif (skb);const struct iphdr *iph;
@@ -2355,7 +2356,8 @@ int tcp_v4_rcv(struct sk_buff *skb) bad_packet:__TCP_INC_STATS (net,TCP_MIB_INERRS ); }else { -tcp_v4_send_reset (NULL , skb); + rst_reason =RST_REASON_NO_SOCKET ; +tcp_v4_send_reset (NULL , skb, rst_reason); } discard_it:
As you can see, we need to add a new 'rst_reason' variable which actually is the same as drop reason. They are the same except for the enum type... Such rst_reasons/drop_reasons are all over the place.
Eric, if you have a strong preference, I can do it as you said.
Well, how about explicitly casting them like this based on the current series. It looks better and clearer and more helpful to people who is reading codes to understand:
net/ipv4/tcp_ipv4.c +1 -1
@@ -1936,7 +1936,7 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -tcp_v4_send_reset (rsk, skb, (u32 )reason); +tcp_v4_send_reset (rsk, skb, (enum sk_rst_reason )reason); discard:kfree_skb_reason (skb, reason);/* Be careful here. If this function gets more complicated and
Thanks for your patience again.
Jason
Quoted 67 lines · Jason Xing
Quoted 9 lines · Jason Xing
Quoted 5 lines
When I said "If you feel the need to put them in a special group, this is fine by me.",
this was really about partitioning the existing enum into groups, if you prefer having a group of 'RES reasons'Are you suggesting copying what we need from enum skb_drop_reason{} to enum sk_rst_reason{}? Why not reusing them directly. I have no idea what the side effect of cast conversion itself is?
Sorry that I'm writing this email. I'm worried my statement is not that clear, so I write one simple snippet which can help me explain well :)
Allow me give
NO_SOCKETas an example:net/ipv4/icmp.c +1
-0@@ -767,6 +767,7 @@ void __icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info, if (!fl4.saddr) fl4.saddr = htonl(INADDR_DUMMY); +trace_icmp_send (skb_in, type, code);icmp_push_reply (sk, &icmp_param, &fl4, &ipc, &rt); ende:ip_rt_put(rt);net/ipv4/tcp_ipv4.c +3
-1@@ -2160,6 +2160,7 @@ int tcp_v4_rcv(struct sk_buff *skb) {struct net *net =dev_net (skb->dev);enum skb_drop_reason drop_reason; +enum sk_rst_reason rst_reason;int sdif =inet_sdif (skb);int dif =inet_iif (skb);const struct iphdr *iph;@@ -2355,7 +2356,8 @@ int tcp_v4_rcv(struct sk_buff *skb) bad_packet:__TCP_INC_STATS (net,TCP_MIB_INERRS ); }else { -tcp_v4_send_reset (NULL , skb); + rst_reason =RST_REASON_NO_SOCKET ; +tcp_v4_send_reset (NULL , skb, rst_reason); } discard_it:As you can see, we need to add a new '
rst_reason' variable which actually is the same as drop reason. They are the same except for the enum type... Such rst_reasons/drop_reasons are all over the place.Eric, if you have a strong preference, I can do it as you said.
Well, how about explicitly casting them like this based on the current series. It looks better and clearer and more helpful to people who is reading codes to understand:
net/ipv4/tcp_ipv4.c +0
-0
net/ipv4/tcp_ipv4.c int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb) · line 1942
Quoted 8 lines
net/ipv4/tcp_ipv4.c +1
-1@@ -1936,7 +1936,7 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -tcp_v4_send_reset (rsk, skb, (u32 )reason); +tcp_v4_send_reset (rsk, skb, (enum sk_rst_reason )reason); discard:kfree_skb_reason (skb, reason);/* Be careful here. If this function gets more complicated and
It makes no sense to declare an enum sk_rst_reason and then convert it to drop_reason
or vice versa.
Next thing you know, compiler guys will add a new -Woption that will forbid such conversions.
Please add to tcp_v4_send_reset() an skb_drop_reason, not a new enum.
skb_drop_reason are simply values that are later converted to strings...
So : Do not declare a new enum.
Quoted 82 lines · Eric Dumazet
Quoted 67 lines · Jason Xing
Quoted 9 lines · Jason Xing
Quoted 5 lines
When I said "If you feel the need to put them in a special group, this is fine by me.",
this was really about partitioning the existing enum into groups, if you prefer having a group of 'RES reasons'Are you suggesting copying what we need from enum skb_drop_reason{} to enum sk_rst_reason{}? Why not reusing them directly. I have no idea what the side effect of cast conversion itself is?
Sorry that I'm writing this email. I'm worried my statement is not that clear, so I write one simple snippet which can help me explain well :)
Allow me give
NO_SOCKETas an example:net/ipv4/icmp.c +1
-0@@ -767,6 +767,7 @@ void __icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info, if (!fl4.saddr) fl4.saddr = htonl(INADDR_DUMMY); +trace_icmp_send (skb_in, type, code);icmp_push_reply (sk, &icmp_param, &fl4, &ipc, &rt); ende:ip_rt_put(rt);net/ipv4/tcp_ipv4.c +3
-1@@ -2160,6 +2160,7 @@ int tcp_v4_rcv(struct sk_buff *skb) {struct net *net =dev_net (skb->dev);enum skb_drop_reason drop_reason; +enum sk_rst_reason rst_reason;int sdif =inet_sdif (skb);int dif =inet_iif (skb);const struct iphdr *iph;@@ -2355,7 +2356,8 @@ int tcp_v4_rcv(struct sk_buff *skb) bad_packet:__TCP_INC_STATS (net,TCP_MIB_INERRS ); }else { -tcp_v4_send_reset (NULL , skb); + rst_reason =RST_REASON_NO_SOCKET ; +tcp_v4_send_reset (NULL , skb, rst_reason); } discard_it:As you can see, we need to add a new '
rst_reason' variable which actually is the same as drop reason. They are the same except for the enum type... Such rst_reasons/drop_reasons are all over the place.Eric, if you have a strong preference, I can do it as you said.
Well, how about explicitly casting them like this based on the current series. It looks better and clearer and more helpful to people who is reading codes to understand:
net/ipv4/tcp_ipv4.c +0
-0net/ipv4/tcp_ipv4.c int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb) · line 1942
Quoted 8 lines
net/ipv4/tcp_ipv4.c +1
-1@@ -1936,7 +1936,7 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -tcp_v4_send_reset (rsk, skb, (u32 )reason); +tcp_v4_send_reset (rsk, skb, (enum sk_rst_reason )reason); discard:kfree_skb_reason (skb, reason);/* Be careful here. If this function gets more complicated and It makes no sense to declare an enum
sk_rst_reasonand then convert it todrop_reason
or vice versa.
Quoted 4 lines
Next thing you know, compiler guys will add a new -Woption that will forbid such conversions.
Please add to
tcp_v4_send_reset()anskb_drop_reason, not a new enum.
Ah... It looks like I didn't make myself clear again. Sorry... Actually I wrote this part many times. My conclusion is that It's not feasible to do this.
REASONS:
If we __only__ need to deal with this passive reset in TCP, it's fine. We pass a skb_drop_reason which should also be converted to u32 type in tcp_v4_send_reset() as you said, it can work. People who use the trace will see the reason with the 'SKB_DROP_REASON' prefix stripped.
But we have to deal with other cases. A few questions are listed here:
1) What about tcp_send_active_reset() in TCP/MPTCP? Passing weird drop reasons? There is no drop reason at all. I think people will get confused. So I believe we should invent new definitions to cope with it.
2) What about the .send_reset callback in the subflow_syn_recv_sock() in MPTCP? The reasons in MPTCP are only definitions (such as MPTCP_RST_EUNSPEC). I don't think we can convert them into the enum skb_drop_reason type.
So where should we group those various reasons?
Introducing a new enum is for extension and compatibility for all kinds of reset reasons.
What do you think?
Thanks,
Jason
Quoted 4 lines
skb_drop_reasonare simply values that are later converted to strings...So : Do not declare a new enum.
Quoted 109 lines · Jason Xing
Quoted 82 lines · Eric Dumazet
Quoted 67 lines · Jason Xing
Quoted 9 lines · Jason Xing
Quoted 5 lines
When I said "If you feel the need to put them in a special group, this is fine by me.",
this was really about partitioning the existing enum into groups, if you prefer having a group of 'RES reasons'Are you suggesting copying what we need from enum skb_drop_reason{} to enum sk_rst_reason{}? Why not reusing them directly. I have no idea what the side effect of cast conversion itself is?
Sorry that I'm writing this email. I'm worried my statement is not that clear, so I write one simple snippet which can help me explain well :)
Allow me give
NO_SOCKETas an example:net/ipv4/icmp.c +1
-0@@ -767,6 +767,7 @@ void __icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info, if (!fl4.saddr) fl4.saddr = htonl(INADDR_DUMMY); +trace_icmp_send (skb_in, type, code);icmp_push_reply (sk, &icmp_param, &fl4, &ipc, &rt); ende:ip_rt_put(rt);net/ipv4/tcp_ipv4.c +3
-1@@ -2160,6 +2160,7 @@ int tcp_v4_rcv(struct sk_buff *skb) {struct net *net =dev_net (skb->dev);enum skb_drop_reason drop_reason; +enum sk_rst_reason rst_reason;int sdif =inet_sdif (skb);int dif =inet_iif (skb);const struct iphdr *iph;@@ -2355,7 +2356,8 @@ int tcp_v4_rcv(struct sk_buff *skb) bad_packet:__TCP_INC_STATS (net,TCP_MIB_INERRS ); }else { -tcp_v4_send_reset (NULL , skb); + rst_reason =RST_REASON_NO_SOCKET ; +tcp_v4_send_reset (NULL , skb, rst_reason); } discard_it:As you can see, we need to add a new '
rst_reason' variable which actually is the same as drop reason. They are the same except for the enum type... Such rst_reasons/drop_reasons are all over the place.Eric, if you have a strong preference, I can do it as you said.
Well, how about explicitly casting them like this based on the current series. It looks better and clearer and more helpful to people who is reading codes to understand:
net/ipv4/tcp_ipv4.c +0
-0net/ipv4/tcp_ipv4.c int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb) · line 1942
Quoted 8 lines
net/ipv4/tcp_ipv4.c +1
-1@@ -1936,7 +1936,7 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -tcp_v4_send_reset (rsk, skb, (u32 )reason); +tcp_v4_send_reset (rsk, skb, (enum sk_rst_reason )reason); discard:kfree_skb_reason (skb, reason);/* Be careful here. If this function gets more complicated and It makes no sense to declare an enum
sk_rst_reasonand then convert it todrop_reason
or vice versa.Quoted 4 lines
Next thing you know, compiler guys will add a new -Woption that will forbid such conversions.
Please add to
tcp_v4_send_reset()anskb_drop_reason, not a new enum.Ah... It looks like I didn't make myself clear again. Sorry... Actually I wrote this part many times. My conclusion is that It's not feasible to do this.
REASONS:
If we__only__need to deal with this passive reset in TCP, it's fine. We pass askb_drop_reasonwhich should also be converted to u32 type intcp_v4_send_reset()as you said, it can work. People who use the trace will see the reason with the 'SKB_DROP_REASON' prefix stripped.But we have to deal with other cases. A few questions are listed here:
1) What abouttcp_send_active_reset()in TCP/MPTCP? Passing weird drop reasons? There is no drop reason at all. I think people will get confused. So I believe we should invent new definitions to cope with it.
2) What about the .send_reset callback in thesubflow_syn_recv_sock()in MPTCP? The reasons in MPTCP are only definitions (such asMPTCP_RST_EUNSPEC). I don't think we can convert them into the enumskb_drop_reasontype.
Quoted 6 lines
So where should we group those various reasons?
Introducing a new enum is for extension and compatibility for all kinds of reset reasons.
What do you think?
I will stop repeating myself.
enums are not what you think.
type safety is there for a reason.
Can you show me another place in networking stacks where we cast enums to others ?
Quoted 118 lines · Eric Dumazet
Quoted 109 lines · Jason Xing
Quoted 82 lines · Eric Dumazet
Quoted 67 lines · Jason Xing
Quoted 9 lines · Jason Xing
Quoted 5 lines
When I said "If you feel the need to put them in a special group, this is fine by me.",
this was really about partitioning the existing enum into groups, if you prefer having a group of 'RES reasons'Are you suggesting copying what we need from enum skb_drop_reason{} to enum sk_rst_reason{}? Why not reusing them directly. I have no idea what the side effect of cast conversion itself is?
Sorry that I'm writing this email. I'm worried my statement is not that clear, so I write one simple snippet which can help me explain well :)
Allow me give
NO_SOCKETas an example:net/ipv4/icmp.c +1
-0@@ -767,6 +767,7 @@ void __icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info, if (!fl4.saddr) fl4.saddr = htonl(INADDR_DUMMY); +trace_icmp_send (skb_in, type, code);icmp_push_reply (sk, &icmp_param, &fl4, &ipc, &rt); ende:ip_rt_put(rt);net/ipv4/tcp_ipv4.c +3
-1@@ -2160,6 +2160,7 @@ int tcp_v4_rcv(struct sk_buff *skb) {struct net *net =dev_net (skb->dev);enum skb_drop_reason drop_reason; +enum sk_rst_reason rst_reason;int sdif =inet_sdif (skb);int dif =inet_iif (skb);const struct iphdr *iph;@@ -2355,7 +2356,8 @@ int tcp_v4_rcv(struct sk_buff *skb) bad_packet:__TCP_INC_STATS (net,TCP_MIB_INERRS ); }else { -tcp_v4_send_reset (NULL , skb); + rst_reason =RST_REASON_NO_SOCKET ; +tcp_v4_send_reset (NULL , skb, rst_reason); } discard_it:As you can see, we need to add a new '
rst_reason' variable which actually is the same as drop reason. They are the same except for the enum type... Such rst_reasons/drop_reasons are all over the place.Eric, if you have a strong preference, I can do it as you said.
Well, how about explicitly casting them like this based on the current series. It looks better and clearer and more helpful to people who is reading codes to understand:
net/ipv4/tcp_ipv4.c +0
-0net/ipv4/tcp_ipv4.c int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb) · line 1942
Quoted 8 lines
net/ipv4/tcp_ipv4.c +1
-1@@ -1936,7 +1936,7 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -tcp_v4_send_reset (rsk, skb, (u32 )reason); +tcp_v4_send_reset (rsk, skb, (enum sk_rst_reason )reason); discard:kfree_skb_reason (skb, reason);/* Be careful here. If this function gets more complicated and It makes no sense to declare an enum
sk_rst_reasonand then convert it todrop_reason
or vice versa.Quoted 4 lines
Next thing you know, compiler guys will add a new -Woption that will forbid such conversions.
Please add to
tcp_v4_send_reset()anskb_drop_reason, not a new enum.Ah... It looks like I didn't make myself clear again. Sorry... Actually I wrote this part many times. My conclusion is that It's not feasible to do this.
REASONS:
If we__only__need to deal with this passive reset in TCP, it's fine. We pass askb_drop_reasonwhich should also be converted to u32 type intcp_v4_send_reset()as you said, it can work. People who use the trace will see the reason with the 'SKB_DROP_REASON' prefix stripped.But we have to deal with other cases. A few questions are listed here:
1) What abouttcp_send_active_reset()in TCP/MPTCP? Passing weird drop reasons? There is no drop reason at all. I think people will get confused. So I believe we should invent new definitions to cope with it.
2) What about the .send_reset callback in thesubflow_syn_recv_sock()in MPTCP? The reasons in MPTCP are only definitions (such asMPTCP_RST_EUNSPEC). I don't think we can convert them into the enumskb_drop_reasontype.Quoted 6 lines
So where should we group those various reasons?
Introducing a new enum is for extension and compatibility for all kinds of reset reasons.
What do you think?
Quoted 8 lines
I will stop repeating myself.
enums are not what you think.
type safety is there for a reason.
Can you show me another place in networking stacks where we cast enums to others ?
No, I've checked this a month ago.
BTW, I don't know the dangers of casting enum types. I know you will answer it, but I still insist on asking, hoping someone seeing this will help me.
Using skb_drop_reason can only deal with the passive reset in TCP. It is just one of all kinds of reset cases :(
Forgive me, I cannot come up with a good way to cover all the cases TT
I've tried..Sorry...
If other experts see this thread, please help me. I would appreciate it. I have strong interests and feel strong responsibility to implement something like this patch series. It can be very useful!!
Thanks again.
Quoted 124 lines · Jason Xing
Quoted 122 lines · Eric Dumazet
Quoted 109 lines · Jason Xing
Quoted 82 lines · Eric Dumazet
Quoted 67 lines · Jason Xing
Quoted 9 lines · Jason Xing
Quoted 5 lines
When I said "If you feel the need to put them in a special group, this is fine by me.",
this was really about partitioning the existing enum into groups, if you prefer having a group of 'RES reasons'Are you suggesting copying what we need from enum skb_drop_reason{} to enum sk_rst_reason{}? Why not reusing them directly. I have no idea what the side effect of cast conversion itself is?
Sorry that I'm writing this email. I'm worried my statement is not that clear, so I write one simple snippet which can help me explain well :)
Allow me give
NO_SOCKETas an example:net/ipv4/icmp.c +1
-0@@ -767,6 +767,7 @@ void __icmp_send(struct sk_buff *skb_in, int type, int code, __be32 info, if (!fl4.saddr) fl4.saddr = htonl(INADDR_DUMMY); +trace_icmp_send (skb_in, type, code);icmp_push_reply (sk, &icmp_param, &fl4, &ipc, &rt); ende:ip_rt_put(rt);net/ipv4/tcp_ipv4.c +3
-1@@ -2160,6 +2160,7 @@ int tcp_v4_rcv(struct sk_buff *skb) {struct net *net =dev_net (skb->dev);enum skb_drop_reason drop_reason; +enum sk_rst_reason rst_reason;int sdif =inet_sdif (skb);int dif =inet_iif (skb);const struct iphdr *iph;@@ -2355,7 +2356,8 @@ int tcp_v4_rcv(struct sk_buff *skb) bad_packet:__TCP_INC_STATS (net,TCP_MIB_INERRS ); }else { -tcp_v4_send_reset (NULL , skb); + rst_reason =RST_REASON_NO_SOCKET ; +tcp_v4_send_reset (NULL , skb, rst_reason); } discard_it:As you can see, we need to add a new '
rst_reason' variable which actually is the same as drop reason. They are the same except for the enum type... Such rst_reasons/drop_reasons are all over the place.Eric, if you have a strong preference, I can do it as you said.
Well, how about explicitly casting them like this based on the current series. It looks better and clearer and more helpful to people who is reading codes to understand:
net/ipv4/tcp_ipv4.c +0
-0net/ipv4/tcp_ipv4.c int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb) · line 1942
Quoted 8 lines
net/ipv4/tcp_ipv4.c +1
-1@@ -1936,7 +1936,7 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb)return 0 ; reset: -tcp_v4_send_reset (rsk, skb, (u32 )reason); +tcp_v4_send_reset (rsk, skb, (enum sk_rst_reason )reason); discard:kfree_skb_reason (skb, reason);/* Be careful here. If this function gets more complicated and It makes no sense to declare an enum
sk_rst_reasonand then convert it todrop_reason
or vice versa.Quoted 4 lines
Next thing you know, compiler guys will add a new -Woption that will forbid such conversions.
Please add to
tcp_v4_send_reset()anskb_drop_reason, not a new enum.Ah... It looks like I didn't make myself clear again. Sorry... Actually I wrote this part many times. My conclusion is that It's not feasible to do this.
REASONS:
If we__only__need to deal with this passive reset in TCP, it's fine. We pass askb_drop_reasonwhich should also be converted to u32 type intcp_v4_send_reset()as you said, it can work. People who use the trace will see the reason with the 'SKB_DROP_REASON' prefix stripped.But we have to deal with other cases. A few questions are listed here:
1) What abouttcp_send_active_reset()in TCP/MPTCP? Passing weird drop reasons? There is no drop reason at all. I think people will get confused. So I believe we should invent new definitions to cope with it.
2) What about the .send_reset callback in thesubflow_syn_recv_sock()in MPTCP? The reasons in MPTCP are only definitions (such asMPTCP_RST_EUNSPEC). I don't think we can convert them into the enumskb_drop_reasontype.Quoted 6 lines
So where should we group those various reasons?
Introducing a new enum is for extension and compatibility for all kinds of reset reasons.
What do you think?
I will stop repeating myself.
enums are not what you think.
Quoted 8 lines
Quoted 4 lines
type safety is there for a reason.
Can you show me another place in networking stacks where we cast enums to others ?
No, I've checked this a month ago.
BTW, I don't know the dangers of casting enum types. I know you will
s/will/won't/
Quoted 15 lines
answer it, but I still insist on asking, hoping someone seeing this will help me.
Using
skb_drop_reasoncan only deal with the passive reset in TCP. It is just one of all kinds of reset cases :(Forgive me, I cannot come up with a good way to cover all the cases TT
I've tried..Sorry...
If other experts see this thread, please help me. I would appreciate it. I have strong interests and feel strong responsibility to implement something like this patch series. It can be very useful!!
Thanks again.
Steven Rostedt
maintainerQuoted 3 lines · On Fri, 19 Apr 2024 16:00:20 +0800 Jason Xing <[email protected]> wrote:
If other experts see this thread, please help me. I would appreciate it. I have strong interests and feel strong responsibility to implement something like this patch series. It can be very useful!!
I'm not a networking expert, but as I'm Cc'd and this is about tracing, I'll jump in to see if I can help. Honestly, reading the thread, it appears that you and Eric are talking past each other.
I believe Eric is concerned about losing the value of the enum. Enums are types, and if you typecast them to another type, they lose the previous type, and all the safety that goes with it.
Now, I do not really understand the problem trying to be solved here. I understand how TCP works but I never looked into the implementation of MPTCP.
You added this:
+static inline enum sk_rst_reason convert_mptcp_reason(u32 reason) +{
+ return reason += RST_REASON_START;
+}
And used it for places like this:
@@ -309,8 +309,13 @@ static struct dst_entry *subflow_v4_route_req(const struct sock *sk,
return dst;
dst_release(dst);
- if (!req->syncookie)
- tcp_request_sock_ops.send_reset(sk, skb, SK_RST_REASON_NOT_SPECIFIED);
+ if (!req->syncookie) {
+ struct mptcp_ext *mpext = mptcp_get_ext(skb);
+ enum sk_rst_reason reason;
+
+ reason = convert_mptcp_reason(mpext->reset_reason);
+ tcp_request_sock_ops.send_reset(sk, skb, reason);
+ }
return NULL;
}
As I don't know this code or how MPTCP works, I do not understand the above. It use to pass to send_reset() SK_RST_REASON_NOT_SPECIFIED. But now it takes a "reset_reason" calls the "convert_mptcp_reason()" to get back a enum value.
If you are mapping the reset_reason to enum sk_rst_reason, why not do it via a real conversion instead of this fragile arithmetic between the two values?
static inline enum sk_rst_reason convert_mptcp_reason(u32 reason)
{
switch(reason) {
case 0: return SK_RST_REASON_MPTCP_RST_EUNSPEC;
case 1: return SK_RST_REASON_MPTCP_RST_EMPTCP;
case 2: return SK_RST_REASON_MPTCP_RST_ERESOURCE;
[..]
default: return SK_RST_REASON_MAX; // or some other error value
]
}I'm not sure if this is any better, but it's not doing any casting and it's easier to understand. It's a simple mapping between the reason and the enum and there's no inherit dependency between the values. Could possibly create enums for the reason numbers and replace the hard coded values with them.
That way that helper function is at least doing a real conversion of one type to another.
But like I said from the beginning. I don't understand the details here and have not spent the time to dig deeper. I just read the thread and I agree with Eric that the arithmetic conversion of reason to an enum looks fragile at best and buggy at worst.
-- Steve
Hello Steven,
Quoted 8 lines · Steven Rostedt
Quoted 3 lines · On Fri, 19 Apr 2024 16:00:20 +0800 Jason Xing <[email protected]> wrote:
If other experts see this thread, please help me. I would appreciate it. I have strong interests and feel strong responsibility to implement something like this patch series. It can be very useful!!
Quoted 7 lines
I'm not a networking expert, but as I'm Cc'd and this is about tracing, I'll jump in to see if I can help. Honestly, reading the thread, it appears that you and Eric are talking past each other.
I believe Eric is concerned about losing the value of the enum. Enums are types, and if you typecast them to another type, they lose the previous type, and all the safety that goes with it.
Ah, I see. Possible lost value in another enum could cause a problem.
Quoted 40 lines
Now, I do not really understand the problem trying to be solved here. I understand how TCP works but I never looked into the implementation of MPTCP.
You added this:
(quoted diff) +4
-0+static inline enum sk_rst_reason convert_mptcp_reason(u32 reason) +{ + return reason += RST_REASON_START; +}And used it for places like this:
(quoted diff) +7
-2@@ -309,8 +309,13 @@ static struct dst_entry *subflow_v4_route_req(const struct sock *sk, return dst; dst_release(dst); - if (!req->syncookie) - tcp_request_sock_ops.send_reset(sk, skb, SK_RST_REASON_NOT_SPECIFIED); + if (!req->syncookie) { + struct mptcp_ext *mpext = mptcp_get_ext(skb); + enum sk_rst_reason reason; + + reason = convert_mptcp_reason(mpext->reset_reason); + tcp_request_sock_ops.send_reset(sk, skb, reason); + } return NULL; }As I don't know this code or how MPTCP works, I do not understand the above. It use to pass to
send_reset()SK_RST_REASON_NOT_SPECIFIED. But now it takes a "reset_reason" calls the "convert_mptcp_reason()" to get back a enum value.If you are mapping the
reset_reasonto enumsk_rst_reason, why not do it via a real conversion instead of this fragile arithmetic between the two values?static inline enum
sk_rst_reasonconvert_mptcp_reason(u32 reason)
Quoted 9 lines
{
switch(reason) {
case 0: return SK_RST_REASON_MPTCP_RST_EUNSPEC;
case 1: return SK_RST_REASON_MPTCP_RST_EMPTCP;
case 2: return SK_RST_REASON_MPTCP_RST_ERESOURCE;
[..]
default: return SK_RST_REASON_MAX; // or some other error value
]
}This code snippet looks much better than mine.
Quoted 6 lines
I'm not sure if this is any better, but it's not doing any casting and it's easier to understand. It's a simple mapping between the reason and the enum and there's no inherit dependency between the values. Could possibly create enums for the reason numbers and replace the hard coded values with them.
Right.
I also need to handle many drop reasons cases like what you do. Due to too many of them, I will try the key-value map instead of switch-case and then see if it works.
Quoted 8 lines
That way that helper function is at least doing a real conversion of one type to another.
But like I said from the beginning. I don't understand the details here and have not spent the time to dig deeper. I just read the thread and I agree with Eric that the arithmetic conversion of reason to an enum looks fragile at best and buggy at worst.
Thanks so much for your help, which I didn't even imagine.
Sure, after one night of investigation, I figured it out. I will drop enum casts without any doubt as Eric and you suggested. But I believe a new enum is needed, grouping various reasons together which help ftrace print the valid string to userspace.
Thanks,
Jason
Quoted 2 lines
-- Steve