Skip to contentSkip to conversation

TCP Extra-option framework for TCP-MD5 and SMC

PATCH Superseded Christoph Paasch 2 participants 56 messages Nov 28, 2017 – Nov 30, 2017

Subsystems (6)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Christoph Paasch

author
#0 lore raw

v2 cover TCP Extra-option framework for TCP-MD5 and SMC

Finally, the second version of this patch-set.

At netdev I talked about this with Eric, and he is looking forward to see TCP-MD5 get out of the hot TCP data-path. He was planning to avoid TCP-MD5 with static jump-labels and also move TCP-MD5 in a separate file.

There are quite a few changes in this new set. I extended much more the extra-option framework. Making the list on a per-socket basis in the request-socket, tcp-socket and time-wait-socket. This eliminates the need for RCU-locking. See patch "tcp_extra_options: Make extra-option list per-socket"

I also added callbacks for response_write/prepare that can be called from the tcp_v4_send_ack/rst, and v6 counterparts. This is needed because in these code-paths the skb and sk have entirely different semantics.

Finally, in the input-path I added a check-callback and a post-process callback.

This is needed because after parsing the option in tcp_parse_options() we also need to act on them. The actions can be either before any other checks (e.g., as is the case for TCP_MD5, because we want to drop segments) or after the validity checks (e.g., as is needed for SMC).

And I also opted SMC into the new framework.

And, I made the TCP-MD5 input code-path use the framework.

There are still a few things that I need to double-check, notably sk_nocaps_add() in TCP_MD5.

And, I need to test ;-)

But I would like to have some early feedback already - especially on the patches prefixed with "tcp_extra_options". If they look good, I can merge them into Mat's patch, which will simplify my patch-juggling.

I want to get this patch-set out to netdev as an RFC as quickly as possible, so that we can make some progress.

Christoph Paasch (24):
tcp: Write options after the header has been fully done
tcp: Pass sock to tcp_options_write instead of tcp_sock
tcp: Pass skb in tcp_options_write
tcp: Allow tcp_fast_parse_options to drop segments
tcp: Make smc_parse_options return 1 on success
tcp_md5: Don't pass along md5-key
tcp_md5: Detect key inside tcp_v4_send_ack instead of passing it as an
argument
tcp_md5: Detect key inside tcp_v6_send_response instead of passing it
as an argument
tcp_extra_options: Extend tcp_extra_options_write
tcp_extra_options: Make extra-option list per-socket
tcp_extra_options: Export module fixes
tcp_extra_options: Add response_prepare callback
tcp_extra_options: Check static branch before _parse
tcp_extra_options: Skip fast-path when extra-options are present
tcp_extra_options: Pass sk to tcp_parse_options instead of tp
tcp_extra_options: Add check-callback right after parsing the option
tcp_extra_options: Allow to parse experimental options
tcp_extra_options: Add post_process callback
tcp_smc: Make SMC use TCP extra-option framework
tcp_md5: Check for TCP_MD5 after TCP Timestamps in
tcp_established_options
tcp_md5: Move TCP-MD5 code out of TCP itself
tcp_md5: Use tcp_extra_options in output path
tcp_md5: Cleanup TCP-code
tcp_md5: Use TCP extra-options on the input path

Mat Martineau (1):
tcp: Register handlers for extra TCP options

 drivers/infiniband/hw/cxgb4/cm.c |    2 +-
 include/linux/inet_diag.h        |    1 +
 include/linux/tcp.h              |   41 +-
 include/linux/tcp_md5.h          |   38 ++
 include/net/inet_sock.h          |    3 +-
 include/net/tcp.h                |  216 ++++---
 net/ipv4/Makefile                |    1 +
 net/ipv4/syncookies.c            |    6 +-
 net/ipv4/tcp.c                   |  372 ++++++++---
 net/ipv4/tcp_diag.c              |   81 +--
 net/ipv4/tcp_input.c             |  134 ++--
 net/ipv4/tcp_ipv4.c              |  553 ++--------------
 net/ipv4/tcp_md5.c               | 1317 ++++++++++++++++++++++++++++++++++++++
 net/ipv4/tcp_minisocks.c         |   76 +--
 net/ipv4/tcp_output.c            |  183 +-----
 net/ipv6/syncookies.c            |    6 +-
 net/ipv6/tcp_ipv6.c              |  386 ++---------
 net/smc/af_smc.c                 |  170 ++++-
 18 files changed, 2148 insertions(+), 1438 deletions(-)
 create mode 100644 include/linux/tcp_md5.h
 create mode 100644 net/ipv4/tcp_md5.c
Signature · 1 line
2.15.0
author
#1 lore raw

PATCH v2 1/25 tcp: Write options after the header has been fully done

This is needed later on for TCP_MD5, as we need the complete TCP-header to compute the TCP_MD5-hash.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv4/tcp_output.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
net/ipv4/tcp_output.c +2 -2
index a4d214c7b506..e0335be600d3 100644
@@ -1126,7 +1126,6 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 		}
 	}
 
-	tcp_options_write((__be32 *)(th + 1), tp, &opts);
 	skb_shinfo(skb)->gso_type = sk->sk_gso_type;
 	if (likely(!(tcb->tcp_flags & TCPHDR_SYN))) {
 		th->window      = htons(tcp_select_window(sk));
@@ -1137,6 +1136,7 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 		 */
 		th->window	= htons(min(tp->rcv_wnd, 65535U));
 	}
+	tcp_options_write((__be32 *)(th + 1), tp, &opts);
 #ifdef CONFIG_TCP_MD5SIG
 	/* Calculate the MD5 hash, as we have all we need now */
 	if (md5) {
@@ -3245,8 +3245,8 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 
 	/* RFC1323: The window in SYN & SYN/ACK segments is never scaled. */
 	th->window = htons(min(req->rsk_rcv_wnd, 65535U));
-	tcp_options_write((__be32 *)(th + 1), NULL, &opts);
 	th->doff = (tcp_header_size >> 2);
+	tcp_options_write((__be32 *)(th + 1), NULL, &opts);
 	__TCP_INC_STATS(sock_net(sk), TCP_MIB_OUTSEGS);
 
 #ifdef CONFIG_TCP_MD5SIG
Signature · 1 line
2.15.0
author
#2 lore raw

PATCH v2 2/25 tcp: Pass sock to tcp_options_write instead of tcp_sock

In reply to Christoph Paasch

Because, to write the extra-options, we need also the request-socket to properly write the correct info.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv4/tcp_output.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
net/ipv4/tcp_output.c +7 -3
index e0335be600d3..6937098fb2b5 100644
@@ -444,10 +444,14 @@ struct tcp_out_options {
  * At least SACK_PERM as the first option is known to lead to a disaster
  * (but it may well be that other scenarios fail similarly).
  */
-static void tcp_options_write(__be32 *ptr, struct tcp_sock *tp,
+static void tcp_options_write(__be32 *ptr, struct sock *sk,
 			      struct tcp_out_options *opts)
 {
 	u16 options = opts->options;	/* mungable copy */
+	struct tcp_sock *tp = NULL;
+
+	if (sk_fullsock(sk))
+		tp = tcp_sk(sk);
 
 	if (unlikely(OPTION_MD5 & options)) {
 		*ptr++ = htonl((TCPOPT_NOP << 24) | (TCPOPT_NOP << 16) |
@@ -1136,7 +1140,7 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 		 */
 		th->window	= htons(min(tp->rcv_wnd, 65535U));
 	}
-	tcp_options_write((__be32 *)(th + 1), tp, &opts);
+	tcp_options_write((__be32 *)(th + 1), sk, &opts);
 #ifdef CONFIG_TCP_MD5SIG
 	/* Calculate the MD5 hash, as we have all we need now */
 	if (md5) {
@@ -3246,7 +3250,7 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 	/* RFC1323: The window in SYN & SYN/ACK segments is never scaled. */
 	th->window = htons(min(req->rsk_rcv_wnd, 65535U));
 	th->doff = (tcp_header_size >> 2);
-	tcp_options_write((__be32 *)(th + 1), NULL, &opts);
+	tcp_options_write((__be32 *)(th + 1), req_to_sk(req), &opts);
 	__TCP_INC_STATS(sock_net(sk), TCP_MIB_OUTSEGS);
 
 #ifdef CONFIG_TCP_MD5SIG
Signature · 1 line
2.15.0
author
#3 lore raw

PATCH v2 3/25 tcp: Pass skb in tcp_options_write

In reply to Christoph Paasch

We will need it to allow TCP_MD5 to compute the hash.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv4/tcp_output.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
net/ipv4/tcp_output.c +3 -3
index 6937098fb2b5..751396f22bb3 100644
@@ -444,7 +444,7 @@ struct tcp_out_options {
  * At least SACK_PERM as the first option is known to lead to a disaster
  * (but it may well be that other scenarios fail similarly).
  */
-static void tcp_options_write(__be32 *ptr, struct sock *sk,
+static void tcp_options_write(__be32 *ptr, struct sk_buff *skb, struct sock *sk,
 			      struct tcp_out_options *opts)
 {
 	u16 options = opts->options;	/* mungable copy */
@@ -1140,7 +1140,7 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 		 */
 		th->window	= htons(min(tp->rcv_wnd, 65535U));
 	}
-	tcp_options_write((__be32 *)(th + 1), sk, &opts);
+	tcp_options_write((__be32 *)(th + 1), skb, sk, &opts);
 #ifdef CONFIG_TCP_MD5SIG
 	/* Calculate the MD5 hash, as we have all we need now */
 	if (md5) {
@@ -3250,7 +3250,7 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 	/* RFC1323: The window in SYN & SYN/ACK segments is never scaled. */
 	th->window = htons(min(req->rsk_rcv_wnd, 65535U));
 	th->doff = (tcp_header_size >> 2);
-	tcp_options_write((__be32 *)(th + 1), req_to_sk(req), &opts);
+	tcp_options_write((__be32 *)(th + 1), skb, req_to_sk(req), &opts);
 	__TCP_INC_STATS(sock_net(sk), TCP_MIB_OUTSEGS);
 
 #ifdef CONFIG_TCP_MD5SIG
Signature · 1 line
2.15.0
author
#4 lore raw

PATCH v2 4/25 tcp: Allow tcp_fast_parse_options to drop segments

In reply to Christoph Paasch

The TCP extra-options parser might decide to drop the segment based on the present option. Especially, for TCP_MD5, we want to do this as soon as possible (thus, right after the parsing).

This patch allows to do this by splitting out tcp_fast_parse_options() in tcp_validate_incoming()

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv4/tcp_input.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)
net/ipv4/tcp_input.c +11 -5
index 734cfc8ff76e..a97f54005452 100644
@@ -3824,6 +3824,8 @@ static bool tcp_parse_aligned_timestamp(struct tcp_sock *tp, const struct tcphdr
 
 /* Fast parse options. This hopes to only see timestamps.
  * If it is wrong it falls back on tcp_parse_options().
+ *
+ * Returns true if we should drop this packet based on present TCP-options.
  */
 static bool tcp_fast_parse_options(const struct net *net,
 				   const struct sk_buff *skb,
@@ -3834,18 +3836,20 @@ static bool tcp_fast_parse_options(const struct net *net,
 	 */
 	if (th->doff == (sizeof(*th) / 4)) {
 		tp->rx_opt.saw_tstamp = 0;
-		return false;
+		goto extra_opt_check;
 	} else if (tp->rx_opt.tstamp_ok &&
 		   th->doff == ((sizeof(*th) + TCPOLEN_TSTAMP_ALIGNED) / 4)) {
 		if (tcp_parse_aligned_timestamp(tp, th))
-			return true;
+			goto extra_opt_check;
 	}
 
 	tcp_parse_options(net, skb, &tp->rx_opt, 1, NULL);
 	if (tp->rx_opt.saw_tstamp && tp->rx_opt.rcv_tsecr)
 		tp->rx_opt.rcv_tsecr -= tp->tsoffset;
 
-	return true;
+extra_opt_check:
+	/* ToDo - will be added here */
+	return false;
 }
 
 #ifdef CONFIG_TCP_MD5SIG
@@ -5165,9 +5169,11 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
 	struct tcp_sock *tp = tcp_sk(sk);
 	bool rst_seq_match = false;
 
+	if (tcp_fast_parse_options(sock_net(sk), skb, th, tp))
+		goto discard;
+
 	/* RFC1323: H1. Apply PAWS check first. */
-	if (tcp_fast_parse_options(sock_net(sk), skb, th, tp) &&
-	    tp->rx_opt.saw_tstamp &&
+	if (tp->rx_opt.saw_tstamp &&
 	    tcp_paws_discard(sk, skb)) {
 		if (!th->rst) {
 			NET_INC_STATS(sock_net(sk), LINUX_MIB_PAWSESTABREJECTED);
Signature · 1 line
2.15.0
#5 lore raw

Re: [MPTCP] [PATCH v2 04/25] tcp: Allow tcp_fast_parse_options to drop segments

Quoted 6 lines · Christoph Paasch

The TCP extra-options parser might decide to drop the segment based on the present option. Especially, for TCP_MD5, we want to do this as soon as possible (thus, right after the parsing).

This patch allows to do this by splitting out tcp_fast_parse_options() in tcp_validate_incoming()

I suggest some additional explanation of how the return value of tcp_parse_fast_options() has changed meaning. The change to the control flow in tcp_validate_incoming() doesn't have quite enough context in the email to see what's going on.

Quoted 61 lines
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_input.c | 16 +++++++++++-----

1 file changed, 11 insertions(+), 5 deletions(-)
net/ipv4/tcp_input.c +11 -5
index 734cfc8ff76e..a97f54005452 100644
@@ -3824,6 +3824,8 @@ static bool tcp_parse_aligned_timestamp(struct tcp_sock *tp, const struct tcphdr
 
 /* Fast parse options. This hopes to only see timestamps.
 * If it is wrong it falls back on tcp_parse_options().
+ *
+ * Returns true if we should drop this packet based on present TCP-options.
 */
 static bool tcp_fast_parse_options(const struct net *net,
 				   const struct sk_buff *skb,
@@ -3834,18 +3836,20 @@ static bool tcp_fast_parse_options(const struct net *net,
 	 */
 	if (th->doff == (sizeof(*th) / 4)) {
 		tp->rx_opt.saw_tstamp = 0;
-		return false;
+		goto extra_opt_check;
 	} else if (tp->rx_opt.tstamp_ok &&
 		   th->doff == ((sizeof(*th) + TCPOLEN_TSTAMP_ALIGNED) / 4)) {
 		if (tcp_parse_aligned_timestamp(tp, th))
-			return true;
+			goto extra_opt_check;
 	}
 
 	tcp_parse_options(net, skb, &tp->rx_opt, 1, NULL);
 	if (tp->rx_opt.saw_tstamp && tp->rx_opt.rcv_tsecr)
 		tp->rx_opt.rcv_tsecr -= tp->tsoffset;
 
-	return true;
+extra_opt_check:
+	/* ToDo - will be added here */
+	return false;
 }
 
 #ifdef CONFIG_TCP_MD5SIG
@@ -5165,9 +5169,11 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
 	struct tcp_sock *tp = tcp_sk(sk);
 	bool rst_seq_match = false;
 
+	if (tcp_fast_parse_options(sock_net(sk), skb, th, tp))
+		goto discard;
+
 	/* RFC1323: H1. Apply PAWS check first. */
-	if (tcp_fast_parse_options(sock_net(sk), skb, th, tp) &&
-	    tp->rx_opt.saw_tstamp &&
+	if (tp->rx_opt.saw_tstamp &&
 	    tcp_paws_discard(sk, skb)) {
 		if (!th->rst) {
 			NET_INC_STATS(sock_net(sk), LINUX_MIB_PAWSESTABREJECTED);
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#6 lore raw

Re: [MPTCP] [PATCH v2 04/25] tcp: Allow tcp_fast_parse_options to drop segments

Quoted 6 lines · Mat Martineau
Quoted 4 lines · Christoph Paasch

The TCP extra-options parser might decide to drop the segment based on the present option. Especially, for TCP_MD5, we want to do this as soon as possible (thus, right after the parsing).

Quoted 7 lines
Quoted 2 lines

This patch allows to do this by splitting out tcp_fast_parse_options() in tcp_validate_incoming()

I suggest some additional explanation of how the return value of tcp_parse_fast_options() has changed meaning. The change to the control flow in tcp_validate_incoming() doesn't have quite enough context in the email to see what's going on.

I made it:

tcp: Allow tcp_fast_parse_options to drop segments

The TCP extra-options parser might decide to drop the segment based on the present option. Especially, for TCP_MD5, we want to do this as soon as possible (thus, right after the parsing).

This patch allows to do this by splitting out tcp_fast_parse_options() in tcp_validate_incoming().

Originally, tcp_fast_parse_options() returned false, when there is no timestamp option (except in the case of the slow-path processing through tcp_parse_options() where it always returns true.

So, the return-value of tcp_fast_parse_options() was kind of inconsistent. With this patch, we make it return true when the segment should get dropped based on the parsed options, and false otherwise.

In tcp_validate_incoming, we will then just check for tp->rx_opt.saw_tstamp to see if we should verify PAWS.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

Christoph

Quoted 66 lines
Quoted 62 lines
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_input.c | 16 +++++++++++-----

1 file changed, 11 insertions(+), 5 deletions(-)
net/ipv4/tcp_input.c +11 -5
index 734cfc8ff76e..a97f54005452 100644
@@ -3824,6 +3824,8 @@ static bool tcp_parse_aligned_timestamp(struct tcp_sock *tp, const struct tcphdr
 
 /* Fast parse options. This hopes to only see timestamps.
 * If it is wrong it falls back on tcp_parse_options().
+ *
+ * Returns true if we should drop this packet based on present TCP-options.
 */
 static bool tcp_fast_parse_options(const struct net *net,
 				   const struct sk_buff *skb,
@@ -3834,18 +3836,20 @@ static bool tcp_fast_parse_options(const struct net *net,
 	 */
 	if (th->doff == (sizeof(*th) / 4)) {
 		tp->rx_opt.saw_tstamp = 0;
-		return false;
+		goto extra_opt_check;
 	} else if (tp->rx_opt.tstamp_ok &&
 		   th->doff == ((sizeof(*th) + TCPOLEN_TSTAMP_ALIGNED) / 4)) {
 		if (tcp_parse_aligned_timestamp(tp, th))
-			return true;
+			goto extra_opt_check;
 	}
 
 	tcp_parse_options(net, skb, &tp->rx_opt, 1, NULL);
 	if (tp->rx_opt.saw_tstamp && tp->rx_opt.rcv_tsecr)
 		tp->rx_opt.rcv_tsecr -= tp->tsoffset;
 
-	return true;
+extra_opt_check:
+	/* ToDo - will be added here */
+	return false;
 }
 
 #ifdef CONFIG_TCP_MD5SIG
@@ -5165,9 +5169,11 @@ static bool tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
 	struct tcp_sock *tp = tcp_sk(sk);
 	bool rst_seq_match = false;
 
+	if (tcp_fast_parse_options(sock_net(sk), skb, th, tp))
+		goto discard;
+
 	/* RFC1323: H1. Apply PAWS check first. */
-	if (tcp_fast_parse_options(sock_net(sk), skb, th, tp) &&
-	    tp->rx_opt.saw_tstamp &&
+	if (tp->rx_opt.saw_tstamp &&
 	    tcp_paws_discard(sk, skb)) {
 		if (!th->rst) {
 			NET_INC_STATS(sock_net(sk), LINUX_MIB_PAWSESTABREJECTED);
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#7 lore raw

PATCH v2 5/25 tcp: Make smc_parse_options return 1 on success

In reply to Christoph Paasch

Will be needed to allow extra_options to parse experimental TCP options.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv4/tcp_input.c | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)
net/ipv4/tcp_input.c +11 -8
index a97f54005452..843061955973 100644
@@ -3668,19 +3668,22 @@ static void tcp_parse_fastopen_option(int len, const unsigned char *cookie,
 	foc->exp = exp_opt;
 }
 
-static void smc_parse_options(const struct tcphdr *th,
-			      struct tcp_options_received *opt_rx,
-			      const unsigned char *ptr,
-			      int opsize)
+static int smc_parse_options(const struct tcphdr *th,
+			     struct tcp_options_received *opt_rx,
+			     const unsigned char *ptr,
+			     int opsize)
 {
 #if IS_ENABLED(CONFIG_SMC)
 	if (static_branch_unlikely(&tcp_have_smc)) {
 		if (th->syn && !(opsize & 1) &&
 		    opsize >= TCPOLEN_EXP_SMC_BASE &&
-		    get_unaligned_be32(ptr) == TCPOPT_SMC_MAGIC)
+		    get_unaligned_be32(ptr) == TCPOPT_SMC_MAGIC) {
 			opt_rx->smc_ok = 1;
+			return 1;
+		}
 	}
 #endif
+	return 0;
 }
 
 /* Look for tcp options. Normally only called on SYN and SYNACK packets.
@@ -3790,9 +3793,9 @@ void tcp_parse_options(const struct net *net,
 					tcp_parse_fastopen_option(opsize -
 						TCPOLEN_EXP_FASTOPEN_BASE,
 						ptr + 2, th->syn, foc, true);
-				else
-					smc_parse_options(th, opt_rx, ptr,
-							  opsize);
+				else if (smc_parse_options(th, opt_rx, ptr,
+							   opsize))
+					break;
 				break;
 
 			}
Signature · 1 line
2.15.0
author
#8 lore raw

PATCH v2 6/25 tcp_md5: Don't pass along md5-key

In reply to Christoph Paasch

It is much cleaner to store the key-pointer in tcp_out_options.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv4/tcp_output.c | 47 +++++++++++++++++++++--------------------------
 1 file changed, 21 insertions(+), 26 deletions(-)
net/ipv4/tcp_output.c +21 -26
index 751396f22bb3..fedaaea83b43 100644
@@ -429,6 +429,7 @@ struct tcp_out_options {
 	__u8 *hash_location;	/* temporary pointer, overloaded */
 	__u32 tsval, tsecr;	/* need to include OPTION_TS */
 	struct tcp_fastopen_cookie *fastopen_cookie;	/* Fast open cookie */
+	struct tcp_md5sig_key *md5; /* TCP_MD5 signature key */
 };
 
 /* Write previously computed TCP options to the packet.
@@ -582,21 +583,20 @@ static void smc_set_option_cond(const struct tcp_sock *tp,
  * network wire format yet.
  */
 static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb,
-				struct tcp_out_options *opts,
-				struct tcp_md5sig_key **md5)
+				struct tcp_out_options *opts)
 {
 	struct tcp_sock *tp = tcp_sk(sk);
 	unsigned int remaining = MAX_TCP_OPTION_SPACE;
 	struct tcp_fastopen_request *fastopen = tp->fastopen_req;
 
 #ifdef CONFIG_TCP_MD5SIG
-	*md5 = tp->af_specific->md5_lookup(sk, sk);
-	if (*md5) {
+	opts->md5 = tp->af_specific->md5_lookup(sk, sk);
+	if (opts->md5) {
 		opts->options |= OPTION_MD5;
 		remaining -= TCPOLEN_MD5SIG_ALIGNED;
 	}
 #else
-	*md5 = NULL;
+	opts->md5 = NULL;
 #endif
 
 	/* We always get an MSS option.  The option bytes which will be seen in
@@ -611,7 +611,7 @@ static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb,
 	opts->mss = tcp_advertise_mss(sk);
 	remaining -= TCPOLEN_MSS_ALIGNED;
 
-	if (likely(sock_net(sk)->ipv4.sysctl_tcp_timestamps && !*md5)) {
+	if (likely(sock_net(sk)->ipv4.sysctl_tcp_timestamps && !opts->md5)) {
 		opts->options |= OPTION_TS;
 		opts->tsval = tcp_skb_timestamp(skb) + tp->tsoffset;
 		opts->tsecr = tp->rx_opt.ts_recent;
@@ -653,14 +653,13 @@ static unsigned int tcp_synack_options(const struct sock *sk,
 				       struct request_sock *req,
 				       unsigned int mss, struct sk_buff *skb,
 				       struct tcp_out_options *opts,
-				       const struct tcp_md5sig_key *md5,
 				       struct tcp_fastopen_cookie *foc)
 {
 	struct inet_request_sock *ireq = inet_rsk(req);
 	unsigned int remaining = MAX_TCP_OPTION_SPACE;
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (md5) {
+	if (opts->md5) {
 		opts->options |= OPTION_MD5;
 		remaining -= TCPOLEN_MD5SIG_ALIGNED;
 
@@ -715,8 +714,7 @@ static unsigned int tcp_synack_options(const struct sock *sk,
  * final wire format yet.
  */
 static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb,
-					struct tcp_out_options *opts,
-					struct tcp_md5sig_key **md5)
+					struct tcp_out_options *opts)
 {
 	struct tcp_sock *tp = tcp_sk(sk);
 	unsigned int size = 0;
@@ -725,13 +723,13 @@ static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb
 	opts->options = 0;
 
 #ifdef CONFIG_TCP_MD5SIG
-	*md5 = tp->af_specific->md5_lookup(sk, sk);
-	if (unlikely(*md5)) {
+	opts->md5 = tp->af_specific->md5_lookup(sk, sk);
+	if (unlikely(opts->md5)) {
 		opts->options |= OPTION_MD5;
 		size += TCPOLEN_MD5SIG_ALIGNED;
 	}
 #else
-	*md5 = NULL;
+	opts->md5 = NULL;
 #endif
 
 	if (likely(tp->rx_opt.tstamp_ok)) {
@@ -1045,7 +1043,6 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	struct tcp_out_options opts;
 	unsigned int tcp_options_size, tcp_header_size;
 	struct sk_buff *oskb = NULL;
-	struct tcp_md5sig_key *md5;
 	struct tcphdr *th;
 	int err;
 
@@ -1074,10 +1071,9 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	memset(&opts, 0, sizeof(opts));
 
 	if (unlikely(tcb->tcp_flags & TCPHDR_SYN))
-		tcp_options_size = tcp_syn_options(sk, skb, &opts, &md5);
+		tcp_options_size = tcp_syn_options(sk, skb, &opts);
 	else
-		tcp_options_size = tcp_established_options(sk, skb, &opts,
-							   &md5);
+		tcp_options_size = tcp_established_options(sk, skb, &opts);
 	tcp_header_size = tcp_options_size + sizeof(struct tcphdr);
 
 	/* if no packet is in qdisc/device queue, then allow XPS to select
@@ -1143,10 +1139,10 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	tcp_options_write((__be32 *)(th + 1), skb, sk, &opts);
 #ifdef CONFIG_TCP_MD5SIG
 	/* Calculate the MD5 hash, as we have all we need now */
-	if (md5) {
+	if (opts.md5) {
 		sk_nocaps_add(sk, NETIF_F_GSO_MASK);
 		tp->af_specific->calc_md5_hash(opts.hash_location,
-					       md5, sk, skb);
+					       opts.md5, sk, skb);
 	}
 #endif
 
@@ -1590,7 +1586,6 @@ unsigned int tcp_current_mss(struct sock *sk)
 	u32 mss_now;
 	unsigned int header_len;
 	struct tcp_out_options opts;
-	struct tcp_md5sig_key *md5;
 
 	mss_now = tp->mss_cache;
 
@@ -1600,7 +1595,7 @@ unsigned int tcp_current_mss(struct sock *sk)
 			mss_now = tcp_sync_mss(sk, mtu);
 	}
 
-	header_len = tcp_established_options(sk, NULL, &opts, &md5) +
+	header_len = tcp_established_options(sk, NULL, &opts) +
 		     sizeof(struct tcphdr);
 	/* The mss_cache is sized based on tp->tcp_header_len, which assumes
 	 * some common options. If this is an odd packet (because we have SACK
@@ -3179,7 +3174,6 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 {
 	struct inet_request_sock *ireq = inet_rsk(req);
 	const struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_key *md5 = NULL;
 	struct tcp_out_options opts;
 	struct sk_buff *skb;
 	int tcp_header_size;
@@ -3225,10 +3219,10 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 
 #ifdef CONFIG_TCP_MD5SIG
 	rcu_read_lock();
-	md5 = tcp_rsk(req)->af_specific->req_md5_lookup(sk, req_to_sk(req));
+	opts.md5 = tcp_rsk(req)->af_specific->req_md5_lookup(sk, req_to_sk(req));
 #endif
 	skb_set_hash(skb, tcp_rsk(req)->txhash, PKT_HASH_TYPE_L4);
-	tcp_header_size = tcp_synack_options(sk, req, mss, skb, &opts, md5,
+	tcp_header_size = tcp_synack_options(sk, req, mss, skb, &opts,
 					     foc) + sizeof(*th);
 
 	skb_push(skb, tcp_header_size);
@@ -3255,9 +3249,10 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 
 #ifdef CONFIG_TCP_MD5SIG
 	/* Okay, we have all we need - do the md5 hash if needed */
-	if (md5)
+	if (opts.md5)
 		tcp_rsk(req)->af_specific->calc_md5_hash(opts.hash_location,
-					       md5, req_to_sk(req), skb);
+							 opts.md5,
+							 req_to_sk(req), skb);
 	rcu_read_unlock();
 #endif
 
Signature · 1 line
2.15.0
#9 lore raw

Re: [MPTCP] [PATCH v2 06/25] tcp_md5: Don't pass along md5-key

Quoted 12 lines · Christoph Paasch

It is much cleaner to store the key-pointer in tcp_out_options.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_output.c | 47 +++++++++++++++++++++--------------------------

1 file changed, 21 insertions(+), 26 deletions(-)
net/ipv4/tcp_output.c +0 -0
index 751396f22bb3..fedaaea83b43 100644

net/ipv4/tcp_output.c struct tcp_out_options { · line 432

Quoted 4 lines
net/ipv4/tcp_output.c +1 -0
@@ -429,6 +429,7 @@ struct tcp_out_options {
 	__u8 *hash_location;	/* temporary pointer, overloaded */
 	__u32 tsval, tsecr;	/* need to include OPTION_TS */
 	struct tcp_fastopen_cookie *fastopen_cookie;	/* Fast open cookie */
+	struct tcp_md5sig_key *md5; /* TCP_MD5 signature key */

The md5 member can be wrapped in an #ifdef CONFIG_TCP_MD5SIG (and propagated to tcp.h when the struct gets moved), with some tweaks below.

Quoted 17 lines
net/ipv4/tcp_output.c +1 -3
 };
 
 /* Write previously computed TCP options to the packet.
@@ -582,21 +583,20 @@ static void smc_set_option_cond(const struct tcp_sock *tp,
 * network wire format yet.
 */
 static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb,
-				struct tcp_out_options *opts,
-				struct tcp_md5sig_key **md5)
+				struct tcp_out_options *opts)
 {
 	struct tcp_sock *tp = tcp_sk(sk);
 	unsigned int remaining = MAX_TCP_OPTION_SPACE;
 	struct tcp_fastopen_request *fastopen = tp->fastopen_req;
 
 #ifdef CONFIG_TCP_MD5SIG
-	*md5 = tp->af_specific->md5_lookup(sk, sk);

net/ipv4/tcp_output.c static void smc_set_option_cond(const struct tcp_sock *tp, · line 600

Quoted 10 lines
net/ipv4/tcp_output.c +3 -2
-	if (*md5) {
+	opts->md5 = tp->af_specific->md5_lookup(sk, sk);
+	if (opts->md5) {
 		opts->options |= OPTION_MD5;
 		remaining -= TCPOLEN_MD5SIG_ALIGNED;
 	}
 #else
-	*md5 = NULL;
+	opts->md5 = NULL;
 #endif

Since *opts is zeroed out, the #else block might not be needed. This would allow the struct member to be configured out too. While this is an intermediate step in the refactoring process, it is helpful to have opts.md5 #ifdef'd from the start.

Quoted 42 lines
net/ipv4/tcp_output.c +3 -6
 	/* We always get an MSS option.  The option bytes which will be seen in
@@ -611,7 +611,7 @@ static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb,
 	opts->mss = tcp_advertise_mss(sk);
 	remaining -= TCPOLEN_MSS_ALIGNED;
 
-	if (likely(sock_net(sk)->ipv4.sysctl_tcp_timestamps && !*md5)) {
+	if (likely(sock_net(sk)->ipv4.sysctl_tcp_timestamps && !opts->md5)) {
 		opts->options |= OPTION_TS;
 		opts->tsval = tcp_skb_timestamp(skb) + tp->tsoffset;
 		opts->tsecr = tp->rx_opt.ts_recent;
@@ -653,14 +653,13 @@ static unsigned int tcp_synack_options(const struct sock *sk,
 				       struct request_sock *req,
 				       unsigned int mss, struct sk_buff *skb,
 				       struct tcp_out_options *opts,
-				       const struct tcp_md5sig_key *md5,
 				       struct tcp_fastopen_cookie *foc)
 {
 	struct inet_request_sock *ireq = inet_rsk(req);
 	unsigned int remaining = MAX_TCP_OPTION_SPACE;
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (md5) {
+	if (opts->md5) {
 		opts->options |= OPTION_MD5;
 		remaining -= TCPOLEN_MD5SIG_ALIGNED;
 
@@ -715,8 +714,7 @@ static unsigned int tcp_synack_options(const struct sock *sk,
 * final wire format yet.
 */
 static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb,
-					struct tcp_out_options *opts,
-					struct tcp_md5sig_key **md5)
+					struct tcp_out_options *opts)
 {
 	struct tcp_sock *tp = tcp_sk(sk);
 	unsigned int size = 0;
@@ -725,13 +723,13 @@ static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb
 	opts->options = 0;
 
 #ifdef CONFIG_TCP_MD5SIG
-	*md5 = tp->af_specific->md5_lookup(sk, sk);

net/ipv4/tcp_output.c static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb · line 733

Quoted 10 lines
net/ipv4/tcp_output.c +3 -2
-	if (unlikely(*md5)) {
+	opts->md5 = tp->af_specific->md5_lookup(sk, sk);
+	if (unlikely(opts->md5)) {
 		opts->options |= OPTION_MD5;
 		size += TCPOLEN_MD5SIG_ALIGNED;
 	}
 #else
-	*md5 = NULL;
+	opts->md5 = NULL;
 #endif

Same as previous comment.

Quoted 91 lines
net/ipv4/tcp_output.c +10 -13
 	if (likely(tp->rx_opt.tstamp_ok)) {
@@ -1045,7 +1043,6 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	struct tcp_out_options opts;
 	unsigned int tcp_options_size, tcp_header_size;
 	struct sk_buff *oskb = NULL;
-	struct tcp_md5sig_key *md5;
 	struct tcphdr *th;
 	int err;
 
@@ -1074,10 +1071,9 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	memset(&opts, 0, sizeof(opts));
 
 	if (unlikely(tcb->tcp_flags & TCPHDR_SYN))
-		tcp_options_size = tcp_syn_options(sk, skb, &opts, &md5);
+		tcp_options_size = tcp_syn_options(sk, skb, &opts);
 	else
-		tcp_options_size = tcp_established_options(sk, skb, &opts,
-							   &md5);
+		tcp_options_size = tcp_established_options(sk, skb, &opts);
 	tcp_header_size = tcp_options_size + sizeof(struct tcphdr);
 
 	/* if no packet is in qdisc/device queue, then allow XPS to select
@@ -1143,10 +1139,10 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	tcp_options_write((__be32 *)(th + 1), skb, sk, &opts);
 #ifdef CONFIG_TCP_MD5SIG
 	/* Calculate the MD5 hash, as we have all we need now */
-	if (md5) {
+	if (opts.md5) {
 		sk_nocaps_add(sk, NETIF_F_GSO_MASK);
 		tp->af_specific->calc_md5_hash(opts.hash_location,
-					       md5, sk, skb);
+					       opts.md5, sk, skb);
 	}
 #endif
 
@@ -1590,7 +1586,6 @@ unsigned int tcp_current_mss(struct sock *sk)
 	u32 mss_now;
 	unsigned int header_len;
 	struct tcp_out_options opts;
-	struct tcp_md5sig_key *md5;
 
 	mss_now = tp->mss_cache;
 
@@ -1600,7 +1595,7 @@ unsigned int tcp_current_mss(struct sock *sk)
 			mss_now = tcp_sync_mss(sk, mtu);
 	}
 
-	header_len = tcp_established_options(sk, NULL, &opts, &md5) +
+	header_len = tcp_established_options(sk, NULL, &opts) +
 		     sizeof(struct tcphdr);
 	/* The mss_cache is sized based on tp->tcp_header_len, which assumes
 	 * some common options. If this is an odd packet (because we have SACK
@@ -3179,7 +3174,6 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 {
 	struct inet_request_sock *ireq = inet_rsk(req);
 	const struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_key *md5 = NULL;
 	struct tcp_out_options opts;
 	struct sk_buff *skb;
 	int tcp_header_size;
@@ -3225,10 +3219,10 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 
 #ifdef CONFIG_TCP_MD5SIG
 	rcu_read_lock();
-	md5 = tcp_rsk(req)->af_specific->req_md5_lookup(sk, req_to_sk(req));
+	opts.md5 = tcp_rsk(req)->af_specific->req_md5_lookup(sk, req_to_sk(req));
 #endif
 	skb_set_hash(skb, tcp_rsk(req)->txhash, PKT_HASH_TYPE_L4);
-	tcp_header_size = tcp_synack_options(sk, req, mss, skb, &opts, md5,
+	tcp_header_size = tcp_synack_options(sk, req, mss, skb, &opts,
 					     foc) + sizeof(*th);
 
 	skb_push(skb, tcp_header_size);
@@ -3255,9 +3249,10 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 
 #ifdef CONFIG_TCP_MD5SIG
 	/* Okay, we have all we need - do the md5 hash if needed */
-	if (md5)
+	if (opts.md5)
 		tcp_rsk(req)->af_specific->calc_md5_hash(opts.hash_location,
-					       md5, req_to_sk(req), skb);
+							 opts.md5,
+							 req_to_sk(req), skb);
 	rcu_read_unlock();
 #endif
 
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#10 lore raw

Re: [MPTCP] [PATCH v2 06/25] tcp_md5: Don't pass along md5-key

Quoted 9 lines · Mat Martineau
Quoted 7 lines · Christoph Paasch

It is much cleaner to store the key-pointer in tcp_out_options.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_output.c | 47 +++++++++++++++++++++--------------------------

1 file changed, 21 insertions(+), 26 deletions(-)
Quoted 12 lines

net/ipv4/tcp_output.c struct tcp_out_options { · line 432

Quoted 9 lines
net/ipv4/tcp_output.c +1 -0
index 751396f22bb3..fedaaea83b43 100644
@@ -429,6 +429,7 @@ struct tcp_out_options {
 	__u8 *hash_location;	/* temporary pointer, overloaded */
 	__u32 tsval, tsecr;	/* need to include OPTION_TS */
 	struct tcp_fastopen_cookie *fastopen_cookie;	/* Fast open cookie */
+	struct tcp_md5sig_key *md5; /* TCP_MD5 signature key */

The md5 member can be wrapped in an #ifdef CONFIG_TCP_MD5SIG (and propagated to tcp.h when the struct gets moved), with some tweaks below.

See my comment below - ifdef'ing will make an if-statement quite messy.

Quoted 25 lines
Quoted 25 lines
net/ipv4/tcp_output.c +3 -4
 };
 
 /* Write previously computed TCP options to the packet.
@@ -582,21 +583,20 @@ static void smc_set_option_cond(const struct tcp_sock *tp,
 * network wire format yet.
 */
 static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb,
-				struct tcp_out_options *opts,
-				struct tcp_md5sig_key **md5)
+				struct tcp_out_options *opts)
 {
 	struct tcp_sock *tp = tcp_sk(sk);
 	unsigned int remaining = MAX_TCP_OPTION_SPACE;
 	struct tcp_fastopen_request *fastopen = tp->fastopen_req;
 
 #ifdef CONFIG_TCP_MD5SIG
-	*md5 = tp->af_specific->md5_lookup(sk, sk);
-	if (*md5) {
+	opts->md5 = tp->af_specific->md5_lookup(sk, sk);
+	if (opts->md5) {
 		opts->options |= OPTION_MD5;
 		remaining -= TCPOLEN_MD5SIG_ALIGNED;
 	}
 #else
Quoted 8 lines

net/ipv4/tcp_output.c static void smc_set_option_cond(const struct tcp_sock *tp, · line 600

Quoted 3 lines
net/ipv4/tcp_output.c +1 -1
-	*md5 = NULL;
+	opts->md5 = NULL;
 #endif

Since *opts is zeroed out, the #else block might not be needed. This would allow the struct member to be configured out too. While this is an intermediate step in the refactoring process, it is helpful to have opts.md5 #ifdef'd from the start.

I removed the opts->md5 = NULL here.

Quoted 7 lines

net/ipv4/tcp_output.c static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb, · line 614

Quoted 7 lines
net/ipv4/tcp_output.c +1 -1
 	/* We always get an MSS option.  The option bytes which will be seen in
@@ -611,7 +611,7 @@ static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb,
 	opts->mss = tcp_advertise_mss(sk);
 	remaining -= TCPOLEN_MSS_ALIGNED;
 
-	if (likely(sock_net(sk)->ipv4.sysctl_tcp_timestamps && !*md5)) {
+	if (likely(sock_net(sk)->ipv4.sysctl_tcp_timestamps && !opts->md5)) {

If I #ifdef opts->md5, this if-statement here will become quite messy.

However in the later patch when I adopt the framework for MD5, I will ifdef it out.

Quoted 32 lines
Quoted 32 lines
net/ipv4/tcp_output.c +2 -4
 		opts->options |= OPTION_TS;
 		opts->tsval = tcp_skb_timestamp(skb) + tp->tsoffset;
 		opts->tsecr = tp->rx_opt.ts_recent;
@@ -653,14 +653,13 @@ static unsigned int tcp_synack_options(const struct sock *sk,
 				       struct request_sock *req,
 				       unsigned int mss, struct sk_buff *skb,
 				       struct tcp_out_options *opts,
-				       const struct tcp_md5sig_key *md5,
 				       struct tcp_fastopen_cookie *foc)
 {
 	struct inet_request_sock *ireq = inet_rsk(req);
 	unsigned int remaining = MAX_TCP_OPTION_SPACE;
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (md5) {
+	if (opts->md5) {
 		opts->options |= OPTION_MD5;
 		remaining -= TCPOLEN_MD5SIG_ALIGNED;
 
@@ -715,8 +714,7 @@ static unsigned int tcp_synack_options(const struct sock *sk,
 * final wire format yet.
 */
 static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb,
-					struct tcp_out_options *opts,
-					struct tcp_md5sig_key **md5)
+					struct tcp_out_options *opts)
 {
 	struct tcp_sock *tp = tcp_sk(sk);
 	unsigned int size = 0;
@@ -725,13 +723,13 @@ static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb
 	opts->options = 0;
 
Quoted 14 lines

net/ipv4/tcp_output.c static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb · line 733

Quoted 12 lines
net/ipv4/tcp_output.c +3 -3
 #ifdef CONFIG_TCP_MD5SIG
-	*md5 = tp->af_specific->md5_lookup(sk, sk);
-	if (unlikely(*md5)) {
+	opts->md5 = tp->af_specific->md5_lookup(sk, sk);
+	if (unlikely(opts->md5)) {
 		opts->options |= OPTION_MD5;
 		size += TCPOLEN_MD5SIG_ALIGNED;
 	}
 #else
-	*md5 = NULL;
+	opts->md5 = NULL;
 #endif

Same as previous comment.

When coming from tcp_current_mss(), opts is not zeroed out. So, I think it's safer to leave this here.

Christoph

Quoted 96 lines
Quoted 92 lines
net/ipv4/tcp_output.c +10 -13
 	if (likely(tp->rx_opt.tstamp_ok)) {
@@ -1045,7 +1043,6 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	struct tcp_out_options opts;
 	unsigned int tcp_options_size, tcp_header_size;
 	struct sk_buff *oskb = NULL;
-	struct tcp_md5sig_key *md5;
 	struct tcphdr *th;
 	int err;
 
@@ -1074,10 +1071,9 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	memset(&opts, 0, sizeof(opts));
 
 	if (unlikely(tcb->tcp_flags & TCPHDR_SYN))
-		tcp_options_size = tcp_syn_options(sk, skb, &opts, &md5);
+		tcp_options_size = tcp_syn_options(sk, skb, &opts);
 	else
-		tcp_options_size = tcp_established_options(sk, skb, &opts,
-							   &md5);
+		tcp_options_size = tcp_established_options(sk, skb, &opts);
 	tcp_header_size = tcp_options_size + sizeof(struct tcphdr);
 
 	/* if no packet is in qdisc/device queue, then allow XPS to select
@@ -1143,10 +1139,10 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	tcp_options_write((__be32 *)(th + 1), skb, sk, &opts);
 #ifdef CONFIG_TCP_MD5SIG
 	/* Calculate the MD5 hash, as we have all we need now */
-	if (md5) {
+	if (opts.md5) {
 		sk_nocaps_add(sk, NETIF_F_GSO_MASK);
 		tp->af_specific->calc_md5_hash(opts.hash_location,
-					       md5, sk, skb);
+					       opts.md5, sk, skb);
 	}
 #endif
 
@@ -1590,7 +1586,6 @@ unsigned int tcp_current_mss(struct sock *sk)
 	u32 mss_now;
 	unsigned int header_len;
 	struct tcp_out_options opts;
-	struct tcp_md5sig_key *md5;
 
 	mss_now = tp->mss_cache;
 
@@ -1600,7 +1595,7 @@ unsigned int tcp_current_mss(struct sock *sk)
 			mss_now = tcp_sync_mss(sk, mtu);
 	}
 
-	header_len = tcp_established_options(sk, NULL, &opts, &md5) +
+	header_len = tcp_established_options(sk, NULL, &opts) +
 		     sizeof(struct tcphdr);
 	/* The mss_cache is sized based on tp->tcp_header_len, which assumes
 	 * some common options. If this is an odd packet (because we have SACK
@@ -3179,7 +3174,6 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 {
 	struct inet_request_sock *ireq = inet_rsk(req);
 	const struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_key *md5 = NULL;
 	struct tcp_out_options opts;
 	struct sk_buff *skb;
 	int tcp_header_size;
@@ -3225,10 +3219,10 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 
 #ifdef CONFIG_TCP_MD5SIG
 	rcu_read_lock();
-	md5 = tcp_rsk(req)->af_specific->req_md5_lookup(sk, req_to_sk(req));
+	opts.md5 = tcp_rsk(req)->af_specific->req_md5_lookup(sk, req_to_sk(req));
 #endif
 	skb_set_hash(skb, tcp_rsk(req)->txhash, PKT_HASH_TYPE_L4);
-	tcp_header_size = tcp_synack_options(sk, req, mss, skb, &opts, md5,
+	tcp_header_size = tcp_synack_options(sk, req, mss, skb, &opts,
 					     foc) + sizeof(*th);
 
 	skb_push(skb, tcp_header_size);
@@ -3255,9 +3249,10 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 
 #ifdef CONFIG_TCP_MD5SIG
 	/* Okay, we have all we need - do the md5 hash if needed */
-	if (md5)
+	if (opts.md5)
 		tcp_rsk(req)->af_specific->calc_md5_hash(opts.hash_location,
-					       md5, req_to_sk(req), skb);
+							 opts.md5,
+							 req_to_sk(req), skb);
 	rcu_read_unlock();
 #endif
 
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

#11 lore raw

Re: [MPTCP] [PATCH v2 06/25] tcp_md5: Don't pass along md5-key

Quoted 66 lines · Christoph Paasch
Quoted 9 lines · Mat Martineau
Quoted 7 lines · Christoph Paasch

It is much cleaner to store the key-pointer in tcp_out_options.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_output.c | 47 +++++++++++++++++++++--------------------------

1 file changed, 21 insertions(+), 26 deletions(-)
Quoted 12 lines

net/ipv4/tcp_output.c struct tcp_out_options { · line 432

Quoted 9 lines
net/ipv4/tcp_output.c +1 -0
index 751396f22bb3..fedaaea83b43 100644
@@ -429,6 +429,7 @@ struct tcp_out_options {
 	__u8 *hash_location;	/* temporary pointer, overloaded */
 	__u32 tsval, tsecr;	/* need to include OPTION_TS */
 	struct tcp_fastopen_cookie *fastopen_cookie;	/* Fast open cookie */
+	struct tcp_md5sig_key *md5; /* TCP_MD5 signature key */

The md5 member can be wrapped in an #ifdef CONFIG_TCP_MD5SIG (and propagated to tcp.h when the struct gets moved), with some tweaks below.

See my comment below - ifdef'ing will make an if-statement quite messy.

Quoted 25 lines
Quoted 25 lines
net/ipv4/tcp_output.c +3 -4
 };
 
 /* Write previously computed TCP options to the packet.
@@ -582,21 +583,20 @@ static void smc_set_option_cond(const struct tcp_sock *tp,
 * network wire format yet.
 */
 static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb,
-				struct tcp_out_options *opts,
-				struct tcp_md5sig_key **md5)
+				struct tcp_out_options *opts)
 {
 	struct tcp_sock *tp = tcp_sk(sk);
 	unsigned int remaining = MAX_TCP_OPTION_SPACE;
 	struct tcp_fastopen_request *fastopen = tp->fastopen_req;
 
 #ifdef CONFIG_TCP_MD5SIG
-	*md5 = tp->af_specific->md5_lookup(sk, sk);
-	if (*md5) {
+	opts->md5 = tp->af_specific->md5_lookup(sk, sk);
+	if (opts->md5) {
 		opts->options |= OPTION_MD5;
 		remaining -= TCPOLEN_MD5SIG_ALIGNED;
 	}
 #else
Quoted 8 lines

net/ipv4/tcp_output.c static void smc_set_option_cond(const struct tcp_sock *tp, · line 600

Quoted 3 lines
net/ipv4/tcp_output.c +1 -1
-	*md5 = NULL;
+	opts->md5 = NULL;
 #endif

Since *opts is zeroed out, the #else block might not be needed. This would allow the struct member to be configured out too. While this is an intermediate step in the refactoring process, it is helpful to have opts.md5 #ifdef'd from the start.

I removed the opts->md5 = NULL here.

Quoted 5 lines
Quoted 5 lines
net/ipv4/tcp_output.c +0 -0
 	/* We always get an MSS option.  The option bytes which will be seen in
@@ -611,7 +611,7 @@ static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb,
 	opts->mss = tcp_advertise_mss(sk);
 	remaining -= TCPOLEN_MSS_ALIGNED;
 
Quoted 8 lines
Quoted 2 lines

net/ipv4/tcp_output.c static unsigned int tcp_syn_options(struct sock *sk, struct sk_buff *skb, · line 614

Quoted 2 lines
net/ipv4/tcp_output.c +1 -1
-	if (likely(sock_net(sk)->ipv4.sysctl_tcp_timestamps && !*md5)) {
+	if (likely(sock_net(sk)->ipv4.sysctl_tcp_timestamps && !opts->md5)) {

If I #ifdef opts->md5, this if-statement here will become quite messy.

However in the later patch when I adopt the framework for MD5, I will ifdef it out.

Ah, ok.

Quoted 47 lines
Quoted 32 lines
Quoted 32 lines
net/ipv4/tcp_output.c +2 -4
 		opts->options |= OPTION_TS;
 		opts->tsval = tcp_skb_timestamp(skb) + tp->tsoffset;
 		opts->tsecr = tp->rx_opt.ts_recent;
@@ -653,14 +653,13 @@ static unsigned int tcp_synack_options(const struct sock *sk,
 				       struct request_sock *req,
 				       unsigned int mss, struct sk_buff *skb,
 				       struct tcp_out_options *opts,
-				       const struct tcp_md5sig_key *md5,
 				       struct tcp_fastopen_cookie *foc)
 {
 	struct inet_request_sock *ireq = inet_rsk(req);
 	unsigned int remaining = MAX_TCP_OPTION_SPACE;
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (md5) {
+	if (opts->md5) {
 		opts->options |= OPTION_MD5;
 		remaining -= TCPOLEN_MD5SIG_ALIGNED;
 
@@ -715,8 +714,7 @@ static unsigned int tcp_synack_options(const struct sock *sk,
 * final wire format yet.
 */
 static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb,
-					struct tcp_out_options *opts,
-					struct tcp_md5sig_key **md5)
+					struct tcp_out_options *opts)
 {
 	struct tcp_sock *tp = tcp_sk(sk);
 	unsigned int size = 0;
@@ -725,13 +723,13 @@ static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb
 	opts->options = 0;
 
Quoted 14 lines

net/ipv4/tcp_output.c static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb · line 733

Quoted 12 lines
net/ipv4/tcp_output.c +3 -3
 #ifdef CONFIG_TCP_MD5SIG
-	*md5 = tp->af_specific->md5_lookup(sk, sk);
-	if (unlikely(*md5)) {
+	opts->md5 = tp->af_specific->md5_lookup(sk, sk);
+	if (unlikely(opts->md5)) {
 		opts->options |= OPTION_MD5;
 		size += TCPOLEN_MD5SIG_ALIGNED;
 	}
 #else
-	*md5 = NULL;
+	opts->md5 = NULL;
 #endif

Same as previous comment.

Quoted 3 lines

When coming from tcp_current_mss(), opts is not zeroed out. So, I think it's safer to leave this here.

I had missed that. Looks like it's not zeroed because all of the struct values are ignored.

Mat

Quoted 89 lines
Quoted 89 lines
Quoted 89 lines
net/ipv4/tcp_output.c +10 -13
 	if (likely(tp->rx_opt.tstamp_ok)) {
@@ -1045,7 +1043,6 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	struct tcp_out_options opts;
 	unsigned int tcp_options_size, tcp_header_size;
 	struct sk_buff *oskb = NULL;
-	struct tcp_md5sig_key *md5;
 	struct tcphdr *th;
 	int err;
 
@@ -1074,10 +1071,9 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	memset(&opts, 0, sizeof(opts));
 
 	if (unlikely(tcb->tcp_flags & TCPHDR_SYN))
-		tcp_options_size = tcp_syn_options(sk, skb, &opts, &md5);
+		tcp_options_size = tcp_syn_options(sk, skb, &opts);
 	else
-		tcp_options_size = tcp_established_options(sk, skb, &opts,
-							   &md5);
+		tcp_options_size = tcp_established_options(sk, skb, &opts);
 	tcp_header_size = tcp_options_size + sizeof(struct tcphdr);
 
 	/* if no packet is in qdisc/device queue, then allow XPS to select
@@ -1143,10 +1139,10 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
 	tcp_options_write((__be32 *)(th + 1), skb, sk, &opts);
 #ifdef CONFIG_TCP_MD5SIG
 	/* Calculate the MD5 hash, as we have all we need now */
-	if (md5) {
+	if (opts.md5) {
 		sk_nocaps_add(sk, NETIF_F_GSO_MASK);
 		tp->af_specific->calc_md5_hash(opts.hash_location,
-					       md5, sk, skb);
+					       opts.md5, sk, skb);
 	}
 #endif
 
@@ -1590,7 +1586,6 @@ unsigned int tcp_current_mss(struct sock *sk)
 	u32 mss_now;
 	unsigned int header_len;
 	struct tcp_out_options opts;
-	struct tcp_md5sig_key *md5;
 
 	mss_now = tp->mss_cache;
 
@@ -1600,7 +1595,7 @@ unsigned int tcp_current_mss(struct sock *sk)
 			mss_now = tcp_sync_mss(sk, mtu);
 	}
 
-	header_len = tcp_established_options(sk, NULL, &opts, &md5) +
+	header_len = tcp_established_options(sk, NULL, &opts) +
 		     sizeof(struct tcphdr);
 	/* The mss_cache is sized based on tp->tcp_header_len, which assumes
 	 * some common options. If this is an odd packet (because we have SACK
@@ -3179,7 +3174,6 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 {
 	struct inet_request_sock *ireq = inet_rsk(req);
 	const struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_key *md5 = NULL;
 	struct tcp_out_options opts;
 	struct sk_buff *skb;
 	int tcp_header_size;
@@ -3225,10 +3219,10 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 
 #ifdef CONFIG_TCP_MD5SIG
 	rcu_read_lock();
-	md5 = tcp_rsk(req)->af_specific->req_md5_lookup(sk, req_to_sk(req));
+	opts.md5 = tcp_rsk(req)->af_specific->req_md5_lookup(sk, req_to_sk(req));
 #endif
 	skb_set_hash(skb, tcp_rsk(req)->txhash, PKT_HASH_TYPE_L4);
-	tcp_header_size = tcp_synack_options(sk, req, mss, skb, &opts, md5,
+	tcp_header_size = tcp_synack_options(sk, req, mss, skb, &opts,
 					     foc) + sizeof(*th);
 
 	skb_push(skb, tcp_header_size);
@@ -3255,9 +3249,10 @@ struct sk_buff *tcp_make_synack(const struct sock *sk, struct dst_entry *dst,
 
 #ifdef CONFIG_TCP_MD5SIG
 	/* Okay, we have all we need - do the md5 hash if needed */
-	if (md5)
+	if (opts.md5)
 		tcp_rsk(req)->af_specific->calc_md5_hash(opts.hash_location,
-					       md5, req_to_sk(req), skb);
+							 opts.md5,
+							 req_to_sk(req), skb);
 	rcu_read_unlock();
 #endif
 
Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

--
2.15.0

--
Mat Martineau
Intel OTC

--
Mat Martineau
Intel OTC

author
#12 lore raw

PATCH v2 7/25 tcp_md5: Detect key inside tcp_v4_send_ack instead of passing it as an argument

In reply to Christoph Paasch

This will simplify to consolidate the TCP_MD5-code into a single place.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv4/tcp_ipv4.c | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)
net/ipv4/tcp_ipv4.c +14 -4
index c6bc0c4d19c6..6f14004bd36d 100644
@@ -736,7 +736,6 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 static void tcp_v4_send_ack(const struct sock *sk,
 			    struct sk_buff *skb, u32 seq, u32 ack,
 			    u32 win, u32 tsval, u32 tsecr, int oif,
-			    struct tcp_md5sig_key *key,
 			    int reply_flags, u8 tos)
 {
 	const struct tcphdr *th = tcp_hdr(skb);
@@ -748,6 +747,9 @@ static void tcp_v4_send_ack(const struct sock *sk,
 #endif
 			];
 	} rep;
+#ifdef CONFIG_TCP_MD5SIG
+	struct tcp_md5sig_key *key;
+#endif
 	struct net *net = sock_net(sk);
 	struct ip_reply_arg arg;
 
@@ -775,6 +777,17 @@ static void tcp_v4_send_ack(const struct sock *sk,
 	rep.th.window  = htons(win);
 
 #ifdef CONFIG_TCP_MD5SIG
+	if (sk->sk_state == TCP_TIME_WAIT) {
+		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
+
+		key = tcp_twsk_md5_key(tcptw);
+	} else if (sk->sk_state == TCP_NEW_SYN_RECV) {
+		key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&ip_hdr(skb)->daddr,
+					AF_INET);
+	} else {
+		BUG();
+	}
+
 	if (key) {
 		int offset = (tsecr) ? 3 : 0;
 
@@ -820,7 +833,6 @@ static void tcp_v4_timewait_ack(struct sock *sk, struct sk_buff *skb)
 			tcp_time_stamp_raw() + tcptw->tw_ts_offset,
 			tcptw->tw_ts_recent,
 			tw->tw_bound_dev_if,
-			tcp_twsk_md5_key(tcptw),
 			tw->tw_transparent ? IP_REPLY_ARG_NOSRCCHECK : 0,
 			tw->tw_tos
 			);
@@ -848,8 +860,6 @@ static void tcp_v4_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
 			tcp_time_stamp_raw() + tcp_rsk(req)->ts_off,
 			req->ts_recent,
 			0,
-			tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&ip_hdr(skb)->daddr,
-					  AF_INET),
 			inet_rsk(req)->no_srccheck ? IP_REPLY_ARG_NOSRCCHECK : 0,
 			ip_hdr(skb)->tos);
 }
Signature · 1 line
2.15.0
#13 lore raw

Re: [MPTCP] [PATCH v2 07/25] tcp_md5: Detect key inside tcp_v4_send_ack instead of passing it as an argument

Quoted 30 lines · Christoph Paasch

This will simplify to consolidate the TCP_MD5-code into a single place.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_ipv4.c | 18 ++++++++++++++----

1 file changed, 14 insertions(+), 4 deletions(-)
net/ipv4/tcp_ipv4.c +3 -1
index c6bc0c4d19c6..6f14004bd36d 100644
@@ -736,7 +736,6 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 static void tcp_v4_send_ack(const struct sock *sk,
 			    struct sk_buff *skb, u32 seq, u32 ack,
 			    u32 win, u32 tsval, u32 tsecr, int oif,
-			    struct tcp_md5sig_key *key,
 			    int reply_flags, u8 tos)
 {
 	const struct tcphdr *th = tcp_hdr(skb);
@@ -748,6 +747,9 @@ static void tcp_v4_send_ack(const struct sock *sk,
 #endif
 			];
 	} rep;
+#ifdef CONFIG_TCP_MD5SIG
+	struct tcp_md5sig_key *key;
+#endif
 	struct net *net = sock_net(sk);
 	struct ip_reply_arg arg;
 

net/ipv4/tcp_ipv4.c static void tcp_v4_send_ack(const struct sock *sk, · line 783

Quoted 7 lines
net/ipv4/tcp_ipv4.c +4 -0
@@ -775,6 +777,17 @@ static void tcp_v4_send_ack(const struct sock *sk,
 	rep.th.window  = htons(win);
 
 #ifdef CONFIG_TCP_MD5SIG
+	if (sk->sk_state == TCP_TIME_WAIT) {
+		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
+
+		key = tcp_twsk_md5_key(tcptw);

Minor nit, could call tcp_twsk_md5_key(tcp_twsk(sk)). But this gets refactored away anyhow.

Quoted 31 lines
net/ipv4/tcp_ipv4.c +7 -3
+	} else if (sk->sk_state == TCP_NEW_SYN_RECV) {
+		key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&ip_hdr(skb)->daddr,
+					AF_INET);
+	} else {
+		BUG();
+	}
+
 	if (key) {
 		int offset = (tsecr) ? 3 : 0;
 
@@ -820,7 +833,6 @@ static void tcp_v4_timewait_ack(struct sock *sk, struct sk_buff *skb)
 			tcp_time_stamp_raw() + tcptw->tw_ts_offset,
 			tcptw->tw_ts_recent,
 			tw->tw_bound_dev_if,
-			tcp_twsk_md5_key(tcptw),
 			tw->tw_transparent ? IP_REPLY_ARG_NOSRCCHECK : 0,
 			tw->tw_tos
 			);
@@ -848,8 +860,6 @@ static void tcp_v4_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
 			tcp_time_stamp_raw() + tcp_rsk(req)->ts_off,
 			req->ts_recent,
 			0,
-			tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&ip_hdr(skb)->daddr,
-					  AF_INET),
 			inet_rsk(req)->no_srccheck ? IP_REPLY_ARG_NOSRCCHECK : 0,
 			ip_hdr(skb)->tos);
 }
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#14 lore raw

Re: [MPTCP] [PATCH v2 07/25] tcp_md5: Detect key inside tcp_v4_send_ack instead of passing it as an argument

Quoted 34 lines · Mat Martineau
Quoted 32 lines · Christoph Paasch

This will simplify to consolidate the TCP_MD5-code into a single place.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_ipv4.c | 18 ++++++++++++++----

1 file changed, 14 insertions(+), 4 deletions(-)
net/ipv4/tcp_ipv4.c +3 -1
index c6bc0c4d19c6..6f14004bd36d 100644
@@ -736,7 +736,6 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 static void tcp_v4_send_ack(const struct sock *sk,
 			    struct sk_buff *skb, u32 seq, u32 ack,
 			    u32 win, u32 tsval, u32 tsecr, int oif,
-			    struct tcp_md5sig_key *key,
 			    int reply_flags, u8 tos)
 {
 	const struct tcphdr *th = tcp_hdr(skb);
@@ -748,6 +747,9 @@ static void tcp_v4_send_ack(const struct sock *sk,
 #endif
 			];
 	} rep;
+#ifdef CONFIG_TCP_MD5SIG
+	struct tcp_md5sig_key *key;
+#endif
 	struct net *net = sock_net(sk);
 	struct ip_reply_arg arg;
 
@@ -775,6 +777,17 @@ static void tcp_v4_send_ack(const struct sock *sk,
 	rep.th.window  = htons(win);
 
Quoted 8 lines

net/ipv4/tcp_ipv4.c static void tcp_v4_send_ack(const struct sock *sk, · line 783

Quoted 5 lines
net/ipv4/tcp_ipv4.c +4 -0
 #ifdef CONFIG_TCP_MD5SIG
+	if (sk->sk_state == TCP_TIME_WAIT) {
+		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
+
+		key = tcp_twsk_md5_key(tcptw);

Minor nit, could call tcp_twsk_md5_key(tcp_twsk(sk)). But this gets refactored away anyhow.

Done, use tcp_twsk(sk) directly.

Christoph

Quoted 36 lines
Quoted 32 lines
net/ipv4/tcp_ipv4.c +7 -3
+	} else if (sk->sk_state == TCP_NEW_SYN_RECV) {
+		key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&ip_hdr(skb)->daddr,
+					AF_INET);
+	} else {
+		BUG();
+	}
+
 	if (key) {
 		int offset = (tsecr) ? 3 : 0;
 
@@ -820,7 +833,6 @@ static void tcp_v4_timewait_ack(struct sock *sk, struct sk_buff *skb)
 			tcp_time_stamp_raw() + tcptw->tw_ts_offset,
 			tcptw->tw_ts_recent,
 			tw->tw_bound_dev_if,
-			tcp_twsk_md5_key(tcptw),
 			tw->tw_transparent ? IP_REPLY_ARG_NOSRCCHECK : 0,
 			tw->tw_tos
 			);
@@ -848,8 +860,6 @@ static void tcp_v4_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
 			tcp_time_stamp_raw() + tcp_rsk(req)->ts_off,
 			req->ts_recent,
 			0,
-			tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&ip_hdr(skb)->daddr,
-					  AF_INET),
 			inet_rsk(req)->no_srccheck ? IP_REPLY_ARG_NOSRCCHECK : 0,
 			ip_hdr(skb)->tos);
 }
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#15 lore raw

PATCH v2 8/25 tcp_md5: Detect key inside tcp_v6_send_response instead of passing it as an argument

In reply to Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv6/tcp_ipv6.c | 113 ++++++++++++++++++++++++++--------------------------
 1 file changed, 57 insertions(+), 56 deletions(-)
net/ipv6/tcp_ipv6.c +57 -56
index 6bb98c93edfe..7075f4a1d701 100644
@@ -777,22 +777,65 @@ static const struct tcp_request_sock_ops tcp_request_sock_ipv6_ops = {
 
 static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32 seq,
 				 u32 ack, u32 win, u32 tsval, u32 tsecr,
-				 int oif, struct tcp_md5sig_key *key, int rst,
-				 u8 tclass, __be32 label)
+				 int oif, int rst, u8 tclass, __be32 label)
 {
 	const struct tcphdr *th = tcp_hdr(skb);
 	struct tcphdr *t1;
-	struct sk_buff *buff;
+	struct sk_buff *buff = NULL;
 	struct flowi6 fl6;
 	struct net *net = sk ? sock_net(sk) : dev_net(skb_dst(skb)->dev);
 	struct sock *ctl_sk = net->ipv6.tcp_sk;
 	unsigned int tot_len = sizeof(struct tcphdr);
 	struct dst_entry *dst;
 	__be32 *topt;
+#ifdef CONFIG_TCP_MD5SIG
+	struct tcp_md5sig_key *key = NULL;
+	const __u8 *hash_location = NULL;
+	struct ipv6hdr *ipv6h = ipv6_hdr(skb);
+#endif
 
 	if (tsecr)
 		tot_len += TCPOLEN_TSTAMP_ALIGNED;
 #ifdef CONFIG_TCP_MD5SIG
+	rcu_read_lock();
+	hash_location = tcp_parse_md5sig_option(th);
+	if (sk && sk_fullsock(sk)) {
+		key = tcp_v6_md5_do_lookup(sk, &ipv6h->saddr);
+	} else if (sk && sk->sk_state == TCP_TIME_WAIT) {
+		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
+
+		key = tcp_twsk_md5_key(tcptw);
+	} else if (sk && sk->sk_state == TCP_NEW_SYN_RECV) {
+		key = tcp_v6_md5_do_lookup(sk, &ipv6h->daddr);
+	} else if (hash_location) {
+		unsigned char newhash[16];
+		struct sock *sk1 = NULL;
+		int genhash;
+
+		/* active side is lost. Try to find listening socket through
+		 * source port, and then find md5 key through listening socket.
+		 * we are not loose security here:
+		 * Incoming packet is checked with md5 hash with finding key,
+		 * no RST generated if md5 hash doesn't match.
+		 */
+		sk1 = inet6_lookup_listener(dev_net(skb_dst(skb)->dev),
+					    &tcp_hashinfo, NULL, 0,
+					    &ipv6h->saddr,
+					    th->source, &ipv6h->daddr,
+					    ntohs(th->source), tcp_v6_iif(skb),
+					    tcp_v6_sdif(skb));
+		if (!sk1)
+			goto out;
+
+		key = tcp_v6_md5_do_lookup(sk1, &ipv6h->saddr);
+		if (!key)
+			goto out;
+
+		genhash = tcp_v6_md5_hash_skb(newhash, key, NULL, skb);
+		if (genhash || memcmp(hash_location, newhash, 16) != 0)
+			goto out;
+	}
+
 	if (key)
 		tot_len += TCPOLEN_MD5SIG_ALIGNED;
 #endif
@@ -800,7 +843,7 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 	buff = alloc_skb(MAX_HEADER + sizeof(struct ipv6hdr) + tot_len,
 			 GFP_ATOMIC);
 	if (!buff)
-		return;
+		goto out;
 
 	skb_reserve(buff, MAX_HEADER + sizeof(struct ipv6hdr) + tot_len);
 
@@ -874,24 +917,21 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 		TCP_INC_STATS(net, TCP_MIB_OUTSEGS);
 		if (rst)
 			TCP_INC_STATS(net, TCP_MIB_OUTRSTS);
-		return;
+		buff = NULL;
 	}
 
+out:
 	kfree_skb(buff);
+
+#if CONFIG_TCP_MD5SIG
+	rcu_read_unlock();
+#endif
 }
 
 static void tcp_v6_send_reset(const struct sock *sk, struct sk_buff *skb)
 {
 	const struct tcphdr *th = tcp_hdr(skb);
 	u32 seq = 0, ack_seq = 0;
-	struct tcp_md5sig_key *key = NULL;
-#ifdef CONFIG_TCP_MD5SIG
-	const __u8 *hash_location = NULL;
-	struct ipv6hdr *ipv6h = ipv6_hdr(skb);
-	unsigned char newhash[16];
-	int genhash;
-	struct sock *sk1 = NULL;
-#endif
 	int oif = 0;
 
 	if (th->rst)
@@ -903,38 +943,6 @@ static void tcp_v6_send_reset(const struct sock *sk, struct sk_buff *skb)
 	if (!sk && !ipv6_unicast_destination(skb))
 		return;
 
-#ifdef CONFIG_TCP_MD5SIG
-	rcu_read_lock();
-	hash_location = tcp_parse_md5sig_option(th);
-	if (sk && sk_fullsock(sk)) {
-		key = tcp_v6_md5_do_lookup(sk, &ipv6h->saddr);
-	} else if (hash_location) {
-		/*
-		 * active side is lost. Try to find listening socket through
-		 * source port, and then find md5 key through listening socket.
-		 * we are not loose security here:
-		 * Incoming packet is checked with md5 hash with finding key,
-		 * no RST generated if md5 hash doesn't match.
-		 */
-		sk1 = inet6_lookup_listener(dev_net(skb_dst(skb)->dev),
-					   &tcp_hashinfo, NULL, 0,
-					   &ipv6h->saddr,
-					   th->source, &ipv6h->daddr,
-					   ntohs(th->source), tcp_v6_iif(skb),
-					   tcp_v6_sdif(skb));
-		if (!sk1)
-			goto out;
-
-		key = tcp_v6_md5_do_lookup(sk1, &ipv6h->saddr);
-		if (!key)
-			goto out;
-
-		genhash = tcp_v6_md5_hash_skb(newhash, key, NULL, skb);
-		if (genhash || memcmp(hash_location, newhash, 16) != 0)
-			goto out;
-	}
-#endif
-
 	if (th->ack)
 		seq = ntohl(th->ack_seq);
 	else
@@ -946,20 +954,14 @@ static void tcp_v6_send_reset(const struct sock *sk, struct sk_buff *skb)
 		trace_tcp_send_reset(sk, skb);
 	}
 
-	tcp_v6_send_response(sk, skb, seq, ack_seq, 0, 0, 0, oif, key, 1, 0, 0);
-
-#ifdef CONFIG_TCP_MD5SIG
-out:
-	rcu_read_unlock();
-#endif
+	tcp_v6_send_response(sk, skb, seq, ack_seq, 0, 0, 0, oif, 1, 0, 0);
 }
 
 static void tcp_v6_send_ack(const struct sock *sk, struct sk_buff *skb, u32 seq,
 			    u32 ack, u32 win, u32 tsval, u32 tsecr, int oif,
-			    struct tcp_md5sig_key *key, u8 tclass,
-			    __be32 label)
+			    u8 tclass, __be32 label)
 {
-	tcp_v6_send_response(sk, skb, seq, ack, win, tsval, tsecr, oif, key, 0,
+	tcp_v6_send_response(sk, skb, seq, ack, win, tsval, tsecr, oif, 0,
 			     tclass, label);
 }
 
@@ -971,7 +973,7 @@ static void tcp_v6_timewait_ack(struct sock *sk, struct sk_buff *skb)
 	tcp_v6_send_ack(sk, skb, tcptw->tw_snd_nxt, tcptw->tw_rcv_nxt,
 			tcptw->tw_rcv_wnd >> tw->tw_rcv_wscale,
 			tcp_time_stamp_raw() + tcptw->tw_ts_offset,
-			tcptw->tw_ts_recent, tw->tw_bound_dev_if, tcp_twsk_md5_key(tcptw),
+			tcptw->tw_ts_recent, tw->tw_bound_dev_if,
 			tw->tw_tclass, cpu_to_be32(tw->tw_flowlabel));
 
 	inet_twsk_put(tw);
@@ -994,7 +996,6 @@ static void tcp_v6_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
 			req->rsk_rcv_wnd >> inet_rsk(req)->rcv_wscale,
 			tcp_time_stamp_raw() + tcp_rsk(req)->ts_off,
 			req->ts_recent, sk->sk_bound_dev_if,
-			tcp_v6_md5_do_lookup(sk, &ipv6_hdr(skb)->daddr),
 			0, 0);
 }
 
Signature · 1 line
2.15.0
author
#16 lore raw

PATCH v2 9/25 tcp: Register handlers for extra TCP options

In reply to Christoph Paasch

From: Mat Martineau <mathew.j.martineau(a)linux.intel.com>

Allow additional TCP options to be handled by registered hook functions.

Registered options have a priority that determines the order in which options are prepared and written. Lower priority numbers are handled first.

Option parsing will call the provided 'parse' function when a TCP option number is not recognized by the normal option parsing code.

The 'prepare' function determines the required space for registered options and store associated data. 'write' adds the option to the TCP header.

A static key and RCU synchronization are used to minimize the performance impact of these extensible TCP features.

Signed-off-by Mat Martineau <mathew.j.martineau(a)linux.intel.com>
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---

Notes:
v2: * Merged my previous commits from v1 into this one
* Moved rep.th.doff = arg.iov[0].iov_len / 4; to after the MD5-code in tcp_v4_send_ack

 drivers/infiniband/hw/cxgb4/cm.c |   2 +-
 include/linux/tcp.h              |  24 +++++++
 include/net/tcp.h                |  46 +++++++++++-
 net/ipv4/syncookies.c            |   2 +-
 net/ipv4/tcp.c                   | 151 +++++++++++++++++++++++++++++++++++++++
 net/ipv4/tcp_input.c             |  16 +++--
 net/ipv4/tcp_ipv4.c              |  87 +++++++++++++++++-----
 net/ipv4/tcp_minisocks.c         |   7 +-
 net/ipv4/tcp_output.c            |  47 ++++++------
 net/ipv6/syncookies.c            |   2 +-
 net/ipv6/tcp_ipv6.c              |  18 +++++
 11 files changed, 351 insertions(+), 51 deletions(-)
drivers/infiniband/hw/cxgb4/cm.c +1 -1
index 21db3b48a617..a1ea5583f07b 100644
@@ -3746,7 +3746,7 @@ static void build_cpl_pass_accept_req(struct sk_buff *skb, int stid , u8 tos)
 	 */
 	memset(&tmp_opt, 0, sizeof(tmp_opt));
 	tcp_clear_options(&tmp_opt);
-	tcp_parse_options(&init_net, skb, &tmp_opt, 0, NULL);
+	tcp_parse_options(&init_net, skb, &tmp_opt, 0, NULL, NULL);
 
 	req = __skb_push(skb, sizeof(*req));
 	memset(req, 0, sizeof(*req));
include/linux/tcp.h +24 -0
index df5d97a85e1a..a2a1676dfc52 100644
@@ -115,6 +115,25 @@ static inline void tcp_clear_options(struct tcp_options_received *rx_opt)
 #endif
 }
 
+#define OPTION_SACK_ADVERTISE	(1 << 0)
+#define OPTION_TS		(1 << 1)
+#define OPTION_MD5		(1 << 2)
+#define OPTION_WSCALE		(1 << 3)
+#define OPTION_FAST_OPEN_COOKIE	(1 << 8)
+#define OPTION_SMC		(1 << 9)
+
+struct tcp_out_options {
+	u16 options;		/* bit field of OPTION_* */
+	u16 mss;		/* 0 to disable */
+	u8 ws;			/* window scale, 0 to disable */
+	u8 num_sack_blocks;	/* number of SACK blocks to include */
+	u8 hash_size;		/* bytes in hash_location */
+	__u8 *hash_location;	/* temporary pointer, overloaded */
+	__u32 tsval, tsecr;	/* need to include OPTION_TS */
+	struct tcp_fastopen_cookie *fastopen_cookie;	/* Fast open cookie */
+	struct tcp_md5sig_key *md5; /* TCP_MD5 signature key */
+};
+
 /* This is the max number of SACKS that we'll generate and process. It's safe
  * to increase this, although since:
  *   size = TCPOLEN_SACK_BASE_ALIGNED (4) + n * TCPOLEN_SACK_PERBLOCK (8)
@@ -399,6 +418,11 @@ static inline struct tcp_sock *tcp_sk(const struct sock *sk)
 	return (struct tcp_sock *)sk;
 }
 
+static inline struct sock *tcp_to_sk(const struct tcp_sock *tp)
+{
+	return (struct sock *)tp;
+}
+
 struct tcp_timewait_sock {
 	struct inet_timewait_sock tw_sk;
 #define tw_rcv_nxt tw_sk.__tw_common.skc_tw_rcv_nxt
include/net/tcp.h +45 -1

56 lines not on this page · raw message

net/ipv4/syncookies.c +1 -1
index fda37f2862c9..b3698ac3f078 100644
@@ -313,7 +313,7 @@ struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb)
 
 	/* check for timestamp cookie support */
 	memset(&tcp_opt, 0, sizeof(tcp_opt));
-	tcp_parse_options(sock_net(sk), skb, &tcp_opt, 0, NULL);
+	tcp_parse_options(sock_net(sk), skb, &tcp_opt, 0, NULL, tp);
 
 	if (tcp_opt.saw_tstamp && tcp_opt.rcv_tsecr) {
 		tsoff = secure_tcp_ts_off(sock_net(sk),
net/ipv4/tcp.c +151 -0

169 lines not on this page · raw message

net/ipv4/tcp_input.c +11 -5
index 843061955973..028da0a4eff5 100644
@@ -3693,7 +3693,7 @@ static int smc_parse_options(const struct tcphdr *th,
 void tcp_parse_options(const struct net *net,
 		       const struct sk_buff *skb,
 		       struct tcp_options_received *opt_rx, int estab,
-		       struct tcp_fastopen_cookie *foc)
+		       struct tcp_fastopen_cookie *foc, struct tcp_sock *tp)
 {
 	const unsigned char *ptr;
 	const struct tcphdr *th = tcp_hdr(skb);
@@ -3798,6 +3798,12 @@ void tcp_parse_options(const struct net *net,
 					break;
 				break;
 
+			default:
+				tcp_extra_options_parse(opcode, opsize, ptr,
+							skb, opt_rx,
+							tcp_to_sk(tp));
+				break;
+
 			}
 			ptr += opsize-2;
 			length -= opsize;
@@ -3846,7 +3852,7 @@ static bool tcp_fast_parse_options(const struct net *net,
 			goto extra_opt_check;
 	}
 
-	tcp_parse_options(net, skb, &tp->rx_opt, 1, NULL);
+	tcp_parse_options(net, skb, &tp->rx_opt, 1, NULL, tp);
 	if (tp->rx_opt.saw_tstamp && tp->rx_opt.rcv_tsecr)
 		tp->rx_opt.rcv_tsecr -= tp->tsoffset;
 
@@ -5512,7 +5518,7 @@ static bool tcp_rcv_fastopen_synack(struct sock *sk, struct sk_buff *synack,
 		/* Get original SYNACK MSS value if user MSS sets mss_clamp */
 		tcp_clear_options(&opt);
 		opt.user_mss = opt.mss_clamp = 0;
-		tcp_parse_options(sock_net(sk), synack, &opt, 0, NULL);
+		tcp_parse_options(sock_net(sk), synack, &opt, 0, NULL, tp);
 		mss = opt.mss_clamp;
 	}
 
@@ -5575,7 +5581,7 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
 	int saved_clamp = tp->rx_opt.mss_clamp;
 	bool fastopen_fail;
 
-	tcp_parse_options(sock_net(sk), skb, &tp->rx_opt, 0, &foc);
+	tcp_parse_options(sock_net(sk), skb, &tp->rx_opt, 0, &foc, tp);
 	if (tp->rx_opt.saw_tstamp && tp->rx_opt.rcv_tsecr)
 		tp->rx_opt.rcv_tsecr -= tp->tsoffset;
 
@@ -6242,7 +6248,7 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,
 	tmp_opt.mss_clamp = af_ops->mss_clamp;
 	tmp_opt.user_mss  = tp->rx_opt.user_mss;
 	tcp_parse_options(sock_net(sk), skb, &tmp_opt, 0,
-			  want_cookie ? NULL : &foc);
+			  want_cookie ? NULL : &foc, tp);
 
 	if (want_cookie && !tmp_opt.saw_tstamp)
 		tcp_clear_options(&tmp_opt);
net/ipv4/tcp_ipv4.c +68 -19

153 lines not on this page · raw message

net/ipv4/tcp_minisocks.c +5 -2
index e36eff0403f4..7f9403d3272c 100644
@@ -97,7 +97,7 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb,
 
 	tmp_opt.saw_tstamp = 0;
 	if (th->doff > (sizeof(*th) >> 2) && tcptw->tw_ts_recent_stamp) {
-		tcp_parse_options(twsk_net(tw), skb, &tmp_opt, 0, NULL);
+		tcp_parse_options(twsk_net(tw), skb, &tmp_opt, 0, NULL, NULL);
 
 		if (tmp_opt.saw_tstamp) {
 			if (tmp_opt.rcv_tsecr)
@@ -538,6 +538,9 @@ struct sock *tcp_create_openreq_child(const struct sock *sk,
 		if (newtp->af_specific->md5_lookup(sk, newsk))
 			newtp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
 #endif
+		if (static_branch_unlikely(&tcp_extra_options_enabled))
+			newtp->tcp_header_len += tcp_extra_options_add_header(sk, newsk);
+
 		if (skb->len >= TCP_MSS_DEFAULT + newtp->tcp_header_len)
 			newicsk->icsk_ack.last_seg_size = skb->len - newtp->tcp_header_len;
 		newtp->rx_opt.mss_clamp = req->mss;
@@ -582,7 +585,7 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
 
 	tmp_opt.saw_tstamp = 0;
 	if (th->doff > (sizeof(struct tcphdr)>>2)) {
-		tcp_parse_options(sock_net(sk), skb, &tmp_opt, 0, NULL);
+		tcp_parse_options(sock_net(sk), skb, &tmp_opt, 0, NULL, NULL);
 
 		if (tmp_opt.saw_tstamp) {
 			tmp_opt.ts_recent = req->ts_recent;
net/ipv4/tcp_output.c +26 -21

115 lines not on this page · raw message

net/ipv6/syncookies.c +1 -1
index e7a3a6b6cf56..0ed3721b03a2 100644
@@ -162,7 +162,7 @@ struct sock *cookie_v6_check(struct sock *sk, struct sk_buff *skb)
 
 	/* check for timestamp cookie support */
 	memset(&tcp_opt, 0, sizeof(tcp_opt));
-	tcp_parse_options(sock_net(sk), skb, &tcp_opt, 0, NULL);
+	tcp_parse_options(sock_net(sk), skb, &tcp_opt, 0, NULL, tp);
 
 	if (tcp_opt.saw_tstamp && tcp_opt.rcv_tsecr) {
 		tsoff = secure_tcpv6_ts_off(sock_net(sk),
net/ipv6/tcp_ipv6.c +18 -0
index 7075f4a1d701..7e4e8788943f 100644
@@ -793,6 +793,7 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 	const __u8 *hash_location = NULL;
 	struct ipv6hdr *ipv6h = ipv6_hdr(skb);
 #endif
+	struct tcp_out_options extraopts;
 
 	if (tsecr)
 		tot_len += TCPOLEN_TSTAMP_ALIGNED;
@@ -840,6 +841,20 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 		tot_len += TCPOLEN_MD5SIG_ALIGNED;
 #endif
 
+
+	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
+		unsigned int remaining = MAX_TCP_OPTION_SPACE - tot_len;
+		u8 extraflags = rst ? TCPHDR_RST : 0;
+
+		if (!rst || !th->ack)
+			extraflags |= TCPHDR_ACK;
+
+		memset(&extraopts, 0, sizeof(extraopts));
+
+		tot_len += tcp_extra_options_prepare(skb, extraflags, remaining,
+						     &extraopts, sk);
+	}
+
 	buff = alloc_skb(MAX_HEADER + sizeof(struct ipv6hdr) + tot_len,
 			 GFP_ATOMIC);
 	if (!buff)
@@ -880,6 +895,9 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 	}
 #endif
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		tcp_extra_options_write(topt, buff, &extraopts, sk);
+
 	memset(&fl6, 0, sizeof(fl6));
 	fl6.daddr = ipv6_hdr(skb)->saddr;
 	fl6.saddr = ipv6_hdr(skb)->daddr;
Signature · 1 line
2.15.0
author
#17 lore raw

PATCH v2 10/25 tcp_extra_options: Extend tcp_extra_options_write

In reply to Christoph Paasch

The sk in tcp_extra_options_write cannot be a const, because we need to be able to modify it (see TCP_MD5 in tcp_transmit_skb).

In tcp_v4_send_ack/reset (and the v6 counterpart), we also need the original skb that triggers the response. Further, the sk must be a const here. So, we need a new tcp_extra_options_response_write callback to handle these cases. It's also cleaner IMO as these responses are quite different from tcp_options_write.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/net/tcp.h     | 13 +++++++++++--
 net/ipv4/tcp.c        | 22 +++++++++++++++++++++-
 net/ipv4/tcp_ipv4.c   |  4 ++--
 net/ipv4/tcp_output.c |  2 +-
 net/ipv6/tcp_ipv6.c   |  2 +-
 5 files changed, 36 insertions(+), 7 deletions(-)
include/net/tcp.h +11 -2
index fff0959dfd91..0981a1b429ca 100644
@@ -2081,7 +2081,11 @@ struct tcp_extra_option_ops {
 				struct tcp_out_options *opts,
 				const struct sock *sk);
 	void (*write)(__be32 *ptr, struct sk_buff *skb,
-		      struct tcp_out_options *opts, const struct sock *sk);
+		      struct tcp_out_options *opts, struct sock *sk);
+	void (*response_write)(__be32 *ptr, struct sk_buff *orig,
+			       struct tcphdr *th,
+			       struct tcp_out_options *opts,
+			       const struct sock *sk);
 	int (*add_header_len)(const struct sock *listener,
 			      const struct sock *sk);
 	struct module		*owner;
@@ -2099,7 +2103,12 @@ unsigned int tcp_extra_options_prepare(struct sk_buff *skb, u8 flags,
 
 void tcp_extra_options_write(__be32 *ptr, struct sk_buff *skb,
 			     struct tcp_out_options *opts,
-			     const struct sock *sk);
+			     struct sock *sk);
+
+void tcp_extra_options_response_write(__be32 *ptr, struct sk_buff *orig,
+				      struct tcphdr *th,
+				      struct tcp_out_options *opts,
+				      const struct sock *sk);
 
 int tcp_extra_options_add_header(const struct sock *listener,
 				 const struct sock *sk);
net/ipv4/tcp.c +21 -1
index 155bee4659fa..487f28222d3c 100644
@@ -3539,7 +3539,7 @@ EXPORT_SYMBOL_GPL(tcp_extra_options_prepare);
  */
 void tcp_extra_options_write(__be32 *ptr, struct sk_buff *skb,
 			     struct tcp_out_options *opts,
-			     const struct sock *sk)
+			     struct sock *sk)
 {
 	struct tcp_extra_option_ops *entry;
 
@@ -3552,6 +3552,26 @@ void tcp_extra_options_write(__be32 *ptr, struct sk_buff *skb,
 }
 EXPORT_SYMBOL_GPL(tcp_extra_options_write);
 
+/* The RCU read lock must be held before calling, and should span both
+ * the call to tcp_extra_options_write and this function to ensure that
+ * tcp_option_list does not change between the two calls.
+ */
+void tcp_extra_options_response_write(__be32 *ptr, struct sk_buff *orig,
+				      struct tcphdr *th,
+				      struct tcp_out_options *opts,
+				      const struct sock *sk)
+{
+	struct tcp_extra_option_ops *entry;
+
+	list_for_each_entry_rcu(entry, &tcp_option_list, list) {
+		if (unlikely(!entry->write))
+			continue;
+
+		entry->response_write(ptr, orig, th, opts, sk);
+	}
+}
+EXPORT_SYMBOL_GPL(tcp_extra_options_response_write);
+
 int tcp_extra_options_add_header(const struct sock *listener,
 				 const struct sock *sk)
 {
net/ipv4/tcp_ipv4.c +2 -2
index ccbb219c7b61..f717069ad778 100644
@@ -696,7 +696,7 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 		used = tcp_extra_options_prepare(NULL, TCPHDR_RST, remaining,
 						 &opts, sk);
 
-		tcp_extra_options_write(&rep.opt[0], NULL, &opts, sk);
+		tcp_extra_options_response_write(&rep.opt[0], skb, &rep.th, &opts, sk);
 		rcu_read_unlock();
 
 		arg.iov[0].iov_len += used;
@@ -829,7 +829,7 @@ static void tcp_v4_send_ack(const struct sock *sk,
 		used = tcp_extra_options_prepare(NULL, TCPHDR_ACK, remaining,
 						 &opts, sk);
 
-		tcp_extra_options_write(&rep.opt[offset], NULL, &opts, sk);
+		tcp_extra_options_response_write(&rep.opt[offset], skb, &rep.th, &opts, sk);
 		rcu_read_unlock();
 
 		arg.iov[0].iov_len += used;
net/ipv4/tcp_output.c +1 -1
index 199a8baf281c..1fd331018e58 100644
@@ -527,7 +527,7 @@ static void tcp_options_write(__be32 *ptr, struct sk_buff *skb, struct sock *sk,
 	smc_options_write(ptr, &options);
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
-		tcp_extra_options_write(ptr, skb, opts, tcp_to_sk(tp));
+		tcp_extra_options_write(ptr, skb, opts, sk);
 }
 
 static void smc_set_option(const struct tcp_sock *tp,
net/ipv6/tcp_ipv6.c +1 -1
index 7e4e8788943f..ef037990ea5e 100644
@@ -896,7 +896,7 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
-		tcp_extra_options_write(topt, buff, &extraopts, sk);
+		tcp_extra_options_response_write(topt, skb, t1, &extraopts, sk);
 
 	memset(&fl6, 0, sizeof(fl6));
 	fl6.daddr = ipv6_hdr(skb)->saddr;
Signature · 1 line
2.15.0
author
#18 lore raw

PATCH v2 11/25 tcp_extra_options: Make extra-option list per-socket

In reply to Christoph Paasch

Each TCP-socket, TCP request-socket and TCP time-wait socket has its own list. That way we can send the proper options in each state.

The list is copied across the different socket-types when needed.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/linux/tcp.h      |   5 ++
 include/net/tcp.h        |  49 +++++++++--
 net/ipv4/tcp.c           | 214 ++++++++++++++++++++++++++++++-----------------
 net/ipv4/tcp_input.c     |   5 ++
 net/ipv4/tcp_ipv4.c      |  14 +++-
 net/ipv4/tcp_minisocks.c |  10 +++
 net/ipv6/tcp_ipv6.c      |   9 ++
 7 files changed, 219 insertions(+), 87 deletions(-)
include/linux/tcp.h +5 -0
index a2a1676dfc52..4f653e3bd33f 100644
@@ -156,6 +156,7 @@ struct tcp_request_sock {
 						  * FastOpen it's the seq#
 						  * after data-in-SYN.
 						  */
+	struct list_head		tcp_option_list;
 };
 
 static inline struct tcp_request_sock *tcp_rsk(const struct request_sock *req)
@@ -391,6 +392,8 @@ struct tcp_sock {
 	 */
 	struct request_sock *fastopen_rsk;
 	u32	*saved_syn;
+
+	struct list_head tcp_option_list;
 };
 
 enum tsq_enum {
@@ -435,6 +438,8 @@ struct tcp_timewait_sock {
 	u32			  tw_last_oow_ack_time;
 
 	long			  tw_ts_recent_stamp;
+
+	struct list_head	  tcp_option_list;
 #ifdef CONFIG_TCP_MD5SIG
 	struct tcp_md5sig_key	  *tw_md5_key;
 #endif
include/net/tcp.h +41 -8

79 lines not on this page · raw message

net/ipv4/tcp.c +139 -75

326 lines not on this page · raw message

net/ipv4/tcp_input.c +5 -0
index 028da0a4eff5..4e84f299c96f 100644
@@ -6133,6 +6133,7 @@ static void tcp_openreq_init(struct request_sock *req,
 #if IS_ENABLED(CONFIG_SMC)
 	ireq->smc_ok = rx_opt->smc_ok;
 #endif
+	INIT_LIST_HEAD(&tcp_rsk(req)->tcp_option_list);
 }
 
 struct request_sock *inet_reqsk_alloc(const struct request_sock_ops *ops,
@@ -6309,6 +6310,10 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,
 		tcp_reqsk_record_syn(sk, req, skb);
 		fastopen_sk = tcp_try_fastopen(sk, skb, req, &foc, dst);
 	}
+
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		tcp_extra_options_copy(sk, req, &tmp_opt);
+
 	if (fastopen_sk) {
 		af_ops->send_synack(fastopen_sk, dst, &fl, req,
 				    &foc, TCP_SYNACK_FASTOPEN);
net/ipv4/tcp_ipv4.c +10 -4
index f717069ad778..4a47bfd12662 100644
@@ -692,12 +692,10 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 
 		memset(&opts, 0, sizeof(opts));
 
-		rcu_read_lock();
 		used = tcp_extra_options_prepare(NULL, TCPHDR_RST, remaining,
 						 &opts, sk);
 
 		tcp_extra_options_response_write(&rep.opt[0], skb, &rep.th, &opts, sk);
-		rcu_read_unlock();
 
 		arg.iov[0].iov_len += used;
 		offset += used / 4;
@@ -825,12 +823,10 @@ static void tcp_v4_send_ack(const struct sock *sk,
 #endif
 
 		memset(&opts, 0, sizeof(opts));
-		rcu_read_lock();
 		used = tcp_extra_options_prepare(NULL, TCPHDR_ACK, remaining,
 						 &opts, sk);
 
 		tcp_extra_options_response_write(&rep.opt[offset], skb, &rep.th, &opts, sk);
-		rcu_read_unlock();
 
 		arg.iov[0].iov_len += used;
 		offset += used / 4;
@@ -952,6 +948,9 @@ static int tcp_v4_send_synack(const struct sock *sk, struct dst_entry *dst,
  */
 static void tcp_v4_reqsk_destructor(struct request_sock *req)
 {
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		tcp_extra_options_destroy(req_to_sk(req));
+
 	kfree(rcu_dereference_protected(inet_rsk(req)->ireq_opt, 1));
 }
 
@@ -1446,6 +1445,11 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
 
 	tcp_initialize_rcv_mss(newsk);
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
+		tcp_extra_options_move(&tcp_rsk(req)->tcp_option_list,
+				       &newtp->tcp_option_list);
+		INIT_LIST_HEAD(&tcp_rsk(req)->tcp_option_list);
+	}
 #ifdef CONFIG_TCP_MD5SIG
 	/* Copy over the MD5 key from the original socket */
 	key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&newinet->inet_daddr,
@@ -1945,6 +1949,8 @@ void tcp_v4_destroy_sock(struct sock *sk)
 	/* Cleans up our, hopefully empty, out_of_order_queue. */
 	skb_rbtree_purge(&tp->out_of_order_queue);
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		tcp_extra_options_destroy(sk);
 #ifdef CONFIG_TCP_MD5SIG
 	/* Clean up the MD5 key list, if any */
 	if (tp->md5sig_info) {
net/ipv4/tcp_minisocks.c +10 -0
index 7f9403d3272c..fb50d3d0d158 100644
@@ -271,6 +271,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 		tcptw->tw_ts_recent_stamp = tp->rx_opt.ts_recent_stamp;
 		tcptw->tw_ts_offset	= tp->tsoffset;
 		tcptw->tw_last_oow_ack_time = 0;
+		INIT_LIST_HEAD(&tcptw->tcp_option_list);
 
 #if IS_ENABLED(CONFIG_IPV6)
 		if (tw->tw_family == PF_INET6) {
@@ -284,6 +285,11 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 		}
 #endif
 
+		if (static_branch_unlikely(&tcp_extra_options_enabled)) {
+			tcp_extra_options_move(&tcp_sk(sk)->tcp_option_list,
+					       &tcptw->tcp_option_list);
+			INIT_LIST_HEAD(&tcp_sk(sk)->tcp_option_list);
+		}
 #ifdef CONFIG_TCP_MD5SIG
 		/*
 		 * The timewait bucket does not have the key DB from the
@@ -334,6 +340,9 @@ void tcp_twsk_destructor(struct sock *sk)
 	if (twsk->tw_md5_key)
 		kfree_rcu(twsk->tw_md5_key, rcu);
 #endif
+
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		tcp_extra_options_destroy(sk);
 }
 EXPORT_SYMBOL_GPL(tcp_twsk_destructor);
 
@@ -463,6 +472,7 @@ struct sock *tcp_create_openreq_child(const struct sock *sk,
 
 		INIT_LIST_HEAD(&newtp->tsq_node);
 		INIT_LIST_HEAD(&newtp->tsorted_sent_queue);
+		INIT_LIST_HEAD(&newtp->tcp_option_list);
 
 		tcp_init_wl(newtp, treq->rcv_isn);
 
net/ipv6/tcp_ipv6.c +9 -0
index ef037990ea5e..0a29ef82db35 100644
@@ -500,6 +500,9 @@ static int tcp_v6_send_synack(const struct sock *sk, struct dst_entry *dst,
 
 static void tcp_v6_reqsk_destructor(struct request_sock *req)
 {
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		tcp_extra_options_destroy(req_to_sk(req));
+
 	kfree(inet_rsk(req)->ipv6_opt);
 	kfree_skb(inet_rsk(req)->pktopts);
 }
@@ -1215,6 +1218,12 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
 	newinet->inet_daddr = newinet->inet_saddr = LOOPBACK4_IPV6;
 	newinet->inet_rcv_saddr = LOOPBACK4_IPV6;
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
+		tcp_extra_options_move(&tcp_rsk(req)->tcp_option_list,
+				       &newtp->tcp_option_list);
+		INIT_LIST_HEAD(&tcp_rsk(req)->tcp_option_list);
+	}
+
 #ifdef CONFIG_TCP_MD5SIG
 	/* Copy over the MD5 key from the original socket */
 	key = tcp_v6_md5_do_lookup(sk, &newsk->sk_v6_daddr);
Signature · 1 line
2.15.0
#19 lore raw

Re: [MPTCP] [PATCH v2 11/25] tcp_extra_options: Make extra-option list per-socket

Quoted 258 lines · Christoph Paasch

Each TCP-socket, TCP request-socket and TCP time-wait socket has its own list. That way we can send the proper options in each state.

The list is copied across the different socket-types when needed.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---

include/linux/tcp.h      |   5 ++
include/net/tcp.h        |  49 +++++++++--
net/ipv4/tcp.c           | 214 ++++++++++++++++++++++++++++++-----------------
net/ipv4/tcp_input.c     |   5 ++
net/ipv4/tcp_ipv4.c      |  14 +++-
net/ipv4/tcp_minisocks.c |  10 +++
net/ipv6/tcp_ipv6.c      |   9 ++
7 files changed, 219 insertions(+), 87 deletions(-)
include/linux/tcp.h +5 -0
index a2a1676dfc52..4f653e3bd33f 100644
@@ -156,6 +156,7 @@ struct tcp_request_sock {
 						  * FastOpen it's the seq#
 						  * after data-in-SYN.
 						  */
+	struct list_head		tcp_option_list;
 };
 
 static inline struct tcp_request_sock *tcp_rsk(const struct request_sock *req)
@@ -391,6 +392,8 @@ struct tcp_sock {
 	 */
 	struct request_sock *fastopen_rsk;
 	u32	*saved_syn;
+
+	struct list_head tcp_option_list;
 };
 
 enum tsq_enum {
@@ -435,6 +438,8 @@ struct tcp_timewait_sock {
 	u32			  tw_last_oow_ack_time;
 
 	long			  tw_ts_recent_stamp;
+
+	struct list_head	  tcp_option_list;
 #ifdef CONFIG_TCP_MD5SIG
 	struct tcp_md5sig_key	  *tw_md5_key;
 #endif
include/net/tcp.h +41 -8
index 0981a1b429ca..1f34b1708990 100644
@@ -2066,31 +2066,56 @@ extern struct static_key_false tcp_have_smc;
 #endif
 
 extern struct static_key_false tcp_extra_options_enabled;
+struct tcp_extra_option_store;
 
 struct tcp_extra_option_ops {
-	struct list_head	list;
 	unsigned char		option_kind;
 	unsigned char		priority;
 	void (*parse)(int opsize, const unsigned char *opptr,
 		      const struct sk_buff *skb,
 		      struct tcp_options_received *opt_rx,
-		      struct sock *sk);
+		      struct sock *sk,
+		      struct tcp_extra_option_store *store);
 	/* Return the number of bytes consumed */
 	unsigned int (*prepare)(struct sk_buff *skb, u8 flags,
 				unsigned int remaining,
 				struct tcp_out_options *opts,
-				const struct sock *sk);
+				const struct sock *sk,
+				struct tcp_extra_option_store *store);
 	void (*write)(__be32 *ptr, struct sk_buff *skb,
-		      struct tcp_out_options *opts, struct sock *sk);
+		      struct tcp_out_options *opts, struct sock *sk,
+		      struct tcp_extra_option_store *store);
 	void (*response_write)(__be32 *ptr, struct sk_buff *orig,
 			       struct tcphdr *th,
 			       struct tcp_out_options *opts,
-			       const struct sock *sk);
+			       const struct sock *sk,
+			       struct tcp_extra_option_store *store);
 	int (*add_header_len)(const struct sock *listener,
-			      const struct sock *sk);
+			      const struct sock *sk,
+			      struct tcp_extra_option_store *store);
+	struct tcp_extra_option_store *(*copy)(struct sock *listener,
+					       struct request_sock *req,
+					       struct tcp_options_received *opt,
+					       struct tcp_extra_option_store *from);
+	struct tcp_extra_option_store *(*move)(struct tcp_extra_option_store *from);
+	void (*destroy)(struct tcp_extra_option_store *store);
 	struct module		*owner;
 };
 
+/* Every extra-option in a TCP-socket gets stored in a _store structure which
+ * has to have tcp_extra_option_store as the first element.
+ *
+ * That way, an extra option registers passes this to tcp_register_extra_option,
+ * and the surrounding structure can contain per-option state.
+ */
+struct tcp_extra_option_store {
+	struct list_head		list;
+	const struct tcp_extra_option_ops	*ops;
+};
+
+struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
+							   const struct sock *sk);
+
 void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
 			     const struct sk_buff *skb,
 			     struct tcp_options_received *opt_rx,
@@ -2113,7 +2138,15 @@ void tcp_extra_options_response_write(__be32 *ptr, struct sk_buff *orig,
 int tcp_extra_options_add_header(const struct sock *listener,
 				 const struct sock *sk);
 
-int tcp_register_extra_option(struct tcp_extra_option_ops *ops);
-void tcp_unregister_extra_option(struct tcp_extra_option_ops *ops);
+int tcp_register_extra_option(struct tcp_extra_option_store *store,
+			      struct sock *sk);
+
+void tcp_extra_options_copy(struct sock *listener,
+			    struct request_sock *req,
+			    struct tcp_options_received *opt);
+
+void tcp_extra_options_move(struct list_head *from, struct list_head *to);
+
+void tcp_extra_options_destroy(struct sock *sk);
 
 #endif	/* _TCP_H */
net/ipv4/tcp.c +40 -31
index 487f28222d3c..01e646873613 100644
@@ -308,8 +308,6 @@ EXPORT_SYMBOL(tcp_sockets_allocated);
 /*
 * Optional TCP option handlers
 */
-static DEFINE_SPINLOCK(tcp_option_list_lock);
-static LIST_HEAD(tcp_option_list);
 DEFINE_STATIC_KEY_FALSE(tcp_extra_options_enabled);
 
 /*
@@ -423,6 +421,7 @@ void tcp_init_sock(struct sock *sk)
 	tcp_init_xmit_timers(sk);
 	INIT_LIST_HEAD(&tp->tsq_node);
 	INIT_LIST_HEAD(&tp->tsorted_sent_queue);
+	INIT_LIST_HEAD(&tp->tcp_option_list);
 
 	icsk->icsk_rto = TCP_TIMEOUT_INIT;
 	tp->mdev_us = jiffies_to_usecs(TCP_TIMEOUT_INIT);
@@ -3479,19 +3478,32 @@ EXPORT_SYMBOL(tcp_md5_hash_key);
 
 #endif
 
-/* Linear search, few entries are expected. The RCU read lock must
- * be held before calling.
- */
-static struct tcp_extra_option_ops *tcp_extra_options_find_kind(unsigned char kind)
+static struct list_head *tcp_extra_options_get_list(const struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;
+	if (sk_fullsock(sk))
+		return &tcp_sk(sk)->tcp_option_list;
+	else if (sk->sk_state == TCP_NEW_SYN_RECV)
+		return &tcp_rsk(inet_reqsk(sk))->tcp_option_list;
+	else if (sk->sk_state == TCP_TIME_WAIT)
+		return &tcp_twsk(sk)->tcp_option_list;
+
+	return NULL;
+}
+
+struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
+							   const struct sock *sk)
+{
+	struct tcp_extra_option_store *entry;
+	struct list_head *lhead;
+
+	lhead = tcp_extra_options_get_list(sk);
 
-	list_for_each_entry_rcu(entry, &tcp_option_list, list) {
-		if (entry->option_kind == kind)
-			return entry;
+	list_for_each_entry(entry, lhead, list) {
+		if (entry->ops->option_kind == kind)
+			break;
 	}
 
-	return NULL;
+	return entry;
 }
 
 void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
@@ -3499,75 +3511,79 @@ void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
 			     struct tcp_options_received *opt_rx,
 			     struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;
+	struct tcp_extra_option_store *entry;
+
+	entry = tcp_extra_options_find_kind(opcode, sk);
 
-	rcu_read_lock();
-	entry = tcp_extra_options_find_kind(opcode);
-	if (entry && entry->parse)
-		entry->parse(opsize, opptr, skb, opt_rx, sk);
-	rcu_read_unlock();
+	if (entry && entry->ops->parse)
+		entry->ops->parse(opsize, opptr, skb, opt_rx, sk, entry);
 }
 EXPORT_SYMBOL_GPL(tcp_extra_options_parse);
 
-/* The RCU read lock must be held before calling, and should span both
- * the call to this function and tcp_extra_options_write to ensure that
- * tcp_option_list does not change between the two calls. To preserve
- * expected option alignment, always returns a multiple of 4 bytes.
- */
 unsigned int tcp_extra_options_prepare(struct sk_buff *skb, u8 flags,
 				       unsigned int remaining,
 				       struct tcp_out_options *opts,
 				       const struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;
+	struct tcp_extra_option_store *entry;
+	struct list_head *lhead;
 	unsigned int used = 0;
 
-	list_for_each_entry_rcu(entry, &tcp_option_list, list) {
-		if (unlikely(!entry->prepare))
+	if (!sk)
+		return 0;
+
+	lhead = tcp_extra_options_get_list(sk);
+
+	list_for_each_entry(entry, lhead, list) {
+		if (unlikely(!entry->ops->prepare))
 			continue;
 
-		used += entry->prepare(skb, flags, remaining - used, opts, sk);
+		used += entry->ops->prepare(skb, flags, remaining - used, opts,
+					    sk, entry);
 	}
 
 	return roundup(used, 4);
 }
 EXPORT_SYMBOL_GPL(tcp_extra_options_prepare);
 
-/* The RCU read lock must be held before calling, and should span both
- * the call to tcp_extra_options_write and this function to ensure that
- * tcp_option_list does not change between the two calls.
- */
 void tcp_extra_options_write(__be32 *ptr, struct sk_buff *skb,
 			     struct tcp_out_options *opts,
 			     struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;
+	struct tcp_extra_option_store *entry;

net/ipv4/tcp.c void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr, · line 3561

Quoted 10 lines
net/ipv4/tcp.c +7 -2
+	struct list_head *lhead;
+
+	if (!sk)
+		return;
 
-	list_for_each_entry_rcu(entry, &tcp_option_list, list) {
-		if (unlikely(!entry->write))
+	lhead = tcp_extra_options_get_list(sk);
+
+	list_for_each_entry_rcu(entry, lhead, list) {

An _rcu iterator snuck in here.

Quoted 19 lines
net/ipv4/tcp.c +2 -6
+		if (unlikely(!entry->ops->write))
 			continue;
 
-		entry->write(ptr, skb, opts, sk);
+		entry->ops->write(ptr, skb, opts, sk, entry);
 	}
 }
 EXPORT_SYMBOL_GPL(tcp_extra_options_write);
 
-/* The RCU read lock must be held before calling, and should span both
- * the call to tcp_extra_options_write and this function to ensure that
- * tcp_option_list does not change between the two calls.
- */
 void tcp_extra_options_response_write(__be32 *ptr, struct sk_buff *orig,
 				      struct tcphdr *th,
 				      struct tcp_out_options *opts,
 				      const struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;

net/ipv4/tcp.c void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr, · line 3582

Quoted 10 lines
net/ipv4/tcp.c +7 -2
+	struct tcp_extra_option_store *entry;
 
-	list_for_each_entry_rcu(entry, &tcp_option_list, list) {
-		if (unlikely(!entry->write))
+	if (!sk)
+		return;
+
+	BUG_ON(!sk_fullsock(sk));
+
+	list_for_each_entry_rcu(entry, &tcp_sk(sk)->tcp_option_list, list) {

Another _rcu

Mat

Quoted 328 lines
net/ipv4/tcp.c +83 -34
+		if (unlikely(!entry->ops->response_write))
 			continue;
 
-		entry->response_write(ptr, orig, th, opts, sk);
+		entry->ops->response_write(ptr, orig, th, opts, sk, entry);
 	}
 }
 EXPORT_SYMBOL_GPL(tcp_extra_options_response_write);
@@ -3575,52 +3591,49 @@ EXPORT_SYMBOL_GPL(tcp_extra_options_response_write);
 int tcp_extra_options_add_header(const struct sock *listener,
 				 const struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;
+	struct tcp_extra_option_store *entry;
 	int tcp_header_len = 0;
 
-	rcu_read_lock();
-	list_for_each_entry_rcu(entry, &tcp_option_list, list) {
-		if (unlikely(!entry->add_header_len))
+	list_for_each_entry(entry, &tcp_sk(sk)->tcp_option_list, list) {
+		if (unlikely(!entry->ops->add_header_len))
 			continue;
 
-		tcp_header_len += entry->add_header_len(listener, sk);
+		tcp_header_len += entry->ops->add_header_len(listener, sk, entry);
 	}
-	rcu_read_unlock();
 
 	return tcp_header_len;
 }
 
-int tcp_register_extra_option(struct tcp_extra_option_ops *ops)
+int tcp_register_extra_option(struct tcp_extra_option_store *store,
+			      struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;
-	struct list_head *add_before = &tcp_option_list;
+	struct list_head *lhead, *add_before;
+	struct tcp_extra_option_store *entry;
 	int ret = 0;
 
-	if (!ops->option_kind)
+	lhead = tcp_extra_options_get_list(sk);
+	add_before = lhead;
+
+	if (!store->ops->option_kind)
 		return -EINVAL;
 
-	if (!try_module_get(ops->owner))
+	if (!try_module_get(store->ops->owner))
 		return -ENOENT;
 
-	spin_lock(&tcp_option_list_lock);
-
-	list_for_each_entry_rcu(entry, &tcp_option_list, list) {
-		if (entry->option_kind == ops->option_kind) {
+	list_for_each_entry(entry, lhead, list) {
+		if (entry->ops->option_kind == store->ops->option_kind) {
 			pr_notice("Option kind %u already registered\n",
-				  ops->option_kind);
-			spin_unlock(&tcp_option_list_lock);
-			module_put(ops->owner);
+				  store->ops->option_kind);
+			module_put(store->ops->owner);
 			return -EEXIST;
 		}
 
-		if (entry->priority <= ops->priority)
+		if (entry->ops->priority <= store->ops->priority)
 			add_before = &entry->list;
 	}
 
-	list_add_tail_rcu(&ops->list, add_before);
-	pr_debug("Option kind %u registered\n", ops->option_kind);
-
-	spin_unlock(&tcp_option_list_lock);
+	list_add_tail(&store->list, add_before);
+	pr_debug("Option kind %u registered\n", store->ops->option_kind);
 
 	static_branch_inc(&tcp_extra_options_enabled);
 
@@ -3628,19 +3641,72 @@ int tcp_register_extra_option(struct tcp_extra_option_ops *ops)
 }
 EXPORT_SYMBOL_GPL(tcp_register_extra_option);
 
-void tcp_unregister_extra_option(struct tcp_extra_option_ops *ops)
+void tcp_extra_options_copy(struct sock *listener,
+			    struct request_sock *req,
+			    struct tcp_options_received *opt)
+{
+	struct tcp_extra_option_store *entry;
+	struct list_head *from, *to;
+
+	from = tcp_extra_options_get_list(listener);
+	to = tcp_extra_options_get_list(req_to_sk(req));
+
+	list_for_each_entry(entry, from, list) {
+		struct tcp_extra_option_store *new;
+
+		if (!try_module_get(entry->ops->owner)) {
+			pr_err("%s Module get failed while copying\n", __func__);
+			continue;
+		}
+
+		new = entry->ops->copy(listener, req, opt, entry);
+		if (!new) {
+			module_put(entry->ops->owner);
+			continue;
+		}
+
+		list_add_tail(&new->list, to);
+
+		static_branch_inc(&tcp_extra_options_enabled);
+	}
+}
+
+void tcp_extra_options_move(struct list_head *from, struct list_head *to)
 {
-	spin_lock(&tcp_option_list_lock);
-	list_del_rcu(&ops->list);
-	spin_unlock(&tcp_option_list_lock);
+	struct tcp_extra_option_store *entry, *tmp;
 
-	synchronize_net();
+	list_for_each_entry_safe(entry, tmp, from, list) {
+		list_del(&entry->list);
 
-	static_branch_dec(&tcp_extra_options_enabled);
+		if (entry->ops->move) {
+			if (entry->ops->move(entry))
+				continue;
+		}
 
-	module_put(ops->owner);
+		list_add_tail(&entry->list, to);
+	}
 }
-EXPORT_SYMBOL_GPL(tcp_unregister_extra_option);
+EXPORT_SYMBOL_GPL(tcp_extra_options_move);
+
+void tcp_extra_options_destroy(struct sock *sk)
+{
+	struct tcp_extra_option_store *entry, *tmp;
+	struct list_head *lhead;
+
+	lhead = tcp_extra_options_get_list(sk);
+
+	list_for_each_entry_safe(entry, tmp, lhead, list) {
+		struct module *owner = entry->ops->owner;
+		list_del(&entry->list);
+
+		entry->ops->destroy(entry);
+
+		static_branch_dec(&tcp_extra_options_enabled);
+
+		module_put(owner);
+	}
+}
+EXPORT_SYMBOL_GPL(tcp_extra_options_destroy);
 
 void tcp_done(struct sock *sk)
 {
@@ -3788,8 +3854,6 @@ void __init tcp_init(void)
 		INIT_HLIST_HEAD(&tcp_hashinfo.bhash[i].chain);
 	}
 
-	INIT_LIST_HEAD(&tcp_option_list);
-
 	cnt = tcp_hashinfo.ehash_mask + 1;
 	sysctl_tcp_max_orphans = cnt / 2;
 
net/ipv4/tcp_input.c +5 -0
index 028da0a4eff5..4e84f299c96f 100644
@@ -6133,6 +6133,7 @@ static void tcp_openreq_init(struct request_sock *req,
 #if IS_ENABLED(CONFIG_SMC)
 	ireq->smc_ok = rx_opt->smc_ok;
 #endif
+	INIT_LIST_HEAD(&tcp_rsk(req)->tcp_option_list);
 }
 
 struct request_sock *inet_reqsk_alloc(const struct request_sock_ops *ops,
@@ -6309,6 +6310,10 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,
 		tcp_reqsk_record_syn(sk, req, skb);
 		fastopen_sk = tcp_try_fastopen(sk, skb, req, &foc, dst);
 	}
+
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		tcp_extra_options_copy(sk, req, &tmp_opt);
+
 	if (fastopen_sk) {
 		af_ops->send_synack(fastopen_sk, dst, &fl, req,
 				    &foc, TCP_SYNACK_FASTOPEN);
net/ipv4/tcp_ipv4.c +10 -4
index f717069ad778..4a47bfd12662 100644
@@ -692,12 +692,10 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 
 		memset(&opts, 0, sizeof(opts));
 
-		rcu_read_lock();
 		used = tcp_extra_options_prepare(NULL, TCPHDR_RST, remaining,
 						 &opts, sk);
 
 		tcp_extra_options_response_write(&rep.opt[0], skb, &rep.th, &opts, sk);
-		rcu_read_unlock();
 
 		arg.iov[0].iov_len += used;
 		offset += used / 4;
@@ -825,12 +823,10 @@ static void tcp_v4_send_ack(const struct sock *sk,
 #endif
 
 		memset(&opts, 0, sizeof(opts));
-		rcu_read_lock();
 		used = tcp_extra_options_prepare(NULL, TCPHDR_ACK, remaining,
 						 &opts, sk);
 
 		tcp_extra_options_response_write(&rep.opt[offset], skb, &rep.th, &opts, sk);
-		rcu_read_unlock();
 
 		arg.iov[0].iov_len += used;
 		offset += used / 4;
@@ -952,6 +948,9 @@ static int tcp_v4_send_synack(const struct sock *sk, struct dst_entry *dst,
 */
 static void tcp_v4_reqsk_destructor(struct request_sock *req)
 {
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		tcp_extra_options_destroy(req_to_sk(req));
+
 	kfree(rcu_dereference_protected(inet_rsk(req)->ireq_opt, 1));
 }
 
@@ -1446,6 +1445,11 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
 
 	tcp_initialize_rcv_mss(newsk);
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
+		tcp_extra_options_move(&tcp_rsk(req)->tcp_option_list,
+				       &newtp->tcp_option_list);
+		INIT_LIST_HEAD(&tcp_rsk(req)->tcp_option_list);
+	}
 #ifdef CONFIG_TCP_MD5SIG
 	/* Copy over the MD5 key from the original socket */
 	key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&newinet->inet_daddr,
@@ -1945,6 +1949,8 @@ void tcp_v4_destroy_sock(struct sock *sk)
 	/* Cleans up our, hopefully empty, out_of_order_queue. */
 	skb_rbtree_purge(&tp->out_of_order_queue);
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		tcp_extra_options_destroy(sk);
 #ifdef CONFIG_TCP_MD5SIG
 	/* Clean up the MD5 key list, if any */
 	if (tp->md5sig_info) {
net/ipv4/tcp_minisocks.c +10 -0
index 7f9403d3272c..fb50d3d0d158 100644
@@ -271,6 +271,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 		tcptw->tw_ts_recent_stamp = tp->rx_opt.ts_recent_stamp;
 		tcptw->tw_ts_offset	= tp->tsoffset;
 		tcptw->tw_last_oow_ack_time = 0;
+		INIT_LIST_HEAD(&tcptw->tcp_option_list);
 
 #if IS_ENABLED(CONFIG_IPV6)
 		if (tw->tw_family == PF_INET6) {
@@ -284,6 +285,11 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 		}
 #endif
 
+		if (static_branch_unlikely(&tcp_extra_options_enabled)) {
+			tcp_extra_options_move(&tcp_sk(sk)->tcp_option_list,
+					       &tcptw->tcp_option_list);
+			INIT_LIST_HEAD(&tcp_sk(sk)->tcp_option_list);
+		}
 #ifdef CONFIG_TCP_MD5SIG
 		/*
 		 * The timewait bucket does not have the key DB from the
@@ -334,6 +340,9 @@ void tcp_twsk_destructor(struct sock *sk)
 	if (twsk->tw_md5_key)
 		kfree_rcu(twsk->tw_md5_key, rcu);
 #endif
+
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		tcp_extra_options_destroy(sk);
 }
 EXPORT_SYMBOL_GPL(tcp_twsk_destructor);
 
@@ -463,6 +472,7 @@ struct sock *tcp_create_openreq_child(const struct sock *sk,
 
 		INIT_LIST_HEAD(&newtp->tsq_node);
 		INIT_LIST_HEAD(&newtp->tsorted_sent_queue);
+		INIT_LIST_HEAD(&newtp->tcp_option_list);
 
 		tcp_init_wl(newtp, treq->rcv_isn);
 
net/ipv6/tcp_ipv6.c +9 -0
index ef037990ea5e..0a29ef82db35 100644
@@ -500,6 +500,9 @@ static int tcp_v6_send_synack(const struct sock *sk, struct dst_entry *dst,
 
 static void tcp_v6_reqsk_destructor(struct request_sock *req)
 {
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		tcp_extra_options_destroy(req_to_sk(req));
+
 	kfree(inet_rsk(req)->ipv6_opt);
 	kfree_skb(inet_rsk(req)->pktopts);
 }
@@ -1215,6 +1218,12 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
 	newinet->inet_daddr = newinet->inet_saddr = LOOPBACK4_IPV6;
 	newinet->inet_rcv_saddr = LOOPBACK4_IPV6;
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
+		tcp_extra_options_move(&tcp_rsk(req)->tcp_option_list,
+				       &newtp->tcp_option_list);
+		INIT_LIST_HEAD(&tcp_rsk(req)->tcp_option_list);
+	}
+
 #ifdef CONFIG_TCP_MD5SIG
 	/* Copy over the MD5 key from the original socket */
 	key = tcp_v6_md5_do_lookup(sk, &newsk->sk_v6_daddr);
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#20 lore raw

Re: [MPTCP] [PATCH v2 11/25] tcp_extra_options: Make extra-option list per-socket

Quoted 294 lines · Mat Martineau
Quoted 258 lines · Christoph Paasch

Each TCP-socket, TCP request-socket and TCP time-wait socket has its own list. That way we can send the proper options in each state.

The list is copied across the different socket-types when needed.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---

include/linux/tcp.h      |   5 ++
include/net/tcp.h        |  49 +++++++++--
net/ipv4/tcp.c           | 214 ++++++++++++++++++++++++++++++-----------------
net/ipv4/tcp_input.c     |   5 ++
net/ipv4/tcp_ipv4.c      |  14 +++-
net/ipv4/tcp_minisocks.c |  10 +++
net/ipv6/tcp_ipv6.c      |   9 ++
7 files changed, 219 insertions(+), 87 deletions(-)
include/linux/tcp.h +5 -0
index a2a1676dfc52..4f653e3bd33f 100644
@@ -156,6 +156,7 @@ struct tcp_request_sock {
 						  * FastOpen it's the seq#
 						  * after data-in-SYN.
 						  */
+	struct list_head		tcp_option_list;
 };
 
 static inline struct tcp_request_sock *tcp_rsk(const struct request_sock *req)
@@ -391,6 +392,8 @@ struct tcp_sock {
 	 */
 	struct request_sock *fastopen_rsk;
 	u32	*saved_syn;
+
+	struct list_head tcp_option_list;
 };
 
 enum tsq_enum {
@@ -435,6 +438,8 @@ struct tcp_timewait_sock {
 	u32			  tw_last_oow_ack_time;
 
 	long			  tw_ts_recent_stamp;
+
+	struct list_head	  tcp_option_list;
 #ifdef CONFIG_TCP_MD5SIG
 	struct tcp_md5sig_key	  *tw_md5_key;
 #endif
include/net/tcp.h +41 -8
index 0981a1b429ca..1f34b1708990 100644
@@ -2066,31 +2066,56 @@ extern struct static_key_false tcp_have_smc;
 #endif
 
 extern struct static_key_false tcp_extra_options_enabled;
+struct tcp_extra_option_store;
 
 struct tcp_extra_option_ops {
-	struct list_head	list;
 	unsigned char		option_kind;
 	unsigned char		priority;
 	void (*parse)(int opsize, const unsigned char *opptr,
 		      const struct sk_buff *skb,
 		      struct tcp_options_received *opt_rx,
-		      struct sock *sk);
+		      struct sock *sk,
+		      struct tcp_extra_option_store *store);
 	/* Return the number of bytes consumed */
 	unsigned int (*prepare)(struct sk_buff *skb, u8 flags,
 				unsigned int remaining,
 				struct tcp_out_options *opts,
-				const struct sock *sk);
+				const struct sock *sk,
+				struct tcp_extra_option_store *store);
 	void (*write)(__be32 *ptr, struct sk_buff *skb,
-		      struct tcp_out_options *opts, struct sock *sk);
+		      struct tcp_out_options *opts, struct sock *sk,
+		      struct tcp_extra_option_store *store);
 	void (*response_write)(__be32 *ptr, struct sk_buff *orig,
 			       struct tcphdr *th,
 			       struct tcp_out_options *opts,
-			       const struct sock *sk);
+			       const struct sock *sk,
+			       struct tcp_extra_option_store *store);
 	int (*add_header_len)(const struct sock *listener,
-			      const struct sock *sk);
+			      const struct sock *sk,
+			      struct tcp_extra_option_store *store);
+	struct tcp_extra_option_store *(*copy)(struct sock *listener,
+					       struct request_sock *req,
+					       struct tcp_options_received *opt,
+					       struct tcp_extra_option_store *from);
+	struct tcp_extra_option_store *(*move)(struct tcp_extra_option_store *from);
+	void (*destroy)(struct tcp_extra_option_store *store);
 	struct module		*owner;
 };
 
+/* Every extra-option in a TCP-socket gets stored in a _store structure which
+ * has to have tcp_extra_option_store as the first element.
+ *
+ * That way, an extra option registers passes this to tcp_register_extra_option,
+ * and the surrounding structure can contain per-option state.
+ */
+struct tcp_extra_option_store {
+	struct list_head		list;
+	const struct tcp_extra_option_ops	*ops;
+};
+
+struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
+							   const struct sock *sk);
+
 void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
 			     const struct sk_buff *skb,
 			     struct tcp_options_received *opt_rx,
@@ -2113,7 +2138,15 @@ void tcp_extra_options_response_write(__be32 *ptr, struct sk_buff *orig,
 int tcp_extra_options_add_header(const struct sock *listener,
 				 const struct sock *sk);
 
-int tcp_register_extra_option(struct tcp_extra_option_ops *ops);
-void tcp_unregister_extra_option(struct tcp_extra_option_ops *ops);
+int tcp_register_extra_option(struct tcp_extra_option_store *store,
+			      struct sock *sk);
+
+void tcp_extra_options_copy(struct sock *listener,
+			    struct request_sock *req,
+			    struct tcp_options_received *opt);
+
+void tcp_extra_options_move(struct list_head *from, struct list_head *to);
+
+void tcp_extra_options_destroy(struct sock *sk);
 
 #endif	/* _TCP_H */
net/ipv4/tcp.c +40 -31
index 487f28222d3c..01e646873613 100644
@@ -308,8 +308,6 @@ EXPORT_SYMBOL(tcp_sockets_allocated);
 /*
 * Optional TCP option handlers
 */
-static DEFINE_SPINLOCK(tcp_option_list_lock);
-static LIST_HEAD(tcp_option_list);
 DEFINE_STATIC_KEY_FALSE(tcp_extra_options_enabled);
 
 /*
@@ -423,6 +421,7 @@ void tcp_init_sock(struct sock *sk)
 	tcp_init_xmit_timers(sk);
 	INIT_LIST_HEAD(&tp->tsq_node);
 	INIT_LIST_HEAD(&tp->tsorted_sent_queue);
+	INIT_LIST_HEAD(&tp->tcp_option_list);
 
 	icsk->icsk_rto = TCP_TIMEOUT_INIT;
 	tp->mdev_us = jiffies_to_usecs(TCP_TIMEOUT_INIT);
@@ -3479,19 +3478,32 @@ EXPORT_SYMBOL(tcp_md5_hash_key);
 
 #endif
 
-/* Linear search, few entries are expected. The RCU read lock must
- * be held before calling.
- */
-static struct tcp_extra_option_ops *tcp_extra_options_find_kind(unsigned char kind)
+static struct list_head *tcp_extra_options_get_list(const struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;
+	if (sk_fullsock(sk))
+		return &tcp_sk(sk)->tcp_option_list;
+	else if (sk->sk_state == TCP_NEW_SYN_RECV)
+		return &tcp_rsk(inet_reqsk(sk))->tcp_option_list;
+	else if (sk->sk_state == TCP_TIME_WAIT)
+		return &tcp_twsk(sk)->tcp_option_list;
+
+	return NULL;
+}
+
+struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
+							   const struct sock *sk)
+{
+	struct tcp_extra_option_store *entry;
+	struct list_head *lhead;
+
+	lhead = tcp_extra_options_get_list(sk);
 
-	list_for_each_entry_rcu(entry, &tcp_option_list, list) {
-		if (entry->option_kind == kind)
-			return entry;
+	list_for_each_entry(entry, lhead, list) {
+		if (entry->ops->option_kind == kind)
+			break;
 	}
 
-	return NULL;
+	return entry;
 }
 
 void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
@@ -3499,75 +3511,79 @@ void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
 			     struct tcp_options_received *opt_rx,
 			     struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;
+	struct tcp_extra_option_store *entry;
+
+	entry = tcp_extra_options_find_kind(opcode, sk);
 
-	rcu_read_lock();
-	entry = tcp_extra_options_find_kind(opcode);
-	if (entry && entry->parse)
-		entry->parse(opsize, opptr, skb, opt_rx, sk);
-	rcu_read_unlock();
+	if (entry && entry->ops->parse)
+		entry->ops->parse(opsize, opptr, skb, opt_rx, sk, entry);
 }
 EXPORT_SYMBOL_GPL(tcp_extra_options_parse);
 
-/* The RCU read lock must be held before calling, and should span both
- * the call to this function and tcp_extra_options_write to ensure that
- * tcp_option_list does not change between the two calls. To preserve
- * expected option alignment, always returns a multiple of 4 bytes.
- */
 unsigned int tcp_extra_options_prepare(struct sk_buff *skb, u8 flags,
 				       unsigned int remaining,
 				       struct tcp_out_options *opts,
 				       const struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;
+	struct tcp_extra_option_store *entry;
+	struct list_head *lhead;
 	unsigned int used = 0;
 
-	list_for_each_entry_rcu(entry, &tcp_option_list, list) {
-		if (unlikely(!entry->prepare))
+	if (!sk)
+		return 0;
+
+	lhead = tcp_extra_options_get_list(sk);
+
+	list_for_each_entry(entry, lhead, list) {
+		if (unlikely(!entry->ops->prepare))
 			continue;
 
-		used += entry->prepare(skb, flags, remaining - used, opts, sk);
+		used += entry->ops->prepare(skb, flags, remaining - used, opts,
+					    sk, entry);
 	}
 
 	return roundup(used, 4);
 }
 EXPORT_SYMBOL_GPL(tcp_extra_options_prepare);
 
-/* The RCU read lock must be held before calling, and should span both
- * the call to tcp_extra_options_write and this function to ensure that
- * tcp_option_list does not change between the two calls.
- */
 void tcp_extra_options_write(__be32 *ptr, struct sk_buff *skb,
 			     struct tcp_out_options *opts,
 			     struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;
+	struct tcp_extra_option_store *entry;

net/ipv4/tcp.c void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr, · line 3561

Quoted 10 lines
net/ipv4/tcp.c +7 -2
+	struct list_head *lhead;
+
+	if (!sk)
+		return;
 
-	list_for_each_entry_rcu(entry, &tcp_option_list, list) {
-		if (unlikely(!entry->write))
+	lhead = tcp_extra_options_get_list(sk);
+
+	list_for_each_entry_rcu(entry, lhead, list) {

An _rcu iterator snuck in here.

Quoted 21 lines
net/ipv4/tcp.c +3 -6
+		if (unlikely(!entry->ops->write))
 			continue;
 
-		entry->write(ptr, skb, opts, sk);
+		entry->ops->write(ptr, skb, opts, sk, entry);
 	}
 }
 EXPORT_SYMBOL_GPL(tcp_extra_options_write);
 
-/* The RCU read lock must be held before calling, and should span both
- * the call to tcp_extra_options_write and this function to ensure that
- * tcp_option_list does not change between the two calls.
- */
 void tcp_extra_options_response_write(__be32 *ptr, struct sk_buff *orig,
 				      struct tcphdr *th,
 				      struct tcp_out_options *opts,
 				      const struct sock *sk)
 {
-	struct tcp_extra_option_ops *entry;
+	struct tcp_extra_option_store *entry;
 
Quoted 10 lines

net/ipv4/tcp.c void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr, · line 3582

Quoted 8 lines
net/ipv4/tcp.c +6 -2
-	list_for_each_entry_rcu(entry, &tcp_option_list, list) {
-		if (unlikely(!entry->write))
+	if (!sk)
+		return;
+
+	BUG_ON(!sk_fullsock(sk));
+
+	list_for_each_entry_rcu(entry, &tcp_sk(sk)->tcp_option_list, list) {

Another _rcu

Thanks, fixed both.

Christoph

author
#21 lore raw

PATCH v2 12/25 tcp_extra_options: Export module fixes

In reply to Christoph Paasch

Otherwise compiling v6 in a module does not work and remove some export_symbol's that were not needed.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv4/tcp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
net/ipv4/tcp.c +1 -1
index 01e646873613..1afd56638854 100644
@@ -309,6 +309,7 @@ EXPORT_SYMBOL(tcp_sockets_allocated);
  * Optional TCP option handlers
  */
 DEFINE_STATIC_KEY_FALSE(tcp_extra_options_enabled);
+EXPORT_SYMBOL_GPL(tcp_extra_options_enabled);
 
 /*
  * TCP splice context
@@ -3518,7 +3519,6 @@ void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
 	if (entry && entry->ops->parse)
 		entry->ops->parse(opsize, opptr, skb, opt_rx, sk, entry);
 }
-EXPORT_SYMBOL_GPL(tcp_extra_options_parse);
 
 unsigned int tcp_extra_options_prepare(struct sk_buff *skb, u8 flags,
 				       unsigned int remaining,
Signature · 1 line
2.15.0
author
#22 lore raw

PATCH v2 13/25 tcp_extra_options: Add response_prepare callback

In reply to Christoph Paasch

In the same way as for the write-callback, having a separate one for the response-case is useful to separate the two use-cases.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/net/tcp.h   | 10 ++++++++++
 net/ipv4/tcp.c      | 34 +++++++++++++++++++++++++++++++++-
 net/ipv4/tcp_ipv4.c | 18 ++++++++++++------
 net/ipv6/tcp_ipv6.c | 10 ++++++++--
 4 files changed, 63 insertions(+), 9 deletions(-)
include/net/tcp.h +10 -0
index 1f34b1708990..0b94356ee5ba 100644
@@ -2085,6 +2085,11 @@ struct tcp_extra_option_ops {
 	void (*write)(__be32 *ptr, struct sk_buff *skb,
 		      struct tcp_out_options *opts, struct sock *sk,
 		      struct tcp_extra_option_store *store);
+	int (*response_prepare)(struct sk_buff *orig, u8 flags,
+				unsigned int remaining,
+				struct tcp_out_options *opts,
+				const struct sock *sk,
+				struct tcp_extra_option_store *store);
 	void (*response_write)(__be32 *ptr, struct sk_buff *orig,
 			       struct tcphdr *th,
 			       struct tcp_out_options *opts,
@@ -2130,6 +2135,11 @@ void tcp_extra_options_write(__be32 *ptr, struct sk_buff *skb,
 			     struct tcp_out_options *opts,
 			     struct sock *sk);
 
+int tcp_extra_options_response_prepare(struct sk_buff *orig, u8 flags,
+				       unsigned int remaining,
+				       struct tcp_out_options *opts,
+				       const struct sock *sk);
+
 void tcp_extra_options_response_write(__be32 *ptr, struct sk_buff *orig,
 				      struct tcphdr *th,
 				      struct tcp_out_options *opts,
net/ipv4/tcp.c +33 -1
index 1afd56638854..81b98a33530f 100644
@@ -3544,7 +3544,6 @@ unsigned int tcp_extra_options_prepare(struct sk_buff *skb, u8 flags,
 
 	return roundup(used, 4);
 }
-EXPORT_SYMBOL_GPL(tcp_extra_options_prepare);
 
 void tcp_extra_options_write(__be32 *ptr, struct sk_buff *skb,
 			     struct tcp_out_options *opts,
@@ -3567,6 +3566,39 @@ void tcp_extra_options_write(__be32 *ptr, struct sk_buff *skb,
 }
 EXPORT_SYMBOL_GPL(tcp_extra_options_write);
 
+int tcp_extra_options_response_prepare(struct sk_buff *orig, u8 flags,
+				       unsigned int remaining,
+				       struct tcp_out_options *opts,
+				       const struct sock *sk)
+{
+	struct tcp_extra_option_store *entry;
+	struct list_head *lhead;
+	unsigned int used = 0;
+
+	if (!sk)
+		return 0;
+
+	lhead = tcp_extra_options_get_list(sk);
+
+	list_for_each_entry(entry, lhead, list) {
+		int ret;
+
+		if (unlikely(!entry->ops->response_prepare))
+			continue;
+
+		ret = entry->ops->response_prepare(orig, flags,
+						   remaining - used, opts,
+						   sk, entry);
+		if (ret < 0)
+			return ret;
+
+		used += ret;
+	}
+
+	return roundup(used, 4);
+}
+EXPORT_SYMBOL_GPL(tcp_extra_options_response_prepare);
+
 void tcp_extra_options_response_write(__be32 *ptr, struct sk_buff *orig,
 				      struct tcphdr *th,
 				      struct tcp_out_options *opts,
net/ipv4/tcp_ipv4.c +12 -6
index 4a47bfd12662..f0db7beda04e 100644
@@ -681,8 +681,8 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
 		unsigned int remaining;
-		unsigned int used;
 		struct tcp_out_options opts;
+		int used;
 
 		remaining = sizeof(rep.opt);
 #ifdef CONFIG_TCP_MD5SIG
@@ -692,8 +692,11 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 
 		memset(&opts, 0, sizeof(opts));
 
-		used = tcp_extra_options_prepare(NULL, TCPHDR_RST, remaining,
-						 &opts, sk);
+		used = tcp_extra_options_response_prepare(skb, TCPHDR_RST,
+							  remaining,
+							  &opts, sk);
+		if (used < 0)
+			return;
 
 		tcp_extra_options_response_write(&rep.opt[0], skb, &rep.th, &opts, sk);
 
@@ -812,8 +815,8 @@ static void tcp_v4_send_ack(const struct sock *sk,
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
 		unsigned int remaining;
-		unsigned int used;
 		struct tcp_out_options opts;
+		int used;
 
 		remaining = sizeof(rep.th) + sizeof(rep.opt) - arg.iov[0].iov_len;
 
@@ -823,8 +826,11 @@ static void tcp_v4_send_ack(const struct sock *sk,
 #endif
 
 		memset(&opts, 0, sizeof(opts));
-		used = tcp_extra_options_prepare(NULL, TCPHDR_ACK, remaining,
-						 &opts, sk);
+		used = tcp_extra_options_response_prepare(skb, TCPHDR_ACK,
+							  remaining,
+							  &opts, sk);
+		if (used < 0)
+			return;
 
 		tcp_extra_options_response_write(&rep.opt[offset], skb, &rep.th, &opts, sk);
 
net/ipv6/tcp_ipv6.c +8 -2
index 0a29ef82db35..2fb688a36731 100644
@@ -848,14 +848,20 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
 		unsigned int remaining = MAX_TCP_OPTION_SPACE - tot_len;
 		u8 extraflags = rst ? TCPHDR_RST : 0;
+		int used;
 
 		if (!rst || !th->ack)
 			extraflags |= TCPHDR_ACK;
 
 		memset(&extraopts, 0, sizeof(extraopts));
 
-		tot_len += tcp_extra_options_prepare(skb, extraflags, remaining,
-						     &extraopts, sk);
+		used = tcp_extra_options_response_prepare(skb, extraflags,
+							  remaining,
+							  &extraopts, sk);
+		if (used < 0)
+			goto out;
+
+		tot_len += used;
 	}
 
 	buff = alloc_skb(MAX_HEADER + sizeof(struct ipv6hdr) + tot_len,
Signature · 1 line
2.15.0
author
#23 lore raw

PATCH v2 14/25 tcp_extra_options: Check static branch before _parse

In reply to Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv4/tcp_input.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
net/ipv4/tcp_input.c +5 -3
index 4e84f299c96f..7acc1f46641c 100644
@@ -3799,9 +3799,11 @@ void tcp_parse_options(const struct net *net,
 				break;
 
 			default:
-				tcp_extra_options_parse(opcode, opsize, ptr,
-							skb, opt_rx,
-							tcp_to_sk(tp));
+				if (static_branch_unlikely(&tcp_extra_options_enabled))
+					tcp_extra_options_parse(opcode, opsize,
+								ptr, skb,
+								opt_rx,
+								tcp_to_sk(tp));
 				break;
 
 			}
Signature · 1 line
2.15.0
#24 lore raw

Re: [MPTCP] [PATCH v2 14/25] tcp_extra_options: Check static branch before _parse

Quoted 10 lines · Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_input.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
net/ipv4/tcp_input.c +0 -0
index 4e84f299c96f..7acc1f46641c 100644

net/ipv4/tcp_input.c void tcp_parse_options(const struct net *net, · line 3802

Quoted 7 lines
net/ipv4/tcp_input.c +1 -3
@@ -3799,9 +3799,11 @@ void tcp_parse_options(const struct net *net,
 				break;
 
 			default:
-				tcp_extra_options_parse(opcode, opsize, ptr,
-							skb, opt_rx,
-							tcp_to_sk(tp));
+				if (static_branch_unlikely(&tcp_extra_options_enabled))

Since this case in the switch statement is only executed for unrecognized options, I had skipped the static branch in the initial implementation. It is more consistent to use the static branch, and there isn't much use in making the function call just to try to iterate over an empty list.

Quoted 11 lines
net/ipv4/tcp_input.c +4 -0
+					tcp_extra_options_parse(opcode, opsize,
+								ptr, skb,
+								opt_rx,
+								tcp_to_sk(tp));
 				break;
 
 			}
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

#25 lore raw

Re: [MPTCP] [PATCH v2 14/25] tcp_extra_options: Check static branch before _parse

Quoted 35 lines · Mat Martineau
Quoted 10 lines · Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_input.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
net/ipv4/tcp_input.c +0 -0
index 4e84f299c96f..7acc1f46641c 100644

net/ipv4/tcp_input.c void tcp_parse_options(const struct net *net, · line 3803

Quoted 8 lines
net/ipv4/tcp_input.c +1 -3
@@ -3799,9 +3799,11 @@ void tcp_parse_options(const struct net *net,
 				break;
 
 			default:
-				tcp_extra_options_parse(opcode, opsize, ptr,
-							skb, opt_rx,
-							tcp_to_sk(tp));
+				if 
 (static_branch_unlikely(&tcp_extra_options_enabled))

Since this case in the switch statement is only executed for unrecognized options, I had skipped the static branch in the initial implementation. It is more consistent to use the static branch, and there isn't much use in making the function call just to try to iterate over an empty list.

Quoted 8 lines
net/ipv4/tcp_input.c +4 -0
+					tcp_extra_options_parse(opcode, 
 opsize,
+								ptr, skb,
+								opt_rx,
+ 
 tcp_to_sk(tp));
				break;
Quoted 5 lines
Quoted 5 lines
			}
Signature · 1 line
2.15.0

I sent my reply and then realized I had one more point to make. We might need to take another look at the use of static branching by the extra options framework. It's one thing to inc/dec tcp_extra_options_enabled when modules are added or removed, but we don't want to be rapidly changing static branches as MPTCP sockets come and go. Does tcp_extra_options_enabled still make sense for per-socket extra_option lists?

--
Mat Martineau
Intel OTC

author
#26 lore raw

Re: [MPTCP] [PATCH v2 14/25] tcp_extra_options: Check static branch before _parse

Quoted 36 lines · Mat Martineau
Quoted 33 lines · Mat Martineau
Quoted 10 lines · Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_input.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
net/ipv4/tcp_input.c +0 -0
index 4e84f299c96f..7acc1f46641c 100644

net/ipv4/tcp_input.c void tcp_parse_options(const struct net *net, · line 3802

Quoted 7 lines
net/ipv4/tcp_input.c +1 -3
@@ -3799,9 +3799,11 @@ void tcp_parse_options(const struct net *net,
 				break;
 
 			default:
-				tcp_extra_options_parse(opcode, opsize, ptr,
-							skb, opt_rx,
-							tcp_to_sk(tp));
+				if (static_branch_unlikely(&tcp_extra_options_enabled))

Since this case in the switch statement is only executed for unrecognized options, I had skipped the static branch in the initial implementation. It is more consistent to use the static branch, and there isn't much use in making the function call just to try to iterate over an empty list.

Quoted 6 lines
net/ipv4/tcp_input.c +4 -0
+					tcp_extra_options_parse(opcode, opsize,
+								ptr, skb,
+								opt_rx,
+ tcp_to_sk(tp));
 				break;
 
Quoted 12 lines
Quoted 5 lines
Quoted 5 lines
net/ipv4/tcp_input.c +0 -0
 			}
Signature · 1 line
2.15.0

I sent my reply and then realized I had one more point to make. We might need to take another look at the use of static branching by the extra options framework. It's one thing to inc/dec tcp_extra_options_enabled when modules are added or removed, but we don't want to be rapidly changing static branches as MPTCP sockets come and go. Does tcp_extra_options_enabled still make sense for per-socket extra_option lists?

You think that the test for whether or not the tcp_option_list is emtpy is sufficient?

I don't know about the cost of static_branch_inc/dec. Is it that bad?

One thing is that having the static branches allows for sure to avoid any cost in the hot data-path. Which is definitely something netdev will want.

If we pay a cost for MPTCP, I think it's fine IMO. (at least in a first step ;))

Christoph

#27 lore raw

Re: [MPTCP] [PATCH v2 14/25] tcp_extra_options: Check static branch before _parse

Quoted 50 lines · Christoph Paasch
Quoted 36 lines · Mat Martineau
Quoted 33 lines · Mat Martineau
Quoted 10 lines · Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_input.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
net/ipv4/tcp_input.c +0 -0
index 4e84f299c96f..7acc1f46641c 100644

net/ipv4/tcp_input.c void tcp_parse_options(const struct net *net, · line 3802

Quoted 7 lines
net/ipv4/tcp_input.c +1 -3
@@ -3799,9 +3799,11 @@ void tcp_parse_options(const struct net *net,
 				break;
 
 			default:
-				tcp_extra_options_parse(opcode, opsize, ptr,
-							skb, opt_rx,
-							tcp_to_sk(tp));
+				if (static_branch_unlikely(&tcp_extra_options_enabled))

Since this case in the switch statement is only executed for unrecognized options, I had skipped the static branch in the initial implementation. It is more consistent to use the static branch, and there isn't much use in making the function call just to try to iterate over an empty list.

Quoted 6 lines
net/ipv4/tcp_input.c +4 -0
+					tcp_extra_options_parse(opcode, opsize,
+								ptr, skb,
+								opt_rx,
+ tcp_to_sk(tp));
 				break;
 
Quoted 12 lines
Quoted 5 lines
Quoted 5 lines
net/ipv4/tcp_input.c +0 -1
 			}
--

2.15.0

I sent my reply and then realized I had one more point to make. We might need to take another look at the use of static branching by the extra options framework. It's one thing to inc/dec tcp_extra_options_enabled when modules are added or removed, but we don't want to be rapidly changing static branches as MPTCP sockets come and go. Does tcp_extra_options_enabled still make sense for per-socket extra_option lists?

Quoted 2 lines

You think that the test for whether or not the tcp_option_list is emtpy is sufficient?

It might be if we wrap it in a "unlikely()".

Quoted 3 lines

I don't know about the cost of static_branch_inc/dec. Is it that bad?

Yeah, it's that bad. The kernel is patching machine code every time the key is enabled or disabled (rewriting no-ops with jump instructions and vice-versa).

"""
* Since this relies on modifying code, the branch modifying functions
* must be considered absolute slow paths (machine wide synchronization etc.).
"""

http://elixir.free-electrons.com/linux/v4.15-rc1/source/include/linux/jump_label.h#L56

Not just a slow path, but an *absolute* slow path!

static_key_deferred and static_key_slow_dec_deferred() are available in jump_label_ratelimit.h, which is one way to keep it from thrashing too much. This rate-limited static key is only used in one file (arch/x86/kvm/lapic.c). That subsystem limits jump label patching to once per second.

Quoted 6 lines

One thing is that having the static branches allows for sure to avoid any cost in the hot data-path. Which is definitely something netdev will want.

If we pay a cost for MPTCP, I think it's fine IMO. (at least in a first step ;))

It would be more straightforward if only MPTCP was paying the cost, but in this case the hit is systemwide if any extra TCP options are in use. The combination of a rate-limiting static_key and checking the option list before the call could be fast enough.

--
Mat Martineau
Intel OTC

author
#28 lore raw

Re: [MPTCP] [PATCH v2 14/25] tcp_extra_options: Check static branch before _parse

Quoted 72 lines · Mat Martineau
Quoted 50 lines · Christoph Paasch
Quoted 36 lines · Mat Martineau
Quoted 33 lines · Mat Martineau
Quoted 10 lines · Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_input.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
net/ipv4/tcp_input.c +0 -0
index 4e84f299c96f..7acc1f46641c 100644

net/ipv4/tcp_input.c void tcp_parse_options(const struct net *net, · line 3802

Quoted 7 lines
net/ipv4/tcp_input.c +1 -3
@@ -3799,9 +3799,11 @@ void tcp_parse_options(const struct net *net,
 				break;
 
 			default:
-				tcp_extra_options_parse(opcode, opsize, ptr,
-							skb, opt_rx,
-							tcp_to_sk(tp));
+				if (static_branch_unlikely(&tcp_extra_options_enabled))

Since this case in the switch statement is only executed for unrecognized options, I had skipped the static branch in the initial implementation. It is more consistent to use the static branch, and there isn't much use in making the function call just to try to iterate over an empty list.

Quoted 6 lines
net/ipv4/tcp_input.c +4 -0
+					tcp_extra_options_parse(opcode, opsize,
+								ptr, skb,
+								opt_rx,
+ tcp_to_sk(tp));
 				break;
 
Quoted 12 lines
Quoted 5 lines
Quoted 5 lines
net/ipv4/tcp_input.c +0 -1
 			}
--

2.15.0

I sent my reply and then realized I had one more point to make. We might need to take another look at the use of static branching by the extra options framework. It's one thing to inc/dec tcp_extra_options_enabled when modules are added or removed, but we don't want to be rapidly changing static branches as MPTCP sockets come and go. Does tcp_extra_options_enabled still make sense for per-socket extra_option lists?

Quoted 2 lines

You think that the test for whether or not the tcp_option_list is emtpy is sufficient?

It might be if we wrap it in a "unlikely()".

Quoted 3 lines

I don't know about the cost of static_branch_inc/dec. Is it that bad?

Yeah, it's that bad. The kernel is patching machine code every time the key is enabled or disabled (rewriting no-ops with jump instructions and vice-versa).

"""
* Since this relies on modifying code, the branch modifying functions
* must be considered absolute slow paths (machine wide synchronization etc.).
"""

http://elixir.free-electrons.com/linux/v4.15-rc1/source/include/linux/jump_label.h#L56

Quoted 7 lines

Not just a slow path, but an *absolute* slow path!

static_key_deferred and static_key_slow_dec_deferred() are available in jump_label_ratelimit.h, which is one way to keep it from thrashing too much. This rate-limited static key is only used in one file (arch/x86/kvm/lapic.c). That subsystem limits jump label patching to once per second.

I see.

Quoted 6 lines
Quoted 6 lines

One thing is that having the static branches allows for sure to avoid any cost in the hot data-path. Which is definitely something netdev will want.

Quoted 6 lines
Quoted 1 line

If we pay a cost for MPTCP, I think it's fine IMO. (at least in a first step ;))

It would be more straightforward if only MPTCP was paying the cost, but in this case the hit is systemwide if any extra TCP options are in use. The combination of a rate-limiting static_key and checking the option list before the call could be fast enough.

If we have rate-limited static keys, do we still need to check the option-list before the call?

Also, what if we rather inc/dec the static key more rarely. E.g., not for every new connection but rather for the very first one and then dec again at the very last one.

I'm trying to cook a patch...

Christoph

Quoted 4 lines

--
Mat Martineau
Intel OTC

author
#29 lore raw

Re: [MPTCP] [PATCH v2 14/25] tcp_extra_options: Check static branch before _parse

Quoted 96 lines · Christoph Paasch
Quoted 72 lines · Mat Martineau
Quoted 50 lines · Christoph Paasch
Quoted 36 lines · Mat Martineau
Quoted 33 lines · Mat Martineau
Quoted 10 lines · Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_input.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
net/ipv4/tcp_input.c +0 -0
index 4e84f299c96f..7acc1f46641c 100644

net/ipv4/tcp_input.c void tcp_parse_options(const struct net *net, · line 3802

Quoted 7 lines
net/ipv4/tcp_input.c +1 -3
@@ -3799,9 +3799,11 @@ void tcp_parse_options(const struct net *net,
 				break;
 
 			default:
-				tcp_extra_options_parse(opcode, opsize, ptr,
-							skb, opt_rx,
-							tcp_to_sk(tp));
+				if (static_branch_unlikely(&tcp_extra_options_enabled))

Since this case in the switch statement is only executed for unrecognized options, I had skipped the static branch in the initial implementation. It is more consistent to use the static branch, and there isn't much use in making the function call just to try to iterate over an empty list.

Quoted 6 lines
net/ipv4/tcp_input.c +4 -0
+					tcp_extra_options_parse(opcode, opsize,
+								ptr, skb,
+								opt_rx,
+ tcp_to_sk(tp));
 				break;
 
Quoted 12 lines
Quoted 5 lines
Quoted 5 lines
net/ipv4/tcp_input.c +0 -1
 			}
--

2.15.0

I sent my reply and then realized I had one more point to make. We might need to take another look at the use of static branching by the extra options framework. It's one thing to inc/dec tcp_extra_options_enabled when modules are added or removed, but we don't want to be rapidly changing static branches as MPTCP sockets come and go. Does tcp_extra_options_enabled still make sense for per-socket extra_option lists?

Quoted 2 lines

You think that the test for whether or not the tcp_option_list is emtpy is sufficient?

It might be if we wrap it in a "unlikely()".

Quoted 3 lines

I don't know about the cost of static_branch_inc/dec. Is it that bad?

Yeah, it's that bad. The kernel is patching machine code every time the key is enabled or disabled (rewriting no-ops with jump instructions and vice-versa).

"""
* Since this relies on modifying code, the branch modifying functions
* must be considered absolute slow paths (machine wide synchronization etc.).
"""

http://elixir.free-electrons.com/linux/v4.15-rc1/source/include/linux/jump_label.h#L56

Quoted 7 lines

Not just a slow path, but an *absolute* slow path!

static_key_deferred and static_key_slow_dec_deferred() are available in jump_label_ratelimit.h, which is one way to keep it from thrashing too much. This rate-limited static key is only used in one file (arch/x86/kvm/lapic.c). That subsystem limits jump label patching to once per second.

I see.

Quoted 6 lines
Quoted 6 lines

One thing is that having the static branches allows for sure to avoid any cost in the hot data-path. Which is definitely something netdev will want.

Quoted 6 lines
Quoted 1 line

If we pay a cost for MPTCP, I think it's fine IMO. (at least in a first step ;))

It would be more straightforward if only MPTCP was paying the cost, but in this case the hit is systemwide if any extra TCP options are in use. The combination of a rate-limiting static_key and checking the option list before the call could be fast enough.

Quoted 8 lines

If we have rate-limited static keys, do we still need to check the option-list before the call?

Also, what if we rather inc/dec the static key more rarely. E.g., not for every new connection but rather for the very first one and then dec again at the very last one.

I'm trying to cook a patch...

This is what I have in mind:

-----
commit ba22182b0b3100539e545313e0eb12a9c466c1fe

Author: Christoph Paasch <cpaasch(a)apple.com>
Date:   Wed Nov 29 13:11:44 2017 -0800

tcp_extra_options: Reduce static_branch_inc/dec through a refcnt

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
include/net/tcp.h +1 -0
index 6db92bbbf3e2..344a0d859b37 100644
@@ -2008,6 +2008,7 @@ struct tcp_extra_option_ops {
 	struct tcp_extra_option_store *(*move)(struct tcp_extra_option_store *from);
 	void (*destroy)(struct tcp_extra_option_store *store);
 	struct module		*owner;
+	atomic_t		*refcnt;
 };
 
 /* Every extra-option in a TCP-socket gets stored in a _store structure which
net/ipv4/tcp.c +10 -3
index 2dcb12c330eb..3067e7b5ad00 100644
@@ -3570,7 +3570,8 @@ int tcp_register_extra_option(struct tcp_extra_option_store *store,
 	list_add_tail(&store->list, add_before);
 	pr_debug("Option kind %u registered\n", store->ops->option_kind);
 
-	static_branch_inc(&tcp_extra_options_enabled);
+	if (atomic_inc_return(store->ops->refcnt) == 1)
+		static_branch_inc(&tcp_extra_options_enabled);
 
 	return ret;
 }
@@ -3602,7 +3603,8 @@ void tcp_extra_options_copy(struct sock *listener,
 
 		list_add_tail(&new->list, to);
 
-		static_branch_inc(&tcp_extra_options_enabled);
+		if (atomic_inc_return(new->ops->refcnt) == 1)
+			static_branch_inc(&tcp_extra_options_enabled);
 	}
 }
 
@@ -3632,11 +3634,16 @@ void tcp_extra_options_destroy(struct sock *sk)
 
 	list_for_each_entry_safe(entry, tmp, lhead, list) {
 		struct module *owner = entry->ops->owner;
+		bool dec_static = false;
 		list_del(&entry->list);
 
+		if (atomic_dec_return(entry->ops->refcnt) == 0)
+			dec_static = true;
+
 		entry->ops->destroy(entry);
 
-		static_branch_dec(&tcp_extra_options_enabled);
+		if (dec_static)
+			static_branch_dec(&tcp_extra_options_enabled);
 
 		module_put(owner);
 	}
#30 lore raw

Re: [MPTCP] [PATCH v2 14/25] tcp_extra_options: Check static branch before _parse

Quoted 92 lines · Christoph Paasch
Quoted 91 lines · Christoph Paasch
Quoted 72 lines · Mat Martineau
Quoted 50 lines · Christoph Paasch
Quoted 36 lines · Mat Martineau
Quoted 33 lines · Mat Martineau
Quoted 10 lines · Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_input.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
net/ipv4/tcp_input.c +0 -0
index 4e84f299c96f..7acc1f46641c 100644

net/ipv4/tcp_input.c void tcp_parse_options(const struct net *net, · line 3802

Quoted 7 lines
net/ipv4/tcp_input.c +1 -3
@@ -3799,9 +3799,11 @@ void tcp_parse_options(const struct net *net,
 				break;
 
 			default:
-				tcp_extra_options_parse(opcode, opsize, ptr,
-							skb, opt_rx,
-							tcp_to_sk(tp));
+				if (static_branch_unlikely(&tcp_extra_options_enabled))

Since this case in the switch statement is only executed for unrecognized options, I had skipped the static branch in the initial implementation. It is more consistent to use the static branch, and there isn't much use in making the function call just to try to iterate over an empty list.

Quoted 6 lines
net/ipv4/tcp_input.c +4 -0
+					tcp_extra_options_parse(opcode, opsize,
+								ptr, skb,
+								opt_rx,
+ tcp_to_sk(tp));
 				break;
 
Quoted 12 lines
Quoted 5 lines
Quoted 5 lines
net/ipv4/tcp_input.c +0 -1
 			}
--

2.15.0

I sent my reply and then realized I had one more point to make. We might need to take another look at the use of static branching by the extra options framework. It's one thing to inc/dec tcp_extra_options_enabled when modules are added or removed, but we don't want to be rapidly changing static branches as MPTCP sockets come and go. Does tcp_extra_options_enabled still make sense for per-socket extra_option lists?

Quoted 2 lines

You think that the test for whether or not the tcp_option_list is emtpy is sufficient?

It might be if we wrap it in a "unlikely()".

Quoted 3 lines

I don't know about the cost of static_branch_inc/dec. Is it that bad?

Yeah, it's that bad. The kernel is patching machine code every time the key is enabled or disabled (rewriting no-ops with jump instructions and vice-versa).

"""
* Since this relies on modifying code, the branch modifying functions
* must be considered absolute slow paths (machine wide synchronization etc.).
"""

http://elixir.free-electrons.com/linux/v4.15-rc1/source/include/linux/jump_label.h#L56

Quoted 7 lines

Not just a slow path, but an *absolute* slow path!

static_key_deferred and static_key_slow_dec_deferred() are available in jump_label_ratelimit.h, which is one way to keep it from thrashing too much. This rate-limited static key is only used in one file (arch/x86/kvm/lapic.c). That subsystem limits jump label patching to once per second.

I see.

Quoted 8 lines
Quoted 7 lines

One thing is that having the static branches allows for sure to avoid any cost in the hot data-path. Which is definitely something netdev will want.

If we pay a cost for MPTCP, I think it's fine IMO. (at least in a first step ;))

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

It would be more straightforward if only MPTCP was paying the cost, but in this case the hit is systemwide if any extra TCP options are in use. The combination of a rate-limiting static_key and checking the option list before the call could be fast enough.

If we have rate-limited static keys, do we still need to check the option-list before the call?

You'd still want the fast path on concurrent regular TCP sockets to be minimally affected, even when other "extra option" sockets exist. I'm not sure if the maintainers would consider it a requirement, but this code is so highly optimized that I'm assuming they would be interested in cutting out the extra CPU cycles.

Quoted 64 lines
Quoted 6 lines

Also, what if we rather inc/dec the static key more rarely. E.g., not for every new connection but rather for the very first one and then dec again at the very last one.

I'm trying to cook a patch...

This is what I have in mind:

-----
commit ba22182b0b3100539e545313e0eb12a9c466c1fe

Author: Christoph Paasch <cpaasch(a)apple.com>
Date:   Wed Nov 29 13:11:44 2017 -0800

tcp_extra_options: Reduce static_branch_inc/dec through a refcnt

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
include/net/tcp.h +1 -0
index 6db92bbbf3e2..344a0d859b37 100644
@@ -2008,6 +2008,7 @@ struct tcp_extra_option_ops {
 	struct tcp_extra_option_store *(*move)(struct tcp_extra_option_store *from);
 	void (*destroy)(struct tcp_extra_option_store *store);
 	struct module		*owner;
+	atomic_t		*refcnt;
 };
 
 /* Every extra-option in a TCP-socket gets stored in a _store structure which
net/ipv4/tcp.c +6 -2
index 2dcb12c330eb..3067e7b5ad00 100644
@@ -3570,7 +3570,8 @@ int tcp_register_extra_option(struct tcp_extra_option_store *store,
 	list_add_tail(&store->list, add_before);
 	pr_debug("Option kind %u registered\n", store->ops->option_kind);
 
-	static_branch_inc(&tcp_extra_options_enabled);
+	if (atomic_inc_return(store->ops->refcnt) == 1)
+		static_branch_inc(&tcp_extra_options_enabled);
 
 	return ret;
 }
@@ -3602,7 +3603,8 @@ void tcp_extra_options_copy(struct sock *listener,
 
 		list_add_tail(&new->list, to);
 
-		static_branch_inc(&tcp_extra_options_enabled);
+		if (atomic_inc_return(new->ops->refcnt) == 1)
+			static_branch_inc(&tcp_extra_options_enabled);
 	}
 }
 
@@ -3632,11 +3634,16 @@ void tcp_extra_options_destroy(struct sock *sk)
 
 	list_for_each_entry_safe(entry, tmp, lhead, list) {
 		struct module *owner = entry->ops->owner;
+		bool dec_static = false;
 		list_del(&entry->list);
 
+		if (atomic_dec_return(entry->ops->refcnt) == 0)

net/ipv4/tcp.c void tcp_extra_options_destroy(struct sock *sk) · line 3649

Quoted 10 lines
net/ipv4/tcp.c +4 -1
+			dec_static = true;
+
 		entry->ops->destroy(entry);
 
-		static_branch_dec(&tcp_extra_options_enabled);
+		if (dec_static)
+			static_branch_dec(&tcp_extra_options_enabled);
 
 		module_put(owner);
 	}

static_branch_{inc,dec} are already reference counts - I didn't explain the performance hit clearly. They aren't always expensive.

The problem is when the static key goes from 0->1 and 1->0. As long as there's at least one extended option socket active on the system, the expensive instruction rewrites won't happen. If you had a process rapidly open and close a socket that enabled/disabled the static key, the system would be drastically impacted. A server with one or more long-lived listening MPTCP sockets would not be affected because the static branch reference count for extended options would always be at least 1.

--
Mat Martineau
Intel OTC

author
#31 lore raw

Re: [MPTCP] [PATCH v2 14/25] tcp_extra_options: Check static branch before _parse

Quoted 180 lines · Mat Martineau
Quoted 92 lines · Christoph Paasch
Quoted 91 lines · Christoph Paasch
Quoted 72 lines · Mat Martineau
Quoted 50 lines · Christoph Paasch
Quoted 36 lines · Mat Martineau
Quoted 33 lines · Mat Martineau
Quoted 10 lines · Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---
net/ipv4/tcp_input.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
net/ipv4/tcp_input.c +0 -0
index 4e84f299c96f..7acc1f46641c 100644

net/ipv4/tcp_input.c void tcp_parse_options(const struct net *net, · line 3802

Quoted 7 lines
net/ipv4/tcp_input.c +1 -3
@@ -3799,9 +3799,11 @@ void tcp_parse_options(const struct net *net,
 				break;
 
 			default:
-				tcp_extra_options_parse(opcode, opsize, ptr,
-							skb, opt_rx,
-							tcp_to_sk(tp));
+				if (static_branch_unlikely(&tcp_extra_options_enabled))

Since this case in the switch statement is only executed for unrecognized options, I had skipped the static branch in the initial implementation. It is more consistent to use the static branch, and there isn't much use in making the function call just to try to iterate over an empty list.

Quoted 6 lines
net/ipv4/tcp_input.c +4 -0
+					tcp_extra_options_parse(opcode, opsize,
+								ptr, skb,
+								opt_rx,
+ tcp_to_sk(tp));
 				break;
 
Quoted 12 lines
Quoted 5 lines
Quoted 5 lines
net/ipv4/tcp_input.c +0 -1
 			}
--

2.15.0

I sent my reply and then realized I had one more point to make. We might need to take another look at the use of static branching by the extra options framework. It's one thing to inc/dec tcp_extra_options_enabled when modules are added or removed, but we don't want to be rapidly changing static branches as MPTCP sockets come and go. Does tcp_extra_options_enabled still make sense for per-socket extra_option lists?

Quoted 2 lines

You think that the test for whether or not the tcp_option_list is emtpy is sufficient?

It might be if we wrap it in a "unlikely()".

Quoted 3 lines

I don't know about the cost of static_branch_inc/dec. Is it that bad?

Yeah, it's that bad. The kernel is patching machine code every time the key is enabled or disabled (rewriting no-ops with jump instructions and vice-versa).

"""
* Since this relies on modifying code, the branch modifying functions
* must be considered absolute slow paths (machine wide synchronization etc.).
"""

http://elixir.free-electrons.com/linux/v4.15-rc1/source/include/linux/jump_label.h#L56

Quoted 7 lines

Not just a slow path, but an *absolute* slow path!

static_key_deferred and static_key_slow_dec_deferred() are available in jump_label_ratelimit.h, which is one way to keep it from thrashing too much. This rate-limited static key is only used in one file (arch/x86/kvm/lapic.c). That subsystem limits jump label patching to once per second.

I see.

Quoted 8 lines
Quoted 7 lines

One thing is that having the static branches allows for sure to avoid any cost in the hot data-path. Which is definitely something netdev will want.

If we pay a cost for MPTCP, I think it's fine IMO. (at least in a first step ;))

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

It would be more straightforward if only MPTCP was paying the cost, but in this case the hit is systemwide if any extra TCP options are in use. The combination of a rate-limiting static_key and checking the option list before the call could be fast enough.

If we have rate-limited static keys, do we still need to check the option-list before the call?

You'd still want the fast path on concurrent regular TCP sockets to be minimally affected, even when other "extra option" sockets exist. I'm not sure if the maintainers would consider it a requirement, but this code is so highly optimized that I'm assuming they would be interested in cutting out the extra CPU cycles.

Quoted 71 lines
Quoted 6 lines

Also, what if we rather inc/dec the static key more rarely. E.g., not for every new connection but rather for the very first one and then dec again at the very last one.

I'm trying to cook a patch...

This is what I have in mind:

-----
commit ba22182b0b3100539e545313e0eb12a9c466c1fe

Author: Christoph Paasch <cpaasch(a)apple.com>
Date:   Wed Nov 29 13:11:44 2017 -0800

tcp_extra_options: Reduce static_branch_inc/dec through a refcnt

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
include/net/tcp.h +1 -0
index 6db92bbbf3e2..344a0d859b37 100644
@@ -2008,6 +2008,7 @@ struct tcp_extra_option_ops {
 	struct tcp_extra_option_store *(*move)(struct tcp_extra_option_store *from);
 	void (*destroy)(struct tcp_extra_option_store *store);
 	struct module		*owner;
+	atomic_t		*refcnt;
 };
 
 /* Every extra-option in a TCP-socket gets stored in a _store structure which
net/ipv4/tcp.c +10 -3
index 2dcb12c330eb..3067e7b5ad00 100644
@@ -3570,7 +3570,8 @@ int tcp_register_extra_option(struct tcp_extra_option_store *store,
 	list_add_tail(&store->list, add_before);
 	pr_debug("Option kind %u registered\n", store->ops->option_kind);
 
-	static_branch_inc(&tcp_extra_options_enabled);
+	if (atomic_inc_return(store->ops->refcnt) == 1)
+		static_branch_inc(&tcp_extra_options_enabled);
 
 	return ret;
 }
@@ -3602,7 +3603,8 @@ void tcp_extra_options_copy(struct sock *listener,
 
 		list_add_tail(&new->list, to);
 
-		static_branch_inc(&tcp_extra_options_enabled);
+		if (atomic_inc_return(new->ops->refcnt) == 1)
+			static_branch_inc(&tcp_extra_options_enabled);
 	}
 }
 
@@ -3632,11 +3634,16 @@ void tcp_extra_options_destroy(struct sock *sk)
 
 	list_for_each_entry_safe(entry, tmp, lhead, list) {
 		struct module *owner = entry->ops->owner;
+		bool dec_static = false;
 		list_del(&entry->list);
 
+		if (atomic_dec_return(entry->ops->refcnt) == 0)
+			dec_static = true;
+
 		entry->ops->destroy(entry);
 
-		static_branch_dec(&tcp_extra_options_enabled);
+		if (dec_static)
+			static_branch_dec(&tcp_extra_options_enabled);
 
Quoted 6 lines

net/ipv4/tcp.c void tcp_extra_options_destroy(struct sock *sk) · line 3649

Quoted 3 lines
net/ipv4/tcp.c +0 -0
 		module_put(owner);
 	}

static_branch_{inc,dec} are already reference counts - I didn't explain the performance hit clearly. They aren't always expensive.

Ok, I see. Thanks for clarifying.

Quoted 7 lines

The problem is when the static key goes from 0->1 and 1->0. As long as there's at least one extended option socket active on the system, the expensive instruction rewrites won't happen. If you had a process rapidly open and close a socket that enabled/disabled the static key, the system would be drastically impacted. A server with one or more long-lived listening MPTCP sockets would not be affected because the static branch reference count for extended options would always be at least 1.

Ok, I think the scenario where we have a single socket appearing/disappearing can be considered a corner-case scenario for now.

Later-on we can optimize that.

Christoph

author
#32 lore raw

PATCH v2 15/25 tcp_extra_options: Skip fast-path when extra-options are present

In reply to Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv4/tcp_input.c | 4 ++++
 1 file changed, 4 insertions(+)
net/ipv4/tcp_input.c +4 -0
index 7acc1f46641c..c2ef1309e877 100644
@@ -5333,6 +5333,10 @@ void tcp_rcv_established(struct sock *sk, struct sk_buff *skb,
 
 	tp->rx_opt.saw_tstamp = 0;
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled) &&
+	    !list_empty(&tp->tcp_option_list))
+		goto slow_path;
+
 	/*	pred_flags is 0xS?10 << 16 + snd_wnd
 	 *	if header_prediction is to be made
 	 *	'S' will always be tp->tcp_header_len >> 2
Signature · 1 line
2.15.0
author
#33 lore raw

PATCH v2 16/25 tcp_extra_options: Pass sk to tcp_parse_options instead of tp

In reply to Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/net/tcp.h        |  2 +-
 net/ipv4/syncookies.c    |  2 +-
 net/ipv4/tcp_input.c     | 13 ++++++-------
 net/ipv4/tcp_minisocks.c |  6 ++++--
 net/ipv6/syncookies.c    |  2 +-
 5 files changed, 13 insertions(+), 12 deletions(-)
include/net/tcp.h +1 -1
index 0b94356ee5ba..55f6e6b39539 100644
@@ -404,7 +404,7 @@ int tcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, int nonblock,
 void tcp_parse_options(const struct net *net, const struct sk_buff *skb,
 		       struct tcp_options_received *opt_rx,
 		       int estab, struct tcp_fastopen_cookie *foc,
-		       struct tcp_sock *tp);
+		       struct sock *sk);
 const u8 *tcp_parse_md5sig_option(const struct tcphdr *th);
 
 /*
net/ipv4/syncookies.c +1 -1
index b3698ac3f078..fc4f91d92b8a 100644
@@ -313,7 +313,7 @@ struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb)
 
 	/* check for timestamp cookie support */
 	memset(&tcp_opt, 0, sizeof(tcp_opt));
-	tcp_parse_options(sock_net(sk), skb, &tcp_opt, 0, NULL, tp);
+	tcp_parse_options(sock_net(sk), skb, &tcp_opt, 0, NULL, sk);
 
 	if (tcp_opt.saw_tstamp && tcp_opt.rcv_tsecr) {
 		tsoff = secure_tcp_ts_off(sock_net(sk),
net/ipv4/tcp_input.c +6 -7
index c2ef1309e877..07a43518affa 100644
@@ -3693,7 +3693,7 @@ static int smc_parse_options(const struct tcphdr *th,
 void tcp_parse_options(const struct net *net,
 		       const struct sk_buff *skb,
 		       struct tcp_options_received *opt_rx, int estab,
-		       struct tcp_fastopen_cookie *foc, struct tcp_sock *tp)
+		       struct tcp_fastopen_cookie *foc, struct sock *sk)
 {
 	const unsigned char *ptr;
 	const struct tcphdr *th = tcp_hdr(skb);
@@ -3802,8 +3802,7 @@ void tcp_parse_options(const struct net *net,
 				if (static_branch_unlikely(&tcp_extra_options_enabled))
 					tcp_extra_options_parse(opcode, opsize,
 								ptr, skb,
-								opt_rx,
-								tcp_to_sk(tp));
+								opt_rx, sk);
 				break;
 
 			}
@@ -3854,7 +3853,7 @@ static bool tcp_fast_parse_options(const struct net *net,
 			goto extra_opt_check;
 	}
 
-	tcp_parse_options(net, skb, &tp->rx_opt, 1, NULL, tp);
+	tcp_parse_options(net, skb, &tp->rx_opt, 1, NULL, tcp_to_sk(tp));
 	if (tp->rx_opt.saw_tstamp && tp->rx_opt.rcv_tsecr)
 		tp->rx_opt.rcv_tsecr -= tp->tsoffset;
 
@@ -5524,7 +5523,7 @@ static bool tcp_rcv_fastopen_synack(struct sock *sk, struct sk_buff *synack,
 		/* Get original SYNACK MSS value if user MSS sets mss_clamp */
 		tcp_clear_options(&opt);
 		opt.user_mss = opt.mss_clamp = 0;
-		tcp_parse_options(sock_net(sk), synack, &opt, 0, NULL, tp);
+		tcp_parse_options(sock_net(sk), synack, &opt, 0, NULL, sk);
 		mss = opt.mss_clamp;
 	}
 
@@ -5587,7 +5586,7 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
 	int saved_clamp = tp->rx_opt.mss_clamp;
 	bool fastopen_fail;
 
-	tcp_parse_options(sock_net(sk), skb, &tp->rx_opt, 0, &foc, tp);
+	tcp_parse_options(sock_net(sk), skb, &tp->rx_opt, 0, &foc, sk);
 	if (tp->rx_opt.saw_tstamp && tp->rx_opt.rcv_tsecr)
 		tp->rx_opt.rcv_tsecr -= tp->tsoffset;
 
@@ -6255,7 +6254,7 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,
 	tmp_opt.mss_clamp = af_ops->mss_clamp;
 	tmp_opt.user_mss  = tp->rx_opt.user_mss;
 	tcp_parse_options(sock_net(sk), skb, &tmp_opt, 0,
-			  want_cookie ? NULL : &foc, tp);
+			  want_cookie ? NULL : &foc, sk);
 
 	if (want_cookie && !tmp_opt.saw_tstamp)
 		tcp_clear_options(&tmp_opt);
net/ipv4/tcp_minisocks.c +4 -2
index fb50d3d0d158..1685c5ab4913 100644
@@ -97,7 +97,8 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb,
 
 	tmp_opt.saw_tstamp = 0;
 	if (th->doff > (sizeof(*th) >> 2) && tcptw->tw_ts_recent_stamp) {
-		tcp_parse_options(twsk_net(tw), skb, &tmp_opt, 0, NULL, NULL);
+		tcp_parse_options(twsk_net(tw), skb, &tmp_opt, 0, NULL,
+				  (struct sock *)tw);
 
 		if (tmp_opt.saw_tstamp) {
 			if (tmp_opt.rcv_tsecr)
@@ -595,7 +596,8 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
 
 	tmp_opt.saw_tstamp = 0;
 	if (th->doff > (sizeof(struct tcphdr)>>2)) {
-		tcp_parse_options(sock_net(sk), skb, &tmp_opt, 0, NULL, NULL);
+		tcp_parse_options(sock_net(sk), skb, &tmp_opt, 0, NULL,
+				  req_to_sk(req));
 
 		if (tmp_opt.saw_tstamp) {
 			tmp_opt.ts_recent = req->ts_recent;
net/ipv6/syncookies.c +1 -1
index 0ed3721b03a2..67ea216b7ffa 100644
@@ -162,7 +162,7 @@ struct sock *cookie_v6_check(struct sock *sk, struct sk_buff *skb)
 
 	/* check for timestamp cookie support */
 	memset(&tcp_opt, 0, sizeof(tcp_opt));
-	tcp_parse_options(sock_net(sk), skb, &tcp_opt, 0, NULL, tp);
+	tcp_parse_options(sock_net(sk), skb, &tcp_opt, 0, NULL, sk);
 
 	if (tcp_opt.saw_tstamp && tcp_opt.rcv_tsecr) {
 		tsoff = secure_tcpv6_ts_off(sock_net(sk),
Signature · 1 line
2.15.0
author
#34 lore raw

PATCH v2 17/25 tcp_extra_options: Add check-callback right after parsing the option

In reply to Christoph Paasch

TCP-options like MD5 can trigger an early drop of the packet. Thus, we need to add a check-callback right after the parsing to implement this drop.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/net/tcp.h        |  8 ++++++++
 net/ipv4/syncookies.c    |  4 ++++
 net/ipv4/tcp.c           | 24 ++++++++++++++++++++++++
 net/ipv4/tcp_input.c     | 11 ++++++++++-
 net/ipv4/tcp_minisocks.c | 12 ++++++++++--
 net/ipv6/syncookies.c    |  4 ++++
 6 files changed, 60 insertions(+), 3 deletions(-)
include/net/tcp.h +8 -0
index 55f6e6b39539..63fe6e02a963 100644
@@ -2076,6 +2076,10 @@ struct tcp_extra_option_ops {
 		      struct tcp_options_received *opt_rx,
 		      struct sock *sk,
 		      struct tcp_extra_option_store *store);
+	bool (*check)(struct sock *sk,
+		      const struct sk_buff *skb,
+		      struct tcp_options_received *opt_rx,
+		      struct tcp_extra_option_store *store);
 	/* Return the number of bytes consumed */
 	unsigned int (*prepare)(struct sk_buff *skb, u8 flags,
 				unsigned int remaining,
@@ -2126,6 +2130,10 @@ void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
 			     struct tcp_options_received *opt_rx,
 			     struct sock *sk);
 
+bool tcp_extra_options_check(struct sock *sk,
+			     const struct sk_buff *skb,
+			     struct tcp_options_received *opt_rx);
+
 unsigned int tcp_extra_options_prepare(struct sk_buff *skb, u8 flags,
 				       unsigned int remaining,
 				       struct tcp_out_options *opts,
net/ipv4/syncookies.c +4 -0
index fc4f91d92b8a..77ddae746b25 100644
@@ -325,6 +325,10 @@ struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb)
 	if (!cookie_timestamp_decode(sock_net(sk), &tcp_opt))
 		goto out;
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled) &&
+	    tcp_extra_options_check(sk, skb, &tcp_opt))
+		goto out;
+
 	ret = NULL;
 	req = inet_reqsk_alloc(&tcp_request_sock_ops, sk, false); /* for safety */
 	if (!req)
net/ipv4/tcp.c +24 -0
index 81b98a33530f..bc7f471bcf96 100644
@@ -3520,6 +3520,30 @@ void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
 		entry->ops->parse(opsize, opptr, skb, opt_rx, sk, entry);
 }
 
+bool tcp_extra_options_check(struct sock *sk,
+			     const struct sk_buff *skb,
+			     struct tcp_options_received *opt_rx)
+{
+	struct tcp_extra_option_store *entry;
+	struct list_head *lhead;
+	bool drop = false;
+
+	lhead = tcp_extra_options_get_list(sk);
+
+	list_for_each_entry(entry, lhead, list) {
+		bool ret = false;
+
+		if (entry->ops->check)
+			ret = entry->ops->check(sk, skb, opt_rx, entry);
+
+		if (ret)
+			drop = true;
+	}
+
+	return drop;
+}
+EXPORT_SYMBOL_GPL(tcp_extra_options_check);
+
 unsigned int tcp_extra_options_prepare(struct sk_buff *skb, u8 flags,
 				       unsigned int remaining,
 				       struct tcp_out_options *opts,
net/ipv4/tcp_input.c +10 -1
index 07a43518affa..23c62d8e4e06 100644
@@ -3858,7 +3858,8 @@ static bool tcp_fast_parse_options(const struct net *net,
 		tp->rx_opt.rcv_tsecr -= tp->tsoffset;
 
 extra_opt_check:
-	/* ToDo - will be added here */
+	if (static_branch_unlikely(&tcp_extra_options_enabled))
+		return tcp_extra_options_check(tcp_to_sk(tp), skb, &tp->rx_opt);
 	return false;
 }
 
@@ -5590,6 +5591,10 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
 	if (tp->rx_opt.saw_tstamp && tp->rx_opt.rcv_tsecr)
 		tp->rx_opt.rcv_tsecr -= tp->tsoffset;
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled) &&
+	    tcp_extra_options_check(sk, skb, &tp->rx_opt))
+		goto discard;
+
 	if (th->ack) {
 		/* rfc793:
 		 * "If the state is SYN-SENT then
@@ -6256,6 +6261,10 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,
 	tcp_parse_options(sock_net(sk), skb, &tmp_opt, 0,
 			  want_cookie ? NULL : &foc, sk);
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled) &&
+	    tcp_extra_options_check(sk, skb, &tmp_opt))
+		goto drop_and_free;
+
 	if (want_cookie && !tmp_opt.saw_tstamp)
 		tcp_clear_options(&tmp_opt);
 
net/ipv4/tcp_minisocks.c +10 -2
index 1685c5ab4913..c3a0a4676fc8 100644
@@ -95,7 +95,7 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb,
 	struct tcp_timewait_sock *tcptw = tcp_twsk((struct sock *)tw);
 	bool paws_reject = false;
 
-	tmp_opt.saw_tstamp = 0;
+	tcp_clear_options(&tmp_opt);
 	if (th->doff > (sizeof(*th) >> 2) && tcptw->tw_ts_recent_stamp) {
 		tcp_parse_options(twsk_net(tw), skb, &tmp_opt, 0, NULL,
 				  (struct sock *)tw);
@@ -109,6 +109,10 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb,
 		}
 	}
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled) &&
+	    tcp_extra_options_check((struct sock *)tw, skb, &tmp_opt))
+		return TCP_TW_SUCCESS;
+
 	if (tw->tw_substate == TCP_FIN_WAIT2) {
 		/* Just repeat all the checks of tcp_rcv_state_process() */
 
@@ -594,7 +598,7 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
 	bool paws_reject = false;
 	bool own_req;
 
-	tmp_opt.saw_tstamp = 0;
+	tcp_clear_options(&tmp_opt);
 	if (th->doff > (sizeof(struct tcphdr)>>2)) {
 		tcp_parse_options(sock_net(sk), skb, &tmp_opt, 0, NULL,
 				  req_to_sk(req));
@@ -612,6 +616,10 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
 		}
 	}
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled) &&
+	    tcp_extra_options_check(req_to_sk(req), skb, &tmp_opt))
+		return NULL;
+
 	/* Check for pure retransmitted SYN. */
 	if (TCP_SKB_CB(skb)->seq == tcp_rsk(req)->rcv_isn &&
 	    flg == TCP_FLAG_SYN &&
net/ipv6/syncookies.c +4 -0
index 67ea216b7ffa..05228c28c992 100644
@@ -174,6 +174,10 @@ struct sock *cookie_v6_check(struct sock *sk, struct sk_buff *skb)
 	if (!cookie_timestamp_decode(sock_net(sk), &tcp_opt))
 		goto out;
 
+	if (static_branch_unlikely(&tcp_extra_options_enabled) &&
+	    tcp_extra_options_check(sk, skb, &tcp_opt))
+		goto out;
+
 	ret = NULL;
 	req = inet_reqsk_alloc(&tcp6_request_sock_ops, sk, false);
 	if (!req)
Signature · 1 line
2.15.0
author
#35 lore raw

PATCH v2 18/25 tcp_extra_options: Allow to parse experimental options

In reply to Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/net/tcp.h    | 7 ++++---
 net/ipv4/tcp.c       | 4 ++--
 net/ipv4/tcp_input.c | 6 ++++++
 3 files changed, 12 insertions(+), 5 deletions(-)
include/net/tcp.h +4 -3
index 63fe6e02a963..f2708d69bb43 100644
@@ -202,6 +202,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOLEN_FASTOPEN_BASE  2
 #define TCPOLEN_EXP_FASTOPEN_BASE  4
 #define TCPOLEN_EXP_SMC_BASE   6
+#define TCPOLEN_EXP_BASE	6
 
 /* But this is what stacks really send out. */
 #define TCPOLEN_TSTAMP_ALIGNED		12
@@ -2069,7 +2070,7 @@ extern struct static_key_false tcp_extra_options_enabled;
 struct tcp_extra_option_store;
 
 struct tcp_extra_option_ops {
-	unsigned char		option_kind;
+	u32			option_kind;
 	unsigned char		priority;
 	void (*parse)(int opsize, const unsigned char *opptr,
 		      const struct sk_buff *skb,
@@ -2122,10 +2123,10 @@ struct tcp_extra_option_store {
 	const struct tcp_extra_option_ops	*ops;
 };
 
-struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
+struct tcp_extra_option_store *tcp_extra_options_find_kind(u32 kind,
 							   const struct sock *sk);
 
-void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
+void tcp_extra_options_parse(u32 opcode, int opsize, const unsigned char *opptr,
 			     const struct sk_buff *skb,
 			     struct tcp_options_received *opt_rx,
 			     struct sock *sk);
net/ipv4/tcp.c +2 -2
index bc7f471bcf96..6d4e87aa1bbf 100644
@@ -3491,7 +3491,7 @@ static struct list_head *tcp_extra_options_get_list(const struct sock *sk)
 	return NULL;
 }
 
-struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
+struct tcp_extra_option_store *tcp_extra_options_find_kind(u32 kind,
 							   const struct sock *sk)
 {
 	struct tcp_extra_option_store *entry;
@@ -3507,7 +3507,7 @@ struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
 	return entry;
 }
 
-void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
+void tcp_extra_options_parse(u32 opcode, int opsize, const unsigned char *opptr,
 			     const struct sk_buff *skb,
 			     struct tcp_options_received *opt_rx,
 			     struct sock *sk)
net/ipv4/tcp_input.c +6 -0
index 23c62d8e4e06..2a3debb2c7fe 100644
@@ -3796,6 +3796,12 @@ void tcp_parse_options(const struct net *net,
 				else if (smc_parse_options(th, opt_rx, ptr,
 							   opsize))
 					break;
+				else if (static_branch_unlikely(&tcp_extra_options_enabled) &&
+					 opsize >= TCPOLEN_EXP_BASE)
+					tcp_extra_options_parse(get_unaligned_be32(ptr),
+								opsize, ptr,
+								skb, opt_rx,
+								sk);
 				break;
 
 			default:
Signature · 1 line
2.15.0
#36 lore raw

Re: [MPTCP] [PATCH v2 18/25] tcp_extra_options: Allow to parse experimental options

Quoted 12 lines · Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---

include/net/tcp.h    | 7 ++++---
net/ipv4/tcp.c       | 4 ++--
net/ipv4/tcp_input.c | 6 ++++++
3 files changed, 12 insertions(+), 5 deletions(-)
include/net/tcp.h +0 -0
index 63fe6e02a963..f2708d69bb43 100644

include/net/tcp.h void tcp_time_wait(struct sock *sk, int state, int timeo); · line 205

Quoted 4 lines
include/net/tcp.h +1 -0
@@ -202,6 +202,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOLEN_FASTOPEN_BASE  2
 #define TCPOLEN_EXP_FASTOPEN_BASE  4
 #define TCPOLEN_EXP_SMC_BASE   6
+#define TCPOLEN_EXP_BASE	6

Minor formatting note: other TCPOLEN_EXP_* defines here use spaces rather than tabs.

Mat

Quoted 68 lines
include/net/tcp.h +3 -3
 /* But this is what stacks really send out. */
 #define TCPOLEN_TSTAMP_ALIGNED		12
@@ -2069,7 +2070,7 @@ extern struct static_key_false tcp_extra_options_enabled;
 struct tcp_extra_option_store;
 
 struct tcp_extra_option_ops {
-	unsigned char		option_kind;
+	u32			option_kind;
 	unsigned char		priority;
 	void (*parse)(int opsize, const unsigned char *opptr,
 		      const struct sk_buff *skb,
@@ -2122,10 +2123,10 @@ struct tcp_extra_option_store {
 	const struct tcp_extra_option_ops	*ops;
 };
 
-struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
+struct tcp_extra_option_store *tcp_extra_options_find_kind(u32 kind,
 							   const struct sock *sk);
 
-void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
+void tcp_extra_options_parse(u32 opcode, int opsize, const unsigned char *opptr,
 			     const struct sk_buff *skb,
 			     struct tcp_options_received *opt_rx,
 			     struct sock *sk);
net/ipv4/tcp.c +2 -2
index bc7f471bcf96..6d4e87aa1bbf 100644
@@ -3491,7 +3491,7 @@ static struct list_head *tcp_extra_options_get_list(const struct sock *sk)
 	return NULL;
 }
 
-struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
+struct tcp_extra_option_store *tcp_extra_options_find_kind(u32 kind,
 							   const struct sock *sk)
 {
 	struct tcp_extra_option_store *entry;
@@ -3507,7 +3507,7 @@ struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
 	return entry;
 }
 
-void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
+void tcp_extra_options_parse(u32 opcode, int opsize, const unsigned char *opptr,
 			     const struct sk_buff *skb,
 			     struct tcp_options_received *opt_rx,
 			     struct sock *sk)
net/ipv4/tcp_input.c +6 -0
index 23c62d8e4e06..2a3debb2c7fe 100644
@@ -3796,6 +3796,12 @@ void tcp_parse_options(const struct net *net,
 				else if (smc_parse_options(th, opt_rx, ptr,
 							   opsize))
 					break;
+				else if (static_branch_unlikely(&tcp_extra_options_enabled) &&
+					 opsize >= TCPOLEN_EXP_BASE)
+					tcp_extra_options_parse(get_unaligned_be32(ptr),
+								opsize, ptr,
+								skb, opt_rx,
+								sk);
 				break;
 
 			default:
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#37 lore raw

Re: [MPTCP] [PATCH v2 18/25] tcp_extra_options: Allow to parse experimental options

Quoted 9 lines · Mat Martineau
Quoted 7 lines · Christoph Paasch
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---

include/net/tcp.h    | 7 ++++---
net/ipv4/tcp.c       | 4 ++--
net/ipv4/tcp_input.c | 6 ++++++
3 files changed, 12 insertions(+), 5 deletions(-)
Quoted 12 lines

include/net/tcp.h void tcp_time_wait(struct sock *sk, int state, int timeo); · line 205

Quoted 9 lines
include/net/tcp.h +1 -0
index 63fe6e02a963..f2708d69bb43 100644
@@ -202,6 +202,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo);
 #define TCPOLEN_FASTOPEN_BASE  2
 #define TCPOLEN_EXP_FASTOPEN_BASE  4
 #define TCPOLEN_EXP_SMC_BASE   6
+#define TCPOLEN_EXP_BASE	6

Minor formatting note: other TCPOLEN_EXP_* defines here use spaces rather than tabs.

Done.

Christoph

Quoted 76 lines

Mat

Quoted 68 lines
include/net/tcp.h +3 -3
 /* But this is what stacks really send out. */
 #define TCPOLEN_TSTAMP_ALIGNED		12
@@ -2069,7 +2070,7 @@ extern struct static_key_false tcp_extra_options_enabled;
 struct tcp_extra_option_store;
 
 struct tcp_extra_option_ops {
-	unsigned char		option_kind;
+	u32			option_kind;
 	unsigned char		priority;
 	void (*parse)(int opsize, const unsigned char *opptr,
 		      const struct sk_buff *skb,
@@ -2122,10 +2123,10 @@ struct tcp_extra_option_store {
 	const struct tcp_extra_option_ops	*ops;
 };
 
-struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
+struct tcp_extra_option_store *tcp_extra_options_find_kind(u32 kind,
 							   const struct sock *sk);
 
-void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
+void tcp_extra_options_parse(u32 opcode, int opsize, const unsigned char *opptr,
 			     const struct sk_buff *skb,
 			     struct tcp_options_received *opt_rx,
 			     struct sock *sk);
net/ipv4/tcp.c +2 -2
index bc7f471bcf96..6d4e87aa1bbf 100644
@@ -3491,7 +3491,7 @@ static struct list_head *tcp_extra_options_get_list(const struct sock *sk)
 	return NULL;
 }
 
-struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
+struct tcp_extra_option_store *tcp_extra_options_find_kind(u32 kind,
 							   const struct sock *sk)
 {
 	struct tcp_extra_option_store *entry;
@@ -3507,7 +3507,7 @@ struct tcp_extra_option_store *tcp_extra_options_find_kind(unsigned char kind,
 	return entry;
 }
 
-void tcp_extra_options_parse(int opcode, int opsize, const unsigned char *opptr,
+void tcp_extra_options_parse(u32 opcode, int opsize, const unsigned char *opptr,
 			     const struct sk_buff *skb,
 			     struct tcp_options_received *opt_rx,
 			     struct sock *sk)
net/ipv4/tcp_input.c +6 -0
index 23c62d8e4e06..2a3debb2c7fe 100644
@@ -3796,6 +3796,12 @@ void tcp_parse_options(const struct net *net,
 				else if (smc_parse_options(th, opt_rx, ptr,
 							   opsize))
 					break;
+				else if (static_branch_unlikely(&tcp_extra_options_enabled) &&
+					 opsize >= TCPOLEN_EXP_BASE)
+					tcp_extra_options_parse(get_unaligned_be32(ptr),
+								opsize, ptr,
+								skb, opt_rx,
+								sk);
 				break;
 
 			default:
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#38 lore raw

PATCH v2 19/25 tcp_extra_options: Add post_process callback

In reply to Christoph Paasch

Some options require that after validation of the segment, we post-process the options (e.g., set certain bits,...)

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/net/tcp.h    |  6 ++++++
 net/ipv4/tcp.c       | 14 ++++++++++++++
 net/ipv4/tcp_input.c |  6 ++++++
 3 files changed, 26 insertions(+)
include/net/tcp.h +6 -0
index f2708d69bb43..a9b99209d726 100644
@@ -2081,6 +2081,9 @@ struct tcp_extra_option_ops {
 		      const struct sk_buff *skb,
 		      struct tcp_options_received *opt_rx,
 		      struct tcp_extra_option_store *store);
+	void (*post_process)(struct sock *sk,
+			     struct tcp_options_received *opt_rx,
+			     struct tcp_extra_option_store *store);
 	/* Return the number of bytes consumed */
 	unsigned int (*prepare)(struct sk_buff *skb, u8 flags,
 				unsigned int remaining,
@@ -2135,6 +2138,9 @@ bool tcp_extra_options_check(struct sock *sk,
 			     const struct sk_buff *skb,
 			     struct tcp_options_received *opt_rx);
 
+void tcp_extra_options_post_process(struct sock *sk,
+				    struct tcp_options_received *opt_rx);
+
 unsigned int tcp_extra_options_prepare(struct sk_buff *skb, u8 flags,
 				       unsigned int remaining,
 				       struct tcp_out_options *opts,
net/ipv4/tcp.c +14 -0
index 6d4e87aa1bbf..5a9269e9ca2b 100644
@@ -3544,6 +3544,20 @@ bool tcp_extra_options_check(struct sock *sk,
 }
 EXPORT_SYMBOL_GPL(tcp_extra_options_check);
 
+void tcp_extra_options_post_process(struct sock *sk,
+				    struct tcp_options_received *opt_rx)
+{
+	struct tcp_extra_option_store *entry;
+	struct list_head *lhead;
+
+	lhead = tcp_extra_options_get_list(sk);
+
+	list_for_each_entry(entry, lhead, list) {
+		if (entry->ops->post_process)
+			entry->ops->post_process(sk, opt_rx, entry);
+	}
+}
+
 unsigned int tcp_extra_options_prepare(struct sk_buff *skb, u8 flags,
 				       unsigned int remaining,
 				       struct tcp_out_options *opts,
net/ipv4/tcp_input.c +6 -0
index 2a3debb2c7fe..dba59c8ae627 100644
@@ -5683,6 +5683,9 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
 			tp->tcp_header_len = sizeof(struct tcphdr);
 		}
 
+		if (static_branch_unlikely(&tcp_extra_options_enabled))
+			tcp_extra_options_post_process(sk, &tp->rx_opt);
+
 		tcp_sync_mss(sk, icsk->icsk_pmtu_cookie);
 		tcp_initialize_rcv_mss(sk);
 
@@ -5776,6 +5779,9 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
 
 		tcp_ecn_rcv_syn(tp, th);
 
+		if (static_branch_unlikely(&tcp_extra_options_enabled))
+			tcp_extra_options_post_process(sk, &tp->rx_opt);
+
 		tcp_mtup_init(sk);
 		tcp_sync_mss(sk, icsk->icsk_pmtu_cookie);
 		tcp_initialize_rcv_mss(sk);
Signature · 1 line
2.15.0
author
#39 lore raw

PATCH v2 20/25 tcp_smc: Make SMC use TCP extra-option framework

In reply to Christoph Paasch
Cc Ursula Braun <ubraun(a)linux.vnet.ibm.com>
Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/linux/tcp.h      |   3 +-
 include/net/inet_sock.h  |   3 +-
 include/net/tcp.h        |   4 --
 net/ipv4/tcp.c           |   6 +-
 net/ipv4/tcp_input.c     |  36 ----------
 net/ipv4/tcp_minisocks.c |  18 -----
 net/ipv4/tcp_output.c    |  54 ---------------
 net/smc/af_smc.c         | 170 +++++++++++++++++++++++++++++++++++++++++++++--
 8 files changed, 167 insertions(+), 127 deletions(-)
include/linux/tcp.h +1 -2
index 4f653e3bd33f..c04b7e4f5309 100644
@@ -257,8 +257,7 @@ struct tcp_sock {
 		syn_fastopen_ch:1, /* Active TFO re-enabling probe */
 		syn_data_acked:1,/* data in SYN is acked by SYN-ACK */
 		save_syn:1,	/* Save headers of SYN packet */
-		is_cwnd_limited:1,/* forward progress limited by snd_cwnd? */
-		syn_smc:1;	/* SYN includes SMC */
+		is_cwnd_limited:1;/* forward progress limited by snd_cwnd? */
 	u32	tlp_high_seq;	/* snd_nxt at the time of TLP retransmit. */
 
 /* RTT measurement */
include/net/inet_sock.h +1 -2
index 39efb968b7a4..8e51b4a69088 100644
@@ -90,8 +90,7 @@ struct inet_request_sock {
 				wscale_ok  : 1,
 				ecn_ok	   : 1,
 				acked	   : 1,
-				no_srccheck: 1,
-				smc_ok	   : 1;
+				no_srccheck: 1;
 	u32                     ir_mark;
 	union {
 		struct ip_options_rcu __rcu	*ireq_opt;
include/net/tcp.h +0 -4
index a9b99209d726..beedb639e664 100644
@@ -2062,10 +2062,6 @@ static inline bool tcp_bpf_ca_needs_ecn(struct sock *sk)
 	return (tcp_call_bpf(sk, BPF_SOCK_OPS_NEEDS_ECN) == 1);
 }
 
-#if IS_ENABLED(CONFIG_SMC)
-extern struct static_key_false tcp_have_smc;
-#endif
-
 extern struct static_key_false tcp_extra_options_enabled;
 struct tcp_extra_option_store;
 
net/ipv4/tcp.c +1 -5
index 5a9269e9ca2b..3090fac3c462 100644
@@ -294,11 +294,6 @@ EXPORT_SYMBOL(sysctl_tcp_mem);
 atomic_long_t tcp_memory_allocated;	/* Current allocated memory. */
 EXPORT_SYMBOL(tcp_memory_allocated);
 
-#if IS_ENABLED(CONFIG_SMC)
-DEFINE_STATIC_KEY_FALSE(tcp_have_smc);
-EXPORT_SYMBOL(tcp_have_smc);
-#endif
-
 /*
  * Current number of TCP sockets.
  */
@@ -3506,6 +3501,7 @@ struct tcp_extra_option_store *tcp_extra_options_find_kind(u32 kind,
 
 	return entry;
 }
+EXPORT_SYMBOL_GPL(tcp_extra_options_find_kind);
 
 void tcp_extra_options_parse(u32 opcode, int opsize, const unsigned char *opptr,
 			     const struct sk_buff *skb,
net/ipv4/tcp_input.c +0 -36
index dba59c8ae627..5dd19c3f37ce 100644
@@ -3668,24 +3668,6 @@ static void tcp_parse_fastopen_option(int len, const unsigned char *cookie,
 	foc->exp = exp_opt;
 }
 
-static int smc_parse_options(const struct tcphdr *th,
-			     struct tcp_options_received *opt_rx,
-			     const unsigned char *ptr,
-			     int opsize)
-{
-#if IS_ENABLED(CONFIG_SMC)
-	if (static_branch_unlikely(&tcp_have_smc)) {
-		if (th->syn && !(opsize & 1) &&
-		    opsize >= TCPOLEN_EXP_SMC_BASE &&
-		    get_unaligned_be32(ptr) == TCPOPT_SMC_MAGIC) {
-			opt_rx->smc_ok = 1;
-			return 1;
-		}
-	}
-#endif
-	return 0;
-}
-
 /* Look for tcp options. Normally only called on SYN and SYNACK packets.
  * But, this can also be called on packets in the established flow when
  * the fast version below fails.
@@ -3793,9 +3775,6 @@ void tcp_parse_options(const struct net *net,
 					tcp_parse_fastopen_option(opsize -
 						TCPOLEN_EXP_FASTOPEN_BASE,
 						ptr + 2, th->syn, foc, true);
-				else if (smc_parse_options(th, opt_rx, ptr,
-							   opsize))
-					break;
 				else if (static_branch_unlikely(&tcp_extra_options_enabled) &&
 					 opsize >= TCPOLEN_EXP_BASE)
 					tcp_extra_options_parse(get_unaligned_be32(ptr),
@@ -5574,16 +5553,6 @@ static bool tcp_rcv_fastopen_synack(struct sock *sk, struct sk_buff *synack,
 	return false;
 }
 
-static void smc_check_reset_syn(struct tcp_sock *tp)
-{
-#if IS_ENABLED(CONFIG_SMC)
-	if (static_branch_unlikely(&tcp_have_smc)) {
-		if (tp->syn_smc && !tp->rx_opt.smc_ok)
-			tp->syn_smc = 0;
-	}
-#endif
-}
-
 static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
 					 const struct tcphdr *th)
 {
@@ -5694,8 +5663,6 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
 		 * is initialized. */
 		tp->copied_seq = tp->rcv_nxt;
 
-		smc_check_reset_syn(tp);
-
 		smp_mb();
 
 		tcp_finish_connect(sk, skb);
@@ -6152,9 +6119,6 @@ static void tcp_openreq_init(struct request_sock *req,
 	ireq->ir_rmt_port = tcp_hdr(skb)->source;
 	ireq->ir_num = ntohs(tcp_hdr(skb)->dest);
 	ireq->ir_mark = inet_request_mark(sk, skb);
-#if IS_ENABLED(CONFIG_SMC)
-	ireq->smc_ok = rx_opt->smc_ok;
-#endif
 	INIT_LIST_HEAD(&tcp_rsk(req)->tcp_option_list);
 }
 
net/ipv4/tcp_minisocks.c +0 -18
index c3a0a4676fc8..641dca38cc1e 100644
@@ -429,21 +429,6 @@ void tcp_ca_openreq_child(struct sock *sk, const struct dst_entry *dst)
 }
 EXPORT_SYMBOL_GPL(tcp_ca_openreq_child);
 
-static void smc_check_reset_syn_req(struct tcp_sock *oldtp,
-				    struct request_sock *req,
-				    struct tcp_sock *newtp)
-{
-#if IS_ENABLED(CONFIG_SMC)
-	struct inet_request_sock *ireq;
-
-	if (static_branch_unlikely(&tcp_have_smc)) {
-		ireq = inet_rsk(req);
-		if (oldtp->syn_smc && !ireq->smc_ok)
-			newtp->syn_smc = 0;
-	}
-#endif
-}
-
 /* This is not only more efficient than what we used to do, it eliminates
  * a lot of code duplication between IPv4/IPv6 SYN recv processing. -DaveM
  *
@@ -461,9 +446,6 @@ struct sock *tcp_create_openreq_child(const struct sock *sk,
 		struct tcp_request_sock *treq = tcp_rsk(req);
 		struct inet_connection_sock *newicsk = inet_csk(newsk);
 		struct tcp_sock *newtp = tcp_sk(newsk);
-		struct tcp_sock *oldtp = tcp_sk(sk);
-
-		smc_check_reset_syn_req(oldtp, req, newtp);
 
 		/* Now setup tcp_sock */
 		newtp->pred_flags = 0;
net/ipv4/tcp_output.c +0 -54

82 lines not on this page · raw message

net/smc/af_smc.c +164 -6

260 lines not on this page · raw message

Signature · 1 line
2.15.0
author
#40 lore raw

PATCH v2 21/25 tcp_md5: Check for TCP_MD5 after TCP Timestamps in tcp_established_options

In reply to Christoph Paasch

It really does not matter, because we never use TCP timestamps when TCP_MD5 is enabled (see tcp_syn_options).

Moving TCP_MD5 a bit lower allows for easier adoption of the tcp_extra_option framework.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 net/ipv4/tcp_output.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
net/ipv4/tcp_output.c +7 -7
index 2c9c6a1f3476..a7de79294ee6 100644
@@ -663,6 +663,13 @@ static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb
 
 	opts->options = 0;
 
+	if (likely(tp->rx_opt.tstamp_ok)) {
+		opts->options |= OPTION_TS;
+		opts->tsval = skb ? tcp_skb_timestamp(skb) + tp->tsoffset : 0;
+		opts->tsecr = tp->rx_opt.ts_recent;
+		size += TCPOLEN_TSTAMP_ALIGNED;
+	}
+
 #ifdef CONFIG_TCP_MD5SIG
 	opts->md5 = tp->af_specific->md5_lookup(sk, sk);
 	if (unlikely(opts->md5)) {
@@ -673,13 +680,6 @@ static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb
 	opts->md5 = NULL;
 #endif
 
-	if (likely(tp->rx_opt.tstamp_ok)) {
-		opts->options |= OPTION_TS;
-		opts->tsval = skb ? tcp_skb_timestamp(skb) + tp->tsoffset : 0;
-		opts->tsecr = tp->rx_opt.ts_recent;
-		size += TCPOLEN_TSTAMP_ALIGNED;
-	}
-
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
 		size += tcp_extra_options_prepare(skb, 0,
 						  MAX_TCP_OPTION_SPACE - size,
Signature · 1 line
2.15.0
author
#41 lore raw

PATCH v2 22/25 tcp_md5: Move TCP-MD5 code out of TCP itself

In reply to Christoph Paasch

This is all just copy-pasting the TCP_MD5-code into functions that are placed in net/ipv4/tcp_md5.c.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/linux/inet_diag.h |    1 +
 include/linux/tcp_md5.h   |  138 ++++++
 include/net/tcp.h         |   77 ----
 net/ipv4/Makefile         |    1 +
 net/ipv4/tcp.c            |  133 +-----
 net/ipv4/tcp_diag.c       |   81 +---
 net/ipv4/tcp_input.c      |   38 --
 net/ipv4/tcp_ipv4.c       |  521 ++-------------------
 net/ipv4/tcp_md5.c        | 1104 +++++++++++++++++++++++++++++++++++++++++++++
 net/ipv4/tcp_minisocks.c  |   26 +-
 net/ipv4/tcp_output.c     |    4 +-
 net/ipv6/tcp_ipv6.c       |  325 +------------
 12 files changed, 1305 insertions(+), 1144 deletions(-)
 create mode 100644 include/linux/tcp_md5.h
 create mode 100644 net/ipv4/tcp_md5.c
include/linux/inet_diag.h +1 -0
index 39faaaf843e1..1ef6727e41c9 100644
@@ -2,6 +2,7 @@
 #ifndef _INET_DIAG_H_
 #define _INET_DIAG_H_ 1
 
+#include <linux/user_namespace.h>
 #include <uapi/linux/inet_diag.h>
 
 struct net;
include/linux/tcp_md5.h new file +138 -0

138 lines not on this page · raw message

include/net/tcp.h +0 -77

95 lines not on this page · raw message

net/ipv4/Makefile +1 -0
index c6c8ad1d4b6d..9262d9a01035 100644
@@ -61,6 +61,7 @@ obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o
 obj-$(CONFIG_TCP_CONG_YEAH) += tcp_yeah.o
 obj-$(CONFIG_TCP_CONG_ILLINOIS) += tcp_illinois.o
 obj-$(CONFIG_NETLABEL) += cipso_ipv4.o
+obj-$(CONFIG_TCP_MD5SIG) += tcp_md5.o
 
 obj-$(CONFIG_XFRM) += xfrm4_policy.o xfrm4_state.o xfrm4_input.o \
 		      xfrm4_output.o xfrm4_protocol.o
net/ipv4/tcp.c +1 -132

145 lines not on this page · raw message

net/ipv4/tcp_diag.c +6 -75

103 lines not on this page · raw message

net/ipv4/tcp_input.c +0 -38
index 5dd19c3f37ce..09778d8104b8 100644
@@ -3848,44 +3848,6 @@ static bool tcp_fast_parse_options(const struct net *net,
 	return false;
 }
 
-#ifdef CONFIG_TCP_MD5SIG
-/*
- * Parse MD5 Signature option
- */
-const u8 *tcp_parse_md5sig_option(const struct tcphdr *th)
-{
-	int length = (th->doff << 2) - sizeof(*th);
-	const u8 *ptr = (const u8 *)(th + 1);
-
-	/* If the TCP option is too short, we can short cut */
-	if (length < TCPOLEN_MD5SIG)
-		return NULL;
-
-	while (length > 0) {
-		int opcode = *ptr++;
-		int opsize;
-
-		switch (opcode) {
-		case TCPOPT_EOL:
-			return NULL;
-		case TCPOPT_NOP:
-			length--;
-			continue;
-		default:
-			opsize = *ptr++;
-			if (opsize < 2 || opsize > length)
-				return NULL;
-			if (opcode == TCPOPT_MD5SIG)
-				return opsize == TCPOLEN_MD5SIG ? ptr : NULL;
-		}
-		ptr += opsize - 2;
-		length -= opsize;
-	}
-	return NULL;
-}
-EXPORT_SYMBOL(tcp_parse_md5sig_option);
-#endif
-
 /* Sorry, PAWS as specified is broken wrt. pure-ACKs -DaveM
  *
  * It is not fatal. If this ACK does _not_ change critical state (seqs, window)
net/ipv4/tcp_ipv4.c +33 -488

643 lines not on this page · raw message

net/ipv4/tcp_md5.c new file +1104 -0

1,104 lines not on this page · raw message

net/ipv4/tcp_minisocks.c +4 -22
index 641dca38cc1e..e9e925d51046 100644
@@ -22,6 +22,7 @@
 #include <linux/module.h>
 #include <linux/slab.h>
 #include <linux/sysctl.h>
+#include <linux/tcp_md5.h>
 #include <linux/workqueue.h>
 #include <linux/static_key.h>
 #include <net/tcp.h>
@@ -296,21 +297,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 			INIT_LIST_HEAD(&tcp_sk(sk)->tcp_option_list);
 		}
 #ifdef CONFIG_TCP_MD5SIG
-		/*
-		 * The timewait bucket does not have the key DB from the
-		 * sock structure. We just make a quick copy of the
-		 * md5 key being used (if indeed we are using one)
-		 * so the timewait ack generating code has the key.
-		 */
-		do {
-			struct tcp_md5sig_key *key;
-			tcptw->tw_md5_key = NULL;
-			key = tp->af_specific->md5_lookup(sk, sk);
-			if (key) {
-				tcptw->tw_md5_key = kmemdup(key, sizeof(*key), GFP_ATOMIC);
-				BUG_ON(tcptw->tw_md5_key && !tcp_alloc_md5sig_pool());
-			}
-		} while (0);
+		tcp_md5_time_wait(sk, tw);
 #endif
 
 		/* Get the TIME_WAIT timeout firing. */
@@ -340,10 +327,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 void tcp_twsk_destructor(struct sock *sk)
 {
 #ifdef CONFIG_TCP_MD5SIG
-	struct tcp_timewait_sock *twsk = tcp_twsk(sk);
-
-	if (twsk->tw_md5_key)
-		kfree_rcu(twsk->tw_md5_key, rcu);
+	tcp_md5_twsk_destructor(sk);
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
@@ -531,9 +515,7 @@ struct sock *tcp_create_openreq_child(const struct sock *sk,
 		}
 		newtp->tsoffset = treq->ts_off;
 #ifdef CONFIG_TCP_MD5SIG
-		newtp->md5sig_info = NULL;	/*XXX*/
-		if (newtp->af_specific->md5_lookup(sk, newsk))
-			newtp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
+		tcp_md5_add_header_len(sk, newsk);
 #endif
 		if (static_branch_unlikely(&tcp_extra_options_enabled))
 			newtp->tcp_header_len += tcp_extra_options_add_header(sk, newsk);
net/ipv4/tcp_output.c +2 -2
index a7de79294ee6..1c040c44fbf4 100644
@@ -42,6 +42,7 @@
 #include <linux/gfp.h>
 #include <linux/module.h>
 #include <linux/static_key.h>
+#include <linux/tcp_md5.h>
 
 #include <trace/events/tcp.h>
 
@@ -3245,8 +3246,7 @@ static void tcp_connect_init(struct sock *sk)
 		tp->tcp_header_len += TCPOLEN_TSTAMP_ALIGNED;
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (tp->af_specific->md5_lookup(sk, sk))
-		tp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
+	tcp_md5_add_header_len(sk, sk);
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
net/ipv6/tcp_ipv6.c +15 -310

401 lines not on this page · raw message

Signature · 1 line
2.15.0
#42 lore raw

Re: [MPTCP] [PATCH v2 22/25] tcp_md5: Move TCP-MD5 code out of TCP itself

Quoted 2,316 lines · Christoph Paasch

This is all just copy-pasting the TCP_MD5-code into functions that are placed in net/ipv4/tcp_md5.c.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---

include/linux/inet_diag.h |    1 +
include/linux/tcp_md5.h   |  138 ++++++
include/net/tcp.h         |   77 ----
net/ipv4/Makefile         |    1 +
net/ipv4/tcp.c            |  133 +-----
net/ipv4/tcp_diag.c       |   81 +---
net/ipv4/tcp_input.c      |   38 --
net/ipv4/tcp_ipv4.c       |  521 ++-------------------
net/ipv4/tcp_md5.c        | 1104 +++++++++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_minisocks.c  |   26 +-
net/ipv4/tcp_output.c     |    4 +-
net/ipv6/tcp_ipv6.c       |  325 +------------
12 files changed, 1305 insertions(+), 1144 deletions(-)

create mode 100644 include/linux/tcp_md5.h
create mode 100644 net/ipv4/tcp_md5.c

include/linux/inet_diag.h +1 -0
index 39faaaf843e1..1ef6727e41c9 100644
@@ -2,6 +2,7 @@
 #ifndef _INET_DIAG_H_
 #define _INET_DIAG_H_ 1
 
+#include <linux/user_namespace.h>
 #include <uapi/linux/inet_diag.h>
 
 struct net;
include/linux/tcp_md5.h new file +138 -0
new file mode 100644
index 000000000000..27f970f0f464
@@ -0,0 +1,138 @@
+#ifndef _LINUX_TCP_MD5_H
+#define _LINUX_TCP_MD5_H
+
+#ifdef CONFIG_TCP_MD5SIG
+
+#include <linux/types.h>
+
+#include <net/tcp.h>
+
+union tcp_md5_addr {
+	struct in_addr  a4;
+#if IS_ENABLED(CONFIG_IPV6)
+	struct in6_addr	a6;
+#endif
+};
+
+/* - key database */
+struct tcp_md5sig_key {
+	struct hlist_node	node;
+	u8			keylen;
+	u8			family; /* AF_INET or AF_INET6 */
+	union tcp_md5_addr	addr;
+	u8			prefixlen;
+	u8			key[TCP_MD5SIG_MAXKEYLEN];
+	struct rcu_head		rcu;
+};
+
+/* - sock block */
+struct tcp_md5sig_info {
+	struct hlist_head	head;
+	struct rcu_head		rcu;
+};
+
+union tcp_md5sum_block {
+	struct tcp4_pseudohdr ip4;
+#if IS_ENABLED(CONFIG_IPV6)
+	struct tcp6_pseudohdr ip6;
+#endif
+};
+
+/* - pool: digest algorithm, hash description and scratch buffer */
+struct tcp_md5sig_pool {
+	struct ahash_request	*md5_req;
+	void			*scratch;
+};
+
+extern const struct tcp_sock_af_ops tcp_sock_ipv4_specific;
+extern const struct tcp_sock_af_ops tcp_sock_ipv6_specific;
+extern const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific;
+
+/* - functions */
+int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
+			const struct sock *sk, const struct sk_buff *skb);
+
+struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
+					 const struct sock *addr_sk);
+
+void tcp_v4_md5_destroy_sock(struct sock *sk);
+
+int tcp_v4_md5_send_response_prepare(struct sk_buff *skb, u8 flags,
+				     unsigned int remaining,
+				     struct tcp_out_options *opts,
+				     const struct sock *sk);
+
+void tcp_v4_md5_send_response_write(__be32 *topt, struct sk_buff *skb,
+				    struct tcphdr *t1,
+				    struct tcp_out_options *opts,
+				    const struct sock *sk);
+
+int tcp_v6_md5_send_response_prepare(struct sk_buff *skb, u8 flags,
+				     unsigned int remaining,
+				     struct tcp_out_options *opts,
+				     const struct sock *sk);
+
+void tcp_v6_md5_send_response_write(__be32 *topt, struct sk_buff *skb,
+				    struct tcphdr *t1,
+				    struct tcp_out_options *opts,
+				    const struct sock *sk);
+
+bool tcp_v4_inbound_md5_hash(const struct sock *sk,
+			     const struct sk_buff *skb);
+
+void tcp_v4_md5_syn_recv_sock(const struct sock *listener, struct sock *sk);
+
+void tcp_v6_md5_syn_recv_sock(const struct sock *listener, struct sock *sk);
+
+void tcp_md5_time_wait(struct sock *sk, struct inet_timewait_sock *tw);
+
+struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
+					 const struct sock *addr_sk);
+
+int tcp_v6_md5_hash_skb(char *md5_hash,
+			const struct tcp_md5sig_key *key,
+			const struct sock *sk,
+			const struct sk_buff *skb);
+
+bool tcp_v6_inbound_md5_hash(const struct sock *sk,
+			     const struct sk_buff *skb);
+
+static inline void tcp_md5_twsk_destructor(struct sock *sk)
+{
+	struct tcp_timewait_sock *twsk = tcp_twsk(sk);
+
+	if (twsk->tw_md5_key)
+		kfree_rcu(twsk->tw_md5_key, rcu);
+}
+
+static inline void tcp_md5_add_header_len(const struct sock *listener,
+					  struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+
+	tp->md5sig_info = NULL;	/*XXX*/
+	if (tp->af_specific->md5_lookup(listener, sk))
+		tp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
+}
+
+int tcp_md5_diag_get_aux(struct sock *sk, bool net_admin, struct sk_buff *skb);
+
+int tcp_md5_diag_get_aux_size(struct sock *sk, bool net_admin);
+
+#else
+
+static inline bool tcp_v4_inbound_md5_hash(const struct sock *sk,
+					   const struct sk_buff *skb)
+{
+	return false;
+}
+
+static inline bool tcp_v6_inbound_md5_hash(const struct sock *sk,
+					   const struct sk_buff *skb)
+{
+	return false;
+}
+
+#endif
+
+#endif /* _LINUX_TCP_MD5_H */
include/net/tcp.h +0 -77
index beedb639e664..058715ab3a45 100644
@@ -406,7 +406,6 @@ void tcp_parse_options(const struct net *net, const struct sk_buff *skb,
 		       struct tcp_options_received *opt_rx,
 		       int estab, struct tcp_fastopen_cookie *foc,
 		       struct sock *sk);
-const u8 *tcp_parse_md5sig_option(const struct tcphdr *th);
 
 /*
 *	TCP v4 functions exported for the inet6 API
@@ -1416,30 +1415,6 @@ static inline void tcp_clear_all_retrans_hints(struct tcp_sock *tp)
 	tp->retransmit_skb_hint = NULL;
 }
 
-union tcp_md5_addr {
-	struct in_addr  a4;
-#if IS_ENABLED(CONFIG_IPV6)
-	struct in6_addr	a6;
-#endif
-};
-
-/* - key database */
-struct tcp_md5sig_key {
-	struct hlist_node	node;
-	u8			keylen;
-	u8			family; /* AF_INET or AF_INET6 */
-	union tcp_md5_addr	addr;
-	u8			prefixlen;
-	u8			key[TCP_MD5SIG_MAXKEYLEN];
-	struct rcu_head		rcu;
-};
-
-/* - sock block */
-struct tcp_md5sig_info {
-	struct hlist_head	head;
-	struct rcu_head		rcu;
-};
-
 /* - pseudo header */
 struct tcp4_pseudohdr {
 	__be32		saddr;
@@ -1456,58 +1431,6 @@ struct tcp6_pseudohdr {
 	__be32		protocol;	/* including padding */
 };
 
-union tcp_md5sum_block {
-	struct tcp4_pseudohdr ip4;
-#if IS_ENABLED(CONFIG_IPV6)
-	struct tcp6_pseudohdr ip6;
-#endif
-};
-
-/* - pool: digest algorithm, hash description and scratch buffer */
-struct tcp_md5sig_pool {
-	struct ahash_request	*md5_req;
-	void			*scratch;
-};
-
-/* - functions */
-int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
-			const struct sock *sk, const struct sk_buff *skb);
-int tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
-		   int family, u8 prefixlen, const u8 *newkey, u8 newkeylen,
-		   gfp_t gfp);
-int tcp_md5_do_del(struct sock *sk, const union tcp_md5_addr *addr,
-		   int family, u8 prefixlen);
-struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
-					 const struct sock *addr_sk);
-
-#ifdef CONFIG_TCP_MD5SIG
-struct tcp_md5sig_key *tcp_md5_do_lookup(const struct sock *sk,
-					 const union tcp_md5_addr *addr,
-					 int family);
-#define tcp_twsk_md5_key(twsk)	((twsk)->tw_md5_key)
-#else
-static inline struct tcp_md5sig_key *tcp_md5_do_lookup(const struct sock *sk,
-					 const union tcp_md5_addr *addr,
-					 int family)
-{
-	return NULL;
-}
-#define tcp_twsk_md5_key(twsk)	NULL
-#endif
-
-bool tcp_alloc_md5sig_pool(void);
-
-struct tcp_md5sig_pool *tcp_get_md5sig_pool(void);
-static inline void tcp_put_md5sig_pool(void)
-{
-	local_bh_enable();
-}
-
-int tcp_md5_hash_skb_data(struct tcp_md5sig_pool *, const struct sk_buff *,
-			  unsigned int header_len);
-int tcp_md5_hash_key(struct tcp_md5sig_pool *hp,
-		     const struct tcp_md5sig_key *key);
-
 /* From tcp_fastopen.c */
 void tcp_fastopen_cache_get(struct sock *sk, u16 *mss,
 			    struct tcp_fastopen_cookie *cookie, int *syn_loss,
net/ipv4/Makefile +0 -0
index c6c8ad1d4b6d..9262d9a01035 100644
@@ -61,6 +61,7 @@ obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o

obj-$(CONFIG_TCP_CONG_YEAH) += tcp_yeah.o
obj-$(CONFIG_TCP_CONG_ILLINOIS) += tcp_illinois.o
obj-$(CONFIG_NETLABEL) += cipso_ipv4.o
+obj-$(CONFIG_TCP_MD5SIG) += tcp_md5.o

obj-$(CONFIG_XFRM) += xfrm4_policy.o xfrm4_state.o xfrm4_input.o \
xfrm4_output.o xfrm4_protocol.o

net/ipv4/tcp.c +1 -132
index 3090fac3c462..ca1ddb2c820c 100644
@@ -271,6 +271,7 @@
 #include <linux/slab.h>
 #include <linux/errqueue.h>
 #include <linux/static_key.h>
+#include <linux/tcp_md5.h>
 
 #include <net/icmp.h>
 #include <net/inet_common.h>
@@ -3342,138 +3343,6 @@ int compat_tcp_getsockopt(struct sock *sk, int level, int optname,
 EXPORT_SYMBOL(compat_tcp_getsockopt);
 #endif
 
-#ifdef CONFIG_TCP_MD5SIG
-static DEFINE_PER_CPU(struct tcp_md5sig_pool, tcp_md5sig_pool);
-static DEFINE_MUTEX(tcp_md5sig_mutex);
-static bool tcp_md5sig_pool_populated = false;
-
-static void __tcp_alloc_md5sig_pool(void)
-{
-	struct crypto_ahash *hash;
-	int cpu;
-
-	hash = crypto_alloc_ahash("md5", 0, CRYPTO_ALG_ASYNC);
-	if (IS_ERR(hash))
-		return;
-
-	for_each_possible_cpu(cpu) {
-		void *scratch = per_cpu(tcp_md5sig_pool, cpu).scratch;
-		struct ahash_request *req;
-
-		if (!scratch) {
-			scratch = kmalloc_node(sizeof(union tcp_md5sum_block) +
-					       sizeof(struct tcphdr),
-					       GFP_KERNEL,
-					       cpu_to_node(cpu));
-			if (!scratch)
-				return;
-			per_cpu(tcp_md5sig_pool, cpu).scratch = scratch;
-		}
-		if (per_cpu(tcp_md5sig_pool, cpu).md5_req)
-			continue;
-
-		req = ahash_request_alloc(hash, GFP_KERNEL);
-		if (!req)
-			return;
-
-		ahash_request_set_callback(req, 0, NULL, NULL);
-
-		per_cpu(tcp_md5sig_pool, cpu).md5_req = req;
-	}
-	/* before setting tcp_md5sig_pool_populated, we must commit all writes
-	 * to memory. See smp_rmb() in tcp_get_md5sig_pool()
-	 */
-	smp_wmb();
-	tcp_md5sig_pool_populated = true;
-}
-
-bool tcp_alloc_md5sig_pool(void)
-{
-	if (unlikely(!tcp_md5sig_pool_populated)) {
-		mutex_lock(&tcp_md5sig_mutex);
-
-		if (!tcp_md5sig_pool_populated)
-			__tcp_alloc_md5sig_pool();
-
-		mutex_unlock(&tcp_md5sig_mutex);
-	}
-	return tcp_md5sig_pool_populated;
-}
-EXPORT_SYMBOL(tcp_alloc_md5sig_pool);
-
-
-/**
- *	tcp_get_md5sig_pool - get md5sig_pool for this user
- *
- *	We use percpu structure, so if we succeed, we exit with preemption
- *	and BH disabled, to make sure another thread or softirq handling
- *	wont try to get same context.
- */
-struct tcp_md5sig_pool *tcp_get_md5sig_pool(void)
-{
-	local_bh_disable();
-
-	if (tcp_md5sig_pool_populated) {
-		/* coupled with smp_wmb() in __tcp_alloc_md5sig_pool() */
-		smp_rmb();
-		return this_cpu_ptr(&tcp_md5sig_pool);
-	}
-	local_bh_enable();
-	return NULL;
-}
-EXPORT_SYMBOL(tcp_get_md5sig_pool);
-
-int tcp_md5_hash_skb_data(struct tcp_md5sig_pool *hp,
-			  const struct sk_buff *skb, unsigned int header_len)
-{
-	struct scatterlist sg;
-	const struct tcphdr *tp = tcp_hdr(skb);
-	struct ahash_request *req = hp->md5_req;
-	unsigned int i;
-	const unsigned int head_data_len = skb_headlen(skb) > header_len ?
-					   skb_headlen(skb) - header_len : 0;
-	const struct skb_shared_info *shi = skb_shinfo(skb);
-	struct sk_buff *frag_iter;
-
-	sg_init_table(&sg, 1);
-
-	sg_set_buf(&sg, ((u8 *) tp) + header_len, head_data_len);
-	ahash_request_set_crypt(req, &sg, NULL, head_data_len);
-	if (crypto_ahash_update(req))
-		return 1;
-
-	for (i = 0; i < shi->nr_frags; ++i) {
-		const struct skb_frag_struct *f = &shi->frags[i];
-		unsigned int offset = f->page_offset;
-		struct page *page = skb_frag_page(f) + (offset >> PAGE_SHIFT);
-
-		sg_set_page(&sg, page, skb_frag_size(f),
-			    offset_in_page(offset));
-		ahash_request_set_crypt(req, &sg, NULL, skb_frag_size(f));
-		if (crypto_ahash_update(req))
-			return 1;
-	}
-
-	skb_walk_frags(skb, frag_iter)
-		if (tcp_md5_hash_skb_data(hp, frag_iter, 0))
-			return 1;
-
-	return 0;
-}
-EXPORT_SYMBOL(tcp_md5_hash_skb_data);
-
-int tcp_md5_hash_key(struct tcp_md5sig_pool *hp, const struct tcp_md5sig_key *key)
-{
-	struct scatterlist sg;
-
-	sg_init_one(&sg, key->key, key->keylen);
-	ahash_request_set_crypt(hp->md5_req, &sg, NULL, key->keylen);
-	return crypto_ahash_update(hp->md5_req);
-}
-EXPORT_SYMBOL(tcp_md5_hash_key);
-
-#endif
-
 static struct list_head *tcp_extra_options_get_list(const struct sock *sk)
 {
 	if (sk_fullsock(sk))
net/ipv4/tcp_diag.c +6 -75
index abbf0edcf6c2..5cfe5dc8f8dd 100644
@@ -15,6 +15,7 @@
 #include <linux/inet_diag.h>
 
 #include <linux/tcp.h>
+#include <linux/tcp_md5.h>
 
 #include <net/netlink.h>
 #include <net/tcp.h>
@@ -37,70 +38,14 @@ static void tcp_diag_get_info(struct sock *sk, struct inet_diag_msg *r,
 		tcp_get_info(sk, info);
 }
 
-#ifdef CONFIG_TCP_MD5SIG
-static void tcp_diag_md5sig_fill(struct tcp_diag_md5sig *info,
-				 const struct tcp_md5sig_key *key)
-{
-	info->tcpm_family = key->family;
-	info->tcpm_prefixlen = key->prefixlen;
-	info->tcpm_keylen = key->keylen;
-	memcpy(info->tcpm_key, key->key, key->keylen);
-
-	if (key->family == AF_INET)
-		info->tcpm_addr[0] = key->addr.a4.s_addr;
-	#if IS_ENABLED(CONFIG_IPV6)
-	else if (key->family == AF_INET6)
-		memcpy(&info->tcpm_addr, &key->addr.a6,
-		       sizeof(info->tcpm_addr));
-	#endif
-}
-
-static int tcp_diag_put_md5sig(struct sk_buff *skb,
-			       const struct tcp_md5sig_info *md5sig)
-{
-	const struct tcp_md5sig_key *key;
-	struct tcp_diag_md5sig *info;
-	struct nlattr *attr;
-	int md5sig_count = 0;
-
-	hlist_for_each_entry_rcu(key, &md5sig->head, node)
-		md5sig_count++;
-	if (md5sig_count == 0)
-		return 0;
-
-	attr = nla_reserve(skb, INET_DIAG_MD5SIG,
-			   md5sig_count * sizeof(struct tcp_diag_md5sig));
-	if (!attr)
-		return -EMSGSIZE;
-
-	info = nla_data(attr);
-	memset(info, 0, md5sig_count * sizeof(struct tcp_diag_md5sig));
-	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
-		tcp_diag_md5sig_fill(info++, key);
-		if (--md5sig_count == 0)
-			break;
-	}
-
-	return 0;
-}
-#endif
-
 static int tcp_diag_get_aux(struct sock *sk, bool net_admin,
 			    struct sk_buff *skb)
 {
 #ifdef CONFIG_TCP_MD5SIG
-	if (net_admin) {
-		struct tcp_md5sig_info *md5sig;
-		int err = 0;
-
-		rcu_read_lock();
-		md5sig = rcu_dereference(tcp_sk(sk)->md5sig_info);
-		if (md5sig)
-			err = tcp_diag_put_md5sig(skb, md5sig);
-		rcu_read_unlock();
-		if (err < 0)
-			return err;
-	}
+	int err = tcp_md5_diag_get_aux(sk, net_admin, skb);
+
+	if (err < 0)
+		return err;
 #endif
 
 	return 0;
@@ -111,21 +56,7 @@ static size_t tcp_diag_get_aux_size(struct sock *sk, bool net_admin)
 	size_t size = 0;
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (net_admin && sk_fullsock(sk)) {
-		const struct tcp_md5sig_info *md5sig;
-		const struct tcp_md5sig_key *key;
-		size_t md5sig_count = 0;
-
-		rcu_read_lock();
-		md5sig = rcu_dereference(tcp_sk(sk)->md5sig_info);
-		if (md5sig) {
-			hlist_for_each_entry_rcu(key, &md5sig->head, node)
-				md5sig_count++;
-		}
-		rcu_read_unlock();
-		size += nla_total_size(md5sig_count *
-				       sizeof(struct tcp_diag_md5sig));
-	}
+	size += tcp_md5_diag_get_aux_size(sk, net_admin);
 #endif
 
 	return size;
net/ipv4/tcp_input.c +0 -38
index 5dd19c3f37ce..09778d8104b8 100644
@@ -3848,44 +3848,6 @@ static bool tcp_fast_parse_options(const struct net *net,
 	return false;
 }
 
-#ifdef CONFIG_TCP_MD5SIG
-/*
- * Parse MD5 Signature option
- */
-const u8 *tcp_parse_md5sig_option(const struct tcphdr *th)
-{
-	int length = (th->doff << 2) - sizeof(*th);
-	const u8 *ptr = (const u8 *)(th + 1);
-
-	/* If the TCP option is too short, we can short cut */
-	if (length < TCPOLEN_MD5SIG)
-		return NULL;
-
-	while (length > 0) {
-		int opcode = *ptr++;
-		int opsize;
-
-		switch (opcode) {
-		case TCPOPT_EOL:
-			return NULL;
-		case TCPOPT_NOP:
-			length--;
-			continue;
-		default:
-			opsize = *ptr++;
-			if (opsize < 2 || opsize > length)
-				return NULL;
-			if (opcode == TCPOPT_MD5SIG)
-				return opsize == TCPOLEN_MD5SIG ? ptr : NULL;
-		}
-		ptr += opsize - 2;
-		length -= opsize;
-	}
-	return NULL;
-}
-EXPORT_SYMBOL(tcp_parse_md5sig_option);
-#endif
-
 /* Sorry, PAWS as specified is broken wrt. pure-ACKs -DaveM
 *
 * It is not fatal. If this ACK does _not_ change critical state (seqs, window)
net/ipv4/tcp_ipv4.c +33 -488
index f0db7beda04e..d05162654848 100644
@@ -62,6 +62,7 @@
 #include <linux/init.h>
 #include <linux/times.h>
 #include <linux/slab.h>
+#include <linux/tcp_md5.h>
 
 #include <net/net_namespace.h>
 #include <net/icmp.h>
@@ -87,11 +88,6 @@
 
 #include <trace/events/tcp.h>
 
-#ifdef CONFIG_TCP_MD5SIG
-static int tcp_v4_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
-			       __be32 daddr, __be32 saddr, const struct tcphdr *th);
-#endif
-
 struct inet_hashinfo tcp_hashinfo;
 EXPORT_SYMBOL(tcp_hashinfo);
 
@@ -602,14 +598,8 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 		struct tcphdr th;
 		__be32 opt[(MAX_TCP_OPTION_SPACE >> 2)];
 	} rep;
+	struct tcp_out_options opts;
 	struct ip_reply_arg arg;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key *key = NULL;
-	const __u8 *hash_location = NULL;
-	unsigned char newhash[16];
-	int genhash;
-	struct sock *sk1 = NULL;
-#endif
 	struct net *net;
 	int offset = 0;
 
@@ -623,6 +613,8 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 	if (!sk && skb_rtable(skb)->rt_type != RTN_LOCAL)
 		return;
 
+	memset(&opts, 0, sizeof(opts));
+
 	/* Swap the send and the receive. */
 	memset(&rep, 0, sizeof(rep));
 	rep.th.dest   = th->source;
@@ -643,55 +635,32 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 	arg.iov[0].iov_len  = sizeof(rep.th);
 
 	net = sk ? sock_net(sk) : dev_net(skb_dst(skb)->dev);
-#ifdef CONFIG_TCP_MD5SIG
-	rcu_read_lock();
-	hash_location = tcp_parse_md5sig_option(th);
-	if (sk && sk_fullsock(sk)) {
-		key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)
-					&ip_hdr(skb)->saddr, AF_INET);
-	} else if (hash_location) {
-		/*
-		 * active side is lost. Try to find listening socket through
-		 * source port, and then find md5 key through listening socket.
-		 * we are not loose security here:
-		 * Incoming packet is checked with md5 hash with finding key,
-		 * no RST generated if md5 hash doesn't match.
-		 */
-		sk1 = __inet_lookup_listener(net, &tcp_hashinfo, NULL, 0,
-					     ip_hdr(skb)->saddr,
-					     th->source, ip_hdr(skb)->daddr,
-					     ntohs(th->source), inet_iif(skb),
-					     tcp_v4_sdif(skb));
-		/* don't send rst if it can't find key */
-		if (!sk1)
-			goto out;
 
-		key = tcp_md5_do_lookup(sk1, (union tcp_md5_addr *)
-					&ip_hdr(skb)->saddr, AF_INET);
-		if (!key)
-			goto out;
+#ifdef CONFIG_TCP_MD5DSIG
+{
+	int ret;
 
+	ret = tcp_v4_md5_send_response_prepare(skb, 0,
+					       MAX_TCP_OPTION_SPACE - arg.iov[0].iov_len
+					       &opts, sk);
 
-		genhash = tcp_v4_md5_hash_skb(newhash, key, NULL, skb);
-		if (genhash || memcmp(hash_location, newhash, 16) != 0)
-			goto out;
+	if (ret == -1)
+		return;
 
-	}
+	arg.iov[0].iov_len += ret;
+}
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
 		unsigned int remaining;
-		struct tcp_out_options opts;
 		int used;
 
 		remaining = sizeof(rep.opt);
 #ifdef CONFIG_TCP_MD5SIG
-		if (key)
+		if (opts.md5)
 			remaining -= TCPOLEN_MD5SIG_ALIGNED;
 #endif
 
-		memset(&opts, 0, sizeof(opts));
-
 		used = tcp_extra_options_response_prepare(skb, TCPHDR_RST,
 							  remaining,
 							  &opts, sk);
@@ -706,19 +675,7 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 	}
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (key) {
-		rep.opt[offset++] = htonl((TCPOPT_NOP << 24) |
-					  (TCPOPT_NOP << 16) |
-					  (TCPOPT_MD5SIG << 8) |
-					  TCPOLEN_MD5SIG);
-		/* Update length and the length the header thinks exists */
-		arg.iov[0].iov_len += TCPOLEN_MD5SIG_ALIGNED;
-		rep.th.doff = arg.iov[0].iov_len / 4;
-
-		tcp_v4_md5_hash_hdr((__u8 *)&rep.opt[offset],
-				    key, ip_hdr(skb)->saddr,
-				    ip_hdr(skb)->daddr, &rep.th);
-	}
+	tcp_v4_md5_send_response_write(&rep.opt[offset], skb, &rep.th, &opts, sk);
 #endif
 	arg.csum = csum_tcpudp_nofold(ip_hdr(skb)->daddr,
 				      ip_hdr(skb)->saddr, /* XXX */
@@ -749,11 +706,6 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 	__TCP_INC_STATS(net, TCP_MIB_OUTSEGS);
 	__TCP_INC_STATS(net, TCP_MIB_OUTRSTS);
 	local_bh_enable();
-
-#ifdef CONFIG_TCP_MD5SIG
-out:
-	rcu_read_unlock();
-#endif
 }
 
 /* The code following below sending ACKs in SYN-RECV and TIME-WAIT states
@@ -770,15 +722,14 @@ static void tcp_v4_send_ack(const struct sock *sk,
 		struct tcphdr th;
 		__be32 opt[(MAX_TCP_OPTION_SPACE >> 2)];
 	} rep;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key *key;
-#endif
+	struct tcp_out_options opts;
 	struct net *net = sock_net(sk);
 	struct ip_reply_arg arg;
 	int offset = 0;
 
 	memset(&rep.th, 0, sizeof(struct tcphdr));
 	memset(&arg, 0, sizeof(arg));
+	memset(&opts, 0, sizeof(opts));
 
 	arg.iov[0].iov_base = (unsigned char *)&rep;
 	arg.iov[0].iov_len  = sizeof(rep.th);
@@ -801,27 +752,28 @@ static void tcp_v4_send_ack(const struct sock *sk,
 	rep.th.window  = htons(win);
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (sk->sk_state == TCP_TIME_WAIT) {
-		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
+{
+	int ret;
 
-		key = tcp_twsk_md5_key(tcptw);
-	} else if (sk->sk_state == TCP_NEW_SYN_RECV) {
-		key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&ip_hdr(skb)->daddr,
-					AF_INET);
-	} else {
-		BUG();
-	}
+	ret = tcp_v4_md5_send_response_prepare(skb, 0,
+					       MAX_TCP_OPTION_SPACE - arg.iov[0].iov_len,
+					       &opts, sk);
+
+	if (ret == -1)
+		return;
+
+	arg.iov[0].iov_len += ret;
+}
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
 		unsigned int remaining;
-		struct tcp_out_options opts;
 		int used;
 
 		remaining = sizeof(rep.th) + sizeof(rep.opt) - arg.iov[0].iov_len;
 
 #ifdef CONFIG_TCP_MD5SIG
-		if (key)
+		if (opts.md5)
 			remaining -= TCPOLEN_MD5SIG_ALIGNED;
 #endif
 
@@ -839,17 +791,7 @@ static void tcp_v4_send_ack(const struct sock *sk,
 	}
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (key) {
-		rep.opt[offset++] = htonl((TCPOPT_NOP << 24) |
-					  (TCPOPT_NOP << 16) |
-					  (TCPOPT_MD5SIG << 8) |
-					  TCPOLEN_MD5SIG);
-		arg.iov[0].iov_len += TCPOLEN_MD5SIG_ALIGNED;
-
-		tcp_v4_md5_hash_hdr((__u8 *) &rep.opt[offset],
-				    key, ip_hdr(skb)->saddr,
-				    ip_hdr(skb)->daddr, &rep.th);
-	}
+	tcp_v4_md5_send_response_write(&rep.opt[offset], skb, &rep.th, &opts, sk);
 #endif
 
 	rep.th.doff = arg.iov[0].iov_len / 4;
@@ -960,374 +902,6 @@ static void tcp_v4_reqsk_destructor(struct request_sock *req)
 	kfree(rcu_dereference_protected(inet_rsk(req)->ireq_opt, 1));
 }
 
-#ifdef CONFIG_TCP_MD5SIG
-/*
- * RFC2385 MD5 checksumming requires a mapping of
- * IP address->MD5 Key.
- * We need to maintain these in the sk structure.
- */
-
-/* Find the Key structure for an address.  */
-struct tcp_md5sig_key *tcp_md5_do_lookup(const struct sock *sk,
-					 const union tcp_md5_addr *addr,
-					 int family)
-{
-	const struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_key *key;
-	const struct tcp_md5sig_info *md5sig;
-	__be32 mask;
-	struct tcp_md5sig_key *best_match = NULL;
-	bool match;
-
-	/* caller either holds rcu_read_lock() or socket lock */
-	md5sig = rcu_dereference_check(tp->md5sig_info,
-				       lockdep_sock_is_held(sk));
-	if (!md5sig)
-		return NULL;
-
-	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
-		if (key->family != family)
-			continue;
-
-		if (family == AF_INET) {
-			mask = inet_make_mask(key->prefixlen);
-			match = (key->addr.a4.s_addr & mask) ==
-				(addr->a4.s_addr & mask);
-#if IS_ENABLED(CONFIG_IPV6)
-		} else if (family == AF_INET6) {
-			match = ipv6_prefix_equal(&key->addr.a6, &addr->a6,
-						  key->prefixlen);
-#endif
-		} else {
-			match = false;
-		}
-
-		if (match && (!best_match ||
-			      key->prefixlen > best_match->prefixlen))
-			best_match = key;
-	}
-	return best_match;
-}
-EXPORT_SYMBOL(tcp_md5_do_lookup);
-
-static struct tcp_md5sig_key *tcp_md5_do_lookup_exact(const struct sock *sk,
-						      const union tcp_md5_addr *addr,
-						      int family, u8 prefixlen)
-{
-	const struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_key *key;
-	unsigned int size = sizeof(struct in_addr);
-	const struct tcp_md5sig_info *md5sig;
-
-	/* caller either holds rcu_read_lock() or socket lock */
-	md5sig = rcu_dereference_check(tp->md5sig_info,
-				       lockdep_sock_is_held(sk));
-	if (!md5sig)
-		return NULL;
-#if IS_ENABLED(CONFIG_IPV6)
-	if (family == AF_INET6)
-		size = sizeof(struct in6_addr);
-#endif
-	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
-		if (key->family != family)
-			continue;
-		if (!memcmp(&key->addr, addr, size) &&
-		    key->prefixlen == prefixlen)
-			return key;
-	}
-	return NULL;
-}
-
-struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
-					 const struct sock *addr_sk)
-{
-	const union tcp_md5_addr *addr;
-
-	addr = (const union tcp_md5_addr *)&addr_sk->sk_daddr;
-	return tcp_md5_do_lookup(sk, addr, AF_INET);
-}
-EXPORT_SYMBOL(tcp_v4_md5_lookup);
-
-/* This can be called on a newly created socket, from other files */
-int tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
-		   int family, u8 prefixlen, const u8 *newkey, u8 newkeylen,
-		   gfp_t gfp)
-{
-	/* Add Key to the list */
-	struct tcp_md5sig_key *key;
-	struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_info *md5sig;
-
-	key = tcp_md5_do_lookup_exact(sk, addr, family, prefixlen);
-	if (key) {
-		/* Pre-existing entry - just update that one. */
-		memcpy(key->key, newkey, newkeylen);
-		key->keylen = newkeylen;
-		return 0;
-	}
-
-	md5sig = rcu_dereference_protected(tp->md5sig_info,
-					   lockdep_sock_is_held(sk));
-	if (!md5sig) {
-		md5sig = kmalloc(sizeof(*md5sig), gfp);
-		if (!md5sig)
-			return -ENOMEM;
-
-		sk_nocaps_add(sk, NETIF_F_GSO_MASK);
-		INIT_HLIST_HEAD(&md5sig->head);
-		rcu_assign_pointer(tp->md5sig_info, md5sig);
-	}
-
-	key = sock_kmalloc(sk, sizeof(*key), gfp);
-	if (!key)
-		return -ENOMEM;
-	if (!tcp_alloc_md5sig_pool()) {
-		sock_kfree_s(sk, key, sizeof(*key));
-		return -ENOMEM;
-	}
-
-	memcpy(key->key, newkey, newkeylen);
-	key->keylen = newkeylen;
-	key->family = family;
-	key->prefixlen = prefixlen;
-	memcpy(&key->addr, addr,
-	       (family == AF_INET6) ? sizeof(struct in6_addr) :
-				      sizeof(struct in_addr));
-	hlist_add_head_rcu(&key->node, &md5sig->head);
-	return 0;
-}
-EXPORT_SYMBOL(tcp_md5_do_add);
-
-int tcp_md5_do_del(struct sock *sk, const union tcp_md5_addr *addr, int family,
-		   u8 prefixlen)
-{
-	struct tcp_md5sig_key *key;
-
-	key = tcp_md5_do_lookup_exact(sk, addr, family, prefixlen);
-	if (!key)
-		return -ENOENT;
-	hlist_del_rcu(&key->node);
-	atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
-	kfree_rcu(key, rcu);
-	return 0;
-}
-EXPORT_SYMBOL(tcp_md5_do_del);
-
-static void tcp_clear_md5_list(struct sock *sk)
-{
-	struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_key *key;
-	struct hlist_node *n;
-	struct tcp_md5sig_info *md5sig;
-
-	md5sig = rcu_dereference_protected(tp->md5sig_info, 1);
-
-	hlist_for_each_entry_safe(key, n, &md5sig->head, node) {
-		hlist_del_rcu(&key->node);
-		atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
-		kfree_rcu(key, rcu);
-	}
-}
-
-static int tcp_v4_parse_md5_keys(struct sock *sk, int optname,
-				 char __user *optval, int optlen)
-{
-	struct tcp_md5sig cmd;
-	struct sockaddr_in *sin = (struct sockaddr_in *)&cmd.tcpm_addr;
-	u8 prefixlen = 32;
-
-	if (optlen < sizeof(cmd))
-		return -EINVAL;
-
-	if (copy_from_user(&cmd, optval, sizeof(cmd)))
-		return -EFAULT;
-
-	if (sin->sin_family != AF_INET)
-		return -EINVAL;
-
-	if (optname == TCP_MD5SIG_EXT &&
-	    cmd.tcpm_flags & TCP_MD5SIG_FLAG_PREFIX) {
-		prefixlen = cmd.tcpm_prefixlen;
-		if (prefixlen > 32)
-			return -EINVAL;
-	}
-
-	if (!cmd.tcpm_keylen)
-		return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin->sin_addr.s_addr,
-				      AF_INET, prefixlen);
-
-	if (cmd.tcpm_keylen > TCP_MD5SIG_MAXKEYLEN)
-		return -EINVAL;
-
-	return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin->sin_addr.s_addr,
-			      AF_INET, prefixlen, cmd.tcpm_key, cmd.tcpm_keylen,
-			      GFP_KERNEL);
-}
-
-static int tcp_v4_md5_hash_headers(struct tcp_md5sig_pool *hp,
-				   __be32 daddr, __be32 saddr,
-				   const struct tcphdr *th, int nbytes)
-{
-	struct tcp4_pseudohdr *bp;
-	struct scatterlist sg;
-	struct tcphdr *_th;
-
-	bp = hp->scratch;
-	bp->saddr = saddr;
-	bp->daddr = daddr;
-	bp->pad = 0;
-	bp->protocol = IPPROTO_TCP;
-	bp->len = cpu_to_be16(nbytes);
-
-	_th = (struct tcphdr *)(bp + 1);
-	memcpy(_th, th, sizeof(*th));
-	_th->check = 0;
-
-	sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th));
-	ahash_request_set_crypt(hp->md5_req, &sg, NULL,
-				sizeof(*bp) + sizeof(*th));
-	return crypto_ahash_update(hp->md5_req);
-}
-
-static int tcp_v4_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
-			       __be32 daddr, __be32 saddr, const struct tcphdr *th)
-{
-	struct tcp_md5sig_pool *hp;
-	struct ahash_request *req;
-
-	hp = tcp_get_md5sig_pool();
-	if (!hp)
-		goto clear_hash_noput;
-	req = hp->md5_req;
-
-	if (crypto_ahash_init(req))
-		goto clear_hash;
-	if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2))
-		goto clear_hash;
-	if (tcp_md5_hash_key(hp, key))
-		goto clear_hash;
-	ahash_request_set_crypt(req, NULL, md5_hash, 0);
-	if (crypto_ahash_final(req))
-		goto clear_hash;
-
-	tcp_put_md5sig_pool();
-	return 0;
-
-clear_hash:
-	tcp_put_md5sig_pool();
-clear_hash_noput:
-	memset(md5_hash, 0, 16);
-	return 1;
-}
-
-int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
-			const struct sock *sk,
-			const struct sk_buff *skb)
-{
-	struct tcp_md5sig_pool *hp;
-	struct ahash_request *req;
-	const struct tcphdr *th = tcp_hdr(skb);
-	__be32 saddr, daddr;
-
-	if (sk) { /* valid for establish/request sockets */
-		saddr = sk->sk_rcv_saddr;
-		daddr = sk->sk_daddr;
-	} else {
-		const struct iphdr *iph = ip_hdr(skb);
-		saddr = iph->saddr;
-		daddr = iph->daddr;
-	}
-
-	hp = tcp_get_md5sig_pool();
-	if (!hp)
-		goto clear_hash_noput;
-	req = hp->md5_req;
-
-	if (crypto_ahash_init(req))
-		goto clear_hash;
-
-	if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, skb->len))
-		goto clear_hash;
-	if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2))
-		goto clear_hash;
-	if (tcp_md5_hash_key(hp, key))
-		goto clear_hash;
-	ahash_request_set_crypt(req, NULL, md5_hash, 0);
-	if (crypto_ahash_final(req))
-		goto clear_hash;
-
-	tcp_put_md5sig_pool();
-	return 0;
-
-clear_hash:
-	tcp_put_md5sig_pool();
-clear_hash_noput:
-	memset(md5_hash, 0, 16);
-	return 1;
-}
-EXPORT_SYMBOL(tcp_v4_md5_hash_skb);
-
-#endif
-
-/* Called with rcu_read_lock() */
-static bool tcp_v4_inbound_md5_hash(const struct sock *sk,
-				    const struct sk_buff *skb)
-{
-#ifdef CONFIG_TCP_MD5SIG
-	/*
-	 * This gets called for each TCP segment that arrives
-	 * so we want to be efficient.
-	 * We have 3 drop cases:
-	 * o No MD5 hash and one expected.
-	 * o MD5 hash and we're not expecting one.
-	 * o MD5 hash and its wrong.
-	 */
-	const __u8 *hash_location = NULL;
-	struct tcp_md5sig_key *hash_expected;
-	const struct iphdr *iph = ip_hdr(skb);
-	const struct tcphdr *th = tcp_hdr(skb);
-	int genhash;
-	unsigned char newhash[16];
-
-	hash_expected = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&iph->saddr,
-					  AF_INET);
-	hash_location = tcp_parse_md5sig_option(th);
-
-	/* We've parsed the options - do we have a hash? */
-	if (!hash_expected && !hash_location)
-		return false;
-
-	if (hash_expected && !hash_location) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5NOTFOUND);
-		return true;
-	}
-
-	if (!hash_expected && hash_location) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5UNEXPECTED);
-		return true;
-	}
-
-	/* Okay, so this is hash_expected and hash_location -
-	 * so we need to calculate the checksum.
-	 */
-	genhash = tcp_v4_md5_hash_skb(newhash,
-				      hash_expected,
-				      NULL, skb);
-
-	if (genhash || memcmp(hash_location, newhash, 16) != 0) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5FAILURE);
-		net_info_ratelimited("MD5 Hash failed for (%pI4, %d)->(%pI4, %d)%s\n",
-				     &iph->saddr, ntohs(th->source),
-				     &iph->daddr, ntohs(th->dest),
-				     genhash ? " tcp_v4_calc_md5_hash failed"
-				     : "");
-		return true;
-	}
-	return false;
-#endif
-	return false;
-}
-
 static void tcp_v4_init_req(struct request_sock *req,
 			    const struct sock *sk_listener,
 			    struct sk_buff *skb)
@@ -1403,9 +977,6 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
 	struct inet_sock *newinet;
 	struct tcp_sock *newtp;
 	struct sock *newsk;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key *key;
-#endif
 	struct ip_options_rcu *inet_opt;
 
 	if (sk_acceptq_is_full(sk))
@@ -1457,20 +1028,7 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
 		INIT_LIST_HEAD(&tcp_rsk(req)->tcp_option_list);
 	}
 #ifdef CONFIG_TCP_MD5SIG
-	/* Copy over the MD5 key from the original socket */
-	key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&newinet->inet_daddr,
-				AF_INET);
-	if (key) {
-		/*
-		 * We're using one, so create a matching key
-		 * on the newsk structure. If we fail to get
-		 * memory, then we end up not copying the key
-		 * across. Shucks.
-		 */
-		tcp_md5_do_add(newsk, (union tcp_md5_addr *)&newinet->inet_daddr,
-			       AF_INET, 32, key->key, key->keylen, GFP_ATOMIC);
-		sk_nocaps_add(newsk, NETIF_F_GSO_MASK);
-	}
+	tcp_v4_md5_syn_recv_sock(sk, newsk);
 #endif
 
 	if (__inet_inherit_port(sk, newsk) < 0)
@@ -1908,14 +1466,6 @@ const struct inet_connection_sock_af_ops ipv4_specific = {
 };
 EXPORT_SYMBOL(ipv4_specific);
 
-#ifdef CONFIG_TCP_MD5SIG
-static const struct tcp_sock_af_ops tcp_sock_ipv4_specific = {
-	.md5_lookup		= tcp_v4_md5_lookup,
-	.calc_md5_hash		= tcp_v4_md5_hash_skb,
-	.md5_parse		= tcp_v4_parse_md5_keys,
-};
-#endif
-
 /* NOTE: A lot of things set to zero explicitly by call to
 *       sk_alloc() so need not be done here.
 */
@@ -1958,12 +1508,7 @@ void tcp_v4_destroy_sock(struct sock *sk)
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
 		tcp_extra_options_destroy(sk);
 #ifdef CONFIG_TCP_MD5SIG
-	/* Clean up the MD5 key list, if any */
-	if (tp->md5sig_info) {
-		tcp_clear_md5_list(sk);
-		kfree_rcu(tp->md5sig_info, rcu);
-		tp->md5sig_info = NULL;
-	}
+	tcp_v4_md5_destroy_sock(sk);
 #endif
 
 	/* Clean up a referenced TCP bind bucket. */
net/ipv4/tcp_md5.c new file +1047 -0
new file mode 100644
index 000000000000..a7fbcda0e505
@@ -0,0 +1,1104 @@
+#include <linux/inet_diag.h>
+#include <linux/inetdevice.h>
+#include <linux/tcp.h>
+#include <linux/tcp_md5.h>
+
+#include <crypto/hash.h>
+
+#include <net/inet6_hashtables.h>
+
+static DEFINE_PER_CPU(struct tcp_md5sig_pool, tcp_md5sig_pool);
+static DEFINE_MUTEX(tcp_md5sig_mutex);
+static bool tcp_md5sig_pool_populated;
+
+#define tcp_twsk_md5_key(twsk)	((twsk)->tw_md5_key)
+
+static void __tcp_alloc_md5sig_pool(void)
+{
+	struct crypto_ahash *hash;
+	int cpu;
+
+	hash = crypto_alloc_ahash("md5", 0, CRYPTO_ALG_ASYNC);
+	if (IS_ERR(hash))
+		return;
+
+	for_each_possible_cpu(cpu) {
+		void *scratch = per_cpu(tcp_md5sig_pool, cpu).scratch;
+		struct ahash_request *req;
+
+		if (!scratch) {
+			scratch = kmalloc_node(sizeof(union tcp_md5sum_block) +
+					       sizeof(struct tcphdr),
+					       GFP_KERNEL,
+					       cpu_to_node(cpu));
+			if (!scratch)
+				return;
+			per_cpu(tcp_md5sig_pool, cpu).scratch = scratch;
+		}
+		if (per_cpu(tcp_md5sig_pool, cpu).md5_req)
+			continue;
+
+		req = ahash_request_alloc(hash, GFP_KERNEL);
+		if (!req)
+			return;
+
+		ahash_request_set_callback(req, 0, NULL, NULL);
+
+		per_cpu(tcp_md5sig_pool, cpu).md5_req = req;
+	}
+	/* before setting tcp_md5sig_pool_populated, we must commit all writes
+	 * to memory. See smp_rmb() in tcp_get_md5sig_pool()
+	 */
+	smp_wmb();
+	tcp_md5sig_pool_populated = true;
+}
+
+static bool tcp_alloc_md5sig_pool(void)
+{
+	if (unlikely(!tcp_md5sig_pool_populated)) {
+		mutex_lock(&tcp_md5sig_mutex);
+
+		if (!tcp_md5sig_pool_populated)
+			__tcp_alloc_md5sig_pool();
+
+		mutex_unlock(&tcp_md5sig_mutex);
+	}
+	return tcp_md5sig_pool_populated;
+}
+
+static void tcp_put_md5sig_pool(void)
+{
+	local_bh_enable();
+}
+
+/**
+ *	tcp_get_md5sig_pool - get md5sig_pool for this user
+ *
+ *	We use percpu structure, so if we succeed, we exit with preemption
+ *	and BH disabled, to make sure another thread or softirq handling
+ *	wont try to get same context.
+ */
+static struct tcp_md5sig_pool *tcp_get_md5sig_pool(void)
+{
+	local_bh_disable();
+
+	if (tcp_md5sig_pool_populated) {
+		/* coupled with smp_wmb() in __tcp_alloc_md5sig_pool() */
+		smp_rmb();
+		return this_cpu_ptr(&tcp_md5sig_pool);
+	}
+	local_bh_enable();
+	return NULL;
+}
+
+static struct tcp_md5sig_key *tcp_md5_do_lookup_exact(const struct sock *sk,
+						      const union tcp_md5_addr *addr,
+						      int family, u8 prefixlen)
+{
+	const struct tcp_sock *tp = tcp_sk(sk);
+	struct tcp_md5sig_key *key;
+	unsigned int size = sizeof(struct in_addr);
+	const struct tcp_md5sig_info *md5sig;
+
+	/* caller either holds rcu_read_lock() or socket lock */
+	md5sig = rcu_dereference_check(tp->md5sig_info,
+				       lockdep_sock_is_held(sk));
+	if (!md5sig)
+		return NULL;
+#if IS_ENABLED(CONFIG_IPV6)
+	if (family == AF_INET6)
+		size = sizeof(struct in6_addr);
+#endif
+	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
+		if (key->family != family)
+			continue;
+		if (!memcmp(&key->addr, addr, size) &&
+		    key->prefixlen == prefixlen)
+			return key;
+	}
+	return NULL;
+}
+
+/* This can be called on a newly created socket, from other files */
+static int tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
+			  int family, u8 prefixlen, const u8 *newkey,
+			  u8 newkeylen, gfp_t gfp)
+{
+	/* Add Key to the list */
+	struct tcp_md5sig_key *key;
+	struct tcp_sock *tp = tcp_sk(sk);
+	struct tcp_md5sig_info *md5sig;
+
+	key = tcp_md5_do_lookup_exact(sk, addr, family, prefixlen);
+	if (key) {
+		/* Pre-existing entry - just update that one. */
+		memcpy(key->key, newkey, newkeylen);
+		key->keylen = newkeylen;
+		return 0;
+	}
+
+	md5sig = rcu_dereference_protected(tp->md5sig_info,
+					   lockdep_sock_is_held(sk));
+	if (!md5sig) {
+		md5sig = kmalloc(sizeof(*md5sig), gfp);
+		if (!md5sig)
+			return -ENOMEM;
+
+		sk_nocaps_add(sk, NETIF_F_GSO_MASK);
+		INIT_HLIST_HEAD(&md5sig->head);
+		rcu_assign_pointer(tp->md5sig_info, md5sig);
+	}
+
+	key = sock_kmalloc(sk, sizeof(*key), gfp);
+	if (!key)
+		return -ENOMEM;
+	if (!tcp_alloc_md5sig_pool()) {
+		sock_kfree_s(sk, key, sizeof(*key));
+		return -ENOMEM;
+	}
+
+	memcpy(key->key, newkey, newkeylen);
+	key->keylen = newkeylen;
+	key->family = family;
+	key->prefixlen = prefixlen;
+	memcpy(&key->addr, addr,
+	       (family == AF_INET6) ? sizeof(struct in6_addr) :
+				      sizeof(struct in_addr));
+	hlist_add_head_rcu(&key->node, &md5sig->head);
+	return 0;
+}
+
+static void tcp_clear_md5_list(struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+	struct tcp_md5sig_key *key;
+	struct hlist_node *n;
+	struct tcp_md5sig_info *md5sig;
+
+	md5sig = rcu_dereference_protected(tp->md5sig_info, 1);
+
+	hlist_for_each_entry_safe(key, n, &md5sig->head, node) {
+		hlist_del_rcu(&key->node);
+		atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
+		kfree_rcu(key, rcu);
+	}
+}
+
+static int tcp_md5_do_del(struct sock *sk, const union tcp_md5_addr *addr,
+			  int family, u8 prefixlen)
+{
+	struct tcp_md5sig_key *key;
+
+	key = tcp_md5_do_lookup_exact(sk, addr, family, prefixlen);
+	if (!key)
+		return -ENOENT;
+	hlist_del_rcu(&key->node);
+	atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
+	kfree_rcu(key, rcu);
+	return 0;
+}
+
+static int tcp_md5_hash_key(struct tcp_md5sig_pool *hp,
+			    const struct tcp_md5sig_key *key)
+{
+	struct scatterlist sg;
+
+	sg_init_one(&sg, key->key, key->keylen);
+	ahash_request_set_crypt(hp->md5_req, &sg, NULL, key->keylen);
+	return crypto_ahash_update(hp->md5_req);
+}
+
+static int tcp_v4_parse_md5_keys(struct sock *sk, int optname,
+				 char __user *optval, int optlen)
+{
+	struct tcp_md5sig cmd;
+	struct sockaddr_in *sin = (struct sockaddr_in *)&cmd.tcpm_addr;
+	u8 prefixlen = 32;
+
+	if (optlen < sizeof(cmd))
+		return -EINVAL;
+
+	if (copy_from_user(&cmd, optval, sizeof(cmd)))
+		return -EFAULT;
+
+	if (sin->sin_family != AF_INET)
+		return -EINVAL;
+
+	if (optname == TCP_MD5SIG_EXT &&
+	    cmd.tcpm_flags & TCP_MD5SIG_FLAG_PREFIX) {
+		prefixlen = cmd.tcpm_prefixlen;
+		if (prefixlen > 32)
+			return -EINVAL;
+	}
+
+	if (!cmd.tcpm_keylen)
+		return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin->sin_addr.s_addr,
+				      AF_INET, prefixlen);
+
+	if (cmd.tcpm_keylen > TCP_MD5SIG_MAXKEYLEN)
+		return -EINVAL;
+
+	return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin->sin_addr.s_addr,
+			      AF_INET, prefixlen, cmd.tcpm_key, cmd.tcpm_keylen,
+			      GFP_KERNEL);
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+static int tcp_v6_parse_md5_keys(struct sock *sk, int optname,
+				 char __user *optval, int optlen)
+{
+	struct tcp_md5sig cmd;
+	struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)&cmd.tcpm_addr;
+	u8 prefixlen;
+
+	if (optlen < sizeof(cmd))
+		return -EINVAL;
+
+	if (copy_from_user(&cmd, optval, sizeof(cmd)))
+		return -EFAULT;
+
+	if (sin6->sin6_family != AF_INET6)
+		return -EINVAL;
+
+	if (optname == TCP_MD5SIG_EXT &&
+	    cmd.tcpm_flags & TCP_MD5SIG_FLAG_PREFIX) {
+		prefixlen = cmd.tcpm_prefixlen;
+		if (prefixlen > 128 || (ipv6_addr_v4mapped(&sin6->sin6_addr) &&
+					prefixlen > 32))
+			return -EINVAL;
+	} else {
+		prefixlen = ipv6_addr_v4mapped(&sin6->sin6_addr) ? 32 : 128;
+	}
+
+	if (!cmd.tcpm_keylen) {
+		if (ipv6_addr_v4mapped(&sin6->sin6_addr))
+			return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin6->sin6_addr.s6_addr32[3],
+					      AF_INET, prefixlen);
+		return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin6->sin6_addr,
+				      AF_INET6, prefixlen);
+	}
+
+	if (cmd.tcpm_keylen > TCP_MD5SIG_MAXKEYLEN)
+		return -EINVAL;
+
+	if (ipv6_addr_v4mapped(&sin6->sin6_addr))
+		return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin6->sin6_addr.s6_addr32[3],
+				      AF_INET, prefixlen, cmd.tcpm_key,
+				      cmd.tcpm_keylen, GFP_KERNEL);
+
+	return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin6->sin6_addr,
+			      AF_INET6, prefixlen, cmd.tcpm_key,
+			      cmd.tcpm_keylen, GFP_KERNEL);
+}
+#endif
+
+static int tcp_v4_md5_hash_headers(struct tcp_md5sig_pool *hp,
+				   __be32 daddr, __be32 saddr,
+				   const struct tcphdr *th, int nbytes)
+{
+	struct tcp4_pseudohdr *bp;
+	struct scatterlist sg;
+	struct tcphdr *_th;
+
+	bp = hp->scratch;
+	bp->saddr = saddr;
+	bp->daddr = daddr;
+	bp->pad = 0;
+	bp->protocol = IPPROTO_TCP;
+	bp->len = cpu_to_be16(nbytes);
+
+	_th = (struct tcphdr *)(bp + 1);
+	memcpy(_th, th, sizeof(*th));
+	_th->check = 0;
+
+	sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th));
+	ahash_request_set_crypt(hp->md5_req, &sg, NULL,
+				sizeof(*bp) + sizeof(*th));
+	return crypto_ahash_update(hp->md5_req);
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+static int tcp_v6_md5_hash_headers(struct tcp_md5sig_pool *hp,
+				   const struct in6_addr *daddr,
+				   const struct in6_addr *saddr,
+				   const struct tcphdr *th, int nbytes)
+{
+	struct tcp6_pseudohdr *bp;
+	struct scatterlist sg;
+	struct tcphdr *_th;
+
+	bp = hp->scratch;
+	/* 1. TCP pseudo-header (RFC2460) */
+	bp->saddr = *saddr;
+	bp->daddr = *daddr;
+	bp->protocol = cpu_to_be32(IPPROTO_TCP);
+	bp->len = cpu_to_be32(nbytes);
+
+	_th = (struct tcphdr *)(bp + 1);
+	memcpy(_th, th, sizeof(*th));
+	_th->check = 0;
+
+	sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th));
+	ahash_request_set_crypt(hp->md5_req, &sg, NULL,
+				sizeof(*bp) + sizeof(*th));
+	return crypto_ahash_update(hp->md5_req);
+}
+#endif
+
+static int tcp_v4_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
+			       __be32 daddr, __be32 saddr,
+			       const struct tcphdr *th)
+{
+	struct tcp_md5sig_pool *hp;
+	struct ahash_request *req;
+
+	hp = tcp_get_md5sig_pool();
+	if (!hp)
+		goto clear_hash_noput;
+	req = hp->md5_req;
+
+	if (crypto_ahash_init(req))
+		goto clear_hash;
+	if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2))
+		goto clear_hash;
+	if (tcp_md5_hash_key(hp, key))
+		goto clear_hash;
+	ahash_request_set_crypt(req, NULL, md5_hash, 0);
+	if (crypto_ahash_final(req))
+		goto clear_hash;
+
+	tcp_put_md5sig_pool();
+	return 0;
+
+clear_hash:
+	tcp_put_md5sig_pool();
+clear_hash_noput:
+	memset(md5_hash, 0, 16);
+	return 1;
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+static int tcp_v6_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
+			       const struct in6_addr *daddr,
+			       struct in6_addr *saddr, const struct tcphdr *th)
+{
+	struct tcp_md5sig_pool *hp;
+	struct ahash_request *req;
+
+	hp = tcp_get_md5sig_pool();
+	if (!hp)
+		goto clear_hash_noput;
+	req = hp->md5_req;
+
+	if (crypto_ahash_init(req))
+		goto clear_hash;
+	if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2))
+		goto clear_hash;
+	if (tcp_md5_hash_key(hp, key))
+		goto clear_hash;
+	ahash_request_set_crypt(req, NULL, md5_hash, 0);
+	if (crypto_ahash_final(req))
+		goto clear_hash;
+
+	tcp_put_md5sig_pool();
+	return 0;
+
+clear_hash:
+	tcp_put_md5sig_pool();
+clear_hash_noput:
+	memset(md5_hash, 0, 16);
+	return 1;
+}
+#endif
+
+/* RFC2385 MD5 checksumming requires a mapping of
+ * IP address->MD5 Key.
+ * We need to maintain these in the sk structure.
+ */
+
+/* Find the Key structure for an address.  */
+static struct tcp_md5sig_key *tcp_md5_do_lookup(const struct sock *sk,
+						const union tcp_md5_addr *addr,
+						int family)
+{
+	const struct tcp_sock *tp = tcp_sk(sk);
+	struct tcp_md5sig_key *key;
+	const struct tcp_md5sig_info *md5sig;
+	__be32 mask;
+	struct tcp_md5sig_key *best_match = NULL;
+	bool match;
+
+	/* caller either holds rcu_read_lock() or socket lock */
+	md5sig = rcu_dereference_check(tp->md5sig_info,
+				       lockdep_sock_is_held(sk));
+	if (!md5sig)
+		return NULL;
+
+	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
+		if (key->family != family)
+			continue;
+
+		if (family == AF_INET) {
+			mask = inet_make_mask(key->prefixlen);
+			match = (key->addr.a4.s_addr & mask) ==
+				(addr->a4.s_addr & mask);
+#if IS_ENABLED(CONFIG_IPV6)
+		} else if (family == AF_INET6) {
+			match = ipv6_prefix_equal(&key->addr.a6, &addr->a6,
+						  key->prefixlen);
+#endif
+		} else {
+			match = false;
+		}
+
+		if (match && (!best_match ||
+			      key->prefixlen > best_match->prefixlen))
+			best_match = key;
+	}
+	return best_match;
+}
+
+/* Parse MD5 Signature option */
+static const u8 *tcp_parse_md5sig_option(const struct tcphdr *th)
+{
+	int length = (th->doff << 2) - sizeof(*th);
+	const u8 *ptr = (const u8 *)(th + 1);
+
+	/* If the TCP option is too short, we can short cut */
+	if (length < TCPOLEN_MD5SIG)
+		return NULL;
+
+	while (length > 0) {
+		int opcode = *ptr++;
+		int opsize;
+
+		switch (opcode) {
+		case TCPOPT_EOL:
+			return NULL;
+		case TCPOPT_NOP:
+			length--;
+			continue;
+		default:
+			opsize = *ptr++;
+			if (opsize < 2 || opsize > length)
+				return NULL;
+			if (opcode == TCPOPT_MD5SIG)
+				return opsize == TCPOLEN_MD5SIG ? ptr : NULL;
+		}
+		ptr += opsize - 2;
+		length -= opsize;
+	}
+	return NULL;
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+static struct tcp_md5sig_key *tcp_v6_md5_do_lookup(const struct sock *sk,
+						   const struct in6_addr *addr)
+{
+	return tcp_md5_do_lookup(sk, (union tcp_md5_addr *)addr, AF_INET6);
+}
+#endif
+
+static int tcp_md5_hash_skb_data(struct tcp_md5sig_pool *hp,
+				 const struct sk_buff *skb,
+				 unsigned int header_len)
+{
+	struct scatterlist sg;
+	const struct tcphdr *tp = tcp_hdr(skb);
+	struct ahash_request *req = hp->md5_req;
+	unsigned int i;
+	const unsigned int head_data_len = skb_headlen(skb) > header_len ?
+					   skb_headlen(skb) - header_len : 0;
+	const struct skb_shared_info *shi = skb_shinfo(skb);
+	struct sk_buff *frag_iter;
+
+	sg_init_table(&sg, 1);
+
+	sg_set_buf(&sg, ((u8 *)tp) + header_len, head_data_len);
+	ahash_request_set_crypt(req, &sg, NULL, head_data_len);
+	if (crypto_ahash_update(req))
+		return 1;
+
+	for (i = 0; i < shi->nr_frags; ++i) {
+		const struct skb_frag_struct *f = &shi->frags[i];
+		unsigned int offset = f->page_offset;
+		struct page *page = skb_frag_page(f) + (offset >> PAGE_SHIFT);
+
+		sg_set_page(&sg, page, skb_frag_size(f),
+			    offset_in_page(offset));
+		ahash_request_set_crypt(req, &sg, NULL, skb_frag_size(f));
+		if (crypto_ahash_update(req))
+			return 1;
+	}
+
+	skb_walk_frags(skb, frag_iter)
+		if (tcp_md5_hash_skb_data(hp, frag_iter, 0))
+			return 1;
+
+	return 0;
+}
+
+int tcp_v4_md5_send_response_prepare(struct sk_buff *skb, u8 flags,
+				     unsigned int remaining,
+				     struct tcp_out_options *opts,
+				     const struct sock *sk)
+{
+	const struct tcphdr *th = tcp_hdr(skb);
+	const struct iphdr *iph = ip_hdr(skb);
+	const __u8 *hash_location = NULL;
+
+	rcu_read_lock();
+	hash_location = tcp_parse_md5sig_option(th);
+	if (sk && sk_fullsock(sk)) {
+		opts->md5 = tcp_md5_do_lookup(sk,
+					      (union tcp_md5_addr *)&iph->daddr,
+					      AF_INET);
+	} else if (sk && sk->sk_state == TCP_TIME_WAIT) {
+		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
+
+		opts->md5 = tcp_twsk_md5_key(tcptw);
+	} else if (sk && sk->sk_state == TCP_NEW_SYN_RECV) {
+		opts->md5 = tcp_md5_do_lookup(sk,
+					      (union tcp_md5_addr *)&iph->daddr,
+					      AF_INET);
+	} else if (hash_location) {
+		unsigned char newhash[16];
+		struct sock *sk1;
+		int genhash;
+
+		/* active side is lost. Try to find listening socket through
+		 * source port, and then find md5 key through listening socket.
+		 * we are not loose security here:
+		 * Incoming packet is checked with md5 hash with finding key,
+		 * no RST generated if md5 hash doesn't match.
+		 */
+		sk1 = __inet_lookup_listener(dev_net(skb_dst(skb)->dev),
+					     &tcp_hashinfo, NULL, 0,
+					     iph->saddr,
+					     th->source, iph->daddr,
+					     ntohs(th->source), inet_iif(skb),
+					     tcp_v4_sdif(skb));
+		/* don't send rst if it can't find key */
+		if (!sk1)
+			goto out_err;
+
+		opts->md5 = tcp_md5_do_lookup(sk1, (union tcp_md5_addr *)
+					      &iph->saddr, AF_INET);
+		if (!opts->md5)
+			goto out_err;
+
+		genhash = tcp_v4_md5_hash_skb(newhash, opts->md5, NULL, skb);
+		if (genhash || memcmp(hash_location, newhash, 16) != 0)
+			goto out_err;
+	}
+
+	if (opts->md5)
+		return TCPOLEN_MD5SIG_ALIGNED;
+
+	rcu_read_unlock();
+	return 0;
+
+out_err:
+	rcu_read_unlock();
+	return -1;
+}
+
+
+void tcp_v4_md5_send_response_write(__be32 *topt, struct sk_buff *skb,
+				    struct tcphdr *t1,
+				    struct tcp_out_options *opts,
+				    const struct sock *sk)
+{
+
+	if (opts->md5) {
+		*topt++ = htonl((TCPOPT_NOP << 24) |
+			        (TCPOPT_NOP << 16) |
+			        (TCPOPT_MD5SIG << 8) |
+			        TCPOLEN_MD5SIG);
+
+		tcp_v4_md5_hash_hdr((__u8 *)topt, opts->md5,
+				    ip_hdr(skb)->saddr,
+				    ip_hdr(skb)->daddr, t1);
+	}
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+int tcp_v6_md5_send_response_prepare(struct sk_buff *skb, u8 flags,
+				     unsigned int remaining,
+				     struct tcp_out_options *opts,
+				     const struct sock *sk)
+{
+	const struct tcphdr *th = tcp_hdr(skb);
+	struct ipv6hdr *ipv6h = ipv6_hdr(skb);
+	const __u8 *hash_location = NULL;
+
+	rcu_read_lock();
+	hash_location = tcp_parse_md5sig_option(th);
+	if (sk && sk_fullsock(sk)) {
+		opts->md5 = tcp_v6_md5_do_lookup(sk, &ipv6h->saddr);
+	} else if (sk && sk->sk_state == TCP_TIME_WAIT) {
+		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
+
+		opts->md5 = tcp_twsk_md5_key(tcptw);
+	} else if (sk && sk->sk_state == TCP_NEW_SYN_RECV) {
+		opts->md5 = tcp_v6_md5_do_lookup(sk, &ipv6h->daddr);
+	} else if (hash_location) {
+		unsigned char newhash[16];
+		struct sock *sk1;
+		int genhash;
+
+		/* active side is lost. Try to find listening socket through
+		 * source port, and then find md5 key through listening socket.
+		 * we are not loose security here:
+		 * Incoming packet is checked with md5 hash with finding key,
+		 * no RST generated if md5 hash doesn't match.
+		 */
+		sk1 = inet6_lookup_listener(dev_net(skb_dst(skb)->dev),
+					    &tcp_hashinfo, NULL, 0,
+					    &ipv6h->saddr,
+					    th->source, &ipv6h->daddr,
+					    ntohs(th->source), tcp_v6_iif(skb),
+					    tcp_v6_sdif(skb));
+		if (!sk1)
+			goto out_err;
+
+		opts->md5 = tcp_v6_md5_do_lookup(sk1, &ipv6h->saddr);
+		if (!opts->md5)
+			goto out_err;
+
+		genhash = tcp_v6_md5_hash_skb(newhash, opts->md5, NULL, skb);
+		if (genhash || memcmp(hash_location, newhash, 16) != 0)
+			goto out_err;
+	}
+
+	if (opts->md5)
+		return TCPOLEN_MD5SIG_ALIGNED;
+
+	rcu_read_unlock();
+	return 0;
+
+out_err:
+	rcu_read_unlock();
+	return -1;
+}
+EXPORT_SYMBOL_GPL(tcp_v6_md5_send_response_prepare);
+
+void tcp_v6_md5_send_response_write(__be32 *topt, struct sk_buff *skb,
+				    struct tcphdr *t1,
+				    struct tcp_out_options *opts,
+				    const struct sock *sk)
+{
+	if (opts->md5) {
+		*topt++ = htonl((TCPOPT_NOP << 24) | (TCPOPT_NOP << 16) |
+				(TCPOPT_MD5SIG << 8) | TCPOLEN_MD5SIG);
+		tcp_v6_md5_hash_hdr((__u8 *)topt, opts->md5,
+				    &ipv6_hdr(skb)->saddr,
+				    &ipv6_hdr(skb)->daddr, t1);
+
+
+		rcu_read_unlock();
+	}
+}
+EXPORT_SYMBOL_GPL(tcp_v6_md5_send_response_write);
+#endif
+
+struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
+					 const struct sock *addr_sk)
+{
+	const union tcp_md5_addr *addr;
+
+	addr = (const union tcp_md5_addr *)&addr_sk->sk_daddr;
+	return tcp_md5_do_lookup(sk, addr, AF_INET);
+}
+EXPORT_SYMBOL(tcp_v4_md5_lookup);
+
+int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
+			const struct sock *sk,
+			const struct sk_buff *skb)
+{
+	struct tcp_md5sig_pool *hp;
+	struct ahash_request *req;
+	const struct tcphdr *th = tcp_hdr(skb);
+	__be32 saddr, daddr;
+
+	if (sk) { /* valid for establish/request sockets */
+		saddr = sk->sk_rcv_saddr;
+		daddr = sk->sk_daddr;
+	} else {
+		const struct iphdr *iph = ip_hdr(skb);
+
+		saddr = iph->saddr;
+		daddr = iph->daddr;
+	}
+
+	hp = tcp_get_md5sig_pool();
+	if (!hp)
+		goto clear_hash_noput;
+	req = hp->md5_req;
+
+	if (crypto_ahash_init(req))
+		goto clear_hash;
+
+	if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, skb->len))
+		goto clear_hash;
+	if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2))
+		goto clear_hash;
+	if (tcp_md5_hash_key(hp, key))
+		goto clear_hash;
+	ahash_request_set_crypt(req, NULL, md5_hash, 0);
+	if (crypto_ahash_final(req))
+		goto clear_hash;
+
+	tcp_put_md5sig_pool();
+	return 0;
+
+clear_hash:
+	tcp_put_md5sig_pool();
+clear_hash_noput:
+	memset(md5_hash, 0, 16);
+	return 1;
+}
+EXPORT_SYMBOL(tcp_v4_md5_hash_skb);
+
+#if IS_ENABLED(CONFIG_IPV6)
+int tcp_v6_md5_hash_skb(char *md5_hash,
+			const struct tcp_md5sig_key *key,
+			const struct sock *sk,
+			const struct sk_buff *skb)
+{
+	const struct in6_addr *saddr, *daddr;
+	struct tcp_md5sig_pool *hp;
+	struct ahash_request *req;
+	const struct tcphdr *th = tcp_hdr(skb);
+
+	if (sk) { /* valid for establish/request sockets */
+		saddr = &sk->sk_v6_rcv_saddr;
+		daddr = &sk->sk_v6_daddr;
+	} else {
+		const struct ipv6hdr *ip6h = ipv6_hdr(skb);
+
+		saddr = &ip6h->saddr;
+		daddr = &ip6h->daddr;
+	}
+
+	hp = tcp_get_md5sig_pool();
+	if (!hp)
+		goto clear_hash_noput;
+	req = hp->md5_req;
+
+	if (crypto_ahash_init(req))
+		goto clear_hash;
+
+	if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, skb->len))
+		goto clear_hash;
+	if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2))
+		goto clear_hash;
+	if (tcp_md5_hash_key(hp, key))
+		goto clear_hash;
+	ahash_request_set_crypt(req, NULL, md5_hash, 0);
+	if (crypto_ahash_final(req))
+		goto clear_hash;
+
+	tcp_put_md5sig_pool();
+	return 0;
+
+clear_hash:
+	tcp_put_md5sig_pool();
+clear_hash_noput:
+	memset(md5_hash, 0, 16);
+	return 1;
+}
+EXPORT_SYMBOL_GPL(tcp_v6_md5_hash_skb);
+#endif
+
+/* Called with rcu_read_lock() */
+bool tcp_v4_inbound_md5_hash(const struct sock *sk,
+			     const struct sk_buff *skb)
+{
+	/* This gets called for each TCP segment that arrives
+	 * so we want to be efficient.
+	 * We have 3 drop cases:
+	 * o No MD5 hash and one expected.
+	 * o MD5 hash and we're not expecting one.
+	 * o MD5 hash and its wrong.
+	 */
+	const __u8 *hash_location = NULL;
+	struct tcp_md5sig_key *hash_expected;
+	const struct iphdr *iph = ip_hdr(skb);
+	const struct tcphdr *th = tcp_hdr(skb);
+	int genhash;
+	unsigned char newhash[16];
+
+	hash_expected = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&iph->saddr,
+					  AF_INET);
+	hash_location = tcp_parse_md5sig_option(th);
+
+	/* We've parsed the options - do we have a hash? */
+	if (!hash_expected && !hash_location)
+		return false;
+
+	if (hash_expected && !hash_location) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5NOTFOUND);
+		return true;
+	}
+
+	if (!hash_expected && hash_location) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5UNEXPECTED);
+		return true;
+	}
+
+	/* Okay, so this is hash_expected and hash_location -
+	 * so we need to calculate the checksum.
+	 */
+	genhash = tcp_v4_md5_hash_skb(newhash,
+				      hash_expected,
+				      NULL, skb);
+
+	if (genhash || memcmp(hash_location, newhash, 16) != 0) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5FAILURE);
+		net_info_ratelimited("MD5 Hash failed for (%pI4, %d)->(%pI4, %d)%s\n",
+				     &iph->saddr, ntohs(th->source),
+				     &iph->daddr, ntohs(th->dest),
+				     genhash ? " tcp_v4_calc_md5_hash failed"
+				     : "");
+		return true;
+	}
+	return false;
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+bool tcp_v6_inbound_md5_hash(const struct sock *sk,
+			     const struct sk_buff *skb)
+{
+	const __u8 *hash_location = NULL;
+	struct tcp_md5sig_key *hash_expected;
+	const struct ipv6hdr *ip6h = ipv6_hdr(skb);
+	const struct tcphdr *th = tcp_hdr(skb);
+	int genhash;
+	u8 newhash[16];
+
+	hash_expected = tcp_v6_md5_do_lookup(sk, &ip6h->saddr);
+	hash_location = tcp_parse_md5sig_option(th);
+
+	/* We've parsed the options - do we have a hash? */
+	if (!hash_expected && !hash_location)
+		return false;
+
+	if (hash_expected && !hash_location) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5NOTFOUND);
+		return true;
+	}
+
+	if (!hash_expected && hash_location) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5UNEXPECTED);
+		return true;
+	}
+
+	/* check the signature */
+	genhash = tcp_v6_md5_hash_skb(newhash,
+				      hash_expected,
+				      NULL, skb);
+
+	if (genhash || memcmp(hash_location, newhash, 16) != 0) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5FAILURE);
+		net_info_ratelimited("MD5 Hash %s for [%pI6c]:%u->[%pI6c]:%u\n",
+				     genhash ? "failed" : "mismatch",
+				     &ip6h->saddr, ntohs(th->source),
+				     &ip6h->daddr, ntohs(th->dest));
+		return true;
+	}
+
+	return false;
+}
+EXPORT_SYMBOL_GPL(tcp_v6_inbound_md5_hash);
+#endif
+
+void tcp_v4_md5_destroy_sock(struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+
+	/* Clean up the MD5 key list, if any */
+	if (tp->md5sig_info) {
+		tcp_clear_md5_list(sk);
+		kfree_rcu(tp->md5sig_info, rcu);
+		tp->md5sig_info = NULL;
+	}
+}
+
+void tcp_v4_md5_syn_recv_sock(const struct sock *listener, struct sock *sk)
+{
+	struct inet_sock *inet = inet_sk(sk);
+	struct tcp_md5sig_key *key;
+
+	/* Copy over the MD5 key from the original socket */
+	key = tcp_md5_do_lookup(listener, (union tcp_md5_addr *)&inet->inet_daddr,
+				AF_INET);
+	if (key) {
+		/* We're using one, so create a matching key
+		 * on the sk structure. If we fail to get
+		 * memory, then we end up not copying the key
+		 * across. Shucks.
+		 */
+		tcp_md5_do_add(sk, (union tcp_md5_addr *)&inet->inet_daddr,
+			       AF_INET, 32, key->key, key->keylen, GFP_ATOMIC);
+		sk_nocaps_add(sk, NETIF_F_GSO_MASK);
+	}
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+void tcp_v6_md5_syn_recv_sock(const struct sock *listener, struct sock *sk)
+{
+	struct tcp_md5sig_key *key;
+
+	/* Copy over the MD5 key from the original socket */
+	key = tcp_v6_md5_do_lookup(listener, &sk->sk_v6_daddr);
+	if (key) {
+		/* We're using one, so create a matching key
+		 * on the newsk structure. If we fail to get
+		 * memory, then we end up not copying the key
+		 * across. Shucks.
+		 */
+		tcp_md5_do_add(sk, (union tcp_md5_addr *)&sk->sk_v6_daddr,
+			       AF_INET6, 128, key->key, key->keylen,
+			       sk_gfp_mask(sk, GFP_ATOMIC));
+	}
+}
+EXPORT_SYMBOL_GPL(tcp_v6_md5_syn_recv_sock);
+
+struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
+					 const struct sock *addr_sk)
+{
+	return tcp_v6_md5_do_lookup(sk, &addr_sk->sk_v6_daddr);
+}
+EXPORT_SYMBOL_GPL(tcp_v6_md5_lookup);
+#endif
+
+void tcp_md5_time_wait(struct sock *sk, struct inet_timewait_sock *tw)
+{
+	struct tcp_timewait_sock *tcptw = tcp_twsk((struct sock *)tw);
+	struct tcp_sock *tp = tcp_sk(sk);
+	struct tcp_md5sig_key *key;
+
+	/* The timewait bucket does not have the key DB from the
+	 * sock structure. We just make a quick copy of the
+	 * md5 key being used (if indeed we are using one)
+	 * so the timewait ack generating code has the key.
+	 */
+	tcptw->tw_md5_key = NULL;
+	key = tp->af_specific->md5_lookup(sk, sk);
+	if (key) {
+		tcptw->tw_md5_key = kmemdup(key, sizeof(*key), GFP_ATOMIC);
+		BUG_ON(tcptw->tw_md5_key && !tcp_alloc_md5sig_pool());
+	}
+}
+
+static void tcp_diag_md5sig_fill(struct tcp_diag_md5sig *info,
+				 const struct tcp_md5sig_key *key)
+{
+	info->tcpm_family = key->family;
+	info->tcpm_prefixlen = key->prefixlen;
+	info->tcpm_keylen = key->keylen;
+	memcpy(info->tcpm_key, key->key, key->keylen);
+
+	if (key->family == AF_INET)
+		info->tcpm_addr[0] = key->addr.a4.s_addr;
+	#if IS_ENABLED(CONFIG_IPV6)
+	else if (key->family == AF_INET6)
+		memcpy(&info->tcpm_addr, &key->addr.a6,
+		       sizeof(info->tcpm_addr));
+	#endif
+}
+
+static int tcp_diag_put_md5sig(struct sk_buff *skb,
+			       const struct tcp_md5sig_info *md5sig)
+{
+	const struct tcp_md5sig_key *key;
+	struct tcp_diag_md5sig *info;
+	struct nlattr *attr;
+	int md5sig_count = 0;
+
+	hlist_for_each_entry_rcu(key, &md5sig->head, node)
+		md5sig_count++;
+	if (md5sig_count == 0)
+		return 0;
+
+	attr = nla_reserve(skb, INET_DIAG_MD5SIG,
+			   md5sig_count * sizeof(struct tcp_diag_md5sig));
+	if (!attr)
+		return -EMSGSIZE;
+
+	info = nla_data(attr);
+	memset(info, 0, md5sig_count * sizeof(struct tcp_diag_md5sig));
+	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
+		tcp_diag_md5sig_fill(info++, key);
+		if (--md5sig_count == 0)
+			break;
+	}
+
+	return 0;
+}
+
+int tcp_md5_diag_get_aux(struct sock *sk, bool net_admin, struct sk_buff *skb)
+{
+	if (net_admin) {
+		struct tcp_md5sig_info *md5sig;
+		int err = 0;
+
+		rcu_read_lock();

net/ipv4/tcp_md5.c line 1057

Quoted 10 lines
net/ipv4/tcp_md5.c new file +10 -0
+		md5sig = rcu_dereference(tcp_sk(sk)->md5sig_info);
+		if (md5sig)
+			err = tcp_diag_put_md5sig(skb, md5sig);
+		rcu_read_unlock();
+		if (err < 0)
+			return err;
+	}
+
+	return 0;
+}

Needs EXPORT_SYMBOL_GPL

Quoted 13 lines
net/ipv4/tcp_md5.c new file +13 -0
+
+int tcp_md5_diag_get_aux_size(struct sock *sk, bool net_admin)
+{
+	int size = 0;
+
+	if (net_admin && sk_fullsock(sk)) {
+		const struct tcp_md5sig_info *md5sig;
+		const struct tcp_md5sig_key *key;
+		size_t md5sig_count = 0;
+
+		rcu_read_lock();
+		md5sig = rcu_dereference(tcp_sk(sk)->md5sig_info);
+		if (md5sig) {

net/ipv4/tcp_md5.c line 1080

Quoted 10 lines
net/ipv4/tcp_md5.c new file +10 -0
+			hlist_for_each_entry_rcu(key, &md5sig->head, node)
+				md5sig_count++;
+		}
+		rcu_read_unlock();
+		size += nla_total_size(md5sig_count *
+				       sizeof(struct tcp_diag_md5sig));
+	}
+
+	return size;
+}

Also needs EXPORT_SYMBOL_GPL

Mat

Quoted 524 lines
net/ipv4/tcp_md5.c new file +24 -0
+
+const struct tcp_sock_af_ops tcp_sock_ipv4_specific = {
+	.md5_lookup	= tcp_v4_md5_lookup,
+	.calc_md5_hash	= tcp_v4_md5_hash_skb,
+	.md5_parse	= tcp_v4_parse_md5_keys,
+};
+
+#if IS_ENABLED(CONFIG_IPV6)
+const struct tcp_sock_af_ops tcp_sock_ipv6_specific = {
+	.md5_lookup	=	tcp_v6_md5_lookup,
+	.calc_md5_hash	=	tcp_v6_md5_hash_skb,
+	.md5_parse	=	tcp_v6_parse_md5_keys,
+};
+EXPORT_SYMBOL_GPL(tcp_sock_ipv6_specific);
+
+const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific = {
+	.md5_lookup	=	tcp_v4_md5_lookup,
+	.calc_md5_hash	=	tcp_v4_md5_hash_skb,
+	.md5_parse	=	tcp_v6_parse_md5_keys,
+};
+EXPORT_SYMBOL_GPL(tcp_sock_ipv6_mapped_specific);
+#endif
+
+
net/ipv4/tcp_minisocks.c +4 -22
index 641dca38cc1e..e9e925d51046 100644
@@ -22,6 +22,7 @@
 #include <linux/module.h>
 #include <linux/slab.h>
 #include <linux/sysctl.h>
+#include <linux/tcp_md5.h>
 #include <linux/workqueue.h>
 #include <linux/static_key.h>
 #include <net/tcp.h>
@@ -296,21 +297,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 			INIT_LIST_HEAD(&tcp_sk(sk)->tcp_option_list);
 		}
 #ifdef CONFIG_TCP_MD5SIG
-		/*
-		 * The timewait bucket does not have the key DB from the
-		 * sock structure. We just make a quick copy of the
-		 * md5 key being used (if indeed we are using one)
-		 * so the timewait ack generating code has the key.
-		 */
-		do {
-			struct tcp_md5sig_key *key;
-			tcptw->tw_md5_key = NULL;
-			key = tp->af_specific->md5_lookup(sk, sk);
-			if (key) {
-				tcptw->tw_md5_key = kmemdup(key, sizeof(*key), GFP_ATOMIC);
-				BUG_ON(tcptw->tw_md5_key && !tcp_alloc_md5sig_pool());
-			}
-		} while (0);
+		tcp_md5_time_wait(sk, tw);
 #endif
 
 		/* Get the TIME_WAIT timeout firing. */
@@ -340,10 +327,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 void tcp_twsk_destructor(struct sock *sk)
 {
 #ifdef CONFIG_TCP_MD5SIG
-	struct tcp_timewait_sock *twsk = tcp_twsk(sk);
-
-	if (twsk->tw_md5_key)
-		kfree_rcu(twsk->tw_md5_key, rcu);
+	tcp_md5_twsk_destructor(sk);
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
@@ -531,9 +515,7 @@ struct sock *tcp_create_openreq_child(const struct sock *sk,
 		}
 		newtp->tsoffset = treq->ts_off;
 #ifdef CONFIG_TCP_MD5SIG
-		newtp->md5sig_info = NULL;	/*XXX*/
-		if (newtp->af_specific->md5_lookup(sk, newsk))
-			newtp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
+		tcp_md5_add_header_len(sk, newsk);
 #endif
 		if (static_branch_unlikely(&tcp_extra_options_enabled))
 			newtp->tcp_header_len += tcp_extra_options_add_header(sk, newsk);
net/ipv4/tcp_output.c +2 -2
index a7de79294ee6..1c040c44fbf4 100644
@@ -42,6 +42,7 @@
 #include <linux/gfp.h>
 #include <linux/module.h>
 #include <linux/static_key.h>
+#include <linux/tcp_md5.h>
 
 #include <trace/events/tcp.h>
 
@@ -3245,8 +3246,7 @@ static void tcp_connect_init(struct sock *sk)
 		tp->tcp_header_len += TCPOLEN_TSTAMP_ALIGNED;
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (tp->af_specific->md5_lookup(sk, sk))
-		tp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
+	tcp_md5_add_header_len(sk, sk);
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
net/ipv6/tcp_ipv6.c +14 -275
index 2fb688a36731..0d3506309bd4 100644
@@ -43,6 +43,7 @@
 #include <linux/ipv6.h>
 #include <linux/icmpv6.h>
 #include <linux/random.h>
+#include <linux/tcp_md5.h>
 
 #include <net/tcp.h>
 #include <net/ndisc.h>
@@ -79,16 +80,6 @@ static int	tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb);
 
 static const struct inet_connection_sock_af_ops ipv6_mapped;
 static const struct inet_connection_sock_af_ops ipv6_specific;
-#ifdef CONFIG_TCP_MD5SIG
-static const struct tcp_sock_af_ops tcp_sock_ipv6_specific;
-static const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific;
-#else
-static struct tcp_md5sig_key *tcp_v6_md5_do_lookup(const struct sock *sk,
-						   const struct in6_addr *addr)
-{
-	return NULL;
-}
-#endif
 
 static void inet6_sk_rx_dst_set(struct sock *sk, const struct sk_buff *skb)
 {
@@ -507,218 +498,6 @@ static void tcp_v6_reqsk_destructor(struct request_sock *req)
 	kfree_skb(inet_rsk(req)->pktopts);
 }
 
-#ifdef CONFIG_TCP_MD5SIG
-static struct tcp_md5sig_key *tcp_v6_md5_do_lookup(const struct sock *sk,
-						   const struct in6_addr *addr)
-{
-	return tcp_md5_do_lookup(sk, (union tcp_md5_addr *)addr, AF_INET6);
-}
-
-static struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
-						const struct sock *addr_sk)
-{
-	return tcp_v6_md5_do_lookup(sk, &addr_sk->sk_v6_daddr);
-}
-
-static int tcp_v6_parse_md5_keys(struct sock *sk, int optname,
-				 char __user *optval, int optlen)
-{
-	struct tcp_md5sig cmd;
-	struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)&cmd.tcpm_addr;
-	u8 prefixlen;
-
-	if (optlen < sizeof(cmd))
-		return -EINVAL;
-
-	if (copy_from_user(&cmd, optval, sizeof(cmd)))
-		return -EFAULT;
-
-	if (sin6->sin6_family != AF_INET6)
-		return -EINVAL;
-
-	if (optname == TCP_MD5SIG_EXT &&
-	    cmd.tcpm_flags & TCP_MD5SIG_FLAG_PREFIX) {
-		prefixlen = cmd.tcpm_prefixlen;
-		if (prefixlen > 128 || (ipv6_addr_v4mapped(&sin6->sin6_addr) &&
-					prefixlen > 32))
-			return -EINVAL;
-	} else {
-		prefixlen = ipv6_addr_v4mapped(&sin6->sin6_addr) ? 32 : 128;
-	}
-
-	if (!cmd.tcpm_keylen) {
-		if (ipv6_addr_v4mapped(&sin6->sin6_addr))
-			return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin6->sin6_addr.s6_addr32[3],
-					      AF_INET, prefixlen);
-		return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin6->sin6_addr,
-				      AF_INET6, prefixlen);
-	}
-
-	if (cmd.tcpm_keylen > TCP_MD5SIG_MAXKEYLEN)
-		return -EINVAL;
-
-	if (ipv6_addr_v4mapped(&sin6->sin6_addr))
-		return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin6->sin6_addr.s6_addr32[3],
-				      AF_INET, prefixlen, cmd.tcpm_key,
-				      cmd.tcpm_keylen, GFP_KERNEL);
-
-	return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin6->sin6_addr,
-			      AF_INET6, prefixlen, cmd.tcpm_key,
-			      cmd.tcpm_keylen, GFP_KERNEL);
-}
-
-static int tcp_v6_md5_hash_headers(struct tcp_md5sig_pool *hp,
-				   const struct in6_addr *daddr,
-				   const struct in6_addr *saddr,
-				   const struct tcphdr *th, int nbytes)
-{
-	struct tcp6_pseudohdr *bp;
-	struct scatterlist sg;
-	struct tcphdr *_th;
-
-	bp = hp->scratch;
-	/* 1. TCP pseudo-header (RFC2460) */
-	bp->saddr = *saddr;
-	bp->daddr = *daddr;
-	bp->protocol = cpu_to_be32(IPPROTO_TCP);
-	bp->len = cpu_to_be32(nbytes);
-
-	_th = (struct tcphdr *)(bp + 1);
-	memcpy(_th, th, sizeof(*th));
-	_th->check = 0;
-
-	sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th));
-	ahash_request_set_crypt(hp->md5_req, &sg, NULL,
-				sizeof(*bp) + sizeof(*th));
-	return crypto_ahash_update(hp->md5_req);
-}
-
-static int tcp_v6_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
-			       const struct in6_addr *daddr, struct in6_addr *saddr,
-			       const struct tcphdr *th)
-{
-	struct tcp_md5sig_pool *hp;
-	struct ahash_request *req;
-
-	hp = tcp_get_md5sig_pool();
-	if (!hp)
-		goto clear_hash_noput;
-	req = hp->md5_req;
-
-	if (crypto_ahash_init(req))
-		goto clear_hash;
-	if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2))
-		goto clear_hash;
-	if (tcp_md5_hash_key(hp, key))
-		goto clear_hash;
-	ahash_request_set_crypt(req, NULL, md5_hash, 0);
-	if (crypto_ahash_final(req))
-		goto clear_hash;
-
-	tcp_put_md5sig_pool();
-	return 0;
-
-clear_hash:
-	tcp_put_md5sig_pool();
-clear_hash_noput:
-	memset(md5_hash, 0, 16);
-	return 1;
-}
-
-static int tcp_v6_md5_hash_skb(char *md5_hash,
-			       const struct tcp_md5sig_key *key,
-			       const struct sock *sk,
-			       const struct sk_buff *skb)
-{
-	const struct in6_addr *saddr, *daddr;
-	struct tcp_md5sig_pool *hp;
-	struct ahash_request *req;
-	const struct tcphdr *th = tcp_hdr(skb);
-
-	if (sk) { /* valid for establish/request sockets */
-		saddr = &sk->sk_v6_rcv_saddr;
-		daddr = &sk->sk_v6_daddr;
-	} else {
-		const struct ipv6hdr *ip6h = ipv6_hdr(skb);
-		saddr = &ip6h->saddr;
-		daddr = &ip6h->daddr;
-	}
-
-	hp = tcp_get_md5sig_pool();
-	if (!hp)
-		goto clear_hash_noput;
-	req = hp->md5_req;
-
-	if (crypto_ahash_init(req))
-		goto clear_hash;
-
-	if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, skb->len))
-		goto clear_hash;
-	if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2))
-		goto clear_hash;
-	if (tcp_md5_hash_key(hp, key))
-		goto clear_hash;
-	ahash_request_set_crypt(req, NULL, md5_hash, 0);
-	if (crypto_ahash_final(req))
-		goto clear_hash;
-
-	tcp_put_md5sig_pool();
-	return 0;
-
-clear_hash:
-	tcp_put_md5sig_pool();
-clear_hash_noput:
-	memset(md5_hash, 0, 16);
-	return 1;
-}
-
-#endif
-
-static bool tcp_v6_inbound_md5_hash(const struct sock *sk,
-				    const struct sk_buff *skb)
-{
-#ifdef CONFIG_TCP_MD5SIG
-	const __u8 *hash_location = NULL;
-	struct tcp_md5sig_key *hash_expected;
-	const struct ipv6hdr *ip6h = ipv6_hdr(skb);
-	const struct tcphdr *th = tcp_hdr(skb);
-	int genhash;
-	u8 newhash[16];
-
-	hash_expected = tcp_v6_md5_do_lookup(sk, &ip6h->saddr);
-	hash_location = tcp_parse_md5sig_option(th);
-
-	/* We've parsed the options - do we have a hash? */
-	if (!hash_expected && !hash_location)
-		return false;
-
-	if (hash_expected && !hash_location) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5NOTFOUND);
-		return true;
-	}
-
-	if (!hash_expected && hash_location) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5UNEXPECTED);
-		return true;
-	}
-
-	/* check the signature */
-	genhash = tcp_v6_md5_hash_skb(newhash,
-				      hash_expected,
-				      NULL, skb);
-
-	if (genhash || memcmp(hash_location, newhash, 16) != 0) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5FAILURE);
-		net_info_ratelimited("MD5 Hash %s for [%pI6c]:%u->[%pI6c]:%u\n",
-				     genhash ? "failed" : "mismatch",
-				     &ip6h->saddr, ntohs(th->source),
-				     &ip6h->daddr, ntohs(th->dest));
-		return true;
-	}
-#endif
-	return false;
-}
-
 static void tcp_v6_init_req(struct request_sock *req,
 			    const struct sock *sk_listener,
 			    struct sk_buff *skb)
@@ -791,60 +570,27 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 	unsigned int tot_len = sizeof(struct tcphdr);
 	struct dst_entry *dst;
 	__be32 *topt;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key *key = NULL;
-	const __u8 *hash_location = NULL;
-	struct ipv6hdr *ipv6h = ipv6_hdr(skb);
-#endif
 	struct tcp_out_options extraopts;
 
+	memset(&extraopts, 0, sizeof(extraopts));
+
 	if (tsecr)
 		tot_len += TCPOLEN_TSTAMP_ALIGNED;
 #ifdef CONFIG_TCP_MD5SIG
-	rcu_read_lock();
-	hash_location = tcp_parse_md5sig_option(th);
-	if (sk && sk_fullsock(sk)) {
-		key = tcp_v6_md5_do_lookup(sk, &ipv6h->saddr);
-	} else if (sk && sk->sk_state == TCP_TIME_WAIT) {
-		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
-
-		key = tcp_twsk_md5_key(tcptw);
-	} else if (sk && sk->sk_state == TCP_NEW_SYN_RECV) {
-		key = tcp_v6_md5_do_lookup(sk, &ipv6h->daddr);
-	} else if (hash_location) {
-		unsigned char newhash[16];
-		struct sock *sk1 = NULL;
-		int genhash;
-
-		/* active side is lost. Try to find listening socket through
-		 * source port, and then find md5 key through listening socket.
-		 * we are not loose security here:
-		 * Incoming packet is checked with md5 hash with finding key,
-		 * no RST generated if md5 hash doesn't match.
-		 */
-		sk1 = inet6_lookup_listener(dev_net(skb_dst(skb)->dev),
-					    &tcp_hashinfo, NULL, 0,
-					    &ipv6h->saddr,
-					    th->source, &ipv6h->daddr,
-					    ntohs(th->source), tcp_v6_iif(skb),
-					    tcp_v6_sdif(skb));
-		if (!sk1)
-			goto out;
+{
+	int ret;
 
-		key = tcp_v6_md5_do_lookup(sk1, &ipv6h->saddr);
-		if (!key)
-			goto out;
+	ret = tcp_v6_md5_send_response_prepare(skb, 0,
+					       MAX_TCP_OPTION_SPACE - tot_len,
+					       &extraopts, sk);
 
-		genhash = tcp_v6_md5_hash_skb(newhash, key, NULL, skb);
-		if (genhash || memcmp(hash_location, newhash, 16) != 0)
-			goto out;
-	}
+	if (ret == -1)
+		goto out;
 
-	if (key)
-		tot_len += TCPOLEN_MD5SIG_ALIGNED;
+	tot_len += ret;
+}
 #endif
 
-
 	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
 		unsigned int remaining = MAX_TCP_OPTION_SPACE - tot_len;
 		u8 extraflags = rst ? TCPHDR_RST : 0;
@@ -853,8 +599,6 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 		if (!rst || !th->ack)
 			extraflags |= TCPHDR_ACK;
 
-		memset(&extraopts, 0, sizeof(extraopts));
-
 		used = tcp_extra_options_response_prepare(skb, extraflags,
 							  remaining,
 							  &extraopts, sk);
@@ -895,13 +639,8 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 	}
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (key) {
-		*topt++ = htonl((TCPOPT_NOP << 24) | (TCPOPT_NOP << 16) |
-				(TCPOPT_MD5SIG << 8) | TCPOLEN_MD5SIG);
-		tcp_v6_md5_hash_hdr((__u8 *)topt, key,
-				    &ipv6_hdr(skb)->saddr,
-				    &ipv6_hdr(skb)->daddr, t1);
-	}
+	if (extraopts.md5)
+		tcp_v6_md5_send_response_write(topt, skb, t1, &extraopts, sk);
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
@@ -949,10 +688,6 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 

out:

	kfree_skb(buff);
net/ipv6/tcp_ipv6.c +0 -4
-
-#if CONFIG_TCP_MD5SIG
-	rcu_read_unlock();
-#endif

}

static void tcp_v6_send_reset(const struct sock *sk, struct sk_buff *skb)

net/ipv6/tcp_ipv6.c +1 -31
@@ -1078,9 +813,6 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
 	struct inet_sock *newinet;
 	struct tcp_sock *newtp;
 	struct sock *newsk;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key *key;
-#endif
 	struct flowi6 fl6;
 
 	if (skb->protocol == htons(ETH_P_IP)) {
@@ -1231,18 +963,7 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
 	}
 
 #ifdef CONFIG_TCP_MD5SIG
-	/* Copy over the MD5 key from the original socket */
-	key = tcp_v6_md5_do_lookup(sk, &newsk->sk_v6_daddr);
-	if (key) {
-		/* We're using one, so create a matching key
-		 * on the newsk structure. If we fail to get
-		 * memory, then we end up not copying the key
-		 * across. Shucks.
-		 */
-		tcp_md5_do_add(newsk, (union tcp_md5_addr *)&newsk->sk_v6_daddr,
-			       AF_INET6, 128, key->key, key->keylen,
-			       sk_gfp_mask(sk, GFP_ATOMIC));
-	}
+	tcp_v6_md5_syn_recv_sock(sk, newsk);
 #endif
 
 	if (__inet_inherit_port(sk, newsk) < 0) {
@@ -1698,14 +1419,6 @@ static const struct inet_connection_sock_af_ops ipv6_specific = {
 	.mtu_reduced	   = tcp_v6_mtu_reduced,
 };
 
-#ifdef CONFIG_TCP_MD5SIG
-static const struct tcp_sock_af_ops tcp_sock_ipv6_specific = {
-	.md5_lookup	=	tcp_v6_md5_lookup,
-	.calc_md5_hash	=	tcp_v6_md5_hash_skb,
-	.md5_parse	=	tcp_v6_parse_md5_keys,
-};
-#endif
-
 /*
 *	TCP over IPv4 via INET6 API
 */
@@ -1728,14 +1441,6 @@ static const struct inet_connection_sock_af_ops ipv6_mapped = {
 	.mtu_reduced	   = tcp_v4_mtu_reduced,
 };
 
-#ifdef CONFIG_TCP_MD5SIG
-static const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific = {
-	.md5_lookup	=	tcp_v4_md5_lookup,
-	.calc_md5_hash	=	tcp_v4_md5_hash_skb,
-	.md5_parse	=	tcp_v6_parse_md5_keys,
-};
-#endif
-
 /* NOTE: A lot of things set to zero explicitly by call to
 *       sk_alloc() so need not be done here.
 */
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#43 lore raw

Re: [MPTCP] [PATCH v2 22/25] tcp_md5: Move TCP-MD5 code out of TCP itself

Quoted 2,348 lines · Mat Martineau
Quoted 2,316 lines · Christoph Paasch

This is all just copy-pasting the TCP_MD5-code into functions that are placed in net/ipv4/tcp_md5.c.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---

include/linux/inet_diag.h |    1 +
include/linux/tcp_md5.h   |  138 ++++++
include/net/tcp.h         |   77 ----
net/ipv4/Makefile         |    1 +
net/ipv4/tcp.c            |  133 +-----
net/ipv4/tcp_diag.c       |   81 +---
net/ipv4/tcp_input.c      |   38 --
net/ipv4/tcp_ipv4.c       |  521 ++-------------------
net/ipv4/tcp_md5.c        | 1104 +++++++++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_minisocks.c  |   26 +-
net/ipv4/tcp_output.c     |    4 +-
net/ipv6/tcp_ipv6.c       |  325 +------------
12 files changed, 1305 insertions(+), 1144 deletions(-)

create mode 100644 include/linux/tcp_md5.h
create mode 100644 net/ipv4/tcp_md5.c

include/linux/inet_diag.h +1 -0
index 39faaaf843e1..1ef6727e41c9 100644
@@ -2,6 +2,7 @@
 #ifndef _INET_DIAG_H_
 #define _INET_DIAG_H_ 1
 
+#include <linux/user_namespace.h>
 #include <uapi/linux/inet_diag.h>
 
 struct net;
include/linux/tcp_md5.h new file +138 -0
new file mode 100644
index 000000000000..27f970f0f464
@@ -0,0 +1,138 @@
+#ifndef _LINUX_TCP_MD5_H
+#define _LINUX_TCP_MD5_H
+
+#ifdef CONFIG_TCP_MD5SIG
+
+#include <linux/types.h>
+
+#include <net/tcp.h>
+
+union tcp_md5_addr {
+	struct in_addr  a4;
+#if IS_ENABLED(CONFIG_IPV6)
+	struct in6_addr	a6;
+#endif
+};
+
+/* - key database */
+struct tcp_md5sig_key {
+	struct hlist_node	node;
+	u8			keylen;
+	u8			family; /* AF_INET or AF_INET6 */
+	union tcp_md5_addr	addr;
+	u8			prefixlen;
+	u8			key[TCP_MD5SIG_MAXKEYLEN];
+	struct rcu_head		rcu;
+};
+
+/* - sock block */
+struct tcp_md5sig_info {
+	struct hlist_head	head;
+	struct rcu_head		rcu;
+};
+
+union tcp_md5sum_block {
+	struct tcp4_pseudohdr ip4;
+#if IS_ENABLED(CONFIG_IPV6)
+	struct tcp6_pseudohdr ip6;
+#endif
+};
+
+/* - pool: digest algorithm, hash description and scratch buffer */
+struct tcp_md5sig_pool {
+	struct ahash_request	*md5_req;
+	void			*scratch;
+};
+
+extern const struct tcp_sock_af_ops tcp_sock_ipv4_specific;
+extern const struct tcp_sock_af_ops tcp_sock_ipv6_specific;
+extern const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific;
+
+/* - functions */
+int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
+			const struct sock *sk, const struct sk_buff *skb);
+
+struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
+					 const struct sock *addr_sk);
+
+void tcp_v4_md5_destroy_sock(struct sock *sk);
+
+int tcp_v4_md5_send_response_prepare(struct sk_buff *skb, u8 flags,
+				     unsigned int remaining,
+				     struct tcp_out_options *opts,
+				     const struct sock *sk);
+
+void tcp_v4_md5_send_response_write(__be32 *topt, struct sk_buff *skb,
+				    struct tcphdr *t1,
+				    struct tcp_out_options *opts,
+				    const struct sock *sk);
+
+int tcp_v6_md5_send_response_prepare(struct sk_buff *skb, u8 flags,
+				     unsigned int remaining,
+				     struct tcp_out_options *opts,
+				     const struct sock *sk);
+
+void tcp_v6_md5_send_response_write(__be32 *topt, struct sk_buff *skb,
+				    struct tcphdr *t1,
+				    struct tcp_out_options *opts,
+				    const struct sock *sk);
+
+bool tcp_v4_inbound_md5_hash(const struct sock *sk,
+			     const struct sk_buff *skb);
+
+void tcp_v4_md5_syn_recv_sock(const struct sock *listener, struct sock *sk);
+
+void tcp_v6_md5_syn_recv_sock(const struct sock *listener, struct sock *sk);
+
+void tcp_md5_time_wait(struct sock *sk, struct inet_timewait_sock *tw);
+
+struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
+					 const struct sock *addr_sk);
+
+int tcp_v6_md5_hash_skb(char *md5_hash,
+			const struct tcp_md5sig_key *key,
+			const struct sock *sk,
+			const struct sk_buff *skb);
+
+bool tcp_v6_inbound_md5_hash(const struct sock *sk,
+			     const struct sk_buff *skb);
+
+static inline void tcp_md5_twsk_destructor(struct sock *sk)
+{
+	struct tcp_timewait_sock *twsk = tcp_twsk(sk);
+
+	if (twsk->tw_md5_key)
+		kfree_rcu(twsk->tw_md5_key, rcu);
+}
+
+static inline void tcp_md5_add_header_len(const struct sock *listener,
+					  struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+
+	tp->md5sig_info = NULL;	/*XXX*/
+	if (tp->af_specific->md5_lookup(listener, sk))
+		tp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
+}
+
+int tcp_md5_diag_get_aux(struct sock *sk, bool net_admin, struct sk_buff *skb);
+
+int tcp_md5_diag_get_aux_size(struct sock *sk, bool net_admin);
+
+#else
+
+static inline bool tcp_v4_inbound_md5_hash(const struct sock *sk,
+					   const struct sk_buff *skb)
+{
+	return false;
+}
+
+static inline bool tcp_v6_inbound_md5_hash(const struct sock *sk,
+					   const struct sk_buff *skb)
+{
+	return false;
+}
+
+#endif
+
+#endif /* _LINUX_TCP_MD5_H */
include/net/tcp.h +0 -77
index beedb639e664..058715ab3a45 100644
@@ -406,7 +406,6 @@ void tcp_parse_options(const struct net *net, const struct sk_buff *skb,
 		       struct tcp_options_received *opt_rx,
 		       int estab, struct tcp_fastopen_cookie *foc,
 		       struct sock *sk);
-const u8 *tcp_parse_md5sig_option(const struct tcphdr *th);
 
 /*
 *	TCP v4 functions exported for the inet6 API
@@ -1416,30 +1415,6 @@ static inline void tcp_clear_all_retrans_hints(struct tcp_sock *tp)
 	tp->retransmit_skb_hint = NULL;
 }
 
-union tcp_md5_addr {
-	struct in_addr  a4;
-#if IS_ENABLED(CONFIG_IPV6)
-	struct in6_addr	a6;
-#endif
-};
-
-/* - key database */
-struct tcp_md5sig_key {
-	struct hlist_node	node;
-	u8			keylen;
-	u8			family; /* AF_INET or AF_INET6 */
-	union tcp_md5_addr	addr;
-	u8			prefixlen;
-	u8			key[TCP_MD5SIG_MAXKEYLEN];
-	struct rcu_head		rcu;
-};
-
-/* - sock block */
-struct tcp_md5sig_info {
-	struct hlist_head	head;
-	struct rcu_head		rcu;
-};
-
 /* - pseudo header */
 struct tcp4_pseudohdr {
 	__be32		saddr;
@@ -1456,58 +1431,6 @@ struct tcp6_pseudohdr {
 	__be32		protocol;	/* including padding */
 };
 
-union tcp_md5sum_block {
-	struct tcp4_pseudohdr ip4;
-#if IS_ENABLED(CONFIG_IPV6)
-	struct tcp6_pseudohdr ip6;
-#endif
-};
-
-/* - pool: digest algorithm, hash description and scratch buffer */
-struct tcp_md5sig_pool {
-	struct ahash_request	*md5_req;
-	void			*scratch;
-};
-
-/* - functions */
-int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
-			const struct sock *sk, const struct sk_buff *skb);
-int tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
-		   int family, u8 prefixlen, const u8 *newkey, u8 newkeylen,
-		   gfp_t gfp);
-int tcp_md5_do_del(struct sock *sk, const union tcp_md5_addr *addr,
-		   int family, u8 prefixlen);
-struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
-					 const struct sock *addr_sk);
-
-#ifdef CONFIG_TCP_MD5SIG
-struct tcp_md5sig_key *tcp_md5_do_lookup(const struct sock *sk,
-					 const union tcp_md5_addr *addr,
-					 int family);
-#define tcp_twsk_md5_key(twsk)	((twsk)->tw_md5_key)
-#else
-static inline struct tcp_md5sig_key *tcp_md5_do_lookup(const struct sock *sk,
-					 const union tcp_md5_addr *addr,
-					 int family)
-{
-	return NULL;
-}
-#define tcp_twsk_md5_key(twsk)	NULL
-#endif
-
-bool tcp_alloc_md5sig_pool(void);
-
-struct tcp_md5sig_pool *tcp_get_md5sig_pool(void);
-static inline void tcp_put_md5sig_pool(void)
-{
-	local_bh_enable();
-}
-
-int tcp_md5_hash_skb_data(struct tcp_md5sig_pool *, const struct sk_buff *,
-			  unsigned int header_len);
-int tcp_md5_hash_key(struct tcp_md5sig_pool *hp,
-		     const struct tcp_md5sig_key *key);
-
 /* From tcp_fastopen.c */
 void tcp_fastopen_cache_get(struct sock *sk, u16 *mss,
 			    struct tcp_fastopen_cookie *cookie, int *syn_loss,
net/ipv4/Makefile +0 -0
index c6c8ad1d4b6d..9262d9a01035 100644
@@ -61,6 +61,7 @@ obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o

obj-$(CONFIG_TCP_CONG_YEAH) += tcp_yeah.o
obj-$(CONFIG_TCP_CONG_ILLINOIS) += tcp_illinois.o
obj-$(CONFIG_NETLABEL) += cipso_ipv4.o
+obj-$(CONFIG_TCP_MD5SIG) += tcp_md5.o

obj-$(CONFIG_XFRM) += xfrm4_policy.o xfrm4_state.o xfrm4_input.o \
xfrm4_output.o xfrm4_protocol.o

net/ipv4/tcp.c +1 -132
index 3090fac3c462..ca1ddb2c820c 100644
@@ -271,6 +271,7 @@
 #include <linux/slab.h>
 #include <linux/errqueue.h>
 #include <linux/static_key.h>
+#include <linux/tcp_md5.h>
 
 #include <net/icmp.h>
 #include <net/inet_common.h>
@@ -3342,138 +3343,6 @@ int compat_tcp_getsockopt(struct sock *sk, int level, int optname,
 EXPORT_SYMBOL(compat_tcp_getsockopt);
 #endif
 
-#ifdef CONFIG_TCP_MD5SIG
-static DEFINE_PER_CPU(struct tcp_md5sig_pool, tcp_md5sig_pool);
-static DEFINE_MUTEX(tcp_md5sig_mutex);
-static bool tcp_md5sig_pool_populated = false;
-
-static void __tcp_alloc_md5sig_pool(void)
-{
-	struct crypto_ahash *hash;
-	int cpu;
-
-	hash = crypto_alloc_ahash("md5", 0, CRYPTO_ALG_ASYNC);
-	if (IS_ERR(hash))
-		return;
-
-	for_each_possible_cpu(cpu) {
-		void *scratch = per_cpu(tcp_md5sig_pool, cpu).scratch;
-		struct ahash_request *req;
-
-		if (!scratch) {
-			scratch = kmalloc_node(sizeof(union tcp_md5sum_block) +
-					       sizeof(struct tcphdr),
-					       GFP_KERNEL,
-					       cpu_to_node(cpu));
-			if (!scratch)
-				return;
-			per_cpu(tcp_md5sig_pool, cpu).scratch = scratch;
-		}
-		if (per_cpu(tcp_md5sig_pool, cpu).md5_req)
-			continue;
-
-		req = ahash_request_alloc(hash, GFP_KERNEL);
-		if (!req)
-			return;
-
-		ahash_request_set_callback(req, 0, NULL, NULL);
-
-		per_cpu(tcp_md5sig_pool, cpu).md5_req = req;
-	}
-	/* before setting tcp_md5sig_pool_populated, we must commit all writes
-	 * to memory. See smp_rmb() in tcp_get_md5sig_pool()
-	 */
-	smp_wmb();
-	tcp_md5sig_pool_populated = true;
-}
-
-bool tcp_alloc_md5sig_pool(void)
-{
-	if (unlikely(!tcp_md5sig_pool_populated)) {
-		mutex_lock(&tcp_md5sig_mutex);
-
-		if (!tcp_md5sig_pool_populated)
-			__tcp_alloc_md5sig_pool();
-
-		mutex_unlock(&tcp_md5sig_mutex);
-	}
-	return tcp_md5sig_pool_populated;
-}
-EXPORT_SYMBOL(tcp_alloc_md5sig_pool);
-
-
-/**
- *	tcp_get_md5sig_pool - get md5sig_pool for this user
- *
- *	We use percpu structure, so if we succeed, we exit with preemption
- *	and BH disabled, to make sure another thread or softirq handling
- *	wont try to get same context.
- */
-struct tcp_md5sig_pool *tcp_get_md5sig_pool(void)
-{
-	local_bh_disable();
-
-	if (tcp_md5sig_pool_populated) {
-		/* coupled with smp_wmb() in __tcp_alloc_md5sig_pool() */
-		smp_rmb();
-		return this_cpu_ptr(&tcp_md5sig_pool);
-	}
-	local_bh_enable();
-	return NULL;
-}
-EXPORT_SYMBOL(tcp_get_md5sig_pool);
-
-int tcp_md5_hash_skb_data(struct tcp_md5sig_pool *hp,
-			  const struct sk_buff *skb, unsigned int header_len)
-{
-	struct scatterlist sg;
-	const struct tcphdr *tp = tcp_hdr(skb);
-	struct ahash_request *req = hp->md5_req;
-	unsigned int i;
-	const unsigned int head_data_len = skb_headlen(skb) > header_len ?
-					   skb_headlen(skb) - header_len : 0;
-	const struct skb_shared_info *shi = skb_shinfo(skb);
-	struct sk_buff *frag_iter;
-
-	sg_init_table(&sg, 1);
-
-	sg_set_buf(&sg, ((u8 *) tp) + header_len, head_data_len);
-	ahash_request_set_crypt(req, &sg, NULL, head_data_len);
-	if (crypto_ahash_update(req))
-		return 1;
-
-	for (i = 0; i < shi->nr_frags; ++i) {
-		const struct skb_frag_struct *f = &shi->frags[i];
-		unsigned int offset = f->page_offset;
-		struct page *page = skb_frag_page(f) + (offset >> PAGE_SHIFT);
-
-		sg_set_page(&sg, page, skb_frag_size(f),
-			    offset_in_page(offset));
-		ahash_request_set_crypt(req, &sg, NULL, skb_frag_size(f));
-		if (crypto_ahash_update(req))
-			return 1;
-	}
-
-	skb_walk_frags(skb, frag_iter)
-		if (tcp_md5_hash_skb_data(hp, frag_iter, 0))
-			return 1;
-
-	return 0;
-}
-EXPORT_SYMBOL(tcp_md5_hash_skb_data);
-
-int tcp_md5_hash_key(struct tcp_md5sig_pool *hp, const struct tcp_md5sig_key *key)
-{
-	struct scatterlist sg;
-
-	sg_init_one(&sg, key->key, key->keylen);
-	ahash_request_set_crypt(hp->md5_req, &sg, NULL, key->keylen);
-	return crypto_ahash_update(hp->md5_req);
-}
-EXPORT_SYMBOL(tcp_md5_hash_key);
-
-#endif
-
 static struct list_head *tcp_extra_options_get_list(const struct sock *sk)
 {
 	if (sk_fullsock(sk))
net/ipv4/tcp_diag.c +6 -75
index abbf0edcf6c2..5cfe5dc8f8dd 100644
@@ -15,6 +15,7 @@
 #include <linux/inet_diag.h>
 
 #include <linux/tcp.h>
+#include <linux/tcp_md5.h>
 
 #include <net/netlink.h>
 #include <net/tcp.h>
@@ -37,70 +38,14 @@ static void tcp_diag_get_info(struct sock *sk, struct inet_diag_msg *r,
 		tcp_get_info(sk, info);
 }
 
-#ifdef CONFIG_TCP_MD5SIG
-static void tcp_diag_md5sig_fill(struct tcp_diag_md5sig *info,
-				 const struct tcp_md5sig_key *key)
-{
-	info->tcpm_family = key->family;
-	info->tcpm_prefixlen = key->prefixlen;
-	info->tcpm_keylen = key->keylen;
-	memcpy(info->tcpm_key, key->key, key->keylen);
-
-	if (key->family == AF_INET)
-		info->tcpm_addr[0] = key->addr.a4.s_addr;
-	#if IS_ENABLED(CONFIG_IPV6)
-	else if (key->family == AF_INET6)
-		memcpy(&info->tcpm_addr, &key->addr.a6,
-		       sizeof(info->tcpm_addr));
-	#endif
-}
-
-static int tcp_diag_put_md5sig(struct sk_buff *skb,
-			       const struct tcp_md5sig_info *md5sig)
-{
-	const struct tcp_md5sig_key *key;
-	struct tcp_diag_md5sig *info;
-	struct nlattr *attr;
-	int md5sig_count = 0;
-
-	hlist_for_each_entry_rcu(key, &md5sig->head, node)
-		md5sig_count++;
-	if (md5sig_count == 0)
-		return 0;
-
-	attr = nla_reserve(skb, INET_DIAG_MD5SIG,
-			   md5sig_count * sizeof(struct tcp_diag_md5sig));
-	if (!attr)
-		return -EMSGSIZE;
-
-	info = nla_data(attr);
-	memset(info, 0, md5sig_count * sizeof(struct tcp_diag_md5sig));
-	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
-		tcp_diag_md5sig_fill(info++, key);
-		if (--md5sig_count == 0)
-			break;
-	}
-
-	return 0;
-}
-#endif
-
 static int tcp_diag_get_aux(struct sock *sk, bool net_admin,
 			    struct sk_buff *skb)
 {
 #ifdef CONFIG_TCP_MD5SIG
-	if (net_admin) {
-		struct tcp_md5sig_info *md5sig;
-		int err = 0;
-
-		rcu_read_lock();
-		md5sig = rcu_dereference(tcp_sk(sk)->md5sig_info);
-		if (md5sig)
-			err = tcp_diag_put_md5sig(skb, md5sig);
-		rcu_read_unlock();
-		if (err < 0)
-			return err;
-	}
+	int err = tcp_md5_diag_get_aux(sk, net_admin, skb);
+
+	if (err < 0)
+		return err;
 #endif
 
 	return 0;
@@ -111,21 +56,7 @@ static size_t tcp_diag_get_aux_size(struct sock *sk, bool net_admin)
 	size_t size = 0;
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (net_admin && sk_fullsock(sk)) {
-		const struct tcp_md5sig_info *md5sig;
-		const struct tcp_md5sig_key *key;
-		size_t md5sig_count = 0;
-
-		rcu_read_lock();
-		md5sig = rcu_dereference(tcp_sk(sk)->md5sig_info);
-		if (md5sig) {
-			hlist_for_each_entry_rcu(key, &md5sig->head, node)
-				md5sig_count++;
-		}
-		rcu_read_unlock();
-		size += nla_total_size(md5sig_count *
-				       sizeof(struct tcp_diag_md5sig));
-	}
+	size += tcp_md5_diag_get_aux_size(sk, net_admin);
 #endif
 
 	return size;
net/ipv4/tcp_input.c +0 -38
index 5dd19c3f37ce..09778d8104b8 100644
@@ -3848,44 +3848,6 @@ static bool tcp_fast_parse_options(const struct net *net,
 	return false;
 }
 
-#ifdef CONFIG_TCP_MD5SIG
-/*
- * Parse MD5 Signature option
- */
-const u8 *tcp_parse_md5sig_option(const struct tcphdr *th)
-{
-	int length = (th->doff << 2) - sizeof(*th);
-	const u8 *ptr = (const u8 *)(th + 1);
-
-	/* If the TCP option is too short, we can short cut */
-	if (length < TCPOLEN_MD5SIG)
-		return NULL;
-
-	while (length > 0) {
-		int opcode = *ptr++;
-		int opsize;
-
-		switch (opcode) {
-		case TCPOPT_EOL:
-			return NULL;
-		case TCPOPT_NOP:
-			length--;
-			continue;
-		default:
-			opsize = *ptr++;
-			if (opsize < 2 || opsize > length)
-				return NULL;
-			if (opcode == TCPOPT_MD5SIG)
-				return opsize == TCPOLEN_MD5SIG ? ptr : NULL;
-		}
-		ptr += opsize - 2;
-		length -= opsize;
-	}
-	return NULL;
-}
-EXPORT_SYMBOL(tcp_parse_md5sig_option);
-#endif
-
 /* Sorry, PAWS as specified is broken wrt. pure-ACKs -DaveM
 *
 * It is not fatal. If this ACK does _not_ change critical state (seqs, window)
net/ipv4/tcp_ipv4.c +33 -488
index f0db7beda04e..d05162654848 100644
@@ -62,6 +62,7 @@
 #include <linux/init.h>
 #include <linux/times.h>
 #include <linux/slab.h>
+#include <linux/tcp_md5.h>
 
 #include <net/net_namespace.h>
 #include <net/icmp.h>
@@ -87,11 +88,6 @@
 
 #include <trace/events/tcp.h>
 
-#ifdef CONFIG_TCP_MD5SIG
-static int tcp_v4_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
-			       __be32 daddr, __be32 saddr, const struct tcphdr *th);
-#endif
-
 struct inet_hashinfo tcp_hashinfo;
 EXPORT_SYMBOL(tcp_hashinfo);
 
@@ -602,14 +598,8 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 		struct tcphdr th;
 		__be32 opt[(MAX_TCP_OPTION_SPACE >> 2)];
 	} rep;
+	struct tcp_out_options opts;
 	struct ip_reply_arg arg;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key *key = NULL;
-	const __u8 *hash_location = NULL;
-	unsigned char newhash[16];
-	int genhash;
-	struct sock *sk1 = NULL;
-#endif
 	struct net *net;
 	int offset = 0;
 
@@ -623,6 +613,8 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 	if (!sk && skb_rtable(skb)->rt_type != RTN_LOCAL)
 		return;
 
+	memset(&opts, 0, sizeof(opts));
+
 	/* Swap the send and the receive. */
 	memset(&rep, 0, sizeof(rep));
 	rep.th.dest   = th->source;
@@ -643,55 +635,32 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 	arg.iov[0].iov_len  = sizeof(rep.th);
 
 	net = sk ? sock_net(sk) : dev_net(skb_dst(skb)->dev);
-#ifdef CONFIG_TCP_MD5SIG
-	rcu_read_lock();
-	hash_location = tcp_parse_md5sig_option(th);
-	if (sk && sk_fullsock(sk)) {
-		key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)
-					&ip_hdr(skb)->saddr, AF_INET);
-	} else if (hash_location) {
-		/*
-		 * active side is lost. Try to find listening socket through
-		 * source port, and then find md5 key through listening socket.
-		 * we are not loose security here:
-		 * Incoming packet is checked with md5 hash with finding key,
-		 * no RST generated if md5 hash doesn't match.
-		 */
-		sk1 = __inet_lookup_listener(net, &tcp_hashinfo, NULL, 0,
-					     ip_hdr(skb)->saddr,
-					     th->source, ip_hdr(skb)->daddr,
-					     ntohs(th->source), inet_iif(skb),
-					     tcp_v4_sdif(skb));
-		/* don't send rst if it can't find key */
-		if (!sk1)
-			goto out;
 
-		key = tcp_md5_do_lookup(sk1, (union tcp_md5_addr *)
-					&ip_hdr(skb)->saddr, AF_INET);
-		if (!key)
-			goto out;
+#ifdef CONFIG_TCP_MD5DSIG
+{
+	int ret;
 
+	ret = tcp_v4_md5_send_response_prepare(skb, 0,
+					       MAX_TCP_OPTION_SPACE - arg.iov[0].iov_len
+					       &opts, sk);
 
-		genhash = tcp_v4_md5_hash_skb(newhash, key, NULL, skb);
-		if (genhash || memcmp(hash_location, newhash, 16) != 0)
-			goto out;
+	if (ret == -1)
+		return;
 
-	}
+	arg.iov[0].iov_len += ret;
+}
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
 		unsigned int remaining;
-		struct tcp_out_options opts;
 		int used;
 
 		remaining = sizeof(rep.opt);
 #ifdef CONFIG_TCP_MD5SIG
-		if (key)
+		if (opts.md5)
 			remaining -= TCPOLEN_MD5SIG_ALIGNED;
 #endif
 
-		memset(&opts, 0, sizeof(opts));
-
 		used = tcp_extra_options_response_prepare(skb, TCPHDR_RST,
 							  remaining,
 							  &opts, sk);
@@ -706,19 +675,7 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 	}
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (key) {
-		rep.opt[offset++] = htonl((TCPOPT_NOP << 24) |
-					  (TCPOPT_NOP << 16) |
-					  (TCPOPT_MD5SIG << 8) |
-					  TCPOLEN_MD5SIG);
-		/* Update length and the length the header thinks exists */
-		arg.iov[0].iov_len += TCPOLEN_MD5SIG_ALIGNED;
-		rep.th.doff = arg.iov[0].iov_len / 4;
-
-		tcp_v4_md5_hash_hdr((__u8 *)&rep.opt[offset],
-				    key, ip_hdr(skb)->saddr,
-				    ip_hdr(skb)->daddr, &rep.th);
-	}
+	tcp_v4_md5_send_response_write(&rep.opt[offset], skb, &rep.th, &opts, sk);
 #endif
 	arg.csum = csum_tcpudp_nofold(ip_hdr(skb)->daddr,
 				      ip_hdr(skb)->saddr, /* XXX */
@@ -749,11 +706,6 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
 	__TCP_INC_STATS(net, TCP_MIB_OUTSEGS);
 	__TCP_INC_STATS(net, TCP_MIB_OUTRSTS);
 	local_bh_enable();
-
-#ifdef CONFIG_TCP_MD5SIG
-out:
-	rcu_read_unlock();
-#endif
 }
 
 /* The code following below sending ACKs in SYN-RECV and TIME-WAIT states
@@ -770,15 +722,14 @@ static void tcp_v4_send_ack(const struct sock *sk,
 		struct tcphdr th;
 		__be32 opt[(MAX_TCP_OPTION_SPACE >> 2)];
 	} rep;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key *key;
-#endif
+	struct tcp_out_options opts;
 	struct net *net = sock_net(sk);
 	struct ip_reply_arg arg;
 	int offset = 0;
 
 	memset(&rep.th, 0, sizeof(struct tcphdr));
 	memset(&arg, 0, sizeof(arg));
+	memset(&opts, 0, sizeof(opts));
 
 	arg.iov[0].iov_base = (unsigned char *)&rep;
 	arg.iov[0].iov_len  = sizeof(rep.th);
@@ -801,27 +752,28 @@ static void tcp_v4_send_ack(const struct sock *sk,
 	rep.th.window  = htons(win);
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (sk->sk_state == TCP_TIME_WAIT) {
-		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
+{
+	int ret;
 
-		key = tcp_twsk_md5_key(tcptw);
-	} else if (sk->sk_state == TCP_NEW_SYN_RECV) {
-		key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&ip_hdr(skb)->daddr,
-					AF_INET);
-	} else {
-		BUG();
-	}
+	ret = tcp_v4_md5_send_response_prepare(skb, 0,
+					       MAX_TCP_OPTION_SPACE - arg.iov[0].iov_len,
+					       &opts, sk);
+
+	if (ret == -1)
+		return;
+
+	arg.iov[0].iov_len += ret;
+}
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
 		unsigned int remaining;
-		struct tcp_out_options opts;
 		int used;
 
 		remaining = sizeof(rep.th) + sizeof(rep.opt) - arg.iov[0].iov_len;
 
 #ifdef CONFIG_TCP_MD5SIG
-		if (key)
+		if (opts.md5)
 			remaining -= TCPOLEN_MD5SIG_ALIGNED;
 #endif
 
@@ -839,17 +791,7 @@ static void tcp_v4_send_ack(const struct sock *sk,
 	}
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (key) {
-		rep.opt[offset++] = htonl((TCPOPT_NOP << 24) |
-					  (TCPOPT_NOP << 16) |
-					  (TCPOPT_MD5SIG << 8) |
-					  TCPOLEN_MD5SIG);
-		arg.iov[0].iov_len += TCPOLEN_MD5SIG_ALIGNED;
-
-		tcp_v4_md5_hash_hdr((__u8 *) &rep.opt[offset],
-				    key, ip_hdr(skb)->saddr,
-				    ip_hdr(skb)->daddr, &rep.th);
-	}
+	tcp_v4_md5_send_response_write(&rep.opt[offset], skb, &rep.th, &opts, sk);
 #endif
 
 	rep.th.doff = arg.iov[0].iov_len / 4;
@@ -960,374 +902,6 @@ static void tcp_v4_reqsk_destructor(struct request_sock *req)
 	kfree(rcu_dereference_protected(inet_rsk(req)->ireq_opt, 1));
 }
 
-#ifdef CONFIG_TCP_MD5SIG
-/*
- * RFC2385 MD5 checksumming requires a mapping of
- * IP address->MD5 Key.
- * We need to maintain these in the sk structure.
- */
-
-/* Find the Key structure for an address.  */
-struct tcp_md5sig_key *tcp_md5_do_lookup(const struct sock *sk,
-					 const union tcp_md5_addr *addr,
-					 int family)
-{
-	const struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_key *key;
-	const struct tcp_md5sig_info *md5sig;
-	__be32 mask;
-	struct tcp_md5sig_key *best_match = NULL;
-	bool match;
-
-	/* caller either holds rcu_read_lock() or socket lock */
-	md5sig = rcu_dereference_check(tp->md5sig_info,
-				       lockdep_sock_is_held(sk));
-	if (!md5sig)
-		return NULL;
-
-	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
-		if (key->family != family)
-			continue;
-
-		if (family == AF_INET) {
-			mask = inet_make_mask(key->prefixlen);
-			match = (key->addr.a4.s_addr & mask) ==
-				(addr->a4.s_addr & mask);
-#if IS_ENABLED(CONFIG_IPV6)
-		} else if (family == AF_INET6) {
-			match = ipv6_prefix_equal(&key->addr.a6, &addr->a6,
-						  key->prefixlen);
-#endif
-		} else {
-			match = false;
-		}
-
-		if (match && (!best_match ||
-			      key->prefixlen > best_match->prefixlen))
-			best_match = key;
-	}
-	return best_match;
-}
-EXPORT_SYMBOL(tcp_md5_do_lookup);
-
-static struct tcp_md5sig_key *tcp_md5_do_lookup_exact(const struct sock *sk,
-						      const union tcp_md5_addr *addr,
-						      int family, u8 prefixlen)
-{
-	const struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_key *key;
-	unsigned int size = sizeof(struct in_addr);
-	const struct tcp_md5sig_info *md5sig;
-
-	/* caller either holds rcu_read_lock() or socket lock */
-	md5sig = rcu_dereference_check(tp->md5sig_info,
-				       lockdep_sock_is_held(sk));
-	if (!md5sig)
-		return NULL;
-#if IS_ENABLED(CONFIG_IPV6)
-	if (family == AF_INET6)
-		size = sizeof(struct in6_addr);
-#endif
-	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
-		if (key->family != family)
-			continue;
-		if (!memcmp(&key->addr, addr, size) &&
-		    key->prefixlen == prefixlen)
-			return key;
-	}
-	return NULL;
-}
-
-struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
-					 const struct sock *addr_sk)
-{
-	const union tcp_md5_addr *addr;
-
-	addr = (const union tcp_md5_addr *)&addr_sk->sk_daddr;
-	return tcp_md5_do_lookup(sk, addr, AF_INET);
-}
-EXPORT_SYMBOL(tcp_v4_md5_lookup);
-
-/* This can be called on a newly created socket, from other files */
-int tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
-		   int family, u8 prefixlen, const u8 *newkey, u8 newkeylen,
-		   gfp_t gfp)
-{
-	/* Add Key to the list */
-	struct tcp_md5sig_key *key;
-	struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_info *md5sig;
-
-	key = tcp_md5_do_lookup_exact(sk, addr, family, prefixlen);
-	if (key) {
-		/* Pre-existing entry - just update that one. */
-		memcpy(key->key, newkey, newkeylen);
-		key->keylen = newkeylen;
-		return 0;
-	}
-
-	md5sig = rcu_dereference_protected(tp->md5sig_info,
-					   lockdep_sock_is_held(sk));
-	if (!md5sig) {
-		md5sig = kmalloc(sizeof(*md5sig), gfp);
-		if (!md5sig)
-			return -ENOMEM;
-
-		sk_nocaps_add(sk, NETIF_F_GSO_MASK);
-		INIT_HLIST_HEAD(&md5sig->head);
-		rcu_assign_pointer(tp->md5sig_info, md5sig);
-	}
-
-	key = sock_kmalloc(sk, sizeof(*key), gfp);
-	if (!key)
-		return -ENOMEM;
-	if (!tcp_alloc_md5sig_pool()) {
-		sock_kfree_s(sk, key, sizeof(*key));
-		return -ENOMEM;
-	}
-
-	memcpy(key->key, newkey, newkeylen);
-	key->keylen = newkeylen;
-	key->family = family;
-	key->prefixlen = prefixlen;
-	memcpy(&key->addr, addr,
-	       (family == AF_INET6) ? sizeof(struct in6_addr) :
-				      sizeof(struct in_addr));
-	hlist_add_head_rcu(&key->node, &md5sig->head);
-	return 0;
-}
-EXPORT_SYMBOL(tcp_md5_do_add);
-
-int tcp_md5_do_del(struct sock *sk, const union tcp_md5_addr *addr, int family,
-		   u8 prefixlen)
-{
-	struct tcp_md5sig_key *key;
-
-	key = tcp_md5_do_lookup_exact(sk, addr, family, prefixlen);
-	if (!key)
-		return -ENOENT;
-	hlist_del_rcu(&key->node);
-	atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
-	kfree_rcu(key, rcu);
-	return 0;
-}
-EXPORT_SYMBOL(tcp_md5_do_del);
-
-static void tcp_clear_md5_list(struct sock *sk)
-{
-	struct tcp_sock *tp = tcp_sk(sk);
-	struct tcp_md5sig_key *key;
-	struct hlist_node *n;
-	struct tcp_md5sig_info *md5sig;
-
-	md5sig = rcu_dereference_protected(tp->md5sig_info, 1);
-
-	hlist_for_each_entry_safe(key, n, &md5sig->head, node) {
-		hlist_del_rcu(&key->node);
-		atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
-		kfree_rcu(key, rcu);
-	}
-}
-
-static int tcp_v4_parse_md5_keys(struct sock *sk, int optname,
-				 char __user *optval, int optlen)
-{
-	struct tcp_md5sig cmd;
-	struct sockaddr_in *sin = (struct sockaddr_in *)&cmd.tcpm_addr;
-	u8 prefixlen = 32;
-
-	if (optlen < sizeof(cmd))
-		return -EINVAL;
-
-	if (copy_from_user(&cmd, optval, sizeof(cmd)))
-		return -EFAULT;
-
-	if (sin->sin_family != AF_INET)
-		return -EINVAL;
-
-	if (optname == TCP_MD5SIG_EXT &&
-	    cmd.tcpm_flags & TCP_MD5SIG_FLAG_PREFIX) {
-		prefixlen = cmd.tcpm_prefixlen;
-		if (prefixlen > 32)
-			return -EINVAL;
-	}
-
-	if (!cmd.tcpm_keylen)
-		return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin->sin_addr.s_addr,
-				      AF_INET, prefixlen);
-
-	if (cmd.tcpm_keylen > TCP_MD5SIG_MAXKEYLEN)
-		return -EINVAL;
-
-	return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin->sin_addr.s_addr,
-			      AF_INET, prefixlen, cmd.tcpm_key, cmd.tcpm_keylen,
-			      GFP_KERNEL);
-}
-
-static int tcp_v4_md5_hash_headers(struct tcp_md5sig_pool *hp,
-				   __be32 daddr, __be32 saddr,
-				   const struct tcphdr *th, int nbytes)
-{
-	struct tcp4_pseudohdr *bp;
-	struct scatterlist sg;
-	struct tcphdr *_th;
-
-	bp = hp->scratch;
-	bp->saddr = saddr;
-	bp->daddr = daddr;
-	bp->pad = 0;
-	bp->protocol = IPPROTO_TCP;
-	bp->len = cpu_to_be16(nbytes);
-
-	_th = (struct tcphdr *)(bp + 1);
-	memcpy(_th, th, sizeof(*th));
-	_th->check = 0;
-
-	sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th));
-	ahash_request_set_crypt(hp->md5_req, &sg, NULL,
-				sizeof(*bp) + sizeof(*th));
-	return crypto_ahash_update(hp->md5_req);
-}
-
-static int tcp_v4_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
-			       __be32 daddr, __be32 saddr, const struct tcphdr *th)
-{
-	struct tcp_md5sig_pool *hp;
-	struct ahash_request *req;
-
-	hp = tcp_get_md5sig_pool();
-	if (!hp)
-		goto clear_hash_noput;
-	req = hp->md5_req;
-
-	if (crypto_ahash_init(req))
-		goto clear_hash;
-	if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2))
-		goto clear_hash;
-	if (tcp_md5_hash_key(hp, key))
-		goto clear_hash;
-	ahash_request_set_crypt(req, NULL, md5_hash, 0);
-	if (crypto_ahash_final(req))
-		goto clear_hash;
-
-	tcp_put_md5sig_pool();
-	return 0;
-
-clear_hash:
-	tcp_put_md5sig_pool();
-clear_hash_noput:
-	memset(md5_hash, 0, 16);
-	return 1;
-}
-
-int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
-			const struct sock *sk,
-			const struct sk_buff *skb)
-{
-	struct tcp_md5sig_pool *hp;
-	struct ahash_request *req;
-	const struct tcphdr *th = tcp_hdr(skb);
-	__be32 saddr, daddr;
-
-	if (sk) { /* valid for establish/request sockets */
-		saddr = sk->sk_rcv_saddr;
-		daddr = sk->sk_daddr;
-	} else {
-		const struct iphdr *iph = ip_hdr(skb);
-		saddr = iph->saddr;
-		daddr = iph->daddr;
-	}
-
-	hp = tcp_get_md5sig_pool();
-	if (!hp)
-		goto clear_hash_noput;
-	req = hp->md5_req;
-
-	if (crypto_ahash_init(req))
-		goto clear_hash;
-
-	if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, skb->len))
-		goto clear_hash;
-	if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2))
-		goto clear_hash;
-	if (tcp_md5_hash_key(hp, key))
-		goto clear_hash;
-	ahash_request_set_crypt(req, NULL, md5_hash, 0);
-	if (crypto_ahash_final(req))
-		goto clear_hash;
-
-	tcp_put_md5sig_pool();
-	return 0;
-
-clear_hash:
-	tcp_put_md5sig_pool();
-clear_hash_noput:
-	memset(md5_hash, 0, 16);
-	return 1;
-}
-EXPORT_SYMBOL(tcp_v4_md5_hash_skb);
-
-#endif
-
-/* Called with rcu_read_lock() */
-static bool tcp_v4_inbound_md5_hash(const struct sock *sk,
-				    const struct sk_buff *skb)
-{
-#ifdef CONFIG_TCP_MD5SIG
-	/*
-	 * This gets called for each TCP segment that arrives
-	 * so we want to be efficient.
-	 * We have 3 drop cases:
-	 * o No MD5 hash and one expected.
-	 * o MD5 hash and we're not expecting one.
-	 * o MD5 hash and its wrong.
-	 */
-	const __u8 *hash_location = NULL;
-	struct tcp_md5sig_key *hash_expected;
-	const struct iphdr *iph = ip_hdr(skb);
-	const struct tcphdr *th = tcp_hdr(skb);
-	int genhash;
-	unsigned char newhash[16];
-
-	hash_expected = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&iph->saddr,
-					  AF_INET);
-	hash_location = tcp_parse_md5sig_option(th);
-
-	/* We've parsed the options - do we have a hash? */
-	if (!hash_expected && !hash_location)
-		return false;
-
-	if (hash_expected && !hash_location) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5NOTFOUND);
-		return true;
-	}
-
-	if (!hash_expected && hash_location) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5UNEXPECTED);
-		return true;
-	}
-
-	/* Okay, so this is hash_expected and hash_location -
-	 * so we need to calculate the checksum.
-	 */
-	genhash = tcp_v4_md5_hash_skb(newhash,
-				      hash_expected,
-				      NULL, skb);
-
-	if (genhash || memcmp(hash_location, newhash, 16) != 0) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5FAILURE);
-		net_info_ratelimited("MD5 Hash failed for (%pI4, %d)->(%pI4, %d)%s\n",
-				     &iph->saddr, ntohs(th->source),
-				     &iph->daddr, ntohs(th->dest),
-				     genhash ? " tcp_v4_calc_md5_hash failed"
-				     : "");
-		return true;
-	}
-	return false;
-#endif
-	return false;
-}
-
 static void tcp_v4_init_req(struct request_sock *req,
 			    const struct sock *sk_listener,
 			    struct sk_buff *skb)
@@ -1403,9 +977,6 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
 	struct inet_sock *newinet;
 	struct tcp_sock *newtp;
 	struct sock *newsk;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key *key;
-#endif
 	struct ip_options_rcu *inet_opt;
 
 	if (sk_acceptq_is_full(sk))
@@ -1457,20 +1028,7 @@ struct sock *tcp_v4_syn_recv_sock(const struct sock *sk, struct sk_buff *skb,
 		INIT_LIST_HEAD(&tcp_rsk(req)->tcp_option_list);
 	}
 #ifdef CONFIG_TCP_MD5SIG
-	/* Copy over the MD5 key from the original socket */
-	key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&newinet->inet_daddr,
-				AF_INET);
-	if (key) {
-		/*
-		 * We're using one, so create a matching key
-		 * on the newsk structure. If we fail to get
-		 * memory, then we end up not copying the key
-		 * across. Shucks.
-		 */
-		tcp_md5_do_add(newsk, (union tcp_md5_addr *)&newinet->inet_daddr,
-			       AF_INET, 32, key->key, key->keylen, GFP_ATOMIC);
-		sk_nocaps_add(newsk, NETIF_F_GSO_MASK);
-	}
+	tcp_v4_md5_syn_recv_sock(sk, newsk);
 #endif
 
 	if (__inet_inherit_port(sk, newsk) < 0)
@@ -1908,14 +1466,6 @@ const struct inet_connection_sock_af_ops ipv4_specific = {
 };
 EXPORT_SYMBOL(ipv4_specific);
 
-#ifdef CONFIG_TCP_MD5SIG
-static const struct tcp_sock_af_ops tcp_sock_ipv4_specific = {
-	.md5_lookup		= tcp_v4_md5_lookup,
-	.calc_md5_hash		= tcp_v4_md5_hash_skb,
-	.md5_parse		= tcp_v4_parse_md5_keys,
-};
-#endif
-
 /* NOTE: A lot of things set to zero explicitly by call to
 *       sk_alloc() so need not be done here.
 */
@@ -1958,12 +1508,7 @@ void tcp_v4_destroy_sock(struct sock *sk)
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
 		tcp_extra_options_destroy(sk);
 #ifdef CONFIG_TCP_MD5SIG
-	/* Clean up the MD5 key list, if any */
-	if (tp->md5sig_info) {
-		tcp_clear_md5_list(sk);
-		kfree_rcu(tp->md5sig_info, rcu);
-		tp->md5sig_info = NULL;
-	}
+	tcp_v4_md5_destroy_sock(sk);
 #endif
 
 	/* Clean up a referenced TCP bind bucket. */
net/ipv4/tcp_md5.c new file +1047 -0
new file mode 100644
index 000000000000..a7fbcda0e505
@@ -0,0 +1,1104 @@
+#include <linux/inet_diag.h>
+#include <linux/inetdevice.h>
+#include <linux/tcp.h>
+#include <linux/tcp_md5.h>
+
+#include <crypto/hash.h>
+
+#include <net/inet6_hashtables.h>
+
+static DEFINE_PER_CPU(struct tcp_md5sig_pool, tcp_md5sig_pool);
+static DEFINE_MUTEX(tcp_md5sig_mutex);
+static bool tcp_md5sig_pool_populated;
+
+#define tcp_twsk_md5_key(twsk)	((twsk)->tw_md5_key)
+
+static void __tcp_alloc_md5sig_pool(void)
+{
+	struct crypto_ahash *hash;
+	int cpu;
+
+	hash = crypto_alloc_ahash("md5", 0, CRYPTO_ALG_ASYNC);
+	if (IS_ERR(hash))
+		return;
+
+	for_each_possible_cpu(cpu) {
+		void *scratch = per_cpu(tcp_md5sig_pool, cpu).scratch;
+		struct ahash_request *req;
+
+		if (!scratch) {
+			scratch = kmalloc_node(sizeof(union tcp_md5sum_block) +
+					       sizeof(struct tcphdr),
+					       GFP_KERNEL,
+					       cpu_to_node(cpu));
+			if (!scratch)
+				return;
+			per_cpu(tcp_md5sig_pool, cpu).scratch = scratch;
+		}
+		if (per_cpu(tcp_md5sig_pool, cpu).md5_req)
+			continue;
+
+		req = ahash_request_alloc(hash, GFP_KERNEL);
+		if (!req)
+			return;
+
+		ahash_request_set_callback(req, 0, NULL, NULL);
+
+		per_cpu(tcp_md5sig_pool, cpu).md5_req = req;
+	}
+	/* before setting tcp_md5sig_pool_populated, we must commit all writes
+	 * to memory. See smp_rmb() in tcp_get_md5sig_pool()
+	 */
+	smp_wmb();
+	tcp_md5sig_pool_populated = true;
+}
+
+static bool tcp_alloc_md5sig_pool(void)
+{
+	if (unlikely(!tcp_md5sig_pool_populated)) {
+		mutex_lock(&tcp_md5sig_mutex);
+
+		if (!tcp_md5sig_pool_populated)
+			__tcp_alloc_md5sig_pool();
+
+		mutex_unlock(&tcp_md5sig_mutex);
+	}
+	return tcp_md5sig_pool_populated;
+}
+
+static void tcp_put_md5sig_pool(void)
+{
+	local_bh_enable();
+}
+
+/**
+ *	tcp_get_md5sig_pool - get md5sig_pool for this user
+ *
+ *	We use percpu structure, so if we succeed, we exit with preemption
+ *	and BH disabled, to make sure another thread or softirq handling
+ *	wont try to get same context.
+ */
+static struct tcp_md5sig_pool *tcp_get_md5sig_pool(void)
+{
+	local_bh_disable();
+
+	if (tcp_md5sig_pool_populated) {
+		/* coupled with smp_wmb() in __tcp_alloc_md5sig_pool() */
+		smp_rmb();
+		return this_cpu_ptr(&tcp_md5sig_pool);
+	}
+	local_bh_enable();
+	return NULL;
+}
+
+static struct tcp_md5sig_key *tcp_md5_do_lookup_exact(const struct sock *sk,
+						      const union tcp_md5_addr *addr,
+						      int family, u8 prefixlen)
+{
+	const struct tcp_sock *tp = tcp_sk(sk);
+	struct tcp_md5sig_key *key;
+	unsigned int size = sizeof(struct in_addr);
+	const struct tcp_md5sig_info *md5sig;
+
+	/* caller either holds rcu_read_lock() or socket lock */
+	md5sig = rcu_dereference_check(tp->md5sig_info,
+				       lockdep_sock_is_held(sk));
+	if (!md5sig)
+		return NULL;
+#if IS_ENABLED(CONFIG_IPV6)
+	if (family == AF_INET6)
+		size = sizeof(struct in6_addr);
+#endif
+	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
+		if (key->family != family)
+			continue;
+		if (!memcmp(&key->addr, addr, size) &&
+		    key->prefixlen == prefixlen)
+			return key;
+	}
+	return NULL;
+}
+
+/* This can be called on a newly created socket, from other files */
+static int tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
+			  int family, u8 prefixlen, const u8 *newkey,
+			  u8 newkeylen, gfp_t gfp)
+{
+	/* Add Key to the list */
+	struct tcp_md5sig_key *key;
+	struct tcp_sock *tp = tcp_sk(sk);
+	struct tcp_md5sig_info *md5sig;
+
+	key = tcp_md5_do_lookup_exact(sk, addr, family, prefixlen);
+	if (key) {
+		/* Pre-existing entry - just update that one. */
+		memcpy(key->key, newkey, newkeylen);
+		key->keylen = newkeylen;
+		return 0;
+	}
+
+	md5sig = rcu_dereference_protected(tp->md5sig_info,
+					   lockdep_sock_is_held(sk));
+	if (!md5sig) {
+		md5sig = kmalloc(sizeof(*md5sig), gfp);
+		if (!md5sig)
+			return -ENOMEM;
+
+		sk_nocaps_add(sk, NETIF_F_GSO_MASK);
+		INIT_HLIST_HEAD(&md5sig->head);
+		rcu_assign_pointer(tp->md5sig_info, md5sig);
+	}
+
+	key = sock_kmalloc(sk, sizeof(*key), gfp);
+	if (!key)
+		return -ENOMEM;
+	if (!tcp_alloc_md5sig_pool()) {
+		sock_kfree_s(sk, key, sizeof(*key));
+		return -ENOMEM;
+	}
+
+	memcpy(key->key, newkey, newkeylen);
+	key->keylen = newkeylen;
+	key->family = family;
+	key->prefixlen = prefixlen;
+	memcpy(&key->addr, addr,
+	       (family == AF_INET6) ? sizeof(struct in6_addr) :
+				      sizeof(struct in_addr));
+	hlist_add_head_rcu(&key->node, &md5sig->head);
+	return 0;
+}
+
+static void tcp_clear_md5_list(struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+	struct tcp_md5sig_key *key;
+	struct hlist_node *n;
+	struct tcp_md5sig_info *md5sig;
+
+	md5sig = rcu_dereference_protected(tp->md5sig_info, 1);
+
+	hlist_for_each_entry_safe(key, n, &md5sig->head, node) {
+		hlist_del_rcu(&key->node);
+		atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
+		kfree_rcu(key, rcu);
+	}
+}
+
+static int tcp_md5_do_del(struct sock *sk, const union tcp_md5_addr *addr,
+			  int family, u8 prefixlen)
+{
+	struct tcp_md5sig_key *key;
+
+	key = tcp_md5_do_lookup_exact(sk, addr, family, prefixlen);
+	if (!key)
+		return -ENOENT;
+	hlist_del_rcu(&key->node);
+	atomic_sub(sizeof(*key), &sk->sk_omem_alloc);
+	kfree_rcu(key, rcu);
+	return 0;
+}
+
+static int tcp_md5_hash_key(struct tcp_md5sig_pool *hp,
+			    const struct tcp_md5sig_key *key)
+{
+	struct scatterlist sg;
+
+	sg_init_one(&sg, key->key, key->keylen);
+	ahash_request_set_crypt(hp->md5_req, &sg, NULL, key->keylen);
+	return crypto_ahash_update(hp->md5_req);
+}
+
+static int tcp_v4_parse_md5_keys(struct sock *sk, int optname,
+				 char __user *optval, int optlen)
+{
+	struct tcp_md5sig cmd;
+	struct sockaddr_in *sin = (struct sockaddr_in *)&cmd.tcpm_addr;
+	u8 prefixlen = 32;
+
+	if (optlen < sizeof(cmd))
+		return -EINVAL;
+
+	if (copy_from_user(&cmd, optval, sizeof(cmd)))
+		return -EFAULT;
+
+	if (sin->sin_family != AF_INET)
+		return -EINVAL;
+
+	if (optname == TCP_MD5SIG_EXT &&
+	    cmd.tcpm_flags & TCP_MD5SIG_FLAG_PREFIX) {
+		prefixlen = cmd.tcpm_prefixlen;
+		if (prefixlen > 32)
+			return -EINVAL;
+	}
+
+	if (!cmd.tcpm_keylen)
+		return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin->sin_addr.s_addr,
+				      AF_INET, prefixlen);
+
+	if (cmd.tcpm_keylen > TCP_MD5SIG_MAXKEYLEN)
+		return -EINVAL;
+
+	return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin->sin_addr.s_addr,
+			      AF_INET, prefixlen, cmd.tcpm_key, cmd.tcpm_keylen,
+			      GFP_KERNEL);
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+static int tcp_v6_parse_md5_keys(struct sock *sk, int optname,
+				 char __user *optval, int optlen)
+{
+	struct tcp_md5sig cmd;
+	struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)&cmd.tcpm_addr;
+	u8 prefixlen;
+
+	if (optlen < sizeof(cmd))
+		return -EINVAL;
+
+	if (copy_from_user(&cmd, optval, sizeof(cmd)))
+		return -EFAULT;
+
+	if (sin6->sin6_family != AF_INET6)
+		return -EINVAL;
+
+	if (optname == TCP_MD5SIG_EXT &&
+	    cmd.tcpm_flags & TCP_MD5SIG_FLAG_PREFIX) {
+		prefixlen = cmd.tcpm_prefixlen;
+		if (prefixlen > 128 || (ipv6_addr_v4mapped(&sin6->sin6_addr) &&
+					prefixlen > 32))
+			return -EINVAL;
+	} else {
+		prefixlen = ipv6_addr_v4mapped(&sin6->sin6_addr) ? 32 : 128;
+	}
+
+	if (!cmd.tcpm_keylen) {
+		if (ipv6_addr_v4mapped(&sin6->sin6_addr))
+			return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin6->sin6_addr.s6_addr32[3],
+					      AF_INET, prefixlen);
+		return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin6->sin6_addr,
+				      AF_INET6, prefixlen);
+	}
+
+	if (cmd.tcpm_keylen > TCP_MD5SIG_MAXKEYLEN)
+		return -EINVAL;
+
+	if (ipv6_addr_v4mapped(&sin6->sin6_addr))
+		return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin6->sin6_addr.s6_addr32[3],
+				      AF_INET, prefixlen, cmd.tcpm_key,
+				      cmd.tcpm_keylen, GFP_KERNEL);
+
+	return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin6->sin6_addr,
+			      AF_INET6, prefixlen, cmd.tcpm_key,
+			      cmd.tcpm_keylen, GFP_KERNEL);
+}
+#endif
+
+static int tcp_v4_md5_hash_headers(struct tcp_md5sig_pool *hp,
+				   __be32 daddr, __be32 saddr,
+				   const struct tcphdr *th, int nbytes)
+{
+	struct tcp4_pseudohdr *bp;
+	struct scatterlist sg;
+	struct tcphdr *_th;
+
+	bp = hp->scratch;
+	bp->saddr = saddr;
+	bp->daddr = daddr;
+	bp->pad = 0;
+	bp->protocol = IPPROTO_TCP;
+	bp->len = cpu_to_be16(nbytes);
+
+	_th = (struct tcphdr *)(bp + 1);
+	memcpy(_th, th, sizeof(*th));
+	_th->check = 0;
+
+	sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th));
+	ahash_request_set_crypt(hp->md5_req, &sg, NULL,
+				sizeof(*bp) + sizeof(*th));
+	return crypto_ahash_update(hp->md5_req);
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+static int tcp_v6_md5_hash_headers(struct tcp_md5sig_pool *hp,
+				   const struct in6_addr *daddr,
+				   const struct in6_addr *saddr,
+				   const struct tcphdr *th, int nbytes)
+{
+	struct tcp6_pseudohdr *bp;
+	struct scatterlist sg;
+	struct tcphdr *_th;
+
+	bp = hp->scratch;
+	/* 1. TCP pseudo-header (RFC2460) */
+	bp->saddr = *saddr;
+	bp->daddr = *daddr;
+	bp->protocol = cpu_to_be32(IPPROTO_TCP);
+	bp->len = cpu_to_be32(nbytes);
+
+	_th = (struct tcphdr *)(bp + 1);
+	memcpy(_th, th, sizeof(*th));
+	_th->check = 0;
+
+	sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th));
+	ahash_request_set_crypt(hp->md5_req, &sg, NULL,
+				sizeof(*bp) + sizeof(*th));
+	return crypto_ahash_update(hp->md5_req);
+}
+#endif
+
+static int tcp_v4_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
+			       __be32 daddr, __be32 saddr,
+			       const struct tcphdr *th)
+{
+	struct tcp_md5sig_pool *hp;
+	struct ahash_request *req;
+
+	hp = tcp_get_md5sig_pool();
+	if (!hp)
+		goto clear_hash_noput;
+	req = hp->md5_req;
+
+	if (crypto_ahash_init(req))
+		goto clear_hash;
+	if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2))
+		goto clear_hash;
+	if (tcp_md5_hash_key(hp, key))
+		goto clear_hash;
+	ahash_request_set_crypt(req, NULL, md5_hash, 0);
+	if (crypto_ahash_final(req))
+		goto clear_hash;
+
+	tcp_put_md5sig_pool();
+	return 0;
+
+clear_hash:
+	tcp_put_md5sig_pool();
+clear_hash_noput:
+	memset(md5_hash, 0, 16);
+	return 1;
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+static int tcp_v6_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
+			       const struct in6_addr *daddr,
+			       struct in6_addr *saddr, const struct tcphdr *th)
+{
+	struct tcp_md5sig_pool *hp;
+	struct ahash_request *req;
+
+	hp = tcp_get_md5sig_pool();
+	if (!hp)
+		goto clear_hash_noput;
+	req = hp->md5_req;
+
+	if (crypto_ahash_init(req))
+		goto clear_hash;
+	if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2))
+		goto clear_hash;
+	if (tcp_md5_hash_key(hp, key))
+		goto clear_hash;
+	ahash_request_set_crypt(req, NULL, md5_hash, 0);
+	if (crypto_ahash_final(req))
+		goto clear_hash;
+
+	tcp_put_md5sig_pool();
+	return 0;
+
+clear_hash:
+	tcp_put_md5sig_pool();
+clear_hash_noput:
+	memset(md5_hash, 0, 16);
+	return 1;
+}
+#endif
+
+/* RFC2385 MD5 checksumming requires a mapping of
+ * IP address->MD5 Key.
+ * We need to maintain these in the sk structure.
+ */
+
+/* Find the Key structure for an address.  */
+static struct tcp_md5sig_key *tcp_md5_do_lookup(const struct sock *sk,
+						const union tcp_md5_addr *addr,
+						int family)
+{
+	const struct tcp_sock *tp = tcp_sk(sk);
+	struct tcp_md5sig_key *key;
+	const struct tcp_md5sig_info *md5sig;
+	__be32 mask;
+	struct tcp_md5sig_key *best_match = NULL;
+	bool match;
+
+	/* caller either holds rcu_read_lock() or socket lock */
+	md5sig = rcu_dereference_check(tp->md5sig_info,
+				       lockdep_sock_is_held(sk));
+	if (!md5sig)
+		return NULL;
+
+	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
+		if (key->family != family)
+			continue;
+
+		if (family == AF_INET) {
+			mask = inet_make_mask(key->prefixlen);
+			match = (key->addr.a4.s_addr & mask) ==
+				(addr->a4.s_addr & mask);
+#if IS_ENABLED(CONFIG_IPV6)
+		} else if (family == AF_INET6) {
+			match = ipv6_prefix_equal(&key->addr.a6, &addr->a6,
+						  key->prefixlen);
+#endif
+		} else {
+			match = false;
+		}
+
+		if (match && (!best_match ||
+			      key->prefixlen > best_match->prefixlen))
+			best_match = key;
+	}
+	return best_match;
+}
+
+/* Parse MD5 Signature option */
+static const u8 *tcp_parse_md5sig_option(const struct tcphdr *th)
+{
+	int length = (th->doff << 2) - sizeof(*th);
+	const u8 *ptr = (const u8 *)(th + 1);
+
+	/* If the TCP option is too short, we can short cut */
+	if (length < TCPOLEN_MD5SIG)
+		return NULL;
+
+	while (length > 0) {
+		int opcode = *ptr++;
+		int opsize;
+
+		switch (opcode) {
+		case TCPOPT_EOL:
+			return NULL;
+		case TCPOPT_NOP:
+			length--;
+			continue;
+		default:
+			opsize = *ptr++;
+			if (opsize < 2 || opsize > length)
+				return NULL;
+			if (opcode == TCPOPT_MD5SIG)
+				return opsize == TCPOLEN_MD5SIG ? ptr : NULL;
+		}
+		ptr += opsize - 2;
+		length -= opsize;
+	}
+	return NULL;
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+static struct tcp_md5sig_key *tcp_v6_md5_do_lookup(const struct sock *sk,
+						   const struct in6_addr *addr)
+{
+	return tcp_md5_do_lookup(sk, (union tcp_md5_addr *)addr, AF_INET6);
+}
+#endif
+
+static int tcp_md5_hash_skb_data(struct tcp_md5sig_pool *hp,
+				 const struct sk_buff *skb,
+				 unsigned int header_len)
+{
+	struct scatterlist sg;
+	const struct tcphdr *tp = tcp_hdr(skb);
+	struct ahash_request *req = hp->md5_req;
+	unsigned int i;
+	const unsigned int head_data_len = skb_headlen(skb) > header_len ?
+					   skb_headlen(skb) - header_len : 0;
+	const struct skb_shared_info *shi = skb_shinfo(skb);
+	struct sk_buff *frag_iter;
+
+	sg_init_table(&sg, 1);
+
+	sg_set_buf(&sg, ((u8 *)tp) + header_len, head_data_len);
+	ahash_request_set_crypt(req, &sg, NULL, head_data_len);
+	if (crypto_ahash_update(req))
+		return 1;
+
+	for (i = 0; i < shi->nr_frags; ++i) {
+		const struct skb_frag_struct *f = &shi->frags[i];
+		unsigned int offset = f->page_offset;
+		struct page *page = skb_frag_page(f) + (offset >> PAGE_SHIFT);
+
+		sg_set_page(&sg, page, skb_frag_size(f),
+			    offset_in_page(offset));
+		ahash_request_set_crypt(req, &sg, NULL, skb_frag_size(f));
+		if (crypto_ahash_update(req))
+			return 1;
+	}
+
+	skb_walk_frags(skb, frag_iter)
+		if (tcp_md5_hash_skb_data(hp, frag_iter, 0))
+			return 1;
+
+	return 0;
+}
+
+int tcp_v4_md5_send_response_prepare(struct sk_buff *skb, u8 flags,
+				     unsigned int remaining,
+				     struct tcp_out_options *opts,
+				     const struct sock *sk)
+{
+	const struct tcphdr *th = tcp_hdr(skb);
+	const struct iphdr *iph = ip_hdr(skb);
+	const __u8 *hash_location = NULL;
+
+	rcu_read_lock();
+	hash_location = tcp_parse_md5sig_option(th);
+	if (sk && sk_fullsock(sk)) {
+		opts->md5 = tcp_md5_do_lookup(sk,
+					      (union tcp_md5_addr *)&iph->daddr,
+					      AF_INET);
+	} else if (sk && sk->sk_state == TCP_TIME_WAIT) {
+		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
+
+		opts->md5 = tcp_twsk_md5_key(tcptw);
+	} else if (sk && sk->sk_state == TCP_NEW_SYN_RECV) {
+		opts->md5 = tcp_md5_do_lookup(sk,
+					      (union tcp_md5_addr *)&iph->daddr,
+					      AF_INET);
+	} else if (hash_location) {
+		unsigned char newhash[16];
+		struct sock *sk1;
+		int genhash;
+
+		/* active side is lost. Try to find listening socket through
+		 * source port, and then find md5 key through listening socket.
+		 * we are not loose security here:
+		 * Incoming packet is checked with md5 hash with finding key,
+		 * no RST generated if md5 hash doesn't match.
+		 */
+		sk1 = __inet_lookup_listener(dev_net(skb_dst(skb)->dev),
+					     &tcp_hashinfo, NULL, 0,
+					     iph->saddr,
+					     th->source, iph->daddr,
+					     ntohs(th->source), inet_iif(skb),
+					     tcp_v4_sdif(skb));
+		/* don't send rst if it can't find key */
+		if (!sk1)
+			goto out_err;
+
+		opts->md5 = tcp_md5_do_lookup(sk1, (union tcp_md5_addr *)
+					      &iph->saddr, AF_INET);
+		if (!opts->md5)
+			goto out_err;
+
+		genhash = tcp_v4_md5_hash_skb(newhash, opts->md5, NULL, skb);
+		if (genhash || memcmp(hash_location, newhash, 16) != 0)
+			goto out_err;
+	}
+
+	if (opts->md5)
+		return TCPOLEN_MD5SIG_ALIGNED;
+
+	rcu_read_unlock();
+	return 0;
+
+out_err:
+	rcu_read_unlock();
+	return -1;
+}
+
+
+void tcp_v4_md5_send_response_write(__be32 *topt, struct sk_buff *skb,
+				    struct tcphdr *t1,
+				    struct tcp_out_options *opts,
+				    const struct sock *sk)
+{
+
+	if (opts->md5) {
+		*topt++ = htonl((TCPOPT_NOP << 24) |
+			        (TCPOPT_NOP << 16) |
+			        (TCPOPT_MD5SIG << 8) |
+			        TCPOLEN_MD5SIG);
+
+		tcp_v4_md5_hash_hdr((__u8 *)topt, opts->md5,
+				    ip_hdr(skb)->saddr,
+				    ip_hdr(skb)->daddr, t1);
+	}
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+int tcp_v6_md5_send_response_prepare(struct sk_buff *skb, u8 flags,
+				     unsigned int remaining,
+				     struct tcp_out_options *opts,
+				     const struct sock *sk)
+{
+	const struct tcphdr *th = tcp_hdr(skb);
+	struct ipv6hdr *ipv6h = ipv6_hdr(skb);
+	const __u8 *hash_location = NULL;
+
+	rcu_read_lock();
+	hash_location = tcp_parse_md5sig_option(th);
+	if (sk && sk_fullsock(sk)) {
+		opts->md5 = tcp_v6_md5_do_lookup(sk, &ipv6h->saddr);
+	} else if (sk && sk->sk_state == TCP_TIME_WAIT) {
+		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
+
+		opts->md5 = tcp_twsk_md5_key(tcptw);
+	} else if (sk && sk->sk_state == TCP_NEW_SYN_RECV) {
+		opts->md5 = tcp_v6_md5_do_lookup(sk, &ipv6h->daddr);
+	} else if (hash_location) {
+		unsigned char newhash[16];
+		struct sock *sk1;
+		int genhash;
+
+		/* active side is lost. Try to find listening socket through
+		 * source port, and then find md5 key through listening socket.
+		 * we are not loose security here:
+		 * Incoming packet is checked with md5 hash with finding key,
+		 * no RST generated if md5 hash doesn't match.
+		 */
+		sk1 = inet6_lookup_listener(dev_net(skb_dst(skb)->dev),
+					    &tcp_hashinfo, NULL, 0,
+					    &ipv6h->saddr,
+					    th->source, &ipv6h->daddr,
+					    ntohs(th->source), tcp_v6_iif(skb),
+					    tcp_v6_sdif(skb));
+		if (!sk1)
+			goto out_err;
+
+		opts->md5 = tcp_v6_md5_do_lookup(sk1, &ipv6h->saddr);
+		if (!opts->md5)
+			goto out_err;
+
+		genhash = tcp_v6_md5_hash_skb(newhash, opts->md5, NULL, skb);
+		if (genhash || memcmp(hash_location, newhash, 16) != 0)
+			goto out_err;
+	}
+
+	if (opts->md5)
+		return TCPOLEN_MD5SIG_ALIGNED;
+
+	rcu_read_unlock();
+	return 0;
+
+out_err:
+	rcu_read_unlock();
+	return -1;
+}
+EXPORT_SYMBOL_GPL(tcp_v6_md5_send_response_prepare);
+
+void tcp_v6_md5_send_response_write(__be32 *topt, struct sk_buff *skb,
+				    struct tcphdr *t1,
+				    struct tcp_out_options *opts,
+				    const struct sock *sk)
+{
+	if (opts->md5) {
+		*topt++ = htonl((TCPOPT_NOP << 24) | (TCPOPT_NOP << 16) |
+				(TCPOPT_MD5SIG << 8) | TCPOLEN_MD5SIG);
+		tcp_v6_md5_hash_hdr((__u8 *)topt, opts->md5,
+				    &ipv6_hdr(skb)->saddr,
+				    &ipv6_hdr(skb)->daddr, t1);
+
+
+		rcu_read_unlock();
+	}
+}
+EXPORT_SYMBOL_GPL(tcp_v6_md5_send_response_write);
+#endif
+
+struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
+					 const struct sock *addr_sk)
+{
+	const union tcp_md5_addr *addr;
+
+	addr = (const union tcp_md5_addr *)&addr_sk->sk_daddr;
+	return tcp_md5_do_lookup(sk, addr, AF_INET);
+}
+EXPORT_SYMBOL(tcp_v4_md5_lookup);
+
+int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
+			const struct sock *sk,
+			const struct sk_buff *skb)
+{
+	struct tcp_md5sig_pool *hp;
+	struct ahash_request *req;
+	const struct tcphdr *th = tcp_hdr(skb);
+	__be32 saddr, daddr;
+
+	if (sk) { /* valid for establish/request sockets */
+		saddr = sk->sk_rcv_saddr;
+		daddr = sk->sk_daddr;
+	} else {
+		const struct iphdr *iph = ip_hdr(skb);
+
+		saddr = iph->saddr;
+		daddr = iph->daddr;
+	}
+
+	hp = tcp_get_md5sig_pool();
+	if (!hp)
+		goto clear_hash_noput;
+	req = hp->md5_req;
+
+	if (crypto_ahash_init(req))
+		goto clear_hash;
+
+	if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, skb->len))
+		goto clear_hash;
+	if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2))
+		goto clear_hash;
+	if (tcp_md5_hash_key(hp, key))
+		goto clear_hash;
+	ahash_request_set_crypt(req, NULL, md5_hash, 0);
+	if (crypto_ahash_final(req))
+		goto clear_hash;
+
+	tcp_put_md5sig_pool();
+	return 0;
+
+clear_hash:
+	tcp_put_md5sig_pool();
+clear_hash_noput:
+	memset(md5_hash, 0, 16);
+	return 1;
+}
+EXPORT_SYMBOL(tcp_v4_md5_hash_skb);
+
+#if IS_ENABLED(CONFIG_IPV6)
+int tcp_v6_md5_hash_skb(char *md5_hash,
+			const struct tcp_md5sig_key *key,
+			const struct sock *sk,
+			const struct sk_buff *skb)
+{
+	const struct in6_addr *saddr, *daddr;
+	struct tcp_md5sig_pool *hp;
+	struct ahash_request *req;
+	const struct tcphdr *th = tcp_hdr(skb);
+
+	if (sk) { /* valid for establish/request sockets */
+		saddr = &sk->sk_v6_rcv_saddr;
+		daddr = &sk->sk_v6_daddr;
+	} else {
+		const struct ipv6hdr *ip6h = ipv6_hdr(skb);
+
+		saddr = &ip6h->saddr;
+		daddr = &ip6h->daddr;
+	}
+
+	hp = tcp_get_md5sig_pool();
+	if (!hp)
+		goto clear_hash_noput;
+	req = hp->md5_req;
+
+	if (crypto_ahash_init(req))
+		goto clear_hash;
+
+	if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, skb->len))
+		goto clear_hash;
+	if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2))
+		goto clear_hash;
+	if (tcp_md5_hash_key(hp, key))
+		goto clear_hash;
+	ahash_request_set_crypt(req, NULL, md5_hash, 0);
+	if (crypto_ahash_final(req))
+		goto clear_hash;
+
+	tcp_put_md5sig_pool();
+	return 0;
+
+clear_hash:
+	tcp_put_md5sig_pool();
+clear_hash_noput:
+	memset(md5_hash, 0, 16);
+	return 1;
+}
+EXPORT_SYMBOL_GPL(tcp_v6_md5_hash_skb);
+#endif
+
+/* Called with rcu_read_lock() */
+bool tcp_v4_inbound_md5_hash(const struct sock *sk,
+			     const struct sk_buff *skb)
+{
+	/* This gets called for each TCP segment that arrives
+	 * so we want to be efficient.
+	 * We have 3 drop cases:
+	 * o No MD5 hash and one expected.
+	 * o MD5 hash and we're not expecting one.
+	 * o MD5 hash and its wrong.
+	 */
+	const __u8 *hash_location = NULL;
+	struct tcp_md5sig_key *hash_expected;
+	const struct iphdr *iph = ip_hdr(skb);
+	const struct tcphdr *th = tcp_hdr(skb);
+	int genhash;
+	unsigned char newhash[16];
+
+	hash_expected = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)&iph->saddr,
+					  AF_INET);
+	hash_location = tcp_parse_md5sig_option(th);
+
+	/* We've parsed the options - do we have a hash? */
+	if (!hash_expected && !hash_location)
+		return false;
+
+	if (hash_expected && !hash_location) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5NOTFOUND);
+		return true;
+	}
+
+	if (!hash_expected && hash_location) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5UNEXPECTED);
+		return true;
+	}
+
+	/* Okay, so this is hash_expected and hash_location -
+	 * so we need to calculate the checksum.
+	 */
+	genhash = tcp_v4_md5_hash_skb(newhash,
+				      hash_expected,
+				      NULL, skb);
+
+	if (genhash || memcmp(hash_location, newhash, 16) != 0) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5FAILURE);
+		net_info_ratelimited("MD5 Hash failed for (%pI4, %d)->(%pI4, %d)%s\n",
+				     &iph->saddr, ntohs(th->source),
+				     &iph->daddr, ntohs(th->dest),
+				     genhash ? " tcp_v4_calc_md5_hash failed"
+				     : "");
+		return true;
+	}
+	return false;
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+bool tcp_v6_inbound_md5_hash(const struct sock *sk,
+			     const struct sk_buff *skb)
+{
+	const __u8 *hash_location = NULL;
+	struct tcp_md5sig_key *hash_expected;
+	const struct ipv6hdr *ip6h = ipv6_hdr(skb);
+	const struct tcphdr *th = tcp_hdr(skb);
+	int genhash;
+	u8 newhash[16];
+
+	hash_expected = tcp_v6_md5_do_lookup(sk, &ip6h->saddr);
+	hash_location = tcp_parse_md5sig_option(th);
+
+	/* We've parsed the options - do we have a hash? */
+	if (!hash_expected && !hash_location)
+		return false;
+
+	if (hash_expected && !hash_location) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5NOTFOUND);
+		return true;
+	}
+
+	if (!hash_expected && hash_location) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5UNEXPECTED);
+		return true;
+	}
+
+	/* check the signature */
+	genhash = tcp_v6_md5_hash_skb(newhash,
+				      hash_expected,
+				      NULL, skb);
+
+	if (genhash || memcmp(hash_location, newhash, 16) != 0) {
+		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5FAILURE);
+		net_info_ratelimited("MD5 Hash %s for [%pI6c]:%u->[%pI6c]:%u\n",
+				     genhash ? "failed" : "mismatch",
+				     &ip6h->saddr, ntohs(th->source),
+				     &ip6h->daddr, ntohs(th->dest));
+		return true;
+	}
+
+	return false;
+}
+EXPORT_SYMBOL_GPL(tcp_v6_inbound_md5_hash);
+#endif
+
+void tcp_v4_md5_destroy_sock(struct sock *sk)
+{
+	struct tcp_sock *tp = tcp_sk(sk);
+
+	/* Clean up the MD5 key list, if any */
+	if (tp->md5sig_info) {
+		tcp_clear_md5_list(sk);
+		kfree_rcu(tp->md5sig_info, rcu);
+		tp->md5sig_info = NULL;
+	}
+}
+
+void tcp_v4_md5_syn_recv_sock(const struct sock *listener, struct sock *sk)
+{
+	struct inet_sock *inet = inet_sk(sk);
+	struct tcp_md5sig_key *key;
+
+	/* Copy over the MD5 key from the original socket */
+	key = tcp_md5_do_lookup(listener, (union tcp_md5_addr *)&inet->inet_daddr,
+				AF_INET);
+	if (key) {
+		/* We're using one, so create a matching key
+		 * on the sk structure. If we fail to get
+		 * memory, then we end up not copying the key
+		 * across. Shucks.
+		 */
+		tcp_md5_do_add(sk, (union tcp_md5_addr *)&inet->inet_daddr,
+			       AF_INET, 32, key->key, key->keylen, GFP_ATOMIC);
+		sk_nocaps_add(sk, NETIF_F_GSO_MASK);
+	}
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+void tcp_v6_md5_syn_recv_sock(const struct sock *listener, struct sock *sk)
+{
+	struct tcp_md5sig_key *key;
+
+	/* Copy over the MD5 key from the original socket */
+	key = tcp_v6_md5_do_lookup(listener, &sk->sk_v6_daddr);
+	if (key) {
+		/* We're using one, so create a matching key
+		 * on the newsk structure. If we fail to get
+		 * memory, then we end up not copying the key
+		 * across. Shucks.
+		 */
+		tcp_md5_do_add(sk, (union tcp_md5_addr *)&sk->sk_v6_daddr,
+			       AF_INET6, 128, key->key, key->keylen,
+			       sk_gfp_mask(sk, GFP_ATOMIC));
+	}
+}
+EXPORT_SYMBOL_GPL(tcp_v6_md5_syn_recv_sock);
+
+struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
+					 const struct sock *addr_sk)
+{
+	return tcp_v6_md5_do_lookup(sk, &addr_sk->sk_v6_daddr);
+}
+EXPORT_SYMBOL_GPL(tcp_v6_md5_lookup);
+#endif
+
+void tcp_md5_time_wait(struct sock *sk, struct inet_timewait_sock *tw)
+{
+	struct tcp_timewait_sock *tcptw = tcp_twsk((struct sock *)tw);
+	struct tcp_sock *tp = tcp_sk(sk);
+	struct tcp_md5sig_key *key;
+
+	/* The timewait bucket does not have the key DB from the
+	 * sock structure. We just make a quick copy of the
+	 * md5 key being used (if indeed we are using one)
+	 * so the timewait ack generating code has the key.
+	 */
+	tcptw->tw_md5_key = NULL;
+	key = tp->af_specific->md5_lookup(sk, sk);
+	if (key) {
+		tcptw->tw_md5_key = kmemdup(key, sizeof(*key), GFP_ATOMIC);
+		BUG_ON(tcptw->tw_md5_key && !tcp_alloc_md5sig_pool());
+	}
+}
+
+static void tcp_diag_md5sig_fill(struct tcp_diag_md5sig *info,
+				 const struct tcp_md5sig_key *key)
+{
+	info->tcpm_family = key->family;
+	info->tcpm_prefixlen = key->prefixlen;
+	info->tcpm_keylen = key->keylen;
+	memcpy(info->tcpm_key, key->key, key->keylen);
+
+	if (key->family == AF_INET)
+		info->tcpm_addr[0] = key->addr.a4.s_addr;
+	#if IS_ENABLED(CONFIG_IPV6)
+	else if (key->family == AF_INET6)
+		memcpy(&info->tcpm_addr, &key->addr.a6,
+		       sizeof(info->tcpm_addr));
+	#endif
+}
+
+static int tcp_diag_put_md5sig(struct sk_buff *skb,
+			       const struct tcp_md5sig_info *md5sig)
+{
+	const struct tcp_md5sig_key *key;
+	struct tcp_diag_md5sig *info;
+	struct nlattr *attr;
+	int md5sig_count = 0;
+
+	hlist_for_each_entry_rcu(key, &md5sig->head, node)
+		md5sig_count++;
+	if (md5sig_count == 0)
+		return 0;
+
+	attr = nla_reserve(skb, INET_DIAG_MD5SIG,
+			   md5sig_count * sizeof(struct tcp_diag_md5sig));
+	if (!attr)
+		return -EMSGSIZE;
+
+	info = nla_data(attr);
+	memset(info, 0, md5sig_count * sizeof(struct tcp_diag_md5sig));
+	hlist_for_each_entry_rcu(key, &md5sig->head, node) {
+		tcp_diag_md5sig_fill(info++, key);
+		if (--md5sig_count == 0)
+			break;
+	}
+
+	return 0;
+}
+
+int tcp_md5_diag_get_aux(struct sock *sk, bool net_admin, struct sk_buff *skb)
+{
+	if (net_admin) {
+		struct tcp_md5sig_info *md5sig;
+		int err = 0;
+
+		rcu_read_lock();

net/ipv4/tcp_md5.c line 1057

Quoted 10 lines
net/ipv4/tcp_md5.c new file +10 -0
+		md5sig = rcu_dereference(tcp_sk(sk)->md5sig_info);
+		if (md5sig)
+			err = tcp_diag_put_md5sig(skb, md5sig);
+		rcu_read_unlock();
+		if (err < 0)
+			return err;
+	}
+
+	return 0;
+}

Needs EXPORT_SYMBOL_GPL

Quoted 17 lines
net/ipv4/tcp_md5.c new file +17 -0
+
+int tcp_md5_diag_get_aux_size(struct sock *sk, bool net_admin)
+{
+	int size = 0;
+
+	if (net_admin && sk_fullsock(sk)) {
+		const struct tcp_md5sig_info *md5sig;
+		const struct tcp_md5sig_key *key;
+		size_t md5sig_count = 0;
+
+		rcu_read_lock();
+		md5sig = rcu_dereference(tcp_sk(sk)->md5sig_info);
+		if (md5sig) {
+			hlist_for_each_entry_rcu(key, &md5sig->head, node)
+				md5sig_count++;
+		}
+		rcu_read_unlock();
Quoted 8 lines

net/ipv4/tcp_md5.c line 1080

Quoted 6 lines
net/ipv4/tcp_md5.c new file +6 -0
+		size += nla_total_size(md5sig_count *
+				       sizeof(struct tcp_diag_md5sig));
+	}
+
+	return size;
+}

Also needs EXPORT_SYMBOL_GPL

Thanks, I'm adding this.

Christoph

Quoted 532 lines

Mat

Quoted 524 lines
net/ipv4/tcp_md5.c new file +24 -0
+
+const struct tcp_sock_af_ops tcp_sock_ipv4_specific = {
+	.md5_lookup	= tcp_v4_md5_lookup,
+	.calc_md5_hash	= tcp_v4_md5_hash_skb,
+	.md5_parse	= tcp_v4_parse_md5_keys,
+};
+
+#if IS_ENABLED(CONFIG_IPV6)
+const struct tcp_sock_af_ops tcp_sock_ipv6_specific = {
+	.md5_lookup	=	tcp_v6_md5_lookup,
+	.calc_md5_hash	=	tcp_v6_md5_hash_skb,
+	.md5_parse	=	tcp_v6_parse_md5_keys,
+};
+EXPORT_SYMBOL_GPL(tcp_sock_ipv6_specific);
+
+const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific = {
+	.md5_lookup	=	tcp_v4_md5_lookup,
+	.calc_md5_hash	=	tcp_v4_md5_hash_skb,
+	.md5_parse	=	tcp_v6_parse_md5_keys,
+};
+EXPORT_SYMBOL_GPL(tcp_sock_ipv6_mapped_specific);
+#endif
+
+
net/ipv4/tcp_minisocks.c +4 -22
index 641dca38cc1e..e9e925d51046 100644
@@ -22,6 +22,7 @@
 #include <linux/module.h>
 #include <linux/slab.h>
 #include <linux/sysctl.h>
+#include <linux/tcp_md5.h>
 #include <linux/workqueue.h>
 #include <linux/static_key.h>
 #include <net/tcp.h>
@@ -296,21 +297,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 			INIT_LIST_HEAD(&tcp_sk(sk)->tcp_option_list);
 		}
 #ifdef CONFIG_TCP_MD5SIG
-		/*
-		 * The timewait bucket does not have the key DB from the
-		 * sock structure. We just make a quick copy of the
-		 * md5 key being used (if indeed we are using one)
-		 * so the timewait ack generating code has the key.
-		 */
-		do {
-			struct tcp_md5sig_key *key;
-			tcptw->tw_md5_key = NULL;
-			key = tp->af_specific->md5_lookup(sk, sk);
-			if (key) {
-				tcptw->tw_md5_key = kmemdup(key, sizeof(*key), GFP_ATOMIC);
-				BUG_ON(tcptw->tw_md5_key && !tcp_alloc_md5sig_pool());
-			}
-		} while (0);
+		tcp_md5_time_wait(sk, tw);
 #endif
 
 		/* Get the TIME_WAIT timeout firing. */
@@ -340,10 +327,7 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 void tcp_twsk_destructor(struct sock *sk)
 {
 #ifdef CONFIG_TCP_MD5SIG
-	struct tcp_timewait_sock *twsk = tcp_twsk(sk);
-
-	if (twsk->tw_md5_key)
-		kfree_rcu(twsk->tw_md5_key, rcu);
+	tcp_md5_twsk_destructor(sk);
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
@@ -531,9 +515,7 @@ struct sock *tcp_create_openreq_child(const struct sock *sk,
 		}
 		newtp->tsoffset = treq->ts_off;
 #ifdef CONFIG_TCP_MD5SIG
-		newtp->md5sig_info = NULL;	/*XXX*/
-		if (newtp->af_specific->md5_lookup(sk, newsk))
-			newtp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
+		tcp_md5_add_header_len(sk, newsk);
 #endif
 		if (static_branch_unlikely(&tcp_extra_options_enabled))
 			newtp->tcp_header_len += tcp_extra_options_add_header(sk, newsk);
net/ipv4/tcp_output.c +2 -2
index a7de79294ee6..1c040c44fbf4 100644
@@ -42,6 +42,7 @@
 #include <linux/gfp.h>
 #include <linux/module.h>
 #include <linux/static_key.h>
+#include <linux/tcp_md5.h>
 
 #include <trace/events/tcp.h>
 
@@ -3245,8 +3246,7 @@ static void tcp_connect_init(struct sock *sk)
 		tp->tcp_header_len += TCPOLEN_TSTAMP_ALIGNED;
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (tp->af_specific->md5_lookup(sk, sk))
-		tp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
+	tcp_md5_add_header_len(sk, sk);
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
net/ipv6/tcp_ipv6.c +14 -275
index 2fb688a36731..0d3506309bd4 100644
@@ -43,6 +43,7 @@
 #include <linux/ipv6.h>
 #include <linux/icmpv6.h>
 #include <linux/random.h>
+#include <linux/tcp_md5.h>
 
 #include <net/tcp.h>
 #include <net/ndisc.h>
@@ -79,16 +80,6 @@ static int	tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb);
 
 static const struct inet_connection_sock_af_ops ipv6_mapped;
 static const struct inet_connection_sock_af_ops ipv6_specific;
-#ifdef CONFIG_TCP_MD5SIG
-static const struct tcp_sock_af_ops tcp_sock_ipv6_specific;
-static const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific;
-#else
-static struct tcp_md5sig_key *tcp_v6_md5_do_lookup(const struct sock *sk,
-						   const struct in6_addr *addr)
-{
-	return NULL;
-}
-#endif
 
 static void inet6_sk_rx_dst_set(struct sock *sk, const struct sk_buff *skb)
 {
@@ -507,218 +498,6 @@ static void tcp_v6_reqsk_destructor(struct request_sock *req)
 	kfree_skb(inet_rsk(req)->pktopts);
 }
 
-#ifdef CONFIG_TCP_MD5SIG
-static struct tcp_md5sig_key *tcp_v6_md5_do_lookup(const struct sock *sk,
-						   const struct in6_addr *addr)
-{
-	return tcp_md5_do_lookup(sk, (union tcp_md5_addr *)addr, AF_INET6);
-}
-
-static struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
-						const struct sock *addr_sk)
-{
-	return tcp_v6_md5_do_lookup(sk, &addr_sk->sk_v6_daddr);
-}
-
-static int tcp_v6_parse_md5_keys(struct sock *sk, int optname,
-				 char __user *optval, int optlen)
-{
-	struct tcp_md5sig cmd;
-	struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)&cmd.tcpm_addr;
-	u8 prefixlen;
-
-	if (optlen < sizeof(cmd))
-		return -EINVAL;
-
-	if (copy_from_user(&cmd, optval, sizeof(cmd)))
-		return -EFAULT;
-
-	if (sin6->sin6_family != AF_INET6)
-		return -EINVAL;
-
-	if (optname == TCP_MD5SIG_EXT &&
-	    cmd.tcpm_flags & TCP_MD5SIG_FLAG_PREFIX) {
-		prefixlen = cmd.tcpm_prefixlen;
-		if (prefixlen > 128 || (ipv6_addr_v4mapped(&sin6->sin6_addr) &&
-					prefixlen > 32))
-			return -EINVAL;
-	} else {
-		prefixlen = ipv6_addr_v4mapped(&sin6->sin6_addr) ? 32 : 128;
-	}
-
-	if (!cmd.tcpm_keylen) {
-		if (ipv6_addr_v4mapped(&sin6->sin6_addr))
-			return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin6->sin6_addr.s6_addr32[3],
-					      AF_INET, prefixlen);
-		return tcp_md5_do_del(sk, (union tcp_md5_addr *)&sin6->sin6_addr,
-				      AF_INET6, prefixlen);
-	}
-
-	if (cmd.tcpm_keylen > TCP_MD5SIG_MAXKEYLEN)
-		return -EINVAL;
-
-	if (ipv6_addr_v4mapped(&sin6->sin6_addr))
-		return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin6->sin6_addr.s6_addr32[3],
-				      AF_INET, prefixlen, cmd.tcpm_key,
-				      cmd.tcpm_keylen, GFP_KERNEL);
-
-	return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin6->sin6_addr,
-			      AF_INET6, prefixlen, cmd.tcpm_key,
-			      cmd.tcpm_keylen, GFP_KERNEL);
-}
-
-static int tcp_v6_md5_hash_headers(struct tcp_md5sig_pool *hp,
-				   const struct in6_addr *daddr,
-				   const struct in6_addr *saddr,
-				   const struct tcphdr *th, int nbytes)
-{
-	struct tcp6_pseudohdr *bp;
-	struct scatterlist sg;
-	struct tcphdr *_th;
-
-	bp = hp->scratch;
-	/* 1. TCP pseudo-header (RFC2460) */
-	bp->saddr = *saddr;
-	bp->daddr = *daddr;
-	bp->protocol = cpu_to_be32(IPPROTO_TCP);
-	bp->len = cpu_to_be32(nbytes);
-
-	_th = (struct tcphdr *)(bp + 1);
-	memcpy(_th, th, sizeof(*th));
-	_th->check = 0;
-
-	sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th));
-	ahash_request_set_crypt(hp->md5_req, &sg, NULL,
-				sizeof(*bp) + sizeof(*th));
-	return crypto_ahash_update(hp->md5_req);
-}
-
-static int tcp_v6_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
-			       const struct in6_addr *daddr, struct in6_addr *saddr,
-			       const struct tcphdr *th)
-{
-	struct tcp_md5sig_pool *hp;
-	struct ahash_request *req;
-
-	hp = tcp_get_md5sig_pool();
-	if (!hp)
-		goto clear_hash_noput;
-	req = hp->md5_req;
-
-	if (crypto_ahash_init(req))
-		goto clear_hash;
-	if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2))
-		goto clear_hash;
-	if (tcp_md5_hash_key(hp, key))
-		goto clear_hash;
-	ahash_request_set_crypt(req, NULL, md5_hash, 0);
-	if (crypto_ahash_final(req))
-		goto clear_hash;
-
-	tcp_put_md5sig_pool();
-	return 0;
-
-clear_hash:
-	tcp_put_md5sig_pool();
-clear_hash_noput:
-	memset(md5_hash, 0, 16);
-	return 1;
-}
-
-static int tcp_v6_md5_hash_skb(char *md5_hash,
-			       const struct tcp_md5sig_key *key,
-			       const struct sock *sk,
-			       const struct sk_buff *skb)
-{
-	const struct in6_addr *saddr, *daddr;
-	struct tcp_md5sig_pool *hp;
-	struct ahash_request *req;
-	const struct tcphdr *th = tcp_hdr(skb);
-
-	if (sk) { /* valid for establish/request sockets */
-		saddr = &sk->sk_v6_rcv_saddr;
-		daddr = &sk->sk_v6_daddr;
-	} else {
-		const struct ipv6hdr *ip6h = ipv6_hdr(skb);
-		saddr = &ip6h->saddr;
-		daddr = &ip6h->daddr;
-	}
-
-	hp = tcp_get_md5sig_pool();
-	if (!hp)
-		goto clear_hash_noput;
-	req = hp->md5_req;
-
-	if (crypto_ahash_init(req))
-		goto clear_hash;
-
-	if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, skb->len))
-		goto clear_hash;
-	if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2))
-		goto clear_hash;
-	if (tcp_md5_hash_key(hp, key))
-		goto clear_hash;
-	ahash_request_set_crypt(req, NULL, md5_hash, 0);
-	if (crypto_ahash_final(req))
-		goto clear_hash;
-
-	tcp_put_md5sig_pool();
-	return 0;
-
-clear_hash:
-	tcp_put_md5sig_pool();
-clear_hash_noput:
-	memset(md5_hash, 0, 16);
-	return 1;
-}
-
-#endif
-
-static bool tcp_v6_inbound_md5_hash(const struct sock *sk,
-				    const struct sk_buff *skb)
-{
-#ifdef CONFIG_TCP_MD5SIG
-	const __u8 *hash_location = NULL;
-	struct tcp_md5sig_key *hash_expected;
-	const struct ipv6hdr *ip6h = ipv6_hdr(skb);
-	const struct tcphdr *th = tcp_hdr(skb);
-	int genhash;
-	u8 newhash[16];
-
-	hash_expected = tcp_v6_md5_do_lookup(sk, &ip6h->saddr);
-	hash_location = tcp_parse_md5sig_option(th);
-
-	/* We've parsed the options - do we have a hash? */
-	if (!hash_expected && !hash_location)
-		return false;
-
-	if (hash_expected && !hash_location) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5NOTFOUND);
-		return true;
-	}
-
-	if (!hash_expected && hash_location) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5UNEXPECTED);
-		return true;
-	}
-
-	/* check the signature */
-	genhash = tcp_v6_md5_hash_skb(newhash,
-				      hash_expected,
-				      NULL, skb);
-
-	if (genhash || memcmp(hash_location, newhash, 16) != 0) {
-		NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5FAILURE);
-		net_info_ratelimited("MD5 Hash %s for [%pI6c]:%u->[%pI6c]:%u\n",
-				     genhash ? "failed" : "mismatch",
-				     &ip6h->saddr, ntohs(th->source),
-				     &ip6h->daddr, ntohs(th->dest));
-		return true;
-	}
-#endif
-	return false;
-}
-
 static void tcp_v6_init_req(struct request_sock *req,
 			    const struct sock *sk_listener,
 			    struct sk_buff *skb)
@@ -791,60 +570,27 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 	unsigned int tot_len = sizeof(struct tcphdr);
 	struct dst_entry *dst;
 	__be32 *topt;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key *key = NULL;
-	const __u8 *hash_location = NULL;
-	struct ipv6hdr *ipv6h = ipv6_hdr(skb);
-#endif
 	struct tcp_out_options extraopts;
 
+	memset(&extraopts, 0, sizeof(extraopts));
+
 	if (tsecr)
 		tot_len += TCPOLEN_TSTAMP_ALIGNED;
 #ifdef CONFIG_TCP_MD5SIG
-	rcu_read_lock();
-	hash_location = tcp_parse_md5sig_option(th);
-	if (sk && sk_fullsock(sk)) {
-		key = tcp_v6_md5_do_lookup(sk, &ipv6h->saddr);
-	} else if (sk && sk->sk_state == TCP_TIME_WAIT) {
-		struct tcp_timewait_sock *tcptw = tcp_twsk(sk);
-
-		key = tcp_twsk_md5_key(tcptw);
-	} else if (sk && sk->sk_state == TCP_NEW_SYN_RECV) {
-		key = tcp_v6_md5_do_lookup(sk, &ipv6h->daddr);
-	} else if (hash_location) {
-		unsigned char newhash[16];
-		struct sock *sk1 = NULL;
-		int genhash;
-
-		/* active side is lost. Try to find listening socket through
-		 * source port, and then find md5 key through listening socket.
-		 * we are not loose security here:
-		 * Incoming packet is checked with md5 hash with finding key,
-		 * no RST generated if md5 hash doesn't match.
-		 */
-		sk1 = inet6_lookup_listener(dev_net(skb_dst(skb)->dev),
-					    &tcp_hashinfo, NULL, 0,
-					    &ipv6h->saddr,
-					    th->source, &ipv6h->daddr,
-					    ntohs(th->source), tcp_v6_iif(skb),
-					    tcp_v6_sdif(skb));
-		if (!sk1)
-			goto out;
+{
+	int ret;
 
-		key = tcp_v6_md5_do_lookup(sk1, &ipv6h->saddr);
-		if (!key)
-			goto out;
+	ret = tcp_v6_md5_send_response_prepare(skb, 0,
+					       MAX_TCP_OPTION_SPACE - tot_len,
+					       &extraopts, sk);
 
-		genhash = tcp_v6_md5_hash_skb(newhash, key, NULL, skb);
-		if (genhash || memcmp(hash_location, newhash, 16) != 0)
-			goto out;
-	}
+	if (ret == -1)
+		goto out;
 
-	if (key)
-		tot_len += TCPOLEN_MD5SIG_ALIGNED;
+	tot_len += ret;
+}
 #endif
 
-
 	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
 		unsigned int remaining = MAX_TCP_OPTION_SPACE - tot_len;
 		u8 extraflags = rst ? TCPHDR_RST : 0;
@@ -853,8 +599,6 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 		if (!rst || !th->ack)
 			extraflags |= TCPHDR_ACK;
 
-		memset(&extraopts, 0, sizeof(extraopts));
-
 		used = tcp_extra_options_response_prepare(skb, extraflags,
 							  remaining,
 							  &extraopts, sk);
@@ -895,13 +639,8 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 	}
 
 #ifdef CONFIG_TCP_MD5SIG
-	if (key) {
-		*topt++ = htonl((TCPOPT_NOP << 24) | (TCPOPT_NOP << 16) |
-				(TCPOPT_MD5SIG << 8) | TCPOLEN_MD5SIG);
-		tcp_v6_md5_hash_hdr((__u8 *)topt, key,
-				    &ipv6_hdr(skb)->saddr,
-				    &ipv6_hdr(skb)->daddr, t1);
-	}
+	if (extraopts.md5)
+		tcp_v6_md5_send_response_write(topt, skb, t1, &extraopts, sk);
 #endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
@@ -949,10 +688,6 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 

out:

	kfree_skb(buff);
net/ipv6/tcp_ipv6.c +0 -4
-
-#if CONFIG_TCP_MD5SIG
-	rcu_read_unlock();
-#endif

}

static void tcp_v6_send_reset(const struct sock *sk, struct sk_buff *skb)

net/ipv6/tcp_ipv6.c +1 -31
@@ -1078,9 +813,6 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
 	struct inet_sock *newinet;
 	struct tcp_sock *newtp;
 	struct sock *newsk;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key *key;
-#endif
 	struct flowi6 fl6;
 
 	if (skb->protocol == htons(ETH_P_IP)) {
@@ -1231,18 +963,7 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
 	}
 
 #ifdef CONFIG_TCP_MD5SIG
-	/* Copy over the MD5 key from the original socket */
-	key = tcp_v6_md5_do_lookup(sk, &newsk->sk_v6_daddr);
-	if (key) {
-		/* We're using one, so create a matching key
-		 * on the newsk structure. If we fail to get
-		 * memory, then we end up not copying the key
-		 * across. Shucks.
-		 */
-		tcp_md5_do_add(newsk, (union tcp_md5_addr *)&newsk->sk_v6_daddr,
-			       AF_INET6, 128, key->key, key->keylen,
-			       sk_gfp_mask(sk, GFP_ATOMIC));
-	}
+	tcp_v6_md5_syn_recv_sock(sk, newsk);
 #endif
 
 	if (__inet_inherit_port(sk, newsk) < 0) {
@@ -1698,14 +1419,6 @@ static const struct inet_connection_sock_af_ops ipv6_specific = {
 	.mtu_reduced	   = tcp_v6_mtu_reduced,
 };
 
-#ifdef CONFIG_TCP_MD5SIG
-static const struct tcp_sock_af_ops tcp_sock_ipv6_specific = {
-	.md5_lookup	=	tcp_v6_md5_lookup,
-	.calc_md5_hash	=	tcp_v6_md5_hash_skb,
-	.md5_parse	=	tcp_v6_parse_md5_keys,
-};
-#endif
-
 /*
 *	TCP over IPv4 via INET6 API
 */
@@ -1728,14 +1441,6 @@ static const struct inet_connection_sock_af_ops ipv6_mapped = {
 	.mtu_reduced	   = tcp_v4_mtu_reduced,
 };
 
-#ifdef CONFIG_TCP_MD5SIG
-static const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific = {
-	.md5_lookup	=	tcp_v4_md5_lookup,
-	.calc_md5_hash	=	tcp_v4_md5_hash_skb,
-	.md5_parse	=	tcp_v6_parse_md5_keys,
-};
-#endif
-
 /* NOTE: A lot of things set to zero explicitly by call to
 *       sk_alloc() so need not be done here.
 */
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#44 lore raw

PATCH v2 23/25 tcp_md5: Use tcp_extra_options in output path

In reply to Christoph Paasch

This patch starts making use of the extra_option framework for TCP_MD5.

One tricky part is that extra_options are called at the end of the option-functions while TCP_MD5 is called at the beginning.

TCP_MD5 is called at the beginning because it wants to disable TCP-timestamps (for option-space reasons). So, in the _prepare-function of the extra options we need to undo the work that was done when enabling TCP timestamps.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/linux/tcp.h      |   8 -
 include/linux/tcp_md5.h  |  65 ------
 net/ipv4/tcp_ipv4.c      |  52 -----
 net/ipv4/tcp_md5.c       | 500 ++++++++++++++++++++++++++++++++++-------------
 net/ipv4/tcp_minisocks.c |  11 --
 net/ipv4/tcp_output.c    |  71 +------
 net/ipv6/tcp_ipv6.c      |  23 ---
 7 files changed, 368 insertions(+), 362 deletions(-)
include/linux/tcp.h +0 -8
index c04b7e4f5309..c1b17b5ffba7 100644
@@ -127,8 +127,6 @@ struct tcp_out_options {
 	u16 mss;		/* 0 to disable */
 	u8 ws;			/* window scale, 0 to disable */
 	u8 num_sack_blocks;	/* number of SACK blocks to include */
-	u8 hash_size;		/* bytes in hash_location */
-	__u8 *hash_location;	/* temporary pointer, overloaded */
 	__u32 tsval, tsecr;	/* need to include OPTION_TS */
 	struct tcp_fastopen_cookie *fastopen_cookie;	/* Fast open cookie */
 	struct tcp_md5sig_key *md5; /* TCP_MD5 signature key */
@@ -379,9 +377,6 @@ struct tcp_sock {
 #ifdef CONFIG_TCP_MD5SIG
 /* TCP AF-Specific parts; only used by MD5 Signature support so far */
 	const struct tcp_sock_af_ops	*af_specific;
-
-/* TCP MD5 Signature Option information */
-	struct tcp_md5sig_info	__rcu *md5sig_info;
 #endif
 
 /* TCP fastopen related information */
@@ -439,9 +434,6 @@ struct tcp_timewait_sock {
 	long			  tw_ts_recent_stamp;
 
 	struct list_head	  tcp_option_list;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key	  *tw_md5_key;
-#endif
 };
 
 static inline struct tcp_timewait_sock *tcp_twsk(const struct sock *sk)
include/linux/tcp_md5.h +0 -65

86 lines not on this page · raw message

net/ipv4/tcp_ipv4.c +0 -52

99 lines not on this page · raw message

net/ipv4/tcp_md5.c +367 -133

665 lines not on this page · raw message

net/ipv4/tcp_minisocks.c +0 -11
index e9e925d51046..de29794ed50f 100644
@@ -22,7 +22,6 @@
 #include <linux/module.h>
 #include <linux/slab.h>
 #include <linux/sysctl.h>
-#include <linux/tcp_md5.h>
 #include <linux/workqueue.h>
 #include <linux/static_key.h>
 #include <net/tcp.h>
@@ -296,9 +295,6 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 					       &tcptw->tcp_option_list);
 			INIT_LIST_HEAD(&tcp_sk(sk)->tcp_option_list);
 		}
-#ifdef CONFIG_TCP_MD5SIG
-		tcp_md5_time_wait(sk, tw);
-#endif
 
 		/* Get the TIME_WAIT timeout firing. */
 		if (timeo < rto)
@@ -326,10 +322,6 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
 
 void tcp_twsk_destructor(struct sock *sk)
 {
-#ifdef CONFIG_TCP_MD5SIG
-	tcp_md5_twsk_destructor(sk);
-#endif
-
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
 		tcp_extra_options_destroy(sk);
 }
@@ -514,9 +506,6 @@ struct sock *tcp_create_openreq_child(const struct sock *sk,
 			newtp->tcp_header_len = sizeof(struct tcphdr);
 		}
 		newtp->tsoffset = treq->ts_off;
-#ifdef CONFIG_TCP_MD5SIG
-		tcp_md5_add_header_len(sk, newsk);
-#endif
 		if (static_branch_unlikely(&tcp_extra_options_enabled))
 			newtp->tcp_header_len += tcp_extra_options_add_header(sk, newsk);
 
net/ipv4/tcp_output.c +1 -70

131 lines not on this page · raw message

net/ipv6/tcp_ipv6.c +0 -23
index 0d3506309bd4..6db6db3d4f98 100644
@@ -576,20 +576,6 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 
 	if (tsecr)
 		tot_len += TCPOLEN_TSTAMP_ALIGNED;
-#ifdef CONFIG_TCP_MD5SIG
-{
-	int ret;
-
-	ret = tcp_v6_md5_send_response_prepare(skb, 0,
-					       MAX_TCP_OPTION_SPACE - tot_len,
-					       &extraopts, sk);
-
-	if (ret == -1)
-		goto out;
-
-	tot_len += ret;
-}
-#endif
 
 	if (static_branch_unlikely(&tcp_extra_options_enabled)) {
 		unsigned int remaining = MAX_TCP_OPTION_SPACE - tot_len;
@@ -638,11 +624,6 @@ static void tcp_v6_send_response(const struct sock *sk, struct sk_buff *skb, u32
 		*topt++ = htonl(tsecr);
 	}
 
-#ifdef CONFIG_TCP_MD5SIG
-	if (extraopts.md5)
-		tcp_v6_md5_send_response_write(topt, skb, t1, &extraopts, sk);
-#endif
-
 	if (static_branch_unlikely(&tcp_extra_options_enabled))
 		tcp_extra_options_response_write(topt, skb, t1, &extraopts, sk);
 
@@ -962,10 +943,6 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
 		INIT_LIST_HEAD(&tcp_rsk(req)->tcp_option_list);
 	}
 
-#ifdef CONFIG_TCP_MD5SIG
-	tcp_v6_md5_syn_recv_sock(sk, newsk);
-#endif
-
 	if (__inet_inherit_port(sk, newsk) < 0) {
 		inet_csk_prepare_forced_close(newsk);
 		tcp_done(newsk);
Signature · 1 line
2.15.0
author
#45 lore raw

PATCH v2 24/25 tcp_md5: Cleanup TCP-code

In reply to Christoph Paasch

Now that we have consolidated the TCP_MD5 output path, we can cleanup TCP and its callbacks to MD5.

These callbacks are solely there to handle the different address-familiese (v4, v6 and v4mapped).

Now that we have isolated the TCP_MD5-code it is acceptable to add a bit more complexity inside tcp_md5.c to handle these address-families at the benefit of getting rid of these callbacks in tcp_sock, together with its assignments in tcp_v4/6_connect,...

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/linux/tcp.h     |   5 -
 include/linux/tcp_md5.h |  18 +--
 include/net/tcp.h       |  24 ----
 net/ipv4/tcp.c          |   2 +-
 net/ipv4/tcp_ipv4.c     |   8 --
 net/ipv4/tcp_md5.c      | 340 +++++++++++++++++++++---------------------------
 net/ipv6/tcp_ipv6.c     |  17 ---
 7 files changed, 153 insertions(+), 261 deletions(-)
include/linux/tcp.h +0 -5
index c1b17b5ffba7..8cf4058963d5 100644
@@ -374,11 +374,6 @@ struct tcp_sock {
 			   * while socket was owned by user.
 			   */
 
-#ifdef CONFIG_TCP_MD5SIG
-/* TCP AF-Specific parts; only used by MD5 Signature support so far */
-	const struct tcp_sock_af_ops	*af_specific;
-#endif
-
 /* TCP fastopen related information */
 	struct tcp_fastopen_request *fastopen_req;
 	/* fastopen_rsk points to request_sock that resulted in this big
include/linux/tcp_md5.h +2 -16
index 1bf9f3d63eb8..377552b2396b 100644
@@ -25,28 +25,14 @@ struct tcp_md5sig_key {
 	struct rcu_head		rcu;
 };
 
-extern const struct tcp_sock_af_ops tcp_sock_ipv4_specific;
-extern const struct tcp_sock_af_ops tcp_sock_ipv6_specific;
-extern const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific;
-
 /* - functions */
-int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
-			const struct sock *sk, const struct sk_buff *skb);
 
-struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
-					 const struct sock *addr_sk);
+int tcp_md5_parse_keys(struct sock *sk, int optname, char __user *optval,
+		       int optlen);
 
 bool tcp_v4_inbound_md5_hash(const struct sock *sk,
 			     const struct sk_buff *skb);
 
-struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
-					 const struct sock *addr_sk);
-
-int tcp_v6_md5_hash_skb(char *md5_hash,
-			const struct tcp_md5sig_key *key,
-			const struct sock *sk,
-			const struct sk_buff *skb);
-
 bool tcp_v6_inbound_md5_hash(const struct sock *sk,
 			     const struct sk_buff *skb);
 
include/net/tcp.h +0 -24
index 058715ab3a45..db95126a6c50 100644
@@ -1731,32 +1731,8 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,
 		     const struct tcp_request_sock_ops *af_ops,
 		     struct sock *sk, struct sk_buff *skb);
 
-/* TCP af-specific functions */
-struct tcp_sock_af_ops {
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key	*(*md5_lookup) (const struct sock *sk,
-						const struct sock *addr_sk);
-	int		(*calc_md5_hash)(char *location,
-					 const struct tcp_md5sig_key *md5,
-					 const struct sock *sk,
-					 const struct sk_buff *skb);
-	int		(*md5_parse)(struct sock *sk,
-				     int optname,
-				     char __user *optval,
-				     int optlen);
-#endif
-};
-
 struct tcp_request_sock_ops {
 	u16 mss_clamp;
-#ifdef CONFIG_TCP_MD5SIG
-	struct tcp_md5sig_key *(*req_md5_lookup)(const struct sock *sk,
-						 const struct sock *addr_sk);
-	int		(*calc_md5_hash) (char *location,
-					  const struct tcp_md5sig_key *md5,
-					  const struct sock *sk,
-					  const struct sk_buff *skb);
-#endif
 	void (*init_req)(struct request_sock *req,
 			 const struct sock *sk_listener,
 			 struct sk_buff *skb);
net/ipv4/tcp.c +1 -1
index ca1ddb2c820c..78d809ec8615 100644
@@ -2800,7 +2800,7 @@ static int do_tcp_setsockopt(struct sock *sk, int level,
 	case TCP_MD5SIG:
 	case TCP_MD5SIG_EXT:
 		/* Read the IP->Key mappings from userspace */
-		err = tp->af_specific->md5_parse(sk, optname, optval, optlen);
+		tcp_md5_parse_keys(sk, optname, optval, optlen);
 		break;
 #endif
 	case TCP_USER_TIMEOUT:
net/ipv4/tcp_ipv4.c +0 -8
index 2f93bc3e7f94..6987003f2db7 100644
@@ -887,10 +887,6 @@ struct request_sock_ops tcp_request_sock_ops __read_mostly = {
 
 static const struct tcp_request_sock_ops tcp_request_sock_ipv4_ops = {
 	.mss_clamp	=	TCP_MSS_DEFAULT,
-#ifdef CONFIG_TCP_MD5SIG
-	.req_md5_lookup	=	tcp_v4_md5_lookup,
-	.calc_md5_hash	=	tcp_v4_md5_hash_skb,
-#endif
 	.init_req	=	tcp_v4_init_req,
 #ifdef CONFIG_SYN_COOKIES
 	.cookie_init_seq =	cookie_v4_init_sequence,
@@ -1428,10 +1424,6 @@ static int tcp_v4_init_sock(struct sock *sk)
 
 	icsk->icsk_af_ops = &ipv4_specific;
 
-#ifdef CONFIG_TCP_MD5SIG
-	tcp_sk(sk)->af_specific = &tcp_sock_ipv4_specific;
-#endif
-
 	return 0;
 }
 
net/ipv4/tcp_md5.c +150 -190

419 lines not on this page · raw message

net/ipv6/tcp_ipv6.c +0 -17
index 6db6db3d4f98..6e5bd3c51b4b 100644
@@ -208,9 +208,6 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr,
 
 		icsk->icsk_af_ops = &ipv6_mapped;
 		sk->sk_backlog_rcv = tcp_v4_do_rcv;
-#ifdef CONFIG_TCP_MD5SIG
-		tp->af_specific = &tcp_sock_ipv6_mapped_specific;
-#endif
 
 		err = tcp_v4_connect(sk, (struct sockaddr *)&sin, sizeof(sin));
 
@@ -218,9 +215,6 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr,
 			icsk->icsk_ext_hdr_len = exthdrlen;
 			icsk->icsk_af_ops = &ipv6_specific;
 			sk->sk_backlog_rcv = tcp_v6_do_rcv;
-#ifdef CONFIG_TCP_MD5SIG
-			tp->af_specific = &tcp_sock_ipv6_specific;
-#endif
 			goto failure;
 		}
 		np->saddr = sk->sk_v6_rcv_saddr;
@@ -543,10 +537,6 @@ struct request_sock_ops tcp6_request_sock_ops __read_mostly = {
 static const struct tcp_request_sock_ops tcp_request_sock_ipv6_ops = {
 	.mss_clamp	=	IPV6_MIN_MTU - sizeof(struct tcphdr) -
 				sizeof(struct ipv6hdr),
-#ifdef CONFIG_TCP_MD5SIG
-	.req_md5_lookup	=	tcp_v6_md5_lookup,
-	.calc_md5_hash	=	tcp_v6_md5_hash_skb,
-#endif
 	.init_req	=	tcp_v6_init_req,
 #ifdef CONFIG_SYN_COOKIES
 	.cookie_init_seq =	cookie_v6_init_sequence,
@@ -820,9 +810,6 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
 
 		inet_csk(newsk)->icsk_af_ops = &ipv6_mapped;
 		newsk->sk_backlog_rcv = tcp_v4_do_rcv;
-#ifdef CONFIG_TCP_MD5SIG
-		newtp->af_specific = &tcp_sock_ipv6_mapped_specific;
-#endif
 
 		newnp->ipv6_mc_list = NULL;
 		newnp->ipv6_ac_list = NULL;
@@ -1429,10 +1416,6 @@ static int tcp_v6_init_sock(struct sock *sk)
 
 	icsk->icsk_af_ops = &ipv6_specific;
 
-#ifdef CONFIG_TCP_MD5SIG
-	tcp_sk(sk)->af_specific = &tcp_sock_ipv6_specific;
-#endif
-
 	return 0;
 }
 
Signature · 1 line
2.15.0
author
#46 lore raw

PATCH v2 25/25 tcp_md5: Use TCP extra-options on the input path

In reply to Christoph Paasch

The checks are now being done through the extra-option framework. For TCP MD5 this means that the check happens a bit later than usual.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>
---
 include/linux/tcp_md5.h | 23 +----------------------
 net/ipv4/tcp_ipv4.c     |  9 ---------
 net/ipv4/tcp_md5.c      | 29 ++++++++++++++++++++++++-----
 net/ipv6/tcp_ipv6.c     |  9 ---------
 4 files changed, 25 insertions(+), 45 deletions(-)
include/linux/tcp_md5.h +1 -22
index 377552b2396b..b28534831483 100644
@@ -30,30 +30,9 @@ struct tcp_md5sig_key {
 int tcp_md5_parse_keys(struct sock *sk, int optname, char __user *optval,
 		       int optlen);
 
-bool tcp_v4_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb);
-
-bool tcp_v6_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb);
-
 int tcp_md5_diag_get_aux(struct sock *sk, bool net_admin, struct sk_buff *skb);
 
 int tcp_md5_diag_get_aux_size(struct sock *sk, bool net_admin);
 
-#else
-
-static inline bool tcp_v4_inbound_md5_hash(const struct sock *sk,
-					   const struct sk_buff *skb)
-{
-	return false;
-}
-
-static inline bool tcp_v6_inbound_md5_hash(const struct sock *sk,
-					   const struct sk_buff *skb)
-{
-	return false;
-}
-
-#endif
-
+#endif /* CONFIG_TCP_MD5SIG */
 #endif /* _LINUX_TCP_MD5_H */
net/ipv4/tcp_ipv4.c +0 -9
index 6987003f2db7..c578eb16b388 100644
@@ -62,7 +62,6 @@
 #include <linux/init.h>
 #include <linux/times.h>
 #include <linux/slab.h>
-#include <linux/tcp_md5.h>
 
 #include <net/net_namespace.h>
 #include <net/icmp.h>
@@ -1238,11 +1237,6 @@ int tcp_v4_rcv(struct sk_buff *skb)
 		struct sock *nsk;
 
 		sk = req->rsk_listener;
-		if (unlikely(tcp_v4_inbound_md5_hash(sk, skb))) {
-			sk_drops_add(sk, skb);
-			reqsk_put(req);
-			goto discard_it;
-		}
 		if (unlikely(sk->sk_state != TCP_LISTEN)) {
 			inet_csk_reqsk_queue_drop_and_put(sk, req);
 			goto lookup;
@@ -1277,9 +1271,6 @@ int tcp_v4_rcv(struct sk_buff *skb)
 	if (!xfrm4_policy_check(sk, XFRM_POLICY_IN, skb))
 		goto discard_and_relse;
 
-	if (tcp_v4_inbound_md5_hash(sk, skb))
-		goto discard_and_relse;
-
 	nf_reset(skb);
 
 	if (tcp_filter(sk, skb))
net/ipv4/tcp_md5.c +24 -5
index f25504c822e7..c9400b7b96ea 100644
@@ -29,6 +29,10 @@ static DEFINE_PER_CPU(struct tcp_md5sig_pool, tcp_md5sig_pool);
 static DEFINE_MUTEX(tcp_md5sig_mutex);
 static bool tcp_md5sig_pool_populated;
 
+static bool tcp_inbound_md5_hash(struct sock *sk, const struct sk_buff *skb,
+				 struct tcp_options_received *opt_rx,
+				 struct tcp_extra_option_store *store);
+
 static unsigned int tcp_md5_extra_option_prepare(struct sk_buff *skb, u8 flags,
 						 unsigned int remaining,
 						 struct tcp_out_options *opts,
@@ -71,6 +75,7 @@ struct tcp_md5_extra_option {
 
 static const struct tcp_extra_option_ops tcp_md5_extra_ops = {
 	.option_kind		= TCPOPT_MD5SIG,
+	.check			= tcp_inbound_md5_hash,
 	.prepare		= tcp_md5_extra_option_prepare,
 	.write			= tcp_md5_extra_option_write,
 	.response_prepare	= tcp_md5_send_response_prepare,
@@ -902,8 +907,8 @@ static struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
 }
 
 /* Called with rcu_read_lock() */
-bool tcp_v4_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb)
+static bool tcp_v4_inbound_md5_hash(const struct sock *sk,
+				    const struct sk_buff *skb)
 {
 	/* This gets called for each TCP segment that arrives
 	 * so we want to be efficient.
@@ -957,8 +962,8 @@ bool tcp_v4_inbound_md5_hash(const struct sock *sk,
 }
 
 #if IS_ENABLED(CONFIG_IPV6)
-bool tcp_v6_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb)
+static bool tcp_v6_inbound_md5_hash(const struct sock *sk,
+				    const struct sk_buff *skb)
 {
 	const __u8 *hash_location = NULL;
 	struct tcp_md5sig_key *hash_expected;
@@ -1000,7 +1005,6 @@ bool tcp_v6_inbound_md5_hash(const struct sock *sk,
 
 	return false;
 }
-EXPORT_SYMBOL_GPL(tcp_v6_inbound_md5_hash);
 
 static struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
 						const struct sock *addr_sk)
@@ -1010,6 +1014,21 @@ static struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
 EXPORT_SYMBOL_GPL(tcp_v6_md5_lookup);
 #endif
 
+static bool tcp_inbound_md5_hash(struct sock *sk, const struct sk_buff *skb,
+				 struct tcp_options_received *opt_rx,
+				 struct tcp_extra_option_store *store)
+{
+	if (skb->protocol == htons(ETH_P_IP)) {
+		return tcp_v4_inbound_md5_hash(sk, skb);
+#if IS_ENABLED(CONFIG_IPV6)
+	} else {
+		return tcp_v6_inbound_md5_hash(sk, skb);
+#endif
+	}
+
+	return false;
+}
+
 static void tcp_diag_md5sig_fill(struct tcp_diag_md5sig *info,
 				 const struct tcp_md5sig_key *key)
 {
net/ipv6/tcp_ipv6.c +0 -9
index 6e5bd3c51b4b..367db77b8bf2 100644
@@ -43,7 +43,6 @@
 #include <linux/ipv6.h>
 #include <linux/icmpv6.h>
 #include <linux/random.h>
-#include <linux/tcp_md5.h>
 
 #include <net/tcp.h>
 #include <net/ndisc.h>
@@ -1174,11 +1173,6 @@ static int tcp_v6_rcv(struct sk_buff *skb)
 
 		sk = req->rsk_listener;
 		tcp_v6_fill_cb(skb, hdr, th);
-		if (tcp_v6_inbound_md5_hash(sk, skb)) {
-			sk_drops_add(sk, skb);
-			reqsk_put(req);
-			goto discard_it;
-		}
 		if (unlikely(sk->sk_state != TCP_LISTEN)) {
 			inet_csk_reqsk_queue_drop_and_put(sk, req);
 			goto lookup;
@@ -1213,9 +1207,6 @@ static int tcp_v6_rcv(struct sk_buff *skb)
 
 	tcp_v6_fill_cb(skb, hdr, th);
 
-	if (tcp_v6_inbound_md5_hash(sk, skb))
-		goto discard_and_relse;
-
 	if (tcp_filter(sk, skb))
 		goto discard_and_relse;
 	th = (const struct tcphdr *)skb->data;
Signature · 1 line
2.15.0
#47 lore raw

Re: [MPTCP] [PATCH v2 25/25] tcp_md5: Use TCP extra-options on the input path

Quoted 10 lines · Christoph Paasch

The checks are now being done through the extra-option framework. For TCP MD5 this means that the check happens a bit later than usual.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---

include/linux/tcp_md5.h | 23 +----------------------
net/ipv4/tcp_ipv4.c     |  9 ---------
net/ipv4/tcp_md5.c      | 29 ++++++++++++++++++++++++-----
net/ipv6/tcp_ipv6.c     |  9 ---------
4 files changed, 25 insertions(+), 45 deletions(-)

There's a lingering #ifdef CONFIG_TCP_MD5SIG in tcp_input.c as well, should that go away too?

Mat

Quoted 184 lines
include/linux/tcp_md5.h +1 -22
index 377552b2396b..b28534831483 100644
@@ -30,30 +30,9 @@ struct tcp_md5sig_key {
 int tcp_md5_parse_keys(struct sock *sk, int optname, char __user *optval,
 		       int optlen);
 
-bool tcp_v4_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb);
-
-bool tcp_v6_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb);
-
 int tcp_md5_diag_get_aux(struct sock *sk, bool net_admin, struct sk_buff *skb);
 
 int tcp_md5_diag_get_aux_size(struct sock *sk, bool net_admin);
 
-#else
-
-static inline bool tcp_v4_inbound_md5_hash(const struct sock *sk,
-					   const struct sk_buff *skb)
-{
-	return false;
-}
-
-static inline bool tcp_v6_inbound_md5_hash(const struct sock *sk,
-					   const struct sk_buff *skb)
-{
-	return false;
-}
-
-#endif
-
+#endif /* CONFIG_TCP_MD5SIG */
 #endif /* _LINUX_TCP_MD5_H */
net/ipv4/tcp_ipv4.c +0 -9
index 6987003f2db7..c578eb16b388 100644
@@ -62,7 +62,6 @@
 #include <linux/init.h>
 #include <linux/times.h>
 #include <linux/slab.h>
-#include <linux/tcp_md5.h>
 
 #include <net/net_namespace.h>
 #include <net/icmp.h>
@@ -1238,11 +1237,6 @@ int tcp_v4_rcv(struct sk_buff *skb)
 		struct sock *nsk;
 
 		sk = req->rsk_listener;
-		if (unlikely(tcp_v4_inbound_md5_hash(sk, skb))) {
-			sk_drops_add(sk, skb);
-			reqsk_put(req);
-			goto discard_it;
-		}
 		if (unlikely(sk->sk_state != TCP_LISTEN)) {
 			inet_csk_reqsk_queue_drop_and_put(sk, req);
 			goto lookup;
@@ -1277,9 +1271,6 @@ int tcp_v4_rcv(struct sk_buff *skb)
 	if (!xfrm4_policy_check(sk, XFRM_POLICY_IN, skb))
 		goto discard_and_relse;
 
-	if (tcp_v4_inbound_md5_hash(sk, skb))
-		goto discard_and_relse;
-
 	nf_reset(skb);
 
 	if (tcp_filter(sk, skb))
net/ipv4/tcp_md5.c +24 -5
index f25504c822e7..c9400b7b96ea 100644
@@ -29,6 +29,10 @@ static DEFINE_PER_CPU(struct tcp_md5sig_pool, tcp_md5sig_pool);
 static DEFINE_MUTEX(tcp_md5sig_mutex);
 static bool tcp_md5sig_pool_populated;
 
+static bool tcp_inbound_md5_hash(struct sock *sk, const struct sk_buff *skb,
+				 struct tcp_options_received *opt_rx,
+				 struct tcp_extra_option_store *store);
+
 static unsigned int tcp_md5_extra_option_prepare(struct sk_buff *skb, u8 flags,
 						 unsigned int remaining,
 						 struct tcp_out_options *opts,
@@ -71,6 +75,7 @@ struct tcp_md5_extra_option {
 
 static const struct tcp_extra_option_ops tcp_md5_extra_ops = {
 	.option_kind		= TCPOPT_MD5SIG,
+	.check			= tcp_inbound_md5_hash,
 	.prepare		= tcp_md5_extra_option_prepare,
 	.write			= tcp_md5_extra_option_write,
 	.response_prepare	= tcp_md5_send_response_prepare,
@@ -902,8 +907,8 @@ static struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
 }
 
 /* Called with rcu_read_lock() */
-bool tcp_v4_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb)
+static bool tcp_v4_inbound_md5_hash(const struct sock *sk,
+				    const struct sk_buff *skb)
 {
 	/* This gets called for each TCP segment that arrives
 	 * so we want to be efficient.
@@ -957,8 +962,8 @@ bool tcp_v4_inbound_md5_hash(const struct sock *sk,
 }
 
 #if IS_ENABLED(CONFIG_IPV6)
-bool tcp_v6_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb)
+static bool tcp_v6_inbound_md5_hash(const struct sock *sk,
+				    const struct sk_buff *skb)
 {
 	const __u8 *hash_location = NULL;
 	struct tcp_md5sig_key *hash_expected;
@@ -1000,7 +1005,6 @@ bool tcp_v6_inbound_md5_hash(const struct sock *sk,
 
 	return false;
 }
-EXPORT_SYMBOL_GPL(tcp_v6_inbound_md5_hash);
 
 static struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
 						const struct sock *addr_sk)
@@ -1010,6 +1014,21 @@ static struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
 EXPORT_SYMBOL_GPL(tcp_v6_md5_lookup);
 #endif
 
+static bool tcp_inbound_md5_hash(struct sock *sk, const struct sk_buff *skb,
+				 struct tcp_options_received *opt_rx,
+				 struct tcp_extra_option_store *store)
+{
+	if (skb->protocol == htons(ETH_P_IP)) {
+		return tcp_v4_inbound_md5_hash(sk, skb);
+#if IS_ENABLED(CONFIG_IPV6)
+	} else {
+		return tcp_v6_inbound_md5_hash(sk, skb);
+#endif
+	}
+
+	return false;
+}
+
 static void tcp_diag_md5sig_fill(struct tcp_diag_md5sig *info,
 				 const struct tcp_md5sig_key *key)
 {
net/ipv6/tcp_ipv6.c +0 -9
index 6e5bd3c51b4b..367db77b8bf2 100644
@@ -43,7 +43,6 @@
 #include <linux/ipv6.h>
 #include <linux/icmpv6.h>
 #include <linux/random.h>
-#include <linux/tcp_md5.h>
 
 #include <net/tcp.h>
 #include <net/ndisc.h>
@@ -1174,11 +1173,6 @@ static int tcp_v6_rcv(struct sk_buff *skb)
 
 		sk = req->rsk_listener;
 		tcp_v6_fill_cb(skb, hdr, th);
-		if (tcp_v6_inbound_md5_hash(sk, skb)) {
-			sk_drops_add(sk, skb);
-			reqsk_put(req);
-			goto discard_it;
-		}
 		if (unlikely(sk->sk_state != TCP_LISTEN)) {
 			inet_csk_reqsk_queue_drop_and_put(sk, req);
 			goto lookup;
@@ -1213,9 +1207,6 @@ static int tcp_v6_rcv(struct sk_buff *skb)
 
 	tcp_v6_fill_cb(skb, hdr, th);
 
-	if (tcp_v6_inbound_md5_hash(sk, skb))
-		goto discard_and_relse;
-
 	if (tcp_filter(sk, skb))
 		goto discard_and_relse;
 	th = (const struct tcphdr *)skb->data;
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#48 lore raw

Re: [MPTCP] [PATCH v2 25/25] tcp_md5: Use TCP extra-options on the input path

Quoted 14 lines · Mat Martineau
Quoted 10 lines · Christoph Paasch

The checks are now being done through the extra-option framework. For TCP MD5 this means that the check happens a bit later than usual.

Signed-off-by Christoph Paasch <cpaasch(a)apple.com>

---

include/linux/tcp_md5.h | 23 +----------------------
net/ipv4/tcp_ipv4.c     |  9 ---------
net/ipv4/tcp_md5.c      | 29 ++++++++++++++++++++++++-----
net/ipv6/tcp_ipv6.c     |  9 ---------
4 files changed, 25 insertions(+), 45 deletions(-)
Quoted 2 lines

There's a lingering #ifdef CONFIG_TCP_MD5SIG in tcp_input.c as well, should that go away too?

Damn, nice catch :)

As you see, I hadn't tested yet ;-)

Christoph

Quoted 192 lines

Mat

Quoted 184 lines
include/linux/tcp_md5.h +1 -22
index 377552b2396b..b28534831483 100644
@@ -30,30 +30,9 @@ struct tcp_md5sig_key {
 int tcp_md5_parse_keys(struct sock *sk, int optname, char __user *optval,
 		       int optlen);
 
-bool tcp_v4_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb);
-
-bool tcp_v6_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb);
-
 int tcp_md5_diag_get_aux(struct sock *sk, bool net_admin, struct sk_buff *skb);
 
 int tcp_md5_diag_get_aux_size(struct sock *sk, bool net_admin);
 
-#else
-
-static inline bool tcp_v4_inbound_md5_hash(const struct sock *sk,
-					   const struct sk_buff *skb)
-{
-	return false;
-}
-
-static inline bool tcp_v6_inbound_md5_hash(const struct sock *sk,
-					   const struct sk_buff *skb)
-{
-	return false;
-}
-
-#endif
-
+#endif /* CONFIG_TCP_MD5SIG */
 #endif /* _LINUX_TCP_MD5_H */
net/ipv4/tcp_ipv4.c +0 -9
index 6987003f2db7..c578eb16b388 100644
@@ -62,7 +62,6 @@
 #include <linux/init.h>
 #include <linux/times.h>
 #include <linux/slab.h>
-#include <linux/tcp_md5.h>
 
 #include <net/net_namespace.h>
 #include <net/icmp.h>
@@ -1238,11 +1237,6 @@ int tcp_v4_rcv(struct sk_buff *skb)
 		struct sock *nsk;
 
 		sk = req->rsk_listener;
-		if (unlikely(tcp_v4_inbound_md5_hash(sk, skb))) {
-			sk_drops_add(sk, skb);
-			reqsk_put(req);
-			goto discard_it;
-		}
 		if (unlikely(sk->sk_state != TCP_LISTEN)) {
 			inet_csk_reqsk_queue_drop_and_put(sk, req);
 			goto lookup;
@@ -1277,9 +1271,6 @@ int tcp_v4_rcv(struct sk_buff *skb)
 	if (!xfrm4_policy_check(sk, XFRM_POLICY_IN, skb))
 		goto discard_and_relse;
 
-	if (tcp_v4_inbound_md5_hash(sk, skb))
-		goto discard_and_relse;
-
 	nf_reset(skb);
 
 	if (tcp_filter(sk, skb))
net/ipv4/tcp_md5.c +24 -5
index f25504c822e7..c9400b7b96ea 100644
@@ -29,6 +29,10 @@ static DEFINE_PER_CPU(struct tcp_md5sig_pool, tcp_md5sig_pool);
 static DEFINE_MUTEX(tcp_md5sig_mutex);
 static bool tcp_md5sig_pool_populated;
 
+static bool tcp_inbound_md5_hash(struct sock *sk, const struct sk_buff *skb,
+				 struct tcp_options_received *opt_rx,
+				 struct tcp_extra_option_store *store);
+
 static unsigned int tcp_md5_extra_option_prepare(struct sk_buff *skb, u8 flags,
 						 unsigned int remaining,
 						 struct tcp_out_options *opts,
@@ -71,6 +75,7 @@ struct tcp_md5_extra_option {
 
 static const struct tcp_extra_option_ops tcp_md5_extra_ops = {
 	.option_kind		= TCPOPT_MD5SIG,
+	.check			= tcp_inbound_md5_hash,
 	.prepare		= tcp_md5_extra_option_prepare,
 	.write			= tcp_md5_extra_option_write,
 	.response_prepare	= tcp_md5_send_response_prepare,
@@ -902,8 +907,8 @@ static struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
 }
 
 /* Called with rcu_read_lock() */
-bool tcp_v4_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb)
+static bool tcp_v4_inbound_md5_hash(const struct sock *sk,
+				    const struct sk_buff *skb)
 {
 	/* This gets called for each TCP segment that arrives
 	 * so we want to be efficient.
@@ -957,8 +962,8 @@ bool tcp_v4_inbound_md5_hash(const struct sock *sk,
 }
 
 #if IS_ENABLED(CONFIG_IPV6)
-bool tcp_v6_inbound_md5_hash(const struct sock *sk,
-			     const struct sk_buff *skb)
+static bool tcp_v6_inbound_md5_hash(const struct sock *sk,
+				    const struct sk_buff *skb)
 {
 	const __u8 *hash_location = NULL;
 	struct tcp_md5sig_key *hash_expected;
@@ -1000,7 +1005,6 @@ bool tcp_v6_inbound_md5_hash(const struct sock *sk,
 
 	return false;
 }
-EXPORT_SYMBOL_GPL(tcp_v6_inbound_md5_hash);
 
 static struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
 						const struct sock *addr_sk)
@@ -1010,6 +1014,21 @@ static struct tcp_md5sig_key *tcp_v6_md5_lookup(const struct sock *sk,
 EXPORT_SYMBOL_GPL(tcp_v6_md5_lookup);
 #endif
 
+static bool tcp_inbound_md5_hash(struct sock *sk, const struct sk_buff *skb,
+				 struct tcp_options_received *opt_rx,
+				 struct tcp_extra_option_store *store)
+{
+	if (skb->protocol == htons(ETH_P_IP)) {
+		return tcp_v4_inbound_md5_hash(sk, skb);
+#if IS_ENABLED(CONFIG_IPV6)
+	} else {
+		return tcp_v6_inbound_md5_hash(sk, skb);
+#endif
+	}
+
+	return false;
+}
+
 static void tcp_diag_md5sig_fill(struct tcp_diag_md5sig *info,
 				 const struct tcp_md5sig_key *key)
 {
net/ipv6/tcp_ipv6.c +0 -9
index 6e5bd3c51b4b..367db77b8bf2 100644
@@ -43,7 +43,6 @@
 #include <linux/ipv6.h>
 #include <linux/icmpv6.h>
 #include <linux/random.h>
-#include <linux/tcp_md5.h>
 
 #include <net/tcp.h>
 #include <net/ndisc.h>
@@ -1174,11 +1173,6 @@ static int tcp_v6_rcv(struct sk_buff *skb)
 
 		sk = req->rsk_listener;
 		tcp_v6_fill_cb(skb, hdr, th);
-		if (tcp_v6_inbound_md5_hash(sk, skb)) {
-			sk_drops_add(sk, skb);
-			reqsk_put(req);
-			goto discard_it;
-		}
 		if (unlikely(sk->sk_state != TCP_LISTEN)) {
 			inet_csk_reqsk_queue_drop_and_put(sk, req);
 			goto lookup;
@@ -1213,9 +1207,6 @@ static int tcp_v6_rcv(struct sk_buff *skb)
 
 	tcp_v6_fill_cb(skb, hdr, th);
 
-	if (tcp_v6_inbound_md5_hash(sk, skb))
-		goto discard_and_relse;
-
 	if (tcp_filter(sk, skb))
 		goto discard_and_relse;
 	th = (const struct tcphdr *)skb->data;
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#49 lore raw
In reply to Christoph Paasch
Quoted 25 lines · Christoph Paasch

Finally, the second version of this patch-set.

At netdev I talked about this with Eric, and he is looking forward to see TCP-MD5 get out of the hot TCP data-path. He was planning to avoid TCP-MD5 with static jump-labels and also move TCP-MD5 in a separate file.

There are quite a few changes in this new set. I extended much more the extra-option framework. Making the list on a per-socket basis in the request-socket, tcp-socket and time-wait-socket. This eliminates the need for RCU-locking. See patch "tcp_extra_options: Make extra-option list per-socket"

I also added callbacks for response_write/prepare that can be called from the tcp_v4_send_ack/rst, and v6 counterparts. This is needed because in these code-paths the skb and sk have entirely different semantics.

Finally, in the input-path I added a check-callback and a post-process callback.

This is needed because after parsing the option in tcp_parse_options() we also need to act on them. The actions can be either before any other checks (e.g., as is the case for TCP_MD5, because we want to drop segments) or after the validity checks (e.g., as is needed for SMC).

And I also opted SMC into the new framework.

Quoted 7 lines

And, I made the TCP-MD5 input code-path use the framework.

There are still a few things that I need to double-check, notably sk_nocaps_add() in TCP_MD5.

And, I need to test ;-)

Oh, and of course, I still need to cleanup a bit. With proper commit-messages,... Also want to see if I should split up some of the patches a bit more.

Christoph

Quoted 71 lines

But I would like to have some early feedback already - especially on the patches prefixed with "tcp_extra_options". If they look good, I can merge them into Mat's patch, which will simplify my patch-juggling.

I want to get this patch-set out to netdev as an RFC as quickly as possible, so that we can make some progress.

Christoph Paasch (24):
tcp: Write options after the header has been fully done
tcp: Pass sock to tcp_options_write instead of tcp_sock
tcp: Pass skb in tcp_options_write
tcp: Allow tcp_fast_parse_options to drop segments
tcp: Make smc_parse_options return 1 on success
tcp_md5: Don't pass along md5-key
tcp_md5: Detect key inside tcp_v4_send_ack instead of passing it as an
argument
tcp_md5: Detect key inside tcp_v6_send_response instead of passing it
as an argument
tcp_extra_options: Extend tcp_extra_options_write
tcp_extra_options: Make extra-option list per-socket
tcp_extra_options: Export module fixes
tcp_extra_options: Add response_prepare callback
tcp_extra_options: Check static branch before _parse
tcp_extra_options: Skip fast-path when extra-options are present
tcp_extra_options: Pass sk to tcp_parse_options instead of tp
tcp_extra_options: Add check-callback right after parsing the option
tcp_extra_options: Allow to parse experimental options
tcp_extra_options: Add post_process callback
tcp_smc: Make SMC use TCP extra-option framework
tcp_md5: Check for TCP_MD5 after TCP Timestamps in
tcp_established_options
tcp_md5: Move TCP-MD5 code out of TCP itself
tcp_md5: Use tcp_extra_options in output path
tcp_md5: Cleanup TCP-code
tcp_md5: Use TCP extra-options on the input path

Mat Martineau (1):
tcp: Register handlers for extra TCP options

 drivers/infiniband/hw/cxgb4/cm.c |    2 +-
 include/linux/inet_diag.h        |    1 +
 include/linux/tcp.h              |   41 +-
 include/linux/tcp_md5.h          |   38 ++
 include/net/inet_sock.h          |    3 +-
 include/net/tcp.h                |  216 ++++---
 net/ipv4/Makefile                |    1 +
 net/ipv4/syncookies.c            |    6 +-
 net/ipv4/tcp.c                   |  372 ++++++++---
 net/ipv4/tcp_diag.c              |   81 +--
 net/ipv4/tcp_input.c             |  134 ++--
 net/ipv4/tcp_ipv4.c              |  553 ++--------------
 net/ipv4/tcp_md5.c               | 1317 ++++++++++++++++++++++++++++++++++++++
 net/ipv4/tcp_minisocks.c         |   76 +--
 net/ipv4/tcp_output.c            |  183 +-----
 net/ipv6/syncookies.c            |    6 +-
 net/ipv6/tcp_ipv6.c              |  386 ++---------
 net/smc/af_smc.c                 |  170 ++++-
 18 files changed, 2148 insertions(+), 1438 deletions(-)
 create mode 100644 include/linux/tcp_md5.h
 create mode 100644 net/ipv4/tcp_md5.c
Signature · 6 lines
2.15.0

_______________________________________________
mptcp mailing list
mptcp(a)lists.01.org
https://lists.01.org/mailman/listinfo/mptcp
#50 lore raw
In reply to Christoph Paasch

Christoph -

Quoted 5 lines · Christoph Paasch

Finally, the second version of this patch-set.

At netdev I talked about this with Eric, and he is looking forward to see TCP-MD5 get out of the hot TCP data-path. He was planning to avoid TCP-MD5 with static jump-labels and also move TCP-MD5 in a separate file.

Good to hear that he's interested.

Quoted 13 lines

There are quite a few changes in this new set. I extended much more the extra-option framework. Making the list on a per-socket basis in the request-socket, tcp-socket and time-wait-socket. This eliminates the need for RCU-locking. See patch "tcp_extra_options: Make extra-option list per-socket"

I also added callbacks for response_write/prepare that can be called from the tcp_v4_send_ack/rst, and v6 counterparts. This is needed because in these code-paths the skb and sk have entirely different semantics.

Finally, in the input-path I added a check-callback and a post-process callback.

Quoted 8 lines

This is needed because after parsing the option in tcp_parse_options() we also need to act on them. The actions can be either before any other checks (e.g., as is the case for TCP_MD5, because we want to drop segments) or after the validity checks (e.g., as is needed for SMC).

And I also opted SMC into the new framework.

And, I made the TCP-MD5 input code-path use the framework.

All of these seem like the right way to go. Nice to have two users of the framework already.

Quoted 6 lines

There are still a few things that I need to double-check, notably sk_nocaps_add() in TCP_MD5.

And, I need to test ;-)

Yeah, that testing :)

Quoted 4 lines

But I would like to have some early feedback already - especially on the patches prefixed with "tcp_extra_options". If they look good, I can merge them into Mat's patch, which will simplify my patch-juggling.

I have a few comments, but no major structural changes. I'll reply to the individual patches where needed.

Quoted 2 lines

I want to get this patch-set out to netdev as an RFC as quickly as possible, so that we can make some progress.

Agreed!

Mat

Quoted 53 lines

Christoph Paasch (24):
tcp: Write options after the header has been fully done
tcp: Pass sock to tcp_options_write instead of tcp_sock
tcp: Pass skb in tcp_options_write
tcp: Allow tcp_fast_parse_options to drop segments
tcp: Make smc_parse_options return 1 on success
tcp_md5: Don't pass along md5-key
tcp_md5: Detect key inside tcp_v4_send_ack instead of passing it as an
argument
tcp_md5: Detect key inside tcp_v6_send_response instead of passing it
as an argument
tcp_extra_options: Extend tcp_extra_options_write
tcp_extra_options: Make extra-option list per-socket
tcp_extra_options: Export module fixes
tcp_extra_options: Add response_prepare callback
tcp_extra_options: Check static branch before _parse
tcp_extra_options: Skip fast-path when extra-options are present
tcp_extra_options: Pass sk to tcp_parse_options instead of tp
tcp_extra_options: Add check-callback right after parsing the option
tcp_extra_options: Allow to parse experimental options
tcp_extra_options: Add post_process callback
tcp_smc: Make SMC use TCP extra-option framework
tcp_md5: Check for TCP_MD5 after TCP Timestamps in
tcp_established_options
tcp_md5: Move TCP-MD5 code out of TCP itself
tcp_md5: Use tcp_extra_options in output path
tcp_md5: Cleanup TCP-code
tcp_md5: Use TCP extra-options on the input path

Mat Martineau (1):
tcp: Register handlers for extra TCP options

drivers/infiniband/hw/cxgb4/cm.c |    2 +-
include/linux/inet_diag.h        |    1 +
include/linux/tcp.h              |   41 +-
include/linux/tcp_md5.h          |   38 ++
include/net/inet_sock.h          |    3 +-
include/net/tcp.h                |  216 ++++---
net/ipv4/Makefile                |    1 +
net/ipv4/syncookies.c            |    6 +-
net/ipv4/tcp.c                   |  372 ++++++++---
net/ipv4/tcp_diag.c              |   81 +--
net/ipv4/tcp_input.c             |  134 ++--
net/ipv4/tcp_ipv4.c              |  553 ++--------------
net/ipv4/tcp_md5.c               | 1317 ++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_minisocks.c         |   76 +--
net/ipv4/tcp_output.c            |  183 +-----
net/ipv6/syncookies.c            |    6 +-
net/ipv6/tcp_ipv6.c              |  386 ++---------
net/smc/af_smc.c                 |  170 ++++-
18 files changed, 2148 insertions(+), 1438 deletions(-)
Quoted 7 lines

create mode 100644 include/linux/tcp_md5.h
create mode 100644 net/ipv4/tcp_md5.c

Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#51 lore raw
Quoted 48 lines · Mat Martineau

Christoph -

Quoted 5 lines · Christoph Paasch

Finally, the second version of this patch-set.

At netdev I talked about this with Eric, and he is looking forward to see TCP-MD5 get out of the hot TCP data-path. He was planning to avoid TCP-MD5 with static jump-labels and also move TCP-MD5 in a separate file.

Good to hear that he's interested.

Quoted 13 lines

There are quite a few changes in this new set. I extended much more the extra-option framework. Making the list on a per-socket basis in the request-socket, tcp-socket and time-wait-socket. This eliminates the need for RCU-locking. See patch "tcp_extra_options: Make extra-option list per-socket"

I also added callbacks for response_write/prepare that can be called from the tcp_v4_send_ack/rst, and v6 counterparts. This is needed because in these code-paths the skb and sk have entirely different semantics.

Finally, in the input-path I added a check-callback and a post-process callback.

Quoted 8 lines

This is needed because after parsing the option in tcp_parse_options() we also need to act on them. The actions can be either before any other checks (e.g., as is the case for TCP_MD5, because we want to drop segments) or after the validity checks (e.g., as is needed for SMC).

And I also opted SMC into the new framework.

And, I made the TCP-MD5 input code-path use the framework.

All of these seem like the right way to go. Nice to have two users of the framework already.

Quoted 6 lines

There are still a few things that I need to double-check, notably sk_nocaps_add() in TCP_MD5.

And, I need to test ;-)

Yeah, that testing :)

Quoted 1 line
Quoted 6 lines
Quoted 3 lines

But I would like to have some early feedback already - especially on the patches prefixed with "tcp_extra_options". If they look good, I can merge them into Mat's patch, which will simplify my patch-juggling.

I have a few comments, but no major structural changes. I'll reply to the individual patches where needed.

Thanks for the feedback! I will reply to each individually.

Christoph

Quoted 73 lines
Quoted 3 lines

I want to get this patch-set out to netdev as an RFC as quickly as possible, so that we can make some progress.

Agreed!

Mat

Quoted 53 lines

Christoph Paasch (24):
tcp: Write options after the header has been fully done
tcp: Pass sock to tcp_options_write instead of tcp_sock
tcp: Pass skb in tcp_options_write
tcp: Allow tcp_fast_parse_options to drop segments
tcp: Make smc_parse_options return 1 on success
tcp_md5: Don't pass along md5-key
tcp_md5: Detect key inside tcp_v4_send_ack instead of passing it as an
argument
tcp_md5: Detect key inside tcp_v6_send_response instead of passing it
as an argument
tcp_extra_options: Extend tcp_extra_options_write
tcp_extra_options: Make extra-option list per-socket
tcp_extra_options: Export module fixes
tcp_extra_options: Add response_prepare callback
tcp_extra_options: Check static branch before _parse
tcp_extra_options: Skip fast-path when extra-options are present
tcp_extra_options: Pass sk to tcp_parse_options instead of tp
tcp_extra_options: Add check-callback right after parsing the option
tcp_extra_options: Allow to parse experimental options
tcp_extra_options: Add post_process callback
tcp_smc: Make SMC use TCP extra-option framework
tcp_md5: Check for TCP_MD5 after TCP Timestamps in
tcp_established_options
tcp_md5: Move TCP-MD5 code out of TCP itself
tcp_md5: Use tcp_extra_options in output path
tcp_md5: Cleanup TCP-code
tcp_md5: Use TCP extra-options on the input path

Mat Martineau (1):
tcp: Register handlers for extra TCP options

drivers/infiniband/hw/cxgb4/cm.c |    2 +-
include/linux/inet_diag.h        |    1 +
include/linux/tcp.h              |   41 +-
include/linux/tcp_md5.h          |   38 ++
include/net/inet_sock.h          |    3 +-
include/net/tcp.h                |  216 ++++---
net/ipv4/Makefile                |    1 +
net/ipv4/syncookies.c            |    6 +-
net/ipv4/tcp.c                   |  372 ++++++++---
net/ipv4/tcp_diag.c              |   81 +--
net/ipv4/tcp_input.c             |  134 ++--
net/ipv4/tcp_ipv4.c              |  553 ++--------------
net/ipv4/tcp_md5.c               | 1317 ++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_minisocks.c         |   76 +--
net/ipv4/tcp_output.c            |  183 +-----
net/ipv6/syncookies.c            |    6 +-
net/ipv6/tcp_ipv6.c              |  386 ++---------
net/smc/af_smc.c                 |  170 ++++-
18 files changed, 2148 insertions(+), 1438 deletions(-)
Quoted 7 lines

create mode 100644 include/linux/tcp_md5.h
create mode 100644 net/ipv4/tcp_md5.c

Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#52 lore raw
In reply to Mat Martineau

Oh, and one more thing.

tcp_extra_options_* is quite long, and function-names are really long, bringing us often close to the 80-char "limit".

What do you think of abbreviating it to "tcp_eo" or "teo" or something like that?

Christoph

Quoted 127 lines · Mat Martineau

Christoph -

Quoted 5 lines · Christoph Paasch

Finally, the second version of this patch-set.

At netdev I talked about this with Eric, and he is looking forward to see TCP-MD5 get out of the hot TCP data-path. He was planning to avoid TCP-MD5 with static jump-labels and also move TCP-MD5 in a separate file.

Good to hear that he's interested.

Quoted 13 lines

There are quite a few changes in this new set. I extended much more the extra-option framework. Making the list on a per-socket basis in the request-socket, tcp-socket and time-wait-socket. This eliminates the need for RCU-locking. See patch "tcp_extra_options: Make extra-option list per-socket"

I also added callbacks for response_write/prepare that can be called from the tcp_v4_send_ack/rst, and v6 counterparts. This is needed because in these code-paths the skb and sk have entirely different semantics.

Finally, in the input-path I added a check-callback and a post-process callback.

Quoted 8 lines

This is needed because after parsing the option in tcp_parse_options() we also need to act on them. The actions can be either before any other checks (e.g., as is the case for TCP_MD5, because we want to drop segments) or after the validity checks (e.g., as is needed for SMC).

And I also opted SMC into the new framework.

And, I made the TCP-MD5 input code-path use the framework.

All of these seem like the right way to go. Nice to have two users of the framework already.

Quoted 6 lines

There are still a few things that I need to double-check, notably sk_nocaps_add() in TCP_MD5.

And, I need to test ;-)

Yeah, that testing :)

Quoted 4 lines

But I would like to have some early feedback already - especially on the patches prefixed with "tcp_extra_options". If they look good, I can merge them into Mat's patch, which will simplify my patch-juggling.

I have a few comments, but no major structural changes. I'll reply to the individual patches where needed.

Quoted 2 lines

I want to get this patch-set out to netdev as an RFC as quickly as possible, so that we can make some progress.

Agreed!

Mat

Quoted 53 lines

Christoph Paasch (24):
tcp: Write options after the header has been fully done
tcp: Pass sock to tcp_options_write instead of tcp_sock
tcp: Pass skb in tcp_options_write
tcp: Allow tcp_fast_parse_options to drop segments
tcp: Make smc_parse_options return 1 on success
tcp_md5: Don't pass along md5-key
tcp_md5: Detect key inside tcp_v4_send_ack instead of passing it as an
argument
tcp_md5: Detect key inside tcp_v6_send_response instead of passing it
as an argument
tcp_extra_options: Extend tcp_extra_options_write
tcp_extra_options: Make extra-option list per-socket
tcp_extra_options: Export module fixes
tcp_extra_options: Add response_prepare callback
tcp_extra_options: Check static branch before _parse
tcp_extra_options: Skip fast-path when extra-options are present
tcp_extra_options: Pass sk to tcp_parse_options instead of tp
tcp_extra_options: Add check-callback right after parsing the option
tcp_extra_options: Allow to parse experimental options
tcp_extra_options: Add post_process callback
tcp_smc: Make SMC use TCP extra-option framework
tcp_md5: Check for TCP_MD5 after TCP Timestamps in
tcp_established_options
tcp_md5: Move TCP-MD5 code out of TCP itself
tcp_md5: Use tcp_extra_options in output path
tcp_md5: Cleanup TCP-code
tcp_md5: Use TCP extra-options on the input path

Mat Martineau (1):
tcp: Register handlers for extra TCP options

drivers/infiniband/hw/cxgb4/cm.c |    2 +-
include/linux/inet_diag.h        |    1 +
include/linux/tcp.h              |   41 +-
include/linux/tcp_md5.h          |   38 ++
include/net/inet_sock.h          |    3 +-
include/net/tcp.h                |  216 ++++---
net/ipv4/Makefile                |    1 +
net/ipv4/syncookies.c            |    6 +-
net/ipv4/tcp.c                   |  372 ++++++++---
net/ipv4/tcp_diag.c              |   81 +--
net/ipv4/tcp_input.c             |  134 ++--
net/ipv4/tcp_ipv4.c              |  553 ++--------------
net/ipv4/tcp_md5.c               | 1317 ++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_minisocks.c         |   76 +--
net/ipv4/tcp_output.c            |  183 +-----
net/ipv6/syncookies.c            |    6 +-
net/ipv6/tcp_ipv6.c              |  386 ++---------
net/smc/af_smc.c                 |  170 ++++-
18 files changed, 2148 insertions(+), 1438 deletions(-)
Quoted 7 lines

create mode 100644 include/linux/tcp_md5.h
create mode 100644 net/ipv4/tcp_md5.c

Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

author
#53 lore raw
Quoted 7 lines · Christoph Paasch

Oh, and one more thing.

tcp_extra_options_* is quite long, and function-names are really long, bringing us often close to the 80-char "limit".

What do you think of abbreviating it to "tcp_eo" or "teo" or something like that?

tcp_xopt_* ? :)

Christoph

Quoted 136 lines

Christoph

Quoted 119 lines · Mat Martineau

Christoph -

Quoted 5 lines · Christoph Paasch

Finally, the second version of this patch-set.

At netdev I talked about this with Eric, and he is looking forward to see TCP-MD5 get out of the hot TCP data-path. He was planning to avoid TCP-MD5 with static jump-labels and also move TCP-MD5 in a separate file.

Good to hear that he's interested.

Quoted 13 lines

There are quite a few changes in this new set. I extended much more the extra-option framework. Making the list on a per-socket basis in the request-socket, tcp-socket and time-wait-socket. This eliminates the need for RCU-locking. See patch "tcp_extra_options: Make extra-option list per-socket"

I also added callbacks for response_write/prepare that can be called from the tcp_v4_send_ack/rst, and v6 counterparts. This is needed because in these code-paths the skb and sk have entirely different semantics.

Finally, in the input-path I added a check-callback and a post-process callback.

Quoted 8 lines

This is needed because after parsing the option in tcp_parse_options() we also need to act on them. The actions can be either before any other checks (e.g., as is the case for TCP_MD5, because we want to drop segments) or after the validity checks (e.g., as is needed for SMC).

And I also opted SMC into the new framework.

And, I made the TCP-MD5 input code-path use the framework.

All of these seem like the right way to go. Nice to have two users of the framework already.

Quoted 6 lines

There are still a few things that I need to double-check, notably sk_nocaps_add() in TCP_MD5.

And, I need to test ;-)

Yeah, that testing :)

Quoted 4 lines

But I would like to have some early feedback already - especially on the patches prefixed with "tcp_extra_options". If they look good, I can merge them into Mat's patch, which will simplify my patch-juggling.

I have a few comments, but no major structural changes. I'll reply to the individual patches where needed.

Quoted 2 lines

I want to get this patch-set out to netdev as an RFC as quickly as possible, so that we can make some progress.

Agreed!

Mat

Quoted 56 lines

Christoph Paasch (24):
tcp: Write options after the header has been fully done
tcp: Pass sock to tcp_options_write instead of tcp_sock
tcp: Pass skb in tcp_options_write
tcp: Allow tcp_fast_parse_options to drop segments
tcp: Make smc_parse_options return 1 on success
tcp_md5: Don't pass along md5-key
tcp_md5: Detect key inside tcp_v4_send_ack instead of passing it as an
argument
tcp_md5: Detect key inside tcp_v6_send_response instead of passing it
as an argument
tcp_extra_options: Extend tcp_extra_options_write
tcp_extra_options: Make extra-option list per-socket
tcp_extra_options: Export module fixes
tcp_extra_options: Add response_prepare callback
tcp_extra_options: Check static branch before _parse
tcp_extra_options: Skip fast-path when extra-options are present
tcp_extra_options: Pass sk to tcp_parse_options instead of tp
tcp_extra_options: Add check-callback right after parsing the option
tcp_extra_options: Allow to parse experimental options
tcp_extra_options: Add post_process callback
tcp_smc: Make SMC use TCP extra-option framework
tcp_md5: Check for TCP_MD5 after TCP Timestamps in
tcp_established_options
tcp_md5: Move TCP-MD5 code out of TCP itself
tcp_md5: Use tcp_extra_options in output path
tcp_md5: Cleanup TCP-code
tcp_md5: Use TCP extra-options on the input path

Mat Martineau (1):
tcp: Register handlers for extra TCP options

drivers/infiniband/hw/cxgb4/cm.c |    2 +-
include/linux/inet_diag.h        |    1 +
include/linux/tcp.h              |   41 +-
include/linux/tcp_md5.h          |   38 ++
include/net/inet_sock.h          |    3 +-
include/net/tcp.h                |  216 ++++---
net/ipv4/Makefile                |    1 +
net/ipv4/syncookies.c            |    6 +-
net/ipv4/tcp.c                   |  372 ++++++++---
net/ipv4/tcp_diag.c              |   81 +--
net/ipv4/tcp_input.c             |  134 ++--
net/ipv4/tcp_ipv4.c              |  553 ++--------------
net/ipv4/tcp_md5.c               | 1317 ++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_minisocks.c         |   76 +--
net/ipv4/tcp_output.c            |  183 +-----
net/ipv6/syncookies.c            |    6 +-
net/ipv6/tcp_ipv6.c              |  386 ++---------
net/smc/af_smc.c                 |  170 ++++-
18 files changed, 2148 insertions(+), 1438 deletions(-)

create mode 100644 include/linux/tcp_md5.h
create mode 100644 net/ipv4/tcp_md5.c

Quoted 8 lines
Quoted 4 lines
Signature · 1 line
2.15.0

--
Mat Martineau
Intel OTC

_______________________________________________
mptcp mailing list
mptcp(a)lists.01.org
https://lists.01.org/mailman/listinfo/mptcp

#54 lore raw
Quoted 10 lines · Christoph Paasch
Quoted 7 lines · Christoph Paasch

Oh, and one more thing.

tcp_extra_options_* is quite long, and function-names are really long, bringing us often close to the 80-char "limit".

What do you think of abbreviating it to "tcp_eo" or "teo" or something like that?

tcp_xopt_* ? :)

I don't have a strong preference, but tcp_extopt_* maps a little better to "extra" or "extended" options (in my mind, anyway).

Mat

Quoted 129 lines
Quoted 129 lines

Christoph

Quoted 124 lines · Mat Martineau

Christoph -

Quoted 5 lines · Christoph Paasch

Finally, the second version of this patch-set.

At netdev I talked about this with Eric, and he is looking forward to see TCP-MD5 get out of the hot TCP data-path. He was planning to avoid TCP-MD5 with static jump-labels and also move TCP-MD5 in a separate file.

Good to hear that he's interested.

Quoted 13 lines

There are quite a few changes in this new set. I extended much more the extra-option framework. Making the list on a per-socket basis in the request-socket, tcp-socket and time-wait-socket. This eliminates the need for RCU-locking. See patch "tcp_extra_options: Make extra-option list per-socket"

I also added callbacks for response_write/prepare that can be called from the tcp_v4_send_ack/rst, and v6 counterparts. This is needed because in these code-paths the skb and sk have entirely different semantics.

Finally, in the input-path I added a check-callback and a post-process callback.

Quoted 8 lines

This is needed because after parsing the option in tcp_parse_options() we also need to act on them. The actions can be either before any other checks (e.g., as is the case for TCP_MD5, because we want to drop segments) or after the validity checks (e.g., as is needed for SMC).

And I also opted SMC into the new framework.

And, I made the TCP-MD5 input code-path use the framework.

All of these seem like the right way to go. Nice to have two users of the framework already.

Quoted 6 lines

There are still a few things that I need to double-check, notably sk_nocaps_add() in TCP_MD5.

And, I need to test ;-)

Yeah, that testing :)

Quoted 4 lines

But I would like to have some early feedback already - especially on the patches prefixed with "tcp_extra_options". If they look good, I can merge them into Mat's patch, which will simplify my patch-juggling.

I have a few comments, but no major structural changes. I'll reply to the individual patches where needed.

Quoted 2 lines

I want to get this patch-set out to netdev as an RFC as quickly as possible, so that we can make some progress.

Agreed!

Mat

Quoted 60 lines

Christoph Paasch (24):
tcp: Write options after the header has been fully done
tcp: Pass sock to tcp_options_write instead of tcp_sock
tcp: Pass skb in tcp_options_write
tcp: Allow tcp_fast_parse_options to drop segments
tcp: Make smc_parse_options return 1 on success
tcp_md5: Don't pass along md5-key
tcp_md5: Detect key inside tcp_v4_send_ack instead of passing it as an
argument
tcp_md5: Detect key inside tcp_v6_send_response instead of passing it
as an argument
tcp_extra_options: Extend tcp_extra_options_write
tcp_extra_options: Make extra-option list per-socket
tcp_extra_options: Export module fixes
tcp_extra_options: Add response_prepare callback
tcp_extra_options: Check static branch before _parse
tcp_extra_options: Skip fast-path when extra-options are present
tcp_extra_options: Pass sk to tcp_parse_options instead of tp
tcp_extra_options: Add check-callback right after parsing the option
tcp_extra_options: Allow to parse experimental options
tcp_extra_options: Add post_process callback
tcp_smc: Make SMC use TCP extra-option framework
tcp_md5: Check for TCP_MD5 after TCP Timestamps in
tcp_established_options
tcp_md5: Move TCP-MD5 code out of TCP itself
tcp_md5: Use tcp_extra_options in output path
tcp_md5: Cleanup TCP-code
tcp_md5: Use TCP extra-options on the input path

Mat Martineau (1):
tcp: Register handlers for extra TCP options

drivers/infiniband/hw/cxgb4/cm.c |    2 +-
include/linux/inet_diag.h        |    1 +
include/linux/tcp.h              |   41 +-
include/linux/tcp_md5.h          |   38 ++
include/net/inet_sock.h          |    3 +-
include/net/tcp.h                |  216 ++++---
net/ipv4/Makefile                |    1 +
net/ipv4/syncookies.c            |    6 +-
net/ipv4/tcp.c                   |  372 ++++++++---
net/ipv4/tcp_diag.c              |   81 +--
net/ipv4/tcp_input.c             |  134 ++--
net/ipv4/tcp_ipv4.c              |  553 ++--------------
net/ipv4/tcp_md5.c               | 1317 ++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_minisocks.c         |   76 +--
net/ipv4/tcp_output.c            |  183 +-----
net/ipv6/syncookies.c            |    6 +-
net/ipv6/tcp_ipv6.c              |  386 ++---------
net/smc/af_smc.c                 |  170 ++++-
18 files changed, 2148 insertions(+), 1438 deletions(-)

create mode 100644 include/linux/tcp_md5.h
create mode 100644 net/ipv4/tcp_md5.c

--
2.15.0

Quoted 8 lines
Quoted 7 lines
Quoted 3 lines

--
Mat Martineau
Intel OTC

_______________________________________________
mptcp mailing list
mptcp(a)lists.01.org
https://lists.01.org/mailman/listinfo/mptcp

--
Mat Martineau
Intel OTC

author
#55 lore raw
Quoted 8 lines · Mat Martineau
Quoted 6 lines · Christoph Paasch
Quoted 5 lines · Christoph Paasch

Oh, and one more thing.

tcp_extra_options_* is quite long, and function-names are really long, bringing us often close to the 80-char "limit".

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

What do you think of abbreviating it to "tcp_eo" or "teo" or something like that?

tcp_xopt_* ? :)

I don't have a strong preference, but tcp_extopt_* maps a little better to "extra" or "extended" options (in my mind, anyway).

Ok, I will go with tcp_extopt_*

Christoph

Quoted 145 lines

Mat

Quoted 129 lines
Quoted 129 lines

Christoph

Quoted 124 lines · Mat Martineau

Christoph -

Quoted 5 lines · Christoph Paasch

Finally, the second version of this patch-set.

At netdev I talked about this with Eric, and he is looking forward to see TCP-MD5 get out of the hot TCP data-path. He was planning to avoid TCP-MD5 with static jump-labels and also move TCP-MD5 in a separate file.

Good to hear that he's interested.

Quoted 13 lines

There are quite a few changes in this new set. I extended much more the extra-option framework. Making the list on a per-socket basis in the request-socket, tcp-socket and time-wait-socket. This eliminates the need for RCU-locking. See patch "tcp_extra_options: Make extra-option list per-socket"

I also added callbacks for response_write/prepare that can be called from the tcp_v4_send_ack/rst, and v6 counterparts. This is needed because in these code-paths the skb and sk have entirely different semantics.

Finally, in the input-path I added a check-callback and a post-process callback.

Quoted 8 lines

This is needed because after parsing the option in tcp_parse_options() we also need to act on them. The actions can be either before any other checks (e.g., as is the case for TCP_MD5, because we want to drop segments) or after the validity checks (e.g., as is needed for SMC).

And I also opted SMC into the new framework.

And, I made the TCP-MD5 input code-path use the framework.

All of these seem like the right way to go. Nice to have two users of the framework already.

Quoted 6 lines

There are still a few things that I need to double-check, notably sk_nocaps_add() in TCP_MD5.

And, I need to test ;-)

Yeah, that testing :)

Quoted 4 lines

But I would like to have some early feedback already - especially on the patches prefixed with "tcp_extra_options". If they look good, I can merge them into Mat's patch, which will simplify my patch-juggling.

I have a few comments, but no major structural changes. I'll reply to the individual patches where needed.

Quoted 2 lines

I want to get this patch-set out to netdev as an RFC as quickly as possible, so that we can make some progress.

Agreed!

Mat

Quoted 60 lines

Christoph Paasch (24):
tcp: Write options after the header has been fully done
tcp: Pass sock to tcp_options_write instead of tcp_sock
tcp: Pass skb in tcp_options_write
tcp: Allow tcp_fast_parse_options to drop segments
tcp: Make smc_parse_options return 1 on success
tcp_md5: Don't pass along md5-key
tcp_md5: Detect key inside tcp_v4_send_ack instead of passing it as an
argument
tcp_md5: Detect key inside tcp_v6_send_response instead of passing it
as an argument
tcp_extra_options: Extend tcp_extra_options_write
tcp_extra_options: Make extra-option list per-socket
tcp_extra_options: Export module fixes
tcp_extra_options: Add response_prepare callback
tcp_extra_options: Check static branch before _parse
tcp_extra_options: Skip fast-path when extra-options are present
tcp_extra_options: Pass sk to tcp_parse_options instead of tp
tcp_extra_options: Add check-callback right after parsing the option
tcp_extra_options: Allow to parse experimental options
tcp_extra_options: Add post_process callback
tcp_smc: Make SMC use TCP extra-option framework
tcp_md5: Check for TCP_MD5 after TCP Timestamps in
tcp_established_options
tcp_md5: Move TCP-MD5 code out of TCP itself
tcp_md5: Use tcp_extra_options in output path
tcp_md5: Cleanup TCP-code
tcp_md5: Use TCP extra-options on the input path

Mat Martineau (1):
tcp: Register handlers for extra TCP options

drivers/infiniband/hw/cxgb4/cm.c |    2 +-
include/linux/inet_diag.h        |    1 +
include/linux/tcp.h              |   41 +-
include/linux/tcp_md5.h          |   38 ++
include/net/inet_sock.h          |    3 +-
include/net/tcp.h                |  216 ++++---
net/ipv4/Makefile                |    1 +
net/ipv4/syncookies.c            |    6 +-
net/ipv4/tcp.c                   |  372 ++++++++---
net/ipv4/tcp_diag.c              |   81 +--
net/ipv4/tcp_input.c             |  134 ++--
net/ipv4/tcp_ipv4.c              |  553 ++--------------
net/ipv4/tcp_md5.c               | 1317 ++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_minisocks.c         |   76 +--
net/ipv4/tcp_output.c            |  183 +-----
net/ipv6/syncookies.c            |    6 +-
net/ipv6/tcp_ipv6.c              |  386 ++---------
net/smc/af_smc.c                 |  170 ++++-
18 files changed, 2148 insertions(+), 1438 deletions(-)

create mode 100644 include/linux/tcp_md5.h
create mode 100644 net/ipv4/tcp_md5.c

--
2.15.0

Quoted 8 lines
Quoted 7 lines
Quoted 3 lines

--
Mat Martineau
Intel OTC

_______________________________________________
mptcp mailing list
mptcp(a)lists.01.org
https://lists.01.org/mailman/listinfo/mptcp

--
Mat Martineau
Intel OTC