Skip to contentSkip to conversation

tcp: avoid the lookup process failing to get sk in ehash table

PATCH Superseded Jason Xing 2 participants 4 messages Jan 14, 2023 – Jan 16, 2023

Subsystems (3)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Jason Xing

author
#0 lore raw

PATCH v2 1/1 tcp: avoid the lookup process failing to get sk in ehash table

From: Jason Xing <[email protected]>

While one cpu is working on looking up the right socket from ehash table, another cpu is done deleting the request socket and is about to add (or is adding) the big socket from the table. It means that we could miss both of them, even though it has little chance.

Let me draw a call trace map of the server side.
CPU 0 CPU 1
----- -----
tcp_v4_rcv() syn_recv_sock()

                            inet_ehash_insert()
                            -> sk_nulls_del_node_init_rcu(osk)
__inet_lookup_established()
                            -> __sk_nulls_add_node_rcu(sk, list)

Notice that the CPU 0 is receiving the data after the final ack during 3-way shakehands and CPU 1 is still handling the final ack.

Why could this be a real problem?
This case is happening only when the final ack and the first data receiving by different CPUs. Then the server receiving data with ACK flag tries to search one proper established socket from ehash table, but apparently it fails as my map shows above. After that, the server fetches a listener socket and then sends a RST because it finds a ACK flag in the skb (data), which obeys RST definition in RFC 793.

Besides, Eric pointed out there's one more race condition where it handles tw socket hashdance. Only by adding to the tail of the list before deleting the old one can we avoid the race if the reader has already begun the bucket traversal and it would possibly miss the head.

Many thanks to Eric for great help from beginning to end.

Fixes 5e0724d027f0 ("tcp/dccp: fix hashdance race for passive sessions")

---
v2:
1) adding the sk node into the tail of list to prevent the race.
2) fix the race condition when handling time-wait socket hashdance.

---
 net/ipv4/inet_hashtables.c    | 10 ++++++++++
 net/ipv4/inet_timewait_sock.c |  6 +++---
 2 files changed, 13 insertions(+), 3 deletions(-)
net/ipv4/inet_hashtables.c +10 -0
index 24a38b56fab9..b0b54ad55507 100644
@@ -650,7 +650,16 @@ bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk)
 	spin_lock(lock);
 	if (osk) {
 		WARN_ON_ONCE(sk->sk_hash != osk->sk_hash);
+		if (sk_hashed(osk))
+			/* Before deleting the node, we insert a new one to make
+			 * sure that the look-up-sk process would not miss either
+			 * of them and that at least one node would exist in ehash
+			 * table all the time. Otherwise there's a tiny chance
+			 * that lookup process could find nothing in ehash table.
+			 */
+			__sk_nulls_add_node_tail_rcu(sk, list);
 		ret = sk_nulls_del_node_init_rcu(osk);
+		goto unlock;
 	} else if (found_dup_sk) {
 		*found_dup_sk = inet_ehash_lookup_by_sk(sk, list);
 		if (*found_dup_sk)
@@ -660,6 +669,7 @@ bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk)
 	if (ret)
 		__sk_nulls_add_node_rcu(sk, list);
 
+unlock:
 	spin_unlock(lock);
 
 	return ret;
net/ipv4/inet_timewait_sock.c +3 -3
index 1d77d992e6e7..6d681ef52bb2 100644
@@ -91,10 +91,10 @@ void inet_twsk_put(struct inet_timewait_sock *tw)
 }
 EXPORT_SYMBOL_GPL(inet_twsk_put);
 
-static void inet_twsk_add_node_rcu(struct inet_timewait_sock *tw,
+static void inet_twsk_add_node_tail_rcu(struct inet_timewait_sock *tw,
 				   struct hlist_nulls_head *list)
 {
-	hlist_nulls_add_head_rcu(&tw->tw_node, list);
+	hlist_nulls_add_tail_rcu(&tw->tw_node, list);
 }
 
 static void inet_twsk_add_bind_node(struct inet_timewait_sock *tw,
@@ -147,7 +147,7 @@ void inet_twsk_hashdance(struct inet_timewait_sock *tw, struct sock *sk,
 
 	spin_lock(lock);
 
-	inet_twsk_add_node_rcu(tw, &ehead->chain);
+	inet_twsk_add_node_tail_rcu(tw, &ehead->chain);
 
 	/* Step 3: Remove SK from hash chain */
 	if (__sk_nulls_del_node_init_rcu(sk))
Signature · 1 line
2.37.3
#1 lore raw
Quoted 22 lines · Jason Xing

From: Jason Xing <[email protected]>

Fixes 5e0724d027f0 ("tcp/dccp: fix hashdance race for passive sessions")

---
v2:
1) adding the sk node into the tail of list to prevent the race.
2) fix the race condition when handling time-wait socket hashdance.

---
 net/ipv4/inet_hashtables.c    | 10 ++++++++++
 net/ipv4/inet_timewait_sock.c |  6 +++---
 2 files changed, 13 insertions(+), 3 deletions(-)
net/ipv4/inet_hashtables.c +0 -0
index 24a38b56fab9..b0b54ad55507 100644

net/ipv4/inet_hashtables.c bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk) · line 653

Quoted 4 lines
net/ipv4/inet_hashtables.c +1 -0
@@ -650,7 +650,16 @@ bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk)
        spin_lock(lock);
        if (osk) {
                WARN_ON_ONCE(sk->sk_hash != osk->sk_hash);
+               if (sk_hashed(osk))

nit: this should be:

if (sk_hashed(osk)) { [1]

    /* multi-line ....
     * .... comment.
     */

ret = sk_nulls_del_node_init_rcu(osk);
goto unlock;
}
if (found_dup_sk) { [2]

1) parentheses needed in [1]
2) No else if in [2], since you added a "goto unlock;"

Quoted 13 lines
net/ipv4/inet_hashtables.c +8 -0
+                       /* Before deleting the node, we insert a new one to make
+                        * sure that the look-up-sk process would not miss either
+                        * of them and that at least one node would exist in ehash
+                        * table all the time. Otherwise there's a tiny chance
+                        * that lookup process could find nothing in ehash table.
+                        */
+                       __sk_nulls_add_node_tail_rcu(sk, list);
                ret = sk_nulls_del_node_init_rcu(osk);
+               goto unlock;
        } else if (found_dup_sk) {
                *found_dup_sk = inet_ehash_lookup_by_sk(sk, list);
                if (*found_dup_sk)

net/ipv4/inet_hashtables.c bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk) · line 673

Quoted 5 lines
net/ipv4/inet_hashtables.c +1 -0
@@ -660,6 +669,7 @@ bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk)
        if (ret)
                __sk_nulls_add_node_rcu(sk, list);
 
+unlock:
        spin_unlock(lock);

Thanks.

author
#2 lore raw
Quoted 39 lines · Eric Dumazet
Quoted 22 lines · Jason Xing

From: Jason Xing <[email protected]>

Fixes 5e0724d027f0 ("tcp/dccp: fix hashdance race for passive sessions")

---
v2:
1) adding the sk node into the tail of list to prevent the race.
2) fix the race condition when handling time-wait socket hashdance.

---
 net/ipv4/inet_hashtables.c    | 10 ++++++++++
 net/ipv4/inet_timewait_sock.c |  6 +++---
 2 files changed, 13 insertions(+), 3 deletions(-)
net/ipv4/inet_hashtables.c +0 -0
index 24a38b56fab9..b0b54ad55507 100644

net/ipv4/inet_hashtables.c bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk) · line 653

Quoted 4 lines
net/ipv4/inet_hashtables.c +1 -0
@@ -650,7 +650,16 @@ bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk)
        spin_lock(lock);
        if (osk) {
                WARN_ON_ONCE(sk->sk_hash != osk->sk_hash);
+               if (sk_hashed(osk))

nit: this should be:

if (sk_hashed(osk)) { [1]

    /* multi-line ....
     * .... comment.
     */
   ret = sk_nulls_del_node_init_rcu(osk);
   goto unlock;
}
Quoted 5 lines

if (found_dup_sk) { [2]

1) parentheses needed in [1]
2) No else if in [2], since you added a "goto unlock;"

I'll do that. It looks much better.

Thanks,
Jason

Quoted 20 lines
Quoted 13 lines
net/ipv4/inet_hashtables.c +8 -0
+                       /* Before deleting the node, we insert a new one to make
+                        * sure that the look-up-sk process would not miss either
+                        * of them and that at least one node would exist in ehash
+                        * table all the time. Otherwise there's a tiny chance
+                        * that lookup process could find nothing in ehash table.
+                        */
+                       __sk_nulls_add_node_tail_rcu(sk, list);
                ret = sk_nulls_del_node_init_rcu(osk);
+               goto unlock;
        } else if (found_dup_sk) {
                *found_dup_sk = inet_ehash_lookup_by_sk(sk, list);
                if (*found_dup_sk)

net/ipv4/inet_hashtables.c bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk) · line 673

Quoted 5 lines
net/ipv4/inet_hashtables.c +1 -0
@@ -660,6 +669,7 @@ bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk)
        if (ret)
                __sk_nulls_add_node_rcu(sk, list);
 
+unlock:
        spin_unlock(lock);

Thanks.

author
#3 lore raw
Quoted 34 lines · Jason Xing
Quoted 32 lines · Eric Dumazet
Quoted 22 lines · Jason Xing

From: Jason Xing <[email protected]>

Fixes 5e0724d027f0 ("tcp/dccp: fix hashdance race for passive sessions")

---
v2:
1) adding the sk node into the tail of list to prevent the race.
2) fix the race condition when handling time-wait socket hashdance.

---
 net/ipv4/inet_hashtables.c    | 10 ++++++++++
 net/ipv4/inet_timewait_sock.c |  6 +++---
 2 files changed, 13 insertions(+), 3 deletions(-)
net/ipv4/inet_hashtables.c +0 -0
index 24a38b56fab9..b0b54ad55507 100644

net/ipv4/inet_hashtables.c bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk) · line 653

Quoted 4 lines
net/ipv4/inet_hashtables.c +1 -0
@@ -650,7 +650,16 @@ bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk)
        spin_lock(lock);
        if (osk) {
                WARN_ON_ONCE(sk->sk_hash != osk->sk_hash);
+               if (sk_hashed(osk))

nit: this should be:

Quoted 7 lines
Quoted 7 lines

if (sk_hashed(osk)) { [1]

    /* multi-line ....
     * .... comment.
     */
   ret = sk_nulls_del_node_init_rcu(osk);
   goto unlock;
}

Well, after I dug into this part, I found something as below. If we enter into the 'if (osk) {', we should always skip the next if-statement which is 'if (found_dup_sk) {' and return a proper value depending on if the osk is hashed.
However, the code as above would leave variable @ret to be true if the sk_hashed(osk) returned false, then It would not go to unlock and then add the node to the list and at last return true which is unexpected.

Quoted 5 lines
Quoted 5 lines

if (found_dup_sk) { [2]

1) parentheses needed in [1]

2) No else if in [2], since you added a "goto unlock;"

I think this modification is fine and makes the code clearer.

Thanks,
Jason

Quoted 27 lines
Quoted 1 line

I'll do that. It looks much better.

Thanks,
Jason

Quoted 20 lines
Quoted 13 lines
net/ipv4/inet_hashtables.c +8 -0
+                       /* Before deleting the node, we insert a new one to make
+                        * sure that the look-up-sk process would not miss either
+                        * of them and that at least one node would exist in ehash
+                        * table all the time. Otherwise there's a tiny chance
+                        * that lookup process could find nothing in ehash table.
+                        */
+                       __sk_nulls_add_node_tail_rcu(sk, list);
                ret = sk_nulls_del_node_init_rcu(osk);
+               goto unlock;
        } else if (found_dup_sk) {
                *found_dup_sk = inet_ehash_lookup_by_sk(sk, list);
                if (*found_dup_sk)

net/ipv4/inet_hashtables.c bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk) · line 673

Quoted 5 lines
net/ipv4/inet_hashtables.c +1 -0
@@ -660,6 +669,7 @@ bool inet_ehash_insert(struct sock *sk, struct sock *osk, bool *found_dup_sk)
        if (ret)
                __sk_nulls_add_node_rcu(sk, list);
 
+unlock:
        spin_unlock(lock);

Thanks.