Skip to contentSkip to conversation

net/smc: fix TOCTOU race between smc_listen_out() and listener close

PATCH Applied Sidraya Jayagond 5 participants 8 messages Aug 3 – Aug 6

Subsystems (4)
lore
Share & tools
lore

Sidraya Jayagond

author maintainer
#0 lore raw

PATCH 1/1 net/smc: fix TOCTOU race between smc_listen_out() and listener close

smc_listen_out() reads lsmc->sk.sk_state without the listener lock, then acquires lock_sock_nested() only after the check passes. This opens a window where smc_close_active() can transition the listener to SMC_CLOSED, call smc_close_cleanup_listen() to drain the accept queue, and release the lock, all between the lockless read and the delayed lock acquisition:

smc_listen_work (smc_hs_wq) smc_close_active()
------------------------------- -------------------------
release_sock(child)
if (sk_state == SMC_LISTEN) TRUE
lock_sock(listener) sk_state = SMC_CLOSED smc_close_cleanup_listen() release_sock(listener) flush_work(tcp_listen_work)
lock_sock_nested(listener)
smc_accept_enqueue(listener, child) /* child enqueued on dead listener */

smc_close_active() flushes only tcp_listen_work. Work items already dispatched onto smc_hs_wq for the CLC handshake continue running unguarded. smc_accept_enqueue() takes a sock_hold() on the child that is never released, so the child smc_sock, its clcsock, and the reference all leak. A remote peer that opens TCP connections while the server calls close() can exhaust kernel memory.

Move lock_sock_nested() to before the sk_state check so that the test and the enqueue are atomic under the listener lock.

Fixes fd57770dd198 ("net/smc: wait for pending work before clcsock release_sock")
---
 net/smc/af_smc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
net/smc/af_smc.c +2 -1
index b5db69073e20..00403175b740 100644
@@ -1931,11 +1931,12 @@ static void smc_listen_out(struct smc_sock *new_smc)
 		atomic_dec(&lsmc->queued_smc_hs);
 
 	release_sock(newsmcsk); /* lock in smc_listen_work() */
+	lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 	if (lsmc->sk.sk_state == SMC_LISTEN) {
-		lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 		smc_accept_enqueue(&lsmc->sk, newsmcsk);
 		release_sock(&lsmc->sk);
 	} else { /* no longer listening */
+		release_sock(&lsmc->sk);
 		smc_close_non_accepted(newsmcsk);
 	}
 
Signature · 1 line
2.53.0
#1 lore raw

net/smc/af_smc.c static void smc_listen_out(struct smc_sock *new_smc) · line 1941

Quoted 13 lines
net/smc/af_smc.c +2 -1
@@ -1931,11 +1931,12 @@ static void smc_listen_out(struct smc_sock *new_smc)
 		atomic_dec(&lsmc->queued_smc_hs);
 
 	release_sock(newsmcsk); /* lock in smc_listen_work() */
+	lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 	if (lsmc->sk.sk_state == SMC_LISTEN) {
-		lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 		smc_accept_enqueue(&lsmc->sk, newsmcsk);
 		release_sock(&lsmc->sk);
 	} else { /* no longer listening */
+		release_sock(&lsmc->sk);
 		smc_close_non_accepted(newsmcsk);
 	}

Do you need to call smc_close_non_accepted() without the lock? otherwise you can have the lock around the whole if/else clause.

#6 lore raw
In reply to Sidraya Jayagond
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 18 lines

smc_listen_out() reads lsmc->sk.sk_state without the listener lock, then acquires lock_sock_nested() only after the check passes. This opens a window where smc_close_active() can transition the listener to SMC_CLOSED, call smc_close_cleanup_listen() to drain the accept queue, and release the lock, all between the lockless read and the delayed lock acquisition:

smc_listen_work (smc_hs_wq) smc_close_active()
------------------------------- -------------------------
release_sock(child)
if (sk_state == SMC_LISTEN) TRUE
lock_sock(listener) sk_state = SMC_CLOSED smc_close_cleanup_listen() release_sock(listener)
[ ... ]
and the enqueue are atomic under the listener lock.

Quoted 3 lines
Fixes fd57770dd198 ("net/smc: wait for pending work before clcsock release_sock")

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author maintainer
#2 lore raw
In reply to Breno Leitao
Quoted 16 lines · Breno Leitao

net/smc/af_smc.c static void smc_listen_out(struct smc_sock *new_smc) · line 1941

Quoted 13 lines
net/smc/af_smc.c +2 -1
@@ -1931,11 +1931,12 @@ static void smc_listen_out(struct smc_sock *new_smc)
 		atomic_dec(&lsmc->queued_smc_hs);
 
 	release_sock(newsmcsk); /* lock in smc_listen_work() */
+	lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 	if (lsmc->sk.sk_state == SMC_LISTEN) {
-		lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 		smc_accept_enqueue(&lsmc->sk, newsmcsk);
 		release_sock(&lsmc->sk);
 	} else { /* no longer listening */
+		release_sock(&lsmc->sk);
 		smc_close_non_accepted(newsmcsk);
 	}

Do you need to call smc_close_non_accepted() without the lock? otherwise you can have the lock around the whole if/else clause.

Yes, smc_close_non_accepted() calls __smc_release() in turn calls smc_close_active(), which in the SMC_ACTIVE case hits smc_close_stream_wait() a blocking wait that can sleep up to SMC_MAX_STREAM_WAIT_TIMEOUT (2 minutes). Holding the listener lock across that would stall any concurrent smc_accept(), smc_listen_out(), or smc_release() on the listener for the same duration.

The release_sock() is intentionally placed before smc_close_non_accepted() to keep the critical section minimal:
lock
check state
enqueue or not
unlock
then do the slow close work without the listener lock held.

Reviewed-by
#3 lore raw
Quoted 27 lines · Sidraya Jayagond
Quoted 16 lines · Breno Leitao

net/smc/af_smc.c static void smc_listen_out(struct smc_sock *new_smc) · line 1941

Quoted 13 lines
net/smc/af_smc.c +2 -1
@@ -1931,11 +1931,12 @@ static void smc_listen_out(struct smc_sock *new_smc)
 		atomic_dec(&lsmc->queued_smc_hs);
 
 	release_sock(newsmcsk); /* lock in smc_listen_work() */
+	lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 	if (lsmc->sk.sk_state == SMC_LISTEN) {
-		lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 		smc_accept_enqueue(&lsmc->sk, newsmcsk);
 		release_sock(&lsmc->sk);
 	} else { /* no longer listening */
+		release_sock(&lsmc->sk);
 		smc_close_non_accepted(newsmcsk);
 	}

Do you need to call smc_close_non_accepted() without the lock? otherwise you can have the lock around the whole if/else clause.

Yes, smc_close_non_accepted() calls __smc_release() in turn calls smc_close_active(), which in the SMC_ACTIVE case hits smc_close_stream_wait() a blocking wait that can sleep up to SMC_MAX_STREAM_WAIT_TIMEOUT (2 minutes). Holding the listener lock across that would stall any concurrent smc_accept(), smc_listen_out(), or smc_release() on the listener for the same duration.

Quoted 7 lines

The release_sock() is intentionally placed before smc_close_non_accepted() to keep the critical section minimal:
lock
check state
enqueue or not
unlock
then do the slow close work without the listener lock held.

That makes sense.

maintainer Reviewed-by
#4 lore raw
In reply to Sidraya Jayagond
Quoted 27 lines · Sidraya Jayagond
Quoted 16 lines · Breno Leitao

net/smc/af_smc.c static void smc_listen_out(struct smc_sock *new_smc) · line 1941

Quoted 13 lines
net/smc/af_smc.c +2 -1
@@ -1931,11 +1931,12 @@ static void smc_listen_out(struct smc_sock *new_smc)
 		atomic_dec(&lsmc->queued_smc_hs);
 
 	release_sock(newsmcsk); /* lock in smc_listen_work() */
+	lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 	if (lsmc->sk.sk_state == SMC_LISTEN) {
-		lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 		smc_accept_enqueue(&lsmc->sk, newsmcsk);
 		release_sock(&lsmc->sk);
 	} else { /* no longer listening */
+		release_sock(&lsmc->sk);
 		smc_close_non_accepted(newsmcsk);
 	}

Do you need to call smc_close_non_accepted() without the lock? otherwise you can have the lock around the whole if/else clause.

Yes, smc_close_non_accepted() calls __smc_release() in turn calls smc_close_active(), which in the SMC_ACTIVE case hits smc_close_stream_wait() a blocking wait that can sleep up to SMC_MAX_STREAM_WAIT_TIMEOUT (2 minutes). Holding the listener lock across that would stall any concurrent smc_accept(), smc_listen_out(), or smc_release() on the listener for the same duration.

Quoted 7 lines

The release_sock() is intentionally placed before smc_close_non_accepted() to keep the critical section minimal:
lock
check state
enqueue or not
unlock
then do the slow close work without the listener lock held.

That patch looks good to me.

Two corrections to the justification though:

1. This newsmcsk sock have never been given to userspace, so smc_close_stream_wait() should return immediately. I think the stronger argument for keeping smc_close_non_accepted() outside the listener lock is what follows in smc_close_active()/__smc_release():
smc_close_final() can wait up to SMC_WR_TX_WAIT_FREE_SLOT_TIME (10s) for a WR slot, and smc_clcsock_release() tears down the TCP socket.

2. SMC_MAX_STREAM_WAIT_TIMEOUT is 2 * HZ, i.e. 2 seconds, not 2 minutes.

Best regards,
Dust

author maintainer
#5 lore raw
Quoted 52 lines · Dust Li
Quoted 27 lines · Sidraya Jayagond
Quoted 16 lines · Breno Leitao

net/smc/af_smc.c static void smc_listen_out(struct smc_sock *new_smc) · line 1941

Quoted 13 lines
net/smc/af_smc.c +2 -1
@@ -1931,11 +1931,12 @@ static void smc_listen_out(struct smc_sock *new_smc)
 		atomic_dec(&lsmc->queued_smc_hs);
 
 	release_sock(newsmcsk); /* lock in smc_listen_work() */
+	lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 	if (lsmc->sk.sk_state == SMC_LISTEN) {
-		lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING);
 		smc_accept_enqueue(&lsmc->sk, newsmcsk);
 		release_sock(&lsmc->sk);
 	} else { /* no longer listening */
+		release_sock(&lsmc->sk);
 		smc_close_non_accepted(newsmcsk);
 	}

Do you need to call smc_close_non_accepted() without the lock? otherwise you can have the lock around the whole if/else clause.

Yes, smc_close_non_accepted() calls __smc_release() in turn calls smc_close_active(), which in the SMC_ACTIVE case hits smc_close_stream_wait() a blocking wait that can sleep up to SMC_MAX_STREAM_WAIT_TIMEOUT (2 minutes). Holding the listener lock across that would stall any concurrent smc_accept(), smc_listen_out(), or smc_release() on the listener for the same duration.

Quoted 7 lines

The release_sock() is intentionally placed before smc_close_non_accepted() to keep the critical section minimal:
lock
check state
enqueue or not
unlock
then do the slow close work without the listener lock held.

That patch looks good to me.

Two corrections to the justification though:

1. This newsmcsk sock have never been given to userspace, so smc_close_stream_wait() should return immediately. I think the stronger argument for keeping smc_close_non_accepted() outside the listener lock is what follows in smc_close_active()/__smc_release():
smc_close_final() can wait up to SMC_WR_TX_WAIT_FREE_SLOT_TIME (10s) for a WR slot, and smc_clcsock_release() tears down the TCP socket.

Quoted 2 lines

2. SMC_MAX_STREAM_WAIT_TIMEOUT is 2 * HZ, i.e. 2 seconds, not 2 minutes.

Thank you for the corrections,Dust. Both points are valid.

Quoted 4 lines

Best regards,
Dust

#7 lore raw
In reply to Sidraya Jayagond
This patch was applied to netdev/net.git (main) by Paolo Abeni <[email protected]>:

Hello:

This patch was applied to netdev/net.git (main)
by Paolo Abeni <[email protected]>:

Quoted 8 lines · On Mon, 3 Aug 2026 09:07:01 +0200 you wrote:

smc_listen_out() reads lsmc->sk.sk_state without the listener lock, then acquires lock_sock_nested() only after the check passes. This opens a window where smc_close_active() can transition the listener to SMC_CLOSED, call smc_close_cleanup_listen() to drain the accept queue, and release the lock, all between the lockless read and the delayed lock acquisition:

[...]

Here is the summary with links:
- [net] net/smc: fix TOCTOU race between smc_listen_out() and listener close

    https://git.kernel.org/netdev/net/c/185a4caeecab

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html