Skip to contentSkip to conversation

nvmet-tcp: two small fixes

PATCH Superseded Geliang Tang 3 participants 6 messages May 19

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover nvmet-tcp: two small fixes

From: Geliang Tang <[email protected]>

During the development of NVMe over MPTCP, two small issues were found in the nvmet-tcp driver. Fix them first.

Geliang Tang (2):
nvmet-tcp: check return value of nvmet_tcp_set_queue_sock
nvmet-tcp: remove duplicate cancel_work_sync in release_queue_work

 drivers/nvme/target/tcp.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH 1/2 nvmet-tcp: check return value of nvmet_tcp_set_queue_sock

From: Geliang Tang <[email protected]>

The return value of nvmet_tcp_set_queue_sock() is currently ignored in nvmet_tcp_tls_handshake_done(). If it fails (e.g., due to the socket not being in TCP_ESTABLISHED state), the socket callbacks will not be properly set, leading to queue and socket leakage.

Fix this by capturing the return value and calling nvmet_tcp_schedule_release_queue() on failure to ensure proper cleanup.

Fixes 675b453e0241 ("nvmet-tcp: enable TLS handshake upcall")
---
 drivers/nvme/target/tcp.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/nvme/target/tcp.c +3 -2
index 164a564ba3b4..8a243d22a511 100644
@@ -1842,10 +1842,11 @@ static void nvmet_tcp_tls_handshake_done(void *data, int status,
 	if (!status)
 		status = nvmet_tcp_tls_key_lookup(queue, peerid);
 
+	if (!status)
+		status = nvmet_tcp_set_queue_sock(queue);
+
 	if (status)
 		nvmet_tcp_schedule_release_queue(queue);
-	else
-		nvmet_tcp_set_queue_sock(queue);
 	kref_put(&queue->kref, nvmet_tcp_release_queue);
 }
 
Signature · 1 line
2.53.0
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] nvmet-tcp: check return value of nvmet_tcp_set_queue_sock

Quoted 24 lines · Geliang Tang

From: Geliang Tang <[email protected]>

The return value of nvmet_tcp_set_queue_sock() is currently ignored in nvmet_tcp_tls_handshake_done(). If it fails (e.g., due to the socket not being in TCP_ESTABLISHED state), the socket callbacks will not be properly set, leading to queue and socket leakage.

Fix this by capturing the return value and calling nvmet_tcp_schedule_release_queue() on failure to ensure proper cleanup.

Fixes 675b453e0241 ("nvmet-tcp: enable TLS handshake upcall")
---
  drivers/nvme/target/tcp.c | 5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)
drivers/nvme/target/tcp.c +0 -0
index 164a564ba3b4..8a243d22a511 100644
@@ -1842,10 +1842,11 @@ static void nvmet_tcp_tls_handshake_done(void *data, int status,
  	if (!status)
  		status = nvmet_tcp_tls_key_lookup(queue, peerid);
 

drivers/nvme/target/tcp.c static void nvmet_tcp_tls_handshake_done(void *data, int status, · line 1852

Quoted 10 lines
drivers/nvme/target/tcp.c +3 -2
+	if (!status)
+		status = nvmet_tcp_set_queue_sock(queue);
+
  	if (status)
  		nvmet_tcp_schedule_release_queue(queue);
-	else
-		nvmet_tcp_set_queue_sock(queue);
  	kref_put(&queue->kref, nvmet_tcp_release_queue);
  }
 

Cheers,

Hannes

Signature · 4 lines
Dr. Hannes Reinecke                  Kernel Storage Architect
[email protected]                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
#3 lore raw

PATCH 2/2 nvmet-tcp: remove duplicate cancel_work_sync in release_queue_work

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

The cancel_work_sync(&queue->io_work) is called twice in nvmet_tcp_release_queue_work(). Remove the second one as it is redundant.

Fixes a208fc5672177 ("nvmet-tcp: fix a race condition between release_queue and io_work")
---
 drivers/nvme/target/tcp.c | 1 -
 1 file changed, 1 deletion(-)
drivers/nvme/target/tcp.c +0 -1
index 8a243d22a511..af7ce384b3cf 100644
@@ -1616,7 +1616,6 @@ static void nvmet_tcp_release_queue_work(struct work_struct *w)
 	nvmet_tcp_uninit_data_in_cmds(queue);
 	nvmet_sq_destroy(&queue->nvme_sq);
 	nvmet_cq_put(&queue->nvme_cq);
-	cancel_work_sync(&queue->io_work);
 	nvmet_tcp_free_cmd_data_in_buffers(queue);
 	/* ->sock will be released by fput() */
 	fput(queue->sock->file);
Signature · 1 line
2.53.0
#4 lore raw

Re: [PATCH 2/2] nvmet-tcp: remove duplicate cancel_work_sync in release_queue_work

Quoted 16 lines · Geliang Tang

From: Geliang Tang <[email protected]>

The cancel_work_sync(&queue->io_work) is called twice in nvmet_tcp_release_queue_work(). Remove the second one as it is redundant.

Fixes a208fc5672177 ("nvmet-tcp: fix a race condition between release_queue and io_work")
---
 drivers/nvme/target/tcp.c | 1 -
 1 file changed, 1 deletion(-)
drivers/nvme/target/tcp.c +0 -0
index 8a243d22a511..af7ce384b3cf 100644

drivers/nvme/target/tcp.c static void nvmet_tcp_release_queue_work(struct work_struct *w) · line 1621

Quoted 7 lines
drivers/nvme/target/tcp.c +0 -1
@@ -1616,7 +1616,6 @@ static void nvmet_tcp_release_queue_work(struct work_struct *w)
 	nvmet_tcp_uninit_data_in_cmds(queue);
 	nvmet_sq_destroy(&queue->nvme_sq);
 	nvmet_cq_put(&queue->nvme_cq);
-	cancel_work_sync(&queue->io_work);
 	nvmet_tcp_free_cmd_data_in_buffers(queue);
 	/* ->sock will be released by fput() */
 	fput(queue->sock->file);

I think that the second cancel_work_sync() is done because nvmet_sq_destroy() can potentially end up calling nvmet_req_complete()

nvmet_sq_destroy() -> nvmet_async_events_failall() -> nvmet_req_complete() -> queue_response()

and this kicks a new io_work.

Maurizio

#5 lore raw

Re: [PATCH 2/2] nvmet-tcp: remove duplicate cancel_work_sync in release_queue_work

Hi Maurizio,

Quoted 32 lines · Maurizio Lombardi
Quoted 25 lines · Geliang Tang

From: Geliang Tang <[email protected]>

The cancel_work_sync(&queue->io_work) is called twice in nvmet_tcp_release_queue_work(). Remove the second one as it is redundant.

Fixes a208fc5672177 ("nvmet-tcp: fix a race condition between

release_queue and io_work")

---
 drivers/nvme/target/tcp.c | 1 -

 1 file changed, 1 deletion(-)
drivers/nvme/target/tcp.c +0 -1
index 8a243d22a511..af7ce384b3cf 100644
@@ -1616,7 +1616,6 @@ static void
 nvmet_tcp_release_queue_work(struct work_struct *w)
  	nvmet_tcp_uninit_data_in_cmds(queue);
  	nvmet_sq_destroy(&queue->nvme_sq);
  	nvmet_cq_put(&queue->nvme_cq);
-	cancel_work_sync(&queue->io_work);
  	nvmet_tcp_free_cmd_data_in_buffers(queue);
  	/* ->sock will be released by fput() */
Quoted 1 line

  fput(queue->sock->file);

I think that the second cancel_work_sync() is done because nvmet_sq_destroy() can potentially end up calling nvmet_req_complete()

Quoted 4 lines

nvmet_sq_destroy() -> nvmet_async_events_failall() -> nvmet_req_complete() -> queue_response()

and this kicks a new io_work.

Thanks for the clarification!

When I was adding code to nvmet_tcp_release_queue_work [1], AI review reported this duplicate cancel_work_sync issue to me.

You are right. I missed that nvmet_sq_destroy() can trigger nvmet_req_complete() -> nvmet_tcp_queue_response(), which queues io_work again. So the second cancel_work_sync() is indeed necessary.

Let's drop this patch.

Thanks,
-Geliang

[1]
https://patchwork.kernel.org/project/mptcp/patch/c796d0260ffebddcaf413f7c5a3d4b3783e814a2.1779159524.git.tanggeliang@kylinos.cn/

Quoted 2 lines

Maurizio