Skip to contentSkip to conversation

nvmet-tcp: fix memory leak when having inflight commands on disconnect

PATCH In review [email protected] 3 participants 4 messages May 24, 2021 – May 27, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvmet-tcp: fix memory leak when having inflight commands on disconnect

From: Elad Grupi <[email protected]>

Some nvme commands might get completed during the execution of nvmet_tcp_release_queue_work. Those commands are being added to resp_list on the queue and never processed after cancel_work_sync is done, causing memory leak of the resources allocated for those commands.

---
 drivers/nvme/target/tcp.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
drivers/nvme/target/tcp.c +28 -0
index f9f34f6caf5e..3c6e36c6ed4a 100644
@@ -1426,6 +1426,33 @@ static void nvmet_tcp_uninit_data_in_cmds(struct nvmet_tcp_queue *queue)
 	}
 }
 
+static void nvmet_tcp_free_resp_list(struct nvmet_tcp_queue *queue)
+{
+	struct list_head *p;
+	struct nvmet_tcp_cmd *cmd;
+	int c = 0;
+
+	nvmet_tcp_process_resp_list(queue);
+
+	list_for_each(p, &queue->resp_send_list) {
+		cmd = list_entry(p, struct nvmet_tcp_cmd, entry);
+		kfree(cmd->iov);
+		sgl_free(cmd->req.sg);
+		c++;
+	}
+
+	WARN_ON(c != queue->send_list_len);
+
+	if (queue->snd_cmd) {
+		kfree(queue->snd_cmd->iov);
+		sgl_free(queue->snd_cmd->req.sg);
+		c++;
+	}
+
+	pr_debug("qid %u send_list_len %d, free %d unprocessed commands\n",
+			queue->nvme_sq.qid, queue->send_list_len, c);
+}
+
 static void nvmet_tcp_release_queue_work(struct work_struct *w)
 {
 	struct nvmet_tcp_queue *queue =
@@ -1441,6 +1468,7 @@ static void nvmet_tcp_release_queue_work(struct work_struct *w)
 	nvmet_tcp_uninit_data_in_cmds(queue);
 	nvmet_sq_destroy(&queue->nvme_sq);
 	cancel_work_sync(&queue->io_work);
+	nvmet_tcp_free_resp_list(queue);
 	sock_release(queue->sock);
 	nvmet_tcp_free_cmds(queue);
 	if (queue->hdr_digest || queue->data_digest)
Signature · 7 lines
2.18.2


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
maintainer
#1 lore raw

drivers/nvme/target/tcp.c static void nvmet_tcp_uninit_data_in_cmds(struct nvmet_tcp_queue *queue) · line 1439

Quoted 12 lines
drivers/nvme/target/tcp.c +11 -0
+static void nvmet_tcp_free_resp_list(struct nvmet_tcp_queue *queue)
+{
+	struct list_head *p;
+	struct nvmet_tcp_cmd *cmd;
+	int c = 0;
+
+	nvmet_tcp_process_resp_list(queue);
+
+	list_for_each(p, &queue->resp_send_list) {
+		cmd = list_entry(p, struct nvmet_tcp_cmd, entry);
+		kfree(cmd->iov);

PDUs need to be unmapped as well in case of writes. That is not handled in nvmet_tcp_uninit_data_in_cmds ? Or is it just reads (data-out commands)?

Quoted 21 lines
drivers/nvme/target/tcp.c +16 -0
+		sgl_free(cmd->req.sg);
+		c++;
+	}
+
+	WARN_ON(c != queue->send_list_len);
+
+	if (queue->snd_cmd) {
+		kfree(queue->snd_cmd->iov);
+		sgl_free(queue->snd_cmd->req.sg);
+		c++;
+	}
+
+	pr_debug("qid %u send_list_len %d, free %d unprocessed commands\n",
+			queue->nvme_sq.qid, queue->send_list_len, c);
+}
+
  static void nvmet_tcp_release_queue_work(struct work_struct *w)
  {
  	struct nvmet_tcp_queue *queue =

drivers/nvme/target/tcp.c static void nvmet_tcp_release_queue_work(struct work_struct *w) · line 1474

Quoted 7 lines
drivers/nvme/target/tcp.c +1 -0
@@ -1441,6 +1468,7 @@ static void nvmet_tcp_release_queue_work(struct work_struct *w)
  	nvmet_tcp_uninit_data_in_cmds(queue);
  	nvmet_sq_destroy(&queue->nvme_sq);
  	cancel_work_sync(&queue->io_work);
+	nvmet_tcp_free_resp_list(queue);
  	sock_release(queue->sock);
  	nvmet_tcp_free_cmds(queue);
  	if (queue->hdr_digest || queue->data_digest)

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#2 lore raw
In reply to [email protected]

drivers/nvme/target/tcp.c static void nvmet_tcp_uninit_data_in_cmds(struct nvmet_tcp_queue *queue) · line 1438

Quoted 12 lines · On Mon, May 24, 2021 at 02:24:41PM +0300, [email protected] wrote:

}

drivers/nvme/target/tcp.c +10 -0
+static void nvmet_tcp_free_resp_list(struct nvmet_tcp_queue *queue)
+{
+	struct list_head *p;
+	struct nvmet_tcp_cmd *cmd;
+	int c = 0;
+
+	nvmet_tcp_process_resp_list(queue);
+
+	list_for_each(p, &queue->resp_send_list) {
+		cmd = list_entry(p, struct nvmet_tcp_cmd, entry);

Please use list_for_each_entry instead.

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#3 lore raw

Sure. I will prepare an update to the patch today.

Thanks,
Elad

-----Original Message-----
From: Christoph Hellwig <[email protected]>
Sent: Wednesday, 26 May 2021 17:05
To: Grupi, Elad
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] nvmet-tcp: fix memory leak when having inflight commands on disconnect

[EXTERNAL EMAIL]

drivers/nvme/target/tcp.c static void nvmet_tcp_uninit_data_in_cmds(struct nvmet_tcp_queue *queue) · line 1438

Quoted 11 lines · On Mon, May 24, 2021 at 02:24:41PM +0300, [email protected] wrote:

}

drivers/nvme/target/tcp.c +9 -0
+static void nvmet_tcp_free_resp_list(struct nvmet_tcp_queue *queue) {
+	struct list_head *p;
+	struct nvmet_tcp_cmd *cmd;
+	int c = 0;
+
+	nvmet_tcp_process_resp_list(queue);
+
+	list_for_each(p, &queue->resp_send_list) {
+		cmd = list_entry(p, struct nvmet_tcp_cmd, entry);

Please use list_for_each_entry instead.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme