Skip to contentSkip to conversation

nvme-host tcp: check if the queue is allocated before stopping it

PATCH Applied Maurizio Lombardi 3 participants 4 messages Aug 1, 2022 – Aug 10, 2022 NVMe

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvme-host tcp: check if the queue is allocated before stopping it

When an error is detected and the host reconnects, the nvme_tcp_error_recovery_work() function is called and starts tearing down the io queues and de-allocating them;
If at the same time the "nvme" process deletes the controller via sysfs, the nvme_tcp_delete_ctrl() gets called and waits until the nvme_tcp_error_recovery_work() finishes its job; then starts tearing down the io queues, but at this point they have already been freed and the mutexes are destroyed.

Calling mutex_lock() against a destroyed mutex triggers a warning:

[ 1299.025575] nvme nvme1: Reconnecting in 10 seconds...
[ 1299.636449] nvme nvme1: Removing ctrl: NQN "blktests-subsystem-1"
[ 1299.645262] ------------[ cut here ]------------
[ 1299.649949] DEBUG_LOCKS_WARN_ON(lock->magic != lock)
[ 1299.649971] WARNING: CPU: 4 PID: 104150 at kernel/locking/mutex.c:579 __mutex_lock+0x2d0/0x7dc
[ 1299.717934] CPU: 4 PID: 104150 Comm: nvme
[ 1299.828075] Call trace:
[ 1299.830526]  __mutex_lock+0x2d0/0x7dc
[ 1299.834203]  mutex_lock_nested+0x64/0xd4
[ 1299.838139]  nvme_tcp_stop_queue+0x54/0xe0 [nvme_tcp]
[ 1299.843211]  nvme_tcp_teardown_io_queues.part.0+0x90/0x280 [nvme_tcp]
[ 1299.849672]  nvme_tcp_delete_ctrl+0x6c/0xf0 [nvme_tcp]
[ 1299.854831]  nvme_do_delete_ctrl+0x108/0x120 [nvme_core]
[ 1299.860181]  nvme_sysfs_delete+0xec/0xf0 [nvme_core]
[ 1299.865179]  dev_attr_store+0x40/0x70

Fix the warning by checking if the queues are allocated in the nvme_tcp_stop_queue(). If they are not, it makes no sense to try to stop them.

---
 drivers/nvme/host/tcp.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/nvme/host/tcp.c +3 -0
index ac74dce3e95b..ab9bb5c98c78 100644
@@ -1619,6 +1619,9 @@ static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
 	struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
 	struct nvme_tcp_queue *queue = &ctrl->queues[qid];
 
+	if (!test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
+		return;
+
 	mutex_lock(&queue->queue_lock);
 	if (test_and_clear_bit(NVME_TCP_Q_LIVE, &queue->flags))
 		__nvme_tcp_stop_queue(queue);
Signature · 1 line
2.31.1
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 6 lines
drivers/nvme/host/tcp.c +0 -0
index ac74dce3e95b..ab9bb5c98c78 100644

drivers/nvme/host/tcp.c static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid) · line 1627

Quoted 9 lines
drivers/nvme/host/tcp.c +3 -0
@@ -1619,6 +1619,9 @@ static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
  	struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
  	struct nvme_tcp_queue *queue = &ctrl->queues[qid];
 
+	if (!test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
+		return;
+
  	mutex_lock(&queue->queue_lock);
  	if (test_and_clear_bit(NVME_TCP_Q_LIVE, &queue->flags))
  		__nvme_tcp_stop_queue(queue);

This looks correct to me,

maintainer
#2 lore raw
Quoted 19 lines
Quoted 7 lines
drivers/nvme/host/tcp.c +0 -0
index ac74dce3e95b..ab9bb5c98c78 100644

drivers/nvme/host/tcp.c static void nvme_tcp_stop_queue(struct nvme_ctrl · line 1627

Quoted 9 lines
drivers/nvme/host/tcp.c +3 -0
@@ -1619,6 +1619,9 @@ static void nvme_tcp_stop_queue(struct nvme_ctrl 
 *nctrl, int qid)
       struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
       struct nvme_tcp_queue *queue = &ctrl->queues[qid];
+    if (!test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
+        return;
+
       mutex_lock(&queue->queue_lock);
       if (test_and_clear_bit(NVME_TCP_Q_LIVE, &queue->flags))
           __nvme_tcp_stop_queue(queue);

This looks correct to me,

Quoted 1 line

Just one nit,

The patch title should start with nvme-tcp: and not nvme-host tcp:
This can be easily addressed when applying the patch