Skip to contentSkip to conversation

nvme: Do not detach nshead when a namespace is removed

PATCH In review Hannes Reinecke 2 participants 2 messages Mar 11, 2021 – Mar 15, 2021 NVMe

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvme: Do not detach nshead when a namespace is removed

'struct nshead' and 'struct ns' have different lifetimes; the nshead can (and occasionally will) stay around even if no namespaces are connected, as it will only be finally removed once the last opener is gone.
Once the system enters this state it becomes very hard to recover, as the only way to remove a namespace is to disconnect the controller, but the nshead is precisely _not_ related to a controller. This leads to the very annoying behaviour that a subsequent 'nvme connect' call will establish a _different_ nshead, _and_ the previous nshead will stay around:

Quoted 1 line

nvme list

/dev/nvme0n1     9329c4c4af01db2c     Linux \
    1          17.18  GB /  17.18  GB      4 KiB +  0 B   5.3.18-4
Quoted 2 lines

nvme disconnect -d /dev/nvme0
nvme list

/dev/nvme0n1                                \
    -1          0.00   B /   0.00   B      1   B +  0 B   n1
Quoted 2 lines

nvme connect
nvme list

/dev/nvme0n1     9329c4c4af01db2c     Linux \
    -1          0.00   B /   0.00   B      1   B +  0 B   5.3.18-4
/dev/nvme0n2     9329c4c4af01db2c     Linux \
    1          17.18  GB /  17.18  GB      4 KiB +  0 B   5.3.18-4

This patch fixes the situation by only removing the nshead from the internal lists once it's being finally removed.

---
 drivers/nvme/host/core.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/nvme/host/core.c +4 -4
index 206e6f883ca8..20fa62ae297a 100644
@@ -542,6 +542,10 @@ static void nvme_free_ns_head(struct kref *ref)
 	struct nvme_ns_head *head =
 		container_of(ref, struct nvme_ns_head, ref);
 
+	mutex_lock(&head->subsys->lock);
+	list_del_init(&head->entry);
+	mutex_unlock(&head->subsys->lock);
+
 	nvme_mpath_remove_disk(head);
 	ida_simple_remove(&head->subsys->ns_ida, head->instance);
 	cleanup_srcu_struct(&head->srcu);
@@ -3983,8 +3987,6 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid,
  out_unlink_ns:
 	mutex_lock(&ctrl->subsys->lock);
 	list_del_rcu(&ns->siblings);
-	if (list_empty(&ns->head->list))
-		list_del_init(&ns->head->entry);
 	mutex_unlock(&ctrl->subsys->lock);
 	nvme_put_ns_head(ns->head);
  out_free_queue:
@@ -4005,8 +4007,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
 
 	mutex_lock(&ns->ctrl->subsys->lock);
 	list_del_rcu(&ns->siblings);
-	if (list_empty(&ns->head->list))
-		list_del_init(&ns->head->entry);
 	mutex_unlock(&ns->ctrl->subsys->lock);
 
 	synchronize_rcu(); /* guarantee not available in head->list */
Signature · 7 lines
2.29.2


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
maintainer
#1 lore raw
Quoted 21 lines

'struct nshead' and 'struct ns' have different lifetimes; the nshead can (and occasionally will) stay around even if no namespaces are connected, as it will only be finally removed once the last opener is gone.
Once the system enters this state it becomes very hard to recover, as the only way to remove a namespace is to disconnect the controller, but the nshead is precisely _not_ related to a controller. This leads to the very annoying behaviour that a subsequent 'nvme connect' call will establish a _different_ nshead, _and_ the previous nshead will stay around:

Quoted 1 line

nvme list

/dev/nvme0n1     9329c4c4af01db2c     Linux \
     1          17.18  GB /  17.18  GB      4 KiB +  0 B   5.3.18-4
Quoted 2 lines

nvme disconnect -d /dev/nvme0
nvme list

/dev/nvme0n1                                \
     -1          0.00   B /   0.00   B      1   B +  0 B   n1
Quoted 2 lines

nvme connect
nvme list

Quoted 7 lines
/dev/nvme0n1     9329c4c4af01db2c     Linux \
     -1          0.00   B /   0.00   B      1   B +  0 B   5.3.18-4
/dev/nvme0n2     9329c4c4af01db2c     Linux \
     1          17.18  GB /  17.18  GB      4 KiB +  0 B   5.3.18-4

This patch fixes the situation by only removing the nshead from the internal lists once it's being finally removed.

I'd say that we should remove the nshead exactly when the last lower ns was removed? Why should it still occupy sysfs and ida resources if it is going away soon?

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme