nvme: allow to re-attach namespaces after all paths are down
PATCH v4 1/1 nvme: allow to re-attach namespaces after all paths are down
We should only remove the ns head from the list of heads per subsystem if the reference count drops to zero. Removing it at the start of nvme_ns_remove() will prevent us from reattaching the namespace to the correct ns head once a path becomes available again.
Changes to v3:
- Simplify if() clause to detect duplicate namespaces Changes to v2:
- Drop memcpy() statement
Changes to v1:
- Always check NSIDs after reattach
--- drivers/nvme/host/core.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
drivers/nvme/host/core.c +8 -3
@@ -546,6 +546,9 @@ static void nvme_free_ns_head(struct kref *ref)struct nvme_ns_head *head = container_of(ref, struct nvme_ns_head, ref); +mutex_lock (&head->subsys->lock); +list_del_init (&head->entry); +mutex_unlock (&head->subsys->lock);nvme_mpath_remove_disk (head);ida_simple_remove (&head->subsys->ns_ida, head->instance);cleanup_srcu_struct (&head->srcu);
@@ -3605,7 +3608,11 @@ static int nvme_init_ns_head(struct nvme_ns *ns, unsigned nsid, head->shared = is_shared; }else { ret = -EINVAL ; -if (!is_shared || !head->shared) { +/* +* If multipath is enabled we might hit an ns head with no +* paths, but that doesn't indicate it's a shared namespace. +*/ +if (!list_empty (&head->list) && (!is_shared || !head->shared)) {dev_err (ctrl->device,"Duplicate unshared namespace %d\n" , nsid); goto out_put_ns_head;
@@ -3764,8 +3771,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)mutex_lock (&ns->ctrl->subsys->lock);list_del_rcu (&ns->siblings); -if (list_empty (&ns->head->list)) -list_del_init (&ns->head->entry);mutex_unlock (&ns->ctrl->subsys->lock);synchronize_rcu ();/* guarantee not available in head->list */
Signature · 7 lines
2.29.2 _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Quoted 5 lines · Hannes Reinecke
We should only remove the ns head from the list of heads per subsystem if the reference count drops to zero. Removing it at the start of
nvme_ns_remove()will prevent us from reattaching the namespace to the correct ns head once a path becomes available again.
This looks fine to me
There are potential corner cases associated with namespace management that can cause problems here, but it would certainly be a user error that needs manual intervention anyway. I suspect those scenarios are very uncommon too, so I think we can deal with that later if it becomes a problem.
Quoted 34 lines
Signed-off-by Hannes Reinecke <[email protected]>--- drivers/nvme/host/core.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)drivers/nvme/host/core.c +8
-1@@ -546,6 +546,9 @@ static void nvme_free_ns_head(struct kref *ref)struct nvme_ns_head *head = container_of(ref, struct nvme_ns_head, ref); +mutex_lock (&head->subsys->lock); +list_del_init (&head->entry); +mutex_unlock (&head->subsys->lock);nvme_mpath_remove_disk (head);ida_simple_remove (&head->subsys->ns_ida, head->instance);cleanup_srcu_struct (&head->srcu);@@ -3605,7 +3608,11 @@ static int nvme_init_ns_head(struct nvme_ns *ns, unsigned nsid, head->shared = is_shared; }else { ret = -EINVAL ; -if (!is_shared || !head->shared) { +/* +* If multipath is enabled we might hit an ns head with no +* paths, but that doesn't indicate it's a shared namespace. +*/ +if (!list_empty (&head->list) && (!is_shared || !head->shared)) {dev_err (ctrl->device,"Duplicate unshared namespace %d\n" , nsid); goto out_put_ns_head;
drivers/nvme/host/core.c static void nvme_ns_remove(struct nvme_ns *ns) · line 3776
Quoted 8 lines
drivers/nvme/host/core.c +0
-2@@ -3764,8 +3771,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)mutex_lock (&ns->ctrl->subsys->lock);list_del_rcu (&ns->siblings); -if (list_empty (&ns->head->list)) -list_del_init (&ns->head->entry);mutex_unlock (&ns->ctrl->subsys->lock);synchronize_rcu ();/* guarantee not available in head->list */
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Quoted 9 lines · Keith Busch
Quoted 5 lines · Hannes Reinecke
We should only remove the ns head from the list of heads per subsystem if the reference count drops to zero. Removing it at the start of
nvme_ns_remove()will prevent us from reattaching the namespace to the correct ns head once a path becomes available again.This looks fine to me
Quoted 8 lines
Reviewed-by Keith Busch <[email protected]>There are potential corner cases associated with namespace management that can cause problems here, but it would certainly be a user error that needs manual intervention anyway. I suspect those scenarios are very uncommon too, so I think we can deal with that later if it becomes a problem.
How so?
We're keeping a copy of the NS IDs around, so after reconnect the namespace would have to provide same set of NS IDs.
If the namespace is different yet providing the same NS IDs we'll have a problem even with the current code, as we're using the ND ID to identify multipathed namespaces.
Or do you have a different scenario in mind?
Cheers,
Hannes
Signature · 9 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions Germany GmbH, 90409 Nürnberg GF: F. Imendörffer, HRB 36809 (AG Nürnberg) _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Keith Busch
maintainerQuoted 22 lines · Hannes Reinecke
Quoted 9 lines · Keith Busch
Quoted 5 lines · Hannes Reinecke
We should only remove the ns head from the list of heads per subsystem if the reference count drops to zero. Removing it at the start of
nvme_ns_remove()will prevent us from reattaching the namespace to the correct ns head once a path becomes available again.This looks fine to me
Quoted 8 lines
Reviewed-by Keith Busch <[email protected]>There are potential corner cases associated with namespace management that can cause problems here, but it would certainly be a user error that needs manual intervention anyway. I suspect those scenarios are very uncommon too, so I think we can deal with that later if it becomes a problem.
How so?
We're keeping a copy of the NS IDs around, so after reconnect the namespace would have to provide same set of NS IDs.
Quoted 5 lines
If the namespace is different yet providing the same NS IDs we'll have a problem even with the current code, as we're using the ND ID to identify multipathed namespaces.
Or do you have a different scenario in mind?
Sorry, I didn't mean to imply this creates a new problem. It's already a problem. :)
This just provides a chance for a recycled ID for a newly created namespace to attach to an in-use head associated with a previously deleted namespace. I am not really concerned about this scenario, though, since it wouldn't work today anyway.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Sagi Grimberg
maintainerQuoted 28 lines
Quoted 22 lines
Quoted 10 lines · Hannes Reinecke
Quoted 5 lines
We should only remove the ns head from the list of heads per subsystem if the reference count drops to zero. Removing it at the start of
nvme_ns_remove()will prevent us from reattaching the namespace to the correct ns head once a path becomes available again.This looks fine to me
Quoted 8 lines
Reviewed-by Keith Busch <[email protected]>There are potential corner cases associated with namespace management that can cause problems here, but it would certainly be a user error that needs manual intervention anyway. I suspect those scenarios are very uncommon too, so I think we can deal with that later if it becomes a problem.
How so?
We're keeping a copy of the NS IDs around, so after reconnect the namespace would have to provide same set of NS IDs.Quoted 5 lines
If the namespace is different yet providing the same NS IDs we'll have a problem even with the current code, as we're using the ND ID to identify multipathed namespaces.
Or do you have a different scenario in mind?
Quoted 7 lines
Sorry, I didn't mean to imply this creates a new problem. It's already a problem. :)
This just provides a chance for a recycled ID for a newly created namespace to attach to an in-use head associated with a previously deleted namespace. I am not really concerned about this scenario, though, since it wouldn't work today anyway.
That is because we keep the device node around even though it doesn't have any bottom devices. Which is what I think Hannes tried to fix in an earlier patch.
Also, if an NVM subsystem happens to recycle nsids, it will trigger pretty surely. So I'm not sure this is an obscure case...
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Keith Busch
maintainerQuoted 36 lines · Sagi Grimberg
Quoted 28 lines
Quoted 22 lines
Quoted 10 lines · Hannes Reinecke
Quoted 5 lines
We should only remove the ns head from the list of heads per subsystem if the reference count drops to zero. Removing it at the start of
nvme_ns_remove()will prevent us from reattaching the namespace to the correct ns head once a path becomes available again.This looks fine to me
Quoted 8 lines
Reviewed-by Keith Busch <[email protected]>There are potential corner cases associated with namespace management that can cause problems here, but it would certainly be a user error that needs manual intervention anyway. I suspect those scenarios are very uncommon too, so I think we can deal with that later if it becomes a problem.
How so?
We're keeping a copy of the NS IDs around, so after reconnect the namespace would have to provide same set of NS IDs.Quoted 5 lines
If the namespace is different yet providing the same NS IDs we'll have a problem even with the current code, as we're using the ND ID to identify multipathed namespaces.
Or do you have a different scenario in mind?
Quoted 7 lines
Sorry, I didn't mean to imply this creates a new problem. It's already a problem. :)
This just provides a chance for a recycled ID for a newly created namespace to attach to an in-use head associated with a previously deleted namespace. I am not really concerned about this scenario, though, since it wouldn't work today anyway.
Quoted 6 lines
That is because we keep the device node around even though it doesn't have any bottom devices. Which is what I think Hannes tried to fix in an earlier patch.
Also, if an NVM subsystem happens to recycle nsids, it will trigger pretty surely. So I'm not sure this is an obscure case...
I'm not very concerned because (1) deleting an in-use namespace is surely a user error, and (2) it currently doesn't work either, and no one's complained yet AFAIK.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Sagi Grimberg
maintainerQuoted 11 lines
Quoted 7 lines
That is because we keep the device node around even though it doesn't have any bottom devices. Which is what I think Hannes tried to fix in an earlier patch.
Also, if an NVM subsystem happens to recycle nsids, it will trigger pretty surely. So I'm not sure this is an obscure case...
I'm not very concerned because (1) deleting an in-use namespace is surely a user error, and (2) it currently doesn't work either, and no one's complained yet AFAIK.
You're not wrong. It's just that one error (on ns X) can manifest in other potentially completely different namespaces created. I guess this kind of thing can happen when ns administration is done by adminX (or scriptX) and the ns mounting is done by userY where what each is doing is unknown to each other.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Christoph Hellwig
maintainerQuoted 12 lines · Sagi Grimberg
Quoted 11 lines
Quoted 7 lines
That is because we keep the device node around even though it doesn't have any bottom devices. Which is what I think Hannes tried to fix in an earlier patch.
Also, if an NVM subsystem happens to recycle nsids, it will trigger pretty surely. So I'm not sure this is an obscure case...
I'm not very concerned because (1) deleting an in-use namespace is surely a user error, and (2) it currently doesn't work either, and no one's complained yet AFAIK.
Quoted 5 lines
You're not wrong. It's just that one error (on ns X) can manifest in other potentially completely different namespaces created. I guess this kind of thing can happen when ns administration is done by adminX (or scriptX) and the ns mounting is done by userY where what each is doing is unknown to each other.
We also check for the matching identifiers in nvme_init_ns_head, so we'll error out there.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
Quoted 12 lines · Sagi Grimberg
Quoted 11 lines
Quoted 7 lines
That is because we keep the device node around even though it doesn't have any bottom devices. Which is what I think Hannes tried to fix in an earlier patch.
Also, if an NVM subsystem happens to recycle nsids, it will trigger pretty surely. So I'm not sure this is an obscure case...
I'm not very concerned because (1) deleting an in-use namespace is surely a user error, and (2) it currently doesn't work either, and no one's complained yet AFAIK.
Quoted 5 lines
You're not wrong. It's just that one error (on ns X) can manifest in other potentially completely different namespaces created. I guess this kind of thing can happen when ns administration is done by adminX (or scriptX) and the ns mounting is done by userY where what each is doing is unknown to each other.
This can only happen if NSFEAT bit 3 is not set and the namespace does not provide a UUID. So if we are concerned about that we can restrict nvme multipathing to exclude these scenarios. But in all other cases CNS 03 _is_ identifying the namespace, and cannot be re-used for other namespaces. And we are keeping a copy of CNS 03 in nshead, so we'll be safe there.
Cheers,
Hannes
Signature · 9 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer _______________________________________________ Linux-nvme mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-nvme
Thanks,
applied to nvme-5.14.
_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme