nvme: fix namespace removal list
PATCH 1/1 nvme: fix namespace removal list
From: Keith Busch <[email protected]>
This function wants to move a list element to another list, so do that instead of using the wrong list splice API.
--- drivers/nvme/host/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
drivers/nvme/host/core.c +4 -3
@@ -3959,9 +3959,10 @@ static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl,mutex_lock (&ctrl->namespaces_lock);list_for_each_entry_safe (ns, next, &ctrl->namespaces, list) { -if (ns->head->ns_id > nsid) -list_splice_init_rcu (&ns->list, &rm_list, - synchronize_rcu); +if (ns->head->ns_id > nsid) { +list_del_rcu (&ns->list); +list_add_tail_rcu (&ns->list, &rm_list); + } }mutex_unlock (&ctrl->namespaces_lock);synchronize_srcu (&ctrl->srcu);
Signature · 1 line
2.43.0
Christoph Hellwig
maintainerdrivers/nvme/host/core.c static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl, · line 3965
Quoted 9 lines · Keith Busch
drivers/nvme/host/core.c +4
-3mutex_lock (&ctrl->namespaces_lock);list_for_each_entry_safe (ns, next, &ctrl->namespaces, list) { -if (ns->head->ns_id > nsid) -list_splice_init_rcu (&ns->list, &rm_list, - synchronize_rcu); +if (ns->head->ns_id > nsid) { +list_del_rcu (&ns->list); +list_add_tail_rcu (&ns->list, &rm_list); + }
Is this actually valid for a (S)RCU protected list? If the entry gets added to the new list before the grace period has completed, we could trick a concurrent traversal into following the new list unless I'm mistaken (although chances I'm mistaken on RCU corner cases aren't that low..).
Quoted 18 lines · Christoph Hellwig
drivers/nvme/host/core.c static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl, · line 3965
Quoted 9 lines · Keith Busch
drivers/nvme/host/core.c +4
-3mutex_lock (&ctrl->namespaces_lock);list_for_each_entry_safe (ns, next, &ctrl->namespaces, list) { -if (ns->head->ns_id > nsid) -list_splice_init_rcu (&ns->list, &rm_list, - synchronize_rcu); +if (ns->head->ns_id > nsid) { +list_del_rcu (&ns->list); +list_add_tail_rcu (&ns->list, &rm_list); + }Is this actually valid for a (S)RCU protected list? If the entry gets added to the new list before the grace period has completed, we could trick a concurrent traversal into following the new list unless I'm mistaken (although chances I'm mistaken on RCU corner cases aren't that low..).
I think we need synchronize_srcu() before we add deleted ns to rm_list.
Thanks,
--Nilay
Paul E. McKenney
maintainerQuoted 21 lines · Nilay Shroff
Quoted 18 lines · Christoph Hellwig
drivers/nvme/host/core.c static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl, · line 3965
Quoted 9 lines · Keith Busch
drivers/nvme/host/core.c +4
-3mutex_lock (&ctrl->namespaces_lock);list_for_each_entry_safe (ns, next, &ctrl->namespaces, list) { -if (ns->head->ns_id > nsid) -list_splice_init_rcu (&ns->list, &rm_list, - synchronize_rcu); +if (ns->head->ns_id > nsid) { +list_del_rcu (&ns->list); +list_add_tail_rcu (&ns->list, &rm_list); + }Is this actually valid for a (S)RCU protected list? If the entry gets added to the new list before the grace period has completed, we could trick a concurrent traversal into following the new list unless I'm mistaken (although chances I'm mistaken on RCU corner cases aren't that low..).
Quoted 1 line
I think we need
synchronize_srcu()before we add deleted ns torm_list.
After we delete it, but yes.
There will also be a window during which the element is not in either list, but given the name "rm_list", I am guessing that this is OK. ;-)
Thanx, Paul
Keith Busch
maintainerQuoted 16 lines · Christoph Hellwig
drivers/nvme/host/core.c static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl, · line 3965
Quoted 9 lines · Keith Busch
drivers/nvme/host/core.c +4
-3mutex_lock (&ctrl->namespaces_lock);list_for_each_entry_safe (ns, next, &ctrl->namespaces, list) { -if (ns->head->ns_id > nsid) -list_splice_init_rcu (&ns->list, &rm_list, - synchronize_rcu); +if (ns->head->ns_id > nsid) { +list_del_rcu (&ns->list); +list_add_tail_rcu (&ns->list, &rm_list); + }Is this actually valid for a (S)RCU protected list? If the entry gets added to the new list before the grace period has completed, we could trick a concurrent traversal into following the new list unless I'm mistaken (although chances I'm mistaken on RCU corner cases aren't that low..).
Good call, you are absolutely right that a sync should happen between the del and the add for readers to consistently iterate this list.
I might be able to weasel out of this though: our namespace list is sorted, and this function wants to append everything from this element all the way to the end to the "rm_list": a reader should get the same result either way, whether if it was torn at this element or the move happened without the reader seeing it.
Now if there was a way to rcu split the list so we don't have to do it element by element...
Christoph Hellwig
maintainerQuoted 9 lines · Keith Busch
Quoted 5 lines
Is this actually valid for a (S)RCU protected list? If the entry gets added to the new list before the grace period has completed, we could trick a concurrent traversal into following the new list unless I'm mistaken (although chances I'm mistaken on RCU corner cases aren't that low..).
Good call, you are absolutely right that a sync should happen between the del and the add for readers to consistently iterate this list.
Quoted 8 lines
I might be able to weasel out of this though: our namespace list is sorted, and this function wants to append everything from this element all the way to the end to the "
rm_list": a reader should get the same result either way, whether if it was torn at this element or the move happened without the reader seeing it.Now if there was a way to rcu split the list so we don't have to do it element by element...
Isn't that exactly what the list_splice helpers do? We'd just need to do that exactly once at the cutoff point and not once per namespace.
Paul E. McKenney
maintainerQuoted 16 lines · Christoph Hellwig
Quoted 15 lines · Keith Busch
Quoted 5 lines
Is this actually valid for a (S)RCU protected list? If the entry gets added to the new list before the grace period has completed, we could trick a concurrent traversal into following the new list unless I'm mistaken (although chances I'm mistaken on RCU corner cases aren't that low..).
Good call, you are absolutely right that a sync should happen between the del and the add for readers to consistently iterate this list.
I might be able to weasel out of this though: our namespace list is sorted, and this function wants to append everything from this element all the way to the end to the "
rm_list": a reader should get the same result either way, whether if it was torn at this element or the move happened without the reader seeing it.
Quoted 5 lines
Quoted 2 lines
Now if there was a way to rcu split the list so we don't have to do it element by element...
Isn't that exactly what the
list_splicehelpers do? We'd just need to do that exactly once at the cutoff point and not once per namespace.
Almost. They move the whole list and Keith wants just part of it moved. It should not be hard to create such a thing, and Keith said (offlist) that he would like to give it a try.
Thanx, Paul
Keith Busch
maintainerQuoted 16 lines · Christoph Hellwig
Quoted 15 lines · Keith Busch
Quoted 5 lines
Is this actually valid for a (S)RCU protected list? If the entry gets added to the new list before the grace period has completed, we could trick a concurrent traversal into following the new list unless I'm mistaken (although chances I'm mistaken on RCU corner cases aren't that low..).
Good call, you are absolutely right that a sync should happen between the del and the add for readers to consistently iterate this list.
I might be able to weasel out of this though: our namespace list is sorted, and this function wants to append everything from this element all the way to the end to the "
rm_list": a reader should get the same result either way, whether if it was torn at this element or the move happened without the reader seeing it.
Quoted 5 lines
Quoted 2 lines
Now if there was a way to rcu split the list so we don't have to do it element by element...
Isn't that exactly what the
list_splicehelpers do? We'd just need to do that exactly once at the cutoff point and not once per namespace.
Not a splice since that takes the entire source list; this usage is more aligned to "list_cut_position()", which doesn't look rcu safe as-is, but I think we can make one!
Paul E. McKenney
maintainerQuoted 16 lines · Christoph Hellwig
drivers/nvme/host/core.c static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl, · line 3965
Quoted 9 lines · Keith Busch
drivers/nvme/host/core.c +4
-3mutex_lock (&ctrl->namespaces_lock);list_for_each_entry_safe (ns, next, &ctrl->namespaces, list) { -if (ns->head->ns_id > nsid) -list_splice_init_rcu (&ns->list, &rm_list, - synchronize_rcu); +if (ns->head->ns_id > nsid) { +list_del_rcu (&ns->list); +list_add_tail_rcu (&ns->list, &rm_list); + }Is this actually valid for a (S)RCU protected list? If the entry gets added to the new list before the grace period has completed, we could trick a concurrent traversal into following the new list unless I'm mistaken (although chances I'm mistaken on RCU corner cases aren't that low..).
Just to confirm, yes, if an RCU reader was referencing the object being deleted from the first list, that reader really could be transported (free of charge!) to the second list.
One (slow) way to avoid this involuntary transportation is to do something like this:
list_del_rcu(&ns->list); synchronize_rcu(); // drain all readers from ns. list_add_tail_rcu(&ns->list, &rm_list);
You could speed things up using synchronize_rcu_expedited, with the usual downsides.
There are faster ways of getting this effect, but the ones that I know of involve putting generation numbers on all objects, duplicating objects on each change, and having readers chase pointers to find the desired version. Usually not worth it, but I don't know enough about this code to judge.
Thanx, Paul