blk-mq: sync the update nr_hw_queues with blk_mq_queue_tag_busy_iter
Subsystems (2)
PATCH 1/1 blk-mq: sync the update nr_hw_queues with blk_mq_queue_tag_busy_iter
From: Jianchao Wang <[email protected]>
commit f5bbbbe4d63577026f908a809f22f5fd5a90ea1f upstream.
For blk-mq, part_in_flight/rw will invoke blk_mq_in_flight/rw to account the inflight requests. It will access the queue_hw_ctx and nr_hw_queues w/o any protection. When updating nr_hw_queues and blk_mq_in_flight/rw occur concurrently, panic comes up.
Before update nr_hw_queues, the q will be frozen. So we could use q_usage_counter to avoid the race. percpu_ref_is_zero is used here so that we will not miss any in-flight request. The access to nr_hw_queues and queue_hw_ctx in blk_mq_queue_tag_busy_iter are under rcu critical section, __blk_mq_update_nr_hw_queues could use synchronize_rcu to ensure the zeroed q_usage_counter to be globally visible.
Backporting Notes
This is a re-backport, landing synchronize_rcu in the right place.
--- block/blk-mq.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
block/blk-mq.c +4 -4
@@ -2331,6 +2331,10 @@ void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues)list_for_each_entry (q, &set->tag_list, tag_set_list)blk_mq_freeze_queue (q); +/* +* Sync with blk_mq_queue_tag_busy_iter. +*/ +synchronize_rcu (); set->nr_hw_queues = nr_hw_queues;list_for_each_entry (q, &set->tag_list, tag_set_list) {
@@ -2346,10 +2350,6 @@ void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues)list_for_each_entry (q, &set->tag_list, tag_set_list)blk_mq_unfreeze_queue (q); -/* -* Sync with blk_mq_queue_tag_busy_iter. -*/ -synchronize_rcu (); }EXPORT_SYMBOL_GPL (blk_mq_update_nr_hw_queues);
Signature · 1 line
2.27.0.278.ge193c7cf3a9-goog
Greg KH
maintainerQuoted 17 lines · Giuliano Procida
From: Jianchao Wang <[email protected]>
commit
f5bbbbe4d63577026f908a809f22f5fd5a90ea1fupstream.For blk-mq, part_in_flight/rw will invoke blk_mq_in_flight/rw to account the inflight requests. It will access the
queue_hw_ctxandnr_hw_queuesw/o any protection. When updatingnr_hw_queuesand blk_mq_in_flight/rw occur concurrently, panic comes up.Before update
nr_hw_queues, the q will be frozen. So we could useq_usage_counterto avoid the race.percpu_ref_is_zerois used here so that we will not miss any in-flight request. The access tonr_hw_queuesandqueue_hw_ctxinblk_mq_queue_tag_busy_iterare under rcu critical section,__blk_mq_update_nr_hw_queuescould usesynchronize_rcuto ensure the zeroedq_usage_counterto be globally visible.
Quoted 3 lines
Backporting Notes
This is a re-backport, landing
synchronize_rcuin the right place.
You sent this twice?
And what stable kernel(s) does it go to?
thanks,
greg k-h
-bouncing mail.
That wasn't clever. Apologies.
This one is for 4.9.
Giuliano.
Quoted 30 lines · Greg KH
Quoted 17 lines · Giuliano Procida
From: Jianchao Wang <[email protected]>
commit
f5bbbbe4d63577026f908a809f22f5fd5a90ea1fupstream.For blk-mq, part_in_flight/rw will invoke blk_mq_in_flight/rw to account the inflight requests. It will access the
queue_hw_ctxandnr_hw_queuesw/o any protection. When updatingnr_hw_queuesand blk_mq_in_flight/rw occur concurrently, panic comes up.Before update
nr_hw_queues, the q will be frozen. So we could useq_usage_counterto avoid the race.percpu_ref_is_zerois used here so that we will not miss any in-flight request. The access tonr_hw_queuesandqueue_hw_ctxinblk_mq_queue_tag_busy_iterare under rcu critical section,__blk_mq_update_nr_hw_queuescould usesynchronize_rcuto ensure the zeroedq_usage_counterto be globally visible.Quoted 3 lines
Backporting Notes
This is a re-backport, landing
synchronize_rcuin the right place.You sent this twice?
And what stable kernel(s) does it go to?
thanks,
greg k-h