Skip to contentSkip to conversation

blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

PATCH Superseded Ming Lei 5 participants 15 messages Aug 25, 2020 – Sep 3, 2020 Block layer

v2 of 8 Superseded Latest is v8 →

lore
Share & tools
lore

Ming Lei

author
#0 lore raw

v2 cover blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

Hi Jens,

The 1st patch add .mq_quiesce_mutex for serializing quiesce/unquiesce, and prepares for replacing srcu with percpu_ref.

The 2nd patch replaces srcu with percpu_ref.

V2:
- add .mq_quiesce_lock
- add comment on patch 2 wrt. handling hctx_lock() failure
- trivial patch style change

Ming Lei (2):
blk-mq: serialize queue quiesce and unquiesce by mutex
blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

 block/blk-core.c       |   2 +
 block/blk-mq-sysfs.c   |   2 -
 block/blk-mq.c         | 125 +++++++++++++++++++++++------------------
 block/blk-sysfs.c      |   6 +-
 include/linux/blk-mq.h |   7 ---
 include/linux/blkdev.h |   6 ++
 6 files changed, 82 insertions(+), 66 deletions(-)
Cc Lai Jiangshan <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Sagi Grimberg <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Chao Leng <[email protected]>
Cc Christoph Hellwig <[email protected]>
Signature · 1 line
2.25.2
author
#1 lore raw

PATCH v2 1/2 blk-mq: serialize queue quiesce and unquiesce by mutex

Add .mq_quiesce_mutext to request queue, so that queue quiesce and unquiesce can be serialized. Meantime we can avoid unnecessary synchronize_rcu() in case that queue has been quiesced already.

Cc Sagi Grimberg <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Chao Leng <[email protected]>
---
 block/blk-core.c       |  2 ++
 block/blk-mq.c         | 11 +++++++++++
 include/linux/blkdev.h |  2 ++
 3 files changed, 15 insertions(+)
block/blk-core.c +2 -0
index d9d632639bd1..ffc57df70064 100644
@@ -561,6 +561,8 @@ struct request_queue *blk_alloc_queue(int node_id)
 	init_waitqueue_head(&q->mq_freeze_wq);
 	mutex_init(&q->mq_freeze_lock);
 
+	mutex_init(&q->mq_quiesce_lock);
+
 	/*
 	 * Init percpu_ref in atomic mode so that it's faster to shutdown.
 	 * See blk_register_queue() for details.
block/blk-mq.c +11 -0
index b3d2785eefe9..817e016ef886 100644
@@ -224,6 +224,11 @@ void blk_mq_quiesce_queue(struct request_queue *q)
 	unsigned int i;
 	bool rcu = false;
 
+	mutex_lock(&q->mq_quiesce_lock);
+
+	if (blk_queue_quiesced(q))
+		goto exit;
+
 	blk_mq_quiesce_queue_nowait(q);
 
 	queue_for_each_hw_ctx(q, hctx, i) {
@@ -234,6 +239,8 @@ void blk_mq_quiesce_queue(struct request_queue *q)
 	}
 	if (rcu)
 		synchronize_rcu();
+ exit:
+	mutex_unlock(&q->mq_quiesce_lock);
 }
 EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
 
@@ -246,10 +253,14 @@ EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
  */
 void blk_mq_unquiesce_queue(struct request_queue *q)
 {
+	mutex_lock(&q->mq_quiesce_lock);
+
 	blk_queue_flag_clear(QUEUE_FLAG_QUIESCED, q);
 
 	/* dispatch requests which are inserted during quiescing */
 	blk_mq_run_hw_queues(q, true);
+
+	mutex_unlock(&q->mq_quiesce_lock);
 }
 EXPORT_SYMBOL_GPL(blk_mq_unquiesce_queue);
 
include/linux/blkdev.h +2 -0
index d8dba550ecac..5ed03066b33e 100644
@@ -569,6 +569,8 @@ struct request_queue {
 	 */
 	struct mutex		mq_freeze_lock;
 
+	struct mutex		mq_quiesce_lock;
+
 	struct blk_mq_tag_set	*tag_set;
 	struct list_head	tag_set_list;
 	struct bio_set		bio_split;
Signature · 1 line
2.25.2
#2 lore raw

Re: [PATCH V2 1/2] blk-mq: serialize queue quiesce and unquiesce by mutex

It doesn't matter. Because the reentry of quiesce&unquiesce queue is not safe, must be avoided by other mechanism. otherwise, exceptions may occur. Introduce mq_quiesce_lock looks saving possible synchronization waits, but it should not happen. If really happen, we need fix it.

Quoted 82 lines · Ming Lei

Add .mq_quiesce_mutext to request queue, so that queue quiesce and unquiesce can be serialized. Meantime we can avoid unnecessary synchronize_rcu() in case that queue has been quiesced already.

Cc Sagi Grimberg <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Chao Leng <[email protected]>
---
  block/blk-core.c       |  2 ++
  block/blk-mq.c         | 11 +++++++++++
  include/linux/blkdev.h |  2 ++
  3 files changed, 15 insertions(+)
block/blk-core.c +2 -0
index d9d632639bd1..ffc57df70064 100644
@@ -561,6 +561,8 @@ struct request_queue *blk_alloc_queue(int node_id)
  	init_waitqueue_head(&q->mq_freeze_wq);
  	mutex_init(&q->mq_freeze_lock);
 
+	mutex_init(&q->mq_quiesce_lock);
+
  	/*
  	 * Init percpu_ref in atomic mode so that it's faster to shutdown.
  	 * See blk_register_queue() for details.
block/blk-mq.c +11 -0
index b3d2785eefe9..817e016ef886 100644
@@ -224,6 +224,11 @@ void blk_mq_quiesce_queue(struct request_queue *q)
  	unsigned int i;
  	bool rcu = false;
 
+	mutex_lock(&q->mq_quiesce_lock);
+
+	if (blk_queue_quiesced(q))
+		goto exit;
+
  	blk_mq_quiesce_queue_nowait(q);
 
  	queue_for_each_hw_ctx(q, hctx, i) {
@@ -234,6 +239,8 @@ void blk_mq_quiesce_queue(struct request_queue *q)
  	}
  	if (rcu)
  		synchronize_rcu();
+ exit:
+	mutex_unlock(&q->mq_quiesce_lock);
  }
  EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
 
@@ -246,10 +253,14 @@ EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue);
   */
  void blk_mq_unquiesce_queue(struct request_queue *q)
  {
+	mutex_lock(&q->mq_quiesce_lock);
+
  	blk_queue_flag_clear(QUEUE_FLAG_QUIESCED, q);
 
  	/* dispatch requests which are inserted during quiescing */
  	blk_mq_run_hw_queues(q, true);
+
+	mutex_unlock(&q->mq_quiesce_lock);
  }
  EXPORT_SYMBOL_GPL(blk_mq_unquiesce_queue);
 
include/linux/blkdev.h +2 -0
index d8dba550ecac..5ed03066b33e 100644
@@ -569,6 +569,8 @@ struct request_queue {
  	 */
  	struct mutex		mq_freeze_lock;
 
+	struct mutex		mq_quiesce_lock;
+
  	struct blk_mq_tag_set	*tag_set;
  	struct list_head	tag_set_list;
  	struct bio_set		bio_split;
author
#3 lore raw

Re: [PATCH V2 1/2] blk-mq: serialize queue quiesce and unquiesce by mutex

Quoted 4 lines · Chao Leng

It doesn't matter. Because the reentry of quiesce&unquiesce queue is not safe, must be avoided by other mechanism. otherwise, exceptions may occur. Introduce mq_quiesce_lock looks saving possible synchronization waits, but it should not happen. If really happen, we need fix it.

Sagi mentioned there may be nested queue quiesce, so I add .mq_quiesce_lock to make this usage easy to support, meantime avoid percpu_ref warning in such usage.

Anyway, not see any problem with adding .mq_quiesce_lock, so I'd suggest to move on with this way.

Thanks,
Ming

#4 lore raw

Re: [PATCH V2 1/2] blk-mq: serialize queue quiesce and unquiesce by mutex

Quoted 6 lines · Ming Lei
Quoted 4 lines · Chao Leng

It doesn't matter. Because the reentry of quiesce&unquiesce queue is not safe, must be avoided by other mechanism. otherwise, exceptions may occur. Introduce mq_quiesce_lock looks saving possible synchronization waits, but it should not happen. If really happen, we need fix it.

Quoted 6 lines

Sagi mentioned there may be nested queue quiesce, so I add .mq_quiesce_lock to make this usage easy to support, meantime avoid percpu_ref warning in such usage.

Anyway, not see any problem with adding .mq_quiesce_lock, so I'd suggest to move on with this way.

I'm not sure there really are any nested queue quiesce paths, but if there are, wouldn't we need to track the "depth" like how a queue freeze works?

#5 lore raw

Re: [PATCH V2 1/2] blk-mq: serialize queue quiesce and unquiesce by mutex

Quoted 10 lines
Quoted 10 lines
Quoted 5 lines

It doesn't matter. Because the reentry of quiesce&unquiesce queue is not safe, must be avoided by other mechanism. otherwise, exceptions may occur. Introduce mq_quiesce_lock looks saving possible synchronization waits, but it should not happen. If really happen, we need fix it.

Sagi mentioned there may be nested queue quiesce, so I add .mq_quiesce_lock to make this usage easy to support, meantime avoid percpu_ref warning in such usage.

Quoted 6 lines
Quoted 2 lines

Anyway, not see any problem with adding .mq_quiesce_lock, so I'd suggest to move on with this way.

I'm not sure there really are any nested queue quiesce paths, but if there are, wouldn't we need to track the "depth" like how a queue freeze works?

We might need it when the async quiesce is implemented for this, because then quiesce will only "start" and in a different context wait for the quiesced event (where we may need to add wakeup for waiters).

This is why I want to see this with the async quiesce piece.

author
#6 lore raw

Re: [PATCH V2 1/2] blk-mq: serialize queue quiesce and unquiesce by mutex

In reply to Keith Busch
Quoted 11 lines · Keith Busch
Quoted 10 lines · Ming Lei
Quoted 4 lines · Chao Leng

It doesn't matter. Because the reentry of quiesce&unquiesce queue is not safe, must be avoided by other mechanism. otherwise, exceptions may occur. Introduce mq_quiesce_lock looks saving possible synchronization waits, but it should not happen. If really happen, we need fix it.

Sagi mentioned there may be nested queue quiesce, so I add .mq_quiesce_lock to make this usage easy to support, meantime avoid percpu_ref warning in such usage.

Quoted 6 lines
Quoted 2 lines

Anyway, not see any problem with adding .mq_quiesce_lock, so I'd suggest to move on with this way.

I'm not sure there really are any nested queue quiesce paths, but if there are, wouldn't we need to track the "depth" like how a queue freeze works?

Both atomic 'depth' and .mq_quiesce_lock can work for nested queue quiesce since we can avoid unnecessary queue quiesce with the mutex. percpu_ref_kill() / percpu_ref_kill_and_confirm() can warn if the percpu_ref has been killed, that is why I think Sagi's suggestion is good.

But 'depth' may cause trouble easily, such as unbalanced quiesce/unquiesce, however no such issue with mutex, at least we don't require the two to be paired strictly so far.

Thanks,
Ming

author
#7 lore raw

PATCH v2 2/2 blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

In reply to Ming Lei

In case of BLK_MQ_F_BLOCKING, blk-mq uses SRCU to mark read critical section during dispatching request, then request queue quiesce is based on SRCU. What we want to get is low cost added in fast path.

With percpu-ref, it is cleaner and simpler & enough for implementing queue quiesce. The main requirement is to make sure all read sections to observe QUEUE_FLAG_QUIESCED once blk_mq_quiesce_queue() returns.

Also it becomes much easier to add interface of async queue quiesce.

Meantime memory footprint can be reduced with per-request-queue percpu-ref.

From implementation viewpoint, in fast path, not see percpu_ref is slower than SRCU, and srcu tree(default option in most distributions) could be slower since memory barrier is required in both lock & unlock, and rcu_read_lock()/rcu_read_unlock() should be much cheap than smp_mb().

1) percpu_ref just hold the rcu_read_lock, then run a check & increase/decrease on the percpu variable:

rcu_read_lock()
if (__ref_is_percpu(ref, &percpu_count))

	this_cpu_inc(*percpu_count);

rcu_read_unlock()

2) srcu tree:

        idx = READ_ONCE(ssp->srcu_idx) & 0x1;
        this_cpu_inc(ssp->sda->srcu_lock_count[idx]);
        smp_mb(); /* B */  /* Avoid leaking the critical section. */

Also from my test on null_blk(blocking), not observe percpu-ref performs worse than srcu, see the following test:

1) test steps:

rmmod null_blk > /dev/null 2>&1
modprobe null_blk nr_devices=1 submit_queues=1 blocking=1

fio --bs=4k --size=512G  --rw=randread --norandommap --direct=1 --ioengine=libaio \
	--iodepth=64 --runtime=60 --group_reporting=1  --name=nullb0 \
	--filename=/dev/nullb0 --numjobs=32

test machine: HP DL380, 16 cpu cores, 2 threads per core, dual sockets/numa, Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz

2) test result:
- srcu quiesce: 6063K IOPS
- percpu-ref quiesce: 6113K IOPS

Cc Lai Jiangshan <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Sagi Grimberg <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Chao Leng <[email protected]>
Cc Christoph Hellwig <[email protected]>
---
 block/blk-mq-sysfs.c   |   2 -
 block/blk-mq.c         | 116 +++++++++++++++++++++--------------------
 block/blk-sysfs.c      |   6 ++-
 include/linux/blk-mq.h |   7 ---
 include/linux/blkdev.h |   4 ++
 5 files changed, 68 insertions(+), 67 deletions(-)
block/blk-mq-sysfs.c +0 -2
index 062229395a50..799db7937105 100644
@@ -38,8 +38,6 @@ static void blk_mq_hw_sysfs_release(struct kobject *kobj)
 
 	cancel_delayed_work_sync(&hctx->run_work);
 
-	if (hctx->flags & BLK_MQ_F_BLOCKING)
-		cleanup_srcu_struct(hctx->srcu);
 	blk_free_flush_queue(hctx->fq);
 	sbitmap_free(&hctx->ctx_map);
 	free_cpumask_var(hctx->cpumask);
block/blk-mq.c +59 -57

243 lines not on this page · raw message

block/blk-sysfs.c +5 -1
index 7dda709f3ccb..56b6c045e30c 100644
@@ -941,9 +941,13 @@ static void blk_release_queue(struct kobject *kobj)
 
 	blk_queue_free_zone_bitmaps(q);
 
-	if (queue_is_mq(q))
+	if (queue_is_mq(q)) {
 		blk_mq_release(q);
 
+		if (q->tag_set->flags & BLK_MQ_F_BLOCKING)
+			percpu_ref_exit(&q->dispatch_counter);
+	}
+
 	blk_trace_shutdown(q);
 	mutex_lock(&q->debugfs_mutex);
 	debugfs_remove_recursive(q->debugfs_dir);
include/linux/blk-mq.h +0 -7
index 9d2d5ad367a4..ea3461298de5 100644
@@ -169,13 +169,6 @@ struct blk_mq_hw_ctx {
 	 * q->unused_hctx_list.
 	 */
 	struct list_head	hctx_list;
-
-	/**
-	 * @srcu: Sleepable RCU. Use as lock when type of the hardware queue is
-	 * blocking (BLK_MQ_F_BLOCKING). Must be the last member - see also
-	 * blk_mq_hw_ctx_size().
-	 */
-	struct srcu_struct	srcu[];
 };
 
 /**
include/linux/blkdev.h +4 -0
index 5ed03066b33e..1ab17bcbc334 100644
@@ -571,6 +571,10 @@ struct request_queue {
 
 	struct mutex		mq_quiesce_lock;
 
+	/* only used for BLK_MQ_F_BLOCKING */
+	struct percpu_ref	dispatch_counter;
+	wait_queue_head_t	mq_quiesce_wq;
+
 	struct blk_mq_tag_set	*tag_set;
 	struct list_head	tag_set_list;
 	struct bio_set		bio_split;
Signature · 1 line
2.25.2
author
#8 lore raw
In reply to Ming Lei
Quoted 25 lines · Ming Lei

Hi Jens,

The 1st patch add .mq_quiesce_mutex for serializing quiesce/unquiesce, and prepares for replacing srcu with percpu_ref.

The 2nd patch replaces srcu with percpu_ref.

V2:
- add .mq_quiesce_lock
- add comment on patch 2 wrt. handling hctx_lock() failure
- trivial patch style change

Ming Lei (2):
blk-mq: serialize queue quiesce and unquiesce by mutex
blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

 block/blk-core.c       |   2 +
 block/blk-mq-sysfs.c   |   2 -
 block/blk-mq.c         | 125 +++++++++++++++++++++++------------------
 block/blk-sysfs.c      |   6 +-
 include/linux/blk-mq.h |   7 ---
 include/linux/blkdev.h |   6 ++
 6 files changed, 82 insertions(+), 66 deletions(-)
Quoted 8 lines
Cc Lai Jiangshan <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Sagi Grimberg <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Chao Leng <[email protected]>
Cc Christoph Hellwig <[email protected]>

Hello Guys,

Is there any objections on the two patches? If not, I'd suggest to move on.

Thanks,
Ming

maintainer
#9 lore raw
Quoted 35 lines · Ming Lei
Quoted 25 lines · Ming Lei

Hi Jens,

The 1st patch add .mq_quiesce_mutex for serializing quiesce/unquiesce, and prepares for replacing srcu with percpu_ref.

The 2nd patch replaces srcu with percpu_ref.

V2:
- add .mq_quiesce_lock
- add comment on patch 2 wrt. handling hctx_lock() failure
- trivial patch style change

Ming Lei (2):
blk-mq: serialize queue quiesce and unquiesce by mutex
blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

 block/blk-core.c       |   2 +
 block/blk-mq-sysfs.c   |   2 -
 block/blk-mq.c         | 125 +++++++++++++++++++++++------------------
 block/blk-sysfs.c      |   6 +-
 include/linux/blk-mq.h |   7 ---
 include/linux/blkdev.h |   6 ++
 6 files changed, 82 insertions(+), 66 deletions(-)
Quoted 8 lines
Cc Lai Jiangshan <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Sagi Grimberg <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Chao Leng <[email protected]>
Cc Christoph Hellwig <[email protected]>
Quoted 3 lines

Hello Guys,

Is there any objections on the two patches? If not, I'd suggest to move> on.

Seems like the nested case is one that should either be handled, or at least detected.

Signature · 1 line
Jens Axboe
#10 lore raw
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

Hi Jens,

The 1st patch add .mq_quiesce_mutex for serializing quiesce/unquiesce, and prepares for replacing srcu with percpu_ref.

The 2nd patch replaces srcu with percpu_ref.

V2:
- add .mq_quiesce_lock
- add comment on patch 2 wrt. handling hctx_lock() failure
- trivial patch style change

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Ming Lei (2):
blk-mq: serialize queue quiesce and unquiesce by mutex
blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

I thought we agreed to have a little more consolidation for blocking and !blocking paths (move fallbacks to common paths).

Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
  block/blk-core.c       |   2 +
  block/blk-mq-sysfs.c   |   2 -
  block/blk-mq.c         | 125 +++++++++++++++++++++++------------------
  block/blk-sysfs.c      |   6 +-
  include/linux/blk-mq.h |   7 ---
  include/linux/blkdev.h |   6 ++
  6 files changed, 82 insertions(+), 66 deletions(-)
Cc Lai Jiangshan <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Sagi Grimberg <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Chao Leng <[email protected]>
Cc Christoph Hellwig <[email protected]>
Quoted 6 lines
Quoted 3 lines

Hello Guys,

Is there any objections on the two patches? If not, I'd suggest to move> on.

Seems like the nested case is one that should either be handled, or at least detected.

Personally, I'd like to see the async quiesce piece as well here, which is the reason why this change was proposed. Don't see a strong urgency to move forward with it before that, especially as this could potentially affect various non-trivial reset flows.

author
#11 lore raw
Quoted 14 lines · Sagi Grimberg
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

Hi Jens,

The 1st patch add .mq_quiesce_mutex for serializing quiesce/unquiesce, and prepares for replacing srcu with percpu_ref.

The 2nd patch replaces srcu with percpu_ref.

V2:
- add .mq_quiesce_lock
- add comment on patch 2 wrt. handling hctx_lock() failure
- trivial patch style change

Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Ming Lei (2):
blk-mq: serialize queue quiesce and unquiesce by mutex
blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

I thought we agreed to have a little more consolidation for blocking and !blocking paths (move fallbacks to common paths).

Could you describe the consolidation one more time for the two paths?

BTW, code will become a little messy if we move queue quiesce handling out of __blk_mq_try_issue_directly(), because we have two conditions to trigger insert request into scheduler queue:

1) hctx_lock() failure

2) blk_queue_quiesced() or blk_mq_hctx_stopped()

The former doesn't need to unlock hctx, however the latter needs that, that is why I don't do the change if that is the consolidation you mentioned.

Quoted 23 lines
Quoted 23 lines
Quoted 22 lines
Quoted 10 lines

  block/blk-core.c       |   2 +
  block/blk-mq-sysfs.c   |   2 -
  block/blk-mq.c         | 125 +++++++++++++++++++++++------------------
  block/blk-sysfs.c      |   6 +-
  include/linux/blk-mq.h |   7 ---
  include/linux/blkdev.h |   6 ++
  6 files changed, 82 insertions(+), 66 deletions(-)
Quoted 8 lines
Cc Lai Jiangshan <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Sagi Grimberg <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Chao Leng <[email protected]>
Cc Christoph Hellwig <[email protected]>

Hello Guys,

Is there any objections on the two patches? If not, I'd suggest to move> on.

Quoted 7 lines
Quoted 2 lines

Seems like the nested case is one that should either be handled, or at least detected.

Personally, I'd like to see the async quiesce piece as well here, which is the reason why this change was proposed. Don't see a strong urgency to move forward with it before that, especially as this could potentially affect various non-trivial reset flows.

OK, it shouldn't be easy to add the interface, will do that in next version.

Thanks,
Ming

author
#12 lore raw
In reply to Jens Axboe
Quoted 36 lines · Jens Axboe
Quoted 35 lines · Ming Lei
Quoted 33 lines · Ming Lei

Hi Jens,

The 1st patch add .mq_quiesce_mutex for serializing quiesce/unquiesce, and prepares for replacing srcu with percpu_ref.

The 2nd patch replaces srcu with percpu_ref.

V2:
- add .mq_quiesce_lock
- add comment on patch 2 wrt. handling hctx_lock() failure
- trivial patch style change

Ming Lei (2):
blk-mq: serialize queue quiesce and unquiesce by mutex
blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

 block/blk-core.c       |   2 +
 block/blk-mq-sysfs.c   |   2 -
 block/blk-mq.c         | 125 +++++++++++++++++++++++------------------
 block/blk-sysfs.c      |   6 +-
 include/linux/blk-mq.h |   7 ---
 include/linux/blkdev.h |   6 ++
 6 files changed, 82 insertions(+), 66 deletions(-)
Cc Lai Jiangshan <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Sagi Grimberg <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Chao Leng <[email protected]>
Cc Christoph Hellwig <[email protected]>
Quoted 6 lines
Quoted 3 lines

Hello Guys,

Is there any objections on the two patches? If not, I'd suggest to move> on.

Seems like the nested case is one that should either be handled, or at least detected.

Yeah, the 1st patch adds mutex for handling nested case correctly and efficiently.

Thanks,
Ming

#13 lore raw
Quoted 37 lines · Ming Lei
Quoted 36 lines · Jens Axboe
Quoted 35 lines · Ming Lei
Quoted 33 lines · Ming Lei

Hi Jens,

The 1st patch add .mq_quiesce_mutex for serializing quiesce/unquiesce, and prepares for replacing srcu with percpu_ref.

The 2nd patch replaces srcu with percpu_ref.

V2:
- add .mq_quiesce_lock
- add comment on patch 2 wrt. handling hctx_lock() failure
- trivial patch style change

Ming Lei (2):
blk-mq: serialize queue quiesce and unquiesce by mutex
blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

 block/blk-core.c       |   2 +
 block/blk-mq-sysfs.c   |   2 -
 block/blk-mq.c         | 125 +++++++++++++++++++++++------------------
 block/blk-sysfs.c      |   6 +-
 include/linux/blk-mq.h |   7 ---
 include/linux/blkdev.h |   6 ++
 6 files changed, 82 insertions(+), 66 deletions(-)
Cc Lai Jiangshan <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Sagi Grimberg <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Chao Leng <[email protected]>
Cc Christoph Hellwig <[email protected]>
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Hello Guys,

Is there any objections on the two patches? If not, I'd suggest to move> on.

Seems like the nested case is one that should either be handled, or at least detected.

Yeah, the 1st patch adds mutex for handling nested case correctly and efficiently.

That doesn't really do anything about handling nested quiesce/unquiesce. It just prevents two threads from doing it at the same time, but anyone can still undo the other's expected queue state. The following on top of your series will at least detect the condition:

---

block/blk-mq.c +1 -0
index ef6c6fa8dab0..52b53f2bb567 100644
@@ -249,6 +249,7 @@ void blk_mq_unquiesce_queue(struct request_queue *q)
 {
 	mutex_lock(&q->mq_quiesce_lock);
 
+	WARN_ON(!blk_queue_quiesced(q));
 	blk_queue_flag_clear(QUEUE_FLAG_QUIESCED, q);
 
 	if (q->tag_set->flags & BLK_MQ_F_BLOCKING)

--

author
#14 lore raw
Quoted 40 lines · Keith Busch
Quoted 39 lines · Ming Lei
Quoted 38 lines · Jens Axboe
Quoted 37 lines · Ming Lei
Quoted 25 lines · Ming Lei

Hi Jens,

The 1st patch add .mq_quiesce_mutex for serializing quiesce/unquiesce, and prepares for replacing srcu with percpu_ref.

The 2nd patch replaces srcu with percpu_ref.

V2:
- add .mq_quiesce_lock
- add comment on patch 2 wrt. handling hctx_lock() failure
- trivial patch style change

Ming Lei (2):
blk-mq: serialize queue quiesce and unquiesce by mutex
blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

 block/blk-core.c       |   2 +
 block/blk-mq-sysfs.c   |   2 -
 block/blk-mq.c         | 125 +++++++++++++++++++++++------------------
 block/blk-sysfs.c      |   6 +-
 include/linux/blk-mq.h |   7 ---
 include/linux/blkdev.h |   6 ++
 6 files changed, 82 insertions(+), 66 deletions(-)
Quoted 8 lines
Cc Lai Jiangshan <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Sagi Grimberg <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Johannes Thumshirn <[email protected]>
Cc Chao Leng <[email protected]>
Cc Christoph Hellwig <[email protected]>

Hello Guys,

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line

Is there any objections on the two patches? If not, I'd suggest to move> on.

Seems like the nested case is one that should either be handled, or at least detected.

Yeah, the 1st patch adds mutex for handling nested case correctly and efficiently.

That doesn't really do anything about handling nested quiesce/unquiesce.

The mutex is required for avoiding warning in percpu_ref_kill_and_confirm() if two queue quiesce are nested, and I will comment on this motivation in next version.

Quoted 2 lines

It just prevents two threads from doing it at the same time, but anyone can still undo the other's expected queue state. The following on top of

Right, the patch itself changes nothing wrt. this point, and it breaks nothing too.

block/blk-mq.c void blk_mq_unquiesce_queue(struct request_queue *q) · line 252

Quoted 12 lines

your series will at least detect the condition:

---

block/blk-mq.c +1 -0
index ef6c6fa8dab0..52b53f2bb567 100644
@@ -249,6 +249,7 @@ void blk_mq_unquiesce_queue(struct request_queue *q)
 {
 	mutex_lock(&q->mq_quiesce_lock);
 
+	WARN_ON(!blk_queue_quiesced(q));

We can't do that simply, because queue unquiesce may be called unconditionally on un-quiesced queue, such as nvme_dev_remove_admin(), nvme_set_queue_dying(), ...

Thanks,
Ming