block/dm: allow DM to defer blk_register_queue() until ready

The original commit e9a823fb34a8b (block: fix warning when I/O elevator is changed as request_queue is being removed) is pretty conflated. "conflated" because the resource being protected by q->sysfs_lock isn't the queue_flags (it is the 'queue' kobj).

q->sysfs_lock serializes __elevator_change() (via elv_iosched_store) from racing with blk_unregister_queue():
1) By holding q->sysfs_lock first, __elevator_change() can complete before a racing blk_unregister_queue().
2) Conversely, __elevator_change() is testing for QUEUE_FLAG_REGISTERED in case elv_iosched_store() loses the race with blk_unregister_queue(), it needs a way to know the 'queue' kobj isn't there.

Expand the scope of blk_unregister_queue()'s q->sysfs_lock use so it is held until after the 'queue' kobj is removed.

Also, blk_unregister_queue() should use q->queue_lock to protect against any concurrent writes to q->queue_flags -- even though chances are the queue is being cleaned up so no concurrent writes are likely.

Fixes e9a823fb34a8b ("block: fix warning when I/O elevator is changed as request_queue is being removed")

Signed-off-by Mike Snitzer <[email protected]>

---
 block/blk-sysfs.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

block/blk-sysfs.c +4 -2

index 870484eaed1f..9272452ff456 100644
@@ -929,12 +929,17 @@ void blk_unregister_queue(struct gendisk *disk)
 	if (WARN_ON(!q))
 		return;
 
+	/*
+	 * Protect against the 'queue' kobj being accessed
+	 * while/after it is removed.
+	 */
 	mutex_lock(&q->sysfs_lock);
-	queue_flag_clear_unlocked(QUEUE_FLAG_REGISTERED, q);
-	mutex_unlock(&q->sysfs_lock);

Quoted 13 lines · Mike Snitzer

The original commit e9a823fb34a8b (block: fix warning when I/O elevator is changed as request_queue is being removed) is pretty conflated. "conflated" because the resource being protected by q->sysfs_lock isn't the queue_flags (it is the 'queue' kobj).

q->sysfs_lock serializes __elevator_change() (via elv_iosched_store) from racing with blk_unregister_queue():
1) By holding q->sysfs_lock first, __elevator_change() can complete before a racing blk_unregister_queue().
2) Conversely, __elevator_change() is testing for QUEUE_FLAG_REGISTERED in case elv_iosched_store() loses the race with blk_unregister_queue(), it needs a way to know the 'queue' kobj isn't there.

Quoted 2 lines

Expand the scope of blk_unregister_queue()'s q->sysfs_lock use so it is held until after the 'queue' kobj is removed.

This way will cause deadlock, see blow.

Quoted 26 lines

Also, blk_unregister_queue() should use q->queue_lock to protect against any concurrent writes to q->queue_flags -- even though chances are the queue is being cleaned up so no concurrent writes are likely.

Fixes e9a823fb34a8b ("block: fix warning when I/O elevator is changed as request_queue is being removed")

Signed-off-by Mike Snitzer <[email protected]>

---
 block/blk-sysfs.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

block/blk-sysfs.c +4 -2

index 870484eaed1f..9272452ff456 100644
@@ -929,12 +929,17 @@ void blk_unregister_queue(struct gendisk *disk)
 	if (WARN_ON(!q))
 		return;
 
+	/*
+	 * Protect against the 'queue' kobj being accessed
+	 * while/after it is removed.
+	 */
 	mutex_lock(&q->sysfs_lock);
-	queue_flag_clear_unlocked(QUEUE_FLAG_REGISTERED, q);
-	mutex_unlock(&q->sysfs_lock);

block/blk-sysfs.c void blk_unregister_queue(struct gendisk *disk) · line 945

Quoted 10 lines

block/blk-sysfs.c +4 -1

 
-	wbt_exit(q);
+	spin_lock_irq(q->queue_lock);
+	queue_flag_clear(QUEUE_FLAG_REGISTERED, q);
+	spin_unlock_irq(q->queue_lock);
 
+	wbt_exit(q);
 
 	if (q->mq_ops)
 		blk_mq_unregister_dev(disk_to_dev(disk), q);

void blk_mq_unregister_dev(struct device *dev, struct request_queue *q)

{
        mutex_lock(&q->sysfs_lock);
        __blk_mq_unregister_dev(dev, q);
        mutex_unlock(&q->sysfs_lock);
}

block/blk-sysfs.c void blk_unregister_queue(struct gendisk *disk) · line 956

Quoted 7 lines

block/blk-sysfs.c +2 -0

@@ -946,4 +951,6 @@ void blk_unregister_queue(struct gendisk *disk)
 	kobject_del(&q->kobj);
 	blk_trace_remove_sysfs(disk_to_dev(disk));
 	kobject_put(&disk_to_dev(disk)->kobj);
+
+	mutex_unlock(&q->sysfs_lock);
 }

Except for above, I remember there is also lockdep warning between sysfs_lock and driver core's lock if the sysfs_lock is extended in this way(I tried it before, but forget the details now), so please just hold queue_lock inside the sysfs lock.

The original commit e9a823fb34a8b (block: fix warning when I/O elevator is changed as request_queue is being removed) is pretty conflated. "conflated" because the resource being protected by q->sysfs_lock isn't the queue_flags (it is the 'queue' kobj).

q->sysfs_lock serializes __elevator_change() (via elv_iosched_store) from racing with blk_unregister_queue():
1) By holding q->sysfs_lock first, __elevator_change() can complete before a racing blk_unregister_queue().
2) Conversely, __elevator_change() is testing for QUEUE_FLAG_REGISTERED in case elv_iosched_store() loses the race with blk_unregister_queue(), it needs a way to know the 'queue' kobj isn't there.

Expand the scope of blk_unregister_queue()'s q->sysfs_lock use so it is held until after the 'queue' kobj is removed.

To do so blk_mq_unregister_dev() must not also take q->sysfs_lock. So rename __blk_mq_unregister_dev() to blk_mq_unregister_dev().

Also, blk_unregister_queue() should use q->queue_lock to protect against any concurrent writes to q->queue_flags -- even though chances are the queue is being cleaned up so no concurrent writes are likely.

Fixes e9a823fb34a8b ("block: fix warning when I/O elevator is changed as request_queue is being removed")

Signed-off-by Mike Snitzer <[email protected]>

---
 block/blk-mq-sysfs.c |  9 +--------
 block/blk-sysfs.c    | 13 ++++++++++---
 2 files changed, 11 insertions(+), 11 deletions(-)

v6: blk_mq_unregister_dev now requires q->sysfs_lock be held, Ming: I am not seeing any lockdep complaints with this. I've tested bio-based, blk-mq and old .request_fn request-based.

block/blk-mq-sysfs.c +1 -8

index 79969c3c234f..a54b4b070f1c 100644
@@ -248,7 +248,7 @@ static int blk_mq_register_hctx(struct blk_mq_hw_ctx *hctx)
 	return ret;
 }
 
-static void __blk_mq_unregister_dev(struct device *dev, struct request_queue *q)
+void blk_mq_unregister_dev(struct device *dev, struct request_queue *q)
 {
 	struct blk_mq_hw_ctx *hctx;
 	int i;

@@ -265,13 +265,6 @@ static void __blk_mq_unregister_dev(struct device *dev, struct request_queue *q)
 	q->mq_sysfs_init_done = false;
 }
 
-void blk_mq_unregister_dev(struct device *dev, struct request_queue *q)
-{
-	mutex_lock(&q->sysfs_lock);
-	__blk_mq_unregister_dev(dev, q);
-	mutex_unlock(&q->sysfs_lock);
-}
-
 void blk_mq_hctx_kobj_init(struct blk_mq_hw_ctx *hctx)
 {
 	kobject_init(&hctx->kobj, &blk_mq_hw_ktype);

block/blk-sysfs.c +8 -3

index 870484eaed1f..9272452ff456 100644
@@ -929,12 +929,17 @@ void blk_unregister_queue(struct gendisk *disk)
 	if (WARN_ON(!q))
 		return;
 
+	/*
+	 * Protect against the 'queue' kobj being accessed
+	 * while/after it is removed.
+	 */
 	mutex_lock(&q->sysfs_lock);
-	queue_flag_clear_unlocked(QUEUE_FLAG_REGISTERED, q);
-	mutex_unlock(&q->sysfs_lock);
 
-	wbt_exit(q);
+	spin_lock_irq(q->queue_lock);
+	queue_flag_clear(QUEUE_FLAG_REGISTERED, q);
+	spin_unlock_irq(q->queue_lock);
 
+	wbt_exit(q);
 
 	if (q->mq_ops)
 		blk_mq_unregister_dev(disk_to_dev(disk), q);

Hi Jens,

I'm submitting this v5 with more feeling now ;)

I've distilled the changes down to be quite minimal. Hopefully this will help you and others review.

I've also done a dry-run of applying 4.16 block changes and then merging dm-4.16; no merge conflicts:
https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/log/?h=block-4.16
https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/log/?h=dm-4.16
https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/log/?h=block-4.16_dm-4.16

And all tests very well for me.

Please consider for 4.16, thanks!

I'm submitting this v5 with more feeling now ;)

Quoted 1 line · Mike Snitzer

I'm submitting this v5 with more feeling now ;)

Hello Mike,

Have these patches been tested with lockdep enabled? The following appeared in the kernel log when after I started testing Jens' for-next tree of this morning:

So you replied to v5, I emailed a v6 out for the relevant patch. Just want to make sure you're testing with either Jens' latest tree or are using my v6 that fixed blk_mq_unregister_dev() to require caller holds q->sysfs_lock ?

Quoted 4 lines · Mike Snitzer

So you replied to v5, I emailed a v6 out for the relevant patch. Just want to make sure you're testing with either Jens' latest tree or are using my v6 that fixed blk_mq_unregister_dev() to require caller holds q->sysfs_lock ?

Hello Mike,

In my test I was using Jens' latest for-next tree (commit 563877ae7dae).

Do I need to do something more to enable lockdep aside from set CONFIG_LOCKDEP_SUPPORT=y ?

Quoted 2 lines · Mike Snitzer

Do I need to do something more to enable lockdep aside from set CONFIG_LOCKDEP_SUPPORT=y ?

Hello Mike,

I think you also need to set CONFIG_PROVE_LOCKING=y.

Quoted 1 line · Mike Snitzer

I'm submitting this v5 with more feeling now ;)

Hello Mike,

Have these patches been tested with lockdep enabled? The following appeared in the kernel log when after I started testing Jens' for-next tree of this morning:

======================================================
WARNING: possible circular locking dependency detected 4.15.0-rc7-dbg+ #1 Not tainted
------------------------------------------------------
02-mq/1211 is trying to acquire lock:
(&q->sysfs_lock){+.+.}, at: [<000000008b65bdad>] queue_attr_store+0x35/0x80

but task is already holding lock:
(kn->count#213){++++}, at: [<000000007a18ad18>] kernfs_fop_write+0xe5/0x190

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:

-> #1 (kn->count#213){++++}:

       kernfs_remove+0x1a/0x30
       kobject_del.part.3+0xe/0x40
       blk_unregister_queue+0xa7/0xe0
       del_gendisk+0x12f/0x260
       sd_remove+0x58/0xc0 [sd_mod]
       device_release_driver_internal+0x15a/0x220
       bus_remove_device+0xf4/0x170
       device_del+0x12f/0x330
       __scsi_remove_device+0xef/0x120 [scsi_mod]
       scsi_forget_host+0x1b/0x60 [scsi_mod]
       scsi_remove_host+0x6f/0x110 [scsi_mod]
       0xffffffffc09ed6e4
       process_one_work+0x21c/0x6d0
       worker_thread+0x35/0x380
       kthread+0x117/0x130
       ret_from_fork+0x24/0x30

-> #0 (&q->sysfs_lock){+.+.}:

       __mutex_lock+0x6c/0x9e0
       queue_attr_store+0x35/0x80
       kernfs_fop_write+0x109/0x190
       __vfs_write+0x1e/0x130
       vfs_write+0xb9/0x1b0
       SyS_write+0x40/0xa0
       entry_SYSCALL_64_fastpath+0x23/0x9a

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(kn->count#213);
                               lock(&q->sysfs_lock);
                               lock(kn->count#213);

lock(&q->sysfs_lock);

*** DEADLOCK ***

3 locks held by 02-mq/1211:

 #0:  (sb_writers#6){.+.+}, at: [<00000000afdb61d3>] vfs_write+0x17f/0x1b0
 #1:  (&of->mutex){+.+.}, at: [<00000000b291cabb>] kernfs_fop_write+0xdd/0x190
 #2:  (kn->count#213){++++}, at: [<000000007a18ad18>] kernfs_fop_write+0xe5/0x190

sysfs write op calls kernfs_fop_write which takes:
of->mutex then kn->count#213 (no idea what that is) then q->sysfs_lock (via queue_attr_store)

vs

blk_unregister_queue takes:
q->sysfs_lock then
kernfs_mutex (via kernfs_remove)
seems lockdep thinks "kernfs_mutex" is "kn->count#213"?

Feels like lockdep code in fs/kernfs/file.c and fs/kernfs/dir.c is triggering false positives.. because these seem like different kernfs locks yet they are reported as "kn->count#213".

Certainly feeling out of my depth with kernfs's locking though.

Quoted 11 lines · Mike Snitzer

sysfs write op calls kernfs_fop_write which takes:
of->mutex then kn->count#213 (no idea what that is) then q->sysfs_lock (via queue_attr_store)

vs

blk_unregister_queue takes:
q->sysfs_lock then
kernfs_mutex (via kernfs_remove)
seems lockdep thinks "kernfs_mutex" is "kn->count#213"?

Quoted 5 lines

Feels like lockdep code in fs/kernfs/file.c and fs/kernfs/dir.c is triggering false positives.. because these seem like different kernfs locks yet they are reported as "kn->count#213".

Certainly feeling out of my depth with kernfs's locking though.

Hello Mike,

I don't think that this is a false positive but rather the following traditional pattern of a potential deadlock involving sysfs attributes:
* One context obtains a mutex from inside a sysfs attribute method:
queue_attr_store() obtains q->sysfs_lock.
* Another context removes a sysfs attribute while holding a mutex:
blk_unregister_queue() removes the queue sysfs attributes while holding q->sysfs_lock.

This can result in a real deadlock because the code that removes sysfs objects waits until all ongoing attribute callbacks have finished.

Since commit 667257e8b298 ("block: properly protect the 'queue' kobj in blk_unregister_queue") modified blk_unregister_queue() such that q->sysfs_lock is held around the kobject_del(&q->kobj) call I think this is a regression introduced by that commit.

Quoted 28 lines · Bart Van Assche

Quoted 11 lines · Mike Snitzer

sysfs write op calls kernfs_fop_write which takes:
of->mutex then kn->count#213 (no idea what that is) then q->sysfs_lock (via queue_attr_store)

vs

blk_unregister_queue takes:
q->sysfs_lock then
kernfs_mutex (via kernfs_remove)
seems lockdep thinks "kernfs_mutex" is "kn->count#213"?

Quoted 5 lines

Feels like lockdep code in fs/kernfs/file.c and fs/kernfs/dir.c is triggering false positives.. because these seem like different kernfs locks yet they are reported as "kn->count#213".

Certainly feeling out of my depth with kernfs's locking though.

Hello Mike,

I don't think that this is a false positive but rather the following traditional pattern of a potential deadlock involving sysfs attributes:
* One context obtains a mutex from inside a sysfs attribute method:
queue_attr_store() obtains q->sysfs_lock.
* Another context removes a sysfs attribute while holding a mutex:
blk_unregister_queue() removes the queue sysfs attributes while holding q->sysfs_lock.

Quoted 7 lines

This can result in a real deadlock because the code that removes sysfs objects waits until all ongoing attribute callbacks have finished.

Since commit 667257e8b298 ("block: properly protect the 'queue' kobj in blk_unregister_queue") modified blk_unregister_queue() such that q->sysfs_lock is held around the kobject_del(&q->kobj) call I think this is a regression introduced by that commit.

Sure, of course it is a regression.

Aside from moving the mutex_unlock(&q->sysfs_lock) above the kobject_del(&q->kobj) I don't know how to fix it.

Though, realistically that'd be an adequate fix (given the way the code was before).

Quoted 39 lines · Mike Snitzer

Quoted 28 lines · Bart Van Assche

Quoted 11 lines · Mike Snitzer

sysfs write op calls kernfs_fop_write which takes:
of->mutex then kn->count#213 (no idea what that is) then q->sysfs_lock (via queue_attr_store)

vs

blk_unregister_queue takes:
q->sysfs_lock then
kernfs_mutex (via kernfs_remove)
seems lockdep thinks "kernfs_mutex" is "kn->count#213"?

Quoted 5 lines

Feels like lockdep code in fs/kernfs/file.c and fs/kernfs/dir.c is triggering false positives.. because these seem like different kernfs locks yet they are reported as "kn->count#213".

Certainly feeling out of my depth with kernfs's locking though.

Hello Mike,

I don't think that this is a false positive but rather the following traditional pattern of a potential deadlock involving sysfs attributes:
* One context obtains a mutex from inside a sysfs attribute method:
queue_attr_store() obtains q->sysfs_lock.
* Another context removes a sysfs attribute while holding a mutex:
blk_unregister_queue() removes the queue sysfs attributes while holding q->sysfs_lock.

Quoted 7 lines

This can result in a real deadlock because the code that removes sysfs objects waits until all ongoing attribute callbacks have finished.

Since commit 667257e8b298 ("block: properly protect the 'queue' kobj in blk_unregister_queue") modified blk_unregister_queue() such that q->sysfs_lock is held around the kobject_del(&q->kobj) call I think this is a regression introduced by that commit.

Quoted 7 lines

Sure, of course it is a regression.

Aside from moving the mutex_unlock(&q->sysfs_lock) above the kobject_del(&q->kobj) I don't know how to fix it.

Though, realistically that'd be an adequate fix (given the way the code was before).

Any chance you apply this and re-run your srp_test that triggered the lockdep splat?

block/blk-sysfs.c +0 -0

index 4a6a40ffd78e..c50e08e9bf17 100644
@@ -952,10 +952,10 @@ void blk_unregister_queue(struct gendisk *disk)
        if (q->request_fn || (q->mq_ops && q->elevator))
                elv_unregister_queue(q);

Quoted 39 lines · Mike Snitzer

Quoted 28 lines · Bart Van Assche

Quoted 11 lines · Mike Snitzer

sysfs write op calls kernfs_fop_write which takes:
of->mutex then kn->count#213 (no idea what that is) then q->sysfs_lock (via queue_attr_store)

vs

blk_unregister_queue takes:
q->sysfs_lock then
kernfs_mutex (via kernfs_remove)
seems lockdep thinks "kernfs_mutex" is "kn->count#213"?

Quoted 5 lines

Feels like lockdep code in fs/kernfs/file.c and fs/kernfs/dir.c is triggering false positives.. because these seem like different kernfs locks yet they are reported as "kn->count#213".

Certainly feeling out of my depth with kernfs's locking though.

Hello Mike,

I don't think that this is a false positive but rather the following traditional pattern of a potential deadlock involving sysfs attributes:
* One context obtains a mutex from inside a sysfs attribute method:
queue_attr_store() obtains q->sysfs_lock.
* Another context removes a sysfs attribute while holding a mutex:
blk_unregister_queue() removes the queue sysfs attributes while holding q->sysfs_lock.

Quoted 7 lines

This can result in a real deadlock because the code that removes sysfs objects waits until all ongoing attribute callbacks have finished.

Since commit 667257e8b298 ("block: properly protect the 'queue' kobj in blk_unregister_queue") modified blk_unregister_queue() such that q->sysfs_lock is held around the kobject_del(&q->kobj) call I think this is a regression introduced by that commit.

Quoted 7 lines

Sure, of course it is a regression.

Aside from moving the mutex_unlock(&q->sysfs_lock) above the kobject_del(&q->kobj) I don't know how to fix it.

Though, realistically that'd be an adequate fix (given the way the code was before).

Any chance you apply this and re-run your srp_test that triggered the lockdep splat?

block/blk-sysfs.c +0 -0

index 4a6a40ffd78e..c50e08e9bf17 100644
@@ -952,10 +952,10 @@ void blk_unregister_queue(struct gendisk *disk)
 	if (q->request_fn || (q->mq_ops && q->elevator))
 		elv_unregister_queue(q);