Skip to contentSkip to conversation

block: don't acquire .sysfs_lock before removing mq & iosched kobjects

PATCH Superseded Ming Lei 2 participants 8 messages Aug 26, 2019 – Aug 27, 2019 Block layer

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Ming Lei

author
#0 lore raw

v3 cover block: don't acquire .sysfs_lock before removing mq & iosched kobjects

Hi,

The 1st 3 patches cleans up current uses on q->sysfs_lock.

The 4th patch adds one helper for checking if queue is registered.

The last patch splits .sysfs_lock into two locks: one is only for sync .store/.show from sysfs, the other one is for pretecting kobjects registering/unregistering. Meantime avoid to acquire .sysfs_lock when removing mq & iosched kobjects, so that the reported deadlock can be fixed.

V3:
- drop the 4th patch in V2, which is wrong, meantime not necesary for fixing this deadlock
- replace comment with one WARN_ON_ONCE() in patch 2
- add reviewed-by tag

V2:
- remove several uses on .sysfs_lock
- Remove blk_mq_register_dev()
- add one helper for checking queue registered
- split .sysfs_lock into two locks

Bart Van Assche (1):
block: Remove blk_mq_register_dev()

Ming Lei (4):
block: don't hold q->sysfs_lock in elevator_init_mq
blk-mq: don't hold q->sysfs_lock in blk_mq_map_swqueue
block: add helper for checking if queue is registered
block: split .sysfs_lock into two locks

 block/blk-core.c       |  1 +
 block/blk-mq-sysfs.c   | 23 ++++------------
 block/blk-mq.c         |  7 -----
 block/blk-sysfs.c      | 50 +++++++++++++++++++++-------------
 block/blk-wbt.c        |  2 +-
 block/blk.h            |  2 +-
 block/elevator.c       | 62 ++++++++++++++++++++++++++++++------------
 include/linux/blk-mq.h |  1 -
 include/linux/blkdev.h |  2 ++
 9 files changed, 86 insertions(+), 64 deletions(-)
Cc Christoph Hellwig <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Greg KH <[email protected]>
Cc Mike Snitzer <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Damien Le Moal <[email protected]>
Signature · 1 line
2.20.1
author
#1 lore raw

PATCH v3 1/5 block: Remove blk_mq_register_dev()

From: Bart Van Assche <[email protected]>

This function has no callers. Hence remove it.

Cc Christoph Hellwig <[email protected]>
Cc Ming Lei <[email protected]>
Cc Hannes Reinecke <[email protected]>
---
 block/blk-mq-sysfs.c   | 11 -----------
 include/linux/blk-mq.h |  1 -
 2 files changed, 12 deletions(-)
block/blk-mq-sysfs.c +0 -11
index d6e1a9bd7131..6ddde3774ebe 100644
@@ -349,17 +349,6 @@ int __blk_mq_register_dev(struct device *dev, struct request_queue *q)
 	return ret;
 }
 
-int blk_mq_register_dev(struct device *dev, struct request_queue *q)
-{
-	int ret;
-
-	mutex_lock(&q->sysfs_lock);
-	ret = __blk_mq_register_dev(dev, q);
-	mutex_unlock(&q->sysfs_lock);
-
-	return ret;
-}
-
 void blk_mq_sysfs_unregister(struct request_queue *q)
 {
 	struct blk_mq_hw_ctx *hctx;
include/linux/blk-mq.h +0 -1
index 21cebe901ac0..62a3bb715899 100644
@@ -253,7 +253,6 @@ struct request_queue *blk_mq_init_sq_queue(struct blk_mq_tag_set *set,
 						const struct blk_mq_ops *ops,
 						unsigned int queue_depth,
 						unsigned int set_flags);
-int blk_mq_register_dev(struct device *, struct request_queue *);
 void blk_mq_unregister_dev(struct device *, struct request_queue *);
 
 int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set);
Signature · 1 line
2.20.1
author
#2 lore raw

PATCH v3 2/5 block: don't hold q->sysfs_lock in elevator_init_mq

In reply to Ming Lei

The original comment says:

	q->sysfs_lock must be held to provide mutual exclusion between
	elevator_switch() and here.

Which is simply wrong. elevator_init_mq() is only called from blk_mq_init_allocated_queue, which is always called before the request queue is registered via blk_register_queue(), for dm-rq or normal rq based driver. However, queue's kobject is only exposed and added to sysfs in blk_register_queue(). So there isn't such race between elevator_switch() and elevator_init_mq().

So avoid to hold q->sysfs_lock in elevator_init_mq().

Cc Christoph Hellwig <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Greg KH <[email protected]>
Cc Mike Snitzer <[email protected]>
Cc Bart Van Assche <[email protected]>
Cc Damien Le Moal <[email protected]>
---
 block/elevator.c | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)
block/elevator.c +5 -9
index 2f17d66d0e61..33c15fb54ed1 100644
@@ -607,23 +607,19 @@ int elevator_init_mq(struct request_queue *q)
 	if (q->nr_hw_queues != 1)
 		return 0;
 
-	/*
-	 * q->sysfs_lock must be held to provide mutual exclusion between
-	 * elevator_switch() and here.
-	 */
-	mutex_lock(&q->sysfs_lock);
+	WARN_ON_ONCE(test_bit(QUEUE_FLAG_REGISTERED, &q->queue_flags));
+
 	if (unlikely(q->elevator))
-		goto out_unlock;
+		goto out;
 
 	e = elevator_get(q, "mq-deadline", false);
 	if (!e)
-		goto out_unlock;
+		goto out;
 
 	err = blk_mq_init_sched(q, e);
 	if (err)
 		elevator_put(e);
-out_unlock:
-	mutex_unlock(&q->sysfs_lock);
+out:
 	return err;
 }
 
Signature · 1 line
2.20.1
author
#3 lore raw

PATCH v3 3/5 blk-mq: don't hold q->sysfs_lock in blk_mq_map_swqueue

In reply to Ming Lei

blk_mq_map_swqueue() is called from blk_mq_init_allocated_queue() and blk_mq_update_nr_hw_queues(). For the former caller, the kobject isn't exposed to userspace yet. For the latter caller, hctx sysfs entries and debugfs are un-registered before updating nr_hw_queues.

On the other hand, commit 2f8f1336a48b ("blk-mq: always free hctx after request queue is freed") moves freeing hctx into queue's release handler, so there won't be race with queue release path too.

So don't hold q->sysfs_lock in blk_mq_map_swqueue().

Cc Christoph Hellwig <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Greg KH <[email protected]>
Cc Mike Snitzer <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 block/blk-mq.c | 7 -------
 1 file changed, 7 deletions(-)
block/blk-mq.c +0 -7
index 6968de9d7402..b0ee0cac737f 100644
@@ -2456,11 +2456,6 @@ static void blk_mq_map_swqueue(struct request_queue *q)
 	struct blk_mq_ctx *ctx;
 	struct blk_mq_tag_set *set = q->tag_set;
 
-	/*
-	 * Avoid others reading imcomplete hctx->cpumask through sysfs
-	 */
-	mutex_lock(&q->sysfs_lock);
-
 	queue_for_each_hw_ctx(q, hctx, i) {
 		cpumask_clear(hctx->cpumask);
 		hctx->nr_ctx = 0;
@@ -2521,8 +2516,6 @@ static void blk_mq_map_swqueue(struct request_queue *q)
 					HCTX_TYPE_DEFAULT, i);
 	}
 
-	mutex_unlock(&q->sysfs_lock);
-
 	queue_for_each_hw_ctx(q, hctx, i) {
 		/*
 		 * If no software queues are mapped to this hardware queue,
Signature · 1 line
2.20.1
author
#4 lore raw

PATCH v3 4/5 block: add helper for checking if queue is registered

In reply to Ming Lei

There are 4 users which check if queue is registered, so add one helper to check it.

Cc Christoph Hellwig <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Greg KH <[email protected]>
Cc Mike Snitzer <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 block/blk-sysfs.c      | 4 ++--
 block/blk-wbt.c        | 2 +-
 block/elevator.c       | 2 +-
 include/linux/blkdev.h | 1 +
 4 files changed, 5 insertions(+), 4 deletions(-)
block/blk-sysfs.c +2 -2
index 977c659dcd18..5b0b5224cfd4 100644
@@ -942,7 +942,7 @@ int blk_register_queue(struct gendisk *disk)
 	if (WARN_ON(!q))
 		return -ENXIO;
 
-	WARN_ONCE(test_bit(QUEUE_FLAG_REGISTERED, &q->queue_flags),
+	WARN_ONCE(blk_queue_registered(q),
 		  "%s is registering an already registered queue\n",
 		  kobject_name(&dev->kobj));
 	blk_queue_flag_set(QUEUE_FLAG_REGISTERED, q);
@@ -1026,7 +1026,7 @@ void blk_unregister_queue(struct gendisk *disk)
 		return;
 
 	/* Return early if disk->queue was never registered. */
-	if (!test_bit(QUEUE_FLAG_REGISTERED, &q->queue_flags))
+	if (!blk_queue_registered(q))
 		return;
 
 	/*
block/blk-wbt.c +1 -1
index 313f45a37e9d..c4d3089e47f7 100644
@@ -656,7 +656,7 @@ void wbt_enable_default(struct request_queue *q)
 		return;
 
 	/* Queue not registered? Maybe shutting down... */
-	if (!test_bit(QUEUE_FLAG_REGISTERED, &q->queue_flags))
+	if (!blk_queue_registered(q))
 		return;
 
 	if (queue_is_mq(q) && IS_ENABLED(CONFIG_BLK_WBT_MQ))
block/elevator.c +1 -1
index 33c15fb54ed1..03d923196569 100644
@@ -656,7 +656,7 @@ static int __elevator_change(struct request_queue *q, const char *name)
 	struct elevator_type *e;
 
 	/* Make sure queue is not in the middle of being removed */
-	if (!test_bit(QUEUE_FLAG_REGISTERED, &q->queue_flags))
+	if (!blk_queue_registered(q))
 		return -ENOENT;
 
 	/*
include/linux/blkdev.h +1 -0
index 167bf879f072..6041755984f4 100644
@@ -647,6 +647,7 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q);
 #define blk_queue_quiesced(q)	test_bit(QUEUE_FLAG_QUIESCED, &(q)->queue_flags)
 #define blk_queue_pm_only(q)	atomic_read(&(q)->pm_only)
 #define blk_queue_fua(q)	test_bit(QUEUE_FLAG_FUA, &(q)->queue_flags)
+#define blk_queue_registered(q)	test_bit(QUEUE_FLAG_REGISTERED, &(q)->queue_flags)
 
 extern void blk_set_pm_only(struct request_queue *q);
 extern void blk_clear_pm_only(struct request_queue *q);
Signature · 1 line
2.20.1
author
#5 lore raw

PATCH v3 5/5 block: split .sysfs_lock into two locks

In reply to Ming Lei

The kernfs built-in lock of 'kn->count' is held in sysfs .show/.store path. Meantime, inside block's .show/.store callback, q->sysfs_lock is required.

However, when mq & iosched kobjects are removed via blk_mq_unregister_dev() & elv_unregister_queue(), q->sysfs_lock is held too. This way causes AB-BA lock because the kernfs built-in lock of 'kn-count' is required inside kobject_del() too, see the lockdep warning[1].

On the other hand, it isn't necessary to acquire q->sysfs_lock for both blk_mq_unregister_dev() & elv_unregister_queue() because clearing REGISTERED flag prevents storing to 'queue/scheduler' from being happened. Also sysfs write(store) is exclusive, so no necessary to hold the lock for elv_unregister_queue() when it is called in switching elevator path.

So split .sysfs_lock into two: one is still named as .sysfs_lock for covering sync .store, the other one is named as .sysfs_dir_lock for covering kobjects and related status change.

sysfs itself can handle the race between add/remove kobjects and showing/storing attributes under kobjects. For switching scheduler via storing to 'queue/scheduler', we use the queue flag of QUEUE_FLAG_REGISTERED with .sysfs_lock for avoiding the race, then we can avoid to hold .sysfs_lock during removing/adding kobjects.

[1] lockdep warning

    ======================================================
    WARNING: possible circular locking dependency detected
    5.3.0-rc3-00044-g73277fc75ea0 #1380 Not tainted
    ------------------------------------------------------
    rmmod/777 is trying to acquire lock:
    00000000ac50e981 (kn->count#202){++++}, at: kernfs_remove_by_name_ns+0x59/0x72

    but task is already holding lock:
    00000000fb16ae21 (&q->sysfs_lock){+.+.}, at: blk_unregister_queue+0x78/0x10b

    which lock already depends on the new lock.

    the existing dependency chain (in reverse order) is:

    -> #1 (&q->sysfs_lock){+.+.}:
           __lock_acquire+0x95f/0xa2f
           lock_acquire+0x1b4/0x1e8
           __mutex_lock+0x14a/0xa9b
           blk_mq_hw_sysfs_show+0x63/0xb6
           sysfs_kf_seq_show+0x11f/0x196
           seq_read+0x2cd/0x5f2
           vfs_read+0xc7/0x18c
           ksys_read+0xc4/0x13e
           do_syscall_64+0xa7/0x295
           entry_SYSCALL_64_after_hwframe+0x49/0xbe

    -> #0 (kn->count#202){++++}:
           check_prev_add+0x5d2/0xc45
           validate_chain+0xed3/0xf94
           __lock_acquire+0x95f/0xa2f
           lock_acquire+0x1b4/0x1e8
           __kernfs_remove+0x237/0x40b
           kernfs_remove_by_name_ns+0x59/0x72
           remove_files+0x61/0x96
           sysfs_remove_group+0x81/0xa4
           sysfs_remove_groups+0x3b/0x44
           kobject_del+0x44/0x94
           blk_mq_unregister_dev+0x83/0xdd
           blk_unregister_queue+0xa0/0x10b
           del_gendisk+0x259/0x3fa
           null_del_dev+0x8b/0x1c3 [null_blk]
           null_exit+0x5c/0x95 [null_blk]
           __se_sys_delete_module+0x204/0x337
           do_syscall_64+0xa7/0x295
           entry_SYSCALL_64_after_hwframe+0x49/0xbe

    other info that might help us debug this:

     Possible unsafe locking scenario:

           CPU0                    CPU1
           ----                    ----
      lock(&q->sysfs_lock);
                                   lock(kn->count#202);
                                   lock(&q->sysfs_lock);
      lock(kn->count#202);

     *** DEADLOCK ***

    2 locks held by rmmod/777:
     #0: 00000000e69bd9de (&lock){+.+.}, at: null_exit+0x2e/0x95 [null_blk]
     #1: 00000000fb16ae21 (&q->sysfs_lock){+.+.}, at: blk_unregister_queue+0x78/0x10b

    stack backtrace:
    CPU: 0 PID: 777 Comm: rmmod Not tainted 5.3.0-rc3-00044-g73277fc75ea0 #1380
    Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS ?-20180724_192412-buildhw-07.phx4
    Call Trace:
     dump_stack+0x9a/0xe6
     check_noncircular+0x207/0x251
     ? print_circular_bug+0x32a/0x32a
     ? find_usage_backwards+0x84/0xb0
     check_prev_add+0x5d2/0xc45
     validate_chain+0xed3/0xf94
     ? check_prev_add+0xc45/0xc45
     ? mark_lock+0x11b/0x804
     ? check_usage_forwards+0x1ca/0x1ca
     __lock_acquire+0x95f/0xa2f
     lock_acquire+0x1b4/0x1e8
     ? kernfs_remove_by_name_ns+0x59/0x72
     __kernfs_remove+0x237/0x40b
     ? kernfs_remove_by_name_ns+0x59/0x72
     ? kernfs_next_descendant_post+0x7d/0x7d
     ? strlen+0x10/0x23
     ? strcmp+0x22/0x44
     kernfs_remove_by_name_ns+0x59/0x72
     remove_files+0x61/0x96
     sysfs_remove_group+0x81/0xa4
     sysfs_remove_groups+0x3b/0x44
     kobject_del+0x44/0x94
     blk_mq_unregister_dev+0x83/0xdd
     blk_unregister_queue+0xa0/0x10b
     del_gendisk+0x259/0x3fa
     ? disk_events_poll_msecs_store+0x12b/0x12b
     ? check_flags+0x1ea/0x204
     ? mark_held_locks+0x1f/0x7a
     null_del_dev+0x8b/0x1c3 [null_blk]
     null_exit+0x5c/0x95 [null_blk]
     __se_sys_delete_module+0x204/0x337
     ? free_module+0x39f/0x39f
     ? blkcg_maybe_throttle_current+0x8a/0x718
     ? rwlock_bug+0x62/0x62
     ? __blkcg_punt_bio_submit+0xd0/0xd0
     ? trace_hardirqs_on_thunk+0x1a/0x20
     ? mark_held_locks+0x1f/0x7a
     ? do_syscall_64+0x4c/0x295
     do_syscall_64+0xa7/0x295
     entry_SYSCALL_64_after_hwframe+0x49/0xbe
    RIP: 0033:0x7fb696cdbe6b
    Code: 73 01 c3 48 8b 0d 1d 20 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 008
    RSP: 002b:00007ffec9588788 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0
    RAX: ffffffffffffffda RBX: 0000559e589137c0 RCX: 00007fb696cdbe6b
    RDX: 000000000000000a RSI: 0000000000000800 RDI: 0000559e58913828
    RBP: 0000000000000000 R08: 00007ffec9587701 R09: 0000000000000000
    R10: 00007fb696d4eae0 R11: 0000000000000206 R12: 00007ffec95889b0
    R13: 00007ffec95896b3 R14: 0000559e58913260 R15: 0000559e589137c0
Cc Christoph Hellwig <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Greg KH <[email protected]>
Cc Mike Snitzer <[email protected]>
Cc Bart Van Assche <[email protected]>
---
 block/blk-core.c       |  1 +
 block/blk-mq-sysfs.c   | 12 +++++------
 block/blk-sysfs.c      | 46 ++++++++++++++++++++++++++----------------
 block/blk.h            |  2 +-
 block/elevator.c       | 46 ++++++++++++++++++++++++++++++++++--------
 include/linux/blkdev.h |  1 +
 6 files changed, 76 insertions(+), 32 deletions(-)
block/blk-core.c +1 -0
index 919629ce4015..2792f7cf7bef 100644
@@ -520,6 +520,7 @@ struct request_queue *blk_alloc_queue_node(gfp_t gfp_mask, int node_id)
 	mutex_init(&q->blk_trace_mutex);
 #endif
 	mutex_init(&q->sysfs_lock);
+	mutex_init(&q->sysfs_dir_lock);
 	spin_lock_init(&q->queue_lock);
 
 	init_waitqueue_head(&q->mq_freeze_wq);
block/blk-mq-sysfs.c +6 -6
index 6ddde3774ebe..a0d3ce30fa08 100644
@@ -270,7 +270,7 @@ void blk_mq_unregister_dev(struct device *dev, struct request_queue *q)
 	struct blk_mq_hw_ctx *hctx;
 	int i;
 
-	lockdep_assert_held(&q->sysfs_lock);
+	lockdep_assert_held(&q->sysfs_dir_lock);
 
 	queue_for_each_hw_ctx(q, hctx, i)
 		blk_mq_unregister_hctx(hctx);
@@ -320,7 +320,7 @@ int __blk_mq_register_dev(struct device *dev, struct request_queue *q)
 	int ret, i;
 
 	WARN_ON_ONCE(!q->kobj.parent);
-	lockdep_assert_held(&q->sysfs_lock);
+	lockdep_assert_held(&q->sysfs_dir_lock);
 
 	ret = kobject_add(q->mq_kobj, kobject_get(&dev->kobj), "%s", "mq");
 	if (ret < 0)
@@ -354,7 +354,7 @@ void blk_mq_sysfs_unregister(struct request_queue *q)
 	struct blk_mq_hw_ctx *hctx;
 	int i;
 
-	mutex_lock(&q->sysfs_lock);
+	mutex_lock(&q->sysfs_dir_lock);
 	if (!q->mq_sysfs_init_done)
 		goto unlock;
 
@@ -362,7 +362,7 @@ void blk_mq_sysfs_unregister(struct request_queue *q)
 		blk_mq_unregister_hctx(hctx);
 
 unlock:
-	mutex_unlock(&q->sysfs_lock);
+	mutex_unlock(&q->sysfs_dir_lock);
 }
 
 int blk_mq_sysfs_register(struct request_queue *q)
@@ -370,7 +370,7 @@ int blk_mq_sysfs_register(struct request_queue *q)
 	struct blk_mq_hw_ctx *hctx;
 	int i, ret = 0;
 
-	mutex_lock(&q->sysfs_lock);
+	mutex_lock(&q->sysfs_dir_lock);
 	if (!q->mq_sysfs_init_done)
 		goto unlock;
 
@@ -381,7 +381,7 @@ int blk_mq_sysfs_register(struct request_queue *q)
 	}
 
 unlock:
-	mutex_unlock(&q->sysfs_lock);
+	mutex_unlock(&q->sysfs_dir_lock);
 
 	return ret;
 }
block/blk-sysfs.c +29 -17

106 lines not on this page · raw message

block/blk.h +1 -1
index de6b2e146d6e..e4619fc5c99a 100644
@@ -188,7 +188,7 @@ int elevator_init_mq(struct request_queue *q);
 int elevator_switch_mq(struct request_queue *q,
 			      struct elevator_type *new_e);
 void __elevator_exit(struct request_queue *, struct elevator_queue *);
-int elv_register_queue(struct request_queue *q);
+int elv_register_queue(struct request_queue *q, bool uevent);
 void elv_unregister_queue(struct request_queue *q);
 
 static inline void elevator_exit(struct request_queue *q,
block/elevator.c +38 -8

92 lines not on this page · raw message

include/linux/blkdev.h +1 -0
index 6041755984f4..e271c3a176fa 100644
@@ -539,6 +539,7 @@ struct request_queue {
 	struct delayed_work	requeue_work;
 
 	struct mutex		sysfs_lock;
+	struct mutex		sysfs_dir_lock;
 
 	/*
 	 * for reusing dead hctx instance in case of updating
Signature · 1 line
2.20.1
#6 lore raw

Re: [PATCH V3 5/5] block: split .sysfs_lock into two locks

Quoted 21 lines · Ming Lei
block/blk-sysfs.c +1 -1
index 5b0b5224cfd4..5941a0176f87 100644
@@ -938,6 +938,7 @@ int blk_register_queue(struct gendisk *disk)
  	int ret;
  	struct device *dev = disk_to_dev(disk);
  	struct request_queue *q = disk->queue;
+	bool has_elevator = false;
 
  	if (WARN_ON(!q))
  		return -ENXIO;
@@ -945,7 +946,6 @@ int blk_register_queue(struct gendisk *disk)
  	WARN_ONCE(blk_queue_registered(q),
  		  "%s is registering an already registered queue\n",
  		  kobject_name(&dev->kobj));
-	blk_queue_flag_set(QUEUE_FLAG_REGISTERED, q);
 
  	/*
  	 * SCSI probing may synchronously create and destroy a lot of

block/blk-sysfs.c int blk_register_queue(struct gendisk *disk) · line 969

Quoted 5 lines
block/blk-sysfs.c +1 -1
@@ -966,7 +966,7 @@ int blk_register_queue(struct gendisk *disk)
  		return ret;
 
  	/* Prevent changes through sysfs until registration is completed. */
-	mutex_lock(&q->sysfs_lock);
+	mutex_lock(&q->sysfs_dir_lock);

Does mutex_lock(&q->sysfs_dir_lock) really protect against changes of the I/O scheduler through sysfs or does it only protect against concurrent sysfs object creation and removal? In other words, should the comment above this mutex lock call be updated?

Quoted 19 lines
block/blk-sysfs.c +6 -7
@@ -987,26 +987,37 @@ int blk_register_queue(struct gendisk *disk)
  		blk_mq_debugfs_register(q);
  	}
 
-	kobject_uevent(&q->kobj, KOBJ_ADD);
-
-	wbt_enable_default(q);
-
-	blk_throtl_register_queue(q);
-
+	/*
+	 * The queue's kobject ADD uevent isn't sent out, also the
+	 * flag of QUEUE_FLAG_REGISTERED isn't set yet, so elevator
+	 * switch won't happen at all.
+	 */
  	if (q->elevator) {
-		ret = elv_register_queue(q);
+		ret = elv_register_queue(q, false);
  		if (ret) {

block/blk-sysfs.c int blk_register_queue(struct gendisk *disk) · line 1005

Quoted 10 lines
block/blk-sysfs.c +2 -2
-			mutex_unlock(&q->sysfs_lock);
-			kobject_uevent(&q->kobj, KOBJ_REMOVE);
+			mutex_unlock(&q->sysfs_dir_lock);
  			kobject_del(&q->kobj);
  			blk_trace_remove_sysfs(dev);
  			kobject_put(&dev->kobj);
  			return ret;
  		}
+		has_elevator = true;
  	}

I think the reference to the kobject ADD event in the comment is misleading. If e.g. a request queue is registered, unregistered and reregistered quickly, can it happen that a udev rule for the ADD event triggered by the first registration is executed in the middle of the second registration? Is setting the REGISTERED flag later sufficient to fix the race against scheduler changes through sysfs? If so, how about leaving out the reference to the kobject ADD event from the above comment?

block/blk-sysfs.c int blk_register_queue(struct gendisk *disk) · line 1016

Quoted 10 lines
block/blk-sysfs.c +10 -0
+	mutex_lock(&q->sysfs_lock);
+	blk_queue_flag_set(QUEUE_FLAG_REGISTERED, q);
+	wbt_enable_default(q);
+	blk_throtl_register_queue(q);
+	mutex_unlock(&q->sysfs_lock);
+
+	/* Now everything is ready and send out KOBJ_ADD uevent */
+	kobject_uevent(&q->kobj, KOBJ_ADD);
+	if (has_elevator)
+		kobject_uevent(&q->elevator->kobj, KOBJ_ADD);

Can it happen that immediately after mutex_unlock(&q->sysfs_lock) a script removes the I/O scheduler and hence makes the value of the 'has_elevator' variable stale? In other words, should emitting KOBJ_ADD also be protected by sysfs_lock?

Quoted 26 lines
block/blk-sysfs.c +4 -2
@@ -1021,6 +1032,7 @@ EXPORT_SYMBOL_GPL(blk_register_queue);
  void blk_unregister_queue(struct gendisk *disk)
  {
  	struct request_queue *q = disk->queue;
+	bool has_elevator;
 
  	if (WARN_ON(!q))
  		return;
@@ -1035,25 +1047,25 @@ void blk_unregister_queue(struct gendisk *disk)
  	 * concurrent elv_iosched_store() calls.
  	 */
  	mutex_lock(&q->sysfs_lock);
-
  	blk_queue_flag_clear(QUEUE_FLAG_REGISTERED, q);
+	has_elevator = !!q->elevator;
+	mutex_unlock(&q->sysfs_lock);
 
+	mutex_lock(&q->sysfs_dir_lock);
  	/*
  	 * Remove the sysfs attributes before unregistering the queue data
  	 * structures that can be modified through sysfs.
  	 */
  	if (queue_is_mq(q))
  		blk_mq_unregister_dev(disk_to_dev(disk), q);
-	mutex_unlock(&q->sysfs_lock);
 

block/blk-sysfs.c void blk_unregister_queue(struct gendisk *disk) · line 1068

Quoted 10 lines
block/blk-sysfs.c +2 -3
  	kobject_uevent(&q->kobj, KOBJ_REMOVE);
  	kobject_del(&q->kobj);
  	blk_trace_remove_sysfs(disk_to_dev(disk));
 
-	mutex_lock(&q->sysfs_lock);
-	if (q->elevator)
+	if (has_elevator)
  		elv_unregister_queue(q);
-	mutex_unlock(&q->sysfs_lock);
+	mutex_unlock(&q->sysfs_dir_lock);

Is it safe to call elv_unregister_queue() if no I/O scheduler is associated with a request queue? If so, have you considered to leave out the 'has_elevator' variable from this function?

Quoted 13 lines
block/blk-sysfs.c +6 -1
@@ -567,10 +580,23 @@ int elevator_switch_mq(struct request_queue *q,
  	lockdep_assert_held(&q->sysfs_lock);
 
  	if (q->elevator) {
-		if (q->elevator->registered)
+		if (q->elevator->registered) {
+			mutex_unlock(&q->sysfs_lock);
+
  			elv_unregister_queue(q);
+
+			mutex_lock(&q->sysfs_lock);
+		}
  		ioc_clear_queue(q);

block/blk-sysfs.c int elevator_switch_mq(struct request_queue *q, · line 600

Quoted 10 lines
block/blk-sysfs.c +8 -0
  		elevator_exit(q, q->elevator);
+
+		/*
+		 * sysfs_lock may be dropped, so re-check if queue is
+		 * unregistered. If yes, don't switch to new elevator
+		 * any more
+		 */
+		if (!blk_queue_registered(q))
+			return 0;
  	}

So elevator_switch_mq() is called with sysfs_lock held and releases and reacquires that mutex? What will happen if e.g. syzbot writes into /sys/block/*/queue/scheduler from multiple threads concurrently? Can that lead to multiple concurrent calls of elv_register_queue() and elv_unregister_queue()? Can that e.g. cause concurrent calls of the following code from elv_register_queue(): kobject_add(&e->kobj, &q->kobj, "%s", "iosched")?

Is it even possible to fix this lock inversion by introducing only one new mutex? I think the sysfs directories and attributes referenced by this patch are as follows:

/sys/block/<q>/queue
/sys/block/<q>/queue/attr
/sys/block/<q>/queue/iosched/attr
/sys/block/<q>/mq
/sys/block/<q>/mq/<n>
/sys/block/<q>/mq/<n>/attr

Isn't the traditional approach to protect such a hierarchy to use one mutex per level? E.g. one mutex to serialize "queue" and "mq" manipulations (sysfs_dir_lock?), one mutex to protect the queue/attr attributes (sysfs_lock?), one mutex to serialize kobj creation in the mq directory, one mutex to protect the mq/<n>/attr attributes and one mutex to protect the I/O scheduler attributes?

Thanks,

Bart.

author
#7 lore raw

Re: [PATCH V3 5/5] block: split .sysfs_lock into two locks

Quoted 20 lines · Bart Van Assche
Quoted 19 lines · Ming Lei
block/blk-sysfs.c +1 -1
index 5b0b5224cfd4..5941a0176f87 100644
@@ -938,6 +938,7 @@ int blk_register_queue(struct gendisk *disk)
  	int ret;
  	struct device *dev = disk_to_dev(disk);
  	struct request_queue *q = disk->queue;
+	bool has_elevator = false;
  	if (WARN_ON(!q))
  		return -ENXIO;
@@ -945,7 +946,6 @@ int blk_register_queue(struct gendisk *disk)
  	WARN_ONCE(blk_queue_registered(q),
  		  "%s is registering an already registered queue\n",
  		  kobject_name(&dev->kobj));
-	blk_queue_flag_set(QUEUE_FLAG_REGISTERED, q);
  	/*
  	 * SCSI probing may synchronously create and destroy a lot of
@@ -966,7 +966,7 @@ int blk_register_queue(struct gendisk *disk)
Quoted 8 lines

block/blk-sysfs.c int blk_register_queue(struct gendisk *disk) · line 969

Quoted 4 lines
block/blk-sysfs.c +1 -1
  		return ret;
  	/* Prevent changes through sysfs until registration is completed. */
-	mutex_lock(&q->sysfs_lock);
+	mutex_lock(&q->sysfs_dir_lock);

Does mutex_lock(&q->sysfs_dir_lock) really protect against changes of the I/O scheduler through sysfs or does it only protect against concurrent sysfs object creation and removal?

It is only for protecting against concurrent sysfs object creation and removal.

Quoted 2 lines

In other words, should the comment above this
mutex lock call be updated?

Yeah, it should be removed.

Quoted 26 lines
Quoted 26 lines
block/blk-sysfs.c +7 -9
@@ -987,26 +987,37 @@ int blk_register_queue(struct gendisk *disk)
  		blk_mq_debugfs_register(q);
  	}
-	kobject_uevent(&q->kobj, KOBJ_ADD);
-
-	wbt_enable_default(q);
-
-	blk_throtl_register_queue(q);
-
+	/*
+	 * The queue's kobject ADD uevent isn't sent out, also the
+	 * flag of QUEUE_FLAG_REGISTERED isn't set yet, so elevator
+	 * switch won't happen at all.
+	 */
  	if (q->elevator) {
-		ret = elv_register_queue(q);
+		ret = elv_register_queue(q, false);
  		if (ret) {
-			mutex_unlock(&q->sysfs_lock);
-			kobject_uevent(&q->kobj, KOBJ_REMOVE);
+			mutex_unlock(&q->sysfs_dir_lock);
  			kobject_del(&q->kobj);
  			blk_trace_remove_sysfs(dev);
  			kobject_put(&dev->kobj);
  			return ret;
Quoted 8 lines

block/blk-sysfs.c int blk_register_queue(struct gendisk *disk) · line 1005

Quoted 3 lines
block/blk-sysfs.c +1 -0
  		}
+		has_elevator = true;
  	}

I think the reference to the kobject ADD event in the comment is misleading. If e.g. a request queue is registered, unregistered and reregistered quickly, can it happen that a udev rule for the ADD event triggered by the first registration is executed in the middle of the second registration? Is

It should happen, but this patch doesn't change anything about this behavior.

Quoted 2 lines

setting the REGISTERED flag later sufficient to fix the race against scheduler changes through sysfs?

Yes, it is enough.

Quoted 2 lines

If so, how about leaving out the reference
to the kobject ADD event from the above comment?

OK.

Quoted 16 lines

block/blk-sysfs.c int blk_register_queue(struct gendisk *disk) · line 1016

Quoted 11 lines
block/blk-sysfs.c +10 -0
+	mutex_lock(&q->sysfs_lock);
+	blk_queue_flag_set(QUEUE_FLAG_REGISTERED, q);
+	wbt_enable_default(q);
+	blk_throtl_register_queue(q);
+	mutex_unlock(&q->sysfs_lock);
+
+	/* Now everything is ready and send out KOBJ_ADD uevent */
+	kobject_uevent(&q->kobj, KOBJ_ADD);
+	if (has_elevator)
+		kobject_uevent(&q->elevator->kobj, KOBJ_ADD);

Can it happen that immediately after mutex_unlock(&q->sysfs_lock) a script removes the I/O scheduler and hence makes the value of the 'has_elevator' variable stale? In other words, should emitting KOBJ_ADD also be protected by sysfs_lock?

Good catch, it could be fine to hold syfs_lock for emitting KOBJ_ADD.

Quoted 28 lines
Quoted 28 lines
block/blk-sysfs.c +4 -3
@@ -1021,6 +1032,7 @@ EXPORT_SYMBOL_GPL(blk_register_queue);
  void blk_unregister_queue(struct gendisk *disk)
  {
  	struct request_queue *q = disk->queue;
+	bool has_elevator;
  	if (WARN_ON(!q))
  		return;
@@ -1035,25 +1047,25 @@ void blk_unregister_queue(struct gendisk *disk)
  	 * concurrent elv_iosched_store() calls.
  	 */
  	mutex_lock(&q->sysfs_lock);
-
  	blk_queue_flag_clear(QUEUE_FLAG_REGISTERED, q);
+	has_elevator = !!q->elevator;
+	mutex_unlock(&q->sysfs_lock);
+	mutex_lock(&q->sysfs_dir_lock);
  	/*
  	 * Remove the sysfs attributes before unregistering the queue data
  	 * structures that can be modified through sysfs.
  	 */
  	if (queue_is_mq(q))
  		blk_mq_unregister_dev(disk_to_dev(disk), q);
-	mutex_unlock(&q->sysfs_lock);
  	kobject_uevent(&q->kobj, KOBJ_REMOVE);
  	kobject_del(&q->kobj);
  	blk_trace_remove_sysfs(disk_to_dev(disk));
-	mutex_lock(&q->sysfs_lock);
Quoted 8 lines

block/blk-sysfs.c void blk_unregister_queue(struct gendisk *disk) · line 1068

Quoted 5 lines
block/blk-sysfs.c +2 -2
-	if (q->elevator)
+	if (has_elevator)
  		elv_unregister_queue(q);
-	mutex_unlock(&q->sysfs_lock);
+	mutex_unlock(&q->sysfs_dir_lock);

Is it safe to call elv_unregister_queue() if no I/O scheduler is associated with a request queue?

No, q->elevator has to be valid for elv_unregister_queue().

Quoted 20 lines

If so, have you considered to leave out the
'has_elevator' variable from this function?

Quoted 17 lines
block/blk-sysfs.c +10 -1
@@ -567,10 +580,23 @@ int elevator_switch_mq(struct request_queue *q,
  	lockdep_assert_held(&q->sysfs_lock);
  	if (q->elevator) {
-		if (q->elevator->registered)
+		if (q->elevator->registered) {
+			mutex_unlock(&q->sysfs_lock);
+
  			elv_unregister_queue(q);
+
+			mutex_lock(&q->sysfs_lock);
+		}
  		ioc_clear_queue(q);
  		elevator_exit(q, q->elevator);
+
+		/*
+		 * sysfs_lock may be dropped, so re-check if queue is
+		 * unregistered. If yes, don't switch to new elevator
Quoted 8 lines

block/blk-sysfs.c int elevator_switch_mq(struct request_queue *q, · line 599

Quoted 5 lines
block/blk-sysfs.c +4 -0
+		 * any more
+		 */
+		if (!blk_queue_registered(q))
+			return 0;
  	}

So elevator_switch_mq() is called with sysfs_lock held and releases and reacquires that mutex?

Yes.

Quoted 2 lines

What will happen if e.g. syzbot writes into /sys/block/*/queue/scheduler from multiple threads concurrently? Can that

It can't happen, sysfs's write on same file is always exclusively protected by one mutex, see kernfs_fop_write(), and should be same for normal fs too.

Quoted 4 lines

lead to multiple concurrent calls of elv_register_queue() and elv_unregister_queue()? Can that e.g. cause concurrent calls of the following code from elv_register_queue(): kobject_add(&e->kobj, &q->kobj, "%s", "iosched")?

No, it won't happen.

Quoted 12 lines

Is it even possible to fix this lock inversion by introducing only one new mutex? I think the sysfs directories and attributes referenced by this patch are as follows:

/sys/block/<q>/queue
/sys/block/<q>/queue/attr
/sys/block/<q>/queue/iosched/attr
/sys/block/<q>/mq
/sys/block/<q>/mq/<n>
/sys/block/<q>/mq/<n>/attr

Quoted 6 lines

Isn't the traditional approach to protect such a hierarchy to use one mutex per level? E.g. one mutex to serialize "queue" and "mq" manipulations (sysfs_dir_lock?), one mutex to protect the queue/attr attributes (sysfs_lock?), one mutex to serialize kobj creation in the mq directory, one mutex to protect the mq/<n>/attr attributes and one mutex to protect the I/O scheduler attributes?

This patch keeps to use sysfs_lock for protecting attributes show/write, meantime don't use it for serializing kobj creation & removal, so far looks good.

I will address your above comments and post V4 for further review.

Thanks,
Ming