block: fix race between adding wbt and normal IO
v3 cover block: fix race between adding wbt and normal IO
Hello,
Yi reported several kernel panics on:
[16687.001777] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000008 ... [16687.163549] pc : __rq_qos_track+0x38/0x60
or
[ 997.690455] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000020 ... [ 997.850347] pc : __rq_qos_done+0x2c/0x50
Turns out it is caused by race between adding wbt and normal IO.
Fix the issue by freezing request queue when adding/deleting rq qos.
V3:
- use ->queue_lock for protecting concurrent adding/deleting rqos on same queue
V2:
- switch to the approach of freezing queue, which is more generic than V1.
Ming Lei (2):
block: fix race between adding/removing rq qos and normal IO
block: mark queue init done at the end of blk_register_queue
block/blk-rq-qos.h | 24 ++++++++++++++++++++++++ block/blk-sysfs.c | 29 +++++++++++++++-------------- 2 files changed, 39 insertions(+), 14 deletions(-)
Signature · 1 line
2.31.1
PATCH v3 1/2 block: fix race between adding/removing rq qos and normal IO
Yi reported several kernel panics on:
[16687.001777] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000008 ... [16687.163549] pc : __rq_qos_track+0x38/0x60
or
[ 997.690455] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000020 ... [ 997.850347] pc : __rq_qos_done+0x2c/0x50
Turns out it is caused by race between adding rq qos(wbt) and normal IO because rq_qos_add can be run when IO is being submitted, fix this issue by freezing queue before adding/deleting rq qos to queue.
rq_qos_exit() needn't to freeze queue because it is called after queue has been frozen.
iolatency calls rq_qos_add() during allocating queue, so freezing won't add delay because queue usage refcount works at atomic mode at that time.
iocost calls rq_qos_add() when writing cgroup attribute file, that is fine to freeze queue at that time since we usually freeze queue when storing to queue sysfs attribute, meantime iocost only exists on the root cgroup.
wbt_init calls it in blk_register_queue() and queue sysfs attribute store(queue_wb_lat_store() when write it 1st time in case of !BLK_WBT_MQ), the following patch will speedup the queue freezing in wbt_init.
--- block/blk-rq-qos.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
block/blk-rq-qos.h +24 -0
@@ -7,6 +7,7 @@#include <linux/blk_types.h> #include <linux/atomic.h> #include <linux/wait.h> +#include <linux/blk-mq.h> #include "blk-mq-debugfs.h"
@@ -99,8 +100,21 @@ static inline void rq_wait_init(struct rq_wait *rq_wait)static inline void rq_qos_add (struct request_queue *q,struct rq_qos *rqos) { +/* +* No IO can be in-flight when adding rqos, so freeze queue, which +* is fine since we only support rq_qos for blk-mq queue. +* +* Reuse ->queue_lock for protecting against other concurrent +* rq_qos adding/deleting +*/ +blk_mq_freeze_queue (q); + +spin_lock_irq (&q->queue_lock); rqos->next = q->rq_qos; q->rq_qos = rqos; +spin_unlock_irq (&q->queue_lock); + +blk_mq_unfreeze_queue (q);if (rqos->ops->debugfs_attrs)blk_mq_debugfs_register_rqos (rqos);
@@ -110,12 +124,22 @@ static inline void rq_qos_del(struct request_queue *q, struct rq_qos *rqos) {struct rq_qos **cur; +/* +* See comment in rq_qos_add() about freezing queue & using +* ->queue_lock. +*/ +blk_mq_freeze_queue (q); + +spin_lock_irq (&q->queue_lock);for (cur = &q->rq_qos; *cur; cur = &(*cur)->next) {if (*cur == rqos) { *cur = rqos->next;break ; } } +spin_unlock_irq (&q->queue_lock); + +blk_mq_unfreeze_queue (q);blk_mq_debugfs_unregister_rqos (rqos); }
Signature · 1 line
2.31.1
Re: [PATCH V3 1/2] block: fix race between adding/removing rq qos and normal IO
Quoted 1 line · Ming Lei
Yi reported several kernel panics on [ ... ]
PATCH v3 2/2 block: mark queue init done at the end of blk_register_queue
In reply to Ming LeiMark queue init done when everything is done well in blk_register_queue(), so that wbt_enable_default() can be run quickly without any RCU period involved since adding rq qos requires to freeze queue.
Also no any side effect by delaying to mark queue init done.
--- block/blk-sysfs.c | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-)
block/blk-sysfs.c +15 -14
@@ -866,20 +866,6 @@ int blk_register_queue(struct gendisk *disk)"%s is registering an already registered queue\n" , kobject_name(&dev->kobj)); -/* -* SCSI probing may synchronously create and destroy a lot of -* request_queues for non-existent devices. Shutting down a fully -* functional queue takes measureable wallclock time as RCU grace -* periods are involved. To avoid excessive latency in these -* cases, a request_queue starts out in a degraded mode which is -* faster to shut down and is made fully functional here as -* request_queues for non-existent devices never get registered. -*/ -if (!blk_queue_init_done (q)) { -blk_queue_flag_set (QUEUE_FLAG_INIT_DONE , q); -percpu_ref_switch_to_percpu (&q->q_usage_counter); - } -blk_queue_update_readahead (q); ret =blk_trace_init_sysfs (dev);
@@ -938,6 +924,21 @@ int blk_register_queue(struct gendisk *disk) ret =0 ; unlock:mutex_unlock (&q->sysfs_dir_lock); + +/* +* SCSI probing may synchronously create and destroy a lot of +* request_queues for non-existent devices. Shutting down a fully +* functional queue takes measureable wallclock time as RCU grace +* periods are involved. To avoid excessive latency in these +* cases, a request_queue starts out in a degraded mode which is +* faster to shut down and is made fully functional here as +* request_queues for non-existent devices never get registered. +*/ +if (!blk_queue_init_done (q)) { +blk_queue_flag_set (QUEUE_FLAG_INIT_DONE , q); +percpu_ref_switch_to_percpu (&q->q_usage_counter); + } +return ret; }EXPORT_SYMBOL_GPL (blk_register_queue);
Signature · 1 line
2.31.1
Re: [PATCH V3 2/2] block: mark queue init done at the end of blk_register_queue
Quoted 3 lines · Ming Lei
Mark queue init done when everything is done well in
blk_register_queue(), so thatwbt_enable_default()can be run quickly without any RCU period involved since adding rq qos requires to freeze queue.
Thanks for the fix.
Quoted 42 lines · Ming Lei
Hello,
Yi reported several kernel panics on:
[16687.001777] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000008 ... [16687.163549] pc : __rq_qos_track+0x38/0x60or
[ 997.690455] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000020 ... [ 997.850347] pc : __rq_qos_done+0x2c/0x50Turns out it is caused by race between adding wbt and normal IO.
Fix the issue by freezing request queue when adding/deleting rq qos.
V3:
- use ->queue_lock for protecting concurrent adding/deleting rqos on same queueV2:
- switch to the approach of freezing queue, which is more generic than V1.Ming Lei (2):
block: fix race between adding/removing rq qos and normal IO
block: mark queue init done at the end ofblk_register_queueblock/blk-rq-qos.h | 24 ++++++++++++++++++++++++ block/blk-sysfs.c | 29 +++++++++++++++-------------- 2 files changed, 39 insertions(+), 14 deletions(-)Cc: Yi Zhang <[email protected]>
Cc: Bart Van Assche <[email protected]>
--
2.31.1
Signature · 2 lines
Best Regards, Yi Zhang
Quoted 32 lines · Ming Lei
Hello,
Yi reported several kernel panics on:
[16687.001777] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000008 ... [16687.163549] pc : __rq_qos_track+0x38/0x60or
[ 997.690455] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000020 ... [ 997.850347] pc : __rq_qos_done+0x2c/0x50Turns out it is caused by race between adding wbt and normal IO.
Fix the issue by freezing request queue when adding/deleting rq qos.
V3:
- use ->queue_lock for protecting concurrent adding/deleting rqos on same queueV2:
- switch to the approach of freezing queue, which is more generic than V1.Ming Lei (2):
block: fix race between adding/removing rq qos and normal IO
block: mark queue init done at the end ofblk_register_queue
Quoted 6 lines
block/blk-rq-qos.h | 24 ++++++++++++++++++++++++ block/blk-sysfs.c | 29 +++++++++++++++-------------- 2 files changed, 39 insertions(+), 14 deletions(-)Cc Yi Zhang <[email protected]>Cc Bart Van Assche <[email protected]>
Hello Jens,
Any chance to merge the fixes if you are fine?
Thanks,
Ming
Quoted 37 lines · Ming Lei
Quoted 36 lines · Ming Lei
Hello,
Yi reported several kernel panics on:
[16687.001777] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000008 ... [16687.163549] pc : __rq_qos_track+0x38/0x60or
[ 997.690455] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000020 ... [ 997.850347] pc : __rq_qos_done+0x2c/0x50Turns out it is caused by race between adding wbt and normal IO.
Fix the issue by freezing request queue when adding/deleting rq qos.
V3:
- use ->queue_lock for protecting concurrent adding/deleting rqos on same queueV2:
- switch to the approach of freezing queue, which is more generic than V1.Ming Lei (2):
block: fix race between adding/removing rq qos and normal IO
block: mark queue init done at the end ofblk_register_queueblock/blk-rq-qos.h | 24 ++++++++++++++++++++++++ block/blk-sysfs.c | 29 +++++++++++++++-------------- 2 files changed, 39 insertions(+), 14 deletions(-)
Quoted 6 lines
Quoted 2 lines
Cc Yi Zhang <[email protected]>Cc Bart Van Assche <[email protected]>Hello Jens,
Any chance to merge the fixes if you are fine?
Hi Jens,
Yi finds that the issue can be triggered in his more tests, and has escalate it in RH BZ, can we fix the issue?
Thanks,
Ming
Quoted 10 lines · Ming Lei
Hello,
Yi reported several kernel panics on:
[16687.001777] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000008 ... [16687.163549] pc : __rq_qos_track+0x38/0x60or
Quoted 7 lines
[ 997.690455] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000020 ... [ 997.850347] pc : __rq_qos_done+0x2c/0x50Turns out it is caused by race between adding wbt and normal IO.
Fix the issue by freezing request queue when adding/deleting rq qos.
Applied, thanks.
Signature · 1 line
Jens Axboe