blk-mq: Don't disable preemption around __blk_mq_run_hw_queue().
PATCH 1/1 blk-mq: Don't disable preemption around __blk_mq_run_hw_queue().
__blk_mq_delay_run_hw_queue() disables preemption to get a stable current CPU number and then invokes __blk_mq_run_hw_queue() if the CPU number is part the mask.
__blk_mq_run_hw_queue() acquires a spin_lock_t which is a sleeping lock on PREEMPT_RT and can't be acquired with disabled preemption.
If it is important that the current CPU matches the requested CPU mask and that the context does not migrate to another CPU while __blk_mq_run_hw_queue() is invoked then it possible to achieve this by disabling migration and keeping the context preemptible.
Disable only migration while testing the CPU mask and invoking __blk_mq_run_hw_queue().
--- block/blk-mq.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
block/blk-mq.c +4 -4
@@ -2083,14 +2083,14 @@ static void __blk_mq_delay_run_hw_queue(return ;if (!async && !(hctx->flags &BLK_MQ_F_BLOCKING )) { -int cpu =get_cpu (); -if (cpumask_test_cpu (cpu, hctx->cpumask)) { +migrate_disable (); +if (cpumask_test_cpu (raw_smp_processor_id (), hctx->cpumask)) {__blk_mq_run_hw_queue (hctx); -put_cpu (); +migrate_enable ();return ; } -put_cpu (); +migrate_enable (); } kblockd_mod_delayed_work_on(blk_mq_hctx_next_cpu(hctx), &hctx->run_work,
Quoted 25 lines · Sebastian Andrzej Siewior
__blk_mq_delay_run_hw_queue()disables preemption to get a stable current CPU number and then invokes__blk_mq_run_hw_queue()if the CPU number is part the mask.
__blk_mq_run_hw_queue()acquires aspin_lock_twhich is a sleeping lock onPREEMPT_RTand can't be acquired with disabled preemption.If it is important that the current CPU matches the requested CPU mask and that the context does not migrate to another CPU while
__blk_mq_run_hw_queue()is invoked then it possible to achieve this by disabling migration and keeping the context preemptible.Disable only migration while testing the CPU mask and invoking
__blk_mq_run_hw_queue().Signed-off-by Sebastian Andrzej Siewior <[email protected]>--- block/blk-mq.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
block/blk-mq.c static void __blk_mq_delay_run_hw_queue( · line 2089
Quoted 10 lines
block/blk-mq.c +3
-3@@ -2083,14 +2083,14 @@ static void __blk_mq_delay_run_hw_queue(return ;if (!async && !(hctx->flags &BLK_MQ_F_BLOCKING )) { -int cpu =get_cpu (); -if (cpumask_test_cpu (cpu, hctx->cpumask)) { +migrate_disable (); +if (cpumask_test_cpu (raw_smp_processor_id (), hctx->cpumask)) {__blk_mq_run_hw_queue (hctx); -put_cpu (); +migrate_enable ();
I think you can replace it with raw_smp_processor_id() directly without disabling migration.
Both async run queue and direct issue can run on cpu not on hctx->cpumask, so it is fine for sync run queue too.
Thanks,
Ming
PATCH v2 1/1 blk-mq: Don't disable preemption around __blk_mq_run_hw_queue().
__blk_mq_delay_run_hw_queue() disables preemption to get a stable current CPU number and then invokes __blk_mq_run_hw_queue() if the CPU number is part the mask.
__blk_mq_run_hw_queue() acquires a spin_lock_t which is a sleeping lock on PREEMPT_RT and can't be acquired with disabled preemption.
It is not required for correctness to invoke __blk_mq_run_hw_queue() on a CPU matching hctx->cpumask. Both (async and direct requests) can run on a CPU not matching hctx->cpumask.
The CPU mask without disabling preemption and invoking __blk_mq_run_hw_queue().
---
v1…v2:
- Drop migrate_disable() as per Ming Lei.
block/blk-mq.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
block/blk-mq.c +1 -5
@@ -2085,14 +2085,10 @@ static void __blk_mq_delay_run_hw_queue(return ;if (!async && !(hctx->flags &BLK_MQ_F_BLOCKING )) { -int cpu =get_cpu (); -if (cpumask_test_cpu (cpu, hctx->cpumask)) { +if (cpumask_test_cpu (raw_smp_processor_id (), hctx->cpumask)) {__blk_mq_run_hw_queue (hctx); -put_cpu ();return ; } - -put_cpu (); } kblockd_mod_delayed_work_on(blk_mq_hctx_next_cpu(hctx), &hctx->run_work,
Quoted 7 lines · Sebastian Andrzej Siewior
__blk_mq_delay_run_hw_queue()disables preemption to get a stable current CPU number and then invokes__blk_mq_run_hw_queue()if the CPU number is part the mask.
__blk_mq_run_hw_queue()acquires aspin_lock_twhich is a sleeping lock onPREEMPT_RTand can't be acquired with disabled preemption.
Quoted 8 lines
It is not required for correctness to invoke
__blk_mq_run_hw_queue()on a CPU matchinghctx->cpumask. Both (async and direct requests) can run on a CPU not matchinghctx->cpumask.The CPU mask without disabling preemption and invoking
__blk_mq_run_hw_queue().Signed-off-by Sebastian Andrzej Siewior <[email protected]>
Looks fine:
Thanks,
Ming
Quoted 8 lines · Sebastian Andrzej Siewior
__blk_mq_delay_run_hw_queue()disables preemption to get a stable current CPU number and then invokes__blk_mq_run_hw_queue()if the CPU number is part the mask.
__blk_mq_run_hw_queue()acquires aspin_lock_twhich is a sleeping lock onPREEMPT_RTand can't be acquired with disabled preemption.[...]
Applied, thanks!
[1/1] blk-mq: Don't disable preemption around __blk_mq_run_hw_queue().
commit: c9198d784fa93d447afe8e4627dfe205f0ce5ec8
Best regards,
Signature · 1 line
Jens Axboe