blk-mq: fix two misuses on RQF_USE_SCHED
PATCH 1/1 blk-mq: fix two misuses on RQF_USE_SCHED
Request allocated from sched tags can't be issued via ->queue_rqs() directly, since driver tag isn't allocated yet. This is the 1st misuse of RQF_USE_SCHED for figuring out plug->has_elevator.
Request allocated from sched tags can't be ended by blk_mq_end_request_batch() too, fix the 2nd RQF_USE_SCHED misuse in blk_mq_add_to_batch().
Without this patch, NVMe uring cmd passthrough IO workload can run into hang easily with real io scheduler.
--- block/blk-mq.c | 6 +++++- include/linux/blk-mq.h | 6 +++++- 2 files changed, 10 insertions(+), 2 deletions(-)
block/blk-mq.c +5 -1
@@ -1302,7 +1302,11 @@ static void blk_add_rq_to_plug(struct blk_plug *plug, struct request *rq)if (!plug->multiple_queues && last && last->q != rq->q) plug->multiple_queues = true; -if (!plug->has_elevator && (rq->rq_flags &RQF_USE_SCHED )) +/* +* Any request allocated from sched tags can't be issued to +* ->queue_rqs() directly +*/ +if (!plug->has_elevator && (rq->rq_flags &RQF_SCHED_TAGS )) plug->has_elevator = true; rq->rq_next =NULL ;rq_list_add (&plug->mq_list, rq);
include/linux/blk-mq.h +5 -1
@@ -852,7 +852,11 @@ static inline bool blk_mq_add_to_batch(struct request *req, struct io_comp_batch *iob, int ioerror, void (*complete)(struct io_comp_batch *)) { - if (!iob || (req->rq_flags & RQF_USE_SCHED) || ioerror || + /* + * blk_mq_end_request_batch() can't end request allocated from + * sched tags + */ + if (!iob || (req->rq_flags & RQF_SCHED_TAGS) || ioerror || (req->end_io && !blk_rq_is_passthrough(req))) return false;
Signature · 1 line
2.40.1
Hi, Ming
Could not hit the error after apply your patch. good job!
Quoted 57 lines · Ming Lei <[email protected]> 于2023年6月24日周六 21:01写道:
Request allocated from sched tags can't be issued via ->queue_rqs() directly, since driver tag isn't allocated yet. This is the 1st misuse of
RQF_USE_SCHEDfor figuring outplug->has_elevator.Request allocated from sched tags can't be ended by
blk_mq_end_request_batch()too, fix the 2ndRQF_USE_SCHEDmisuse inblk_mq_add_to_batch().Without this patch, NVMe uring cmd passthrough IO workload can run into hang easily with real io scheduler.
Fixes dd6216bb16e8 ("blk-mq: make sure elevator callbacks aren't called for passthrough request")Reported-by Guangwu Zhang <[email protected]>Closes https://lore.kernel.org/linux-block/CAGS2=YrBjpLPOKa-gzcKuuOG60AGth5794PNCDwatdnnscB9ug@mail.gmail.com/ open hereCc Christoph Hellwig <[email protected]>Signed-off-by Ming Lei <[email protected]>--- block/blk-mq.c | 6 +++++- include/linux/blk-mq.h | 6 +++++- 2 files changed, 10 insertions(+), 2 deletions(-)block/blk-mq.c +5
-1@@ -1302,7 +1302,11 @@ static void blk_add_rq_to_plug(struct blk_plug *plug, struct request *rq)if (!plug->multiple_queues && last && last->q != rq->q) plug->multiple_queues = true; -if (!plug->has_elevator && (rq->rq_flags &RQF_USE_SCHED )) +/* +* Any request allocated from sched tags can't be issued to +* ->queue_rqs() directly +*/ +if (!plug->has_elevator && (rq->rq_flags &RQF_SCHED_TAGS )) plug->has_elevator = true; rq->rq_next =NULL ;rq_list_add (&plug->mq_list, rq);include/linux/blk-mq.h +5
-2@@ -852,7 +852,11 @@ static inline bool blk_mq_add_to_batch(struct request *req, struct io_comp_batch *iob, int ioerror, void (*complete)(struct io_comp_batch *)) { - if (!iob || (req->rq_flags & RQF_USE_SCHED) || ioerror || + /* + * blk_mq_end_request_batch() can't end request allocated from + * sched tags + */ + if (!iob || (req->rq_flags & RQF_SCHED_TAGS) || ioerror || (req->end_io && !blk_rq_is_passthrough(req))) return false; --
Quoted 2 lines
2.40.1
--
Guangwu Zhang
Thanks
Quoted 9 lines · Ming Lei
Request allocated from sched tags can't be issued via ->queue_rqs() directly, since driver tag isn't allocated yet. This is the 1st misuse of
RQF_USE_SCHEDfor figuring outplug->has_elevator.Request allocated from sched tags can't be ended by
blk_mq_end_request_batch()too, fix the 2ndRQF_USE_SCHEDmisuse inblk_mq_add_to_batch().[...]
Applied, thanks!
[1/1] blk-mq: fix two misuses on RQF_USE_SCHED
commit: c6b7a3a26e809c9d2a51ae303764c1d2994f31cf
Best regards,
Signature · 1 line
Jens Axboe