keep passthrough request out of the I/O schedulers
Hi Jens,
this is my respin of Ming's "blk-mq: handle passthrough request as really passthrough" series. The first patch is a slightly tweaked version of Ming's first patch, while the 2 others are new based on the discussion.
This isn't meant to shut down the discussion on wether to use scheduler tags for passthrough or not, but I'd like to see the bug fixed (and a series I have that needs it unblocked).
PATCH 1/3 blk-mq: don't queue plugged passthrough requests into scheduler
From: Ming Lei <[email protected]>
Passthrough) request should never be queued to the I/O scheduler, as scheduling these opaque requests doens't make sense, and I/O schedulers might required req->bio to be always valid.
We never let passthrough request cross scheduler before commit 1c2d2fff6dc0 ("block: wire-up support for passthrough plugging"), restored this behavior even for passthrough requests issued under a plug.
[hch: use blk_mq_insert_requests for passthrough requests,
fix up the commit message and comments]
--- block/blk-mq.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
block/blk-mq.c +6 -2
@@ -2711,6 +2711,7 @@ static void blk_mq_dispatch_plug_list(struct blk_plug *plug, bool from_sched)struct request *requeue_list =NULL ;struct request **requeue_lastp = &requeue_list;unsigned int depth =0 ; +bool is_passthrough = false;LIST_HEAD (list); do {
@@ -2719,7 +2720,9 @@ static void blk_mq_dispatch_plug_list(struct blk_plug *plug, bool from_sched)if (!this_hctx) { this_hctx = rq->mq_hctx; this_ctx = rq->mq_ctx; - }else if (this_hctx != rq->mq_hctx || this_ctx != rq->mq_ctx) { + is_passthrough =blk_rq_is_passthrough (rq); + }else if (this_hctx != rq->mq_hctx || this_ctx != rq->mq_ctx || + is_passthrough !=blk_rq_is_passthrough (rq)) {rq_list_add_tail (&requeue_lastp, rq);continue ; }
@@ -2731,7 +2734,8 @@ static void blk_mq_dispatch_plug_list(struct blk_plug *plug, bool from_sched)trace_block_unplug (this_hctx->queue, depth, !from_sched);percpu_ref_get (&this_hctx->queue->q_usage_counter); -if (this_hctx->queue->elevator) { +/* passthrough requests should never be issued to the I/O scheduler */ +if (this_hctx->queue->elevator && !is_passthrough) { this_hctx->queue->elevator->type->ops.insert_requests (this_hctx, &list,0 );blk_mq_run_hw_queue (this_hctx, from_sched);
Signature · 1 line
2.39.2
Re: [PATCH 1/3] blk-mq: don't queue plugged passthrough requests into scheduler
Quoted 1 line · Christoph Hellwig
Passthrough) request should never be queued to the I/O scheduler,
Passthrough) request -> Passthrough requests
Quoted 1 line
as scheduling these opaque requests doens't make sense, and I/O
doens't -> doesn't
Quoted 1 line
schedulers might required
req->bioto be always valid.
required -> require
Quoted 1 line
We never let passthrough request cross scheduler before commit
request -> requests
Quoted 3 lines
1c2d2fff6dc0("block: wire-up support for passthrough plugging"), restored this behavior even for passthrough requests issued under a plug.
The above sentence needs to be edited for clarity.
Otherwise this patch looks good to me.
Thanks,
Bart.
Re: [PATCH 1/3] blk-mq: don't queue plugged passthrough requests into scheduler
In reply to Christoph HellwigQuoted 10 lines · Christoph Hellwig
Passthrough) request should never be queued to the I/O scheduler, as scheduling these opaque requests doens't make sense, and I/O schedulers might required
req->bioto be always valid.We never let passthrough request cross scheduler before commit
1c2d2fff6dc0("block: wire-up support for passthrough plugging"), restored this behavior even for passthrough requests issued under a plug.[...]
Applied, thanks!
[1/3] blk-mq: don't queue plugged passthrough requests into scheduler
commit: d97217e7f024bbe9aa62aea070771234c2879358
[2/3] blk-mq: remove RQF_ELVPRIV
commit: fdcab6cddef24a26b86d798814b3c25057e53c21
[3/3] blk-mq: make sure elevator callbacks aren't called for passthrough request
commit: 59f86a9c69ad379650839b41bb01be213bfac9e3Best regards,
Signature · 1 line
Jens Axboe
PATCH 2/3 blk-mq: remove RQF_ELVPRIV
In reply to Christoph HellwigRQF_ELVPRIV is set for all non-flush requests that have RQF_ELV set. Expand this condition in the two users of the flag and remove it.
--- block/blk-mq-debugfs.c | 1 - block/blk-mq-sched.h | 4 ++-- block/blk-mq.c | 6 ++---- include/linux/blk-mq.h | 2 -- 4 files changed, 4 insertions(+), 9 deletions(-)
block/blk-mq-debugfs.c +0 -1
@@ -248,7 +248,6 @@ static const char *const rqf_name[] = {RQF_NAME (DONTPREP ),RQF_NAME (FAILED ),RQF_NAME (QUIET ), -RQF_NAME (ELVPRIV ),RQF_NAME (IO_STAT ),RQF_NAME (PM ), RQF_NAME(HASHED),
block/blk-mq-sched.h +2 -2
@@ -58,11 +58,11 @@ static inline void blk_mq_sched_completed_request(struct request *rq, u64 now)static inline void blk_mq_sched_requeue_request (struct request *rq) { -if (rq->rq_flags &RQF_ELV ) { +if ((rq->rq_flags &RQF_ELV ) && !op_is_flush (rq->cmd_flags)) {struct request_queue *q = rq->q;struct elevator_queue *e = q->elevator; -if ((rq->rq_flags &RQF_ELVPRIV ) && e->type->ops.requeue_request) +if (e->type->ops.requeue_request) e->type->ops.requeue_request (rq); } }
block/blk-mq.c +2 -4
@@ -393,10 +393,8 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data,RB_CLEAR_NODE (&rq->rb_node);if (!op_is_flush (data->cmd_flags) && - e->type->ops.prepare_request) { + e->type->ops.prepare_request) e->type->ops.prepare_request (rq); - rq->rq_flags |=RQF_ELVPRIV ; - } }return rq;
@@ -696,7 +694,7 @@ void blk_mq_free_request(struct request *rq)struct request_queue *q = rq->q;struct blk_mq_hw_ctx *hctx = rq->mq_hctx; -if ((rq->rq_flags &RQF_ELVPRIV ) && +if ((rq->rq_flags &RQF_ELV ) && !op_is_flush (rq->cmd_flags) && q->elevator->type->ops.finish_request) q->elevator->type->ops.finish_request (rq);
include/linux/blk-mq.h +0 -2
@@ -42,8 +42,6 @@ typedef __u32 __bitwise req_flags_t;#define RQF_FAILED ((__force req_flags_t)(1 << 10))/* don't warn about errors */ #define RQF_QUIET ((__force req_flags_t)(1 << 11)) -/* elevator private data attached */ -#define RQF_ELVPRIV ((__force req_flags_t)(1 << 12))/* account into disk and partition IO statistics */ #define RQF_IO_STAT ((__force req_flags_t)(1 << 13))/* runtime pm request */
Signature · 1 line
2.39.2
Re: [PATCH 2/3] blk-mq: remove RQF_ELVPRIV
Quoted 4 lines · Christoph Hellwig
RQF_ELVPRIVis set for all non-flush requests that haveRQF_ELVset. Expand this condition in the two users of the flag and remove it.Signed-off-by Christoph Hellwig <[email protected]>
Thanks,
Ming
Re: [PATCH 2/3] blk-mq: remove RQF_ELVPRIV
In reply to Christoph HellwigQuoted 29 lines · Christoph Hellwig
RQF_ELVPRIVis set for all non-flush requests that haveRQF_ELVset. Expand this condition in the two users of the flag and remove it.Signed-off-by Christoph Hellwig <[email protected]>--- block/blk-mq-debugfs.c | 1 - block/blk-mq-sched.h | 4 ++-- block/blk-mq.c | 6 ++---- include/linux/blk-mq.h | 2 -- 4 files changed, 4 insertions(+), 9 deletions(-)block/blk-mq-debugfs.c +0
-1@@ -248,7 +248,6 @@ static const char *const rqf_name[] = {RQF_NAME (DONTPREP ),RQF_NAME (FAILED ),RQF_NAME (QUIET ), -RQF_NAME (ELVPRIV ),RQF_NAME (IO_STAT ),RQF_NAME (PM ), RQF_NAME(HASHED),block/blk-mq-sched.h +0
-0@@ -58,11 +58,11 @@ static inline void blk_mq_sched_completed_request(struct request *rq, u64 now)
block/blk-mq-sched.h static inline void blk_mq_sched_completed_request(struct request *rq, u64 now) · line 66
Quoted 10 lines
block/blk-mq-sched.h +2
-2static inline void blk_mq_sched_requeue_request (struct request *rq) { -if (rq->rq_flags &RQF_ELV ) { +if ((rq->rq_flags &RQF_ELV ) && !op_is_flush (rq->cmd_flags)) {struct request_queue *q = rq->q;struct elevator_queue *e = q->elevator; -if ((rq->rq_flags &RQF_ELVPRIV ) && e->type->ops.requeue_request) +if (e->type->ops.requeue_request) e->type->ops.requeue_request (rq);
The above actually changes current behavior since RQF_ELVPRIV is only set iff the following condition is true:
(rq->rq_flags & RQF_ELV) && !op_is_flush(rq->cmd_flags) && e->type->ops.prepare_request.
Quoted 19 lines
block/blk-mq-sched.h +0
-0} }block/blk-mq.c +1
-3@@ -393,10 +393,8 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data,RB_CLEAR_NODE (&rq->rb_node);if (!op_is_flush (data->cmd_flags) && - e->type->ops.prepare_request) { + e->type->ops.prepare_request) e->type->ops.prepare_request (rq); - rq->rq_flags |=RQF_ELVPRIV ; - } }return rq;
block/blk-mq.c void blk_mq_free_request(struct request *rq) · line 699
Quoted 7 lines
block/blk-mq.c +1
-1@@ -696,7 +694,7 @@ void blk_mq_free_request(struct request *rq)struct request_queue *q = rq->q;struct blk_mq_hw_ctx *hctx = rq->mq_hctx; -if ((rq->rq_flags &RQF_ELVPRIV ) && +if ((rq->rq_flags &RQF_ELV ) && !op_is_flush (rq->cmd_flags) && q->elevator->type->ops.finish_request) q->elevator->type->ops.finish_request (rq);
Same with above.
Thanks,
Ming
Re: [PATCH 2/3] blk-mq: remove RQF_ELVPRIV
Quoted 9 lines · Ming Lei
block/blk-mq-sched.h static inline void blk_mq_sched_completed_request(struct request *rq, u64 now) · line 66
Quoted 3 lines
block/blk-mq-sched.h +1
-1-if ((rq->rq_flags &RQF_ELVPRIV ) && e->type->ops.requeue_request) +if (e->type->ops.requeue_request) e->type->ops.requeue_request (rq);The above actually changes current behavior since
RQF_ELVPRIVis only set iff the following condition is true:(rq->rq_flags & RQF_ELV) && !op_is_flush(rq->cmd_flags) && e->type->ops.prepare_request.
It would require an I/O scheduler that implements .requeue_request but not .prepare_request, which doesn't exist and also is rather pointless as this .requeue_request method would never get called in the current code.
So no, no behavior change in practice.
Re: [PATCH 2/3] blk-mq: remove RQF_ELVPRIV
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Ming Lei
block/blk-mq-sched.h static inline void blk_mq_sched_completed_request(struct request *rq, u64 now) · line 66
Quoted 3 lines
block/blk-mq-sched.h +1
-1-if ((rq->rq_flags &RQF_ELVPRIV ) && e->type->ops.requeue_request) +if (e->type->ops.requeue_request) e->type->ops.requeue_request (rq);The above actually changes current behavior since
RQF_ELVPRIVis only set iff the following condition is true:
Quoted 8 lines
Quoted 2 lines
(rq->rq_flags & RQF_ELV) && !op_is_flush(rq->cmd_flags) && e->type->ops.prepare_request.It would require an I/O scheduler that implements .requeue_request but not .prepare_request, which doesn't exist and also is rather pointless as this .requeue_request method would never get called in the current code.
So no, no behavior change in practice.
Fair enough, just found that all three schedulers have implemented e->type->ops.prepare_request.
Thanks,
Ming
Jens Axboe
maintainerRe: [PATCH 2/3] blk-mq: remove RQF_ELVPRIV
Quoted 12 lines · Ming Lei
Quoted 11 lines · Christoph Hellwig
Quoted 9 lines · Ming Lei
block/blk-mq-sched.h static inline void blk_mq_sched_completed_request(struct request *rq, u64 now) · line 66
Quoted 3 lines
block/blk-mq-sched.h +1
-1-if ((rq->rq_flags &RQF_ELVPRIV ) && e->type->ops.requeue_request) +if (e->type->ops.requeue_request) e->type->ops.requeue_request (rq);The above actually changes current behavior since
RQF_ELVPRIVis only set iff the following condition is true:(rq->rq_flags & RQF_ELV) && !op_is_flush(rq->cmd_flags) && e->type->ops.prepare_request.
Quoted 8 lines
Quoted 5 lines
It would require an I/O scheduler that implements .requeue_request but not .prepare_request, which doesn't exist and also is rather pointless as this .requeue_request method would never get called in the current code.
So no, no behavior change in practice.
Fair enough, just found that all three schedulers have implemented
e->type->ops.prepare_request.
We should probably make this requirement explicit though, seems very fragile to depend on it just because it's the status quo.
Signature · 1 line
Jens Axboe
Re: [PATCH 2/3] blk-mq: remove RQF_ELVPRIV
In reply to Christoph HellwigQuoted 2 lines · Christoph Hellwig
RQF_ELVPRIVis set for all non-flush requests that haveRQF_ELVset. Expand this condition in the two users of the flag and remove it.
PATCH 3/3 blk-mq: make sure elevator callbacks aren't called for passthrough request
In reply to Christoph HellwigIn case of q->elevator, passthrought request can still be marked as RQF_ELV, so some elevator callbacks will be called for them.
Fix this by splitting RQF_SCHED_TAGS, which is set for all requests that are issued on a queue that uses an I/O scheduler, and RQF_USE_SCHED for non-flush, non-passthrough requests on such a queue.
Roughly based on two different patches from
Ming Lei <[email protected]>.
--- block/blk-mq-debugfs.c | 3 ++- block/blk-mq-sched.h | 6 ++--- block/blk-mq.c | 53 +++++++++++++++++++++++------------------- block/blk-mq.h | 6 ++--- include/linux/blk-mq.h | 12 ++++++---- 5 files changed, 44 insertions(+), 36 deletions(-)
block/blk-mq-debugfs.c +2 -1
@@ -246,6 +246,8 @@ static const char *const rqf_name[] = {RQF_NAME (MIXED_MERGE ),RQF_NAME (MQ_INFLIGHT ),RQF_NAME (DONTPREP ), +RQF_NAME (SCHED_TAGS ), +RQF_NAME (USE_SCHED ),RQF_NAME (FAILED ),RQF_NAME (QUIET ), RQF_NAME(IO_STAT),
@@ -255,7 +257,6 @@ static const char *const rqf_name[] = {RQF_NAME (SPECIAL_PAYLOAD ),RQF_NAME (ZONE_WRITE_LOCKED ),RQF_NAME (TIMED_OUT ), -RQF_NAME (ELV ), RQF_NAME(RESV), };#undef RQF_NAME
block/blk-mq-sched.h +3 -3
@@ -37,7 +37,7 @@ static inline bool blk_mq_sched_allow_merge(struct request_queue *q, struct request *rq, struct bio *bio) { - if (rq->rq_flags & RQF_ELV) { + if (rq->rq_flags & RQF_USE_SCHED) { struct elevator_queue *e = q->elevator;if (e->type->ops.allow_merge)
@@ -48,7 +48,7 @@ blk_mq_sched_allow_merge(struct request_queue *q, struct request *rq,static inline void blk_mq_sched_completed_request (struct request *rq,u64 now) { -if (rq->rq_flags &RQF_ELV ) { +if (rq->rq_flags &RQF_USE_SCHED ) {struct elevator_queue *e = rq->q->elevator;if (e->type->ops.completed_request)
@@ -58,7 +58,7 @@ static inline void blk_mq_sched_completed_request(struct request *rq, u64 now)static inline void blk_mq_sched_requeue_request (struct request *rq) { -if ((rq->rq_flags &RQF_ELV ) && !op_is_flush (rq->cmd_flags)) { +if (rq->rq_flags &RQF_USE_SCHED ) {struct request_queue *q = rq->q;struct elevator_queue *e = q->elevator;
block/blk-mq.c +29 -24
119 lines not on this page · raw message
block/blk-mq.h +3 -3
@@ -226,9 +226,9 @@ static inline bool blk_mq_is_shared_tags(unsigned int flags)static inline struct blk_mq_tags *blk_mq_tags_from_data (struct blk_mq_alloc_data *data) { -if (!(data->rq_flags &RQF_ELV )) -return data->hctx->tags; -return data->hctx->sched_tags; +if (data->rq_flags &RQF_SCHED_TAGS ) +return data->hctx->sched_tags; +return data->hctx->tags; }static inline bool blk_mq_hctx_stopped (struct blk_mq_hw_ctx *hctx)
include/linux/blk-mq.h +7 -5
@@ -38,6 +38,10 @@ typedef __u32 __bitwise req_flags_t;#define RQF_MQ_INFLIGHT ((__force req_flags_t)(1 << 6))/* don't call prep for this one */ #define RQF_DONTPREP ((__force req_flags_t)(1 << 7)) +/* use hctx->sched_tags */ +#define RQF_SCHED_TAGS ((__force req_flags_t)(1 << 8)) +/* use and I/O scheduler for this request */ +#define RQF_USE_SCHED ((__force req_flags_t)(1 << 9))/* vaguely specified driver internal error. Ignored by the block layer */ #define RQF_FAILED ((__force req_flags_t)(1 << 10))/* don't warn about errors */
@@ -57,9 +61,7 @@ typedef __u32 __bitwise req_flags_t;#define RQF_ZONE_WRITE_LOCKED ((__force req_flags_t)(1 << 19))/* ->timeout has been called, don't expire again */ #define RQF_TIMED_OUT ((__force req_flags_t)(1 << 21)) -/* queue has elevator attached */ -#define RQF_ELV ((__force req_flags_t)(1 << 22)) -#define RQF_RESV ((__force req_flags_t)(1 << 23)) +#define RQF_RESV ((__force req_flags_t)(1 << 23))/* flags that prevent us from merging requests: */ #define RQF_NOMERGE_FLAGS \
@@ -842,7 +844,7 @@ void blk_mq_end_request_batch(struct io_comp_batch *ib);*/ static inline bool blk_mq_need_time_stamp (struct request *rq) { -return (rq->rq_flags & (RQF_IO_STAT |RQF_STATS |RQF_ELV )); +return (rq->rq_flags & (RQF_IO_STAT |RQF_STATS |RQF_USE_SCHED )); }static inline bool blk_mq_is_reserved_rq (struct request *rq)
@@ -858,7 +860,7 @@ static inline bool blk_mq_add_to_batch(struct request *req, struct io_comp_batch *iob, int ioerror, void (*complete)(struct io_comp_batch *)) { - if (!iob || (req->rq_flags & RQF_ELV) || ioerror || + if (!iob || (req->rq_flags & RQF_USE_SCHED) || ioerror || (req->end_io && !blk_rq_is_passthrough(req))) return false;
Signature · 1 line
2.39.2
Re: [PATCH 3/3] blk-mq: make sure elevator callbacks aren't called for passthrough request
Quoted 7 lines · Christoph Hellwig
In case of
q->elevator, passthrought request can still be marked asRQF_ELV, so some elevator callbacks will be called for them.Fix this by splitting
RQF_SCHED_TAGS, which is set for all requests that are issued on a queue that uses an I/O scheduler, andRQF_USE_SCHEDfor non-flush, non-passthrough requests on such a queue.
Quoted 5 lines
Roughly based on two different patches from
Ming Lei <[email protected]>.Signed-off-by Christoph Hellwig <[email protected]>---
Thanks,
Ming
Re: [PATCH 3/3] blk-mq: make sure elevator callbacks aren't called for passthrough request
In reply to Christoph HellwigQuoted 1 line · Christoph Hellwig
In case of
q->elevator, passthrought request can still be marked as
passthrought -> passthrough
include/linux/blk-mq.h typedef __u32 __bitwise req_flags_t; · line 44
Quoted 4 lines
include/linux/blk-mq.h +4
-0+/* use hctx->sched_tags */ +#define RQF_SCHED_TAGS ((__force req_flags_t)(1 << 8)) +/* use and I/O scheduler for this request */ +#define RQF_USE_SCHED ((__force req_flags_t)(1 << 9))
and -> an
Otherwise this patch looks good to me.
Thanks,
Bart.
Jens Axboe
maintainerQuoted 9 lines · Christoph Hellwig
Hi Jens,
this is my respin of Ming's "blk-mq: handle passthrough request as really passthrough" series. The first patch is a slightly tweaked version of Ming's first patch, while the 2 others are new based on the discussion.
This isn't meant to shut down the discussion on wether to use scheduler tags for passthrough or not, but I'd like to see the bug fixed (and a series I have that needs it unblocked).
I think the series stands fine on its own.
Signature · 1 line
Jens Axboe