blk-flush: fix rq->flush.seq for post-flush requests
PATCH 1/2 blk-flush: fix rq->flush.seq for post-flush requests
From: Chengming Zhou <[email protected]>
If the policy == (REQ_FSEQ_DATA | REQ_FSEQ_POSTFLUSH), it means that the data sequence and post-flush sequence need to be done for this request.
The rq->flush.seq should record what sequences have been done (or don't need to be done). So in this case, pre-flush doesn't need to be done, we should init rq->flush.seq to REQ_FSEQ_PREFLUSH not REQ_FSEQ_POSTFLUSH.
Of course, this doesn't cause any problem in fact, since pre-flush and post-flush sequence do the same thing for now.
But we'd better fix this value, and the next patch will depend on this value to be correct.
--- block/blk-flush.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
block/blk-flush.c +1 -1
@@ -448,7 +448,7 @@ bool blk_insert_flush(struct request *rq)* the post flush, and then just pass the command on. */ blk_rq_init_flush (rq); - rq->flush.seq |=REQ_FSEQ_POSTFLUSH ; + rq->flush.seq |=REQ_FSEQ_PREFLUSH ;spin_lock_irq (&fq->mq_flush_lock); fq->flush_data_in_flight++;spin_unlock_irq (&fq->mq_flush_lock);
Signature · 1 line
2.41.0
PATCH 2/2 blk-flush: don't need to end rq twice for non post-flush requests
From: Chengming Zhou <[email protected]>
Now we unconditionally blk_rq_init_flush() to replace rq->end_io to make rq return twice back to the flush state machine for post-flush.
Obviously, non post-flush requests don't need it, they don't need to end request twice, so they don't need to replace rq->end_io callback. And the same for requests with the FUA bit on hardware with FUA support.
So we move blk_rq_init_flush() to REQ_FSEQ_DATA stage and only replace rq->end_io if it needs post-flush. Otherwise, it can end like normal request and doesn't need to return back to the flush state machine.
There are also some other good points:
1. all requests on hardware with FUA support won't have post-flush, so all of them don't need to end twice.
2. non post-flush requests won't have RQF_FLUSH_SEQ rq_flags set, so they can merge like normal requests.
3. we don't account non post-flush requests in flush_data_in_flight, since there is no point to defer pending flush for these requests.
--- block/blk-flush.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-)
block/blk-flush.c +18 -4
@@ -93,6 +93,7 @@ enum {static void blk_kick_flush (struct request_queue *q,struct blk_flush_queue *fq,blk_opf_t flags); +static void blk_rq_init_flush (struct request *rq);static inline struct blk_flush_queue *blk_get_flush_queue (struct request_queue *q,struct blk_mq_ctx *ctx)
@@ -187,7 +188,15 @@ static void blk_flush_complete_seq(struct request *rq,break ;case REQ_FSEQ_DATA : - fq->flush_data_in_flight++; +/* +* Only for requests that need post-flush, +* we need to do rq->end_io replacement trick +* to return back to the flush state machine. +*/ +if (!(rq->flush.seq &REQ_FSEQ_POSTFLUSH )) { +blk_rq_init_flush (rq); + fq->flush_data_in_flight++; + }spin_lock (&q->requeue_lock);list_move_tail (&rq->queuelist, &q->flush_list);spin_unlock (&q->requeue_lock);
@@ -202,7 +211,13 @@ static void blk_flush_complete_seq(struct request *rq,* normal completion and end it. */ list_del_init (&rq->queuelist); -blk_flush_restore_request (rq); +/* +* Only for requests that had rq->end_io replaced, +* we need to restore rq->end_io and make it a normal +* request before the second end. +*/ +if (rq->rq_flags &RQF_FLUSH_SEQ ) +blk_flush_restore_request (rq);blk_mq_end_request (rq, error);break ;
@@ -389,7 +404,6 @@ static enum rq_end_io_ret mq_flush_data_end_io(struct request *rq,static void blk_rq_init_flush (struct request *rq) { - rq->flush.seq =0 ; rq->rq_flags |=RQF_FLUSH_SEQ ; rq->flush.saved_end_io = rq->end_io;/* Usually NULL */ rq->end_io = mq_flush_data_end_io;
@@ -424,6 +438,7 @@ bool blk_insert_flush(struct request *rq)* the request accounting. */ rq->cmd_flags |=REQ_SYNC ; + rq->flush.seq =0 ;switch (policy) {case 0 :
@@ -458,7 +473,6 @@ bool blk_insert_flush(struct request *rq)* Mark the request as part of a flush sequence and submit it * for further processing to the flush state machine. */ -blk_rq_init_flush (rq);spin_lock_irq (&fq->mq_flush_lock);blk_flush_complete_seq (rq, fq,REQ_FSEQ_ACTIONS & ~policy,0 );spin_unlock_irq (&fq->mq_flush_lock);
Signature · 1 line
2.41.0
Quoted 11 lines · On Mon, Jul 10, 2023 at 02:47:04PM +0800, [email protected] wrote:
From: Chengming Zhou <[email protected]>
If the policy == (
REQ_FSEQ_DATA|REQ_FSEQ_POSTFLUSH), it means that the data sequence and post-flush sequence need to be done for this request.The
rq->flush.seqshould record what sequences have been done (or don't need to be done). So in this case, pre-flush doesn't need to be done, we should initrq->flush.seqtoREQ_FSEQ_PREFLUSHnotREQ_FSEQ_POSTFLUSH.Of course, this doesn't cause any problem in fact, since pre-flush and post-flush sequence do the same thing for now.
I wonder if it really doesn't cause any problems, but the change for sure looks good:
It should probably go before your other flush optimizations and maybe grow a fixes tag.
Re: [PATCH 2/2] blk-flush: don't need to end rq twice for non post-flush requests
In reply to [email protected]Quoted 5 lines · On Mon, Jul 10, 2023 at 02:47:05PM +0800, [email protected] wrote:
From: Chengming Zhou <[email protected]>
Now we unconditionally
blk_rq_init_flush()to replacerq->end_ioto make rq return twice back to the flush state machine for post-flush.
Quoted 7 lines
Obviously, non post-flush requests don't need it, they don't need to end request twice, so they don't need to replace
rq->end_iocallback. And the same for requests with the FUA bit on hardware with FUA support.So we move
blk_rq_init_flush()toREQ_FSEQ_DATAstage and only replacerq->end_ioif it needs post-flush. Otherwise, it can end like normal request and doesn't need to return back to the flush state machine.
I really like the idea behind this optimization, but I kinda hate adding more magic to the already way too magic flush sequence.
I wonder if a better idea would be to kill the flush sequence entirely, and just split the flush_queue into a preflush and a postflush queue. This would remove a field from struct request and lead to more readable code as well.
Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · On Mon, Jul 10, 2023 at 02:47:04PM +0800, [email protected] wrote:
From: Chengming Zhou <[email protected]>
If the policy == (
REQ_FSEQ_DATA|REQ_FSEQ_POSTFLUSH), it means that the data sequence and post-flush sequence need to be done for this request.The
rq->flush.seqshould record what sequences have been done (or don't need to be done). So in this case, pre-flush doesn't need to be done, we should initrq->flush.seqtoREQ_FSEQ_PREFLUSHnotREQ_FSEQ_POSTFLUSH.Of course, this doesn't cause any problem in fact, since pre-flush and post-flush sequence do the same thing for now.
Quoted 7 lines
I wonder if it really doesn't cause any problems, but the change for sure looks good:
Reviewed-by Christoph Hellwig <[email protected]>It should probably go before your other flush optimizations and maybe grow a fixes tag.
Ok, will add a Fixes tag and send it as a separate patch since it's a bug fix.
Thanks.
Quoted 17 lines · Chengming Zhou
Quoted 16 lines · Christoph Hellwig
Quoted 11 lines · On Mon, Jul 10, 2023 at 02:47:04PM +0800, [email protected] wrote:
From: Chengming Zhou <[email protected]>
If the policy == (
REQ_FSEQ_DATA|REQ_FSEQ_POSTFLUSH), it means that the data sequence and post-flush sequence need to be done for this request.The
rq->flush.seqshould record what sequences have been done (or don't need to be done). So in this case, pre-flush doesn't need to be done, we should initrq->flush.seqtoREQ_FSEQ_PREFLUSHnotREQ_FSEQ_POSTFLUSH.Of course, this doesn't cause any problem in fact, since pre-flush and post-flush sequence do the same thing for now.
I wonder if it really doesn't cause any problems, but the change for sure looks good:
Quoted 7 lines
Quoted 4 lines
Reviewed-by Christoph Hellwig <[email protected]>It should probably go before your other flush optimizations and maybe grow a fixes tag.
Ok, will add a Fixes tag and send it as a separate patch since it's a bug fix.
Well, I should put it in that series before other flush optimizations instead.
Quoted 1 line · Chengming Zhou
Ok, will add a Fixes tag and send it as a separate patch since it's a bug fix.
Btw, it's probably not worth resending patch 2 until we've figured out and dealt with the SATA flush regression that Chuck reported.
Quoted 5 lines · Christoph Hellwig
Quoted 1 line · Chengming Zhou
Ok, will add a Fixes tag and send it as a separate patch since it's a bug fix.
Btw, it's probably not worth resending patch 2 until we've figured out and dealt with the SATA flush regression that Chuck reported.
Ok, I will not resend patch 2. As for the patch 1, should I resend it as a separate patch or just put it in that series [1] before other flush optimizations ?
I search on the block mail list, find the issue [2] you mentioned, will look into it too.
[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/linux-block/[email protected]/
Quoted 9 lines · Chengming Zhou
Quoted 5 lines · Christoph Hellwig
Quoted 1 line · Chengming Zhou
Ok, will add a Fixes tag and send it as a separate patch since it's a bug fix.
Btw, it's probably not worth resending patch 2 until we've figured out and dealt with the SATA flush regression that Chuck reported.
Ok, I will not resend patch 2. As for the patch 1, should I resend it as a separate patch or just put it in that series [1] before other flush optimizations ?
I'd wait a bit for debugging the regression. For the worst case we'll have to revert the patch, which currently can be done cleanly, but can't be with that patch.
Re: [PATCH 2/2] blk-flush: don't need to end rq twice for non post-flush requests
In reply to Christoph HellwigQuoted 10 lines · Christoph Hellwig
Quoted 9 lines · On Mon, Jul 10, 2023 at 02:47:05PM +0800, [email protected] wrote:
From: Chengming Zhou <[email protected]>
Now we unconditionally
blk_rq_init_flush()to replacerq->end_ioto make rq return twice back to the flush state machine for post-flush.Obviously, non post-flush requests don't need it, they don't need to end request twice, so they don't need to replace
rq->end_iocallback. And the same for requests with the FUA bit on hardware with FUA support.
Quoted 6 lines
Quoted 3 lines
So we move
blk_rq_init_flush()toREQ_FSEQ_DATAstage and only replacerq->end_ioif it needs post-flush. Otherwise, it can end like normal request and doesn't need to return back to the flush state machine.I really like the idea behind this optimization, but I kinda hate adding more magic to the already way too magic flush sequence.
Yes, agree.
Quoted 5 lines
I wonder if a better idea would be to kill the flush sequence entirely, and just split the
flush_queueinto a preflush and a postflush queue. This would remove a field from struct request and lead to more readable code as well.
I have thought about this for some time, it seems feasible. So I implement it today and test it using blktests, it works.
I will send the patchset soon.
Thanks.