block: pop cached rq before potentially blocking rq_qos_throttle()
Jens Axboe
maintainerPATCH 1/1 block: pop cached rq before potentially blocking rq_qos_throttle()
If rq_qos_throttle() ends up blocking, then we will have invalidated and flushed our current plug. Since blk_mq_get_cached_request() hasn't popped the cached request off the plug list just yet, we end holding a pointer to a request that is no longer valid. This insta-crashes with rq->mq_hctx being NULL in the validity checks just after.
Pop the request off the cached list before doing rq_qos_throttle() to avoid using a potentially stale request.
---
block/blk-mq.c +8 -3
@@ -2765,15 +2765,20 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q,return NULL ; } -rq_qos_throttle (q, *bio); -if (blk_mq_get_hctx_type ((*bio)->bi_opf) != rq->mq_hctx->type)return NULL ;if (op_is_flush (rq->cmd_flags) !=op_is_flush ((*bio)->bi_opf))return NULL ; - rq->cmd_flags = (*bio)->bi_opf; +/* +* If any qos ->throttle() end up blocking, we will have flushed the +* plug and hence killed the cached_rq list as well. Pop this entry +* before we throttle. +*/ plug->cached_rq =rq_list_next (rq); +rq_qos_throttle (q, *bio); + + rq->cmd_flags = (*bio)->bi_opf;INIT_LIST_HEAD (&rq->queuelist);return rq; }
Signature · 1 line
Jens Axboe
Quoted 43 lines · Jens Axboe
If
rq_qos_throttle()ends up blocking, then we will have invalidated and
flushed our current plug. Sinceblk_mq_get_cached_request()hasn't popped the cached request off the plug list just yet, we end holding a
pointer to a request that is no longer valid. This insta-crashes withrq->mq_hctxbeing NULL in the validity checks just after.Pop the request off the cached list before doing
rq_qos_throttle()to avoid using a potentially stale request.Fixes 0a5aa8d161d1 ("block: fix blk_mq_attempt_bio_merge and
rq_qos_throttleprotection")Reported-by Dylan Yudaken <[email protected]>Signed-off-by Jens Axboe <[email protected]>---
block/blk-mq.c +0
-0@@ -2765,15 +2765,20 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, return NULL; }
- rq_qos_throttle(q, *bio);
-
if (blk_mq_get_hctx_type((*bio)->bi_opf) != rq->mq_hctx-Quoted 1 line
type)
return NULL;
if (op_is_flush(rq->cmd_flags)!= op_is_flush((*bio)-Quoted 1 line
bi_opf))return NULL;
- rq->cmd_flags = (*bio)->bi_opf;
+ /*
+ * If any qos ->throttle() end up blocking, we will have flushed the
+ * plug and hence killed thecached_rqlist as well. Pop this entry
+ * before we throttle.
Quoted 8 lines
+ */
plug->cached_rq =rq_list_next(rq);
+ rq_qos_throttle(q, *bio);
+
+ rq->cmd_flags = (*bio)->bi_opf;
INIT_LIST_HEAD(&rq->queuelist);
return rq;
}
This fixes the problems I was seeing
Quoted 6 lines · Jens Axboe
If
rq_qos_throttle()ends up blocking, then we will have invalidated and flushed our current plug. Sinceblk_mq_get_cached_request()hasn't popped the cached request off the plug list just yet, we end holding a pointer to a request that is no longer valid. This insta-crashes withrq->mq_hctxbeing NULL in the validity checks just after.
Quoted 6 lines
Pop the request off the cached list before doing
rq_qos_throttle()to avoid using a potentially stale request.Fixes 0a5aa8d161d1 ("block: fix blk_mq_attempt_bio_merge and rq_qos_throttle protection")Reported-by Dylan Yudaken <[email protected]>Signed-off-by Jens Axboe <[email protected]>
Thank you for the fix and sorry for the trouble. The patch passed my test set:
BTW, may I know the workload or script which triggers the failure? I'm interested in if we can add a blktests test case to exercises qos throttle and prevent similar bugs in the future.
Signature · 1 line
Shin'ichiro Kawasaki
Jens Axboe
maintainerQuoted 14 lines · Shinichiro Kawasaki
Quoted 6 lines · Jens Axboe
If
rq_qos_throttle()ends up blocking, then we will have invalidated and flushed our current plug. Sinceblk_mq_get_cached_request()hasn't popped the cached request off the plug list just yet, we end holding a pointer to a request that is no longer valid. This insta-crashes withrq->mq_hctxbeing NULL in the validity checks just after.Quoted 6 lines
Pop the request off the cached list before doing
rq_qos_throttle()to avoid using a potentially stale request.Fixes 0a5aa8d161d1 ("block: fix blk_mq_attempt_bio_merge and rq_qos_throttle protection")Reported-by Dylan Yudaken <[email protected]>Signed-off-by Jens Axboe <[email protected]>
Quoted 8 lines
Thank you for the fix and sorry for the trouble. The patch passed my test set:
Tested-by Shin'ichiro Kawasaki <[email protected]>BTW, may I know the workload or script which triggers the failure? I'm interested in if we can add a blktests test case to exercises qos throttle and prevent similar bugs in the future.
Not sure if there are others, but specifically blk-iocost will do an explicit schedule() for some conditions. See the bottom of ioc_rqos_throttle().
Signature · 1 line
Jens Axboe
Quoted 20 lines · Jens Axboe
Quoted 19 lines · Shinichiro Kawasaki
Quoted 6 lines · Jens Axboe
If
rq_qos_throttle()ends up blocking, then we will have invalidated and flushed our current plug. Sinceblk_mq_get_cached_request()hasn't popped the cached request off the plug list just yet, we end holding a pointer to a request that is no longer valid. This insta-crashes withrq->mq_hctxbeing NULL in the validity checks just after.Quoted 6 lines
Pop the request off the cached list before doing
rq_qos_throttle()to avoid using a potentially stale request.Fixes 0a5aa8d161d1 ("block: fix blk_mq_attempt_bio_merge and rq_qos_throttle protection")Reported-by Dylan Yudaken <[email protected]>Signed-off-by Jens Axboe <[email protected]>Thank you for the fix and sorry for the trouble. The patch passed my test set:
Tested-by Shin'ichiro Kawasaki <[email protected]>
Quoted 7 lines
Quoted 3 lines
BTW, may I know the workload or script which triggers the failure? I'm interested in if we can add a blktests test case to exercises qos throttle and prevent similar bugs in the future.
Not sure if there are others, but specifically blk-iocost will do an explicit
schedule()for some conditions. See the bottom ofioc_rqos_throttle().
Thanks. Will try to create a script which triggers the schedule() in ioc_rqos_throttle().
Signature · 1 line
Shin'ichiro Kawasaki