block: Fix use-after-free while iterating over requests
PATCH 1/1 block: Fix use-after-free while iterating over requests
Observes below crash while accessing (use-after-free) request queue member of struct request.
191.784789: <2> Unable to handle kernel paging request at virtual
address ffffff81429a4440
...
191.786174: <2> CPU: 3 PID: 213 Comm: kworker/3:1H Tainted: G S
O 5.4.61-qgki-debug-ge45de39 #1
...
191.786226: <2> Workqueue: kblockd blk_mq_timeout_work 191.786242: <2> pstate: 20c00005 (nzCv daif +PAN +UAO) 191.786261: <2> pc : bt_for_each+0x114/0x1a4 191.786274: <2> lr : bt_for_each+0xe0/0x1a4 ... 191.786494: <2> Call trace: 191.786507: <2> bt_for_each+0x114/0x1a4 191.786519: <2> blk_mq_queue_tag_busy_iter+0x60/0xd4 191.786532: <2> blk_mq_timeout_work+0x54/0xe8 191.786549: <2> process_one_work+0x2cc/0x568 191.786562: <2> worker_thread+0x28c/0x518 191.786577: <2> kthread+0x160/0x170 191.786594: <2> ret_from_fork+0x10/0x18 191.786615: <2> Code: 0b080148 f9404929 f8685921 b4fffe01 (f9400028) 191.786630: <2> ---[ end trace 0f1f51d79ab3f955 ]--- 191.786643: <2> Kernel panic - not syncing: Fatal exception
Fix this by updating the freed request with NULL. This could avoid accessing the already free request from other contexts while iterating over the requests.
--- block/blk-mq.c | 1 + block/blk-mq.h | 1 + 2 files changed, 2 insertions(+)
block/blk-mq.c +1 -0
@@ -492,6 +492,7 @@ static void __blk_mq_free_request(struct request *rq)blk_crypto_free_request (rq);blk_pm_mark_last_busy (rq); + hctx->tags->rqs[rq->tag] =NULL ; rq->mq_hctx =NULL ;if (rq->tag !=BLK_MQ_NO_TAG )blk_mq_put_tag (hctx->tags, ctx, rq->tag);
block/blk-mq.h +1 -0
@@ -224,6 +224,7 @@ static inline int __blk_mq_active_requests(struct blk_mq_hw_ctx *hctx)static inline void __blk_mq_put_driver_tag (struct blk_mq_hw_ctx *hctx,struct request *rq) { + hctx->tags->rqs[rq->tag] =NULL ;blk_mq_put_tag (hctx->tags, rq->mq_ctx, rq->tag); rq->tag =BLK_MQ_NO_TAG ;
Signature · 1 line
2.7.4
Quoted 53 lines · Pradeep P V K
Observes below crash while accessing (use-after-free) request queue member of struct request.
191.784789: <2> Unable to handle kernel paging request at virtual
addressffffff81429a4440
...
191.786174: <2> CPU: 3 PID: 213 Comm: kworker/3:1H Tainted: G S
O 5.4.61-qgki-debug-ge45de39 #1
...191.786226: <2> Workqueue: kblockd blk_mq_timeout_work 191.786242: <2> pstate: 20c00005 (nzCv daif +PAN +UAO) 191.786261: <2> pc : bt_for_each+0x114/0x1a4 191.786274: <2> lr : bt_for_each+0xe0/0x1a4 ... 191.786494: <2> Call trace: 191.786507: <2> bt_for_each+0x114/0x1a4 191.786519: <2> blk_mq_queue_tag_busy_iter+0x60/0xd4 191.786532: <2> blk_mq_timeout_work+0x54/0xe8 191.786549: <2> process_one_work+0x2cc/0x568 191.786562: <2> worker_thread+0x28c/0x518 191.786577: <2> kthread+0x160/0x170 191.786594: <2> ret_from_fork+0x10/0x18 191.786615: <2> Code: 0b080148 f9404929 f8685921 b4fffe01 (f9400028) 191.786630: <2> ---[ end trace 0f1f51d79ab3f955 ]--- 191.786643: <2> Kernel panic - not syncing: Fatal exceptionFix this by updating the freed request with NULL. This could avoid accessing the already free request from other contexts while iterating over the requests.
Signed-off-by Pradeep P V K <[email protected]>--- block/blk-mq.c | 1 + block/blk-mq.h | 1 + 2 files changed, 2 insertions(+)block/blk-mq.c +1
-0@@ -492,6 +492,7 @@ static void __blk_mq_free_request(struct request *rq)blk_crypto_free_request (rq);blk_pm_mark_last_busy (rq); + hctx->tags->rqs[rq->tag] =NULL ; rq->mq_hctx =NULL ;if (rq->tag !=BLK_MQ_NO_TAG )blk_mq_put_tag (hctx->tags, ctx, rq->tag);block/blk-mq.h +0
-0
block/blk-mq.h static inline int __blk_mq_active_requests(struct blk_mq_hw_ctx *hctx) · line 229
Quoted 6 lines
block/blk-mq.h +1
-0@@ -224,6 +224,7 @@ static inline int __blk_mq_active_requests(struct blk_mq_hw_ctx *hctx)static inline void __blk_mq_put_driver_tag (struct blk_mq_hw_ctx *hctx,struct request *rq) { + hctx->tags->rqs[rq->tag] =NULL ;blk_mq_put_tag (hctx->tags, rq->mq_ctx, rq->tag); rq->tag =BLK_MQ_NO_TAG ;
Is this perhaps a block driver bug instead of a block layer core bug? If this would be a block layer core bug, it would have been reported before.
Bart.
Quoted 55 lines · Bart Van Assche
Quoted 54 lines · Pradeep P V K
Observes below crash while accessing (use-after-free) request queue member of struct request.
191.784789: <2> Unable to handle kernel paging request at virtual
addressffffff81429a4440
...
191.786174: <2> CPU: 3 PID: 213 Comm: kworker/3:1H Tainted: G S
O 5.4.61-qgki-debug-ge45de39 #1
...191.786226: <2> Workqueue: kblockd blk_mq_timeout_work 191.786242: <2> pstate: 20c00005 (nzCv daif +PAN +UAO) 191.786261: <2> pc : bt_for_each+0x114/0x1a4 191.786274: <2> lr : bt_for_each+0xe0/0x1a4 ... 191.786494: <2> Call trace: 191.786507: <2> bt_for_each+0x114/0x1a4 191.786519: <2> blk_mq_queue_tag_busy_iter+0x60/0xd4 191.786532: <2> blk_mq_timeout_work+0x54/0xe8 191.786549: <2> process_one_work+0x2cc/0x568 191.786562: <2> worker_thread+0x28c/0x518 191.786577: <2> kthread+0x160/0x170 191.786594: <2> ret_from_fork+0x10/0x18 191.786615: <2> Code: 0b080148 f9404929 f8685921 b4fffe01 (f9400028) 191.786630: <2> ---[ end trace 0f1f51d79ab3f955 ]--- 191.786643: <2> Kernel panic - not syncing: Fatal exceptionFix this by updating the freed request with NULL. This could avoid accessing the already free request from other contexts while iterating over the requests.
Signed-off-by Pradeep P V K <[email protected]>--- block/blk-mq.c | 1 + block/blk-mq.h | 1 + 2 files changed, 2 insertions(+)block/blk-mq.c +1
-0@@ -492,6 +492,7 @@ static void __blk_mq_free_request(struct request *rq)blk_crypto_free_request (rq);blk_pm_mark_last_busy (rq); + hctx->tags->rqs[rq->tag] =NULL ; rq->mq_hctx =NULL ;if (rq->tag !=BLK_MQ_NO_TAG )blk_mq_put_tag (hctx->tags, ctx, rq->tag);block/blk-mq.h +0
-0@@ -224,6 +224,7 @@ static inline int __blk_mq_active_requests(struct blk_mq_hw_ctx *hctx) static inline void __blk_mq_put_driver_tag(struct blk_mq_hw_ctx *hctx,
Quoted 8 lines
block/blk-mq.h static inline int __blk_mq_active_requests(struct blk_mq_hw_ctx *hctx) · line 229
Quoted 5 lines
block/blk-mq.h +1
-0struct request *rq) { + hctx->tags->rqs[rq->tag] =NULL ;blk_mq_put_tag (hctx->tags, rq->mq_ctx, rq->tag); rq->tag =BLK_MQ_NO_TAG ;Is this perhaps a block driver bug instead of a block layer core bug? If this would be a block layer core bug, it would have been reported before.
Isn't this the same issue which as been reported many times:
https://lore.kernel.org/linux-block/[email protected]/
https://lore.kernel.org/linux-block/[email protected]/
But I never saw a crash, just kasan report.
Thanks,
John
Quoted 68 lines · John Garry
Quoted 56 lines · Bart Van Assche
Quoted 55 lines · Pradeep P V K
Observes below crash while accessing (use-after-free) request queue member of struct request.
191.784789: <2> Unable to handle kernel paging request at virtual address
ffffff81429a4440
...
191.786174: <2> CPU: 3 PID: 213 Comm: kworker/3:1H Tainted: G S O 5.4.61-qgki-debug-ge45de39 #1
...
191.786226: <2> Workqueue: kblockdblk_mq_timeout_work191.786242: <2> pstate:20c00005(nzCv daif +PAN +UAO) 191.786261: <2> pc :bt_for_each+0x114/0x1a4
191.786274: <2> lr :bt_for_each+0xe0/0x1a4
...
191.786494: <2> Call trace:
191.786507: <2>bt_for_each+0x114/0x1a4
191.786519: <2>blk_mq_queue_tag_busy_iter+0x60/0xd4191.786532: <2>blk_mq_timeout_work+0x54/0xe8191.786549: <2>process_one_work+0x2cc/0x568
191.786562: <2>worker_thread+0x28c/0x518
191.786577: <2>kthread+0x160/0x170
191.786594: <2>ret_from_fork+0x10/0x18
191.786615: <2> Code:0b080148f9404929f8685921b4fffe01(f9400028) 191.786630: <2> ---[ end trace0f1f51d79ab3f955]--- 191.786643: <2> Kernel panic - not syncing: Fatal exceptionFix this by updating the freed request with NULL. This could avoid accessing the already free request from other contexts while iterating over the requests.
Signed-off-by Pradeep P V K <[email protected]>---
block/blk-mq.c| 1 +
block/blk-mq.h| 1 +2 files changed, 2 insertions(+)block/blk-mq.c +1
-0@@ -492,6 +492,7 @@ static void __blk_mq_free_request(struct request *rq) blk_crypto_free_request(rq); blk_pm_mark_last_busy(rq); + hctx->tags->rqs[rq->tag] = NULL; rq->mq_hctx = NULL; if (rq->tag != BLK_MQ_NO_TAG) blk_mq_put_tag(hctx->tags, ctx, rq->tag);block/blk-mq.h +0
-0@@ -224,6 +224,7 @@ static inline int __blk_mq_active_requests(struct blk_mq_hw_ctx *hctx) static inline void __blk_mq_put_driver_tag(struct blk_mq_hw_ctx *hctx,Quoted 8 lines
block/blk-mq.h static inline int __blk_mq_active_requests(struct blk_mq_hw_ctx *hctx) · line 229
Quoted 5 lines
block/blk-mq.h +1
-0struct request *rq) { + hctx->tags->rqs[rq->tag] = NULL; blk_mq_put_tag(hctx->tags, rq->mq_ctx, rq->tag); rq->tag = BLK_MQ_NO_TAG;Is this perhaps a block driver bug instead of a block layer core bug? If this would be a block layer core bug, it would have been reported before.
Isn't this the same issue which as been reported many times:
Quoted 8 lines
https://lore.kernel.org/linux-block/[email protected]/
https://lore.kernel.org/linux-block/[email protected]/
But I never saw a crash, just kasan report.
And if that above were a concern, I would have thought one would need to use a WRITE_ONCE() here; otherwise we might have a race condition where other CPUs still see the old value, no?
Cheers,
Hannes
Signature · 4 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
Quoted 72 lines · Hannes Reinecke
Quoted 71 lines · John Garry
Quoted 56 lines · Bart Van Assche
Quoted 55 lines · Pradeep P V K
Observes below crash while accessing (use-after-free) request queue member of struct request.
191.784789: <2> Unable to handle kernel paging request at virtual address
ffffff81429a4440
...
191.786174: <2> CPU: 3 PID: 213 Comm: kworker/3:1H Tainted: G S O 5.4.61-qgki-debug-ge45de39 #1
...
191.786226: <2> Workqueue: kblockdblk_mq_timeout_work191.786242: <2> pstate:20c00005(nzCv daif +PAN +UAO) 191.786261: <2> pc :bt_for_each+0x114/0x1a4
191.786274: <2> lr :bt_for_each+0xe0/0x1a4
...
191.786494: <2> Call trace:
191.786507: <2>bt_for_each+0x114/0x1a4
191.786519: <2>blk_mq_queue_tag_busy_iter+0x60/0xd4191.786532: <2>blk_mq_timeout_work+0x54/0xe8191.786549: <2>process_one_work+0x2cc/0x568
191.786562: <2>worker_thread+0x28c/0x518
191.786577: <2>kthread+0x160/0x170
191.786594: <2>ret_from_fork+0x10/0x18
191.786615: <2> Code:0b080148f9404929f8685921b4fffe01(f9400028) 191.786630: <2> ---[ end trace0f1f51d79ab3f955]--- 191.786643: <2> Kernel panic - not syncing: Fatal exceptionFix this by updating the freed request with NULL. This could avoid accessing the already free request from other contexts while iterating over the requests.
Signed-off-by Pradeep P V K <[email protected]>---
block/blk-mq.c| 1 +
block/blk-mq.h| 1 +2 files changed, 2 insertions(+)block/blk-mq.c +1
-0@@ -492,6 +492,7 @@ static void __blk_mq_free_request(struct request *rq) blk_crypto_free_request(rq); blk_pm_mark_last_busy(rq); + hctx->tags->rqs[rq->tag] = NULL; rq->mq_hctx = NULL; if (rq->tag != BLK_MQ_NO_TAG) blk_mq_put_tag(hctx->tags, ctx, rq->tag);block/blk-mq.h +0
-0@@ -224,6 +224,7 @@ static inline int __blk_mq_active_requests(struct blk_mq_hw_ctx *hctx) static inline void __blk_mq_put_driver_tag(struct blk_mq_hw_ctx *hctx,Quoted 8 lines
block/blk-mq.h static inline int · line 230
Quoted 5 lines
block/blk-mq.h +1
-0struct request *rq) { + hctx->tags->rqs[rq->tag] = NULL; blk_mq_put_tag(hctx->tags, rq->mq_ctx, rq->tag); rq->tag = BLK_MQ_NO_TAG;Is this perhaps a block driver bug instead of a block layer core bug? If this would be a block layer core bug, it would have been reported before.
Isn't this the same issue which as been reported many times:
Quoted 6 lines
Quoted 5 lines
https://lore.kernel.org/linux-block/[email protected]/
But I never saw a crash, just kasan report.
And if that above were a concern, I would have thought one would need to use a
WRITE_ONCE()here; otherwise we might have a race condition where other CPUs still see the old value, no?
From further looking at the history here, Kashyap tried the same approach and preference was to not add anything to the fast path:
https://lore.kernel.org/linux-block/[email protected]/
I had another solution for this which clears any references when we free the sched tags, but never posted as it looked a bit crazy and I was busy, below.
---->8------
block/blk-mq-sched.c +2 -3
@@ -608,7 +608,7 @@ static void blk_mq_sched_free_tags(struct blk_mq_tag_set *set, unsigned int hctx_idx) {if (hctx->sched_tags) { -blk_mq_free_rqs (set, hctx->sched_tags, hctx_idx); +blk_mq_free_rqs (set, hctx->sched_tags, hctx_idx,NULL );blk_mq_free_rq_map (hctx->sched_tags); hctx->sched_tags =NULL ; }
@@ -711,10 +711,9 @@ void blk_mq_sched_free_requests(struct request_queue *q) {struct blk_mq_hw_ctx *hctx;int i;queue_for_each_hw_ctx (q, hctx, i) { -if (hctx->sched_tags) -blk_mq_free_rqs (q->tag_set, hctx->sched_tags, i); +blk_mq_free_rqs (q->tag_set, hctx->sched_tags, i, hctx->tags); } }
block/blk-mq-tag.c +1 -1
@@ -515,7 +515,7 @@ int blk_mq_tag_update_depth(struct blk_mq_hw_ctx *hctx,return -ENOMEM ; } -blk_mq_free_rqs (set, *tagsptr, hctx->queue_num); +blk_mq_free_rqs (set, *tagsptr, hctx->queue_num,NULL );blk_mq_free_rq_map (*tagsptr); *tagsptr = new; }else {
block/blk-mq.c +7 -4
@@ -2256,12 +2256,12 @@ blk_qc_t blk_mq_submit_bio(struct bio *bio)EXPORT_SYMBOL_GPL (blk_mq_submit_bio);/* only for request based dm */ void blk_mq_free_rqs (struct blk_mq_tag_set *set,struct blk_mq_tags *tags, -unsigned int hctx_idx) +unsigned int hctx_idx,struct blk_mq_tags *references) {struct page *page;if (tags->rqs && set->ops->exit_request) { -int i; +int i, j;for (i =0 ; i < tags->nr_tags; i++) {struct request *rq = tags->static_rqs[i];
@@ -2269,6 +2269,12 @@ void blk_mq_free_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags,if (!rq)continue ; set->ops->exit_request (set, rq, hctx_idx); +for (j =0 ; references && j < references->nr_tags; j++) { +struct request *old =cmpxchg (&references->rqs[j], rq,0 ); + } tags->static_rqs[i] =NULL ; } }
@@ -2425,7 +2431,7 @@ int blk_mq_alloc_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags,return 0 ; fail: -blk_mq_free_rqs (set, tags, hctx_idx); +blk_mq_free_rqs (set, tags, hctx_idx,NULL );return -ENOMEM ; }
@@ -2755,7 +2761,7 @@ static void blk_mq_free_map_and_requests(struct blk_mq_tag_set *set, unsigned int hctx_idx) {if (set->tags && set->tags[hctx_idx]) { -blk_mq_free_rqs (set, set->tags[hctx_idx], hctx_idx); +blk_mq_free_rqs (set, set->tags[hctx_idx], hctx_idx,NULL );blk_mq_free_rq_map (set->tags[hctx_idx]); set->tags[hctx_idx] =NULL ; }
block/blk-mq.h +1 -1
@@ -52,7 +52,7 @@ struct request *blk_mq_dequeue_from_ctx(struct blk_mq_hw_ctx *hctx,* Internal helpers for allocating/freeing the request map */ void blk_mq_free_rqs (struct blk_mq_tag_set *set,struct blk_mq_tags *tags, -unsigned int hctx_idx); +unsigned int hctx_idx,struct blk_mq_tags *references);void blk_mq_free_rq_map (struct blk_mq_tags *tags); struct blk_mq_tags *blk_mq_alloc_rq_map(struct blk_mq_tag_set *set, unsigned int hctx_idx,
Quoted 74 lines · Hannes Reinecke
Quoted 73 lines · John Garry
Quoted 58 lines · Bart Van Assche
Quoted 57 lines · Pradeep P V K
Observes below crash while accessing (use-after-free) request queue member of struct request.
191.784789: <2> Unable to handle kernel paging request at virtual address
ffffff81429a4440
...
191.786174: <2> CPU: 3 PID: 213 Comm: kworker/3:1H Tainted: G S O 5.4.61-qgki-debug-ge45de39 #1
...
191.786226: <2> Workqueue: kblockdblk_mq_timeout_work191.786242: <2> pstate:20c00005(nzCv daif +PAN +UAO) 191.786261: <2> pc :bt_for_each+0x114/0x1a4
191.786274: <2> lr :bt_for_each+0xe0/0x1a4
...
191.786494: <2> Call trace:
191.786507: <2>bt_for_each+0x114/0x1a4
191.786519: <2>blk_mq_queue_tag_busy_iter+0x60/0xd4191.786532: <2>blk_mq_timeout_work+0x54/0xe8191.786549: <2>process_one_work+0x2cc/0x568
191.786562: <2>worker_thread+0x28c/0x518
191.786577: <2>kthread+0x160/0x170
191.786594: <2>ret_from_fork+0x10/0x18
191.786615: <2> Code:0b080148f9404929f8685921b4fffe01(f9400028) 191.786630: <2> ---[ end trace0f1f51d79ab3f955]--- 191.786643: <2> Kernel panic - not syncing: Fatal exceptionFix this by updating the freed request with NULL. This could avoid accessing the already free request from other contexts while iterating over the requests.
Signed-off-by Pradeep P V K <[email protected]>---
block/blk-mq.c| 1 +
block/blk-mq.h| 1 +2 files changed, 2 insertions(+)block/blk-mq.c +1
-0@@ -492,6 +492,7 @@ static void __blk_mq_free_request(struct request *rq) blk_crypto_free_request(rq); blk_pm_mark_last_busy(rq); + hctx->tags->rqs[rq->tag] = NULL; rq->mq_hctx = NULL; if (rq->tag != BLK_MQ_NO_TAG) blk_mq_put_tag(hctx->tags, ctx, rq->tag);block/blk-mq.h +0
-0@@ -224,6 +224,7 @@ static inline int __blk_mq_active_requests(struct blk_mq_hw_ctx *hctx)static inline void __blk_mq_put_driver_tag(struct
blk_mq_hw_ctx*hctx,
struct request *rq)Quoted 8 lines
Quoted 4 lines
{
+ hctx->tags->rqs[rq->tag] = NULL;blk_mq_put_tag(hctx->tags, rq->mq_ctx, rq->tag);rq->tag=BLK_MQ_NO_TAG;Is this perhaps a block driver bug instead of a block layer core bug? If this would be a block layer core bug, it would have been reported before.
Isn't this the same issue which as been reported many times:
Quoted 8 lines
Quoted 5 lines
https://lore.kernel.org/linux-block/[email protected]/
But I never saw a crash, just kasan report.
And if that above were a concern, I would have thought one would need to use a
WRITE_ONCE()here; otherwise we might have a race condition where other CPUs still see the old value, no?
Hi Hannes,
Freeing tag->rqs and tags->static_rqs with kfree_rcu() is probably a better solution than clearing request pointers. Even when using WRITE_ONCE() to clear tag pointers, it is still possible that another thread read the tag pointer before the WRITE_ONCE() and uses it after the WRITE_ONCE() has finished.
Thanks,
Bart.