virtio_blk: fix race between start and stop queue
Subsystems (3)
PATCH 1/1 virtio_blk: fix race between start and stop queue
When there isn't enough vring descriptor for adding to vq, blk-mq will be put as stopped state until some of pending descriptors are completed & freed.
Unfortunately, the vq's interrupt may come just before blk-mq's BLK_MQ_S_STOPPED flag is set, so the blk-mq will still be kept as stopped even though lots of descriptors are completed and freed in the interrupt handler. The worst case is that all pending descriptors are freed in the interrupt handler, and the queue is kept as stopped forever.
This patch fixes the problem by starting/stopping blk-mq with holding vq_lock.
---
v1:
- remove 'queue_stopped' flag
drivers/block/virtio_blk.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/block/virtio_blk.c +2 -2
@@ -147,11 +147,11 @@ static void virtblk_done(struct virtqueue *vq)if (unlikely (virtqueue_is_broken (vq)))break ; }while (!virtqueue_enable_cb (vq)); -spin_unlock_irqrestore (&vblk->vq_lock, flags);/* In case queue is stopped waiting for more buffers. */ if (req_done)blk_mq_start_stopped_hw_queues (vblk->disk->queue, true); +spin_unlock_irqrestore (&vblk->vq_lock, flags); }static int virtio_queue_rq (struct blk_mq_hw_ctx *hctx,struct request *req)
@@ -205,8 +205,8 @@ static int virtio_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *req) err =__virtblk_add_req (vblk->vq, vbr, vbr->sg, num);if (err) {virtqueue_kick (vblk->vq); -spin_unlock_irqrestore (&vblk->vq_lock, flags);blk_mq_stop_hw_queue (hctx); +spin_unlock_irqrestore (&vblk->vq_lock, flags);/* Out of mem doesn't actually happen, since we fall back * to direct descriptors */ if (err == -ENOMEM || err == -ENOSPC )
Signature · 1 line
1.7.9.5
Jens Axboe
maintainerQuoted 11 lines · Ming Lei
When there isn't enough vring descriptor for adding to vq, blk-mq will be put as stopped state until some of pending descriptors are completed & freed.
Unfortunately, the vq's interrupt may come just before blk-mq's
BLK_MQ_S_STOPPEDflag is set, so the blk-mq will still be kept as stopped even though lots of descriptors are completed and freed in the interrupt handler. The worst case is that all pending descriptors are freed in the interrupt handler, and the queue is kept as stopped forever.
Quoted 2 lines
This patch fixes the problem by starting/stopping blk-mq with holding
vq_lock.
Thanks, this looks good, I'll apply it for 3.16 (with a stable marker, even if it is an unlikely event).
Signature · 1 line
Jens Axboe
Quoted 13 lines · Jens Axboe
Quoted 12 lines · Ming Lei
When there isn't enough vring descriptor for adding to vq, blk-mq will be put as stopped state until some of pending descriptors are completed & freed.
Unfortunately, the vq's interrupt may come just before blk-mq's
BLK_MQ_S_STOPPEDflag is set, so the blk-mq will still be kept as stopped even though lots of descriptors are completed and freed in the interrupt handler. The worst case is that all pending descriptors are freed in the interrupt handler, and the queue is kept as stopped forever.
Quoted 6 lines
Quoted 2 lines
This patch fixes the problem by starting/stopping blk-mq with holding
vq_lock.Thanks, this looks good, I'll apply it for 3.16 (with a stable marker, even if it is an unlikely event).
Thanks.
It shouldn't be very difficult to happen in case of non-indirect descriptor, and it is easy to reproduce when module parameter of 'virtblk_queue_depth' is bigger than vq->num_free for non-indirect case.
Thanks,
Signature · 1 line
Ming Lei
Jens Axboe
maintainerQuoted 21 lines · Ming Lei
Quoted 13 lines · Jens Axboe
Quoted 12 lines · Ming Lei
When there isn't enough vring descriptor for adding to vq, blk-mq will be put as stopped state until some of pending descriptors are completed & freed.
Unfortunately, the vq's interrupt may come just before blk-mq's
BLK_MQ_S_STOPPEDflag is set, so the blk-mq will still be kept as stopped even though lots of descriptors are completed and freed in the interrupt handler. The worst case is that all pending descriptors are freed in the interrupt handler, and the queue is kept as stopped forever.Quoted 6 lines
Quoted 2 lines
This patch fixes the problem by starting/stopping blk-mq with holding
vq_lock.Thanks, this looks good, I'll apply it for 3.16 (with a stable marker, even if it is an unlikely event).
Quoted 6 lines
Thanks.
It shouldn't be very difficult to happen in case of non-indirect descriptor, and it is easy to reproduce when module parameter of '
virtblk_queue_depth' is bigger thanvq->num_freefor non-indirect case.
I agree, it can definitely be setup so that it would not be hard to trigger. But I don't recall seeing any hang bugs since 3.13 was released, which would seem to indicate that it doesn't happen a lot in the wild with default settings.
Signature · 1 line
Jens Axboe