Skip to contentSkip to conversation

virtio_blk: always post notifications under the lock

PATCH In review Andrew Boyer 4 participants 24 messages Jan 7, 2025 – Feb 24, 2025 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 virtio_blk: always post notifications under the lock

Commit af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support") added notification data support to the core virtio driver code. When this feature is enabled, the notification includes the updated producer index for the queue. Thus it is now critical that notifications arrive in order.

The virtio_blk driver has historically not worried about notification ordering. Modify it so that the prepare and kick steps are both done under the vq lock.

Fixes af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support")
Cc Viktor Prutyanov <[email protected]>
---
 drivers/block/virtio_blk.c | 19 ++++---------------
 1 file changed, 4 insertions(+), 15 deletions(-)
drivers/block/virtio_blk.c +4 -15
index 3efe378f1386..14d9e66bb844 100644
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
 	struct virtio_blk *vblk = hctx->queue->queuedata;
 	struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-	bool kick;
 
 	spin_lock_irq(&vq->lock);
-	kick = virtqueue_kick_prepare(vq->vq);
+	virtqueue_kick(vq->vq);
 	spin_unlock_irq(&vq->lock);
-
-	if (kick)
-		virtqueue_notify(vq->vq);
 }
 
 static blk_status_t virtblk_fail_to_queue(struct request *req, int rc)
@@ -432,7 +428,6 @@ static blk_status_t virtio_queue_rq(struct blk_mq_hw_ctx *hctx,
 	struct virtblk_req *vbr = blk_mq_rq_to_pdu(req);
 	unsigned long flags;
 	int qid = hctx->queue_num;
-	bool notify = false;
 	blk_status_t status;
 	int err;
 
@@ -454,12 +449,10 @@ static blk_status_t virtio_queue_rq(struct blk_mq_hw_ctx *hctx,
 		return virtblk_fail_to_queue(req, err);
 	}
 
-	if (bd->last && virtqueue_kick_prepare(vblk->vqs[qid].vq))
-		notify = true;
+	if (bd->last)
+		virtqueue_kick(vblk->vqs[qid].vq);
 	spin_unlock_irqrestore(&vblk->vqs[qid].lock, flags);
 
-	if (notify)
-		virtqueue_notify(vblk->vqs[qid].vq);
 	return BLK_STS_OK;
 }
 
@@ -476,7 +469,6 @@ static void virtblk_add_req_batch(struct virtio_blk_vq *vq,
 {
 	struct request *req;
 	unsigned long flags;
-	bool kick;
 
 	spin_lock_irqsave(&vq->lock, flags);
 
@@ -492,11 +484,8 @@ static void virtblk_add_req_batch(struct virtio_blk_vq *vq,
 		}
 	}
 
-	kick = virtqueue_kick_prepare(vq->vq);
+	virtqueue_kick(vq->vq);
 	spin_unlock_irqrestore(&vq->lock, flags);
-
-	if (kick)
-		virtqueue_notify(vq->vq);
 }
 
 static void virtio_queue_rqs(struct rq_list *rqlist)
Signature · 1 line
2.17.1
#1 lore raw
Quoted 10 lines · Andrew Boyer

Commit af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support") added notification data support to the core virtio driver code. When this feature is enabled, the notification includes the updated producer index for the queue. Thus it is now critical that notifications arrive in order.

The virtio_blk driver has historically not worried about notification ordering. Modify it so that the prepare and kick steps are both done under the vq lock.

Do we have performance numbers when VIRTIO_F_NOTIFICATION_DATA is not negotiated?

We need to make sure it doesn't introduce any regression in the case like virtualization setup since now there could be an vmexit when holding the virtqueue lock.

Thanks

Quoted 78 lines
Fixes af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support")
Cc Viktor Prutyanov <[email protected]>
---
 drivers/block/virtio_blk.c | 19 ++++---------------
 1 file changed, 4 insertions(+), 15 deletions(-)
drivers/block/virtio_blk.c +4 -16
index 3efe378f1386..14d9e66bb844 100644
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
        struct virtio_blk *vblk = hctx->queue->queuedata;
        struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-       bool kick;
 
        spin_lock_irq(&vq->lock);
-       kick = virtqueue_kick_prepare(vq->vq);
+       virtqueue_kick(vq->vq);
        spin_unlock_irq(&vq->lock);
-
-       if (kick)
-               virtqueue_notify(vq->vq);
 }
 
 static blk_status_t virtblk_fail_to_queue(struct request *req, int rc)
@@ -432,7 +428,6 @@ static blk_status_t virtio_queue_rq(struct blk_mq_hw_ctx *hctx,
        struct virtblk_req *vbr = blk_mq_rq_to_pdu(req);
        unsigned long flags;
        int qid = hctx->queue_num;
-       bool notify = false;
        blk_status_t status;
        int err;
 
@@ -454,12 +449,10 @@ static blk_status_t virtio_queue_rq(struct blk_mq_hw_ctx *hctx,
                return virtblk_fail_to_queue(req, err);
        }
 
-       if (bd->last && virtqueue_kick_prepare(vblk->vqs[qid].vq))
-               notify = true;
+       if (bd->last)
+               virtqueue_kick(vblk->vqs[qid].vq);
        spin_unlock_irqrestore(&vblk->vqs[qid].lock, flags);
 
-       if (notify)
-               virtqueue_notify(vblk->vqs[qid].vq);
        return BLK_STS_OK;
 }
 
@@ -476,7 +469,6 @@ static void virtblk_add_req_batch(struct virtio_blk_vq *vq,
 {
        struct request *req;
        unsigned long flags;
-       bool kick;
 
        spin_lock_irqsave(&vq->lock, flags);
 
@@ -492,11 +484,8 @@ static void virtblk_add_req_batch(struct virtio_blk_vq *vq,
                }
        }
 
-       kick = virtqueue_kick_prepare(vq->vq);
+       virtqueue_kick(vq->vq);
        spin_unlock_irqrestore(&vq->lock, flags);
-
-       if (kick)
-               virtqueue_notify(vq->vq);
 }
 
 static void virtio_queue_rqs(struct rq_list *rqlist)
--

2.17.1

#2 lore raw
In reply to Andrew Boyer
Quoted 10 lines · Andrew Boyer

Commit af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support") added notification data support to the core virtio driver code. When this feature is enabled, the notification includes the updated producer index for the queue. Thus it is now critical that notifications arrive in order.

The virtio_blk driver has historically not worried about notification ordering. Modify it so that the prepare and kick steps are both done under the vq lock.

Quoted 6 lines
Fixes af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support")
Cc Viktor Prutyanov <[email protected]>

Hmm. Not good, notify can be very slow, holding a lock is a bad idea. Basically, virtqueue_notify must work ouside of locks, this means af8ececda185 is broken and we did not notice.

Let's fix it please.

Try some kind of compare and swap scheme where we detect that index was updated since? Will allow skipping a notification, too.

Quoted 70 lines
---
 drivers/block/virtio_blk.c | 19 ++++---------------
 1 file changed, 4 insertions(+), 15 deletions(-)
drivers/block/virtio_blk.c +4 -15
index 3efe378f1386..14d9e66bb844 100644
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
 	struct virtio_blk *vblk = hctx->queue->queuedata;
 	struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-	bool kick;
 
 	spin_lock_irq(&vq->lock);
-	kick = virtqueue_kick_prepare(vq->vq);
+	virtqueue_kick(vq->vq);
 	spin_unlock_irq(&vq->lock);
-
-	if (kick)
-		virtqueue_notify(vq->vq);
 }
 
 static blk_status_t virtblk_fail_to_queue(struct request *req, int rc)
@@ -432,7 +428,6 @@ static blk_status_t virtio_queue_rq(struct blk_mq_hw_ctx *hctx,
 	struct virtblk_req *vbr = blk_mq_rq_to_pdu(req);
 	unsigned long flags;
 	int qid = hctx->queue_num;
-	bool notify = false;
 	blk_status_t status;
 	int err;
 
@@ -454,12 +449,10 @@ static blk_status_t virtio_queue_rq(struct blk_mq_hw_ctx *hctx,
 		return virtblk_fail_to_queue(req, err);
 	}
 
-	if (bd->last && virtqueue_kick_prepare(vblk->vqs[qid].vq))
-		notify = true;
+	if (bd->last)
+		virtqueue_kick(vblk->vqs[qid].vq);
 	spin_unlock_irqrestore(&vblk->vqs[qid].lock, flags);
 
-	if (notify)
-		virtqueue_notify(vblk->vqs[qid].vq);
 	return BLK_STS_OK;
 }
 
@@ -476,7 +469,6 @@ static void virtblk_add_req_batch(struct virtio_blk_vq *vq,
 {
 	struct request *req;
 	unsigned long flags;
-	bool kick;
 
 	spin_lock_irqsave(&vq->lock, flags);
 
@@ -492,11 +484,8 @@ static void virtblk_add_req_batch(struct virtio_blk_vq *vq,
 		}
 	}
 
-	kick = virtqueue_kick_prepare(vq->vq);
+	virtqueue_kick(vq->vq);
 	spin_unlock_irqrestore(&vq->lock, flags);
-
-	if (kick)
-		virtqueue_notify(vq->vq);
 }
 
 static void virtio_queue_rqs(struct rq_list *rqlist)
Signature · 1 line
2.17.1
#3 lore raw
In reply to Andrew Boyer

Am 07.01.25 um 19:25 schrieb Andrew Boyer:

Quoted 28 lines

Commit af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support") added notification data support to the core virtio driver code. When this feature is enabled, the notification includes the updated producer index for the queue. Thus it is now critical that notifications arrive in order.

The virtio_blk driver has historically not worried about notification ordering. Modify it so that the prepare and kick steps are both done under the vq lock.

Fixes af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support")
Cc Viktor Prutyanov <[email protected]>
---
  drivers/block/virtio_blk.c | 19 ++++---------------
  1 file changed, 4 insertions(+), 15 deletions(-)
drivers/block/virtio_blk.c +0 -0
index 3efe378f1386..14d9e66bb844 100644
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
  {
  	struct virtio_blk *vblk = hctx->queue->queuedata;
  	struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 386

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
-	bool kick;
 
  	spin_lock_irq(&vq->lock);
-	kick = virtqueue_kick_prepare(vq->vq);
+	virtqueue_kick(vq->vq);
  	spin_unlock_irq(&vq->lock);
-
-	if (kick)
-		virtqueue_notify(vq->vq);
  }

I would assume this will be a performance nightmare for normal IO.

#4 lore raw
Quoted 34 lines · Christian Borntraeger

Am 07.01.25 um 19:25 schrieb Andrew Boyer:

Quoted 31 lines

Commit af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support") added notification data support to the core virtio driver code. When this feature is enabled, the notification includes the updated producer index for the queue. Thus it is now critical that notifications arrive in order.

The virtio_blk driver has historically not worried about notification ordering. Modify it so that the prepare and kick steps are both done under the vq lock.

Fixes af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support")
Cc Viktor Prutyanov <[email protected]>
---
  drivers/block/virtio_blk.c | 19 ++++---------------
  1 file changed, 4 insertions(+), 15 deletions(-)
drivers/block/virtio_blk.c +0 -2
index 3efe378f1386..14d9e66bb844 100644
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
  {
  	struct virtio_blk *vblk = hctx->queue->queuedata;
  	struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-	bool kick;
  	spin_lock_irq(&vq->lock);
-	kick = virtqueue_kick_prepare(vq->vq);
Quoted 8 lines

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 386

Quoted 6 lines
drivers/block/virtio_blk.c +1 -3
+	virtqueue_kick(vq->vq);
  	spin_unlock_irq(&vq->lock);
-
-	if (kick)
-		virtqueue_notify(vq->vq);
  }

I would assume this will be a performance nightmare for normal IO.

Indeed.
AMD guys, can't device survive with reordered notifications? Basically just drop a notification if you see index going back?

Signature · 1 line
MST
#5 lore raw
Quoted 36 lines · Michael S. Tsirkin
Quoted 34 lines · Christian Borntraeger

Am 07.01.25 um 19:25 schrieb Andrew Boyer:

Quoted 31 lines

Commit af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support") added notification data support to the core virtio driver code. When this feature is enabled, the notification includes the updated producer index for the queue. Thus it is now critical that notifications arrive in order.

The virtio_blk driver has historically not worried about notification ordering. Modify it so that the prepare and kick steps are both done under the vq lock.

Fixes af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature support")
Cc Viktor Prutyanov <[email protected]>
---
 drivers/block/virtio_blk.c | 19 ++++---------------
 1 file changed, 4 insertions(+), 15 deletions(-)
drivers/block/virtio_blk.c +0 -2
index 3efe378f1386..14d9e66bb844 100644
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
   struct virtio_blk *vblk = hctx->queue->queuedata;
   struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
   spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
Quoted 9 lines
Quoted 8 lines

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 386

Quoted 6 lines
drivers/block/virtio_blk.c +1 -3
+   virtqueue_kick(vq->vq);
   spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
 }

I would assume this will be a performance nightmare for normal IO.

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

I don't have any sort of setup where I can try it but I would appreciate it if someone else could.

Quoted 5 lines

Hmm. Not good, notify can be very slow, holding a lock is a bad idea. Basically, virtqueue_notify must work ouside of locks, this means af8ececda185 is broken and we did not notice.

Let's fix it please.

With so many broken kernels already in the wild, I think disabling F_NOTIFICATION_DATA for virtio-blk would be a reasonable solution.

Quoted 2 lines

Try some kind of compare and swap scheme where we detect that index was updated since? Will allow skipping a notification, too.

Do you have an idea of how this might be done? Anything I've come up with involves a lock. Would it be doable to have a lock for the vq management stuff and a second one to post notifications?

Quoted 3 lines

AMD guys, can't device survive with reordered notifications? Basically just drop a notification if you see index going back?

This is the driver lying to us about the state of the queue; it's not going to be possible for us to work around it in hardware. For starters, how would we detect queue wrap around?

Thank you,
Andrew

(Apologies for sending twice, mail client issue)

A message that is not in the archive (sent off-list or not yet received): [email protected]

#7 lore raw
Quoted 77 lines · Andrew

On Jan 9, 2025, at 8:42 AM, Michael S. Tsirkin <[email protected]> wrote:

On Thu, Jan 09, 2025 at 01:01:20PM +0100, Christian Borntraeger wrote:

Am 07.01.25 um 19:25 schrieb Andrew Boyer:

Commit af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature
support") added notification data support to the core virtio driver code. When this feature is enabled, the notification includes the updated producer index for the queue. Thus it is now critical that notifications arrive in order.

The virtio_blk driver has historically not worried about notification
ordering. Modify it so that the prepare and kick steps are both done
under the vq lock.

Fixes af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA

feature support")
Cc: Viktor Prutyanov <[email protected]>
Cc: [email protected]
Cc: [email protected]
---
drivers/block/virtio_blk.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-)

diff --git a/drivers/block/virtio_blk.c b/drivers/block/ virtio_blk.c
index 3efe378f1386..14d9e66bb844 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
{
struct virtio_blk *vblk = hctx->queue->queuedata; struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
- bool kick;
spin_lock_irq(&vq->lock);
- kick = virtqueue_kick_prepare(vq->vq);
+ virtqueue_kick(vq->vq);
spin_unlock_irq(&vq->lock);
-
- if (kick)
- virtqueue_notify(vq->vq);
}

I would assume this will be a performance nightmare for normal IO.

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

I don't have any sort of setup where I can try it but I would appreciate it if someone else could.

Hmm. Not good, notify can be very slow, holding a lock is a bad idea. Basically, virtqueue_notify must work ouside of locks, this means af8ececda185 is broken and we did not notice.

Quoted 5 lines

Let's fix it please.

With so many broken kernels already in the wild, I think disabling F_NOTIFICATION_DATA for virtio-blk would be a reasonable solution.

Some devices might fail feature negotiation then.
I am not sure they are broken, devices might simply be able to handle out of order values.

Quoted 10 lines

Try some kind of compare and swap scheme where we detect that index was updated since? Will allow skipping a notification, too.

Do you have an idea of how this might be done? Anything I've come up with involves a lock.

Would it be doable to have a lock for the vq management stuff and a second one to post notifications?

and only for when F_NOTIFICATION_DATA is set. not terrible ok I think.

Quoted 6 lines

AMD guys, can't device survive with reordered notifications? Basically just drop a notification if you see index going back?

Quoted 6 lines

This is the driver lying to us about the state of the queue; it's not going to be possible for us to work around it in hardware. For starters, how would we detect queue wrap around?

Thank you,
Andrew

The index is a running value for split, for wrap arounds, there is a special bit for that. No?

Quoted 6 lines

    --
    MST
#8 lore raw
Quoted 88 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 80 lines · Andrew

On Jan 9, 2025, at 8:42 AM, Michael S. Tsirkin <[email protected]> wrote:

On Thu, Jan 09, 2025 at 01:01:20PM +0100, Christian Borntraeger wrote:

Am 07.01.25 um 19:25 schrieb Andrew Boyer:

Commit af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature
support") added notification data support to the core virtio driver code. When this feature is enabled, the notification includes the updated producer index for the queue. Thus it is now critical that notifications arrive in order.

The virtio_blk driver has historically not worried about notification
ordering. Modify it so that the prepare and kick steps are both done
under the vq lock.

Fixes af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA

feature support")
Cc: Viktor Prutyanov <[email protected]>
Cc: [email protected]
Cc: [email protected]
---
drivers/block/virtio_blk.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-)

diff --git a/drivers/block/virtio_blk.c b/drivers/block/ virtio_blk.c
index 3efe378f1386..14d9e66bb844 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
{
struct virtio_blk *vblk = hctx->queue->queuedata; struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
- bool kick;
spin_lock_irq(&vq->lock);
- kick = virtqueue_kick_prepare(vq->vq);
+ virtqueue_kick(vq->vq);
spin_unlock_irq(&vq->lock);
-
- if (kick)
- virtqueue_notify(vq->vq);
}

I would assume this will be a performance nightmare for normal IO.

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

I don't have any sort of setup where I can try it but I would appreciate it if someone else could.

Hmm. Not good, notify can be very slow, holding a lock is a bad idea. Basically, virtqueue_notify must work ouside of locks, this means af8ececda185 is broken and we did not notice.

Let's fix it please.

Quoted 7 lines
Quoted 2 lines

With so many broken kernels already in the wild, I think disabling F_NOTIFICATION_DATA for virtio-blk would be a reasonable solution.

Some devices might fail feature negotiation then.
I am not sure they are broken, devices might simply be able to handle out of order values.

A driver which does not support F_NOTIFICATION_DATA should just clear that bit. Surely devices which support it would also support not enabling it? Otherwise pre-6.4 kernels wouldn't work at all.

Quoted 25 lines
Quoted 11 lines

Try some kind of compare and swap scheme where we detect that index was updated since? Will allow skipping a notification, too.

Do you have an idea of how this might be done? Anything I've come up with involves a lock.

Would it be doable to have a lock for the vq management stuff and a second one to post notifications?

and only for when F_NOTIFICATION_DATA is set. not terrible ok I think.

Quoted 10 lines

AMD guys, can't device survive with reordered notifications? Basically just drop a notification if you see index going back?

This is the driver lying to us about the state of the queue; it's not going to be possible for us to work around it in hardware. For starters, how would we detect queue wrap around?

Quoted 6 lines
Quoted 2 lines

Thank you,
Andrew

The index is a running value for split, for wrap arounds, there is a special bit for that. No?

This is a hardware block used for many different interfaces and devices. When the notification write comes through, the doorbell block updates the queue state and schedules the queue for work. If a second notification comes in and overwrites that update before the queue is able to run (going backwards but not wrapping), we'll have no way of detecting it.

-Andrew

#9 lore raw
Quoted 126 lines · Andrew
Quoted 88 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 80 lines · Andrew

On Jan 9, 2025, at 8:42 AM, Michael S. Tsirkin <[email protected]> wrote:

On Thu, Jan 09, 2025 at 01:01:20PM +0100, Christian Borntraeger wrote:

Am 07.01.25 um 19:25 schrieb Andrew Boyer:

Commit af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature
support") added notification data support to the core virtio driver code. When this feature is enabled, the notification includes the updated producer index for the queue. Thus it is now critical that notifications arrive in order.

The virtio_blk driver has historically not worried about notification
ordering. Modify it so that the prepare and kick steps are both done
under the vq lock.

Fixes af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA

feature support")
Cc: Viktor Prutyanov <[email protected]>
Cc: [email protected]
Cc: [email protected]
---
drivers/block/virtio_blk.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-)

diff --git a/drivers/block/virtio_blk.c b/drivers/block/ virtio_blk.c
index 3efe378f1386..14d9e66bb844 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
{
struct virtio_blk *vblk = hctx->queue->queuedata; struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
- bool kick;
spin_lock_irq(&vq->lock);
- kick = virtqueue_kick_prepare(vq->vq);
+ virtqueue_kick(vq->vq);
spin_unlock_irq(&vq->lock);
-
- if (kick)
- virtqueue_notify(vq->vq);
}

I would assume this will be a performance nightmare for normal IO.

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

I don't have any sort of setup where I can try it but I would appreciate it if someone else could.

Hmm. Not good, notify can be very slow, holding a lock is a bad idea. Basically, virtqueue_notify must work ouside of locks, this means af8ececda185 is broken and we did not notice.

Let's fix it please.

Quoted 7 lines
Quoted 2 lines

With so many broken kernels already in the wild, I think disabling F_NOTIFICATION_DATA for virtio-blk would be a reasonable solution.

Some devices might fail feature negotiation then.
I am not sure they are broken, devices might simply be able to handle out of order values.

A driver which does not support F_NOTIFICATION_DATA should just clear that bit. Surely devices which support it would also support not enabling it? Otherwise pre-6.4 kernels wouldn't work at all.

Quoted 28 lines
Quoted 11 lines

Try some kind of compare and swap scheme where we detect that index was updated since? Will allow skipping a notification, too.

Do you have an idea of how this might be done? Anything I've come up with involves a lock.

Would it be doable to have a lock for the vq management stuff and a second one to post notifications?

and only for when F_NOTIFICATION_DATA is set. not terrible ok I think.

Quoted 12 lines

AMD guys, can't device survive with reordered notifications? Basically just drop a notification if you see index going back?

This is the driver lying to us about the state of the queue; it's not going to be possible for us to work around it in hardware. For starters, how would we detect queue wrap around?

Thank you,
Andrew

Quoted 7 lines
Quoted 3 lines

The index is a running value for split, for wrap arounds, there is a special bit for that. No?

This is a hardware block used for many different interfaces and devices. When the notification write comes through, the doorbell block updates the queue state and schedules the queue for work. If a second notification comes in and overwrites that update before the queue is able to run (going backwards but not wrapping), we'll have no way of detecting it.

-Andrew

Do you not have programmable hardware that can compare current queue state and the doorbell *before* overwriting it?

#10 lore raw

(repost) Re: [PATCH] virtio_blk: always post notifications under the lock

In reply to a message not in the archive
Quoted 77 lines · Andrew

On Jan 9, 2025, at 8:42 AM, Michael S. Tsirkin <[email protected]> wrote:

On Thu, Jan 09, 2025 at 01:01:20PM +0100, Christian Borntraeger wrote:

Am 07.01.25 um 19:25 schrieb Andrew Boyer:

Commit af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA feature
support") added notification data support to the core virtio driver code. When this feature is enabled, the notification includes the updated producer index for the queue. Thus it is now critical that notifications arrive in order.

The virtio_blk driver has historically not worried about notification
ordering. Modify it so that the prepare and kick steps are both done
under the vq lock.

Fixes af8ececda185 ("virtio: add VIRTIO_F_NOTIFICATION_DATA

feature support")
Cc: Viktor Prutyanov <[email protected]>
Cc: [email protected]
Cc: [email protected]
---
drivers/block/virtio_blk.c | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-)

diff --git a/drivers/block/virtio_blk.c b/drivers/block/ virtio_blk.c
index 3efe378f1386..14d9e66bb844 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
{
struct virtio_blk *vblk = hctx->queue->queuedata; struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
- bool kick;
spin_lock_irq(&vq->lock);
- kick = virtqueue_kick_prepare(vq->vq);
+ virtqueue_kick(vq->vq);
spin_unlock_irq(&vq->lock);
-
- if (kick)
- virtqueue_notify(vq->vq);
}

I would assume this will be a performance nightmare for normal IO.

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

I don't have any sort of setup where I can try it but I would appreciate it if someone else could.

Hmm. Not good, notify can be very slow, holding a lock is a bad idea. Basically, virtqueue_notify must work ouside of locks, this means af8ececda185 is broken and we did not notice.

Quoted 5 lines

Let's fix it please.

With so many broken kernels already in the wild, I think disabling F_NOTIFICATION_DATA for virtio-blk would be a reasonable solution.

Some devices might fail feature negotiation then.
I am not sure they are broken, devices might simply be able to handle out of order values.

Quoted 10 lines

Try some kind of compare and swap scheme where we detect that index was updated since? Will allow skipping a notification, too.

Do you have an idea of how this might be done? Anything I've come up with involves a lock.

Would it be doable to have a lock for the vq management stuff and a second one to post notifications?

and only for when F_NOTIFICATION_DATA is set. not terrible ok I think.

Quoted 6 lines

AMD guys, can't device survive with reordered notifications? Basically just drop a notification if you see index going back?

Quoted 6 lines

This is the driver lying to us about the state of the queue; it's not going to be possible for us to work around it in hardware. For starters, how would we detect queue wrap around?

Thank you,
Andrew

The index is a running value for split, for wrap arounds, there is a special bit for that. No?

Quoted 6 lines

    --
    MST
#11 lore raw
In reply to a message not in the archive

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
drivers/block/virtio_blk.c +0 -2
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
  {
    struct virtio_blk *vblk = hctx->queue->queuedata;
    struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
    spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
Quoted 9 lines
Quoted 8 lines

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 386

Quoted 6 lines
drivers/block/virtio_blk.c +1 -3
+   virtqueue_kick(vq->vq);
    spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
  }

I would assume this will be a performance nightmare for normal IO.

Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

#12 lore raw
Quoted 29 lines · Christian Borntraeger

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
  struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
  struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
  spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
  spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
 }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

If holding a lock at all is unworkable, do you have a suggestion for how we might fix the correctness issue? Because this is definitely a correctness issue.

I don't see anything in the spec about requiring support for out-of-order notifications.

-Andrew

#13 lore raw
Quoted 37 lines · Andrew
Quoted 29 lines · Christian Borntraeger

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
  struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
  struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
  spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
  spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
 }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

Quoted 6 lines

If holding a lock at all is unworkable, do you have a suggestion for how we might fix the correctness issue? Because this is definitely a correctness issue.

I don't see anything in the spec about requiring support for out-of-order notifications.

-Andrew

Yes we didn't think it through in the spec :(
So driver made one assumption, for performance,
and the device made another one, for simplicity.

The data needed for the device to solve it, is there in the notification. We can make the safe choice with the lock, and add a feature flag for devices who can handle out of order, or make the current choice and add a feature flag for devices that need a lock.

Let's continue the discussion in another thread whether a hardware workaround is possible.

Signature · 1 line
MST
#14 lore raw
In reply to Christian Borntraeger
Quoted 22 lines · Christian Borntraeger

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
  {
    struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
    struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
    spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
    spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
  }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

The issue is with hardware that wants a copy of an index sent to it in a notification. Now, you have a race:

thread 1:

	index = 1
		-> 			-> send 1 to hardware

thread 2:

	index = 2
		-> send 2 to hardware

the spec unfortunately does not say whether that is legal.

As far as I could tell, the device can easily use the wrap counter inside the notification to detect this and simply discard the "1" notification.

If not, I'd like to understand why.

Thanks!

Signature · 1 line
MST
#15 lore raw
Quoted 52 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 22 lines · Christian Borntraeger

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
  struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
  struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
  spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
  spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
 }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

The issue is with hardware that wants a copy of an index sent to it in a notification. Now, you have a race:

thread 1:

index = 1
-> -> send 1 to hardware

thread 2:

index = 2
-> send 2 to hardware

Quoted 8 lines

the spec unfortunately does not say whether that is legal.

As far as I could tell, the device can easily use the wrap counter inside the notification to detect this and simply discard the "1" notification.

If not, I'd like to understand why.

"Easily"?

This is a hardware doorbell block used for many different interfaces and devices. When the notification write comes through, the doorbell block updates the queue state and schedules the queue for work. If a second notification comes in and overwrites that update before the queue is able to run (going backwards but not wrapping), we'll have no way of detecting it.

-Andrew

#16 lore raw
Quoted 59 lines · Andrew
Quoted 59 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 22 lines · Christian Borntraeger

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
  struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
  struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
  spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
  spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
 }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

The issue is with hardware that wants a copy of an index sent to it in a notification. Now, you have a race:

thread 1:

index = 1
-> -> send 1 to hardware

thread 2:

index = 2
-> send 2 to hardware

the spec unfortunately does not say whether that is legal.

As far as I could tell, the device can easily use the wrap counter inside the notification to detect this and simply discard the "1" notification.

Quoted 8 lines
Quoted 1 line

If not, I'd like to understand why.

"Easily"?

This is a hardware doorbell block used for many different interfaces and devices. When the notification write comes through, the doorbell block updates the queue state and schedules the queue for work. If a second notification comes in and overwrites that update before the queue is able to run (going backwards but not wrapping), we'll have no way of detecting it.

-Andrew

Does not this work?

notification includes two values:

1. offset
2. wrap_counter

if ((offset2 < offset1 && wrap_counter2 == wrap_counter1) ||

     offset1 > offset1 && wrap_counter2 != wrap_counter1)) {
	printf("going backwards, discard offset2");
}
#17 lore raw
Quoted 79 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 59 lines · Andrew
Quoted 59 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 22 lines · Christian Borntraeger

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
 struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
 struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
 spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
 spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
 }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

The issue is with hardware that wants a copy of an index sent to it in a notification. Now, you have a race:

thread 1:

index = 1
-> -> send 1 to hardware

thread 2:

index = 2
-> send 2 to hardware

the spec unfortunately does not say whether that is legal.

As far as I could tell, the device can easily use the wrap counter inside the notification to detect this and simply discard the "1" notification.

Quoted 8 lines
Quoted 1 line

If not, I'd like to understand why.

"Easily"?

This is a hardware doorbell block used for many different interfaces and devices. When the notification write comes through, the doorbell block updates the queue state and schedules the queue for work. If a second notification comes in and overwrites that update before the queue is able to run (going backwards but not wrapping), we'll have no way of detecting it.

-Andrew

Does not this work?

notification includes two values:

Quoted 8 lines

1. offset
2. wrap_counter

if ((offset2 < offset1 && wrap_counter2 == wrap_counter1) ||

    offset1 > offset1 && wrap_counter2 != wrap_counter1)) {
       printf("going backwards, discard offset2");
}

No, Michael, this does not work in our programmable hardware device, because it is not software.

-Andrew

#18 lore raw
Quoted 89 lines · Andrew
Quoted 80 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 59 lines · Andrew
Quoted 59 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 22 lines · Christian Borntraeger

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
 struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
 struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
 spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
 spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
 }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

The issue is with hardware that wants a copy of an index sent to it in a notification. Now, you have a race:

thread 1:

index = 1
-> -> send 1 to hardware

thread 2:

index = 2
-> send 2 to hardware

the spec unfortunately does not say whether that is legal.

As far as I could tell, the device can easily use the wrap counter inside the notification to detect this and simply discard the "1" notification.

Quoted 8 lines
Quoted 1 line

If not, I'd like to understand why.

"Easily"?

This is a hardware doorbell block used for many different interfaces and devices. When the notification write comes through, the doorbell block updates the queue state and schedules the queue for work. If a second notification comes in and overwrites that update before the queue is able to run (going backwards but not wrapping), we'll have no way of detecting it.

-Andrew

Does not this work?

notification includes two values:

Quoted 8 lines

1. offset
2. wrap_counter

if ((offset2 < offset1 && wrap_counter2 == wrap_counter1) ||

    offset1 > offset1 && wrap_counter2 != wrap_counter1)) {
       printf("going backwards, discard offset2");
}
Quoted 4 lines

No, Michael, this does not work in our programmable hardware device, because it is not software.

-Andrew

Should we send a patch to disable this feature and cc stable first then consider how to fix this?

Thanks

#19 lore raw
Quoted 92 lines · Jason Wang
Quoted 91 lines · Andrew
Quoted 80 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 59 lines · Andrew
Quoted 59 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 22 lines · Christian Borntraeger

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
 struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
 struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
 spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
 spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
 }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

The issue is with hardware that wants a copy of an index sent to it in a notification. Now, you have a race:

thread 1:

index = 1
-> -> send 1 to hardware

thread 2:

index = 2
-> send 2 to hardware

the spec unfortunately does not say whether that is legal.

As far as I could tell, the device can easily use the wrap counter inside the notification to detect this and simply discard the "1" notification.

Quoted 8 lines
Quoted 1 line

If not, I'd like to understand why.

"Easily"?

This is a hardware doorbell block used for many different interfaces and devices. When the notification write comes through, the doorbell block updates the queue state and schedules the queue for work. If a second notification comes in and overwrites that update before the queue is able to run (going backwards but not wrapping), we'll have no way of detecting it.

-Andrew

Does not this work?

notification includes two values:

Quoted 8 lines

1. offset
2. wrap_counter

if ((offset2 < offset1 && wrap_counter2 == wrap_counter1) ||

    offset1 > offset1 && wrap_counter2 != wrap_counter1)) {
       printf("going backwards, discard offset2");
}

No, Michael, this does not work in our programmable hardware device, because it is not software.

Quoted 7 lines
Quoted 2 lines

-Andrew

Should we send a patch to disable this feature and cc stable first then consider how to fix this?

Thanks

Given this has been out there for a long time, I do not see how this will contribute anything useful.

Signature · 1 line
MST
#20 lore raw
In reply to Boyer, Andrew
Quoted 88 lines · Andrew
Quoted 79 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 59 lines · Andrew
Quoted 59 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 22 lines · Christian Borntraeger

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
 struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
 struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
 spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
 spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
 }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

The issue is with hardware that wants a copy of an index sent to it in a notification. Now, you have a race:

thread 1:

index = 1
-> -> send 1 to hardware

thread 2:

index = 2
-> send 2 to hardware

the spec unfortunately does not say whether that is legal.

As far as I could tell, the device can easily use the wrap counter inside the notification to detect this and simply discard the "1" notification.

Quoted 8 lines
Quoted 1 line

If not, I'd like to understand why.

"Easily"?

This is a hardware doorbell block used for many different interfaces and devices. When the notification write comes through, the doorbell block updates the queue state and schedules the queue for work. If a second notification comes in and overwrites that update before the queue is able to run (going backwards but not wrapping), we'll have no way of detecting it.

-Andrew

Does not this work?

notification includes two values:

Quoted 8 lines

1. offset
2. wrap_counter

if ((offset2 < offset1 && wrap_counter2 == wrap_counter1) ||

    offset1 > offset1 && wrap_counter2 != wrap_counter1)) {
       printf("going backwards, discard offset2");
}
Quoted 4 lines

No, Michael, this does not work in our programmable hardware device, because it is not software.

-Andrew

Of course. The hardware equivalent.

#21 lore raw
In reply to Michael S. Tsirkin

Am 22.01.25 um 23:07 schrieb Michael S. Tsirkin:

Quoted 50 lines · Christian Borntraeger
Quoted 22 lines

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
  {
    struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
    struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
    spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
    spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
  }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

The issue is with hardware that wants a copy of an index sent to it in a notification. Now, you have a race:

thread 1:

	index = 1
		-> 			-> send 1 to hardware

thread 2:

	index = 2
		-> send 2 to hardware
Quoted 8 lines

the spec unfortunately does not say whether that is legal.

As far as I could tell, the device can easily use the wrap counter inside the notification to detect this and simply discard the "1" notification.

If not, I'd like to understand why.

Yes I agree with you here.
I just want to emphasize that holding a lock during notify is not a workable solution.

#22 lore raw
Quoted 58 lines · Christian Borntraeger

Am 22.01.25 um 23:07 schrieb Michael S. Tsirkin:

Quoted 56 lines · Christian Borntraeger
Quoted 22 lines

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
  {
    struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
    struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
    spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
    spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
  }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

The issue is with hardware that wants a copy of an index sent to it in a notification. Now, you have a race:

thread 1:

	index = 1
		-> 			-> send 1 to hardware

thread 2:

	index = 2
		-> send 2 to hardware

the spec unfortunately does not say whether that is legal.

As far as I could tell, the device can easily use the wrap counter inside the notification to detect this and simply discard the "1" notification.

Quoted 4 lines
Quoted 1 line

If not, I'd like to understand why.

Yes I agree with you here.
I just want to emphasize that holding a lock during notify is not a workable solution.

Andrew, what do you say?

Signature · 1 line
MST
#23 lore raw
Quoted 67 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 60 lines · Christian Borntraeger

Am 22.01.25 um 23:07 schrieb Michael S. Tsirkin:

Quoted 57 lines · Christian Borntraeger
Quoted 22 lines

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
   struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
   struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
   spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
   spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
 }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

The issue is with hardware that wants a copy of an index sent to it in a notification. Now, you have a race:

thread 1:

index = 1

           ->                      -> send 1 to hardware

thread 2:

index = 2

           -> send 2 to hardware

the spec unfortunately does not say whether that is legal.

As far as I could tell, the device can easily use the wrap counter inside the notification to detect this and simply discard the "1" notification.

If not, I'd like to understand why.

Quoted 9 lines
Quoted 2 lines

Yes I agree with you here.
I just want to emphasize that holding a lock during notify is not a workable solution.

Andrew, what do you say?

--
MST

"can *easily* use" -> No.

Our doorbell hardware is configurable, but not infinitely programmable. None of the suggested workarounds are feasible for hardware in the face of incorrect driver behavior. We have marked the feature as broken in the linux kernel driver and moved on.

Thanks,
Andrew

#24 lore raw
Quoted 77 lines · Andrew
Quoted 67 lines · Michael S. Tsirkin

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 60 lines · Christian Borntraeger

Am 22.01.25 um 23:07 schrieb Michael S. Tsirkin:

Quoted 57 lines · Christian Borntraeger
Quoted 22 lines

Am 22.01.25 um 15:44 schrieb Boyer, Andrew:
[...]

Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
drivers/block/virtio_blk.c +0 -0
@@ -379,14 +379,10 @@ static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx)
 {
   struct virtio_blk *vblk = hctx->queue->queuedata;

drivers/block/virtio_blk.c static void virtio_commit_rqs(struct blk_mq_hw_ctx *hctx) · line 385

Quoted 10 lines
drivers/block/virtio_blk.c +1 -5
   struct virtio_blk_vq *vq = &vblk->vqs[hctx->queue_num];
-   bool kick;
   spin_lock_irq(&vq->lock);
-   kick = virtqueue_kick_prepare(vq->vq);
+   virtqueue_kick(vq->vq);
   spin_unlock_irq(&vq->lock);
-
-   if (kick)
-           virtqueue_notify(vq->vq);
 }

I would assume this will be a performance nightmare for normal IO.

Quoted 7 lines
Quoted 4 lines

Hello Michael and Christian and Jason,
Thank you for taking a look.

Is the performance concern that the vmexit might lead to the underlying virtual storage stack doing the work immediately? Any other job posting to the same queue would presumably be blocked on a vmexit when it goes to attempt its own notification. That would be almost the same as having the other job block on a lock during the operation, although I guess if you are skipping notifications somehow it would look different.

The performance concern is that you hold a lock and then exit. Exits are expensive and can schedule so you will increase the lock holding time significantly. This is begging for lock holder preemption. Really, dont do it.

The issue is with hardware that wants a copy of an index sent to it in a notification. Now, you have a race:

thread 1:

index = 1

           ->                      -> send 1 to hardware

thread 2:

index = 2

           -> send 2 to hardware

the spec unfortunately does not say whether that is legal.

As far as I could tell, the device can easily use the wrap counter inside the notification to detect this and simply discard the "1" notification.

If not, I'd like to understand why.

Quoted 9 lines
Quoted 2 lines

Yes I agree with you here.
I just want to emphasize that holding a lock during notify is not a workable solution.

Andrew, what do you say?

--
MST

Quoted 7 lines

"can *easily* use" -> No.

Our doorbell hardware is configurable, but not infinitely programmable. None of the suggested workarounds are feasible for hardware in the face of incorrect driver behavior. We have marked the feature as broken in the linux kernel driver and moved on.

Thanks,
Andrew

Thanks Andrew, I understand.
My question would be, is the feature worth it for you?

I see two way to move forward
1.- amend spec to say notifications can be reordered
2.- amend spec to say they can not be reordered and add a new feature bit
saying they can be
add a new lock for driver to hold in case the feature is off

1 is less work but if 2 gives you a big performance advantage, I can do it.

Signature · 1 line
MST