Skip to contentSkip to conversation

vhost: cache avail index in vhost_enable_notify()

RFC PATCH In review Stefano Garzarella 3 participants 6 messages Jan 13, 2022 – Jan 14, 2022

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 vhost: cache avail index in vhost_enable_notify()

In vhost_enable_notify() we enable the notifications and we read the avail index to check if new buffers have become available in the meantime. In this case, the device would go to re-read avail index to access the descriptor.

As we already do in other place, we can cache the value in avail_idx and compare it with last_avail_idx to check if there are new buffers available.

---
 drivers/vhost/vhost.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/vhost/vhost.c +2 -1
index 59edb5a1ffe2..07363dff559e 100644
@@ -2543,8 +2543,9 @@ bool vhost_enable_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq)
 		       &vq->avail->idx, r);
 		return false;
 	}
+	vq->avail_idx = vhost16_to_cpu(vq, avail_idx);
 
-	return vhost16_to_cpu(vq, avail_idx) != vq->avail_idx;
+	return vq->avail_idx != vq->last_avail_idx;
 }
 EXPORT_SYMBOL_GPL(vhost_enable_notify);
 
Signature · 1 line
2.31.1
#1 lore raw
Quoted 10 lines · Stefano Garzarella

In vhost_enable_notify() we enable the notifications and we read the avail index to check if new buffers have become available in the meantime. In this case, the device would go to re-read avail index to access the descriptor.

As we already do in other place, we can cache the value in avail_idx and compare it with last_avail_idx to check if there are new buffers available.

I guess we can ... but what's the point?

Quoted 21 lines
---
 drivers/vhost/vhost.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/vhost/vhost.c +2 -1
index 59edb5a1ffe2..07363dff559e 100644
@@ -2543,8 +2543,9 @@ bool vhost_enable_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq)
 		       &vq->avail->idx, r);
 		return false;
 	}
+	vq->avail_idx = vhost16_to_cpu(vq, avail_idx);
 
-	return vhost16_to_cpu(vq, avail_idx) != vq->avail_idx;
+	return vq->avail_idx != vq->last_avail_idx;
 }
 EXPORT_SYMBOL_GPL(vhost_enable_notify);
 
Signature · 1 line
2.31.1
#2 lore raw
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Stefano Garzarella

In vhost_enable_notify() we enable the notifications and we read the avail index to check if new buffers have become available in the meantime. In this case, the device would go to re-read avail index to access the descriptor.

Quoted 8 lines
Quoted 5 lines

As we already do in other place, we can cache the value in avail_idx and compare it with last_avail_idx to check if there are new buffers available.

I guess we can ... but what's the point?

That without this patch if avail index is new, then device when will call vhost_get_vq_desc() will find old value in cache and will read it again.

With this patch we also do the same path and update the cache every time we read avail index.

I marked it RFC because I don't know if it's worth it :-)

Stefano

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

#3 lore raw
Quoted 19 lines · Stefano Garzarella
Quoted 6 lines · Michael S. Tsirkin
Quoted 5 lines · Stefano Garzarella

In vhost_enable_notify() we enable the notifications and we read the avail index to check if new buffers have become available in the meantime. In this case, the device would go to re-read avail index to access the descriptor.

Quoted 8 lines
Quoted 5 lines

As we already do in other place, we can cache the value in avail_idx and compare it with last_avail_idx to check if there are new buffers available.

I guess we can ... but what's the point?

That without this patch if avail index is new, then device when will call vhost_get_vq_desc() will find old value in cache and will read it again.

Quoted 6 lines

With this patch we also do the same path and update the cache every time we read avail index.

I marked it RFC because I don't know if it's worth it :-)

Stefano

Pls include info like this in commit log. Thanks!

Signature · 1 line
MST
#4 lore raw
In reply to Stefano Garzarella
Quoted 11 lines · Stefano Garzarella

In vhost_enable_notify() we enable the notifications and we read the avail index to check if new buffers have become available in the meantime. In this case, the device would go to re-read avail index to access the descriptor.

As we already do in other place, we can cache the value in avail_idx and compare it with last_avail_idx to check if there are new buffers available.

Patch looks fine but I guess we won't get performance improvement since it doesn't save any userspace/VM memory access?

Thanks

Quoted 22 lines
---
 drivers/vhost/vhost.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/vhost/vhost.c +2 -2
index 59edb5a1ffe2..07363dff559e 100644
@@ -2543,8 +2543,9 @@ bool vhost_enable_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq)
                       &vq->avail->idx, r);
                return false;
        }
+       vq->avail_idx = vhost16_to_cpu(vq, avail_idx);
 
-       return vhost16_to_cpu(vq, avail_idx) != vq->avail_idx;
+       return vq->avail_idx != vq->last_avail_idx;
 }
 EXPORT_SYMBOL_GPL(vhost_enable_notify);
 
--

2.31.1

#5 lore raw
Quoted 7 lines · Jason Wang
Quoted 6 lines · Stefano Garzarella

In vhost_enable_notify() we enable the notifications and we read the avail index to check if new buffers have become available in the meantime. In this case, the device would go to re-read avail index to access the descriptor.

Quoted 8 lines
Quoted 5 lines

As we already do in other place, we can cache the value in avail_idx and compare it with last_avail_idx to check if there are new buffers available.

Patch looks fine but I guess we won't get performance improvement since it doesn't save any userspace/VM memory access?

It should save the memory access when vhost_enable_notify() find something new in the VQ, so in this path:

vhost_enable_notify() <- VM memory access for avail index
== true

vhost_disable_notify()
...

vhost_get_vq_desc() <- VM memory access for avail index
with the patch applied, this access is avoided since avail index is cached

In any case, I don't expect this to be a very common path, indeed we usually use unlikely() for this path:

    if (unlikely(vhost_enable_notify(dev, vq))) {
        vhost_disable_notify(dev, vq);
        continue;
    }

So I don't expect a significant performance increase.

v1 coming with a better commit description.

Thanks,
Stefano