vhost/vsock: fix device IOTLB feature lifecycle
Subsystems (6)
v3 cover vhost/vsock: fix device IOTLB feature lifecycle
vhost-vsock leaves the device IOTLB attached when userspace clears VIRTIO_F_ACCESS_PLATFORM. It also replaces an existing IOTLB with an empty one whenever a later feature update keeps ACCESS_PLATFORM enabled.
Patch 1 detaches and frees the old IOTLB when ACCESS_PLATFORM is cleared. Each virtqueue drops its IOTLB pointer and metadata cache while holding its own mutex. The old IOTLB remains alive until all virtqueues have dropped their references. Queued IOTLB miss messages are then cleared and blocked readers are woken.
Patch 2 keeps the current IOTLB when ACCESS_PLATFORM remains enabled, so a runtime logging update does not discard valid translations.
Changes since v2:
- update each virtqueue under its own mutex instead of locking all virtqueues at once, as suggested by Michael S. Tsirkin
- keep the old IOTLB alive until all per-virtqueue references are gone
- clarify that the teardown drops queued IOTLB miss messages
Changes since v1:
- discard the existing IOTLB instead of returning -EBUSY
- split IOTLB teardown and preservation into separate patches
- clear queued IOTLB miss messages and wake blocked readers
Jia Jia (2):
vhost/vsock: discard IOTLB when ACCESS_PLATFORM is cleared vhost/vsock: keep IOTLB across feature updates
drivers/vhost/vsock.c | 42 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 6 deletions(-)
Signature · 1 line
2.34.1
PATCH v3 1/2 vhost/vsock: discard IOTLB when ACCESS_PLATFORM is cleared
vhost_vsock_set_features() leaves the device IOTLB attached when userspace clears VIRTIO_F_ACCESS_PLATFORM. Descriptors can therefore continue to use translations installed before the feature change, including HVAs made stale by a later memory table update.
Detach the device IOTLB before acknowledging a feature mask without ACCESS_PLATFORM. Serialize each virtqueue handoff with its own mutex while clearing its IOTLB pointer, resetting its metadata cache, and updating its acknowledged features. Keep the old IOTLB alive until all virtqueues have dropped their references, then free it.
Also drop queued IOTLB miss messages and wake readers now that the device no longer accepts IOTLB updates.
--- drivers/vhost/vsock.c | 39 ++++++++++++++++++++++++++++++++++----- 1 file changed, 34 insertions(+), 5 deletions(-)
drivers/vhost/vsock.c +34 -5
@@ -851,6 +851,30 @@ static int vhost_vsock_set_cid(struct vhost_vsock *vsock, u64 guest_cid)return 0 ; } +/* Caller must hold the device mutex. */ +static void vhost_vsock_clear_iotlb (struct vhost_vsock *vsock,u64 features) +{ +struct vhost_iotlb *iotlb; +struct vhost_virtqueue *vq; +int i; + + iotlb = vsock->dev.iotlb; + vsock->dev.iotlb =NULL ; + +for (i =0 ; i <ARRAY_SIZE (vsock->vqs); i++) { +mutex_lock (&vsock->vqs[i].mutex); + vq = &vsock->vqs[i]; + vq->iotlb =NULL ; +memset (vq->meta_iotlb,0 ,sizeof (vq->meta_iotlb)); + vq->acked_features = features; +mutex_unlock (&vsock->vqs[i].mutex); + } + +vhost_clear_msg (&vsock->dev); +vhost_iotlb_free (iotlb); +wake_up_interruptible_poll (&vsock->dev.wait,EPOLLIN |EPOLLRDNORM ); +} +static int vhost_vsock_set_features (struct vhost_vsock *vsock,u64 features) {struct vhost_virtqueue *vq;
@@ -872,11 +896,16 @@ static int vhost_vsock_set_features(struct vhost_vsock *vsock, u64 features) vsock->seqpacket_allow = features & (1ULL <<VIRTIO_VSOCK_F_SEQPACKET ); -for (i =0 ; i <ARRAY_SIZE (vsock->vqs); i++) { - vq = &vsock->vqs[i]; -mutex_lock (&vq->mutex); - vq->acked_features = features; -mutex_unlock (&vq->mutex); +if (!(features & (1ULL <<VIRTIO_F_ACCESS_PLATFORM )) && + vsock->dev.iotlb) { +vhost_vsock_clear_iotlb (vsock, features); + }else { +for (i =0 ; i <ARRAY_SIZE (vsock->vqs); i++) { + vq = &vsock->vqs[i]; +mutex_lock (&vq->mutex); + vq->acked_features = features; +mutex_unlock (&vq->mutex); + } }mutex_unlock (&vsock->dev.mutex);return 0 ;
Signature · 1 line
2.34.1
Stefano Garzarella
maintainerRe: [PATCH v3 1/2] vhost/vsock: discard IOTLB when ACCESS_PLATFORM is cleared
Quoted 32 lines · Jia Jia
vhost_vsock_set_features()leaves the device IOTLB attached when userspace clearsVIRTIO_F_ACCESS_PLATFORM. Descriptors can therefore continue to use translations installed before the feature change, including HVAs made stale by a later memory table update.Detach the device IOTLB before acknowledging a feature mask without
ACCESS_PLATFORM. Serialize each virtqueue handoff with its own mutex while clearing its IOTLB pointer, resetting its metadata cache, and updating its acknowledged features. Keep the old IOTLB alive until all virtqueues have dropped their references, then free it.Also drop queued IOTLB miss messages and wake readers now that the device no longer accepts IOTLB updates.
Fixes e13a6915a03f ("vhost/vsock: add IOTLB API support")Suggested-by Michael S. Tsirkin <[email protected]>Signed-off-by Jia Jia <[email protected]>---
drivers/vhost/vsock.c| 39 ++++++++++++++++++++++++++++++++++-----1 file changed, 34 insertions(+), 5 deletions(-)drivers/vhost/vsock.c +3
-0@@ -851,6 +851,30 @@ static int vhost_vsock_set_cid(struct vhost_vsock *vsock, u64 guest_cid)return 0 ; } +/* Caller must hold the device mutex. */ +static void vhost_vsock_clear_iotlb (struct vhost_vsock *vsock,u64 features) +{
drivers/vhost/vsock.c static int vhost_vsock_set_cid(struct vhost_vsock *vsock, u64 guest_cid) · line 866
Quoted 10 lines
drivers/vhost/vsock.c +10
-0+struct vhost_iotlb *iotlb; +struct vhost_virtqueue *vq; +int i; + + iotlb = vsock->dev.iotlb; + vsock->dev.iotlb =NULL ; + +for (i =0 ; i <ARRAY_SIZE (vsock->vqs); i++) { +mutex_lock (&vsock->vqs[i].mutex); + vq = &vsock->vqs[i];
You can assing vq before the mutex_lock() and use it there too (and in mutex_unlock()), as we do in all other places in this file.
drivers/vhost/vsock.c static int vhost_vsock_set_cid(struct vhost_vsock *vsock, u64 guest_cid) · line 876
Quoted 10 lines
drivers/vhost/vsock.c +10
-0+ vq->iotlb =NULL ; +memset (vq->meta_iotlb,0 ,sizeof (vq->meta_iotlb)); + vq->acked_features = features; +mutex_unlock (&vsock->vqs[i].mutex); + } + +vhost_clear_msg (&vsock->dev); +vhost_iotlb_free (iotlb); +wake_up_interruptible_poll (&vsock->dev.wait,EPOLLIN |EPOLLRDNORM ); +}
I don't see anything vsock specific here. Would it be better to move this to vhost.c and reuse some of the functions we have there?
I mean something like this (untested and may be incomplete):
void vhost_clear_device_iotlb(struct vhost_dev *d)
{
struct vhost_iotlb *iotlb;
int i;
iotlb = d->iotlb;
d->iotlb = NULL;
for (i = 0; i < d->nvqs; ++i) {
struct vhost_virtqueue *vq = d->vqs[i];
mutex_lock(&vq->mutex);
vq->iotlb = NULL;
__vhost_vq_meta_reset(vq);
mutex_unlock(&vq->mutex);
}
vhost_clear_msg(d);
vhost_iotlb_free(iotlb);
wake_up_interruptible_poll(&d->wait, EPOLLIN | EPOLLRDNORM);
}EXPORT_SYMBOL_GPL(vhost_clear_device_iotlb);
Quoted 13 lines
drivers/vhost/vsock.c +1
-5+static int vhost_vsock_set_features (struct vhost_vsock *vsock,u64 features) {struct vhost_virtqueue *vq;@@ -872,11 +896,16 @@ static int vhost_vsock_set_features(struct vhost_vsock *vsock, u64 features) vsock->seqpacket_allow = features & (1ULL <<VIRTIO_VSOCK_F_SEQPACKET ); -for (i =0 ; i <ARRAY_SIZE (vsock->vqs); i++) { - vq = &vsock->vqs[i]; -mutex_lock (&vq->mutex); - vq->acked_features = features; -mutex_unlock (&vq->mutex);
drivers/vhost/vsock.c static int vhost_vsock_set_features(struct vhost_vsock *vsock, u64 features) · line 908
Quoted 10 lines
drivers/vhost/vsock.c +10
-0+if (!(features & (1ULL <<VIRTIO_F_ACCESS_PLATFORM )) && + vsock->dev.iotlb) { +vhost_vsock_clear_iotlb (vsock, features); + }else { +for (i =0 ; i <ARRAY_SIZE (vsock->vqs); i++) { + vq = &vsock->vqs[i]; +mutex_lock (&vq->mutex); + vq->acked_features = features; +mutex_unlock (&vq->mutex); + }
TBH I don't like this mix.
Why assigning acked_features inside vhost_vsock_clear_iotlb()?
IMO it's confusing. I see that you're saving another loop around the VQs, but this code is not easy to understand IMO.
I think we have 2 options:
1. leave the loop for acked_features and don't set it in vhost_vsock_clear_iotlb() (less code touched by this patch). This is also what to do if we move the clear_iotlb() function in vhost.c.
2. change the code to have a single for loop with if block inside to reset IOTLB stuff if needed. In this case maybe better to avoid the function and move the entire code here.
I prefer 1 with clear_iotlb() in vhost.c, but I'm not fully against 2.
Thanks,
Stefano
Quoted 6 lines
drivers/vhost/vsock.c +0
-0}mutex_unlock (&vsock->dev.mutex);return 0 ;Signature · 1 line
2.34.1
Re: [PATCH v3 1/2] vhost/vsock: discard IOTLB when ACCESS_PLATFORM is cleared
Quoted 2 lines
You can assing vq before the
mutex_lock()and use it there too (and inmutex_unlock()), as we do in all other places in this file.
Sure, will do.
Quoted 26 lines
I don't see anything vsock specific here. Would it be better to move this to
vhost.cand reuse some of the functions we have there?I mean something like this (untested and may be incomplete):
void
vhost_clear_device_iotlb(struct vhost_dev *d){ struct vhost_iotlb *iotlb; int i; iotlb = d->iotlb; d->iotlb = NULL; for (i = 0; i < d->nvqs; ++i) { struct vhost_virtqueue *vq = d->vqs[i]; mutex_lock(&vq->mutex); vq->iotlb = NULL; __vhost_vq_meta_reset(vq); mutex_unlock(&vq->mutex); } vhost_clear_msg(d); vhost_iotlb_free(iotlb); wake_up_interruptible_poll(&d->wait, EPOLLIN | EPOLLRDNORM); }
Quoted 1 line
EXPORT_SYMBOL_GPL(vhost_clear_device_iotlb);
Thanks for the detailed feedback. There has been some evolution behind this patch. The work originally started from a vhost-scsi bug. After Stefan Hajnoczi pointed out that vhost-net and vhost-vsock might have the same underlying issue, the first version used a feature-change rejection approach. The vhost-net patch from that stage is:
https://lore.kernel.org/all/[email protected]/
Michael S. Tsirkin then suggested that simply rejecting the change was not the best way to handle the IOTLB transition, and suggested discarding the existing IOTLB instead. That led to the current vhost-vsock patch.
The vhost-net patch already preserves an existing IOTLB when ACCESS_PLATFORM remains enabled, as in patch 2 here. For clearing ACCESS_PLATFORM, the two versions currently use different policies:
vhost-net rejects the live change with -EBUSY, while this vhost-vsock series accepts the change and tears down the old IOTLB. The vhost-net patch also addresses a separate live IN_ORDER transition issue.
I initially kept the helper in vsock.c because I was unsure whether changing the common vhost code was appropriate. Looking at the code, however, the IOTLB teardown has no dependency on vsock- or net-specific state. Unless there are other concerns, I will move the helper to vhost.c in the next revision and apply the same IOTLB teardown handling to vhost-net as well. I will keep the acked_features updates in each backend's own loop, while retaining the vhost-net-specific checks for IN_ORDER and other unsafe feature changes.
Quoted 16 lines
TBH I don't like this mix.
Why assigning
acked_featuresinsidevhost_vsock_clear_iotlb()?IMO it's confusing. I see that you're saving another loop around the VQs, but this code is not easy to understand IMO.
I think we have 2 options:
1. leave the loop foracked_featuresand don't set it invhost_vsock_clear_iotlb()(less code touched by this patch). This is also what to do if we move theclear_iotlb()function invhost.c.
2. change the code to have a single for loop with if block inside to reset IOTLB stuff if needed. In this case maybe better to avoid the function and move the entire code here.
Quoted 1 line
I prefer 1 with
clear_iotlb()invhost.c, but I'm not fully against 2.
Thanks for understanding that the original intent was to avoid an extra loop. I agree that assigning acked_features in vhost_vsock_clear_iotlb() is confusing, so I'll follow option 1 in the next version.
Thanks,
Jia
PATCH v3 2/2 vhost/vsock: keep IOTLB across feature updates
In reply to Jia JiaVHOST_SET_FEATURES is also used to update logging while a device is running. When ACCESS_PLATFORM stays enabled, allocating a new empty IOTLB on every call drops valid translations and forces avoidable misses.
Initialize the device IOTLB only when one does not already exist.
--- drivers/vhost/vsock.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/vhost/vsock.c +2 -1
@@ -889,7 +889,8 @@ static int vhost_vsock_set_features(struct vhost_vsock *vsock, u64 features) goto err; } -if ((features & (1ULL <<VIRTIO_F_ACCESS_PLATFORM ))) { +if ((features & (1ULL <<VIRTIO_F_ACCESS_PLATFORM )) && + !vsock->dev.iotlb) {if (vhost_init_device_iotlb (&vsock->dev)) goto err; }
Signature · 1 line
2.34.1
Stefano Garzarella
maintainerRe: [PATCH v3 2/2] vhost/vsock: keep IOTLB across feature updates
Quoted 7 lines · Jia Jia
VHOST_SET_FEATURESis also used to update logging while a device is running. WhenACCESS_PLATFORMstays enabled, allocating a new empty IOTLB on every call drops valid translations and forces avoidable misses.Initialize the device IOTLB only when one does not already exist.
Do we need a Fixes tag here?
Quoted 10 lines
Signed-off-by Jia Jia <[email protected]>---
drivers/vhost/vsock.c| 3 ++-1 file changed, 2 insertions(+), 1 deletion(-)drivers/vhost/vsock.c +0
-0
drivers/vhost/vsock.c static int vhost_vsock_set_features(struct vhost_vsock *vsock, u64 features) · line 893
Quoted 6 lines
drivers/vhost/vsock.c +2
-1@@ -889,7 +889,8 @@ static int vhost_vsock_set_features(struct vhost_vsock *vsock, u64 features) goto err; } -if ((features & (1ULL <<VIRTIO_F_ACCESS_PLATFORM ))) { +if ((features & (1ULL <<VIRTIO_F_ACCESS_PLATFORM )) && + !vsock->dev.iotlb) {
Do we need to do the same in vhost-net too?
Thanks,
Stefano
Quoted 6 lines
drivers/vhost/vsock.c +0
-0if (vhost_init_device_iotlb (&vsock->dev)) goto err; }Signature · 1 line
2.34.1
Re: [PATCH v3 2/2] vhost/vsock: keep IOTLB across feature updates
Quoted 1 line
Do we need a Fixes tag here?
I overlooked the Fixes tag. Thanks for pointing that out. I will add:
Quoted 1 line
Do we need to do the same in vhost-net too?
Yes, vhost-net has the same IOTLB lifecycle issue. I previously posted a separate patch for vhost-net:
https://lore.kernel.org/all/[email protected]/
I will update that patch in the next version to use the common IOTLB teardown handling for the ACCESS_PLATFORM transition, while retaining the vhost-net-specific checks for IN_ORDER and other unsafe feature changes.
Thanks,
Jia