Fix expected set_vq_state behavior on vdpa_sim
Subsystems (2)
Eugenio Pérez
maintainercover Fix expected set_vq_state behavior on vdpa_sim
The use of set_vq_state is to indicate vdpa device the state of a virtqueue. In the case of split, it means the avail_idx. This is mandatory for use cases like live migration.
However, vdpa_sim reset the vq state at vdpasim_queue_ready since it calls vringh_init_iotlb.
Also, to starting from an used_idx different than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.
This series fixes both problems allowing to migrate to a vdpa_sim_net device.
Eugenio Pérez (2):
vdpa_sim: not reset state in vdpasim_queue_ready
vringh: fetch used_idx from vring at vringh_init_iotlb
drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 ++ drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 2 files changed, 25 insertions(+), 2 deletions(-)
Signature · 1 line
2.31.1
Eugenio Pérez
maintainerPATCH 1/2 vdpa_sim: not reset state in vdpasim_queue_ready
vdpasim_queue_ready calls vringh_init_iotlb, which resets split indexes. But it can be called after setting a ring base with vdpasim_set_vq_state.
Fix it by stashing them. They're still resetted in vdpasim_vq_reset.
This was discovered and tested live migrating the vdpa_sim_net device.
--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/vdpa/vdpa_sim/vdpa_sim.c +2 -0
@@ -66,6 +66,7 @@ static void vdpasim_vq_notify(struct vringh *vring)static void vdpasim_queue_ready (struct vdpasim *vdpasim,unsigned int idx) {struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; +uint16_t last_avail_idx = vq->vring.last_avail_idx; vringh_init_iotlb(&vq->vring, vdpasim->features, vq->num, false, (struct vring_desc *)(uintptr_t)vq->desc_addr,
@@ -74,6 +75,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) (struct vring_used *) (uintptr_t)vq->device_addr); + vq->vring.last_avail_idx = last_avail_idx; vq->vring.notify = vdpasim_vq_notify; }
Signature · 1 line
2.31.1
Re: [PATCH 1/2] vdpa_sim: not reset state in vdpasim_queue_ready
Quoted 28 lines · Eugenio Pérez
vdpasim_queue_readycallsvringh_init_iotlb, which resets split indexes. But it can be called after setting a ring base withvdpasim_set_vq_state.Fix it by stashing them. They're still resetted in
vdpasim_vq_reset.This was discovered and tested live migrating the
vdpa_sim_netdevice.Fixes 2c53d0f64c06 ("vdpasim: vDPA device simulator")Signed-off-by Eugenio Pérez <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 ++ 1 file changed, 2 insertions(+)drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-0@@ -66,6 +66,7 @@ static void vdpasim_vq_notify(struct vringh *vring)static void vdpasim_queue_ready (struct vdpasim *vdpasim,unsigned int idx) {struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; +uint16_t last_avail_idx = vq->vring.last_avail_idx; vringh_init_iotlb(&vq->vring, vdpasim->features, vq->num, false, (struct vring_desc *)(uintptr_t)vq->desc_addr,
drivers/vdpa/vdpa_sim/vdpa_sim.c static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) · line 78
Quoted 4 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-0@@ -74,6 +75,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) (struct vring_used *) (uintptr_t)vq->device_addr); + vq->vring.last_avail_idx = last_avail_idx;
Does this need to be serialized with the datapath?
E.g in set_vq_state() we do:
spin_lock(&vdpasim->lock);vrh->last_avail_idx = state->split.avail_index;spin_unlock(&vdpasim->lock);
Thanks
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +0
-1vq->vring.notify = vdpasim_vq_notify; } --2.31.1
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Eugenio Perez Martin
maintainerRe: [PATCH 1/2] vdpa_sim: not reset state in vdpasim_queue_ready
Quoted 37 lines · Jason Wang
Quoted 28 lines · Eugenio Pérez
vdpasim_queue_readycallsvringh_init_iotlb, which resets split indexes. But it can be called after setting a ring base withvdpasim_set_vq_state.Fix it by stashing them. They're still resetted in
vdpasim_vq_reset.This was discovered and tested live migrating the
vdpa_sim_netdevice.Fixes 2c53d0f64c06 ("vdpasim: vDPA device simulator")Signed-off-by Eugenio Pérez <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 ++ 1 file changed, 2 insertions(+)drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-0@@ -66,6 +66,7 @@ static void vdpasim_vq_notify(struct vringh *vring)static void vdpasim_queue_ready (struct vdpasim *vdpasim,unsigned int idx) {struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; +uint16_t last_avail_idx = vq->vring.last_avail_idx; vringh_init_iotlb(&vq->vring, vdpasim->features, vq->num, false, (struct vring_desc *)(uintptr_t)vq->desc_addr,drivers/vdpa/vdpa_sim/vdpa_sim.c static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) · line 78
Quoted 4 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-0@@ -74,6 +75,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) (struct vring_used *) (uintptr_t)vq->device_addr); + vq->vring.last_avail_idx = last_avail_idx;Does this need to be serialized with the datapath?
Quoted 6 lines
E.g in
set_vq_state()we do:
spin_lock(&vdpasim->lock);vrh->last_avail_idx=state->split.avail_index;spin_unlock(&vdpasim->lock);
vdpasim_queue_ready is called from vdpasim_set_vq_ready, which holds these locks.
Maybe it's too much indirection and to embed vdpasim_queue_ready in vdpasim_set_vq_ready would be clearer for the future?
Thanks!
Jason Wang
Acked-byRe: [PATCH 1/2] vdpa_sim: not reset state in vdpasim_queue_ready
Quoted 41 lines · Eugenio Perez Martin
Quoted 31 lines · Jason Wang
Quoted 30 lines · Eugenio Pérez
vdpasim_queue_readycallsvringh_init_iotlb, which resets split indexes. But it can be called after setting a ring base withvdpasim_set_vq_state.Fix it by stashing them. They're still resetted in
vdpasim_vq_reset.This was discovered and tested live migrating the
vdpa_sim_netdevice.Fixes 2c53d0f64c06 ("vdpasim: vDPA device simulator")Signed-off-by Eugenio Pérez <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 ++ 1 file changed, 2 insertions(+)drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-0@@ -66,6 +66,7 @@ static void vdpasim_vq_notify(struct vringh *vring)static void vdpasim_queue_ready (struct vdpasim *vdpasim,unsigned int idx) {struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; +uint16_t last_avail_idx = vq->vring.last_avail_idx; vringh_init_iotlb(&vq->vring, vdpasim->features, vq->num, false, (struct vring_desc *)(uintptr_t)vq->desc_addr,@@ -74,6 +75,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) (struct vring_used *) (uintptr_t)vq->device_addr);Quoted 9 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) · line 78
Quoted 1 line
drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-0+ vq->vring.last_avail_idx = last_avail_idx;Does this need to be serialized with the datapath?
E.g in
set_vq_state()we do:
spin_lock(&vdpasim->lock);vrh->last_avail_idx=state->split.avail_index;spin_unlock(&vdpasim->lock);
Quoted 5 lines
vdpasim_queue_readyis called fromvdpasim_set_vq_ready, which holds these locks.Maybe it's too much indirection and to embed
vdpasim_queue_readyinvdpasim_set_vq_readywould be clearer for the future?
Nope, I miss the caller.
Thanks
Quoted 3 lines
Thanks!
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Lei Yang
Tested-byRe: [PATCH 1/2] vdpa_sim: not reset state in vdpasim_queue_ready
The patch was tested by QE in a test environment and regression tested using vdpa_sim device with virtio_vdpa and vhost_vdpa;There are no new issues caused by this patch.
Quoted 62 lines · Jason Wang <[email protected]> 于2023年1月29日周日 13:56写道:
Quoted 41 lines · Eugenio Perez Martin
Quoted 31 lines · Jason Wang
Quoted 30 lines · Eugenio Pérez
vdpasim_queue_readycallsvringh_init_iotlb, which resets split indexes. But it can be called after setting a ring base withvdpasim_set_vq_state.Fix it by stashing them. They're still resetted in
vdpasim_vq_reset.This was discovered and tested live migrating the
vdpa_sim_netdevice.Fixes 2c53d0f64c06 ("vdpasim: vDPA device simulator")Signed-off-by Eugenio Pérez <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 2 ++ 1 file changed, 2 insertions(+)drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-0@@ -66,6 +66,7 @@ static void vdpasim_vq_notify(struct vringh *vring)static void vdpasim_queue_ready (struct vdpasim *vdpasim,unsigned int idx) {struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; +uint16_t last_avail_idx = vq->vring.last_avail_idx; vringh_init_iotlb(&vq->vring, vdpasim->features, vq->num, false, (struct vring_desc *)(uintptr_t)vq->desc_addr,@@ -74,6 +75,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) (struct vring_used *) (uintptr_t)vq->device_addr);Quoted 9 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx) · line 78
Quoted 1 line
drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-0+ vq->vring.last_avail_idx = last_avail_idx;Does this need to be serialized with the datapath?
E.g in
set_vq_state()we do:
spin_lock(&vdpasim->lock);vrh->last_avail_idx=state->split.avail_index;spin_unlock(&vdpasim->lock);Quoted 5 lines
vdpasim_queue_readyis called fromvdpasim_set_vq_ready, which holds these locks.Maybe it's too much indirection and to embed
vdpasim_queue_readyinvdpasim_set_vq_readywould be clearer for the future?Nope, I miss the caller.
Acked-by Jason Wang <[email protected]>Thanks
Quoted 3 lines
Thanks!
Eugenio Pérez
maintainerPATCH 2/2 vringh: fetch used_idx from vring at vringh_init_iotlb
In reply to Eugenio PérezStarting from an used_idx different than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.
While callers like vdpa_sim set avail_idx directly it does not set used_idx. Instead of let the caller do the assignment, fetch it from the guest at initialization like vhost-kernel do.
To perform the same at vring_kernel_init and vring_user_init is left for the future.
--- drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)
drivers/vhost/vringh.c +23 -2
@@ -1301,6 +1301,17 @@ static inline int putused_iotlb(const struct vringh *vrh,return 0 ; } +/** +* vringh_update_used_idx - fetch used idx from driver's used split vring +* @vrh: The vring. +* +* Returns -errno or 0. +*/ +static inline int vringh_update_used_idx (struct vringh *vrh) +{ +return getu16_iotlb (vrh, &vrh->last_used_idx, &vrh->vring.used->idx); +} +/** * vringh_init_iotlb - initialize a vringh for a ring with IOTLB. * @vrh: the vringh to initialize.
@@ -1319,8 +1330,18 @@ int vringh_init_iotlb(struct vringh *vrh, u64 features,struct vring_avail *avail, struct vring_used *used) { - return vringh_init_kern(vrh, features, num, weak_barriers, -desc ,avail ,used ); + int r = vringh_init_kern(vrh, features, num, weak_barriers, desc, + avail, used); + +if (r !=0 ) +return r; + +/* Consider the ring not initialized */ +if ((void *)desc == used) +return 0 ; + +return vringh_update_used_idx (vrh); + }EXPORT_SYMBOL (vringh_init_iotlb);
Signature · 1 line
2.31.1
Re: [PATCH 2/2] vringh: fetch used_idx from vring at vringh_init_iotlb
Quoted 42 lines · Eugenio Pérez
Starting from an
used_idxdifferent than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.While callers like
vdpa_simsetavail_idxdirectly it does not setused_idx. Instead of let the caller do the assignment, fetch it from the guest at initialization like vhost-kernel do.To perform the same at
vring_kernel_initandvring_user_initis left for the future.Signed-off-by Eugenio Pérez <[email protected]>--- drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)drivers/vhost/vringh.c +11
-0@@ -1301,6 +1301,17 @@ static inline int putused_iotlb(const struct vringh *vrh,return 0 ; } +/** +* vringh_update_used_idx - fetch used idx from driver's used split vring +* @vrh: The vring. +* +* Returns -errno or 0. +*/ +static inline int vringh_update_used_idx (struct vringh *vrh) +{ +return getu16_iotlb (vrh, &vrh->last_used_idx, &vrh->vring.used->idx); +} +/** * vringh_init_iotlb - initialize a vringh for a ring with IOTLB. * @vrh: the vringh to initialize.
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1332
Quoted 3 lines
drivers/vhost/vringh.c +0
-0@@ -1319,8 +1330,18 @@ int vringh_init_iotlb(struct vringh *vrh, u64 features,struct vring_avail *avail, struct vring_used *used) {
While at this, I wonder if it's better to have a dedicated parameter for last_avail_idx?
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1341
Quoted 11 lines
drivers/vhost/vringh.c +9
-2-return vringh_init_kern (vrh, features, num, weak_barriers, - desc, avail, used); +int r =vringh_init_kern (vrh, features, num, weak_barriers, desc, + avail, used); + +if (r !=0 ) +return r; + +/* Consider the ring not initialized */ +if ((void *)desc == used) +return 0 ;
I don't understand when we can get this (actually it should be a bug of the caller).
Thanks
Quoted 9 lines
drivers/vhost/vringh.c +3
-1+ +return vringh_update_used_idx (vrh); + }EXPORT_SYMBOL (vringh_init_iotlb); --2.31.1
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Eugenio Perez Martin
maintainerRe: [PATCH 2/2] vringh: fetch used_idx from vring at vringh_init_iotlb
Quoted 42 lines · Jason Wang
Quoted 40 lines · Eugenio Pérez
Starting from an
used_idxdifferent than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.While callers like
vdpa_simsetavail_idxdirectly it does not setused_idx. Instead of let the caller do the assignment, fetch it from the guest at initialization like vhost-kernel do.To perform the same at
vring_kernel_initandvring_user_initis left for the future.Signed-off-by Eugenio Pérez <[email protected]>--- drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)drivers/vhost/vringh.c +11
-0@@ -1301,6 +1301,17 @@ static inline int putused_iotlb(const struct vringh *vrh,return 0 ; } +/** +* vringh_update_used_idx - fetch used idx from driver's used split vring +* @vrh: The vring. +* +* Returns -errno or 0. +*/ +static inline int vringh_update_used_idx (struct vringh *vrh) +{ +return getu16_iotlb (vrh, &vrh->last_used_idx, &vrh->vring.used->idx); +} +/** * vringh_init_iotlb - initialize a vringh for a ring with IOTLB.
Quoted 9 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1332
Quoted 5 lines
drivers/vhost/vringh.c +0
-0* @vrh: the vringh to initialize. @@ -1319,8 +1330,18 @@ int vringh_init_iotlb(struct vringh *vrh, u64 features,struct vring_avail *avail, struct vring_used *used) {While at this, I wonder if it's better to have a dedicated parameter for
last_avail_idx?
I also had that thought. To directly assign last_avail_idx is not a specially elegant API IMO.
Maybe expose a way to fetch used_idx from device vring and pass used_idx as parameter too?
Quoted 6 lines
Quoted 6 lines
drivers/vhost/vringh.c +4
-2-return vringh_init_kern (vrh, features, num, weak_barriers, - desc, avail, used); +int r =vringh_init_kern (vrh, features, num, weak_barriers, desc, + avail, used); + +if (r !=0 )
Quoted 9 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1341
Quoted 5 lines
drivers/vhost/vringh.c +5
-0+return r; + +/* Consider the ring not initialized */ +if ((void *)desc == used) +return 0 ;I don't understand when we can get this (actually it should be a bug of the caller).
You can see it in vdpasim_vq_reset.
Note that to consider desc == 0 to be an uninitialized ring is a bug IMO. QEMU considers it that way also, but the standard does not forbid any ring to be at address 0. Especially if we use vIOMMU.
So I think the best way to know if we can use the vringh is either this way, or provide an explicit "initialized" boolean attribute. Maybe a new "bool is_initialized(vrh)" is enough, if we don't want to add new attributes.
Thanks!
Quoted 12 lines
Thanks
Quoted 9 lines
drivers/vhost/vringh.c +3
-1+ +return vringh_update_used_idx (vrh); + }EXPORT_SYMBOL (vringh_init_iotlb); --2.31.1
Re: [PATCH 2/2] vringh: fetch used_idx from vring at vringh_init_iotlb
Quoted 54 lines · Eugenio Perez Martin
Quoted 42 lines · Jason Wang
Quoted 40 lines · Eugenio Pérez
Starting from an
used_idxdifferent than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.While callers like
vdpa_simsetavail_idxdirectly it does not setused_idx. Instead of let the caller do the assignment, fetch it from the guest at initialization like vhost-kernel do.To perform the same at
vring_kernel_initandvring_user_initis left for the future.Signed-off-by Eugenio Pérez <[email protected]>--- drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)drivers/vhost/vringh.c +11
-0@@ -1301,6 +1301,17 @@ static inline int putused_iotlb(const struct vringh *vrh,return 0 ; } +/** +* vringh_update_used_idx - fetch used idx from driver's used split vring +* @vrh: The vring. +* +* Returns -errno or 0. +*/ +static inline int vringh_update_used_idx (struct vringh *vrh) +{ +return getu16_iotlb (vrh, &vrh->last_used_idx, &vrh->vring.used->idx); +} +/** * vringh_init_iotlb - initialize a vringh for a ring with IOTLB. Quoted 9 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1332
Quoted 5 lines
drivers/vhost/vringh.c +0
-0* @vrh: the vringh to initialize. @@ -1319,8 +1330,18 @@ int vringh_init_iotlb(struct vringh *vrh, u64 features,struct vring_avail *avail, struct vring_used *used) {While at this, I wonder if it's better to have a dedicated parameter for
last_avail_idx?
Quoted 5 lines
I also had that thought. To directly assign
last_avail_idxis not a specially elegant API IMO.Maybe expose a way to fetch
used_idxfrom device vring and passused_idxas parameter too?
If I was not wrong, we can start from last_avail_idx, for used_idx it is only needed for inflight descriptors which might require other APIs?
(All the current vDPA user of vringh is doing in order processing)
Quoted 19 lines
Quoted 16 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1341
Quoted 12 lines
drivers/vhost/vringh.c +9
-2-return vringh_init_kern (vrh, features, num, weak_barriers, - desc, avail, used); +int r =vringh_init_kern (vrh, features, num, weak_barriers, desc, + avail, used); + +if (r !=0 ) +return r; + +/* Consider the ring not initialized */ +if ((void *)desc == used) +return 0 ;I don't understand when we can get this (actually it should be a bug of the caller).
You can see it in
vdpasim_vq_reset.
Quoted 8 lines
Note that to consider desc == 0 to be an uninitialized ring is a bug IMO. QEMU considers it that way also, but the standard does not forbid any ring to be at address 0. Especially if we use vIOMMU.
So I think the best way to know if we can use the vringh is either this way, or provide an explicit "initialized" boolean attribute. Maybe a new "bool
is_initialized(vrh)" is enough, if we don't want to add new attributes.
I wonder if we can avoid this in the simulator level instead of the vringh (anyhow it only exposes a vringh_init_xxx() helper now).
Thanks
Quoted 16 lines
Thanks!
Quoted 12 lines
Thanks
Quoted 9 lines
drivers/vhost/vringh.c +3
-1+ +return vringh_update_used_idx (vrh); + }EXPORT_SYMBOL (vringh_init_iotlb); --2.31.1
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Eugenio Perez Martin
maintainerRe: [PATCH 2/2] vringh: fetch used_idx from vring at vringh_init_iotlb
Quoted 60 lines · Jason Wang
Quoted 57 lines · Eugenio Perez Martin
Quoted 42 lines · Jason Wang
Quoted 40 lines · Eugenio Pérez
Starting from an
used_idxdifferent than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.While callers like
vdpa_simsetavail_idxdirectly it does not setused_idx. Instead of let the caller do the assignment, fetch it from the guest at initialization like vhost-kernel do.To perform the same at
vring_kernel_initandvring_user_initis left for the future.Signed-off-by Eugenio Pérez <[email protected]>--- drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)drivers/vhost/vringh.c +11
-0@@ -1301,6 +1301,17 @@ static inline int putused_iotlb(const struct vringh *vrh,return 0 ; } +/** +* vringh_update_used_idx - fetch used idx from driver's used split vring +* @vrh: The vring. +* +* Returns -errno or 0. +*/ +static inline int vringh_update_used_idx (struct vringh *vrh) +{ +return getu16_iotlb (vrh, &vrh->last_used_idx, &vrh->vring.used->idx); +} +/** * vringh_init_iotlb - initialize a vringh for a ring with IOTLB. Quoted 9 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1332
Quoted 5 lines
drivers/vhost/vringh.c +0
-0* @vrh: the vringh to initialize. @@ -1319,8 +1330,18 @@ int vringh_init_iotlb(struct vringh *vrh, u64 features,struct vring_avail *avail, struct vring_used *used) {While at this, I wonder if it's better to have a dedicated parameter for
last_avail_idx?I also had that thought. To directly assign
last_avail_idxis not a specially elegant API IMO.
Quoted 9 lines
Quoted 2 lines
Maybe expose a way to fetch
used_idxfrom device vring and passused_idxas parameter too?If I was not wrong, we can start from
last_avail_idx, forused_idxit is only needed for inflight descriptors which might require other APIs?(All the current vDPA user of vringh is doing in order processing)
That was actually my first attempt and it works equally well for the moment, but it diverges from vhost-kernel behavior for little benefit.
To assign both values at set_vring_base mean that if vDPA introduces an (hypothetical) VHOST_VDPA_F_INFLIGHT backend feature in the future, the initialization process would vary a lot:
* Without that feature, the used_idx starts with 0, and the avail one is 0 or whatever value the user set with vring_set_base.
* With that feature, the device will read guest's used_idx as vhost-kernel? We would enable a new ioctl to set it, or expand set_base to include used_idx, effectively diverting from vhost-kernel?
To me the wisest option is to move this with vhost-kernel. Maybe we need to add a feature bit to know that the hypervisor can trust the device will do "the right thing" (VHOST_VDPA_F_FETCH_USED_AT_ENABLE?), but we should keep it orthogonal to inflight descriptor migration in my opinion.
Having said that, I'm totally ok to do it otherwise (or to expand the patch message if needed).
Quoted 23 lines
Quoted 23 lines
Quoted 16 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1341
Quoted 12 lines
drivers/vhost/vringh.c +9
-2-return vringh_init_kern (vrh, features, num, weak_barriers, - desc, avail, used); +int r =vringh_init_kern (vrh, features, num, weak_barriers, desc, + avail, used); + +if (r !=0 ) +return r; + +/* Consider the ring not initialized */ +if ((void *)desc == used) +return 0 ;I don't understand when we can get this (actually it should be a bug of the caller).
You can see it in
vdpasim_vq_reset.Note that to consider desc == 0 to be an uninitialized ring is a bug IMO. QEMU considers it that way also, but the standard does not forbid any ring to be at address 0. Especially if we use vIOMMU.
Quoted 8 lines
Quoted 4 lines
So I think the best way to know if we can use the vringh is either this way, or provide an explicit "initialized" boolean attribute. Maybe a new "bool
is_initialized(vrh)" is enough, if we don't want to add new attributes.I wonder if we can avoid this in the simulator level instead of the vringh (anyhow it only exposes a
vringh_init_xxx()helper now).
In my opinion that is a mistake if other drivers will use it to implement the emulated control virtqueue. And it requires more changes. But it is doable for sure.
Thanks!
Re: [PATCH 2/2] vringh: fetch used_idx from vring at vringh_init_iotlb
Quoted 75 lines · Eugenio Perez Martin
Quoted 60 lines · Jason Wang
Quoted 57 lines · Eugenio Perez Martin
Quoted 42 lines · Jason Wang
Quoted 40 lines · Eugenio Pérez
Starting from an
used_idxdifferent than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.While callers like
vdpa_simsetavail_idxdirectly it does not setused_idx. Instead of let the caller do the assignment, fetch it from the guest at initialization like vhost-kernel do.To perform the same at
vring_kernel_initandvring_user_initis left for the future.Signed-off-by Eugenio Pérez <[email protected]>--- drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)drivers/vhost/vringh.c +11
-0@@ -1301,6 +1301,17 @@ static inline int putused_iotlb(const struct vringh *vrh,return 0 ; } +/** +* vringh_update_used_idx - fetch used idx from driver's used split vring +* @vrh: The vring. +* +* Returns -errno or 0. +*/ +static inline int vringh_update_used_idx (struct vringh *vrh) +{ +return getu16_iotlb (vrh, &vrh->last_used_idx, &vrh->vring.used->idx); +} +/** * vringh_init_iotlb - initialize a vringh for a ring with IOTLB. Quoted 9 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1332
Quoted 5 lines
drivers/vhost/vringh.c +0
-0* @vrh: the vringh to initialize. @@ -1319,8 +1330,18 @@ int vringh_init_iotlb(struct vringh *vrh, u64 features,struct vring_avail *avail, struct vring_used *used) {While at this, I wonder if it's better to have a dedicated parameter for
last_avail_idx?I also had that thought. To directly assign
last_avail_idxis not a specially elegant API IMO.Quoted 9 lines
Quoted 2 lines
Maybe expose a way to fetch
used_idxfrom device vring and passused_idxas parameter too?If I was not wrong, we can start from
last_avail_idx, forused_idxit is only needed for inflight descriptors which might require other APIs?(All the current vDPA user of vringh is doing in order processing)
That was actually my first attempt and it works equally well for the moment, but it diverges from vhost-kernel behavior for little benefit.
Quoted 8 lines
To assign both values at
set_vring_basemean that if vDPA introduces an (hypothetical)VHOST_VDPA_F_INFLIGHTbackend feature in the future, the initialization process would vary a lot:
* Without that feature, theused_idxstarts with 0, and the avail one is 0 or whatever value the user set withvring_set_base.
* With that feature, the device will read guest'sused_idxas vhost-kernel? We would enable a new ioctl to set it, or expandset_baseto includeused_idx, effectively diverting from vhost-kernel?
Adding Longpeng who is looking at this.
We can leave this to the caller to decide.
Btw, a question, at which case the device used index does not equal to the used index in the vring when the device is suspended? I think the correct way is to flush the pending used indexes before suspending. Otherwise we need an API to get pending used indices?
Quoted 6 lines
To me the wisest option is to move this with vhost-kernel. Maybe we need to add a feature bit to know that the hypervisor can trust the device will do "the right thing" (
VHOST_VDPA_F_FETCH_USED_AT_ENABLE?), but we should keep it orthogonal to inflight descriptor migration in my opinion.
I think we need to understand if there are any other possible use cases for setting used idx other than inflight stuff.
Quoted 3 lines
Having said that, I'm totally ok to do it otherwise (or to expand the patch message if needed).
I tend to do that in another series (not mix with the fixes).
Quoted 29 lines
Quoted 29 lines
Quoted 28 lines
Quoted 17 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1341
Quoted 13 lines
drivers/vhost/vringh.c +9
-2-return vringh_init_kern (vrh, features, num, weak_barriers, - desc, avail, used); +int r =vringh_init_kern (vrh, features, num, weak_barriers, desc, + avail, used); + +if (r !=0 ) +return r; + +/* Consider the ring not initialized */ +if ((void *)desc == used) +return 0 ;I don't understand when we can get this (actually it should be a bug of the caller).
You can see it in
vdpasim_vq_reset.Note that to consider desc == 0 to be an uninitialized ring is a bug IMO. QEMU considers it that way also, but the standard does not forbid any ring to be at address 0. Especially if we use vIOMMU.
So I think the best way to know if we can use the vringh is either this way, or provide an explicit "initialized" boolean attribute. Maybe a new "bool
is_initialized(vrh)" is enough, if we don't want to add new attributes.
Quoted 7 lines
Quoted 3 lines
I wonder if we can avoid this in the simulator level instead of the vringh (anyhow it only exposes a
vringh_init_xxx()helper now).In my opinion that is a mistake if other drivers will use it to implement the emulated control virtqueue. And it requires more changes. But it is doable for sure.
The problem is, there's no reset API in vringh, that's why you need to do if ((void *)desc == used) which depends on behaviour of the vringh user.
So I think we should either:
1) move that check in vdpa_sim (since it's not guaranteed that all the vringh users will make desc equal to used during reset)
or
2) introduce a vringh_reset_xxx()
1) seems a good step for -stable.
Thanks
Quoted 3 lines
Thanks!
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Eugenio Perez Martin
maintainerRe: [PATCH 2/2] vringh: fetch used_idx from vring at vringh_init_iotlb
Quoted 87 lines · Jason Wang
Quoted 75 lines · Eugenio Perez Martin
Quoted 60 lines · Jason Wang
Quoted 57 lines · Eugenio Perez Martin
Quoted 42 lines · Jason Wang
Quoted 40 lines · Eugenio Pérez
Starting from an
used_idxdifferent than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.While callers like
vdpa_simsetavail_idxdirectly it does not setused_idx. Instead of let the caller do the assignment, fetch it from the guest at initialization like vhost-kernel do.To perform the same at
vring_kernel_initandvring_user_initis left for the future.Signed-off-by Eugenio Pérez <[email protected]>--- drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)drivers/vhost/vringh.c +11
-0@@ -1301,6 +1301,17 @@ static inline int putused_iotlb(const struct vringh *vrh,return 0 ; } +/** +* vringh_update_used_idx - fetch used idx from driver's used split vring +* @vrh: The vring. +* +* Returns -errno or 0. +*/ +static inline int vringh_update_used_idx (struct vringh *vrh) +{ +return getu16_iotlb (vrh, &vrh->last_used_idx, &vrh->vring.used->idx); +} +/** * vringh_init_iotlb - initialize a vringh for a ring with IOTLB. Quoted 9 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1332
Quoted 5 lines
drivers/vhost/vringh.c +0
-0* @vrh: the vringh to initialize. @@ -1319,8 +1330,18 @@ int vringh_init_iotlb(struct vringh *vrh, u64 features,struct vring_avail *avail, struct vring_used *used) {While at this, I wonder if it's better to have a dedicated parameter for
last_avail_idx?I also had that thought. To directly assign
last_avail_idxis not a specially elegant API IMO.Quoted 9 lines
Quoted 2 lines
Maybe expose a way to fetch
used_idxfrom device vring and passused_idxas parameter too?If I was not wrong, we can start from
last_avail_idx, forused_idxit is only needed for inflight descriptors which might require other APIs?(All the current vDPA user of vringh is doing in order processing)
That was actually my first attempt and it works equally well for the moment, but it diverges from vhost-kernel behavior for little benefit.
Quoted 8 lines
To assign both values at
set_vring_basemean that if vDPA introduces an (hypothetical)VHOST_VDPA_F_INFLIGHTbackend feature in the future, the initialization process would vary a lot:
* Without that feature, theused_idxstarts with 0, and the avail one is 0 or whatever value the user set withvring_set_base.
* With that feature, the device will read guest'sused_idxas vhost-kernel? We would enable a new ioctl to set it, or expandset_baseto includeused_idx, effectively diverting from vhost-kernel?
Quoted 2 lines
Adding Longpeng who is looking at this.
Sorry, I'll CC [email protected] in future series like this one.
Quoted 8 lines
We can leave this to the caller to decide.
Btw, a question, at which case the device used index does not equal to the used index in the vring when the device is suspended? I think the correct way is to flush the pending used indexes before suspending. Otherwise we need an API to get pending used indices?
Quoted 1 line
Quoted 9 lines
Quoted 5 lines
To me the wisest option is to move this with vhost-kernel. Maybe we need to add a feature bit to know that the hypervisor can trust the device will do "the right thing" (
VHOST_VDPA_F_FETCH_USED_AT_ENABLE?), but we should keep it orthogonal to inflight descriptor migration in my opinion.I think we need to understand if there are any other possible use cases for setting used idx other than inflight stuff.
Answering this and the previous comment, I cannot think in any case outside of inflight migration. I'm just trying to avoid different behavior between vhost-kernel and vhost-vdpa, and to make features as orthogonal as possible.
Quoted 39 lines
Quoted 3 lines
Having said that, I'm totally ok to do it otherwise (or to expand the patch message if needed).
I tend to do that in another series (not mix with the fixes).
Quoted 33 lines
Quoted 32 lines
Quoted 20 lines
Quoted 17 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1341
Quoted 13 lines
drivers/vhost/vringh.c +9
-2-return vringh_init_kern (vrh, features, num, weak_barriers, - desc, avail, used); +int r =vringh_init_kern (vrh, features, num, weak_barriers, desc, + avail, used); + +if (r !=0 ) +return r; + +/* Consider the ring not initialized */ +if ((void *)desc == used) +return 0 ;I don't understand when we can get this (actually it should be a bug of the caller).
You can see it in
vdpasim_vq_reset.Quoted 8 lines
Note that to consider desc == 0 to be an uninitialized ring is a bug IMO. QEMU considers it that way also, but the standard does not forbid any ring to be at address 0. Especially if we use vIOMMU.
So I think the best way to know if we can use the vringh is either this way, or provide an explicit "initialized" boolean attribute. Maybe a new "bool
is_initialized(vrh)" is enough, if we don't want to add new attributes.I wonder if we can avoid this in the simulator level instead of the vringh (anyhow it only exposes a
vringh_init_xxx()helper now).
Quoted 8 lines
Quoted 3 lines
In my opinion that is a mistake if other drivers will use it to implement the emulated control virtqueue. And it requires more changes. But it is doable for sure.
The problem is, there's no reset API in vringh, that's why you need to do if ((void *)desc == used) which depends on behaviour of the vringh user.
That's a very good point indeed.
Quoted 11 lines
So I think we should either:
1) move that check in
vdpa_sim(since it's not guaranteed that all the vringh users will make desc equal to used during reset)or
2) introduce a
vringh_reset_xxx()1) seems a good step for -stable.
We can go to 1 for sure. So let's set last_used_idx at vdpasim_set_vq_state, same value as last_avail_idx, and stash it at vdpasim_queue_ready.
Do I need to resend the previous patch in this series?
Do we need to offer a new feature flag indicating we will set used_idx with avail_idx?
Thanks!
Michael S. Tsirkin
maintainerRe: [PATCH 2/2] vringh: fetch used_idx from vring at vringh_init_iotlb
Quoted 182 lines · Eugenio Perez Martin
Quoted 87 lines · Jason Wang
Quoted 75 lines · Eugenio Perez Martin
Quoted 60 lines · Jason Wang
Quoted 57 lines · Eugenio Perez Martin
Quoted 42 lines · Jason Wang
Quoted 40 lines · Eugenio Pérez
Starting from an
used_idxdifferent than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.While callers like
vdpa_simsetavail_idxdirectly it does not setused_idx. Instead of let the caller do the assignment, fetch it from the guest at initialization like vhost-kernel do.To perform the same at
vring_kernel_initandvring_user_initis left for the future.Signed-off-by Eugenio Pérez <[email protected]>--- drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)drivers/vhost/vringh.c +11
-0@@ -1301,6 +1301,17 @@ static inline int putused_iotlb(const struct vringh *vrh,return 0 ; } +/** +* vringh_update_used_idx - fetch used idx from driver's used split vring +* @vrh: The vring. +* +* Returns -errno or 0. +*/ +static inline int vringh_update_used_idx (struct vringh *vrh) +{ +return getu16_iotlb (vrh, &vrh->last_used_idx, &vrh->vring.used->idx); +} +/** * vringh_init_iotlb - initialize a vringh for a ring with IOTLB. Quoted 9 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1332
Quoted 5 lines
drivers/vhost/vringh.c +0
-0* @vrh: the vringh to initialize. @@ -1319,8 +1330,18 @@ int vringh_init_iotlb(struct vringh *vrh, u64 features,struct vring_avail *avail, struct vring_used *used) {While at this, I wonder if it's better to have a dedicated parameter for
last_avail_idx?I also had that thought. To directly assign
last_avail_idxis not a specially elegant API IMO.Quoted 9 lines
Quoted 2 lines
Maybe expose a way to fetch
used_idxfrom device vring and passused_idxas parameter too?If I was not wrong, we can start from
last_avail_idx, forused_idxit is only needed for inflight descriptors which might require other APIs?(All the current vDPA user of vringh is doing in order processing)
That was actually my first attempt and it works equally well for the moment, but it diverges from vhost-kernel behavior for little benefit.
Quoted 8 lines
To assign both values at
set_vring_basemean that if vDPA introduces an (hypothetical)VHOST_VDPA_F_INFLIGHTbackend feature in the future, the initialization process would vary a lot:
* Without that feature, theused_idxstarts with 0, and the avail one is 0 or whatever value the user set withvring_set_base.
* With that feature, the device will read guest'sused_idxas vhost-kernel? We would enable a new ioctl to set it, or expandset_baseto includeused_idx, effectively diverting from vhost-kernel?Quoted 2 lines
Adding Longpeng who is looking at this.
Sorry, I'll CC [email protected] in future series like this one.
Quoted 8 lines
We can leave this to the caller to decide.
Btw, a question, at which case the device used index does not equal to the used index in the vring when the device is suspended? I think the correct way is to flush the pending used indexes before suspending. Otherwise we need an API to get pending used indices?
Quoted 1 line
Quoted 9 lines
Quoted 5 lines
To me the wisest option is to move this with vhost-kernel. Maybe we need to add a feature bit to know that the hypervisor can trust the device will do "the right thing" (
VHOST_VDPA_F_FETCH_USED_AT_ENABLE?), but we should keep it orthogonal to inflight descriptor migration in my opinion.I think we need to understand if there are any other possible use cases for setting used idx other than inflight stuff.
Answering this and the previous comment, I cannot think in any case outside of inflight migration. I'm just trying to avoid different behavior between vhost-kernel and vhost-vdpa, and to make features as orthogonal as possible.
Quoted 39 lines
Quoted 3 lines
Having said that, I'm totally ok to do it otherwise (or to expand the patch message if needed).
I tend to do that in another series (not mix with the fixes).
Quoted 33 lines
Quoted 32 lines
Quoted 20 lines
Quoted 17 lines
drivers/vhost/vringh.c int vringh_init_iotlb(struct vringh *vrh, u64 features, · line 1341
Quoted 13 lines
drivers/vhost/vringh.c +9
-2-return vringh_init_kern (vrh, features, num, weak_barriers, - desc, avail, used); +int r =vringh_init_kern (vrh, features, num, weak_barriers, desc, + avail, used); + +if (r !=0 ) +return r; + +/* Consider the ring not initialized */ +if ((void *)desc == used) +return 0 ;I don't understand when we can get this (actually it should be a bug of the caller).
You can see it in
vdpasim_vq_reset.Quoted 8 lines
Note that to consider desc == 0 to be an uninitialized ring is a bug IMO. QEMU considers it that way also, but the standard does not forbid any ring to be at address 0. Especially if we use vIOMMU.
So I think the best way to know if we can use the vringh is either this way, or provide an explicit "initialized" boolean attribute. Maybe a new "bool
is_initialized(vrh)" is enough, if we don't want to add new attributes.I wonder if we can avoid this in the simulator level instead of the vringh (anyhow it only exposes a
vringh_init_xxx()helper now).Quoted 8 lines
Quoted 3 lines
In my opinion that is a mistake if other drivers will use it to implement the emulated control virtqueue. And it requires more changes. But it is doable for sure.
The problem is, there's no reset API in vringh, that's why you need to do if ((void *)desc == used) which depends on behaviour of the vringh user.
That's a very good point indeed.
Quoted 11 lines
So I think we should either:
1) move that check in
vdpa_sim(since it's not guaranteed that all the vringh users will make desc equal to used during reset)or
2) introduce a
vringh_reset_xxx()1) seems a good step for -stable.
We can go to 1 for sure. So let's set
last_used_idxatvdpasim_set_vq_state, same value aslast_avail_idx, and stash it atvdpasim_queue_ready.
Quoted 6 lines
Do I need to resend the previous patch in this series?
Do we need to offer a new feature flag indicating we will set
used_idxwithavail_idx?Thanks!
Jason did you forget to answer or did I miss it?
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerRe: [PATCH 2/2] vringh: fetch used_idx from vring at vringh_init_iotlb
In reply to Eugenio PérezQuoted 6 lines · Eugenio Pérez
Starting from an
used_idxdifferent than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.
Quoted 8 lines
While callers like
vdpa_simsetavail_idxdirectly it does not setused_idx. Instead of let the caller do the assignment, fetch it from the guest at initialization like vhost-kernel do.To perform the same at
vring_kernel_initandvring_user_initis left for the future.Signed-off-by Eugenio Pérez <[email protected]>
So I applied 1/2 and dropped 2/2 for now, right?
Quoted 49 lines
--- drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)drivers/vhost/vringh.c +23
-2@@ -1301,6 +1301,17 @@ static inline int putused_iotlb(const struct vringh *vrh,return 0 ; } +/** +* vringh_update_used_idx - fetch used idx from driver's used split vring +* @vrh: The vring. +* +* Returns -errno or 0. +*/ +static inline int vringh_update_used_idx (struct vringh *vrh) +{ +return getu16_iotlb (vrh, &vrh->last_used_idx, &vrh->vring.used->idx); +} +/** * vringh_init_iotlb - initialize a vringh for a ring with IOTLB. * @vrh: the vringh to initialize. @@ -1319,8 +1330,18 @@ int vringh_init_iotlb(struct vringh *vrh, u64 features,struct vring_avail *avail, struct vring_used *used) { - return vringh_init_kern(vrh, features, num, weak_barriers, -desc ,avail ,used ); + int r = vringh_init_kern(vrh, features, num, weak_barriers, desc, + avail, used); + +if (r !=0 ) +return r; + +/* Consider the ring not initialized */ +if ((void *)desc == used) +return 0 ; + +return vringh_update_used_idx (vrh); + }EXPORT_SYMBOL (vringh_init_iotlb);Signature · 1 line
2.31.1
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Eugenio Perez Martin
maintainerRe: [PATCH 2/2] vringh: fetch used_idx from vring at vringh_init_iotlb
Quoted 12 lines · Michael S. Tsirkin
Quoted 10 lines · Eugenio Pérez
Starting from an
used_idxdifferent than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.While callers like
vdpa_simsetavail_idxdirectly it does not setused_idx. Instead of let the caller do the assignment, fetch it from the guest at initialization like vhost-kernel do.
Quoted 8 lines
Quoted 4 lines
To perform the same at
vring_kernel_initandvring_user_initis left for the future.Signed-off-by Eugenio Pérez <[email protected]>So I applied 1/2 and dropped 2/2 for now, right?
Yes, please. 2/2 needs tweaking, I'll address them ASAP.
Thanks!
Quoted 50 lines
Quoted 49 lines
--- drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)drivers/vhost/vringh.c +23
-3@@ -1301,6 +1301,17 @@ static inline int putused_iotlb(const struct vringh *vrh,return 0 ; } +/** +* vringh_update_used_idx - fetch used idx from driver's used split vring +* @vrh: The vring. +* +* Returns -errno or 0. +*/ +static inline int vringh_update_used_idx (struct vringh *vrh) +{ +return getu16_iotlb (vrh, &vrh->last_used_idx, &vrh->vring.used->idx); +} +/** * vringh_init_iotlb - initialize a vringh for a ring with IOTLB. * @vrh: the vringh to initialize. @@ -1319,8 +1330,18 @@ int vringh_init_iotlb(struct vringh *vrh, u64 features,struct vring_avail *avail, struct vring_used *used) { - return vringh_init_kern(vrh, features, num, weak_barriers, -desc ,avail ,used ); + int r = vringh_init_kern(vrh, features, num, weak_barriers, desc, + avail, used); + +if (r !=0 ) +return r; + +/* Consider the ring not initialized */ +if ((void *)desc == used) +return 0 ; + +return vringh_update_used_idx (vrh); + }EXPORT_SYMBOL (vringh_init_iotlb); --2.31.1
Michael S. Tsirkin
maintainerQuoted 7 lines · Eugenio Pérez
The use of
set_vq_stateis to indicate vdpa device the state of a virtqueue. In the case of split, it means theavail_idx. This is mandatory for use cases like live migration.However,
vdpa_simreset the vq state atvdpasim_queue_readysince it callsvringh_init_iotlb.
Quoted 6 lines
Also, to starting from an
used_idxdifferent than 0 is needed in use cases like virtual machine migration. Not doing so and letting the caller set an avail idx different than 0 causes destination device to try to use old buffers that source driver already recover and are not available anymore.This series fixes both problems allowing to migrate to a
vdpa_sim_netdevice.
Jason problems you pointed out are all consmetic do you ack the patchset? Or expect another revision?
Quoted 11 lines
Eugenio Pérez (2):
vdpa_sim: not reset state invdpasim_queue_ready
vringh: fetchused_idxfrom vring atvringh_init_iotlbdrivers/vdpa/vdpa_sim/vdpa_sim.c | 2 ++ drivers/vhost/vringh.c | 25 +++++++++++++++++++++++-- 2 files changed, 25 insertions(+), 2 deletions(-)Signature · 1 line
2.31.1
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization