Skip to contentSkip to conversation

vdpa/mlx5: Clear vq ready indication upon device reset

PATCH In review Eli Cohen 2 participants 5 messages Jun 2, 2021 – Jun 3, 2021

Subsystems (2)
lore
Share & tools
lore

Eli Cohen

author
#0 lore raw

PATCH 1/1 vdpa/mlx5: Clear vq ready indication upon device reset

After device reset, the virtqueues are not ready so clear the ready field.

Failing to do so can result in virtio_vdpa failing to load if the device was previously used by vhost_vdpa and the old values are ready. virtio_vdpa expects to find VQs in "not ready" state.

Fixes 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
---
 drivers/vdpa/mlx5/net/mlx5_vnet.c | 1 +
 1 file changed, 1 insertion(+)
drivers/vdpa/mlx5/net/mlx5_vnet.c +1 -0
index 02a05492204c..e8bc0842b44c 100644
@@ -862,6 +862,7 @@ static void destroy_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtq
 		return;
 	}
 	umems_destroy(ndev, mvq);
+	mvq->ready = false;
 }
 
 static u32 get_rqpn(struct mlx5_vdpa_virtqueue *mvq, bool fw)
Signature · 1 line
2.31.1
Acked-by on v1 1/1
#1 lore raw
Quoted 9 lines · Eli Cohen

After device reset, the virtqueues are not ready so clear the ready field.

Failing to do so can result in virtio_vdpa failing to load if the device was previously used by vhost_vdpa and the old values are ready. virtio_vdpa expects to find VQs in "not ready" state.

Fixes 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
Quoted 16 lines
---
  drivers/vdpa/mlx5/net/mlx5_vnet.c | 1 +
  1 file changed, 1 insertion(+)
drivers/vdpa/mlx5/net/mlx5_vnet.c +1 -0
index 02a05492204c..e8bc0842b44c 100644
@@ -862,6 +862,7 @@ static void destroy_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtq
  		return;
  	}
  	umems_destroy(ndev, mvq);
+	mvq->ready = false;
  }
 
  static u32 get_rqpn(struct mlx5_vdpa_virtqueue *mvq, bool fw)
#2 lore raw
Quoted 9 lines · Jason Wang
Quoted 7 lines · Eli Cohen

After device reset, the virtqueues are not ready so clear the ready field.

Failing to do so can result in virtio_vdpa failing to load if the device was previously used by vhost_vdpa and the old values are ready. virtio_vdpa expects to find VQs in "not ready" state.

Quoted 5 lines
Quoted 2 lines
Fixes 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")

A second thought.

destroy_virtqueue() could be called many places.

One of them is the mlx5_vdpa_change_map(), if this is case, this looks wrong.

It looks to me it's simpler to do this in clear_virtqueues() which can only be called during reset.

Thanks

Quoted 18 lines
Quoted 18 lines

---
  drivers/vdpa/mlx5/net/mlx5_vnet.c | 1 +

  1 file changed, 1 insertion(+)
drivers/vdpa/mlx5/net/mlx5_vnet.c +0 -0
index 02a05492204c..e8bc0842b44c 100644
@@ -862,6 +862,7 @@ static void destroy_virtqueue(struct 

mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtq
          return;
      }
      umems_destroy(ndev, mvq);
+    mvq->ready = false;
  }
    static u32 get_rqpn(struct mlx5_vdpa_virtqueue *mvq, bool fw)

author
#3 lore raw
Quoted 19 lines · Jason Wang
Quoted 9 lines · Jason Wang
Quoted 7 lines · Eli Cohen

After device reset, the virtqueues are not ready so clear the ready field.

Failing to do so can result in virtio_vdpa failing to load if the device was previously used by vhost_vdpa and the old values are ready. virtio_vdpa expects to find VQs in "not ready" state.

Quoted 6 lines
Quoted 3 lines
Fixes 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5

devices")

Quoted 6 lines

A second thought.

destroy_virtqueue() could be called many places.

One of them is the mlx5_vdpa_change_map(), if this is case, this looks wrong.

Right, although most likely VQs become ready only after all map changes occur becuase I did not encounter any issue while testing.

Quoted 3 lines

It looks to me it's simpler to do this in clear_virtqueues() which can only be called during reset.

There is no clear_virtqueues() function. You probably mean to insert a call in mlx5_vdpa_set_status() in case it performs reset. This function will go over all virtqueues and clear their ready flag.

Alternatively we can add boolean argument to teardown_driver() that signifies if we are in reset flow and in this case we clear ready.

Quoted 24 lines

Thanks

Quoted 19 lines
Quoted 19 lines

---
  drivers/vdpa/mlx5/net/mlx5_vnet.c | 1 +

  1 file changed, 1 insertion(+)
drivers/vdpa/mlx5/net/mlx5_vnet.c +0 -0
index 02a05492204c..e8bc0842b44c 100644
@@ -862,6 +862,7 @@ static void destroy_virtqueue(struct

mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtq
          return;
      }
      umems_destroy(ndev, mvq);
+    mvq->ready = false;
  }
    static u32 get_rqpn(struct mlx5_vdpa_virtqueue *mvq, bool fw)

#4 lore raw
Quoted 16 lines · Eli Cohen
Quoted 15 lines · Jason Wang
Quoted 13 lines · Jason Wang
Quoted 10 lines · Eli Cohen

After device reset, the virtqueues are not ready so clear the ready field.

Failing to do so can result in virtio_vdpa failing to load if the device was previously used by vhost_vdpa and the old values are ready. virtio_vdpa expects to find VQs in "not ready" state.

Fixes 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5

devices")

Quoted 8 lines
Quoted 6 lines

A second thought.

destroy_virtqueue() could be called many places.

One of them is the mlx5_vdpa_change_map(), if this is case, this looks wrong.

Right, although most likely VQs become ready only after all map changes occur becuase I did not encounter any issue while testing.

Yes, but it's not guaranteed that the map won't be changed. Userspace can update the mapping when new memory is plugged into the guest for example.

Quoted 5 lines
Quoted 2 lines

It looks to me it's simpler to do this in clear_virtqueues() which can only be called during reset.

There is no clear_virtqueues() function. You probably mean to insert a call in mlx5_vdpa_set_status() in case it performs reset. This function will go over all virtqueues and clear their ready flag.

Right.

Quoted 3 lines

Alternatively we can add boolean argument to teardown_driver() that signifies if we are in reset flow and in this case we clear ready.

Yes, but doing in set_status() seems easier.

Thanks

Quoted 22 lines
Quoted 22 lines

Thanks

Quoted 18 lines
Quoted 18 lines
---
   drivers/vdpa/mlx5/net/mlx5_vnet.c | 1 +
   1 file changed, 1 insertion(+)
drivers/vdpa/mlx5/net/mlx5_vnet.c +0 -0
index 02a05492204c..e8bc0842b44c 100644
@@ -862,6 +862,7 @@ static void destroy_virtqueue(struct

mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtq
          return;
      }
      umems_destroy(ndev, mvq);
+    mvq->ready = false;
  }
    static u32 get_rqpn(struct mlx5_vdpa_virtqueue *mvq, bool fw)