Skip to contentSkip to conversation

vdpa/sfc: handle sync issue between qemu and vhost-user

PATCH Superseded [email protected] 3 participants 6 messages Jun 27, 2022 – Jul 1, 2022 virtio core

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 vdpa/sfc: handle sync issue between qemu and vhost-user

From: Abhimanyu Saini <[email protected]>

When DPDK app is running in the VF, it sometimes rings the doorbell before dev_config has had a chance to complete and hence it misses the event. As workaround, ring the doorbell when vDPA reports the notify_area to QEMU.

Fixes 5e7596ba7cb3 ("vdpa/sfc: introduce Xilinx vDPA driver")
---
 drivers/vdpa/sfc/sfc_vdpa_ops.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/vdpa/sfc/sfc_vdpa_ops.c +14 -0
index b3d9b6c..63aa52d 100644
@@ -794,6 +794,8 @@
 	int vfio_dev_fd;
 	efx_rc_t rc;
 	unsigned int bar_offset;
+	volatile void *doorbell;
+	struct rte_pci_device *pci_dev;
 	struct rte_vdpa_device *vdpa_dev;
 	struct sfc_vdpa_ops_data *ops_data;
 	struct vfio_region_info reg = { .argsz = sizeof(reg) };
@@ -856,6 +858,18 @@
 	sfc_vdpa_info(dev, "vDPA ops get_notify_area :: offset : 0x%" PRIx64,
 		      *offset);
 
+	pci_dev = sfc_vdpa_adapter_by_dev_handle(dev)->pdev;
+	doorbell = (uint8_t *)pci_dev->mem_resource[reg.index].addr + *offset;
+
+	/*
+	 * virtio-net driver in VM sends queue notifications before
+	 * vDPA has a chance to setup the queues and notification area,
+	 * and hence the HW misses these doorbell notifications.
+	 * Since, it is safe to send duplicate doorbell, send another
+	 * doorbell from vDPA driver as workaround for this timing issue.
+	 */
+	rte_write16(qid, doorbell);
+
 	return 0;
 }
 
Signature · 1 line
1.8.3.1
#1 lore raw

When you send a new version, please, don't forget to specify -v <next-version-number> on part format and use --in-reply-to with the first mail ID. See contributors guidelines.

Also, new version should make it clear what is changed. See below.

Quoted 8 lines · On 6/27/22 11:49, [email protected] wrote:

From: Abhimanyu Saini <[email protected]>

When DPDK app is running in the VF, it sometimes rings the doorbell before dev_config has had a chance to complete and hence it misses the event. As workaround, ring the doorbell when vDPA reports the notify_area to QEMU.

Fixes 5e7596ba7cb3 ("vdpa/sfc: introduce Xilinx vDPA driver")

Above seems to be wrong. Nearby code is added later, so, it should be:

Fixes 630be406dcbf ("vdpa/sfc: get queue notify area info")
Quoted 5 lines

---

Version changelog here.

Quoted 35 lines
  drivers/vdpa/sfc/sfc_vdpa_ops.c | 14 ++++++++++++++
  1 file changed, 14 insertions(+)
drivers/vdpa/sfc/sfc_vdpa_ops.c +14 -0
index b3d9b6c..63aa52d 100644
@@ -794,6 +794,8 @@
  	int vfio_dev_fd;
  	efx_rc_t rc;
  	unsigned int bar_offset;
+	volatile void *doorbell;
+	struct rte_pci_device *pci_dev;
  	struct rte_vdpa_device *vdpa_dev;
  	struct sfc_vdpa_ops_data *ops_data;
  	struct vfio_region_info reg = { .argsz = sizeof(reg) };
@@ -856,6 +858,18 @@
  	sfc_vdpa_info(dev, "vDPA ops get_notify_area :: offset : 0x%" PRIx64,
  		      *offset);
 
+	pci_dev = sfc_vdpa_adapter_by_dev_handle(dev)->pdev;
+	doorbell = (uint8_t *)pci_dev->mem_resource[reg.index].addr + *offset;
+
+	/*
+	 * virtio-net driver in VM sends queue notifications before
+	 * vDPA has a chance to setup the queues and notification area,
+	 * and hence the HW misses these doorbell notifications.
+	 * Since, it is safe to send duplicate doorbell, send another
+	 * doorbell from vDPA driver as workaround for this timing issue.
+	 */
+	rte_write16(qid, doorbell);
+
  	return 0;
  }
 
#2 lore raw

PATCH v2 1/1 vdpa/sfc: handle sync issue between qemu and vhost-user

In reply to [email protected]

From: Abhimanyu Saini <[email protected]>

When DPDK app is running in the VF, it sometimes rings the doorbell before dev_config has had a chance to complete and hence it misses the event. As workaround, ring the doorbell when vDPA reports the notify_area to QEMU.

Fixes 630be406dcbf ("vdpa/sfc: get queue notify area info")

---
v1:
* Update the commit id that this patch fixes

 drivers/vdpa/sfc/sfc_vdpa_ops.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/vdpa/sfc/sfc_vdpa_ops.c +14 -0
index b3d9b6c..63aa52d 100644
@@ -794,6 +794,8 @@
 	int vfio_dev_fd;
 	efx_rc_t rc;
 	unsigned int bar_offset;
+	volatile void *doorbell;
+	struct rte_pci_device *pci_dev;
 	struct rte_vdpa_device *vdpa_dev;
 	struct sfc_vdpa_ops_data *ops_data;
 	struct vfio_region_info reg = { .argsz = sizeof(reg) };
@@ -856,6 +858,18 @@
 	sfc_vdpa_info(dev, "vDPA ops get_notify_area :: offset : 0x%" PRIx64,
 		      *offset);
 
+	pci_dev = sfc_vdpa_adapter_by_dev_handle(dev)->pdev;
+	doorbell = (uint8_t *)pci_dev->mem_resource[reg.index].addr + *offset;
+
+	/*
+	 * virtio-net driver in VM sends queue notifications before
+	 * vDPA has a chance to setup the queues and notification area,
+	 * and hence the HW misses these doorbell notifications.
+	 * Since, it is safe to send duplicate doorbell, send another
+	 * doorbell from vDPA driver as workaround for this timing issue.
+	 */
+	rte_write16(qid, doorbell);
+
 	return 0;
 }
 
Signature · 1 line
1.8.3.1
Acked-by on v2 1/1
#3 lore raw
Quoted 7 lines · On 6/28/22 08:29, [email protected] wrote:

From: Abhimanyu Saini <[email protected]>

When DPDK app is running in the VF, it sometimes rings the doorbell before dev_config has had a chance to complete and hence it misses the event. As workaround, ring the doorbell when vDPA reports the notify_area to QEMU.

Quoted 5 lines
Fixes 630be406dcbf ("vdpa/sfc: get queue notify area info")
Reviewed-by on v2 1/1
#4 lore raw
In reply to [email protected]
Quoted 10 lines · On 6/28/22 07:29, [email protected] wrote:

From: Abhimanyu Saini <[email protected]>

When DPDK app is running in the VF, it sometimes rings the doorbell before dev_config has had a chance to complete and hence it misses the event. As workaround, ring the doorbell when vDPA reports the notify_area to QEMU.

Fixes 630be406dcbf ("vdpa/sfc: get queue notify area info")
Quoted 9 lines

---
v1:
* Update the commit id that this patch fixes

  drivers/vdpa/sfc/sfc_vdpa_ops.c | 14 ++++++++++++++
  1 file changed, 14 insertions(+)

Thanks,
Maxime

#5 lore raw
In reply to [email protected]
Quoted 10 lines · On 6/28/22 07:29, [email protected] wrote:

From: Abhimanyu Saini <[email protected]>

When DPDK app is running in the VF, it sometimes rings the doorbell before dev_config has had a chance to complete and hence it misses the event. As workaround, ring the doorbell when vDPA reports the notify_area to QEMU.

Fixes 630be406dcbf ("vdpa/sfc: get queue notify area info")
Quoted 9 lines

---
v1:
* Update the commit id that this patch fixes

  drivers/vdpa/sfc/sfc_vdpa_ops.c | 14 ++++++++++++++
  1 file changed, 14 insertions(+)

Applied to dpdk-next-virtio/main.

Thanks,
Maxime