Skip to contentSkip to conversation

net/virtio: fix indirect descriptors reconnection

PATCH Superseded Xuan Ding 3 participants 9 messages Aug 19, 2021 – Oct 21, 2021 Network drivers

lore
Share & tools
lore

Xuan Ding

author
#0 lore raw

PATCH 1/1 net/virtio: fix indirect descriptors reconnection

Since packed indirect descriptors are added and initialized when initializing vring, the reconnection path also needs to be considered.

Fixes 381f39ebb78a ("net/virtio: fix packed ring indirect descricptors setup")
---
 drivers/net/virtio/virtqueue.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/net/virtio/virtqueue.c +14 -0
index 1f9af3c31b..47229f35c4 100644
@@ -208,6 +208,20 @@ virtqueue_txvq_reset_packed(struct virtqueue *vq)
 			rte_pktmbuf_free(dxp->cookie);
 			dxp->cookie = NULL;
 		}
+
+		struct virtio_tx_region *txr;
+		txr = txvq->virtio_net_hdr_mz->addr;
+		/* first indirect descriptor is always the tx header */
+		struct vring_packed_desc *start_dp =
+			txr[desc_idx].tx_packed_indir;
+		vring_desc_init_indirect_packed(start_dp,
+		      RTE_DIM(txr[desc_idx].tx_packed_indir));
+		start_dp->addr = txvq->virtio_net_hdr_mem
+			+ desc_idx * sizeof(*txr)
+			+ offsetof(struct virtio_tx_region,
+				   tx_hdr);
+		start_dp->len = vq->hw->vtnet_hdr_size;
+
 	}
 
 	vring_desc_init_packed(vq, size);
Signature · 1 line
2.17.1
#1 lore raw

Hi Xuan,

Quoted 17 lines · Xuan Ding

Since packed indirect descriptors are added and initialized when initializing vring, the reconnection path also needs to be considered.

Fixes 381f39ebb78a ("net/virtio: fix packed ring indirect descricptors setup")
---
  drivers/net/virtio/virtqueue.c | 14 ++++++++++++++
  1 file changed, 14 insertions(+)
drivers/net/virtio/virtqueue.c +0 -0
index 1f9af3c31b..47229f35c4 100644

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 210

Quoted 3 lines
drivers/net/virtio/virtqueue.c +0 -0
@@ -208,6 +208,20 @@ virtqueue_txvq_reset_packed(struct virtqueue *vq)
  			rte_pktmbuf_free(dxp->cookie);
  			dxp->cookie = NULL;
  		}

It makes me think we might save quite some bytes by not allocating indirect descriptors when feature is not negotiated, but it might have a cost in term of performance.

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 212

Quoted 2 lines
drivers/net/virtio/virtqueue.c +2 -0
+
+		struct virtio_tx_region *txr;

Don't mix declarations within code.

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 216

Quoted 4 lines
drivers/net/virtio/virtqueue.c +4 -0
+		txr = txvq->virtio_net_hdr_mz->addr;
+		/* first indirect descriptor is always the tx header */
+		struct vring_packed_desc *start_dp =
+			txr[desc_idx].tx_packed_indir;

Ditto.

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 227

Quoted 12 lines
drivers/net/virtio/virtqueue.c +8 -0
+		vring_desc_init_indirect_packed(start_dp,
+		      RTE_DIM(txr[desc_idx].tx_packed_indir));
+		start_dp->addr = txvq->virtio_net_hdr_mem
+			+ desc_idx * sizeof(*txr)
+			+ offsetof(struct virtio_tx_region,
+				   tx_hdr);
+		start_dp->len = vq->hw->vtnet_hdr_size;
+
  	}
 
  	vring_desc_init_packed(vq, size);

Thanks,
Maxime

author
#2 lore raw

Hi Maxime,

Quoted 23 lines

-----Original Message-----
From: Maxime Coquelin <[email protected]>
Sent: Thursday, September 23, 2021 4:16 PM
To: Ding, Xuan <[email protected]>; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; [email protected]; Liu, Yong <[email protected]>
Subject: Re: [PATCH] net/virtio: fix indirect descriptors reconnection

Hi Xuan,

Quoted 12 lines · Xuan Ding

Since packed indirect descriptors are added and initialized when initializing vring, the reconnection path also needs to be considered.

Fixes 381f39ebb78a ("net/virtio: fix packed ring indirect descricptors setup")
---
  drivers/net/virtio/virtqueue.c | 14 ++++++++++++++
  1 file changed, 14 insertions(+)
Quoted 12 lines

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 210

Quoted 8 lines
drivers/net/virtio/virtqueue.c +0 -0
index 1f9af3c31b..47229f35c4 100644
@@ -208,6 +208,20 @@ virtqueue_txvq_reset_packed(struct virtqueue *vq)
  			rte_pktmbuf_free(dxp->cookie);
  			dxp->cookie = NULL;
  		}

It makes me think we might save quite some bytes by not allocating indirect descriptors when feature is not negotiated, but it might have a cost in term of performance.

Do you mean we allocate the indirect descriptors after checking the indirect feature in virtio TX queue, instead of allocating it at initialization?
If so, that makes sense, and hence we don't to initialize it in two places.

Quoted 6 lines

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 212

Quoted 3 lines
drivers/net/virtio/virtqueue.c +2 -0
+
+		struct virtio_tx_region *txr;

Don't mix declarations within code.

Quoted 6 lines

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 216

Quoted 4 lines
drivers/net/virtio/virtqueue.c +4 -0
+		txr = txvq->virtio_net_hdr_mz->addr;
+		/* first indirect descriptor is always the tx header */
+		struct vring_packed_desc *start_dp =
+			txr[desc_idx].tx_packed_indir;

Ditto.

Thanks, will update in next version.

Regards,
Xuan

Quoted 16 lines

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 227

Quoted 13 lines
drivers/net/virtio/virtqueue.c +8 -0
+		vring_desc_init_indirect_packed(start_dp,
+		      RTE_DIM(txr[desc_idx].tx_packed_indir));
+		start_dp->addr = txvq->virtio_net_hdr_mem
+			+ desc_idx * sizeof(*txr)
+			+ offsetof(struct virtio_tx_region,
+				   tx_hdr);
+		start_dp->len = vq->hw->vtnet_hdr_size;
+
  	}
 
  	vring_desc_init_packed(vq, size);

Thanks,
Maxime

author
#3 lore raw

Hi Maxime,

Quoted 47 lines

-----Original Message-----
From: Ding, Xuan
Sent: Thursday, September 23, 2021 4:36 PM
To: Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; [email protected]; Liu, Yong <[email protected]>
Subject: RE: [PATCH] net/virtio: fix indirect descriptors reconnection

Hi Maxime,

Quoted 23 lines

-----Original Message-----
From: Maxime Coquelin <[email protected]>
Sent: Thursday, September 23, 2021 4:16 PM
To: Ding, Xuan <[email protected]>; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; [email protected]; Liu, Yong <[email protected]>
Subject: Re: [PATCH] net/virtio: fix indirect descriptors reconnection

Hi Xuan,

Quoted 12 lines · Xuan Ding

Since packed indirect descriptors are added and initialized when initializing vring, the reconnection path also needs to be considered.

Fixes 381f39ebb78a ("net/virtio: fix packed ring indirect descricptors setup")
---
  drivers/net/virtio/virtqueue.c | 14 ++++++++++++++
  1 file changed, 14 insertions(+)
Quoted 12 lines

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 210

Quoted 8 lines
drivers/net/virtio/virtqueue.c +0 -0
index 1f9af3c31b..47229f35c4 100644
@@ -208,6 +208,20 @@ virtqueue_txvq_reset_packed(struct virtqueue *vq)
  			rte_pktmbuf_free(dxp->cookie);
  			dxp->cookie = NULL;
  		}

It makes me think we might save quite some bytes by not allocating indirect descriptors when feature is not negotiated, but it might have a cost in term of performance.

Quoted 4 lines

Do you mean we allocate the indirect descriptors after checking the indirect feature in virtio TX queue,
instead of allocating it at initialization?
If so, that makes sense, and hence we don't to initialize it in two places.

Sorry for replying myself, as for performance, I can have a try and test the impact on performance.

Thanks,
Xuan

Quoted 35 lines
Quoted 7 lines

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 212

Quoted 4 lines
drivers/net/virtio/virtqueue.c +2 -0
+
+		struct virtio_tx_region *txr;

Don't mix declarations within code.

Quoted 6 lines

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 216

Quoted 4 lines
drivers/net/virtio/virtqueue.c +4 -0
+		txr = txvq->virtio_net_hdr_mz->addr;
+		/* first indirect descriptor is always the tx header */
+		struct vring_packed_desc *start_dp =
+			txr[desc_idx].tx_packed_indir;

Ditto.

Thanks, will update in next version.

Regards,
Xuan

Quoted 16 lines

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 227

Quoted 13 lines
drivers/net/virtio/virtqueue.c +8 -0
+		vring_desc_init_indirect_packed(start_dp,
+		      RTE_DIM(txr[desc_idx].tx_packed_indir));
+		start_dp->addr = txvq->virtio_net_hdr_mem
+			+ desc_idx * sizeof(*txr)
+			+ offsetof(struct virtio_tx_region,
+				   tx_hdr);
+		start_dp->len = vq->hw->vtnet_hdr_size;
+
  	}
 
  	vring_desc_init_packed(vq, size);

Thanks,
Maxime

#4 lore raw
Quoted 49 lines · Xuan

Hi Maxime,

Quoted 47 lines

-----Original Message-----
From: Ding, Xuan
Sent: Thursday, September 23, 2021 4:36 PM
To: Maxime Coquelin <[email protected]>; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; [email protected]; Liu, Yong <[email protected]>
Subject: RE: [PATCH] net/virtio: fix indirect descriptors reconnection

Hi Maxime,

Quoted 35 lines

-----Original Message-----
From: Maxime Coquelin <[email protected]>
Sent: Thursday, September 23, 2021 4:16 PM
To: Ding, Xuan <[email protected]>; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; [email protected]; Liu, Yong <[email protected]>
Subject: Re: [PATCH] net/virtio: fix indirect descriptors reconnection

Hi Xuan,

Quoted 17 lines · Xuan Ding

Since packed indirect descriptors are added and initialized when initializing vring, the reconnection path also needs to be considered.

Fixes 381f39ebb78a ("net/virtio: fix packed ring indirect descricptors setup")
---
   drivers/net/virtio/virtqueue.c | 14 ++++++++++++++
   1 file changed, 14 insertions(+)
drivers/net/virtio/virtqueue.c +0 -0
index 1f9af3c31b..47229f35c4 100644

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 210

Quoted 3 lines
drivers/net/virtio/virtqueue.c +0 -0
@@ -208,6 +208,20 @@ virtqueue_txvq_reset_packed(struct virtqueue *vq)
   			rte_pktmbuf_free(dxp->cookie);
   			dxp->cookie = NULL;
   		}

It makes me think we might save quite some bytes by not allocating indirect descriptors when feature is not negotiated, but it might have a cost in term of performance.

Quoted 7 lines
Quoted 4 lines

Do you mean we allocate the indirect descriptors after checking the indirect feature in virtio TX queue,
instead of allocating it at initialization?
If so, that makes sense, and hence we don't to initialize it in two places.

Sorry for replying myself, as for performance, I can have a try and test the impact on performance.

Thanks for looking at this, but maybe that could be done later, after having the fix merged first if you don't have the bandwidth?

Maxime

Quoted 39 lines

Thanks,
Xuan

Quoted 35 lines
Quoted 7 lines

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 212

Quoted 4 lines
drivers/net/virtio/virtqueue.c +2 -0
+
+		struct virtio_tx_region *txr;

Don't mix declarations within code.

Quoted 6 lines

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 216

Quoted 4 lines
drivers/net/virtio/virtqueue.c +4 -0
+		txr = txvq->virtio_net_hdr_mz->addr;
+		/* first indirect descriptor is always the tx header */
+		struct vring_packed_desc *start_dp =
+			txr[desc_idx].tx_packed_indir;

Ditto.

Thanks, will update in next version.

Regards,
Xuan

Quoted 16 lines

drivers/net/virtio/virtqueue.c virtqueue_txvq_reset_packed(struct virtqueue *vq) · line 227

Quoted 13 lines
drivers/net/virtio/virtqueue.c +8 -0
+		vring_desc_init_indirect_packed(start_dp,
+		      RTE_DIM(txr[desc_idx].tx_packed_indir));
+		start_dp->addr = txvq->virtio_net_hdr_mem
+			+ desc_idx * sizeof(*txr)
+			+ offsetof(struct virtio_tx_region,
+				   tx_hdr);
+		start_dp->len = vq->hw->vtnet_hdr_size;
+
   	}
 
   	vring_desc_init_packed(vq, size);

Thanks,
Maxime

Tested-by on v1 1/1
#5 lore raw
In reply to Xuan Ding
Quoted 17 lines

-----Original Message-----
From: Ding, Xuan <[email protected]>
Sent: 2021?8?19? 13:35
To: [email protected]; Xia, Chenbo <[email protected]>
Cc: [email protected]; Hu, Jiayu <[email protected]>; Wang, Yinan <[email protected]>; Ding, Xuan <[email protected]>; [email protected]; Liu, Yong <[email protected]>
Subject: [PATCH] net/virtio: fix indirect descriptors reconnection

Since packed indirect descriptors are added and initialized when initializing vring, the reconnection path also needs to be considered.

Fixes 381f39ebb78a ("net/virtio: fix packed ring indirect descricptors setup")
author
#6 lore raw

PATCH v2 1/1 net/virtio: fix indirect descriptors reconnection

In reply to Xuan Ding

Add initialization for packed ring indirect descriptors in reconnection path.

Fixes 381f39ebb78a ("net/virtio: fix packed ring indirect descricptors setup")

---

v2:
* Fix the position of some declarations.

---
 drivers/net/virtio/virtqueue.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/net/virtio/virtqueue.c +14 -0
index 1f9af3c31b..65bf792eb0 100644
@@ -186,6 +186,8 @@ virtqueue_txvq_reset_packed(struct virtqueue *vq)
 	struct vq_desc_extra *dxp;
 	struct virtnet_tx *txvq;
 	uint16_t desc_idx;
+	struct virtio_tx_region *txr;
+	struct vring_packed_desc *start_dp;
 
 	vq->vq_used_cons_idx = 0;
 	vq->vq_desc_head_idx = 0;
@@ -198,6 +200,7 @@ virtqueue_txvq_reset_packed(struct virtqueue *vq)
 	vq->vq_packed.event_flags_shadow = 0;
 
 	txvq = &vq->txq;
+	txr = txvq->virtio_net_hdr_mz->addr;
 	memset(txvq->mz->addr, 0, txvq->mz->len);
 	memset(txvq->virtio_net_hdr_mz->addr, 0,
 		txvq->virtio_net_hdr_mz->len);
@@ -208,6 +211,17 @@ virtqueue_txvq_reset_packed(struct virtqueue *vq)
 			rte_pktmbuf_free(dxp->cookie);
 			dxp->cookie = NULL;
 		}
+
+		if (virtio_with_feature(vq->hw, VIRTIO_RING_F_INDIRECT_DESC)) {
+			/* first indirect descriptor is always the tx header */
+			start_dp = txr[desc_idx].tx_packed_indir;
+			vring_desc_init_indirect_packed(start_dp,
+							RTE_DIM(txr[desc_idx].tx_packed_indir));
+			start_dp->addr = txvq->virtio_net_hdr_mem
+					 + desc_idx * sizeof(*txr)
+					 + offsetof(struct virtio_tx_region, tx_hdr);
+			start_dp->len = vq->hw->vtnet_hdr_size;
+		}
 	}
 
 	vring_desc_init_packed(vq, size);
Signature · 1 line
2.17.1
Reviewed-by on v2 1/1
#7 lore raw
Quoted 9 lines · Xuan Ding

Add initialization for packed ring indirect descriptors in reconnection path.

Fixes 381f39ebb78a ("net/virtio: fix packed ring indirect descricptors setup")
Quoted 7 lines

---

v2:
* Fix the position of some declarations.

---
  drivers/net/virtio/virtqueue.c | 14 ++++++++++++++
  1 file changed, 14 insertions(+)

Thanks!
Maxime

#8 lore raw
In reply to Xuan Ding
Quoted 9 lines · Xuan Ding

Add initialization for packed ring indirect descriptors in reconnection path.

Fixes 381f39ebb78a ("net/virtio: fix packed ring indirect descricptors setup")
Quoted 8 lines

---

v2:
* Fix the position of some declarations.

---
  drivers/net/virtio/virtqueue.c | 14 ++++++++++++++
  1 file changed, 14 insertions(+)

Applied to dpdk-next-virtio/main.

Thanks,
Maxime