Skip to contentSkip to conversation

Fixes in virtio vectorized functions

PATCH Applied Thierry Herbelot 3 participants 14 messages Sep 11, 2019 – Oct 16, 2019 Network drivers

lore
Share & tools
lore
#0 lore raw

cover Fixes in virtio vectorized functions

Two fixes in the vectorized functions of virtio.

Thibaut Collet (2):
virtio: fix rx stats with vectorized functions
virtio: get all pending rx packets with vectorized functions

 drivers/net/virtio/virtio_rxtx.c             |  2 +-
 drivers/net/virtio/virtio_rxtx.h             |  2 ++
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 10 ++++++++--
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 10 ++++++++--
 4 files changed, 19 insertions(+), 5 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/2 virtio: fix rx stats with vectorized functions

From: Thibaut Collet <[email protected]>

With vectorized functions, only the rx stats for number of packets is incremented.
Update also the other statistics.
Performance impact is about 2%

Fixes fc3d66212fed ("virtio: add vector Rx")
---
 drivers/net/virtio/virtio_rxtx.c             | 2 +-
 drivers/net/virtio/virtio_rxtx.h             | 2 ++
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 5 +++++
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 5 +++++
 4 files changed, 13 insertions(+), 1 deletion(-)
drivers/net/virtio/virtio_rxtx.c +1 -1
index 27ead19fbe81..6dd62bf51863 100644
@@ -1083,7 +1083,7 @@ virtio_discard_rxbuf_inorder(struct virtqueue *vq, struct rte_mbuf *m)
 	}
 }
 
-static inline void
+void
 virtio_update_packet_stats(struct virtnet_stats *stats, struct rte_mbuf *mbuf)
 {
 	uint32_t s = mbuf->pkt_len;
drivers/net/virtio/virtio_rxtx.h +2 -0
index 685cc4f8104c..1eb8dae227ee 100644
@@ -59,5 +59,7 @@ struct virtnet_ctl {
 };
 
 int virtio_rxq_vec_setup(struct virtnet_rx *rxvq);
+void virtio_update_packet_stats(struct virtnet_stats *stats,
+				struct rte_mbuf *mbuf);
 
 #endif /* _VIRTIO_RXTX_H_ */
drivers/net/virtio/virtio_rxtx_simple_neon.c +5 -0
index cdc2a4d28ed5..70e89fc428e0 100644
@@ -47,6 +47,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 
 	uint8x16_t shuf_msk1 = {
@@ -105,6 +106,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		uint64x2_t desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -204,5 +206,8 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++)
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);
+
 	return nb_pkts_received;
 }
drivers/net/virtio/virtio_rxtx_simple_sse.c +5 -0
index af76708d66ae..cb1610e71563 100644
@@ -48,6 +48,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 	__m128i shuf_msk1, shuf_msk2, len_adjust;
 
@@ -107,6 +108,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		__m128i desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -190,5 +192,8 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++)
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);
+
 	return nb_pkts_received;
 }
Signature · 1 line
2.20.1
Reviewed-by on v1 1/2
#2 lore raw

Re: [dpdk-dev] [PATCH 1/2] virtio: fix rx stats with vectorized functions

Quoted 13 lines · Thierry Herbelot

From: Thibaut Collet <[email protected]>

With vectorized functions, only the rx stats for number of packets is incremented.
Update also the other statistics.
Performance impact is about 2%

Fixes fc3d66212fed ("virtio: add vector Rx")
Quoted 8 lines
---
 drivers/net/virtio/virtio_rxtx.c             | 2 +-
 drivers/net/virtio/virtio_rxtx.h             | 2 ++
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 5 +++++
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 5 +++++
 4 files changed, 13 insertions(+), 1 deletion(-)

Thanks,
Maxime

#3 lore raw

PATCH 1/1 net/virtio: fix Rx stats with altivec vectorized functions

In reply to Thierry Herbelot

With vectorized functions, only the rx stats for number of packets is incremented.
Update also the other statistics.
Performance impact is about 2%

---
 drivers/net/virtio/virtio_rxtx_simple_altivec.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/net/virtio/virtio_rxtx_simple_altivec.c +5 -0
index 1cd732dec..f856396ab 100644
@@ -49,6 +49,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 	const vector unsigned char zero = {0};
 
@@ -108,6 +109,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		vector unsigned char desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -199,5 +201,8 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++)
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);
+
 	return nb_pkts_received;
 }
Signature · 1 line
2.18.1
#4 lore raw

PATCH 2/2 virtio: get all pending rx packets with vectorized functions

In reply to Thierry Herbelot

From: Thibaut Collet <[email protected]>

The loop to read packets does not take all packets as the number of available packets (nb_used) is decremented in the loop. Take all available packets provides a performance improvement of 3%.

---
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 5 +++--
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 5 +++--
 2 files changed, 6 insertions(+), 4 deletions(-)
drivers/net/virtio/virtio_rxtx_simple_neon.c +3 -2
index 70e89fc428e0..992e71f010eb 100644
@@ -42,7 +42,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct virtnet_rx *rxvq = rx_queue;
 	struct virtqueue *vq = rxvq->vq;
 	struct virtio_hw *hw = vq->hw;
-	uint16_t nb_used;
+	uint16_t nb_used, nb_total;
 	uint16_t desc_idx;
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
@@ -106,9 +106,10 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
-		nb_pkts_received < nb_used;) {
+		nb_pkts_received < nb_total;) {
 		uint64x2_t desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		uint64x2_t mbp[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		uint64x2_t pkt_mb[RTE_VIRTIO_DESC_PER_LOOP];
drivers/net/virtio/virtio_rxtx_simple_sse.c +3 -2
index cb1610e71563..f9ec4ae69986 100644
@@ -43,7 +43,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct virtnet_rx *rxvq = rx_queue;
 	struct virtqueue *vq = rxvq->vq;
 	struct virtio_hw *hw = vq->hw;
-	uint16_t nb_used;
+	uint16_t nb_used, nb_total;
 	uint16_t desc_idx;
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
@@ -108,9 +108,10 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
-		nb_pkts_received < nb_used;) {
+		nb_pkts_received < nb_total;) {
 		__m128i desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		__m128i mbp[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		__m128i pkt_mb[RTE_VIRTIO_DESC_PER_LOOP];
Signature · 1 line
2.20.1
Reviewed-by on v1 2/2
#5 lore raw

Re: [dpdk-dev] [PATCH 2/2] virtio: get all pending rx packets with vectorized functions

Quoted 14 lines · Thierry Herbelot

From: Thibaut Collet <[email protected]>

The loop to read packets does not take all packets as the number of available packets (nb_used) is decremented in the loop. Take all available packets provides a performance improvement of 3%.

Quoted 6 lines
---
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 5 +++--
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 5 +++--
 2 files changed, 6 insertions(+), 4 deletions(-)

Thanks,
Maxime

#6 lore raw

PATCH 1/1 net/virtio: fix Rx stats with altivec vectorized functions

In reply to Thierry Herbelot

With vectorized functions, only the rx stats for number of packets is incremented.
Update also the other statistics.
Performance impact is about 2%

---
 drivers/net/virtio/virtio_rxtx_simple_altivec.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/net/virtio/virtio_rxtx_simple_altivec.c +5 -0
index 1cd732dec..f856396ab 100644
@@ -49,6 +49,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 	const vector unsigned char zero = {0};
 
@@ -108,6 +109,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		vector unsigned char desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -199,5 +201,8 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++)
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);
+
 	return nb_pkts_received;
 }
Signature · 1 line
2.18.1
Reviewed-by on v1 1/1
#7 lore raw

Re: [dpdk-dev] [PATCH] net/virtio: fix Rx stats with altivec vectorized functions

Thanks David for the fix,

Quoted 11 lines · David Christensen

With vectorized functions, only the rx stats for number of packets is incremented.
Update also the other statistics.
Performance impact is about 2%

The commit message seems like a copy-paste, so claim about performance impact may be not true, same for the Fixes line, which points to a commit when altivec was not here yet.

I'll fix that when appyling:

Thanks,
Maxime

Quoted 35 lines
---
 drivers/net/virtio/virtio_rxtx_simple_altivec.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/net/virtio/virtio_rxtx_simple_altivec.c +5 -0
index 1cd732dec..f856396ab 100644
@@ -49,6 +49,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
 	struct rte_mbuf **sw_ring_end;
+	struct rte_mbuf **ref_rx_pkts;
 	uint16_t nb_pkts_received = 0;
 	const vector unsigned char zero = {0};
 
@@ -108,6 +109,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
 		nb_pkts_received < nb_used;) {
 		vector unsigned char desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
@@ -199,5 +201,8 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	vq->vq_used_cons_idx += nb_pkts_received;
 	vq->vq_free_cnt += nb_pkts_received;
 	rxvq->stats.packets += nb_pkts_received;
+	for (nb_used = 0; nb_used < nb_pkts_received; nb_used++)
+		virtio_update_packet_stats(&rxvq->stats, ref_rx_pkts[nb_used]);
+
 	return nb_pkts_received;
 }
#8 lore raw

Re: [dpdk-dev] [PATCH] net/virtio: fix Rx stats with altivec vectorized functions

In reply to David Christensen
Quoted 12 lines · David Christensen

With vectorized functions, only the rx stats for number of packets is incremented.
Update also the other statistics.
Performance impact is about 2%

Quoted 5 lines
---
 drivers/net/virtio/virtio_rxtx_simple_altivec.c | 5 +++++
 1 file changed, 5 insertions(+)

Applied to dpdk-next-virtio/master.

Thanks,
Maxime

#9 lore raw

PATCH 1/1 net/virtio: get all pending Rx packets in altivec vectorized paths

In reply to Thierry Herbelot

The loop to read packets does not take all packets as the number of available packets (nb_used) is decremented in the loop. Take all available packets provides a performance improvement of 3%.

---
 drivers/net/virtio/virtio_rxtx_simple_altivec.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio_rxtx_simple_altivec.c +3 -2
index f856396ab..47225f412 100644
@@ -44,7 +44,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct virtnet_rx *rxvq = rx_queue;
 	struct virtqueue *vq = rxvq->vq;
 	struct virtio_hw *hw = vq->hw;
-	uint16_t nb_used;
+	uint16_t nb_used, nb_total;
 	uint16_t desc_idx;
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
@@ -109,9 +109,10 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
-		nb_pkts_received < nb_used;) {
+		nb_pkts_received < nb_total;) {
 		vector unsigned char desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		vector unsigned char mbp[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		vector unsigned char pkt_mb[RTE_VIRTIO_DESC_PER_LOOP];
Signature · 1 line
2.18.1
Reviewed-by on v1 1/1
#10 lore raw

Re: [dpdk-dev] [PATCH] net/virtio: get all pending Rx packets in altivec vectorized paths

Quoted 4 lines · David Christensen

The loop to read packets does not take all packets as the number of available packets (nb_used) is decremented in the loop. Take all available packets provides a performance improvement of 3%.

Quoted 9 lines

Same comments as for other patch, I'll fix that whiile applying.

Thanks!
Maxime

Quoted 31 lines
---
 drivers/net/virtio/virtio_rxtx_simple_altivec.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/net/virtio/virtio_rxtx_simple_altivec.c +3 -2
index f856396ab..47225f412 100644
@@ -44,7 +44,7 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct virtnet_rx *rxvq = rx_queue;
 	struct virtqueue *vq = rxvq->vq;
 	struct virtio_hw *hw = vq->hw;
-	uint16_t nb_used;
+	uint16_t nb_used, nb_total;
 	uint16_t desc_idx;
 	struct vring_used_elem *rused;
 	struct rte_mbuf **sw_ring;
@@ -109,9 +109,10 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 			virtqueue_notify(vq);
 	}
 
+	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
-		nb_pkts_received < nb_used;) {
+		nb_pkts_received < nb_total;) {
 		vector unsigned char desc[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		vector unsigned char mbp[RTE_VIRTIO_DESC_PER_LOOP / 2];
 		vector unsigned char pkt_mb[RTE_VIRTIO_DESC_PER_LOOP];
#11 lore raw

Re: [dpdk-dev] [PATCH] net/virtio: get all pending Rx packets in altivec vectorized paths

In reply to David Christensen
Quoted 13 lines · David Christensen

The loop to read packets does not take all packets as the number of available packets (nb_used) is decremented in the loop. Take all available packets provides a performance improvement of 3%.

Quoted 5 lines
---
 drivers/net/virtio/virtio_rxtx_simple_altivec.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

Applied to dpdk-next-virtio/master.

Thanks,
Maxime

#12 lore raw
In reply to Thierry Herbelot

Hi David,

Quoted 6 lines · Thierry Herbelot

Two fixes in the vectorized functions of virtio.

Thibaut Collet (2):
virtio: fix rx stats with vectorized functions
virtio: get all pending rx packets with vectorized functions

Quoted 6 lines
 drivers/net/virtio/virtio_rxtx.c             |  2 +-
 drivers/net/virtio/virtio_rxtx.h             |  2 ++
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 10 ++++++++--
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 10 ++++++++--
 4 files changed, 19 insertions(+), 5 deletions(-)

Could you do the same fix for the new Power vectorized Rx path?

Thanks,
Maxime

#13 lore raw
In reply to Thierry Herbelot
Quoted 6 lines · Thierry Herbelot

Two fixes in the vectorized functions of virtio.

Thibaut Collet (2):
virtio: fix rx stats with vectorized functions
virtio: get all pending rx packets with vectorized functions

Quoted 6 lines
 drivers/net/virtio/virtio_rxtx.c             |  2 +-
 drivers/net/virtio/virtio_rxtx.h             |  2 ++
 drivers/net/virtio/virtio_rxtx_simple_neon.c | 10 ++++++++--
 drivers/net/virtio/virtio_rxtx_simple_sse.c  | 10 ++++++++--
 4 files changed, 19 insertions(+), 5 deletions(-)

Applied to dpdk-next-virtio/master.

Thanks,
Maxime