net/virtio: fix vectorized Rx queue stuck
PATCH 1/1 net/virtio: fix vectorized Rx queue stuck
When Rx burst size >= Rx queue size, all descriptors in used queue consumed without rearm, the next Rx burst found no new packets and returned directly without rearm as well.
This patch rearms available queue at once after rx_burst to avoid vq hungry.
--- drivers/net/virtio/virtio_rxtx_simple_altivec.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_neon.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_sse.c | 12 ++++++------ 3 files changed, 18 insertions(+), 18 deletions(-)
drivers/net/virtio/virtio_rxtx_simple_altivec.c +6 -6
@@ -102,12 +102,6 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,rte_prefetch0 (rused); -if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { -virtio_rxq_rearm_vec (rxvq); -if (unlikely (virtqueue_kick_prepare (vq))) -virtqueue_notify (vq); - } - nb_total = nb_used; ref_rx_pkts = rx_pkts; for (nb_pkts_received = 0;
@@ -204,5 +198,11 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,for (nb_used =0 ; nb_used < nb_pkts_received; nb_used++)virtio_update_packet_stats (&rxvq->stats, ref_rx_pkts[nb_used]); +if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { +virtio_rxq_rearm_vec (rxvq); +if (unlikely (virtqueue_kick_prepare (vq))) +virtqueue_notify (vq); + } +return nb_pkts_received; }
drivers/net/virtio/virtio_rxtx_simple_neon.c +6 -6
@@ -100,12 +100,6 @@ virtio_recv_pkts_vec(void *rx_queue,rte_prefetch_non_temporal (rused); -if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { -virtio_rxq_rearm_vec (rxvq); -if (unlikely (virtqueue_kick_prepare (vq))) -virtqueue_notify (vq); - } - nb_total = nb_used; ref_rx_pkts = rx_pkts; for (nb_pkts_received = 0;
@@ -210,5 +204,11 @@ virtio_recv_pkts_vec(void *rx_queue,for (nb_used =0 ; nb_used < nb_pkts_received; nb_used++)virtio_update_packet_stats (&rxvq->stats, ref_rx_pkts[nb_used]); +if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { +virtio_rxq_rearm_vec (rxvq); +if (unlikely (virtqueue_kick_prepare (vq))) +virtqueue_notify (vq); + } +return nb_pkts_received; }
drivers/net/virtio/virtio_rxtx_simple_sse.c +6 -6
@@ -100,12 +100,6 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,rte_prefetch0 (rused); -if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { -virtio_rxq_rearm_vec (rxvq); -if (unlikely (virtqueue_kick_prepare (vq))) -virtqueue_notify (vq); - } - nb_total = nb_used; ref_rx_pkts = rx_pkts; for (nb_pkts_received = 0;
@@ -194,5 +188,11 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,for (nb_used =0 ; nb_used < nb_pkts_received; nb_used++)virtio_update_packet_stats (&rxvq->stats, ref_rx_pkts[nb_used]); +if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { +virtio_rxq_rearm_vec (rxvq); +if (unlikely (virtqueue_kick_prepare (vq))) +virtqueue_notify (vq); + } +return nb_pkts_received; }
Signature · 1 line
2.25.1
+@谢华伟(此时此刻） <[email protected]>
Quoted 119 lines
-----Original Message-----
From: Xueming Li <[email protected]>
Sent: Wednesday, April 14, 2021 12:27 PM
Cc: [email protected]; Xueming(Steven) Li <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; Maxime Coquelin <[email protected]>; Chenbo Xia <[email protected]>; Jerin Jacob <[email protected]>; Ruifeng Wang <[email protected]>; Bruce Richardson <[email protected]>; Konstantin Ananyev <[email protected]>; Jianfeng Tan <[email protected]>; Jianbo Liu <[email protected]>; Yuanhan Liu <[email protected]>
Subject: [PATCH] net/virtio: fix vectorized Rx queue stuckWhen Rx burst size >= Rx queue size, all descriptors in used queue consumed without rearm, the next Rx burst found no new packets and returned directly without rearm as well.
This patch rearms available queue at once after
rx_burstto avoid vq hungry.Fixes fc3d66212fed ("virtio: add vector Rx")Fixes 2d7c37194ee4 ("net/virtio: add NEON based Rx handler")Fixes 52b5a707e6ca ("net/virtio: add Altivec Rx")Signed-off-by Xueming Li <[email protected]>--- drivers/net/virtio/virtio_rxtx_simple_altivec.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_neon.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_sse.c | 12 ++++++------ 3 files changed, 18 insertions(+), 18 deletions(-)drivers/net/virtio/virtio_rxtx_simple_altivec.c +6
-6@@ -102,12 +102,6 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,rte_prefetch0 (rused); -if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { -virtio_rxq_rearm_vec (rxvq); -if (unlikely (virtqueue_kick_prepare (vq))) -virtqueue_notify (vq); - } - nb_total = nb_used; ref_rx_pkts = rx_pkts; for (nb_pkts_received = 0;@@ -204,5 +198,11 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,for (nb_used =0 ; nb_used < nb_pkts_received; nb_used++)virtio_update_packet_stats (&rxvq->stats, ref_rx_pkts[nb_used]); +if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { +virtio_rxq_rearm_vec (rxvq); +if (unlikely (virtqueue_kick_prepare (vq))) +virtqueue_notify (vq); + } +return nb_pkts_received; }drivers/net/virtio/virtio_rxtx_simple_neon.c +6
-6@@ -100,12 +100,6 @@ virtio_recv_pkts_vec(void *rx_queue,rte_prefetch_non_temporal (rused); -if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { -virtio_rxq_rearm_vec (rxvq); -if (unlikely (virtqueue_kick_prepare (vq))) -virtqueue_notify (vq); - } - nb_total = nb_used; ref_rx_pkts = rx_pkts; for (nb_pkts_received = 0;@@ -210,5 +204,11 @@ virtio_recv_pkts_vec(void *rx_queue,for (nb_used =0 ; nb_used < nb_pkts_received; nb_used++)virtio_update_packet_stats (&rxvq->stats, ref_rx_pkts[nb_used]); +if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { +virtio_rxq_rearm_vec (rxvq); +if (unlikely (virtqueue_kick_prepare (vq))) +virtqueue_notify (vq); + } +return nb_pkts_received; }drivers/net/virtio/virtio_rxtx_simple_sse.c +6
-7@@ -100,12 +100,6 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,rte_prefetch0 (rused); -if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { -virtio_rxq_rearm_vec (rxvq); -if (unlikely (virtqueue_kick_prepare (vq))) -virtqueue_notify (vq); - } - nb_total = nb_used; ref_rx_pkts = rx_pkts; for (nb_pkts_received = 0;@@ -194,5 +188,11 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,for (nb_used =0 ; nb_used < nb_pkts_received; nb_used++)virtio_update_packet_stats (&rxvq->stats, ref_rx_pkts[nb_used]); +if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { +virtio_rxq_rearm_vec (rxvq); +if (unlikely (virtqueue_kick_prepare (vq))) +virtqueue_notify (vq); + } +return nb_pkts_received; } --2.25.1
PATCH 1/1 net/virtio: fix vectorized Rx queue stuck
In reply to Xueming LiFrom: ".Xueming Li" <[email protected]>
When Rx queue worked in vectorized mode and rxd <= 512, under traffic of high PPS rate, testpmd often start and receive packets of rxd without further growth.
Testpmd started with rxq flush which tried to rx MAX_PKT_BURST(512) packets and drop. When Rx burst size >= Rx queue size, all descriptors in used queue consumed without rearm, device can't receive more packets. The next Rx burst returned at once since no used descriptors found, rearm logic was skipped, rx vq kept in starving state.
To avoid rx vq starving, this patch always check the available queue, rearm if needed even no used descriptor reported by device.
--- drivers/net/virtio/virtio_rxtx_simple_altivec.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_neon.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_sse.c | 12 ++++++------ 3 files changed, 18 insertions(+), 18 deletions(-)
drivers/net/virtio/virtio_rxtx_simple_altivec.c +6 -6
@@ -85,6 +85,12 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,if (unlikely (nb_pkts <RTE_VIRTIO_DESC_PER_LOOP ))return 0 ; +if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { +virtio_rxq_rearm_vec (rxvq); +if (unlikely (virtqueue_kick_prepare (vq))) +virtqueue_notify (vq); + } + nb_used =virtqueue_nused (vq);rte_compiler_barrier ();
@@ -102,12 +108,6 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,rte_prefetch0 (rused); -if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { -virtio_rxq_rearm_vec (rxvq); -if (unlikely (virtqueue_kick_prepare (vq))) -virtqueue_notify (vq); - } - nb_total = nb_used; ref_rx_pkts = rx_pkts; for (nb_pkts_received = 0;
drivers/net/virtio/virtio_rxtx_simple_neon.c +6 -6
@@ -84,6 +84,12 @@ virtio_recv_pkts_vec(void *rx_queue,if (unlikely (nb_pkts <RTE_VIRTIO_DESC_PER_LOOP ))return 0 ; +if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { +virtio_rxq_rearm_vec (rxvq); +if (unlikely (virtqueue_kick_prepare (vq))) +virtqueue_notify (vq); + } +/* virtqueue_nused has a load-acquire or rte_io_rmb inside */ nb_used =virtqueue_nused (vq);
@@ -100,12 +106,6 @@ virtio_recv_pkts_vec(void *rx_queue,rte_prefetch_non_temporal (rused); -if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { -virtio_rxq_rearm_vec (rxvq); -if (unlikely (virtqueue_kick_prepare (vq))) -virtqueue_notify (vq); - } - nb_total = nb_used; ref_rx_pkts = rx_pkts; for (nb_pkts_received = 0;
drivers/net/virtio/virtio_rxtx_simple_sse.c +6 -6
@@ -85,6 +85,12 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,if (unlikely (nb_pkts <RTE_VIRTIO_DESC_PER_LOOP ))return 0 ; +if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { +virtio_rxq_rearm_vec (rxvq); +if (unlikely (virtqueue_kick_prepare (vq))) +virtqueue_notify (vq); + } + nb_used =virtqueue_nused (vq);if (unlikely (nb_used ==0 ))
@@ -100,12 +106,6 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,rte_prefetch0 (rused); -if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { -virtio_rxq_rearm_vec (rxvq); -if (unlikely (virtqueue_kick_prepare (vq))) -virtqueue_notify (vq); - } - nb_total = nb_used; ref_rx_pkts = rx_pkts; for (nb_pkts_received = 0;
Signature · 1 line
2.25.1
Quoted 48 lines
When Rx queue worked in vectorized mode and rxd <= 512, under traffic of high PPS rate, testpmd often start and receive packets of rxd without further growth.
Testpmd started with rxq flush which tried to rx
MAX_PKT_BURST(512)packets and drop. When Rx burst size >= Rx queue size, all descriptors in used queue consumed without rearm, device can't receive more packets. The next Rx burst returned at once since no used descriptors found, rearm logic was skipped, rx vq kept in starving state.To avoid rx vq starving, this patch always check the available queue, rearm if needed even no used descriptor reported by device.
Fixes fc3d66212fed ("virtio: add vector Rx")Cc 谢华伟(此时此刻） <[email protected]>Fixes 2d7c37194ee4 ("net/virtio: add NEON based Rx handler")Fixes 52b5a707e6ca ("net/virtio: add Altivec Rx")Signed-off-by Xueming Li <[email protected]>--- drivers/net/virtio/virtio_rxtx_simple_altivec.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_neon.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_sse.c | 12 ++++++------ 3 files changed, 18 insertions(+), 18 deletions(-)drivers/net/virtio/virtio_rxtx_simple_altivec.c +6
-0@@ -85,6 +85,12 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,if (unlikely (nb_pkts <RTE_VIRTIO_DESC_PER_LOOP ))return 0 ; +if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { +virtio_rxq_rearm_vec (rxvq); +if (unlikely (virtqueue_kick_prepare (vq))) +virtqueue_notify (vq); + } + nb_used =virtqueue_nused (vq);rte_compiler_barrier ();@@ -102,12 +108,6 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,rte_prefetch0 (rused);
drivers/net/virtio/virtio_rxtx_simple_altivec.c virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, · line 113
Quoted 10 lines
drivers/net/virtio/virtio_rxtx_simple_altivec.c +0
-6-if (vq->vq_free_cnt >=RTE_VIRTIO_VPMD_RX_REARM_THRESH ) { -virtio_rxq_rearm_vec (rxvq); -if (unlikely (virtqueue_kick_prepare (vq))) -virtqueue_notify (vq); - } - nb_total = nb_used; ref_rx_pkts = rx_pkts; for (nb_pkts_received = 0;
Quoted 23 lines · Xueming Li
From: ".Xueming Li" <[email protected]>
When Rx queue worked in vectorized mode and rxd <= 512, under traffic of high PPS rate, testpmd often start and receive packets of rxd without further growth.
Testpmd started with rxq flush which tried to rx
MAX_PKT_BURST(512)packets and drop. When Rx burst size >= Rx queue size, all descriptors in used queue consumed without rearm, device can't receive more packets. The next Rx burst returned at once since no used descriptors found, rearm logic was skipped, rx vq kept in starving state.To avoid rx vq starving, this patch always check the available queue, rearm if needed even no used descriptor reported by device.
Fixes fc3d66212fed ("virtio: add vector Rx")Cc 谢华伟(此时此刻） <[email protected]>Fixes 2d7c37194ee4 ("net/virtio: add NEON based Rx handler")Fixes 52b5a707e6ca ("net/virtio: add Altivec Rx")
Quoted 7 lines
Signed-off-by Xueming Li <[email protected]>--- drivers/net/virtio/virtio_rxtx_simple_altivec.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_neon.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_sse.c | 12 ++++++------ 3 files changed, 18 insertions(+), 18 deletions(-)
Thanks,
Maxime
Maxime Coquelin
AppliedQuoted 23 lines · Xueming Li
From: ".Xueming Li" <[email protected]>
When Rx queue worked in vectorized mode and rxd <= 512, under traffic of high PPS rate, testpmd often start and receive packets of rxd without further growth.
Testpmd started with rxq flush which tried to rx
MAX_PKT_BURST(512)packets and drop. When Rx burst size >= Rx queue size, all descriptors in used queue consumed without rearm, device can't receive more packets. The next Rx burst returned at once since no used descriptors found, rearm logic was skipped, rx vq kept in starving state.To avoid rx vq starving, this patch always check the available queue, rearm if needed even no used descriptor reported by device.
Fixes fc3d66212fed ("virtio: add vector Rx")Cc 谢华伟(此时此刻） <[email protected]>Fixes 2d7c37194ee4 ("net/virtio: add NEON based Rx handler")Fixes 52b5a707e6ca ("net/virtio: add Altivec Rx")
Quoted 7 lines
Signed-off-by Xueming Li <[email protected]>--- drivers/net/virtio/virtio_rxtx_simple_altivec.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_neon.c | 12 ++++++------ drivers/net/virtio/virtio_rxtx_simple_sse.c | 12 ++++++------ 3 files changed, 18 insertions(+), 18 deletions(-)
Applied to dpdk-next-virtio/main.
Thanks,
Maxime