net/af_xdp: free mbuf when allocate Tx queue fails
PATCH 1/1 net/af_xdp: free mbuf when allocate Tx queue fails
When we fail to allocate enough slots in tx queue for transmitting packets, we need to free the corresponding mbufs.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +6 -2
@@ -276,7 +276,8 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); -return 0 ; + nb_pkts =0 ; + goto out; }for (i =0 ; i < nb_pkts; i++) {
@@ -296,7 +297,6 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) valid++; tx_bytes += mbuf->pkt_len; } -rte_pktmbuf_free (mbuf); }xsk_ring_prod__submit (&txq->tx, nb_pkts);
@@ -311,6 +311,10 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) txq->stats.tx_pkts += valid; txq->stats.tx_bytes += tx_bytes; + out: +for (i =0 ; i < nb_pkts; i++) +rte_pktmbuf_free (bufs[i]); +return nb_pkts; }
Signature · 1 line
2.17.1
Quoted 3 lines · Xiaolong Ye
When we fail to allocate enough slots in tx queue for transmitting packets, we need to free the corresponding mbufs.
You'd better let the application retry on its own...
Quoted 13 lines
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, · line 283
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +2
-1@@ -276,7 +276,8 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); -return 0 ; + nb_pkts =0 ; + goto out;
...and free back the buffers you got from the umem ring here.
}
Quoted 3 lines
for (i = 0; i < nb_pkts; i++) {Signature · 1 line
David Marchand
Hi, David
Quoted 7 lines · David Marchand
Quoted 3 lines · Xiaolong Ye
When we fail to allocate enough slots in tx queue for transmitting packets, we need to free the corresponding mbufs.
You'd better let the application retry on its own...
Make sense.
Quoted 17 lines
Quoted 17 lines
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0@@ -276,7 +276,8 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 282
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +2
-1if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); -return 0 ; + nb_pkts =0 ; + goto out;...and free back the buffers you got from the umem ring here.
Agree, will send a new version.
Thanks,
Xiaolong
Quoted 9 lines
}Quoted 3 lines
for (i = 0; i < nb_pkts; i++) {Signature · 1 line
David Marchand
PATCH 1/1 net/af_xdp: enqueue buf ring when allocate Tx queue fails
In reply to Xiaolong YeWhen it fails to allocate enough slots in Tx queue for transmitting packets, we need to return the dequeued addrs to buf ring.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/af_xdp/rte_eth_af_xdp.c +1 -0
@@ -276,6 +276,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_pkts,NULL );return 0 ; }
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH] net/af_xdp: enqueue buf ring when allocate Tx queue fails
Quoted 26 lines · Xiaolong Ye
When it fails to allocate enough slots in Tx queue for transmitting packets, we need to return the dequeued addrs to buf ring.
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 1 + 1 file changed, 1 insertion(+)drivers/net/af_xdp/rte_eth_af_xdp.c +1
-0@@ -276,6 +276,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_pkts,NULL );return 0; }
Quoted 3 lines
--
2.17.1
Looks good to me.
But I have an additional question.
After the for loop that only picks the mbufs it can copy to the xdp desc, is it normal to call xsk_ring_prod__submit(&txq->tx, nb_pkts); rather than with valid count ?
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH] net/af_xdp: enqueue buf ring when allocate Tx queue fails
Quoted 35 lines · David Marchand
Quoted 26 lines · Xiaolong Ye
When it fails to allocate enough slots in Tx queue for transmitting packets, we need to return the dequeued addrs to buf ring.
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 1 + 1 file changed, 1 insertion(+)drivers/net/af_xdp/rte_eth_af_xdp.c +1
-0@@ -276,6 +276,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_pkts,NULL );return 0; }Quoted 3 lines
--
2.17.1Looks good to me.
But I have an additional question.
Quoted 3 lines
After the for loop that only picks the mbufs it can copy to the xdp desc, is it normal to call
xsk_ring_prod__submit(&txq->tx, nb_pkts); rather than with valid count ?
Good catch, I think it needs to submit valid count other than nb_pkts, and return back (nb_pkts - count) addrs to buf ring.
I'll send a fix patch for it.
Thanks,
Xiaolong
Quoted 3 lines
Signature · 2 lines
-- David Marchand
PATCH 1/1 net/af_xdp: submit valid count to Tx queue
In reply to Xiaolong YeSince tx callback only picks mbufs that can be copied to the xdp desc, it should call xsk_ring_prod__submit with valid count other than nb_pkts.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +1 -1
@@ -300,7 +300,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)rte_pktmbuf_free (mbuf); } -xsk_ring_prod__submit (&txq->tx, nb_pkts); +xsk_ring_prod__submit (&txq->tx, valid);kick_tx (txq);
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH] net/af_xdp: submit valid count to Tx queue
Quoted 17 lines · Xiaolong Ye
Since tx callback only picks mbufs that can be copied to the xdp desc, it should call
xsk_ring_prod__submitwith valid count other thannb_pkts.Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Reported-by David Marchand <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, · line 305
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-1@@ -300,7 +300,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)rte_pktmbuf_free (mbuf ); } -xsk_ring_prod__submit (&txq->tx, nb_pkts); +xsk_ring_prod__submit (&txq->tx, valid);
Err, well, I think we will have an issue here.
Taking from the 5.1.0-rc4 sources I have:
static inline __u32 xsk_prod_nb_free(struct xsk_ring_prod *r, __u32 nb) {
__u32 free_entries = r->cached_cons - r->cached_prod;
if (free_entries >= nb)
return free_entries;
/* Refresh the local tail pointer.
* cached_cons is r->size bigger than the real consumer pointer so
* that this addition can be avoided in the more frequently
* executed code that computs free_entries in the beginning of
* this function. Without this optimization it whould have been
* free_entries = r->cached_prod - r->cached_cons + r->size. */
r->cached_cons = *r->consumer + r->size;
return r->cached_cons - r->cached_prod;
}
static inline size_t xsk_ring_prod__reserve(struct xsk_ring_prod *prod,
size_t nb, __u32 *idx)
{
if (unlikely(xsk_prod_nb_free(prod, nb) < nb))
return 0;
*idx = prod->cached_prod;
prod->cached_prod += nb;
return nb;
}static inline void xsk_ring_prod__submit(struct xsk_ring_prod *prod, size_t nb)
{
/* Make sure everything has been written to the ring before
signalling
* this to the kernel.
*/
smp_wmb();
*prod->producer += nb;
}If we reserve N slots, but only submit n slots, we end up with an incorrect opinion of the number of available slots later.
Either xsk_ring_prod__submit should also update cached_prod (but I am not sure it was the intent of this api), or we must ensure that both reserve and submit are consistent.
Did I miss some subtility ?
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH] net/af_xdp: submit valid count to Tx queue
Quoted 75 lines · David Marchand
Quoted 17 lines · Xiaolong Ye
Since tx callback only picks mbufs that can be copied to the xdp desc, it should call
xsk_ring_prod__submitwith valid count other thannb_pkts.Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Reported-by David Marchand <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, · line 305
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-1@@ -300,7 +300,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)rte_pktmbuf_free (mbuf ); } -xsk_ring_prod__submit (&txq->tx, nb_pkts); +xsk_ring_prod__submit (&txq->tx, valid);Err, well, I think we will have an issue here.
Taking from the 5.1.0-rc4 sources I have:
static inline
__u32xsk_prod_nb_free(struct xsk_ring_prod *r, __u32 nb){
__u32free_entries=r->cached_cons-r->cached_prod;if (
free_entries>= nb)
returnfree_entries;/* Refresh the local tail pointer.
*cached_consisr->sizebigger than the real consumer pointer so
* that this addition can be avoided in the more frequently
* executed code that computsfree_entriesin the beginning of
* this function. Without this optimization it whould have been
*free_entries=r->cached_prod-r->cached_cons+r->size. */
r->cached_cons=*r->consumer+r->size;return
r->cached_cons-r->cached_prod;
}static inline
size_txsk_ring_prod__reserve(structxsk_ring_prod*prod,size_t nb, __u32 *idx) { if (unlikely(xsk_prod_nb_free(prod, nb) < nb)) return 0; *idx = prod->cached_prod; prod->cached_prod += nb; return nb; }static inline void
xsk_ring_prod__submit(struct xsk_ring_prod *prod, size_t nb){ /* Make sure everything has been written to the ring before signalling * this to the kernel. */ smp_wmb(); *prod->producer += nb; }
Quoted 5 lines
If we reserve N slots, but only submit n slots, we end up with an incorrect opinion of the number of available slots later.
Eitherxsk_ring_prod__submitshould also updatecached_prod(but I am not sure it was the intent of this api), or we must ensure that both reserve and submit are consistent.
I think you are right, current design does have the flaw, I haven't thought of it before :( So in order to make sure both reserve and submit are consistent, what about we check the valid count of mbuf at the beginning, then reserve the valid count slots?
Thanks,
Xiaolong
Quoted 6 lines
Did I miss some subtility ?
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH] net/af_xdp: submit valid count to Tx queue
Hello Xiaolong,
Quoted 8 lines · Ye Xiaolong
Quoted 1 line · David Marchand
If we reserve N slots, but only submit n slots, we end up with an
incorrect
Quoted 4 lines
opinion of the number of available slots later.
Eitherxsk_ring_prod__submitshould also updatecached_prod(but I am not sure it was the intent of this api), or we must ensure that both reserve and submit are consistent.
Quoted 9 lines
I think you are right, current design does have the flaw, I haven't thought of
it before :( So in order to make sure both reserve and submit are consistent, what
about we check the valid count of mbuf at the beginning, then reserve the valid
count slots?
Ok, I can see other places to inspect in the driver: reserve_fill_queue() for the same issue, and eth_af_xdp_rx() for a similar issue but with xsk_ring_cons__peek()/xsk_ring_cons__release() ?
Validating the needed slots count before reserving/peeking in the prod/cons rings seems the most simple fix.
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH] net/af_xdp: submit valid count to Tx queue
Quoted 21 lines · David Marchand
Hello Xiaolong,
Quoted 8 lines · Ye Xiaolong
Quoted 1 line · David Marchand
If we reserve N slots, but only submit n slots, we end up with an
incorrect
Quoted 4 lines
opinion of the number of available slots later.
Eitherxsk_ring_prod__submitshould also updatecached_prod(but I am not sure it was the intent of this api), or we must ensure that both reserve and submit are consistent.Quoted 9 lines
I think you are right, current design does have the flaw, I haven't thought of
it before :( So in order to make sure both reserve and submit are consistent, what
about we check the valid count of mbuf at the beginning, then reserve the valid
count slots?
Quoted 5 lines
Ok, I can see other places to inspect in the driver:
reserve_fill_queue()for the same issue, andeth_af_xdp_rx()for a similar issue but with xsk_ring_cons__peek()/xsk_ring_cons__release() ?
Validating the needed slots count before reserving/peeking in the prod/cons rings seems the most simple fix.
Ok, then I'll handle them in a following patch.
Thanks,
Xiaolong
Quoted 4 lines
Signature · 2 lines
-- David Marchand
PATCH v2 1/2 net/af_xdp: enqueue buf ring when allocate Tx queue fails
In reply to David MarchandWhen it fails to allocate enough slots in Tx queue for transmitting packets, we need to return the dequeued addrs to buf ring.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/af_xdp/rte_eth_af_xdp.c +1 -0
@@ -276,6 +276,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_pkts,NULL );return 0 ; }
Signature · 1 line
2.17.1
PATCH v2 2/2 net/af_xdp: make reserve/submit peek/release consistent
As David pointed out, if we reserve N slots, but only submit n slots, we would end up with an incorrect opinion of the number of available slots later, we also would get wrong idx when we call xsk_ring_prod__reserve next time. It also applies to xsk_ring_cons__peek()/xsk_ring_cons__release().
This patch ensures that both reserve/submit and peek/release are consistent.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 80 +++++++++++++++-------------- 1 file changed, 41 insertions(+), 39 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +41 -39
@@ -138,22 +138,19 @@ reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size) {struct xsk_ring_prod *fq = &umem->fq;uint32_t idx; -int i, ret; - - ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); -if (unlikely (!ret)) { -AF_XDP_LOG (ERR ,"Failed to reserve enough fq descs.\n" ); -return ret; - } +int i;for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr;void *addr =NULL ;if (rte_ring_dequeue (umem->buf_ring, &addr)) { - i--;break ; } - fq_addr =xsk_ring_prod__fill_addr (fq, idx++); +if (unlikely (!xsk_ring_prod__reserve (fq,1 , &idx))) { +AF_XDP_LOG (WARNING ,"Failed to reserve 1 fq desc.\n" ); +break ; + } + fq_addr =xsk_ring_prod__fill_addr (fq, idx); *fq_addr = (uint64_t )addr; }
@@ -179,6 +176,9 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE ); +if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, nb_pkts) !=0 )) +return 0 ; + rcvd =xsk_ring_cons__peek (rx, nb_pkts, &idx_rx);if (rcvd ==0 )return 0 ;
@@ -186,9 +186,6 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); -if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, rcvd) !=0 )) -return 0 ; -for (i =0 ; i < rcvd; i++) {const struct xdp_desc *desc;uint64_t addr;
@@ -211,6 +208,10 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)xsk_ring_cons__release (rx, rcvd); +/* free the extra mbufs */ +for (; rcvd < nb_pkts; rcvd++) +rte_pktmbuf_free (mbufs[rcvd]); +/* statistics */ rxq->stats.rx_pkts += (rcvd - dropped); rxq->stats.rx_bytes += rx_bytes;
@@ -261,55 +262,56 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct xsk_umem_info *umem = txq->pair->umem;struct rte_mbuf *mbuf;void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ]; +struct rte_mbuf *valid_bufs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long tx_bytes =0 ; -int i, valid =0 ; +int i; +uint16_t nb_valid =0 ;uint32_t idx_tx; +uint32_t buf_len =ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM ; nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE );pull_umem_cq (umem, nb_pkts); - nb_pkts =rte_ring_dequeue_bulk (umem->buf_ring, addrs, - nb_pkts,NULL ); -if (nb_pkts ==0 ) +for (i =0 ; i < nb_pkts; i++) { +if (bufs[i]->pkt_len <= buf_len) + valid_bufs[nb_valid++] = bufs[i]; +else +rte_pktmbuf_free (bufs[i]); + } + + nb_valid =rte_ring_dequeue_bulk (umem->buf_ring, addrs, + nb_valid,NULL ); +if (nb_valid ==0 )return 0 ; -if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) { +if (xsk_ring_prod__reserve (&txq->tx, nb_valid, &idx_tx) != nb_valid) {kick_tx (txq); -rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_pkts,NULL ); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_valid,NULL );return 0 ; } -for (i =0 ; i < nb_pkts; i++) { +for (i =0 ; i < nb_valid; i++) {struct xdp_desc *desc;void *pkt; -uint32_t buf_len =ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_DATA_HEADROOM ; desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); - mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; - desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->mz->addr, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); - valid++; - tx_bytes += mbuf->pkt_len; - } + mbuf = valid_bufs[i]; + desc->addr = (uint64_t )addrs[i]; + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->mz->addr, + desc->addr); + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + tx_bytes += mbuf->pkt_len;rte_pktmbuf_free (mbuf); } -xsk_ring_prod__submit (&txq->tx, nb_pkts); +xsk_ring_prod__submit (&txq->tx, nb_valid);kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - - txq->stats.err_pkts += nb_pkts - valid; - txq->stats.tx_pkts += valid; + txq->stats.err_pkts += nb_pkts - nb_valid; + txq->stats.tx_pkts += nb_valid; txq->stats.tx_bytes += tx_bytes;return nb_pkts;
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH 2/2] net/af_xdp: make reserve/submit peek/release consistent
Quoted 50 lines · Xiaolong Ye
As David pointed out, if we reserve N slots, but only submit n slots, we would end up with an incorrect opinion of the number of available slots later, we also would get wrong idx when we call
xsk_ring_prod__reservenext time. It also applies to xsk_ring_cons__peek()/xsk_ring_cons__release().This patch ensures that both reserve/submit and peek/release are consistent.
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Reported-by David Marchand <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 80 +++++++++++++++-------------- 1 file changed, 41 insertions(+), 39 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +6
-9@@ -138,22 +138,19 @@ reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size) {struct xsk_ring_prod *fq = &umem->fq;uint32_t idx; -int i, ret; - - ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); -if (unlikely (!ret)) { -AF_XDP_LOG (ERR ,"Failed to reserve enough fq descs.\n" ); -return ret; - } +int i;for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr;void *addr =NULL ;if (rte_ring_dequeue (umem->buf_ring, &addr)) { - i--;break ; } - fq_addr =xsk_ring_prod__fill_addr (fq, idx++); +if (unlikely (!xsk_ring_prod__reserve (fq,1 , &idx))) { + AF_XDP_LOG(WARNING, "Failed to reserve 1 fq desc.\n"); +break ; + } + fq_addr =xsk_ring_prod__fill_addr (fq, idx); *fq_addr = (uint64_t )addr;
Quoted 3 lines
}
I just spotted that reserve_fill_queue always returns 0.
I understand that xsk_configure expects an errors when not succeeding in populating this ring.
And for this, it expects a non zero value for this.
How about something like (neither tested nor compiled):
static inline intreserve_fill_queue(struct xsk_umem_info *umem, int reserve_size)
{
struct xsk_ring_prod *fq = &umem->fq;
void *addrs[reserve_size];
uint32_t idx;
int i, ret;
if (rte_ring_dequeue_bulk(umem->buf_ring, &addrs, reserve_size, NULL)
!= reserve_size) {
AF_XDP_LOG(DEBUG, "Failed to get enough buffers for fq.\n");
return -1;
}
ret = xsk_ring_prod__reserve(fq, reserve_size, &idx);
if (unlikely(!ret)) {
AF_XDP_LOG(DEBUG, "Failed to reserve enough fq descs.\n");
rte_ring_enqueue_bulk(umem->buf_ring, &addrs, reserve_size,
NULL);
return -1;
}
for (i = 0; i < reserve_size; i++) {
__u64 *fq_addr;
fq_addr = xsk_ring_prod__fill_addr(fq, idx++);
*fq_addr = (uint64_t)addrs[i];
}
xsk_ring_prod__submit(fq, reserve_size);
return 0;
}@@ -179,6 +176,9 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs,
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 184
Quoted 12 lines
uint16_tnb_pkts)nb_pkts = RTE_MIN(nb_pkts, ETH_AF_XDP_TX_BATCH_SIZE);+ if (unlikely(rte_pktmbuf_alloc_bulk(rxq->mb_pool, mbufs,
nb_pkts)
!= 0))drivers/net/af_xdp/rte_eth_af_xdp.c +2
-0+return 0 ; + rcvd =xsk_ring_cons__peek (rx, nb_pkts, &idx_rx);if (rcvd ==0 )return 0 ;
When xsk_ring_cons__peek() returns 0, we will leak nb_pkts freshly allocated mbufs.
See below for a suggestion.
@@ -186,9 +186,6 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs,
Quoted 12 lines
uint16_tnb_pkts)
if (xsk_prod_nb_free(fq, free_thresh)>=free_thresh)
(void)reserve_fill_queue(umem,ETH_AF_XDP_RX_BATCH_SIZE);- if (unlikely(rte_pktmbuf_alloc_bulk(rxq->mb_pool, mbufs, rcvd) !=
0))drivers/net/af_xdp/rte_eth_af_xdp.c +0
-2-return 0 ; -for (i =0 ; i < rcvd; i++) {const struct xdp_desc *desc;uint64_t addr;
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, · line 216
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +4
-0@@ -211,6 +208,10 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)xsk_ring_cons__release (rx ,rcvd ); +/* free the extra mbufs */ +for (; rcvd < nb_pkts; rcvd++) +rte_pktmbuf_free (mbufs[rcvd]); +
You can move this block after the statistic update...
/* statistics */
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0rxq->stats.rx_pkts += (rcvd - dropped);rxq->stats.rx_bytes += rx_bytes;
... then define a out: label.
And those mbufs are still clean and coming from a single mempool, we can put them back as a single bulk.
Something like (again, untested):
out:
if (count != nb_pkts) {
rte_mempool_put_bulk(rxq->mb_pool, &mbufs[count],
nb_pkts - count);
}
return count;
}And you would jump to this label when xsk_ring_cons__peek() == 0. What do you think ?
@@ -261,55 +262,56 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs,
Quoted 23 lines
uint16_tnb_pkts)struct xsk_umem_info *umem = txq->pair->umem; struct rte_mbuf *mbuf; void *addrs[ETH_AF_XDP_TX_BATCH_SIZE];drivers/net/af_xdp/rte_eth_af_xdp.c +4
-1+struct rte_mbuf *valid_bufs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long tx_bytes =0 ; -int i, valid =0 ; +int i; +uint16_t nb_valid =0 ;uint32_t idx_tx; + uint32_t buf_len = ETH_AF_XDP_FRAME_SIZE -
ETH_AF_XDP_DATA_HEADROOM;nb_pkts = RTE_MIN(nb_pkts, ETH_AF_XDP_TX_BATCH_SIZE); pull_umem_cq(umem, nb_pkts);drivers/net/af_xdp/rte_eth_af_xdp.c +1
-3- nb_pkts =rte_ring_dequeue_bulk (umem->buf_ring, addrs, - nb_pkts,NULL ); -if (nb_pkts ==0 ) +for (i =0 ; i < nb_pkts; i++) {
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 286
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +9
-0+if (bufs[i]->pkt_len <= buf_len) + valid_bufs[nb_valid++] = bufs[i]; +else +rte_pktmbuf_free (bufs[i]); + } + + nb_valid =rte_ring_dequeue_bulk (umem->buf_ring, addrs, + nb_valid,NULL ); +if (nb_valid ==0 )return 0 ;
You can't return 0.
You have stolen buffers from the caller with the previous check on pktlen. When the application resends this bulk or frees the whole bulk, we will have mbuf reuse bugs.
Thinking about this, why would this happen ?
This limitation should be handled by properly reporting the mtu. The application would then know it can't send those too big mbufs.
If I missed something else and/or if you still don't trust the application, I think the tx burst function should go like as described below.
The first thing to do is check the mbufs length.
At the first invalid length, you break from the loop at index i (i is the invalid buffer index).
Then dequeue i - 1 buffers from buf_ring.
Reserve i - 1 slots in tx.
And return i buffers have been sent (plus a tx error stat += 1).
You need to carefully take into account each step and free the buffer to buf_ring when relevant and free the mbufs properly.
- if (xsk_ring_prod__reserve(&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {
Quoted 59 lines
+ if (
xsk_ring_prod__reserve(&txq->tx, nb_valid, &idx_tx)!=nb_valid) { kick_tx(txq); - rte_ring_enqueue_bulk(umem->buf_ring, addrs, nb_pkts, NULL); + rte_ring_enqueue_bulk(umem->buf_ring, addrs, nb_valid, NULL); return 0; }drivers/net/af_xdp/rte_eth_af_xdp.c +12
-22-for (i =0 ; i < nb_pkts; i++) { +for (i =0 ; i < nb_valid; i++) {struct xdp_desc *desc;void *pkt; -uint32_t buf_len =ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_DATA_HEADROOM ; desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); - mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; - desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->mz->addr, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); - valid++; - tx_bytes += mbuf->pkt_len; - } + mbuf = valid_bufs[i]; + desc->addr = (uint64_t )addrs[i]; + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->mz->addr, + desc->addr); + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + tx_bytes += mbuf->pkt_len;rte_pktmbuf_free (mbuf); } -xsk_ring_prod__submit (&txq->tx, nb_pkts); +xsk_ring_prod__submit (&txq->tx, nb_valid);kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - - txq->stats.err_pkts += nb_pkts - valid; - txq->stats.tx_pkts += valid; + txq->stats.err_pkts += nb_pkts - nb_valid; + txq->stats.tx_pkts += nb_valid; txq->stats.tx_bytes += tx_bytes;return nb_pkts; --2.17.1
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH 2/2] net/af_xdp: make reserve/submit peek/release consistent
Hi, David
Thanks for you detailed review comment.
Quoted 51 lines · David Marchand
Quoted 49 lines · Xiaolong Ye
As David pointed out, if we reserve N slots, but only submit n slots, we would end up with an incorrect opinion of the number of available slots later, we also would get wrong idx when we call
xsk_ring_prod__reservenext time. It also applies to xsk_ring_cons__peek()/xsk_ring_cons__release().This patch ensures that both reserve/submit and peek/release are consistent.
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Reported-by David Marchand <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 80 +++++++++++++++-------------- 1 file changed, 41 insertions(+), 39 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +6
-9@@ -138,22 +138,19 @@ reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size) {struct xsk_ring_prod *fq = &umem->fq;uint32_t idx; -int i, ret; - - ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); -if (unlikely (!ret)) { -AF_XDP_LOG (ERR ,"Failed to reserve enough fq descs.\n" ); -return ret; - } +int i;for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr;void *addr =NULL ;if (rte_ring_dequeue (umem->buf_ring, &addr)) { - i--;break ; } - fq_addr =xsk_ring_prod__fill_addr (fq, idx++); +if (unlikely (!xsk_ring_prod__reserve (fq,1 , &idx))) { + AF_XDP_LOG(WARNING, "Failed to reserve 1 fq desc.\n"); +break ; + } + fq_addr =xsk_ring_prod__fill_addr (fq, idx);
Quoted 8 lines
Quoted 4 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0*fq_addr = (uint64_t )addr;}I just spotted that
reserve_fill_queuealways returns 0.
I understand thatxsk_configureexpects an errors when not succeeding in populating this ring.
And for this, it expects a non zero value for this.
You are right, reserve_fill_queue does need retrun a non zero value when it fails to populate the ring.
Quoted 28 lines
How about something like (neither tested nor compiled):
static inline int
reserve_fill_queue(struct xsk_umem_info *umem, int reserve_size){ struct xsk_ring_prod *fq = &umem->fq; void *addrs[reserve_size]; uint32_t idx; int i, ret; if (rte_ring_dequeue_bulk(umem->buf_ring, &addrs, reserve_size, NULL) != reserve_size) { AF_XDP_LOG(DEBUG, "Failed to get enough buffers for fq.\n"); return -1; } ret = xsk_ring_prod__reserve(fq, reserve_size, &idx); if (unlikely(!ret)) { AF_XDP_LOG(DEBUG, "Failed to reserve enough fq descs.\n"); rte_ring_enqueue_bulk(umem->buf_ring, &addrs, reserve_size, NULL); return -1; } for (i = 0; i < reserve_size; i++) { __u64 *fq_addr;
Quoted 8 lines
fq_addr = xsk_ring_prod__fill_addr(fq, idx++);
*fq_addr = (uint64_t)addrs[i];
}
xsk_ring_prod__submit(fq, reserve_size);
return 0;
}Sounds better, I'll adopt it in my new version.
Quoted 67 lines
(quoted diff) +0
-0@@ -179,6 +176,9 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, > uint16_t nb_pkts)drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 184
Quoted 11 lines
nb_pkts = RTE_MIN(nb_pkts, ETH_AF_XDP_TX_BATCH_SIZE);+ if (unlikely(rte_pktmbuf_alloc_bulk(rxq->mb_pool, mbufs,
nb_pkts)
!= 0))drivers/net/af_xdp/rte_eth_af_xdp.c +2
-0+return 0 ; + rcvd =xsk_ring_cons__peek (rx, nb_pkts, &idx_rx);if (rcvd ==0 )return 0 ;When
xsk_ring_cons__peek()returns 0, we will leaknb_pktsfreshly allocated mbufs.
See below for a suggestion.(quoted diff) +0
-0@@ -186,9 +186,6 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, > uint16_t nb_pkts) > if (xsk_prod_nb_free(fq, free_thresh) >= free_thresh) > (void)reserve_fill_queue(umem, ETH_AF_XDP_RX_BATCH_SIZE);Quoted 9 lines
- if (unlikely(rte_pktmbuf_alloc_bulk(rxq->mb_pool, mbufs, rcvd) !=
0))drivers/net/af_xdp/rte_eth_af_xdp.c +0
-2-return 0 ; -for (i =0 ; i < rcvd; i++) {const struct xdp_desc *desc;uint64_t addr;drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, · line 216
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +4
-0@@ -211,6 +208,10 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)xsk_ring_cons__release (rx ,rcvd ); +/* free the extra mbufs */ +for (; rcvd < nb_pkts; rcvd++) +rte_pktmbuf_free (mbufs[rcvd]); +You can move this block after the statistic update...
/* statistics */Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0rxq->stats.rx_pkts += (rcvd - dropped);rxq->stats.rx_bytes += rx_bytes;... then define a out: label.
And those mbufs are still clean and coming from a single mempool, we can put them back as a single bulk.
Something like (again, untested):out:
if (count != nb_pkts) { rte_mempool_put_bulk(rxq->mb_pool, &mbufs[count], nb_pkts - count); } return count; }
Quoted 2 lines
And you would jump to this label when
xsk_ring_cons__peek()== 0. What do you think ?
I think these are all sensible suggestions, will do.
Quoted 37 lines
(quoted diff) +0
-0@@ -261,55 +262,56 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, > uint16_t nb_pkts) > struct xsk_umem_info *umem = txq->pair->umem; > struct rte_mbuf *mbuf; > void *addrs[ETH_AF_XDP_TX_BATCH_SIZE]; > + struct rte_mbuf *valid_bufs[ETH_AF_XDP_TX_BATCH_SIZE]; > unsigned long tx_bytes = 0; > - int i, valid = 0; > + int i; > + uint16_t nb_valid = 0; > uint32_t idx_tx;Quoted 13 lines
+
uint32_tbuf_len=ETH_AF_XDP_FRAME_SIZE-ETH_AF_XDP_DATA_HEADROOM;nb_pkts = RTE_MIN(nb_pkts, ETH_AF_XDP_TX_BATCH_SIZE); pull_umem_cq(umem, nb_pkts);drivers/net/af_xdp/rte_eth_af_xdp.c +1
-3- nb_pkts =rte_ring_dequeue_bulk (umem->buf_ring, addrs, - nb_pkts,NULL ); -if (nb_pkts ==0 ) +for (i =0 ; i < nb_pkts; i++) {drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 286
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +9
-0+if (bufs[i]->pkt_len <= buf_len) + valid_bufs[nb_valid++] = bufs[i]; +else +rte_pktmbuf_free (bufs[i]); + } + + nb_valid =rte_ring_dequeue_bulk (umem->buf_ring, addrs, + nb_valid,NULL ); +if (nb_valid ==0 )return 0 ;
Quoted 9 lines
You can't return 0.
You have stolen buffers from the caller with the previous check on pktlen. When the application resends this bulk or frees the whole bulk, we will have mbuf reuse bugs.Thinking about this, why would this happen ?
This limitation should be handled by properly reporting the mtu. The application would then know it can't send those too big mbufs.
I think we can rely on mtu to handle the limitation, and assume that in this tx function, all pktlen are valid. Will change in next version.
Thanks,
Xiaolong
Quoted 79 lines
If I missed something else and/or if you still don't trust the application, I think the tx burst function should go like as described below.
The first thing to do is check the mbufs length.
At the first invalid length, you break from the loop at index i (i is the invalid buffer index).
Then dequeue i - 1 buffers frombuf_ring.
Reserve i - 1 slots in tx.And return i buffers have been sent (plus a tx error stat += 1).
You need to carefully take into account each step and free the buffer to
buf_ringwhen relevant and free the mbufs properly.- if (
xsk_ring_prod__reserve(&txq->tx, nb_pkts, &idx_tx)!=nb_pkts) {Quoted 59 lines
+ if (
xsk_ring_prod__reserve(&txq->tx, nb_valid, &idx_tx)!=nb_valid) { kick_tx(txq); - rte_ring_enqueue_bulk(umem->buf_ring, addrs, nb_pkts, NULL); + rte_ring_enqueue_bulk(umem->buf_ring, addrs, nb_valid, NULL); return 0; }drivers/net/af_xdp/rte_eth_af_xdp.c +12
-22-for (i =0 ; i < nb_pkts; i++) { +for (i =0 ; i < nb_valid; i++) {struct xdp_desc *desc;void *pkt; -uint32_t buf_len =ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_DATA_HEADROOM ; desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); - mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; - desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->mz->addr, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); - valid++; - tx_bytes += mbuf->pkt_len; - } + mbuf = valid_bufs[i]; + desc->addr = (uint64_t )addrs[i]; + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->mz->addr, + desc->addr); + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + tx_bytes += mbuf->pkt_len;rte_pktmbuf_free (mbuf); } -xsk_ring_prod__submit (&txq->tx, nb_pkts); +xsk_ring_prod__submit (&txq->tx, nb_valid);kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - - txq->stats.err_pkts += nb_pkts - valid; - txq->stats.tx_pkts += valid; + txq->stats.err_pkts += nb_pkts - nb_valid; + txq->stats.tx_pkts += nb_valid; txq->stats.tx_bytes += tx_bytes;return nb_pkts; --2.17.1
Signature · 1 line
David Marchand
v2 cover some fixes
In reply to David MarchandThis patchset provides some fixes to af_xdp pmd, at first, I just added a simple error handling when Tx queue allocation fails, then David suggested a better way to do it and pointed out the inconsistent issue of reserve/submit ops (for Tx queue) and peek/release ops (for Rx queue), the second patch addressed this.
v2 changes:
- adopt David's suggestion to refactor the code
Xiaolong Ye (2):
net/af_xdp: enqueue buf ring when allocate Tx queue fails net/af_xdp: make reserve/submit peek/release consistent
drivers/net/af_xdp/rte_eth_af_xdp.c | 78 +++++++++++++++-------------- 1 file changed, 40 insertions(+), 38 deletions(-)
Signature · 1 line
2.17.1
PATCH v2 1/2 net/af_xdp: enqueue buf ring when allocate Tx queue fails
When it fails to allocate enough slots in Tx queue for transmitting packets, we need to return the dequeued addrs to buf ring.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/af_xdp/rte_eth_af_xdp.c +1 -0
@@ -276,6 +276,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_pkts,NULL );return 0 ; }
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH v2 1/2] net/af_xdp: enqueue buf ring when allocate Tx queue fails
Quoted 26 lines · Xiaolong Ye
When it fails to allocate enough slots in Tx queue for transmitting packets, we need to return the dequeued addrs to buf ring.
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 1 + 1 file changed, 1 insertion(+)drivers/net/af_xdp/rte_eth_af_xdp.c +1
-0@@ -276,6 +276,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_pkts,NULL );return 0; }
Quoted 4 lines
--
2.17.1
Signature · 1 line
David Marchand
PATCH v2 2/2 net/af_xdp: make reserve/submit peek/release consistent
In reply to Xiaolong YeAs David pointed out, if we reserve N slots for Tx, but only submit n slots, we would end up with an incorrect opinion of the number of available slots later, we also would get wrong idx when we call xsk_ring_prod__reserve next time. It also applies to xsk_ring_cons__peek()/xsk_ring_cons__release().
This patch ensures that both reserve/submit and peek/release are consistent.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 77 +++++++++++++++-------------- 1 file changed, 39 insertions(+), 38 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +39 -38
@@ -134,30 +134,34 @@ static const struct rte_eth_link pmd_link = { };static inline int -reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) +reserve_fill_queue (struct xsk_umem_info *umem,uint16_t reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +void *addrs[reserve_size];uint32_t idx; -int i, ret; +uint16_t i; + +if (rte_ring_dequeue_bulk (umem->buf_ring, addrs, reserve_size,NULL ) + != reserve_size) { +AF_XDP_LOG (DEBUG ,"Failed to get enough buffers for fq.\n" ); +return -1 ; + } - ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); -if (unlikely (!ret)) { +if (unlikely (!xsk_ring_prod__reserve (fq, reserve_size, &idx))) {AF_XDP_LOG (ERR ,"Failed to reserve enough fq descs.\n" ); -return ret; +rte_ring_enqueue_bulk (umem->buf_ring, addrs, + reserve_size,NULL ); +return -1 ; }for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr; -void *addr =NULL ; -if (rte_ring_dequeue (umem->buf_ring, &addr)) { - i--; -break ; - } + fq_addr =xsk_ring_prod__fill_addr (fq, idx++); - *fq_addr = (uint64_t )addr; + *fq_addr = (uint64_t )addrs[i]; } -xsk_ring_prod__submit (fq, i); +xsk_ring_prod__submit (fq, reserve_size);return 0 ; }
@@ -174,21 +178,20 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbufs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long dropped =0 ;unsigned long rx_bytes =0 ; -uint16_t count =0 ;int rcvd, i; nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE ); +if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, nb_pkts) !=0 )) +return 0 ; + rcvd =xsk_ring_cons__peek (rx, nb_pkts, &idx_rx);if (rcvd ==0 ) -return 0 ; + goto out;if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); -if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, rcvd) !=0 )) -return 0 ; -for (i =0 ; i < rcvd; i++) {const struct xdp_desc *desc;uint64_t addr;
@@ -204,7 +207,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)rte_pktmbuf_pkt_len (mbufs[i]) = len;rte_pktmbuf_data_len (mbufs[i]) = len; rx_bytes += len; - bufs[count++] = mbufs[i]; + bufs[i] = mbufs[i];rte_ring_enqueue (umem->buf_ring, (void *)addr); }
@@ -215,7 +218,12 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) rxq->stats.rx_pkts += (rcvd - dropped); rxq->stats.rx_bytes += rx_bytes; -return count; + out: +if (rcvd != nb_pkts) +rte_mempool_put_bulk (rxq->mb_pool, (void **)&mbufs[rcvd], + nb_pkts - rcvd); + +return rcvd; } static void
@@ -262,7 +270,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbuf;void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long tx_bytes =0 ; -int i, valid =0 ; +int i;uint32_t idx_tx; nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE );
@@ -283,20 +291,18 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)for (i =0 ; i < nb_pkts; i++) {struct xdp_desc *desc;void *pkt; -uint32_t buf_len =ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_DATA_HEADROOM ; + desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; - desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->mz->addr, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); - valid++; - tx_bytes += mbuf->pkt_len; - } + + desc->addr = (uint64_t )addrs[i]; + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->mz->addr, + desc->addr); + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + tx_bytes += mbuf->pkt_len; +rte_pktmbuf_free (mbuf); }
@@ -304,12 +310,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - - txq->stats.err_pkts += nb_pkts - valid; - txq->stats.tx_pkts += valid; + txq->stats.tx_pkts += nb_pkts; txq->stats.tx_bytes += tx_bytes;return nb_pkts;
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH v2 2/2] net/af_xdp: make reserve/submit peek/release consistent
Quoted 38 lines · Xiaolong Ye
As David pointed out, if we reserve N slots for Tx, but only submit n slots, we would end up with an incorrect opinion of the number of available slots later, we also would get wrong idx when we call
xsk_ring_prod__reservenext time. It also applies to xsk_ring_cons__peek()/xsk_ring_cons__release().This patch ensures that both reserve/submit and peek/release are consistent.
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Suggested-by David Marchand <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 77 +++++++++++++++-------------- 1 file changed, 39 insertions(+), 38 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +6
-2@@ -134,30 +134,34 @@ static const struct rte_eth_link pmd_link = { };static inline int -reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) +reserve_fill_queue (struct xsk_umem_info *umem,uint16_t reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +void *addrs[reserve_size];uint32_t idx; -int i, ret; +uint16_t i; + +if (rte_ring_dequeue_bulk (umem->buf_ring, addrs, reserve_size,NULL ) + != reserve_size) {
drivers/net/af_xdp/rte_eth_af_xdp.c static const struct rte_eth_link pmd_link = { · line 154
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +4
-2+ AF_XDP_LOG(DEBUG, "Failed to get enough buffers for fq.\n"); +return -1 ; + } - ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); -if (unlikely (!ret)) { +if (unlikely (!xsk_ring_prod__reserve (fq, reserve_size, &idx))) {AF_XDP_LOG (ERR ,"Failed to reserve enough fq descs.\n" );
Is this situation happening a lot ?
If this is the case, I would prefer see this as a DEBUG message rather than ERR.
- return ret;
Quoted 24 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +6
-7+rte_ring_enqueue_bulk (umem->buf_ring, addrs, + reserve_size,NULL ); +return -1 ; }for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr; -void *addr =NULL ; -if (rte_ring_dequeue (umem->buf_ring, &addr)) { - i--; -break ; - } + fq_addr =xsk_ring_prod__fill_addr (fq, idx++); - *fq_addr = (uint64_t )addr; + *fq_addr = (uint64_t )addrs[i]; } -xsk_ring_prod__submit (fq, i); +xsk_ring_prod__submit (fq, reserve_size);return 0 ; }
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, · line 185
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +0
-1@@ -174,21 +178,20 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbufs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long dropped =0 ;unsigned long rx_bytes =0 ; -uint16_t count =0 ;int rcvd, i; nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE );
Nothing to do with your patch, but it should be ETH_AF_XDP_ "R" X_BATCH_SIZE (we have two occurences of this in eth_af_xdp_rx). I can send a follow up patch after yours.
Or you do it, your choice :-)
Quoted 30 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +4
-4+if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, nb_pkts) !=0 )) +return 0 ; + rcvd =xsk_ring_cons__peek (rx, nb_pkts, &idx_rx);if (rcvd ==0 ) -return 0 ; + goto out;if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); -if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, rcvd) != 0)) -return 0 ; -for (i = 0; i < rcvd; i++) { const struct xdp_desc *desc; uint64_t addr;drivers/net/af_xdp/rte_eth_af_xdp.c +1
-1@@ -204,7 +207,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)rte_pktmbuf_pkt_len (mbufs [i]) = len;rte_pktmbuf_data_len (mbufs[i]) = len; rx_bytes += len; - bufs[count++] = mbufs[i]; + bufs[i] = mbufs[i];rte_ring_enqueue (umem->buf_ring, (void *)addr);}
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, · line 223
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-1@@ -215,7 +218,12 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) rxq->stats.rx_pkts += (rcvd - dropped); rxq->stats.rx_bytes += rx_bytes; -return count; + out:
No space before label.
+ if (rcvd != nb_pkts)
Quoted 40 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +5
-1+rte_mempool_put_bulk (rxq->mb_pool, (void **)&mbufs[rcvd], + nb_pkts - rcvd); + +return rcvd; } static void@@ -262,7 +270,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbuf;void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long tx_bytes =0 ; -int i, valid =0 ; +int i;uint32_t idx_tx;nb_pkts = RTE_MIN(nb_pkts, ETH_AF_XDP_TX_BATCH_SIZE);drivers/net/af_xdp/rte_eth_af_xdp.c +2
-12@@ -283,20 +291,18 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)for (i =0 ; i < nb_pkts; i++) {struct xdp_desc *desc;void *pkt; -uint32_t buf_len =ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_DATA_HEADROOM ; + desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; - desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->mz->addr, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); - valid++; - tx_bytes += mbuf->pkt_len; - } +
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, · line 308
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +8
-0+ desc->addr = (uint64_t )addrs[i]; + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->mz->addr, + desc->addr); + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + tx_bytes += mbuf->pkt_len; +rte_pktmbuf_free (mbuf); }
I can see that the buffers in umem can embed ETH_AF_XDP_FRAME_SIZE - ETH_AF_XDP_DATA_HEADROOM bytes.
And this driver does not support multi segment.
So we are missing a check in eth_dev_mtu_set().
The min_mtu / max_mtu fields are not filled in eth_dev_info().
You should fix this in a preparation patch before this change.
@@ -304,12 +310,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs,
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 311
Quoted 10 lines
uint16_tnb_pkts)kick_tx(txq);drivers/net/af_xdp/rte_eth_af_xdp.c +0
-5-if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - - txq->stats.err_pkts += nb_pkts - valid;
err_pkts stats is not used anymore afaics.
- txq->stats.tx_pkts += valid;
Quoted 8 lines
+ txq->stats.tx_pkts += nb_pkts; txq->stats.tx_bytes += tx_bytes; return nb_pkts;--
2.17.1
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v2 2/2] net/af_xdp: make reserve/submit peek/release consistent
Quoted 45 lines · David Marchand
Quoted 43 lines · Xiaolong Ye
As David pointed out, if we reserve N slots for Tx, but only submit n slots, we would end up with an incorrect opinion of the number of available slots later, we also would get wrong idx when we call
xsk_ring_prod__reservenext time. It also applies to xsk_ring_cons__peek()/xsk_ring_cons__release().This patch ensures that both reserve/submit and peek/release are consistent.
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Suggested-by David Marchand <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 77 +++++++++++++++-------------- 1 file changed, 39 insertions(+), 38 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +9
-2@@ -134,30 +134,34 @@ static const struct rte_eth_link pmd_link = { };static inline int -reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) +reserve_fill_queue (struct xsk_umem_info *umem,uint16_t reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +void *addrs[reserve_size];uint32_t idx; -int i, ret; +uint16_t i; + +if (rte_ring_dequeue_bulk (umem->buf_ring, addrs, reserve_size,NULL ) + != reserve_size) { + AF_XDP_LOG(DEBUG, "Failed to get enough buffers for fq.\n"); +return -1 ; + }
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c static const struct rte_eth_link pmd_link = { · line 156
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-2- ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); -if (unlikely (!ret)) { +if (unlikely (!xsk_ring_prod__reserve (fq, reserve_size, &idx))) {AF_XDP_LOG (ERR ,"Failed to reserve enough fq descs.\n" );Is this situation happening a lot ?
If this is the case, I would prefer see this as a DEBUG message rather than ERR.
Agree DEBUG lable is better here.
Quoted 34 lines
- return ret;
Quoted 31 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +6
-8+rte_ring_enqueue_bulk (umem->buf_ring, addrs, + reserve_size,NULL ); +return -1 ; }for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr; -void *addr =NULL ; -if (rte_ring_dequeue (umem->buf_ring, &addr)) { - i--; -break ; - } + fq_addr =xsk_ring_prod__fill_addr (fq, idx++); - *fq_addr = (uint64_t )addr; + *fq_addr = (uint64_t )addrs[i]; } -xsk_ring_prod__submit (fq, i); +xsk_ring_prod__submit (fq, reserve_size);return 0 ; }@@ -174,21 +178,20 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbufs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long dropped =0 ;unsigned long rx_bytes =0 ; -uint16_t count =0 ;int rcvd, i;
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 184
Quoted 2 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE );Nothing to do with your patch, but it should be
ETH_AF_XDP_"R"X_BATCH_SIZE(we have two occurences of this ineth_af_xdp_rx). I can send a follow up patch after yours.
Or you do it, your choice :-)
Good catch, I'll submit a separate patch for this.
Quoted 37 lines
Quoted 37 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +4
-4+if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, nb_pkts) !=0 )) +return 0 ; + rcvd =xsk_ring_cons__peek (rx, nb_pkts, &idx_rx);if (rcvd ==0 ) -return 0 ; + goto out;if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); -if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, rcvd) != 0)) -return 0 ; -for (i = 0; i < rcvd; i++) { const struct xdp_desc *desc; uint64_t addr;drivers/net/af_xdp/rte_eth_af_xdp.c +1
-1@@ -204,7 +207,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)rte_pktmbuf_pkt_len (mbufs [i]) = len;rte_pktmbuf_data_len (mbufs[i]) = len; rx_bytes += len; - bufs[count++] = mbufs[i]; + bufs[i] = mbufs[i];rte_ring_enqueue (umem->buf_ring, (void *)addr);}drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0@@ -215,7 +218,12 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) rxq->stats.rx_pkts += (rcvd - dropped); rxq->stats.rx_bytes += rx_bytes;
Quoted 5 lines
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 228
Quoted 3 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-1-return count; + out:No space before label.
Ouch, will remove this unwanted space.
Quoted 59 lines
+ if (rcvd !=
nb_pkts)Quoted 40 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +5
-1+rte_mempool_put_bulk (rxq->mb_pool, (void **)&mbufs[rcvd], + nb_pkts - rcvd); + +return rcvd; } static void@@ -262,7 +270,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbuf;void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long tx_bytes =0 ; -int i, valid =0 ; +int i;uint32_t idx_tx;nb_pkts = RTE_MIN(nb_pkts, ETH_AF_XDP_TX_BATCH_SIZE);drivers/net/af_xdp/rte_eth_af_xdp.c +2
-12@@ -283,20 +291,18 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)for (i =0 ; i < nb_pkts; i++) {struct xdp_desc *desc;void *pkt; -uint32_t buf_len =ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_DATA_HEADROOM ; + desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; - desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->mz->addr, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); - valid++; - tx_bytes += mbuf->pkt_len; - } +drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, · line 308
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +8
-0+ desc->addr = (uint64_t )addrs[i]; + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->mz->addr, + desc->addr); + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + tx_bytes += mbuf->pkt_len; +rte_pktmbuf_free (mbuf); }I can see that the buffers in umem can embed
ETH_AF_XDP_FRAME_SIZE-ETH_AF_XDP_DATA_HEADROOMbytes.
And this driver does not support multi segment.
So we are missing a check ineth_dev_mtu_set().
Themin_mtu/max_mtufields are not filled ineth_dev_info().
Quoted 1 line
You should fix this in a preparation patch before this change.
Got it, will submit a preparation patch in next version.
Quoted 7 lines
(quoted diff) +0
-0@@ -304,12 +310,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, > uint16_t nb_pkts)Quoted 3 lines
kick_tx(txq);
Quoted 8 lines
drivers/net/af_xdp/rte_eth_af_xdp.c eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) · line 311
Quoted 6 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +0
-5-if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - - txq->stats.err_pkts += nb_pkts - valid;
err_pktsstats is not used anymore afaics.
Will delete it in next version.
Thanks,
Xiaolong
Quoted 15 lines
-
txq->stats.tx_pkts+= valid;Quoted 8 lines
+ txq->stats.tx_pkts += nb_pkts; txq->stats.tx_bytes += tx_bytes; return nb_pkts;--
2.17.1Signature · 1 line
David Marchand
v3 cover some fixes for AF_XDP pmd
In reply to David MarchandThis patchset provides some fixes to af_xdp pmd, at first, I just added a simple error handling when Tx queue allocation fails, then David suggested a better way to do it and pointed out the inconsistent issue of reserve/submit ops (for Tx queue) and peek/release ops (for Rx queue), the third patch addressed this.
v3 changes:
- address David's review comments
- add one patch to specify the mtu range
- add one fix patch for typo
v2 changes:
- adopt David's suggestion to refactor the code
Xiaolong Ye (4):
net/af_xdp: enqueue buf ring when allocate Tx queue fails net/af_xdp: specify minimal and maximal MTU net/af_xdp: make reserve/submit peek/release consistent net/af_xdp: fix typos in Rx function
drivers/net/af_xdp/rte_eth_af_xdp.c | 96 +++++++++++++++++------------ 1 file changed, 55 insertions(+), 41 deletions(-)
Signature · 1 line
2.17.1
PATCH v3 1/4 net/af_xdp: enqueue buf ring when allocate Tx queue fails
When it fails to allocate enough slots in Tx queue for transmitting packets, we need to return the dequeued addrs to buf ring.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/af_xdp/rte_eth_af_xdp.c +1 -0
@@ -276,6 +276,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_pkts,NULL );return 0 ; }
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH v3 1/4] net/af_xdp: enqueue buf ring when allocate Tx queue fails
Quoted 26 lines · Xiaolong Ye
When it fails to allocate enough slots in Tx queue for transmitting packets, we need to return the dequeued addrs to buf ring.
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 1 + 1 file changed, 1 insertion(+)drivers/net/af_xdp/rte_eth_af_xdp.c +1
-0@@ -276,6 +276,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_pkts,NULL );return 0; }
Quoted 3 lines
--
2.17.1
When there is no change, please keep Review tags from previous versions.
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v3 1/4] net/af_xdp: enqueue buf ring when allocate Tx queue fails
Quoted 28 lines · David Marchand
Quoted 26 lines · Xiaolong Ye
When it fails to allocate enough slots in Tx queue for transmitting packets, we need to return the dequeued addrs to buf ring.
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 1 + 1 file changed, 1 insertion(+)drivers/net/af_xdp/rte_eth_af_xdp.c +1
-0@@ -276,6 +276,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_pkts,NULL );return 0; }
Quoted 7 lines
Quoted 3 lines
--
2.17.1When there is no change, please keep Review tags from previous versions.
Reviewed-by David Marchand <[email protected]>
Got it, will add your review tag in next version.
Thanks,
Xiaolong
Quoted 3 lines
Signature · 2 lines
-- David Marchand
PATCH v3 2/4 net/af_xdp: specify minimal and maximal MTU
In reply to Xiaolong YeSince AF_XDP pmd doesn't support multi segment, we need to add a valid check in eth_dev_mtu_set.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
drivers/net/af_xdp/rte_eth_af_xdp.c +12 -0
@@ -351,6 +351,9 @@ eth_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_rx_queues =1 ; dev_info->max_tx_queues =1 ; + dev_info->min_mtu =ETHER_MIN_MTU ; + dev_info->max_mtu =ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM ; + dev_info->default_rxportconf.nb_queues =1 ; dev_info->default_txportconf.nb_queues =1 ; dev_info->default_rxportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ;
@@ -654,6 +657,15 @@ eth_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu)int ret;int s; +if (mtu >ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM || + mtu <ETHER_MIN_MTU ) { +AF_XDP_LOG (ERR ,"Unsupported MTU of %d. " +"max mtu: %d, min mtu: %d" , mtu, +ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM , +ETHER_MIN_MTU ); +return -EINVAL ; + } + s =socket (PF_INET ,SOCK_DGRAM ,0 );if (s <0 )return -EINVAL ;
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH v3 2/4] net/af_xdp: specify minimal and maximal MTU
Quoted 3 lines · Xiaolong Ye
Since
AF_XDPpmd doesn't support multi segment, we need to add a valid check ineth_dev_mtu_set.
How about:
Properly report mtu capability in port device info.
Quoted 10 lines
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0
drivers/net/af_xdp/rte_eth_af_xdp.c eth_dev_info(struct rte_eth_dev *dev, struct · line 359
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +3
-0@@ -351,6 +351,9 @@ eth_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_rx_queues = 1; dev_info->max_tx_queues =1 ; + dev_info->min_mtu =ETHER_MIN_MTU ; + dev_info->max_mtu =ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM ; +
dev_info->default_rxportconf.nb_queues = 1;
Quoted 6 lines
dev_info->default_txportconf.nb_queues = 1; dev_info->default_rxportconf.ring_size = ETH_AF_XDP_DFLT_NUM_DESCS;drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0@@ -654,6 +657,15 @@ eth_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu)int ret;int s;
drivers/net/af_xdp/rte_eth_af_xdp.c eth_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu) · line 669
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +9
-0+if (mtu >ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM || + mtu <ETHER_MIN_MTU ) { +AF_XDP_LOG (ERR ,"Unsupported MTU of %d. " +"max mtu: %d, min mtu: %d" , mtu, +ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM , +ETHER_MIN_MTU ); +return -EINVAL ; + } +
Sorry, I suppose my previous mail was confusing.
If you provide min/max_mtu, ethdev will enforce those checks for you and you don't need to care about it.
See:
https://git.dpdk.org/dpdk/tree/lib/librte_ethdev/rte_ethdev.c?h=v19.05-rc1#n2630
With this block removed, you can add my review tag.
Thanks.
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v3 2/4] net/af_xdp: specify minimal and maximal MTU
Quoted 8 lines · David Marchand
Quoted 3 lines · Xiaolong Ye
Since
AF_XDPpmd doesn't support multi segment, we need to add a valid check ineth_dev_mtu_set.How about:
Properly report mtu capability in port device info.
Sounds better.
Quoted 46 lines
Reported-by David Marchand <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>Quoted 10 lines
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0drivers/net/af_xdp/rte_eth_af_xdp.c eth_dev_info(struct rte_eth_dev *dev, struct · line 359
Quoted 9 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +3
-0@@ -351,6 +351,9 @@ eth_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_rx_queues = 1; dev_info->max_tx_queues =1 ; + dev_info->min_mtu =ETHER_MIN_MTU ; + dev_info->max_mtu =ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM ; +dev_info->default_rxportconf.nb_queues = 1;Quoted 6 lines
dev_info->default_txportconf.nb_queues = 1; dev_info->default_rxportconf.ring_size = ETH_AF_XDP_DFLT_NUM_DESCS;drivers/net/af_xdp/rte_eth_af_xdp.c +0
-0@@ -654,6 +657,15 @@ eth_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu)int ret;int s;drivers/net/af_xdp/rte_eth_af_xdp.c eth_dev_mtu_set(struct rte_eth_dev *dev, uint16_t mtu) · line 669
Quoted 10 lines
drivers/net/af_xdp/rte_eth_af_xdp.c +9
-0+if (mtu >ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM || + mtu <ETHER_MIN_MTU ) { +AF_XDP_LOG (ERR ,"Unsupported MTU of %d. " +"max mtu: %d, min mtu: %d" , mtu, +ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM , +ETHER_MIN_MTU ); +return -EINVAL ; + } +Sorry, I suppose my previous mail was confusing.
If you provide min/max_mtu, ethdev will enforce those checks for you and you don't need to care about it.
Quoted 4 lines
See:
https://git.dpdk.org/dpdk/tree/lib/librte_ethdev/rte_ethdev.c?h=v19.05-rc1#n2630With this block removed, you can add my review tag.
Got it, will remove it in next version.
Thanks,
Xiaolong
Quoted 5 lines
Thanks.
Signature · 1 line
David Marchand
PATCH v3 3/4 net/af_xdp: make reserve/submit peek/release consistent
In reply to Xiaolong YeAs David pointed out, if we reserve N slots for Tx, but only submit n slots, we would end up with an incorrect opinion of the number of available slots later, we also would get wrong idx when we call xsk_ring_prod__reserve next time. It also applies to xsk_ring_cons__peek()/xsk_ring_cons__release().
This patch ensures that both reserve/submit and peek/release are consistent.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 79 +++++++++++++++-------------- 1 file changed, 40 insertions(+), 39 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +40 -39
@@ -134,30 +134,34 @@ static const struct rte_eth_link pmd_link = { };static inline int -reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) +reserve_fill_queue (struct xsk_umem_info *umem,uint16_t reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +void *addrs[reserve_size];uint32_t idx; -int i, ret; +uint16_t i; + +if (rte_ring_dequeue_bulk (umem->buf_ring, addrs, reserve_size,NULL ) + != reserve_size) { +AF_XDP_LOG (DEBUG ,"Failed to get enough buffers for fq.\n" ); +return -1 ; + } - ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); -if (unlikely (!ret)) { -AF_XDP_LOG (ERR ,"Failed to reserve enough fq descs.\n" ); -return ret; +if (unlikely (!xsk_ring_prod__reserve (fq, reserve_size, &idx))) { +AF_XDP_LOG (DEBUG ,"Failed to reserve enough fq descs.\n" ); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, + reserve_size,NULL ); +return -1 ; }for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr; -void *addr =NULL ; -if (rte_ring_dequeue (umem->buf_ring, &addr)) { - i--; -break ; - } + fq_addr =xsk_ring_prod__fill_addr (fq, idx++); - *fq_addr = (uint64_t )addr; + *fq_addr = (uint64_t )addrs[i]; } -xsk_ring_prod__submit (fq, i); +xsk_ring_prod__submit (fq, reserve_size);return 0 ; }
@@ -174,21 +178,20 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbufs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long dropped =0 ;unsigned long rx_bytes =0 ; -uint16_t count =0 ;int rcvd, i; nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE ); +if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, nb_pkts) !=0 )) +return 0 ; + rcvd =xsk_ring_cons__peek (rx, nb_pkts, &idx_rx);if (rcvd ==0 ) -return 0 ; + goto out;if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); -if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, rcvd) !=0 )) -return 0 ; -for (i =0 ; i < rcvd; i++) {const struct xdp_desc *desc;uint64_t addr;
@@ -204,7 +207,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)rte_pktmbuf_pkt_len (mbufs[i]) = len;rte_pktmbuf_data_len (mbufs[i]) = len; rx_bytes += len; - bufs[count++] = mbufs[i]; + bufs[i] = mbufs[i];rte_ring_enqueue (umem->buf_ring, (void *)addr); }
@@ -215,7 +218,12 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) rxq->stats.rx_pkts += (rcvd - dropped); rxq->stats.rx_bytes += rx_bytes; -return count; +out: +if (rcvd != nb_pkts) +rte_mempool_put_bulk (rxq->mb_pool, (void **)&mbufs[rcvd], + nb_pkts - rcvd); + +return rcvd; } static void
@@ -262,7 +270,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbuf;void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long tx_bytes =0 ; -int i, valid =0 ; +int i;uint32_t idx_tx; nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE );
@@ -283,20 +291,18 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)for (i =0 ; i < nb_pkts; i++) {struct xdp_desc *desc;void *pkt; -uint32_t buf_len =ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_DATA_HEADROOM ; + desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; - desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->mz->addr, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); - valid++; - tx_bytes += mbuf->pkt_len; - } + + desc->addr = (uint64_t )addrs[i]; + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->mz->addr, + desc->addr); + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + tx_bytes += mbuf->pkt_len; +rte_pktmbuf_free (mbuf); }
@@ -304,12 +310,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - - txq->stats.err_pkts += nb_pkts - valid; - txq->stats.tx_pkts += valid; + txq->stats.tx_pkts += nb_pkts; txq->stats.tx_bytes += tx_bytes;return nb_pkts;
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH v3 3/4] net/af_xdp: make reserve/submit peek/release consistent
Quoted 183 lines · Xiaolong Ye
As David pointed out, if we reserve N slots for Tx, but only submit n slots, we would end up with an incorrect opinion of the number of available slots later, we also would get wrong idx when we call
xsk_ring_prod__reservenext time. It also applies to xsk_ring_cons__peek()/xsk_ring_cons__release().This patch ensures that both reserve/submit and peek/release are consistent.
Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")Suggested-by David Marchand <[email protected]>Signed-off-by Xiaolong Ye <[email protected]>--- drivers/net/af_xdp/rte_eth_af_xdp.c | 79 +++++++++++++++-------------- 1 file changed, 40 insertions(+), 39 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +17
-13@@ -134,30 +134,34 @@ static const struct rte_eth_link pmd_link = { };static inline int -reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) +reserve_fill_queue (struct xsk_umem_info *umem,uint16_t reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +void *addrs[reserve_size];uint32_t idx; -int i, ret; +uint16_t i; + +if (rte_ring_dequeue_bulk (umem->buf_ring, addrs, reserve_size,NULL ) + != reserve_size) { + AF_XDP_LOG(DEBUG, "Failed to get enough buffers for fq.\n"); +return -1 ; + } - ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); -if (unlikely (!ret)) { -AF_XDP_LOG (ERR ,"Failed to reserve enough fq descs.\n" ); -return ret; +if (unlikely (!xsk_ring_prod__reserve (fq, reserve_size, &idx))) { +AF_XDP_LOG (DEBUG ,"Failed to reserve enough fq descs.\n" ); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, + reserve_size,NULL ); +return -1 ; }for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr; -void *addr =NULL ; -if (rte_ring_dequeue (umem->buf_ring, &addr)) { - i--; -break ; - } + fq_addr =xsk_ring_prod__fill_addr (fq, idx++); - *fq_addr = (uint64_t )addr; + *fq_addr = (uint64_t )addrs[i]; } -xsk_ring_prod__submit (fq, i); +xsk_ring_prod__submit (fq, reserve_size);return 0;}
drivers/net/af_xdp/rte_eth_af_xdp.c +4
-5@@ -174,21 +178,20 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbufs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long dropped =0 ;unsigned long rx_bytes =0 ; -uint16_t count =0 ;int rcvd, i; nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE ); +if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, nb_pkts) !=0 )) +return 0 ; + rcvd =xsk_ring_cons__peek (rx, nb_pkts, &idx_rx);if (rcvd ==0 ) -return 0 ; + goto out;if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); -if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, rcvd) != 0)) -return 0 ; -for (i = 0; i < rcvd; i++) { const struct xdp_desc *desc; uint64_t addr;drivers/net/af_xdp/rte_eth_af_xdp.c +1
-1@@ -204,7 +207,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)rte_pktmbuf_pkt_len (mbufs [i]) = len;rte_pktmbuf_data_len (mbufs[i]) = len; rx_bytes += len; - bufs[count++] = mbufs[i]; + bufs[i] = mbufs[i];rte_ring_enqueue (umem->buf_ring, (void *)addr);}drivers/net/af_xdp/rte_eth_af_xdp.c +6
-1@@ -215,7 +218,12 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) rxq->stats.rx_pkts += (rcvd - dropped); rxq->stats.rx_bytes += rx_bytes; -return count; +out: +if (rcvd != nb_pkts) +rte_mempool_put_bulk (rxq->mb_pool, (void **)&mbufs[rcvd], + nb_pkts - rcvd); + +return rcvd; }static void
drivers/net/af_xdp/rte_eth_af_xdp.c +1
-1@@ -262,7 +270,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbuf;void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long tx_bytes =0 ; -int i, valid =0 ; +int i;uint32_t idx_tx;nb_pkts = RTE_MIN(nb_pkts, ETH_AF_XDP_TX_BATCH_SIZE);drivers/net/af_xdp/rte_eth_af_xdp.c +11
-18@@ -283,20 +291,18 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)for (i =0 ; i < nb_pkts; i++) {struct xdp_desc *desc;void *pkt; -uint32_t buf_len =ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_DATA_HEADROOM ; + desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; - desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->mz->addr, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); - valid++; - tx_bytes += mbuf->pkt_len; - } + + desc->addr = (uint64_t )addrs[i]; + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->mz->addr, + desc->addr); + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + tx_bytes += mbuf->pkt_len; +rte_pktmbuf_free (mbuf); }@@ -304,12 +310,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)kick_tx (txq ); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - - txq->stats.err_pkts += nb_pkts - valid; - txq->stats.tx_pkts += valid; + txq->stats.tx_pkts += nb_pkts; txq->stats.tx_bytes += tx_bytes;return nb_pkts;
Quoted 4 lines
--
2.17.1
Signature · 1 line
David Marchand
PATCH v3 4/4 net/af_xdp: fix typos in Rx function
In reply to Xiaolong YeWe should use ETH_AF_XDP_RX_BATCH_SIZE in Rx function rather than ETH_AF_XDP_TX_BATCH_SIZE.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +2 -2
@@ -175,12 +175,12 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct xsk_ring_prod *fq = &umem->fq;uint32_t idx_rx =0 ;uint32_t free_thresh = fq->size >>1 ; -struct rte_mbuf *mbufs[ETH_AF_XDP_TX_BATCH_SIZE ]; +struct rte_mbuf *mbufs[ETH_AF_XDP_RX_BATCH_SIZE ];unsigned long dropped =0 ;unsigned long rx_bytes =0 ;int rcvd, i; - nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE ); + nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_RX_BATCH_SIZE );if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, nb_pkts) !=0 ))return 0 ;
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH v3 4/4] net/af_xdp: fix typos in Rx function
Quoted 6 lines · Xiaolong Ye
We should use
ETH_AF_XDP_RX_BATCH_SIZEin Rx function rather thanETH_AF_XDP_TX_BATCH_SIZE.Fixes f1debd77efaf ("net/af_xdp: introduce AF_XDP PMD")
Quoted 25 lines
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/net/af_xdp/rte_eth_af_xdp.c +2
-2@@ -175,12 +175,12 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct xsk_ring_prod *fq = &umem->fq;uint32_t idx_rx =0 ;uint32_t free_thresh = fq->size >>1 ; -struct rte_mbuf *mbufs[ETH_AF_XDP_TX_BATCH_SIZE ]; +struct rte_mbuf *mbufs[ETH_AF_XDP_RX_BATCH_SIZE ];unsigned long dropped =0 ;unsigned long rx_bytes =0 ;int rcvd, i; - nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE ); + nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_RX_BATCH_SIZE ); if (unlikely(rte_pktmbuf_alloc_bulk(rxq->mb_pool, mbufs, nb_pkts)!= 0))
Quoted 4 lines
return 0;--
2.17.1
Signature · 1 line
David Marchand
v4 cover some fixes for AF_XDP pmd
In reply to David MarchandThis patchset provides some fixes to af_xdp pmd, at first, I just added a simple error handling when Tx queue allocation fails, then David suggested a better way to do it and pointed out the inconsistent issue of reserve/submit ops (for Tx queue) and peek/release ops (for Rx queue), the third patch addressed this.
v4 changes:
- remove unnecessary mtu valid check in eth_dev_mtu_set
- add Reported-by and Reviewed-by tags of David
v3 changes:
- address David's review comments
- add one patch to specify the mtu range
- add one fix patch for typo
v2 changes:
- adopt David's suggestion to refactor the code
Xiaolong Ye (4):
net/af_xdp: enqueue buf ring when allocate Tx queue fails net/af_xdp: specify minimal and maximal MTU net/af_xdp: make reserve/submit peek/release consistent net/af_xdp: fix typos in Rx function
drivers/net/af_xdp/rte_eth_af_xdp.c | 87 +++++++++++++++-------------- 1 file changed, 46 insertions(+), 41 deletions(-)
Signature · 1 line
2.17.1
PATCH v4 1/4 net/af_xdp: enqueue buf ring when allocate Tx queue fails
When it fails to allocate enough slots in Tx queue for transmitting packets, we need to return the dequeued addrs to buf ring.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/af_xdp/rte_eth_af_xdp.c +1 -0
@@ -276,6 +276,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)if (xsk_ring_prod__reserve (&txq->tx, nb_pkts, &idx_tx) != nb_pkts) {kick_tx (txq); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, nb_pkts,NULL );return 0 ; }
Signature · 1 line
2.17.1
PATCH v4 2/4 net/af_xdp: specify minimal and maximal MTU
In reply to Xiaolong YeProperly report mtu capability in port device info.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/net/af_xdp/rte_eth_af_xdp.c +3 -0
@@ -351,6 +351,9 @@ eth_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_rx_queues =1 ; dev_info->max_tx_queues =1 ; + dev_info->min_mtu =ETHER_MIN_MTU ; + dev_info->max_mtu =ETH_AF_XDP_FRAME_SIZE -ETH_AF_XDP_DATA_HEADROOM ; + dev_info->default_rxportconf.nb_queues =1 ; dev_info->default_txportconf.nb_queues =1 ; dev_info->default_rxportconf.ring_size =ETH_AF_XDP_DFLT_NUM_DESCS ;
Signature · 1 line
2.17.1
PATCH v4 3/4 net/af_xdp: make reserve/submit peek/release consistent
In reply to Xiaolong YeAs David pointed out, if we reserve N slots for Tx, but only submit n slots, we would end up with an incorrect opinion of the number of available slots later, we also would get wrong idx when we call xsk_ring_prod__reserve next time. It also applies to xsk_ring_cons__peek()/xsk_ring_cons__release().
This patch ensures that both reserve/submit and peek/release are consistent.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 79 +++++++++++++++-------------- 1 file changed, 40 insertions(+), 39 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +40 -39
@@ -134,30 +134,34 @@ static const struct rte_eth_link pmd_link = { };static inline int -reserve_fill_queue (struct xsk_umem_info *umem,int reserve_size) +reserve_fill_queue (struct xsk_umem_info *umem,uint16_t reserve_size) {struct xsk_ring_prod *fq = &umem->fq; +void *addrs[reserve_size];uint32_t idx; -int i, ret; +uint16_t i; + +if (rte_ring_dequeue_bulk (umem->buf_ring, addrs, reserve_size,NULL ) + != reserve_size) { +AF_XDP_LOG (DEBUG ,"Failed to get enough buffers for fq.\n" ); +return -1 ; + } - ret =xsk_ring_prod__reserve (fq, reserve_size, &idx); -if (unlikely (!ret)) { -AF_XDP_LOG (ERR ,"Failed to reserve enough fq descs.\n" ); -return ret; +if (unlikely (!xsk_ring_prod__reserve (fq, reserve_size, &idx))) { +AF_XDP_LOG (DEBUG ,"Failed to reserve enough fq descs.\n" ); +rte_ring_enqueue_bulk (umem->buf_ring, addrs, + reserve_size,NULL ); +return -1 ; }for (i =0 ; i < reserve_size; i++) {__u64 *fq_addr; -void *addr =NULL ; -if (rte_ring_dequeue (umem->buf_ring, &addr)) { - i--; -break ; - } + fq_addr =xsk_ring_prod__fill_addr (fq, idx++); - *fq_addr = (uint64_t )addr; + *fq_addr = (uint64_t )addrs[i]; } -xsk_ring_prod__submit (fq, i); +xsk_ring_prod__submit (fq, reserve_size);return 0 ; }
@@ -174,21 +178,20 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbufs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long dropped =0 ;unsigned long rx_bytes =0 ; -uint16_t count =0 ;int rcvd, i; nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE ); +if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, nb_pkts) !=0 )) +return 0 ; + rcvd =xsk_ring_cons__peek (rx, nb_pkts, &idx_rx);if (rcvd ==0 ) -return 0 ; + goto out;if (xsk_prod_nb_free (fq, free_thresh) >= free_thresh) (void )reserve_fill_queue (umem,ETH_AF_XDP_RX_BATCH_SIZE ); -if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, rcvd) !=0 )) -return 0 ; -for (i =0 ; i < rcvd; i++) {const struct xdp_desc *desc;uint64_t addr;
@@ -204,7 +207,7 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)rte_pktmbuf_pkt_len (mbufs[i]) = len;rte_pktmbuf_data_len (mbufs[i]) = len; rx_bytes += len; - bufs[count++] = mbufs[i]; + bufs[i] = mbufs[i];rte_ring_enqueue (umem->buf_ring, (void *)addr); }
@@ -215,7 +218,12 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) rxq->stats.rx_pkts += (rcvd - dropped); rxq->stats.rx_bytes += rx_bytes; -return count; +out: +if (rcvd != nb_pkts) +rte_mempool_put_bulk (rxq->mb_pool, (void **)&mbufs[rcvd], + nb_pkts - rcvd); + +return rcvd; } static void
@@ -262,7 +270,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct rte_mbuf *mbuf;void *addrs[ETH_AF_XDP_TX_BATCH_SIZE ];unsigned long tx_bytes =0 ; -int i, valid =0 ; +int i;uint32_t idx_tx; nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE );
@@ -283,20 +291,18 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)for (i =0 ; i < nb_pkts; i++) {struct xdp_desc *desc;void *pkt; -uint32_t buf_len =ETH_AF_XDP_FRAME_SIZE - -ETH_AF_XDP_DATA_HEADROOM ; + desc =xsk_ring_prod__tx_desc (&txq->tx, idx_tx + i); mbuf = bufs[i]; -if (mbuf->pkt_len <= buf_len) { - desc->addr = (uint64_t )addrs[valid]; - desc->len = mbuf->pkt_len; - pkt =xsk_umem__get_data (umem->mz->addr, - desc->addr); - rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), - desc->len); - valid++; - tx_bytes += mbuf->pkt_len; - } + + desc->addr = (uint64_t )addrs[i]; + desc->len = mbuf->pkt_len; + pkt =xsk_umem__get_data (umem->mz->addr, + desc->addr); + rte_memcpy(pkt, rte_pktmbuf_mtod(mbuf, void *), + desc->len); + tx_bytes += mbuf->pkt_len; +rte_pktmbuf_free (mbuf); }
@@ -304,12 +310,7 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)kick_tx (txq); -if (valid < nb_pkts) -rte_ring_enqueue_bulk (umem->buf_ring, &addrs[valid], - nb_pkts - valid,NULL ); - - txq->stats.err_pkts += nb_pkts - valid; - txq->stats.tx_pkts += valid; + txq->stats.tx_pkts += nb_pkts; txq->stats.tx_bytes += tx_bytes;return nb_pkts;
Signature · 1 line
2.17.1
PATCH v4 4/4 net/af_xdp: fix typos in Rx function
In reply to Xiaolong YeWe should use ETH_AF_XDP_RX_BATCH_SIZE in Rx function rather than ETH_AF_XDP_TX_BATCH_SIZE.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +2 -2
@@ -175,12 +175,12 @@ eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)struct xsk_ring_prod *fq = &umem->fq;uint32_t idx_rx =0 ;uint32_t free_thresh = fq->size >>1 ; -struct rte_mbuf *mbufs[ETH_AF_XDP_TX_BATCH_SIZE ]; +struct rte_mbuf *mbufs[ETH_AF_XDP_RX_BATCH_SIZE ];unsigned long dropped =0 ;unsigned long rx_bytes =0 ;int rcvd, i; - nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_TX_BATCH_SIZE ); + nb_pkts =RTE_MIN (nb_pkts,ETH_AF_XDP_RX_BATCH_SIZE );if (unlikely (rte_pktmbuf_alloc_bulk (rxq->mb_pool, mbufs, nb_pkts) !=0 ))return 0 ;
Signature · 1 line
2.17.1
Re: [dpdk-dev] [PATCH v4 4/4] net/af_xdp: fix typos in Rx function
Re: [dpdk-dev] [PATCH v4 0/4] some fixes for AF_XDP pmd
In reply to Xiaolong YeQuoted 27 lines · Xiaolong Ye
This patchset provides some fixes to
af_xdppmd, at first, I just added a simple error handling when Tx queue allocation fails, then David suggested a better way to do it and pointed out the inconsistent issue of reserve/submit ops (for Tx queue) and peek/release ops (for Rx queue), the third patch addressed this.v4 changes:
- remove unnecessary mtu valid check in
eth_dev_mtu_set
- add Reported-by and Reviewed-by tags of Davidv3 changes:
- address David's review comments
- add one patch to specify the mtu range
- add one fix patch for typov2 changes:
- adopt David's suggestion to refactor the code
Xiaolong Ye (4):
net/af_xdp: enqueue buf ring when allocate Tx queue failsnet/af_xdp: specify minimal and maximal MTUnet/af_xdp: make reserve/submit peek/release consistentnet/af_xdp: fix typos in Rx function
Quoted 7 lines
drivers/net/af_xdp/rte_eth_af_xdp.c | 87 +++++++++++++++-------------- 1 file changed, 46 insertions(+), 41 deletions(-)--
2.17.1
LGTM.
Thanks Xiaolong.
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v4 0/4] some fixes for AF_XDP pmd
Quoted 32 lines · David Marchand
Quoted 30 lines · Xiaolong Ye
This patchset provides some fixes to
af_xdppmd, at first, I just added a simple error handling when Tx queue allocation fails, then David suggested a better way to do it and pointed out the inconsistent issue of reserve/submit ops (for Tx queue) and peek/release ops (for Rx queue), the third patch addressed this.v4 changes:
- remove unnecessary mtu valid check in
eth_dev_mtu_set
- add Reported-by and Reviewed-by tags of Davidv3 changes:
- address David's review comments
- add one patch to specify the mtu range
- add one fix patch for typov2 changes:
- adopt David's suggestion to refactor the code
Xiaolong Ye (4):
net/af_xdp: enqueue buf ring when allocate Tx queue failsnet/af_xdp: specify minimal and maximal MTUnet/af_xdp: make reserve/submit peek/release consistentnet/af_xdp: fix typos in Rx functiondrivers/net/af_xdp/rte_eth_af_xdp.c | 87 +++++++++++++++-------------- 1 file changed, 46 insertions(+), 41 deletions(-)
Quoted 6 lines
Quoted 4 lines
--
2.17.1LGTM.
Thanks Xiaolong.
Thanks a lot for your guidance. :)
Quoted 4 lines
Signature · 2 lines
-- David Marchand
Re: [dpdk-dev] [PATCH v4 0/4] some fixes for AF_XDP pmd
In reply to David MarchandQuoted 33 lines · David Marchand
On Wed, Apr 17, 2019 at 3:55 PM Xiaolong Ye <[email protected] <mailto:[email protected]>> wrote:
This patchset provides some fixes to
af_xdppmd, at first, I just added a simple error handling when Tx queue allocation fails, then David suggested a better way to do it and pointed out the inconsistent issue of reserve/submit ops (for Tx queue) and peek/release ops (for Rx queue), the third patch addressed this.v4 changes:
- remove unnecessary mtu valid check in
eth_dev_mtu_set
- add Reported-by and Reviewed-by tags of Davidv3 changes:
- address David's review comments
- add one patch to specify the mtu range
- add one fix patch for typov2 changes:
- adopt David's suggestion to refactor the code
Xiaolong Ye (4):
net/af_xdp: enqueue buf ring when allocate Tx queue failsnet/af_xdp: specify minimal and maximal MTUnet/af_xdp: make reserve/submit peek/release consistentnet/af_xdp: fix typos in Rx functiondrivers/net/af_xdp/rte_eth_af_xdp.c | 87 +++++++++++++++-------------- 1 file changed, 46 insertions(+), 41 deletions(-)
Quoted 6 lines
--
2.17.1LGTM.
Thanks Xiaolong.
Series applied to dpdk-next-net/master, thanks.