net: fec: fix some issues of ndo_xdp_xmit()
Subsystems (4)
[email protected]
maintainerv2 cover net: fec: fix some issues of ndo_xdp_xmit()
From: Wei Fang <[email protected]>
We encountered some issues when testing the ndo_xdp_xmit() interface of the fec driver on i.MX8MP and i.MX93 platforms. These issues are easy to reproduce, and the specific reproduction steps are as follows.
step1: The ethernet port of a board (board A) is connected to the EQOS port of i.MX8MP/i.MX93, and the FEC port of i.MX8MP/i.MX93 is connected to another ethernet port, such as a switch port.
step2: Board A uses the pktgen_sample03_burst_single_flow.sh to generate and send packets to i.MX8MP/i.MX93. The command is shown below. ./pktgen_sample03_burst_single_flow.sh -i eth0 -d 192.168.6.8 -m \
56:bf:0d:68:b0:9e -s 1500
step3: i.MX8MP/i.MX93 use the xdp_redirect bfp program to redirect the XDP frames from EQOS port to FEC port. The command is shown below. ./xdp_redirect eth1 eth0
After a few moments, the warning or error logs will be printed in the console, for more details, please refer to the commit message of each patch.
Wei Fang (4):
net: fec: dynamically set the NETDEV_XDP_ACT_NDO_XMIT feature of XDP
net: fec: recycle pages for transmitted XDP frames
net: fec: increase the size of tx ring and update tx_wake_threshold
net: fec: use netdev_err_once() instead of netdev_err()
drivers/net/ethernet/freescale/fec.h | 17 ++- drivers/net/ethernet/freescale/fec_main.c | 166 +++++++++++++++------- 2 files changed, 127 insertions(+), 56 deletions(-)
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH v2 1/4 net: fec: dynamically set the NETDEV_XDP_ACT_NDO_XMIT feature of XDP
From: Wei Fang <[email protected]>
When a XDP program is installed or uninstalled, fec_restart() will be invoked to reset MAC and buffer descriptor rings. It's reasonable not to transmit any packet during the process of reset. However, the NETDEV_XDP_ACT_NDO_XMIT bit of xdp_features is enabled by default, that is to say, it's possible that the fec_enet_xdp_xmit() will be invoked even if the process of reset is not finished. In this case, the redirected XDP frames might be dropped and available transmit BDs may be incorrectly deemed insufficient. So this patch disable the NETDEV_XDP_ACT_NDO_XMIT feature by default and dynamically configure this feature when the bpf program is installed or uninstalled.
---
V2 change:
No change.
--- drivers/net/ethernet/freescale/fec_main.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
drivers/net/ethernet/freescale/fec_main.c +9 -4
@@ -3732,12 +3732,18 @@ static int fec_enet_bpf(struct net_device *dev, struct netdev_bpf *bpf)if (fep->quirks &FEC_QUIRK_SWAP_FRAME )return -EOPNOTSUPP ; +if (!bpf->prog) +xdp_features_clear_redirect_target (dev); +if (is_run) {napi_disable (&fep->napi);netif_tx_disable (dev); } old_prog =xchg (&fep->xdp_prog, bpf->prog); +if (old_prog) +bpf_prog_put (old_prog); +fec_restart (dev);if (is_run) {
@@ -3745,8 +3751,8 @@ static int fec_enet_bpf(struct net_device *dev, struct netdev_bpf *bpf)netif_tx_start_all_queues (dev); } -if (old_prog) -bpf_prog_put (old_prog); +if (bpf->prog) +xdp_features_set_redirect_target (dev, false);return 0 ;
@@ -4016,8 +4022,7 @@ static int fec_enet_init(struct net_device *ndev)if (!(fep->quirks &FEC_QUIRK_SWAP_FRAME )) ndev->xdp_features =NETDEV_XDP_ACT_BASIC | -NETDEV_XDP_ACT_REDIRECT | -NETDEV_XDP_ACT_NDO_XMIT ; +NETDEV_XDP_ACT_REDIRECT ;fec_restart (ndev);
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH v2 2/4 net: fec: recycle pages for transmitted XDP frames
In reply to [email protected]From: Wei Fang <[email protected]>
Once the XDP frames have been successfully transmitted through the ndo_xdp_xmit() interface, it's the driver responsibility to free the frames so that the page_pool can recycle the pages and reuse them. However, this action is not implemented in the fec driver. This leads to a user-visible problem that the console will print the following warning log.
[ 157.568851] page_pool_release_retry() stalled pool shutdown 1389 inflight 60 sec [ 217.983446] page_pool_release_retry() stalled pool shutdown 1389 inflight 120 sec [ 278.399006] page_pool_release_retry() stalled pool shutdown 1389 inflight 181 sec [ 338.812885] page_pool_release_retry() stalled pool shutdown 1389 inflight 241 sec [ 399.226946] page_pool_release_retry() stalled pool shutdown 1389 inflight 302 sec
Therefore, to solve this issue, we free XDP frames via xdp_return_frame() while cleaning the tx BD ring.
---
V2 change:
No change.
--- drivers/net/ethernet/freescale/fec.h | 15 ++- drivers/net/ethernet/freescale/fec_main.c | 148 +++++++++++++++------- 2 files changed, 115 insertions(+), 48 deletions(-)
drivers/net/ethernet/freescale/fec.h +14 -1
@@ -544,10 +544,23 @@ enum { XDP_STATS_TOTAL, }; +enum fec_txbuf_type { +FEC_TXBUF_T_SKB , +FEC_TXBUF_T_XDP_NDO , +}; + +struct fec_tx_buffer { +union { +struct sk_buff *skb; +struct xdp_frame *xdp; + }; +enum fec_txbuf_type type; +}; +struct fec_enet_priv_tx_q {struct bufdesc_prop bd;unsigned char *tx_bounce[TX_RING_SIZE ]; -struct sk_buff *tx_skbuff[TX_RING_SIZE ]; +struct fec_tx_buffer tx_buf[TX_RING_SIZE ];unsigned short tx_stop_threshold;unsigned short tx_wake_threshold;
drivers/net/ethernet/freescale/fec_main.c +101 -47
216 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH V2 net 2/4] net: fec: recycle pages for transmitted XDP frames
From: Wei Fang <[email protected]>
Date: Thu, 6 Jul 2023 16:10:10 +0800
Quoted 8 lines
From: Wei Fang <[email protected]>
Once the XDP frames have been successfully transmitted through the
ndo_xdp_xmit()interface, it's the driver responsibility to free the frames so that thepage_poolcan recycle the pages and reuse them. However, this action is not implemented in the fec driver. This leads to a user-visible problem that the console will print the following warning log.
[...]
drivers/net/ethernet/freescale/fec_main.c static void fec_enet_bd_init(struct net_device *dev) · line 977
Quoted 7 lines
drivers/net/ethernet/freescale/fec_main.c +7
-0+if (txq->tx_buf[i].xdp) { +xdp_return_frame (txq->tx_buf[i].xdp); + txq->tx_buf[i].xdp =NULL ; + } + +/* restore default tx buffer type: FEC_TXBUF_T_SKB */ + txq->tx_buf[i].type =FEC_TXBUF_T_SKB ;
Here and in the related places below: maybe set ::type dynamically when sending to either SKB or XDP instead of setting it only for XDP and then restoring each time?
drivers/net/ethernet/freescale/fec_main.c static void fec_enet_bd_init(struct net_device *dev) · line 982
Quoted 5 lines
drivers/net/ethernet/freescale/fec_main.c +1
-0} + bdp->cbd_bufaddr =cpu_to_fec32 (0 ); bdp =fec_enet_get_nextdesc (bdp, &txq->bd); }
[...]
Thanks,
Olek
Wei Fang
maintainerRE: [PATCH V2 net 2/4] net: fec: recycle pages for transmitted XDP frames
Quoted 27 lines
-----Original Message-----
From: Alexander Lobakin <[email protected]>
Sent: 2023年7月6日 19:59
To: Wei Fang <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Shenwei Wang <[email protected]>; Clark Wang <[email protected]>; [email protected]; dl-linux-imx <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH V2 net 2/4] net: fec: recycle pages for transmitted XDP framesFrom: Wei Fang <[email protected]>
Date: Thu, 6 Jul 2023 16:10:10 +0800Quoted 8 lines
From: Wei Fang <[email protected]>
Once the XDP frames have been successfully transmitted through the
ndo_xdp_xmit()interface, it's the driver responsibility to free the frames so that thepage_poolcan recycle the pages and reuse them. However, this action is not implemented in the fec driver. This leads to a user-visible problem that the console will print the following warning log.[...]
Quoted 11 lines
drivers/net/ethernet/freescale/fec_main.c static void fec_enet_bd_init(struct net_device *dev) · line 977
Quoted 7 lines
drivers/net/ethernet/freescale/fec_main.c +7
-0+if (txq->tx_buf[i].xdp) { +xdp_return_frame (txq->tx_buf[i].xdp); + txq->tx_buf[i].xdp =NULL ; + } + +/* restore default tx buffer type: FEC_TXBUF_T_SKB */ + txq->tx_buf[i].type =FEC_TXBUF_T_SKB ;Here and in the related places below: maybe set ::type dynamically when sending to either SKB or XDP instead of setting it only for XDP and then restoring each time?
I also considered this method. but when the skb has frags or needs to be TSO, only the last tx_buf of the skb needs to store the skb pointer, but all the tx_buf of the skb needs to set the type explicitly, I think it is a bit mess and not concise. So I restore the type to default when recycle the BDs. Anyway, it;s just a difference in implement, if you guys insist it's better to set the type explicitly, I will modify the patch. Thanks!
Quoted 10 lines
drivers/net/ethernet/freescale/fec_main.c static void fec_enet_bd_init(struct net_device *dev) · line 982
Quoted 6 lines
drivers/net/ethernet/freescale/fec_main.c +1
-0} + bdp->cbd_bufaddr =cpu_to_fec32 (0 ); bdp =fec_enet_get_nextdesc (bdp, &txq->bd); }[...]
Thanks,
Olek
Re: [PATCH V2 net 2/4] net: fec: recycle pages for transmitted XDP frames
From: Wei Fang <[email protected]>
Date: Fri, 7 Jul 2023 01:54:04 +0000
Quoted 38 lines
Quoted 27 lines
-----Original Message-----
From: Alexander Lobakin <[email protected]>
Sent: 2023年7月6日 19:59
To: Wei Fang <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Shenwei Wang <[email protected]>; Clark Wang <[email protected]>; [email protected]; dl-linux-imx <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH V2 net 2/4] net: fec: recycle pages for transmitted XDP framesFrom: Wei Fang <[email protected]>
Date: Thu, 6 Jul 2023 16:10:10 +0800Quoted 8 lines
From: Wei Fang <[email protected]>
Once the XDP frames have been successfully transmitted through the
ndo_xdp_xmit()interface, it's the driver responsibility to free the frames so that thepage_poolcan recycle the pages and reuse them. However, this action is not implemented in the fec driver. This leads to a user-visible problem that the console will print the following warning log.[...]
Quoted 11 lines
drivers/net/ethernet/freescale/fec_main.c static void fec_enet_bd_init(struct net_device *dev) · line 977
Quoted 7 lines
drivers/net/ethernet/freescale/fec_main.c +7
-0+if (txq->tx_buf[i].xdp) { +xdp_return_frame (txq->tx_buf[i].xdp); + txq->tx_buf[i].xdp =NULL ; + } + +/* restore default tx buffer type: FEC_TXBUF_T_SKB */ + txq->tx_buf[i].type =FEC_TXBUF_T_SKB ;Here and in the related places below: maybe set ::type dynamically when sending to either SKB or XDP instead of setting it only for XDP and then restoring each time?
Quoted 6 lines
I also considered this method. but when the skb has frags or needs to be TSO, only the last
tx_bufof the skb needs to store the skb pointer, but all thetx_bufof the skb needs to set the type explicitly, I think it is a bit mess and not concise. So I restore the type to default when recycle the BDs. Anyway, it;s just a difference in implement, if you guys insist it's better to set the type explicitly, I will modify the patch. Thanks!
Just more of personal preference, no problems. Moreover, your explanation makes sense to me.
Quoted 10 lines
Quoted 10 lines
drivers/net/ethernet/freescale/fec_main.c static void fec_enet_bd_init(struct net_device *dev) · line 982
Quoted 6 lines
drivers/net/ethernet/freescale/fec_main.c +1
-0} + bdp->cbd_bufaddr =cpu_to_fec32 (0 ); bdp =fec_enet_get_nextdesc (bdp, &txq->bd); }[...]
Thanks,
Olek
Thanks,
Olek
[email protected]
maintainerPATCH v2 3/4 net: fec: increase the size of tx ring and update tx_wake_threshold
In reply to [email protected]From: Wei Fang <[email protected]>
When the XDP feature is enabled and with heavy XDP frames to be transmitted, there is a considerable probability that available tx BDs are insufficient. This will lead to some XDP frames to be discarded and the "NOT enough BD for SG!" error log will appear in the console (as shown below).
[ 160.013112] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.023116] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.028926] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.038946] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.044758] fec 30be0000.ethernet eth0: NOT enough BD for SG!
In the case of heavy XDP traffic, sometimes the speed of recycling tx BDs may be slower than the speed of sending XDP frames. There may be several specific reasons, such as the interrupt is not responsed in time, the efficiency of the NAPI callback function is too low due to all the queues (tx queues and rx queues) share the same NAPI, and so on.
After trying various methods, I think that increase the size of tx BD ring is simple and effective. Maybe the best resolution is that allocate NAPI for each queue to improve the efficiency of the NAPI callback, but this change is a bit big and I didn't try this method. Perheps this method will be implemented in a future patch.
This patch also updates the tx_wake_threshold of tx ring which is related to the size of tx ring in the previous logic. Otherwise, the tx_wake_threshold will be too high (403 BDs), which is more likely to impact the slow path in the case of heavy XDP traffic, because XDP path and slow path share the tx BD rings. According to Jakub's suggestion, the tx_wake_threshold is at least equal to tx_stop_threshold + 2 * MAX_SKB_FRAGS, if a queue of hundreds of entries is overflowing, we should be able to apply a hysteresis of a few tens of entries.
---
V2 change:
1. Keep the tx_stop_threshold unchanged and change the tx_wake_threshold to tx_stop_threshold + 2 * MAX_SKB_FRAGS.
2. Split the change, the netdev_err_once() is moved to a separate patch.
3. Modify the commit message and subject ppropriately.
--- drivers/net/ethernet/freescale/fec.h | 2 +- drivers/net/ethernet/freescale/fec_main.c | 3 +-- 2 files changed, 2 insertions(+), 3 deletions(-)
drivers/net/ethernet/freescale/fec.h +1 -1
@@ -355,7 +355,7 @@ struct bufdesc_ex {#define RX_RING_SIZE (FEC_ENET_RX_FRPPG * FEC_ENET_RX_PAGES)#define FEC_ENET_TX_FRSIZE 2048#define FEC_ENET_TX_FRPPG (PAGE_SIZE / FEC_ENET_TX_FRSIZE) -#define TX_RING_SIZE 512/* Must be power of two */ +#define TX_RING_SIZE 1024/* Must be power of two */ #define TX_RING_MOD_MASK 511/* for this to work */ #define BD_ENET_RX_INT 0x00800000
drivers/net/ethernet/freescale/fec_main.c +1 -2
@@ -3349,8 +3349,7 @@ static int fec_enet_alloc_queue(struct net_device *ndev) fep->total_tx_ring_size += fep->tx_queue[i]->bd.ring_size; txq->tx_stop_threshold =FEC_MAX_SKB_DESCS ; - txq->tx_wake_threshold = - (txq->bd.ring_size - txq->tx_stop_threshold) /2 ; + txq->tx_wake_threshold =FEC_MAX_SKB_DESCS +2 *MAX_SKB_FRAGS ; txq->tso_hdrs = dma_alloc_coherent(&fep->pdev->dev, txq->bd.ring_size * TSO_HEADER_SIZE,
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH v2 4/4 net: fec: use netdev_err_once() instead of netdev_err()
In reply to [email protected]From: Wei Fang <[email protected]>
In the case of heavy XDP traffic to be transmitted, the console will print the error log continuously if there are lack of enough BDs to accommodate the frames. The log looks like below.
[ 160.013112] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.023116] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.028926] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.038946] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.044758] fec 30be0000.ethernet eth0: NOT enough BD for SG!
Not only will this log be replicated and redundant, it will also degrade XDP performance. So we use netdev_err_once() instead of netdev_err() now.
---
V2 change:
This is a new patch which is separated from the patch 3.
--- drivers/net/ethernet/freescale/fec_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/freescale/fec_main.c +1 -1
@@ -3836,7 +3836,7 @@ static int fec_enet_txq_xmit_frame(struct fec_enet_private *fep, entries_free =fec_enet_get_free_txdesc_num (txq);if (entries_free <MAX_SKB_FRAGS +1 ) { -netdev_err (fep->netdev,"NOT enough BD for SG!\n" ); +netdev_err_once (fep->netdev,"NOT enough BD for SG!\n" );return -EBUSY ; }
Signature · 1 line
2.25.1
From: Wei Fang <[email protected]>
Date: Thu, 6 Jul 2023 16:10:08 +0800
Quoted 1 line
From: Wei Fang <[email protected]>
Please sync your Git commit author settings wrt Git email settings, so that there wouldn't be "From:" for your own commits.
Quoted 28 lines
We encountered some issues when testing the
ndo_xdp_xmit()interface of the fec driver on i.MX8MP and i.MX93 platforms. These issues are easy to reproduce, and the specific reproduction steps are as follows.step1: The ethernet port of a board (board A) is connected to the EQOS port of i.MX8MP/i.MX93, and the FEC port of i.MX8MP/i.MX93 is connected to another ethernet port, such as a switch port.
step2: Board A uses the
pktgen_sample03_burst_single_flow.shto generate and send packets to i.MX8MP/i.MX93. The command is shown below. ./pktgen_sample03_burst_single_flow.sh -i eth0 -d 192.168.6.8 -m \
56:bf:0d:68:b0:9e -s 1500step3: i.MX8MP/i.MX93 use the
xdp_redirectbfp program to redirect the XDP frames from EQOS port to FEC port. The command is shown below. ./xdp_redirect eth1 eth0After a few moments, the warning or error logs will be printed in the console, for more details, please refer to the commit message of each patch.
Wei Fang (4):
net: fec: dynamically set theNETDEV_XDP_ACT_NDO_XMITfeature of XDP
net: fec: recycle pages for transmitted XDP frames
net: fec: increase the size of tx ring and updatetx_wake_threshold
net: fec: usenetdev_err_once()instead ofnetdev_err()
Quoted 3 lines
drivers/net/ethernet/freescale/fec.h | 17 ++- drivers/net/ethernet/freescale/fec_main.c | 166 +++++++++++++++------- 2 files changed, 127 insertions(+), 56 deletions(-)
Thanks,
Olek
This series was applied to netdev/net.git (main) by Paolo Abeni <[email protected]>:
Hello:
This series was applied to netdev/net.git (main)
by Paolo Abeni <[email protected]>:
Quoted 11 lines · On Thu, 6 Jul 2023 16:10:08 +0800 you wrote:
From: Wei Fang <[email protected]>
We encountered some issues when testing the
ndo_xdp_xmit()interface of the fec driver on i.MX8MP and i.MX93 platforms. These issues are easy to reproduce, and the specific reproduction steps are as follows.step1: The ethernet port of a board (board A) is connected to the EQOS port of i.MX8MP/i.MX93, and the FEC port of i.MX8MP/i.MX93 is connected to another ethernet port, such as a switch port.
[...]
Here is the summary with links:
- [V2,net,1/4] net: fec: dynamically set the NETDEV_XDP_ACT_NDO_XMIT feature of XDP
https://git.kernel.org/netdev/net/c/be7ecbe7ec7d
- [V2,net,2/4] net: fec: recycle pages for transmitted XDP frames
https://git.kernel.org/netdev/net/c/20f797399035 - [V2,net,3/4] net: fec: increase the size of tx ring and update tx_wake_threshold https://git.kernel.org/netdev/net/c/56b3c6ba53d0 - [V2,net,4/4] net: fec: use netdev_err_once() instead of netdev_err() https://git.kernel.org/netdev/net/c/84a109471987
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html