net/mlx4_en: avoid indirect call in TX completion
Subsystems (4)
PATCH 1/1 net/mlx4_en: avoid indirect call in TX completion
Commit 9ecc2d86171a ("net/mlx4_en: add xdp forwarding and data write support") brought another indirect call in fast path.
Use INDIRECT_CALL_2() helper to avoid the cost of the indirect call when/if CONFIG_RETPOLINE=y
--- drivers/net/ethernet/mellanox/mlx4/en_tx.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx4/en_tx.c +13 -1
@@ -43,6 +43,7 @@#include <linux/ip.h> #include <linux/ipv6.h> #include <linux/moduleparam.h> +#include <linux/indirect_call_wrapper.h> #include "mlx4_en.h"
@@ -261,6 +262,10 @@ static void mlx4_en_stamp_wqe(struct mlx4_en_priv *priv, } } +INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv, +struct mlx4_en_tx_ring *ring, +int index,u64 timestamp, +int napi_mode)); u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv, struct mlx4_en_tx_ring *ring,
@@ -329,6 +334,11 @@ u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv,return tx_info->nr_txbb; } +INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv, +struct mlx4_en_tx_ring *ring, +int index,u64 timestamp, +int napi_mode)); + u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv, struct mlx4_en_tx_ring *ring, int index, u64 timestamp,
@@ -449,7 +459,9 @@ bool mlx4_en_process_tx_cq(struct net_device *dev, timestamp =mlx4_en_get_cqe_ts (cqe);/* free next descriptor */ - last_nr_txbb = ring->free_tx_desc ( + last_nr_txbb =INDIRECT_CALL_2 (ring->free_tx_desc, + mlx4_en_free_tx_desc, + mlx4_en_recycle_tx_desc, priv, ring, ring_index, timestamp, napi_budget);
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Quoted 11 lines · Eric Dumazet
Commit
9ecc2d86171a("net/mlx4_en: add xdp forwarding and data write support")
brought another indirect call in fast path.Use
INDIRECT_CALL_2()helper to avoid the cost of the indirect call when/ifCONFIG_RETPOLINE=ySigned-off-by Eric Dumazet <[email protected]>Cc Tariq Toukan <[email protected]>Cc Willem de Bruijn <[email protected]>---
Hi Eric, I believe net-next is still closed.
But FWIW,
Quoted 64 lines
drivers/net/ethernet/mellanox/mlx4/en_tx.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)drivers/net/ethernet/mellanox/mlx4/en_tx.c +0
-0index
4d5ca302c067126b8627cb4809485b45c10e2460..a30edb436f4af11526e04c09623840288ebe4a29100644drivers/net/ethernet/mellanox/mlx4/en_tx.c +4
-0@@ -43,6 +43,7 @@#include <linux/ip.h> #include <linux/ipv6.h> #include <linux/moduleparam.h> +#include <linux/indirect_call_wrapper.h> #include "mlx4_en.h" @@ -261,6 +262,10 @@ static void mlx4_en_stamp_wqe(struct mlx4_en_priv *priv, } } +INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv, + struct mlx4_en_tx_ring *ring, + int index, u64timestamp,
drivers/net/ethernet/mellanox/mlx4/en_tx.c +4
-0+ int napi_mode)); u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv, struct mlx4_en_tx_ring *ring,@@ -329,6 +334,11 @@ u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv, return tx_info->nr_txbb; } +INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv, + struct mlx4_en_tx_ring *ring, + int index, u64timestamp,
drivers/net/ethernet/mellanox/mlx4/en_tx.c +5
-1+ int napi_mode)); + u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv, struct mlx4_en_tx_ring *ring, int index, u64 timestamp,@@ -449,7 +459,9 @@ bool mlx4_en_process_tx_cq(struct net_device *dev, timestamp =mlx4_en_get_cqe_ts (cqe);/* free next descriptor */ - last_nr_txbb = ring->free_tx_desc( + last_nr_txbb =INDIRECT_CALL_2 (ring- >free_tx_desc, + mlx4_en_free_tx_ desc, + mlx4_en_recycle_
tx_desc,drivers/net/ethernet/mellanox/mlx4/en_tx.c +0
-0priv, ring, ring_index, timestamp, napi_budget);
Quoted 14 lines · Saeed Mahameed
Quoted 11 lines · Eric Dumazet
Commit
9ecc2d86171a("net/mlx4_en: add xdp forwarding and data write support")
brought another indirect call in fast path.Use
INDIRECT_CALL_2()helper to avoid the cost of the indirect call when/ifCONFIG_RETPOLINE=ySigned-off-by Eric Dumazet <[email protected]>Cc Tariq Toukan <[email protected]>Cc Willem de Bruijn <[email protected]>---
Quoted 6 lines
Hi Eric, I believe net-next is still closed.
But FWIW,
Reviewed-by Saeed Mahameed <[email protected]>
Well, this can be pushed to net then, since this is a trivial patch that helps performance.
With this COVID-19 thing, we need more capacity from the serving fleet (Youtube and all..)
distributed all over the world and using a high number of CX-3 NIC.
Thanks.
Quoted 65 lines
Quoted 65 lines
drivers/net/ethernet/mellanox/mlx4/en_tx.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)drivers/net/ethernet/mellanox/mlx4/en_tx.c +0
-0index
4d5ca302c067126b8627cb4809485b45c10e2460..a30edb436f4af11526e04c09623840288ebe4a29100644drivers/net/ethernet/mellanox/mlx4/en_tx.c +4
-0@@ -43,6 +43,7 @@#include <linux/ip.h> #include <linux/ipv6.h> #include <linux/moduleparam.h> +#include <linux/indirect_call_wrapper.h> #include "mlx4_en.h" @@ -261,6 +262,10 @@ static void mlx4_en_stamp_wqe(struct mlx4_en_priv *priv, } } +INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv, + struct mlx4_en_tx_ring *ring, + int index, u64timestamp,
drivers/net/ethernet/mellanox/mlx4/en_tx.c +4
-0+ int napi_mode)); u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv, struct mlx4_en_tx_ring *ring,@@ -329,6 +334,11 @@ u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv, return tx_info->nr_txbb; } +INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv, + struct mlx4_en_tx_ring *ring, + int index, u64timestamp,
drivers/net/ethernet/mellanox/mlx4/en_tx.c +5
-1+ int napi_mode)); + u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv, struct mlx4_en_tx_ring *ring, int index, u64 timestamp,@@ -449,7 +459,9 @@ bool mlx4_en_process_tx_cq(struct net_device *dev, timestamp =mlx4_en_get_cqe_ts (cqe);/* free next descriptor */ - last_nr_txbb = ring->free_tx_desc( + last_nr_txbb =INDIRECT_CALL_2 (ring- >free_tx_desc, + mlx4_en_free_tx_ desc, + mlx4_en_recycle_
tx_desc,
priv, ring,ring_index, timestamp,napi_budget);
Quoted 27 lines · Eric Dumazet
Quoted 15 lines · Saeed Mahameed
Quoted 5 lines · Eric Dumazet
Commit
9ecc2d86171a("net/mlx4_en: add xdp forwarding and data write
support")
brought another indirect call in fast path.Quoted 8 lines
Use
INDIRECT_CALL_2()helper to avoid the cost of the indirect call
when/ifCONFIG_RETPOLINE=ySigned-off-by Eric Dumazet <[email protected]>Cc Tariq Toukan <[email protected]>Cc Willem de Bruijn <[email protected]>---
Quoted 6 lines
Hi Eric, I believe net-next is still closed.
But FWIW,
Reviewed-by Saeed Mahameed <[email protected]>Well, this can be pushed to net then, since this is a trivial patch that helps performance.
Quoted 7 lines
With this COVID-19 thing, we need more capacity from the serving fleet
(Youtube and all..)
distributed all over the world and using a high number of CX-3 NIC.Thanks.
I don't mind, either net or net-next is fine by me, up to Dave,
although we are missing the Fixes Tag ..
support")
Quoted 37 lines · Saeed Mahameed
Quoted 27 lines · Eric Dumazet
Quoted 15 lines · Saeed Mahameed
Quoted 5 lines · Eric Dumazet
Commit
9ecc2d86171a("net/mlx4_en: add xdp forwarding and data write
support")
brought another indirect call in fast path.Quoted 8 lines
Use
INDIRECT_CALL_2()helper to avoid the cost of the indirect call
when/ifCONFIG_RETPOLINE=ySigned-off-by Eric Dumazet <[email protected]>Cc Tariq Toukan <[email protected]>Cc Willem de Bruijn <[email protected]>---
Quoted 6 lines
Hi Eric, I believe net-next is still closed.
But FWIW,
Reviewed-by Saeed Mahameed <[email protected]>Well, this can be pushed to net then, since this is a trivial patch that helps performance.
Quoted 7 lines
With this COVID-19 thing, we need more capacity from the serving fleet
(Youtube and all..)
distributed all over the world and using a high number of CX-3 NIC.Thanks.
Quoted 6 lines
I don't mind, either net or net-next is fine by me, up to Dave,
although we are missing the Fixes Tag ..
Fixes 9ecc2d86171a ("net/mlx4_en: add xdp forwarding and data writesupport")
I chose not adding this tag, because at the time of this commit, we had not yet the INDIRECT_CALL() infrastructure.
I want to avoid requests from stable teams about backporting too many patches :)
David Miller
maintainerFrom: Eric Dumazet <[email protected]>
Date: Wed, 15 Apr 2020 09:46:52 -0700
Quoted 7 lines
Commit
9ecc2d86171a("net/mlx4_en: add xdp forwarding and data write support") brought another indirect call in fast path.Use
INDIRECT_CALL_2()helper to avoid the cost of the indirect call when/ifCONFIG_RETPOLINE=ySigned-off-by Eric Dumazet <[email protected]>
I've decided to apply this to 'net' and queue it up for -stable, thanks.
Quoted 6 lines · David Miller
From: Eric Dumazet <[email protected]>
Date: Wed, 15 Apr 2020 09:46:52 -0700Quoted 3 lines
Commit
9ecc2d86171a("net/mlx4_en: add xdp forwarding and data write support") brought another indirect call in fast path.
Quoted 7 lines
Quoted 4 lines
Use
INDIRECT_CALL_2()helper to avoid the cost of the indirect call when/ifCONFIG_RETPOLINE=ySigned-off-by Eric Dumazet <[email protected]>I've decided to apply this to 'net' and queue it up for -stable, thanks.
SGTM, thanks David !