Skip to contentSkip to conversation

net/mlx4_en: avoid indirect call in TX completion

PATCH In review Eric Dumazet 4 participants 7 messages Apr 15, 2020 – Apr 18, 2020

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net/mlx4_en: avoid indirect call in TX completion

Commit 9ecc2d86171a ("net/mlx4_en: add xdp forwarding and data write support") brought another indirect call in fast path.

Use INDIRECT_CALL_2() helper to avoid the cost of the indirect call when/if CONFIG_RETPOLINE=y

Cc Tariq Toukan <[email protected]>
Cc Willem de Bruijn <[email protected]>
---
 drivers/net/ethernet/mellanox/mlx4/en_tx.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx4/en_tx.c +13 -1
index 4d5ca302c067126b8627cb4809485b45c10e2460..a30edb436f4af11526e04c09623840288ebe4a29 100644
@@ -43,6 +43,7 @@
 #include <linux/ip.h>
 #include <linux/ipv6.h>
 #include <linux/moduleparam.h>
+#include <linux/indirect_call_wrapper.h>
 
 #include "mlx4_en.h"
 
@@ -261,6 +262,10 @@ static void mlx4_en_stamp_wqe(struct mlx4_en_priv *priv,
 	}
 }
 
+INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv,
+						   struct mlx4_en_tx_ring *ring,
+						   int index, u64 timestamp,
+						   int napi_mode));
 
 u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv,
 			 struct mlx4_en_tx_ring *ring,
@@ -329,6 +334,11 @@ u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv,
 	return tx_info->nr_txbb;
 }
 
+INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv,
+						      struct mlx4_en_tx_ring *ring,
+						      int index, u64 timestamp,
+						      int napi_mode));
+
 u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv,
 			    struct mlx4_en_tx_ring *ring,
 			    int index, u64 timestamp,
@@ -449,7 +459,9 @@ bool mlx4_en_process_tx_cq(struct net_device *dev,
 				timestamp = mlx4_en_get_cqe_ts(cqe);
 
 			/* free next descriptor */
-			last_nr_txbb = ring->free_tx_desc(
+			last_nr_txbb = INDIRECT_CALL_2(ring->free_tx_desc,
+						       mlx4_en_free_tx_desc,
+						       mlx4_en_recycle_tx_desc,
 					priv, ring, ring_index,
 					timestamp, napi_budget);
 
Signature · 1 line
2.26.0.110.g2183baf09c-goog
Reviewed-by on v1 1/1
#1 lore raw
Quoted 11 lines · Eric Dumazet

Commit 9ecc2d86171a ("net/mlx4_en: add xdp forwarding and data write support")
brought another indirect call in fast path.

Use INDIRECT_CALL_2() helper to avoid the cost of the indirect call when/if CONFIG_RETPOLINE=y

Cc Tariq Toukan <[email protected]>
Cc Willem de Bruijn <[email protected]>

---

Hi Eric, I believe net-next is still closed.

But FWIW,

Quoted 64 lines
 drivers/net/ethernet/mellanox/mlx4/en_tx.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx4/en_tx.c +0 -0

index 4d5ca302c067126b8627cb4809485b45c10e2460..a30edb436f4af11526e04c09623 840288ebe4a29 100644

drivers/net/ethernet/mellanox/mlx4/en_tx.c +4 -0
@@ -43,6 +43,7 @@
 #include <linux/ip.h>
 #include <linux/ipv6.h>
 #include <linux/moduleparam.h>
+#include <linux/indirect_call_wrapper.h>
 
 #include "mlx4_en.h"
 
@@ -261,6 +262,10 @@ static void mlx4_en_stamp_wqe(struct
 mlx4_en_priv *priv,
 	}
 }
 
+INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_free_tx_desc(struct
 mlx4_en_priv *priv,
+						   struct
 mlx4_en_tx_ring *ring,
+						   int index, u64

timestamp,

drivers/net/ethernet/mellanox/mlx4/en_tx.c +4 -0
+						   int napi_mode));
 
 u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv,
 			 struct mlx4_en_tx_ring *ring,
@@ -329,6 +334,11 @@ u32 mlx4_en_free_tx_desc(struct mlx4_en_priv
 *priv,
 	return tx_info->nr_txbb;
 }
 
+INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_recycle_tx_desc(struct
 mlx4_en_priv *priv,
+						      struct
 mlx4_en_tx_ring *ring,
+						      int index, u64

timestamp,

drivers/net/ethernet/mellanox/mlx4/en_tx.c +5 -1
+						      int napi_mode));
+
 u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv,
 			    struct mlx4_en_tx_ring *ring,
 			    int index, u64 timestamp,
@@ -449,7 +459,9 @@ bool mlx4_en_process_tx_cq(struct net_device
 *dev,
 				timestamp = mlx4_en_get_cqe_ts(cqe);
 
 			/* free next descriptor */
-			last_nr_txbb = ring->free_tx_desc(
+			last_nr_txbb = INDIRECT_CALL_2(ring-
 >free_tx_desc,
+						       mlx4_en_free_tx_
 desc,
+						       mlx4_en_recycle_

tx_desc,

drivers/net/ethernet/mellanox/mlx4/en_tx.c +0 -0
 					priv, ring, ring_index,
 					timestamp, napi_budget);
#2 lore raw
Quoted 14 lines · Saeed Mahameed
Quoted 11 lines · Eric Dumazet

Commit 9ecc2d86171a ("net/mlx4_en: add xdp forwarding and data write support")
brought another indirect call in fast path.

Use INDIRECT_CALL_2() helper to avoid the cost of the indirect call when/if CONFIG_RETPOLINE=y

Cc Tariq Toukan <[email protected]>
Cc Willem de Bruijn <[email protected]>

---

Quoted 6 lines

Hi Eric, I believe net-next is still closed.

But FWIW,

Well, this can be pushed to net then, since this is a trivial patch that helps performance.

With this COVID-19 thing, we need more capacity from the serving fleet (Youtube and all..)
distributed all over the world and using a high number of CX-3 NIC.

Thanks.

Quoted 65 lines
Quoted 65 lines
 drivers/net/ethernet/mellanox/mlx4/en_tx.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx4/en_tx.c +0 -0

index 4d5ca302c067126b8627cb4809485b45c10e2460..a30edb436f4af11526e04c09623 840288ebe4a29 100644

drivers/net/ethernet/mellanox/mlx4/en_tx.c +4 -0
@@ -43,6 +43,7 @@
 #include <linux/ip.h>
 #include <linux/ipv6.h>
 #include <linux/moduleparam.h>
+#include <linux/indirect_call_wrapper.h>
 
 #include "mlx4_en.h"
 
@@ -261,6 +262,10 @@ static void mlx4_en_stamp_wqe(struct
 mlx4_en_priv *priv,
      }
 }
 
+INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_free_tx_desc(struct
 mlx4_en_priv *priv,
+                                                struct
 mlx4_en_tx_ring *ring,
+                                                int index, u64

timestamp,

drivers/net/ethernet/mellanox/mlx4/en_tx.c +4 -0
+                                                int napi_mode));
 
 u32 mlx4_en_free_tx_desc(struct mlx4_en_priv *priv,
                       struct mlx4_en_tx_ring *ring,
@@ -329,6 +334,11 @@ u32 mlx4_en_free_tx_desc(struct mlx4_en_priv
 *priv,
      return tx_info->nr_txbb;
 }
 
+INDIRECT_CALLABLE_DECLARE(u32 mlx4_en_recycle_tx_desc(struct
 mlx4_en_priv *priv,
+                                                   struct
 mlx4_en_tx_ring *ring,
+                                                   int index, u64

timestamp,

drivers/net/ethernet/mellanox/mlx4/en_tx.c +5 -1
+                                                   int napi_mode));
+
 u32 mlx4_en_recycle_tx_desc(struct mlx4_en_priv *priv,
                          struct mlx4_en_tx_ring *ring,
                          int index, u64 timestamp,
@@ -449,7 +459,9 @@ bool mlx4_en_process_tx_cq(struct net_device
 *dev,
                              timestamp = mlx4_en_get_cqe_ts(cqe);
 
                      /* free next descriptor */
-                     last_nr_txbb = ring->free_tx_desc(
+                     last_nr_txbb = INDIRECT_CALL_2(ring-
 >free_tx_desc,
+                                                    mlx4_en_free_tx_
 desc,
+                                                    mlx4_en_recycle_

tx_desc,
priv, ring, ring_index, timestamp, napi_budget);

#3 lore raw
Quoted 27 lines · Eric Dumazet
Quoted 15 lines · Saeed Mahameed
Quoted 5 lines · Eric Dumazet

Commit 9ecc2d86171a ("net/mlx4_en: add xdp forwarding and data write
support")
brought another indirect call in fast path.

Quoted 8 lines

Use INDIRECT_CALL_2() helper to avoid the cost of the indirect call
when/if CONFIG_RETPOLINE=y

Cc Tariq Toukan <[email protected]>
Cc Willem de Bruijn <[email protected]>

---

Quoted 6 lines

Hi Eric, I believe net-next is still closed.

But FWIW,

Well, this can be pushed to net then, since this is a trivial patch that helps performance.

Quoted 7 lines

With this COVID-19 thing, we need more capacity from the serving fleet
(Youtube and all..)
distributed all over the world and using a high number of CX-3 NIC.

Thanks.

I don't mind, either net or net-next is fine by me, up to Dave,

although we are missing the Fixes Tag ..

Fixes 9ecc2d86171a ("net/mlx4_en: add xdp forwarding and data write

support")

#4 lore raw
Quoted 37 lines · Saeed Mahameed
Quoted 27 lines · Eric Dumazet
Quoted 15 lines · Saeed Mahameed
Quoted 5 lines · Eric Dumazet

Commit 9ecc2d86171a ("net/mlx4_en: add xdp forwarding and data write
support")
brought another indirect call in fast path.

Quoted 8 lines

Use INDIRECT_CALL_2() helper to avoid the cost of the indirect call
when/if CONFIG_RETPOLINE=y

Cc Tariq Toukan <[email protected]>
Cc Willem de Bruijn <[email protected]>

---

Quoted 6 lines

Hi Eric, I believe net-next is still closed.

But FWIW,

Well, this can be pushed to net then, since this is a trivial patch that helps performance.

Quoted 7 lines

With this COVID-19 thing, we need more capacity from the serving fleet
(Youtube and all..)
distributed all over the world and using a high number of CX-3 NIC.

Thanks.

Quoted 6 lines

I don't mind, either net or net-next is fine by me, up to Dave,

although we are missing the Fixes Tag ..

Fixes 9ecc2d86171a ("net/mlx4_en: add xdp forwarding and data write

support")

I chose not adding this tag, because at the time of this commit, we had not yet the INDIRECT_CALL() infrastructure.

I want to avoid requests from stable teams about backporting too many patches :)

maintainer
#5 lore raw
In reply to Eric Dumazet

From: Eric Dumazet <[email protected]>
Date: Wed, 15 Apr 2020 09:46:52 -0700

Quoted 7 lines

Commit 9ecc2d86171a ("net/mlx4_en: add xdp forwarding and data write support") brought another indirect call in fast path.

Use INDIRECT_CALL_2() helper to avoid the cost of the indirect call when/if CONFIG_RETPOLINE=y

I've decided to apply this to 'net' and queue it up for -stable, thanks.

#6 lore raw
Quoted 6 lines · David Miller

From: Eric Dumazet <[email protected]>
Date: Wed, 15 Apr 2020 09:46:52 -0700

Quoted 3 lines

Commit 9ecc2d86171a ("net/mlx4_en: add xdp forwarding and data write support") brought another indirect call in fast path.

Quoted 7 lines
Quoted 4 lines

Use INDIRECT_CALL_2() helper to avoid the cost of the indirect call when/if CONFIG_RETPOLINE=y

I've decided to apply this to 'net' and queue it up for -stable, thanks.

SGTM, thanks David !