Skip to contentSkip to conversation

mlx4: better BIG-TCP support

PATCH Applied Eric Dumazet 3 participants 8 messages Dec 7, 2022 – Dec 8, 2022

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v2 cover mlx4: better BIG-TCP support

mlx4 uses a bounce buffer in TX whenever the tx descriptors wrap around the right edge of the ring.

Size of this bounce buffer was hard coded and can be increased if/when needed.

v2: roundup MLX4_TX_BOUNCE_BUFFER_SIZE (Tariq)

Eric Dumazet (3):
net/mlx4: rename two constants
net/mlx4: MLX4_TX_BOUNCE_BUFFER_SIZE depends on MAX_SKB_FRAGS net/mlx4: small optimization in mlx4_en_xmit()

 drivers/net/ethernet/mellanox/mlx4/en_tx.c   | 18 ++++++++++--------
 drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 18 +++++++++++++-----
 2 files changed, 23 insertions(+), 13 deletions(-)
Signature · 1 line
2.39.0.rc0.267.gcb52ba06e7-goog
#3 lore raw

PATCH v2 2/3 net/mlx4: MLX4_TX_BOUNCE_BUFFER_SIZE depends on MAX_SKB_FRAGS

Google production kernel has increased MAX_SKB_FRAGS to 45 for BIG-TCP rollout.

Unfortunately mlx4 TX bounce buffer is not big enough whenever an skb has up to 45 page fragments.

This can happen often with TCP TX zero copy, as one frag usually holds 4096 bytes of payload (order-0 page).

Tested:
Kernel built with MAX_SKB_FRAGS=45
ip link set dev eth0 gso_max_size 185000
netperf -t TCP_SENDFILE

I made sure that "ethtool -G eth0 tx 64" was properly working, ring->full_size being set to 15.

---
 drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
drivers/net/ethernet/mellanox/mlx4/mlx4_en.h +12 -4
index 7cc288db2a64f75ffe64882e3c25b90715e68855..3d4226ddba5e6582e9420d853b6535b806219e55 100644
@@ -89,8 +89,18 @@
 #define MLX4_EN_FILTER_HASH_SHIFT 4
 #define MLX4_EN_FILTER_EXPIRY_QUOTA 60
 
-/* Typical TSO descriptor with 16 gather entries is 352 bytes... */
-#define MLX4_TX_BOUNCE_BUFFER_SIZE 512
+#define CTRL_SIZE	sizeof(struct mlx4_wqe_ctrl_seg)
+#define DS_SIZE		sizeof(struct mlx4_wqe_data_seg)
+
+/* Maximal size of the bounce buffer:
+ * 256 bytes for LSO headers.
+ * CTRL_SIZE for control desc.
+ * DS_SIZE if skb->head contains some payload.
+ * MAX_SKB_FRAGS frags.
+ */
+#define MLX4_TX_BOUNCE_BUFFER_SIZE \
+	ALIGN(256 + CTRL_SIZE + DS_SIZE + MAX_SKB_FRAGS * DS_SIZE, TXBB_SIZE)
+
 #define MLX4_MAX_DESC_TXBBS	   (MLX4_TX_BOUNCE_BUFFER_SIZE / TXBB_SIZE)
 
 /*
@@ -217,9 +227,7 @@ struct mlx4_en_tx_info {
 
 
 #define MLX4_EN_BIT_DESC_OWN	0x80000000
-#define CTRL_SIZE	sizeof(struct mlx4_wqe_ctrl_seg)
 #define MLX4_EN_MEMTYPE_PAD	0x100
-#define DS_SIZE		sizeof(struct mlx4_wqe_data_seg)
 
 
 struct mlx4_en_tx_desc {
Signature · 1 line
2.39.0.rc0.267.gcb52ba06e7-goog
#1 lore raw

PATCH v2 1/3 net/mlx4: rename two constants

In reply to Eric Dumazet

MAX_DESC_SIZE is really the size of the bounce buffer used when reaching the right side of TX ring buffer.

MAX_DESC_TXBBS get a MLX4_ prefix.

---
 drivers/net/ethernet/mellanox/mlx4/en_tx.c   | 10 ++++++----
 drivers/net/ethernet/mellanox/mlx4/mlx4_en.h |  4 ++--
 2 files changed, 8 insertions(+), 6 deletions(-)
drivers/net/ethernet/mellanox/mlx4/en_tx.c +6 -4
index 43a4102e9c091758b33aa7377dcb82cab7c43a94..8372aeb392a28cf36a454e1b8a4783bc2b2056eb 100644
@@ -65,7 +65,7 @@ int mlx4_en_create_tx_ring(struct mlx4_en_priv *priv,
 	ring->size = size;
 	ring->size_mask = size - 1;
 	ring->sp_stride = stride;
-	ring->full_size = ring->size - HEADROOM - MAX_DESC_TXBBS;
+	ring->full_size = ring->size - HEADROOM - MLX4_MAX_DESC_TXBBS;
 
 	tmp = size * sizeof(struct mlx4_en_tx_info);
 	ring->tx_info = kvmalloc_node(tmp, GFP_KERNEL, node);
@@ -77,9 +77,11 @@ int mlx4_en_create_tx_ring(struct mlx4_en_priv *priv,
 	en_dbg(DRV, priv, "Allocated tx_info ring at addr:%p size:%d\n",
 		 ring->tx_info, tmp);
 
-	ring->bounce_buf = kmalloc_node(MAX_DESC_SIZE, GFP_KERNEL, node);
+	ring->bounce_buf = kmalloc_node(MLX4_TX_BOUNCE_BUFFER_SIZE,
+					GFP_KERNEL, node);
 	if (!ring->bounce_buf) {
-		ring->bounce_buf = kmalloc(MAX_DESC_SIZE, GFP_KERNEL);
+		ring->bounce_buf = kmalloc(MLX4_TX_BOUNCE_BUFFER_SIZE,
+					   GFP_KERNEL);
 		if (!ring->bounce_buf) {
 			err = -ENOMEM;
 			goto err_info;
@@ -909,7 +911,7 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)
 	/* Align descriptor to TXBB size */
 	desc_size = ALIGN(real_size, TXBB_SIZE);
 	nr_txbb = desc_size >> LOG_TXBB_SIZE;
-	if (unlikely(nr_txbb > MAX_DESC_TXBBS)) {
+	if (unlikely(nr_txbb > MLX4_MAX_DESC_TXBBS)) {
 		if (netif_msg_tx_err(priv))
 			en_warn(priv, "Oversized header or SG list\n");
 		goto tx_drop_count;
drivers/net/ethernet/mellanox/mlx4/mlx4_en.h +2 -2
index e132ff4c82f2d33045f6c9aeecaaa409a41e0b0d..7cc288db2a64f75ffe64882e3c25b90715e68855 100644
@@ -90,8 +90,8 @@
 #define MLX4_EN_FILTER_EXPIRY_QUOTA 60
 
 /* Typical TSO descriptor with 16 gather entries is 352 bytes... */
-#define MAX_DESC_SIZE		512
-#define MAX_DESC_TXBBS		(MAX_DESC_SIZE / TXBB_SIZE)
+#define MLX4_TX_BOUNCE_BUFFER_SIZE 512
+#define MLX4_MAX_DESC_TXBBS	   (MLX4_TX_BOUNCE_BUFFER_SIZE / TXBB_SIZE)
 
 /*
  * OS related constants and tunables
Signature · 1 line
2.39.0.rc0.267.gcb52ba06e7-goog
#5 lore raw

PATCH v2 3/3 net/mlx4: small optimization in mlx4_en_xmit()

In reply to Eric Dumazet

Test against MLX4_MAX_DESC_TXBBS only matters if the TX bounce buffer is going to be used.

Cc Tariq Toukan <[email protected]>
Cc Wei Wang <[email protected]>
---
 drivers/net/ethernet/mellanox/mlx4/en_tx.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/net/ethernet/mellanox/mlx4/en_tx.c +5 -5
index 8372aeb392a28cf36a454e1b8a4783bc2b2056eb..c5758637b7bed67021a9f3e9c5283033f68639a3 100644
@@ -911,11 +911,6 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)
 	/* Align descriptor to TXBB size */
 	desc_size = ALIGN(real_size, TXBB_SIZE);
 	nr_txbb = desc_size >> LOG_TXBB_SIZE;
-	if (unlikely(nr_txbb > MLX4_MAX_DESC_TXBBS)) {
-		if (netif_msg_tx_err(priv))
-			en_warn(priv, "Oversized header or SG list\n");
-		goto tx_drop_count;
-	}
 
 	bf_ok = ring->bf_enabled;
 	if (skb_vlan_tag_present(skb)) {
@@ -943,6 +938,11 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)
 	if (likely(index + nr_txbb <= ring->size))
 		tx_desc = ring->buf + (index << LOG_TXBB_SIZE);
 	else {
+		if (unlikely(nr_txbb > MLX4_MAX_DESC_TXBBS)) {
+			if (netif_msg_tx_err(priv))
+				en_warn(priv, "Oversized header or SG list\n");
+			goto tx_drop_count;
+		}
 		tx_desc = (struct mlx4_en_tx_desc *) ring->bounce_buf;
 		bounce = true;
 		bf_ok = false;
Signature · 1 line
2.39.0.rc0.267.gcb52ba06e7-goog
Reviewed-by on v2 1/3
#2 lore raw

Re: [PATCH v2 net-next 1/3] net/mlx4: rename two constants

In reply to Eric Dumazet
Quoted 12 lines · Eric Dumazet

MAX_DESC_SIZE is really the size of the bounce buffer used when reaching the right side of TX ring buffer.

MAX_DESC_TXBBS get a MLX4_ prefix.

---
  drivers/net/ethernet/mellanox/mlx4/en_tx.c   | 10 ++++++----
  drivers/net/ethernet/mellanox/mlx4/mlx4_en.h |  4 ++--
  2 files changed, 8 insertions(+), 6 deletions(-)

Thanks for your series!

Reviewed-by on v2 2/3
#4 lore raw

Re: [PATCH v2 net-next 2/3] net/mlx4: MLX4_TX_BOUNCE_BUFFER_SIZE depends on MAX_SKB_FRAGS

In reply to Eric Dumazet
Quoted 14 lines · Eric Dumazet

Google production kernel has increased MAX_SKB_FRAGS to 45 for BIG-TCP rollout.

Unfortunately mlx4 TX bounce buffer is not big enough whenever an skb has up to 45 page fragments.

This can happen often with TCP TX zero copy, as one frag usually holds 4096 bytes of payload (order-0 page).

Tested:
Kernel built with MAX_SKB_FRAGS=45
ip link set dev eth0 gso_max_size 185000
netperf -t TCP_SENDFILE

Quoted 6 lines

I made sure that "ethtool -G eth0 tx 64" was properly working, ring->full_size being set to 15.

Reviewed-by on v2 3/3
#6 lore raw

Re: [PATCH v2 net-next 3/3] net/mlx4: small optimization in mlx4_en_xmit()

In reply to Eric Dumazet
Quoted 10 lines · Eric Dumazet

Test against MLX4_MAX_DESC_TXBBS only matters if the TX bounce buffer is going to be used.

Cc Tariq Toukan <[email protected]>
Cc Wei Wang <[email protected]>
---
  drivers/net/ethernet/mellanox/mlx4/en_tx.c | 10 +++++-----
  1 file changed, 5 insertions(+), 5 deletions(-)
#7 lore raw
In reply to Eric Dumazet
This series was applied to netdev/net-next.git (master) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (master) by Jakub Kicinski <[email protected]>:

Quoted 9 lines · On Wed, 7 Dec 2022 14:12:34 +0000 you wrote:

mlx4 uses a bounce buffer in TX whenever the tx descriptors wrap around the right edge of the ring.

Size of this bounce buffer was hard coded and can be increased if/when needed.

v2: roundup MLX4_TX_BOUNCE_BUFFER_SIZE (Tariq)

[...]

Here is the summary with links:
- [v2,net-next,1/3] net/mlx4: rename two constants

    https://git.kernel.org/netdev/net-next/c/35f31ff0c0b6
  - [v2,net-next,2/3] net/mlx4: MLX4_TX_BOUNCE_BUFFER_SIZE depends on MAX_SKB_FRAGS
    https://git.kernel.org/netdev/net-next/c/26782aad00cc
  - [v2,net-next,3/3] net/mlx4: small optimization in mlx4_en_xmit()
    https://git.kernel.org/netdev/net-next/c/0e706f7961a4

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html