Skip to contentSkip to conversation

mlx5 updates 2022-01-06

GIT PULL Applied Saeed Mahameed 2 participants 17 messages Jan 7, 2022

Subsystems (8)
lore
Share & tools
lore
#0 lore raw

From: Saeed Mahameed <[email protected]>

Hi Dave, Hi Jakub,

This series adds misc updates.
For more information please see tag log below.

Please pull and let me know if there is any problem.

Thanks,
Saeed.

The following changes since commit 710ad98c363a66a0cd8526465426c5c5f8377ee0:

veth: Do not record rx queue hint in veth_xmit (2022-01-06 13:49:54 +0000)

are available in the Git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/mlx5-updates-2022-01-06

for you to fetch changes up to 745a13061aa079b36d83ba7f36dc051dbfccd888:

Documentation: devlink: mlx5.rst: Fix htmldoc build warning (2022-01-06 16:22:55 -0800)

----------------------------------------------------------------
mlx5-updates-2022-01-06

1) Expose FEC per lane block counters via ethtool

2) Trivial fixes/updates/cleanup to mlx5e netdev driver

3) Fix htmldoc build warning

4) Spread mlx5 SFs (sub-functions) to all available CPU cores: Commits 1..5

Shay Drory Says:
================
Before this patchset, mlx5 subfunction shared the same IRQs (MSI-X) with their peers subfunctions, causing them to use same CPU cores.

In large scale, this is very undesirable, SFs use small number of cpu cores and all of them will be packed on the same CPU cores, not utilizing all CPU cores in the system.

In this patchset we want to achieve two things.
a) Spread IRQs used by SFs to all cpu cores
b) Pack less SFs in the same IRQ, will result in multiple IRQs per core.

In this patchset, we spread SFs over all online cpus available to mlx5 irqs in Round-Robin manner. e.g.: Whenever a SF is created, pick the next CPU core with least number of SF IRQs bound to it, SFs will share IRQs on the same core until a certain limit, when such limit is reached, we request a new IRQ and add it to that CPU core IRQ pool, when out of IRQs, pick any IRQ with least number of SF users.

This enhancement is done in order to achieve a better distribution of the SFs over all the available CPUs, which reduces application latency, as shown bellow.

Machine details:
Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz with 56 cores. PCI Express 3 with BW of 126 Gb/s.
ConnectX-5 Ex; EDR IB (100Gb/s) and 100GbE; dual-port QSFP28; PCIe4.0 x16.

Base line test description:
Single SF on the system. One instance of netperf is running on-top the SF.
Numbers: latency = 15.136 usec, CPU Util = 35%

Test description:
There are 250 SFs on the system. There are 3 instances of netperf running, on-top three different SFs, in parallel.

Perf numbers:

 # netperf     SFs         latency(usec)     latency    CPU utilization
   affinity    affinity    (lower is better) increase %
 1 cpu=0       cpu={0}     ~23 (app 1-3)     35%        75%
 2 cpu=0,2,4   cpu={0}     app 1: 21.625     30%        68% (CPU 0)
                           app 2-3: 16.5     9%         15% (CPU 2,4)
 3 cpu=0       cpu={0,2,4} app 1: ~16        7%         84% (CPU 0)
                           app 2-3: ~17.9    14%        22% (CPU 2,4)
 4 cpu=0,2,4   cpu={0,2,4} 15.2 (app 1-3)    0%         33% (CPU 0,2,4)

- The first two entries (#1 and #2) show current state. e.g.: SFs are using the same CPU. The last two entries (#3 and #4) shows the latency reduction improvement of this patch. e.g.: SFs are on different CPUs.
- Whenever we use several CPUs, in case there is a different CPU utilization, write the utilization of each CPU separately.
- Whenever the latency result of the netperf instances were different, write the latency of each netperf instances separately.

Commands:
- for netperf CPU=0:

$ for i in {1..3}; do taskset -c 0 netperf -H 1${i}.1.1.1 -t TCP_RR  -- \
  -o RT_LATENCY -r8 & done

- for netperf CPU=0,2,4

$ for i in {1..3}; do taskset -c $(( ($i - 1) * 2  )) netperf -H \
  1${i}.1.1.1 -t TCP_RR  -- -o RT_LATENCY -r8 & done

================

----------------------------------------------------------------
Aya Levin (1):
net/mlx5e: Fix feature check per profile

Gal Pressman (2):
net/mlx5e: Move HW-GRO and CQE compression check to fix features flow net/mlx5e: Add recovery flow in case of error CQE

Lama Kayal (1):
net/mlx5e: Expose FEC counters via ethtool

Maher Sanalla (1):
net/mlx5: Update log_max_qp value to FW max capability

Maor Dickman (1):
net/mlx5e: Unblock setting vid 0 for VF in case PF isn't eswitch manager

Roi Dayan (1):
net/mlx5e: TC, Remove redundant error logging

Saeed Mahameed (3):
net/mlx5: mlx5e_hv_vhca_stats_create return type to void net/mlx5e: Refactor set_pflag_cqe_based_moder
Documentation: devlink: mlx5.rst: Fix htmldoc build warning

Shay Drory (5):
net/mlx5: Introduce control IRQ request API net/mlx5: Move affinity assignment into irq_request net/mlx5: Split irq_pool_affinity logic to new file net/mlx5: Introduce API for bulk request and release of IRQs net/mlx5: SF, Use all available cpu for setting cpu affinity

 Documentation/networking/devlink/mlx5.rst          |   1 +
 drivers/infiniband/hw/mlx5/odp.c                   |   6 -
 drivers/net/ethernet/mellanox/mlx5/core/Makefile   |   2 +-
 drivers/net/ethernet/mellanox/mlx5/core/en.h       |   2 +-
 .../ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c |   8 +-
 .../ethernet/mellanox/mlx5/core/en/hv_vhca_stats.h |  13 +-
 .../ethernet/mellanox/mlx5/core/en/tc/act/mirred.c |   8 -
 .../net/ethernet/mellanox/mlx5/core/en_ethtool.c   |  19 +-
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c  |  10 +-
 drivers/net/ethernet/mellanox/mlx5/core/en_rx.c    |  20 +-
 drivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 101 ++++++-
 drivers/net/ethernet/mellanox/mlx5/core/eq.c       | 104 +++++--
 .../net/ethernet/mellanox/mlx5/core/esw/legacy.c   |   2 +-
 .../net/ethernet/mellanox/mlx5/core/irq_affinity.c | 226 +++++++++++++++
 drivers/net/ethernet/mellanox/mlx5/core/main.c     |   8 +-
 drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h |  30 +-
 drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c  | 308 +++++++++++----------
 drivers/net/ethernet/mellanox/mlx5/core/pci_irq.h  |  39 +++
 include/linux/mlx5/eq.h                            |   4 +-
 19 files changed, 675 insertions(+), 236 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/pci_irq.h
#1 lore raw

PATCH 1/15 net/mlx5: mlx5e_hv_vhca_stats_create return type to void

From: Saeed Mahameed <[email protected]>

Callers of this functions ignore its return value, as reported by Wang Qing, in one of the return paths, it returns positive values.

Since return value is ignored anyways, void out the return type of the function.

---
 .../ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c  |  8 +++-----
 .../ethernet/mellanox/mlx5/core/en/hv_vhca_stats.h  | 13 +++----------
 2 files changed, 6 insertions(+), 15 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c +3 -5
index 074ffa4fa5af..b4f3bd7d346e 100644
@@ -120,14 +120,14 @@ static void mlx5e_hv_vhca_stats_cleanup(struct mlx5_hv_vhca_agent *agent)
 	cancel_delayed_work_sync(&priv->stats_agent.work);
 }
 
-int mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv)
+void mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv)
 {
 	int buf_len = mlx5e_hv_vhca_stats_buf_size(priv);
 	struct mlx5_hv_vhca_agent *agent;
 
 	priv->stats_agent.buf = kvzalloc(buf_len, GFP_KERNEL);
 	if (!priv->stats_agent.buf)
-		return -ENOMEM;
+		return;
 
 	agent = mlx5_hv_vhca_agent_create(priv->mdev->hv_vhca,
 					  MLX5_HV_VHCA_AGENT_STATS,
@@ -142,13 +142,11 @@ int mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv)
 				    PTR_ERR(agent));
 
 		kvfree(priv->stats_agent.buf);
-		return IS_ERR_OR_NULL(agent);
+		return;
 	}
 
 	priv->stats_agent.agent = agent;
 	INIT_DELAYED_WORK(&priv->stats_agent.work, mlx5e_hv_vhca_stats_work);
-
-	return 0;
 }
 
 void mlx5e_hv_vhca_stats_destroy(struct mlx5e_priv *priv)
drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.h +3 -10
index 664463faf77b..29c8c6d3260f 100644
@@ -7,19 +7,12 @@
 
 #if IS_ENABLED(CONFIG_PCI_HYPERV_INTERFACE)
 
-int mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv);
+void mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv);
 void mlx5e_hv_vhca_stats_destroy(struct mlx5e_priv *priv);
 
 #else
-
-static inline int mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv)
-{
-	return 0;
-}
-
-static inline void mlx5e_hv_vhca_stats_destroy(struct mlx5e_priv *priv)
-{
-}
+static inline void mlx5e_hv_vhca_stats_create(struct mlx5e_priv *priv) {}
+static inline void mlx5e_hv_vhca_stats_destroy(struct mlx5e_priv *priv) {}
 #endif
 
 #endif /* __MLX5_EN_STATS_VHCA_H__ */
Signature · 1 line
2.33.1
#2 lore raw

Re: [PATCH net-next 01/15] net/mlx5: mlx5e_hv_vhca_stats_create return type to void

This series was applied to netdev/net-next.git (master) by Saeed Mahameed <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (master) by Saeed Mahameed <[email protected]>:

Quoted 9 lines · On Thu, 6 Jan 2022 16:29:42 -0800 you wrote:

From: Saeed Mahameed <[email protected]>

Callers of this functions ignore its return value, as reported by Wang Qing, in one of the return paths, it returns positive values.

Since return value is ignored anyways, void out the return type of the function.

[...]

Here is the summary with links:
- [net-next,01/15] net/mlx5: mlx5e_hv_vhca_stats_create return type to void

    https://git.kernel.org/netdev/net-next/c/20f80ffcedfa
  - [net-next,02/15] net/mlx5: Introduce control IRQ request API
    https://git.kernel.org/netdev/net-next/c/5256a46bf538
  - [net-next,03/15] net/mlx5: Move affinity assignment into irq_request
    https://git.kernel.org/netdev/net-next/c/30c6afa735db
  - [net-next,04/15] net/mlx5: Split irq_pool_affinity logic to new file
    https://git.kernel.org/netdev/net-next/c/424544df97b0
  - [net-next,05/15] net/mlx5: Introduce API for bulk request and release of IRQs
    https://git.kernel.org/netdev/net-next/c/79b60ca83b6f
  - [net-next,06/15] net/mlx5: SF, Use all available cpu for setting cpu affinity
    https://git.kernel.org/netdev/net-next/c/061f5b23588a
  - [net-next,07/15] net/mlx5: Update log_max_qp value to FW max capability
    https://git.kernel.org/netdev/net-next/c/f79a609ea6bf
  - [net-next,08/15] net/mlx5e: Expose FEC counters via ethtool
    https://git.kernel.org/netdev/net-next/c/0a1498ebfa55
  - [net-next,09/15] net/mlx5e: Unblock setting vid 0 for VF in case PF isn't eswitch manager
    https://git.kernel.org/netdev/net-next/c/7846665d3504
  - [net-next,10/15] net/mlx5e: Fix feature check per profile
    https://git.kernel.org/netdev/net-next/c/bc2a7b5c6b37
  - [net-next,11/15] net/mlx5e: Move HW-GRO and CQE compression check to fix features flow
    https://git.kernel.org/netdev/net-next/c/b5f42903704f
  - [net-next,12/15] net/mlx5e: Refactor set_pflag_cqe_based_moder
    https://git.kernel.org/netdev/net-next/c/be23511eb5c4
  - [net-next,13/15] net/mlx5e: TC, Remove redundant error logging
    https://git.kernel.org/netdev/net-next/c/68511b48bfbe
  - [net-next,14/15] net/mlx5e: Add recovery flow in case of error CQE
    https://git.kernel.org/netdev/net-next/c/5dd29f40b25f
  - [net-next,15/15] Documentation: devlink: mlx5.rst: Fix htmldoc build warning
    https://git.kernel.org/netdev/net-next/c/745a13061aa0

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
#3 lore raw

PATCH 2/15 net/mlx5: Introduce control IRQ request API

In reply to Saeed Mahameed

From: Shay Drory <[email protected]>

Currently, IRQ layer have a separate flow for ctrl and comp IRQs, and the distinction between ctrl and comp IRQs is done in the IRQ layer.

In order to ease the coding and maintenance of the IRQ layer, introduce a new API for requesting control IRQs - mlx5_ctrl_irq_request(struct mlx5_core_dev *dev).

---
 drivers/net/ethernet/mellanox/mlx5/core/eq.c  |  5 +-
 .../ethernet/mellanox/mlx5/core/mlx5_irq.h    |  1 +
 .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 62 ++++++++++++++++---
 3 files changed, 58 insertions(+), 10 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eq.c +4 -1
index b695aad71ee1..1eb0326a489b 100644
@@ -289,7 +289,10 @@ create_map_eq(struct mlx5_core_dev *dev, struct mlx5_eq *eq,
 	mlx5_init_fbc(eq->frag_buf.frags, log_eq_stride, log_eq_size, &eq->fbc);
 	init_eq_buf(eq);
 
-	eq->irq = mlx5_irq_request(dev, vecidx, param->affinity);
+	if (vecidx == MLX5_IRQ_EQ_CTRL)
+		eq->irq = mlx5_ctrl_irq_request(dev);
+	else
+		eq->irq = mlx5_irq_request(dev, vecidx, param->affinity);
 	if (IS_ERR(eq->irq)) {
 		err = PTR_ERR(eq->irq);
 		goto err_buf;
drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h +1 -0
index 8116815663a7..7028e4b43837 100644
@@ -22,6 +22,7 @@ int mlx5_set_msix_vec_count(struct mlx5_core_dev *dev, int devfn,
 			    int msix_vec_count);
 int mlx5_get_default_msix_vec_count(struct mlx5_core_dev *dev, int num_vfs);
 
+struct mlx5_irq *mlx5_ctrl_irq_request(struct mlx5_core_dev *dev);
 struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, u16 vecidx,
 				  struct cpumask *affinity);
 void mlx5_irq_release(struct mlx5_irq *irq);
drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +53 -9

96 lines not on this page · raw message

Signature · 1 line
2.33.1
#4 lore raw

PATCH 3/15 net/mlx5: Move affinity assignment into irq_request

In reply to Saeed Mahameed

From: Shay Drory <[email protected]>

Move affinity binding of the IRQ to irq_request function in order to bind the IRQ before inserting it to the xarray.

After this change, the IRQ is ready for use when inserted to the xarray.

---
 .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 22 ++++++++-----------
 1 file changed, 9 insertions(+), 13 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +9 -13
index 510a9b91ff9a..656a55114600 100644
@@ -215,7 +215,8 @@ static bool irq_pool_is_sf_pool(struct mlx5_irq_pool *pool)
 	return !strncmp("mlx5_sf", pool->name, strlen("mlx5_sf"));
 }
 
-static struct mlx5_irq *irq_request(struct mlx5_irq_pool *pool, int i)
+static struct mlx5_irq *irq_request(struct mlx5_irq_pool *pool, int i,
+				    struct cpumask *affinity)
 {
 	struct mlx5_core_dev *dev = pool->dev;
 	char name[MLX5_MAX_IRQ_NAME];
@@ -244,6 +245,10 @@ static struct mlx5_irq *irq_request(struct mlx5_irq_pool *pool, int i)
 		err = -ENOMEM;
 		goto err_cpumask;
 	}
+	if (affinity) {
+		cpumask_copy(irq->mask, affinity);
+		irq_set_affinity_hint(irq->irqn, irq->mask);
+	}
 	irq->pool = pool;
 	irq->refcount = 1;
 	irq->index = i;
@@ -255,6 +260,7 @@ static struct mlx5_irq *irq_request(struct mlx5_irq_pool *pool, int i)
 	}
 	return irq;
 err_xa:
+	irq_set_affinity_hint(irq->irqn, NULL);
 	free_cpumask_var(irq->mask);
 err_cpumask:
 	free_irq(irq->irqn, &irq->nh);
@@ -304,7 +310,6 @@ int mlx5_irq_get_index(struct mlx5_irq *irq)
 static struct mlx5_irq *irq_pool_create_irq(struct mlx5_irq_pool *pool,
 					    struct cpumask *affinity)
 {
-	struct mlx5_irq *irq;
 	u32 irq_index;
 	int err;
 
@@ -312,12 +317,7 @@ static struct mlx5_irq *irq_pool_create_irq(struct mlx5_irq_pool *pool,
 		       GFP_KERNEL);
 	if (err)
 		return ERR_PTR(err);
-	irq = irq_request(pool, irq_index);
-	if (IS_ERR(irq))
-		return irq;
-	cpumask_copy(irq->mask, affinity);
-	irq_set_affinity_hint(irq->irqn, irq->mask);
-	return irq;
+	return irq_request(pool, irq_index, affinity);
 }
 
 /* looking for the irq with the smallest refcount and the same affinity */
@@ -392,11 +392,7 @@ irq_pool_request_vector(struct mlx5_irq_pool *pool, int vecidx,
 		irq_get_locked(irq);
 		goto unlock;
 	}
-	irq = irq_request(pool, vecidx);
-	if (IS_ERR(irq) || !affinity)
-		goto unlock;
-	cpumask_copy(irq->mask, affinity);
-	irq_set_affinity_hint(irq->irqn, irq->mask);
+	irq = irq_request(pool, vecidx, affinity);
 unlock:
 	mutex_unlock(&pool->lock);
 	return irq;
Signature · 1 line
2.33.1
#5 lore raw

PATCH 4/15 net/mlx5: Split irq_pool_affinity logic to new file

In reply to Saeed Mahameed

From: Shay Drory <[email protected]>

The downstream patches add more functionality to irq_pool_affinity. Move the irq_pool_affinity logic to a new file in order to ease the coding and maintenance of it.

---
 .../net/ethernet/mellanox/mlx5/core/Makefile  |   2 +-
 .../mellanox/mlx5/core/irq_affinity.c         |  99 ++++++++++++++++
 .../ethernet/mellanox/mlx5/core/mlx5_irq.h    |  11 ++
 .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 111 +++---------------
 .../net/ethernet/mellanox/mlx5/core/pci_irq.h |  31 +++++
 5 files changed, 157 insertions(+), 97 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/pci_irq.h
drivers/net/ethernet/mellanox/mlx5/core/Makefile +1 -1
index 33904bc87efa..fcfd38fa9e6c 100644
@@ -109,7 +109,7 @@ mlx5_core-$(CONFIG_MLX5_SW_STEERING) += steering/dr_domain.o steering/dr_table.o
 #
 # SF device
 #
-mlx5_core-$(CONFIG_MLX5_SF) += sf/vhca_event.o sf/dev/dev.o sf/dev/driver.o
+mlx5_core-$(CONFIG_MLX5_SF) += sf/vhca_event.o sf/dev/dev.o sf/dev/driver.o irq_affinity.o
 
 #
 # SF manager
drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c new file +99 -0

99 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h +11 -0
index 7028e4b43837..db58f5e3f457 100644
@@ -31,4 +31,15 @@ int mlx5_irq_detach_nb(struct mlx5_irq *irq, struct notifier_block *nb);
 struct cpumask *mlx5_irq_get_affinity_mask(struct mlx5_irq *irq);
 int mlx5_irq_get_index(struct mlx5_irq *irq);
 
+struct mlx5_irq_pool;
+#ifdef CONFIG_MLX5_SF
+struct mlx5_irq *mlx5_irq_affinity_request(struct mlx5_irq_pool *pool,
+					   const struct cpumask *req_mask);
+#else
+static inline struct mlx5_irq *
+mlx5_irq_affinity_request(struct mlx5_irq_pool *pool, const struct cpumask *req_mask)
+{
+	return ERR_PTR(-EOPNOTSUPP);
+}
+#endif
 #endif /* __MLX5_IRQ_H__ */
drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +15 -96

178 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/pci_irq.h new file +31 -0
new file mode 100644
index 000000000000..5fee4ce57d6c
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#ifndef __PCI_IRQ_H__
+#define __PCI_IRQ_H__
+
+#include <linux/mlx5/driver.h>
+
+#define MLX5_MAX_IRQ_NAME (32)
+/* max irq_index is 2047, so four chars */
+#define MLX5_MAX_IRQ_IDX_CHARS (4)
+#define MLX5_EQ_REFS_PER_IRQ (2)
+
+struct mlx5_irq;
+
+struct mlx5_irq_pool {
+	char name[MLX5_MAX_IRQ_NAME - MLX5_MAX_IRQ_IDX_CHARS];
+	struct xa_limit xa_num_irqs;
+	struct mutex lock; /* sync IRQs creations */
+	struct xarray irqs;
+	u32 max_threshold;
+	u32 min_threshold;
+	struct mlx5_core_dev *dev;
+};
+
+struct mlx5_irq *mlx5_irq_alloc(struct mlx5_irq_pool *pool, int i,
+				const struct cpumask *affinity);
+int mlx5_irq_get_locked(struct mlx5_irq *irq);
+int mlx5_irq_read_locked(struct mlx5_irq *irq);
+
+#endif /* __PCI_IRQ_H__ */
Signature · 1 line
2.33.1
#6 lore raw

PATCH 5/15 net/mlx5: Introduce API for bulk request and release of IRQs

In reply to Saeed Mahameed

From: Shay Drory <[email protected]>

Currently IRQs are requested one by one. To balance spreading IRQs among cpus using such scheme requires remembering cpu mask for the cpus used for a given device. This complicates the IRQ allocation scheme in subsequent patch.

Hence, prepare the code for bulk IRQs allocation. This enables spreading IRQs among cpus in subsequent patch.

---
 drivers/infiniband/hw/mlx5/odp.c              |  6 --
 drivers/net/ethernet/mellanox/mlx5/core/eq.c  | 98 ++++++++++++-------
 .../ethernet/mellanox/mlx5/core/mlx5_irq.h    |  5 +-
 .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 69 ++++++++++++-
 include/linux/mlx5/eq.h                       |  4 +-
 5 files changed, 135 insertions(+), 47 deletions(-)
drivers/infiniband/hw/mlx5/odp.c +0 -6
index 91eb615b89ee..86842cd580ba 100644
@@ -1541,16 +1541,10 @@ int mlx5r_odp_create_eq(struct mlx5_ib_dev *dev, struct mlx5_ib_pf_eq *eq)
 
 	eq->irq_nb.notifier_call = mlx5_ib_eq_pf_int;
 	param = (struct mlx5_eq_param) {
-		.irq_index = MLX5_IRQ_EQ_CTRL,
 		.nent = MLX5_IB_NUM_PF_EQE,
 	};
 	param.mask[0] = 1ull << MLX5_EVENT_TYPE_PAGE_FAULT;
-	if (!zalloc_cpumask_var(&param.affinity, GFP_KERNEL)) {
-		err = -ENOMEM;
-		goto err_wq;
-	}
 	eq->core = mlx5_eq_create_generic(dev->mdev, &param);
-	free_cpumask_var(param.affinity);
 	if (IS_ERR(eq->core)) {
 		err = PTR_ERR(eq->core);
 		goto err_wq;
drivers/net/ethernet/mellanox/mlx5/core/eq.c +65 -33

226 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h +4 -1
index db58f5e3f457..f2ad3c21802f 100644
@@ -23,9 +23,12 @@ int mlx5_set_msix_vec_count(struct mlx5_core_dev *dev, int devfn,
 int mlx5_get_default_msix_vec_count(struct mlx5_core_dev *dev, int num_vfs);
 
 struct mlx5_irq *mlx5_ctrl_irq_request(struct mlx5_core_dev *dev);
+void mlx5_ctrl_irq_release(struct mlx5_irq *ctrl_irq);
 struct mlx5_irq *mlx5_irq_request(struct mlx5_core_dev *dev, u16 vecidx,
 				  struct cpumask *affinity);
-void mlx5_irq_release(struct mlx5_irq *irq);
+int mlx5_irqs_request_vectors(struct mlx5_core_dev *dev, u16 *cpus, int nirqs,
+			      struct mlx5_irq **irqs);
+void mlx5_irqs_release_vectors(struct mlx5_irq **irqs, int nirqs);
 int mlx5_irq_attach_nb(struct mlx5_irq *irq, struct notifier_block *nb);
 int mlx5_irq_detach_nb(struct mlx5_irq *irq, struct notifier_block *nb);
 struct cpumask *mlx5_irq_get_affinity_mask(struct mlx5_irq *irq);
drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +64 -5

83 lines not on this page · raw message

include/linux/mlx5/eq.h +2 -2
index ea3ff5a8ced3..3705a382276b 100644
@@ -9,13 +9,13 @@
 #define MLX5_NUM_SPARE_EQE (0x80)
 
 struct mlx5_eq;
+struct mlx5_irq;
 struct mlx5_core_dev;
 
 struct mlx5_eq_param {
-	u8             irq_index;
 	int            nent;
 	u64            mask[4];
-	cpumask_var_t  affinity;
+	struct mlx5_irq *irq;
 };
 
 struct mlx5_eq *
Signature · 1 line
2.33.1
#7 lore raw

PATCH 6/15 net/mlx5: SF, Use all available cpu for setting cpu affinity

In reply to Saeed Mahameed

From: Shay Drory <[email protected]>

Currently all SFs are using the same CPUs. Spreading SF over CPUs, in round-robin manner, in order to achieve better distribution of the SFs over available CPUs.

---
 drivers/net/ethernet/mellanox/mlx5/core/eq.c  |  11 +-
 .../mellanox/mlx5/core/irq_affinity.c         | 139 +++++++++++++++++-
 .../ethernet/mellanox/mlx5/core/mlx5_irq.h    |  13 ++
 .../net/ethernet/mellanox/mlx5/core/pci_irq.c |  64 ++++----
 .../net/ethernet/mellanox/mlx5/core/pci_irq.h |   8 +
 5 files changed, 201 insertions(+), 34 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/eq.c +10 -1
index 14547b6f2894..48a45aa54a3c 100644
@@ -798,7 +798,10 @@ static void comp_irqs_release(struct mlx5_core_dev *dev)
 {
 	struct mlx5_eq_table *table = dev->priv.eq_table;
 
-	mlx5_irqs_release_vectors(table->comp_irqs, table->num_comp_eqs);
+	if (mlx5_core_is_sf(dev))
+		mlx5_irq_affinity_irqs_release(dev, table->comp_irqs, table->num_comp_eqs);
+	else
+		mlx5_irqs_release_vectors(table->comp_irqs, table->num_comp_eqs);
 	kfree(table->comp_irqs);
 }
 
@@ -814,6 +817,12 @@ static int comp_irqs_request(struct mlx5_core_dev *dev)
 	table->comp_irqs = kcalloc(ncomp_eqs, sizeof(*table->comp_irqs), GFP_KERNEL);
 	if (!table->comp_irqs)
 		return -ENOMEM;
+	if (mlx5_core_is_sf(dev)) {
+		ret = mlx5_irq_affinity_irqs_request_auto(dev, ncomp_eqs, table->comp_irqs);
+		if (ret < 0)
+			goto free_irqs;
+		return ret;
+	}
 
 	cpus = kcalloc(ncomp_eqs, sizeof(*cpus), GFP_KERNEL);
 	if (!cpus) {
drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c +133 -6

165 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/mlx5_irq.h +13 -0
index f2ad3c21802f..23cb63fa4588 100644
@@ -36,13 +36,26 @@ int mlx5_irq_get_index(struct mlx5_irq *irq);
 
 struct mlx5_irq_pool;
 #ifdef CONFIG_MLX5_SF
+int mlx5_irq_affinity_irqs_request_auto(struct mlx5_core_dev *dev, int nirqs,
+					struct mlx5_irq **irqs);
 struct mlx5_irq *mlx5_irq_affinity_request(struct mlx5_irq_pool *pool,
 					   const struct cpumask *req_mask);
+void mlx5_irq_affinity_irqs_release(struct mlx5_core_dev *dev, struct mlx5_irq **irqs,
+				    int num_irqs);
 #else
+static inline int mlx5_irq_affinity_irqs_request_auto(struct mlx5_core_dev *dev, int nirqs,
+						      struct mlx5_irq **irqs)
+{
+	return -EOPNOTSUPP;
+}
+
 static inline struct mlx5_irq *
 mlx5_irq_affinity_request(struct mlx5_irq_pool *pool, const struct cpumask *req_mask)
 {
 	return ERR_PTR(-EOPNOTSUPP);
 }
+
+static inline void mlx5_irq_affinity_irqs_release(struct mlx5_core_dev *dev,
+						  struct mlx5_irq **irqs, int num_irqs) {}
 #endif
 #endif /* __MLX5_IRQ_H__ */
drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +37 -27

146 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/pci_irq.h +8 -0
index 5fee4ce57d6c..5c7e68bee43a 100644
@@ -20,12 +20,20 @@ struct mlx5_irq_pool {
 	struct xarray irqs;
 	u32 max_threshold;
 	u32 min_threshold;
+	u16 *irqs_per_cpu;
 	struct mlx5_core_dev *dev;
 };
 
+struct mlx5_irq_pool *mlx5_irq_pool_get(struct mlx5_core_dev *dev);
+static inline bool mlx5_irq_pool_is_sf_pool(struct mlx5_irq_pool *pool)
+{
+	return !strncmp("mlx5_sf", pool->name, strlen("mlx5_sf"));
+}
+
 struct mlx5_irq *mlx5_irq_alloc(struct mlx5_irq_pool *pool, int i,
 				const struct cpumask *affinity);
 int mlx5_irq_get_locked(struct mlx5_irq *irq);
 int mlx5_irq_read_locked(struct mlx5_irq *irq);
+int mlx5_irq_put(struct mlx5_irq *irq);
 
 #endif /* __PCI_IRQ_H__ */
Signature · 1 line
2.33.1
#8 lore raw

PATCH 7/15 net/mlx5: Update log_max_qp value to FW max capability

In reply to Saeed Mahameed

From: Maher Sanalla <[email protected]>

log_max_qp in driver's default profile #2 was set to 18, but FW actually supports 17 at the most - a situation that led to the concerning print when the driver is loaded:
"log_max_qp value in current profile is 18, changing to HCA capabaility limit (17)"

The expected behavior from mlx5_profile #2 is to match the maximum FW capability in regards to log_max_qp. Thus, log_max_qp in profile #2 is initialized to a defined static value (0xff) - which basically means that when loading this profile, log_max_qp value will be what the currently installed FW supports at most.

---
 drivers/net/ethernet/mellanox/mlx5/core/main.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/main.c +6 -2
index 6b225bb5751a..2c774f367199 100644
@@ -98,6 +98,8 @@ enum {
 	MLX5_ATOMIC_REQ_MODE_HOST_ENDIANNESS = 0x1,
 };
 
+#define LOG_MAX_SUPPORTED_QPS 0xff
+
 static struct mlx5_profile profile[] = {
 	[0] = {
 		.mask           = 0,
@@ -109,7 +111,7 @@ static struct mlx5_profile profile[] = {
 	[2] = {
 		.mask		= MLX5_PROF_MASK_QP_SIZE |
 				  MLX5_PROF_MASK_MR_CACHE,
-		.log_max_qp	= 18,
+		.log_max_qp	= LOG_MAX_SUPPORTED_QPS,
 		.mr_cache[0]	= {
 			.size	= 500,
 			.limit	= 250
@@ -523,7 +525,9 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)
 		 to_fw_pkey_sz(dev, 128));
 
 	/* Check log_max_qp from HCA caps to set in current profile */
-	if (MLX5_CAP_GEN_MAX(dev, log_max_qp) < prof->log_max_qp) {
+	if (prof->log_max_qp == LOG_MAX_SUPPORTED_QPS) {
+		prof->log_max_qp = MLX5_CAP_GEN_MAX(dev, log_max_qp);
+	} else if (MLX5_CAP_GEN_MAX(dev, log_max_qp) < prof->log_max_qp) {
 		mlx5_core_warn(dev, "log_max_qp value in current profile is %d, changing it to HCA capability limit (%d)\n",
 			       prof->log_max_qp,
 			       MLX5_CAP_GEN_MAX(dev, log_max_qp));
Signature · 1 line
2.33.1
#9 lore raw

PATCH 8/15 net/mlx5e: Expose FEC counters via ethtool

In reply to Saeed Mahameed

From: Lama Kayal <[email protected]>

Add FEC counters' statistics of corrected_blocks and uncorrectable_blocks, along with their lanes via ethtool.

HW supports corrected_blocks and uncorrectable_blocks counters both for RS-FEC mode and FC-FEC mode. In FC mode these counters are accumulated per lane, while in RS mode the correction method crosses lanes, thus only total corrected_blocks and uncorrectable_blocks are reported in this mode.

---
 .../ethernet/mellanox/mlx5/core/en_stats.c    | 101 +++++++++++++++++-
 1 file changed, 98 insertions(+), 3 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +98 -3
index 73fcd9fb17dd..7e7c0c1019f6 100644
@@ -35,6 +35,7 @@
 #include "en_accel/tls.h"
 #include "en_accel/en_accel.h"
 #include "en/ptp.h"
+#include "en/port.h"
 
 static unsigned int stats_grps_num(struct mlx5e_priv *priv)
 {
@@ -1158,12 +1159,99 @@ static MLX5E_DECLARE_STATS_GRP_OP_UPDATE_STATS(phy)
 	mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_PPCNT, 0, 0);
 }
 
-void mlx5e_stats_fec_get(struct mlx5e_priv *priv,
-			 struct ethtool_fec_stats *fec_stats)
+static int fec_num_lanes(struct mlx5_core_dev *dev)
+{
+	u32 out[MLX5_ST_SZ_DW(pmlp_reg)] = {};
+	u32 in[MLX5_ST_SZ_DW(pmlp_reg)] = {};
+	int err;
+
+	MLX5_SET(pmlp_reg, in, local_port, 1);
+	err = mlx5_core_access_reg(dev, in, sizeof(in), out, sizeof(out),
+				   MLX5_REG_PMLP, 0, 0);
+	if (err)
+		return 0;
+
+	return MLX5_GET(pmlp_reg, out, width);
+}
+
+static int fec_active_mode(struct mlx5_core_dev *mdev)
+{
+	unsigned long fec_active_long;
+	u32 fec_active;
+
+	if (mlx5e_get_fec_mode(mdev, &fec_active, NULL))
+		return MLX5E_FEC_NOFEC;
+
+	fec_active_long = fec_active;
+	return find_first_bit(&fec_active_long, sizeof(unsigned long) * BITS_PER_BYTE);
+}
+
+#define MLX5E_STATS_SET_FEC_BLOCK(idx) ({ \
+	fec_stats->corrected_blocks.lanes[(idx)] = \
+		MLX5E_READ_CTR64_BE_F(ppcnt, phys_layer_cntrs, \
+				      fc_fec_corrected_blocks_lane##idx); \
+	fec_stats->uncorrectable_blocks.lanes[(idx)] = \
+		MLX5E_READ_CTR64_BE_F(ppcnt, phys_layer_cntrs, \
+				      fc_fec_uncorrectable_blocks_lane##idx); \
+})
+
+static void fec_set_fc_stats(struct ethtool_fec_stats *fec_stats,
+			     u32 *ppcnt, u8 lanes)
+{
+	if (lanes > 3) { /* 4 lanes */
+		MLX5E_STATS_SET_FEC_BLOCK(3);
+		MLX5E_STATS_SET_FEC_BLOCK(2);
+	}
+	if (lanes > 1) /* 2 lanes */
+		MLX5E_STATS_SET_FEC_BLOCK(1);
+	if (lanes > 0) /* 1 lane */
+		MLX5E_STATS_SET_FEC_BLOCK(0);
+}
+
+static void fec_set_rs_stats(struct ethtool_fec_stats *fec_stats, u32 *ppcnt)
+{
+	fec_stats->corrected_blocks.total =
+		MLX5E_READ_CTR64_BE_F(ppcnt, phys_layer_cntrs,
+				      rs_fec_corrected_blocks);
+	fec_stats->uncorrectable_blocks.total =
+		MLX5E_READ_CTR64_BE_F(ppcnt, phys_layer_cntrs,
+				      rs_fec_uncorrectable_blocks);
+}
+
+static void fec_set_block_stats(struct mlx5e_priv *priv,
+				struct ethtool_fec_stats *fec_stats)
+{
+	struct mlx5_core_dev *mdev = priv->mdev;
+	u32 out[MLX5_ST_SZ_DW(ppcnt_reg)] = {};
+	u32 in[MLX5_ST_SZ_DW(ppcnt_reg)] = {};
+	int sz = MLX5_ST_SZ_BYTES(ppcnt_reg);
+	int mode = fec_active_mode(mdev);
+
+	if (mode == MLX5E_FEC_NOFEC)
+		return;
+
+	MLX5_SET(ppcnt_reg, in, local_port, 1);
+	MLX5_SET(ppcnt_reg, in, grp, MLX5_PHYSICAL_LAYER_COUNTERS_GROUP);
+	if (mlx5_core_access_reg(mdev, in, sz, outl, sz, MLX5_REG_PPCNT, 0, 0))
+		return;
+
+	switch (mode) {
+	case MLX5E_FEC_RS_528_514:
+	case MLX5E_FEC_RS_544_514:
+	case MLX5E_FEC_LLRS_272_257_1:
+		fec_set_rs_stats(fec_stats, out);
+		return;
+	case MLX5E_FEC_FIRECODE:
+		fec_set_fc_stats(fec_stats, out, fec_num_lanes(mdev));
+	}
+}
+
+static void fec_set_corrected_bits_total(struct mlx5e_priv *priv,
+					 struct ethtool_fec_stats *fec_stats)
 {
 	u32 ppcnt_phy_statistical[MLX5_ST_SZ_DW(ppcnt_reg)];
 	struct mlx5_core_dev *mdev = priv->mdev;
-	u32 in[MLX5_ST_SZ_DW(ppcnt_reg)] = {0};
+	u32 in[MLX5_ST_SZ_DW(ppcnt_reg)] = {};
 	int sz = MLX5_ST_SZ_BYTES(ppcnt_reg);
 
 	if (!MLX5_CAP_PCAM_FEATURE(mdev, ppcnt_statistical_group))
@@ -1181,6 +1269,13 @@ void mlx5e_stats_fec_get(struct mlx5e_priv *priv,
 				      phy_corrected_bits);
 }
 
+void mlx5e_stats_fec_get(struct mlx5e_priv *priv,
+			 struct ethtool_fec_stats *fec_stats)
+{
+	fec_set_corrected_bits_total(priv, fec_stats);
+	fec_set_block_stats(priv, fec_stats);
+}
+
 #define PPORT_ETH_EXT_OFF(c) \
 	MLX5_BYTE_OFF(ppcnt_reg, \
 		      counter_set.eth_extended_cntrs_grp_data_layout.c##_high)
Signature · 1 line
2.33.1
#10 lore raw

PATCH 9/15 net/mlx5e: Unblock setting vid 0 for VF in case PF isn't eswitch manager

In reply to Saeed Mahameed

From: Maor Dickman <[email protected]>

When using libvirt to passthrough VF to VM it will always set the VF vlan to 0 even if user didn’t request it, this will cause libvirt to fail to boot in case the PF isn't eswitch owner.

Example of such case is the DPU host PF which isn't eswitch manager, so any attempt to passthrough VF of it using libvirt will fail.

Fix it by not returning error in case set VF vlan is called with vid 0.

---
 drivers/net/ethernet/mellanox/mlx5/core/esw/legacy.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/esw/legacy.c +1 -1
index 2b52f7c09152..9d17206d1625 100644
@@ -431,7 +431,7 @@ int mlx5_eswitch_set_vport_vlan(struct mlx5_eswitch *esw,
 	int err = 0;
 
 	if (!mlx5_esw_allowed(esw))
-		return -EPERM;
+		return vlan ? -EPERM : 0;
 
 	if (vlan || qos)
 		set_flags = SET_VLAN_STRIP | SET_VLAN_INSERT;
Signature · 1 line
2.33.1
#11 lore raw

PATCH 10/15 net/mlx5e: Fix feature check per profile

In reply to Saeed Mahameed

From: Aya Levin <[email protected]>

Remove redundant space when constructing the feature's enum. Validate against the indented enum value.

Fixes 6c72cb05d4b8 ("net/mlx5e: Use bitmap field for profile features")
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +1 -1
index c5f959a9e14b..812e6810cb3b 100644
@@ -984,7 +984,7 @@ struct mlx5e_profile {
 };
 
 #define mlx5e_profile_feature_cap(profile, feature)	\
-	((profile)->features & (MLX5E_PROFILE_FEATURE_## feature))
+	((profile)->features & BIT(MLX5E_PROFILE_FEATURE_##feature))
 
 void mlx5e_build_ptys2ethtool_map(void);
 
Signature · 1 line
2.33.1
#12 lore raw

PATCH 11/15 net/mlx5e: Move HW-GRO and CQE compression check to fix features flow

In reply to Saeed Mahameed

From: Gal Pressman <[email protected]>

Feature dependencies should be resolved in fix features rather than in set features flow. Move the check that disables HW-GRO in case CQE compression is enabled from set_feature_hw_gro() to mlx5e_fix_features().

---
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +5 -5
index 31c911182498..d36489363a26 100644
@@ -3605,11 +3605,6 @@ static int set_feature_hw_gro(struct net_device *netdev, bool enable)
 	new_params = priv->channels.params;
 
 	if (enable) {
-		if (MLX5E_GET_PFLAG(&new_params, MLX5E_PFLAG_RX_CQE_COMPRESS)) {
-			netdev_warn(netdev, "Can't set HW-GRO when CQE compress is active\n");
-			err = -EINVAL;
-			goto out;
-		}
 		new_params.packet_merge.type = MLX5E_PACKET_MERGE_SHAMPO;
 		new_params.packet_merge.shampo.match_criteria_type =
 			MLX5_RQC_SHAMPO_MATCH_CRITERIA_TYPE_EXTENDED;
@@ -3871,6 +3866,11 @@ static netdev_features_t mlx5e_fix_features(struct net_device *netdev,
 		features &= ~NETIF_F_RXHASH;
 		if (netdev->features & NETIF_F_RXHASH)
 			netdev_warn(netdev, "Disabling rxhash, not supported when CQE compress is active\n");
+
+		if (features & NETIF_F_GRO_HW) {
+			netdev_warn(netdev, "Disabling HW-GRO, not supported when CQE compress is active\n");
+			features &= ~NETIF_F_GRO_HW;
+		}
 	}
 
 	if (mlx5e_is_uplink_rep(priv))
Signature · 1 line
2.33.1
#13 lore raw

PATCH 12/15 net/mlx5e: Refactor set_pflag_cqe_based_moder

In reply to Saeed Mahameed

From: Saeed Mahameed <[email protected]>

Rearrange the code and use cqe_mode_to_period_mode() helper.

---
 .../ethernet/mellanox/mlx5/core/en_ethtool.c  | 19 +++++++------------
 1 file changed, 7 insertions(+), 12 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +7 -12
index 536fcb2c5e90..57d755db1cf5 100644
@@ -1883,24 +1883,19 @@ static int set_pflag_cqe_based_moder(struct net_device *netdev, bool enable,
 				     bool is_rx_cq)
 {
 	struct mlx5e_priv *priv = netdev_priv(netdev);
-	struct mlx5_core_dev *mdev = priv->mdev;
-	struct mlx5e_params new_params;
-	bool mode_changed;
 	u8 cq_period_mode, current_cq_period_mode;
+	struct mlx5e_params new_params;
+
+	if (enable && !MLX5_CAP_GEN(priv->mdev, cq_period_start_from_cqe))
+		return -EOPNOTSUPP;
+
+	cq_period_mode = cqe_mode_to_period_mode(enable);
 
-	cq_period_mode = enable ?
-		MLX5_CQ_PERIOD_MODE_START_FROM_CQE :
-		MLX5_CQ_PERIOD_MODE_START_FROM_EQE;
 	current_cq_period_mode = is_rx_cq ?
 		priv->channels.params.rx_cq_moderation.cq_period_mode :
 		priv->channels.params.tx_cq_moderation.cq_period_mode;
-	mode_changed = cq_period_mode != current_cq_period_mode;
-
-	if (cq_period_mode == MLX5_CQ_PERIOD_MODE_START_FROM_CQE &&
-	    !MLX5_CAP_GEN(mdev, cq_period_start_from_cqe))
-		return -EOPNOTSUPP;
 
-	if (!mode_changed)
+	if (cq_period_mode == current_cq_period_mode)
 		return 0;
 
 	new_params = priv->channels.params;
Signature · 1 line
2.33.1
#14 lore raw

PATCH 13/15 net/mlx5e: TC, Remove redundant error logging

In reply to Saeed Mahameed

From: Roi Dayan <[email protected]>

Remove redundant and trivial error logging when trying to offload mirred device with unsupported devices. Using OVS could hit those a lot and the errors are still logged in extack.

---
 .../net/ethernet/mellanox/mlx5/core/en/tc/act/mirred.c    | 8 --------
 1 file changed, 8 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en/tc/act/mirred.c +0 -8
index a0832b86016c..c614fc7fdc9c 100644
@@ -45,14 +45,10 @@ verify_uplink_forwarding(struct mlx5e_priv *priv,
 					termination_table_raw_traffic)) {
 		NL_SET_ERR_MSG_MOD(extack,
 				   "devices are both uplink, can't offload forwarding");
-			pr_err("devices %s %s are both uplink, can't offload forwarding\n",
-			       priv->netdev->name, out_dev->name);
 			return -EOPNOTSUPP;
 	} else if (out_dev != rep_priv->netdev) {
 		NL_SET_ERR_MSG_MOD(extack,
 				   "devices are not the same uplink, can't offload forwarding");
-		pr_err("devices %s %s are both uplink but not the same, can't offload forwarding\n",
-		       priv->netdev->name, out_dev->name);
 		return -EOPNOTSUPP;
 	}
 	return 0;
@@ -160,10 +156,6 @@ tc_act_can_offload_mirred(struct mlx5e_tc_act_parse_state *parse_state,
 	}
 
 	NL_SET_ERR_MSG_MOD(extack, "devices are not on same switch HW, can't offload forwarding");
-	netdev_warn(priv->netdev,
-		    "devices %s %s not on same switch HW, can't offload forwarding\n",
-		    netdev_name(priv->netdev),
-		    out_dev->name);
 
 	return false;
 }
Signature · 1 line
2.33.1
#15 lore raw

PATCH 14/15 net/mlx5e: Add recovery flow in case of error CQE

In reply to Saeed Mahameed

From: Gal Pressman <[email protected]>

The rep legacy RQ completion handling was missing the appropriate handling of error CQEs (dump the CQE and queue a recover work), fix it by calling trigger_report() when needed.

Since all CQE handling flows do the exact same error CQE handling, extract it to a common helper function.

---
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   | 20 ++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +11 -9
index f09b57c31ed7..96e260fd7987 100644
@@ -1603,6 +1603,12 @@ static void trigger_report(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
 	}
 }
 
+static void mlx5e_handle_rx_err_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
+{
+	trigger_report(rq, cqe);
+	rq->stats->wqe_err++;
+}
+
 static void mlx5e_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
 {
 	struct mlx5_wq_cyc *wq = &rq->wqe.wq;
@@ -1616,8 +1622,7 @@ static void mlx5e_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
 	cqe_bcnt = be32_to_cpu(cqe->byte_cnt);
 
 	if (unlikely(MLX5E_RX_ERR_CQE(cqe))) {
-		trigger_report(rq, cqe);
-		rq->stats->wqe_err++;
+		mlx5e_handle_rx_err_cqe(rq, cqe);
 		goto free_wqe;
 	}
 
@@ -1670,7 +1675,7 @@ static void mlx5e_handle_rx_cqe_rep(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
 	cqe_bcnt = be32_to_cpu(cqe->byte_cnt);
 
 	if (unlikely(MLX5E_RX_ERR_CQE(cqe))) {
-		rq->stats->wqe_err++;
+		mlx5e_handle_rx_err_cqe(rq, cqe);
 		goto free_wqe;
 	}
 
@@ -1719,8 +1724,7 @@ static void mlx5e_handle_rx_cqe_mpwrq_rep(struct mlx5e_rq *rq, struct mlx5_cqe64
 	wi->consumed_strides += cstrides;
 
 	if (unlikely(MLX5E_RX_ERR_CQE(cqe))) {
-		trigger_report(rq, cqe);
-		rq->stats->wqe_err++;
+		mlx5e_handle_rx_err_cqe(rq, cqe);
 		goto mpwrq_cqe_out;
 	}
 
@@ -1988,8 +1992,7 @@ static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cq
 	wi->consumed_strides += cstrides;
 
 	if (unlikely(MLX5E_RX_ERR_CQE(cqe))) {
-		trigger_report(rq, cqe);
-		stats->wqe_err++;
+		mlx5e_handle_rx_err_cqe(rq, cqe);
 		goto mpwrq_cqe_out;
 	}
 
@@ -2058,8 +2061,7 @@ static void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cq
 	wi->consumed_strides += cstrides;
 
 	if (unlikely(MLX5E_RX_ERR_CQE(cqe))) {
-		trigger_report(rq, cqe);
-		rq->stats->wqe_err++;
+		mlx5e_handle_rx_err_cqe(rq, cqe);
 		goto mpwrq_cqe_out;
 	}
 
Signature · 1 line
2.33.1
#16 lore raw

PATCH 15/15 Documentation: devlink: mlx5.rst: Fix htmldoc build warning

In reply to Saeed Mahameed

From: Saeed Mahameed <[email protected]>

Fix the following build warning:

Documentation/networking/devlink/mlx5.rst:13: WARNING: Error parsing content block for the "list-table" directive:
+uniform two-level bullet list expected, but row 2 does not contain the same number of items as row 1 (2 vs 3).
...

Add the missing item in the first row.

Fixes 0844fa5f7b89 ("net/mlx5: Let user configure io_eq_size param")
---
 Documentation/networking/devlink/mlx5.rst | 1 +
 1 file changed, 1 insertion(+)
Documentation/networking/devlink/mlx5.rst +1 -0
index 38e94ed65936..29ad304e6fba 100644
@@ -17,6 +17,7 @@ Parameters
      - Validation
    * - ``enable_roce``
      - driverinit
+     - Type: Boolean
    * - ``io_eq_size``
      - driverinit
      - The range is between 64 and 4096.
Signature · 1 line
2.33.1