Skip to contentSkip to conversation

mlx5 updates 2023-06-16

GIT PULL Applied Saeed Mahameed 4 participants 23 messages Jun 16, 2023 – Jun 19, 2023

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

From: Saeed Mahameed <[email protected]>

Hi Dave, Hi Jakub,

This series adds misc updates to mlx5 driver.

For more information please see tag log below.

Please pull and let me know if there is any problem.

Thanks,
Saeed.

The following changes since commit 5a6f6873606e03a0a95afe40ba5e84bb6e28a26f:

ip, ip6: Fix splice to raw and ping sockets (2023-06-16 11:45:16 -0700)

are available in the Git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/mlx5-updates-2023-06-16

for you to fetch changes up to 5f2cf757f9c56255470c23a2a4a5574a34edad4b:

net/mlx5: Remove unused ecpu field from struct mlx5_sf_table (2023-06-16 12:02:09 -0700)

----------------------------------------------------------------
mlx5-updates-2023-06-16

1) Added a new event handler to firmware sync reset, which is used to support firmware sync reset flow on smart NIC. Adding this new stage to the flow enables the firmware to ensure host PFs unload before ECPFs unload, to avoid race of PFs recovery.

2) Debugfs for mlx5 eswitch bridge offloads

3) Added two new counters for vport stats

4) Minor Fixups and cleanups for net-next branch

----------------------------------------------------------------
Daniel Jurgens (2):
net/mlx5: Fix the macro for accessing EC VF vports net/mlx5: DR, update query of HCA caps for EC VFs

Gal Pressman (1):
net/mlx5e: Remove mlx5e_dbg() and msglvl support

Jiri Pirko (1):
net/mlx5: Remove unused ecpu field from struct mlx5_sf_table

Juhee Kang (1):
net/mlx5: Add header file for events

Moshe Shemesh (4):
net/mlx5: Ack on sync_reset_request only if PF can do reset_now net/mlx5: Expose timeout for sync reset unload stage net/mlx5: Check DTOR entry value is not zero net/mlx5: Handle sync reset unload event

Or Har-Toov (2):
net/mlx5: Expose bits for local loopback counter net/mlx5e: Add local loopback counter to vport stats

Saeed Mahameed (1):
net/mlx5: E-Switch, remove redundant else statements

Vlad Buslov (3):
net/mlx5: Create eswitch debugfs root directory net/mlx5: Bridge, pass net device when linking vport to bridge net/mlx5: Bridge, expose FDB state via debugfs

 .../ethernet/mellanox/mlx5/counters.rst            |  10 ++
 drivers/net/ethernet/mellanox/mlx5/core/Makefile   |   3 +-
 drivers/net/ethernet/mellanox/mlx5/core/en.h       |  10 --
 .../ethernet/mellanox/mlx5/core/en/port_buffer.c   |  44 +++---
 .../ethernet/mellanox/mlx5/core/en/rep/bridge.c    |   9 +-
 drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c  |   8 +-
 drivers/net/ethernet/mellanox/mlx5/core/en_dcbnl.c |  26 ++--
 .../net/ethernet/mellanox/mlx5/core/en_ethtool.c   |  18 +--
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c  |   5 +-
 drivers/net/ethernet/mellanox/mlx5/core/en_stats.c |  25 +++-
 .../net/ethernet/mellanox/mlx5/core/esw/bridge.c   |  39 ++---
 .../net/ethernet/mellanox/mlx5/core/esw/bridge.h   |  12 +-
 .../mellanox/mlx5/core/esw/bridge_debugfs.c        |  89 ++++++++++++
 .../ethernet/mellanox/mlx5/core/esw/bridge_priv.h  |   6 +
 .../net/ethernet/mellanox/mlx5/core/esw/legacy.c   |   6 +-
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.c  |   4 +
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.h  |   4 +
 drivers/net/ethernet/mellanox/mlx5/core/events.c   |   2 +-
 drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c | 160 ++++++++++++++++++---
 drivers/net/ethernet/mellanox/mlx5/core/health.c   |   1 +
 drivers/net/ethernet/mellanox/mlx5/core/lag/mp.c   |   2 +-
 .../net/ethernet/mellanox/mlx5/core/lag/mpesw.c    |   2 +-
 .../net/ethernet/mellanox/mlx5/core/lib/events.h   |  40 ++++++
 drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h |  34 -----
 drivers/net/ethernet/mellanox/mlx5/core/lib/tout.c |   7 +-
 drivers/net/ethernet/mellanox/mlx5/core/lib/tout.h |   1 +
 drivers/net/ethernet/mellanox/mlx5/core/main.c     |   3 +
 .../net/ethernet/mellanox/mlx5/core/mlx5_core.h    |   7 +
 .../net/ethernet/mellanox/mlx5/core/sf/devlink.c   |   1 -
 .../ethernet/mellanox/mlx5/core/steering/dr_cmd.c  |   4 +-
 drivers/net/ethernet/mellanox/mlx5/core/vport.c    |   6 -
 include/linux/mlx5/device.h                        |   1 +
 include/linux/mlx5/mlx5_ifc.h                      |  13 +-
 33 files changed, 435 insertions(+), 167 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_debugfs.c
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/lib/events.h
#1 lore raw

PATCH 1/15 net/mlx5: Ack on sync_reset_request only if PF can do reset_now

From: Moshe Shemesh <[email protected]>

Verify at reset_request stage that PF is capable to do reset_now. In case PF is not capable, notify the firmware that the sync reset can not happen and so firmware will abort the sync reset at early stage and will not send reset_now event to any PF.

---
 .../ethernet/mellanox/mlx5/core/fw_reset.c    | 57 ++++++++++++++-----
 1 file changed, 44 insertions(+), 13 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +44 -13
index 50022e7565f1..952cc340b510 100644
@@ -276,6 +276,44 @@ static void mlx5_fw_live_patch_event(struct work_struct *work)
 		mlx5_core_err(dev, "Failed to reload FW tracer\n");
 }
 
+static int mlx5_check_dev_ids(struct mlx5_core_dev *dev, u16 dev_id)
+{
+	struct pci_bus *bridge_bus = dev->pdev->bus;
+	struct pci_dev *sdev;
+	u16 sdev_id;
+	int err;
+
+	/* Check that all functions under the pci bridge are PFs of
+	 * this device otherwise fail this function.
+	 */
+	list_for_each_entry(sdev, &bridge_bus->devices, bus_list) {
+		err = pci_read_config_word(sdev, PCI_DEVICE_ID, &sdev_id);
+		if (err)
+			return err;
+		if (sdev_id != dev_id) {
+			mlx5_core_warn(dev, "unrecognized dev_id (0x%x)\n", sdev_id);
+			return -EPERM;
+		}
+	}
+	return 0;
+}
+
+static bool mlx5_is_reset_now_capable(struct mlx5_core_dev *dev)
+{
+	u16 dev_id;
+	int err;
+
+	if (!MLX5_CAP_GEN(dev, fast_teardown)) {
+		mlx5_core_warn(dev, "fast teardown is not supported by firmware\n");
+		return -EOPNOTSUPP;
+	}
+
+	err = pci_read_config_word(dev->pdev, PCI_DEVICE_ID, &dev_id);
+	if (err)
+		return false;
+	return (!mlx5_check_dev_ids(dev, dev_id));
+}
+
 static void mlx5_sync_reset_request_event(struct work_struct *work)
 {
 	struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset,
@@ -283,7 +321,8 @@ static void mlx5_sync_reset_request_event(struct work_struct *work)
 	struct mlx5_core_dev *dev = fw_reset->dev;
 	int err;
 
-	if (test_bit(MLX5_FW_RESET_FLAGS_NACK_RESET_REQUEST, &fw_reset->reset_flags)) {
+	if (test_bit(MLX5_FW_RESET_FLAGS_NACK_RESET_REQUEST, &fw_reset->reset_flags) ||
+	    !mlx5_is_reset_now_capable(dev)) {
 		err = mlx5_fw_reset_set_reset_sync_nack(dev);
 		mlx5_core_warn(dev, "PCI Sync FW Update Reset Nack %s",
 			       err ? "Failed" : "Sent");
@@ -303,26 +342,18 @@ static int mlx5_pci_link_toggle(struct mlx5_core_dev *dev)
 {
 	struct pci_bus *bridge_bus = dev->pdev->bus;
 	struct pci_dev *bridge = bridge_bus->self;
-	u16 reg16, dev_id, sdev_id;
 	unsigned long timeout;
 	struct pci_dev *sdev;
+	u16 reg16, dev_id;
 	int cap, err;
 	u32 reg32;
 
-	/* Check that all functions under the pci bridge are PFs of
-	 * this device otherwise fail this function.
-	 */
 	err = pci_read_config_word(dev->pdev, PCI_DEVICE_ID, &dev_id);
 	if (err)
 		return err;
-	list_for_each_entry(sdev, &bridge_bus->devices, bus_list) {
-		err = pci_read_config_word(sdev, PCI_DEVICE_ID, &sdev_id);
-		if (err)
-			return err;
-		if (sdev_id != dev_id)
-			return -EPERM;
-	}
-
+	err = mlx5_check_dev_ids(dev, dev_id);
+	if (err)
+		return err;
 	cap = pci_find_capability(bridge, PCI_CAP_ID_EXP);
 	if (!cap)
 		return -EOPNOTSUPP;
Signature · 1 line
2.40.1
#3 lore raw

PATCH 2/15 net/mlx5: Expose timeout for sync reset unload stage

In reply to Saeed Mahameed

From: Moshe Shemesh <[email protected]>

Expose new timoueout in Default Timeouts Register to be used on sync reset flow running on smart NIC. In this flow the driver should know how much time to wait from getting unload request till firmware will ask the PF to continue to next stage of the flow.

---
 drivers/net/ethernet/mellanox/mlx5/core/lib/tout.c | 4 +++-
 drivers/net/ethernet/mellanox/mlx5/core/lib/tout.h | 1 +
 include/linux/mlx5/mlx5_ifc.h                      | 4 +++-
 3 files changed, 7 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/lib/tout.c +3 -1
index 696e45e2bd06..a87d0178ebf3 100644
@@ -24,7 +24,8 @@ static const u32 tout_def_sw_val[MAX_TIMEOUT_TYPES] = {
 	[MLX5_TO_TEARDOWN_MS] = 3000,
 	[MLX5_TO_FSM_REACTIVATE_MS] = 5000,
 	[MLX5_TO_RECLAIM_PAGES_MS] = 5000,
-	[MLX5_TO_RECLAIM_VFS_PAGES_MS] = 120000
+	[MLX5_TO_RECLAIM_VFS_PAGES_MS] = 120000,
+	[MLX5_TO_RESET_UNLOAD_MS] = 300000
 };
 
 static void tout_set(struct mlx5_core_dev *dev, u64 val, enum mlx5_timeouts_types type)
@@ -146,6 +147,7 @@ static int tout_query_dtor(struct mlx5_core_dev *dev)
 	MLX5_TIMEOUT_FILL(fsm_reactivate_to, out, dev, MLX5_TO_FSM_REACTIVATE_MS, 0);
 	MLX5_TIMEOUT_FILL(reclaim_pages_to, out, dev, MLX5_TO_RECLAIM_PAGES_MS, 0);
 	MLX5_TIMEOUT_FILL(reclaim_vfs_pages_to, out, dev, MLX5_TO_RECLAIM_VFS_PAGES_MS, 0);
+	MLX5_TIMEOUT_FILL(reset_unload_to, out, dev, MLX5_TO_RESET_UNLOAD_MS, 0);
 
 	return 0;
 }
drivers/net/ethernet/mellanox/mlx5/core/lib/tout.h +1 -0
index bc9e9aeda847..99e0a05526fe 100644
@@ -26,6 +26,7 @@ enum mlx5_timeouts_types {
 	MLX5_TO_FSM_REACTIVATE_MS,
 	MLX5_TO_RECLAIM_PAGES_MS,
 	MLX5_TO_RECLAIM_VFS_PAGES_MS,
+	MLX5_TO_RESET_UNLOAD_MS,
 
 	MAX_TIMEOUT_TYPES
 };
include/linux/mlx5/mlx5_ifc.h +3 -1
index 1f4f62cb9f34..14892e795808 100644
@@ -3117,7 +3117,9 @@ struct mlx5_ifc_dtor_reg_bits {
 
 	struct mlx5_ifc_default_timeout_bits reclaim_vfs_pages_to;
 
-	u8         reserved_at_1c0[0x40];
+	struct mlx5_ifc_default_timeout_bits reset_unload_to;
+
+	u8         reserved_at_1c0[0x20];
 };
 
 enum {
Signature · 1 line
2.40.1
#4 lore raw

PATCH 3/15 net/mlx5: Check DTOR entry value is not zero

In reply to Saeed Mahameed

From: Moshe Shemesh <[email protected]>

The Default Timeout Register (DTOR) provides timeout values to driver for flows that are device dependent. Zero value for DTOR entry is not valid and should not be used. In case of reading zero value from DTOR, the driver should use the hard coded SW default value instead.

---
 drivers/net/ethernet/mellanox/mlx5/core/lib/tout.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/lib/tout.c +2 -1
index a87d0178ebf3..e223e0e46433 100644
@@ -119,7 +119,8 @@ u64 _mlx5_tout_ms(struct mlx5_core_dev *dev, enum mlx5_timeouts_types type)
 #define MLX5_TIMEOUT_FILL(fld, reg_out, dev, to_type, to_extra) \
 	({ \
 	u64 fw_to = MLX5_TIMEOUT_QUERY(fld, reg_out); \
-	tout_set(dev, fw_to + (to_extra), to_type); \
+	if (fw_to) \
+		tout_set(dev, fw_to + (to_extra), to_type); \
 	fw_to; \
 	})
 
Signature · 1 line
2.40.1
#5 lore raw

PATCH 4/15 net/mlx5: Handle sync reset unload event

In reply to Saeed Mahameed

From: Moshe Shemesh <[email protected]>

Added a new event handler to firmware sync reset, which is used to support firmware sync reset flow on smart NIC. Adding this new stage to the flow enables the firmware to ensure host PFs unload before ECPFs unload, to avoid race of PFs recovery.

If firmware sends sync_reset_unload event to driver the driver should unload and close all HW resources of the function. Once the driver finishes unloading part, it can't get any more events from firmware as event queues are closed, so it polls the reset state field to know when to continue to next stage of the sync reset flow.

Added capability bit for supporting sync_reset_unload event.

---
 .../ethernet/mellanox/mlx5/core/fw_reset.c    | 103 +++++++++++++++++-
 .../net/ethernet/mellanox/mlx5/core/main.c    |   3 +
 include/linux/mlx5/device.h                   |   1 +
 include/linux/mlx5/mlx5_ifc.h                 |   3 +-
 4 files changed, 104 insertions(+), 6 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +98 -5
index 952cc340b510..7af2b14ab5d8 100644
@@ -21,6 +21,7 @@ struct mlx5_fw_reset {
 	struct workqueue_struct *wq;
 	struct work_struct fw_live_patch_work;
 	struct work_struct reset_request_work;
+	struct work_struct reset_unload_work;
 	struct work_struct reset_reload_work;
 	struct work_struct reset_now_work;
 	struct work_struct reset_abort_work;
@@ -30,6 +31,26 @@ struct mlx5_fw_reset {
 	int ret;
 };
 
+enum {
+	MLX5_FW_RST_STATE_IDLE = 0,
+	MLX5_FW_RST_STATE_TOGGLE_REQ = 4,
+};
+
+enum {
+	MLX5_RST_STATE_BIT_NUM = 12,
+	MLX5_RST_ACK_BIT_NUM = 22,
+};
+
+static u8 mlx5_get_fw_rst_state(struct mlx5_core_dev *dev)
+{
+	return (ioread32be(&dev->iseg->initializing) >> MLX5_RST_STATE_BIT_NUM) & 0xF;
+}
+
+static void mlx5_set_fw_rst_ack(struct mlx5_core_dev *dev)
+{
+	iowrite32be(BIT(MLX5_RST_ACK_BIT_NUM), &dev->iseg->initializing);
+}
+
 static int mlx5_fw_reset_enable_remote_dev_reset_set(struct devlink *devlink, u32 id,
 						     struct devlink_param_gset_ctx *ctx)
 {
@@ -155,7 +176,7 @@ int mlx5_fw_reset_set_live_patch(struct mlx5_core_dev *dev)
 	return mlx5_reg_mfrl_set(dev, MLX5_MFRL_REG_RESET_LEVEL0, 0, 0, false);
 }
 
-static void mlx5_fw_reset_complete_reload(struct mlx5_core_dev *dev)
+static void mlx5_fw_reset_complete_reload(struct mlx5_core_dev *dev, bool unloaded)
 {
 	struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset;
 
@@ -163,7 +184,8 @@ static void mlx5_fw_reset_complete_reload(struct mlx5_core_dev *dev)
 	if (test_bit(MLX5_FW_RESET_FLAGS_PENDING_COMP, &fw_reset->reset_flags)) {
 		complete(&fw_reset->done);
 	} else {
-		mlx5_unload_one(dev, false);
+		if (!unloaded)
+			mlx5_unload_one(dev, false);
 		if (mlx5_health_wait_pci_up(dev))
 			mlx5_core_err(dev, "reset reload flow aborted, PCI reads still not working\n");
 		else
@@ -204,7 +226,7 @@ static void mlx5_sync_reset_reload_work(struct work_struct *work)
 
 	mlx5_sync_reset_clear_reset_requested(dev, false);
 	mlx5_enter_error_state(dev, true);
-	mlx5_fw_reset_complete_reload(dev);
+	mlx5_fw_reset_complete_reload(dev, false);
 }
 
 #define MLX5_RESET_POLL_INTERVAL	(HZ / 10)
@@ -458,7 +480,70 @@ static void mlx5_sync_reset_now_event(struct work_struct *work)
 	mlx5_enter_error_state(dev, true);
 done:
 	fw_reset->ret = err;
-	mlx5_fw_reset_complete_reload(dev);
+	mlx5_fw_reset_complete_reload(dev, false);
+}
+
+static void mlx5_sync_reset_unload_event(struct work_struct *work)
+{
+	struct mlx5_fw_reset *fw_reset;
+	struct mlx5_core_dev *dev;
+	unsigned long timeout;
+	bool reset_action;
+	u8 rst_state;
+	int err;
+
+	fw_reset = container_of(work, struct mlx5_fw_reset, reset_unload_work);
+	dev = fw_reset->dev;
+
+	if (mlx5_sync_reset_clear_reset_requested(dev, false))
+		return;
+
+	mlx5_core_warn(dev, "Sync Reset Unload. Function is forced down.\n");
+
+	err = mlx5_cmd_fast_teardown_hca(dev);
+	if (err)
+		mlx5_core_warn(dev, "Fast teardown failed, unloading, err %d\n", err);
+	else
+		mlx5_enter_error_state(dev, true);
+
+	if (test_bit(MLX5_FW_RESET_FLAGS_PENDING_COMP, &fw_reset->reset_flags))
+		mlx5_unload_one_devl_locked(dev, false);
+	else
+		mlx5_unload_one(dev, false);
+
+	mlx5_set_fw_rst_ack(dev);
+	mlx5_core_warn(dev, "Sync Reset Unload done, device reset expected\n");
+
+	reset_action = false;
+	timeout = jiffies + msecs_to_jiffies(mlx5_tout_ms(dev, RESET_UNLOAD));
+	do {
+		rst_state = mlx5_get_fw_rst_state(dev);
+		if (rst_state == MLX5_FW_RST_STATE_TOGGLE_REQ ||
+		    rst_state == MLX5_FW_RST_STATE_IDLE) {
+			reset_action = true;
+			break;
+		}
+		msleep(20);
+	} while (!time_after(jiffies, timeout));
+
+	if (!reset_action) {
+		mlx5_core_err(dev, "Got timeout waiting for sync reset action, state = %u\n",
+			      rst_state);
+		fw_reset->ret = -ETIMEDOUT;
+		goto done;
+	}
+
+	mlx5_core_warn(dev, "Sync Reset, got reset action. rst_state = %u\n", rst_state);
+	if (rst_state == MLX5_FW_RST_STATE_TOGGLE_REQ) {
+		err = mlx5_pci_link_toggle(dev);
+		if (err) {
+			mlx5_core_warn(dev, "mlx5_pci_link_toggle failed, err %d\n", err);
+			fw_reset->ret = err;
+		}
+	}
+
+done:
+	mlx5_fw_reset_complete_reload(dev, true);
 }
 
 static void mlx5_sync_reset_abort_event(struct work_struct *work)
@@ -483,6 +568,9 @@ static void mlx5_sync_reset_events_handle(struct mlx5_fw_reset *fw_reset, struct
 	case MLX5_SYNC_RST_STATE_RESET_REQUEST:
 		queue_work(fw_reset->wq, &fw_reset->reset_request_work);
 		break;
+	case MLX5_SYNC_RST_STATE_RESET_UNLOAD:
+		queue_work(fw_reset->wq, &fw_reset->reset_unload_work);
+		break;
 	case MLX5_SYNC_RST_STATE_RESET_NOW:
 		queue_work(fw_reset->wq, &fw_reset->reset_now_work);
 		break;
@@ -517,10 +605,13 @@ static int fw_reset_event_notifier(struct notifier_block *nb, unsigned long acti
 int mlx5_fw_reset_wait_reset_done(struct mlx5_core_dev *dev)
 {
 	unsigned long pci_sync_update_timeout = mlx5_tout_ms(dev, PCI_SYNC_UPDATE);
-	unsigned long timeout = msecs_to_jiffies(pci_sync_update_timeout);
 	struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset;
+	unsigned long timeout;
 	int err;
 
+	if (MLX5_CAP_GEN(dev, pci_sync_for_fw_update_with_driver_unload))
+		pci_sync_update_timeout += mlx5_tout_ms(dev, RESET_UNLOAD);
+	timeout = msecs_to_jiffies(pci_sync_update_timeout);
 	if (!wait_for_completion_timeout(&fw_reset->done, timeout)) {
 		mlx5_core_warn(dev, "FW sync reset timeout after %lu seconds\n",
 			       pci_sync_update_timeout / 1000);
@@ -557,6 +648,7 @@ void mlx5_drain_fw_reset(struct mlx5_core_dev *dev)
 	set_bit(MLX5_FW_RESET_FLAGS_DROP_NEW_REQUESTS, &fw_reset->reset_flags);
 	cancel_work_sync(&fw_reset->fw_live_patch_work);
 	cancel_work_sync(&fw_reset->reset_request_work);
+	cancel_work_sync(&fw_reset->reset_unload_work);
 	cancel_work_sync(&fw_reset->reset_reload_work);
 	cancel_work_sync(&fw_reset->reset_now_work);
 	cancel_work_sync(&fw_reset->reset_abort_work);
@@ -595,6 +687,7 @@ int mlx5_fw_reset_init(struct mlx5_core_dev *dev)
 
 	INIT_WORK(&fw_reset->fw_live_patch_work, mlx5_fw_live_patch_event);
 	INIT_WORK(&fw_reset->reset_request_work, mlx5_sync_reset_request_event);
+	INIT_WORK(&fw_reset->reset_unload_work, mlx5_sync_reset_unload_event);
 	INIT_WORK(&fw_reset->reset_reload_work, mlx5_sync_reset_reload_work);
 	INIT_WORK(&fw_reset->reset_now_work, mlx5_sync_reset_now_event);
 	INIT_WORK(&fw_reset->reset_abort_work, mlx5_sync_reset_abort_event);
drivers/net/ethernet/mellanox/mlx5/core/main.c +3 -0
index 6fa314f8e5ee..88dbea6631d5 100644
@@ -619,6 +619,9 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)
 
 	if (MLX5_CAP_GEN_MAX(dev, pci_sync_for_fw_update_event))
 		MLX5_SET(cmd_hca_cap, set_hca_cap, pci_sync_for_fw_update_event, 1);
+	if (MLX5_CAP_GEN_MAX(dev, pci_sync_for_fw_update_with_driver_unload))
+		MLX5_SET(cmd_hca_cap, set_hca_cap,
+			 pci_sync_for_fw_update_with_driver_unload, 1);
 
 	if (MLX5_CAP_GEN_MAX(dev, num_vhca_ports))
 		MLX5_SET(cmd_hca_cap,
include/linux/mlx5/device.h +1 -0
index c0af74efd3cb..80cc12a9a531 100644
@@ -716,6 +716,7 @@ enum sync_rst_state_type {
 	MLX5_SYNC_RST_STATE_RESET_REQUEST	= 0x0,
 	MLX5_SYNC_RST_STATE_RESET_NOW		= 0x1,
 	MLX5_SYNC_RST_STATE_RESET_ABORT		= 0x2,
+	MLX5_SYNC_RST_STATE_RESET_UNLOAD	= 0x3,
 };
 
 struct mlx5_eqe_sync_fw_update {
include/linux/mlx5/mlx5_ifc.h +2 -1
index 14892e795808..d61dcb5d7cd5 100644
@@ -1755,7 +1755,8 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 	u8         reserved_at_328[0x2];
 	u8	   relaxed_ordering_read[0x1];
 	u8         log_max_pd[0x5];
-	u8         reserved_at_330[0x7];
+	u8         reserved_at_330[0x6];
+	u8         pci_sync_for_fw_update_with_driver_unload[0x1];
 	u8         vnic_env_cnt_steering_fail[0x1];
 	u8         reserved_at_338[0x1];
 	u8         q_counter_aggregation[0x1];
Signature · 1 line
2.40.1
#6 lore raw

PATCH 5/15 net/mlx5: Create eswitch debugfs root directory

In reply to Saeed Mahameed

From: Vlad Buslov <[email protected]>

Following patch in series uses the new directory for bridge FDB debugfs. The new directory is intended for all future eswitch-specific debugfs files.

---
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 4 ++++
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.h | 3 +++
 2 files changed, 7 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +4 -0
index 2af9c4646bc7..5aaedbf71783 100644
@@ -31,6 +31,7 @@
  */
 
 #include <linux/etherdevice.h>
+#include <linux/debugfs.h>
 #include <linux/mlx5/driver.h>
 #include <linux/mlx5/mlx5_ifc.h>
 #include <linux/mlx5/vport.h>
@@ -1765,6 +1766,7 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev)
 	esw->manager_vport = mlx5_eswitch_manager_vport(dev);
 	esw->first_host_vport = mlx5_eswitch_first_host_vport_num(dev);
 
+	esw->debugfs_root = debugfs_create_dir("esw", mlx5_debugfs_get_dev_root(dev));
 	esw->work_queue = create_singlethread_workqueue("mlx5_esw_wq");
 	if (!esw->work_queue) {
 		err = -ENOMEM;
@@ -1818,6 +1820,7 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev)
 abort:
 	if (esw->work_queue)
 		destroy_workqueue(esw->work_queue);
+	debugfs_remove_recursive(esw->debugfs_root);
 	kfree(esw);
 unregister_param:
 	devl_params_unregister(priv_to_devlink(dev), mlx5_eswitch_params,
@@ -1844,6 +1847,7 @@ void mlx5_eswitch_cleanup(struct mlx5_eswitch *esw)
 	mutex_destroy(&esw->offloads.decap_tbl_lock);
 	esw_offloads_cleanup(esw);
 	mlx5_esw_vports_cleanup(esw);
+	debugfs_remove_recursive(esw->debugfs_root);
 	kfree(esw);
 	devl_params_unregister(priv_to_devlink(esw->dev), mlx5_eswitch_params,
 			       ARRAY_SIZE(mlx5_eswitch_params));
drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +3 -0
index 266b60fefe25..bcbab06759c4 100644
@@ -304,6 +304,8 @@ enum {
 	MLX5_ESW_FDB_CREATED = BIT(0),
 };
 
+struct dentry;
+
 struct mlx5_eswitch {
 	struct mlx5_core_dev    *dev;
 	struct mlx5_nb          nb;
@@ -312,6 +314,7 @@ struct mlx5_eswitch {
 	struct hlist_head       mc_table[MLX5_L2_ADDR_HASH_SIZE];
 	struct esw_mc_addr mc_promisc;
 	/* end of legacy */
+	struct dentry *debugfs_root;
 	struct workqueue_struct *work_queue;
 	struct xarray vports;
 	u32 flags;
Signature · 1 line
2.40.1
#7 lore raw

PATCH 6/15 net/mlx5: Bridge, pass net device when linking vport to bridge

In reply to Saeed Mahameed

From: Vlad Buslov <[email protected]>

Following patch requires access to additional data in bridge net_device. Pass the whole structure down the stack instead of adding necessary fields as function arguments one-by-one.

---
 .../mellanox/mlx5/core/en/rep/bridge.c        |  9 +++--
 .../ethernet/mellanox/mlx5/core/esw/bridge.c  | 35 ++++++++++---------
 .../ethernet/mellanox/mlx5/core/esw/bridge.h  | 10 +++---
 3 files changed, 29 insertions(+), 25 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en/rep/bridge.c +4 -5
index fd191925ab4b..560800246573 100644
@@ -136,7 +136,6 @@ static int mlx5_esw_bridge_port_changeupper(struct notifier_block *nb, void *ptr
 	struct mlx5_eswitch *esw = br_offloads->esw;
 	u16 vport_num, esw_owner_vhca_id;
 	struct netlink_ext_ack *extack;
-	int ifindex = upper->ifindex;
 	int err = 0;
 
 	if (!netif_is_bridge_master(upper))
@@ -150,15 +149,15 @@ static int mlx5_esw_bridge_port_changeupper(struct notifier_block *nb, void *ptr
 
 	if (mlx5_esw_bridge_is_local(dev, rep, esw))
 		err = info->linking ?
-			mlx5_esw_bridge_vport_link(ifindex, vport_num, esw_owner_vhca_id,
+			mlx5_esw_bridge_vport_link(upper, vport_num, esw_owner_vhca_id,
 						   br_offloads, extack) :
-			mlx5_esw_bridge_vport_unlink(ifindex, vport_num, esw_owner_vhca_id,
+			mlx5_esw_bridge_vport_unlink(upper, vport_num, esw_owner_vhca_id,
 						     br_offloads, extack);
 	else if (mlx5_esw_bridge_dev_same_hw(rep, esw))
 		err = info->linking ?
-			mlx5_esw_bridge_vport_peer_link(ifindex, vport_num, esw_owner_vhca_id,
+			mlx5_esw_bridge_vport_peer_link(upper, vport_num, esw_owner_vhca_id,
 							br_offloads, extack) :
-			mlx5_esw_bridge_vport_peer_unlink(ifindex, vport_num, esw_owner_vhca_id,
+			mlx5_esw_bridge_vport_peer_unlink(upper, vport_num, esw_owner_vhca_id,
 							  br_offloads, extack);
 
 	return err;
drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c +19 -16
index bea7cc645461..eaa9b328abd5 100644
@@ -834,7 +834,7 @@ mlx5_esw_bridge_egress_miss_flow_create(struct mlx5_flow_table *egress_ft,
 	return handle;
 }
 
-static struct mlx5_esw_bridge *mlx5_esw_bridge_create(int ifindex,
+static struct mlx5_esw_bridge *mlx5_esw_bridge_create(struct net_device *br_netdev,
 						      struct mlx5_esw_bridge_offloads *br_offloads)
 {
 	struct mlx5_esw_bridge *bridge;
@@ -858,7 +858,7 @@ static struct mlx5_esw_bridge *mlx5_esw_bridge_create(int ifindex,
 		goto err_mdb_ht;
 
 	INIT_LIST_HEAD(&bridge->fdb_list);
-	bridge->ifindex = ifindex;
+	bridge->ifindex = br_netdev->ifindex;
 	bridge->refcnt = 1;
 	bridge->ageing_time = clock_t_to_jiffies(BR_DEFAULT_AGEING_TIME);
 	bridge->vlan_proto = ETH_P_8021Q;
@@ -898,14 +898,14 @@ static void mlx5_esw_bridge_put(struct mlx5_esw_bridge_offloads *br_offloads,
 }
 
 static struct mlx5_esw_bridge *
-mlx5_esw_bridge_lookup(int ifindex, struct mlx5_esw_bridge_offloads *br_offloads)
+mlx5_esw_bridge_lookup(struct net_device *br_netdev, struct mlx5_esw_bridge_offloads *br_offloads)
 {
 	struct mlx5_esw_bridge *bridge;
 
 	ASSERT_RTNL();
 
 	list_for_each_entry(bridge, &br_offloads->bridges, list) {
-		if (bridge->ifindex == ifindex) {
+		if (bridge->ifindex == br_netdev->ifindex) {
 			mlx5_esw_bridge_get(bridge);
 			return bridge;
 		}
@@ -918,7 +918,7 @@ mlx5_esw_bridge_lookup(int ifindex, struct mlx5_esw_bridge_offloads *br_offloads
 			return ERR_PTR(err);
 	}
 
-	bridge = mlx5_esw_bridge_create(ifindex, br_offloads);
+	bridge = mlx5_esw_bridge_create(br_netdev, br_offloads);
 	if (IS_ERR(bridge) && list_empty(&br_offloads->bridges))
 		mlx5_esw_bridge_ingress_table_cleanup(br_offloads);
 	return bridge;
@@ -1601,15 +1601,15 @@ static int mlx5_esw_bridge_vport_cleanup(struct mlx5_esw_bridge_offloads *br_off
 	return 0;
 }
 
-static int mlx5_esw_bridge_vport_link_with_flags(int ifindex, u16 vport_num, u16 esw_owner_vhca_id,
-						 u16 flags,
+static int mlx5_esw_bridge_vport_link_with_flags(struct net_device *br_netdev, u16 vport_num,
+						 u16 esw_owner_vhca_id, u16 flags,
 						 struct mlx5_esw_bridge_offloads *br_offloads,
 						 struct netlink_ext_ack *extack)
 {
 	struct mlx5_esw_bridge *bridge;
 	int err;
 
-	bridge = mlx5_esw_bridge_lookup(ifindex, br_offloads);
+	bridge = mlx5_esw_bridge_lookup(br_netdev, br_offloads);
 	if (IS_ERR(bridge)) {
 		NL_SET_ERR_MSG_MOD(extack, "Error checking for existing bridge with same ifindex");
 		return PTR_ERR(bridge);
@@ -1627,15 +1627,16 @@ static int mlx5_esw_bridge_vport_link_with_flags(int ifindex, u16 vport_num, u16
 	return err;
 }
 
-int mlx5_esw_bridge_vport_link(int ifindex, u16 vport_num, u16 esw_owner_vhca_id,
+int mlx5_esw_bridge_vport_link(struct net_device *br_netdev, u16 vport_num, u16 esw_owner_vhca_id,
 			       struct mlx5_esw_bridge_offloads *br_offloads,
 			       struct netlink_ext_ack *extack)
 {
-	return mlx5_esw_bridge_vport_link_with_flags(ifindex, vport_num, esw_owner_vhca_id, 0,
+	return mlx5_esw_bridge_vport_link_with_flags(br_netdev, vport_num, esw_owner_vhca_id, 0,
 						     br_offloads, extack);
 }
 
-int mlx5_esw_bridge_vport_unlink(int ifindex, u16 vport_num, u16 esw_owner_vhca_id,
+int mlx5_esw_bridge_vport_unlink(struct net_device *br_netdev, u16 vport_num,
+				 u16 esw_owner_vhca_id,
 				 struct mlx5_esw_bridge_offloads *br_offloads,
 				 struct netlink_ext_ack *extack)
 {
@@ -1647,7 +1648,7 @@ int mlx5_esw_bridge_vport_unlink(int ifindex, u16 vport_num, u16 esw_owner_vhca_
 		NL_SET_ERR_MSG_MOD(extack, "Port is not attached to any bridge");
 		return -EINVAL;
 	}
-	if (port->bridge->ifindex != ifindex) {
+	if (port->bridge->ifindex != br_netdev->ifindex) {
 		NL_SET_ERR_MSG_MOD(extack, "Port is attached to another bridge");
 		return -EINVAL;
 	}
@@ -1658,23 +1659,25 @@ int mlx5_esw_bridge_vport_unlink(int ifindex, u16 vport_num, u16 esw_owner_vhca_
 	return err;
 }
 
-int mlx5_esw_bridge_vport_peer_link(int ifindex, u16 vport_num, u16 esw_owner_vhca_id,
+int mlx5_esw_bridge_vport_peer_link(struct net_device *br_netdev, u16 vport_num,
+				    u16 esw_owner_vhca_id,
 				    struct mlx5_esw_bridge_offloads *br_offloads,
 				    struct netlink_ext_ack *extack)
 {
 	if (!MLX5_CAP_ESW(br_offloads->esw->dev, merged_eswitch))
 		return 0;
 
-	return mlx5_esw_bridge_vport_link_with_flags(ifindex, vport_num, esw_owner_vhca_id,
+	return mlx5_esw_bridge_vport_link_with_flags(br_netdev, vport_num, esw_owner_vhca_id,
 						     MLX5_ESW_BRIDGE_PORT_FLAG_PEER,
 						     br_offloads, extack);
 }
 
-int mlx5_esw_bridge_vport_peer_unlink(int ifindex, u16 vport_num, u16 esw_owner_vhca_id,
+int mlx5_esw_bridge_vport_peer_unlink(struct net_device *br_netdev, u16 vport_num,
+				      u16 esw_owner_vhca_id,
 				      struct mlx5_esw_bridge_offloads *br_offloads,
 				      struct netlink_ext_ack *extack)
 {
-	return mlx5_esw_bridge_vport_unlink(ifindex, vport_num, esw_owner_vhca_id, br_offloads,
+	return mlx5_esw_bridge_vport_unlink(br_netdev, vport_num, esw_owner_vhca_id, br_offloads,
 					    extack);
 }
 
drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h +6 -4
index a9dd18c73d6a..2f7ad3bdba5e 100644
@@ -43,16 +43,18 @@ struct mlx5_esw_bridge_offloads {
 
 struct mlx5_esw_bridge_offloads *mlx5_esw_bridge_init(struct mlx5_eswitch *esw);
 void mlx5_esw_bridge_cleanup(struct mlx5_eswitch *esw);
-int mlx5_esw_bridge_vport_link(int ifindex, u16 vport_num, u16 esw_owner_vhca_id,
+int mlx5_esw_bridge_vport_link(struct net_device *br_netdev, u16 vport_num, u16 esw_owner_vhca_id,
 			       struct mlx5_esw_bridge_offloads *br_offloads,
 			       struct netlink_ext_ack *extack);
-int mlx5_esw_bridge_vport_unlink(int ifindex, u16 vport_num, u16 esw_owner_vhca_id,
+int mlx5_esw_bridge_vport_unlink(struct net_device *br_netdev, u16 vport_num, u16 esw_owner_vhca_id,
 				 struct mlx5_esw_bridge_offloads *br_offloads,
 				 struct netlink_ext_ack *extack);
-int mlx5_esw_bridge_vport_peer_link(int ifindex, u16 vport_num, u16 esw_owner_vhca_id,
+int mlx5_esw_bridge_vport_peer_link(struct net_device *br_netdev, u16 vport_num,
+				    u16 esw_owner_vhca_id,
 				    struct mlx5_esw_bridge_offloads *br_offloads,
 				    struct netlink_ext_ack *extack);
-int mlx5_esw_bridge_vport_peer_unlink(int ifindex, u16 vport_num, u16 esw_owner_vhca_id,
+int mlx5_esw_bridge_vport_peer_unlink(struct net_device *br_netdev, u16 vport_num,
+				      u16 esw_owner_vhca_id,
 				      struct mlx5_esw_bridge_offloads *br_offloads,
 				      struct netlink_ext_ack *extack);
 void mlx5_esw_bridge_fdb_update_used(struct net_device *dev, u16 vport_num, u16 esw_owner_vhca_id,
Signature · 1 line
2.40.1
#8 lore raw

PATCH 7/15 net/mlx5: Bridge, expose FDB state via debugfs

In reply to Saeed Mahameed

From: Vlad Buslov <[email protected]>

For debugging purposes expose offloaded FDB state (flags, counters, etc.) via debugfs inside 'esw' root directory. Example debugfs file output:

$ cat mlx5/0000\:08\:00.0/esw/bridge/bridge1/fdb
DEV              MAC               VLAN              PACKETS                BYTES              LASTUSE FLAGS
enp8s0f0_1       e4:0a:05:08:00:06    2                    2                  204           4295567112   0x0
enp8s0f0_0       e4:0a:05:08:00:03    2                    3                  278           4295567112   0x0
---
 .../net/ethernet/mellanox/mlx5/core/Makefile  |  3 +-
 .../ethernet/mellanox/mlx5/core/esw/bridge.c  |  4 +
 .../ethernet/mellanox/mlx5/core/esw/bridge.h  |  2 +
 .../mellanox/mlx5/core/esw/bridge_debugfs.c   | 89 +++++++++++++++++++
 .../mellanox/mlx5/core/esw/bridge_priv.h      |  6 ++
 5 files changed, 103 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_debugfs.c
drivers/net/ethernet/mellanox/mlx5/core/Makefile +2 -1
index ddf1e352f51d..35f00700a4d6 100644
@@ -75,7 +75,8 @@ mlx5_core-$(CONFIG_MLX5_ESWITCH)   += esw/acl/helper.o \
 				      esw/acl/egress_lgcy.o esw/acl/egress_ofld.o \
 				      esw/acl/ingress_lgcy.o esw/acl/ingress_ofld.o
 
-mlx5_core-$(CONFIG_MLX5_BRIDGE)    += esw/bridge.o esw/bridge_mcast.o en/rep/bridge.o
+mlx5_core-$(CONFIG_MLX5_BRIDGE)    += esw/bridge.o esw/bridge_mcast.o esw/bridge_debugfs.o \
+				      en/rep/bridge.o
 
 mlx5_core-$(CONFIG_THERMAL)        += thermal.o
 mlx5_core-$(CONFIG_MLX5_MPFS)      += lib/mpfs.o
drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.c +4 -0
index eaa9b328abd5..f4fe1daa4afd 100644
@@ -863,6 +863,7 @@ static struct mlx5_esw_bridge *mlx5_esw_bridge_create(struct net_device *br_netd
 	bridge->ageing_time = clock_t_to_jiffies(BR_DEFAULT_AGEING_TIME);
 	bridge->vlan_proto = ETH_P_8021Q;
 	list_add(&bridge->list, &br_offloads->bridges);
+	mlx5_esw_bridge_debugfs_init(br_netdev, bridge);
 
 	return bridge;
 
@@ -886,6 +887,7 @@ static void mlx5_esw_bridge_put(struct mlx5_esw_bridge_offloads *br_offloads,
 	if (--bridge->refcnt)
 		return;
 
+	mlx5_esw_bridge_debugfs_cleanup(bridge);
 	mlx5_esw_bridge_egress_table_cleanup(bridge);
 	mlx5_esw_bridge_mcast_disable(bridge);
 	list_del(&bridge->list);
@@ -1904,6 +1906,7 @@ struct mlx5_esw_bridge_offloads *mlx5_esw_bridge_init(struct mlx5_eswitch *esw)
 	xa_init(&br_offloads->ports);
 	br_offloads->esw = esw;
 	esw->br_offloads = br_offloads;
+	mlx5_esw_bridge_debugfs_offloads_init(br_offloads);
 
 	return br_offloads;
 }
@@ -1919,6 +1922,7 @@ void mlx5_esw_bridge_cleanup(struct mlx5_eswitch *esw)
 
 	mlx5_esw_bridge_flush(br_offloads);
 	WARN_ON(!xa_empty(&br_offloads->ports));
+	mlx5_esw_bridge_debugfs_offloads_cleanup(br_offloads);
 
 	esw->br_offloads = NULL;
 	kvfree(br_offloads);
drivers/net/ethernet/mellanox/mlx5/core/esw/bridge.h +2 -0
index 2f7ad3bdba5e..c2c7c70d99eb 100644
@@ -10,6 +10,7 @@
 #include <linux/xarray.h>
 #include "eswitch.h"
 
+struct dentry;
 struct mlx5_flow_table;
 struct mlx5_flow_group;
 
@@ -17,6 +18,7 @@ struct mlx5_esw_bridge_offloads {
 	struct mlx5_eswitch *esw;
 	struct list_head bridges;
 	struct xarray ports;
+	struct dentry *debugfs_root;
 
 	struct notifier_block netdev_nb;
 	struct notifier_block nb_blk;
drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_debugfs.c new file +89 -0

89 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/esw/bridge_priv.h +6 -0
index c9595801bdb4..4911cc32161b 100644
@@ -199,6 +199,7 @@ struct mlx5_esw_bridge {
 	int refcnt;
 	struct list_head list;
 	struct mlx5_esw_bridge_offloads *br_offloads;
+	struct dentry *debugfs_dir;
 
 	struct list_head fdb_list;
 	struct rhashtable fdb_ht;
@@ -241,4 +242,9 @@ void mlx5_esw_bridge_port_mdb_vlan_flush(struct mlx5_esw_bridge_port *port,
 					 struct mlx5_esw_bridge_vlan *vlan);
 void mlx5_esw_bridge_mdb_flush(struct mlx5_esw_bridge *bridge);
 
+void mlx5_esw_bridge_debugfs_offloads_init(struct mlx5_esw_bridge_offloads *br_offloads);
+void mlx5_esw_bridge_debugfs_offloads_cleanup(struct mlx5_esw_bridge_offloads *br_offloads);
+void mlx5_esw_bridge_debugfs_init(struct net_device *br_netdev, struct mlx5_esw_bridge *bridge);
+void mlx5_esw_bridge_debugfs_cleanup(struct mlx5_esw_bridge *bridge);
+
 #endif /* _MLX5_ESW_BRIDGE_PRIVATE_ */
Signature · 1 line
2.40.1
#15 lore raw

PATCH 8/15 net/mlx5: E-Switch, remove redundant else statements

In reply to Saeed Mahameed

From: Saeed Mahameed <[email protected]>

These else statement blocks are redundant since the if block already jumps to the function abort label.

---
 drivers/net/ethernet/mellanox/mlx5/core/esw/legacy.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/esw/legacy.c +2 -4
index fabe49a35a5c..255bc8b749f9 100644
@@ -285,9 +285,8 @@ static int _mlx5_eswitch_set_vepa_locked(struct mlx5_eswitch *esw,
 	if (IS_ERR(flow_rule)) {
 		err = PTR_ERR(flow_rule);
 		goto out;
-	} else {
-		esw->fdb_table.legacy.vepa_uplink_rule = flow_rule;
 	}
+	esw->fdb_table.legacy.vepa_uplink_rule = flow_rule;
 
 	/* Star rule to forward all traffic to uplink vport */
 	memset(&dest, 0, sizeof(dest));
@@ -299,9 +298,8 @@ static int _mlx5_eswitch_set_vepa_locked(struct mlx5_eswitch *esw,
 	if (IS_ERR(flow_rule)) {
 		err = PTR_ERR(flow_rule);
 		goto out;
-	} else {
-		esw->fdb_table.legacy.vepa_star_rule = flow_rule;
 	}
+	esw->fdb_table.legacy.vepa_star_rule = flow_rule;
 
 out:
 	kvfree(spec);
Signature · 1 line
2.40.1
#16 lore raw

PATCH 9/15 net/mlx5e: Remove mlx5e_dbg() and msglvl support

In reply to Saeed Mahameed

From: Gal Pressman <[email protected]>

The msglvl support was implemented using the mlx5e_dbg() macro which is rarely used in the driver, and is not very useful when you can just use dynamic debug instead.
Remove mlx5e_dbg() and convert its usages to netdev_dbg().

---
 drivers/net/ethernet/mellanox/mlx5/core/en.h  | 10 -----
 .../mellanox/mlx5/core/en/port_buffer.c       | 44 ++++++++++---------
 .../net/ethernet/mellanox/mlx5/core/en_arfs.c |  8 ++--
 .../ethernet/mellanox/mlx5/core/en_dcbnl.c    | 26 +++++------
 .../ethernet/mellanox/mlx5/core/en_ethtool.c  | 18 ++------
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  5 +--
 6 files changed, 45 insertions(+), 66 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en.h +0 -10
index ceabe57c511a..b1807bfb815f 100644
@@ -165,15 +165,6 @@ struct page_pool;
 #define MLX5E_MAX_KLM_PER_WQE(mdev) \
 	MLX5E_KLM_ENTRIES_PER_WQE(MLX5_SEND_WQE_BB * mlx5e_get_max_sq_aligned_wqebbs(mdev))
 
-#define MLX5E_MSG_LEVEL			NETIF_MSG_LINK
-
-#define mlx5e_dbg(mlevel, priv, format, ...)                    \
-do {                                                            \
-	if (NETIF_MSG_##mlevel & (priv)->msglevel)              \
-		netdev_warn(priv->netdev, format,               \
-			    ##__VA_ARGS__);                     \
-} while (0)
-
 #define mlx5e_state_dereference(priv, p) \
 	rcu_dereference_protected((p), lockdep_is_held(&(priv)->state_lock))
 
@@ -880,7 +871,6 @@ struct mlx5e_priv {
 #endif
 	/* priv data path fields - end */
 
-	u32                        msglevel;
 	unsigned long              state;
 	struct mutex               state_lock; /* Protects Interface state */
 	struct mlx5e_rq            drop_rq;
drivers/net/ethernet/mellanox/mlx5/core/en/port_buffer.c +23 -21

99 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c +4 -4
index bed0c2d043e7..933a7772a7a3 100644
@@ -570,10 +570,10 @@ static struct mlx5_flow_handle *arfs_add_rule(struct mlx5e_priv *priv,
 	if (IS_ERR(rule)) {
 		err = PTR_ERR(rule);
 		priv->channel_stats[arfs_rule->rxq]->rq.arfs_err++;
-		mlx5e_dbg(HW, priv,
-			  "%s: add rule(filter id=%d, rq idx=%d, ip proto=0x%x) failed,err=%d\n",
-			  __func__, arfs_rule->filter_id, arfs_rule->rxq,
-			  tuple->ip_proto, err);
+		netdev_dbg(priv->netdev,
+			   "%s: add rule(filter id=%d, rq idx=%d, ip proto=0x%x) failed,err=%d\n",
+			   __func__, arfs_rule->filter_id, arfs_rule->rxq,
+			   tuple->ip_proto, err);
 	}
 
 out:
drivers/net/ethernet/mellanox/mlx5/core/en_dcbnl.c +13 -13
index ebee52a8361a..8705cffc747f 100644
@@ -275,10 +275,10 @@ static int mlx5e_dcbnl_ieee_setets_core(struct mlx5e_priv *priv, struct ieee_ets
 	memcpy(priv->dcbx.tc_tsa, ets->tc_tsa, sizeof(ets->tc_tsa));
 
 	for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++) {
-		mlx5e_dbg(HW, priv, "%s: prio_%d <=> tc_%d\n",
-			  __func__, i, ets->prio_tc[i]);
-		mlx5e_dbg(HW, priv, "%s: tc_%d <=> tx_bw_%d%%, group_%d\n",
-			  __func__, i, tc_tx_bw[i], tc_group[i]);
+		netdev_dbg(priv->netdev, "%s: prio_%d <=> tc_%d\n",
+			   __func__, i, ets->prio_tc[i]);
+		netdev_dbg(priv->netdev, "%s: tc_%d <=> tx_bw_%d%%, group_%d\n",
+			   __func__, i, tc_tx_bw[i], tc_group[i]);
 	}
 
 	return err;
@@ -399,9 +399,9 @@ static int mlx5e_dcbnl_ieee_setpfc(struct net_device *dev,
 	}
 
 	if (!ret) {
-		mlx5e_dbg(HW, priv,
-			  "%s: PFC per priority bit mask: 0x%x\n",
-			  __func__, pfc->pfc_en);
+		netdev_dbg(dev,
+			   "%s: PFC per priority bit mask: 0x%x\n",
+			   __func__, pfc->pfc_en);
 	}
 	return ret;
 }
@@ -611,8 +611,8 @@ static int mlx5e_dcbnl_ieee_setmaxrate(struct net_device *netdev,
 	}
 
 	for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++) {
-		mlx5e_dbg(HW, priv, "%s: tc_%d <=> max_bw %d Gbps\n",
-			  __func__, i, max_bw_value[i]);
+		netdev_dbg(netdev, "%s: tc_%d <=> max_bw %d Gbps\n",
+			   __func__, i, max_bw_value[i]);
 	}
 
 	return mlx5_modify_port_ets_rate_limit(mdev, max_bw_value, max_bw_unit);
@@ -640,10 +640,10 @@ static u8 mlx5e_dcbnl_setall(struct net_device *netdev)
 		ets.tc_rx_bw[i] = cee_cfg->pg_bw_pct[i];
 		ets.tc_tsa[i]   = IEEE_8021QAZ_TSA_ETS;
 		ets.prio_tc[i]  = cee_cfg->prio_to_pg_map[i];
-		mlx5e_dbg(HW, priv,
-			  "%s: Priority group %d: tx_bw %d, rx_bw %d, prio_tc %d\n",
-			  __func__, i, ets.tc_tx_bw[i], ets.tc_rx_bw[i],
-			  ets.prio_tc[i]);
+		netdev_dbg(netdev,
+			   "%s: Priority group %d: tx_bw %d, rx_bw %d, prio_tc %d\n",
+			   __func__, i, ets.tc_tx_bw[i], ets.tc_rx_bw[i],
+			   ets.prio_tc[i]);
 	}
 
 	err = mlx5e_dbcnl_validate_ets(netdev, &ets, true);
drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +3 -15
index 1f5a2110d31f..27861b68ced5 100644
@@ -1689,16 +1689,6 @@ static int mlx5e_set_fecparam(struct net_device *netdev,
 	return 0;
 }
 
-static u32 mlx5e_get_msglevel(struct net_device *dev)
-{
-	return ((struct mlx5e_priv *)netdev_priv(dev))->msglevel;
-}
-
-static void mlx5e_set_msglevel(struct net_device *dev, u32 val)
-{
-	((struct mlx5e_priv *)netdev_priv(dev))->msglevel = val;
-}
-
 static int mlx5e_set_phys_id(struct net_device *dev,
 			     enum ethtool_phys_id_state state)
 {
@@ -1952,9 +1942,9 @@ int mlx5e_modify_rx_cqe_compression_locked(struct mlx5e_priv *priv, bool new_val
 	if (err)
 		return err;
 
-	mlx5e_dbg(DRV, priv, "MLX5E: RxCqeCmprss was turned %s\n",
-		  MLX5E_GET_PFLAG(&priv->channels.params,
-				  MLX5E_PFLAG_RX_CQE_COMPRESS) ? "ON" : "OFF");
+	netdev_dbg(priv->netdev, "MLX5E: RxCqeCmprss was turned %s\n",
+		   MLX5E_GET_PFLAG(&priv->channels.params,
+				   MLX5E_PFLAG_RX_CQE_COMPRESS) ? "ON" : "OFF");
 
 	return 0;
 }
@@ -2444,8 +2434,6 @@ const struct ethtool_ops mlx5e_ethtool_ops = {
 	.get_priv_flags    = mlx5e_get_priv_flags,
 	.set_priv_flags    = mlx5e_set_priv_flags,
 	.self_test         = mlx5e_self_test,
-	.get_msglevel      = mlx5e_get_msglevel,
-	.set_msglevel      = mlx5e_set_msglevel,
 	.get_fec_stats     = mlx5e_get_fec_stats,
 	.get_fecparam      = mlx5e_get_fecparam,
 	.set_fecparam      = mlx5e_set_fecparam,
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +2 -3
index a7c526ee5024..c564ac86ff8a 100644
@@ -2401,7 +2401,7 @@ static int mlx5e_channel_stats_alloc(struct mlx5e_priv *priv, int ix, int cpu)
 	/* Asymmetric dynamic memory allocation.
 	 * Freed in mlx5e_priv_arrays_free, not on channel closure.
 	 */
-	mlx5e_dbg(DRV, priv, "Creating channel stats %d\n", ix);
+	netdev_dbg(priv->netdev, "Creating channel stats %d\n", ix);
 	priv->channel_stats[ix] = kvzalloc_node(sizeof(**priv->channel_stats),
 						GFP_KERNEL, cpu_to_node(cpu));
 	if (!priv->channel_stats[ix])
@@ -2779,7 +2779,7 @@ int mlx5e_update_tx_netdev_queues(struct mlx5e_priv *priv)
 	if (MLX5E_GET_PFLAG(&priv->channels.params, MLX5E_PFLAG_TX_PORT_TS))
 		num_txqs += ntc;
 
-	mlx5e_dbg(DRV, priv, "Setting num_txqs %d\n", num_txqs);
+	netdev_dbg(priv->netdev, "Setting num_txqs %d\n", num_txqs);
 	err = netif_set_real_num_tx_queues(priv->netdev, num_txqs);
 	if (err)
 		netdev_warn(priv->netdev, "netif_set_real_num_tx_queues failed, %d\n", err);
@@ -5585,7 +5585,6 @@ int mlx5e_priv_init(struct mlx5e_priv *priv,
 	/* priv init */
 	priv->mdev        = mdev;
 	priv->netdev      = netdev;
-	priv->msglevel    = MLX5E_MSG_LEVEL;
 	priv->max_nch     = nch;
 	priv->max_opened_tc = 1;
 
Signature · 1 line
2.40.1
#17 lore raw

PATCH 10/15 net/mlx5: Expose bits for local loopback counter

In reply to Saeed Mahameed

From: Or Har-Toov <[email protected]>

Add needed HW bits for querying local loopback counter and the HCA capability for it.

---
 include/linux/mlx5/mlx5_ifc.h | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
include/linux/mlx5/mlx5_ifc.h +4 -2
index d61dcb5d7cd5..354c7e326eab 100644
@@ -1758,7 +1758,7 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 	u8         reserved_at_330[0x6];
 	u8         pci_sync_for_fw_update_with_driver_unload[0x1];
 	u8         vnic_env_cnt_steering_fail[0x1];
-	u8         reserved_at_338[0x1];
+	u8         vport_counter_local_loopback[0x1];
 	u8         q_counter_aggregation[0x1];
 	u8         q_counter_other_vport[0x1];
 	u8         log_max_xrcd[0x5];
@@ -5190,7 +5190,9 @@ struct mlx5_ifc_query_vport_counter_out_bits {
 
 	struct mlx5_ifc_traffic_counter_bits transmitted_eth_multicast;
 
-	u8         reserved_at_680[0xa00];
+	struct mlx5_ifc_traffic_counter_bits local_loopback;
+
+	u8         reserved_at_700[0x980];
 };
 
 enum {
Signature · 1 line
2.40.1
#18 lore raw

PATCH 11/15 net/mlx5e: Add local loopback counter to vport stats

In reply to Saeed Mahameed

From: Or Har-Toov <[email protected]>

Add counter for number of unicast, multicast and broadcast packets/ octets that were loopback.

---
 .../ethernet/mellanox/mlx5/counters.rst       | 10 ++++++++
 .../ethernet/mellanox/mlx5/core/en_stats.c    | 23 ++++++++++++++++++-
 2 files changed, 32 insertions(+), 1 deletion(-)
Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst +10 -0
index 6b2d1fe74ecf..a395df9c2751 100644
@@ -797,6 +797,16 @@ Counters on the NIC port that is connected to a eSwitch.
        RoCE/UD/RC traffic) [#accel]_.
      - Acceleration
 
+   * - `vport_loopback_packets`
+     - Unicast, multicast and broadcast packets that were loop-back (received
+       and transmitted), IB/Eth  [#accel]_.
+     - Acceleration
+
+   * - `vport_loopback_bytes`
+     - Unicast, multicast and broadcast bytes that were loop-back (received
+       and transmitted), IB/Eth  [#accel]_.
+     - Acceleration
+
    * - `rx_steer_missed_packets`
      - Number of packets that was received by the NIC, however was discarded
        because it did not match any flow in the NIC flow table.
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +22 -1
index f1d9596905c6..25a6c596300d 100644
@@ -748,11 +748,22 @@ static const struct counter_desc vport_stats_desc[] = {
 		VPORT_COUNTER_OFF(transmitted_ib_multicast.octets) },
 };
 
+static const struct counter_desc vport_loopback_stats_desc[] = {
+	{ "vport_loopback_packets",
+		VPORT_COUNTER_OFF(local_loopback.packets) },
+	{ "vport_loopback_bytes",
+		VPORT_COUNTER_OFF(local_loopback.octets) },
+};
+
 #define NUM_VPORT_COUNTERS		ARRAY_SIZE(vport_stats_desc)
+#define NUM_VPORT_LOOPBACK_COUNTERS(dev) \
+	(MLX5_CAP_GEN(dev, vport_counter_local_loopback) ? \
+	 ARRAY_SIZE(vport_loopback_stats_desc) : 0)
 
 static MLX5E_DECLARE_STATS_GRP_OP_NUM_STATS(vport)
 {
-	return NUM_VPORT_COUNTERS;
+	return NUM_VPORT_COUNTERS +
+		NUM_VPORT_LOOPBACK_COUNTERS(priv->mdev);
 }
 
 static MLX5E_DECLARE_STATS_GRP_OP_FILL_STRS(vport)
@@ -761,6 +772,11 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STRS(vport)
 
 	for (i = 0; i < NUM_VPORT_COUNTERS; i++)
 		strcpy(data + (idx++) * ETH_GSTRING_LEN, vport_stats_desc[i].format);
+
+	for (i = 0; i < NUM_VPORT_LOOPBACK_COUNTERS(priv->mdev); i++)
+		strcpy(data + (idx++) * ETH_GSTRING_LEN,
+		       vport_loopback_stats_desc[i].format);
+
 	return idx;
 }
 
@@ -771,6 +787,11 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STATS(vport)
 	for (i = 0; i < NUM_VPORT_COUNTERS; i++)
 		data[idx++] = MLX5E_READ_CTR64_BE(priv->stats.vport.query_vport_out,
 						  vport_stats_desc, i);
+
+	for (i = 0; i < NUM_VPORT_LOOPBACK_COUNTERS(priv->mdev); i++)
+		data[idx++] = MLX5E_READ_CTR64_BE(priv->stats.vport.query_vport_out,
+						  vport_loopback_stats_desc, i);
+
 	return idx;
 }
 
Signature · 1 line
2.40.1
#19 lore raw

PATCH 12/15 net/mlx5: Fix the macro for accessing EC VF vports

In reply to Saeed Mahameed

From: Daniel Jurgens <[email protected]>

The last value is not set correctly. This results in representors not being created for all EC VFs when the base value is higher than 0.

Fixes a7719b29a821 ("net/mlx5: Add management of EC VF vports")
---
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.h | 1 +
 1 file changed, 1 insertion(+)
drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +1 -0
index bcbab06759c4..7064609f4998 100644
@@ -668,6 +668,7 @@ void mlx5e_tc_clean_fdb_peer_flows(struct mlx5_eswitch *esw);
 			  index,					\
 			  vport,					\
 			  MLX5_CAP_GEN_2((esw->dev), ec_vf_vport_base),	\
+			  MLX5_CAP_GEN_2((esw->dev), ec_vf_vport_base) +\
 			  (last) - 1)
 
 struct mlx5_eswitch *mlx5_devlink_eswitch_get(struct devlink *devlink);
Signature · 1 line
2.40.1
#20 lore raw

PATCH 13/15 net/mlx5: DR, update query of HCA caps for EC VFs

In reply to Saeed Mahameed

From: Daniel Jurgens <[email protected]>

This change is needed to use EC VFs with metadata based steering.

There was an assumption that vport was equal to function ID. That's not the case for EC VF functions. Adjust to function ID and set the ec_vf_function bit accordingly.

Fixes 9ac0b128248e ("net/mlx5: Update vport caps query/set for EC VFs")
---
 drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h       | 7 +++++++
 drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c | 4 +++-
 drivers/net/ethernet/mellanox/mlx5/core/vport.c           | 6 ------
 3 files changed, 10 insertions(+), 7 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +7 -0
index 60673f98de2b..c4be257c043d 100644
@@ -358,4 +358,11 @@ static inline bool mlx5_core_is_ec_vf_vport(const struct mlx5_core_dev *dev, u16
 
 	return (vport_num >= base_vport && vport_num < max_vport);
 }
+
+static inline int mlx5_vport_to_func_id(const struct mlx5_core_dev *dev, u16 vport, bool ec_vf_func)
+{
+	return ec_vf_func ? vport - mlx5_core_ec_vf_vport_base(dev)
+			  : vport;
+}
+
 #endif /* __MLX5_CORE_H__ */
drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c +3 -1
index 1aa525e509f1..7491911ebcb5 100644
@@ -34,6 +34,7 @@ int mlx5dr_cmd_query_esw_vport_context(struct mlx5_core_dev *mdev,
 int mlx5dr_cmd_query_gvmi(struct mlx5_core_dev *mdev, bool other_vport,
 			  u16 vport_number, u16 *gvmi)
 {
+	bool ec_vf_func = other_vport ? mlx5_core_is_ec_vf_vport(mdev, vport_number) : false;
 	u32 in[MLX5_ST_SZ_DW(query_hca_cap_in)] = {};
 	int out_size;
 	void *out;
@@ -46,7 +47,8 @@ int mlx5dr_cmd_query_gvmi(struct mlx5_core_dev *mdev, bool other_vport,
 
 	MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP);
 	MLX5_SET(query_hca_cap_in, in, other_function, other_vport);
-	MLX5_SET(query_hca_cap_in, in, function_id, vport_number);
+	MLX5_SET(query_hca_cap_in, in, function_id, mlx5_vport_to_func_id(mdev, vport_number, ec_vf_func));
+	MLX5_SET(query_hca_cap_in, in, ec_vf_function, ec_vf_func);
 	MLX5_SET(query_hca_cap_in, in, op_mod,
 		 MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE << 1 |
 		 HCA_CAP_OPMOD_GET_CUR);
drivers/net/ethernet/mellanox/mlx5/core/vport.c +0 -6
index 6d3984dd5b21..5a31fb47ffa5 100644
@@ -1161,12 +1161,6 @@ u64 mlx5_query_nic_system_image_guid(struct mlx5_core_dev *mdev)
 }
 EXPORT_SYMBOL_GPL(mlx5_query_nic_system_image_guid);
 
-static int mlx5_vport_to_func_id(const struct mlx5_core_dev *dev, u16 vport, bool ec_vf_func)
-{
-	return ec_vf_func ? vport - mlx5_core_ec_vf_vport_base(dev)
-			  : vport;
-}
-
 int mlx5_vport_get_other_func_cap(struct mlx5_core_dev *dev, u16 vport, void *out,
 				  u16 opmod)
 {
Signature · 1 line
2.40.1
#21 lore raw

PATCH 14/15 net/mlx5: Add header file for events

In reply to Saeed Mahameed

From: Juhee Kang <[email protected]>

Separate the event API defined in the generic mlx5.h header into a dedicated header. And remove the TODO comment in commit 69c1280b1f3b ("net/mlx5: Device events, Use async events chain").

---
 .../ethernet/mellanox/mlx5/core/en_stats.c    |  2 +-
 .../net/ethernet/mellanox/mlx5/core/events.c  |  2 +-
 .../net/ethernet/mellanox/mlx5/core/health.c  |  1 +
 .../net/ethernet/mellanox/mlx5/core/lag/mp.c  |  2 +-
 .../ethernet/mellanox/mlx5/core/lag/mpesw.c   |  2 +-
 .../ethernet/mellanox/mlx5/core/lib/events.h  | 40 +++++++++++++++++++
 .../ethernet/mellanox/mlx5/core/lib/mlx5.h    | 34 ----------------
 7 files changed, 45 insertions(+), 38 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/lib/events.h
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +1 -1
index 25a6c596300d..4d77055abd4b 100644
@@ -30,7 +30,7 @@
  * SOFTWARE.
  */
 
-#include "lib/mlx5.h"
+#include "lib/events.h"
 #include "en.h"
 #include "en_accel/ktls.h"
 #include "en_accel/en_accel.h"
drivers/net/ethernet/mellanox/mlx5/core/events.c +1 -1
index 718cf09c28ce..3ec892d51f57 100644
@@ -5,7 +5,7 @@
 
 #include "mlx5_core.h"
 #include "lib/eq.h"
-#include "lib/mlx5.h"
+#include "lib/events.h"
 
 struct mlx5_event_nb {
 	struct mlx5_nb  nb;
drivers/net/ethernet/mellanox/mlx5/core/health.c +1 -0
index 210100a4064a..187cb2c464f8 100644
@@ -39,6 +39,7 @@
 #include "mlx5_core.h"
 #include "lib/eq.h"
 #include "lib/mlx5.h"
+#include "lib/events.h"
 #include "lib/pci_vsc.h"
 #include "lib/tout.h"
 #include "diag/fw_tracer.h"
drivers/net/ethernet/mellanox/mlx5/core/lag/mp.c +1 -1
index 976caa8e6922..b1aa494c76ba 100644
@@ -7,7 +7,7 @@
 #include "lag/mp.h"
 #include "mlx5_core.h"
 #include "eswitch.h"
-#include "lib/mlx5.h"
+#include "lib/events.h"
 
 static bool __mlx5_lag_is_multipath(struct mlx5_lag *ldev)
 {
drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +1 -1
index 0e869a76dfe4..4bf15391525c 100644
@@ -6,7 +6,7 @@
 #include "lag/lag.h"
 #include "eswitch.h"
 #include "esw/acl/ofld.h"
-#include "lib/mlx5.h"
+#include "lib/events.h"
 
 static void mlx5_mpesw_metadata_cleanup(struct mlx5_lag *ldev)
 {
drivers/net/ethernet/mellanox/mlx5/core/lib/events.h new file +40 -0
new file mode 100644
index 000000000000..a0f7faea317b
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#ifndef __LIB_EVENTS_H__
+#define __LIB_EVENTS_H__
+
+#include "mlx5_core.h"
+
+#define PORT_MODULE_EVENT_MODULE_STATUS_MASK 0xF
+#define PORT_MODULE_EVENT_ERROR_TYPE_MASK    0xF
+
+enum port_module_event_status_type {
+	MLX5_MODULE_STATUS_PLUGGED   = 0x1,
+	MLX5_MODULE_STATUS_UNPLUGGED = 0x2,
+	MLX5_MODULE_STATUS_ERROR     = 0x3,
+	MLX5_MODULE_STATUS_DISABLED  = 0x4,
+	MLX5_MODULE_STATUS_NUM,
+};
+
+enum  port_module_event_error_type {
+	MLX5_MODULE_EVENT_ERROR_POWER_BUDGET_EXCEEDED    = 0x0,
+	MLX5_MODULE_EVENT_ERROR_LONG_RANGE_FOR_NON_MLNX  = 0x1,
+	MLX5_MODULE_EVENT_ERROR_BUS_STUCK                = 0x2,
+	MLX5_MODULE_EVENT_ERROR_NO_EEPROM_RETRY_TIMEOUT  = 0x3,
+	MLX5_MODULE_EVENT_ERROR_ENFORCE_PART_NUMBER_LIST = 0x4,
+	MLX5_MODULE_EVENT_ERROR_UNKNOWN_IDENTIFIER       = 0x5,
+	MLX5_MODULE_EVENT_ERROR_HIGH_TEMPERATURE         = 0x6,
+	MLX5_MODULE_EVENT_ERROR_BAD_CABLE                = 0x7,
+	MLX5_MODULE_EVENT_ERROR_PCIE_POWER_SLOT_EXCEEDED = 0xc,
+	MLX5_MODULE_EVENT_ERROR_NUM,
+};
+
+struct mlx5_pme_stats {
+	u64 status_counters[MLX5_MODULE_STATUS_NUM];
+	u64 error_counters[MLX5_MODULE_EVENT_ERROR_NUM];
+};
+
+void mlx5_get_pme_stats(struct mlx5_core_dev *dev, struct mlx5_pme_stats *stats);
+int mlx5_notifier_call_chain(struct mlx5_events *events, unsigned int event, void *data);
+#endif
drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h +0 -34
index ccf12f7db6f0..2b5826a785c4 100644
@@ -45,40 +45,6 @@ int mlx5_crdump_enable(struct mlx5_core_dev *dev);
 void mlx5_crdump_disable(struct mlx5_core_dev *dev);
 int mlx5_crdump_collect(struct mlx5_core_dev *dev, u32 *cr_data);
 
-/* TODO move to lib/events.h */
-
-#define PORT_MODULE_EVENT_MODULE_STATUS_MASK 0xF
-#define PORT_MODULE_EVENT_ERROR_TYPE_MASK    0xF
-
-enum port_module_event_status_type {
-	MLX5_MODULE_STATUS_PLUGGED   = 0x1,
-	MLX5_MODULE_STATUS_UNPLUGGED = 0x2,
-	MLX5_MODULE_STATUS_ERROR     = 0x3,
-	MLX5_MODULE_STATUS_DISABLED  = 0x4,
-	MLX5_MODULE_STATUS_NUM,
-};
-
-enum  port_module_event_error_type {
-	MLX5_MODULE_EVENT_ERROR_POWER_BUDGET_EXCEEDED    = 0x0,
-	MLX5_MODULE_EVENT_ERROR_LONG_RANGE_FOR_NON_MLNX  = 0x1,
-	MLX5_MODULE_EVENT_ERROR_BUS_STUCK                = 0x2,
-	MLX5_MODULE_EVENT_ERROR_NO_EEPROM_RETRY_TIMEOUT  = 0x3,
-	MLX5_MODULE_EVENT_ERROR_ENFORCE_PART_NUMBER_LIST = 0x4,
-	MLX5_MODULE_EVENT_ERROR_UNKNOWN_IDENTIFIER       = 0x5,
-	MLX5_MODULE_EVENT_ERROR_HIGH_TEMPERATURE         = 0x6,
-	MLX5_MODULE_EVENT_ERROR_BAD_CABLE                = 0x7,
-	MLX5_MODULE_EVENT_ERROR_PCIE_POWER_SLOT_EXCEEDED = 0xc,
-	MLX5_MODULE_EVENT_ERROR_NUM,
-};
-
-struct mlx5_pme_stats {
-	u64 status_counters[MLX5_MODULE_STATUS_NUM];
-	u64 error_counters[MLX5_MODULE_EVENT_ERROR_NUM];
-};
-
-void mlx5_get_pme_stats(struct mlx5_core_dev *dev, struct mlx5_pme_stats *stats);
-int mlx5_notifier_call_chain(struct mlx5_events *events, unsigned int event, void *data);
-
 static inline struct net *mlx5_core_net(struct mlx5_core_dev *dev)
 {
 	return devlink_net(priv_to_devlink(dev));
Signature · 1 line
2.40.1
#22 lore raw

PATCH 15/15 net/mlx5: Remove unused ecpu field from struct mlx5_sf_table

In reply to Saeed Mahameed

From: Jiri Pirko <[email protected]>

"ecpu" field in struct mlx5_sf_table is not used anywhere. Remove it.

---
 drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c | 1 -
 1 file changed, 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c +0 -1
index 9c02e5ea797c..6a3fa30b2bf2 100644
@@ -28,7 +28,6 @@ struct mlx5_sf_table {
 	struct mutex sf_state_lock; /* Serializes sf state among user cmds & vhca event handler. */
 	struct notifier_block esw_nb;
 	struct notifier_block vhca_nb;
-	u8 ecpu: 1;
 };
 
 static struct mlx5_sf *
Signature · 1 line
2.40.1
maintainer
#9 lore raw

Re: [net-next 07/15] net/mlx5: Bridge, expose FDB state via debugfs

In reply to Saeed Mahameed
Quoted 4 lines · On Fri, 16 Jun 2023 13:11:05 -0700 Saeed Mahameed wrote:
$ cat mlx5/0000\:08\:00.0/esw/bridge/bridge1/fdb
DEV              MAC               VLAN              PACKETS                BYTES              LASTUSE FLAGS
enp8s0f0_1       e4:0a:05:08:00:06    2                    2                  204           4295567112   0x0
enp8s0f0_0       e4:0a:05:08:00:03    2                    3                  278           4295567112   0x0

The flags here are the only thing that's mlx5 specific? Why not add an API for dumping this kind of stats that other drivers can reuse?

The rest of the patches LGTM

Signature · 1 line
pw-bot: cr
#2 lore raw

Re: [net-next 01/15] net/mlx5: Ack on sync_reset_request only if PF can do reset_now

In reply to Saeed Mahameed
This series was applied to netdev/net-next.git (main) by Saeed Mahameed <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (main) by Saeed Mahameed <[email protected]>:

Quoted 8 lines · On Fri, 16 Jun 2023 13:10:59 -0700 you wrote:

From: Moshe Shemesh <[email protected]>

Verify at reset_request stage that PF is capable to do reset_now. In case PF is not capable, notify the firmware that the sync reset can not happen and so firmware will abort the sync reset at early stage and will not send reset_now event to any PF.

[...]

Here is the summary with links:
- [net-next,01/15] net/mlx5: Ack on sync_reset_request only if PF can do reset_now

    https://git.kernel.org/netdev/net-next/c/92501fa6e421
  - [net-next,02/15] net/mlx5: Expose timeout for sync reset unload stage
    https://git.kernel.org/netdev/net-next/c/8bb42ed4210e
  - [net-next,03/15] net/mlx5: Check DTOR entry value is not zero
    https://git.kernel.org/netdev/net-next/c/6f8551f8d9e4
  - [net-next,04/15] net/mlx5: Handle sync reset unload event
    https://git.kernel.org/netdev/net-next/c/7a9770f1bfea
  - [net-next,05/15] net/mlx5: Create eswitch debugfs root directory
    https://git.kernel.org/netdev/net-next/c/f405787a0aba
  - [net-next,06/15] net/mlx5: Bridge, pass net device when linking vport to bridge
    https://git.kernel.org/netdev/net-next/c/ade19f0d6a3a
  - [net-next,07/15] net/mlx5: Bridge, expose FDB state via debugfs
    https://git.kernel.org/netdev/net-next/c/791eb78285e8
  - [net-next,08/15] net/mlx5: E-Switch, remove redundant else statements
    https://git.kernel.org/netdev/net-next/c/8a955da230d3
  - [net-next,09/15] net/mlx5e: Remove mlx5e_dbg() and msglvl support
    https://git.kernel.org/netdev/net-next/c/559f4c32ebff
  - [net-next,10/15] net/mlx5: Expose bits for local loopback counter
    https://git.kernel.org/netdev/net-next/c/0bd2e6fc78fd
  - [net-next,11/15] net/mlx5e: Add local loopback counter to vport stats
    https://git.kernel.org/netdev/net-next/c/c8013a1f714f
  - [net-next,12/15] net/mlx5: Fix the macro for accessing EC VF vports
    https://git.kernel.org/netdev/net-next/c/b3bd68925ebb
  - [net-next,13/15] net/mlx5: DR, update query of HCA caps for EC VFs
    https://git.kernel.org/netdev/net-next/c/8bbe544e0380
  - [net-next,14/15] net/mlx5: Add header file for events
    https://git.kernel.org/netdev/net-next/c/2bd3b292955f
  - [net-next,15/15] net/mlx5: Remove unused ecpu field from struct mlx5_sf_table
    https://git.kernel.org/netdev/net-next/c/5f2cf757f9c5

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
#10 lore raw

Re: [net-next 07/15] net/mlx5: Bridge, expose FDB state via debugfs

In reply to Jakub Kicinski
Quoted 7 lines · Jakub Kicinski
Quoted 4 lines · On Fri, 16 Jun 2023 13:11:05 -0700 Saeed Mahameed wrote:
$ cat mlx5/0000\:08\:00.0/esw/bridge/bridge1/fdb
DEV              MAC               VLAN              PACKETS                BYTES              LASTUSE FLAGS
enp8s0f0_1       e4:0a:05:08:00:06    2                    2                  204           4295567112   0x0
enp8s0f0_0       e4:0a:05:08:00:03    2                    3                  278           4295567112   0x0

The flags here are the only thing that's mlx5 specific?

Not exactly. This debugfs exposes the state of our bridge offload layer. For example, when VF representors from different eswitches are added to the same bridge every FDB entry on such bridge will have multiple underlying offloaded steering rules (one per eswitch instance connected to the bridge). User will observe the entries in all connected 'fdb' debugfs' (all except the 'main' entry will have flag MLX5_ESW_BRIDGE_FLAG_PEER set) and their corresponding counters will increment only on the eswitch instance that is actually processing the packets, which depends on the mode (when bonding device is added to the bridge in single FDB LAG mode all traffic appears on eswitch 0, without it the the traffic is on the eswitch of parent uplink of the VF). I understand that this is rather convoluted but this is exactly why we are going with debugfs.

Quoted 2 lines

Why not add an API for dumping this kind of stats that other drivers can reuse?

As explained in previous paragraph we would like to expose internal mlx5 bridge layer for debug purposes, not to design generic bridge FDB counter interface. Also, the debugging needs of our implementation may not correspond to other drivers because we don't have a 'hardware switch' on our NIC, so we do things like learning and ageing in software, and have to deal with multiple possible mode of operations (single FDB vs merged eswitch from previous example, etc.).

Quoted 2 lines

The rest of the patches LGTM

maintainer
#11 lore raw

Re: [net-next 07/15] net/mlx5: Bridge, expose FDB state via debugfs

Quoted 26 lines · On Mon, 19 Jun 2023 11:37:30 +0300 Vlad Buslov wrote:
Quoted 7 lines · Jakub Kicinski
Quoted 4 lines · On Fri, 16 Jun 2023 13:11:05 -0700 Saeed Mahameed wrote:
$ cat mlx5/0000\:08\:00.0/esw/bridge/bridge1/fdb
DEV              MAC               VLAN              PACKETS                BYTES              LASTUSE FLAGS
enp8s0f0_1       e4:0a:05:08:00:06    2                    2                  204           4295567112   0x0
enp8s0f0_0       e4:0a:05:08:00:03    2                    3                  278           4295567112   0x0  

The flags here are the only thing that's mlx5 specific?

Not exactly. This debugfs exposes the state of our bridge offload layer. For example, when VF representors from different eswitches are added to the same bridge every FDB entry on such bridge will have multiple underlying offloaded steering rules (one per eswitch instance connected to the bridge). User will observe the entries in all connected 'fdb' debugfs' (all except the 'main' entry will have flag MLX5_ESW_BRIDGE_FLAG_PEER set) and their corresponding counters will increment only on the eswitch instance that is actually processing the packets, which depends on the mode (when bonding device is added to the bridge in single FDB LAG mode all traffic appears on eswitch 0, without it the the traffic is on the eswitch of parent uplink of the VF). I understand that this is rather convoluted but this is exactly why we are going with debugfs.

Quoted 2 lines

Why not add an API for dumping this kind of stats that other drivers can reuse?

Quoted 7 lines

As explained in previous paragraph we would like to expose internal mlx5 bridge layer for debug purposes, not to design generic bridge FDB counter interface. Also, the debugging needs of our implementation may not correspond to other drivers because we don't have a 'hardware switch' on our NIC, so we do things like learning and ageing in software, and have to deal with multiple possible mode of operations (single FDB vs merged eswitch from previous example, etc.).

Looks like my pw-bot shenanigans backfired / crashed, patches didn't get marked as Changes Requested and Dave applied the series :S

I understand the motivation but the information is easy enough to understand to potentially tempt a user to start depending on it for production needs. Then another vendor may get asked to implement similar but not exactly the same set of stats etc. etc.

Do you have customer who will need this?

At the very least please follow up to make the files readable to only root. Normal users should never look at debugfs IMO.

#12 lore raw

Re: [net-next 07/15] net/mlx5: Bridge, expose FDB state via debugfs

Quoted 35 lines · Jakub Kicinski
Quoted 26 lines · On Mon, 19 Jun 2023 11:37:30 +0300 Vlad Buslov wrote:
Quoted 7 lines · Jakub Kicinski
Quoted 4 lines · On Fri, 16 Jun 2023 13:11:05 -0700 Saeed Mahameed wrote:
$ cat mlx5/0000\:08\:00.0/esw/bridge/bridge1/fdb
DEV              MAC               VLAN              PACKETS                BYTES              LASTUSE FLAGS
enp8s0f0_1       e4:0a:05:08:00:06    2                    2                  204           4295567112   0x0
enp8s0f0_0       e4:0a:05:08:00:03    2                    3                  278           4295567112   0x0  

The flags here are the only thing that's mlx5 specific?

Not exactly. This debugfs exposes the state of our bridge offload layer. For example, when VF representors from different eswitches are added to the same bridge every FDB entry on such bridge will have multiple underlying offloaded steering rules (one per eswitch instance connected to the bridge). User will observe the entries in all connected 'fdb' debugfs' (all except the 'main' entry will have flag MLX5_ESW_BRIDGE_FLAG_PEER set) and their corresponding counters will increment only on the eswitch instance that is actually processing the packets, which depends on the mode (when bonding device is added to the bridge in single FDB LAG mode all traffic appears on eswitch 0, without it the the traffic is on the eswitch of parent uplink of the VF). I understand that this is rather convoluted but this is exactly why we are going with debugfs.

Quoted 2 lines

Why not add an API for dumping this kind of stats that other drivers can reuse?

Quoted 7 lines

As explained in previous paragraph we would like to expose internal mlx5 bridge layer for debug purposes, not to design generic bridge FDB counter interface. Also, the debugging needs of our implementation may not correspond to other drivers because we don't have a 'hardware switch' on our NIC, so we do things like learning and ageing in software, and have to deal with multiple possible mode of operations (single FDB vs merged eswitch from previous example, etc.).

Quoted 7 lines

Looks like my pw-bot shenanigans backfired / crashed, patches didn't get marked as Changes Requested and Dave applied the series :S

I understand the motivation but the information is easy enough to understand to potentially tempt a user to start depending on it for production needs. Then another vendor may get asked to implement similar but not exactly the same set of stats etc. etc.

That could happen (although consider that bridge offload functionality significantly predates mlx5 implementation and apparently no one really needed that until now), but such API would supplement, not replace the debugfs since we would like to have per-eswitch FDB state exposed together with our internal flags and everything as explained in my previous email.

Quoted 2 lines

Do you have customer who will need this?

Yes. But strictly for debugging (by human), not for building some proprietary weird user-space switch-controller application that would query this in normal mode of operation, if I understand your concern correctly.

Quoted 3 lines

At the very least please follow up to make the files readable to only root. Normal users should never look at debugfs IMO.

Hmm, all other debugfs' in mlx5 that I tend to use for switching-related functionality debugging seems to be 0444 (lag, steering, tc hairpin). Why would this one be any different?

maintainer
#13 lore raw

Re: [net-next 07/15] net/mlx5: Bridge, expose FDB state via debugfs

Quoted 8 lines · On Mon, 19 Jun 2023 21:34:02 +0300 Vlad Buslov wrote:
Quoted 7 lines

Looks like my pw-bot shenanigans backfired / crashed, patches didn't get marked as Changes Requested and Dave applied the series :S

I understand the motivation but the information is easy enough to understand to potentially tempt a user to start depending on it for production needs. Then another vendor may get asked to implement similar but not exactly the same set of stats etc. etc.

Quoted 6 lines

That could happen (although consider that bridge offload functionality significantly predates mlx5 implementation and apparently no one really needed that until now), but such API would supplement, not replace the debugfs since we would like to have per-eswitch FDB state exposed together with our internal flags and everything as explained in my previous email.

Because crossing between eswitches incurs additional cost?

Quoted 7 lines
Quoted 1 line

Do you have customer who will need this?

Yes. But strictly for debugging (by human), not for building some proprietary weird user-space switch-controller application that would query this in normal mode of operation, if I understand your concern correctly.

Quoted 6 lines
Quoted 2 lines

At the very least please follow up to make the files readable to only root. Normal users should never look at debugfs IMO.

Hmm, all other debugfs' in mlx5 that I tend to use for switching-related functionality debugging seems to be 0444 (lag, steering, tc hairpin). Why would this one be any different?

Querying the stats seems generally useful, so I'd like to narrow down the access as much as possible. This way if the usage spreads we'll hear complaints and can go back to creating a more appropriate API.

#14 lore raw

Re: [net-next 07/15] net/mlx5: Bridge, expose FDB state via debugfs

Quoted 9 lines · Jakub Kicinski
Quoted 8 lines · On Mon, 19 Jun 2023 21:34:02 +0300 Vlad Buslov wrote:
Quoted 7 lines

Looks like my pw-bot shenanigans backfired / crashed, patches didn't get marked as Changes Requested and Dave applied the series :S

I understand the motivation but the information is easy enough to understand to potentially tempt a user to start depending on it for production needs. Then another vendor may get asked to implement similar but not exactly the same set of stats etc. etc.

Quoted 8 lines
Quoted 6 lines

That could happen (although consider that bridge offload functionality significantly predates mlx5 implementation and apparently no one really needed that until now), but such API would supplement, not replace the debugfs since we would like to have per-eswitch FDB state exposed together with our internal flags and everything as explained in my previous email.

Because crossing between eswitches incurs additional cost?

It is not about performance. I install multiple steering rules (one per eswitch), I would like to understand which one is processing the packets when something goes wrong (main or peer). User/field engineer complains that some FDB is (not) aged out according to the expectations, I would like them to dump the file several times while running traffic to see how the lastused and counters changed during that. Just the basic debugging stuff because, again, ConnectX doesn't implement 802.1D in hardware so all the FDB management is done purely in software and we need a way to expose the state.

Quoted 11 lines
Quoted 11 lines
Quoted 2 lines

Do you have customer who will need this?

Yes. But strictly for debugging (by human), not for building some proprietary weird user-space switch-controller application that would query this in normal mode of operation, if I understand your concern correctly.

Quoted 2 lines

At the very least please follow up to make the files readable to only root. Normal users should never look at debugfs IMO.

Quoted 7 lines
Quoted 3 lines

Hmm, all other debugfs' in mlx5 that I tend to use for switching-related functionality debugging seems to be 0444 (lag, steering, tc hairpin). Why would this one be any different?

Querying the stats seems generally useful, so I'd like to narrow down the access as much as possible. This way if the usage spreads we'll hear complaints and can go back to creating a more appropriate API.

Ack.