Skip to contentSkip to conversation

devlink: Add boot-time eswitch mode defaults

PATCH Superseded Mark Bloch 2 participants 14 messages Jun 3 – Jun 5

Subsystems (9)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Mark Bloch

author maintainer
#0 lore raw

v2 cover devlink: Add boot-time eswitch mode defaults

This series adds a devlink_eswitch_mode= kernel command line parameter for applying a default devlink eswitch mode during device initialization.

Following the discussion with Jakub[1] and the feedback on the RFC postings, this version keeps the scope limited to a boot-time devlink eswitch mode default only.

The option selects either all devlink handles or an explicit comma separated handle list:

devlink_eswitch_mode=[*]:switchdev devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:switchdev_inactive

The supported modes are legacy, switchdev and switchdev_inactive. The selected mode is applied through the existing eswitch_mode_set() devlink operation, the same operation used by the devlink eswitch mode command.

The preparatory patches move registration points that expose the devlink instance before the driver is ready for a registration-time eswitch mode change. Where registration is moved later, the matching unregister path is moved earlier so unregister notifications are sent from devl_unregister() before object teardown. The final patch adds the parser and applies the default from devlink core when a matching instance is registered and after a successful devlink reload that performed DRIVER_REINIT.

Patch 1 skips devlink health recovery notifications while a devlink instance is not registered. Health state and counters are still updated, but there is no registered instance for userspace to observe or receive notifications from yet. This lets drivers move registration later without hitting health notification registration assertions during early initialization.

Patch 2 moves netdevsim devlink registration after device initialization, so registration-time defaults can call eswitch_mode_set() after simulator state is ready. It also unregisters devlink before netdevsim tears down the objects that were registered before devlink became visible.

Patch 3 clears the mlx5 FW reset-in-progress bit before reloading after a firmware reset.

Patch 4 moves mlx5 devlink registration after device initialization, including the lightweight init path, and moves unregister before the matching teardown.

Patch 5 moves octeontx2 AF devlink registration after SR-IOV setup and switch lock initialization.

Patch 6 moves octeontx2 PF devlink registration after PF SR-IOV state setup.

Patch 7 adds the devlink_eswitch_mode= parser, documentation, registration-time default application and successful reload default application.

Changelog:

v1 -> v2:

- Move default eswitch mode application into devlink core. The default is now applied during devlink registration and after a successful devlink reload that performed DRIVER_REINIT.

- Remove the exported devl_apply_default_esw_mode() driver API and the mlx5 driver-side call to it.

- Skip devlink health recovery notifications while the devlink instance is not registered, so drivers can move registration later without early health work hitting registration assertions.

- Move mlx5 devlink registration after device initialization, including the lightweight init path, so the core can apply the default through the normal registration flow.

- Move the matching netdevsim and mlx5 unregister paths before object teardown, so unregister notifications come from devl_unregister() and the later object teardown paths run while the devlink instance is no longer registered.

- Add registration-ordering preparation patches for netdevsim and octeontx2 AF/PF, so their eswitch state is ready before registration-time defaults may call eswitch_mode_set().

[1] https://lore.kernel.org/all/[email protected]/
RFC V1 : https://lore.kernel.org/all/[email protected]/
RFC V2 : https://lore.kernel.org/all/[email protected]/
v1 : https://lore.kernel.org/all/[email protected]/

Mark Bloch (7):
devlink: Skip health recover notifications before register
netdevsim: Register devlink after device init net/mlx5: Clear FW reset-in-progress bit before reload net/mlx5: Register devlink after device init
octeontx2-af: Register devlink after SR-IOV init
octeontx2-pf: Register devlink after SR-IOV state init
devlink: Add eswitch mode boot defaults

 .../admin-guide/kernel-parameters.txt         |  25 ++
 .../networking/devlink/devlink-defaults.rst   |  80 ++++++
 Documentation/networking/devlink/index.rst    |   1 +
 .../net/ethernet/marvell/octeontx2/af/rvu.c   |  24 +-
 .../ethernet/marvell/octeontx2/nic/otx2_pf.c  |  17 +-
 .../ethernet/mellanox/mlx5/core/fw_reset.c    |  28 +-
 .../net/ethernet/mellanox/mlx5/core/main.c    |  34 ++-
 drivers/net/netdevsim/dev.c                   |  15 +-
 net/devlink/core.c                            | 261 ++++++++++++++++++
 net/devlink/dev.c                             |   3 +
 net/devlink/devl_internal.h                   |   1 +
 net/devlink/health.c                          |   3 +-
 12 files changed, 443 insertions(+), 49 deletions(-)
 create mode 100644 Documentation/networking/devlink/devlink-defaults.rst

base-commit: dfcc2ff12925d99e858eaf539eaa4aaaf81fe2a6

Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH v2 1/7 devlink: Skip health recover notifications before register

devlink health reports can be generated before the devlink instance is registered. This can happen during driver initialization when a driver creates health reporters early and its health polling detects an error before devlink_register() is reached.

devlink health still records the report state and counters in that case, but userspace cannot observe the devlink instance yet and there is no registered handle to notify through. Skip the netlink notification while the devlink instance is not registered instead of asserting registration.

This keeps later userspace queries useful after registration while avoiding a warning from early health reports.

---
 net/devlink/health.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
net/devlink/health.c +1 -2
index ea7a334e939b..376e79497771 100644
@@ -513,9 +513,8 @@ static void devlink_recover_notify(struct devlink_health_reporter *reporter,
 	int err;
 
 	WARN_ON(cmd != DEVLINK_CMD_HEALTH_REPORTER_RECOVER);
-	ASSERT_DEVLINK_REGISTERED(devlink);
 
-	if (!devlink_nl_notify_need(devlink))
+	if (!__devl_is_registered(devlink) || !devlink_nl_notify_need(devlink))
 		return;
 
 	msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
Signature · 1 line
2.34.1
author maintainer
#2 lore raw

PATCH v2 2/7 netdevsim: Register devlink after device init

In reply to Mark Bloch

devl_register() makes the devlink instance visible to userspace. A later patch also makes registration the point where devlink core may call eswitch_mode_set() to apply a boot-time default eswitch mode.

Move netdevsim registration after all objects (resources, params, regions, traps, debugfs etc) are initialized, and after the initial eswitch mode is set to legacy.

Move devl_unregister() to the beginning of nsim_drv_remove(), before those devlink objects are torn down. This keeps devlink register/unregister as the notification barrier and makes the later object teardown paths run after devlink is no longer registered, so they do not emit their own netlink DEL notifications.

---
 drivers/net/netdevsim/dev.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
drivers/net/netdevsim/dev.c +7 -8
index aed9ad5f1b43..7cf4102b049e 100644
@@ -1680,13 +1680,9 @@ int nsim_drv_probe(struct nsim_bus_dev *nsim_bus_dev)
 		goto err_devlink_unlock;
 	}
 
-	err = devl_register(devlink);
-	if (err)
-		goto err_vfc_free;
-
 	err = nsim_dev_resources_register(devlink);
 	if (err)
-		goto err_dl_unregister;
+		goto err_vfc_free;
 
 	err = devl_params_register(devlink, nsim_devlink_params,
 				   ARRAY_SIZE(nsim_devlink_params));
@@ -1733,9 +1729,14 @@ int nsim_drv_probe(struct nsim_bus_dev *nsim_bus_dev)
 		goto err_hwstats_exit;
 
 	nsim_dev->esw_mode = DEVLINK_ESWITCH_MODE_LEGACY;
+	err = devl_register(devlink);
+	if (err)
+		goto err_port_del_all;
 	devl_unlock(devlink);
 	return 0;
 
+err_port_del_all:
+	nsim_dev_port_del_all(nsim_dev);
 err_hwstats_exit:
 	nsim_dev_hwstats_exit(nsim_dev);
 err_psample_exit:
@@ -1757,8 +1758,6 @@ int nsim_drv_probe(struct nsim_bus_dev *nsim_bus_dev)
 			       ARRAY_SIZE(nsim_devlink_params));
 err_resource_unregister:
 	devl_resources_unregister(devlink);
-err_dl_unregister:
-	devl_unregister(devlink);
 err_vfc_free:
 	kfree(nsim_dev->vfconfigs);
 err_devlink_unlock:
@@ -1797,6 +1796,7 @@ void nsim_drv_remove(struct nsim_bus_dev *nsim_bus_dev)
 	struct devlink *devlink = priv_to_devlink(nsim_dev);
 
 	devl_lock(devlink);
+	devl_unregister(devlink);
 	nsim_dev_reload_destroy(nsim_dev);
 
 	nsim_bpf_dev_exit(nsim_dev);
@@ -1804,7 +1804,6 @@ void nsim_drv_remove(struct nsim_bus_dev *nsim_bus_dev)
 	devl_params_unregister(devlink, nsim_devlink_params,
 			       ARRAY_SIZE(nsim_devlink_params));
 	devl_resources_unregister(devlink);
-	devl_unregister(devlink);
 	kfree(nsim_dev->vfconfigs);
 	kfree(nsim_dev->fa_cookie);
 	mutex_destroy(&nsim_dev->progs_list_lock);
Signature · 1 line
2.34.1
author maintainer
#3 lore raw

PATCH v2 3/7 net/mlx5: Clear FW reset-in-progress bit before reload

In reply to Mark Bloch

mlx5 sets MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS when acknowledging a sync reset request. This bit blocks devlink reload and other devlink operations while the firmware reset is running, but it was kept set until after the driver reload finished.

Clear the reset-in-progress bit once the reset unload flow is done and PCI access is back, before reloading the device. For a reset initiated through devlink, clear it before completing the reload waiter. For a reset reported through an asynchronous firmware event, keep the unload flow outside devl_lock, then take devl_lock before clearing the bit and reloading through the devl-locked load helper.

---
 .../ethernet/mellanox/mlx5/core/fw_reset.c    | 28 +++++++++++--------
 1 file changed, 17 insertions(+), 11 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +17 -11
index 07440c58713a..7283e5b49eed 100644
@@ -238,24 +238,30 @@ static void mlx5_fw_reset_complete_reload(struct mlx5_core_dev *dev)
 {
 	struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset;
 	struct devlink *devlink = priv_to_devlink(dev);
+	int err;
 
 	/* if this is the driver that initiated the fw reset, devlink completed the reload */
 	if (test_bit(MLX5_FW_RESET_FLAGS_PENDING_COMP, &fw_reset->reset_flags)) {
+		clear_bit(MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS,
+			  &fw_reset->reset_flags);
 		complete(&fw_reset->done);
-	} else {
-		mlx5_sync_reset_unload_flow(dev, false);
-		if (mlx5_health_wait_pci_up(dev))
-			mlx5_core_err(dev, "reset reload flow aborted, PCI reads still not working\n");
-		else
-			mlx5_load_one(dev, true);
-		devl_lock(devlink);
-		devlink_remote_reload_actions_performed(devlink, 0,
-							BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT) |
-							BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE));
-		devl_unlock(devlink);
+		return;
 	}
 
+	mlx5_sync_reset_unload_flow(dev, false);
+	err = mlx5_health_wait_pci_up(dev);
+
+	devl_lock(devlink);
 	clear_bit(MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS, &fw_reset->reset_flags);
+	if (err)
+		mlx5_core_err(dev, "reset reload flow aborted, PCI reads still not working\n");
+	else
+		mlx5_load_one_devl_locked(dev, true);
+
+	devlink_remote_reload_actions_performed(devlink, 0,
+						BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT) |
+						BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE));
+	devl_unlock(devlink);
 }
 
 static void mlx5_stop_sync_reset_poll(struct mlx5_core_dev *dev)
Signature · 1 line
2.34.1
author maintainer
#4 lore raw

PATCH v2 4/7 net/mlx5: Register devlink after device init

In reply to Mark Bloch

devl_register() makes the devlink instance visible to userspace. A later patch also makes registration the point where devlink core may call eswitch_mode_set() to apply a boot-time default eswitch mode.

Move mlx5 devlink registration after mlx5 device initialization completes, including the lightweight init path, so registration-time devlink operations see initialized driver state.

Move devl_unregister() before the matching teardown paths, so unregister notifications are emitted from devl_unregister() before mlx5 removes the devlink objects.

Add a devl-locked uninit helper so failed nested devlink setup can unwind the initialized device before the instance is registered.

---
 .../net/ethernet/mellanox/mlx5/core/main.c    | 34 ++++++++++++++-----
 1 file changed, 25 insertions(+), 9 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/main.c +25 -9
index 0c6e4efe38c8..ab3d3ff10f1a 100644
@@ -1455,31 +1455,40 @@ int mlx5_init_one_devl_locked(struct mlx5_core_dev *dev)
 	return err;
 }
 
+static void mlx5_uninit_one_devl_locked(struct mlx5_core_dev *dev);
+
 int mlx5_init_one(struct mlx5_core_dev *dev)
 {
 	struct devlink *devlink = priv_to_devlink(dev);
 	int err;
 
 	devl_lock(devlink);
+	err = mlx5_init_one_devl_locked(dev);
+	if (err)
+		goto unlock;
+
 	if (dev->shd) {
 		err = devl_nested_devlink_set(dev->shd, devlink);
 		if (err)
-			goto unlock;
+			goto err_uninit;
 	}
+
 	devl_register(devlink);
-	err = mlx5_init_one_devl_locked(dev);
-	if (err)
-		devl_unregister(devlink);
+	devl_unlock(devlink);
+	return 0;
+
+err_uninit:
+	mlx5_uninit_one_devl_locked(dev);
 unlock:
 	devl_unlock(devlink);
 	return err;
 }
 
-void mlx5_uninit_one(struct mlx5_core_dev *dev)
+static void mlx5_uninit_one_devl_locked(struct mlx5_core_dev *dev)
 {
 	struct devlink *devlink = priv_to_devlink(dev);
 
-	devl_lock(devlink);
+	devl_assert_locked(devlink);
 	mutex_lock(&dev->intf_state_mutex);
 
 	mlx5_hwmon_dev_unregister(dev);
@@ -1501,7 +1510,15 @@ void mlx5_uninit_one(struct mlx5_core_dev *dev)
 	mlx5_function_teardown(dev, true);
 out:
 	mutex_unlock(&dev->intf_state_mutex);
+}
+
+void mlx5_uninit_one(struct mlx5_core_dev *dev)
+{
+	struct devlink *devlink = priv_to_devlink(dev);
+
+	devl_lock(devlink);
 	devl_unregister(devlink);
+	mlx5_uninit_one_devl_locked(dev);
 	devl_unlock(devlink);
 }
 
@@ -1635,7 +1652,6 @@ int mlx5_init_one_light(struct mlx5_core_dev *dev)
 	int err;
 
 	devl_lock(devlink);
-	devl_register(devlink);
 	dev->state = MLX5_DEVICE_STATE_UP;
 	err = mlx5_function_enable(dev, true, mlx5_tout_ms(dev, FW_PRE_INIT_TIMEOUT));
 	if (err) {
@@ -1655,6 +1671,7 @@ int mlx5_init_one_light(struct mlx5_core_dev *dev)
 		goto query_hca_caps_err;
 	}
 
+	devl_register(devlink);
 	devl_unlock(devlink);
 	return 0;
 
@@ -1662,7 +1679,6 @@ int mlx5_init_one_light(struct mlx5_core_dev *dev)
 	mlx5_function_disable(dev, true);
 out:
 	dev->state = MLX5_DEVICE_STATE_INTERNAL_ERROR;
-	devl_unregister(devlink);
 	devl_unlock(devlink);
 	return err;
 }
@@ -1672,8 +1688,8 @@ void mlx5_uninit_one_light(struct mlx5_core_dev *dev)
 	struct devlink *devlink = priv_to_devlink(dev);
 
 	devl_lock(devlink);
-	mlx5_devlink_params_unregister(priv_to_devlink(dev));
 	devl_unregister(devlink);
+	mlx5_devlink_params_unregister(priv_to_devlink(dev));
 	devl_unlock(devlink);
 	if (dev->state != MLX5_DEVICE_STATE_UP)
 		return;
Signature · 1 line
2.34.1
author maintainer
#5 lore raw

PATCH v2 5/7 octeontx2-af: Register devlink after SR-IOV init

In reply to Mark Bloch

A later patch makes devlink registration the point where devlink core may call eswitch_mode_set() to apply a boot-time default eswitch mode.

Move octeontx2 AF devlink registration after SR-IOV is enabled and the representor switch lock is initialized, so the AF eswitch mode set path sees the state it depends on.

If devlink registration fails after SR-IOV setup, unregister interrupts before disabling SR-IOV. This keeps the AF-VF mailbox IRQ handlers synchronized before the AF-VF mailbox workqueue is destroyed.

---
 .../net/ethernet/marvell/octeontx2/af/rvu.c   | 24 ++++++++++---------
 1 file changed, 13 insertions(+), 11 deletions(-)
drivers/net/ethernet/marvell/octeontx2/af/rvu.c +13 -11
index 3cf131508ecf..c2b52eb4ffab 100644
@@ -3545,6 +3545,7 @@ static void rvu_update_module_params(struct rvu *rvu)
 static int rvu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
 	struct device *dev = &pdev->dev;
+	bool sriov_done = false;
 	struct rvu *rvu;
 	int    err;
 
@@ -3634,26 +3635,27 @@ static int rvu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 		goto err_flr;
 	}
 
-	err = rvu_register_dl(rvu);
-	if (err) {
-		dev_err(dev, "%s: Failed to register devlink\n", __func__);
-		goto err_irq;
-	}
-
 	rvu_setup_rvum_blk_revid(rvu);
 
 	/* Enable AF's VFs (if any) */
 	err = rvu_enable_sriov(rvu);
 	if (err) {
 		dev_err(dev, "%s: Failed to enable sriov\n", __func__);
-		goto err_dl;
+		goto err_irq;
+	}
+	sriov_done = true;
+
+	mutex_init(&rvu->rswitch.switch_lock);
+
+	err = rvu_register_dl(rvu);
+	if (err) {
+		dev_err(dev, "%s: Failed to register devlink\n", __func__);
+		goto err_irq;
 	}
 
 	/* Initialize debugfs */
 	rvu_dbg_init(rvu);
 
-	mutex_init(&rvu->rswitch.switch_lock);
-
 	if (rvu->fwdata)
 		ptp_start(rvu, rvu->fwdata->sclk, rvu->fwdata->ptp_ext_clk_rate,
 			  rvu->fwdata->ptp_ext_tstamp);
@@ -3662,10 +3664,10 @@ static int rvu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	rvu_alloc_cint_qint_mem(rvu, &rvu->pf[RVU_AFPF], BLKADDR_NIX0,
 				(rvu->hw->block[BLKADDR_NIX0].lf.max));
 	return 0;
-err_dl:
-	rvu_unregister_dl(rvu);
 err_irq:
 	rvu_unregister_interrupts(rvu);
+	if (sriov_done)
+		rvu_disable_sriov(rvu);
 err_flr:
 	rvu_flr_wq_destroy(rvu);
 err_mbox:
Signature · 1 line
2.34.1
author maintainer
#6 lore raw

PATCH v2 6/7 octeontx2-pf: Register devlink after SR-IOV state init

In reply to Mark Bloch

A later patch makes devlink registration the point where devlink core may call eswitch_mode_set() to apply a boot-time default eswitch mode.

Move octeontx2 PF devlink registration after PF SR-IOV configuration state is initialized, so representor creation has the state it needs.

Add a separate unwind label so failures after devlink registration unregister devlink before cleaning up SR-IOV state.

---
 .../ethernet/marvell/octeontx2/nic/otx2_pf.c    | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +9 -8
index ee623476e5ff..9afe6cf0ea01 100644
@@ -3278,14 +3278,14 @@ static int otx2_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	if (err)
 		goto err_mcam_flow_del;
 
-	err = otx2_register_dl(pf);
-	if (err)
-		goto err_mcam_flow_del;
-
 	/* Initialize SR-IOV resources */
 	err = otx2_sriov_vfcfg_init(pf);
 	if (err)
-		goto err_pf_sriov_init;
+		goto err_shutdown_tc;
+
+	err = otx2_register_dl(pf);
+	if (err)
+		goto err_sriov_cleannup;
 
 	/* Enable link notifications */
 	otx2_cgx_config_linkevents(pf, true);
@@ -3293,7 +3293,7 @@ static int otx2_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	pf->af_xdp_zc_qidx = bitmap_zalloc(qcount, GFP_KERNEL);
 	if (!pf->af_xdp_zc_qidx) {
 		err = -ENOMEM;
-		goto err_sriov_cleannup;
+		goto err_dl_unregister;
 	}
 
 #ifdef CONFIG_DCB
@@ -3310,10 +3310,11 @@ static int otx2_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 err_free_zc_bmap:
 	bitmap_free(pf->af_xdp_zc_qidx);
 #endif
+err_dl_unregister:
+	otx2_unregister_dl(pf);
 err_sriov_cleannup:
 	otx2_sriov_vfcfg_cleanup(pf);
-err_pf_sriov_init:
-	otx2_unregister_dl(pf);
+err_shutdown_tc:
 	otx2_shutdown_tc(pf);
 err_mcam_flow_del:
 	otx2_mcam_flow_del(pf);
Signature · 1 line
2.34.1
author maintainer
#7 lore raw

PATCH v2 7/7 devlink: Add eswitch mode boot defaults

In reply to Mark Bloch

Add devlink_eswitch_mode= command line support for setting a default eswitch mode during device initialization.

The supported syntax selects either all devlink handles or one explicit comma-separated handle list:

devlink_eswitch_mode=[*]:<mode> devlink_eswitch_mode=[<handle>[,<handle>...]]:<mode>

where <mode> is one of legacy, switchdev or switchdev_inactive. All selected handles receive the same mode. Assigning different modes to different handle lists in the same parameter value is not supported.

The default is applied through the existing eswitch_mode_set() devlink operation, matching the userspace devlink eswitch mode command. devlink core applies it when a matching devlink instance is registered and after a successful devlink reload that performed DRIVER_REINIT, so rebuilt device state returns to the requested boot default.

Document the devlink_eswitch_mode= syntax and duplicate handle handling.

---
 .../admin-guide/kernel-parameters.txt         |  25 ++
 .../networking/devlink/devlink-defaults.rst   |  80 ++++++
 Documentation/networking/devlink/index.rst    |   1 +
 net/devlink/core.c                            | 261 ++++++++++++++++++
 net/devlink/dev.c                             |   3 +
 net/devlink/devl_internal.h                   |   1 +
 6 files changed, 371 insertions(+)
 create mode 100644 Documentation/networking/devlink/devlink-defaults.rst
Documentation/admin-guide/kernel-parameters.txt +25 -0
index 063c11ca33e5..7af9f2898d92 100644
@@ -1264,6 +1264,31 @@ Kernel parameters
 	dell_smm_hwmon.fan_max=
 			[HW] Maximum configurable fan speed.
 
+	devlink_eswitch_mode=
+			[NET]
+			Format:
+			[<selector>]:<mode>
+
+			<selector>:
+			* | <handle>[,<handle>...]
+
+			<handle>:
+			<bus-name>/<dev-name>
+
+			Configure default devlink eswitch mode for matching
+			devlink instances during device initialization.
+
+			<mode>:
+			legacy | switchdev | switchdev_inactive
+
+			Examples:
+			devlink_eswitch_mode=[*]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:legacy
+
+			See Documentation/networking/devlink/devlink-defaults.rst
+			for the full syntax.
+
 	dfltcc=		[HW,S390]
 			Format: { on | off | def_only | inf_only | always }
 			on:       s390 zlib hardware support for compression on
Documentation/networking/devlink/devlink-defaults.rst new file +80 -0

80 lines not on this page · raw message

Documentation/networking/devlink/index.rst +1 -0
index f7ba7dcf477d..0d27a7008b14 100644
@@ -56,6 +56,7 @@ general.
    :maxdepth: 1
 
    devlink-dpipe
+   devlink-defaults
    devlink-eswitch-attr
    devlink-flash
    devlink-health
net/devlink/core.c +261 -0

292 lines not on this page · raw message

net/devlink/dev.c +3 -0
index 57b2b8f03543..0b4a831465e8 100644
@@ -478,6 +478,9 @@ int devlink_reload(struct devlink *devlink, struct net *dest_net,
 		return err;
 
 	WARN_ON(!(*actions_performed & BIT(action)));
+	if (*actions_performed & BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT))
+		devlink_apply_default_esw_mode(devlink);
+
 	/* Catch driver on updating the remote action within devlink reload */
 	WARN_ON(memcmp(remote_reload_stats, devlink->stats.remote_reload_stats,
 		       sizeof(remote_reload_stats)));
net/devlink/devl_internal.h +1 -0
index e4e48ee2da5a..12557b65248d 100644
@@ -71,6 +71,7 @@ extern struct genl_family devlink_nl_family;
 struct devlink *__devlink_alloc(const struct devlink_ops *ops, size_t priv_size,
 				struct net *net, struct device *dev,
 				const struct device_driver *dev_driver);
+void devlink_apply_default_esw_mode(struct devlink *devlink);
 
 #define devl_warn(devlink, format, args...)				\
 	do {								\
Signature · 1 line
2.34.1
reviewer
#8 lore raw

Re: [PATCH net-next V2 7/7] devlink: Add eswitch mode boot defaults

Hi.

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1270

Quoted 12 lines · Mark Bloch
Documentation/admin-guide/kernel-parameters.txt +4 -0
index 063c11ca33e5..7af9f2898d92 100644
@@ -1264,6 +1264,31 @@ Kernel parameters
 	dell_smm_hwmon.fan_max=
 			[HW] Maximum configurable fan speed.
 
+	devlink_eswitch_mode=
+			[NET]
+			Format:
+			[<selector>]:<mode>

It appears (please correct me if I am mistaken) that the '[' and ']' above don't mean "optional" but instead they are required characters...

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1273

Quoted 3 lines
Documentation/admin-guide/kernel-parameters.txt +3 -0
+
+			<selector>:
+			* | <handle>[,<handle>...]

while here they mean "optional".

That is confusing (inconsistent). Also, if the square brackets are always required around the <selector>, what purpose do they serve?

Quoted 21 lines
Documentation/admin-guide/kernel-parameters.txt +18 -0
+
+			<handle>:
+			<bus-name>/<dev-name>
+
+			Configure default devlink eswitch mode for matching
+			devlink instances during device initialization.
+
+			<mode>:
+			legacy | switchdev | switchdev_inactive
+
+			Examples:
+			devlink_eswitch_mode=[*]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:legacy
+
+			See Documentation/networking/devlink/devlink-defaults.rst
+			for the full syntax.
+
 	dfltcc=		[HW,S390]
 			Format: { on | off | def_only | inf_only | always }
 			on:       s390 zlib hardware support for compression on
Signature · 1 line
~Randy
author maintainer
#9 lore raw

Re: [PATCH net-next V2 7/7] devlink: Add eswitch mode boot defaults

Quoted 19 lines · Randy Dunlap

Hi.

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1270

Quoted 12 lines · Mark Bloch
Documentation/admin-guide/kernel-parameters.txt +4 -0
index 063c11ca33e5..7af9f2898d92 100644
@@ -1264,6 +1264,31 @@ Kernel parameters
 	dell_smm_hwmon.fan_max=
 			[HW] Maximum configurable fan speed.
 
+	devlink_eswitch_mode=
+			[NET]
+			Format:
+			[<selector>]:<mode>

It appears (please correct me if I am mistaken) that the '[' and ']' above don't mean "optional" but instead they are required characters...

Quoted 8 lines

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1273

Quoted 3 lines
Documentation/admin-guide/kernel-parameters.txt +3 -0
+
+			<selector>:
+			* | <handle>[,<handle>...]

while here they mean "optional".

That is confusing (inconsistent). Also, if the square brackets are always required around the <selector>, what purpose do they serve?

Yes, you are right, this is confusing. The outer square brackets are part of the syntax and are required, while the brackets in "[,<handle>...]" mean that additional handles are optional.

I couldn't find a better way to describe this. What I want to say is that the selector is always wrapped in square brackets. Inside the brackets it can either be "*" to match all devices, or a comma separated list of handles. If "*" is not used, then at least one handle has to be provided.

Maybe it would be clearer to spell it out explicitly, something like:

Format:
[<selector>]:<mode>

The '[' and ']' characters are literal and required.

<selector>:
* | <handle>[,<handle>...]

If '*' is not used, <selector> must contain at least one <handle>.

Does that sound like a reasonable way to document it?

Mark

Quoted 24 lines
Quoted 22 lines
Documentation/admin-guide/kernel-parameters.txt +18 -0
+
+			<handle>:
+			<bus-name>/<dev-name>
+
+			Configure default devlink eswitch mode for matching
+			devlink instances during device initialization.
+
+			<mode>:
+			legacy | switchdev | switchdev_inactive
+
+			Examples:
+			devlink_eswitch_mode=[*]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:legacy
+
+			See Documentation/networking/devlink/devlink-defaults.rst
+			for the full syntax.
+
 	dfltcc=		[HW,S390]
 			Format: { on | off | def_only | inf_only | always }
 			on:       s390 zlib hardware support for compression on
reviewer
#10 lore raw

Re: [PATCH net-next V2 7/7] devlink: Add eswitch mode boot defaults

Quoted 45 lines · Mark Bloch
Quoted 19 lines · Randy Dunlap

Hi.

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1270

Quoted 12 lines · Mark Bloch
Documentation/admin-guide/kernel-parameters.txt +4 -0
index 063c11ca33e5..7af9f2898d92 100644
@@ -1264,6 +1264,31 @@ Kernel parameters
 	dell_smm_hwmon.fan_max=
 			[HW] Maximum configurable fan speed.
 
+	devlink_eswitch_mode=
+			[NET]
+			Format:
+			[<selector>]:<mode>

It appears (please correct me if I am mistaken) that the '[' and ']' above don't mean "optional" but instead they are required characters...

Quoted 8 lines

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1273

Quoted 3 lines
Documentation/admin-guide/kernel-parameters.txt +3 -0
+
+			<selector>:
+			* | <handle>[,<handle>...]

while here they mean "optional".

That is confusing (inconsistent). Also, if the square brackets are always required around the <selector>, what purpose do they serve?

Yes, you are right, this is confusing. The outer square brackets are part of the syntax and are required, while the brackets in "[,<handle>...]" mean that additional handles are optional.

I couldn't find a better way to describe this. What I want to say is that the selector is always wrapped in square brackets. Inside the brackets it can either be "*" to match all devices, or a comma separated list of handles. If "*" is not used, then at least one handle has to be provided.

Maybe it would be clearer to spell it out explicitly, something like:

Format:
[<selector>]:<mode>

Quoted 8 lines

The '[' and ']' characters are literal and required.

<selector>:
* | <handle>[,<handle>...]

If '*' is not used, <selector> must contain at least one <handle>.

Does that sound like a reasonable way to document it?

Yes, that helps a little bit. Better than nothing.

But why are they required at all?

Quoted 24 lines
Quoted 23 lines
Quoted 21 lines
Documentation/admin-guide/kernel-parameters.txt +18 -0
+
+			<handle>:
+			<bus-name>/<dev-name>
+
+			Configure default devlink eswitch mode for matching
+			devlink instances during device initialization.
+
+			<mode>:
+			legacy | switchdev | switchdev_inactive
+
+			Examples:
+			devlink_eswitch_mode=[*]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:legacy
+
+			See Documentation/networking/devlink/devlink-defaults.rst
+			for the full syntax.
+
 	dfltcc=		[HW,S390]
 			Format: { on | off | def_only | inf_only | always }
 			on:       s390 zlib hardware support for compression on
Signature · 1 line
~Randy
author maintainer
#11 lore raw

Re: [PATCH net-next V2 7/7] devlink: Add eswitch mode boot defaults

Quoted 53 lines · Randy Dunlap
Quoted 50 lines · Mark Bloch
Quoted 19 lines · Randy Dunlap

Hi.

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1270

Quoted 12 lines · Mark Bloch
Documentation/admin-guide/kernel-parameters.txt +4 -0
index 063c11ca33e5..7af9f2898d92 100644
@@ -1264,6 +1264,31 @@ Kernel parameters
 	dell_smm_hwmon.fan_max=
 			[HW] Maximum configurable fan speed.
 
+	devlink_eswitch_mode=
+			[NET]
+			Format:
+			[<selector>]:<mode>

It appears (please correct me if I am mistaken) that the '[' and ']' above don't mean "optional" but instead they are required characters...

Quoted 8 lines

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1273

Quoted 3 lines
Documentation/admin-guide/kernel-parameters.txt +3 -0
+
+			<selector>:
+			* | <handle>[,<handle>...]

while here they mean "optional".

That is confusing (inconsistent). Also, if the square brackets are always required around the <selector>, what purpose do they serve?

Yes, you are right, this is confusing. The outer square brackets are part of the syntax and are required, while the brackets in "[,<handle>...]" mean that additional handles are optional.

I couldn't find a better way to describe this. What I want to say is that the selector is always wrapped in square brackets. Inside the brackets it can either be "*" to match all devices, or a comma separated list of handles. If "*" is not used, then at least one handle has to be provided.

Maybe it would be clearer to spell it out explicitly, something like:

Format:
[<selector>]:<mode>

The '[' and ']' characters are literal and required.

<selector>:
* | <handle>[,<handle>...]

Quoted 7 lines
Quoted 3 lines

If '*' is not used, <selector> must contain at least one <handle>.

Does that sound like a reasonable way to document it?

Yes, that helps a little bit. Better than nothing.

But why are they required at all?

Jiri suggested using the square brackets, and I liked that they made the selector look like a grouped argument. But if that is too confusing, I can also drop them and use a simpler separator, for example:

	devlink_eswitch_mode=
			[NET]
			Format:
			<selector>=<mode>

			<selector>:
			* | <handle>[,<handle>...]

			<handle>:
			<bus-name>/<dev-name>

			Configure default devlink eswitch mode for matching
			devlink instances during device initialization.

			<mode>:
			legacy | switchdev | switchdev_inactive

			Examples:
			devlink_eswitch_mode=*=switchdev
			devlink_eswitch_mode=pci/0000:08:00.0=switchdev
			devlink_eswitch_mode=pci/0000:08:00.0,pci/0000:09:00.1=switchdev_inactive

Does this look better to you?

Mark

Quoted 26 lines
Quoted 25 lines
Quoted 24 lines
Quoted 22 lines
Documentation/admin-guide/kernel-parameters.txt +18 -0
+
+			<handle>:
+			<bus-name>/<dev-name>
+
+			Configure default devlink eswitch mode for matching
+			devlink instances during device initialization.
+
+			<mode>:
+			legacy | switchdev | switchdev_inactive
+
+			Examples:
+			devlink_eswitch_mode=[*]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:legacy
+
+			See Documentation/networking/devlink/devlink-defaults.rst
+			for the full syntax.
+
 	dfltcc=		[HW,S390]
 			Format: { on | off | def_only | inf_only | always }
 			on:       s390 zlib hardware support for compression on
reviewer
#12 lore raw

Re: [PATCH net-next V2 7/7] devlink: Add eswitch mode boot defaults

Quoted 90 lines · Mark Bloch
Quoted 53 lines · Randy Dunlap
Quoted 50 lines · Mark Bloch
Quoted 19 lines · Randy Dunlap

Hi.

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1270

Quoted 12 lines · Mark Bloch
Documentation/admin-guide/kernel-parameters.txt +4 -0
index 063c11ca33e5..7af9f2898d92 100644
@@ -1264,6 +1264,31 @@ Kernel parameters
 	dell_smm_hwmon.fan_max=
 			[HW] Maximum configurable fan speed.
 
+	devlink_eswitch_mode=
+			[NET]
+			Format:
+			[<selector>]:<mode>

It appears (please correct me if I am mistaken) that the '[' and ']' above don't mean "optional" but instead they are required characters...

Quoted 8 lines

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1273

Quoted 3 lines
Documentation/admin-guide/kernel-parameters.txt +3 -0
+
+			<selector>:
+			* | <handle>[,<handle>...]

while here they mean "optional".

That is confusing (inconsistent). Also, if the square brackets are always required around the <selector>, what purpose do they serve?

Yes, you are right, this is confusing. The outer square brackets are part of the syntax and are required, while the brackets in "[,<handle>...]" mean that additional handles are optional.

I couldn't find a better way to describe this. What I want to say is that the selector is always wrapped in square brackets. Inside the brackets it can either be "*" to match all devices, or a comma separated list of handles. If "*" is not used, then at least one handle has to be provided.

Maybe it would be clearer to spell it out explicitly, something like:

Format:
[<selector>]:<mode>

The '[' and ']' characters are literal and required.

<selector>:
* | <handle>[,<handle>...]

Quoted 7 lines
Quoted 3 lines

If '*' is not used, <selector> must contain at least one <handle>.

Does that sound like a reasonable way to document it?

Yes, that helps a little bit. Better than nothing.

But why are they required at all?

Jiri suggested using the square brackets, and I liked that they made the selector look like a grouped argument. But if that is too confusing, I can also drop them and use a simpler separator, for example:

	devlink_eswitch_mode=
			[NET]
			Format:
			<selector>=<mode>

			<selector>:
			* | <handle>[,<handle>...]

			<handle>:
			<bus-name>/<dev-name>

			Configure default devlink eswitch mode for matching
			devlink instances during device initialization.

			<mode>:
			legacy | switchdev | switchdev_inactive

			Examples:
			devlink_eswitch_mode=*=switchdev
			devlink_eswitch_mode=pci/0000:08:00.0=switchdev
			devlink_eswitch_mode=pci/0000:08:00.0,pci/0000:09:00.1=switchdev_inactive
Quoted 1 line

Does this look better to you?

Yes, that looks much better to me.
But you should do whatever you think is right.

Quoted 26 lines
Quoted 25 lines
Quoted 24 lines
Quoted 23 lines
Quoted 21 lines
Documentation/admin-guide/kernel-parameters.txt +18 -0
+
+			<handle>:
+			<bus-name>/<dev-name>
+
+			Configure default devlink eswitch mode for matching
+			devlink instances during device initialization.
+
+			<mode>:
+			legacy | switchdev | switchdev_inactive
+
+			Examples:
+			devlink_eswitch_mode=[*]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:legacy
+
+			See Documentation/networking/devlink/devlink-defaults.rst
+			for the full syntax.
+
 	dfltcc=		[HW,S390]
 			Format: { on | off | def_only | inf_only | always }
 			on:       s390 zlib hardware support for compression on
Signature · 1 line
~Randy
author maintainer
#13 lore raw

Re: [PATCH net-next V2 7/7] devlink: Add eswitch mode boot defaults

Quoted 93 lines · Randy Dunlap
Quoted 90 lines · Mark Bloch
Quoted 53 lines · Randy Dunlap
Quoted 50 lines · Mark Bloch
Quoted 19 lines · Randy Dunlap

Hi.

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1270

Quoted 12 lines · Mark Bloch
Documentation/admin-guide/kernel-parameters.txt +4 -0
index 063c11ca33e5..7af9f2898d92 100644
@@ -1264,6 +1264,31 @@ Kernel parameters
 	dell_smm_hwmon.fan_max=
 			[HW] Maximum configurable fan speed.
 
+	devlink_eswitch_mode=
+			[NET]
+			Format:
+			[<selector>]:<mode>

It appears (please correct me if I am mistaken) that the '[' and ']' above don't mean "optional" but instead they are required characters...

Quoted 8 lines

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 1273

Quoted 3 lines
Documentation/admin-guide/kernel-parameters.txt +3 -0
+
+			<selector>:
+			* | <handle>[,<handle>...]

while here they mean "optional".

That is confusing (inconsistent). Also, if the square brackets are always required around the <selector>, what purpose do they serve?

Yes, you are right, this is confusing. The outer square brackets are part of the syntax and are required, while the brackets in "[,<handle>...]" mean that additional handles are optional.

I couldn't find a better way to describe this. What I want to say is that the selector is always wrapped in square brackets. Inside the brackets it can either be "*" to match all devices, or a comma separated list of handles. If "*" is not used, then at least one handle has to be provided.

Maybe it would be clearer to spell it out explicitly, something like:

Format:
[<selector>]:<mode>

The '[' and ']' characters are literal and required.

<selector>:
* | <handle>[,<handle>...]

Quoted 7 lines
Quoted 3 lines

If '*' is not used, <selector> must contain at least one <handle>.

Does that sound like a reasonable way to document it?

Yes, that helps a little bit. Better than nothing.

But why are they required at all?

Jiri suggested using the square brackets, and I liked that they made the selector look like a grouped argument. But if that is too confusing, I can also drop them and use a simpler separator, for example:

	devlink_eswitch_mode=
			[NET]
			Format:
			<selector>=<mode>

			<selector>:
			* | <handle>[,<handle>...]

			<handle>:
			<bus-name>/<dev-name>

			Configure default devlink eswitch mode for matching
			devlink instances during device initialization.

			<mode>:
			legacy | switchdev | switchdev_inactive

			Examples:
			devlink_eswitch_mode=*=switchdev
			devlink_eswitch_mode=pci/0000:08:00.0=switchdev
			devlink_eswitch_mode=pci/0000:08:00.0,pci/0000:09:00.1=switchdev_inactive
Quoted 4 lines
Quoted 1 line

Does this look better to you?

Yes, that looks much better to me.
But you should do whatever you think is right.

Good syntax should not be confusing. If it was not clear to you, then it will probably not be clear to others either, so I am fine with changing it.

I will post v3 with this change.

Thanks for the input!

Mark

Quoted 28 lines
Quoted 27 lines
Quoted 26 lines
Quoted 25 lines
Quoted 24 lines
Quoted 22 lines
Documentation/admin-guide/kernel-parameters.txt +18 -0
+
+			<handle>:
+			<bus-name>/<dev-name>
+
+			Configure default devlink eswitch mode for matching
+			devlink instances during device initialization.
+
+			<mode>:
+			legacy | switchdev | switchdev_inactive
+
+			Examples:
+			devlink_eswitch_mode=[*]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0]:switchdev
+			devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:legacy
+
+			See Documentation/networking/devlink/devlink-defaults.rst
+			for the full syntax.
+
 	dfltcc=		[HW,S390]
 			Format: { on | off | def_only | inf_only | always }
 			on:       s390 zlib hardware support for compression on