Skip to contentSkip to conversation

devlink: Add boot-time eswitch mode defaults

PATCH Superseded Mark Bloch 2 participants 8 messages Jul 14

Subsystems (6)

v6 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Mark Bloch

author maintainer
#0 lore raw

v6 cover devlink: Add boot-time eswitch mode defaults

This series adds a devlink_eswitch_mode= kernel command line parameter for setting a default devlink eswitch mode during boot.

Following the discussion with Jakub[1] and the feedback on the RFC postings, this version keeps the scope limited to a boot-time devlink eswitch mode default only.

The option selects either all devlink handles or an explicit comma-separated handle list:

devlink_eswitch_mode=*=switchdev devlink_eswitch_mode=pci/0000:08:00.0,pci/0000:09:00.1=switchdev_inactive

The supported modes are legacy, switchdev and switchdev_inactive. The selected mode is applied through the existing eswitch_mode_set() devlink operation, the same operation used by the devlink eswitch mode command.

Registration may happen while a driver holds the devlink lock and continues device initialization. Devlink core marks the default as pending and applies it from devl_unlock() once the instance is registered, before releasing the lock. This prevents userspace from racing with the boot default.

After a successful reload that performed DRIVER_REINIT, devlink core already holds the devlink instance lock and the driver completed reload_up(), so the default is applied directly from the reload path.

Patch 1 clears the mlx5 FW reset-in-progress bit before reload.

Patch 2 factors the common eswitch mode set validation into a helper.

Patch 3 adds the devlink_eswitch_mode= parser and documentation.

Patch 4 applies parsed defaults from devlink core.

Changelog:

v5 -> v6:

- Dropped regular work and applied a pending default directly from devl_unlock() while the devlink instance lock is still held.

- Dropped the driver API and mlx5-specific default application patches.

v4 -> v5:

- Moved the default eswitch mode code into a separate file, per Jiri's comment.

- Dropped the delayed workqueue and switched to regular work triggered via devl_unlock(), per Jiri's comment.

- Renamed some functions to better align with devlink code.

v3 -> v4:

- Rework registration time apply to use per devlink work queued from devl_unlock(), instead of calling eswitch_mode_set() directly from devl_register().

- Apply the default directly after successful DRIVER_REINIT devlink reload, where the devlink lock is already held and reload_up() has completed.

- Add devl_apply_default_esw_mode() for drivers that know their exact ready point.

- Drop the driver registration-ordering preparation patches that are no longer needed with the async registration apply path.

v2 -> v3:

- Change the devlink_eswitch_mode= API syntax to use <selector>=<mode> instead of [<selector>]:<mode>, following a comment from Randy Dunlap.

v1 -> v2:

- Move default eswitch mode application into devlink core. The default is now applied during devlink registration and after a successful devlink reload that performed DRIVER_REINIT.

- Remove the exported devl_apply_default_esw_mode() driver API and the mlx5 driver-side call to it.

- Skip devlink health recovery notifications while the devlink instance is not registered, so drivers can move registration later without early health work hitting registration assertions.

- Move mlx5 devlink registration after device initialization, including the lightweight init path, so the core can apply the default through the normal registration flow.

- Move the matching netdevsim and mlx5 unregister paths before object teardown, so unregister notifications come from devl_unregister() and the later object teardown paths run while the devlink instance is no longer registered.

- Add registration-ordering preparation patches for netdevsim and octeontx2 AF/PF, so their eswitch state is ready before registration-time defaults may call eswitch_mode_set().

[1] lore.kernel.org/r/[email protected]/ RFC v1: lore.kernel.org/r/[email protected]/ RFC v2: lore.kernel.org/r/[email protected]/
v1: lore.kernel.org/r/[email protected]/
v2: lore.kernel.org/all/[email protected]/
v3: lore.kernel.org/all/[email protected]/
v4: lore.kernel.org/all/[email protected]/
v5: lore.kernel.org/all/[email protected]/

Mark Bloch (4):
net/mlx5: Clear FW reset-in-progress bit before reload
devlink: Factor out eswitch mode setting
devlink: Parse eswitch mode boot defaults
devlink: Apply eswitch mode boot defaults

 .../admin-guide/kernel-parameters.txt         |  25 ++
 .../networking/devlink/devlink-defaults.rst   |  78 +++++
 Documentation/networking/devlink/index.rst    |   1 +
 .../ethernet/mellanox/mlx5/core/fw_reset.c    |  28 +-
 net/devlink/Makefile                          |   2 +-
 net/devlink/core.c                            |  10 +
 net/devlink/default.c                         | 303 ++++++++++++++++++
 net/devlink/dev.c                             |  33 +-
 net/devlink/devl_internal.h                   |  10 +
 9 files changed, 471 insertions(+), 19 deletions(-)
 create mode 100644 Documentation/networking/devlink/devlink-defaults.rst
 create mode 100644 net/devlink/default.c

base-commit: f6f3b36c15ed44de1fbb44e645e4fae8c4a4453e

Signature · 1 line
2.43.0
author maintainer
#1 lore raw

PATCH v6 1/4 net/mlx5: Clear FW reset-in-progress bit before reload

mlx5 sets MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS when acknowledging a sync reset request. This bit blocks devlink reload and other devlink operations while the firmware reset is running, but it was kept set until after the driver reload finished.

Clear the reset-in-progress bit once the reset unload flow is done and PCI access is back, before reloading the device. For a reset initiated through devlink, clear it before completing the reload waiter. For a reset reported through an asynchronous firmware event, keep the unload flow outside devl_lock, then take devl_lock before clearing the bit and reloading through the devl-locked load helper.

---
 .../ethernet/mellanox/mlx5/core/fw_reset.c    | 28 +++++++++++--------
 1 file changed, 17 insertions(+), 11 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +17 -11
index 07440c58713a..7283e5b49eed 100644
@@ -238,24 +238,30 @@ static void mlx5_fw_reset_complete_reload(struct mlx5_core_dev *dev)
 {
 	struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset;
 	struct devlink *devlink = priv_to_devlink(dev);
+	int err;
 
 	/* if this is the driver that initiated the fw reset, devlink completed the reload */
 	if (test_bit(MLX5_FW_RESET_FLAGS_PENDING_COMP, &fw_reset->reset_flags)) {
+		clear_bit(MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS,
+			  &fw_reset->reset_flags);
 		complete(&fw_reset->done);
-	} else {
-		mlx5_sync_reset_unload_flow(dev, false);
-		if (mlx5_health_wait_pci_up(dev))
-			mlx5_core_err(dev, "reset reload flow aborted, PCI reads still not working\n");
-		else
-			mlx5_load_one(dev, true);
-		devl_lock(devlink);
-		devlink_remote_reload_actions_performed(devlink, 0,
-							BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT) |
-							BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE));
-		devl_unlock(devlink);
+		return;
 	}
 
+	mlx5_sync_reset_unload_flow(dev, false);
+	err = mlx5_health_wait_pci_up(dev);
+
+	devl_lock(devlink);
 	clear_bit(MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS, &fw_reset->reset_flags);
+	if (err)
+		mlx5_core_err(dev, "reset reload flow aborted, PCI reads still not working\n");
+	else
+		mlx5_load_one_devl_locked(dev, true);
+
+	devlink_remote_reload_actions_performed(devlink, 0,
+						BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT) |
+						BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE));
+	devl_unlock(devlink);
 }
 
 static void mlx5_stop_sync_reset_poll(struct mlx5_core_dev *dev)
Signature · 1 line
2.43.0
author maintainer
#2 lore raw

PATCH v6 2/4 devlink: Factor out eswitch mode setting

In reply to Mark Bloch

Move the common eswitch mode set checks into a small helper and use it from the netlink eswitch set command. This makes the same validation available to the devlink core path that applies eswitch mode defaults.

---
 net/devlink/dev.c           | 27 ++++++++++++++++++++-------
 net/devlink/devl_internal.h |  3 +++
 2 files changed, 23 insertions(+), 7 deletions(-)
net/devlink/dev.c +20 -7
index bcf001554e84..119ef105d0a7 100644
@@ -702,6 +702,25 @@ int devlink_nl_eswitch_get_doit(struct sk_buff *skb, struct genl_info *info)
 	return genlmsg_reply(msg, info);
 }
 
+int devlink_eswitch_mode_set(struct devlink *devlink,
+			     enum devlink_eswitch_mode mode,
+			     struct netlink_ext_ack *extack)
+{
+	const struct devlink_ops *ops = devlink->ops;
+	int err;
+
+	devl_assert_locked(devlink);
+
+	if (!ops->eswitch_mode_set)
+		return -EOPNOTSUPP;
+
+	err = devlink_rates_check(devlink, devlink_rate_is_node, extack);
+	if (err)
+		return err;
+
+	return ops->eswitch_mode_set(devlink, mode, extack);
+}
+
 int devlink_nl_eswitch_set_doit(struct sk_buff *skb, struct genl_info *info)
 {
 	struct devlink *devlink = devlink_nl_ctx(info)->devlink;
@@ -712,14 +731,8 @@ int devlink_nl_eswitch_set_doit(struct sk_buff *skb, struct genl_info *info)
 	u16 mode;
 
 	if (info->attrs[DEVLINK_ATTR_ESWITCH_MODE]) {
-		if (!ops->eswitch_mode_set)
-			return -EOPNOTSUPP;
-		err = devlink_rates_check(devlink, devlink_rate_is_node,
-					  info->extack);
-		if (err)
-			return err;
 		mode = nla_get_u16(info->attrs[DEVLINK_ATTR_ESWITCH_MODE]);
-		err = ops->eswitch_mode_set(devlink, mode, info->extack);
+		err = devlink_eswitch_mode_set(devlink, mode, info->extack);
 		if (err)
 			return err;
 	}
net/devlink/devl_internal.h +3 -0
index cdf894ba5a9d..af43b7163f78 100644
@@ -348,6 +348,9 @@ bool devlink_rate_is_node(const struct devlink_rate *devlink_rate);
 int devlink_rates_check(struct devlink *devlink,
 			bool (*rate_filter)(const struct devlink_rate *),
 			struct netlink_ext_ack *extack);
+int devlink_eswitch_mode_set(struct devlink *devlink,
+			     enum devlink_eswitch_mode mode,
+			     struct netlink_ext_ack *extack);
 
 /* Linecards */
 unsigned int devlink_linecard_index(struct devlink_linecard *linecard);
Signature · 1 line
2.43.0
maintainer Reviewed-by on v6 2/4
#3 lore raw

Re: [PATCH net-next V6 2/4] devlink: Factor out eswitch mode setting

Quoted 5 lines · Tue, Jul 14, 2026 at 08:17:28AM +0200, [email protected] wrote:

Move the common eswitch mode set checks into a small helper and use it from the netlink eswitch set command. This makes the same validation available to the devlink core path that applies eswitch mode defaults.

author maintainer
#4 lore raw

PATCH v6 3/4 devlink: Parse eswitch mode boot defaults

In reply to Mark Bloch

Add devlink_eswitch_mode= kernel command line parsing for a default eswitch mode.

The supported syntax selects either all devlink handles or one explicit comma-separated handle list:

devlink_eswitch_mode=*=<mode>

devlink_eswitch_mode=<handle>[,<handle>...]=<mode>

where <mode> is one of legacy, switchdev or switchdev_inactive. All selected handles receive the same mode. Assigning different modes to different handle lists in the same parameter value is not supported.

Store the parsed selector and mode in devlink core so the default can be applied by a downstream patch.

Document the devlink_eswitch_mode= syntax and duplicate handle handling.

---
 .../admin-guide/kernel-parameters.txt         |  25 ++
 .../networking/devlink/devlink-defaults.rst   |  78 ++++++
 Documentation/networking/devlink/index.rst    |   1 +
 net/devlink/Makefile                          |   2 +-
 net/devlink/core.c                            |   7 +
 net/devlink/default.c                         | 237 ++++++++++++++++++
 net/devlink/devl_internal.h                   |   2 +
 7 files changed, 351 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/networking/devlink/devlink-defaults.rst
 create mode 100644 net/devlink/default.c
Documentation/admin-guide/kernel-parameters.txt +25 -0
index b5493a7f8f22..117300dd589c 100644
@@ -1249,6 +1249,31 @@ Kernel parameters
 	dell_smm_hwmon.fan_max=
 			[HW] Maximum configurable fan speed.
 
+	devlink_eswitch_mode=
+			[NET]
+			Format:
+			<selector>=<mode>
+
+			<selector>:
+			* | <handle>[,<handle>...]
+
+			<handle>:
+			<bus-name>/<dev-name>
+
+			Configure default devlink eswitch mode for matching
+			devlink instances during device initialization.
+
+			<mode>:
+			legacy | switchdev | switchdev_inactive
+
+			Examples:
+			devlink_eswitch_mode=*=switchdev
+			devlink_eswitch_mode=pci/0000:08:00.0=switchdev
+			devlink_eswitch_mode=pci/0000:08:00.0,pci/0000:09:00.1=switchdev_inactive
+
+			See Documentation/networking/devlink/devlink-defaults.rst
+			for the full syntax.
+
 	dfltcc=		[HW,S390]
 			Format: { on | off | def_only | inf_only | always }
 			on:       s390 zlib hardware support for compression on
Documentation/networking/devlink/devlink-defaults.rst new file +78 -0

78 lines not on this page · raw message

Documentation/networking/devlink/index.rst +1 -0
index 4745148fecf4..134d2f319922 100644
@@ -56,6 +56,7 @@ general.
    :maxdepth: 1
 
    devlink-dpipe
+   devlink-defaults
    devlink-eswitch-attr
    devlink-flash
    devlink-health
net/devlink/Makefile +1 -1
index 8f2adb5e5836..99ca0ef7cf1e 100644
@@ -1,4 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
 
-obj-y := core.o netlink.o netlink_gen.o dev.o port.o sb.o dpipe.o \
+obj-y := core.o netlink.o netlink_gen.o dev.o default.o port.o sb.o dpipe.o \
 	 resource.o param.o region.o health.o trap.o rate.o linecard.o sh_dev.o
net/devlink/core.c +7 -0
index c53a42e17a58..fc14ee5d9dcf 100644
@@ -598,6 +598,10 @@ static int __init devlink_init(void)
 {
 	int err;
 
+	err = devlink_default_esw_mode_init();
+	if (err)
+		goto out;
+
 	err = register_pernet_subsys(&devlink_pernet_ops);
 	if (err)
 		goto out;
@@ -613,7 +617,10 @@ static int __init devlink_init(void)
 out_unreg_pernet_subsys:
 	unregister_pernet_subsys(&devlink_pernet_ops);
 out:
+	if (err)
+		devlink_default_esw_mode_cleanup();
 	WARN_ON(err);
+
 	return err;
 }
 
net/devlink/default.c new file +237 -0

237 lines not on this page · raw message

net/devlink/devl_internal.h +2 -0
index af43b7163f78..fe9ad58515d4 100644
@@ -71,6 +71,8 @@ extern struct genl_family devlink_nl_family;
 struct devlink *__devlink_alloc(const struct devlink_ops *ops, size_t priv_size,
 				struct net *net, struct device *dev,
 				const struct device_driver *dev_driver);
+int devlink_default_esw_mode_init(void);
+void devlink_default_esw_mode_cleanup(void);
 
 #define devl_warn(devlink, format, args...)				\
 	do {								\
Signature · 1 line
2.43.0
maintainer Reviewed-by on v6 3/4
#5 lore raw

Re: [PATCH net-next V6 3/4] devlink: Parse eswitch mode boot defaults

Quoted 14 lines · Tue, Jul 14, 2026 at 08:17:29AM +0200, [email protected] wrote:

Add devlink_eswitch_mode= kernel command line parsing for a default eswitch mode.

The supported syntax selects either all devlink handles or one explicit comma-separated handle list:

devlink_eswitch_mode=*=<mode>

devlink_eswitch_mode=<handle>[,<handle>...]=<mode>

where <mode> is one of legacy, switchdev or switchdev_inactive. All selected handles receive the same mode. Assigning different modes to different handle lists in the same parameter value is not supported.

Quoted 6 lines

Store the parsed selector and mode in devlink core so the default can be applied by a downstream patch.

Document the devlink_eswitch_mode= syntax and duplicate handle handling.

author maintainer
#6 lore raw

PATCH v6 4/4 devlink: Apply eswitch mode boot defaults

In reply to Mark Bloch

Apply parsed devlink_eswitch_mode= defaults after devlink registration and after successful reload.

Mark the default mode as pending when a devlink instance is allocated. Before devl_unlock() releases the instance lock, apply a pending default when the instance is registered.

Clear the pending state before calling into the driver so the boot default remains a one-shot operation even if the mode change fails.

For successful reloads that performed DRIVER_REINIT, devlink_reload() already holds the devlink instance lock and the driver has completed reload_up(). Clear the pending state and apply the default directly from the reload path.

Treat an explicit user eswitch mode request as consuming the pending default mode.

---
 net/devlink/core.c          |  3 ++
 net/devlink/default.c       | 70 +++++++++++++++++++++++++++++++++++--
 net/devlink/dev.c           |  6 ++++
 net/devlink/devl_internal.h |  5 +++
 4 files changed, 82 insertions(+), 2 deletions(-)
net/devlink/core.c +3 -0
index fc14ee5d9dcf..cea4afc27dd9 100644
@@ -317,6 +317,7 @@ EXPORT_SYMBOL_GPL(devl_trylock);
 
 void devl_unlock(struct devlink *devlink)
 {
+	devlink_default_esw_mode_apply_pending(devlink);
 	mutex_unlock(&devlink->lock);
 }
 EXPORT_SYMBOL_GPL(devl_unlock);
@@ -429,6 +430,7 @@ void devl_unregister(struct devlink *devlink)
 	ASSERT_DEVLINK_REGISTERED(devlink);
 	devl_assert_locked(devlink);
 
+	devlink_default_esw_mode_apply_pending_clear(devlink);
 	devlink_notify_unregister(devlink);
 	xa_clear_mark(&devlinks, devlink->index, DEVLINK_REGISTERED);
 	devlink_rel_put(devlink);
@@ -490,6 +492,7 @@ struct devlink *__devlink_alloc(const struct devlink_ops *ops, size_t priv_size,
 	INIT_LIST_HEAD(&devlink->trap_group_list);
 	INIT_LIST_HEAD(&devlink->trap_policer_list);
 	INIT_RCU_WORK(&devlink->rwork, devlink_release);
+	devlink_default_esw_mode_instance_init(devlink);
 	lockdep_register_key(&devlink->lock_key);
 	mutex_init(&devlink->lock);
 	lockdep_set_class(&devlink->lock, &devlink->lock_key);
net/devlink/default.c +68 -2

94 lines not on this page · raw message

net/devlink/dev.c +6 -0
index 119ef105d0a7..611bb6bfd492 100644
@@ -478,6 +478,11 @@ int devlink_reload(struct devlink *devlink, struct net *dest_net,
 		return err;
 
 	WARN_ON(!(*actions_performed & BIT(action)));
+	if (*actions_performed & BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT)) {
+		devlink_default_esw_mode_apply_pending_clear(devlink);
+		devlink_default_esw_mode_apply_locked(devlink);
+	}
+
 	/* Catch driver on updating the remote action within devlink reload */
 	WARN_ON(memcmp(remote_reload_stats, devlink->stats.remote_reload_stats,
 		       sizeof(remote_reload_stats)));
@@ -731,6 +736,7 @@ int devlink_nl_eswitch_set_doit(struct sk_buff *skb, struct genl_info *info)
 	u16 mode;
 
 	if (info->attrs[DEVLINK_ATTR_ESWITCH_MODE]) {
+		devlink_default_esw_mode_apply_pending_clear(devlink);
 		mode = nla_get_u16(info->attrs[DEVLINK_ATTR_ESWITCH_MODE]);
 		err = devlink_eswitch_mode_set(devlink, mode, info->extack);
 		if (err)
net/devlink/devl_internal.h +5 -0
index fe9ad58515d4..97f53394b1c0 100644
@@ -58,6 +58,7 @@ struct devlink {
 	struct mutex lock;
 	struct lock_class_key lock_key;
 	u8 reload_failed:1;
+	u8 default_esw_mode_apply_pending:1;
 	refcount_t refcount;
 	struct rcu_work rwork;
 	struct devlink_rel *rel;
@@ -73,6 +74,10 @@ struct devlink *__devlink_alloc(const struct devlink_ops *ops, size_t priv_size,
 				const struct device_driver *dev_driver);
 int devlink_default_esw_mode_init(void);
 void devlink_default_esw_mode_cleanup(void);
+void devlink_default_esw_mode_instance_init(struct devlink *devlink);
+void devlink_default_esw_mode_apply_locked(struct devlink *devlink);
+void devlink_default_esw_mode_apply_pending(struct devlink *devlink);
+void devlink_default_esw_mode_apply_pending_clear(struct devlink *devlink);
 
 #define devl_warn(devlink, format, args...)				\
 	do {								\
Signature · 1 line
2.43.0
maintainer Reviewed-by on v6 4/4
#7 lore raw

Re: [PATCH net-next V6 4/4] devlink: Apply eswitch mode boot defaults

Quoted 10 lines · Tue, Jul 14, 2026 at 08:17:30AM +0200, [email protected] wrote:

Apply parsed devlink_eswitch_mode= defaults after devlink registration and after successful reload.

Mark the default mode as pending when a devlink instance is allocated. Before devl_unlock() releases the instance lock, apply a pending default when the instance is registered.

Clear the pending state before calling into the driver so the boot default remains a one-shot operation even if the mode change fails.

Quoted 8 lines

For successful reloads that performed DRIVER_REINIT, devlink_reload() already holds the devlink instance lock and the driver has completed reload_up(). Clear the pending state and apply the default directly from the reload path.

Treat an explicit user eswitch mode request as consuming the pending default mode.

Looks fine to me now. One nit below:

[..]

Quoted 1 line

+void devlink_default_esw_mode_instance_init(struct devlink *devlink)

Forgotten "instance" here. With this fixed, feel free to add my

[..]