devlink: Add boot-time eswitch mode defaults
Subsystems (6)
Mark Bloch
maintainerv6 cover devlink: Add boot-time eswitch mode defaults
This series adds a devlink_eswitch_mode= kernel command line parameter for setting a default devlink eswitch mode during boot.
Following the discussion with Jakub[1] and the feedback on the RFC postings, this version keeps the scope limited to a boot-time devlink eswitch mode default only.
The option selects either all devlink handles or an explicit comma-separated handle list:
devlink_eswitch_mode=*=switchdev devlink_eswitch_mode=pci/0000:08:00.0,pci/0000:09:00.1=switchdev_inactive
The supported modes are legacy, switchdev and switchdev_inactive. The selected mode is applied through the existing eswitch_mode_set() devlink operation, the same operation used by the devlink eswitch mode command.
Registration may happen while a driver holds the devlink lock and continues device initialization. Devlink core marks the default as pending and applies it from devl_unlock() once the instance is registered, before releasing the lock. This prevents userspace from racing with the boot default.
After a successful reload that performed DRIVER_REINIT, devlink core already holds the devlink instance lock and the driver completed reload_up(), so the default is applied directly from the reload path.
Patch 1 clears the mlx5 FW reset-in-progress bit before reload.
Patch 2 factors the common eswitch mode set validation into a helper.
Patch 3 adds the devlink_eswitch_mode= parser and documentation.
Patch 4 applies parsed defaults from devlink core.
Changelog:
v5 -> v6:
- Dropped regular work and applied a pending default directly from devl_unlock() while the devlink instance lock is still held.
- Dropped the driver API and mlx5-specific default application patches.
v4 -> v5:
- Moved the default eswitch mode code into a separate file, per Jiri's comment.
- Dropped the delayed workqueue and switched to regular work triggered via devl_unlock(), per Jiri's comment.
- Renamed some functions to better align with devlink code.
v3 -> v4:
- Rework registration time apply to use per devlink work queued from devl_unlock(), instead of calling eswitch_mode_set() directly from devl_register().
- Apply the default directly after successful DRIVER_REINIT devlink reload, where the devlink lock is already held and reload_up() has completed.
- Add devl_apply_default_esw_mode() for drivers that know their exact ready point.
- Drop the driver registration-ordering preparation patches that are no longer needed with the async registration apply path.
v2 -> v3:
- Change the devlink_eswitch_mode= API syntax to use <selector>=<mode> instead of [<selector>]:<mode>, following a comment from Randy Dunlap.
v1 -> v2:
- Move default eswitch mode application into devlink core. The default is now applied during devlink registration and after a successful devlink reload that performed DRIVER_REINIT.
- Remove the exported devl_apply_default_esw_mode() driver API and the mlx5 driver-side call to it.
- Skip devlink health recovery notifications while the devlink instance is not registered, so drivers can move registration later without early health work hitting registration assertions.
- Move mlx5 devlink registration after device initialization, including the lightweight init path, so the core can apply the default through the normal registration flow.
- Move the matching netdevsim and mlx5 unregister paths before object teardown, so unregister notifications come from devl_unregister() and the later object teardown paths run while the devlink instance is no longer registered.
- Add registration-ordering preparation patches for netdevsim and octeontx2 AF/PF, so their eswitch state is ready before registration-time defaults may call eswitch_mode_set().
[1] lore.kernel.org/r/[email protected]/ RFC v1: lore.kernel.org/r/[email protected]/ RFC v2: lore.kernel.org/r/[email protected]/
v1: lore.kernel.org/r/[email protected]/
v2: lore.kernel.org/all/[email protected]/
v3: lore.kernel.org/all/[email protected]/
v4: lore.kernel.org/all/[email protected]/
v5: lore.kernel.org/all/[email protected]/
Mark Bloch (4):
net/mlx5: Clear FW reset-in-progress bit before reload
devlink: Factor out eswitch mode setting
devlink: Parse eswitch mode boot defaults
devlink: Apply eswitch mode boot defaults
.../admin-guide/kernel-parameters.txt | 25 ++ .../networking/devlink/devlink-defaults.rst | 78 +++++ Documentation/networking/devlink/index.rst | 1 + .../ethernet/mellanox/mlx5/core/fw_reset.c | 28 +- net/devlink/Makefile | 2 +- net/devlink/core.c | 10 + net/devlink/default.c | 303 ++++++++++++++++++ net/devlink/dev.c | 33 +- net/devlink/devl_internal.h | 10 + 9 files changed, 471 insertions(+), 19 deletions(-) create mode 100644 Documentation/networking/devlink/devlink-defaults.rst create mode 100644 net/devlink/default.c
base-commit: f6f3b36c15ed44de1fbb44e645e4fae8c4a4453e
Signature · 1 line
2.43.0
Mark Bloch
maintainerPATCH v6 1/4 net/mlx5: Clear FW reset-in-progress bit before reload
mlx5 sets MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS when acknowledging a sync reset request. This bit blocks devlink reload and other devlink operations while the firmware reset is running, but it was kept set until after the driver reload finished.
Clear the reset-in-progress bit once the reset unload flow is done and PCI access is back, before reloading the device. For a reset initiated through devlink, clear it before completing the reload waiter. For a reset reported through an asynchronous firmware event, keep the unload flow outside devl_lock, then take devl_lock before clearing the bit and reloading through the devl-locked load helper.
--- .../ethernet/mellanox/mlx5/core/fw_reset.c | 28 +++++++++++-------- 1 file changed, 17 insertions(+), 11 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +17 -11
@@ -238,24 +238,30 @@ static void mlx5_fw_reset_complete_reload(struct mlx5_core_dev *dev) {struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset;struct devlink *devlink =priv_to_devlink (dev); +int err;/* if this is the driver that initiated the fw reset, devlink completed the reload */ if (test_bit (MLX5_FW_RESET_FLAGS_PENDING_COMP , &fw_reset->reset_flags)) { +clear_bit (MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS , + &fw_reset->reset_flags);complete (&fw_reset->done); - }else { -mlx5_sync_reset_unload_flow (dev, false); -if (mlx5_health_wait_pci_up (dev)) -mlx5_core_err (dev,"reset reload flow aborted, PCI reads still not working\n" ); -else -mlx5_load_one (dev, true); -devl_lock (devlink); -devlink_remote_reload_actions_performed (devlink,0 , -BIT (DEVLINK_RELOAD_ACTION_DRIVER_REINIT ) | -BIT (DEVLINK_RELOAD_ACTION_FW_ACTIVATE )); -devl_unlock (devlink); +return ; } +mlx5_sync_reset_unload_flow (dev, false); + err =mlx5_health_wait_pci_up (dev); + +devl_lock (devlink);clear_bit (MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS , &fw_reset->reset_flags); +if (err) +mlx5_core_err (dev,"reset reload flow aborted, PCI reads still not working\n" ); +else +mlx5_load_one_devl_locked (dev, true); + +devlink_remote_reload_actions_performed (devlink,0 , +BIT (DEVLINK_RELOAD_ACTION_DRIVER_REINIT ) | +BIT (DEVLINK_RELOAD_ACTION_FW_ACTIVATE )); +devl_unlock (devlink); }static void mlx5_stop_sync_reset_poll (struct mlx5_core_dev *dev)
Signature · 1 line
2.43.0
Mark Bloch
maintainerPATCH v6 2/4 devlink: Factor out eswitch mode setting
In reply to Mark BlochMove the common eswitch mode set checks into a small helper and use it from the netlink eswitch set command. This makes the same validation available to the devlink core path that applies eswitch mode defaults.
--- net/devlink/dev.c | 27 ++++++++++++++++++++------- net/devlink/devl_internal.h | 3 +++ 2 files changed, 23 insertions(+), 7 deletions(-)
net/devlink/dev.c +20 -7
@@ -702,6 +702,25 @@ int devlink_nl_eswitch_get_doit(struct sk_buff *skb, struct genl_info *info)return genlmsg_reply (msg, info); } +int devlink_eswitch_mode_set (struct devlink *devlink, +enum devlink_eswitch_mode mode, +struct netlink_ext_ack *extack) +{ +const struct devlink_ops *ops = devlink->ops; +int err; + +devl_assert_locked (devlink); + +if (!ops->eswitch_mode_set) +return -EOPNOTSUPP ; + + err =devlink_rates_check (devlink, devlink_rate_is_node, extack); +if (err) +return err; + +return ops->eswitch_mode_set (devlink, mode, extack); +} +int devlink_nl_eswitch_set_doit (struct sk_buff *skb,struct genl_info *info) {struct devlink *devlink =devlink_nl_ctx (info)->devlink;
@@ -712,14 +731,8 @@ int devlink_nl_eswitch_set_doit(struct sk_buff *skb, struct genl_info *info)u16 mode;if (info->attrs[DEVLINK_ATTR_ESWITCH_MODE ]) { -if (!ops->eswitch_mode_set) -return -EOPNOTSUPP ; - err =devlink_rates_check (devlink, devlink_rate_is_node, - info->extack); -if (err) -return err; mode =nla_get_u16 (info->attrs[DEVLINK_ATTR_ESWITCH_MODE ]); - err = ops->eswitch_mode_set (devlink, mode, info->extack); + err =devlink_eswitch_mode_set (devlink, mode, info->extack);if (err)return err; }
net/devlink/devl_internal.h +3 -0
@@ -348,6 +348,9 @@ bool devlink_rate_is_node(const struct devlink_rate *devlink_rate);int devlink_rates_check (struct devlink *devlink,bool (*rate_filter)(const struct devlink_rate *),struct netlink_ext_ack *extack); +int devlink_eswitch_mode_set (struct devlink *devlink, +enum devlink_eswitch_mode mode, +struct netlink_ext_ack *extack);/* Linecards */ unsigned int devlink_linecard_index (struct devlink_linecard *linecard);
Signature · 1 line
2.43.0
Re: [PATCH net-next V6 2/4] devlink: Factor out eswitch mode setting
Quoted 5 lines · Tue, Jul 14, 2026 at 08:17:28AM +0200, [email protected] wrote:
Move the common eswitch mode set checks into a small helper and use it from the netlink eswitch set command. This makes the same validation available to the devlink core path that applies eswitch mode defaults.
Signed-off-by Mark Bloch <[email protected]>
Mark Bloch
maintainerPATCH v6 3/4 devlink: Parse eswitch mode boot defaults
In reply to Mark BlochAdd devlink_eswitch_mode= kernel command line parsing for a default eswitch mode.
The supported syntax selects either all devlink handles or one explicit comma-separated handle list:
devlink_eswitch_mode=*=<mode>
devlink_eswitch_mode=<handle>[,<handle>...]=<mode>
where <mode> is one of legacy, switchdev or switchdev_inactive. All selected handles receive the same mode. Assigning different modes to different handle lists in the same parameter value is not supported.
Store the parsed selector and mode in devlink core so the default can be applied by a downstream patch.
Document the devlink_eswitch_mode= syntax and duplicate handle handling.
--- .../admin-guide/kernel-parameters.txt | 25 ++ .../networking/devlink/devlink-defaults.rst | 78 ++++++ Documentation/networking/devlink/index.rst | 1 + net/devlink/Makefile | 2 +- net/devlink/core.c | 7 + net/devlink/default.c | 237 ++++++++++++++++++ net/devlink/devl_internal.h | 2 + 7 files changed, 351 insertions(+), 1 deletion(-) create mode 100644 Documentation/networking/devlink/devlink-defaults.rst create mode 100644 net/devlink/default.c
Documentation/admin-guide/kernel-parameters.txt +25 -0
@@ -1249,6 +1249,31 @@ Kernel parameters dell_smm_hwmon.fan_max= [HW] Maximum configurable fan speed. + devlink_eswitch_mode= + [NET] + Format: + <selector>=<mode> + + <selector>: + * | <handle>[,<handle>...] + + <handle>: + <bus-name>/<dev-name> + + Configure default devlink eswitch mode for matching + devlink instances during device initialization. + + <mode>: + legacy | switchdev | switchdev_inactive + + Examples: + devlink_eswitch_mode=*=switchdev + devlink_eswitch_mode=pci/0000:08:00.0=switchdev + devlink_eswitch_mode=pci/0000:08:00.0,pci/0000:09:00.1=switchdev_inactive + + See Documentation/networking/devlink/devlink-defaults.rst + for the full syntax. + dfltcc= [HW,S390] Format: { on | off | def_only | inf_only | always } on: s390 zlib hardware support for compression on
Documentation/networking/devlink/devlink-defaults.rst new file +78 -0
78 lines not on this page · raw message
Documentation/networking/devlink/index.rst +1 -0
@@ -56,6 +56,7 @@ general. :maxdepth: 1 devlink-dpipe + devlink-defaults devlink-eswitch-attr devlink-flash devlink-health
net/devlink/Makefile +1 -1
@@ -1,4 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 -obj-y := core.o netlink.o netlink_gen.o dev.o port.o sb.o dpipe.o \ +obj-y := core.o netlink.o netlink_gen.o dev.o default.o port.o sb.o dpipe.o \ resource.o param.o region.o health.o trap.o rate.o linecard.o sh_dev.o
net/devlink/core.c +7 -0
@@ -598,6 +598,10 @@ static int __init devlink_init(void) {int err; + err =devlink_default_esw_mode_init (); +if (err) + goto out; + err =register_pernet_subsys (&devlink_pernet_ops);if (err) goto out;
@@ -613,7 +617,10 @@ static int __init devlink_init(void) out_unreg_pernet_subsys:unregister_pernet_subsys (&devlink_pernet_ops); out: +if (err) +devlink_default_esw_mode_cleanup ();WARN_ON (err); +return err; }
net/devlink/default.c new file +237 -0
237 lines not on this page · raw message
net/devlink/devl_internal.h +2 -0
@@ -71,6 +71,8 @@ extern struct genl_family devlink_nl_family;struct devlink *__devlink_alloc (const struct devlink_ops *ops,size_t priv_size,struct net *net,struct device *dev,const struct device_driver *dev_driver); +int devlink_default_esw_mode_init (void ); +void devlink_default_esw_mode_cleanup (void ); #define devl_warn(devlink, format, args...) \ do { \
Signature · 1 line
2.43.0
Re: [PATCH net-next V6 3/4] devlink: Parse eswitch mode boot defaults
Quoted 14 lines · Tue, Jul 14, 2026 at 08:17:29AM +0200, [email protected] wrote:
Add devlink_eswitch_mode= kernel command line parsing for a default eswitch mode.
The supported syntax selects either all devlink handles or one explicit comma-separated handle list:
devlink_eswitch_mode=*=<mode>
devlink_eswitch_mode=<handle>[,<handle>...]=<mode>
where <mode> is one of legacy, switchdev or
switchdev_inactive. All selected handles receive the same mode. Assigning different modes to different handle lists in the same parameter value is not supported.
Quoted 6 lines
Store the parsed selector and mode in devlink core so the default can be applied by a downstream patch.
Document the devlink_eswitch_mode= syntax and duplicate handle handling.
Signed-off-by Mark Bloch <[email protected]>
Mark Bloch
maintainerPATCH v6 4/4 devlink: Apply eswitch mode boot defaults
In reply to Mark BlochApply parsed devlink_eswitch_mode= defaults after devlink registration and after successful reload.
Mark the default mode as pending when a devlink instance is allocated. Before devl_unlock() releases the instance lock, apply a pending default when the instance is registered.
Clear the pending state before calling into the driver so the boot default remains a one-shot operation even if the mode change fails.
For successful reloads that performed DRIVER_REINIT, devlink_reload() already holds the devlink instance lock and the driver has completed reload_up(). Clear the pending state and apply the default directly from the reload path.
Treat an explicit user eswitch mode request as consuming the pending default mode.
--- net/devlink/core.c | 3 ++ net/devlink/default.c | 70 +++++++++++++++++++++++++++++++++++-- net/devlink/dev.c | 6 ++++ net/devlink/devl_internal.h | 5 +++ 4 files changed, 82 insertions(+), 2 deletions(-)
net/devlink/core.c +3 -0
@@ -317,6 +317,7 @@ EXPORT_SYMBOL_GPL(devl_trylock);void devl_unlock (struct devlink *devlink) { +devlink_default_esw_mode_apply_pending (devlink);mutex_unlock (&devlink->lock); }EXPORT_SYMBOL_GPL (devl_unlock);
@@ -429,6 +430,7 @@ void devl_unregister(struct devlink *devlink)ASSERT_DEVLINK_REGISTERED (devlink);devl_assert_locked (devlink); +devlink_default_esw_mode_apply_pending_clear (devlink);devlink_notify_unregister (devlink);xa_clear_mark (&devlinks, devlink->index,DEVLINK_REGISTERED );devlink_rel_put (devlink);
@@ -490,6 +492,7 @@ struct devlink *__devlink_alloc(const struct devlink_ops *ops, size_t priv_size,INIT_LIST_HEAD (&devlink->trap_group_list);INIT_LIST_HEAD (&devlink->trap_policer_list);INIT_RCU_WORK (&devlink->rwork, devlink_release); +devlink_default_esw_mode_instance_init (devlink);lockdep_register_key (&devlink->lock_key);mutex_init (&devlink->lock);lockdep_set_class (&devlink->lock, &devlink->lock_key);
net/devlink/default.c +68 -2
94 lines not on this page · raw message
net/devlink/dev.c +6 -0
@@ -478,6 +478,11 @@ int devlink_reload(struct devlink *devlink, struct net *dest_net,return err;WARN_ON (!(*actions_performed &BIT (action))); +if (*actions_performed &BIT (DEVLINK_RELOAD_ACTION_DRIVER_REINIT )) { +devlink_default_esw_mode_apply_pending_clear (devlink); +devlink_default_esw_mode_apply_locked (devlink); + } +/* Catch driver on updating the remote action within devlink reload */ WARN_ON (memcmp (remote_reload_stats, devlink->stats.remote_reload_stats,sizeof (remote_reload_stats)));
@@ -731,6 +736,7 @@ int devlink_nl_eswitch_set_doit(struct sk_buff *skb, struct genl_info *info)u16 mode;if (info->attrs[DEVLINK_ATTR_ESWITCH_MODE ]) { +devlink_default_esw_mode_apply_pending_clear (devlink); mode =nla_get_u16 (info->attrs[DEVLINK_ATTR_ESWITCH_MODE ]); err =devlink_eswitch_mode_set (devlink, mode, info->extack);if (err)
net/devlink/devl_internal.h +5 -0
@@ -58,6 +58,7 @@ struct devlink {struct mutex lock;struct lock_class_key lock_key; u8 reload_failed:1; + u8 default_esw_mode_apply_pending:1;refcount_t refcount;struct rcu_work rwork;struct devlink_rel *rel;
@@ -73,6 +74,10 @@ struct devlink *__devlink_alloc(const struct devlink_ops *ops, size_t priv_size, const struct device_driver *dev_driver);int devlink_default_esw_mode_init (void );void devlink_default_esw_mode_cleanup (void ); +void devlink_default_esw_mode_instance_init (struct devlink *devlink); +void devlink_default_esw_mode_apply_locked (struct devlink *devlink); +void devlink_default_esw_mode_apply_pending (struct devlink *devlink); +void devlink_default_esw_mode_apply_pending_clear (struct devlink *devlink); #define devl_warn(devlink, format, args...) \ do { \
Signature · 1 line
2.43.0
Re: [PATCH net-next V6 4/4] devlink: Apply eswitch mode boot defaults
Quoted 10 lines · Tue, Jul 14, 2026 at 08:17:30AM +0200, [email protected] wrote:
Apply parsed devlink_eswitch_mode= defaults after devlink registration and after successful reload.
Mark the default mode as pending when a devlink instance is allocated. Before
devl_unlock()releases the instance lock, apply a pending default when the instance is registered.Clear the pending state before calling into the driver so the boot default remains a one-shot operation even if the mode change fails.
Quoted 8 lines
For successful reloads that performed
DRIVER_REINIT,devlink_reload()already holds the devlink instance lock and the driver has completedreload_up(). Clear the pending state and apply the default directly from the reload path.Treat an explicit user eswitch mode request as consuming the pending default mode.
Looks fine to me now. One nit below:
[..]
Quoted 1 line
+void
devlink_default_esw_mode_instance_init(struct devlink *devlink)
Forgotten "instance" here. With this fixed, feel free to add my
[..]