devlink: Add boot-time eswitch mode defaults
Subsystems (6)
Mark Bloch
maintainerv5 cover devlink: Add boot-time eswitch mode defaults
This series adds a devlink_eswitch_mode= kernel command line parameter for setting a default devlink eswitch mode during boot.
Following the discussion with Jakub[1] and the feedback on the RFC postings, this version keeps the scope limited to a boot-time devlink eswitch mode default only.
The option selects either all devlink handles or an explicit comma-separated handle list:
devlink_eswitch_mode=*=switchdev devlink_eswitch_mode=pci/0000:08:00.0,pci/0000:09:00.1=switchdev_inactive
The supported modes are legacy, switchdev and switchdev_inactive. The selected mode is applied through the existing eswitch_mode_set() devlink operation, the same operation used by the devlink eswitch mode command.
Registration may happen before a driver is ready to change eswitch mode, so devlink core marks the default as pending and queues async work from devl_unlock() once the instance is registered. The worker takes the devlink instance lock before calling into the driver.
After a successful reload that performed DRIVER_REINIT, devlink core already holds the devlink instance lock and the driver completed reload_up(), so the default is applied directly from the reload path.
Drivers that know exactly when the device is ready can call devl_apply_default_esw_mode() directly. mlx5 uses this after initial probe, when the device is initialized and the devlink lock is already held.
Patch 1 clears the mlx5 FW reset-in-progress bit before reload.
Patch 2 factors the common eswitch mode set validation into a helper.
Patch 3 adds the devlink_eswitch_mode= parser and documentation.
Patch 4 applies parsed defaults from devlink core.
Patch 5 adds devl_apply_default_esw_mode() for drivers.
Patch 6 wires mlx5 to apply the default after initial probe.
Changelog:
v4 -> v5:
- Moved the default eswitch mode code into a separate file, per Jiri's comment.
- Dropped the delayed workqueue and switched to regular work triggered via devl_unlock(), per Jiri's comment.
- Renamed some functions to better align with devlink code.
v3 -> v4:
- Rework registration time apply to use per devlink delayed work instead of calling eswitch_mode_set() directly from devl_register().
- Apply the default directly after successful DRIVER_REINIT devlink reload, where the devlink lock is already held and reload_up() has completed.
- Add devl_apply_default_esw_mode() for drivers that know their exact ready point.
- Drop the driver registration-ordering preparation patches that are no longer needed with the async registration apply path.
v2 -> v3:
- Change the devlink_eswitch_mode= API syntax to use <selector>=<mode> instead of [<selector>]:<mode>, following a comment from Randy Dunlap.
v1 -> v2:
- Move default eswitch mode application into devlink core. The default is now applied during devlink registration and after a successful devlink reload that performed DRIVER_REINIT.
- Remove the exported devl_apply_default_esw_mode() driver API and the mlx5 driver-side call to it.
- Skip devlink health recovery notifications while the devlink instance is not registered, so drivers can move registration later without early health work hitting registration assertions.
- Move mlx5 devlink registration after device initialization, including the lightweight init path, so the core can apply the default through the normal registration flow.
- Move the matching netdevsim and mlx5 unregister paths before object teardown, so unregister notifications come from devl_unregister() and the later object teardown paths run while the devlink instance is no longer registered.
- Add registration-ordering preparation patches for netdevsim and octeontx2 AF/PF, so their eswitch state is ready before registration-time defaults may call eswitch_mode_set().
[1] lore.kernel.org/r/[email protected]/ RFC v1: lore.kernel.org/r/[email protected]/ RFC v2: lore.kernel.org/r/[email protected]/
v1: lore.kernel.org/r/[email protected]/
v2: lore.kernel.org/all/[email protected]/
v3: lore.kernel.org/all/[email protected]/
v4: lore.kernel.org/all/[email protected]/
Mark Bloch (6):
net/mlx5: Clear FW reset-in-progress bit before reload
devlink: Factor out eswitch mode setting
devlink: Parse eswitch mode boot defaults
devlink: Apply eswitch mode boot defaults
devlink: Add API to apply eswitch mode boot default net/mlx5: Apply devlink eswitch mode boot default on probe
.../admin-guide/kernel-parameters.txt | 25 ++ .../networking/devlink/devlink-defaults.rst | 78 ++++ Documentation/networking/devlink/index.rst | 1 + .../ethernet/mellanox/mlx5/core/fw_reset.c | 28 +- .../net/ethernet/mellanox/mlx5/core/main.c | 13 + include/net/devlink.h | 1 + net/devlink/Makefile | 2 +- net/devlink/core.c | 13 + net/devlink/default.c | 364 ++++++++++++++++++ net/devlink/dev.c | 33 +- net/devlink/devl_internal.h | 12 + 11 files changed, 551 insertions(+), 19 deletions(-) create mode 100644 Documentation/networking/devlink/devlink-defaults.rst create mode 100644 net/devlink/default.c
base-commit: 4a13f31a92f35161b499bf29638336885259da78
Signature · 1 line
2.43.0
Mark Bloch
maintainerPATCH v5 1/6 net/mlx5: Clear FW reset-in-progress bit before reload
mlx5 sets MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS when acknowledging a sync reset request. This bit blocks devlink reload and other devlink operations while the firmware reset is running, but it was kept set until after the driver reload finished.
Clear the reset-in-progress bit once the reset unload flow is done and PCI access is back, before reloading the device. For a reset initiated through devlink, clear it before completing the reload waiter. For a reset reported through an asynchronous firmware event, keep the unload flow outside devl_lock, then take devl_lock before clearing the bit and reloading through the devl-locked load helper.
--- .../ethernet/mellanox/mlx5/core/fw_reset.c | 28 +++++++++++-------- 1 file changed, 17 insertions(+), 11 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +17 -11
@@ -238,24 +238,30 @@ static void mlx5_fw_reset_complete_reload(struct mlx5_core_dev *dev) {struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset;struct devlink *devlink =priv_to_devlink (dev); +int err;/* if this is the driver that initiated the fw reset, devlink completed the reload */ if (test_bit (MLX5_FW_RESET_FLAGS_PENDING_COMP , &fw_reset->reset_flags)) { +clear_bit (MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS , + &fw_reset->reset_flags);complete (&fw_reset->done); - }else { -mlx5_sync_reset_unload_flow (dev, false); -if (mlx5_health_wait_pci_up (dev)) -mlx5_core_err (dev,"reset reload flow aborted, PCI reads still not working\n" ); -else -mlx5_load_one (dev, true); -devl_lock (devlink); -devlink_remote_reload_actions_performed (devlink,0 , -BIT (DEVLINK_RELOAD_ACTION_DRIVER_REINIT ) | -BIT (DEVLINK_RELOAD_ACTION_FW_ACTIVATE )); -devl_unlock (devlink); +return ; } +mlx5_sync_reset_unload_flow (dev, false); + err =mlx5_health_wait_pci_up (dev); + +devl_lock (devlink);clear_bit (MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS , &fw_reset->reset_flags); +if (err) +mlx5_core_err (dev,"reset reload flow aborted, PCI reads still not working\n" ); +else +mlx5_load_one_devl_locked (dev, true); + +devlink_remote_reload_actions_performed (devlink,0 , +BIT (DEVLINK_RELOAD_ACTION_DRIVER_REINIT ) | +BIT (DEVLINK_RELOAD_ACTION_FW_ACTIVATE )); +devl_unlock (devlink); }static void mlx5_stop_sync_reset_poll (struct mlx5_core_dev *dev)
Signature · 1 line
2.43.0
Mark Bloch
maintainerPATCH v5 2/6 devlink: Factor out eswitch mode setting
In reply to Mark BlochMove the common eswitch mode set checks into a small helper and use it from the netlink eswitch set command. Making the same validation available to the devlink core path that applies eswitch mode defaults.
--- net/devlink/dev.c | 27 ++++++++++++++++++++------- net/devlink/devl_internal.h | 3 +++ 2 files changed, 23 insertions(+), 7 deletions(-)
net/devlink/dev.c +20 -7
@@ -702,6 +702,25 @@ int devlink_nl_eswitch_get_doit(struct sk_buff *skb, struct genl_info *info)return genlmsg_reply (msg, info); } +int devlink_eswitch_mode_set (struct devlink *devlink, +enum devlink_eswitch_mode mode, +struct netlink_ext_ack *extack) +{ +const struct devlink_ops *ops = devlink->ops; +int err; + +devl_assert_locked (devlink); + +if (!ops->eswitch_mode_set) +return -EOPNOTSUPP ; + + err =devlink_rates_check (devlink, devlink_rate_is_node, extack); +if (err) +return err; + +return ops->eswitch_mode_set (devlink, mode, extack); +} +int devlink_nl_eswitch_set_doit (struct sk_buff *skb,struct genl_info *info) {struct devlink *devlink =devlink_nl_ctx (info)->devlink;
@@ -712,14 +731,8 @@ int devlink_nl_eswitch_set_doit(struct sk_buff *skb, struct genl_info *info)u16 mode;if (info->attrs[DEVLINK_ATTR_ESWITCH_MODE ]) { -if (!ops->eswitch_mode_set) -return -EOPNOTSUPP ; - err =devlink_rates_check (devlink, devlink_rate_is_node, - info->extack); -if (err) -return err; mode =nla_get_u16 (info->attrs[DEVLINK_ATTR_ESWITCH_MODE ]); - err = ops->eswitch_mode_set (devlink, mode, info->extack); + err =devlink_eswitch_mode_set (devlink, mode, info->extack);if (err)return err; }
net/devlink/devl_internal.h +3 -0
@@ -348,6 +348,9 @@ bool devlink_rate_is_node(const struct devlink_rate *devlink_rate);int devlink_rates_check (struct devlink *devlink,bool (*rate_filter)(const struct devlink_rate *),struct netlink_ext_ack *extack); +int devlink_eswitch_mode_set (struct devlink *devlink, +enum devlink_eswitch_mode mode, +struct netlink_ext_ack *extack);/* Linecards */ unsigned int devlink_linecard_index (struct devlink_linecard *linecard);
Signature · 1 line
2.43.0
Mark Bloch
maintainerPATCH v5 3/6 devlink: Parse eswitch mode boot defaults
In reply to Mark BlochAdd devlink_eswitch_mode= kernel command line parsing for a default eswitch mode.
The supported syntax selects either all devlink handles or one explicit comma-separated handle list:
devlink_eswitch_mode=*=<mode>
devlink_eswitch_mode=<handle>[,<handle>...]=<mode>
where <mode> is one of legacy, switchdev or switchdev_inactive. All selected handles receive the same mode. Assigning different modes to different handle lists in the same parameter value is not supported.
Store the parsed selector and mode in devlink core so the default can be applied by a downstream patch.
Document the devlink_eswitch_mode= syntax and duplicate handle handling.
--- .../admin-guide/kernel-parameters.txt | 25 ++ .../networking/devlink/devlink-defaults.rst | 78 ++++++ Documentation/networking/devlink/index.rst | 1 + net/devlink/Makefile | 2 +- net/devlink/core.c | 7 + net/devlink/default.c | 237 ++++++++++++++++++ net/devlink/devl_internal.h | 2 + 7 files changed, 351 insertions(+), 1 deletion(-) create mode 100644 Documentation/networking/devlink/devlink-defaults.rst create mode 100644 net/devlink/default.c
Documentation/admin-guide/kernel-parameters.txt +25 -0
@@ -1249,6 +1249,31 @@ Kernel parameters dell_smm_hwmon.fan_max= [HW] Maximum configurable fan speed. + devlink_eswitch_mode= + [NET] + Format: + <selector>=<mode> + + <selector>: + * | <handle>[,<handle>...] + + <handle>: + <bus-name>/<dev-name> + + Configure default devlink eswitch mode for matching + devlink instances during device initialization. + + <mode>: + legacy | switchdev | switchdev_inactive + + Examples: + devlink_eswitch_mode=*=switchdev + devlink_eswitch_mode=pci/0000:08:00.0=switchdev + devlink_eswitch_mode=pci/0000:08:00.0,pci/0000:09:00.1=switchdev_inactive + + See Documentation/networking/devlink/devlink-defaults.rst + for the full syntax. + dfltcc= [HW,S390] Format: { on | off | def_only | inf_only | always } on: s390 zlib hardware support for compression on
Documentation/networking/devlink/devlink-defaults.rst new file +78 -0
78 lines not on this page · raw message
Documentation/networking/devlink/index.rst +1 -0
@@ -56,6 +56,7 @@ general. :maxdepth: 1 devlink-dpipe + devlink-defaults devlink-eswitch-attr devlink-flash devlink-health
net/devlink/Makefile +1 -1
@@ -1,4 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 -obj-y := core.o netlink.o netlink_gen.o dev.o port.o sb.o dpipe.o \ +obj-y := core.o netlink.o netlink_gen.o dev.o default.o port.o sb.o dpipe.o \ resource.o param.o region.o health.o trap.o rate.o linecard.o sh_dev.o
net/devlink/core.c +7 -0
@@ -598,6 +598,10 @@ static int __init devlink_init(void) {int err; + err =devlink_default_esw_mode_init (); +if (err) + goto out; + err =register_pernet_subsys (&devlink_pernet_ops);if (err) goto out;
@@ -613,7 +617,10 @@ static int __init devlink_init(void) out_unreg_pernet_subsys:unregister_pernet_subsys (&devlink_pernet_ops); out: +if (err) +devlink_default_esw_mode_cleanup ();WARN_ON (err); +return err; }
net/devlink/default.c new file +237 -0
237 lines not on this page · raw message
net/devlink/devl_internal.h +2 -0
@@ -71,6 +71,8 @@ extern struct genl_family devlink_nl_family;struct devlink *__devlink_alloc (const struct devlink_ops *ops,size_t priv_size,struct net *net,struct device *dev,const struct device_driver *dev_driver); +int devlink_default_esw_mode_init (void ); +void devlink_default_esw_mode_cleanup (void ); #define devl_warn(devlink, format, args...) \ do { \
Signature · 1 line
2.43.0
Mark Bloch
maintainerPATCH v5 4/6 devlink: Apply eswitch mode boot defaults
In reply to Mark BlochApply parsed devlink_eswitch_mode= defaults after devlink registration and after successful reload.
devl_register() may still be called before the device is ready for an eswitch mode change. Keep the registration path passive and let the regular devl_unlock() path queue the async apply work once the instance is registered and the default is still pending.
The queueing path runs while the devlink instance lock is held, so the queued work gets its devlink reference before the caller drops the lock. The worker then takes the devlink instance lock normally and applies the default only if the instance is still registered and the default is still pending.
For successful reloads that performed DRIVER_REINIT, devlink_reload() already holds the devlink instance lock and the driver has completed reload_up(). Clear pending work and apply the default directly from the reload path instead of queueing work.
Preserve the user configured mode when it is set before devlink applies the default.
--- net/devlink/core.c | 6 ++ net/devlink/default.c | 112 +++++++++++++++++++++++++++++++++++- net/devlink/dev.c | 6 ++ net/devlink/devl_internal.h | 7 +++ 4 files changed, 129 insertions(+), 2 deletions(-)
net/devlink/core.c +6 -0
@@ -317,6 +317,7 @@ EXPORT_SYMBOL_GPL(devl_trylock);void devl_unlock (struct devlink *devlink) { +devlink_default_esw_mode_queue_apply_work (devlink);mutex_unlock (&devlink->lock); }EXPORT_SYMBOL_GPL (devl_unlock);
@@ -429,6 +430,7 @@ void devl_unregister(struct devlink *devlink)ASSERT_DEVLINK_REGISTERED (devlink);devl_assert_locked (devlink); +devlink_default_esw_mode_apply_pending_clear (devlink);devlink_notify_unregister (devlink);xa_clear_mark (&devlinks, devlink->index,DEVLINK_REGISTERED );devlink_rel_put (devlink);
@@ -490,6 +492,7 @@ struct devlink *__devlink_alloc(const struct devlink_ops *ops, size_t priv_size,INIT_LIST_HEAD (&devlink->trap_group_list);INIT_LIST_HEAD (&devlink->trap_policer_list);INIT_RCU_WORK (&devlink->rwork, devlink_release); +devlink_default_esw_mode_instance_init (devlink);lockdep_register_key (&devlink->lock_key);mutex_init (&devlink->lock);lockdep_set_class (&devlink->lock, &devlink->lock_key);
@@ -537,6 +540,9 @@ void devlink_free(struct devlink *devlink)devl_lock (devlink);WARN_ON (devlink_rates_check (devlink,NULL ,NULL ));devl_unlock (devlink); + +devlink_default_esw_mode_instance_cleanup (devlink); +devlink_rel_put (devlink);WARN_ON (!list_empty (&devlink->trap_policer_list));
net/devlink/default.c +110 -2
148 lines not on this page · raw message
net/devlink/dev.c +6 -0
@@ -478,6 +478,11 @@ int devlink_reload(struct devlink *devlink, struct net *dest_net,return err;WARN_ON (!(*actions_performed &BIT (action))); +if (*actions_performed &BIT (DEVLINK_RELOAD_ACTION_DRIVER_REINIT )) { +devlink_default_esw_mode_apply_pending_clear (devlink); +devlink_default_esw_mode_apply_locked (devlink); + } +/* Catch driver on updating the remote action within devlink reload */ WARN_ON (memcmp (remote_reload_stats, devlink->stats.remote_reload_stats,sizeof (remote_reload_stats)));
@@ -731,6 +736,7 @@ int devlink_nl_eswitch_set_doit(struct sk_buff *skb, struct genl_info *info)u16 mode;if (info->attrs[DEVLINK_ATTR_ESWITCH_MODE ]) { +devlink_default_esw_mode_apply_pending_clear (devlink); mode =nla_get_u16 (info->attrs[DEVLINK_ATTR_ESWITCH_MODE ]); err =devlink_eswitch_mode_set (devlink, mode, info->extack);if (err)
net/devlink/devl_internal.h +7 -0
@@ -58,8 +58,10 @@ struct devlink {struct mutex lock;struct lock_class_key lock_key; u8 reload_failed:1; + u8 default_esw_mode_apply_pending:1;refcount_t refcount;struct rcu_work rwork; +struct work_struct default_esw_mode_apply_work;struct devlink_rel *rel;struct xarray nested_rels;char priv[]__aligned(NETDEV_ALIGN) ;
@@ -73,6 +75,11 @@ struct devlink *__devlink_alloc(const struct devlink_ops *ops, size_t priv_size, const struct device_driver *dev_driver);int devlink_default_esw_mode_init (void );void devlink_default_esw_mode_cleanup (void ); +void devlink_default_esw_mode_instance_init (struct devlink *devlink); +void devlink_default_esw_mode_apply_locked (struct devlink *devlink); +void devlink_default_esw_mode_queue_apply_work (struct devlink *devlink); +void devlink_default_esw_mode_apply_pending_clear (struct devlink *devlink); +void devlink_default_esw_mode_instance_cleanup (struct devlink *devlink); #define devl_warn(devlink, format, args...) \ do { \
Signature · 1 line
2.43.0
Jiri Pirko
maintainerRe: [PATCH net-next V5 4/6] devlink: Apply eswitch mode boot defaults
Quoted 8 lines · Tue, Jul 07, 2026 at 07:45:25PM +0200, [email protected] wrote:
Apply parsed devlink_eswitch_mode= defaults after devlink registration and after successful reload.
devl_register()may still be called before the device is ready for an eswitch mode change. Keep the registration path passive and let the regulardevl_unlock()path queue the async apply work once the instance is registered and the default is still pending.
Quoted 5 lines
The queueing path runs while the devlink instance lock is held, so the queued work gets its devlink reference before the caller drops the lock. The worker then takes the devlink instance lock normally and applies the default only if the instance is still registered and the default is still pending.
This is very code-descriptive. What's the benefit of that?
Quoted 9 lines
For successful reloads that performed
DRIVER_REINIT,devlink_reload()already holds the devlink instance lock and the driver has completedreload_up(). Clear pending work and apply the default directly from the reload path instead of queueing work.Preserve the user configured mode when it is set before devlink applies the default.
[..]
Quoted 15 lines
net/devlink/default.c +15
-0+void devlink_default_esw_mode_apply_locked (struct devlink *devlink) +{ +const struct devlink_ops *ops = devlink->ops; +int err; + +devl_assert_locked (devlink); + +if (!devlink_default_esw_mode_match (devlink)) +return ; + +if (!ops->eswitch_mode_set) { +if (!devlink_default_esw_mode_match_all) +devl_warn (devlink, +"devlink_eswitch_mode= selected this device but eswitch mode setting is not supported\n" ); +return ;
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 233
Quoted 10 lines
net/devlink/default.c +10
-0+ } + + err =devlink_eswitch_mode_set (devlink, devlink_default_esw_mode,NULL ); +if (err) +devl_warn (devlink, +"Couldn't apply default eswitch mode, err %d\n" , + err); +} + +void devlink_default_esw_mode_queue_apply_work (struct devlink *devlink)
eswitch/esw - we call it "eswitch" consistently everywhere. Why "esw" here?
Quoted 12 lines
net/devlink/default.c +12
-0+{ +devl_assert_locked (devlink); + +if (!devlink_default_esw_mode_enabled || !devlink_default_esw_mode_wq) +return ; +if (!devlink->default_esw_mode_apply_pending || + !__devl_is_registered (devlink)) +return ; +if (!devlink_try_get (devlink)) +return ; +if (!queue_work (devlink_default_esw_mode_wq, + &devlink->default_esw_mode_apply_work))
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 255
Quoted 10 lines
net/devlink/default.c +10
-0+devlink_put (devlink); +} + +static void devlink_default_esw_mode_apply_work (struct work_struct *work) +{ +struct devlink *devlink; + + devlink = container_of(work, struct devlink, + default_esw_mode_apply_work); +
What happens if userspace eswitch mode set happens now? Any userspace attempt should cancel the default apply. I don't see such mechanism in your patches, did I miss it?
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 268
Quoted 13 lines
net/devlink/default.c +13
-0+devl_lock (devlink); + +if (devl_is_registered (devlink) && + devlink->default_esw_mode_apply_pending) { +devlink_default_esw_mode_apply_locked (devlink); + devlink->default_esw_mode_apply_pending = false; + } + +devl_unlock (devlink); +devlink_put (devlink); +} + +void devlink_default_esw_mode_instance_init (struct devlink *devlink)
Why "_instance_"? Care to drop?
Quoted 4 lines
net/devlink/default.c +4
-0+{ +INIT_WORK (&devlink->default_esw_mode_apply_work, + devlink_default_esw_mode_apply_work); + devlink->default_esw_mode_apply_pending = true;
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 282
Quoted 10 lines
net/devlink/default.c +10
-0+} + +void devlink_default_esw_mode_apply_pending_clear (struct devlink *devlink) +{ +devl_assert_locked (devlink); + + devlink->default_esw_mode_apply_pending = false; +} + +void devlink_default_esw_mode_instance_cleanup (struct devlink *devlink)
Why "_instance_"? Care to drop?
Quoted 9 lines
net/devlink/default.c +5
-0+{ +if (cancel_work_sync (&devlink->default_esw_mode_apply_work)) +devlink_put (devlink); +} +static int
__initdevlink_default_esw_mode_setup(char *str){ devlink_default_esw_mode_param = str;
net/devlink/default.c int __init devlink_default_esw_mode_init(void) · line 334
Quoted 10 lines
net/devlink/default.c +7
-0@@ -228,10 +325,21 @@ int __init devlink_default_esw_mode_init(void)return err; } + devlink_default_esw_mode_wq =alloc_workqueue ("devlink_default_esw_mode" , +WQ_UNBOUND |WQ_MEM_RECLAIM , +0 ); +if (!devlink_default_esw_mode_wq) { + devlink_default_esw_mode_param =NULL ; +devlink_default_esw_mode_nodes_clear (); +pr_warn ("devlink: devlink_eswitch_mode parameter ignored, failed to allocate workqueue\n" );
Why you don't "return" here? I think that we don't need to allow the case wq is not allocated.
Quoted 11 lines
net/devlink/default.c +4
-0+ } +return 0 ; }void __init devlink_default_esw_mode_cleanup (void ) { +if (devlink_default_esw_mode_wq) +destroy_workqueue (devlink_default_esw_mode_wq);devlink_default_esw_mode_nodes_clear (); }
[..]
Mark Bloch
maintainerRe: [PATCH net-next V5 4/6] devlink: Apply eswitch mode boot defaults
Quoted 9 lines · Jiri Pirko
Quoted 8 lines · Tue, Jul 07, 2026 at 07:45:25PM +0200, [email protected] wrote:
Apply parsed devlink_eswitch_mode= defaults after devlink registration and after successful reload.
devl_register()may still be called before the device is ready for an eswitch mode change. Keep the registration path passive and let the regulardevl_unlock()path queue the async apply work once the instance is registered and the default is still pending.
Quoted 7 lines
Quoted 5 lines
The queueing path runs while the devlink instance lock is held, so the queued work gets its devlink reference before the caller drops the lock. The worker then takes the devlink instance lock normally and applies the default only if the instance is still registered and the default is still pending.
This is very code-descriptive. What's the benefit of that?
The point is that there is still a window before the queued work runs where the user can explicitly set the eswitch mode. If they do, the default will no longer be pending, so the worker will skip applying it.
I'll reword.
Quoted 35 lines
Quoted 11 lines
For successful reloads that performed
DRIVER_REINIT,devlink_reload()already holds the devlink instance lock and the driver has completedreload_up(). Clear pending work and apply the default directly from the reload path instead of queueing work.Preserve the user configured mode when it is set before devlink applies the default.
[..]
Quoted 20 lines
net/devlink/default.c +20
-0+void devlink_default_esw_mode_apply_locked (struct devlink *devlink) +{ +const struct devlink_ops *ops = devlink->ops; +int err; + +devl_assert_locked (devlink); + +if (!devlink_default_esw_mode_match (devlink)) +return ; + +if (!ops->eswitch_mode_set) { +if (!devlink_default_esw_mode_match_all) +devl_warn (devlink, +"devlink_eswitch_mode= selected this device but eswitch mode setting is not supported\n" ); +return ; + } + + err =devlink_eswitch_mode_set (devlink, devlink_default_esw_mode,NULL ); +if (err) + devl_warn(devlink,
Quoted 8 lines
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 233
Quoted 5 lines
net/devlink/default.c +5
-0+"Couldn't apply default eswitch mode, err %d\n" , + err); +} + +void devlink_default_esw_mode_queue_apply_work (struct devlink *devlink)eswitch/esw - we call it "eswitch" consistently everywhere. Why "esw" here?
Ack
Quoted 21 lines
Quoted 21 lines
net/devlink/default.c +18
-0+{ +devl_assert_locked (devlink); + +if (!devlink_default_esw_mode_enabled || !devlink_default_esw_mode_wq) +return ; +if (!devlink->default_esw_mode_apply_pending || + !__devl_is_registered (devlink)) +return ; +if (!devlink_try_get (devlink)) +return ; +if (!queue_work (devlink_default_esw_mode_wq, + &devlink->default_esw_mode_apply_work)) +devlink_put (devlink); +} + +static void devlink_default_esw_mode_apply_work (struct work_struct *work) +{ +struct devlink *devlink;
Quoted 8 lines
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 255
Quoted 4 lines
net/devlink/default.c +4
-0+ + devlink = container_of(work, struct devlink, + default_esw_mode_apply_work); +What happens if userspace eswitch mode set happens now? Any userspace attempt should cancel the default apply. I don't see such mechanism in your patches, did I miss it?
devlink_nl_eswitch_set_doit() calls devlink_default_esw_mode_apply_pending_clear(), which clears the pending bit.
So if a user sets the eswitch mode before the queued default work applies it, the worker will see that the default is no longer pending and will do nothing
Quoted 18 lines
Quoted 6 lines
net/devlink/default.c +3
-0+devl_lock (devlink); + + if (devl_is_registered(devlink) &&net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 268
Quoted 10 lines
net/devlink/default.c +10
-0+ devlink->default_esw_mode_apply_pending) { +devlink_default_esw_mode_apply_locked (devlink); + devlink->default_esw_mode_apply_pending = false; + } + +devl_unlock (devlink); +devlink_put (devlink); +} + +void devlink_default_esw_mode_instance_init (struct devlink *devlink)Why "_instance_"? Care to drop?
Ack
Quoted 18 lines
Quoted 6 lines
net/devlink/default.c +4
-0+{ +INIT_WORK (&devlink->default_esw_mode_apply_work, + devlink_default_esw_mode_apply_work); + devlink->default_esw_mode_apply_pending = true;net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 282
Quoted 10 lines
net/devlink/default.c +10
-0+} + +void devlink_default_esw_mode_apply_pending_clear (struct devlink *devlink) +{ +devl_assert_locked (devlink); + + devlink->default_esw_mode_apply_pending = false; +} + +void devlink_default_esw_mode_instance_cleanup (struct devlink *devlink)Why "_instance_"? Care to drop?
Ack
Quoted 14 lines
Quoted 14 lines
net/devlink/default.c +5
-0+{ +if (cancel_work_sync (&devlink->default_esw_mode_apply_work)) +devlink_put (devlink); +} +static int
__initdevlink_default_esw_mode_setup(char *str){ devlink_default_esw_mode_param = str;net/devlink/default.c +0
-0@@ -228,10 +325,21 @@ int __init devlink_default_esw_mode_init(void)return err; }
Quoted 10 lines
net/devlink/default.c int __init devlink_default_esw_mode_init(void) · line 334
Quoted 7 lines
net/devlink/default.c +7
-0+ devlink_default_esw_mode_wq =alloc_workqueue ("devlink_default_esw_mode" , +WQ_UNBOUND |WQ_MEM_RECLAIM , +0 ); +if (!devlink_default_esw_mode_wq) { + devlink_default_esw_mode_param =NULL ; +devlink_default_esw_mode_nodes_clear (); +pr_warn ("devlink: devlink_eswitch_mode parameter ignored, failed to allocate workqueue\n" );Why you don't "return" here? I think that we don't need to allow the case wq is not allocated.
The function returns right after this block. It is not treated as a valid “workqueue unavailable” mode, the parsed defaults are cleared, the parameter is ignored, and no default eswitch mode will be applied.
I kept it as a non critical failure so we do not abort the whole devlink init just because the default-mode workqueue could not be allocated.
That said, I can make this more explicit by returning 0 directly from this error path.
Mark
Quoted 15 lines
Quoted 13 lines
net/devlink/default.c +4
-0+ } +return 0 ; }void __init devlink_default_esw_mode_cleanup (void ) { +if (devlink_default_esw_mode_wq) +destroy_workqueue (devlink_default_esw_mode_wq);devlink_default_esw_mode_nodes_clear (); }[..]
Jiri Pirko
maintainerRe: [PATCH net-next V5 4/6] devlink: Apply eswitch mode boot defaults
Quoted 103 lines · Thu, Jul 09, 2026 at 07:45:20AM +0200, [email protected] wrote:
Quoted 9 lines · Jiri Pirko
Quoted 8 lines · Tue, Jul 07, 2026 at 07:45:25PM +0200, [email protected] wrote:
Apply parsed devlink_eswitch_mode= defaults after devlink registration and after successful reload.
devl_register()may still be called before the device is ready for an eswitch mode change. Keep the registration path passive and let the regulardevl_unlock()path queue the async apply work once the instance is registered and the default is still pending.Quoted 7 lines
Quoted 5 lines
The queueing path runs while the devlink instance lock is held, so the queued work gets its devlink reference before the caller drops the lock. The worker then takes the devlink instance lock normally and applies the default only if the instance is still registered and the default is still pending.
This is very code-descriptive. What's the benefit of that?
The point is that there is still a window before the queued work runs where the user can explicitly set the eswitch mode. If they do, the default will no longer be pending, so the worker will skip applying it.
I'll reword.
Quoted 35 lines
Quoted 11 lines
For successful reloads that performed
DRIVER_REINIT,devlink_reload()already holds the devlink instance lock and the driver has completedreload_up(). Clear pending work and apply the default directly from the reload path instead of queueing work.Preserve the user configured mode when it is set before devlink applies the default.
[..]
Quoted 20 lines
net/devlink/default.c +20
-0+void devlink_default_esw_mode_apply_locked (struct devlink *devlink) +{ +const struct devlink_ops *ops = devlink->ops; +int err; + +devl_assert_locked (devlink); + +if (!devlink_default_esw_mode_match (devlink)) +return ; + +if (!ops->eswitch_mode_set) { +if (!devlink_default_esw_mode_match_all) +devl_warn (devlink, +"devlink_eswitch_mode= selected this device but eswitch mode setting is not supported\n" ); +return ; + } + + err =devlink_eswitch_mode_set (devlink, devlink_default_esw_mode,NULL ); +if (err) + devl_warn(devlink,Quoted 8 lines
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 233
Quoted 5 lines
net/devlink/default.c +5
-0+"Couldn't apply default eswitch mode, err %d\n" , + err); +} + +void devlink_default_esw_mode_queue_apply_work (struct devlink *devlink)eswitch/esw - we call it "eswitch" consistently everywhere. Why "esw" here?
Ack
Quoted 21 lines
Quoted 21 lines
net/devlink/default.c +18
-0+{ +devl_assert_locked (devlink); + +if (!devlink_default_esw_mode_enabled || !devlink_default_esw_mode_wq) +return ; +if (!devlink->default_esw_mode_apply_pending || + !__devl_is_registered (devlink)) +return ; +if (!devlink_try_get (devlink)) +return ; +if (!queue_work (devlink_default_esw_mode_wq, + &devlink->default_esw_mode_apply_work)) +devlink_put (devlink); +} + +static void devlink_default_esw_mode_apply_work (struct work_struct *work) +{ +struct devlink *devlink;Quoted 8 lines
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 255
Quoted 4 lines
net/devlink/default.c +4
-0+ + devlink = container_of(work, struct devlink, + default_esw_mode_apply_work); +What happens if userspace eswitch mode set happens now? Any userspace attempt should cancel the default apply. I don't see such mechanism in your patches, did I miss it?
Quoted 7 lines
devlink_nl_eswitch_set_doit()callsdevlink_default_esw_mode_apply_pending_clear(), which clears the pending bit.So if a user sets the eswitch mode before the queued default work applies it, the worker will see that the default is no longer pending and will do nothing
Okay.
Quoted 68 lines
Quoted 4 lines
Quoted 15 lines
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 268
Quoted 13 lines
net/devlink/default.c +13
-0+devl_lock (devlink); + +if (devl_is_registered (devlink) && + devlink->default_esw_mode_apply_pending) { +devlink_default_esw_mode_apply_locked (devlink); + devlink->default_esw_mode_apply_pending = false; + } + +devl_unlock (devlink); +devlink_put (devlink); +} + +void devlink_default_esw_mode_instance_init (struct devlink *devlink)Why "_instance_"? Care to drop?
Ack
Quoted 18 lines
Quoted 6 lines
net/devlink/default.c +4
-0+{ +INIT_WORK (&devlink->default_esw_mode_apply_work, + devlink_default_esw_mode_apply_work); + devlink->default_esw_mode_apply_pending = true;net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 282
Quoted 10 lines
net/devlink/default.c +10
-0+} + +void devlink_default_esw_mode_apply_pending_clear (struct devlink *devlink) +{ +devl_assert_locked (devlink); + + devlink->default_esw_mode_apply_pending = false; +} + +void devlink_default_esw_mode_instance_cleanup (struct devlink *devlink)Why "_instance_"? Care to drop?
Ack
Quoted 14 lines
Quoted 14 lines
net/devlink/default.c +5
-0+{ +if (cancel_work_sync (&devlink->default_esw_mode_apply_work)) +devlink_put (devlink); +} +static int
__initdevlink_default_esw_mode_setup(char *str){ devlink_default_esw_mode_param = str;net/devlink/default.c +0
-0@@ -228,10 +325,21 @@ int __init devlink_default_esw_mode_init(void)return err; }Quoted 10 lines
net/devlink/default.c int __init devlink_default_esw_mode_init(void) · line 334
Quoted 7 lines
net/devlink/default.c +7
-0+ devlink_default_esw_mode_wq =alloc_workqueue ("devlink_default_esw_mode" , +WQ_UNBOUND |WQ_MEM_RECLAIM , +0 ); +if (!devlink_default_esw_mode_wq) { + devlink_default_esw_mode_param =NULL ; +devlink_default_esw_mode_nodes_clear (); +pr_warn ("devlink: devlink_eswitch_mode parameter ignored, failed to allocate workqueue\n" );Why you don't "return" here? I think that we don't need to allow the case wq is not allocated.
Quoted 1 line
The function returns right after this block. It is not treated
What I ment was "return error".
Quoted 7 lines
as a valid “workqueue unavailable” mode, the parsed defaults are cleared, the parameter is ignored, and no default eswitch mode will be applied.
I kept it as a non critical failure so we do not abort the whole devlink init just because the default-mode workqueue could not be allocated.
Why to treat it like this? Is there any other example of such flow in devlink? I don't see the benefit, only potential confusion in very unlikely case the alloc_workqueue fails. Am I wrong? If not, just bail out here.
Quoted 22 lines
That said, I can make this more explicit by returning 0 directly from this error path.
Mark
Quoted 15 lines
Quoted 13 lines
net/devlink/default.c +4
-0+ } +return 0 ; }void __init devlink_default_esw_mode_cleanup (void ) { +if (devlink_default_esw_mode_wq) +destroy_workqueue (devlink_default_esw_mode_wq);devlink_default_esw_mode_nodes_clear (); }[..]
Mark Bloch
maintainerRe: [PATCH net-next V5 4/6] devlink: Apply eswitch mode boot defaults
Quoted 192 lines · Jiri Pirko
Quoted 103 lines · Thu, Jul 09, 2026 at 07:45:20AM +0200, [email protected] wrote:
Quoted 9 lines · Jiri Pirko
Quoted 8 lines · Tue, Jul 07, 2026 at 07:45:25PM +0200, [email protected] wrote:
Apply parsed devlink_eswitch_mode= defaults after devlink registration and after successful reload.
devl_register()may still be called before the device is ready for an eswitch mode change. Keep the registration path passive and let the regulardevl_unlock()path queue the async apply work once the instance is registered and the default is still pending.Quoted 7 lines
Quoted 5 lines
The queueing path runs while the devlink instance lock is held, so the queued work gets its devlink reference before the caller drops the lock. The worker then takes the devlink instance lock normally and applies the default only if the instance is still registered and the default is still pending.
This is very code-descriptive. What's the benefit of that?
The point is that there is still a window before the queued work runs where the user can explicitly set the eswitch mode. If they do, the default will no longer be pending, so the worker will skip applying it.
I'll reword.
Quoted 35 lines
Quoted 11 lines
For successful reloads that performed
DRIVER_REINIT,devlink_reload()already holds the devlink instance lock and the driver has completedreload_up(). Clear pending work and apply the default directly from the reload path instead of queueing work.Preserve the user configured mode when it is set before devlink applies the default.
[..]
Quoted 20 lines
net/devlink/default.c +20
-0+void devlink_default_esw_mode_apply_locked (struct devlink *devlink) +{ +const struct devlink_ops *ops = devlink->ops; +int err; + +devl_assert_locked (devlink); + +if (!devlink_default_esw_mode_match (devlink)) +return ; + +if (!ops->eswitch_mode_set) { +if (!devlink_default_esw_mode_match_all) +devl_warn (devlink, +"devlink_eswitch_mode= selected this device but eswitch mode setting is not supported\n" ); +return ; + } + + err =devlink_eswitch_mode_set (devlink, devlink_default_esw_mode,NULL ); +if (err) + devl_warn(devlink,Quoted 8 lines
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 233
Quoted 5 lines
net/devlink/default.c +5
-0+"Couldn't apply default eswitch mode, err %d\n" , + err); +} + +void devlink_default_esw_mode_queue_apply_work (struct devlink *devlink)eswitch/esw - we call it "eswitch" consistently everywhere. Why "esw" here?
Ack
Quoted 21 lines
Quoted 21 lines
net/devlink/default.c +18
-0+{ +devl_assert_locked (devlink); + +if (!devlink_default_esw_mode_enabled || !devlink_default_esw_mode_wq) +return ; +if (!devlink->default_esw_mode_apply_pending || + !__devl_is_registered (devlink)) +return ; +if (!devlink_try_get (devlink)) +return ; +if (!queue_work (devlink_default_esw_mode_wq, + &devlink->default_esw_mode_apply_work)) +devlink_put (devlink); +} + +static void devlink_default_esw_mode_apply_work (struct work_struct *work) +{ +struct devlink *devlink;Quoted 8 lines
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 255
Quoted 4 lines
net/devlink/default.c +4
-0+ + devlink = container_of(work, struct devlink, + default_esw_mode_apply_work); +What happens if userspace eswitch mode set happens now? Any userspace attempt should cancel the default apply. I don't see such mechanism in your patches, did I miss it?
Quoted 7 lines
devlink_nl_eswitch_set_doit()callsdevlink_default_esw_mode_apply_pending_clear(), which clears the pending bit.So if a user sets the eswitch mode before the queued default work applies it, the worker will see that the default is no longer pending and will do nothing
Okay.
Quoted 68 lines
Quoted 4 lines
Quoted 15 lines
net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 268
Quoted 13 lines
net/devlink/default.c +13
-0+devl_lock (devlink); + +if (devl_is_registered (devlink) && + devlink->default_esw_mode_apply_pending) { +devlink_default_esw_mode_apply_locked (devlink); + devlink->default_esw_mode_apply_pending = false; + } + +devl_unlock (devlink); +devlink_put (devlink); +} + +void devlink_default_esw_mode_instance_init (struct devlink *devlink)Why "_instance_"? Care to drop?
Ack
Quoted 18 lines
Quoted 6 lines
net/devlink/default.c +4
-0+{ +INIT_WORK (&devlink->default_esw_mode_apply_work, + devlink_default_esw_mode_apply_work); + devlink->default_esw_mode_apply_pending = true;net/devlink/default.c static int __init devlink_default_esw_mode_parse(char *str) · line 282
Quoted 10 lines
net/devlink/default.c +10
-0+} + +void devlink_default_esw_mode_apply_pending_clear (struct devlink *devlink) +{ +devl_assert_locked (devlink); + + devlink->default_esw_mode_apply_pending = false; +} + +void devlink_default_esw_mode_instance_cleanup (struct devlink *devlink)Why "_instance_"? Care to drop?
Ack
Quoted 14 lines
Quoted 14 lines
net/devlink/default.c +5
-0+{ +if (cancel_work_sync (&devlink->default_esw_mode_apply_work)) +devlink_put (devlink); +} +static int
__initdevlink_default_esw_mode_setup(char *str){ devlink_default_esw_mode_param = str;net/devlink/default.c +0
-0@@ -228,10 +325,21 @@ int __init devlink_default_esw_mode_init(void)return err; }Quoted 10 lines
net/devlink/default.c int __init devlink_default_esw_mode_init(void) · line 334
Quoted 7 lines
net/devlink/default.c +7
-0+ devlink_default_esw_mode_wq =alloc_workqueue ("devlink_default_esw_mode" , +WQ_UNBOUND |WQ_MEM_RECLAIM , +0 ); +if (!devlink_default_esw_mode_wq) { + devlink_default_esw_mode_param =NULL ; +devlink_default_esw_mode_nodes_clear (); +pr_warn ("devlink: devlink_eswitch_mode parameter ignored, failed to allocate workqueue\n" );Why you don't "return" here? I think that we don't need to allow the case wq is not allocated.
Quoted 1 line
The function returns right after this block. It is not treated
What I ment was "return error".
Quoted 4 lines
as a valid “workqueue unavailable” mode, the parsed defaults are cleared, the parameter is ignored, and no default eswitch mode will be applied.
Quoted 8 lines
Quoted 3 lines
I kept it as a non critical failure so we do not abort the whole devlink init just because the default-mode workqueue could not be allocated.
Why to treat it like this? Is there any other example of such flow in devlink? I don't see the benefit, only potential confusion in very unlikely case the
alloc_workqueuefails. Am I wrong? If not, just bail out here.
As I'm dropping the work queue it doesn't matter. I'll treat failed parsing as error then.
Mark
Quoted 25 lines
Quoted 25 lines
That said, I can make this more explicit by returning 0 directly from this error path.
Mark
Quoted 15 lines
Quoted 13 lines
net/devlink/default.c +4
-0+ } +return 0 ; }void __init devlink_default_esw_mode_cleanup (void ) { +if (devlink_default_esw_mode_wq) +destroy_workqueue (devlink_default_esw_mode_wq);devlink_default_esw_mode_nodes_clear (); }[..]
Mark Bloch
maintainerPATCH v5 5/6 devlink: Add API to apply eswitch mode boot default
In reply to Mark BlochAdd devl_apply_default_esw_mode() for drivers that can apply the devlink_eswitch_mode= boot default once their device is ready instead of waiting for the asynchronous registration work.
--- include/net/devlink.h | 1 + net/devlink/default.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)
include/net/devlink.h +1 -0
@@ -1661,6 +1661,7 @@ static inline struct devlink *devlink_alloc(const struct devlink_ops *ops,int devl_register (struct devlink *devlink);void devl_unregister (struct devlink *devlink); +void devl_apply_default_esw_mode (struct devlink *devlink);void devlink_register (struct devlink *devlink);void devlink_unregister (struct devlink *devlink);void devlink_free (struct devlink *devlink);
net/devlink/default.c +19 -0
@@ -2,6 +2,7 @@/* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ #include <linux/init.h> +#include <linux/export.h> #include <linux/list.h> #include <linux/slab.h> #include <linux/string.h>
@@ -279,6 +280,24 @@ void devlink_default_esw_mode_apply_pending_clear(struct devlink *devlink) devlink->default_esw_mode_apply_pending = false; } +/** +* devl_apply_default_esw_mode - Apply devlink eswitch mode boot default +* @devlink: devlink +* +* Apply the devlink eswitch mode selected by the devlink_eswitch_mode= +* kernel command line parameter, if any matches @devlink. +* +* The caller must hold the devlink instance lock. +*/ +void devl_apply_default_esw_mode (struct devlink *devlink) +{ +devl_assert_locked (devlink); + + devlink->default_esw_mode_apply_pending = false; +devlink_default_esw_mode_apply_locked (devlink); +} +EXPORT_SYMBOL_GPL (devl_apply_default_esw_mode); +void devlink_default_esw_mode_instance_cleanup (struct devlink *devlink) {if (cancel_work_sync (&devlink->default_esw_mode_apply_work))
Signature · 1 line
2.43.0
Mark Bloch
maintainerPATCH v5 6/6 net/mlx5: Apply devlink eswitch mode boot default on probe
In reply to Mark BlochApply devlink_eswitch_mode= boot defaults for mlx5 after the initial probe finishes device initialization while holding the devlink instance lock.
At this point the devlink instance is registered and mlx5 can perform an eswitch mode change. Calling devl_apply_default_esw_mode() also clears any pending default apply work queued by devl_register(), so the queued work will not apply the same default again.
Keep this call in mlx5_init_one() rather than the lower-level devl-locked init helper. That helper is also used by devlink reload, and devlink core already applies the boot default after a successful DRIVER_REINIT reload.
--- drivers/net/ethernet/mellanox/mlx5/core/main.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/main.c +13 -0
@@ -1392,6 +1392,17 @@ static void mlx5_unload(struct mlx5_core_dev *dev)mlx5_free_bfreg (dev, &dev->priv.bfreg); } +static void mlx5_devl_apply_default_esw_mode (struct mlx5_core_dev *dev) +{ +struct devlink *devlink =priv_to_devlink (dev); + +if (!MLX5_ESWITCH_MANAGER (dev)) +return ; + +devl_assert_locked (devlink); +devl_apply_default_esw_mode (devlink); +} +int mlx5_init_one_devl_locked (struct mlx5_core_dev *dev) {bool light_probe =mlx5_dev_is_lightweight (dev);
@@ -1471,6 +1482,8 @@ int mlx5_init_one(struct mlx5_core_dev *dev) err =mlx5_init_one_devl_locked (dev);if (err)devl_unregister (devlink); +else +mlx5_devl_apply_default_esw_mode (dev); unlock:devl_unlock (devlink);return err;
Signature · 1 line
2.43.0
Jiri Pirko
maintainerRe: [PATCH net-next V5 6/6] net/mlx5: Apply devlink eswitch mode boot default on probe
Quoted 42 lines · Tue, Jul 07, 2026 at 07:45:27PM +0200, [email protected] wrote:
Apply devlink_eswitch_mode= boot defaults for mlx5 after the initial probe finishes device initialization while holding the devlink instance lock.
At this point the devlink instance is registered and mlx5 can perform an eswitch mode change. Calling
devl_apply_default_esw_mode()also clears any pending default apply work queued bydevl_register(), so the queued work will not apply the same default again.Keep this call in
mlx5_init_one()rather than the lower-level devl-locked init helper. That helper is also used by devlink reload, and devlink core already applies the boot default after a successfulDRIVER_REINITreload.Signed-off-by Mark Bloch <[email protected]>---
drivers/net/ethernet/mellanox/mlx5/core/main.c| 13 +++++++++++++1 file changed, 13 insertions(+)drivers/net/ethernet/mellanox/mlx5/core/main.c +11
-0@@ -1392,6 +1392,17 @@ static void mlx5_unload(struct mlx5_core_dev *dev)mlx5_free_bfreg (dev, &dev->priv.bfreg); } +static void mlx5_devl_apply_default_esw_mode (struct mlx5_core_dev *dev) +{ +struct devlink *devlink =priv_to_devlink (dev); + +if (!MLX5_ESWITCH_MANAGER (dev)) +return ; + +devl_assert_locked (devlink); +devl_apply_default_esw_mode (devlink); +} +int mlx5_init_one_devl_locked (struct mlx5_core_dev *dev) {bool light_probe =mlx5_dev_is_lightweight (dev);
drivers/net/ethernet/mellanox/mlx5/core/main.c int mlx5_init_one(struct mlx5_core_dev *dev) · line 1486
Quoted 5 lines
drivers/net/ethernet/mellanox/mlx5/core/main.c +2
-0@@ -1471,6 +1482,8 @@ int mlx5_init_one(struct mlx5_core_dev *dev) err =mlx5_init_one_devl_locked (dev);if (err)devl_unregister (devlink); +else +mlx5_devl_apply_default_esw_mode (dev);
I don't understand why this patch is needed at all. Just leave the job to the devlink core, no? That was the point to not pollute drivers with code like this. Is it some kind of leftover?
Quoted 6 lines
unlock:
devl_unlock(devlink); return err;Signature · 1 line
2.43.0
Mark Bloch
maintainerRe: [PATCH net-next V5 6/6] net/mlx5: Apply devlink eswitch mode boot default on probe
Quoted 44 lines · Jiri Pirko
Quoted 43 lines · Tue, Jul 07, 2026 at 07:45:27PM +0200, [email protected] wrote:
Apply devlink_eswitch_mode= boot defaults for mlx5 after the initial probe finishes device initialization while holding the devlink instance lock.
At this point the devlink instance is registered and mlx5 can perform an eswitch mode change. Calling
devl_apply_default_esw_mode()also clears any pending default apply work queued bydevl_register(), so the queued work will not apply the same default again.Keep this call in
mlx5_init_one()rather than the lower-level devl-locked init helper. That helper is also used by devlink reload, and devlink core already applies the boot default after a successfulDRIVER_REINITreload.Signed-off-by Mark Bloch <[email protected]>---
drivers/net/ethernet/mellanox/mlx5/core/main.c| 13 +++++++++++++1 file changed, 13 insertions(+)drivers/net/ethernet/mellanox/mlx5/core/main.c +11
-0@@ -1392,6 +1392,17 @@ static void mlx5_unload(struct mlx5_core_dev *dev)mlx5_free_bfreg (dev, &dev->priv.bfreg); } +static void mlx5_devl_apply_default_esw_mode (struct mlx5_core_dev *dev) +{ +struct devlink *devlink =priv_to_devlink (dev); + +if (!MLX5_ESWITCH_MANAGER (dev)) +return ; + +devl_assert_locked (devlink); +devl_apply_default_esw_mode (devlink); +} +int mlx5_init_one_devl_locked (struct mlx5_core_dev *dev) {bool light_probe =mlx5_dev_is_lightweight (dev);@@ -1471,6 +1482,8 @@ int mlx5_init_one(struct mlx5_core_dev *dev) err =mlx5_init_one_devl_locked (dev);
Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/main.c int mlx5_init_one(struct mlx5_core_dev *dev) · line 1486
Quoted 4 lines
drivers/net/ethernet/mellanox/mlx5/core/main.c +2
-0if (err)devl_unregister (devlink); +else +mlx5_devl_apply_default_esw_mode (dev);I don't understand why this patch is needed at all. Just leave the job to the devlink core, no? That was the point to not pollute drivers with code like this. Is it some kind of leftover?
It was discussed with Jakub here:
https://lore.kernel.org/all/[email protected]/
The main reason is timing. If the default is applied only by devlink core, it has to wait until the driver drops the devlink lock. For mlx5, that usually happens very late in the init sequence. I wanted drivers to be able to apply the default as soon as the driver is ready for it, because on NICs with a DPU the host PF can remain stuck until the ECPF moves to switchdev.
This API is also useful beyond the initial devlink registration path. Follow-up patches will use it for driver controlled paths that are not covered by the devlink core, such as recovery and FW reset.
There is also a race window where userspace may take the devlink lock before the core gets a chance to apply the default. Letting the driver explicitly apply the default at the right point avoids that scenario.
Thinking about this again, maybe the simpler approach is to apply the default from devl_unlock(). That would avoid the whole workqueue infra.
I avoided doing that earlier because applying a default mode as a side effect of devl_unlock() feels a bit odd. But compared to adding dedicated workqueue handling maybe it is the lesser evil here.
What do you think?
About the extra API, I still think it's useful and would like to keep it if possible.
Mark
Quoted 9 lines
Quoted 9 lines
unlock:
devl_unlock(devlink); return err;Signature · 1 line
2.43.0
Jiri Pirko
maintainerRe: [PATCH net-next V5 6/6] net/mlx5: Apply devlink eswitch mode boot default on probe
Quoted 56 lines · Thu, Jul 09, 2026 at 08:00:19AM +0200, [email protected] wrote:
Quoted 44 lines · Jiri Pirko
Quoted 43 lines · Tue, Jul 07, 2026 at 07:45:27PM +0200, [email protected] wrote:
Apply devlink_eswitch_mode= boot defaults for mlx5 after the initial probe finishes device initialization while holding the devlink instance lock.
At this point the devlink instance is registered and mlx5 can perform an eswitch mode change. Calling
devl_apply_default_esw_mode()also clears any pending default apply work queued bydevl_register(), so the queued work will not apply the same default again.Keep this call in
mlx5_init_one()rather than the lower-level devl-locked init helper. That helper is also used by devlink reload, and devlink core already applies the boot default after a successfulDRIVER_REINITreload.Signed-off-by Mark Bloch <[email protected]>---
drivers/net/ethernet/mellanox/mlx5/core/main.c| 13 +++++++++++++1 file changed, 13 insertions(+)drivers/net/ethernet/mellanox/mlx5/core/main.c +11
-0@@ -1392,6 +1392,17 @@ static void mlx5_unload(struct mlx5_core_dev *dev)mlx5_free_bfreg (dev, &dev->priv.bfreg); } +static void mlx5_devl_apply_default_esw_mode (struct mlx5_core_dev *dev) +{ +struct devlink *devlink =priv_to_devlink (dev); + +if (!MLX5_ESWITCH_MANAGER (dev)) +return ; + +devl_assert_locked (devlink); +devl_apply_default_esw_mode (devlink); +} +int mlx5_init_one_devl_locked (struct mlx5_core_dev *dev) {bool light_probe =mlx5_dev_is_lightweight (dev);@@ -1471,6 +1482,8 @@ int mlx5_init_one(struct mlx5_core_dev *dev) err =mlx5_init_one_devl_locked (dev);Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/main.c int mlx5_init_one(struct mlx5_core_dev *dev) · line 1486
Quoted 4 lines
drivers/net/ethernet/mellanox/mlx5/core/main.c +2
-0if (err)devl_unregister (devlink); +else +mlx5_devl_apply_default_esw_mode (dev);I don't understand why this patch is needed at all. Just leave the job to the devlink core, no? That was the point to not pollute drivers with code like this. Is it some kind of leftover?
Quoted 5 lines
It was discussed with Jakub here:
https://lore.kernel.org/all/[email protected]/The main reason is timing. If the default is applied only by devlink core, it has to wait until the driver drops the devlink lock.
I don't follow.
<quote>
devl_lock(devlink);
if (dev->shd) {
err = devl_nested_devlink_set(dev->shd, devlink);
if (err)
goto unlock;
}
devl_register(devlink);
err = mlx5_init_one_devl_locked(dev);
if (err)
devl_unregister(devlink);
unlock:
devl_unlock(devlink);</quote>
devlink lock is droped right after.
Quoted 17 lines
For mlx5, that usually happens very late in the init sequence. I wanted drivers to be able to apply the default as soon as the driver is ready for it, because on NICs with a DPU the host PF can remain stuck until the ECPF moves to switchdev.
This API is also useful beyond the initial devlink registration path. Follow-up patches will use it for driver controlled paths that are not covered by the devlink core, such as recovery and FW reset.
There is also a race window where userspace may take the devlink lock before the core gets a chance to apply the default. Letting the driver explicitly apply the default at the right point avoids that scenario.
Thinking about this again, maybe the simpler approach is to apply the default from
devl_unlock(). That would avoid the whole workqueue infra.
Quoted 5 lines
I avoided doing that earlier because applying a default mode as a side effect of
devl_unlock()feels a bit odd. But compared to adding dedicated workqueue handling maybe it is the lesser evil here.What do you think?
I was under impression that you need the work to resolve nested locking. If not, drop it, sure.
Quoted 17 lines
About the extra API, I still think it's useful and would like to keep it if possible.
Mark
Quoted 9 lines
Quoted 9 lines
unlock:
devl_unlock(devlink); return err;Signature · 1 line
2.43.0
Mark Bloch
maintainerRe: [PATCH net-next V5 6/6] net/mlx5: Apply devlink eswitch mode boot default on probe
Quoted 78 lines · Jiri Pirko
Quoted 56 lines · Thu, Jul 09, 2026 at 08:00:19AM +0200, [email protected] wrote:
Quoted 44 lines · Jiri Pirko
Quoted 43 lines · Tue, Jul 07, 2026 at 07:45:27PM +0200, [email protected] wrote:
Apply devlink_eswitch_mode= boot defaults for mlx5 after the initial probe finishes device initialization while holding the devlink instance lock.
At this point the devlink instance is registered and mlx5 can perform an eswitch mode change. Calling
devl_apply_default_esw_mode()also clears any pending default apply work queued bydevl_register(), so the queued work will not apply the same default again.Keep this call in
mlx5_init_one()rather than the lower-level devl-locked init helper. That helper is also used by devlink reload, and devlink core already applies the boot default after a successfulDRIVER_REINITreload.Signed-off-by Mark Bloch <[email protected]>---
drivers/net/ethernet/mellanox/mlx5/core/main.c| 13 +++++++++++++1 file changed, 13 insertions(+)drivers/net/ethernet/mellanox/mlx5/core/main.c +11
-0@@ -1392,6 +1392,17 @@ static void mlx5_unload(struct mlx5_core_dev *dev)mlx5_free_bfreg (dev, &dev->priv.bfreg); } +static void mlx5_devl_apply_default_esw_mode (struct mlx5_core_dev *dev) +{ +struct devlink *devlink =priv_to_devlink (dev); + +if (!MLX5_ESWITCH_MANAGER (dev)) +return ; + +devl_assert_locked (devlink); +devl_apply_default_esw_mode (devlink); +} +int mlx5_init_one_devl_locked (struct mlx5_core_dev *dev) {bool light_probe =mlx5_dev_is_lightweight (dev);@@ -1471,6 +1482,8 @@ int mlx5_init_one(struct mlx5_core_dev *dev) err =mlx5_init_one_devl_locked (dev);Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/main.c int mlx5_init_one(struct mlx5_core_dev *dev) · line 1486
Quoted 4 lines
drivers/net/ethernet/mellanox/mlx5/core/main.c +2
-0if (err)devl_unregister (devlink); +else +mlx5_devl_apply_default_esw_mode (dev);I don't understand why this patch is needed at all. Just leave the job to the devlink core, no? That was the point to not pollute drivers with code like this. Is it some kind of leftover?
Quoted 5 lines
It was discussed with Jakub here:
https://lore.kernel.org/all/[email protected]/The main reason is timing. If the default is applied only by devlink core, it has to wait until the driver drops the devlink lock.
I don't follow.
<quote>
devl_lock(devlink); if (dev->shd) { err = devl_nested_devlink_set(dev->shd, devlink); if (err) goto unlock; } devl_register(devlink); err = mlx5_init_one_devl_locked(dev); if (err) devl_unregister(devlink); unlock: devl_unlock(devlink);
Quoted 3 lines
</quote>
devlink lock is droped right after.
Earlier versions had the fw reset / recovery flows covered as well. The regular probe/init in mlx5 need some changes in mlx5 so I can move it earlier, left the API just to make sure we are aligned and it could be used be a later patches.
Anyway, I guess I can drop this as well and continue with followup patches just to keep forward progress.
Quoted 20 lines
Quoted 20 lines
For mlx5, that usually happens very late in the init sequence. I wanted drivers to be able to apply the default as soon as the driver is ready for it, because on NICs with a DPU the host PF can remain stuck until the ECPF moves to switchdev.
This API is also useful beyond the initial devlink registration path. Follow-up patches will use it for driver controlled paths that are not covered by the devlink core, such as recovery and FW reset.
There is also a race window where userspace may take the devlink lock before the core gets a chance to apply the default. Letting the driver explicitly apply the default at the right point avoids that scenario.
Thinking about this again, maybe the simpler approach is to apply the default from
devl_unlock(). That would avoid the whole workqueue infra.
Quoted 8 lines
Quoted 5 lines
I avoided doing that earlier because applying a default mode as a side effect of
devl_unlock()feels a bit odd. But compared to adding dedicated workqueue handling maybe it is the lesser evil here.What do you think?
I was under impression that you need the work to resolve nested locking. If not, drop it, sure.
I think that should be fine. Once the instance is registered and the devlink lock is dropped, the driver should be ready for normal devlink operations anyway, since a user could trigger the same path at that point, so we just don't drop the lock and call the driver directly, should work.
The only caveat I can think of is a driver holding some other lock while dropping the devlink lock, and then trying to take that same lock again from the mode-setting callback. But that sounds like a driver bug rather than something devlink should work around.
I’ll drop the workqueue approach and rework this for v6 :)
Mark
Quoted 19 lines
Quoted 19 lines
About the extra API, I still think it's useful and would like to keep it if possible.
Mark
Quoted 9 lines
Quoted 9 lines
unlock:
devl_unlock(devlink); return err;Signature · 1 line
2.43.0