mlxsw: Make the driver ops-locked
Subsystems (3)
Ido Schimmel
maintainercover mlxsw: Make the driver ops-locked
Make the driver ops-locked in order to allow ethtool operations to be invoked without RTNL being held.
An ops-locked driver has most of its NDOs, all of its ethtool operations and some net device notifications run with the netdev instance lock held.
In the specific case of mlxsw, the driver is not using any functions that acquire this lock nor functions that expect the lock to be held for an ops-locked driver. Therefore, converting its NDOs to run with the lock being held is trivial except for a small quirk which is handled in patch #1.
The driver does not generate any net device notifications, so there is no risk of nested notifications of the ops-locked types. For the notifications that run under the instance lock, RTNL is also held, and the driver does not acquire the instance lock itself, so no changes are required in its notifier handling.
Ethtool operations can be invoked without RTNL except for two operations that are annotated in patch #2.
Lastly, patch #3 converts the driver to be ops-locked.
A probe on rtnl_lock() shows it is no longer taken when dumping statistics:
# perf probe --add rtnl_lock
Before:
# perf stat -e probe:rtnl_lock -- ethtool -S swp1 --all-groups
[...]
1 probe:rtnl_lockAfter:
# perf stat -e probe:rtnl_lock -- ethtool -S swp1 --all-groups
[...]
0 probe:rtnl_lockNo issues were reported after running a full regression with a debug config that has lockdep enabled.
Ido Schimmel (3):
mlxsw: Convert to async version of ndo_set_rx_mode
mlxsw: ethtool: Prepare for RTNL-less ethtool operations
mlxsw: Tell the core to use the netdev instance lock
drivers/net/ethernet/mellanox/mlxsw/spectrum.c | 8 ++++++-- drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c | 2 ++ 2 files changed, 8 insertions(+), 2 deletions(-)
Signature · 1 line
2.54.0
Ido Schimmel
maintainerPATCH 1/3 mlxsw: Convert to async version of ndo_set_rx_mode
Commit c5b9b518adab ("mlxsw: spectrum: Add set_rx_mode ndo stub") added a stub for ndo_set_rx_mode to prevent dev_ifsioc() from returning an error for the SIOCADDMULTI and SIOCDELMULTI cases.
Since then dev_ifsioc() was taught to also accept ndo_set_rx_mode_async and commit 3cbd22938877 ("net: warn ops-locked drivers still using ndo_set_rx_mode") modified register_netdevice() to warn when registering an ops-locked net device that still uses ndo_set_rx_mode instead of ndo_set_rx_mode_async.
In preparation for converting the driver to be ops-locked, convert the ndo_set_rx_mode stub to a ndo_set_rx_mode_async stub.
--- drivers/net/ethernet/mellanox/mlxsw/spectrum.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlxsw/spectrum.c +5 -2
@@ -663,8 +663,11 @@ static netdev_tx_t mlxsw_sp_port_xmit(struct sk_buff *skb,return NETDEV_TX_OK ; } -static void mlxsw_sp_set_rx_mode (struct net_device *dev) +static int mlxsw_sp_set_rx_mode_async (struct net_device *dev, +struct netdev_hw_addr_list *uc, +struct netdev_hw_addr_list *mc) { +return 0 ; }static int mlxsw_sp_port_set_mac_address (struct net_device *dev,void *p)
@@ -1191,7 +1194,7 @@ static const struct net_device_ops mlxsw_sp_port_netdev_ops = { .ndo_stop = mlxsw_sp_port_stop, .ndo_start_xmit = mlxsw_sp_port_xmit, .ndo_setup_tc = mlxsw_sp_setup_tc, - .ndo_set_rx_mode = mlxsw_sp_set_rx_mode, + .ndo_set_rx_mode_async = mlxsw_sp_set_rx_mode_async, .ndo_set_mac_address = mlxsw_sp_port_set_mac_address, .ndo_change_mtu = mlxsw_sp_port_change_mtu, .ndo_get_stats64 = mlxsw_sp_port_get_stats64,
Signature · 1 line
2.54.0
Ido Schimmel
maintainerPATCH 2/3 mlxsw: ethtool: Prepare for RTNL-less ethtool operations
In reply to Ido SchimmelA subsequent patch is going to make the driver ops-locked and allow ethtool operations to run without RTNL. In preparation for this change, tell the core about a couple of ethtool operations that should remain under RTNL:
1. Set pause parameters: Configures the port's headroom buffer which is also configured by RTNL-only paths such as DCB and qdisc. These paths can probably be converted to acquire the netdev instance lock, but this operation in not frequently called (unlike stats query), so avoid the added complexity for now.
2. Get link state: Calls ethtool_op_get_link() which requires RTNL. See commit 1105ef941c1a ("net: ethtool: keep rtnl_lock for ops using ethtool_op_get_link()").
All the other operations do not access shared resources, do not invoke helpers that require RTNL or already have the appropriate locking in place.
--- drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c +2 -0
@@ -1262,6 +1262,8 @@ mlxsw_sp_set_module_power_mode(struct net_device *dev,const struct ethtool_ops mlxsw_sp_port_ethtool_ops = { .cap_link_lanes_supported = true, + .op_needs_rtnl =ETHTOOL_OP_NEEDS_RTNL_SPAUSEPARAM | +ETHTOOL_OP_NEEDS_RTNL_GLINK , .get_drvinfo = mlxsw_sp_port_get_drvinfo, .get_link = ethtool_op_get_link, .get_link_ext_state = mlxsw_sp_port_get_link_ext_state,
Signature · 1 line
2.54.0
Ido Schimmel
maintainerPATCH 3/3 mlxsw: Tell the core to use the netdev instance lock
In reply to Ido SchimmelAfter the previous changes the driver is now ready to have its net device and ethtool operations invoked with the netdev instance lock held.
Tell the core about it by setting request_ops_lock to true.
--- drivers/net/ethernet/mellanox/mlxsw/spectrum.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/ethernet/mellanox/mlxsw/spectrum.c +1 -0
@@ -1552,6 +1552,7 @@ static int mlxsw_sp_port_create(struct mlxsw_sp *mlxsw_sp, u16 local_port, dev->vlan_features |=NETIF_F_IP_CSUM |NETIF_F_IPV6_CSUM ; dev->lltx = true; dev->netns_immutable = true; + dev->request_ops_lock = true; dev->min_mtu =ETH_MIN_MTU ; dev->max_mtu =MLXSW_PORT_MAX_MTU -MLXSW_PORT_ETH_FRAME_HDR ;
Signature · 1 line
2.54.0
Quoted 36 lines · Ido Schimmel
Make the driver ops-locked in order to allow ethtool operations to be invoked without RTNL being held.
An ops-locked driver has most of its NDOs, all of its ethtool operations and some net device notifications run with the netdev instance lock held.
In the specific case of mlxsw, the driver is not using any functions that acquire this lock nor functions that expect the lock to be held for an ops-locked driver. Therefore, converting its NDOs to run with the lock being held is trivial except for a small quirk which is handled in patch #1.
The driver does not generate any net device notifications, so there is no risk of nested notifications of the ops-locked types. For the notifications that run under the instance lock, RTNL is also held, and the driver does not acquire the instance lock itself, so no changes are required in its notifier handling.
Ethtool operations can be invoked without RTNL except for two operations that are annotated in patch #2.
Lastly, patch #3 converts the driver to be ops-locked.
A probe on
rtnl_lock()shows it is no longer taken when dumping statistics:# perf probe --add rtnl_lockBefore:
# perf stat -e probe:rtnl_lock -- ethtool -S swp1 --all-groups [...] 1 probe:rtnl_lock
Quoted 6 lines
After:
# perf stat -e probe:rtnl_lock -- ethtool -S swp1 --all-groups [...] 0 probe:rtnl_lock
We probably want to add noinline_for_tracing to rtnl_lock() and friends...
This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:
Quoted 8 lines · On Wed, 8 Jul 2026 15:39:30 +0300 you wrote:
Make the driver ops-locked in order to allow ethtool operations to be invoked without RTNL being held.
An ops-locked driver has most of its NDOs, all of its ethtool operations and some net device notifications run with the netdev instance lock held.
[...]
Here is the summary with links:
- [net-next,1/3] mlxsw: Convert to async version of ndo_set_rx_mode
https://git.kernel.org/netdev/net-next/c/a5faaa079c61 - [net-next,2/3] mlxsw: ethtool: Prepare for RTNL-less ethtool operations https://git.kernel.org/netdev/net-next/c/925a17fe4309 - [net-next,3/3] mlxsw: Tell the core to use the netdev instance lock https://git.kernel.org/netdev/net-next/c/0501dd682648
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html