Skip to contentSkip to conversation

mlxsw: Make the driver ops-locked

PATCH Applied Ido Schimmel 3 participants 6 messages Jul 8 – Jul 21

Subsystems (3)
lore
Share & tools
lore

Ido Schimmel

author maintainer
#0 lore raw

cover mlxsw: Make the driver ops-locked

Make the driver ops-locked in order to allow ethtool operations to be invoked without RTNL being held.

An ops-locked driver has most of its NDOs, all of its ethtool operations and some net device notifications run with the netdev instance lock held.

In the specific case of mlxsw, the driver is not using any functions that acquire this lock nor functions that expect the lock to be held for an ops-locked driver. Therefore, converting its NDOs to run with the lock being held is trivial except for a small quirk which is handled in patch #1.

The driver does not generate any net device notifications, so there is no risk of nested notifications of the ops-locked types. For the notifications that run under the instance lock, RTNL is also held, and the driver does not acquire the instance lock itself, so no changes are required in its notifier handling.

Ethtool operations can be invoked without RTNL except for two operations that are annotated in patch #2.

Lastly, patch #3 converts the driver to be ops-locked.

A probe on rtnl_lock() shows it is no longer taken when dumping statistics:

 # perf probe --add rtnl_lock

Before:

 # perf stat -e probe:rtnl_lock -- ethtool -S swp1 --all-groups
 [...]
                  1      probe:rtnl_lock

After:

 # perf stat -e probe:rtnl_lock -- ethtool -S swp1 --all-groups
 [...]
                  0      probe:rtnl_lock

No issues were reported after running a full regression with a debug config that has lockdep enabled.

Ido Schimmel (3):
mlxsw: Convert to async version of ndo_set_rx_mode
mlxsw: ethtool: Prepare for RTNL-less ethtool operations
mlxsw: Tell the core to use the netdev instance lock

 drivers/net/ethernet/mellanox/mlxsw/spectrum.c         | 8 ++++++--
 drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c | 2 ++
 2 files changed, 8 insertions(+), 2 deletions(-)
Signature · 1 line
2.54.0
author maintainer
#1 lore raw

PATCH 1/3 mlxsw: Convert to async version of ndo_set_rx_mode

Commit c5b9b518adab ("mlxsw: spectrum: Add set_rx_mode ndo stub") added a stub for ndo_set_rx_mode to prevent dev_ifsioc() from returning an error for the SIOCADDMULTI and SIOCDELMULTI cases.

Since then dev_ifsioc() was taught to also accept ndo_set_rx_mode_async and commit 3cbd22938877 ("net: warn ops-locked drivers still using ndo_set_rx_mode") modified register_netdevice() to warn when registering an ops-locked net device that still uses ndo_set_rx_mode instead of ndo_set_rx_mode_async.

In preparation for converting the driver to be ops-locked, convert the ndo_set_rx_mode stub to a ndo_set_rx_mode_async stub.

---
 drivers/net/ethernet/mellanox/mlxsw/spectrum.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlxsw/spectrum.c +5 -2
index 82569162d2e5..3ee1272dcf0e 100644
@@ -663,8 +663,11 @@ static netdev_tx_t mlxsw_sp_port_xmit(struct sk_buff *skb,
 	return NETDEV_TX_OK;
 }
 
-static void mlxsw_sp_set_rx_mode(struct net_device *dev)
+static int mlxsw_sp_set_rx_mode_async(struct net_device *dev,
+				      struct netdev_hw_addr_list *uc,
+				      struct netdev_hw_addr_list *mc)
 {
+	return 0;
 }
 
 static int mlxsw_sp_port_set_mac_address(struct net_device *dev, void *p)
@@ -1191,7 +1194,7 @@ static const struct net_device_ops mlxsw_sp_port_netdev_ops = {
 	.ndo_stop		= mlxsw_sp_port_stop,
 	.ndo_start_xmit		= mlxsw_sp_port_xmit,
 	.ndo_setup_tc           = mlxsw_sp_setup_tc,
-	.ndo_set_rx_mode	= mlxsw_sp_set_rx_mode,
+	.ndo_set_rx_mode_async	= mlxsw_sp_set_rx_mode_async,
 	.ndo_set_mac_address	= mlxsw_sp_port_set_mac_address,
 	.ndo_change_mtu		= mlxsw_sp_port_change_mtu,
 	.ndo_get_stats64	= mlxsw_sp_port_get_stats64,
Signature · 1 line
2.54.0
author maintainer
#2 lore raw

PATCH 2/3 mlxsw: ethtool: Prepare for RTNL-less ethtool operations

In reply to Ido Schimmel

A subsequent patch is going to make the driver ops-locked and allow ethtool operations to run without RTNL. In preparation for this change, tell the core about a couple of ethtool operations that should remain under RTNL:

1. Set pause parameters: Configures the port's headroom buffer which is also configured by RTNL-only paths such as DCB and qdisc. These paths can probably be converted to acquire the netdev instance lock, but this operation in not frequently called (unlike stats query), so avoid the added complexity for now.

2. Get link state: Calls ethtool_op_get_link() which requires RTNL. See commit 1105ef941c1a ("net: ethtool: keep rtnl_lock for ops using ethtool_op_get_link()").

All the other operations do not access shared resources, do not invoke helpers that require RTNL or already have the appropriate locking in place.

---
 drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c +2 -0
index 7f78b1ef61cc..3bdb532d833b 100644
@@ -1262,6 +1262,8 @@ mlxsw_sp_set_module_power_mode(struct net_device *dev,
 
 const struct ethtool_ops mlxsw_sp_port_ethtool_ops = {
 	.cap_link_lanes_supported	= true,
+	.op_needs_rtnl			= ETHTOOL_OP_NEEDS_RTNL_SPAUSEPARAM |
+					  ETHTOOL_OP_NEEDS_RTNL_GLINK,
 	.get_drvinfo			= mlxsw_sp_port_get_drvinfo,
 	.get_link			= ethtool_op_get_link,
 	.get_link_ext_state		= mlxsw_sp_port_get_link_ext_state,
Signature · 1 line
2.54.0
author maintainer
#3 lore raw

PATCH 3/3 mlxsw: Tell the core to use the netdev instance lock

In reply to Ido Schimmel

After the previous changes the driver is now ready to have its net device and ethtool operations invoked with the netdev instance lock held.

Tell the core about it by setting request_ops_lock to true.

---
 drivers/net/ethernet/mellanox/mlxsw/spectrum.c | 1 +
 1 file changed, 1 insertion(+)
drivers/net/ethernet/mellanox/mlxsw/spectrum.c +1 -0
index 3ee1272dcf0e..815e8d8e3185 100644
@@ -1552,6 +1552,7 @@ static int mlxsw_sp_port_create(struct mlxsw_sp *mlxsw_sp, u16 local_port,
 	dev->vlan_features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM;
 	dev->lltx = true;
 	dev->netns_immutable = true;
+	dev->request_ops_lock = true;
 
 	dev->min_mtu = ETH_MIN_MTU;
 	dev->max_mtu = MLXSW_PORT_MAX_MTU - MLXSW_PORT_ETH_FRAME_HDR;
Signature · 1 line
2.54.0
#4 lore raw
In reply to Ido Schimmel
Quoted 36 lines · Ido Schimmel

Make the driver ops-locked in order to allow ethtool operations to be invoked without RTNL being held.

An ops-locked driver has most of its NDOs, all of its ethtool operations and some net device notifications run with the netdev instance lock held.

In the specific case of mlxsw, the driver is not using any functions that acquire this lock nor functions that expect the lock to be held for an ops-locked driver. Therefore, converting its NDOs to run with the lock being held is trivial except for a small quirk which is handled in patch #1.

The driver does not generate any net device notifications, so there is no risk of nested notifications of the ops-locked types. For the notifications that run under the instance lock, RTNL is also held, and the driver does not acquire the instance lock itself, so no changes are required in its notifier handling.

Ethtool operations can be invoked without RTNL except for two operations that are annotated in patch #2.

Lastly, patch #3 converts the driver to be ops-locked.

A probe on rtnl_lock() shows it is no longer taken when dumping statistics:

 # perf probe --add rtnl_lock

Before:

 # perf stat -e probe:rtnl_lock -- ethtool -S swp1 --all-groups
 [...]
                  1      probe:rtnl_lock
Quoted 6 lines

After:

 # perf stat -e probe:rtnl_lock -- ethtool -S swp1 --all-groups
 [...]
                  0      probe:rtnl_lock

We probably want to add noinline_for_tracing to rtnl_lock() and friends...

#5 lore raw
In reply to Ido Schimmel
This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:

Quoted 8 lines · On Wed, 8 Jul 2026 15:39:30 +0300 you wrote:

Make the driver ops-locked in order to allow ethtool operations to be invoked without RTNL being held.

An ops-locked driver has most of its NDOs, all of its ethtool operations and some net device notifications run with the netdev instance lock held.

[...]

Here is the summary with links:
- [net-next,1/3] mlxsw: Convert to async version of ndo_set_rx_mode

    https://git.kernel.org/netdev/net-next/c/a5faaa079c61
  - [net-next,2/3] mlxsw: ethtool: Prepare for RTNL-less ethtool operations
    https://git.kernel.org/netdev/net-next/c/925a17fe4309
  - [net-next,3/3] mlxsw: Tell the core to use the netdev instance lock
    https://git.kernel.org/netdev/net-next/c/0501dd682648

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html