net: ethtool: let ops locked drivers run without rtnl_lock
Status from this message: This series was applied to netdev/net-next.git (main)
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v2 Latest12 patches 24 repliesApplied 12/12
- v114 patches 11 repliesSuperseded
Patches in v2 All 12 applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/12 | net: ethtool: serialize broadcast notification sequence allocationnet/ net/ net/ | +17 | 1 reply | Applied |
| 2/12 | net: ethtool: relax ethnl_req_get_phydev() locking assertiondrivers/ include/ net/ +2 more | +21 | 1 reply | Applied |
| 3/12 | net: ethtool: make dev->hwprov ops-protecteddrivers/ drivers/ include/ +3 more | +28 | 1 reply | Applied |
| 4/12 | net: ethtool: optionally skip rtnl_lock on Netlink path for GET opsdrivers/ drivers/ include/ +3 more | +85 | 1 reply | Applied |
| 5/12 | net: ethtool: optionally skip rtnl_lock on Netlink path for SET opsdrivers/ drivers/ drivers/ +9 more | +67 | 4 replies | Applied |
| 6/12 | net: ethtool: optionally skip rtnl_lock in cable test handlersnet/ | +4 | 2 replies | Applied |
| 7/12 | net: ethtool: optionally skip rtnl_lock in ethnl_tsinfo_dumpit()net/ | +15 | 1 reply | Applied |
| 8/12 | net: ethtool: optionally skip rtnl_lock in ethnl_act_module_fw_flash()include/ net/ | +3 | 1 reply | Applied |
| 9/12 | net: ethtool: optionally skip rtnl_lock in RSS context handlersnet/ | +6 | 1 reply | Applied |
| 10/12 | net: ethtool: ioctl: concentrate the lockingnet/ | +25 | 1 reply | Applied |
| 11/12 | net: ethtool: optionally skip rtnl_lock on IOCTL pathinclude/ net/ | +82 | 1 reply | Applied |
| 12/12 | docs: net: ethtool: document ops-locked drivers and op_needs_rtnlDocumentation/ Documentation/ include/ | +25 | 1 reply | Applied |
Reviews trailers per patch
All 12 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.
| Patch | Nicolai Buchwitz | Maxime Chevallier | Stanislav Fomichev | Jacob Keller | Eric Dumazet |
|---|---|---|---|---|---|
| Series | A-b | R-b | |||
| 1/12 | R-b | ||||
| 2/12 | R-b | R-b | R-b | ||
| 3/12 | R-b | ||||
| 4/12 | R-b | ||||
| 5/12 | R-b | ||||
| 6/12 | R-b | R-b | |||
| 7/12 | R-b | ||||
| 8/12 | R-b | ||||
| 9/12 | R-b | ||||
| 10/12 | R-b | ||||
| 11/12 | R-b | ||||
| 12/12 | R-b | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 5 reviewers.
Cover letter [PATCH net-next v2 00/12] net: ethtool: let ops locked drivers run without rtnl_lock
With the ethtool_get_link_ksettings() situation hopefully ironed out the previous series (commit 6a5d837f0ce2) let's return to the main part of the series.
We have been slowly moving towards removing the rtnl_lock dependency in driver ops since the concept of "ops-locked" drivers have been introduced last year. Since last year will take the netdev instance lock before invoking any ndo or ethtool op of "ops-locked" drivers.
We dipped our toes into rtnl_lock-less ops with the queue binding API. Queue stats, NAPI, and other netdev-netlink objects are also queried without holding rtnl_lock already. It's time to take the next logical step and lift the requirement from ethtool ops.
The direct motivation for this patchset is that ethtool ops often involve communicating with device FW, and may take a long time to complete. Aggressive polling of device state on machines with 10+ NICs have been shown to significantly increase rtnl_lock pressure.
There's a handful of areas which still need rtnl_lock (see below). I decided to convert everything to rtnl_lock-less by default, and add a set of flags which let the drivers request rtnl_lock to still be taken. I don't love this, but I'm worried that opt-in would be even more confusing.
Known issues / exclusions:
- qdiscs - qdisc configuration currently assumes rtnl_lock, this is mostly impacting set_channels callback. qdisc config is probably the easiest one of the exclusions to tackle, it's fairly self-contained.
- features - even tho feature changes are (correctly) plumbed to the driver thru ndos they are part of ethtool uAPI. ethtool itself calls netdev_features_change() if it has spotted device feature change before vs after to the callback. Some drivers also call netdev_features_change() directly in response to various changes, e.g. setting priv flags.
Since features have to propagate to upper and lower devices anything that touches features is quite hard to move from under rtnl_lock.
- phylink - phylink and SFP depend on rtnl_lock today, I suspect that this is purely for historic reasons. I started poking at it and don't really see a need for a global lock. But accessing the netdev instance lock from the SFP entry points will require some attention from the phylink folks.
- phydev - similar to phylink, looks quite doable. But no ops-locked driver currently has a phydev (fbnic only uses phylink) so phydev related paths retain a ASSERT_RTNL() for now.
Tested on mlx5, bnxt and fbnic.
Jakub Kicinski (12):
net: ethtool: serialize broadcast notification sequence allocation
net: ethtool: relax ethnl_req_get_phydev() locking assertion
net: ethtool: make dev->hwprov ops-protected
net: ethtool: optionally skip rtnl_lock on Netlink path for GET ops
net: ethtool: optionally skip rtnl_lock on Netlink path for SET ops
net: ethtool: optionally skip rtnl_lock in cable test handlers
net: ethtool: optionally skip rtnl_lock in ethnl_tsinfo_dumpit()
net: ethtool: optionally skip rtnl_lock in ethnl_act_module_fw_flash()
net: ethtool: optionally skip rtnl_lock in RSS context handlers
net: ethtool: ioctl: concentrate the locking
net: ethtool: optionally skip rtnl_lock on IOCTL path
docs: net: ethtool: document ops-locked drivers and op_needs_rtnl
Documentation/networking/netdev-features.rst | 7 + Documentation/networking/netdevices.rst | 17 ++- include/linux/ethtool.h | 36 ++++- include/linux/netdevice.h | 3 + include/linux/phy_link_topology.h | 5 + include/net/netdev_lock.h | 11 ++ net/ethtool/common.h | 76 +++++++++++ net/ethtool/netlink.h | 8 +- .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 4 + drivers/net/ethernet/google/gve/gve_ethtool.c | 6 +- .../ethernet/mellanox/mlx5/core/en_ethtool.c | 3 + .../net/ethernet/mellanox/mlx5/core/en_rep.c | 2 + .../mellanox/mlx5/core/ipoib/ethtool.c | 2 + .../net/ethernet/meta/fbnic/fbnic_ethtool.c | 5 + .../ethernet/microsoft/mana/mana_ethtool.c | 2 + drivers/net/netdevsim/ethtool.c | 1 + drivers/net/phy/phy_device.c | 3 + drivers/net/phy/phy_link_topology.c | 10 ++ net/core/dev_ioctl.c | 4 +- net/ethtool/cabletest.c | 12 +- net/ethtool/ioctl.c | 123 ++++++++++++++---- net/ethtool/mm.c | 5 +- net/ethtool/module.c | 6 +- net/ethtool/netlink.c | 62 ++++++--- net/ethtool/phy.c | 1 - net/ethtool/rss.c | 21 +-- net/ethtool/tsconfig.c | 10 +- net/ethtool/tsinfo.c | 32 +++-- 28 files changed, 363 insertions(+), 114 deletions(-)
Signature · 1 line
2.54.0