Skip to contentSkip to conversation

net/mlx5: LAG, preparations for LAG over VFs/SFs

PATCH In review Tariq Toukan 4 participants 32 messages Nov 16 – Nov 19

Subsystems (5)
lore
Share & tools
lore

Tariq Toukan

author maintainer
#0 lore raw

v2 cover net/mlx5: LAG, preparations for LAG over VFs/SFs

Hi,

This series by Shay is the first part of enabling a LAG over functions that do not represent a physical port - a "virtual LAG" (virt LAG) of VFs or SFs acting as nested e-switch managers. New FW lets a VF/SF front the physical uplinks through a multi-uplink e-switch; such a function reports num_lag_ports < 2 and drives port selection via the e-switch (port_select_eswitch) rather than the HW port-selection flow table. A virt LAG is therefore a software-managed, shared-FDB MPESW LAG: the driver builds the LAG demux and vport-LAG in software.

This part prepares the LAG core. A follow-up series adds the e-switch and representor support and the patch that admits a virt LAG; until then no ldev is marked as a virt LAG and the virt LAG paths added here are unused.

The patches fall into two parts.

LAG code cleanup (patches 5, 6, 7, 8, 9)
- Finish removing the fixed MLX5_MAX_PORTS sizing the first series
began: convert the remaining structures to dynamic, port-count-based allocation and inline iteration. A virt LAG's member count is unrelated to the physical port count.
- Rename the LAG "pf" naming to "fn", since a member is now any function (PF, VF or SF), not necessarily a PF.
- Resolve the queue-affinity member indices only in that mode, so the other port-select modes are not rejected for indices they never use.

Virtual LAG core (patches 1, 2, 3, 4, 10)
- Distinguish VFs/SFs from their PF in the SW system image GUID, so they group into their own LAG instead of with the PF.
- Select the MPESW master from the live member snapshot: VF/SF members are dynamic and not marked by the devcom PAIR event, so the master is chosen from the current membership (tie-broken by PCI BDF for VFs, sfnum for SFs).
- Skip the phys-LAG prerequisites (port_select_flow_table, create_lag_when_not_master_up, ...) for a virt LAG, and require port_select_eswitch instead.
- Treat a virt LAG as a software-managed LAG (demux + vport-LAG in

    SW).

- Do not add a device to an already-active virt LAG.

Regards,
Tariq

V2:
- Split the patch set to ease the review: this is v1 patches 1-10; v1 patches 11-15 will be sent as a follow-up series.
- Patch 1: define mlx5_virt_lag_is_supported() in include/linux/mlx5/driver.h, so mlx5_ib can use it in the follow-up series.
- Patch 3: fix the commit message: a LAG of VFs/SFs still issues CREATE_LAG, but a VF/SF issuing it does not create a LAG of physical ports (Sashiko).
- Patch 10: retitled, and the commit message says the rejection applies only to a virt LAG, and why the other join paths are safe (Sashiko).

V1:
https://lore.kernel.org/all/[email protected]/

Answers to sashiko comments on v1:

net/mlx5: Distinguish VFs/SFs in the SW system image GUID for LAG grouping

Q: A VF/SF that reports multi_uplink_eswitch, lag_master and num_lag_ports < 2 now gets a key that differs from its PF's key, and mlx5_same_hw_devs() requires the keys to match exactly. Does this break PF to VF/SF same-HW detection, e.g. mlx5e_tc_is_vf_tunnel() and mirred offload in NIC mode? Should mlx5_same_hw_devs() use a key without the VF/SF markers?

[SD] No, this is the point of the patch. A VF/SF that reports multi_uplink_eswitch and lag_master (and num_lag_ports < 2) is a nested eswitch manager. That makes it a different switch domain from its PF, not one more vport of the PF's FDB, so it must not be identified as the same HW as the PF. Without the marker it would also be grouped into the PF's LAG, which is what the patch prevents.
A VF/SF that does not report these caps, i.e. every VF/SF that exists today, keeps its key unchanged and is still the same HW as its PF, as the commit message says. Dropping the markers inside mlx5_same_hw_devs() would make a nested eswitch manager the same HW as its PF again, i.e. undo the patch.

net/mlx5: LAG, skip phys LAG prerequisites for a virt LAG

Review

Quoted 2 lines
(quoted diff) +2 -0
+	if (mlx5_eswitch_mode(dev0) != MLX5_ESWITCH_OFFLOADS ||
+	    !MLX5_CAP_PORT_SELECTION(dev0, port_select_eswitch) ||

Q: mlx5_virt_lag_is_supported() does not check port_select_eswitch. A VF/SF that reports multi_uplink_eswitch and lag_master but not port_select_eswitch would get the VF/SF marker in its SW GUID and join a virt ldev, yet MPESW could never be enabled on it. Should port_select_eswitch be checked in mlx5_virt_lag_is_supported() instead, so the two checks agree?

[SD] The two checks are at different stages on purpose. A function with the first two and without the third doesn't exist.
Even if it were, the result is benign: the function stays an inactive member and esw_multiport fails cleanly with -EOPNOTSUPP, nothing else changes.

net/mlx5: LAG, treat virt LAG as a software managed LAG

Review

Quoted 5 lines
(quoted diff) +2 -2
-static int mlx5_lag_is_sw_lag(struct mlx5_core_dev *dev)
+int mlx5_lag_is_sw_managed(struct mlx5_core_dev *dev)
 {
-	return mlx5_lag_is_sd(dev);
+	struct mlx5_lag *ldev = mlx5_lag_dev(dev);

Q: If dev is NULL, does mlx5_lag_dev(dev) NULL-deref dev->priv.lag? When called from mlx5_lag_multipath_check_prereq() via the FIB notifier, pf->dev can be NULL due to concurrent removal in mlx5_ldev_remove_mdev(). __mlx5_lag_is_sd() was safe (lockless pointer compare). Does this need a NULL check?

[SD] mlx5_lag_is_sw_managed() assumes a valid dev pointer, like the rest of the mlx5_lag_is_* APIs. It is true mlx5_lag_multipath_check_prereq() can provide an invalid device pointer, but that is an existing bug there which should be handled in its own patch.

net/mlx5: LAG, decouple iteration end-of-walk from MLX5_MAX_PORTS

Review

Quoted 3 lines
(quoted diff) +1 -1
-		if (i < MLX5_MAX_PORTS) {
+		if (i < ldev->ports) {
 			slave = bond_slave_get_rcu(ndev_tmp);

Q: Should this compare against ldev->max_funcs, like the other converted call sites? A PORT-marked member can sit at an index in [ldev->ports, max_funcs) if more PFs than num_lag_ports join the ldev and a lower-index one is removed later. Physical ports also share the xarray with VFs/SFs. Wouldn't that leave the bond silently not offloaded?

[SD] No. mlx5 creates an ldev per sw_guid, and PFs have a different sw_guid than VFs/SFs, so only PFs share this ldev. The only case with more PFs than num_lag_ports in one ldev is SD, and for SD the netdev notifier, and with it mlx5_handle_changeupper_event(), is not registered. So a PORT-marked member here always sits at an index below ldev->ports.

Review

Quoted 2 lines
(quoted diff) +1 -1
-	err = xa_alloc(&ldev->pfs, &idx, pf, XA_LIMIT(0, MLX5_MAX_PORTS - 1),
+	err = xa_alloc(&ldev->pfs, &idx, pf, XA_LIMIT(0, U8_MAX - 1),

Q: Physical members and Socket Direct secondaries share the xarray index space. Can a physical member get an index >= MLX5_MAX_PORTS and make mlx5_ldev_add_netdev() write beyond tracker.netdev_state[]?

[SD] Such HW doesn't exist; it needs LAG over SD with more than MLX5_MAX_PORTS / 2 ports, and when it does MLX5_MAX_PORTS will be increased. To protect a new HW running an old driver, a check that num_lag_ports times the SD group size fits in MLX5_MAX_PORTS will be added to mlx5_lag_is_supported() separately.

Q: Can removing the xa_alloc() limit let member indices reach 8 or more while pf_metadata[], devs[], the cong counter array, tracker.netdev_state[] and the port_sel rules[] are still sized MLX5_MAX_PORTS? Could the limit stay at MLX5_MAX_PORTS - 1 until all of these users are converted?

[SD] No. In this series a 9th member is impossible: an ldev holds only PFs and SD secondaries, and no HW has more than MLX5_MAX_PORTS of them. VFs/SFs join an ldev only in the follow-up series.
The limit can't be kept until all arrays are converted, since not all of them will be. Some arrays are used only by a PF LAG, others by both a PF LAG and a virt LAG. The shared ones, the fns xarray included, are converted in this series (pf_metadata[] and devs[]; the cong counter array is converted as well). The PF LAG only ones, tracker.netdev_state[] and the port_sel rules[], keep their static MLX5_MAX_PORTS size: they are walked only over PORT-marked members, in the hash and queue affinity modes. A virt member is never PORT-marked and sits in its own ldev, and a virt LAG supports only MPESW, so these arrays never see more than MLX5_MAX_PORTS entries.

Review

Quoted 3 lines
(quoted diff) +3 -0
+	/* max_funcs is the iteration ceiling; grow it, never shrink. */
+	if (idx >= ldev->max_funcs)
+		ldev->max_funcs = idx + 1;

Q: Is this store safe against readers that only hold lag_lock? mlx5_lag_get_next_peer_mdev(), mlx5_lag_get_slave_port() and mlx5_lag_query_cong_counters() read max_funcs more than once per walk, so a concurrent add could make the sentinel and the loop condition disagree, and the body could see a pf whose dev is not set yet.

[SD] all the readers above running only while LAG is active, and mlx5_lag_query_cong_counters() and mlx5_lag_get_slave_port() check whether lag is active explicitly before walking over the lag

Review

Quoted 2 lines
(quoted diff) +1 -1
-								   ldev->ports);
+								   ldev->max_funcs);

Q: The commit message says "No functional change". Is that accurate? max_slaves sizes the master's egress ACL, so the size of the hardware flow table changes whenever max_funcs != ports.

[SD] Only the ACL size changes, and only for SD, where more PFs than ports join the ldev and max_funcs > ports. The rules installed are the same as before, so there is no functional change.

Q: end_flow_index is inclusive, so the master egress group spans max_slaves + 1 entries in a table created with max_fte = max_slaves. If max_slaves equals a flow table pool size, e.g. 128, can the group creation fail?

[SD] This is theoretical only; there is no such use case. In this series max_slaves is at most MLX5_MAX_PORTS, and the table is rounded up to 128 entries, so the group fits.

Q: The patch also changes the xa_alloc() limit in mlx5_ldev_add_mdev(), which now accepts a 9th member, and the found check in mlx5_handle_changeupper_event(), which now uses i < ldev->ports. Could the commit message describe these changes?

[SD] The xa_alloc() change is already in the commit message ("remove the xa_alloc() limit of MLX5_MAX_PORTS"), and the answer above explains why it can't let in a 9th member. mlx5_handle_changeupper_event() runs only for a PF LAG without SD: the netdev notifier is registered only when !mlx5_sd_is_supported(). There each LAG port is one PF at an xarray index below ldev->ports (num_lag_ports), so "i < ldev->ports" is true in exactly the same cases as the old end-of-walk check.

net/mlx5: LAG: replace fixed MLX5_MAX_PORTS arrays with dynamic allocation

Review

Quoted 3 lines
(quoted diff) +3 -0
+	ldev = mlx5_lag_dev(dev);
+	num_ports = ldev ? max_t(int, ldev->ports, 1) : 1;
+	mdev = kcalloc(num_ports, sizeof(mdev[0]), GFP_KERNEL);

Q: Could mlx5_lag_query_cong_counters() keep a freed ldev across this sleeping allocation if teardown removes the association and drops the last LAG reference before the query resumes?

[SD] No. The query runs on the IB device of dev, and dev leaves the LAG only in its own teardown, after its IB device is unregistered.

Q: Can this loop write past the end of mdev[]? The array has max(ldev->ports, 1) entries, mlx5_ldev_for_each() writes one entry per PORT-marked member, and nothing caps the PORT-marked members at ldev->ports. Should the loop break once idx >= num_ports?

[SD] It can't. The loop runs only for a non-virt LAG, and the PORT-marked members of such an ldev are the PFs of one card, at most num_lag_ports == ldev->ports of them (an SD secondary is not PORT-marked and a virt member is excluded by the !ldev->virt_lag check right here).

Q: Can mlx5_mpesw_metadata_cleanup() index past the end of pf_metadata[]? The buffer is sized by ldev->max_fns when MPESW is enabled, but sd_lag_init() can add SD secondaries afterwards and grow max_fns, and the cleanup loop stops at the current max_fns.

[SD] this is pre-existing bug, MPESW should failed if SD Secondary is down. this will be handle in separate patch.

net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan

Review

Quoted 4 lines
(quoted diff) +4 -0
+	mlx5_lag_for_each(i, 0, ldev, filter) {
+		pf = mlx5_lag_pf(ldev, i);
+		...
+		mlx5_rescan_drivers_locked(pf->dev);

Q: Is it safe to iterate the LAG PFs and deref pf->dev while ldev->lock is dropped?

[SD] Valid. Membership can be removed by SD cleanup while ldev->lock is dropped; this will be fixed in a separate patch.

Q: Does using the sparse XArray index directly as a dense-array index cause OOB writes elsewhere? With Socket Direct the XArray index (i) for a primary port can exceed ldev->ports, and it is used directly in mlx5_infer_tx_affinity_mapping() (ports[idx]) and mlx5_handle_changelowerstate_event() (tracker->netdev_state[idx]) - could these overflow the heap/stack buffers?

[SD] All these buffers aren't used in LAG over SD; only MPESW is, and MPESW doesn't use these APIs.

net/mlx5: LAG, do not add a device to an already-active virt LAG

Q: After this change only -EOPNOTSUPP is final. Every other error from __mlx5_lag_dev_add_mdev() is retried with no limit. xa_alloc() returns -EBUSY once all 255 slots are in use, and membership does not change while the probe waits. Can mlx5_load() loop forever? Would it make sense to retry only on -EAGAIN?

[SD] This is pre-existing: mlx5_lag_add_mdev() retried every error before this series as well. for example future HW can return -EBUSY. The fix will be sent as a separate patch.

Shay Drory (10):
net/mlx5: Distinguish VFs/SFs in the SW system image GUID for LAG
grouping
net/mlx5: LAG, select the MPESW master from the live member snapshot net/mlx5: LAG, skip phys LAG prerequisites for a virt LAG net/mlx5: LAG, treat virt LAG as a software managed LAG net/mlx5: LAG, decouple iteration end-of-walk from MLX5_MAX_PORTS net/mlx5: LAG, rename PF naming to fn
net/mlx5: LAG, resolve affinity indices only in queue-affinity mode net/mlx5: LAG: replace fixed MLX5_MAX_PORTS arrays with dynamic
allocation
net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan net/mlx5: LAG, do not add a device to an already-active virt LAG

 .../mellanox/mlx5/core/eswitch_offloads.c     |   4 +
 .../ethernet/mellanox/mlx5/core/lag/debugfs.c |   2 +-
 .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 568 ++++++++++--------
 .../net/ethernet/mellanox/mlx5/core/lag/lag.h |  62 +-
 .../net/ethernet/mellanox/mlx5/core/lag/mp.c  |  29 +-
 .../ethernet/mellanox/mlx5/core/lag/mpesw.c   | 125 ++--
 .../ethernet/mellanox/mlx5/core/lag/mpesw.h   |   2 +-
 .../mellanox/mlx5/core/lag/port_sel.c         |  25 +-
 .../mellanox/mlx5/core/lag/shared_fdb.c       |  44 +-
 .../net/ethernet/mellanox/mlx5/core/lib/sd.c  |  32 +-
 .../ethernet/mellanox/mlx5/core/sf/dev/dev.h  |  12 +
 .../net/ethernet/mellanox/mlx5/core/vport.c   |  40 +-
 include/linux/mlx5/driver.h                   |  11 +-
 13 files changed, 573 insertions(+), 383 deletions(-)

base-commit: 7ccc5e05043b1bb84326b19a2b05938d8f7a2ae6

Signature · 1 line
2.44.0
author maintainer
#1 lore raw

PATCH v2 1/10 net/mlx5: Distinguish VFs/SFs in the SW system image GUID for LAG grouping

From: Shay Drory <[email protected]>

mlx5_query_nic_sw_system_image_guid() builds the key used to group devices that may form a LAG. A PF and its VFs/SFs share the same FW system image GUID, so without a discriminator a VF/SF that can join a virt LAG would be grouped with its PF.

Appending bare discriminator bytes at conditional offsets is not enough:
distinct capability sets can collapse to the same key (a PF ending in a load_balance byte of 0x01 vs a VF whose marker is 0x01). Give each discriminator a fixed position instead, and add the VF and SF as such fields. Trailing zero fields are trimmed, so keys of devices that set none of the new fields are unchanged.

Set the VF/SF markers only when mlx5_virt_lag_is_supported(). The same key also backs mlx5_same_hw_devs(), so a VF/SF that cannot join a virt LAG keeps its current key and is still identified as the same HW as its PF.

---
 .../net/ethernet/mellanox/mlx5/core/vport.c   | 40 +++++++++++++++++--
 include/linux/mlx5/driver.h                   | 11 ++++-
 2 files changed, 47 insertions(+), 4 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/vport.c +37 -3
index 78f3d15b599d..bab43df6a7d7 100644
@@ -1308,10 +1308,37 @@ u64 mlx5_query_nic_system_image_guid(struct mlx5_core_dev *mdev)
 }
 EXPORT_SYMBOL_GPL(mlx5_query_nic_system_image_guid);
 
+static u8 mlx5_sw_guid_load_balance(const struct mlx5_core_dev *mdev)
+{
+	if (MLX5_CAP_GEN_2(mdev, load_balance_id) &&
+	    MLX5_CAP_GEN_2(mdev, lag_per_mp_group))
+		return MLX5_CAP_GEN_2(mdev, load_balance_id);
+	return 0;
+}
+
+static u8 mlx5_sw_guid_sf(const struct mlx5_core_dev *mdev)
+{
+	return mlx5_core_is_sf(mdev) && mlx5_virt_lag_is_supported(mdev);
+}
+
+static u8 mlx5_sw_guid_vf(const struct mlx5_core_dev *mdev)
+{
+	return mlx5_core_is_vf(mdev) && mlx5_virt_lag_is_supported(mdev);
+}
+
+static u8 (* const mlx5_sw_guid_fields[])(const struct mlx5_core_dev *mdev) = {
+	mlx5_sw_guid_load_balance,
+	mlx5_sw_guid_vf,
+	mlx5_sw_guid_sf,
+	/* New fields should be added at the end */
+};
+
 void mlx5_query_nic_sw_system_image_guid(struct mlx5_core_dev *mdev, u8 *buf,
 					 u8 *len)
 {
 	u64 fw_system_image_guid;
+	u8 trim_len;
+	int i;
 
 	*len = 0;
 
@@ -1321,10 +1348,17 @@ void mlx5_query_nic_sw_system_image_guid(struct mlx5_core_dev *mdev, u8 *buf,
 
 	memcpy(buf, &fw_system_image_guid, sizeof(fw_system_image_guid));
 	*len += sizeof(fw_system_image_guid);
+	trim_len = *len;
 
-	if (MLX5_CAP_GEN_2(mdev, load_balance_id) &&
-	    MLX5_CAP_GEN_2(mdev, lag_per_mp_group))
-		buf[(*len)++] = MLX5_CAP_GEN_2(mdev, load_balance_id);
+	for (i = 0; i < ARRAY_SIZE(mlx5_sw_guid_fields); i++) {
+		u8 val = mlx5_sw_guid_fields[i](mdev);
+
+		buf[(*len)++] = val;
+		if (val)
+			trim_len = *len;
+	}
+
+	*len = trim_len;
 }
 
 bool mlx5_vport_use_vhca_id_as_func_id(struct mlx5_core_dev *dev,
include/linux/mlx5/driver.h +10 -1
index 84e9f6bdb7c1..e28b9c23c165 100644
@@ -1270,6 +1270,15 @@ static inline int mlx5_lag_is_lacp_owner(struct mlx5_core_dev *dev)
 		    MLX5_CAP_GEN(dev, lag_master);
 }
 
+static inline bool mlx5_virt_lag_is_supported(const struct mlx5_core_dev *dev)
+{
+	if (!MLX5_CAP_GEN(dev, multi_uplink_eswitch) ||
+	    !MLX5_CAP_GEN(dev, lag_master) ||
+	    MLX5_CAP_GEN(dev, num_lag_ports) >= 2)
+		return false;
+	return true;
+}
+
 static inline u16 mlx5_core_max_ec_vfs(const struct mlx5_core_dev *dev)
 {
 	return dev->priv.sriov.max_ec_vfs;
@@ -1399,7 +1408,7 @@ static inline struct net *mlx5_core_net(struct mlx5_core_dev *dev)
 	return devlink_net(priv_to_devlink(dev));
 }
 
-#define MLX5_SW_IMAGE_GUID_MAX_BYTES 9
+#define MLX5_SW_IMAGE_GUID_MAX_BYTES 11
 
 static inline bool mlx5_data_direct_supported(struct mlx5_core_dev *mdev)
 {
Signature · 1 line
2.44.0
#2 lore raw

Re: [PATCH net-next V2 01/10] net/mlx5: Distinguish VFs/SFs in the SW system image GUID for LAG grouping

Sashiko has reviewed this patch and found no issues.
Quoted 19 lines

From: Shay Drory <[email protected]>

mlx5_query_nic_sw_system_image_guid() builds the key used to group devices that may form a LAG. A PF and its VFs/SFs share the same FW system image GUID, so without a discriminator a VF/SF that can join a virt LAG would be grouped with its PF.

Appending bare discriminator bytes at conditional offsets is not enough:
distinct capability sets can collapse to the same key (a PF ending in a load_balance byte of 0x01 vs a VF whose marker is 0x01). Give each discriminator a fixed position instead, and add the VF and SF as such fields. Trailing zero fields are trimmed, so keys of devices that set none of the new fields are unchanged.

Set the VF/SF markers only when mlx5_virt_lag_is_supported(). The same key also backs mlx5_same_hw_devs(), so a VF/SF that cannot join a virt LAG keeps its current key and is still identified as the same HW as its PF.

Quoted 4 lines

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

Re: [PATCH net-next V2 01/10] net/mlx5: Distinguish VFs/SFs in the SW system image GUID for LAG grouping

In reply to Tariq Toukan
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.

Critical: 0 · High: 0 · Medium: 1 · Low: 0

If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).

- [Medium] The patch changes the SW system image GUID of a VF/SF that passes mlx5_virt_lag_is_supported(): mlx5_sw_guid_vf() or…

--

net/mlx5: Distinguish VFs/SFs in the SW system image GUID for LAG grouping

This patch changes mlx5_query_nic_sw_system_image_guid() so that each discriminator byte after the FW system image GUID has a fixed position, and trailing zero fields are trimmed. It adds VF and SF marker bytes that are only set when mlx5_virt_lag_is_supported() is true, and it raises MLX5_SW_IMAGE_GUID_MAX_BYTES from 9 to 11.

Quoted 15 lines
drivers/net/ethernet/mellanox/mlx5/core/vport.c +7 -0
index 78f3d15b599d0..bab43df6a7d72 100644
@@ -1308,10 +1308,37 @@ u64 mlx5_query_nic_system_image_guid(struct mlx5_core_dev *mdev)
 }
 EXPORT_SYMBOL_GPL(mlx5_query_nic_system_image_guid);
 
+static u8 mlx5_sw_guid_load_balance(const struct mlx5_core_dev *mdev)
+{
+	if (MLX5_CAP_GEN_2(mdev, load_balance_id) &&
+	    MLX5_CAP_GEN_2(mdev, lag_per_mp_group))
+		return MLX5_CAP_GEN_2(mdev, load_balance_id);
+	return 0;
+}

drivers/net/ethernet/mellanox/mlx5/core/vport.c u64 mlx5_query_nic_system_image_guid(struct mlx5_core_dev *mdev) · line 1327

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/vport.c +10 -0
+
+static u8 mlx5_sw_guid_sf(const struct mlx5_core_dev *mdev)
+{
+	return mlx5_core_is_sf(mdev) && mlx5_virt_lag_is_supported(mdev);
+}
+
+static u8 mlx5_sw_guid_vf(const struct mlx5_core_dev *mdev)
+{
+	return mlx5_core_is_vf(mdev) && mlx5_virt_lag_is_supported(mdev);
+}

[Severity: Medium]
Does this break the same-HW check between a PF and a VF/SF that passes mlx5_virt_lag_is_supported()?

Suppose the VF/SF's own caps report multi_uplink_eswitch=1, lag_master=1 and num_lag_ports < 2. The VF key then becomes guid + [lb|0] + 0x01, which is 10 bytes. The SF key becomes guid + [lb|0] + 0x00 + 0x01, which is 11 bytes. A PF never sets either marker, so its key is still 8 or 9 bytes.

mlx5_same_hw_devs() requires the lengths to match:

drivers/net/ethernet/mellanox/mlx5/core/dev.c:mlx5_same_hw_devs() {
    ...
	return plen && flen && flen == plen &&
		!memcmp(fsystem_guid, psystem_guid, flen);
}

That means it now returns false for any PF paired with such a VF/SF.

mlx5e_tc_is_vf_tunnel() only reaches that comparison when out_mdev is a PF and route_mdev is a VF/SF:

drivers/net/ethernet/mellanox/mlx5/core/en_tc.c:mlx5e_tc_is_vf_tunnel() {
    ...
	if (route_mdev->coredev_type != MLX5_COREDEV_VF &&
	    route_mdev->coredev_type != MLX5_COREDEV_SF)
		return false;

	return mlx5e_same_hw_devs(out_priv, route_priv);
}

For a virt-LAG-capable VF/SF route device, will it now always return false? These are the paths that would be affected:

mlx5e_set_vf_tunnel()
mlx5e_tc_is_vf_tunnel()

    mlx5e_same_hw_devs()
      mlx5_same_hw_devs()

- In mlx5e_set_vf_tunnel() the check fails and the code jumps to out with err = 0. The encap flow is still offloaded, but without the VF vport source port rewrite (MLX5_ESW_DEST_CHAIN_WITH_SRC_PORT_CHANGE plus the VPORT_TO_REG metadata). The encapsulated packet then goes straight to the uplink and skips FDB processing of the VF vport.

- mlx5e_update_vf_tunnel() returns -ENODEV on route updates.

- On decap, mlx5e_tc_tun_route_lookup() no longer sets rx_tun_attr->decap_vport to the VF vport.

- In mirred_nic.c, the NIC-mode redirect check

	if (priv->netdev->netdev_ops != out_dev->netdev_ops ||
	    !mlx5e_same_hw_devs(priv, netdev_priv(out_dev))) {

now rejects redirects between a PF netdev and such a VF/SF netdev with "devices are not on same switch HW".

The commit message says:

"so a VF/SF that cannot join a virt LAG keeps its current key and is still identified as the same HW as its PF."

That covers VFs/SFs that cannot join a virt LAG. It doesn't cover the ones that can, and none of the consumers above are updated.

Should mlx5_same_hw_devs() compare without the VF/SF markers? Or should the LAG grouping key be kept separate from the key used for same-HW detection?

Quoted 7 lines
drivers/net/ethernet/mellanox/mlx5/core/vport.c +7 -0
+
+static u8 (* const mlx5_sw_guid_fields[])(const struct mlx5_core_dev *mdev) = {
+	mlx5_sw_guid_load_balance,
+	mlx5_sw_guid_vf,
+	mlx5_sw_guid_sf,
+	/* New fields should be added at the end */
+};

[ ... ]

Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com
#4 lore raw

Re: [PATCH net-next V2 01/10] net/mlx5: Distinguish VFs/SFs in the SW system image GUID for LAG grouping

Quoted 113 lines · On Wed, 18 Nov 2026 10:31:52 +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.

Critical: 0 · High: 0 · Medium: 1 · Low: 0

If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).

- [Medium] The patch changes the SW system image GUID of a VF/SF that passes mlx5_virt_lag_is_supported(): mlx5_sw_guid_vf() or…

--

net/mlx5: Distinguish VFs/SFs in the SW system image GUID for LAG grouping

This patch changes mlx5_query_nic_sw_system_image_guid() so that each discriminator byte after the FW system image GUID has a fixed position, and trailing zero fields are trimmed. It adds VF and SF marker bytes that are only set when mlx5_virt_lag_is_supported() is true, and it raises MLX5_SW_IMAGE_GUID_MAX_BYTES from 9 to 11.

Quoted 15 lines
drivers/net/ethernet/mellanox/mlx5/core/vport.c +7 -0
index 78f3d15b599d0..bab43df6a7d72 100644
@@ -1308,10 +1308,37 @@ u64 mlx5_query_nic_system_image_guid(struct mlx5_core_dev *mdev)
 }
 EXPORT_SYMBOL_GPL(mlx5_query_nic_system_image_guid);
 
+static u8 mlx5_sw_guid_load_balance(const struct mlx5_core_dev *mdev)
+{
+	if (MLX5_CAP_GEN_2(mdev, load_balance_id) &&
+	    MLX5_CAP_GEN_2(mdev, lag_per_mp_group))
+		return MLX5_CAP_GEN_2(mdev, load_balance_id);
+	return 0;
+}

drivers/net/ethernet/mellanox/mlx5/core/vport.c u64 mlx5_query_nic_system_image_guid(struct mlx5_core_dev *mdev) · line 1327

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/vport.c +10 -0
+
+static u8 mlx5_sw_guid_sf(const struct mlx5_core_dev *mdev)
+{
+	return mlx5_core_is_sf(mdev) && mlx5_virt_lag_is_supported(mdev);
+}
+
+static u8 mlx5_sw_guid_vf(const struct mlx5_core_dev *mdev)
+{
+	return mlx5_core_is_vf(mdev) && mlx5_virt_lag_is_supported(mdev);
+}

[Severity: Medium]
Does this break the same-HW check between a PF and a VF/SF that passes mlx5_virt_lag_is_supported()?

Suppose the VF/SF's own caps report multi_uplink_eswitch=1, lag_master=1 and num_lag_ports < 2. The VF key then becomes guid + [lb|0] + 0x01, which is 10 bytes. The SF key becomes guid + [lb|0] + 0x00 + 0x01, which is 11 bytes. A PF never sets either marker, so its key is still 8 or 9 bytes.

mlx5_same_hw_devs() requires the lengths to match:

drivers/net/ethernet/mellanox/mlx5/core/dev.c:mlx5_same_hw_devs() {
    ...
	return plen && flen && flen == plen &&
		!memcmp(fsystem_guid, psystem_guid, flen);
}

That means it now returns false for any PF paired with such a VF/SF.

mlx5e_tc_is_vf_tunnel() only reaches that comparison when out_mdev is a PF and route_mdev is a VF/SF:

drivers/net/ethernet/mellanox/mlx5/core/en_tc.c:mlx5e_tc_is_vf_tunnel() {
    ...
	if (route_mdev->coredev_type != MLX5_COREDEV_VF &&
	    route_mdev->coredev_type != MLX5_COREDEV_SF)
		return false;

	return mlx5e_same_hw_devs(out_priv, route_priv);
}

For a virt-LAG-capable VF/SF route device, will it now always return false? These are the paths that would be affected:

mlx5e_set_vf_tunnel()
mlx5e_tc_is_vf_tunnel()

    mlx5e_same_hw_devs()
      mlx5_same_hw_devs()

- In mlx5e_set_vf_tunnel() the check fails and the code jumps to out with err = 0. The encap flow is still offloaded, but without the VF vport source port rewrite (MLX5_ESW_DEST_CHAIN_WITH_SRC_PORT_CHANGE plus the VPORT_TO_REG metadata). The encapsulated packet then goes straight to the uplink and skips FDB processing of the VF vport.

- mlx5e_update_vf_tunnel() returns -ENODEV on route updates.

- On decap, mlx5e_tc_tun_route_lookup() no longer sets rx_tun_attr->decap_vport to the VF vport.

- In mirred_nic.c, the NIC-mode redirect check

	if (priv->netdev->netdev_ops != out_dev->netdev_ops ||
	    !mlx5e_same_hw_devs(priv, netdev_priv(out_dev))) {

now rejects redirects between a PF netdev and such a VF/SF netdev with "devices are not on same switch HW".

The commit message says:

"so a VF/SF that cannot join a virt LAG keeps its current key and is still identified as the same HW as its PF."

Quoted 6 lines

That covers VFs/SFs that cannot join a virt LAG. It doesn't cover the ones that can, and none of the consumers above are updated.

Should mlx5_same_hw_devs() compare without the VF/SF markers? Or should the LAG grouping key be kept separate from the key used for same-HW detection?

No, this is the point of the patch. A VF/SF that reports multi_uplink_eswitch and lag_master (and num_lag_ports < 2) is a nested eswitch manager. That makes it a different switch domain from its PF, not one more vport of the PF's FDB, so it must not be identified as the same HW as the PF. Without the marker it would also be grouped into the PF's LAG, which is what the patch prevents.
A VF/SF that does not report these caps, i.e. every VF/SF that exists today, keeps its key unchanged and is still the same HW as its PF, as the commit message says. Dropping the markers inside mlx5_same_hw_devs() would make a nested eswitch manager the same HW as its PF again, i.e. undo the patch.

Quoted 12 lines

drivers/net/ethernet/mellanox/mlx5/core/vport.c u64 mlx5_query_nic_system_image_guid(struct mlx5_core_dev *mdev) · line 1334

Quoted 7 lines
drivers/net/ethernet/mellanox/mlx5/core/vport.c +7 -0
+
+static u8 (* const mlx5_sw_guid_fields[])(const struct mlx5_core_dev *mdev) = {
+	mlx5_sw_guid_load_balance,
+	mlx5_sw_guid_vf,
+	mlx5_sw_guid_sf,
+	/* New fields should be added at the end */
+};

[ ... ]

--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com

author maintainer
#5 lore raw

PATCH v2 2/10 net/mlx5: LAG, select the MPESW master from the live member snapshot

In reply to Tariq Toukan

From: Shay Drory <[email protected]>

The LAG master is marked by the devcom PAIR event when all physical ports have registered (mlx5_lag_num_devs() == ldev->ports). That does not work for a virtual LAG of VFs/SFs: those functions are unrelated to the physical port count.

Select the master from the current membership snapshot in mlx5_lag_enable_mpesw() via mlx5_virt_lag_mark_master(), and clear it in mlx5_lag_disable_mpesw(). Unlike the regular path, it walks all LAG members.
mlx5_get_dev_index() is identical for a PF and all its VFs/SFs, so tie-break by a stable per-member key is used - PCI BDF for VFs, sfnum for SFs.

This is gated by a new ldev->virt_lag bit identifying this type of LAG.

---
 .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 46 +++++++++++++++++--
 .../net/ethernet/mellanox/mlx5/core/lag/lag.h |  3 ++
 .../ethernet/mellanox/mlx5/core/lag/mpesw.c   | 42 +++++++++++++----
 .../ethernet/mellanox/mlx5/core/sf/dev/dev.h  | 12 +++++
 4 files changed, 90 insertions(+), 13 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +43 -3
index 9330aab24c02..c7ece82fd457 100644
@@ -39,6 +39,7 @@
 #include "lib/mlx5.h"
 #include "lib/devcom.h"
 #include "mlx5_core.h"
+#include "sf/dev/dev.h"
 #include "eswitch.h"
 #include "esw/acl/ofld.h"
 #include "lag.h"
@@ -598,7 +599,43 @@ static void mlx5_lag_mark_master(struct mlx5_lag *ldev)
 		xa_set_mark(&ldev->pfs, master_xa_idx, MLX5_LAG_XA_MARK_MASTER);
 }
 
-static void mlx5_lag_clear_master(struct mlx5_lag *ldev)
+static u64 mlx5_lag_member_tiebreak(struct mlx5_core_dev *dev)
+{
+	struct pci_dev *pdev = dev->pdev;
+
+	if (mlx5_core_is_sf(dev))
+		return mlx5_sf_coredev_sfnum(dev);
+
+	return ((u64)pci_domain_nr(pdev->bus) << 16) | pci_dev_id(pdev);
+}
+
+void mlx5_virt_lag_mark_master(struct mlx5_lag *ldev)
+{
+	u64 lowest_tiebreak = U64_MAX;
+	int lowest_dev_idx = INT_MAX;
+	int master_xa_idx = -1;
+	struct lag_func *pf;
+	u64 tiebreak;
+	int dev_idx;
+	int i;
+
+	mlx5_lag_for_each(i, 0, ldev, MLX5_LAG_FILTER_ALL) {
+		pf = mlx5_lag_pf(ldev, i);
+		dev_idx = mlx5_get_dev_index(pf->dev);
+		tiebreak = mlx5_lag_member_tiebreak(pf->dev);
+		if (dev_idx < lowest_dev_idx ||
+		    (dev_idx == lowest_dev_idx && tiebreak < lowest_tiebreak)) {
+			lowest_dev_idx = dev_idx;
+			lowest_tiebreak = tiebreak;
+			master_xa_idx = i;
+		}
+	}
+
+	if (master_xa_idx >= 0)
+		xa_set_mark(&ldev->pfs, master_xa_idx, MLX5_LAG_XA_MARK_MASTER);
+}
+
+void mlx5_lag_clear_master(struct mlx5_lag *ldev)
 {
 	unsigned long idx = 0;
 	void *entry;
@@ -618,7 +655,7 @@ static int mlx5_lag_devcom_event(int event, void *my_data, void *event_data)
 	int idx;
 
 	ldev = mlx5_lag_dev(dev);
-	if (!ldev)
+	if (!ldev || ldev->virt_lag)
 		return 0;
 
 	mutex_lock(&ldev->lock);
@@ -1784,10 +1821,13 @@ struct mlx5_devcom_comp_dev *mlx5_lag_get_devcom_comp(struct mlx5_lag *ldev)
 {
 	struct mlx5_devcom_comp_dev *devcom = NULL;
 	struct lag_func *pf;
+	u32 filter;
 	int i;
 
 	mutex_lock(&ldev->lock);
-	i = mlx5_get_next_lag_func(ldev, 0, MLX5_LAG_FILTER_PORTS);
+	filter = ldev->virt_lag ? MLX5_LAG_FILTER_ALL :
+		MLX5_LAG_FILTER_PORTS;
+	i = mlx5_get_next_lag_func(ldev, 0, filter);
 	if (i < MLX5_MAX_PORTS) {
 		pf = mlx5_lag_pf(ldev, i);
 		devcom = pf->dev->priv.hca_devcom_comp;
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h +3 -0
index 8552792b9035..21c83f51fcb5 100644
@@ -84,6 +84,7 @@ struct mlx5_lag {
 	unsigned long		  state_flags;
 	u8			  ports;
 	u8			  buckets;
+	bool			  virt_lag;
 	int			  mode_changes_in_progress;
 	u8			  *v2p_map;
 	struct kref               ref;
@@ -225,6 +226,8 @@ static inline bool mlx5_lag_shared_fdb_supported(struct mlx5_lag *ldev)
 }
 #endif
 bool mlx5_lag_check_prereq(struct mlx5_lag *ldev);
+void mlx5_virt_lag_mark_master(struct mlx5_lag *ldev);
+void mlx5_lag_clear_master(struct mlx5_lag *ldev);
 bool mlx5_lag_is_sd(struct mlx5_core_dev *dev);
 int mlx5_lag_demux_init(struct mlx5_core_dev *dev,
 			struct mlx5_flow_table_attr *ft_attr);
drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +32 -10

79 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/sf/dev/dev.h +12 -0
index 3ab0449c770c..382162baabdb 100644
@@ -19,6 +19,13 @@ struct mlx5_sf_dev {
 	u16 fn_id;
 };
 
+static inline u32 mlx5_sf_coredev_sfnum(struct mlx5_core_dev *mdev)
+{
+	struct auxiliary_device *adev = mlx5_sf_coredev_to_adev(mdev);
+
+	return container_of(adev, struct mlx5_sf_dev, adev)->sfnum;
+}
+
 struct mlx5_sf_peer_devlink_event_ctx {
 	u16 fn_id;
 	struct devlink *devlink;
@@ -68,6 +75,11 @@ static inline bool mlx5_sf_dev_allocated(const struct mlx5_core_dev *dev)
 	return false;
 }
 
+static inline u32 mlx5_sf_coredev_sfnum(struct mlx5_core_dev *mdev)
+{
+	return 0;
+}
+
 #endif
 
 #endif
Signature · 1 line
2.44.0
#6 lore raw

Re: [PATCH net-next V2 02/10] net/mlx5: LAG, select the MPESW master from the live member snapshot

Sashiko has reviewed this patch and found no issues.
Quoted 15 lines

From: Shay Drory <[email protected]>

The LAG master is marked by the devcom PAIR event when all physical ports have registered (mlx5_lag_num_devs() == ldev->ports). That does not work for a virtual LAG of VFs/SFs: those functions are unrelated to the physical port count.

Select the master from the current membership snapshot in mlx5_lag_enable_mpesw() via mlx5_virt_lag_mark_master(), and clear it in mlx5_lag_disable_mpesw(). Unlike the regular path, it walks all LAG members.
mlx5_get_dev_index() is identical for a PF and all its VFs/SFs, so tie-break by a stable per-member key is used - PCI BDF for VFs, sfnum for SFs.

Quoted 6 lines

This is gated by a new ldev->virt_lag bit identifying this type of LAG.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
author maintainer
#7 lore raw

PATCH v2 3/10 net/mlx5: LAG, skip phys LAG prerequisites for a virt LAG

In reply to Tariq Toukan

From: Shay Drory <[email protected]>

mlx5_lag_enable_mpesw() gates on port_select_flow_table and create_lag_when_not_master_up, and on mlx5_lag_check_prereq(). Those describe the phys LAG: the port-selection flow table and the physical-LAG prerequisites (e.g. member count equal to the physical port count).

A LAG of VFs/SFs still issues the same CREATE_LAG command in MPESW mode from its master. Issued by a VF/SF, though, the command has different
semantics: it does not create a LAG of physical ports, so these checks do not apply and would reject it outright. Require them only when it is a phys LAG; a LAG of VFs/SFs instead requires port_select_eswitch.

---
 .../ethernet/mellanox/mlx5/core/lag/mpesw.c   | 31 ++++++++++++++++---
 1 file changed, 26 insertions(+), 5 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +26 -5
index 0bf5b4754a44..fd70fbebb1de 100644
@@ -107,9 +107,32 @@ static bool mlx5_lag_has_sd_group(struct mlx5_lag *ldev)
 	return false;
 }
 
+static bool mlx5_phys_mpesw_validate(struct mlx5_lag *ldev,
+				     struct mlx5_core_dev *dev0)
+{
+	if (mlx5_eswitch_mode(dev0) != MLX5_ESWITCH_OFFLOADS ||
+	    !MLX5_CAP_PORT_SELECTION(dev0, port_select_flow_table) ||
+	    !MLX5_CAP_GEN(dev0, create_lag_when_not_master_up) ||
+	    !mlx5_lag_check_prereq(ldev) ||
+	    !mlx5_lag_shared_fdb_supported_filter(ldev, MLX5_LAG_FILTER_ALL))
+		return false;
+	return true;
+}
+
+static bool mlx5_virt_mpesw_validate(struct mlx5_lag *ldev,
+				     struct mlx5_core_dev *dev0)
+{
+	if (mlx5_eswitch_mode(dev0) != MLX5_ESWITCH_OFFLOADS ||
+	    !MLX5_CAP_PORT_SELECTION(dev0, port_select_eswitch) ||
+	    !mlx5_lag_shared_fdb_supported_filter(ldev, MLX5_LAG_FILTER_ALL))
+		return false;
+	return true;
+}
+
 static int mlx5_lag_enable_mpesw(struct mlx5_lag *ldev)
 {
 	struct mlx5_core_dev *dev0;
+	bool valid;
 	int idx;
 	int err;
 
@@ -133,11 +156,9 @@ static int mlx5_lag_enable_mpesw(struct mlx5_lag *ldev)
 	}
 
 	dev0 = mlx5_lag_pf(ldev, idx)->dev;
-	if (mlx5_eswitch_mode(dev0) != MLX5_ESWITCH_OFFLOADS ||
-	    !MLX5_CAP_PORT_SELECTION(dev0, port_select_flow_table) ||
-	    !MLX5_CAP_GEN(dev0, create_lag_when_not_master_up) ||
-	    !mlx5_lag_check_prereq(ldev) ||
-	    !mlx5_lag_shared_fdb_supported_filter(ldev, MLX5_LAG_FILTER_ALL)) {
+	valid = ldev->virt_lag ? mlx5_virt_mpesw_validate(ldev, dev0) :
+		mlx5_phys_mpesw_validate(ldev, dev0);
+	if (!valid) {
 		err = -EOPNOTSUPP;
 		goto err_clear_master;
 	}
Signature · 1 line
2.44.0
#8 lore raw

Re: [PATCH net-next V2 03/10] net/mlx5: LAG, skip phys LAG prerequisites for a virt LAG

Sashiko has reviewed this patch and found no issues.
Quoted 14 lines

From: Shay Drory <[email protected]>

mlx5_lag_enable_mpesw() gates on port_select_flow_table and create_lag_when_not_master_up, and on mlx5_lag_check_prereq(). Those describe the phys LAG: the port-selection flow table and the physical-LAG prerequisites (e.g. member count equal to the physical port count).

A LAG of VFs/SFs still issues the same CREATE_LAG command in MPESW mode from its master. Issued by a VF/SF, though, the command has different
semantics: it does not create a LAG of physical ports, so these checks do not apply and would reject it outright. Require them only when it is a phys LAG; a LAG of VFs/SFs instead requires port_select_eswitch.

Quoted 4 lines

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
author maintainer
#9 lore raw

PATCH v2 4/10 net/mlx5: LAG, treat virt LAG as a software managed LAG

In reply to Tariq Toukan

From: Shay Drory <[email protected]>

mlx5_lag_is_sw_managed() (renamed from mlx5_lag_is_sw_lag()) marks the LAG paths the driver builds in software - the LAG demux and vport-LAG - which must skip the FW vport-LAG commands. A virt LAG is software-managed like an SD LAG.

In the same manner, block multipath LAG over virt LAG as well.

---
 drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c | 12 +++++++-----
 drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h |  1 +
 drivers/net/ethernet/mellanox/mlx5/core/lag/mp.c  |  4 ++--
 3 files changed, 10 insertions(+), 7 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +7 -5
index c7ece82fd457..669b2167a82d 100644
@@ -173,9 +173,11 @@ static u32 mlx5_lag_dev_group_id(struct mlx5_core_dev *dev)
 	return 0;
 }
 
-static int mlx5_lag_is_sw_lag(struct mlx5_core_dev *dev)
+bool mlx5_lag_is_sw_managed(struct mlx5_core_dev *dev)
 {
-	return mlx5_lag_is_sd(dev);
+	struct mlx5_lag *ldev = mlx5_lag_dev(dev);
+
+	return mlx5_lag_is_sd(dev) || (ldev && ldev->virt_lag);
 }
 
 int mlx5_cmd_create_vport_lag(struct mlx5_core_dev *dev)
@@ -184,7 +186,7 @@ int mlx5_cmd_create_vport_lag(struct mlx5_core_dev *dev)
 	struct mlx5_lag *ldev = mlx5_lag_dev(dev);
 	int ret;
 
-	if (mlx5_lag_is_sw_lag(dev)) {
+	if (mlx5_lag_is_sw_managed(dev)) {
 		if (!ldev)
 			return -ENODEV;
 
@@ -206,7 +208,7 @@ int mlx5_cmd_destroy_vport_lag(struct mlx5_core_dev *dev)
 	u32 in[MLX5_ST_SZ_DW(destroy_vport_lag_in)] = {};
 	struct mlx5_lag *ldev = mlx5_lag_dev(dev);
 
-	if (mlx5_lag_is_sw_lag(dev)) {
+	if (mlx5_lag_is_sw_managed(dev)) {
 		if (!ldev)
 			return 0;
 
@@ -1911,7 +1913,7 @@ int mlx5_lag_demux_init(struct mlx5_core_dev *dev,
 
 	xa_init(&pf->lag_demux_rules);
 
-	if (mlx5_lag_is_sw_lag(dev))
+	if (mlx5_lag_is_sw_managed(dev))
 		return mlx5_lag_demux_ft_fg_init(dev, ft_attr, pf);
 
 	return mlx5_lag_demux_fw_init(dev, ft_attr, pf);
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h +1 -0
index 21c83f51fcb5..94e07870e506 100644
@@ -229,6 +229,7 @@ bool mlx5_lag_check_prereq(struct mlx5_lag *ldev);
 void mlx5_virt_lag_mark_master(struct mlx5_lag *ldev);
 void mlx5_lag_clear_master(struct mlx5_lag *ldev);
 bool mlx5_lag_is_sd(struct mlx5_core_dev *dev);
+bool mlx5_lag_is_sw_managed(struct mlx5_core_dev *dev);
 int mlx5_lag_demux_init(struct mlx5_core_dev *dev,
 			struct mlx5_flow_table_attr *ft_attr);
 void mlx5_lag_demux_cleanup(struct mlx5_core_dev *dev);
drivers/net/ethernet/mellanox/mlx5/core/lag/mp.c +2 -2
index 65c76bd748c6..2077fc5d80c7 100644
@@ -26,8 +26,8 @@ static bool mlx5_lag_multipath_check_prereq(struct mlx5_lag *ldev)
 	if (__mlx5_lag_is_active(ldev) && !__mlx5_lag_is_multipath(ldev))
 		return false;
 
-	if (__mlx5_lag_is_sd(ldev, mlx5_lag_pf(ldev, idx0)->dev) ||
-	    __mlx5_lag_is_sd(ldev, mlx5_lag_pf(ldev, idx1)->dev))
+	if (mlx5_lag_is_sw_managed(mlx5_lag_pf(ldev, idx0)->dev) ||
+	    mlx5_lag_is_sw_managed(mlx5_lag_pf(ldev, idx1)->dev))
 		return false;
 
 	if (ldev->ports > MLX5_LAG_MULTIPATH_OFFLOADS_SUPPORTED_PORTS)
Signature · 1 line
2.44.0
#10 lore raw

Re: [PATCH net-next V2 04/10] net/mlx5: LAG, treat virt LAG as a software managed LAG

Sashiko has reviewed this patch and found no issues.
Quoted 7 lines

From: Shay Drory <[email protected]>

mlx5_lag_is_sw_managed() (renamed from mlx5_lag_is_sw_lag()) marks the LAG paths the driver builds in software - the LAG demux and vport-LAG - which must skip the FW vport-LAG commands. A virt LAG is software-managed like an SD LAG.

Quoted 6 lines

In the same manner, block multipath LAG over virt LAG as well.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
author maintainer
#11 lore raw

PATCH v2 5/10 net/mlx5: LAG, decouple iteration end-of-walk from MLX5_MAX_PORTS

In reply to Tariq Toukan

From: Shay Drory <[email protected]>

The LAG xarray iterators returned MLX5_MAX_PORTS as their end-of-walk sentinel, and the open-coded callers compared the current index against MLX5_MAX_PORTS to detect the end of iteration. This ties iteration to a compile-time maximum rather than to the actual number of LAG members.

Add ldev->max_funcs, and use it as the loop bound. Update the open-coded callers accordingly. When a member is added at an xarray index >= ldev->max_funcs, increase max_funcs.
In addition, remove the xa_alloc() limit of MLX5_MAX_PORTS.

No functional change; this only removes the dependency on MLX5_MAX_PORTS.

---
 .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 22 +++++++++++--------
 .../net/ethernet/mellanox/mlx5/core/lag/lag.h |  3 ++-
 .../ethernet/mellanox/mlx5/core/lag/mpesw.c   |  2 +-
 .../mellanox/mlx5/core/lag/shared_fdb.c       |  2 +-
 4 files changed, 17 insertions(+), 12 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +13 -9
index 669b2167a82d..cbef6ab1e38a 100644
@@ -395,6 +395,7 @@ static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev)
 		mlx5_core_err(dev, "Failed to init multipath lag err=%d\n",
 			      err);
 
+	ldev->max_funcs = ldev->ports;
 	return ldev;
 
 err_wq:
@@ -1283,7 +1284,7 @@ static void mlx5_lag_assert_locked_transition(struct mlx5_lag *ldev, u32 filter)
 	lockdep_assert_held(&ldev->lock);
 
 	i = mlx5_get_next_lag_func(ldev, 0, filter);
-	if (i < MLX5_MAX_PORTS) {
+	if (i < ldev->max_funcs) {
 		pf = mlx5_lag_pf(ldev, i);
 		if (filter == MLX5_LAG_FILTER_PORTS ||
 		    filter == MLX5_LAG_FILTER_ALL)
@@ -1830,7 +1831,7 @@ struct mlx5_devcom_comp_dev *mlx5_lag_get_devcom_comp(struct mlx5_lag *ldev)
 	filter = ldev->virt_lag ? MLX5_LAG_FILTER_ALL :
 		MLX5_LAG_FILTER_PORTS;
 	i = mlx5_get_next_lag_func(ldev, 0, filter);
-	if (i < MLX5_MAX_PORTS) {
+	if (i < ldev->max_funcs) {
 		pf = mlx5_lag_pf(ldev, i);
 		devcom = pf->dev->priv.hca_devcom_comp;
 	}
@@ -2094,7 +2095,7 @@ static int mlx5_handle_changeupper_event(struct mlx5_lag *ldev,
 				break;
 			}
 		}
-		if (i < MLX5_MAX_PORTS) {
+		if (i < ldev->ports) {
 			if (netif_is_bond_master(upper)) {
 				slave = bond_slave_get_rcu(ndev_tmp);
 				if (slave)
@@ -2359,13 +2360,16 @@ int mlx5_ldev_add_mdev(struct mlx5_lag *ldev,
 	if (!pf)
 		return -ENOMEM;
 
-	err = xa_alloc(&ldev->pfs, &idx, pf, XA_LIMIT(0, MLX5_MAX_PORTS - 1),
+	err = xa_alloc(&ldev->pfs, &idx, pf, XA_LIMIT(0, U8_MAX - 1),
 		       GFP_KERNEL);
 	if (err) {
 		kfree(pf);
 		return err;
 	}
 
+	/* max_funcs is the iteration ceiling; grow it, never shrink. */
+	if (idx >= ldev->max_funcs)
+		ldev->max_funcs = idx + 1;
 	pf->idx = idx;
 	pf->dev = dev;
 	pf->group_id = group_id;
@@ -2394,7 +2398,7 @@ void mlx5_ldev_remove_mdev(struct mlx5_lag *ldev,
 		if (pf->dev == dev)
 			break;
 	}
-	if (i >= MLX5_MAX_PORTS)
+	if (i >= ldev->max_funcs)
 		return;
 
 	if (pf->port_change_nb.nb.notifier_call)
@@ -2607,7 +2611,7 @@ int mlx5_get_next_lag_func(struct mlx5_lag *ldev, int start_idx, u32 filter)
 					 MLX5_LAG_XA_MARK_PORT, start_idx)
 			if (pf->dev)
 				return idx;
-		return MLX5_MAX_PORTS;
+		return ldev->max_funcs;
 	}
 
 	xa_for_each_start(&ldev->pfs, idx, pf, start_idx) {
@@ -2617,7 +2621,7 @@ int mlx5_get_next_lag_func(struct mlx5_lag *ldev, int start_idx, u32 filter)
 		    filter == pf->group_id)
 			return idx;
 	}
-	return MLX5_MAX_PORTS;
+	return ldev->max_funcs;
 }
 
 bool mlx5_lag_is_roce(struct mlx5_core_dev *dev)
@@ -2867,7 +2871,7 @@ struct mlx5_core_dev *mlx5_lag_get_next_peer_mdev(struct mlx5_core_dev *dev, int
 	if (!ldev)
 		goto unlock;
 
-	if (*i == MLX5_MAX_PORTS)
+	if (*i == ldev->max_funcs)
 		goto unlock;
 	mlx5_lag_for_each(idx, *i, ldev, mlx5_lag_get_filter(ldev, dev)) {
 		pf = mlx5_lag_pf(ldev, idx);
@@ -2875,7 +2879,7 @@ struct mlx5_core_dev *mlx5_lag_get_next_peer_mdev(struct mlx5_core_dev *dev, int
 			break;
 	}
 
-	if (idx == MLX5_MAX_PORTS) {
+	if (idx == ldev->max_funcs) {
 		*i = idx;
 		goto unlock;
 	}
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h +2 -1
index 94e07870e506..1dcd0b87997f 100644
@@ -83,6 +83,7 @@ struct mlx5_lag {
 	unsigned long		  mode_flags;
 	unsigned long		  state_flags;
 	u8			  ports;
+	u8			  max_funcs;
 	u8			  buckets;
 	bool			  virt_lag;
 	int			  mode_changes_in_progress;
@@ -289,7 +290,7 @@ static inline bool mlx5_lag_is_supported(struct mlx5_core_dev *dev)
 #define mlx5_lag_for_each(i, start_index, ldev, filter) \
 	for (int tmp = start_index; \
 	     tmp = mlx5_get_next_lag_func(ldev, tmp, filter), \
-	     i = tmp, tmp < MLX5_MAX_PORTS; tmp++)
+	     i = tmp, tmp < (ldev)->max_funcs; tmp++)
 
 #define mlx5_lag_for_each_reverse(i, start_index, end_index, ldev, filter) \
 	for (int tmp = start_index, tmp1 = end_index; \
drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +1 -1
index fd70fbebb1de..45e92d71e62c 100644
@@ -223,7 +223,7 @@ void mlx5_mpesw_sd_devcoms_unlock(struct mlx5_lag *ldev)
 	struct mlx5_devcom_comp_dev *sd_devcom;
 	int i;
 
-	mlx5_ldev_for_each_reverse(i, MLX5_MAX_PORTS, 0, ldev) {
+	mlx5_ldev_for_each_reverse(i, ldev->max_funcs, 0, ldev) {
 		sd_devcom = mlx5_sd_get_devcom(mlx5_lag_pf(ldev, i)->dev);
 		if (sd_devcom)
 			mlx5_devcom_comp_unlock(sd_devcom);
drivers/net/ethernet/mellanox/mlx5/core/lag/shared_fdb.c +1 -1
index 7c7e6c832a93..29048b0c2797 100644
@@ -71,7 +71,7 @@ static int mlx5_lag_create_single_fdb_filter(struct mlx5_lag *ldev, u32 filter)
 
 		err = mlx5_eswitch_offloads_single_fdb_add_one(master_esw,
 							       slave_esw,
-							       ldev->ports);
+							       ldev->max_funcs);
 		if (err)
 			goto err;
 	}
Signature · 1 line
2.44.0
#12 lore raw

Re: [PATCH net-next V2 05/10] net/mlx5: LAG, decouple iteration end-of-walk from MLX5_MAX_PORTS

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] net/mlx5: LAG, max_funcs sentinel mismatch in changeupper and alloc --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] net/mlx5: LAG, max_funcs sentinel mismatch in changeupper and alloc
--

commit 78f2f8069167c6a38a6b670a3fc665eb9f73b7c1
Author: Shay Drory <[email protected]>

net/mlx5: LAG, decouple iteration end-of-walk from MLX5_MAX_PORTS

This commit replaces MLX5_MAX_PORTS with a dynamic ldev->max_funcs ceiling as the end-of-walk sentinel for LAG xarray iterators and removes the MLX5_MAX_PORTS upper bound on xa_alloc().

Quoted 12 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +1 -0
index 669b2167a82d2..cbef6ab1e38a8 100644
@@ -395,6 +395,7 @@ static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev)
 		mlx5_core_err(dev, "Failed to init multipath lag err=%d\n",
 			      err);
 
+	ldev->max_funcs = ldev->ports;
 	return ldev;
 
 err_wq:

[ ... ]

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c static int mlx5_handle_changeupper_event(struct mlx5_lag *ldev, · line 2101

Quoted 9 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +1 -1
@@ -2094,7 +2095,7 @@ static int mlx5_handle_changeupper_event(struct mlx5_lag *ldev,
 				break;
 			}
 		}
-		if (i < MLX5_MAX_PORTS) {
+		if (i < ldev->ports) {
 			if (netif_is_bond_master(upper)) {
 				slave = bond_slave_get_rcu(ndev_tmp);
 				if (slave)

[Severity: Medium]
Should mlx5_handle_changeupper_event() check i < ldev->max_funcs instead of i < ldev->ports, and should ldev->max_funcs be initialized before registering the notifiers in mlx5_lag_dev_alloc()?

mlx5_ldev_for_each() terminates with i == ldev->max_funcs when no matching netdev is found. In mlx5_lag_dev_alloc(), register_netdevice_notifier_net() and mlx5_lag_mp_init() run while ldev->max_funcs is still 0 and ldev->ports is at least 2:

CPU0 CPU1
mlx5_lag_dev_alloc()
ldev->ports = ...;
register_netdevice_notifier_net()

                                      mlx5_lag_netdev_event()
                                        mlx5_handle_changeupper_event()
                                          mlx5_ldev_for_each(i, 0, ldev)
                                            /* ends with i = 0, idx = -1 */
                                          if (i < ldev->ports)
                                            bond_status |= (1 << idx);
                                              <-- shift by -1

mlx5_lag_mp_init()
ldev->max_funcs = ldev->ports;

Conversely, once xa_alloc() assigns a port an xarray index >= ldev->ports and grows ldev->max_funcs, a matching slave increments idx and breaks out of mlx5_ldev_for_each() with i >= ldev->ports, so if (i < ldev->ports) skips updating bond_status and has_inactive for that slave.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
#13 lore raw

Re: [PATCH net-next V2 05/10] net/mlx5: LAG, decouple iteration end-of-walk from MLX5_MAX_PORTS

In reply to Tariq Toukan
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider.

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider.

Critical: 0 · High: 1 · Medium: 2 · Low: 2

If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).

- [High] mlx5_ldev_add_mdev() raises the xa_alloc() range from XA_LIMIT(0, MLX5_MAX_PORTS - 1) to XA_LIMIT(0, U8_MAX - 1).
- [Medium] mlx5_handle_changeupper_event() now tests whether the port was found with if (i < ldev->ports).
- [Medium] The end-of-walk sentinel used to be the compile-time constant MLX5_MAX_PORTS.
- [Low] In mlx5_lag_create_single_fdb_filter() (shared_fdb.c), the max_slaves argument to mlx5_eswitch_offloads_single_fdb_add_one()…
- [Low] mlx5_lag_dev_alloc() sets ldev->max_funcs = ldev->ports only at the very end.

--

net/mlx5: LAG, decouple iteration end-of-walk from MLX5_MAX_PORTS

This patch adds ldev->max_funcs and uses it as the end-of-walk sentinel for the LAG xarray iterators in place of MLX5_MAX_PORTS. max_funcs grows when a member is added at a higher xarray index. The patch also raises the xa_alloc() limit in mlx5_ldev_add_mdev() from MLX5_MAX_PORTS - 1 to U8_MAX - 1.

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev) · line 399

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +1 -0
index 669b2167a82d2..cbef6ab1e38a8 100644
@@ -395,6 +395,7 @@ static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev)
 		mlx5_core_err(dev, "Failed to init multipath lag err=%d\n",
 			      err);
 
+	ldev->max_funcs = ldev->ports;
 	return ldev;

[Severity: Low]
Can a netdev event see max_funcs == 0 here?

By the time this assignment runs, register_netdevice_notifier_net() has already made ldev->nb live. mlx5_lag_mp_init() has also registered the FIB notifier, which dumps the FIB and can take a while. Until this line, max_funcs is still 0 from kzalloc.

Suppose a NETDEV_CHANGEUPPER for a bond in the same netns arrives in that window. mlx5_handle_changeupper_event() then runs mlx5_ldev_for_each() with max_funcs == 0:

- mlx5_get_next_lag_func() returns 0.
- The loop exits with i = 0 and idx still -1.
- The new "i < ldev->ports" check passes.

The handler then executes:

    bond_status |= (1 << idx);

This is a shift by -1. Would it be better to set max_funcs right after ldev->ports is assigned, before any notifier is registered?

[ ... ]

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c static int mlx5_handle_changeupper_event(struct mlx5_lag *ldev, · line 2098

Quoted 6 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +1 -1
@@ -2094,7 +2095,7 @@ static int mlx5_handle_changeupper_event(struct mlx5_lag *ldev,
 				break;
 			}
 		}
-		if (i < MLX5_MAX_PORTS) {
+		if (i < ldev->ports) {

[Severity: Medium]
Should this compare against ldev->max_funcs, like the other converted call sites? mlx5_ldev_for_each() now ends with i == ldev->max_funcs. This patch also explicitly allows members at xarray indices >= ldev->ports, since mlx5_ldev_add_mdev() grows max_funcs for that case.

xa_alloc() hands out the lowest free index to both port and SD members. For example, after a PF is unbound and rebound, SD secondaries added by sd_lag_init() can hold the low indices, so the matched port PF can end up at an index >= ports.

In that case the loop breaks with i equal to that index, "i < ldev->ports" is false, and the PF is treated as not found. Its bond_status bit is never set and has_inactive is not computed for it, so this never becomes true:

    is_in_lag = num_slaves == ldev->ports &&
        bond_status == GENMASK(ldev->ports - 1, 0);

As a result, the bond is never offloaded. Before this patch, indices were capped at 7, so any found entry passed the check.

The notifier is only registered when !mlx5_sd_is_supported(dev), so this may need a mixed or unusual topology to trigger. The check is still the same at the end of the series.

[ ... ]

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c int mlx5_ldev_add_mdev(struct mlx5_lag *ldev, · line 2364

Quoted 7 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +1 -1
@@ -2359,13 +2360,16 @@ int mlx5_ldev_add_mdev(struct mlx5_lag *ldev,
 	if (!pf)
 		return -ENOMEM;
 
-	err = xa_alloc(&ldev->pfs, &idx, pf, XA_LIMIT(0, MLX5_MAX_PORTS - 1),
+	err = xa_alloc(&ldev->pfs, &idx, pf, XA_LIMIT(0, U8_MAX - 1),
 		       GFP_KERNEL);

[Severity: High]
Is it safe to drop the MLX5_MAX_PORTS limit before the arrays that depend on it are resized?

This limit was the only thing that kept both the member count and every xarray index below MLX5_MAX_PORTS (8). Neither __mlx5_lag_dev_add_mdev() nor sd_lag_init() checks the member count.

At this commit, once an ldev has more than 8 members, these writes look like they go out of bounds:

- mlx5_mpesw_metadata_set() writes ldev->lag_mpesw.pf_metadata[i] (u32[MLX5_MAX_PORTS]) under MLX5_LAG_FILTER_ALL. lag_mpesw is the last member of struct mlx5_lag, so this writes past the kzalloc'd ldev.

- mlx5_lag_rescan_devices_locked_filter() fills the on-stack devs[MLX5_MAX_PORTS] with devs[num_devs++] without any bound check.

- mlx5_lag_query_cong_counters() fills a buffer of MLX5_MAX_PORTS entries with mdev[idx++].

- struct lag_tracker::netdev_state[MLX5_MAX_PORTS] is written at the port's xarray index in mlx5_ldev_add_netdev() and in mlx5_handle_changelowerstate_event():

      tracker->netdev_state[idx] = *lag_lower_info;

- struct mlx5_lag_definer::rules[MLX5_MAX_PORTS * MLX5_LAG_MAX_HASH_BUCKETS] is indexed by i * ldev->buckets + j in port_sel.c, where i is the port's xarray index.

Before this patch, such a configuration failed cleanly: xa_alloc() returned -EBUSY and sd_lag_init() rolled back the secondaries. An 8-port SD configuration, for example, would now give 16 members.

Later patches in the series fix some of these:

- "net/mlx5: LAG: replace fixed MLX5_MAX_PORTS arrays with dynamic allocation" makes pf_metadata kcalloc(max_fns) and sizes the cong-counter buffer from ports.
- "net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan" removes devs[].

At the end of the series, though, lag_tracker::netdev_state[] and mlx5_lag_definer::rules[] are still sized by MLX5_MAX_PORTS and indexed by xarray indices that are no longer bounded.

The commit message also says:

No functional change; this only removes the dependency on MLX5_MAX_PORTS.

Isn't dropping the xa_alloc() bound a functional change in itself?

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c int mlx5_ldev_add_mdev(struct mlx5_lag *ldev, · line 2374

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +3 -0
 	if (err) {
 		kfree(pf);
 		return err;
 	}
 
+	/* max_funcs is the iteration ceiling; grow it, never shrink. */
+	if (idx >= ldev->max_funcs)
+		ldev->max_funcs = idx + 1;
 	pf->idx = idx;
 	pf->dev = dev;

[Severity: Medium]
Can walkers that don't hold ldev->lock see an inconsistent max_funcs?

The end-of-walk sentinel used to be a constant. It is now this field, which is written with a plain store under ldev->lock only. Each iteration of mlx5_lag_for_each() loads it twice: once as the return value of mlx5_get_next_lag_func(), and again in the macro test:

    i = tmp, tmp < (ldev)->max_funcs; tmp++)

Several walkers don't hold ldev->lock:

- mlx5_mpesw_sd_devcoms_lock(), called from mlx5_mpesw_work() and mlx5_lag_disable_change(), runs under the HCA devcom lock but before mutex_lock(&ldev->lock).
- mlx5_ldev_add_netdev(), mlx5_lag_get_next_peer_mdev() and mlx5_lag_query_cong_counters() hold only lag_lock.
- mlx5_handle_changeupper_event() holds only RCU.

sd_lag_init() adds secondaries while holding only ldev->lock. The writer order is:

1. xa_alloc() publishes a zeroed pf at idx == old max_funcs.
2. max_funcs = idx + 1.
3. pf->dev = dev.
4. The PORT mark is set.

Suppose a reader's walk ends while the new pf isn't valid yet. It gets sentinel N back from mlx5_get_next_lag_func(), then re-reads max_funcs as N + 1. The loop body then runs once with i = N, skipping the !pf->dev check and the PORT-mark filter. In mlx5_mpesw_sd_devcoms_lock() that means:

    sd_devcom = mlx5_sd_get_devcom(mlx5_lag_pf(ldev, i)->dev);

runs with a NULL dev, which mlx5_get_sd() dereferences. In mlx5_lag_get_next_peer_mdev(), a NULL or non-port dev could be returned as a peer.

Also, on weakly ordered architectures nothing orders this store against the xarray slot, so xa_load(N) could return NULL as well.

[ ... ]

drivers/net/ethernet/mellanox/mlx5/core/lag/shared_fdb.c static int mlx5_lag_create_single_fdb_filter(struct mlx5_lag *ldev, u32 filter) · line 74

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/shared_fdb.c +1 -1
index 7c7e6c832a93d..29048b0c27977 100644
@@ -71,7 +71,7 @@ static int mlx5_lag_create_single_fdb_filter(struct mlx5_lag *ldev, u32 filter)
 
 		err = mlx5_eswitch_offloads_single_fdb_add_one(master_esw,
 							       slave_esw,
-							       ldev->ports);
+							       ldev->max_funcs);

[Severity: Low]
Is this change intended? This argument isn't an end-of-walk sentinel. It is max_slaves, which goes through esw_set_master_egress_rule() to esw_master_egress_create_resources(). There it becomes ft_attr.max_fte and the bounce group's end_flow_index for the master's egress ACL table in firmware.

With SD, secondaries take extra xarray indices, so max_funcs can exceed ports (for example, 4 on a 2-port SD card) and the ACL table is created larger. max_funcs is also a high-water mark of xarray indices that never shrinks, not a slave count.

Bounce rules are only added for SD primaries, so ldev->ports seems to have been sufficient already.

This doesn't match "No functional change" in the commit message. Could the change be explained in the commit message, or dropped?

Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com
#14 lore raw

Re: [PATCH net-next V2 05/10] net/mlx5: LAG, decouple iteration end-of-walk from MLX5_MAX_PORTS

Quoted 59 lines · On Wed, 18 Nov 2026 10:31:54 +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider.

Critical: 0 · High: 1 · Medium: 2 · Low: 2

If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).

- [High] mlx5_ldev_add_mdev() raises the xa_alloc() range from XA_LIMIT(0, MLX5_MAX_PORTS - 1) to XA_LIMIT(0, U8_MAX - 1).
- [Medium] mlx5_handle_changeupper_event() now tests whether the port was found with if (i < ldev->ports).
- [Medium] The end-of-walk sentinel used to be the compile-time constant MLX5_MAX_PORTS.
- [Low] In mlx5_lag_create_single_fdb_filter() (shared_fdb.c), the max_slaves argument to mlx5_eswitch_offloads_single_fdb_add_one()…
- [Low] mlx5_lag_dev_alloc() sets ldev->max_funcs = ldev->ports only at the very end.

--

net/mlx5: LAG, decouple iteration end-of-walk from MLX5_MAX_PORTS

This patch adds ldev->max_funcs and uses it as the end-of-walk sentinel for the LAG xarray iterators in place of MLX5_MAX_PORTS. max_funcs grows when a member is added at a higher xarray index. The patch also raises the xa_alloc() limit in mlx5_ldev_add_mdev() from MLX5_MAX_PORTS - 1 to U8_MAX - 1.

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev) · line 399

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +1 -0
index 669b2167a82d2..cbef6ab1e38a8 100644
@@ -395,6 +395,7 @@ static struct mlx5_lag *mlx5_lag_dev_alloc(struct mlx5_core_dev *dev)
 		mlx5_core_err(dev, "Failed to init multipath lag err=%d\n",
 			      err);
 
+	ldev->max_funcs = ldev->ports;
 	return ldev;

[Severity: Low]
Can a netdev event see max_funcs == 0 here?

By the time this assignment runs, register_netdevice_notifier_net() has already made ldev->nb live. mlx5_lag_mp_init() has also registered the FIB notifier, which dumps the FIB and can take a while. Until this line, max_funcs is still 0 from kzalloc.

Suppose a NETDEV_CHANGEUPPER for a bond in the same netns arrives in that window. mlx5_handle_changeupper_event() then runs mlx5_ldev_for_each() with max_funcs == 0:

- mlx5_get_next_lag_func() returns 0.
- The loop exits with i = 0 and idx still -1.
- The new "i < ldev->ports" check passes.

Quoted 6 lines

The handler then executes:

    bond_status |= (1 << idx);

This is a shift by -1. Would it be better to set max_funcs right after ldev->ports is assigned, before any notifier is registered?

The window is very narrow and harmless: bond_status stays 0 and the handler returns early, so nothing is changed. It will be fixed in a follow-up.

Quoted 27 lines

[ ... ]

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c static int mlx5_handle_changeupper_event(struct mlx5_lag *ldev, · line 2098

Quoted 6 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +1 -1
@@ -2094,7 +2095,7 @@ static int mlx5_handle_changeupper_event(struct mlx5_lag *ldev,
 				break;
 			}
 		}
-		if (i < MLX5_MAX_PORTS) {
+		if (i < ldev->ports) {

[Severity: Medium]
Should this compare against ldev->max_funcs, like the other converted call sites? mlx5_ldev_for_each() now ends with i == ldev->max_funcs. This patch also explicitly allows members at xarray indices >= ldev->ports, since mlx5_ldev_add_mdev() grows max_funcs for that case.

xa_alloc() hands out the lowest free index to both port and SD members. For example, after a PF is unbound and rebound, SD secondaries added by sd_lag_init() can hold the low indices, so the matched port PF can end up at an index >= ports.

In that case the loop breaks with i equal to that index, "i < ldev->ports" is false, and the PF is treated as not found. Its bond_status bit is never set and has_inactive is not computed for it, so this never becomes true:

    is_in_lag = num_slaves == ldev->ports &&
        bond_status == GENMASK(ldev->ports - 1, 0);
Quoted 6 lines

As a result, the bond is never offloaded. Before this patch, indices were capped at 7, so any found entry passed the check.

The notifier is only registered when !mlx5_sd_is_supported(dev), so this may need a mixed or unusual topology to trigger. The check is still the same at the end of the series.

No. mlx5 creates an ldev per sw_guid, and PFs have a different sw_guid than VFs/SFs, so only PFs share this ldev. The only case with more PFs than num_lag_ports in one ldev is SD, and for SD the netdev notifier, and with it mlx5_handle_changeupper_event(), is not registered. So a PORT-marked member here always sits at an index below ldev->ports.

Quoted 53 lines

[ ... ]

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c int mlx5_ldev_add_mdev(struct mlx5_lag *ldev, · line 2364

Quoted 7 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +1 -1
@@ -2359,13 +2360,16 @@ int mlx5_ldev_add_mdev(struct mlx5_lag *ldev,
 	if (!pf)
 		return -ENOMEM;
 
-	err = xa_alloc(&ldev->pfs, &idx, pf, XA_LIMIT(0, MLX5_MAX_PORTS - 1),
+	err = xa_alloc(&ldev->pfs, &idx, pf, XA_LIMIT(0, U8_MAX - 1),
 		       GFP_KERNEL);

[Severity: High]
Is it safe to drop the MLX5_MAX_PORTS limit before the arrays that depend on it are resized?

This limit was the only thing that kept both the member count and every xarray index below MLX5_MAX_PORTS (8). Neither __mlx5_lag_dev_add_mdev() nor sd_lag_init() checks the member count.

At this commit, once an ldev has more than 8 members, these writes look like they go out of bounds:

- mlx5_mpesw_metadata_set() writes ldev->lag_mpesw.pf_metadata[i] (u32[MLX5_MAX_PORTS]) under MLX5_LAG_FILTER_ALL. lag_mpesw is the last member of struct mlx5_lag, so this writes past the kzalloc'd ldev.

- mlx5_lag_rescan_devices_locked_filter() fills the on-stack devs[MLX5_MAX_PORTS] with devs[num_devs++] without any bound check.

- mlx5_lag_query_cong_counters() fills a buffer of MLX5_MAX_PORTS entries with mdev[idx++].

- struct lag_tracker::netdev_state[MLX5_MAX_PORTS] is written at the port's xarray index in mlx5_ldev_add_netdev() and in mlx5_handle_changelowerstate_event():

      tracker->netdev_state[idx] = *lag_lower_info;

- struct mlx5_lag_definer::rules[MLX5_MAX_PORTS * MLX5_LAG_MAX_HASH_BUCKETS] is indexed by i * ldev->buckets + j in port_sel.c, where i is the port's xarray index.

Before this patch, such a configuration failed cleanly: xa_alloc() returned -EBUSY and sd_lag_init() rolled back the secondaries. An 8-port SD configuration, for example, would now give 16 members.

Later patches in the series fix some of these:

- "net/mlx5: LAG: replace fixed MLX5_MAX_PORTS arrays with dynamic allocation" makes pf_metadata kcalloc(max_fns) and sizes the cong-counter buffer from ports.
- "net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan" removes devs[].

Quoted 3 lines

At the end of the series, though, lag_tracker::netdev_state[] and mlx5_lag_definer::rules[] are still sized by MLX5_MAX_PORTS and indexed by xarray indices that are no longer bounded.

In this series a 9th member is impossible: an ldev holds only PFs and SD secondaries, and no HW has more than MLX5_MAX_PORTS of them. VFs/SFs join an ldev only in the follow-up series.
The limit can't be kept until all arrays are converted, since not all of them will be. Some arrays are used only by a PF LAG, others by both a PF LAG and a virt LAG. The shared ones, the fns xarray included, are converted in this series (pf_metadata[] and devs[]; the cong counter array is converted as well). The PF LAG only ones, tracker.netdev_state[] and the port_sel rules[], keep their static MLX5_MAX_PORTS size: they are walked only over PORT-marked members, in the hash and queue affinity modes. A virt member is never PORT-marked and sits in its own ldev, and a virt LAG supports only MPESW, so these arrays never see more than MLX5_MAX_PORTS entries.

Quoted 6 lines

The commit message also says:

No functional change; this only removes the dependency on MLX5_MAX_PORTS.

Isn't dropping the xa_alloc() bound a functional change in itself?

The xa_alloc() change is already in the commit message ("remove the xa_alloc() limit of MLX5_MAX_PORTS"), and the answer above explains why it can't let in a 9th member.

Quoted 5 lines
Quoted 5 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +0 -0
 	if (err) {
 		kfree(pf);
 		return err;
 	}
 
Quoted 9 lines

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c int mlx5_ldev_add_mdev(struct mlx5_lag *ldev, · line 2374

Quoted 5 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +3 -0
+	/* max_funcs is the iteration ceiling; grow it, never shrink. */
+	if (idx >= ldev->max_funcs)
+		ldev->max_funcs = idx + 1;
 	pf->idx = idx;
 	pf->dev = dev;

[Severity: Medium]
Can walkers that don't hold ldev->lock see an inconsistent max_funcs?

No, all these walkers run either during LAG activation or while the LAG is active, and membership doesn't change then.

Quoted 63 lines

The end-of-walk sentinel used to be a constant. It is now this field, which is written with a plain store under ldev->lock only. Each iteration of mlx5_lag_for_each() loads it twice: once as the return value of mlx5_get_next_lag_func(), and again in the macro test:

    i = tmp, tmp < (ldev)->max_funcs; tmp++)

Several walkers don't hold ldev->lock:

- mlx5_mpesw_sd_devcoms_lock(), called from mlx5_mpesw_work() and mlx5_lag_disable_change(), runs under the HCA devcom lock but before mutex_lock(&ldev->lock).
- mlx5_ldev_add_netdev(), mlx5_lag_get_next_peer_mdev() and mlx5_lag_query_cong_counters() hold only lag_lock.
- mlx5_handle_changeupper_event() holds only RCU.

sd_lag_init() adds secondaries while holding only ldev->lock. The writer order is:

1. xa_alloc() publishes a zeroed pf at idx == old max_funcs.
2. max_funcs = idx + 1.
3. pf->dev = dev.
4. The PORT mark is set.

Suppose a reader's walk ends while the new pf isn't valid yet. It gets sentinel N back from mlx5_get_next_lag_func(), then re-reads max_funcs as N + 1. The loop body then runs once with i = N, skipping the !pf->dev check and the PORT-mark filter. In mlx5_mpesw_sd_devcoms_lock() that means:

    sd_devcom = mlx5_sd_get_devcom(mlx5_lag_pf(ldev, i)->dev);

runs with a NULL dev, which mlx5_get_sd() dereferences. In mlx5_lag_get_next_peer_mdev(), a NULL or non-port dev could be returned as a peer.

Also, on weakly ordered architectures nothing orders this store against the xarray slot, so xa_load(N) could return NULL as well.
[ ... ]

drivers/net/ethernet/mellanox/mlx5/core/lag/shared_fdb.c static int mlx5_lag_create_single_fdb_filter(struct mlx5_lag *ldev, u32 filter) · line 74

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/shared_fdb.c +1 -1
index 7c7e6c832a93d..29048b0c27977 100644
@@ -71,7 +71,7 @@ static int mlx5_lag_create_single_fdb_filter(struct mlx5_lag *ldev, u32 filter)
 
 		err = mlx5_eswitch_offloads_single_fdb_add_one(master_esw,
 							       slave_esw,
-							       ldev->ports);
+							       ldev->max_funcs);

[Severity: Low]
Is this change intended? This argument isn't an end-of-walk sentinel. It is max_slaves, which goes through esw_set_master_egress_rule() to esw_master_egress_create_resources(). There it becomes ft_attr.max_fte and the bounce group's end_flow_index for the master's egress ACL table in firmware.

With SD, secondaries take extra xarray indices, so max_funcs can exceed ports (for example, 4 on a 2-port SD card) and the ACL table is created larger. max_funcs is also a high-water mark of xarray indices that never shrinks, not a slave count.

Quoted 5 lines

Bounce rules are only added for SD primaries, so ldev->ports seems to have been sufficient already.

This doesn't match "No functional change" in the commit message. Could the change be explained in the commit message, or dropped?

Only the ACL size changes, and only for SD, where more PFs than ports join the ldev and max_funcs > ports. The rules installed are the same as before, so there is no functional change.

Quoted 2 lines

--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com

author maintainer
#15 lore raw

PATCH v2 6/10 net/mlx5: LAG, rename PF naming to fn

In reply to Tariq Toukan

From: Shay Drory <[email protected]>

The LAG member bookkeeping is named after physical functions: the xarray is ldev->pfs, the lookup helpers are mlx5_lag_pf(), mlx5_lag_pf_by_dev(), mlx5_lag_pf_by_dev_idx() and mlx5_lag_dev_get_master_pf(), and the local variables holding the result are pf.

A follow-up series enables virt LAG, which means LAG members won't always be physical functions.

Rename the xarray to ldev->fns, the helpers to mlx5_lag_fn*(), and the locals to fn, so the naming matches struct lag_fn.

No functional change.

---
 .../ethernet/mellanox/mlx5/core/lag/debugfs.c |   2 +-
 .../net/ethernet/mellanox/mlx5/core/lag/lag.c | 472 +++++++++---------
 .../net/ethernet/mellanox/mlx5/core/lag/lag.h |  54 +-
 .../net/ethernet/mellanox/mlx5/core/lag/mp.c  |  29 +-
 .../ethernet/mellanox/mlx5/core/lag/mpesw.c   |  42 +-
 .../mellanox/mlx5/core/lag/port_sel.c         |  25 +-
 .../mellanox/mlx5/core/lag/shared_fdb.c       |  44 +-
 .../net/ethernet/mellanox/mlx5/core/lib/sd.c  |  32 +-
 8 files changed, 354 insertions(+), 346 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/lag/debugfs.c +1 -1
index b6125c7bd91b..116432cb7ce5 100644
@@ -154,7 +154,7 @@ static int members_show(struct seq_file *file, void *priv)
 	mutex_lock(&ldev->lock);
 	mlx5_ldev_for_each(i, 0, ldev)
 		seq_printf(file, "%s\n",
-			   dev_name(mlx5_lag_pf(ldev, i)->dev->device));
+			   dev_name(mlx5_lag_fn(ldev, i)->dev->device));
 	mutex_unlock(&ldev->lock);
 
 	return 0;
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +237 -235

1,161 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h +27 -27

115 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/lag/mp.c +16 -13
index 2077fc5d80c7..587638278f34 100644
@@ -26,15 +26,15 @@ static bool mlx5_lag_multipath_check_prereq(struct mlx5_lag *ldev)
 	if (__mlx5_lag_is_active(ldev) && !__mlx5_lag_is_multipath(ldev))
 		return false;
 
-	if (mlx5_lag_is_sw_managed(mlx5_lag_pf(ldev, idx0)->dev) ||
-	    mlx5_lag_is_sw_managed(mlx5_lag_pf(ldev, idx1)->dev))
+	if (mlx5_lag_is_sw_managed(mlx5_lag_fn(ldev, idx0)->dev) ||
+	    mlx5_lag_is_sw_managed(mlx5_lag_fn(ldev, idx1)->dev))
 		return false;
 
 	if (ldev->ports > MLX5_LAG_MULTIPATH_OFFLOADS_SUPPORTED_PORTS)
 		return false;
 
-	return mlx5_esw_multipath_prereq(mlx5_lag_pf(ldev, idx0)->dev,
-					 mlx5_lag_pf(ldev, idx1)->dev);
+	return mlx5_esw_multipath_prereq(mlx5_lag_fn(ldev, idx0)->dev,
+					 mlx5_lag_fn(ldev, idx1)->dev);
 }
 
 bool mlx5_lag_is_multipath(struct mlx5_core_dev *dev)
@@ -59,11 +59,15 @@ static void mlx5_lag_set_port_affinity(struct mlx5_lag *ldev,
 {
 	int idx0 = mlx5_lag_get_dev_index_by_seq(ldev, MLX5_LAG_P1);
 	int idx1 = mlx5_lag_get_dev_index_by_seq(ldev, MLX5_LAG_P2);
+	struct mlx5_core_dev *dev0, *dev1;
 	struct lag_tracker tracker = {};
 
 	if (idx0 < 0 || idx1 < 0 || !__mlx5_lag_is_multipath(ldev))
 		return;
 
+	dev0 = mlx5_lag_fn(ldev, idx0)->dev;
+	dev1 = mlx5_lag_fn(ldev, idx1)->dev;
+
 	switch (port) {
 	case MLX5_LAG_NORMAL_AFFINITY:
 		tracker.netdev_state[idx0].tx_enabled = true;
@@ -84,18 +88,17 @@ static void mlx5_lag_set_port_affinity(struct mlx5_lag *ldev,
 		tracker.netdev_state[idx1].link_up = true;
 		break;
 	default:
-		mlx5_core_warn(mlx5_lag_pf(ldev, idx0)->dev,
-			       "Invalid affinity port %d", port);
+		mlx5_core_warn(dev0, "Invalid affinity port %d", port);
 		return;
 	}
 
 	if (tracker.netdev_state[idx0].tx_enabled)
-		mlx5_notifier_call_chain(mlx5_lag_pf(ldev, idx0)->dev->priv.events,
+		mlx5_notifier_call_chain(dev0->priv.events,
 					 MLX5_DEV_EVENT_PORT_AFFINITY,
 					 (void *)0);
 
 	if (tracker.netdev_state[idx1].tx_enabled)
-		mlx5_notifier_call_chain(mlx5_lag_pf(ldev, idx1)->dev->priv.events,
+		mlx5_notifier_call_chain(dev1->priv.events,
 					 MLX5_DEV_EVENT_PORT_AFFINITY,
 					 (void *)0);
 
@@ -150,7 +153,7 @@ mlx5_lag_get_next_fib_dev(struct mlx5_lag *ldev,
 		fib_dev = fib_info_nh(fi, i)->fib_nh_dev;
 		ldev_idx = mlx5_lag_dev_get_netdev_idx(ldev, fib_dev);
 		if (ldev_idx >= 0)
-			return mlx5_lag_pf(ldev, ldev_idx)->netdev;
+			return mlx5_lag_fn(ldev, ldev_idx)->netdev;
 	}
 
 	return NULL;
@@ -182,7 +185,7 @@ static void mlx5_lag_fib_route_event(struct mlx5_lag *ldev, unsigned long event,
 	    mp->fib.dst_len <= fen_info->dst_len &&
 	    !(mp->fib.dst_len == fen_info->dst_len &&
 	      fi->fib_priority < mp->fib.priority)) {
-		mlx5_core_dbg(mlx5_lag_pf(ldev, idx)->dev,
+		mlx5_core_dbg(mlx5_lag_fn(ldev, idx)->dev,
 			      "Multipath entry with lower priority was rejected\n");
 		return;
 	}
@@ -198,7 +201,7 @@ static void mlx5_lag_fib_route_event(struct mlx5_lag *ldev, unsigned long event,
 	}
 
 	if (nh_dev0 == nh_dev1) {
-		mlx5_core_warn(mlx5_lag_pf(ldev, idx)->dev,
+		mlx5_core_warn(mlx5_lag_fn(ldev, idx)->dev,
 			       "Multipath offload doesn't support routes with multiple nexthops of the same device");
 		return;
 	}
@@ -207,7 +210,7 @@ static void mlx5_lag_fib_route_event(struct mlx5_lag *ldev, unsigned long event,
 		if (__mlx5_lag_is_active(ldev)) {
 			mlx5_ldev_for_each(i, 0, ldev) {
 				dev_idx++;
-				if (mlx5_lag_pf(ldev, i)->netdev == nh_dev0)
+				if (mlx5_lag_fn(ldev, i)->netdev == nh_dev0)
 					break;
 			}
 			mlx5_lag_set_port_affinity(ldev, dev_idx);
@@ -244,7 +247,7 @@ static void mlx5_lag_fib_nexthop_event(struct mlx5_lag *ldev,
 	/* nh added/removed */
 	if (event == FIB_EVENT_NH_DEL) {
 		mlx5_ldev_for_each(i, 0, ldev) {
-			if (mlx5_lag_pf(ldev, i)->netdev == fib_nh->fib_nh_dev)
+			if (mlx5_lag_fn(ldev, i)->netdev == fib_nh->fib_nh_dev)
 				break;
 			dev_idx++;
 		}
drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +21 -21

114 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/lag/port_sel.c +14 -11
index 2a034b2a3eee..183e18910095 100644
@@ -50,7 +50,7 @@ static int mlx5_lag_create_port_sel_table(struct mlx5_lag *ldev,
 	if (first_idx < 0)
 		return -EINVAL;
 
-	dev = mlx5_lag_pf(ldev, first_idx)->dev;
+	dev = mlx5_lag_fn(ldev, first_idx)->dev;
 	ft_attr.max_fte = ldev->ports * ldev->buckets;
 	ft_attr.level = MLX5_LAG_FT_LEVEL_DEFINER;
 
@@ -79,6 +79,7 @@ static int mlx5_lag_create_port_sel_table(struct mlx5_lag *ldev,
 	flow_act.flags |= FLOW_ACT_NO_APPEND;
 	mlx5_ldev_for_each(i, 0, ldev) {
 		for (j = 0; j < ldev->buckets; j++) {
+			struct mlx5_core_dev *mdev;
 			u8 affinity;
 
 			idx = i * ldev->buckets + j;
@@ -87,9 +88,8 @@ static int mlx5_lag_create_port_sel_table(struct mlx5_lag *ldev,
 			/* affinity is 1-indexed device index,
 			 * use reverse lookup.
 			 */
-			dest.vport.vhca_id =
-				MLX5_CAP_GEN(mlx5_lag_pf_by_dev_idx(ldev, affinity - 1)->dev,
-					     vhca_id);
+			mdev = mlx5_lag_fn_by_dev_idx(ldev, affinity - 1)->dev;
+			dest.vport.vhca_id = MLX5_CAP_GEN(mdev, vhca_id);
 			lag_definer->rules[idx] = mlx5_add_flow_rules(lag_definer->ft,
 								      NULL, &flow_act,
 								      &dest, 1);
@@ -311,7 +311,7 @@ mlx5_lag_create_definer(struct mlx5_lag *ldev, enum netdev_lag_hash hash,
 	if (first_idx < 0)
 		return ERR_PTR(-EINVAL);
 
-	dev = mlx5_lag_pf(ldev, first_idx)->dev;
+	dev = mlx5_lag_fn(ldev, first_idx)->dev;
 	lag_definer = kzalloc_obj(*lag_definer);
 	if (!lag_definer)
 		return ERR_PTR(-ENOMEM);
@@ -360,7 +360,7 @@ static void mlx5_lag_destroy_definer(struct mlx5_lag *ldev,
 	if (first_idx < 0)
 		return;
 
-	dev = mlx5_lag_pf(ldev, first_idx)->dev;
+	dev = mlx5_lag_fn(ldev, first_idx)->dev;
 	mlx5_ldev_for_each(i, 0, ldev) {
 		for (j = 0; j < ldev->buckets; j++) {
 			idx = i * ldev->buckets + j;
@@ -524,7 +524,7 @@ static int mlx5_lag_create_ttc_table(struct mlx5_lag *ldev)
 	if (first_idx < 0)
 		return -EINVAL;
 
-	dev = mlx5_lag_pf(ldev, first_idx)->dev;
+	dev = mlx5_lag_fn(ldev, first_idx)->dev;
 	mlx5_lag_set_outer_ttc_params(ldev, &ttc_params);
 	port_sel->outer.ttc = mlx5_create_ttc_table(dev, &ttc_params);
 	return PTR_ERR_OR_ZERO(port_sel->outer.ttc);
@@ -540,7 +540,7 @@ static int mlx5_lag_create_inner_ttc_table(struct mlx5_lag *ldev)
 	if (first_idx < 0)
 		return -EINVAL;
 
-	dev = mlx5_lag_pf(ldev, first_idx)->dev;
+	dev = mlx5_lag_fn(ldev, first_idx)->dev;
 	mlx5_lag_set_inner_ttc_params(ldev, &ttc_params);
 	port_sel->inner.ttc = mlx5_create_inner_ttc_table(dev, &ttc_params);
 	return PTR_ERR_OR_ZERO(port_sel->inner.ttc);
@@ -594,6 +594,9 @@ static int __mlx5_lag_modify_definers_destinations(struct mlx5_lag *ldev,
 
 	mlx5_ldev_for_each(i, 0, ldev) {
 		for (j = 0; j < ldev->buckets; j++) {
+			struct mlx5_core_dev *mdev;
+			u8 affinity;
+
 			idx = i * ldev->buckets + j;
 			if (ldev->v2p_map[idx] == ports[idx])
 				continue;
@@ -601,9 +604,9 @@ static int __mlx5_lag_modify_definers_destinations(struct mlx5_lag *ldev,
 			/* ports[] contains 1-indexed device indices,
 			 * use reverse lookup.
 			 */
-			dest.vport.vhca_id =
-				MLX5_CAP_GEN(mlx5_lag_pf_by_dev_idx(ldev, ports[idx] - 1)->dev,
-					     vhca_id);
+			affinity = ports[idx];
+			mdev = mlx5_lag_fn_by_dev_idx(ldev, affinity - 1)->dev;
+			dest.vport.vhca_id = MLX5_CAP_GEN(mdev, vhca_id);
 			err = mlx5_modify_rule_destination(def->rules[idx], &dest, NULL);
 			if (err)
 				return err;
drivers/net/ethernet/mellanox/mlx5/core/lag/shared_fdb.c +22 -22

150 lines not on this page · raw message

drivers/net/ethernet/mellanox/mlx5/core/lib/sd.c +16 -16
index 4cdc50cd6f03..f843efbca660 100644
@@ -300,7 +300,7 @@ static int sd_lag_state_show(struct seq_file *file, void *priv)
 {
 	struct mlx5_core_dev *dev = file->private;
 	struct mlx5_lag *ldev;
-	struct lag_func *pf;
+	struct lag_fn *fn;
 	bool active = false;
 	int i;
 
@@ -310,9 +310,9 @@ static int sd_lag_state_show(struct seq_file *file, void *priv)
 
 	mutex_lock(&ldev->lock);
 	mlx5_ldev_for_each(i, 0, ldev) {
-		pf = mlx5_lag_pf(ldev, i);
-		if (pf->dev == dev) {
-			active = pf->sd_fdb_active;
+		fn = mlx5_lag_fn(ldev, i);
+		if (fn->dev == dev) {
+			active = fn->sd_fdb_active;
 			break;
 		}
 	}
@@ -334,7 +334,7 @@ static void sd_lag_init(struct mlx5_core_dev *dev)
 	struct mlx5_sd *sd = mlx5_get_sd(primary);
 	struct mlx5_core_dev *pos, *to;
 	struct mlx5_lag *ldev;
-	struct lag_func *pf;
+	struct lag_fn *fn;
 	int err;
 	int i;
 
@@ -346,14 +346,14 @@ static void sd_lag_init(struct mlx5_core_dev *dev)
 	}
 
 	mutex_lock(&ldev->lock);
-	pf = mlx5_lag_pf_by_dev(ldev, primary);
-	if (!pf) {
+	fn = mlx5_lag_fn_by_dev(ldev, primary);
+	if (!fn) {
 		sd_warn(primary, "%s: primary not registered in ldev, skipping\n",
 			__func__);
 		goto out;
 	}
 
-	pf->group_id = sd->group_id;
+	fn->group_id = sd->group_id;
 
 	mlx5_sd_for_each_secondary(i, primary, pos) {
 		err = mlx5_ldev_add_mdev(ldev, pos, sd->group_id);
@@ -372,7 +372,7 @@ static void sd_lag_init(struct mlx5_core_dev *dev)
 	to = pos;
 	mlx5_sd_for_each_secondary_to(i, primary, to, pos)
 		mlx5_ldev_remove_mdev(ldev, pos);
-	pf->group_id = 0;
+	fn->group_id = 0;
 	mutex_unlock(&ldev->lock);
 }
 
@@ -381,7 +381,7 @@ static void sd_lag_cleanup(struct mlx5_core_dev *dev)
 	struct mlx5_core_dev *primary = mlx5_sd_get_primary(dev);
 	struct mlx5_core_dev *pos;
 	struct mlx5_lag *ldev;
-	struct lag_func *pf;
+	struct lag_fn *fn;
 	int i;
 
 	ldev = mlx5_lag_dev(primary);
@@ -392,9 +392,9 @@ static void sd_lag_cleanup(struct mlx5_core_dev *dev)
 	mlx5_sd_for_each_secondary(i, primary, pos)
 		mlx5_ldev_remove_mdev(ldev, pos);
 
-	pf = mlx5_lag_pf_by_dev(ldev, primary);
-	if (pf)
-		pf->group_id = 0;
+	fn = mlx5_lag_fn_by_dev(ldev, primary);
+	if (fn)
+		fn->group_id = 0;
 	mutex_unlock(&ldev->lock);
 }
 
@@ -989,7 +989,7 @@ static void mlx5_sd_activate_shared_fdb(struct mlx5_core_dev *primary)
 	struct mlx5_sd *sd = mlx5_get_sd(primary);
 	struct mlx5_core_dev *pos;
 	struct mlx5_lag *ldev;
-	struct lag_func *pf;
+	struct lag_fn *fn;
 	int err;
 	int i;
 
@@ -1009,8 +1009,8 @@ static void mlx5_sd_activate_shared_fdb(struct mlx5_core_dev *primary)
 
 	/* Check if SD FDB is already active for this group */
 	mlx5_lag_for_each(i, 0, ldev, sd->group_id) {
-		pf = mlx5_lag_pf(ldev, i);
-		if (pf->sd_fdb_active)
+		fn = mlx5_lag_fn(ldev, i);
+		if (fn->sd_fdb_active)
 			goto unlock;
 		break;
 	}
Signature · 1 line
2.44.0
#16 lore raw

Re: [PATCH net-next V2 06/10] net/mlx5: LAG, rename PF naming to fn

Sashiko has reviewed this patch and found no issues.
Quoted 14 lines

From: Shay Drory <[email protected]>

The LAG member bookkeeping is named after physical functions: the xarray is ldev->pfs, the lookup helpers are mlx5_lag_pf(), mlx5_lag_pf_by_dev(), mlx5_lag_pf_by_dev_idx() and mlx5_lag_dev_get_master_pf(), and the local variables holding the result are pf.

A follow-up series enables virt LAG, which means LAG members won't always be physical functions.

Rename the xarray to ldev->fns, the helpers to mlx5_lag_fn*(), and the locals to fn, so the naming matches struct lag_fn.

Quoted 6 lines

No functional change.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
#17 lore raw

Re: [PATCH net-next V2 06/10] net/mlx5: LAG, rename PF naming to fn

In reply to Tariq Toukan
Thank you for your contribution! Sashiko AI review found more severe issue(s) in other patches of this series. This mail is only an FYI, in case you want to address the nit picks below when you…

Thank you for your contribution! Sashiko AI review found more severe issue(s) in other patches of this series. This mail is only an FYI, in case you want to address the nit picks below when you respin.

Low priority / nit picks for your consideration:

- [Low] The commit message misstates what this patch renames.

Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but it is still present at the end of the series.

--

net/mlx5: LAG, rename PF naming to fn

This commit renames the LAG member bookkeeping from PF naming to fn. The ldev->pfs xarray becomes ldev->fns, the mlx5_lag_pf*() helpers become mlx5_lag_fn*(), and the pf locals become fn. The rename prepares for virt LAG, where LAG members will not always be physical functions.

Quoted 2 lines

Rename the xarray to ldev->fns, the helpers to mlx5_lag_fn*(), and the locals to fn, so the naming matches struct lag_fn.

[Severity: Low]
Is this description accurate? struct lag_fn doesn't exist before this patch. The patch creates it by renaming the struct in lag.h:

-struct lag_func {
+struct lag_fn {

It also renames a field of struct mlx5_lag:

- u8 max_funcs;
+ u8 max_fns;

The commit message mentions neither rename.

The patch also makes a few small changes that are not renames:

- a new esw local in mlx5_lag_drop_rule_cleanup()
- dev0/dev1 moved above the switch in mlx5_lag_set_port_affinity()
- new mdev/affinity locals in mlx5_lag_create_port_sel_table() and __mlx5_lag_modify_definers_destinations()

Could the commit message mention these as well?

A few spots look like they were missed by the rename. The pf_idx and pf_mdev locals remain in mlx5_lag_get_devices_oper_speed(), mlx5_lag_get_devices_max_speed(), mlx5_lag_set_vports_agg_speed() and mlx5_lag_reset_vports_speed():

	mlx5_ldev_for_each(pf_idx, 0, ldev) {
		...
		pf_mdev = fn->dev;

mlx5_get_next_lag_func() and mlx5_get_pre_lag_func() walk the renamed fns xarray but keep their old names.

In mlx5_lag_mpesw_port_change_event(), the local now has the same name as the struct tag. The container_of() continuation lines are still aligned for the old, longer name:

	struct lag_fn *lag_fn = container_of(mlx5_nb,
						 struct lag_fn,
						 port_change_nb);

Was that intended?

Quoted 4 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +0 -0
index cbef6ab1e38a8..74e7e7161c95d 100644

[ ... ]

Quoted 4 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +0 -0
@@ -2390,24 +2392,24 @@ int mlx5_ldev_add_mdev(struct mlx5_lag *ldev,
 void mlx5_ldev_remove_mdev(struct mlx5_lag *ldev,
 			   struct mlx5_core_dev *dev)
 {

[ ... ]

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c int mlx5_ldev_add_mdev(struct mlx5_lag *ldev, · line 2413

Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +3 -3
-	pf->dev = NULL;
+	fn->dev = NULL;
 	dev->priv.lag = NULL;
-	xa_erase(&ldev->pfs, pf->idx);
-	kfree(pf);
+	xa_erase(&ldev->fns, fn->idx);
+	kfree(fn);
 }

[Severity: High]
This is a pre-existing issue and was not introduced by this patch. Since this code is being touched: is it safe to free the lag_fn here while readers holding only lag_lock may still be using it?

mlx5_ldev_remove_mdev() runs with only ldev->lock held. It is called from:

- mlx5_unload()->mlx5_lag_remove_mdev()
- sd_lag_cleanup()
- the error path of sd_lag_init()

It erases the entry and frees it right away with kfree(). There is no RCU grace period, and lag_lock is not taken.

Several exported helpers walk the same ldev->fns xarray while holding only lag_lock, without ldev->lock. For example:

mlx5_lag_get_next_peer_mdev() {
	spin_lock_irqsave(&lag_lock, flags);
	...
	mlx5_lag_for_each(idx, *i, ldev, mlx5_lag_get_filter(ldev, dev)) {
		fn = mlx5_lag_fn(ldev, idx);
		if (fn->dev != dev)
			break;
	}
	...
	fn = mlx5_lag_fn(ldev, idx);
	peer_dev = fn->dev;
	...
}

mlx5_lag_is_master(), mlx5_lag_is_shared_fdb() and mlx5_lag_is_sd() read fn->dev, fn->group_id and fn->sd_fdb_active through mlx5_lag_get_filter() and mlx5_lag_fn_by_dev(). mlx5_lag_get_slave_port() and mlx5_lag_active_backup_get_netdev() also dereference entries while holding only lag_lock.

Can this interleaving happen between two devices in the same ldev?

CPU0 (peer device A) CPU1 (device B unload)
mlx5_esw_is_owner()
mlx5_lag_is_master()

    spin_lock_irqsave(&lag_lock)
    mlx5_lag_get_filter()
      mlx5_lag_fn_by_dev()
        xa_for_each() loads fn
                                 mlx5_lag_remove_mdev()
                                   mutex_lock(&ldev->lock)
                                   mlx5_ldev_remove_mdev()
                                     xa_erase(&ldev->fns, fn->idx)
                                     kfree(fn)
        fn->dev == dev   <- reads freed memory

If mlx5_lag_get_next_peer_mdev() hits the same window, it returns the stale fn->dev to its caller as a peer mlx5_core_dev pointer.

The baseline already has the same xa_erase() and kfree() in mlx5_ldev_remove_mdev(), and nothing later in the series changes this.

Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com
author maintainer
#18 lore raw

PATCH v2 7/10 net/mlx5: LAG, resolve affinity indices only in queue-affinity mode

In reply to Tariq Toukan

From: Shay Drory <[email protected]>

mlx5_cmd_create_lag() resolved the first two members up front via mlx5_lag_get_dev_index_by_seq() and returned -EINVAL if either failed, but only the queue-affinity mode uses them.

Move the lookup into that case.

---
 .../net/ethernet/mellanox/mlx5/core/lag/lag.c    | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +9 -7
index 74e7e7161c95..5c9f200c9637 100644
@@ -95,23 +95,25 @@ static int mlx5_cmd_create_lag(struct mlx5_core_dev *dev, struct mlx5_lag *ldev,
 	int port_sel_mode = get_port_sel_mode(mode, flags);
 	u32 in[MLX5_ST_SZ_DW(create_lag_in)] = {};
 	u8 *ports = ldev->v2p_map;
-	int idx0, idx1;
 	void *lag_ctx;
 
 	lag_ctx = MLX5_ADDR_OF(create_lag_in, in, ctx);
 	MLX5_SET(create_lag_in, in, opcode, MLX5_CMD_OP_CREATE_LAG);
 	MLX5_SET(lagc, lag_ctx, fdb_selection_mode, fdb_sel_mode);
-	idx0 = mlx5_lag_get_dev_index_by_seq(ldev, 0);
-	idx1 = mlx5_lag_get_dev_index_by_seq(ldev, 1);
+	switch (port_sel_mode) {
+	case MLX5_LAG_PORT_SELECT_MODE_QUEUE_AFFINITY: {
+		int idx0, idx1;
 
-	if (idx0 < 0 || idx1 < 0)
-		return -EINVAL;
+		idx0 = mlx5_lag_get_dev_index_by_seq(ldev, 0);
+		idx1 = mlx5_lag_get_dev_index_by_seq(ldev, 1);
+
+		if (idx0 < 0 || idx1 < 0)
+			return -EINVAL;
 
-	switch (port_sel_mode) {
-	case MLX5_LAG_PORT_SELECT_MODE_QUEUE_AFFINITY:
 		MLX5_SET(lagc, lag_ctx, tx_remap_affinity_1, ports[idx0]);
 		MLX5_SET(lagc, lag_ctx, tx_remap_affinity_2, ports[idx1]);
 		break;
+	}
 	case MLX5_LAG_PORT_SELECT_MODE_PORT_SELECT_FT: {
 		u8 active_port;
 		int ret;
Signature · 1 line
2.44.0
#19 lore raw

Re: [PATCH net-next V2 07/10] net/mlx5: LAG, resolve affinity indices only in queue-affinity mode

Sashiko has reviewed this patch and found no issues.
Quoted 6 lines

From: Shay Drory <[email protected]>

mlx5_cmd_create_lag() resolved the first two members up front via mlx5_lag_get_dev_index_by_seq() and returned -EINVAL if either failed, but only the queue-affinity mode uses them.

Quoted 6 lines

Move the lookup into that case.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
author maintainer
#20 lore raw

PATCH v2 8/10 net/mlx5: LAG: replace fixed MLX5_MAX_PORTS arrays with dynamic allocation

In reply to Tariq Toukan

From: Shay Drory <[email protected]>

Converts two fixed-size arrays sized to MLX5_MAX_PORTS into dynamically allocated pointers:

- lag_mpesw.pf_metadata in mpesw.c/mpesw.h, sized by ldev->max_fns
- mdev array in mlx5_lag_query_cong_counters(), sized by ldev->ports

The cong-counters array is sized by ldev->ports, which is fixed at ldev allocation, rather than by the LAG mode, so a mode change between the allocation and filling the array cannot overflow it. The query loop is bounded by the number of entries actually filled. A virt LAG is not aggregated and queries only the given device, as its members are not LAG ports.

Thus, removes the hard dependency on MLX5_MAX_PORTS.

---
 drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c | 15 +++++++--------
 .../net/ethernet/mellanox/mlx5/core/lag/mpesw.c   | 12 ++++++++++++
 .../net/ethernet/mellanox/mlx5/core/lag/mpesw.h   |  2 +-
 3 files changed, 20 insertions(+), 9 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +7 -8
index 5c9f200c9637..a7b12353b722 100644
@@ -2916,7 +2916,9 @@ int mlx5_lag_query_cong_counters(struct mlx5_core_dev *dev,
 	if (!out)
 		return -ENOMEM;
 
-	mdev = kvzalloc(sizeof(mdev[0]) * MLX5_MAX_PORTS, GFP_KERNEL);
+	ldev = mlx5_lag_dev(dev);
+	num_ports = ldev ? max_t(int, ldev->ports, 1) : 1;
+	mdev = kcalloc(num_ports, sizeof(mdev[0]), GFP_KERNEL);
 	if (!mdev) {
 		ret = -ENOMEM;
 		goto free_out;
@@ -2925,20 +2927,17 @@ int mlx5_lag_query_cong_counters(struct mlx5_core_dev *dev,
 	memset(values, 0, sizeof(*values) * num_counters);
 
 	spin_lock_irqsave(&lag_lock, flags);
-	ldev = mlx5_lag_dev(dev);
-	if (ldev && __mlx5_lag_is_active(ldev)) {
-		num_ports = ldev->ports;
+	if (ldev && __mlx5_lag_is_active(ldev) && !ldev->virt_lag) {
 		mlx5_ldev_for_each(i, 0, ldev) {
 			fn = mlx5_lag_fn(ldev, i);
 			mdev[idx++] = fn->dev;
 		}
 	} else {
-		num_ports = 1;
-		mdev[MLX5_LAG_P1] = dev;
+		mdev[idx++] = dev;
 	}
 	spin_unlock_irqrestore(&lag_lock, flags);
 
-	for (i = 0; i < num_ports; ++i) {
+	for (i = 0; i < idx; ++i) {
 		u32 in[MLX5_ST_SZ_DW(query_cong_statistics_in)] = {};
 
 		MLX5_SET(query_cong_statistics_in, in, opcode,
@@ -2953,7 +2952,7 @@ int mlx5_lag_query_cong_counters(struct mlx5_core_dev *dev,
 	}
 
 free_mdev:
-	kvfree(mdev);
+	kfree(mdev);
 free_out:
 	kvfree(out);
 	return ret;
drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +12 -0
index 49602f830615..8a867a04916a 100644
@@ -15,6 +15,9 @@ static void mlx5_mpesw_metadata_cleanup(struct mlx5_lag *ldev)
 	u32 pf_metadata;
 	int i;
 
+	if (!ldev->lag_mpesw.pf_metadata)
+		return;
+
 	mlx5_lag_for_each(i, 0, ldev, MLX5_LAG_FILTER_ALL) {
 		dev = mlx5_lag_fn(ldev, i)->dev;
 		esw = dev->priv.eswitch;
@@ -27,6 +30,9 @@ static void mlx5_mpesw_metadata_cleanup(struct mlx5_lag *ldev)
 		mlx5_esw_match_metadata_free(esw, pf_metadata);
 		ldev->lag_mpesw.pf_metadata[i] = 0;
 	}
+
+	kfree(ldev->lag_mpesw.pf_metadata);
+	ldev->lag_mpesw.pf_metadata = NULL;
 }
 
 static int mlx5_mpesw_metadata_set(struct mlx5_lag *ldev)
@@ -36,6 +42,12 @@ static int mlx5_mpesw_metadata_set(struct mlx5_lag *ldev)
 	u32 pf_metadata;
 	int i, err;
 
+	ldev->lag_mpesw.pf_metadata =
+		kcalloc(ldev->max_fns, sizeof(*ldev->lag_mpesw.pf_metadata),
+			GFP_KERNEL);
+	if (!ldev->lag_mpesw.pf_metadata)
+		return -ENOMEM;
+
 	mlx5_lag_for_each(i, 0, ldev, MLX5_LAG_FILTER_ALL) {
 		dev = mlx5_lag_fn(ldev, i)->dev;
 		esw = dev->priv.eswitch;
drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.h +1 -1
index 5099723ba0f7..6296e3edd38c 100644
@@ -9,7 +9,7 @@
 
 struct lag_mpesw {
 	struct work_struct mpesw_work;
-	u32 pf_metadata[MLX5_MAX_PORTS];
+	u32 *pf_metadata;
 };
 
 enum mpesw_op {
Signature · 1 line
2.44.0
#21 lore raw

Re: [PATCH net-next V2 08/10] net/mlx5: LAG: replace fixed MLX5_MAX_PORTS arrays with dynamic allocation

Sashiko has reviewed this patch and found no issues.
Quoted 15 lines

From: Shay Drory <[email protected]>

Converts two fixed-size arrays sized to MLX5_MAX_PORTS into dynamically allocated pointers:

- lag_mpesw.pf_metadata in mpesw.c/mpesw.h, sized by ldev->max_fns
- mdev array in mlx5_lag_query_cong_counters(), sized by ldev->ports

The cong-counters array is sized by ldev->ports, which is fixed at ldev allocation, rather than by the LAG mode, so a mode change between the allocation and filling the array cannot overflow it. The query loop is bounded by the number of entries actually filled. A virt LAG is not aggregated and queries only the given device, as its members are not LAG ports.

Quoted 6 lines

Thus, removes the hard dependency on MLX5_MAX_PORTS.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
#22 lore raw

Re: [PATCH net-next V2 08/10] net/mlx5: LAG: replace fixed MLX5_MAX_PORTS arrays with dynamic allocation

In reply to Tariq Toukan
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider.

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider.

Critical: 0 · High: 1 · Medium: 0 · Low: 1

If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).

- [High] Possible heap out-of-bounds read/write in mlx5_mpesw_metadata_cleanup()…
- [Low] The commit message ends with "Thus, removes the hard dependency on MLX5_MAX_PORTS", which overstates what the patch does.

--

net/mlx5: LAG: replace fixed MLX5_MAX_PORTS arrays with dynamic allocation

This converts lag_mpesw.pf_metadata and the mdev array in mlx5_lag_query_cong_counters() from fixed MLX5_MAX_PORTS sized arrays to dynamically allocated buffers. pf_metadata is sized by ldev->max_fns when MPESW is enabled. The cong counters array is sized by ldev->ports.

Quoted 1 line

Thus, removes the hard dependency on MLX5_MAX_PORTS.

[Severity: Low]
Is this statement accurate? This patch converts only two arrays.

This patch leaves the devs[MLX5_MAX_PORTS] scratch array in mlx5_lag_rescan_devices_locked_filter() in place. The later series patch "drop MLX5_MAX_PORTS scratch array from device rescan" removes it.

Some arrays tied to the LAG member index space are still there at the end of the series. That index space is no longer capped at MLX5_MAX_PORTS, because "decouple iteration end-of-walk from MLX5_MAX_PORTS" moved xa_alloc() to XA_LIMIT(0, U8_MAX - 1):

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h:struct lag_tracker {
    ...
    struct netdev_lag_lower_state_info  netdev_state[MLX5_MAX_PORTS];
    ...
}
drivers/net/ethernet/mellanox/mlx5/core/lag/port_sel.h:struct mlx5_lag_port_sel {
    ...
    struct mlx5_flow_handle *rules[MLX5_MAX_PORTS * MLX5_LAG_MAX_HASH_BUCKETS];
    ...
}

Could the commit message say that these remain? Or should they be converted or bounded as well?

Quoted 25 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +6 -0
index 49602f8306155..8a867a04916a8 100644
@@ -15,6 +15,9 @@ static void mlx5_mpesw_metadata_cleanup(struct mlx5_lag *ldev)
 	u32 pf_metadata;
 	int i;
 
+	if (!ldev->lag_mpesw.pf_metadata)
+		return;
+
 	mlx5_lag_for_each(i, 0, ldev, MLX5_LAG_FILTER_ALL) {
 		dev = mlx5_lag_fn(ldev, i)->dev;
 		esw = dev->priv.eswitch;
@@ -27,6 +30,9 @@ static void mlx5_mpesw_metadata_cleanup(struct mlx5_lag *ldev)
 		mlx5_esw_match_metadata_free(esw, pf_metadata);
 		ldev->lag_mpesw.pf_metadata[i] = 0;
 	}
+
+	kfree(ldev->lag_mpesw.pf_metadata);
+	ldev->lag_mpesw.pf_metadata = NULL;
 }
 
 static int mlx5_mpesw_metadata_set(struct mlx5_lag *ldev)

drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c static int mlx5_mpesw_metadata_set(struct mlx5_lag *ldev) · line 50

Quoted 9 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +6 -0
@@ -36,6 +42,12 @@ static int mlx5_mpesw_metadata_set(struct mlx5_lag *ldev)
 	u32 pf_metadata;
 	int i, err;
 
+	ldev->lag_mpesw.pf_metadata =
+		kcalloc(ldev->max_fns, sizeof(*ldev->lag_mpesw.pf_metadata),
+			GFP_KERNEL);
+	if (!ldev->lag_mpesw.pf_metadata)
+		return -ENOMEM;
+

[Severity: High]
Can ldev->max_fns grow after this allocation while MPESW is still active?

The allocated length is not stored anywhere. Yet mlx5_mpesw_metadata_cleanup() walks the members with mlx5_lag_for_each(), which checks against the current max_fns:

#define mlx5_lag_for_each(i, start_index, ldev, filter) \
	for (int tmp = start_index; \
	     tmp = mlx5_get_next_lag_func(ldev, tmp, filter), \
	     i = tmp, tmp < (ldev)->max_fns; tmp++)

mlx5_ldev_add_mdev() grows max_fns without looking at the LAG mode:

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c:mlx5_ldev_add_mdev() {
    ...
	/* max_fns is the iteration ceiling; grow it, never shrink. */
	if (idx >= ldev->max_fns)
		ldev->max_fns = idx + 1;
    ...
}

sd_lag_init() calls it for each SD secondary, also without checking ldev->mode:

drivers/net/ethernet/mellanox/mlx5/core/lib/sd.c:sd_lag_init() {
    ...
	mlx5_sd_for_each_secondary(i, primary, pos) {
		err = mlx5_ldev_add_mdev(ldev, pos, sd->group_id);
    ...
}

Here is one possible sequence on an SD card:

devlink esw_multiport=true
mlx5_lag_enable_mpesw()
mlx5_mpesw_metadata_set()
kcalloc(max_fns = 2)

secondary PF probes later

    mlx5e_probe() -> mlx5_sd_init() -> sd_lag_init()
      mlx5_ldev_add_mdev()     /* idx 2, max_fns = 3 */

devlink esw_multiport=false, or unload
mlx5_lag_disable_mpesw()
mlx5_mpesw_metadata_cleanup()
reads and writes pf_metadata[2]

SD secondaries have fn->dev set, so the MLX5_LAG_FILTER_ALL walk visits them. Three things then go wrong for any index past the allocated count:

- cleanup reads pf_metadata[i] past the end of the slab object.
- if that stale value is non-zero, it calls mlx5_esw_acl_ingress_vport_metadata_update() on that eswitch, and mlx5_esw_match_metadata_free() with an ida id that was never allocated.
- it writes 0 to pf_metadata[i] past the end of the buffer.

With the old embedded u32 pf_metadata[MLX5_MAX_PORTS], a member added later at an index below MLX5_MAX_PORTS read a zero slot and was skipped.

The last patch in the series ("do not add a device to an already-active virt LAG") says SD secondaries join before MPESW can be enabled. Is that enforced anywhere?

- mlx5_devlink_esw_multiport_set()->mlx5_lag_mpesw_enable() does not check SD readiness.
- mlx5_phys_mpesw_validate()->mlx5_lag_check_prereq() only checks mlx5_lag_num_devs(ldev) == ldev->ports. That counts port members, not SD secondaries.

Would it be safer to record the allocated length and bound the cleanup loop by it? Another option is to refuse max_fns growth and SD joins while MPESW is active.

[ ... ]

Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com
#23 lore raw

Re: [PATCH net-next V2 08/10] net/mlx5: LAG: replace fixed MLX5_MAX_PORTS arrays with dynamic allocation

Quoted 45 lines · On Wed, 18 Nov 2026 10:31:57 +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider.

Critical: 0 · High: 1 · Medium: 0 · Low: 1

If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).

- [High] Possible heap out-of-bounds read/write in mlx5_mpesw_metadata_cleanup()…
- [Low] The commit message ends with "Thus, removes the hard dependency on MLX5_MAX_PORTS", which overstates what the patch does.

--

net/mlx5: LAG: replace fixed MLX5_MAX_PORTS arrays with dynamic allocation

This converts lag_mpesw.pf_metadata and the mdev array in mlx5_lag_query_cong_counters() from fixed MLX5_MAX_PORTS sized arrays to dynamically allocated buffers. pf_metadata is sized by ldev->max_fns when MPESW is enabled. The cong counters array is sized by ldev->ports.

Quoted 1 line

Thus, removes the hard dependency on MLX5_MAX_PORTS.

[Severity: Low]
Is this statement accurate? This patch converts only two arrays.

This patch leaves the devs[MLX5_MAX_PORTS] scratch array in mlx5_lag_rescan_devices_locked_filter() in place. The later series patch "drop MLX5_MAX_PORTS scratch array from device rescan" removes it.

Some arrays tied to the LAG member index space are still there at the end of the series. That index space is no longer capped at MLX5_MAX_PORTS, because "decouple iteration end-of-walk from MLX5_MAX_PORTS" moved xa_alloc() to XA_LIMIT(0, U8_MAX - 1):

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h:struct lag_tracker {
    ...
    struct netdev_lag_lower_state_info  netdev_state[MLX5_MAX_PORTS];
    ...
}
Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/port_sel.h:struct mlx5_lag_port_sel {
    ...
    struct mlx5_flow_handle *rules[MLX5_MAX_PORTS * MLX5_LAG_MAX_HASH_BUCKETS];
    ...
}

Could the commit message say that these remain? Or should they be converted or bounded as well?

The sentence refers to the two arrays this patch converts. tracker.netdev_state[] and the port_sel rules[] keep their MLX5_MAX_PORTS size on purpose.

Quoted 28 lines
Quoted 28 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +6 -0
index 49602f8306155..8a867a04916a8 100644
@@ -15,6 +15,9 @@ static void mlx5_mpesw_metadata_cleanup(struct mlx5_lag *ldev)
 	u32 pf_metadata;
 	int i;
 
+	if (!ldev->lag_mpesw.pf_metadata)
+		return;
+
 	mlx5_lag_for_each(i, 0, ldev, MLX5_LAG_FILTER_ALL) {
 		dev = mlx5_lag_fn(ldev, i)->dev;
 		esw = dev->priv.eswitch;
@@ -27,6 +30,9 @@ static void mlx5_mpesw_metadata_cleanup(struct mlx5_lag *ldev)
 		mlx5_esw_match_metadata_free(esw, pf_metadata);
 		ldev->lag_mpesw.pf_metadata[i] = 0;
 	}
+
+	kfree(ldev->lag_mpesw.pf_metadata);
+	ldev->lag_mpesw.pf_metadata = NULL;
 }
 
 static int mlx5_mpesw_metadata_set(struct mlx5_lag *ldev)
@@ -36,6 +42,12 @@ static int mlx5_mpesw_metadata_set(struct mlx5_lag *ldev)
 	u32 pf_metadata;
 	int i, err;
 
Quoted 10 lines

drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c static int mlx5_mpesw_metadata_set(struct mlx5_lag *ldev) · line 50

Quoted 6 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c +6 -0
+	ldev->lag_mpesw.pf_metadata =
+		kcalloc(ldev->max_fns, sizeof(*ldev->lag_mpesw.pf_metadata),
+			GFP_KERNEL);
+	if (!ldev->lag_mpesw.pf_metadata)
+		return -ENOMEM;
+

[Severity: High]
Can ldev->max_fns grow after this allocation while MPESW is still active?

It shouldn't. This is a pre-existing bug: MPESW shouldn't be enabled while an SD secondary hasn't probed yet. A separate fix that rejects MPESW in this case is in the pipe.

Quoted 74 lines

The allocated length is not stored anywhere. Yet mlx5_mpesw_metadata_cleanup() walks the members with mlx5_lag_for_each(), which checks against the current max_fns:

#define mlx5_lag_for_each(i, start_index, ldev, filter) \
	for (int tmp = start_index; \
	     tmp = mlx5_get_next_lag_func(ldev, tmp, filter), \
	     i = tmp, tmp < (ldev)->max_fns; tmp++)

mlx5_ldev_add_mdev() grows max_fns without looking at the LAG mode:

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c:mlx5_ldev_add_mdev() {
    ...
	/* max_fns is the iteration ceiling; grow it, never shrink. */
	if (idx >= ldev->max_fns)
		ldev->max_fns = idx + 1;
    ...
}

sd_lag_init() calls it for each SD secondary, also without checking ldev->mode:

drivers/net/ethernet/mellanox/mlx5/core/lib/sd.c:sd_lag_init() {
    ...
	mlx5_sd_for_each_secondary(i, primary, pos) {
		err = mlx5_ldev_add_mdev(ldev, pos, sd->group_id);
    ...
}

Here is one possible sequence on an SD card:

devlink esw_multiport=true
mlx5_lag_enable_mpesw()
mlx5_mpesw_metadata_set()
kcalloc(max_fns = 2)

secondary PF probes later

    mlx5e_probe() -> mlx5_sd_init() -> sd_lag_init()
      mlx5_ldev_add_mdev()     /* idx 2, max_fns = 3 */

devlink esw_multiport=false, or unload
mlx5_lag_disable_mpesw()
mlx5_mpesw_metadata_cleanup()
reads and writes pf_metadata[2]

SD secondaries have fn->dev set, so the MLX5_LAG_FILTER_ALL walk visits them. Three things then go wrong for any index past the allocated count:

- cleanup reads pf_metadata[i] past the end of the slab object.
- if that stale value is non-zero, it calls mlx5_esw_acl_ingress_vport_metadata_update() on that eswitch, and mlx5_esw_match_metadata_free() with an ida id that was never allocated.
- it writes 0 to pf_metadata[i] past the end of the buffer.

With the old embedded u32 pf_metadata[MLX5_MAX_PORTS], a member added later at an index below MLX5_MAX_PORTS read a zero slot and was skipped.

The last patch in the series ("do not add a device to an already-active virt LAG") says SD secondaries join before MPESW can be enabled. Is that enforced anywhere?

- mlx5_devlink_esw_multiport_set()->mlx5_lag_mpesw_enable() does not check SD readiness.
- mlx5_phys_mpesw_validate()->mlx5_lag_check_prereq() only checks mlx5_lag_num_devs(ldev) == ldev->ports. That counts port members, not SD secondaries.

Would it be safer to record the allocated length and bound the cleanup loop by it? Another option is to refuse max_fns growth and SD joins while MPESW is active.
[ ... ]

--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com

author maintainer
#24 lore raw

PATCH v2 9/10 net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan

In reply to Tariq Toukan

From: Shay Drory <[email protected]>

mlx5_lag_rescan_devices_locked_filter() collected the participating LAG members into a fixed-size devs[MLX5_MAX_PORTS] array only to iterate over it a second time for the rescan calls. Drop the scratch array and re-walk the members directly with mlx5_lag_for_each().

This removes another dependency on the compile-time MLX5_MAX_PORTS bound with no functional change.

---
 drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +7 -5
index a7b12353b722..3e88b3f51612 100644
@@ -1340,9 +1340,7 @@ void mlx5_lag_rescan_dev_locked(struct mlx5_lag *ldev,
 static void mlx5_lag_rescan_devices_locked_filter(struct mlx5_lag *ldev,
 						  bool enable, u32 filter)
 {
-	struct mlx5_core_dev *devs[MLX5_MAX_PORTS];
 	struct lag_fn *fn;
-	int num_devs = 0;
 	int i;
 
 	mlx5_lag_assert_locked_transition(ldev, filter);
@@ -1356,12 +1354,16 @@ static void mlx5_lag_rescan_devices_locked_filter(struct mlx5_lag *ldev,
 			fn->dev->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
 		else
 			fn->dev->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
-		devs[num_devs++] = fn->dev;
 	}
 
 	mlx5_lag_drop_lock_for_reps(ldev, filter);
-	for (i = 0; i < num_devs; i++)
-		mlx5_rescan_drivers_locked(devs[i]);
+	mlx5_lag_for_each(i, 0, ldev, filter) {
+		fn = mlx5_lag_fn(ldev, i);
+		if (fn->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV)
+			continue;
+
+		mlx5_rescan_drivers_locked(fn->dev);
+	}
 	mlx5_lag_retake_lock_after_reps(ldev);
 }
 
Signature · 1 line
2.44.0
#25 lore raw

Re: [PATCH net-next V2 09/10] net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan

Sashiko has reviewed this patch and found no issues.
Quoted 7 lines

From: Shay Drory <[email protected]>

mlx5_lag_rescan_devices_locked_filter() collected the participating LAG members into a fixed-size devs[MLX5_MAX_PORTS] array only to iterate over it a second time for the rescan calls. Drop the scratch array and re-walk the members directly with mlx5_lag_for_each().

Quoted 7 lines

This removes another dependency on the compile-time MLX5_MAX_PORTS bound with no functional change.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
#26 lore raw

Re: [PATCH net-next V2 09/10] net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan

In reply to Tariq Toukan
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.

Critical: 0 · High: 1 · Medium: 0 · Low: 0

If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).

- [High] In mlx5_lag_rescan_devices_locked_filter() (drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c), this patch…

--

net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan

This commit removes the devs[MLX5_MAX_PORTS] array from mlx5_lag_rescan_devices_locked_filter(). The post-unlock rescan loop now walks the LAG members again with mlx5_lag_for_each(). The commit message describes this as having no functional change.

Quoted 4 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +0 -0
index a7b12353b7225..3e88b3f51612e 100644

[ ... ]

Quoted 7 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +0 -1
@@ -1356,12 +1354,16 @@ static void mlx5_lag_rescan_devices_locked_filter(struct mlx5_lag *ldev,
 			fn->dev->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
 		else
 			fn->dev->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
-		devs[num_devs++] = fn->dev;
 	}
 

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c static void mlx5_lag_rescan_devices_locked_filter(struct mlx5_lag *ldev, · line 1366

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +7 -2
 	mlx5_lag_drop_lock_for_reps(ldev, filter);
-	for (i = 0; i < num_devs; i++)
-		mlx5_rescan_drivers_locked(devs[i]);
+	mlx5_lag_for_each(i, 0, ldev, filter) {
+		fn = mlx5_lag_fn(ldev, i);
+		if (fn->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV)
+			continue;
+
+		mlx5_rescan_drivers_locked(fn->dev);
+	}

[Severity: High]
This loop now walks ldev->fns after mlx5_lag_drop_lock_for_reps() has released ldev->lock. Can a concurrent SD member removal free a lag_fn while the walk is in progress?

On each step, mlx5_get_next_lag_func() reads xarray entries and fn->dev. mlx5_lag_fn() then does an xa_load(), and the loop dereferences fn->dev->priv.flags. The only thing guarding membership while the lock is dropped is ldev->mode_changes_in_progress. The comment in mlx5_lag_drop_lock_for_reps() says:

	/* Keep PF membership stable while ldev->lock is dropped. Device add
	 * and remove paths observe mode_changes_in_progress and retry.
	 */

The SD paths in lib/sd.c don't appear to check that counter, though. sd_lag_cleanup() takes the lock directly:

	mutex_lock(&ldev->lock);
	mlx5_sd_for_each_secondary(i, primary, pos)
		mlx5_ldev_remove_mdev(ldev, pos);

mlx5_ldev_remove_mdev() then frees the entry right away, with no RCU deferral:

	fn->dev = NULL;
	dev->priv.lag = NULL;
	xa_erase(&ldev->fns, fn->idx);
	kfree(fn);

sd_lag_cleanup() is serialized only by sd->devcom. For MLX5_LAG_FILTER_ALL, this function runs under the HCA devcom instead. mlx5_do_bond_work() takes only the HCA devcom trylock plus ldev->lock, and not mlx5_mpesw_sd_devcoms_lock(). It reaches this function as follows:

mlx5_do_bond_work()
mlx5_do_bond()

    mlx5_lag_shared_fdb_create(ldev, ..., MLX5_LAG_MODE_SRIOV, 0)
      mlx5_lag_remove_devices_filter(ldev, MLX5_LAG_FILTER_ALL)
        mlx5_lag_rescan_devices_locked_filter()

mlx5_disable_lag() -> mlx5_lag_shared_fdb_destroy(ldev, 0) reaches it the same way. Group id 0 maps to MLX5_LAG_FILTER_ALL, which matches every member, SD secondaries included.

At the same time, removing an SD member's eth aux driver can run:

mlx5e_remove() / mlx5e_suspend()
mlx5_sd_cleanup()

    sd_lag_cleanup()
      mlx5_ldev_remove_mdev()

This removal can come from a sysfs unbind of mlx5_core.eth.N. It can also come from a reload of an SD secondary whose hca_devcom_comp is NULL or in a different component. Neither case holds the HCA devcom that the bond work holds.

If that happens, mlx5_get_next_lag_func() could read fn->dev from a freed lag_fn. mlx5_lag_fn() could also return NULL after xa_erase(), and the loop would then dereference it.

The old loop touched only the devs[] core_dev pointers once the lock was dropped. Those pointers stay valid across an aux unbind, so lag_fn and xarray state was never accessed without the lock.

A similar question applies to sd_lag_init(), which also takes ldev->lock without checking mode_changes_in_progress and calls mlx5_ldev_add_mdev(). Could the second loop then rescan a newly added member whose MLX5_PRIV_FLAGS_DISABLE_IB_ADEV flag the first loop never updated?

Given the above, is the "no functional change" statement in the commit message accurate?

Would it be safer to keep a snapshot taken under the lock? For example, a dynamically sized array, or a reference on each core_dev collected before mlx5_lag_drop_lock_for_reps().

As far as I can tell, the last patch in the series doesn't touch this function or the SD paths, so this would still be present at the end of the series.

Quoted 3 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +0 -0
 	mlx5_lag_retake_lock_after_reps(ldev);
 }
 
Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com
#27 lore raw

Re: [PATCH net-next V2 09/10] net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan

Quoted 37 lines · On Wed, 18 Nov 2026 10:31:59 +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.

Critical: 0 · High: 1 · Medium: 0 · Low: 0

If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).

- [High] In mlx5_lag_rescan_devices_locked_filter() (drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c), this patch…

--

net/mlx5: LAG, drop MLX5_MAX_PORTS scratch array from device rescan

This commit removes the devs[MLX5_MAX_PORTS] array from mlx5_lag_rescan_devices_locked_filter(). The post-unlock rescan loop now walks the LAG members again with mlx5_lag_for_each(). The commit message describes this as having no functional change.

Quoted 4 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +0 -0
index a7b12353b7225..3e88b3f51612e 100644

[ ... ]

Quoted 7 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +0 -1
@@ -1356,12 +1354,16 @@ static void mlx5_lag_rescan_devices_locked_filter(struct mlx5_lag *ldev,
 			fn->dev->priv.flags &= ~MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
 		else
 			fn->dev->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_IB_ADEV;
-		devs[num_devs++] = fn->dev;
 	}
 
Quoted 15 lines

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c static void mlx5_lag_rescan_devices_locked_filter(struct mlx5_lag *ldev, · line 1366

Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +7 -2
 	mlx5_lag_drop_lock_for_reps(ldev, filter);
-	for (i = 0; i < num_devs; i++)
-		mlx5_rescan_drivers_locked(devs[i]);
+	mlx5_lag_for_each(i, 0, ldev, filter) {
+		fn = mlx5_lag_fn(ldev, i);
+		if (fn->dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV)
+			continue;
+
+		mlx5_rescan_drivers_locked(fn->dev);
+	}

[Severity: High]
This loop now walks ldev->fns after mlx5_lag_drop_lock_for_reps() has released ldev->lock. Can a concurrent SD member removal free a lag_fn while the walk is in progress?

Valid. The race between SD join/leave and a LAG transition that drops ldev->lock is pre-existing; SD join/leave doesn't observe mode_changes_in_progress. It will be fixed in a follow-up series.

Quoted 82 lines

On each step, mlx5_get_next_lag_func() reads xarray entries and fn->dev. mlx5_lag_fn() then does an xa_load(), and the loop dereferences fn->dev->priv.flags. The only thing guarding membership while the lock is dropped is ldev->mode_changes_in_progress. The comment in mlx5_lag_drop_lock_for_reps() says:

	/* Keep PF membership stable while ldev->lock is dropped. Device add
	 * and remove paths observe mode_changes_in_progress and retry.
	 */

The SD paths in lib/sd.c don't appear to check that counter, though. sd_lag_cleanup() takes the lock directly:

	mutex_lock(&ldev->lock);
	mlx5_sd_for_each_secondary(i, primary, pos)
		mlx5_ldev_remove_mdev(ldev, pos);

mlx5_ldev_remove_mdev() then frees the entry right away, with no RCU deferral:

	fn->dev = NULL;
	dev->priv.lag = NULL;
	xa_erase(&ldev->fns, fn->idx);
	kfree(fn);

sd_lag_cleanup() is serialized only by sd->devcom. For MLX5_LAG_FILTER_ALL, this function runs under the HCA devcom instead. mlx5_do_bond_work() takes only the HCA devcom trylock plus ldev->lock, and not mlx5_mpesw_sd_devcoms_lock(). It reaches this function as follows:

mlx5_do_bond_work()
mlx5_do_bond()

    mlx5_lag_shared_fdb_create(ldev, ..., MLX5_LAG_MODE_SRIOV, 0)
      mlx5_lag_remove_devices_filter(ldev, MLX5_LAG_FILTER_ALL)
        mlx5_lag_rescan_devices_locked_filter()

mlx5_disable_lag() -> mlx5_lag_shared_fdb_destroy(ldev, 0) reaches it the same way. Group id 0 maps to MLX5_LAG_FILTER_ALL, which matches every member, SD secondaries included.

At the same time, removing an SD member's eth aux driver can run:

mlx5e_remove() / mlx5e_suspend()
mlx5_sd_cleanup()

    sd_lag_cleanup()
      mlx5_ldev_remove_mdev()

This removal can come from a sysfs unbind of mlx5_core.eth.N. It can also come from a reload of an SD secondary whose hca_devcom_comp is NULL or in a different component. Neither case holds the HCA devcom that the bond work holds.

If that happens, mlx5_get_next_lag_func() could read fn->dev from a freed lag_fn. mlx5_lag_fn() could also return NULL after xa_erase(), and the loop would then dereference it.

The old loop touched only the devs[] core_dev pointers once the lock was dropped. Those pointers stay valid across an aux unbind, so lag_fn and xarray state was never accessed without the lock.

A similar question applies to sd_lag_init(), which also takes ldev->lock without checking mode_changes_in_progress and calls mlx5_ldev_add_mdev(). Could the second loop then rescan a newly added member whose MLX5_PRIV_FLAGS_DISABLE_IB_ADEV flag the first loop never updated?

Given the above, is the "no functional change" statement in the commit message accurate?

Would it be safer to keep a snapshot taken under the lock? For example, a dynamically sized array, or a reference on each core_dev collected before mlx5_lag_drop_lock_for_reps().

As far as I can tell, the last patch in the series doesn't touch this function or the SD paths, so this would still be present at the end of the series.

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c static void mlx5_lag_rescan_devices_locked_filter(struct mlx5_lag *ldev, · line 1369

Quoted 3 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +0 -0
 	mlx5_lag_retake_lock_after_reps(ldev);
 }
 

--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com

author maintainer
#28 lore raw

PATCH v2 10/10 net/mlx5: LAG, do not add a device to an already-active virt LAG

In reply to Tariq Toukan

From: Shay Drory <[email protected]>

mlx5_ldev_add_mdev() would add a newly probed device to the LAG even when the LAG is already active, mutating membership while getters may be iterating it and while the active mapping assumes a fixed member set.

A LAG of VFs/SFs is the only one whose members come independently of its activation. Reject the join of a virt LAG capable function when the group's virt LAG is active. The device stays out of the LAG until it is reprobed and the LAG is disabled.

The other join paths cannot reach an active LAG. A LAG of PFs is activated only once all the PFs of the card are members, so there is no further PF that could join it, and SD secondaries join at load, before MPESW can be enabled. Hence the check is limited to virt LAG capable functions, and it is a no-op for existing configurations.

In addition, do not register the eswitch devcom component of such a device, so it does not pair with the eswitches of the LAG members.

---
 .../net/ethernet/mellanox/mlx5/core/eswitch_offloads.c   | 4 ++++
 drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c        | 9 +++++++++
 drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h        | 5 +++++
 3 files changed, 18 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +4 -0
index 474784131091..1323114ce7f9 100644
@@ -3572,6 +3572,10 @@ void mlx5_esw_offloads_devcom_init(struct mlx5_eswitch *esw,
 	xa_init(&esw->offloads.peer_flows);
 	mutex_init(&esw->offloads.peer_mutex);
 
+	/* do not pair a virt LAG capable VF/SF that is not part of a LAG */
+	if (!mlx5_core_is_pf(esw->dev) && mlx5_virt_lag_nonmember(esw->dev))
+		return;
+
 	if (!MLX5_CAP_ESW(esw->dev, merged_eswitch))
 		return;
 
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +9 -0
index 3e88b3f51612..abcab76fced0 100644
@@ -2448,6 +2448,11 @@ static int __mlx5_lag_dev_add_mdev(struct mlx5_core_dev *dev)
 		mutex_unlock(&ldev->lock);
 		return -EAGAIN;
 	}
+
+	if (mlx5_virt_lag_is_supported(dev) && __mlx5_lag_is_active(ldev)) {
+		mutex_unlock(&ldev->lock);
+		return -EOPNOTSUPP;
+	}
 	mlx5_ldev_get(ldev);
 	err = mlx5_ldev_add_mdev(ldev, dev, 0);
 	if (err) {
@@ -2536,6 +2541,10 @@ void mlx5_lag_add_mdev(struct mlx5_core_dev *dev)
 	err = __mlx5_lag_dev_add_mdev(dev);
 	mlx5_devcom_comp_unlock(dev->priv.hca_devcom_comp);
 
+	if (err == -EOPNOTSUPP) {
+		mlx5_lag_unregister_hca_devcom_comp(dev);
+		return;
+	}
 	if (err) {
 		msleep(100);
 		goto recheck;
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h +5 -0
index 330f5ab83a18..61afccba68ca 100644
@@ -271,6 +271,11 @@ static inline void mlx5_lag_set_vports_agg_speed(struct mlx5_lag *ldev) {}
 static inline void mlx5_lag_reset_vports_speed(struct mlx5_lag *ldev) {}
 #endif
 
+static inline bool mlx5_virt_lag_nonmember(struct mlx5_core_dev *dev)
+{
+	return mlx5_virt_lag_is_supported(dev) && !mlx5_lag_dev(dev);
+}
+
 static inline bool mlx5_lag_is_supported(struct mlx5_core_dev *dev)
 {
 	if (!MLX5_CAP_GEN(dev, vport_group_manager) ||
Signature · 1 line
2.44.0
#29 lore raw

Re: [PATCH net-next V2 10/10] net/mlx5: LAG, do not add a device to an already-active virt LAG

Sashiko has reviewed this patch and found no issues.
Quoted 17 lines

From: Shay Drory <[email protected]>

mlx5_ldev_add_mdev() would add a newly probed device to the LAG even when the LAG is already active, mutating membership while getters may be iterating it and while the active mapping assumes a fixed member set.

A LAG of VFs/SFs is the only one whose members come independently of its activation. Reject the join of a virt LAG capable function when the group's virt LAG is active. The device stays out of the LAG until it is reprobed and the LAG is disabled.

The other join paths cannot reach an active LAG. A LAG of PFs is activated only once all the PFs of the card are members, so there is no further PF that could join it, and SD secondaries join at load, before MPESW can be enabled. Hence the check is limited to virt LAG capable functions, and it is a no-op for existing configurations.

Quoted 7 lines

In addition, do not register the eswitch devcom component of such a device, so it does not pair with the eswitches of the LAG members.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
#30 lore raw

Re: [PATCH net-next V2 10/10] net/mlx5: LAG, do not add a device to an already-active virt LAG

In reply to Tariq Toukan
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.

Critical: 0 · High: 0 (1) · Medium: 1 · Low: 0

If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).

- [Medium] In this tree, both parts of the patch hit a predicate mismatch.

Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but the patch now cleans up the HCA devcom component on its own new -EOPNOTSUPP path…

--

net/mlx5: LAG, do not add a device to an already-active virt LAG

This patch stops a virt LAG capable function from joining a LAG whose mode is already active. In that case __mlx5_lag_dev_add_mdev() returns -EOPNOTSUPP, and mlx5_lag_add_mdev() then unregisters the HCA devcom component. The patch also makes mlx5_esw_offloads_devcom_init() skip eswitch devcom registration for a non-PF virt LAG capable function that has no ldev.

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c static int __mlx5_lag_dev_add_mdev(struct mlx5_core_dev *dev) · line 2455

Quoted 13 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +5 -0
index 3e88b3f51612e..abcab76fced0b 100644
@@ -2448,6 +2448,11 @@ static int __mlx5_lag_dev_add_mdev(struct mlx5_core_dev *dev)
 		mutex_unlock(&ldev->lock);
 		return -EAGAIN;
 	}
+
+	if (mlx5_virt_lag_is_supported(dev) && __mlx5_lag_is_active(ldev)) {
+		mutex_unlock(&ldev->lock);
+		return -EOPNOTSUPP;
+	}

[Severity: Medium]

Can this condition ever be true in this tree?

mlx5_lag_add_mdev() is the only caller of __mlx5_lag_dev_add_mdev(). It returns early unless mlx5_lag_is_supported() passes:

mlx5_lag_add_mdev() {
	if (!mlx5_lag_is_supported(dev))
		return;
	...
}

mlx5_lag_is_supported() requires num_lag_ports >= 2:

	if (!MLX5_CAP_GEN(dev, vport_group_manager) ||
	    !MLX5_CAP_GEN(dev, lag_master) ||
	    MLX5_CAP_GEN(dev, num_lag_ports) < 2 ||
	...

mlx5_virt_lag_is_supported(), on the other hand, requires num_lag_ports < 2:

	if (!MLX5_CAP_GEN(dev, multi_uplink_eswitch) ||
	    !MLX5_CAP_GEN(dev, lag_master) ||
	    MLX5_CAP_GEN(dev, num_lag_ports) >= 2)
		return false;

No device can satisfy both, so this -EOPNOTSUPP return and the new cleanup branch in mlx5_lag_add_mdev() look unreachable.

The commit message says:

"mlx5_ldev_add_mdev() would add a newly probed device to the LAG even when the LAG is already active"

Can that happen today for a virt LAG capable function? If this check only prepares for a later series that enables virt LAG, could the commit message say so?

The same mismatch seems to affect the eswitch change. A non-PF virt LAG capable function never reaches mlx5_ldev_add_mdev() here, because sd_lag_init() only handles PF SD secondaries. So dev->priv.lag stays NULL for it, and mlx5_virt_lag_nonmember() is always true:

static inline bool mlx5_virt_lag_nonmember(struct mlx5_core_dev *dev)

{
	return mlx5_virt_lag_is_supported(dev) && !mlx5_lag_dev(dev);
}

As a result, the new early return in mlx5_esw_offloads_devcom_init() fires for every such VF/SF eswitch:

	if (!mlx5_core_is_pf(esw->dev) && mlx5_virt_lag_nonmember(esw->dev))
		return;

Before this patch, a VF/SF eswitch with merged_eswitch would register MLX5_DEVCOM_ESW_OFFLOADS and send ESW_OFFLOADS_DEVCOM_PAIR. The existing MLX5_VPORT_MANAGER() / !mlx5_lag_is_supported() gate further down does not stop a non-PF, because MLX5_VPORT_MANAGER() requires mlx5_core_is_pf().

Does this drop eswitch devcom pairing for those VF/SF eswitches? That would contradict "it is a no-op for existing configurations".

The commit message also limits the devcom change to "such a device", meaning one rejected from an active LAG. The condition, however, matches any virt LAG capable VF/SF that has no ldev.

Once virt LAG joins are allowed, devices that are left out because mlx5_lag_dev_alloc(), mlx5_ldev_add_mdev() or mlx5_lag_register_hca_devcom_comp() failed would also be treated as non-members. Is that intended?

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c void mlx5_lag_add_mdev(struct mlx5_core_dev *dev) · line 2547

Quoted 11 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +4 -0
 	mlx5_ldev_get(ldev);
 	err = mlx5_ldev_add_mdev(ldev, dev, 0);
 	if (err) {
@@ -2536,6 +2541,10 @@ void mlx5_lag_add_mdev(struct mlx5_core_dev *dev)
 	err = __mlx5_lag_dev_add_mdev(dev);
 	mlx5_devcom_comp_unlock(dev->priv.hca_devcom_comp);
 
+	if (err == -EOPNOTSUPP) {
+		mlx5_lag_unregister_hca_devcom_comp(dev);
+		return;
+	}

[Severity: High]

This isn't a bug introduced by this patch, but this new path unregisters the HCA devcom component, while the nearby failure paths in __mlx5_lag_dev_add_mdev() still leave it registered.

If no peer ldev exists and mlx5_lag_dev_alloc() or mlx5_ldev_add_mdev() fails, __mlx5_lag_dev_add_mdev() returns 0 with dev->priv.lag still NULL:

	if (!ldev) {
		ldev = mlx5_lag_dev_alloc(dev);
		if (!ldev) {
			mlx5_core_err(dev, "Failed to alloc lag dev\n");
			return 0;
		}
		err = mlx5_ldev_add_mdev(ldev, dev, 0);
		if (err) {
			mlx5_core_err(dev, "Failed to add mdev to lag dev\n");
			mlx5_ldev_put(ldev);
			return 0;
		}
		return 0;
	}

mlx5_lag_add_mdev() treats that as success, sends LAG_DEVCOM_PAIR and keeps dev->priv.hca_devcom_comp. On teardown, mlx5_lag_remove_mdev() returns before it reaches mlx5_lag_unregister_hca_devcom_comp():

	ldev = mlx5_lag_dev(dev);
	if (!ldev)
		return;

The comp_dev's data still points at this mlx5_core_dev. Does that comp_dev stay on the shared component list after the core dev is freed?

If so, a peer with the same key that probes or reloads would get the freed dev back from mlx5_devcom_get_next_peer_data() in __mlx5_lag_dev_add_mdev(). It would then call mlx5_lag_dev(tmp_dev) on freed memory.

A peer's LAG_DEVCOM_PAIR/UNPAIR event would likewise call mlx5_lag_devcom_event() with the freed dev. That reads dev->priv.lag and can call mutex_lock() on a garbage ldev.

Could the component also be unregistered on those allocation failure paths?

Quoted 3 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +0 -0
 	if (err) {
 		msleep(100);
 		goto recheck;

[ ... ]

Signature · 1 line
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com
#31 lore raw

Re: [PATCH net-next V2 10/10] net/mlx5: LAG, do not add a device to an already-active virt LAG

Quoted 73 lines · On Wed, 18 Nov 2026 10:32:00 +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider.

Critical: 0 · High: 0 (1) · Medium: 1 · Low: 0

If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough).

- [Medium] In this tree, both parts of the patch hit a predicate mismatch.

Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but the patch now cleans up the HCA devcom component on its own new -EOPNOTSUPP path…

--

net/mlx5: LAG, do not add a device to an already-active virt LAG

This patch stops a virt LAG capable function from joining a LAG whose mode is already active. In that case __mlx5_lag_dev_add_mdev() returns -EOPNOTSUPP, and mlx5_lag_add_mdev() then unregisters the HCA devcom component. The patch also makes mlx5_esw_offloads_devcom_init() skip eswitch devcom registration for a non-PF virt LAG capable function that has no ldev.

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c static int __mlx5_lag_dev_add_mdev(struct mlx5_core_dev *dev) · line 2455

Quoted 13 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +5 -0
index 3e88b3f51612e..abcab76fced0b 100644
@@ -2448,6 +2448,11 @@ static int __mlx5_lag_dev_add_mdev(struct mlx5_core_dev *dev)
 		mutex_unlock(&ldev->lock);
 		return -EAGAIN;
 	}
+
+	if (mlx5_virt_lag_is_supported(dev) && __mlx5_lag_is_active(ldev)) {
+		mutex_unlock(&ldev->lock);
+		return -EOPNOTSUPP;
+	}

[Severity: Medium]

Can this condition ever be true in this tree?

mlx5_lag_add_mdev() is the only caller of __mlx5_lag_dev_add_mdev(). It returns early unless mlx5_lag_is_supported() passes:

mlx5_lag_add_mdev() {
	if (!mlx5_lag_is_supported(dev))
		return;
	...
}

mlx5_lag_is_supported() requires num_lag_ports >= 2:

	if (!MLX5_CAP_GEN(dev, vport_group_manager) ||
	    !MLX5_CAP_GEN(dev, lag_master) ||
	    MLX5_CAP_GEN(dev, num_lag_ports) < 2 ||
	...

mlx5_virt_lag_is_supported(), on the other hand, requires num_lag_ports < 2:

	if (!MLX5_CAP_GEN(dev, multi_uplink_eswitch) ||
	    !MLX5_CAP_GEN(dev, lag_master) ||
	    MLX5_CAP_GEN(dev, num_lag_ports) >= 2)
		return false;

No device can satisfy both, so this -EOPNOTSUPP return and the new cleanup branch in mlx5_lag_add_mdev() look unreachable.

Quoted 8 lines

The commit message says:

"mlx5_ldev_add_mdev() would add a newly probed device to the LAG even when the LAG is already active"

Can that happen today for a virt LAG capable function? If this check only prepares for a later series that enables virt LAG, could the commit message say so?

No, not in this series, and that is intended. As the cover letter says, this series only prepares the LAG core. The follow-up series admits virt LAG capable functions in mlx5_lag_add_mdev(), and only then can the check fire.

Quoted 16 lines

The same mismatch seems to affect the eswitch change. A non-PF virt LAG capable function never reaches mlx5_ldev_add_mdev() here, because sd_lag_init() only handles PF SD secondaries. So dev->priv.lag stays NULL for it, and mlx5_virt_lag_nonmember() is always true:

static inline bool mlx5_virt_lag_nonmember(struct mlx5_core_dev *dev)

{
	return mlx5_virt_lag_is_supported(dev) && !mlx5_lag_dev(dev);
}

As a result, the new early return in mlx5_esw_offloads_devcom_init() fires for every such VF/SF eswitch:

	if (!mlx5_core_is_pf(esw->dev) && mlx5_virt_lag_nonmember(esw->dev))
		return;
Quoted 7 lines

Before this patch, a VF/SF eswitch with merged_eswitch would register MLX5_DEVCOM_ESW_OFFLOADS and send ESW_OFFLOADS_DEVCOM_PAIR. The existing MLX5_VPORT_MANAGER() / !mlx5_lag_is_supported() gate further down does not stop a non-PF, because MLX5_VPORT_MANAGER() requires mlx5_core_is_pf().

Does this drop eswitch devcom pairing for those VF/SF eswitches? That would contradict "it is a no-op for existing configurations".

No existing VF/SF passes mlx5_virt_lag_is_supported(). New FW will allow a VF/SF to report multi_uplink_eswitch and lag_master with num_lag_ports < 2, so the skip changes nothing for existing configurations.

For such a VF/SF, skipping the pairing outside a LAG is intended. It manages its own nested E-switch, and it pairs with other E-switches only as a member of the virt LAG's shared FDB.

Quoted 8 lines

The commit message also limits the devcom change to "such a device", meaning one rejected from an active LAG. The condition, however, matches any virt LAG capable VF/SF that has no ldev.

Once virt LAG joins are allowed, devices that are left out because mlx5_lag_dev_alloc(), mlx5_ldev_add_mdev() or mlx5_lag_register_hca_devcom_comp() failed would also be treated as non-members. Is that intended?

Yes. Whatever the reason the device was left out, it is not a LAG member, so it must not pair with the LAG members' E-switches.

Quoted 65 lines

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c void mlx5_lag_add_mdev(struct mlx5_core_dev *dev) · line 2547

Quoted 11 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +4 -0
 	mlx5_ldev_get(ldev);
 	err = mlx5_ldev_add_mdev(ldev, dev, 0);
 	if (err) {
@@ -2536,6 +2541,10 @@ void mlx5_lag_add_mdev(struct mlx5_core_dev *dev)
 	err = __mlx5_lag_dev_add_mdev(dev);
 	mlx5_devcom_comp_unlock(dev->priv.hca_devcom_comp);
 
+	if (err == -EOPNOTSUPP) {
+		mlx5_lag_unregister_hca_devcom_comp(dev);
+		return;
+	}

[Severity: High]

This isn't a bug introduced by this patch, but this new path unregisters the HCA devcom component, while the nearby failure paths in __mlx5_lag_dev_add_mdev() still leave it registered.

If no peer ldev exists and mlx5_lag_dev_alloc() or mlx5_ldev_add_mdev() fails, __mlx5_lag_dev_add_mdev() returns 0 with dev->priv.lag still NULL:

	if (!ldev) {
		ldev = mlx5_lag_dev_alloc(dev);
		if (!ldev) {
			mlx5_core_err(dev, "Failed to alloc lag dev\n");
			return 0;
		}
		err = mlx5_ldev_add_mdev(ldev, dev, 0);
		if (err) {
			mlx5_core_err(dev, "Failed to add mdev to lag dev\n");
			mlx5_ldev_put(ldev);
			return 0;
		}
		return 0;
	}

mlx5_lag_add_mdev() treats that as success, sends LAG_DEVCOM_PAIR and keeps dev->priv.hca_devcom_comp. On teardown, mlx5_lag_remove_mdev() returns before it reaches mlx5_lag_unregister_hca_devcom_comp():

	ldev = mlx5_lag_dev(dev);
	if (!ldev)
		return;

The comp_dev's data still points at this mlx5_core_dev. Does that comp_dev stay on the shared component list after the core dev is freed?

If so, a peer with the same key that probes or reloads would get the freed dev back from mlx5_devcom_get_next_peer_data() in __mlx5_lag_dev_add_mdev(). It would then call mlx5_lag_dev(tmp_dev) on freed memory.

A peer's LAG_DEVCOM_PAIR/UNPAIR event would likewise call mlx5_lag_devcom_event() with the freed dev. That reads dev->priv.lag and can call mutex_lock() on a garbage ldev.

Could the component also be unregistered on those allocation failure paths?

drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c void mlx5_lag_add_mdev(struct mlx5_core_dev *dev) · line 2550

Quoted 3 lines
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +0 -0
 	if (err) {
 		msleep(100);
 		goto recheck;

[ ... ]

--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261116103010.1129666-1-tariqt%40nvidia.com