Skip to contentSkip to conversation

net: mana: Fix probe/remove error path bugs

PATCH Superseded Erni Sri Satya Vennela 2 participants 12 messages Apr 15 – Apr 20

Subsystems (4)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover net: mana: Fix probe/remove error path bugs

Fix five bugs in mana_probe()/mana_remove() error handling that can cause warnings on uninitialized work structs, NULL pointer dereferences, masked errors, and resource leaks when early probe steps fail.

Patches 1-2 move work struct initialization (link_change_work and gf_stats_work) to before any error path that could trigger mana_remove(), preventing WARN_ON in __flush_work() or debug object warnings when sync cancellation runs on uninitialized work structs.

Patch 3 guards mana_remove() against double invocation. If PM resume fails, mana_probe() calls mana_remove() which sets gdma_context and driver_data to NULL. A failed resume does not unbind the driver, so when the device is eventually unbound, mana_remove() is called again and dereferences NULL, causing a kernel panic. An early return on NULL gdma_context or driver_data makes the second call harmless.

Patch 4 prevents add_adev() from overwriting a port probe error, which could leave the driver in a broken state with NULL ports while reporting success.

Patch 5 changes 'goto out' to 'break' in mana_remove()'s port loop so that mana_destroy_eq() is always reached, preventing EQ leaks when a NULL port is encountered.
---
Changes in v3:
* Add patch 3: net: mana: Guard mana_remove against double invocation.
* Fix inaccurate comments.
* Correct Fixes tag from ca9c54d2d6a5 to 1e2d0824a9c3. Changes in v2:
* Apply the patchset in net instead of net-next.
---
Erni Sri Satya Vennela (5):
net: mana: Init link_change_work before potential error paths in probe
net: mana: Init gf_stats_work before potential error paths in probe
net: mana: Guard mana_remove against double invocation
net: mana: Don't overwrite port probe error with add_adev result
net: mana: Fix EQ leak in mana_remove on NULL port

 drivers/net/ethernet/microsoft/mana/mana_en.c | 35 +++++++++++--------
 1 file changed, 20 insertions(+), 15 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v3 1/5 net: mana: Init link_change_work before potential error paths in probe

Move INIT_WORK(link_change_work) to right after the mana_context allocation, before any error path that could reach mana_remove().

Previously, if mana_create_eq() or mana_query_device_cfg() failed, mana_probe() would jump to the error path which calls mana_remove(). mana_remove() unconditionally calls disable_work_sync(link_change_work), but the work struct had not been initialized yet. This can trigger CONFIG_DEBUG_OBJECTS_WORK enabled.

Fixes 54133f9b4b53 ("net: mana: Support HW link state events")

---
Changes in v3:
* No change.
Changes in v2:
* Apply the patch in net instead of net-next.

---
 drivers/net/ethernet/microsoft/mana/mana_en.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/microsoft/mana/mana_en.c +2 -2
index 6302432b9bf6..e3e4b6de6668 100644
@@ -3631,6 +3631,8 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
 
 		ac->gdma_dev = gd;
 		gd->driver_data = ac;
+
+		INIT_WORK(&ac->link_change_work, mana_link_state_handle);
 	}
 
 	err = mana_create_eq(ac);
@@ -3648,8 +3650,6 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
 
 	if (!resuming) {
 		ac->num_ports = num_ports;
-
-		INIT_WORK(&ac->link_change_work, mana_link_state_handle);
 	} else {
 		if (ac->num_ports != num_ports) {
 			dev_err(dev, "The number of vPorts changed: %d->%d\n",
Signature · 1 line
2.34.1
reviewer Reviewed-by on v3 1/5
#2 lore raw

Re: [PATCH net v3 1/5] net: mana: Init link_change_work before potential error paths in probe

Quoted 9 lines · Erni Sri Satya Vennela

Move INIT_WORK(link_change_work) to right after the mana_context allocation, before any error path that could reach mana_remove().

Previously, if mana_create_eq() or mana_query_device_cfg() failed, mana_probe() would jump to the error path which calls mana_remove(). mana_remove() unconditionally calls disable_work_sync(link_change_work), but the work struct had not been initialized yet. This can trigger CONFIG_DEBUG_OBJECTS_WORK enabled.

Quoted 7 lines
Fixes 54133f9b4b53 ("net: mana: Support HW link state events")

---
Changes in v3:
* No change.
Changes in v2:
* Apply the patch in net instead of net-next.

#3 lore raw

PATCH v3 2/5 net: mana: Init gf_stats_work before potential error paths in probe

In reply to Erni Sri Satya Vennela

Move INIT_DELAYED_WORK(gf_stats_work) to before mana_create_eq(), while keeping schedule_delayed_work() at its original location.

Previously, if any function between mana_create_eq() and the INIT_DELAYED_WORK call failed, mana_probe() would call mana_remove() which unconditionally calls cancel_delayed_work_sync(gf_stats_work) in __flush_work() or debug object warnings with CONFIG_DEBUG_OBJECTS_WORK enabled.

Fixes be4f1d67ec56 ("net: mana: Add standard counter rx_missed_errors")

---
Changes in v3:
* No change
Changes in v2:
* Apply the patch in net instead of net-next.

---
 drivers/net/ethernet/microsoft/mana/mana_en.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ethernet/microsoft/mana/mana_en.c +2 -1
index e3e4b6de6668..468ed60a8a00 100644
@@ -3635,6 +3635,8 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
 		INIT_WORK(&ac->link_change_work, mana_link_state_handle);
 	}
 
+	INIT_DELAYED_WORK(&ac->gf_stats_work, mana_gf_stats_work_handler);
+
 	err = mana_create_eq(ac);
 	if (err) {
 		dev_err(dev, "Failed to create EQs: %d\n", err);
@@ -3709,7 +3711,6 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
 
 	err = add_adev(gd, "eth");
 
-	INIT_DELAYED_WORK(&ac->gf_stats_work, mana_gf_stats_work_handler);
 	schedule_delayed_work(&ac->gf_stats_work, MANA_GF_STATS_PERIOD);
 
 out:
Signature · 1 line
2.34.1
reviewer Reviewed-by on v3 2/5
#4 lore raw

Re: [PATCH net v3 2/5] net: mana: Init gf_stats_work before potential error paths in probe

Quoted 9 lines · Erni Sri Satya Vennela

Move INIT_DELAYED_WORK(gf_stats_work) to before mana_create_eq(), while keeping schedule_delayed_work() at its original location.

Previously, if any function between mana_create_eq() and the INIT_DELAYED_WORK call failed, mana_probe() would call mana_remove() which unconditionally calls cancel_delayed_work_sync(gf_stats_work) in __flush_work() or debug object warnings with CONFIG_DEBUG_OBJECTS_WORK enabled.

Quoted 7 lines
Fixes be4f1d67ec56 ("net: mana: Add standard counter rx_missed_errors")

---
Changes in v3:
* No change
Changes in v2:
* Apply the patch in net instead of net-next.

#5 lore raw

PATCH v3 3/5 net: mana: Guard mana_remove against double invocation

In reply to Erni Sri Satya Vennela

If PM resume fails (e.g., mana_attach() returns an error), mana_probe() calls mana_remove(), which tears down the device and sets gd->gdma_context = NULL and gd->driver_data = NULL.

However, a failed resume callback does not automatically unbind the driver. When the device is eventually unbound, mana_remove() is invoked a second time. Without a NULL check, it dereferences gc->dev with gc == NULL, causing a kernel panic.

Add an early return if gdma_context or driver_data is NULL so the second invocation is harmless. Move the dev = gc->dev assignment after the guard so it cannot dereference NULL.

Fixes ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")

---
Changes in v3:
* Add this patch to the patchset

---
 drivers/net/ethernet/microsoft/mana/mana_en.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/net/ethernet/microsoft/mana/mana_en.c +6 -1
index 468ed60a8a00..ce1b7ec46a27 100644
@@ -3731,11 +3731,16 @@ void mana_remove(struct gdma_dev *gd, bool suspending)
 	struct gdma_context *gc = gd->gdma_context;
 	struct mana_context *ac = gd->driver_data;
 	struct mana_port_context *apc;
-	struct device *dev = gc->dev;
+	struct device *dev;
 	struct net_device *ndev;
 	int err;
 	int i;
 
+	if (!gc || !ac)
+		return;
+
+	dev = gc->dev;
+
 	disable_work_sync(&ac->link_change_work);
 	cancel_delayed_work_sync(&ac->gf_stats_work);
 
Signature · 1 line
2.34.1
reviewer
#6 lore raw

Re: [PATCH net v3 3/5] net: mana: Guard mana_remove against double invocation

This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.

For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
net: mana: Guard mana_remove against double invocation

If PM resume fails (e.g., mana_attach() returns an error), mana_probe() calls mana_remove(), which tears down the device and sets gd->gdma_context = NULL and gd->driver_data = NULL. However, a failed resume callback does not automatically unbind the driver. When the device is eventually unbound, mana_remove() is invoked a second time. Without a NULL check, it dereferences gc->dev with gc == NULL, causing a kernel panic. Add an early return if gdma_context or driver_data is NULL so the second invocation is harmless.

Quoted 1 line
Fixes ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")

The Fixes: tag points to ca9c54d2d6a5, which introduced mana_remove() without NULL checks. However, the exploitable double invocation scenario was actually introduced by commit 635096a86edb ("net: mana: Support hibernation and kexec"), which added suspend/resume support and made mana_probe() call mana_remove() on error.

Should the Fixes: tag be:

Fixes 635096a86edb ("net: mana: Support hibernation and kexec")
Quoted 21 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +6 -1
index 9d095a6fb56c..2ab7c89e2fed 100644
@@ -3685,11 +3685,16 @@ void mana_remove(struct gdma_dev *gd, bool suspending)
 	struct gdma_context *gc = gd->gdma_context;
 	struct mana_context *ac = gd->driver_data;
 	struct mana_port_context *apc;
-	struct device *dev = gc->dev;
+	struct device *dev;
 	struct net_device *ndev;
 	int err;
 	int i;
 
+	if (!gc || !ac)
+		return;
+
+	dev = gc->dev;
+
 	disable_work_sync(&ac->link_change_work);
 	cancel_delayed_work_sync(&ac->gf_stats_work);
#7 lore raw

Re: [PATCH net v3 3/5] net: mana: Guard mana_remove against double invocation

Quoted 42 lines · Simon Horman

This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.

For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
net: mana: Guard mana_remove against double invocation

If PM resume fails (e.g., mana_attach() returns an error), mana_probe() calls mana_remove(), which tears down the device and sets gd->gdma_context = NULL and gd->driver_data = NULL. However, a failed resume callback does not automatically unbind the driver. When the device is eventually unbound, mana_remove() is invoked a second time. Without a NULL check, it dereferences gc->dev with gc == NULL, causing a kernel panic. Add an early return if gdma_context or driver_data is NULL so the second invocation is harmless.

Quoted 1 line
Fixes ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")

The Fixes: tag points to ca9c54d2d6a5, which introduced mana_remove() without NULL checks. However, the exploitable double invocation scenario was actually introduced by commit 635096a86edb ("net: mana: Support hibernation and kexec"), which added suspend/resume support and made mana_probe() call mana_remove() on error.

Should the Fixes: tag be:

Fixes 635096a86edb ("net: mana: Support hibernation and kexec")
Quoted 14 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +1 -1
index 9d095a6fb56c..2ab7c89e2fed 100644
@@ -3685,11 +3685,16 @@ void mana_remove(struct gdma_dev *gd, bool suspending)
 	struct gdma_context *gc = gd->gdma_context;
 	struct mana_context *ac = gd->driver_data;
 	struct mana_port_context *apc;
-	struct device *dev = gc->dev;
+	struct device *dev;
 	struct net_device *ndev;
 	int err;
 	int i;
 
Quoted 7 lines

drivers/net/ethernet/microsoft/mana/mana_en.c void mana_remove(struct gdma_dev *gd, bool suspending) · line 3745

Quoted 7 lines
drivers/net/ethernet/microsoft/mana/mana_en.c +5 -0
+	if (!gc || !ac)
+		return;
+
+	dev = gc->dev;
+
 	disable_work_sync(&ac->link_change_work);
 	cancel_delayed_work_sync(&ac->gf_stats_work);

Thankyou for the correction, Simon.
I'll make this change in the next version.

- Vennela

#8 lore raw

PATCH v3 4/5 net: mana: Don't overwrite port probe error with add_adev result

In reply to Erni Sri Satya Vennela

In mana_probe(), if mana_probe_port() fails for any port, the error is stored in 'err' and the loop breaks. However, the subsequent unconditional 'err = add_adev(gd, "eth")' overwrites this error. If add_adev() succeeds, mana_probe() returns success despite ports being left in a partially initialized state (ac->ports[i] == NULL).

Only call add_adev() when there is no prior error, so the probe correctly fails and triggers mana_remove() cleanup.

Fixes ced82fce77e9 ("net: mana: Probe rdma device in mana driver")

---
Changes in v3:
* Fix inaccurate comments.
Changes in v2:
* Apply the patch in net instead of net-next.

---
 drivers/net/ethernet/microsoft/mana/mana_en.c | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)
drivers/net/ethernet/microsoft/mana/mana_en.c +8 -9
index ce1b7ec46a27..39b18577fb51 100644
@@ -3680,10 +3680,9 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
 	if (!resuming) {
 		for (i = 0; i < ac->num_ports; i++) {
 			err = mana_probe_port(ac, i, &ac->ports[i]);
-			/* we log the port for which the probe failed and stop
-			 * probes for subsequent ports.
-			 * Note that we keep running ports, for which the probes
-			 * were successful, unless add_adev fails too
+			/* Log the port for which the probe failed, stop probing
+			 * subsequent ports, and skip add_adev.
+			 * mana_remove() will clean up already-probed ports.
 			 */
 			if (err) {
 				dev_err(dev, "Probe Failed for port %d\n", i);
@@ -3697,10 +3696,9 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
 			enable_work(&apc->queue_reset_work);
 			err = mana_attach(ac->ports[i]);
 			rtnl_unlock();
-			/* we log the port for which the attach failed and stop
-			 * attach for subsequent ports
-			 * Note that we keep running ports, for which the attach
-			 * were successful, unless add_adev fails too
+			/* Log the port for which the attach failed, stop
+			 * attaching subsequent ports, and skip add_adev.
+			 * mana_remove() will clean up already-attached ports.
 			 */
 			if (err) {
 				dev_err(dev, "Attach Failed for port %d\n", i);
@@ -3709,7 +3707,8 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
 		}
 	}
 
-	err = add_adev(gd, "eth");
+	if (!err)
+		err = add_adev(gd, "eth");
 
 	schedule_delayed_work(&ac->gf_stats_work, MANA_GF_STATS_PERIOD);
 
Signature · 1 line
2.34.1
reviewer
#9 lore raw

Re: [PATCH net v3 4/5] net: mana: Don't overwrite port probe error with add_adev result

This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.

For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
net: mana: Don't overwrite port probe error with add_adev result

In mana_probe(), if mana_probe_port() fails for any port, the error is stored in 'err' and the loop breaks. However, the subsequent unconditional 'err = add_adev(gd, "eth")' overwrites this error. If add_adev() succeeds, mana_probe() returns success despite ports being left in a partially initialized state (ac->ports[i] == NULL). Only call add_adev() when there is no prior error, so the probe correctly fails and triggers mana_remove() cleanup.

Quoted 10 lines

In mana_probe(), if mana_probe_port() fails for any port, the error is stored in 'err' and the loop breaks. However, the subsequent unconditional 'err = add_adev(gd, "eth")' overwrites this error. If add_adev() succeeds, mana_probe() returns success despite ports being left in a partially initialized state (ac->ports[i] == NULL).

Only call add_adev() when there is no prior error, so the probe correctly fails and triggers mana_remove() cleanup.

Fixes ced82fce77e9 ("net: mana: Probe rdma device in mana driver")

The existing Fixes: tag points to commit ced82fce77e9 ("net: mana: Probe rdma device in mana driver"), but was the bug actually introduced there?

Looking at the commit history, commit a69839d4327d ("net: mana: Add support for auxiliary device") added the unconditional 'err = add_adev(gd);' call that overwrites the error from mana_probe_port(). Commit ced82fce77e9 only modified the add_adev signature from add_adev(gd) to add_adev(gd, "eth") but did not introduce the buggy pattern.

Should the Fixes: tag be:

Fixes a69839d4327d ("net: mana: Add support for auxiliary device")
#10 lore raw

Re: [PATCH net v3 4/5] net: mana: Don't overwrite port probe error with add_adev result

Quoted 29 lines · Simon Horman

This is an AI-generated review of your patch. The human sending this email has considered the AI review valid, or at least plausible.

For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
net: mana: Don't overwrite port probe error with add_adev result

In mana_probe(), if mana_probe_port() fails for any port, the error is stored in 'err' and the loop breaks. However, the subsequent unconditional 'err = add_adev(gd, "eth")' overwrites this error. If add_adev() succeeds, mana_probe() returns success despite ports being left in a partially initialized state (ac->ports[i] == NULL). Only call add_adev() when there is no prior error, so the probe correctly fails and triggers mana_remove() cleanup.

Quoted 10 lines

In mana_probe(), if mana_probe_port() fails for any port, the error is stored in 'err' and the loop breaks. However, the subsequent unconditional 'err = add_adev(gd, "eth")' overwrites this error. If add_adev() succeeds, mana_probe() returns success despite ports being left in a partially initialized state (ac->ports[i] == NULL).

Only call add_adev() when there is no prior error, so the probe correctly fails and triggers mana_remove() cleanup.

Fixes ced82fce77e9 ("net: mana: Probe rdma device in mana driver")

The existing Fixes: tag points to commit ced82fce77e9 ("net: mana: Probe rdma device in mana driver"), but was the bug actually introduced there?

Quoted 8 lines

Looking at the commit history, commit a69839d4327d ("net: mana: Add support for auxiliary device") added the unconditional 'err = add_adev(gd);' call that overwrites the error from mana_probe_port(). Commit ced82fce77e9 only modified the add_adev signature from add_adev(gd) to add_adev(gd, "eth") but did not introduce the buggy pattern.

Should the Fixes: tag be:

Fixes a69839d4327d ("net: mana: Add support for auxiliary device")

Thankyou for the correction, Simon.
I'll make this change in the next version.

- Vennela

#11 lore raw

PATCH v3 5/5 net: mana: Fix EQ leak in mana_remove on NULL port

In reply to Erni Sri Satya Vennela

In mana_remove(), when a NULL port is encountered in the port iteration loop, 'goto out' skips the mana_destroy_eq(ac) call, leaking the event queues allocated earlier by mana_create_eq().

This can happen when mana_probe_port() fails for port 0, leaving ac->ports[0] as NULL. On driver unload or error cleanup, mana_remove() hits the NULL entry and jumps past mana_destroy_eq().

Change 'goto out' to 'break' so the for-loop exits normally and mana_destroy_eq() is always reached. Remove the now-unreferenced out:
label.

Fixes 1e2d0824a9c3 ("net: mana: Add support for EQ sharing")

---
Changes in v3;
* Update Fixes tag to appropriate commit id.
Changes in v2:
* Apply the patch in net instead of net-next.

---
 drivers/net/ethernet/microsoft/mana/mana_en.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/microsoft/mana/mana_en.c +2 -2
index 39b18577fb51..98e2fcc797ca 100644
@@ -3752,7 +3752,7 @@ void mana_remove(struct gdma_dev *gd, bool suspending)
 		if (!ndev) {
 			if (i == 0)
 				dev_err(dev, "No net device to remove\n");
-			goto out;
+			break;
 		}
 
 		apc = netdev_priv(ndev);
@@ -3783,7 +3783,7 @@ void mana_remove(struct gdma_dev *gd, bool suspending)
 	}
 
 	mana_destroy_eq(ac);
-out:
+
 	if (ac->per_port_queue_reset_wq) {
 		destroy_workqueue(ac->per_port_queue_reset_wq);
 		ac->per_port_queue_reset_wq = NULL;
Signature · 1 line
2.34.1