net/mana: add device reset support

From: Wei Hu <[email protected]>

Add support for handling hardware service reset events in the MANA driver. When the MANA kernel driver receives a hardware service event, it initiates a device reset and notifies userspace via IBV_EVENT_DEVICE_FATAL. The MANA PMD handles this by performing an automatic teardown and recovery sequence.

The driver uses ethdev recovery events (ERR_RECOVERING, RECOVERY_SUCCESS, RECOVERY_FAILED) to notify upper layers of the reset lifecycle, and a PCI device removal event callback to distinguish hot-remove from service reset.

Changes since v6:
- Rebased onto latest upstream for-main
- Replaced removed RTE_ETH_DEV_TO_PCI macro with RTE_CLASS_TO_BUS_DEVICE (upstream commit 4757b8df04 removed the old bus-specific ethdev convenience macros)

Changes since v5:
- Replaced RCU QSBR with per-queue atomic burst_state using a single-variable CAS design: bit 0 is the in-burst flag, bits 1+ encode device state. The data path uses CAS(0→1) to enter burst and fetch_and(~1) to exit. The reset path uses fetch_or to set state bits and polls bit 0 to drain in-flight bursts. This eliminates the two-variable Dekker pattern and the need for sequential consistency (seq_cst) ordering.
- Removed librte_rcu dependency
- Removed __rte_no_thread_safety_analysis annotations (no longer needed after mutex conversion)
- Moved ERR_RECOVERING event emission before acquiring reset_ops_lock and before mana_reset_enter, so upper layers (e.g. netvsc) can switch data path before mana stops queues. Emitting outside the lock avoids deadlock if the callback calls dev_stop or dev_close.
- Replaced MANA_OPS_*_LOCK macros with mana_reset_trylock() helper function and explicit per-operation wrappers
- Removed unused rte_alarm.h and rte_lock_annotations.h includes
- Added RECOVERY_FAILED event when mana_reset_enter fails internally, so the application always receives a terminal event
- Added mana_clear_burst_state() helper to clear per-queue burst_state on failure paths (reset_failed, dev_stop_lock, dev_close_lock) preventing permanent silent packet drop after a failed reset

Changes since v4:
- Fixed stale rte_spinlock_unlock call in mana_intr_handler that was missed during the spinlock-to-mutex conversion, causing a -Wincompatible-pointer-types warning

Changes since v3:
- Converted reset_ops_lock from rte_spinlock_t to pthread_mutex_t with PTHREAD_PROCESS_SHARED, since the lock is held across blocking IB verbs calls and IPC with 5s timeout
- Removed rte_dev_event_callback_unregister retry loop to avoid
deadlock: the callback itself blocks on reset_ops_lock, so retrying on -EAGAIN while holding the lock is a deadlock
- Introduced mana_join_reset_thread() helper using CAS on reset_thread_active to prevent double-join undefined behavior
- Added reset thread join in mana_dev_uninit to prevent thread leak on device removal
- Fixed ibv handle leak: priv->ib_ctx is now only set to NULL after ibv_close_device succeeds
- Fixed misleading "All secondary threads are quiescent" log in mana_mp_reset_enter — changed to "Secondary doorbell pages unmapped" since actual quiescence is enforced by the primary's per-queue atomic flag check before IPC is sent
- Changed event list in mana.rst to RST definition list style
- Squashed documentation into the feature patch per convention

Changes since v2:
- Fixed dev_state_qsv memory leak on device removal
- Fixed reset thread TCB/stack leak: reset_thread_active is now only cleared by the joiner, not the thread itself
- Fixed second reset crash: removed reset thread join logic from mana_dev_close (inner function) to avoid corrupting dev_state when called from mana_reset_enter
- Made reset_thread_active RTE_ATOMIC(bool) with explicit ordering
- Added retry loop for rte_dev_event_callback_unregister on -EAGAIN
- Initialized condvar/mutex with PTHREAD_PROCESS_SHARED since priv is in hugepage shared memory
- Added re-check of dev_state after lock acquisition in mana_intr_handler to prevent racing with pci_remove_event_cb
- Replaced (void *)0 with NULL in mp.c
- Added lock ownership comment block at mana_reset_enter
- Documented rte_dev_event_monitor_start() requirement
- Added mana.rst documentation and release note

Changes since v1:
- Removed net/netvsc patch from this series
- Simplified reset exit: mana_reset_exit calls mana_reset_exit_delay directly instead of spawning a thread
- Added __rte_no_thread_safety_analysis annotations for clang
- Switched to rte_thread_create_internal_control
- Fixed declaration-after-statement style issues
- Removed unnecessary blank lines and stale comments

Wei Hu (1):
net/mana: add device reset support

 doc/guides/nics/mana.rst               |   39 +
 doc/guides/rel_notes/release_26_07.rst |    8 +
 drivers/net/mana/mana.c                | 1049 ++++++++++++++++++++++--
 drivers/net/mana/mana.h                |   46 +-
 drivers/net/mana/mp.c                  |   89 +-
 drivers/net/mana/mr.c                  |    6 +-
 drivers/net/mana/rx.c                  |   23 +-
 drivers/net/mana/tx.c                  |   44 +-
 8 files changed, 1196 insertions(+), 108 deletions(-)

2.34.1

From: Wei Hu <[email protected]>

Add support for handling hardware reset events in the MANA driver. When the MANA kernel driver receives a hardware service event, it initiates a device reset and notifies userspace via IBV_EVENT_DEVICE_FATAL. The DPDK driver handles this by performing an automatic teardown and recovery sequence.

The reset flow has two phases. In the enter phase, running on the EAL interrupt thread, the driver transitions the device state, waits for data path threads to drain using per-queue atomic flags, stops queues, tears down IB resources, and frees per-queue MR caches. A control thread is then spawned to handle the exit phase:
it waits for the hardware to recover, unregisters the interrupt handler, re-probes the PCI device, reinitializes MR caches, and restarts queues.

Each queue has an atomic burst_state variable where bit 0 is the in-burst flag and bits 1+ encode device state. The data path uses a single compare-and-swap (0 to 1) to enter a burst, which fails immediately if the reset path has set any state bits. The reset path sets state bits via atomic fetch-or and polls bit 0 to wait for in-flight bursts to drain. This single-variable design avoids the need for sequential consistency ordering.

A per-device mutex serializes the reset path with ethdev operations. The mutex uses PTHREAD_PROCESS_SHARED for multi-process support and is held across blocking IB verbs calls. A trylock helper encapsulates the lock acquisition and device state check for all ethdev operation wrappers. Operations that cannot wait (configure, queue setup) return -EBUSY during reset, while dev_stop and dev_close join the reset thread before acquiring the lock to ensure proper sequencing. A CAS-based helper prevents double-join of the reset thread.

Multi-process support is included: secondary processes unmap and remap doorbell pages via IPC during the reset enter and exit phases. Data path functions in both primary and secondary processes check the device state atomically and return early when the device is not active.

The driver emits RTE_ETH_EVENT_ERR_RECOVERING before entering the reset path so that upper layers (e.g. netvsc) can switch their data path before queues are stopped. The event is emitted outside the reset lock to avoid deadlock if the callback calls dev_stop or dev_close. On completion, the driver emits RECOVERY_SUCCESS or RECOVERY_FAILED. If the enter phase fails internally, RECOVERY_FAILED is sent immediately so the application receives a terminal event. A PCI device removal event callback distinguishes hot-remove from service reset.

Documentation for the device reset feature is added in the MANA NIC guide and the 26.07 release notes.

---
 doc/guides/nics/mana.rst               |   39 +
 doc/guides/rel_notes/release_26_07.rst |    8 +
 drivers/net/mana/mana.c                | 1049 ++++++++++++++++++++++--
 drivers/net/mana/mana.h                |   46 +-
 drivers/net/mana/mp.c                  |   89 +-
 drivers/net/mana/mr.c                  |    6 +-
 drivers/net/mana/rx.c                  |   23 +-
 drivers/net/mana/tx.c                  |   44 +-
 8 files changed, 1196 insertions(+), 108 deletions(-)

index 0fcab6e2f6..136adf8808 100644
@@ -71,3 +71,42 @@ The user can specify below argument in devargs.
     The default value is not set,
     meaning all the NICs will be probed and loaded.
     User can specify multiple mac=xx:xx:xx:xx:xx:xx arguments for up to 8 NICs.
+
+Device Reset Support
+--------------------
+
+The MANA PMD supports automatic recovery from hardware service reset events.
+When the MANA kernel driver receives a hardware service event,
+it initiates a device reset and notifies userspace
+via ``IBV_EVENT_DEVICE_FATAL``.
+
+The driver handles this transparently through a two-phase reset flow:
+
+* **Enter phase**: The driver stops the data path,
+  waits for all in-flight burst calls to drain
+  using per-queue atomic flags,
+  tears down IB resources and queues,
+  and unmaps secondary process doorbell pages.
+
+* **Exit phase**: After a delay for hardware recovery,
+  a control thread re-probes the PCI device,
+  reinstalls the interrupt handler,
+  reinitializes resources, and restarts queues.
+
+The driver emits the following ethdev recovery events
+to notify upper layers (e.g. netvsc) of the reset lifecycle:
+
+``RTE_ETH_EVENT_ERR_RECOVERING``
+   Reset has started.
+
+``RTE_ETH_EVENT_RECOVERY_SUCCESS``
+   Device has recovered successfully.
+
+``RTE_ETH_EVENT_RECOVERY_FAILED``
+   Recovery failed.
+
+To distinguish a PCI hot-remove from a service reset,
+the driver registers for PCI device removal events.
+This requires the application to call ``rte_dev_event_monitor_start()``
+for removal events to be delivered
+(e.g. testpmd ``--hot-plug-handling`` option).

index bd0cec2709..58e8c2422e 100644
@@ -122,6 +122,14 @@ New Features
   Added AGENTS.md file for AI review
   and supporting scripts to review patches and documentation.
 
+* **Added device reset support to the MANA PMD.**
+
+  Added automatic recovery from hardware service reset events
+  in the MANA poll mode driver. The driver uses ethdev recovery events
+  (``RTE_ETH_EVENT_ERR_RECOVERING``, ``RTE_ETH_EVENT_RECOVERY_SUCCESS``,
+  ``RTE_ETH_EVENT_RECOVERY_FAILED``) to notify upper layers of the
+  reset lifecycle.
+
 
 Removed Items
 -------------

index c4045141bc..8914f4cf04 100644
@@ -314,8 +314,10 @@ mana_mr_btree_init(struct mana_mr_btree *bt, int n, int socket)
 void
 mana_mr_btree_free(struct mana_mr_btree *bt)
 {
-	rte_free(bt->table);
-	memset(bt, 0, sizeof(*bt));
+	if (bt && bt->table) {
+		rte_free(bt->table);
+		memset(bt, 0, sizeof(*bt));
+	}
 }
 
 int

index 1b8ba1f3a9..53b19c67ed 100644
@@ -36,6 +36,11 @@ mana_rq_ring_doorbell(struct mana_rxq *rxq)
 		db_page = process_priv->db_page;
 	}
 
+	if (!db_page) {
+		DP_LOG(ERR, "db_page is NULL, cannot ring RX doorbell");
+		return -EINVAL;
+	}
+
 	/* Hardware Spec specifies that software client should set 0 for
 	 * wqe_cnt for Receive Queues.
 	 */

@@ -172,7 +177,7 @@ mana_stop_rx_queues(struct rte_eth_dev *dev)
 
 	for (i = 0; i < priv->num_queues; i++)
 		if (dev->data->rx_queue_state[i] == RTE_ETH_QUEUE_STATE_STOPPED)
-			return -EINVAL;
+			return 0;
 
 	if (priv->rwq_qp) {
 		ret = ibv_destroy_qp(priv->rwq_qp);

@@ -256,6 +261,9 @@ mana_start_rx_queues(struct rte_eth_dev *dev)
 		struct mana_rxq *rxq = dev->data->rx_queues[i];
 		struct ibv_wq_init_attr wq_attr = {};
 
+		rxq->rxq_idx = i;
+		DRV_LOG(DEBUG, "assigning rxq_idx to %d", i);
+
 		manadv_set_context_attr(priv->ib_ctx,
 			MANADV_CTX_ATTR_BUF_ALLOCATORS,
 			(void *)((uintptr_t)&(struct manadv_ctx_allocators){

@@ -451,6 +459,16 @@ mana_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n)
 	uint32_t pkt_len;
 	uint32_t i;
 	int polled = 0;
+	uint32_t expected = 0;
+
+	/* Single atomic CAS: enter burst only if device is active (0→1).
+	 * Fails immediately if reset path has set state bits.
+	 */
+	if (unlikely(!rte_atomic_compare_exchange_strong_explicit(
+			&rxq->burst_state, &expected, 1,
+			rte_memory_order_acquire,
+			rte_memory_order_relaxed)))
+		return 0;
 
 repoll:
 	/* Polling on new completions if we have no backlog */

@@ -592,6 +610,9 @@ mana_rx_burst(void *dpdk_rxq, struct rte_mbuf **pkts, uint16_t pkts_n)
 				wqe_consumed, ret);
 	}
 
+	rte_atomic_fetch_and_explicit(&rxq->burst_state, ~(uint32_t)1,
+				     rte_memory_order_release);
+
 	return pkt_received;
 }
 

2.34.1

This is is a rather complex set of state transitions so admit to relying on AI as backup for tracking this. It still sees some errors here.

Worth asking the question, "what does mlx5 do?" and "should DPDK EAL be doing this at the PCI layer instead?"

---

Much better - this addresses the RCU, the macros, the thread-safety-analysis suppressions, and the callback-under-lock deadlock. The single-variable burst_state CAS is a clean way to do the drain and the acquire/release reasoning checks out.

One structural thing remains. The enter phase still runs the heavy teardown on the EAL interrupt thread under reset_ops_lock: dev_stop, then mana_mp_req_on_rxtx(RESET_ENTER) which is a blocking rte_mp_request_sync with a multi-second timeout, then dev_close with its ibv calls. You already moved the exit phase to a control thread because intr_callback_unregister cannot run on the interrupt thread; the same argument applies to blocking IPC and verbs teardown. A slow or absent secondary will stall the interrupt thread for the MP timeout, and this is the blocking-under-a-sleeping-mutex pattern. Please have the interrupt handler just set state and drain, then hand the rest of the teardown to the control thread. That also removes the last lock hand-off between functions/threads, so each function can own its lock.

Smaller points:
- RECOVERY_SUCCESS/FAILED are emitted from the reset thread. If the callback calls dev_stop/dev_close, mana_join_reset_thread() joins the current thread (EDEADLK, leaked handle). INTR_RMV is fine since it runs on the dev-event thread.
- The burst_state comment says bits 1+ encode device state, but only RESET_ENTER<<1 is ever stored - it is effectively a single "blocked" flag.

MLX5 doesn't have this device service model and will not attempt device recovery, it just quits and tells the app the device is removed via sending out RTE_ETH_EVENT_INTR_RMV.

This service event has very limited visibility over the PCI, as the device is servicing itself while keeping all the PCI endpoints alive. PCI device persists during the reset. Only the DMA queues need to be recreated.

I think processing the service events entirely in the mana driver is the right thing to do.

Yes, we should move all the blocking calls from the system interrupt thread to the MANA control thread.