Skip to contentSkip to conversation

fix pthread mutexes for multi-process

PATCH Superseded Stephen Hemminger 3 participants 33 messages Apr 13 – May 8

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover fix pthread mutexes for multi-process

Several drivers and the ethdev layer initialize pthread mutexes in shared memory with default (process-private) attributes. This is undefined behavior when secondary processes use them.

This series adds PTHREAD_PROCESS_SHARED to all affected mutexes. All are on control paths (firmware mailbox, hotplug, flow ops, PHY negotiation) where sleeping is acceptable.

See POSIX spec:
https://pubs.opengroup.org/onlinepubs/009696899/functions/pthread_mutexattr_getpshared.html

Bugzilla ID: 662

Stephen Hemminger (6):
ethdev: fix flow_ops_mutex for multi-process net/failsafe: fix hotplug_mutex for multi-process net/atlantic: fix mbox_mutex for multi-process net/axgbe: fix mutexes for multi-process net/bnxt: fix mutexes for multi-process net/hinic: fix mutexes for multi-process

 drivers/net/atlantic/atl_ethdev.c      | 14 +++++++++++++-
 drivers/net/axgbe/axgbe_ethdev.c       | 19 +++++++++++++++----
 drivers/net/bnxt/bnxt_ethdev.c         | 11 ++++++-----
 drivers/net/bnxt/bnxt_txq.c            |  3 ++-
 drivers/net/bnxt/bnxt_util.c           | 13 +++++++++++++
 drivers/net/bnxt/bnxt_util.h           |  2 ++
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c     |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c  |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c |  2 +-
 drivers/net/failsafe/failsafe.c        | 15 ++++++++++++---
 drivers/net/hinic/base/hinic_compat.h  | 13 ++++++++++++-
 lib/ethdev/ethdev_driver.c             | 18 +++++++++++++++++-
 12 files changed, 95 insertions(+), 19 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH 1/6 ethdev: fix flow_ops_mutex for multi-process

The flow_ops_mutex in rte_eth_dev_data is located in shared memory that is accessed by both primary and secondary processes. However, the mutex is initialized without PTHREAD_PROCESS_SHARED attribute, which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures in shared memory that are accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

This bug has existed since the flow_ops_mutex was added and affects any multi-process DPDK application using rte_flow APIs.

Bugzilla ID: 662

Fixes 80d1a9aff7f6 ("ethdev: make flow API thread safe")
---
 lib/ethdev/ethdev_driver.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)
lib/ethdev/ethdev_driver.c +17 -1
index 2ec665be0f..d12395efe0 100644
@@ -89,6 +89,22 @@ eth_dev_set_dummy_fops(struct rte_eth_dev *eth_dev)
 	eth_dev->recycle_rx_descriptors_refill = rte_eth_recycle_rx_descriptors_refill_dummy;
 }
 
+/*
+ * Initialize mutex for process-shared access.
+ * The rte_eth_dev_data structure is in shared memory, so any mutex
+ * protecting it must use PTHREAD_PROCESS_SHARED.
+ */
+static void
+eth_dev_flow_ops_mutex_init(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
+
 RTE_EXPORT_INTERNAL_SYMBOL(rte_eth_dev_allocate)
 struct rte_eth_dev *
 rte_eth_dev_allocate(const char *name)
@@ -135,7 +151,7 @@ rte_eth_dev_allocate(const char *name)
 	eth_dev->data->port_id = port_id;
 	eth_dev->data->backer_port_id = RTE_MAX_ETHPORTS;
 	eth_dev->data->mtu = RTE_ETHER_MTU;
-	pthread_mutex_init(&eth_dev->data->flow_ops_mutex, NULL);
+	eth_dev_flow_ops_mutex_init(&eth_dev->data->flow_ops_mutex);
 	RTE_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY);
 	eth_dev_shared_data->allocated_ports++;
 
Signature · 1 line
2.53.0
#2 lore raw

PATCH 2/6 net/failsafe: fix hotplug_mutex for multi-process

In reply to Stephen Hemminger

The failsafe driver supports secondary process attachment via rte_eth_dev_attach_secondary() in rte_pmd_failsafe_probe(). The hotplug_mutex in fs_priv protects shared state but is initialized without PTHREAD_PROCESS_SHARED attribute.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

Also add missing pthread_mutexattr_destroy() call to avoid resource leak.

Bugzilla ID: 662

Fixes 655fcd68c7d2 ("net/failsafe: fix hotplug races")
---
 drivers/net/failsafe/failsafe.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
drivers/net/failsafe/failsafe.c +12 -3
index 3e590d38f7..aa3fe53b22 100644
@@ -144,11 +144,20 @@ fs_mutex_init(struct fs_priv *priv)
 	/* Allow mutex relocks for the thread holding the mutex. */
 	ret = pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
 	if (ret) {
-		ERROR("Cannot set mutex type - %s", strerror(ret));
-		return ret;
+		ERROR("Cannot set mutex recursive - %s", strerror(ret));
+		goto out;
+	}
+	/* Allow mutex to be shared between processes. */
+	ret = pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	if (ret) {
+		ERROR("Cannot set mutex pshared - %s", strerror(ret));
+		goto out;
 	}
+	pthread_mutex_init(&priv->hotplug_mutex, &attr);
 
-	return pthread_mutex_init(&priv->hotplug_mutex, &attr);
+out:
+	pthread_mutexattr_destroy(&attr);
+	return ret;
 }
 
 static int
Signature · 1 line
2.53.0
#3 lore raw

PATCH 3/6 net/atlantic: fix mbox_mutex for multi-process

In reply to Stephen Hemminger

The Atlantic driver supports secondary processes, as shown by the explicit check in eth_atl_dev_init(). The mbox_mutex in aq_hw_s is located in dev_private which is in shared memory accessible by both primary and secondary processes.

However, the mutex is initialized without PTHREAD_PROCESS_SHARED attribute, which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures in shared memory that are accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

Bugzilla ID: 662

Fixes e9924638f5c9 ("net/atlantic: add FW mailbox guard mutex")
---
 drivers/net/atlantic/atl_ethdev.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/net/atlantic/atl_ethdev.c +13 -1
index 2925dc2478..7282f9b691 100644
@@ -3,6 +3,7 @@
  */
 
 #include <rte_string_fns.h>
+#include <pthread.h>
 #include <ethdev_pci.h>
 #include <rte_alarm.h>
 
@@ -355,6 +356,17 @@ atl_disable_intr(struct aq_hw_s *hw)
 	hw_atl_itr_irq_msk_clearlsw_set(hw, 0xffffffff);
 }
 
+static void
+atl_init_mutex(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
+
 static int
 eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 {
@@ -405,7 +417,7 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 
 	hw->aq_nic_cfg = &adapter->hw_cfg;
 
-	pthread_mutex_init(&hw->mbox_mutex, NULL);
+	atl_init_mutex(&hw->mbox_mutex);
 
 	/* disable interrupt */
 	atl_disable_intr(hw);
Signature · 1 line
2.53.0
#4 lore raw

PATCH 4/6 net/axgbe: fix mutexes for multi-process

In reply to Stephen Hemminger

The AXGBE driver supports secondary processes, as shown by the explicit check in eth_axgbe_dev_init(). The xpcs_mutex, i2c_mutex, an_mutex, and phy_mutex in axgbe_port are located in dev_private which is in shared memory accessible by both primary and secondary processes.

However, the mutexes are initialized without PTHREAD_PROCESS_SHARED attribute, which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures in shared memory that are accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

Bugzilla ID: 662

Fixes 572890ef6625 ("net/axgbe: add structs for MAC init and reset")
---
 drivers/net/axgbe/axgbe_ethdev.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)
drivers/net/axgbe/axgbe_ethdev.c +15 -4
index cfcd880961..4aadf0993e 100644
@@ -194,6 +194,17 @@ static const struct axgbe_xstats axgbe_xstats_strings[] = {
 #define AMD_PCI_AXGBE_DEVICE_V2A 0x1458
 #define AMD_PCI_AXGBE_DEVICE_V2B 0x1459
 
+static void
+axgbe_init_mutex(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
+
 static const struct rte_pci_id pci_id_axgbe_map[] = {
 	{RTE_PCI_DEVICE(AMD_PCI_VENDOR_ID, AMD_PCI_AXGBE_DEVICE_V2A)},
 	{RTE_PCI_DEVICE(AMD_PCI_VENDOR_ID, AMD_PCI_AXGBE_DEVICE_V2B)},
@@ -2419,10 +2430,10 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 
 	pdata->tx_desc_count = AXGBE_MAX_RING_DESC;
 	pdata->rx_desc_count = AXGBE_MAX_RING_DESC;
-	pthread_mutex_init(&pdata->xpcs_mutex, NULL);
-	pthread_mutex_init(&pdata->i2c_mutex, NULL);
-	pthread_mutex_init(&pdata->an_mutex, NULL);
-	pthread_mutex_init(&pdata->phy_mutex, NULL);
+	axgbe_init_mutex(&pdata->xpcs_mutex);
+	axgbe_init_mutex(&pdata->i2c_mutex);
+	axgbe_init_mutex(&pdata->an_mutex);
+	axgbe_init_mutex(&pdata->phy_mutex);
 
 	ret = pdata->phy_if.phy_init(pdata);
 	if (ret) {
Signature · 1 line
2.53.0
#5 lore raw

PATCH 5/6 net/bnxt: fix mutexes for multi-process

In reply to Stephen Hemminger

The BNXT driver supports secondary processes, as shown by the explicit check in bnxt_dev_init(). Multiple mutexes are located in structures allocated in shared memory (dev_private and rte_zmalloc'd structures) that are accessible by both primary and secondary processes.

However, the mutexes are initialized without PTHREAD_PROCESS_SHARED attribute, which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures in shared memory that are accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

This patch adds a helper function and updates all mutex initializations in the BNXT driver:
- flow_lock, def_cp_lock, health_check_lock, err_recovery_lock in struct bnxt (bnxt_ethdev.c)
- vfr_start_lock in rep_info (bnxt_ethdev.c)
- txq_lock in struct bnxt_tx_queue (bnxt_txq.c)
- bnxt_ulp_mutex in session state (bnxt_ulp.c)
- flow_db_lock in cfg_data (bnxt_ulp_tf.c, bnxt_ulp_tfc.c)

Bugzilla ID: 662

Fixes 1cb3d39a48f7 ("net/bnxt: synchronize between flow related functions")
Fixes 5526c8025d4d ("net/bnxt: fix race between interrupt handler and dev config")
Fixes b59e4be2b6a7 ("net/bnxt: fix VF representor port add")
---
 drivers/net/bnxt/bnxt_ethdev.c         | 11 ++++++-----
 drivers/net/bnxt/bnxt_txq.c            |  3 ++-
 drivers/net/bnxt/bnxt_util.c           | 13 +++++++++++++
 drivers/net/bnxt/bnxt_util.h           |  2 ++
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c     |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c  |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c |  2 +-
 7 files changed, 26 insertions(+), 9 deletions(-)
drivers/net/bnxt/bnxt_ethdev.c +6 -5
index b677f9491d..3f55ad041d 100644
@@ -5899,10 +5899,10 @@ static int bnxt_get_config(struct bnxt *bp)
 static int
 bnxt_init_locks(struct bnxt *bp)
 {
-	pthread_mutex_init(&bp->flow_lock, NULL);
-	pthread_mutex_init(&bp->def_cp_lock, NULL);
-	pthread_mutex_init(&bp->health_check_lock, NULL);
-	pthread_mutex_init(&bp->err_recovery_lock, NULL);
+	bnxt_init_mutex(&bp->flow_lock);
+	bnxt_init_mutex(&bp->def_cp_lock);
+	bnxt_init_mutex(&bp->health_check_lock);
+	bnxt_init_mutex(&bp->err_recovery_lock);
 
 	return 0;
 }
@@ -6920,7 +6920,8 @@ static int bnxt_init_rep_info(struct bnxt *bp)
 	for (i = 0; i < BNXT_MAX_CFA_CODE; i++)
 		bp->cfa_code_map[i] = BNXT_VF_IDX_INVALID;
 
-	return pthread_mutex_init(&bp->rep_info->vfr_start_lock, NULL);
+	bnxt_init_mutex(&bp->rep_info->vfr_start_lock);
+	return 0;
 }
 
 static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
drivers/net/bnxt/bnxt_txq.c +2 -1
index 7752f06eb7..8c834acb1d 100644
@@ -204,7 +204,8 @@ int bnxt_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
 		goto err;
 	}
 
-	return pthread_mutex_init(&txq->txq_lock, NULL);
+	bnxt_init_mutex(&txq->txq_lock);
+	return 0;
 err:
 	bnxt_tx_queue_release_op(eth_dev, queue_idx);
 	return rc;
drivers/net/bnxt/bnxt_util.c +13 -0
index aa184496c2..c3f0a03f25 100644
@@ -3,6 +3,7 @@
  * All rights reserved.
  */
 
+#include <pthread.h>
 #include <inttypes.h>
 #include <rte_ether.h>
 
@@ -37,3 +38,15 @@ uint8_t hweight32(uint32_t word32)
 	res = res + (res >> 8);
 	return (res + (res >> 16)) & 0x000000FF;
 }
+
+/* Initialize mutex so that it can be shared between processes */
+void
+bnxt_init_mutex(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
drivers/net/bnxt/bnxt_util.h +2 -0
index 416a6a2609..e1b45d1bb5 100644
@@ -16,4 +16,6 @@
 int bnxt_check_zero_bytes(const uint8_t *bytes, int len);
 void bnxt_eth_hw_addr_random(uint8_t *mac_addr);
 uint8_t hweight32(uint32_t word32);
+void bnxt_init_mutex(pthread_mutex_t *mutex);
+
 #endif /* _BNXT_UTIL_H_ */
drivers/net/bnxt/tf_ulp/bnxt_ulp.c +1 -1
index 0c03ae7a83..c2c93859ff 100644
@@ -214,7 +214,7 @@ ulp_session_init(struct bnxt *bp,
 			session->pci_info.domain = pci_addr->domain;
 			session->pci_info.bus = pci_addr->bus;
 			memcpy(session->dsn, bp->dsn, sizeof(session->dsn));
-			pthread_mutex_init(&session->bnxt_ulp_mutex, NULL);
+			bnxt_init_mutex(&session->bnxt_ulp_mutex);
 			STAILQ_INSERT_TAIL(&bnxt_ulp_session_list,
 					   session, next);
 		}
drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c +1 -1
index bc347de202..d87120aea3 100644
@@ -1469,7 +1469,7 @@ ulp_tf_init(struct bnxt *bp,
 		goto jump_to_error;
 	}
 
-	pthread_mutex_init(&bp->ulp_ctx->cfg_data->flow_db_lock, NULL);
+	bnxt_init_mutex(&bp->ulp_ctx->cfg_data->flow_db_lock);
 
 	/* Initialize ulp dparms with values devargs passed */
 	rc = ulp_tf_dparms_init(bp, bp->ulp_ctx);
drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c +1 -1
index ad44ec93ca..0c3d1f7dae 100644
@@ -1038,7 +1038,7 @@ ulp_tfc_init(struct bnxt *bp,
 		goto jump_to_error;
 	}
 
-	pthread_mutex_init(&bp->ulp_ctx->cfg_data->flow_db_lock, NULL);
+	bnxt_init_mutex(&bp->ulp_ctx->cfg_data->flow_db_lock);
 
 	rc = ulp_tfc_dparms_init(bp, bp->ulp_ctx, ulp_dev_id);
 	if (rc) {
Signature · 1 line
2.53.0
Acked-by on v1 5/6
#6 lore raw

RE: [PATCH 5/6] net/bnxt: fix mutexes for multi-process

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Monday, April 13, 2026 1:15 PM
To: [email protected]
Cc: Stephen Hemminger <[email protected]>; [email protected]; Kishore Padmanabha <[email protected]>; Ajit Khaparde <[email protected]>; Venkat Duvvuru <[email protected]>; Kalesh AP <[email protected]>; Somnath Kotur <[email protected]>
Subject: [PATCH 5/6] net/bnxt: fix mutexes for multi-process

The BNXT driver supports secondary processes, as shown by the explicit check in bnxt_dev_init(). Multiple mutexes are located in structures allocated in shared memory (dev_private and rte_zmalloc'd structures) that are accessible by both primary and secondary processes.

However, the mutexes are initialized without PTHREAD_PROCESS_SHARED attribute, which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures in shared memory that are accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

This patch adds a helper function and updates all mutex initializations in the BNXT driver:
- flow_lock, def_cp_lock, health_check_lock, err_recovery_lock in struct bnxt (bnxt_ethdev.c)
- vfr_start_lock in rep_info (bnxt_ethdev.c)
- txq_lock in struct bnxt_tx_queue (bnxt_txq.c)
- bnxt_ulp_mutex in session state (bnxt_ulp.c)
- flow_db_lock in cfg_data (bnxt_ulp_tf.c, bnxt_ulp_tfc.c)

Bugzilla ID: 662

Fixes 1cb3d39a48f7 ("net/bnxt: synchronize between flow related

functions")

Fixes 5526c8025d4d ("net/bnxt: fix race between interrupt handler and dev

config")

Fixes b59e4be2b6a7 ("net/bnxt: fix VF representor port add")
---
 drivers/net/bnxt/bnxt_ethdev.c         | 11 ++++++-----
 drivers/net/bnxt/bnxt_txq.c            |  3 ++-
 drivers/net/bnxt/bnxt_util.c           | 13 +++++++++++++
 drivers/net/bnxt/bnxt_util.h           |  2 ++
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c     |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c  |  2 +-
drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c |  2 +-
 7 files changed, 26 insertions(+), 9 deletions(-)
(quoted diff) +0 -0

b/drivers/net/bnxt/bnxt_ethdev.c index b677f9491d..3f55ad041d 100644

drivers/net/bnxt/bnxt_ethdev.c +0 -0
@@ -5899,10 +5899,10 @@ static int bnxt_get_config(struct bnxt *bp)

static int bnxt_init_locks(struct bnxt *bp) {
- pthread_mutex_init(&bp->flow_lock, NULL);
- pthread_mutex_init(&bp->def_cp_lock, NULL);
- pthread_mutex_init(&bp->health_check_lock, NULL);
- pthread_mutex_init(&bp->err_recovery_lock, NULL);
+ bnxt_init_mutex(&bp->flow_lock);
+ bnxt_init_mutex(&bp->def_cp_lock);
+ bnxt_init_mutex(&bp->health_check_lock);
+ bnxt_init_mutex(&bp->err_recovery_lock);

return 0;
}
@@ -6920,7 +6920,8 @@ static int bnxt_init_rep_info(struct bnxt *bp)
for (i = 0; i < BNXT_MAX_CFA_CODE; i++)
bp->cfa_code_map[i] = BNXT_VF_IDX_INVALID;

- return pthread_mutex_init(&bp->rep_info->vfr_start_lock, NULL);
+ bnxt_init_mutex(&bp->rep_info->vfr_start_lock);
+ return 0;
}

static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev, diff --git
a/drivers/net/bnxt/bnxt_txq.c b/drivers/net/bnxt/bnxt_txq.c index 7752f06eb7..8c834acb1d 100644

drivers/net/bnxt/bnxt_txq.c +0 -0
@@ -204,7 +204,8 @@ int bnxt_tx_queue_setup_op(struct rte_eth_dev

*eth_dev,
goto err;
}

- return pthread_mutex_init(&txq->txq_lock, NULL);
+ bnxt_init_mutex(&txq->txq_lock);
+ return 0;
err:
bnxt_tx_queue_release_op(eth_dev, queue_idx); return rc;

drivers/net/bnxt/bnxt_util.c +11 -0
index aa184496c2..c3f0a03f25 100644
@@ -3,6 +3,7 @@
  * All rights reserved.
  */
 
+#include <pthread.h>
 #include <inttypes.h>
 #include <rte_ether.h>
 
@@ -37,3 +38,15 @@ uint8_t hweight32(uint32_t word32)
 	res = res + (res >> 8);
 	return (res + (res >> 16)) & 0x000000FF;  }
+
+/* Initialize mutex so that it can be shared between processes */ void
+bnxt_init_mutex(pthread_mutex_t *mutex) {
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
drivers/net/bnxt/bnxt_util.h +0 -0
index 416a6a2609..e1b45d1bb5 100644
@@ -16,4 +16,6 @@
 int bnxt_check_zero_bytes(const uint8_t *bytes, int len);  void

bnxt_eth_hw_addr_random(uint8_t *mac_addr); uint8_t hweight32(uint32_t word32);
+void bnxt_init_mutex(pthread_mutex_t *mutex);
+
#endif /* _BNXT_UTIL_H_ */

(quoted diff) +0 -0

b/drivers/net/bnxt/tf_ulp/bnxt_ulp.c
index 0c03ae7a83..c2c93859ff 100644

drivers/net/bnxt/tf_ulp/bnxt_ulp.c +0 -0
@@ -214,7 +214,7 @@ ulp_session_init(struct bnxt *bp,
 			session->pci_info.domain = pci_addr->domain;
 			session->pci_info.bus = pci_addr->bus;
 			memcpy(session->dsn, bp->dsn,

sizeof(session->dsn));
- pthread_mutex_init(&session->bnxt_ulp_mutex,
NULL);
+ bnxt_init_mutex(&session->bnxt_ulp_mutex);
STAILQ_INSERT_TAIL(&bnxt_ulp_session_list,
session, next);
}

(quoted diff) +0 -0

b/drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c
index bc347de202..d87120aea3 100644

drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c +1 -1
@@ -1469,7 +1469,7 @@ ulp_tf_init(struct bnxt *bp,
 		goto jump_to_error;
 	}
 
-	pthread_mutex_init(&bp->ulp_ctx->cfg_data->flow_db_lock, NULL);
+	bnxt_init_mutex(&bp->ulp_ctx->cfg_data->flow_db_lock);
 
 	/* Initialize ulp dparms with values devargs passed */
 	rc = ulp_tf_dparms_init(bp, bp->ulp_ctx); diff --git

a/drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c
b/drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c
index ad44ec93ca..0c3d1f7dae 100644

drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c +1 -1
@@ -1038,7 +1038,7 @@ ulp_tfc_init(struct bnxt *bp,
 		goto jump_to_error;
 	}
 
-	pthread_mutex_init(&bp->ulp_ctx->cfg_data->flow_db_lock, NULL);
+	bnxt_init_mutex(&bp->ulp_ctx->cfg_data->flow_db_lock);
 
 	rc = ulp_tfc_dparms_init(bp, bp->ulp_ctx, ulp_dev_id);
 	if (rc) {

--
2.53.0

#7 lore raw

PATCH 6/6 net/hinic: fix mutexes for multi-process

In reply to Stephen Hemminger

The hinic driver supports secondary processes, as shown by the explicit check in hinic_dev_init(). Multiple mutexes are located in structures allocated in shared memory (dev_private and rte_zmalloc'd structures) that are accessible by both primary and secondary processes.

However, the mutexes are initialized without PTHREAD_PROCESS_SHARED attribute, which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures in shared memory that are accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

Fix the hinic_mutex_init() wrapper function to set the PTHREAD_PROCESS_SHARED attribute on all mutexes.

Note: all callers to hinic_mutex_init() pass NULL as second argument, so it could be simplified but that would entail more changes to base vendor code.

Bugzilla ID: 662

Fixes ae865766b334 ("net/hinic: replace spinlock with mutex")
---
 drivers/net/hinic/base/hinic_compat.h | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
drivers/net/hinic/base/hinic_compat.h +12 -1
index d3994c50e9..18148a119e 100644
@@ -197,10 +197,21 @@ static inline u16 ilog2(u32 n)
 	return res;
 }
 
+/*
+ * Initialize mutex for process-shared access.
+ * Structures may be in shared memory accessible by multiple processes,
+ * so mutexes must use PTHREAD_PROCESS_SHARED.
+ */
 static inline int hinic_mutex_init(pthread_mutex_t *pthreadmutex,
 					const pthread_mutexattr_t *mattr)
 {
-	return pthread_mutex_init(pthreadmutex, mattr);
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(pthreadmutex, mattr ? mattr : &attr);
+	pthread_mutexattr_destroy(&attr);
+	return 0;
 }
 
 static inline int hinic_mutex_destroy(pthread_mutex_t *pthreadmutex)
Signature · 1 line
2.53.0
#8 lore raw

v2 cover fix process shared pthread mutexes

In reply to Stephen Hemminger

Several drivers and the ethdev layer initialize pthread mutexes in shared memory with default (process-private) attributes. This is undefined behavior when secondary processes use them.

This series adds PTHREAD_PROCESS_SHARED to all affected mutexes. All are on control paths (firmware mailbox, hotplug, flow ops, PHY negotiation) where sleeping is acceptable.

See POSIX spec:
https://pubs.opengroup.org/onlinepubs/009696899/functions/pthread_mutexattr_getpshared.html

Bugzilla ID: 662

v2 - fix build on Windows which does not need this.

Stephen Hemminger (6):
ethdev: fix flow_ops_mutex for multi-process net/failsafe: fix hotplug_mutex for multi-process net/atlantic: fix mbox_mutex for multi-process net/axgbe: fix mutexes for multi-process net/bnxt: fix mutexes for multi-process net/hinic: fix mutexes for multi-process

 drivers/net/atlantic/atl_ethdev.c      | 14 +++++++++++++-
 drivers/net/axgbe/axgbe_ethdev.c       | 19 +++++++++++++++----
 drivers/net/bnxt/bnxt_ethdev.c         | 11 ++++++-----
 drivers/net/bnxt/bnxt_txq.c            |  3 ++-
 drivers/net/bnxt/bnxt_util.c           | 13 +++++++++++++
 drivers/net/bnxt/bnxt_util.h           |  2 ++
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c     |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c  |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c |  2 +-
 drivers/net/failsafe/failsafe.c        | 15 ++++++++++++---
 drivers/net/hinic/base/hinic_compat.h  | 13 ++++++++++++-
 lib/ethdev/ethdev_driver.c             | 22 +++++++++++++++++++++-
 12 files changed, 99 insertions(+), 19 deletions(-)
Signature · 1 line
2.53.0
#9 lore raw

PATCH v2 1/6 ethdev: fix flow_ops_mutex for multi-process

The flow_ops_mutex in rte_eth_dev_data is located in shared memory that is accessed by both primary and secondary processes. However, the mutex is initialized without PTHREAD_PROCESS_SHARED attribute, which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures in shared memory that are accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

This bug has existed since the flow_ops_mutex was added and affects any multi-process DPDK application using rte_flow APIs.

Bugzilla ID: 662

Fixes 80d1a9aff7f6 ("ethdev: make flow API thread safe")
---
 lib/ethdev/ethdev_driver.c | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)
lib/ethdev/ethdev_driver.c +21 -1
index 2ec665be0f..60d0246226 100644
@@ -89,6 +89,26 @@ eth_dev_set_dummy_fops(struct rte_eth_dev *eth_dev)
 	eth_dev->recycle_rx_descriptors_refill = rte_eth_recycle_rx_descriptors_refill_dummy;
 }
 
+/*
+ * Initialize mutex for process-shared access.
+ * The rte_eth_dev_data structure is in shared memory, so any mutex
+ * protecting it must use PTHREAD_PROCESS_SHARED.
+ */
+static void
+eth_dev_flow_ops_mutex_init(pthread_mutex_t *mutex)
+{
+#ifndef RTE_EXEC_ENV_WINDOWS
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+#else
+	pthread_mutex_init(mutex, NULL);
+#endif
+}
+
 RTE_EXPORT_INTERNAL_SYMBOL(rte_eth_dev_allocate)
 struct rte_eth_dev *
 rte_eth_dev_allocate(const char *name)
@@ -135,7 +155,7 @@ rte_eth_dev_allocate(const char *name)
 	eth_dev->data->port_id = port_id;
 	eth_dev->data->backer_port_id = RTE_MAX_ETHPORTS;
 	eth_dev->data->mtu = RTE_ETHER_MTU;
-	pthread_mutex_init(&eth_dev->data->flow_ops_mutex, NULL);
+	eth_dev_flow_ops_mutex_init(&eth_dev->data->flow_ops_mutex);
 	RTE_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY);
 	eth_dev_shared_data->allocated_ports++;
 
Signature · 1 line
2.53.0
#10 lore raw

PATCH v2 2/6 net/failsafe: fix hotplug_mutex for multi-process

In reply to Stephen Hemminger

The failsafe driver supports secondary process attachment via rte_eth_dev_attach_secondary() in rte_pmd_failsafe_probe(). The hotplug_mutex in fs_priv protects shared state but is initialized without PTHREAD_PROCESS_SHARED attribute.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

Also add missing pthread_mutexattr_destroy() call to avoid resource leak.

Bugzilla ID: 662

Fixes 655fcd68c7d2 ("net/failsafe: fix hotplug races")
---
 drivers/net/failsafe/failsafe.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
drivers/net/failsafe/failsafe.c +12 -3
index 3e590d38f7..aa3fe53b22 100644
@@ -144,11 +144,20 @@ fs_mutex_init(struct fs_priv *priv)
 	/* Allow mutex relocks for the thread holding the mutex. */
 	ret = pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
 	if (ret) {
-		ERROR("Cannot set mutex type - %s", strerror(ret));
-		return ret;
+		ERROR("Cannot set mutex recursive - %s", strerror(ret));
+		goto out;
+	}
+	/* Allow mutex to be shared between processes. */
+	ret = pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	if (ret) {
+		ERROR("Cannot set mutex pshared - %s", strerror(ret));
+		goto out;
 	}
+	pthread_mutex_init(&priv->hotplug_mutex, &attr);
 
-	return pthread_mutex_init(&priv->hotplug_mutex, &attr);
+out:
+	pthread_mutexattr_destroy(&attr);
+	return ret;
 }
 
 static int
Signature · 1 line
2.53.0
#11 lore raw

PATCH v2 3/6 net/atlantic: fix mbox_mutex for multi-process

In reply to Stephen Hemminger

The Atlantic driver supports secondary processes, as shown by the explicit check in eth_atl_dev_init(). The mbox_mutex in aq_hw_s is located in dev_private which is in shared memory accessible by both primary and secondary processes.

However, the mutex is initialized without PTHREAD_PROCESS_SHARED attribute, which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures in shared memory that are accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

Bugzilla ID: 662

Fixes e9924638f5c9 ("net/atlantic: add FW mailbox guard mutex")
---
 drivers/net/atlantic/atl_ethdev.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/net/atlantic/atl_ethdev.c +13 -1
index 2925dc2478..7282f9b691 100644
@@ -3,6 +3,7 @@
  */
 
 #include <rte_string_fns.h>
+#include <pthread.h>
 #include <ethdev_pci.h>
 #include <rte_alarm.h>
 
@@ -355,6 +356,17 @@ atl_disable_intr(struct aq_hw_s *hw)
 	hw_atl_itr_irq_msk_clearlsw_set(hw, 0xffffffff);
 }
 
+static void
+atl_init_mutex(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
+
 static int
 eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 {
@@ -405,7 +417,7 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 
 	hw->aq_nic_cfg = &adapter->hw_cfg;
 
-	pthread_mutex_init(&hw->mbox_mutex, NULL);
+	atl_init_mutex(&hw->mbox_mutex);
 
 	/* disable interrupt */
 	atl_disable_intr(hw);
Signature · 1 line
2.53.0
#12 lore raw

PATCH v2 4/6 net/axgbe: fix mutexes for multi-process

In reply to Stephen Hemminger

The AXGBE driver supports secondary processes, as shown by the explicit check in eth_axgbe_dev_init(). The xpcs_mutex, i2c_mutex, an_mutex, and phy_mutex in axgbe_port are located in dev_private which is in shared memory accessible by both primary and secondary processes.

However, the mutexes are initialized without PTHREAD_PROCESS_SHARED attribute, which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures in shared memory that are accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

Bugzilla ID: 662

Fixes 572890ef6625 ("net/axgbe: add structs for MAC init and reset")
---
 drivers/net/axgbe/axgbe_ethdev.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)
drivers/net/axgbe/axgbe_ethdev.c +15 -4
index cfcd880961..4aadf0993e 100644
@@ -194,6 +194,17 @@ static const struct axgbe_xstats axgbe_xstats_strings[] = {
 #define AMD_PCI_AXGBE_DEVICE_V2A 0x1458
 #define AMD_PCI_AXGBE_DEVICE_V2B 0x1459
 
+static void
+axgbe_init_mutex(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
+
 static const struct rte_pci_id pci_id_axgbe_map[] = {
 	{RTE_PCI_DEVICE(AMD_PCI_VENDOR_ID, AMD_PCI_AXGBE_DEVICE_V2A)},
 	{RTE_PCI_DEVICE(AMD_PCI_VENDOR_ID, AMD_PCI_AXGBE_DEVICE_V2B)},
@@ -2419,10 +2430,10 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 
 	pdata->tx_desc_count = AXGBE_MAX_RING_DESC;
 	pdata->rx_desc_count = AXGBE_MAX_RING_DESC;
-	pthread_mutex_init(&pdata->xpcs_mutex, NULL);
-	pthread_mutex_init(&pdata->i2c_mutex, NULL);
-	pthread_mutex_init(&pdata->an_mutex, NULL);
-	pthread_mutex_init(&pdata->phy_mutex, NULL);
+	axgbe_init_mutex(&pdata->xpcs_mutex);
+	axgbe_init_mutex(&pdata->i2c_mutex);
+	axgbe_init_mutex(&pdata->an_mutex);
+	axgbe_init_mutex(&pdata->phy_mutex);
 
 	ret = pdata->phy_if.phy_init(pdata);
 	if (ret) {
Signature · 1 line
2.53.0
#13 lore raw

PATCH v2 5/6 net/bnxt: fix mutexes for multi-process

In reply to Stephen Hemminger

The BNXT driver supports secondary processes, as shown by the explicit check in bnxt_dev_init(). Multiple mutexes are located in structures allocated in shared memory (dev_private and rte_zmalloc'd structures) that are accessible by both primary and secondary processes.

However, the mutexes are initialized without PTHREAD_PROCESS_SHARED attribute, which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures in shared memory that are accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

This patch adds a helper function and updates all mutex initializations in the BNXT driver:
- flow_lock, def_cp_lock, health_check_lock, err_recovery_lock in struct bnxt (bnxt_ethdev.c)
- vfr_start_lock in rep_info (bnxt_ethdev.c)
- txq_lock in struct bnxt_tx_queue (bnxt_txq.c)
- bnxt_ulp_mutex in session state (bnxt_ulp.c)
- flow_db_lock in cfg_data (bnxt_ulp_tf.c, bnxt_ulp_tfc.c)

Bugzilla ID: 662

Fixes 1cb3d39a48f7 ("net/bnxt: synchronize between flow related functions")
Fixes 5526c8025d4d ("net/bnxt: fix race between interrupt handler and dev config")
Fixes b59e4be2b6a7 ("net/bnxt: fix VF representor port add")
---
 drivers/net/bnxt/bnxt_ethdev.c         | 11 ++++++-----
 drivers/net/bnxt/bnxt_txq.c            |  3 ++-
 drivers/net/bnxt/bnxt_util.c           | 13 +++++++++++++
 drivers/net/bnxt/bnxt_util.h           |  2 ++
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c     |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c  |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c |  2 +-
 7 files changed, 26 insertions(+), 9 deletions(-)
drivers/net/bnxt/bnxt_ethdev.c +6 -5
index b677f9491d..3f55ad041d 100644
@@ -5899,10 +5899,10 @@ static int bnxt_get_config(struct bnxt *bp)
 static int
 bnxt_init_locks(struct bnxt *bp)
 {
-	pthread_mutex_init(&bp->flow_lock, NULL);
-	pthread_mutex_init(&bp->def_cp_lock, NULL);
-	pthread_mutex_init(&bp->health_check_lock, NULL);
-	pthread_mutex_init(&bp->err_recovery_lock, NULL);
+	bnxt_init_mutex(&bp->flow_lock);
+	bnxt_init_mutex(&bp->def_cp_lock);
+	bnxt_init_mutex(&bp->health_check_lock);
+	bnxt_init_mutex(&bp->err_recovery_lock);
 
 	return 0;
 }
@@ -6920,7 +6920,8 @@ static int bnxt_init_rep_info(struct bnxt *bp)
 	for (i = 0; i < BNXT_MAX_CFA_CODE; i++)
 		bp->cfa_code_map[i] = BNXT_VF_IDX_INVALID;
 
-	return pthread_mutex_init(&bp->rep_info->vfr_start_lock, NULL);
+	bnxt_init_mutex(&bp->rep_info->vfr_start_lock);
+	return 0;
 }
 
 static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
drivers/net/bnxt/bnxt_txq.c +2 -1
index 7752f06eb7..8c834acb1d 100644
@@ -204,7 +204,8 @@ int bnxt_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
 		goto err;
 	}
 
-	return pthread_mutex_init(&txq->txq_lock, NULL);
+	bnxt_init_mutex(&txq->txq_lock);
+	return 0;
 err:
 	bnxt_tx_queue_release_op(eth_dev, queue_idx);
 	return rc;
drivers/net/bnxt/bnxt_util.c +13 -0
index aa184496c2..c3f0a03f25 100644
@@ -3,6 +3,7 @@
  * All rights reserved.
  */
 
+#include <pthread.h>
 #include <inttypes.h>
 #include <rte_ether.h>
 
@@ -37,3 +38,15 @@ uint8_t hweight32(uint32_t word32)
 	res = res + (res >> 8);
 	return (res + (res >> 16)) & 0x000000FF;
 }
+
+/* Initialize mutex so that it can be shared between processes */
+void
+bnxt_init_mutex(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
drivers/net/bnxt/bnxt_util.h +2 -0
index 416a6a2609..e1b45d1bb5 100644
@@ -16,4 +16,6 @@
 int bnxt_check_zero_bytes(const uint8_t *bytes, int len);
 void bnxt_eth_hw_addr_random(uint8_t *mac_addr);
 uint8_t hweight32(uint32_t word32);
+void bnxt_init_mutex(pthread_mutex_t *mutex);
+
 #endif /* _BNXT_UTIL_H_ */
drivers/net/bnxt/tf_ulp/bnxt_ulp.c +1 -1
index 0c03ae7a83..c2c93859ff 100644
@@ -214,7 +214,7 @@ ulp_session_init(struct bnxt *bp,
 			session->pci_info.domain = pci_addr->domain;
 			session->pci_info.bus = pci_addr->bus;
 			memcpy(session->dsn, bp->dsn, sizeof(session->dsn));
-			pthread_mutex_init(&session->bnxt_ulp_mutex, NULL);
+			bnxt_init_mutex(&session->bnxt_ulp_mutex);
 			STAILQ_INSERT_TAIL(&bnxt_ulp_session_list,
 					   session, next);
 		}
drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c +1 -1
index bc347de202..d87120aea3 100644
@@ -1469,7 +1469,7 @@ ulp_tf_init(struct bnxt *bp,
 		goto jump_to_error;
 	}
 
-	pthread_mutex_init(&bp->ulp_ctx->cfg_data->flow_db_lock, NULL);
+	bnxt_init_mutex(&bp->ulp_ctx->cfg_data->flow_db_lock);
 
 	/* Initialize ulp dparms with values devargs passed */
 	rc = ulp_tf_dparms_init(bp, bp->ulp_ctx);
drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c +1 -1
index ad44ec93ca..0c3d1f7dae 100644
@@ -1038,7 +1038,7 @@ ulp_tfc_init(struct bnxt *bp,
 		goto jump_to_error;
 	}
 
-	pthread_mutex_init(&bp->ulp_ctx->cfg_data->flow_db_lock, NULL);
+	bnxt_init_mutex(&bp->ulp_ctx->cfg_data->flow_db_lock);
 
 	rc = ulp_tfc_dparms_init(bp, bp->ulp_ctx, ulp_dev_id);
 	if (rc) {
Signature · 1 line
2.53.0
#14 lore raw

PATCH v2 6/6 net/hinic: fix mutexes for multi-process

In reply to Stephen Hemminger

The hinic driver supports secondary processes, as shown by the explicit check in hinic_dev_init(). Multiple mutexes are located in structures allocated in shared memory (dev_private and rte_zmalloc'd structures) that are accessible by both primary and secondary processes.

However, the mutexes are initialized without PTHREAD_PROCESS_SHARED attribute, which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them. When a mutex protects data structures in shared memory that are accessed by multiple processes, pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

Fix the hinic_mutex_init() wrapper function to set the PTHREAD_PROCESS_SHARED attribute on all mutexes.

Note: all callers to hinic_mutex_init() pass NULL as second argument, so it could be simplified but that would entail more changes to base vendor code.

Bugzilla ID: 662

Fixes ae865766b334 ("net/hinic: replace spinlock with mutex")
---
 drivers/net/hinic/base/hinic_compat.h | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
drivers/net/hinic/base/hinic_compat.h +12 -1
index d3994c50e9..18148a119e 100644
@@ -197,10 +197,21 @@ static inline u16 ilog2(u32 n)
 	return res;
 }
 
+/*
+ * Initialize mutex for process-shared access.
+ * Structures may be in shared memory accessible by multiple processes,
+ * so mutexes must use PTHREAD_PROCESS_SHARED.
+ */
 static inline int hinic_mutex_init(pthread_mutex_t *pthreadmutex,
 					const pthread_mutexattr_t *mattr)
 {
-	return pthread_mutex_init(pthreadmutex, mattr);
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(pthreadmutex, mattr ? mattr : &attr);
+	pthread_mutexattr_destroy(&attr);
+	return 0;
 }
 
 static inline int hinic_mutex_destroy(pthread_mutex_t *pthreadmutex)
Signature · 1 line
2.53.0
#15 lore raw

Re: [PATCH v2 0/6] fix process shared pthread mutexes

In reply to Stephen Hemminger
Quoted 4 lines · On Tue, 14 Apr 2026 07:39:52 -0700 Stephen Hemminger…

Several drivers and the ethdev layer initialize pthread mutexes in shared memory with default (process-private) attributes. This is undefined behavior when secondary processes use them.

Quoted 8 lines

This series adds PTHREAD_PROCESS_SHARED to all affected mutexes. All are on control paths (firmware mailbox, hotplug, flow ops, PHY negotiation) where sleeping is acceptable.

See POSIX spec:
https://pubs.opengroup.org/onlinepubs/009696899/functions/pthread_mutexattr_getpshared.html

Bugzilla ID: 662

I would like someone to review and ack some or all of these patches.

#16 lore raw

RE: [PATCH v2 0/6] fix process shared pthread mutexes

In reply to Stephen Hemminger
Quoted 40 lines

Several drivers and the ethdev layer initialize pthread mutexes in shared memory with default (process-private) attributes. This is undefined behavior when secondary processes use them.

This series adds PTHREAD_PROCESS_SHARED to all affected mutexes. All are on control paths (firmware mailbox, hotplug, flow ops, PHY negotiation) where sleeping is acceptable.

See POSIX spec:
https://pubs.opengroup.org/onlinepubs/009696899/functions/pthread_mutexat
tr_getpshared.html

Bugzilla ID: 662

v2 - fix build on Windows which does not need this.

Stephen Hemminger (6):
ethdev: fix flow_ops_mutex for multi-process net/failsafe: fix hotplug_mutex for multi-process net/atlantic: fix mbox_mutex for multi-process net/axgbe: fix mutexes for multi-process net/bnxt: fix mutexes for multi-process net/hinic: fix mutexes for multi-process

 drivers/net/atlantic/atl_ethdev.c      | 14 +++++++++++++-
 drivers/net/axgbe/axgbe_ethdev.c       | 19 +++++++++++++++----
 drivers/net/bnxt/bnxt_ethdev.c         | 11 ++++++-----
 drivers/net/bnxt/bnxt_txq.c            |  3 ++-
 drivers/net/bnxt/bnxt_util.c           | 13 +++++++++++++
 drivers/net/bnxt/bnxt_util.h           |  2 ++
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c     |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c  |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c |  2 +-
 drivers/net/failsafe/failsafe.c        | 15 ++++++++++++---
 drivers/net/hinic/base/hinic_compat.h  | 13 ++++++++++++-
 lib/ethdev/ethdev_driver.c             | 22 +++++++++++++++++++++-
 12 files changed, 99 insertions(+), 19 deletions(-)
Quoted 1 line

--

LGTM
Just as a generic thought :
- as these new functions is practically identical in all drivers, would it make sense to create some helper function in drivcers/common or somewhere in other place and use it everywhere.
With or without suggested change:

Quoted 1 line

2.53.0

author Applied
#17 lore raw

Re: [PATCH v2 0/6] fix process shared pthread mutexes

In reply to Stephen Hemminger
Quoted 23 lines · On Tue, 14 Apr 2026 07:39:52 -0700 Stephen Hemminger…

Several drivers and the ethdev layer initialize pthread mutexes in shared memory with default (process-private) attributes. This is undefined behavior when secondary processes use them.

This series adds PTHREAD_PROCESS_SHARED to all affected mutexes. All are on control paths (firmware mailbox, hotplug, flow ops, PHY negotiation) where sleeping is acceptable.

See POSIX spec:
https://pubs.opengroup.org/onlinepubs/009696899/functions/pthread_mutexattr_getpshared.html

Bugzilla ID: 662

v2 - fix build on Windows which does not need this.

Stephen Hemminger (6):
ethdev: fix flow_ops_mutex for multi-process net/failsafe: fix hotplug_mutex for multi-process net/atlantic: fix mbox_mutex for multi-process net/axgbe: fix mutexes for multi-process net/bnxt: fix mutexes for multi-process net/hinic: fix mutexes for multi-process

Quoted 14 lines
 drivers/net/atlantic/atl_ethdev.c      | 14 +++++++++++++-
 drivers/net/axgbe/axgbe_ethdev.c       | 19 +++++++++++++++----
 drivers/net/bnxt/bnxt_ethdev.c         | 11 ++++++-----
 drivers/net/bnxt/bnxt_txq.c            |  3 ++-
 drivers/net/bnxt/bnxt_util.c           | 13 +++++++++++++
 drivers/net/bnxt/bnxt_util.h           |  2 ++
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c     |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c  |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c |  2 +-
 drivers/net/failsafe/failsafe.c        | 15 ++++++++++++---
 drivers/net/hinic/base/hinic_compat.h  | 13 ++++++++++++-
 lib/ethdev/ethdev_driver.c             | 22 +++++++++++++++++++++-
 12 files changed, 99 insertions(+), 19 deletions(-)

Applied to next-net with helper.

#18 lore raw

v3 cover af_packet: cleanup and test

In reply to Stephen Hemminger

A couple of small patches for the af_packet PMD.
Fix indentation and add a basic test.

Stephen Hemminger (2):
net/af_packet: fix indentation
test: add test for af_packet

v3 - fix unchecked return value in test

 app/test/meson.build                      |    1 +
 app/test/test_pmd_af_packet.c             | 1174 +++++++++++++++++++++
 drivers/net/af_packet/rte_eth_af_packet.c |   64 +-
 3 files changed, 1207 insertions(+), 32 deletions(-)
 create mode 100644 app/test/test_pmd_af_packet.c
Signature · 1 line
2.53.0
#19 lore raw

PATCH v3 1/2 net/af_packet: fix indentation

DPDK code is supposed to be indented with TABS not spaces. Also, used "unsigned int" to keep checkpatch happy.

---
 drivers/net/af_packet/rte_eth_af_packet.c | 64 +++++++++++------------
 1 file changed, 32 insertions(+), 32 deletions(-)
drivers/net/af_packet/rte_eth_af_packet.c +32 -32
index 0ee94e71ea..8303ff5ca9 100644
@@ -79,7 +79,7 @@ struct __rte_cache_aligned pkt_tx_queue {
 };
 
 struct pmd_internals {
-	unsigned nb_queues;
+	unsigned int nb_queues;
 
 	int if_index;
 	char *if_name;
@@ -602,7 +602,7 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 static int
 eth_link_update(struct rte_eth_dev *dev,
-                int wait_to_complete __rte_unused)
+		int wait_to_complete __rte_unused)
 {
 	const struct pmd_internals *internals = dev->data->dev_private;
 	struct rte_eth_link *dev_link = &dev->data->dev_link;
@@ -622,11 +622,11 @@ eth_link_update(struct rte_eth_dev *dev,
 
 static int
 eth_rx_queue_setup(struct rte_eth_dev *dev,
-                   uint16_t rx_queue_id,
-                   uint16_t nb_rx_desc __rte_unused,
-                   unsigned int socket_id __rte_unused,
-                   const struct rte_eth_rxconf *rx_conf __rte_unused,
-                   struct rte_mempool *mb_pool)
+		   uint16_t rx_queue_id,
+		   uint16_t nb_rx_desc __rte_unused,
+		   unsigned int socket_id __rte_unused,
+		   const struct rte_eth_rxconf *rx_conf __rte_unused,
+		   struct rte_mempool *mb_pool)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
 	struct pkt_rx_queue *pkt_q = &internals->rx_queue[rx_queue_id];
@@ -660,10 +660,10 @@ eth_rx_queue_setup(struct rte_eth_dev *dev,
 
 static int
 eth_tx_queue_setup(struct rte_eth_dev *dev,
-                   uint16_t tx_queue_id,
-                   uint16_t nb_tx_desc __rte_unused,
-                   unsigned int socket_id __rte_unused,
-                   const struct rte_eth_txconf *tx_conf __rte_unused)
+		   uint16_t tx_queue_id,
+		   uint16_t nb_tx_desc __rte_unused,
+		   unsigned int socket_id __rte_unused,
+		   const struct rte_eth_txconf *tx_conf __rte_unused)
 {
 
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -790,8 +790,8 @@ static const struct eth_dev_ops ops = {
  */
 static int
 open_packet_iface(const char *key __rte_unused,
-                  const char *value __rte_unused,
-                  void *extra_args)
+		  const char *value __rte_unused,
+		  void *extra_args)
 {
 	int *sockfd = extra_args;
 
@@ -854,17 +854,17 @@ get_fanout(const char *fanout_mode, int if_index)
 
 static int
 rte_pmd_init_internals(struct rte_vdev_device *dev,
-                       const int sockfd,
-                       const unsigned nb_queues,
-                       unsigned int blocksize,
-                       unsigned int blockcnt,
-                       unsigned int framesize,
-                       unsigned int framecnt,
+		       const int sockfd,
+		       const unsigned int nb_queues,
+		       unsigned int blocksize,
+		       unsigned int blockcnt,
+		       unsigned int framesize,
+		       unsigned int framecnt,
 		       unsigned int qdisc_bypass,
 		       const char *fanout_mode,
-                       struct pmd_internals **internals,
-                       struct rte_eth_dev **eth_dev,
-                       struct rte_kvargs *kvlist)
+		       struct pmd_internals **internals,
+		       struct rte_eth_dev **eth_dev,
+		       struct rte_kvargs *kvlist)
 {
 	const char *name = rte_vdev_device_name(dev);
 	const unsigned int numa_node = dev->device.numa_node;
@@ -890,7 +890,7 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 	if (pair == NULL) {
 		PMD_LOG(ERR,
 			"%s: no interface specified for AF_PACKET ethdev",
-		        name);
+			name);
 		return -1;
 	}
 
@@ -899,7 +899,7 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 		name, numa_node);
 
 	*internals = rte_zmalloc_socket(name, sizeof(**internals),
-	                                0, numa_node);
+					0, numa_node);
 	if (*internals == NULL)
 		return -1;
 
@@ -1140,8 +1140,8 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 
 static int
 rte_eth_from_packet(struct rte_vdev_device *dev,
-                    int const *sockfd,
-                    struct rte_kvargs *kvlist)
+		    int const *sockfd,
+		    struct rte_kvargs *kvlist)
 {
 	const char *name = rte_vdev_device_name(dev);
 	struct pmd_internals *internals = NULL;
@@ -1172,7 +1172,7 @@ rte_eth_from_packet(struct rte_vdev_device *dev,
 			if (qpairs < 1) {
 				PMD_LOG(ERR,
 					"%s: invalid qpairs value",
-				        name);
+					name);
 				return -1;
 			}
 			continue;
@@ -1182,7 +1182,7 @@ rte_eth_from_packet(struct rte_vdev_device *dev,
 			if (!blocksize) {
 				PMD_LOG(ERR,
 					"%s: invalid blocksize value",
-				        name);
+					name);
 				return -1;
 			}
 			continue;
@@ -1192,7 +1192,7 @@ rte_eth_from_packet(struct rte_vdev_device *dev,
 			if (!framesize) {
 				PMD_LOG(ERR,
 					"%s: invalid framesize value",
-				        name);
+					name);
 				return -1;
 			}
 			continue;
@@ -1202,7 +1202,7 @@ rte_eth_from_packet(struct rte_vdev_device *dev,
 			if (!framecount) {
 				PMD_LOG(ERR,
 					"%s: invalid framecount value",
-				        name);
+					name);
 				return -1;
 			}
 			continue;
@@ -1226,7 +1226,7 @@ rte_eth_from_packet(struct rte_vdev_device *dev,
 	if (framesize > blocksize) {
 		PMD_LOG(ERR,
 			"%s: AF_PACKET MMAP frame size exceeds block size!",
-		        name);
+			name);
 		return -1;
 	}
 
@@ -1301,7 +1301,7 @@ rte_pmd_af_packet_probe(struct rte_vdev_device *dev)
 	if (rte_kvargs_count(kvlist, ETH_AF_PACKET_IFACE_ARG) == 1) {
 
 		ret = rte_kvargs_process(kvlist, ETH_AF_PACKET_IFACE_ARG,
-		                         &open_packet_iface, &sockfd);
+					 &open_packet_iface, &sockfd);
 		if (ret < 0)
 			goto exit;
 	}
Signature · 1 line
2.53.0
#20 lore raw

PATCH v3 2/2 test: add test for af_packet

In reply to Stephen Hemminger

Add a test for AF_PACKET PMD.
This test was generated with Claude AI based off of existing test_pmd_ring.c with some cleanup afterwards.

---
 app/test/meson.build          |    1 +
 app/test/test_pmd_af_packet.c | 1174 +++++++++++++++++++++++++++++++++
 2 files changed, 1175 insertions(+)
 create mode 100644 app/test/test_pmd_af_packet.c
app/test/meson.build +1 -0
index 7d458f9c07..24f71fb987 100644
@@ -141,6 +141,7 @@ source_file_deps = {
     'test_per_lcore.c': [],
     'test_pflock.c': [],
     'test_pie.c': ['sched'],
+    'test_pmd_af_packet.c': ['net_af_packet', 'ethdev', 'bus_vdev'],
     'test_pmd_perf.c': ['ethdev', 'net'] + packet_burst_generator_deps,
     'test_pmd_ring.c': ['net_ring', 'ethdev', 'bus_vdev'],
     'test_pmd_ring_perf.c': ['ethdev', 'net_ring', 'bus_vdev'],
app/test/test_pmd_af_packet.c new file +1174 -0

1,174 lines not on this page · raw message

Signature · 1 line
2.53.0
#21 lore raw

v4 cover af_packet: cleanup and test

In reply to Stephen Hemminger

A couple of small patches for the af_packet PMD.
Fix indentation and add a basic test.

Stephen Hemminger (2):
net/af_packet: fix indentation
test: add test for af_packet

v4 - fix test to use stdatomic correctly

Stephen Hemminger (2):
net/af_packet: fix indentation
test: add test for af_packet

 app/test/meson.build                      |    1 +
 app/test/test_pmd_af_packet.c             | 1122 +++++++++++++++++++++
 drivers/net/af_packet/rte_eth_af_packet.c |   64 +-
 3 files changed, 1155 insertions(+), 32 deletions(-)
 create mode 100644 app/test/test_pmd_af_packet.c
Signature · 1 line
2.53.0
#22 lore raw

PATCH v4 1/2 net/af_packet: fix indentation

DPDK code is supposed to be indented with TABS not spaces. Also, used "unsigned int" to keep checkpatch happy.

---
 drivers/net/af_packet/rte_eth_af_packet.c | 64 +++++++++++------------
 1 file changed, 32 insertions(+), 32 deletions(-)
drivers/net/af_packet/rte_eth_af_packet.c +32 -32
index 0ee94e71ea..8303ff5ca9 100644
@@ -79,7 +79,7 @@ struct __rte_cache_aligned pkt_tx_queue {
 };
 
 struct pmd_internals {
-	unsigned nb_queues;
+	unsigned int nb_queues;
 
 	int if_index;
 	char *if_name;
@@ -602,7 +602,7 @@ eth_dev_close(struct rte_eth_dev *dev)
 
 static int
 eth_link_update(struct rte_eth_dev *dev,
-                int wait_to_complete __rte_unused)
+		int wait_to_complete __rte_unused)
 {
 	const struct pmd_internals *internals = dev->data->dev_private;
 	struct rte_eth_link *dev_link = &dev->data->dev_link;
@@ -622,11 +622,11 @@ eth_link_update(struct rte_eth_dev *dev,
 
 static int
 eth_rx_queue_setup(struct rte_eth_dev *dev,
-                   uint16_t rx_queue_id,
-                   uint16_t nb_rx_desc __rte_unused,
-                   unsigned int socket_id __rte_unused,
-                   const struct rte_eth_rxconf *rx_conf __rte_unused,
-                   struct rte_mempool *mb_pool)
+		   uint16_t rx_queue_id,
+		   uint16_t nb_rx_desc __rte_unused,
+		   unsigned int socket_id __rte_unused,
+		   const struct rte_eth_rxconf *rx_conf __rte_unused,
+		   struct rte_mempool *mb_pool)
 {
 	struct pmd_internals *internals = dev->data->dev_private;
 	struct pkt_rx_queue *pkt_q = &internals->rx_queue[rx_queue_id];
@@ -660,10 +660,10 @@ eth_rx_queue_setup(struct rte_eth_dev *dev,
 
 static int
 eth_tx_queue_setup(struct rte_eth_dev *dev,
-                   uint16_t tx_queue_id,
-                   uint16_t nb_tx_desc __rte_unused,
-                   unsigned int socket_id __rte_unused,
-                   const struct rte_eth_txconf *tx_conf __rte_unused)
+		   uint16_t tx_queue_id,
+		   uint16_t nb_tx_desc __rte_unused,
+		   unsigned int socket_id __rte_unused,
+		   const struct rte_eth_txconf *tx_conf __rte_unused)
 {
 
 	struct pmd_internals *internals = dev->data->dev_private;
@@ -790,8 +790,8 @@ static const struct eth_dev_ops ops = {
  */
 static int
 open_packet_iface(const char *key __rte_unused,
-                  const char *value __rte_unused,
-                  void *extra_args)
+		  const char *value __rte_unused,
+		  void *extra_args)
 {
 	int *sockfd = extra_args;
 
@@ -854,17 +854,17 @@ get_fanout(const char *fanout_mode, int if_index)
 
 static int
 rte_pmd_init_internals(struct rte_vdev_device *dev,
-                       const int sockfd,
-                       const unsigned nb_queues,
-                       unsigned int blocksize,
-                       unsigned int blockcnt,
-                       unsigned int framesize,
-                       unsigned int framecnt,
+		       const int sockfd,
+		       const unsigned int nb_queues,
+		       unsigned int blocksize,
+		       unsigned int blockcnt,
+		       unsigned int framesize,
+		       unsigned int framecnt,
 		       unsigned int qdisc_bypass,
 		       const char *fanout_mode,
-                       struct pmd_internals **internals,
-                       struct rte_eth_dev **eth_dev,
-                       struct rte_kvargs *kvlist)
+		       struct pmd_internals **internals,
+		       struct rte_eth_dev **eth_dev,
+		       struct rte_kvargs *kvlist)
 {
 	const char *name = rte_vdev_device_name(dev);
 	const unsigned int numa_node = dev->device.numa_node;
@@ -890,7 +890,7 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 	if (pair == NULL) {
 		PMD_LOG(ERR,
 			"%s: no interface specified for AF_PACKET ethdev",
-		        name);
+			name);
 		return -1;
 	}
 
@@ -899,7 +899,7 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 		name, numa_node);
 
 	*internals = rte_zmalloc_socket(name, sizeof(**internals),
-	                                0, numa_node);
+					0, numa_node);
 	if (*internals == NULL)
 		return -1;
 
@@ -1140,8 +1140,8 @@ rte_pmd_init_internals(struct rte_vdev_device *dev,
 
 static int
 rte_eth_from_packet(struct rte_vdev_device *dev,
-                    int const *sockfd,
-                    struct rte_kvargs *kvlist)
+		    int const *sockfd,
+		    struct rte_kvargs *kvlist)
 {
 	const char *name = rte_vdev_device_name(dev);
 	struct pmd_internals *internals = NULL;
@@ -1172,7 +1172,7 @@ rte_eth_from_packet(struct rte_vdev_device *dev,
 			if (qpairs < 1) {
 				PMD_LOG(ERR,
 					"%s: invalid qpairs value",
-				        name);
+					name);
 				return -1;
 			}
 			continue;
@@ -1182,7 +1182,7 @@ rte_eth_from_packet(struct rte_vdev_device *dev,
 			if (!blocksize) {
 				PMD_LOG(ERR,
 					"%s: invalid blocksize value",
-				        name);
+					name);
 				return -1;
 			}
 			continue;
@@ -1192,7 +1192,7 @@ rte_eth_from_packet(struct rte_vdev_device *dev,
 			if (!framesize) {
 				PMD_LOG(ERR,
 					"%s: invalid framesize value",
-				        name);
+					name);
 				return -1;
 			}
 			continue;
@@ -1202,7 +1202,7 @@ rte_eth_from_packet(struct rte_vdev_device *dev,
 			if (!framecount) {
 				PMD_LOG(ERR,
 					"%s: invalid framecount value",
-				        name);
+					name);
 				return -1;
 			}
 			continue;
@@ -1226,7 +1226,7 @@ rte_eth_from_packet(struct rte_vdev_device *dev,
 	if (framesize > blocksize) {
 		PMD_LOG(ERR,
 			"%s: AF_PACKET MMAP frame size exceeds block size!",
-		        name);
+			name);
 		return -1;
 	}
 
@@ -1301,7 +1301,7 @@ rte_pmd_af_packet_probe(struct rte_vdev_device *dev)
 	if (rte_kvargs_count(kvlist, ETH_AF_PACKET_IFACE_ARG) == 1) {
 
 		ret = rte_kvargs_process(kvlist, ETH_AF_PACKET_IFACE_ARG,
-		                         &open_packet_iface, &sockfd);
+					 &open_packet_iface, &sockfd);
 		if (ret < 0)
 			goto exit;
 	}
Signature · 1 line
2.53.0
#23 lore raw

PATCH v4 2/2 test: add test for af_packet

In reply to Stephen Hemminger

Add a test for AF_PACKET PMD.
This test was generated with Claude AI based off of existing test_pmd_ring.c with some cleanup afterwards.

---
 app/test/meson.build          |    1 +
 app/test/test_pmd_af_packet.c | 1122 +++++++++++++++++++++++++++++++++
 2 files changed, 1123 insertions(+)
 create mode 100644 app/test/test_pmd_af_packet.c
app/test/meson.build +1 -0
index 7d458f9c07..24f71fb987 100644
@@ -141,6 +141,7 @@ source_file_deps = {
     'test_per_lcore.c': [],
     'test_pflock.c': [],
     'test_pie.c': ['sched'],
+    'test_pmd_af_packet.c': ['net_af_packet', 'ethdev', 'bus_vdev'],
     'test_pmd_perf.c': ['ethdev', 'net'] + packet_burst_generator_deps,
     'test_pmd_ring.c': ['net_ring', 'ethdev', 'bus_vdev'],
     'test_pmd_ring_perf.c': ['ethdev', 'net_ring', 'bus_vdev'],
app/test/test_pmd_af_packet.c new file +1122 -0

1,122 lines not on this page · raw message

Signature · 1 line
2.53.0
#24 lore raw

v3 cover fix use of pthread mutex between processes

In reply to Stephen Hemminger

Several drivers and the ethdev layer initialize pthread mutexes in shared memory with default (process-private) attributes. This is undefined behavior when secondary processes use them.

This series adds PTHREAD_PROCESS_SHARED to all affected mutexes. All are on control paths (firmware mailbox, hotplug, flow ops, PHY negotiation) where sleeping is acceptable.

See POSIX spec:
https://pubs.opengroup.org/onlinepubs/009696899/functions/pthread_mutexattr_getpshared.html

Stephen Hemminger (7):
eal: add helper to initialize process-shared mutex
ethdev: fix flow_ops_mutex for multi-process net/failsafe: fix hotplug_mutex for multi-process net/atlantic: fix mbox_mutex for multi-process net/axgbe: fix mutexes for multi-process net/bnxt: fix mutexes for multi-process net/hinic: fix mutexes for multi-process

 drivers/net/atlantic/atl_ethdev.c      |  3 ++-
 drivers/net/axgbe/axgbe_ethdev.c       | 14 +++++++-------
 drivers/net/bnxt/bnxt_ethdev.c         | 11 ++++++-----
 drivers/net/bnxt/bnxt_txq.c            |  3 ++-
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c     |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c  |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c |  2 +-
 drivers/net/failsafe/failsafe.c        | 15 ++++++++++++---
 drivers/net/hinic/base/hinic_compat.h  |  6 ++++--
 lib/eal/include/rte_thread.h           | 15 +++++++++++++++
 lib/eal/unix/rte_thread.c              | 12 ++++++++++++
 lib/eal/windows/rte_thread.c           | 10 ++++++++++
 lib/ethdev/ethdev_driver.c             |  2 +-
 13 files changed, 74 insertions(+), 23 deletions(-)
Signature · 1 line
2.53.0
#25 lore raw

PATCH v3 1/7 eal: add helper to initialize process-shared mutex

Drivers and libraries that place a pthread_mutex_t in shared memory to coordinate primary and secondary DPDK processes must initialize the mutex with PTHREAD_PROCESS_SHARED, otherwise behaviour is undefined. Add an internal helper so this is done in one place rather than copied into every driver.

Errors from the attribute calls are ignored: on platforms that do not support cross-process mutex sharing the mutex still works within a single process.

Bugzilla ID: 662
Cc: [email protected]

---
 lib/eal/include/rte_thread.h | 15 +++++++++++++++
 lib/eal/unix/rte_thread.c    | 12 ++++++++++++
 lib/eal/windows/rte_thread.c | 10 ++++++++++
 3 files changed, 37 insertions(+)
lib/eal/include/rte_thread.h +15 -0
index 8da9d4d3fb..27d5b9c594 100644
@@ -3,6 +3,7 @@
  * Copyright (C) 2022 Microsoft Corporation
  */
 
+#include <pthread.h>
 #include <stdint.h>
 
 #include <rte_os.h>
@@ -463,6 +464,20 @@ int rte_thread_value_set(rte_thread_key key, const void *value);
  */
 void *rte_thread_value_get(rte_thread_key key);
 
+/**
+ * Initialize a pthread mutex for use in shared memory accessible by
+ * multiple DPDK processes.
+ *
+ * Sets PTHREAD_PROCESS_SHARED so the mutex can synchronize threads
+ * across DPDK primary and secondary processes when the mutex resides
+ * in shared memory (e.g. hugepage memory).
+ *
+ * @param mutex
+ *   The mutex to initialize.
+ */
+__rte_internal
+void rte_thread_mutex_init_shared(pthread_mutex_t *mutex);
+
 #ifdef __cplusplus
 }
 #endif
lib/eal/unix/rte_thread.c +12 -0
index 950c0848ba..f3a6bd2ad3 100644
@@ -419,3 +419,15 @@ rte_thread_get_affinity_by_id(rte_thread_t thread_id,
 	return pthread_getaffinity_np((pthread_t)thread_id.opaque_id,
 		sizeof(*cpuset), cpuset);
 }
+
+RTE_EXPORT_INTERNAL_SYMBOL(rte_thread_mutex_init_shared)
+void
+rte_thread_mutex_init_shared(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
lib/eal/windows/rte_thread.c +10 -0
index 85e5a57346..19bb9ed61d 100644
@@ -621,3 +621,13 @@ rte_thread_get_affinity_by_id(rte_thread_t thread_id,
 	}
 	return ret;
 }
+
+/* Note: Windows does not have PTHREAD_PROCESS_SHARED
+ * and DPDK on Windows only supports single process model.
+ */
+RTE_EXPORT_INTERNAL_SYMBOL(rte_thread_mutex_init_shared)
+void
+rte_thread_mutex_init_shared(pthread_mutex_t *mutex)
+{
+	pthread_mutex_init(mutex, NULL);
+}
Signature · 1 line
2.53.0
#26 lore raw

PATCH v3 2/7 ethdev: fix flow_ops_mutex for multi-process

In reply to Stephen Hemminger

The flow_ops_mutex in rte_eth_dev_data lives in shared memory and is accessed by both primary and secondary processes. It must be initialized with PTHREAD_PROCESS_SHARED.

Bugzilla ID: 662

Fixes 80d1a9aff7f6 ("ethdev: make flow API thread safe")
---
 lib/ethdev/ethdev_driver.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/ethdev/ethdev_driver.c +1 -1
index 2ec665be0f..70ddce5bfc 100644
@@ -135,7 +135,7 @@ rte_eth_dev_allocate(const char *name)
 	eth_dev->data->port_id = port_id;
 	eth_dev->data->backer_port_id = RTE_MAX_ETHPORTS;
 	eth_dev->data->mtu = RTE_ETHER_MTU;
-	pthread_mutex_init(&eth_dev->data->flow_ops_mutex, NULL);
+	rte_thread_mutex_init_shared(&eth_dev->data->flow_ops_mutex);
 	RTE_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY);
 	eth_dev_shared_data->allocated_ports++;
 
Signature · 1 line
2.53.0
#27 lore raw

PATCH v3 3/7 net/failsafe: fix hotplug_mutex for multi-process

In reply to Stephen Hemminger

The failsafe driver supports secondary process attachment via rte_eth_dev_attach_secondary() in rte_pmd_failsafe_probe(). The hotplug_mutex protects shared state but was initialized without PTHREAD_PROCESS_SHARED.

The mutex is also recursive, so the EAL helper does not fit; set the attribute alongside the recursive type. Also add the missing pthread_mutexattr_destroy() call.

Bugzilla ID: 662

Fixes 655fcd68c7d2 ("net/failsafe: fix hotplug races")
---
 drivers/net/failsafe/failsafe.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
drivers/net/failsafe/failsafe.c +12 -3
index 3e590d38f7..aa3fe53b22 100644
@@ -144,11 +144,20 @@ fs_mutex_init(struct fs_priv *priv)
 	/* Allow mutex relocks for the thread holding the mutex. */
 	ret = pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
 	if (ret) {
-		ERROR("Cannot set mutex type - %s", strerror(ret));
-		return ret;
+		ERROR("Cannot set mutex recursive - %s", strerror(ret));
+		goto out;
+	}
+	/* Allow mutex to be shared between processes. */
+	ret = pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	if (ret) {
+		ERROR("Cannot set mutex pshared - %s", strerror(ret));
+		goto out;
 	}
+	pthread_mutex_init(&priv->hotplug_mutex, &attr);
 
-	return pthread_mutex_init(&priv->hotplug_mutex, &attr);
+out:
+	pthread_mutexattr_destroy(&attr);
+	return ret;
 }
 
 static int
Signature · 1 line
2.53.0
#28 lore raw

PATCH v3 4/7 net/atlantic: fix mbox_mutex for multi-process

In reply to Stephen Hemminger

The Atlantic driver supports secondary processes. The mbox_mutex in aq_hw_s lives in dev_private (shared memory) and must be initialized with PTHREAD_PROCESS_SHARED.

Bugzilla ID: 662

Fixes e9924638f5c9 ("net/atlantic: add FW mailbox guard mutex")
---
 drivers/net/atlantic/atl_ethdev.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/atlantic/atl_ethdev.c +2 -1
index 2925dc2478..f8744da221 100644
@@ -5,6 +5,7 @@
 #include <rte_string_fns.h>
 #include <ethdev_pci.h>
 #include <rte_alarm.h>
+#include <rte_thread.h>
 
 #include "atl_ethdev.h"
 #include "atl_common.h"
@@ -405,7 +406,7 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 
 	hw->aq_nic_cfg = &adapter->hw_cfg;
 
-	pthread_mutex_init(&hw->mbox_mutex, NULL);
+	rte_thread_mutex_init_shared(&hw->mbox_mutex);
 
 	/* disable interrupt */
 	atl_disable_intr(hw);
Signature · 1 line
2.53.0
#29 lore raw

PATCH v3 5/7 net/axgbe: fix mutexes for multi-process

In reply to Stephen Hemminger

The AXGBE driver supports secondary processes. The xpcs_mutex, i2c_mutex, an_mutex and phy_mutex in axgbe_port live in dev_private (shared memory) and must be initialized with PTHREAD_PROCESS_SHARED.

Also, make headers use standard <> format when accessing DPDK EAL includes.

Bugzilla ID: 662

Fixes 572890ef6625 ("net/axgbe: add structs for MAC init and reset")
---
 drivers/net/axgbe/axgbe_ethdev.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
drivers/net/axgbe/axgbe_ethdev.c +7 -7
index cfcd880961..9491f18b7c 100644
@@ -8,10 +8,10 @@
 #include "axgbe_common.h"
 #include "axgbe_phy.h"
 #include "axgbe_regs.h"
-#include "rte_time.h"
-
-#include "eal_filesystem.h"
 
+#include <rte_thread.h>
+#include <rte_time.h>
+#include <eal_filesystem.h>
 #include <rte_vect.h>
 
 #ifdef RTE_ARCH_X86
@@ -2419,10 +2419,10 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 
 	pdata->tx_desc_count = AXGBE_MAX_RING_DESC;
 	pdata->rx_desc_count = AXGBE_MAX_RING_DESC;
-	pthread_mutex_init(&pdata->xpcs_mutex, NULL);
-	pthread_mutex_init(&pdata->i2c_mutex, NULL);
-	pthread_mutex_init(&pdata->an_mutex, NULL);
-	pthread_mutex_init(&pdata->phy_mutex, NULL);
+	rte_thread_mutex_init_shared(&pdata->xpcs_mutex);
+	rte_thread_mutex_init_shared(&pdata->i2c_mutex);
+	rte_thread_mutex_init_shared(&pdata->an_mutex);
+	rte_thread_mutex_init_shared(&pdata->phy_mutex);
 
 	ret = pdata->phy_if.phy_init(pdata);
 	if (ret) {
Signature · 1 line
2.53.0
#30 lore raw

PATCH v3 6/7 net/bnxt: fix mutexes for multi-process

In reply to Stephen Hemminger

The BNXT driver supports secondary processes. Several mutexes live in structures allocated in shared memory (dev_private and rte_zmalloc'd structures) and must be initialized with PTHREAD_PROCESS_SHARED:

- flow_lock, def_cp_lock, health_check_lock, err_recovery_lock in struct bnxt
- vfr_start_lock in rep_info
- txq_lock in struct bnxt_tx_queue
- bnxt_ulp_mutex in session state
- flow_db_lock in cfg_data

Bugzilla ID: 662

Fixes 1cb3d39a48f7 ("net/bnxt: synchronize between flow related functions")
Fixes 5526c8025d4d ("net/bnxt: fix race between interrupt handler and dev config")
Fixes b59e4be2b6a7 ("net/bnxt: fix VF representor port add")
---
 drivers/net/bnxt/bnxt_ethdev.c         | 11 ++++++-----
 drivers/net/bnxt/bnxt_txq.c            |  3 ++-
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c     |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c  |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c |  2 +-
 5 files changed, 11 insertions(+), 9 deletions(-)
drivers/net/bnxt/bnxt_ethdev.c +6 -5
index b677f9491d..4c5c042ed8 100644
@@ -5899,10 +5899,10 @@ static int bnxt_get_config(struct bnxt *bp)
 static int
 bnxt_init_locks(struct bnxt *bp)
 {
-	pthread_mutex_init(&bp->flow_lock, NULL);
-	pthread_mutex_init(&bp->def_cp_lock, NULL);
-	pthread_mutex_init(&bp->health_check_lock, NULL);
-	pthread_mutex_init(&bp->err_recovery_lock, NULL);
+	rte_thread_mutex_init_shared(&bp->flow_lock);
+	rte_thread_mutex_init_shared(&bp->def_cp_lock);
+	rte_thread_mutex_init_shared(&bp->health_check_lock);
+	rte_thread_mutex_init_shared(&bp->err_recovery_lock);
 
 	return 0;
 }
@@ -6920,7 +6920,8 @@ static int bnxt_init_rep_info(struct bnxt *bp)
 	for (i = 0; i < BNXT_MAX_CFA_CODE; i++)
 		bp->cfa_code_map[i] = BNXT_VF_IDX_INVALID;
 
-	return pthread_mutex_init(&bp->rep_info->vfr_start_lock, NULL);
+	rte_thread_mutex_init_shared(&bp->rep_info->vfr_start_lock);
+	return 0;
 }
 
 static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
drivers/net/bnxt/bnxt_txq.c +2 -1
index 7752f06eb7..03407c556a 100644
@@ -204,7 +204,8 @@ int bnxt_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
 		goto err;
 	}
 
-	return pthread_mutex_init(&txq->txq_lock, NULL);
+	rte_thread_mutex_init_shared(&txq->txq_lock);
+	return 0;
 err:
 	bnxt_tx_queue_release_op(eth_dev, queue_idx);
 	return rc;
drivers/net/bnxt/tf_ulp/bnxt_ulp.c +1 -1
index 0c03ae7a83..c28cf4f2b5 100644
@@ -214,7 +214,7 @@ ulp_session_init(struct bnxt *bp,
 			session->pci_info.domain = pci_addr->domain;
 			session->pci_info.bus = pci_addr->bus;
 			memcpy(session->dsn, bp->dsn, sizeof(session->dsn));
-			pthread_mutex_init(&session->bnxt_ulp_mutex, NULL);
+			rte_thread_mutex_init_shared(&session->bnxt_ulp_mutex);
 			STAILQ_INSERT_TAIL(&bnxt_ulp_session_list,
 					   session, next);
 		}
drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c +1 -1
index bc347de202..6ca9d3309b 100644
@@ -1469,7 +1469,7 @@ ulp_tf_init(struct bnxt *bp,
 		goto jump_to_error;
 	}
 
-	pthread_mutex_init(&bp->ulp_ctx->cfg_data->flow_db_lock, NULL);
+	rte_thread_mutex_init_shared(&bp->ulp_ctx->cfg_data->flow_db_lock);
 
 	/* Initialize ulp dparms with values devargs passed */
 	rc = ulp_tf_dparms_init(bp, bp->ulp_ctx);
drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c +1 -1
index ad44ec93ca..fa98b2bca9 100644
@@ -1038,7 +1038,7 @@ ulp_tfc_init(struct bnxt *bp,
 		goto jump_to_error;
 	}
 
-	pthread_mutex_init(&bp->ulp_ctx->cfg_data->flow_db_lock, NULL);
+	rte_thread_mutex_init_shared(&bp->ulp_ctx->cfg_data->flow_db_lock);
 
 	rc = ulp_tfc_dparms_init(bp, bp->ulp_ctx, ulp_dev_id);
 	if (rc) {
Signature · 1 line
2.53.0
#31 lore raw

PATCH v3 7/7 net/hinic: fix mutexes for multi-process

In reply to Stephen Hemminger

The hinic driver supports secondary processes. Mutexes in structures allocated in shared memory must be initialized with PTHREAD_PROCESS_SHARED. All callers of hinic_mutex_init() pass NULL as the attribute argument, so route it through the EAL helper and drop the unused parameter handling.

Bugzilla ID: 662

Fixes ae865766b334 ("net/hinic: replace spinlock with mutex")
---
 drivers/net/hinic/base/hinic_compat.h | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/net/hinic/base/hinic_compat.h +4 -2
index d3994c50e9..707a3b92b9 100644
@@ -19,6 +19,7 @@
 #include <rte_spinlock.h>
 #include <rte_cycles.h>
 #include <rte_log.h>
+#include <rte_thread.h>
 
 typedef uint8_t   u8;
 typedef int8_t    s8;
@@ -198,9 +199,10 @@ static inline u16 ilog2(u32 n)
 }
 
 static inline int hinic_mutex_init(pthread_mutex_t *pthreadmutex,
-					const pthread_mutexattr_t *mattr)
+					__rte_unused const pthread_mutexattr_t *mattr)
 {
-	return pthread_mutex_init(pthreadmutex, mattr);
+	rte_thread_mutex_init_shared(pthreadmutex);
+	return 0;
 }
 
 static inline int hinic_mutex_destroy(pthread_mutex_t *pthreadmutex)
Signature · 1 line
2.53.0
author Applied
#32 lore raw

Re: [PATCH v3 0/7] fix use of pthread mutex between processes

In reply to Stephen Hemminger
Quoted 12 lines · On Wed, 29 Apr 2026 11:46:37 -0700 Stephen Hemminger…

Several drivers and the ethdev layer initialize pthread mutexes in shared memory with default (process-private) attributes. This is undefined behavior when secondary processes use them.

This series adds PTHREAD_PROCESS_SHARED to all affected mutexes. All are on control paths (firmware mailbox, hotplug, flow ops, PHY negotiation) where sleeping is acceptable.

See POSIX spec:
https://pubs.opengroup.org/onlinepubs/009696899/functions/pthread_mutexattr_getpshared.html

Quoted 8 lines

Stephen Hemminger (7):
eal: add helper to initialize process-shared mutex
ethdev: fix flow_ops_mutex for multi-process net/failsafe: fix hotplug_mutex for multi-process net/atlantic: fix mbox_mutex for multi-process net/axgbe: fix mutexes for multi-process net/bnxt: fix mutexes for multi-process net/hinic: fix mutexes for multi-process

Applied to next-net