Skip to contentSkip to conversation

net/mlx5e: Add mutual exclusion between PSP and PTP TX port timestamping

PATCH In review Tariq Toukan 3 participants 3 messages Jul 29 – Aug 10

Subsystems (6)
lore
Share & tools
lore

Tariq Toukan

author maintainer
#0 lore raw

PATCH 1/1 net/mlx5e: Add mutual exclusion between PSP and PTP TX port timestamping

From: Carolina Jubran <[email protected]>

The WQE flow_table_metadata field is shared by multiple TX offloads:
IPsec, MACsec, PTP timestamping and PSP each write to it.

Commit [1] resolved the conflicts between IPsec/MACsec and PTP by shifting their markers to bits [8+], leaving PTP's 8-bit metadata index in bits [7:0] without overlap.
But then PSP support was added in commit [2], which writes a 32-bit keyid across all bits of flow_table_metadata. Unlike IPsec and MACsec, PTP timestamping applies independently of encryption -- a PSP-encrypted packet can also require a HW timestamp. When both write to the same WQE, the values corrupt each other, leading to wrong PTP timestamp tracking and potentially wrong PSP encryption keys.

Prevent the conflict at configuration time by blocking:
- Enabling TX-port-TS when PSP has active TX keys.
- Adding PSP TX keys when TX-port-TS is already enabled.

[1] Commit 2ac207381c37 ("net/mlx5e: Prevent WQE metadata conflicts between timestamping and offloads")
[2] Commit 89ee2d92f66c ("net/mlx5e: Support PSP offload functionality")

Fixes 89ee2d92f66c ("net/mlx5e: Support PSP offload functionality")
---
 .../ethernet/mellanox/mlx5/core/en_accel/psp.c   | 16 +++++++++++++++-
 .../ethernet/mellanox/mlx5/core/en_accel/psp.h   | 11 +++++++++++
 .../net/ethernet/mellanox/mlx5/core/en_ethtool.c |  8 ++++++++
 3 files changed, 34 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +15 -1
index d9adb993e64d..8d76dd488d77 100644
@@ -993,6 +993,20 @@ static int mlx5e_psp_assoc_add(struct psp_dev *psd, struct psp_assoc *pas,
 	struct psp_key *nkey;
 	int err;
 
+	/* Mutual exclusion with TX-port-TS (shared WQE metadata). Reserve
+	 * tx_key_cnt under state_lock with the TS check so ethtool cannot
+	 * enable TX-port-TS until key creation completes or fails.
+	 */
+	mutex_lock(&priv->state_lock);
+	if (MLX5E_GET_PFLAG(&priv->channels.params, MLX5E_PFLAG_TX_PORT_TS)) {
+		mutex_unlock(&priv->state_lock);
+		NL_SET_ERR_MSG_MOD(extack,
+				   "TX-port-TS is active, PSP TX keys cannot be added");
+		return -EBUSY;
+	}
+	atomic_inc(&psp->tx_key_cnt);
+	mutex_unlock(&priv->state_lock);
+
 	mdev = priv->mdev;
 	nkey = (struct psp_key *)pas->drv_data;
 
@@ -1001,11 +1015,11 @@ static int mlx5e_psp_assoc_add(struct psp_dev *psd, struct psp_assoc *pas,
 					 MLX5_ACCEL_OBJ_PSP_KEY,
 					 &nkey->id);
 	if (err) {
+		atomic_dec(&psp->tx_key_cnt);
 		mlx5_core_err(mdev, "Failed to create encryption key (err = %d)\n", err);
 		return err;
 	}
 
-	atomic_inc(&psp->tx_key_cnt);
 	return 0;
 }
 
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.h +11 -0
index 6b62fef0d9a7..315c4b2101c2 100644
@@ -30,6 +30,11 @@ struct mlx5e_psp {
 	atomic_t tx_drop;
 };
 
+static inline bool mlx5e_psp_tx_keys_active(const struct mlx5e_priv *priv)
+{
+	return priv->psp && atomic_read(&priv->psp->tx_key_cnt);
+}
+
 static inline bool mlx5_is_psp_device(struct mlx5_core_dev *mdev)
 {
 	if (!MLX5_CAP_GEN(mdev, psp))
@@ -52,6 +57,11 @@ void mlx5e_psp_unregister(struct mlx5e_priv *priv);
 int mlx5e_psp_init(struct mlx5e_priv *priv);
 void mlx5e_psp_cleanup(struct mlx5e_priv *priv);
 #else
+static inline bool mlx5e_psp_tx_keys_active(const struct mlx5e_priv *priv)
+{
+	return false;
+}
+
 static inline int mlx5_accel_psp_fs_init_rx_tables(struct mlx5e_priv *priv)
 {
 	return 0;
@@ -74,4 +84,5 @@ static inline void mlx5e_psp_unregister(struct mlx5e_priv *priv) { }
 static inline int mlx5e_psp_init(struct mlx5e_priv *priv) { return 0; }
 static inline void mlx5e_psp_cleanup(struct mlx5e_priv *priv) { }
 #endif /* CONFIG_MLX5_EN_PSP */
+
 #endif /* __MLX5E_ACCEL_PSP_H__ */
drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +8 -0
index 112926d07634..98941088d100 100644
@@ -42,6 +42,7 @@
 #include "en/ptp.h"
 #include "lib/clock.h"
 #include "en/fs_ethtool.h"
+#include "en_accel/psp.h"
 
 #define LANES_UNKNOWN		 0
 
@@ -2387,6 +2388,13 @@ static int set_pflag_tx_port_ts(struct net_device *netdev, bool enable)
 			   __func__);
 		return -EINVAL;
 	}
+
+	if (enable && mlx5e_psp_tx_keys_active(priv)) {
+		netdev_err(priv->netdev,
+			   "%s: PSP TX keys are active, TX-port-TS cannot be enabled\n",
+			   __func__);
+		return -EBUSY;
+	}
 	MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_TX_PORT_TS, enable);
 	/* No need to verify SQ stop room as
 	 * ptpsq.txqsq.stop_room <= generic_sq->stop_room, and both

base-commit: 3bd438a58e910db5dc369aa25dfed1fc95f1b596

Signature · 1 line
2.44.0
maintainer
#1 lore raw
Quoted 12 lines · On Wed, 29 Jul 2026 09:51:37 +0300 Tariq Toukan wrote:

The WQE flow_table_metadata field is shared by multiple TX offloads:
IPsec, MACsec, PTP timestamping and PSP each write to it.

Commit [1] resolved the conflicts between IPsec/MACsec and PTP by shifting their markers to bits [8+], leaving PTP's 8-bit metadata index in bits [7:0] without overlap.
But then PSP support was added in commit [2], which writes a 32-bit keyid across all bits of flow_table_metadata. Unlike IPsec and MACsec, PTP timestamping applies independently of encryption -- a PSP-encrypted packet can also require a HW timestamp. When both write to the same WQE, the values corrupt each other, leading to wrong PTP timestamp tracking and potentially wrong PSP encryption keys.

Is the concern about Tx? (please clarify)
If so, Tx HW timestamps are never guaranteed, why can't we just skip timestamping for PSP packets? Only TCP supports PSP, and upstream TCP and HW timestamps are pretty much mutually exclusive.

Quoted 3 lines

Prevent the conflict at configuration time by blocking:
- Enabling TX-port-TS when PSP has active TX keys.
- Adding PSP TX keys when TX-port-TS is already enabled.

Why catch PSP at the Tx key programming? PSP docs very explicitly require drivers to _not_ enable the feature by default. And tx keys can't be added if it's disabled. So it would make much more sense to catch the enable / config callback of PSP?

Signature · 1 line
pw-bot: cr
#2 lore raw
Quoted 13 lines · Jakub Kicinski
Quoted 12 lines · On Wed, 29 Jul 2026 09:51:37 +0300 Tariq Toukan wrote:

The WQE flow_table_metadata field is shared by multiple TX offloads:
IPsec, MACsec, PTP timestamping and PSP each write to it.

Commit [1] resolved the conflicts between IPsec/MACsec and PTP by shifting their markers to bits [8+], leaving PTP's 8-bit metadata index in bits [7:0] without overlap.
But then PSP support was added in commit [2], which writes a 32-bit keyid across all bits of flow_table_metadata. Unlike IPsec and MACsec, PTP timestamping applies independently of encryption -- a PSP-encrypted packet can also require a HW timestamp. When both write to the same WQE, the values corrupt each other, leading to wrong PTP timestamp tracking and potentially wrong PSP encryption keys.

Quoted 4 lines

Is the concern about Tx? (please clarify)
If so, Tx HW timestamps are never guaranteed, why can't we just skip timestamping for PSP packets? Only TCP supports PSP, and upstream TCP and HW timestamps are pretty much mutually exclusive.

Yes, this is about Tx

You are right. I missed that PSP is TCP only, so it never selects ptpsq. Thanks for the clarification.

I'll drop this patch and I will revisit how to avoid a metadata collision without adding overhead on the datapath.

Thanks!

Quoted 8 lines
Quoted 4 lines

Prevent the conflict at configuration time by blocking:
- Enabling TX-port-TS when PSP has active TX keys.
- Adding PSP TX keys when TX-port-TS is already enabled.

Why catch PSP at the Tx key programming? PSP docs very explicitly require drivers to _not_ enable the feature by default. And tx keys can't be added if it's disabled. So it would make much more sense to catch the enable / config callback of PSP?