net/mlx5e: Add mutual exclusion between PSP and PTP TX port timestamping
Subsystems (6)
Tariq Toukan
maintainerPATCH 1/1 net/mlx5e: Add mutual exclusion between PSP and PTP TX port timestamping
From: Carolina Jubran <[email protected]>
The WQE flow_table_metadata field is shared by multiple TX offloads:
IPsec, MACsec, PTP timestamping and PSP each write to it.
Commit [1] resolved the conflicts between IPsec/MACsec and PTP by shifting their markers to bits [8+], leaving PTP's 8-bit metadata index in bits [7:0] without overlap.
But then PSP support was added in commit [2], which writes a 32-bit keyid across all bits of flow_table_metadata. Unlike IPsec and MACsec, PTP timestamping applies independently of encryption -- a PSP-encrypted packet can also require a HW timestamp. When both write to the same WQE, the values corrupt each other, leading to wrong PTP timestamp tracking and potentially wrong PSP encryption keys.
Prevent the conflict at configuration time by blocking:
- Enabling TX-port-TS when PSP has active TX keys.
- Adding PSP TX keys when TX-port-TS is already enabled.
[1] Commit 2ac207381c37 ("net/mlx5e: Prevent WQE metadata conflicts between timestamping and offloads")
[2] Commit 89ee2d92f66c ("net/mlx5e: Support PSP offload functionality")
--- .../ethernet/mellanox/mlx5/core/en_accel/psp.c | 16 +++++++++++++++- .../ethernet/mellanox/mlx5/core/en_accel/psp.h | 11 +++++++++++ .../net/ethernet/mellanox/mlx5/core/en_ethtool.c | 8 ++++++++ 3 files changed, 34 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +15 -1
@@ -993,6 +993,20 @@ static int mlx5e_psp_assoc_add(struct psp_dev *psd, struct psp_assoc *pas,struct psp_key *nkey;int err; +/* Mutual exclusion with TX-port-TS (shared WQE metadata). Reserve +* tx_key_cnt under state_lock with the TS check so ethtool cannot +* enable TX-port-TS until key creation completes or fails. +*/ +mutex_lock (&priv->state_lock); +if (MLX5E_GET_PFLAG (&priv->channels.params,MLX5E_PFLAG_TX_PORT_TS )) { +mutex_unlock (&priv->state_lock); +NL_SET_ERR_MSG_MOD (extack, +"TX-port-TS is active, PSP TX keys cannot be added" ); +return -EBUSY ; + } +atomic_inc (&psp->tx_key_cnt); +mutex_unlock (&priv->state_lock); + mdev = priv->mdev; nkey = (struct psp_key *)pas->drv_data;
@@ -1001,11 +1015,11 @@ static int mlx5e_psp_assoc_add(struct psp_dev *psd, struct psp_assoc *pas,MLX5_ACCEL_OBJ_PSP_KEY , &nkey->id);if (err) { +atomic_dec (&psp->tx_key_cnt);mlx5_core_err (mdev,"Failed to create encryption key (err = %d)\n" , err);return err; } -atomic_inc (&psp->tx_key_cnt);return 0 ; }
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.h +11 -0
@@ -30,6 +30,11 @@ struct mlx5e_psp {atomic_t tx_drop; }; +static inline bool mlx5e_psp_tx_keys_active (const struct mlx5e_priv *priv) +{ +return priv->psp &&atomic_read (&priv->psp->tx_key_cnt); +} +static inline bool mlx5_is_psp_device (struct mlx5_core_dev *mdev) {if (!MLX5_CAP_GEN (mdev, psp))
@@ -52,6 +57,11 @@ void mlx5e_psp_unregister(struct mlx5e_priv *priv);int mlx5e_psp_init (struct mlx5e_priv *priv);void mlx5e_psp_cleanup (struct mlx5e_priv *priv);#else +static inline bool mlx5e_psp_tx_keys_active (const struct mlx5e_priv *priv) +{ +return false; +} +static inline int mlx5_accel_psp_fs_init_rx_tables (struct mlx5e_priv *priv) {return 0 ;
@@ -74,4 +84,5 @@ static inline void mlx5e_psp_unregister(struct mlx5e_priv *priv) { }static inline int mlx5e_psp_init (struct mlx5e_priv *priv) {return 0 ; }static inline void mlx5e_psp_cleanup (struct mlx5e_priv *priv) { }#endif /* CONFIG_MLX5_EN_PSP */ +#endif /* __MLX5E_ACCEL_PSP_H__ */
drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +8 -0
@@ -42,6 +42,7 @@#include "en/ptp.h" #include "lib/clock.h" #include "en/fs_ethtool.h" +#include "en_accel/psp.h" #define LANES_UNKNOWN 0
@@ -2387,6 +2388,13 @@ static int set_pflag_tx_port_ts(struct net_device *netdev, bool enable) __func__);return -EINVAL ; } + +if (enable &&mlx5e_psp_tx_keys_active (priv)) { +netdev_err (priv->netdev, +"%s: PSP TX keys are active, TX-port-TS cannot be enabled\n" , + __func__); +return -EBUSY ; + }MLX5E_SET_PFLAG (&new_params,MLX5E_PFLAG_TX_PORT_TS , enable);/* No need to verify SQ stop room as * ptpsq.txqsq.stop_room <= generic_sq->stop_room, and both
base-commit: 3bd438a58e910db5dc369aa25dfed1fc95f1b596
Signature · 1 line
2.44.0
Jakub Kicinski
maintainerQuoted 12 lines · On Wed, 29 Jul 2026 09:51:37 +0300 Tariq Toukan wrote:
The WQE
flow_table_metadatafield is shared by multiple TX offloads:
IPsec, MACsec, PTP timestamping and PSP each write to it.Commit [1] resolved the conflicts between IPsec/MACsec and PTP by shifting their markers to bits [8+], leaving PTP's 8-bit metadata index in bits [7:0] without overlap.
But then PSP support was added in commit [2], which writes a 32-bit keyid across all bits offlow_table_metadata. Unlike IPsec and MACsec, PTP timestamping applies independently of encryption -- a PSP-encrypted packet can also require a HW timestamp. When both write to the same WQE, the values corrupt each other, leading to wrong PTP timestamp tracking and potentially wrong PSP encryption keys.
Is the concern about Tx? (please clarify)
If so, Tx HW timestamps are never guaranteed, why can't we just skip timestamping for PSP packets? Only TCP supports PSP, and upstream TCP and HW timestamps are pretty much mutually exclusive.
Quoted 3 lines
Prevent the conflict at configuration time by blocking:
- Enabling TX-port-TS when PSP has active TX keys.
- Adding PSP TX keys when TX-port-TS is already enabled.
Why catch PSP at the Tx key programming? PSP docs very explicitly require drivers to _not_ enable the feature by default. And tx keys can't be added if it's disabled. So it would make much more sense to catch the enable / config callback of PSP?
Signature · 1 line
pw-bot: cr
Quoted 13 lines · Jakub Kicinski
Quoted 12 lines · On Wed, 29 Jul 2026 09:51:37 +0300 Tariq Toukan wrote:
The WQE
flow_table_metadatafield is shared by multiple TX offloads:
IPsec, MACsec, PTP timestamping and PSP each write to it.Commit [1] resolved the conflicts between IPsec/MACsec and PTP by shifting their markers to bits [8+], leaving PTP's 8-bit metadata index in bits [7:0] without overlap.
But then PSP support was added in commit [2], which writes a 32-bit keyid across all bits offlow_table_metadata. Unlike IPsec and MACsec, PTP timestamping applies independently of encryption -- a PSP-encrypted packet can also require a HW timestamp. When both write to the same WQE, the values corrupt each other, leading to wrong PTP timestamp tracking and potentially wrong PSP encryption keys.
Quoted 4 lines
Is the concern about Tx? (please clarify)
If so, Tx HW timestamps are never guaranteed, why can't we just skip timestamping for PSP packets? Only TCP supports PSP, and upstream TCP and HW timestamps are pretty much mutually exclusive.
Yes, this is about Tx
You are right. I missed that PSP is TCP only, so it never selects ptpsq. Thanks for the clarification.
I'll drop this patch and I will revisit how to avoid a metadata collision without adding overhead on the datapath.
Thanks!
Quoted 8 lines
Quoted 4 lines
Prevent the conflict at configuration time by blocking:
- Enabling TX-port-TS when PSP has active TX keys.
- Adding PSP TX keys when TX-port-TS is already enabled.Why catch PSP at the Tx key programming? PSP docs very explicitly require drivers to _not_ enable the feature by default. And tx keys can't be added if it's disabled. So it would make much more sense to catch the enable / config callback of PSP?