net/mlx5e: Add support for HW-GRO to PSP
Subsystems (8)
Tariq Toukan
maintainercover net/mlx5e: Add support for HW-GRO to PSP
Hi,
Ingress PSP packets cannot be merged by the HW-GRO HW state machine because they are not decapsulated and the current HW-GRO state machine does not understand PSP.
This series by Cosmin decapsulates PSP packets in steering, which allows the now-decapsulated PSP packets (== TCP) to go through HW-GRO and be aggregated. The SPI and PSP version from the PSP header are handed off to the driver in the CQE metadata fields. They are used to terminate the HW GRO session on mismatch, and are required to construct the skb extension which is used higher up in the stack.
kperf tests on a pair of CX7 NICs with 200Gbps link speed:
Streams Gbps no HW-GRO Gbps HW-GRO Speedup
------- -------------- ----------- -------
1 28 58 2.07x
2 67 102 1.52x
4 136 180 1.32x
8 175 183 1.05xRegards,
Tariq
Some internal Sashiko findings, plus Cosmin's comments:
Review
Quoted 2 lines
(quoted diff) +2
-0+ cfg.wait_hw_stats_settle() + after = cfg.netnl.qstats_get({"ifindex": cfg.ifindex}, dump=True)[0]
Could this lead to flaky test failures on multiqueue NICs?
By hardcoding the netlink dump array access to index [0], the test statically verifies statistics for the first queue only. Since the test sends traffic over an ephemeral random port, Receive Side Scaling (RSS) will hash this 4-tuple and could deliver the flow to any active RX queue. If the flow lands on a queue other than queue 0, will the test fail to observe the hardware GRO packet increments?
[CR] The premise is wrong. [0] is not queue 0, it's the result for the requested dev.
Review run_session(struct ynl_sock *ys, struct opts *opts, · line 263
Quoted 12 lines
(quoted diff) +8
-0@@ -224,6 +253,28 @@ run_session(struct ynl_sock *ys, struct opts *opts, fprintf(stderr, "WARN: echo but no data sock\n"); send_ack(comm_sock); } + if (cmd("data send")) { + __u32 len; + + if (data_sock < 0) { + fprintf(stderr, "WARN: send but no data sock\n"); + send_err(comm_sock); + continue; + }
Is there a race condition between processing the control command and accepting the data socket?
If the Python client initiates the data socket connection and immediately sends the "data send" command on the control socket, both the server_sock (accept queue) and comm_sock (control connection) could become readable simultaneously before the responder calls poll().
Because the comm_sock is processed first in run_session():
tools/testing/selftests/drivers/net/psp_responder.c:run_session() {
...
/* comm sock */
if (pfds[1].revents & POLLIN) {
...
}
/* server sock */
if (pfds[0].revents & POLLIN) {
data_sock = accept(server_sock, NULL, NULL);
}
...
}does this cause the responder to handle the "data send" command before accepting the data socket, resulting in data_sock < 0 and an immediate failure? [CR] Possible in theory, not happening in practice today because "data send" is only used after a few round trips from the Python test, so it is guaranteed that the new socket is accepted by the time the command gets executed. Fixing this class of races in the responder would be nice, but outside the scope of this series.
Review macsec_fs_rx_add_rule(struct mlx5_macsec_fs *macsec_fs, · line 1761
Quoted 10 lines
(quoted diff) +1
-2@@ -1757,11 +1755,10 @@ macsec_fs_rx_add_rule(struct mlx5_macsec_fs *macsec_fs, rx_tables = &rx_fs->tables; ft_crypto = &rx_tables->ft_crypto; - /* Set bit[31 - 30] macsec marker - 0x01 */ /* Set bit[15-0] fs id */ MLX5_SET(set_action_in, action, action_type, MLX5_ACTION_TYPE_SET); MLX5_SET(set_action_in, action, field, MLX5_ACTION_IN_FIELD_METADATA_REG_B); - MLX5_SET(set_action_in, action, data, macsec_fs_set_rx_fs_id(fs_id)); + MLX5_SET(set_action_in, action, data, fs_id);
Does removing the BIT(30) marker from the MACsec fs_id metadata cause a regression by exposing RoCE MACsec to an aliasing issue?
By removing BIT(30) from the fs_id in this patch, the fs_id written to METADATA_REG_B becomes a small integer (e.g., 1, 2).
Later in mlx5_macsec_fs_add_roce_rule_rx(), the hardware rule ft_macsec_op_check enforces MACsec on RoCE traffic by matching metadata_reg_c_5 exactly against this same fs_id value.
Could any unencrypted packet that has its REG_B set to this small integer by another NIC feature (such as TC rules setting packet marks, or PSP IPsec decryption) falsely pass the MACsec check and bypass encryption enforcement? [CR] Preexisting, TC rules could set the fsid to the the previous value even before. 'PSP IPsec' decryption doesn't make sense.
Review void mlx5e_macsec_offload_handle_rx_skb(struct net_device *netdev, · line 1723
Quoted 9 lines
(quoted diff) +1
-1@@ -1717,7 +1717,7 @@ void mlx5e_macsec_offload_handle_rx_skb(struct net_device *netdev, if (!macsec) return; - fs_id = MLX5_MACSEC_RX_METADAT_HANDLE(macsec_meta_data); + fs_id = MLX5_MACSEC_RX_METADATA_HANDLE(macsec_meta_data); rcu_read_lock(); sc_xarray_element = xa_load(&macsec->sc_xarray, fs_id);
This is a pre-existing issue, but can this xa_load() race with an immediate kfree() during deletion, causing a use-after-free?
In the deletion path, mlx5e_macsec_del_rxsc() calls xa_erase() and then immediately frees the element without an RCU grace period:
mlx5e_macsec_del_rxsc() {
...
xa_erase(&macsec->sc_xarray, rx_sc->sc_xarray_element->fs_id);
dst_release(&rx_sc->md_dst->dst);
kfree(rx_sc->sc_xarray_element);
...
}Since mlx5e_macsec_offload_handle_rx_skb() retrieves sc_xarray_element locklessly under rcu_read_lock() via xa_load(), would a concurrent deletion result in dereferencing freed memory?
[CR] Preexisting.
Review
Quoted 9 lines
(quoted diff) +9
-0+static inline int mlx5e_accel_block_tc_offload(struct mlx5_core_dev *mdev) +{ + int ret = 0; + + mutex_lock(&mdev->offload_block.lock); + if (mdev->offload_block.num_accel) + ret = -EBUSY; + else + mdev->offload_block.num_tc++;
This isn't a bug, but does the new naming here swap the semantics of the variables?
In the original logic, num_block_tc counted the number of blockers on TC (which are IPsec/accel rules). By renaming it to num_tc, mlx5e_accel_block_tc_offload() now increments num_tc when an accel rule is added.
Would it be clearer to keep "block" in the names, e.g., num_block_tc, so it doesn't look like num_tc is tracking the number of TC rules? [CR] Intentional, "offload_block.num_{tc, accel}" sounds better.
Review
Quoted 9 lines
(quoted diff) +9
-0+static inline int mlx5e_accel_block_tc_offload(struct mlx5_core_dev *mdev) +{ + int ret = 0; + + mutex_lock(&mdev->offload_block.lock); + if (mdev->offload_block.num_accel) + ret = -EBUSY; + else + mdev->offload_block.num_tc++;
Are the semantic meanings of the num_tc and num_accel variables inverted? When an Accel rule is added here in mlx5e_accel_block_tc_offload(), it increments num_tc.
[CR] Same thing, it's an intentional rename.
Change-Id: Ib52fd4bfc205ea2e4da47e07a54cc0b727cc225a
Cosmin Ratiu (13):
net/mlx5e: Generalize TC <-> IPsec mutual exclusion net/mlx5e: ipsec: Block TC offload when IPsec is enabled net/mlx5e: psp: Block TC offload when PSP is enabled net/mlx5e: macsec: Block TC offload when MACsec is enabled net/mlx5e: psp: Move RX marker from ft_metadata to flow_tag net/mlx5e: ipsec: Move RX marker from ft_metadata to flow_tag net/mlx5e: macsec: Move RX marker from ft_metadata to flow_tag net/mlx5e: psp: Handle HW-decapsulated RX PSP packets net/mlx5e: psp: Add an rx_decap steering table net/mlx5e: shampo: Flush session on PSP mismatch net/mlx5e: psp: Dynamically reconfigure based on SHAMPO mode
selftests: drv-net: psp: Fix responder parsing
selftests: drv-net: psp: Add a test for PSP with HW-GRO
.../net/ethernet/mellanox/mlx5/core/en/fs.h | 1 + .../mellanox/mlx5/core/en_accel/en_accel.h | 31 ++ .../mellanox/mlx5/core/en_accel/flow_tag.h | 45 +++ .../mellanox/mlx5/core/en_accel/ipsec_fs.c | 70 ++-- .../mellanox/mlx5/core/en_accel/ipsec_rxtx.h | 9 +- .../mellanox/mlx5/core/en_accel/macsec.c | 34 +- .../mellanox/mlx5/core/en_accel/macsec.h | 5 +- .../mellanox/mlx5/core/en_accel/psp.c | 319 ++++++++++++++++-- .../mellanox/mlx5/core/en_accel/psp.h | 2 + .../mellanox/mlx5/core/en_accel/psp_rxtx.c | 21 +- .../mellanox/mlx5/core/en_accel/psp_rxtx.h | 44 ++- .../net/ethernet/mellanox/mlx5/core/en_main.c | 8 +- .../net/ethernet/mellanox/mlx5/core/en_rx.c | 32 +- .../net/ethernet/mellanox/mlx5/core/en_tc.c | 46 +-- .../net/ethernet/mellanox/mlx5/core/en_tc.h | 7 +- .../mellanox/mlx5/core/lib/macsec_fs.c | 17 +- .../mellanox/mlx5/core/lib/macsec_fs.h | 9 +- .../net/ethernet/mellanox/mlx5/core/main.c | 3 + include/linux/mlx5/driver.h | 7 +- tools/testing/selftests/drivers/net/psp.py | 129 ++++++- .../selftests/drivers/net/psp_responder.c | 87 +++-- 21 files changed, 763 insertions(+), 163 deletions(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/flow_tag.h
base-commit: 2bb54b49e9d522f54dc9c0fe10ba40fbc56041c8
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 1/13 net/mlx5e: Generalize TC <-> IPsec mutual exclusion
From: Cosmin Ratiu <[email protected]>
There is a mechanism to mutually exclude TC offload and IPsec offload from the same interface (commit [1]) due to ordering issues between SW and HW paths.
TC offload makes use of flow_tag to carry the tc mark (even when 0). Upcoming changes to accel protocols will make use of flow_tag to carry the protocol marker. As the flow_tag cannot be partially modified by a steering rule, the last rule setting the flow_tag will overwrite any previous ones.
This means that TC offload cannot be active at the same time with any of the currently implemented accel protocols (IPsec, MACsec, PSP).
Generalize the TC <-> IPsec mutual exclusion mechanism to be usable by more accel protocols:
- move the existing mlx5e_ipsec_{,un}block_tc_offload functions to en_accel.h, rename them to mlx5e_accel_{,un}block_tc_offload.
- rename the mdev counter from num_block_ipsec to num_accel.
- rename is_tc_ipsec_order_check_needed -> is_tc_accel_check_needed and make it not bail out when IPsec isn't configured.
- replace the condition use of the esw write lock as a protection for incrementing the counter with a new mutex instead. The esw might not be available, and it wasn't used correctly on the decrement path anyway, allowing races to happen. Using a dedicated mutex for these counters makes it clear and avoids races.
- add underflow warnings for the two counters to catch future miscounting bugs.
- move counters and the new lock into a dedicated struct in mlx5_core_dev named 'offload_block'. Now offload_block.num_tc means the number of tc blocks due to accel rules and ofload_block.num_accel means the number of accel blocks due to tc rules.
[1] commit c8e350e62fc5 ("net/mlx5e: Make TC and IPsec offloads mutually exclusive on a netdev")
--- .../mellanox/mlx5/core/en_accel/en_accel.h | 22 ++++++++ .../mellanox/mlx5/core/en_accel/ipsec_fs.c | 54 +++---------------- .../net/ethernet/mellanox/mlx5/core/en_tc.c | 46 +++++++++------- .../net/ethernet/mellanox/mlx5/core/main.c | 3 ++ include/linux/mlx5/driver.h | 7 ++- 5 files changed, 64 insertions(+), 68 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h +22 -0
@@ -218,6 +218,28 @@ static inline void mlx5e_accel_tx_finish(struct mlx5e_txqsq *sq,#endif } +static inline int mlx5e_accel_block_tc_offload (struct mlx5_core_dev *mdev) +{ +int ret =0 ; + +mutex_lock (&mdev->offload_block.lock); +if (mdev->offload_block.num_accel) + ret = -EBUSY ; +else + mdev->offload_block.num_tc++; +mutex_unlock (&mdev->offload_block.lock); + +return ret; +} + +static inline void mlx5e_accel_unblock_tc_offload (struct mlx5_core_dev *mdev) +{ +mutex_lock (&mdev->offload_block.lock); +if (!WARN_ON_ONCE (!mdev->offload_block.num_tc)) + mdev->offload_block.num_tc--; +mutex_unlock (&mdev->offload_block.lock); +} +static inline int mlx5e_accel_init_rx (struct mlx5e_priv *priv) {return mlx5e_ktls_init_rx (priv);
drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c +7 -47
101 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlx5/core/en_tc.c +27 -19
89 lines not on this page · raw message
drivers/net/ethernet/mellanox/mlx5/core/main.c +3 -0
@@ -1810,6 +1810,7 @@ int mlx5_mdev_init(struct mlx5_core_dev *dev, int profile_idx)lockdep_register_key (&dev->lock_key);mutex_init (&dev->intf_state_mutex);lockdep_set_class (&dev->intf_state_mutex, &dev->lock_key); +mutex_init (&dev->offload_block.lock);mutex_init (&dev->mlx5e_res.uplink_netdev_lock);mutex_init (&dev->wc_state_lock);
@@ -1898,6 +1899,7 @@ int mlx5_mdev_init(struct mlx5_core_dev *dev, int profile_idx)mutex_destroy (&priv->alloc_mutex);mutex_destroy (&priv->bfregs.wc_head.lock);mutex_destroy (&priv->bfregs.reg_head.lock); +mutex_destroy (&dev->offload_block.lock);mutex_destroy (&dev->intf_state_mutex);lockdep_unregister_key (&dev->lock_key);return err;
@@ -1925,6 +1927,7 @@ void mlx5_mdev_uninit(struct mlx5_core_dev *dev)mutex_destroy (&priv->bfregs.reg_head.lock);mutex_destroy (&dev->wc_state_lock);mutex_destroy (&dev->mlx5e_res.uplink_netdev_lock); +mutex_destroy (&dev->offload_block.lock);mutex_destroy (&dev->intf_state_mutex);lockdep_unregister_key (&dev->lock_key); }
include/linux/mlx5/driver.h +5 -2
@@ -788,8 +788,11 @@ struct mlx5_core_dev {u32 vsc_addr;struct mlx5_hv_vhca *hv_vhca;struct mlx5_hwmon *hwmon; -u64 num_block_tc; -u64 num_block_ipsec; +struct { +struct mutex lock; +u64 num_tc; +u64 num_accel; + } offload_block;#ifdef CONFIG_MLX5_MACSEC struct mlx5_macsec_fs *macsec_fs;/* MACsec notifier chain to sync MACsec core and IB database */
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 2/13 net/mlx5e: ipsec: Block TC offload when IPsec is enabled
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
Currently the mutual exclusion mechanism is only used in packet offload mode. But with the upcoming changes to flow_tag, all RX IPsec flows need to block TC offload, so do that with the help of a small helper.
--- .../ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c +10 -3
@@ -2575,11 +2575,18 @@ void mlx5e_accel_ipsec_fs_read_stats(struct mlx5e_priv *priv, void *ipsec_stats) } } +static bool accel_ipsec_should_block_tc (struct mlx5e_ipsec_sa_entry *sa_entry) +{ +return sa_entry->attrs.type ==XFRM_DEV_OFFLOAD_PACKET || + sa_entry->attrs.dir ==XFRM_DEV_OFFLOAD_IN ; +} +int mlx5e_accel_ipsec_fs_add_rule (struct mlx5e_ipsec_sa_entry *sa_entry) { +bool block_tc =accel_ipsec_should_block_tc (sa_entry);int err; -if (sa_entry->attrs.type ==XFRM_DEV_OFFLOAD_PACKET ) { +if (block_tc) { err =mlx5e_accel_block_tc_offload (sa_entry->ipsec->mdev);if (err)return err;
@@ -2596,7 +2603,7 @@ int mlx5e_accel_ipsec_fs_add_rule(struct mlx5e_ipsec_sa_entry *sa_entry)return 0 ; err_out: -if (sa_entry->attrs.type ==XFRM_DEV_OFFLOAD_PACKET ) +if (block_tc)mlx5e_accel_unblock_tc_offload (sa_entry->ipsec->mdev);return err; }
@@ -2611,7 +2618,7 @@ void mlx5e_accel_ipsec_fs_del_rule(struct mlx5e_ipsec_sa_entry *sa_entry)if (ipsec_rule->pkt_reformat)mlx5_packet_reformat_dealloc (mdev, ipsec_rule->pkt_reformat); -if (sa_entry->attrs.type ==XFRM_DEV_OFFLOAD_PACKET ) +if (accel_ipsec_should_block_tc (sa_entry))mlx5e_accel_unblock_tc_offload (mdev);if (sa_entry->attrs.dir ==XFRM_DEV_OFFLOAD_OUT ) {
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 3/13 net/mlx5e: psp: Block TC offload when PSP is enabled
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
Make use of the mutual exclusion mechanism from the previous patch to make sure only PSP or TC offload is active on a device. Later in the series, the PSP protocol marker will move to the flow_tag, which is also used by TC offload.
--- .../net/ethernet/mellanox/mlx5/core/en_accel/psp.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +13 -1
@@ -6,6 +6,7 @@#include "mlx5_core.h" #include "psp.h" #include "lib/crypto.h" +#include "en_accel/en_accel.h" #include "en_accel/psp.h" #include "fs_core.h"
@@ -526,6 +527,7 @@ accel_psp_fs_rx_decrypt_ft_create(struct mlx5e_psp_fs *fs,static void accel_psp_fs_rx_destroy (struct mlx5e_psp_fs *fs) {struct mlx5_ttc_table *ttc =mlx5e_fs_get_ttc (fs->fs, false); +bool tc_blocked = fs->rx.ft;int i;/* disconnect */
@@ -535,6 +537,8 @@ static void accel_psp_fs_rx_destroy(struct mlx5e_psp_fs *fs) }accel_psp_fs_rx_check_ft_destroy (&fs->check);accel_psp_fs_rx_ft_destroy (&fs->rx); +if (tc_blocked) +mlx5e_accel_unblock_tc_offload (fs->mdev); } static int accel_psp_fs_rx_create(struct mlx5e_psp_fs *fs,
@@ -543,10 +547,16 @@ static int accel_psp_fs_rx_create(struct mlx5e_psp_fs *fs,struct mlx5_ttc_table *ttc =mlx5e_fs_get_ttc (fs->fs, false);int i, err; + err =mlx5e_accel_block_tc_offload (fs->mdev); +if (err) { +NL_SET_ERR_MSG (extack,"TC offload active, cannot enable PSP" ); +return err; + } + err =accel_psp_fs_rx_ft_create (fs, &fs->rx);if (err) {NL_SET_ERR_MSG (extack,"Failed creating RX steering table" ); -return err; + goto err_unblock_tc; } err =accel_psp_fs_rx_check_ft_create (fs, &fs->check);
@@ -583,6 +593,8 @@ static int accel_psp_fs_rx_create(struct mlx5e_psp_fs *fs,accel_psp_fs_rx_check_ft_destroy (&fs->check); err_ft:accel_psp_fs_rx_ft_destroy (&fs->rx); +err_unblock_tc: +mlx5e_accel_unblock_tc_offload (fs->mdev);return err; }
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 4/13 net/mlx5e: macsec: Block TC offload when MACsec is enabled
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
The MACsec protocol marker will soon move to flow_tag and that will make TC offload unusable at the same time as MACsec on the same device.
This patch makes use of the mutual exclusion mechanism to make sure that TC and MACsec cannot be both active at the same time.
One extra bit of logic is in macsec_upd_secy_hw_address(), where existing macsec rules are drained then readded. During the two loops it's possible a mistimed TC filter add to throw a wrench into things and prevent the 2nd loop from adding anything, since accel rules are now blocked. Fix that by keeping a best-effort TC block across the entire operation.
--- .../mellanox/mlx5/core/en_accel/macsec.c | 28 +++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c +26 -2
@@ -10,6 +10,7 @@#include "en.h" #include "lib/aso.h" #include "lib/crypto.h" +#include "en_accel/en_accel.h" #include "en_accel/macsec.h" #define MLX5_MACSEC_EPN_SCOPE_MID 0x80000000L
@@ -324,6 +325,8 @@ static void mlx5e_macsec_cleanup_sa_fs(struct mlx5e_macsec *macsec,mlx5_macsec_fs_del_rule (macsec->mdev->macsec_fs, sa->macsec_rule, action, netdev, fs_id); sa->macsec_rule =NULL ; +if (!is_tx) +mlx5e_accel_unblock_tc_offload (macsec->mdev); } static void mlx5e_macsec_cleanup_sa(struct mlx5e_macsec *macsec,
@@ -343,6 +346,7 @@ static int mlx5e_macsec_init_sa_fs(struct macsec_context *ctx,const struct macsec_tx_sc *tx_sc = &ctx->secy->tx_sc;struct mlx5_macsec_rule_attrs rule_attrs;union mlx5_macsec_rule *macsec_rule; +int err =0 ;if (is_tx && tx_sc->encoding_sa != sa->assoc_num)return 0 ;
@@ -353,13 +357,26 @@ static int mlx5e_macsec_init_sa_fs(struct macsec_context *ctx, rule_attrs.action = (is_tx) ?MLX5_ACCEL_MACSEC_ACTION_ENCRYPT :MLX5_ACCEL_MACSEC_ACTION_DECRYPT ; +if (!is_tx) { + err =mlx5e_accel_block_tc_offload (priv->mdev); +if (err) +return err; + } + macsec_rule =mlx5_macsec_fs_add_rule (macsec_fs, ctx, &rule_attrs, fs_id); -if (!macsec_rule) -return -ENOMEM ; +if (!macsec_rule) { + err = -ENOMEM ; + goto out_unblock_tc; + } sa->macsec_rule = macsec_rule;return 0 ; + +out_unblock_tc: +if (!is_tx) +mlx5e_accel_unblock_tc_offload (priv->mdev); +return err; } static int mlx5e_macsec_init_sa(struct macsec_context *ctx,
@@ -1137,7 +1154,12 @@ static int macsec_upd_secy_hw_address(struct macsec_context *ctx,struct mlx5e_macsec_sa *rx_sa;struct list_head *list;int i, err =0 ; +bool block_tc; +/* Best-effort TC block across the operation, to prevent a mistimed TC +* filter add from preventing the 2nd loop from happening. +*/ + block_tc =mlx5e_accel_block_tc_offload (priv->mdev) ==0 ; list = &macsec_device->macsec_rx_sc_list_head;list_for_each_entry_safe (rx_sc, tmp, list, rx_sc_list_element) {
@@ -1168,6 +1190,8 @@ static int macsec_upd_secy_hw_address(struct macsec_context *ctx,memcpy (macsec_device->dev_addr, dev->dev_addr, dev->addr_len); out: +if (block_tc) +mlx5e_accel_unblock_tc_offload (priv->mdev);return err; }
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 5/13 net/mlx5e: psp: Move RX marker from ft_metadata to flow_tag
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
Move PSP RX marker from ft_metadata (set via modify_hdr action) to flow_tag (set via flow_context). This frees ft_metadata for storing SPI for future decapsulated PSP packets.
All mlx5e accel protos have to move to avoid misinterpreting packets with high bits of ft_metadata as IPsec or Macsec. This patch is the first step, defining a new header, bit layout and macros.
--- .../mellanox/mlx5/core/en_accel/flow_tag.h | 35 +++++++++++++++++++ .../mellanox/mlx5/core/en_accel/psp.c | 34 ++++-------------- .../mellanox/mlx5/core/en_accel/psp_rxtx.h | 10 +++--- 3 files changed, 47 insertions(+), 32 deletions(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/flow_tag.h
drivers/net/ethernet/mellanox/mlx5/core/en_accel/flow_tag.h new file +35 -0
@@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5E_FLOW_TAG_H__ +#define __MLX5E_FLOW_TAG_H__ + +#include <linux/bits.h> +#include <linux/kernel.h> +#include <linux/types.h> +#include <linux/mlx5/device.h> + +/* Unified accel flow_tag layout in CQE sop_drop_qpn [23:0]: +* +* [23:21] = protocol ID (3 bits): +* 0 = none (default) +* 3 = PSP (HW decrypted, PSP header present) +* 1,2,4-7 = reserved +* [20:16] = reserved +* [15:0] = used by other subsystems (e.g. TC). +*/ +#define MLX5E_ACCEL_FLOW_TAG_PROTO_MASK GENMASK(23, 21) +#define MLX5E_ACCEL_FLOW_TAG_PROTO_NONE (0 << 21) +#define MLX5E_ACCEL_FLOW_TAG_PROTO_PSP (3 << 21) + +static inline u32 mlx5e_accel_flow_tag (struct mlx5_cqe64 *cqe) +{ +return be32_to_cpu (cqe->sop_drop_qpn) &0xFFFFFF ; +} + +static inline u32 mlx5e_accel_flow_tag_proto (struct mlx5_cqe64 *cqe) +{ +return mlx5e_accel_flow_tag (cqe) &MLX5E_ACCEL_FLOW_TAG_PROTO_MASK ; +} + +#endif /* __MLX5E_FLOW_TAG_H__ */
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +7 -27
@@ -8,6 +8,7 @@#include "lib/crypto.h" #include "en_accel/en_accel.h" #include "en_accel/psp.h" +#include "en_accel/psp_rxtx.h" #include "fs_core.h" enum accel_fs_psp_type {
@@ -42,7 +43,6 @@ struct mlx5e_psp_rx_decrypt_table {struct mlx5_flow_table *ft;struct mlx5_flow_group *miss_group;struct mlx5_flow_handle *miss_rule; -struct mlx5_modify_hdr *rx_modify_hdr;struct mlx5_flow_handle *rule; };
@@ -408,10 +408,6 @@ accel_psp_fs_rx_decrypt_ft_destroy(struct mlx5e_psp_fs *fs, struct mlx5e_psp_rx_decrypt_table *decrypt) {accel_psp_fs_del_flow_rule (&decrypt->rule); -if (decrypt->rx_modify_hdr) { -mlx5_modify_header_dealloc (fs->mdev, decrypt->rx_modify_hdr); - decrypt->rx_modify_hdr =NULL ; - }accel_psp_fs_del_flow_rule (&decrypt->miss_rule);accel_psp_fs_destroy_flow_group (&decrypt->miss_group);accel_psp_fs_destroy_ft (&decrypt->ft);
@@ -431,8 +427,6 @@ accel_psp_fs_rx_decrypt_ft_create(struct mlx5e_psp_fs *fs,struct mlx5e_psp_rx_decrypt_table *decrypt, struct mlx5_flow_destination *default_dest) { - u8 action[MLX5_UN_SZ_BYTES(set_add_copy_action_in_auto)] = {}; -struct mlx5_modify_hdr *modify_hdr =NULL ; struct mlx5_flow_table_attr ft_attr = {};struct mlx5_flow_destination dest = {};struct mlx5_core_dev *mdev = fs->mdev;
@@ -482,28 +476,14 @@ accel_psp_fs_rx_decrypt_ft_create(struct mlx5e_psp_fs *fs,/* Add PSP RX decrypt rule */ setup_fte_udp_psp (spec,PSP_DEFAULT_UDP_PORT ); - flow_act.crypto.type =MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_PSP ; -/* Set bit[31, 30] PSP marker */ -#define MLX5E_PSP_MARKER_BIT (BIT(30) | BIT(31)) -MLX5_SET (set_action_in, action, action_type,MLX5_ACTION_TYPE_SET ); -MLX5_SET (set_action_in, action, field,MLX5_ACTION_IN_FIELD_METADATA_REG_B ); -MLX5_SET (set_action_in, action, data,MLX5E_PSP_MARKER_BIT ); -MLX5_SET (set_action_in, action, offset,0 ); -MLX5_SET (set_action_in, action, length,32 ); - - modify_hdr =mlx5_modify_header_alloc (mdev,MLX5_FLOW_NAMESPACE_KERNEL ,1 , action); -if (IS_ERR (modify_hdr)) { - err =PTR_ERR (modify_hdr); -mlx5_core_err (mdev,"fail to alloc psp set modify_header_id err=%d\n" , err); - modify_hdr =NULL ; - goto out_err; - } - decrypt->rx_modify_hdr = modify_hdr; +/* Set PSP marker via flow_tag */ + spec->flow_context.flags =FLOW_CONTEXT_HAS_TAG ; + spec->flow_context.flow_tag =MLX5E_ACCEL_FLOW_TAG_PROTO_PSP ; + + flow_act.crypto.type =MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_PSP ; flow_act.action =MLX5_FLOW_CONTEXT_ACTION_FWD_DEST | -MLX5_FLOW_CONTEXT_ACTION_CRYPTO_DECRYPT | -MLX5_FLOW_CONTEXT_ACTION_MOD_HDR ; - flow_act.modify_hdr = modify_hdr; +MLX5_FLOW_CONTEXT_ACTION_CRYPTO_DECRYPT ; dest.type =MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE ; dest.ft = fs->check.ft; rule =mlx5_add_flow_rules (decrypt->ft, spec, &flow_act, &dest,1 );
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp_rxtx.h +5 -5
@@ -9,10 +9,7 @@#include <net/psp.h> #include "en.h" #include "en/txrx.h" - -/* Bit30: PSP marker, Bit22-0: PSP obj id */ -#define MLX5_PSP_METADATA_MARKER (metadata) ((((metadata) >> 30) & 0x3) == 0x3) -#define MLX5_PSP_METADATA_HANDLE (metadata) ((metadata) & GENMASK(22, 0)) +#include "en_accel/flow_tag.h" struct mlx5e_accel_tx_psp_state {u32 tailen;
@@ -82,7 +79,10 @@ static inline unsigned int mlx5e_psp_tx_ids_len(struct mlx5e_accel_tx_psp_statestatic inline bool mlx5e_psp_is_rx_flow (struct mlx5_cqe64 *cqe) { -return MLX5_PSP_METADATA_MARKER (be32_to_cpu (cqe->ft_metadata)); +u32 proto =mlx5e_accel_flow_tag_proto (cqe); + +return proto ==MLX5E_ACCEL_FLOW_TAG_PROTO_PSP ; + } bool mlx5e_psp_offload_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 6/13 net/mlx5e: ipsec: Move RX marker from ft_metadata to flow_tag
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
IPsec used BIT(31) in cqe.ft_metadata as a marker for ingress IPsec-decrypted packets. This will conflict with PSP which stores SPI in ft_metadata - an SPI with bit 31 set would falsely trigger mlx5_ipsec_is_rx_flow().
Define a new marker for IPsec in flow_tag and use it in steering rules. The obj_id stays in ft_metadata bits[23:0].
Remove the unused MARKER and SYNDROM (sic) macros.
This is only done for IPsec NIC RX flows. The esw path uses a different mechanism and is unaffected: RX handling in mlx5e_rep_tc_receive() makes use of a few bits in ft_metadata to detect and hand off IPsec packets to mlx5e_ipsec_offload_handle_rx_skb().
An additional complication is that mlx5e_cqe_regb_chain() used the high order bits of ft_metadata to differentiate between TC chains and protocol markers. With the IPSec marker moving to flow_tag, this would falsely trigger mlx5e_cqe_regb_chain() to believe the chain is set, when in fact, ft_metadata is something completely different. Solve that by requiring that the new flow_tag proto is NONE, since only those packets can carry a chain id in ft_metadata.
--- .../net/ethernet/mellanox/mlx5/core/en_accel/flow_tag.h | 4 +++- .../net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c | 9 +++++++-- .../ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h | 9 +++++---- drivers/net/ethernet/mellanox/mlx5/core/en_tc.h | 7 +++++-- 4 files changed, 20 insertions(+), 9 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/flow_tag.h +3 -1
@@ -13,13 +13,15 @@* * [23:21] = protocol ID (3 bits): * 0 = none (default) +* 1 = IPsec * 3 = PSP (HW decrypted, PSP header present) -* 1,2,4-7 = reserved +* 2,4-7 = reserved * [20:16] = reserved * [15:0] = used by other subsystems (e.g. TC). */ #define MLX5E_ACCEL_FLOW_TAG_PROTO_MASK GENMASK(23, 21)#define MLX5E_ACCEL_FLOW_TAG_PROTO_NONE (0 << 21) +#define MLX5E_ACCEL_FLOW_TAG_PROTO_IPSEC (1 << 21)#define MLX5E_ACCEL_FLOW_TAG_PROTO_PSP (3 << 21)static inline u32 mlx5e_accel_flow_tag (struct mlx5_cqe64 *cqe)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c +7 -2
@@ -5,6 +5,7 @@#include "en.h" #include "en/fs.h" #include "en_accel/en_accel.h" +#include "en_accel/flow_tag.h" #include "eswitch.h" #include "ipsec.h" #include "fs_core.h"
@@ -190,7 +191,7 @@ static void ipsec_rx_rule_add_match_obj(struct mlx5e_ipsec_sa_entry *sa_entry, misc_parameters_2.metadata_reg_c_2);MLX5_SET (fte_match_param, spec->match_value, misc_parameters_2.metadata_reg_c_2, - sa_entry->ipsec_obj_id |BIT (31 )); + sa_entry->ipsec_obj_id); spec->match_criteria_enable |=MLX5_MATCH_MISC_PARAMETERS_2 ; }
@@ -2066,7 +2067,7 @@ static int rx_add_rule(struct mlx5e_ipsec_sa_entry *sa_entry)if (!attrs->drop) {if (rx != ipsec->rx_esw) err =setup_modify_header (ipsec, attrs->type, - sa_entry->ipsec_obj_id |BIT (31 ), + sa_entry->ipsec_obj_id,XFRM_DEV_OFFLOAD_IN , &flow_act);else err =mlx5_esw_ipsec_rx_setup_modify_header (sa_entry, &flow_act);
@@ -2099,6 +2100,10 @@ static int rx_add_rule(struct mlx5e_ipsec_sa_entry *sa_entry) flow_act.action |=MLX5_FLOW_CONTEXT_ACTION_DROP ; else flow_act.action |= MLX5_FLOW_CONTEXT_ACTION_FWD_DEST; +if (!attrs->drop && rx != ipsec->rx_esw) { + spec->flow_context.flags |=FLOW_CONTEXT_HAS_TAG ; + spec->flow_context.flow_tag =MLX5E_ACCEL_FLOW_TAG_PROTO_IPSEC ; + } dest[0 ].type =MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE ; dest[0 ].ft = rx->ft.status; dest[1 ].type =MLX5_FLOW_DESTINATION_TYPE_COUNTER ;
drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h +5 -4
@@ -38,10 +38,9 @@#include <net/xfrm.h> #include "en.h" #include "en/txrx.h" +#include "en_accel/flow_tag.h" -/* Bit31: IPsec marker, Bit30: reserved, Bit29-24: IPsec syndrome, Bit23-0: IPsec obj id */ -#define MLX5_IPSEC_METADATA_MARKER (metadata) ((((metadata) >> 30) & 0x3) == 0x2) -#define MLX5_IPSEC_METADATA_SYNDROM (metadata) (((metadata) >> 24) & GENMASK(5, 0)) +/* IPsec obj id in ft_metadata bits[23:0] */ #define MLX5_IPSEC_METADATA_HANDLE (metadata) ((metadata) & GENMASK(23, 0))struct mlx5e_accel_tx_ipsec_state {
@@ -74,7 +73,9 @@ static inline unsigned int mlx5e_ipsec_tx_ids_len(struct mlx5e_accel_tx_ipsec_ststatic inline bool mlx5_ipsec_is_rx_flow (struct mlx5_cqe64 *cqe) { -return MLX5_IPSEC_METADATA_MARKER (be32_to_cpu (cqe->ft_metadata)); +u32 proto =mlx5e_accel_flow_tag_proto (cqe); + +return proto ==MLX5E_ACCEL_FLOW_TAG_PROTO_IPSEC ; }static inline bool mlx5e_ipsec_eseg_meta (struct mlx5_wqe_eth_seg *eseg)
drivers/net/ethernet/mellanox/mlx5/core/en_tc.h +5 -2
@@ -36,6 +36,7 @@#include <net/pkt_cls.h> #include "en.h" #include "eswitch.h" +#include "en_accel/flow_tag.h" #include "en/tc_ct.h" #include "en/tc_tun.h" #include "en/tc/int_port.h"
@@ -370,11 +371,13 @@ struct mlx5e_tc_table *mlx5e_tc_table_alloc(void);void mlx5e_tc_table_free (struct mlx5e_tc_table *tc);static inline bool mlx5e_cqe_regb_chain (struct mlx5_cqe64 *cqe) { -u32 chain, reg_b; +u32 flow_tag, reg_b, chain; + flow_tag =mlx5e_accel_flow_tag_proto (cqe); reg_b =be32_to_cpu (cqe->ft_metadata); -if (reg_b >> (MLX5E_TC_TABLE_CHAIN_TAG_BITS +ESW_ZONE_ID_BITS )) +if (flow_tag !=MLX5E_ACCEL_FLOW_TAG_PROTO_NONE || + (reg_b >> (MLX5E_TC_TABLE_CHAIN_TAG_BITS +ESW_ZONE_ID_BITS )))return false; chain = reg_b &MLX5E_TC_TABLE_CHAIN_TAG_MASK ;
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 7/13 net/mlx5e: macsec: Move RX marker from ft_metadata to flow_tag
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
MACsec uses BIT(30) in cqe.ft_metadata as a marker to identify MACsec-decrypted packets. This will conflict with PSP SPI values with bit 30 set, which would falsely trigger mlx5e_macsec_is_rx_flow().
Define a new marker for MACsec in flow_tag and use it in steering rules. The fs_id stays in ft_metadata bits[15:0].
Correct typos of MLX5_MACSEC_RX_METADAT_HANDLE and MLX5_MACEC_RX_FS_ID_MAX while touching this.
Delete the METADATA_MARKER macro.
Set flow_tag on both with-SCI and without-SCI crypto rules.
--- .../mellanox/mlx5/core/en_accel/flow_tag.h | 4 +++- .../mellanox/mlx5/core/en_accel/macsec.c | 6 +++--- .../mellanox/mlx5/core/en_accel/macsec.h | 5 ++++- .../ethernet/mellanox/mlx5/core/lib/macsec_fs.c | 17 ++++++++++------- .../ethernet/mellanox/mlx5/core/lib/macsec_fs.h | 9 ++++----- 5 files changed, 24 insertions(+), 17 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/flow_tag.h +3 -1
@@ -14,14 +14,16 @@* [23:21] = protocol ID (3 bits): * 0 = none (default) * 1 = IPsec +* 2 = MACsec * 3 = PSP (HW decrypted, PSP header present) -* 2,4-7 = reserved +* 4-7 = reserved * [20:16] = reserved * [15:0] = used by other subsystems (e.g. TC). */ #define MLX5E_ACCEL_FLOW_TAG_PROTO_MASK GENMASK(23, 21)#define MLX5E_ACCEL_FLOW_TAG_PROTO_NONE (0 << 21)#define MLX5E_ACCEL_FLOW_TAG_PROTO_IPSEC (1 << 21) +#define MLX5E_ACCEL_FLOW_TAG_PROTO_MACSEC (2 << 21)#define MLX5E_ACCEL_FLOW_TAG_PROTO_PSP (3 << 21)static inline u32 mlx5e_accel_flow_tag (struct mlx5_cqe64 *cqe)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c +3 -3
@@ -752,12 +752,12 @@ static int mlx5e_macsec_add_rxsc(struct macsec_context *ctx)* a non-NULL md_dst with uninitialised contents. */ err =xa_alloc (&macsec->sc_xarray, &sc_xarray_element->fs_id, sc_xarray_element, -XA_LIMIT (1 ,MLX5_MACEC_RX_FS_ID_MAX ),GFP_KERNEL ); +XA_LIMIT (1 ,MLX5_MACSEC_RX_FS_ID_MAX ),GFP_KERNEL );if (err) {if (err == -EBUSY )netdev_err (ctx->netdev,"MACsec offload: unable to create entry for RX SC (%d Rx SCs already allocated)\n" , -MLX5_MACEC_RX_FS_ID_MAX ); +MLX5_MACSEC_RX_FS_ID_MAX ); goto destroy_md_dst; }
@@ -1724,7 +1724,7 @@ void mlx5e_macsec_offload_handle_rx_skb(struct net_device *netdev,if (!macsec)return ; - fs_id =MLX5_MACSEC_RX_METADAT_HANDLE (macsec_meta_data); + fs_id =MLX5_MACSEC_RX_METADATA_HANDLE (macsec_meta_data);rcu_read_lock (); sc_xarray_element =xa_load (&macsec->sc_xarray, fs_id);
drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.h +4 -1
@@ -9,6 +9,7 @@#include <linux/mlx5/driver.h> #include <net/macsec.h> #include <net/dst_metadata.h> +#include "en_accel/flow_tag.h" #include "lib/macsec_fs.h" struct mlx5e_priv ;
@@ -31,7 +32,9 @@ static inline bool mlx5e_macsec_skb_is_offload(struct sk_buff *skb)static inline bool mlx5e_macsec_is_rx_flow (struct mlx5_cqe64 *cqe) { -return MLX5_MACSEC_METADATA_MARKER (be32_to_cpu (cqe->ft_metadata)); +u32 proto =mlx5e_accel_flow_tag_proto (cqe); + +return proto ==MLX5E_ACCEL_FLOW_TAG_PROTO_MACSEC ; } void mlx5e_macsec_offload_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb,
drivers/net/ethernet/mellanox/mlx5/core/lib/macsec_fs.c +10 -7
@@ -8,6 +8,7 @@#include <linux/mlx5/macsec.h> #include "fs_core.h" #include "lib/macsec_fs.h" +#include "en_accel/flow_tag.h" #include "mlx5_core.h" /* MACsec TX flow steering */
@@ -45,9 +46,6 @@#define MLX5_SECTAG_HEADER_SIZE_WITHOUT_SCI 0x8#define MLX5_SECTAG_HEADER_SIZE_WITH_SCI (MLX5_SECTAG_HEADER_SIZE_WITHOUT_SCI + MACSEC_SCI_LEN) -/* MACsec fs_id handling for steering */ -#define macsec_fs_set_rx_fs_id (fs_id) ((fs_id) | BIT(30)) -struct mlx5_sectag_header {__be16 ethertype;u8 tci_an;
@@ -1757,11 +1755,10 @@ macsec_fs_rx_add_rule(struct mlx5_macsec_fs *macsec_fs, rx_tables = &rx_fs->tables; ft_crypto = &rx_tables->ft_crypto; -/* Set bit[31 - 30] macsec marker - 0x01 */ /* Set bit[15-0] fs id */ MLX5_SET (set_action_in, action, action_type,MLX5_ACTION_TYPE_SET );MLX5_SET (set_action_in, action, field,MLX5_ACTION_IN_FIELD_METADATA_REG_B ); -MLX5_SET (set_action_in, action, data,macsec_fs_set_rx_fs_id (fs_id)); +MLX5_SET (set_action_in, action, data, fs_id);MLX5_SET (set_action_in, action, offset,0 );MLX5_SET (set_action_in, action, length,32 );
@@ -1778,6 +1775,9 @@ macsec_fs_rx_add_rule(struct mlx5_macsec_fs *macsec_fs,/* Rx crypto table with SCI rule */ macsec_fs_rx_setup_fte (spec, &flow_act, attrs, true); + spec->flow_context.flags |=FLOW_CONTEXT_HAS_TAG ; + spec->flow_context.flow_tag =MLX5E_ACCEL_FLOW_TAG_PROTO_MACSEC ; + flow_act.modify_hdr = modify_hdr; flow_act.action =MLX5_FLOW_CONTEXT_ACTION_FWD_DEST | MLX5_FLOW_CONTEXT_ACTION_CRYPTO_DECRYPT |
@@ -1803,6 +1803,9 @@ macsec_fs_rx_add_rule(struct mlx5_macsec_fs *macsec_fs,macsec_fs_rx_setup_fte (spec, &flow_act, attrs, false); + spec->flow_context.flags |=FLOW_CONTEXT_HAS_TAG ; + spec->flow_context.flow_tag =MLX5E_ACCEL_FLOW_TAG_PROTO_MACSEC ; + flow_act.modify_hdr = modify_hdr; flow_act.action =MLX5_FLOW_CONTEXT_ACTION_FWD_DEST | MLX5_FLOW_CONTEXT_ACTION_CRYPTO_DECRYPT |
@@ -2160,8 +2163,8 @@ static int mlx5_macsec_fs_add_roce_rule_rx(struct mlx5_macsec_fs *macsec_fs, u32 spec->match_criteria_enable |=MLX5_MATCH_MISC_PARAMETERS_2 ;MLX5_SET_TO_ONES (fte_match_param, spec->match_criteria, misc_parameters_2.metadata_reg_c_5); -MLX5_SET (fte_match_param, spec->match_value, misc_parameters_2.metadata_reg_c_5, -macsec_fs_set_rx_fs_id (fs_id)); +MLX5_SET (fte_match_param, spec->match_value, + misc_parameters_2.metadata_reg_c_5, fs_id); flow_act.action =MLX5_FLOW_CONTEXT_ACTION_ALLOW ; new_rule =mlx5_add_flow_rules (rx_fs->roce.ft_macsec_op_check, spec, &flow_act,NULL ,0 );
drivers/net/ethernet/mellanox/mlx5/core/lib/macsec_fs.h +4 -5
@@ -6,11 +6,10 @@#ifdef CONFIG_MLX5_MACSEC -/* Bit31 - 30: MACsec marker, Bit15-0: MACsec id */ -#define MLX5_MACEC_RX_FS_ID_MAX USHRT_MAX/* Must be power of two */ -#define MLX5_MACSEC_RX_FS_ID_MASK MLX5_MACEC_RX_FS_ID_MAX -#define MLX5_MACSEC_METADATA_MARKER (metadata) ((((metadata) >> 30) & 0x3) == 0x1) -#define MLX5_MACSEC_RX_METADAT_HANDLE (metadata) ((metadata) & MLX5_MACSEC_RX_FS_ID_MASK) +/* MACsec fs_id in ft_metadata bits[15:0] */ +#define MLX5_MACSEC_RX_FS_ID_MAX USHRT_MAX/* Must be power of two */ +#define MLX5_MACSEC_RX_FS_ID_MASK MLX5_MACSEC_RX_FS_ID_MAX +#define MLX5_MACSEC_RX_METADATA_HANDLE (metadata) ((metadata) & MLX5_MACSEC_RX_FS_ID_MASK)/* MACsec TX flow steering */ #define MLX5_ETH_WQE_FT_META_MACSEC_MASK \
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 8/13 net/mlx5e: psp: Handle HW-decapsulated RX PSP packets
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
mlx5e_psp_offload_handle_rx_skb() handles RX PSP packets.
Add an additional flow tag marker and helpers for decapsulated PSP packets and extend the handler to construct the PSP skb extension from the CQE fields (SPI in ft_metadata and PSP version in flow_tag).
--- .../mellanox/mlx5/core/en_accel/flow_tag.h | 10 ++++-- .../mellanox/mlx5/core/en_accel/psp_rxtx.c | 21 +++++++++-- .../mellanox/mlx5/core/en_accel/psp_rxtx.h | 36 ++++++++++++++++++- 3 files changed, 61 insertions(+), 6 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/flow_tag.h +8 -2
@@ -16,8 +16,10 @@* 1 = IPsec * 2 = MACsec * 3 = PSP (HW decrypted, PSP header present) -* 4-7 = reserved -* [20:16] = reserved +* 4 = PSP decap (HW decrypted & decapsulated) +* 5-7 = reserved +* [20:17] = PSP version (4 bits, valid when proto == PSP decap) +* 16 = reserved * [15:0] = used by other subsystems (e.g. TC). */ #define MLX5E_ACCEL_FLOW_TAG_PROTO_MASK GENMASK(23, 21)
@@ -25,6 +27,10 @@#define MLX5E_ACCEL_FLOW_TAG_PROTO_IPSEC (1 << 21)#define MLX5E_ACCEL_FLOW_TAG_PROTO_MACSEC (2 << 21)#define MLX5E_ACCEL_FLOW_TAG_PROTO_PSP (3 << 21) +#define MLX5E_ACCEL_FLOW_TAG_PROTO_PSP_DECAP (4 << 21) + +#define MLX5E_ACCEL_FLOW_TAG_PSP_VER_SHIFT 17 +#define MLX5E_ACCEL_FLOW_TAG_PSP_VER_MASK (0xF << MLX5E_ACCEL_FLOW_TAG_PSP_VER_SHIFT)static inline u32 mlx5e_accel_flow_tag (struct mlx5_cqe64 *cqe) {
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp_rxtx.c +18 -3
@@ -118,10 +118,25 @@ bool mlx5e_psp_offload_handle_rx_skb(struct net_device *netdev, struct sk_buff * {struct mlx5e_priv *priv =netdev_priv (netdev);u16 dev_id = priv->psp->psd->id; -bool strip_icv = true; -u8 generation =0 ; +struct psp_skb_ext *pse; + +if (mlx5e_psp_is_decap (cqe)) { +/* UDP + PSP headers and PSP trailer removed by HW. +* Construct the PSP extension from CQE metadata. +*/ + pse =skb_ext_add (skb,SKB_EXT_PSP ); +if (unlikely (!pse)) + goto drop; + + pse->spi =mlx5e_psp_get_spi (cqe); + pse->version =mlx5e_psp_get_version (cqe); + pse->dev_id = dev_id; + pse->generation =0 ; + skb->decrypted =1 ; +return false; + } -if (psp_dev_rcv (skb, dev_id, generation, strip_icv)) +if (psp_dev_rcv (skb, dev_id,0 , true)) goto drop; skb->decrypted =1 ;
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp_rxtx.h +35 -1
@@ -81,10 +81,29 @@ static inline bool mlx5e_psp_is_rx_flow(struct mlx5_cqe64 *cqe) {u32 proto =mlx5e_accel_flow_tag_proto (cqe); -return proto ==MLX5E_ACCEL_FLOW_TAG_PROTO_PSP ; +return proto ==MLX5E_ACCEL_FLOW_TAG_PROTO_PSP || + proto ==MLX5E_ACCEL_FLOW_TAG_PROTO_PSP_DECAP ; } +static inline bool mlx5e_psp_is_decap (struct mlx5_cqe64 *cqe) +{ +u32 proto =mlx5e_accel_flow_tag_proto (cqe); + +return proto ==MLX5E_ACCEL_FLOW_TAG_PROTO_PSP_DECAP ; +} + +static inline u8 mlx5e_psp_get_version (struct mlx5_cqe64 *cqe) +{ +return (mlx5e_accel_flow_tag (cqe) &MLX5E_ACCEL_FLOW_TAG_PSP_VER_MASK ) >> +MLX5E_ACCEL_FLOW_TAG_PSP_VER_SHIFT ; +} + +static inline __be32 mlx5e_psp_get_spi (struct mlx5_cqe64 *cqe) +{ +return cqe->ft_metadata; +} +bool mlx5e_psp_offload_handle_rx_skb (struct net_device *netdev,struct sk_buff *skb,struct mlx5_cqe64 *cqe);#else
@@ -110,6 +129,21 @@ static inline bool mlx5e_psp_is_rx_flow(struct mlx5_cqe64 *cqe)return false; } +static inline bool mlx5e_psp_is_decap (struct mlx5_cqe64 *cqe) +{ +return false; +} + +static inline u8 mlx5e_psp_get_version (struct mlx5_cqe64 *cqe) +{ +return 0 ; +} + +static inline __be32 mlx5e_psp_get_spi (struct mlx5_cqe64 *cqe) +{ +return 0 ; +} +static inline bool mlx5e_psp_offload_handle_rx_skb (struct net_device *netdev,struct sk_buff *skb,struct mlx5_cqe64 *cqe)
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 9/13 net/mlx5e: psp: Add an rx_decap steering table
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
Introduce an additional steering table for PSP transport mode decapsulation, containing:
- one rule per supported PSP version which does:
- transport mode decap (removes UDP+PSP headers and PSP trailer)
- recomputes iph->tot_len
- recomputes IP checksum
- reparses packet headers
- copy SPI into reg_b (which ends up as cqe.ft_metadata)
- set a decap marker and the PSP version in the flow_tag, so the RX handler can make sense of the packet
- default drop rule for unsupported PSP versions (per PSP spec).
Packets are forwarded to the previously added rx table, where:
- one rule forwards UDP traffic to the UDP default destination.
- default rule forwards traffic to the TTC table.
The reason is to avoid steering loops. If packets were to be injected into the TTC directly after rx_decap, it may be possible to create a steering loop with RX packets of the form IP|UDP|PSP|UDP|PSP... The rx flow table guarantees that packets go through PSP steering at most once.
The steering mode is saved in a new field 'fs.decap_enabled'. Updating the mode is done through accel_psp_fs_rx_reconfigure(), which creates the decap steering table if needed and possible. It then uses an atomic rule update to redirect traffic to the new table. This is now invoked with decap_wanted == false.
The intention is for failures creating the new table to not block feature reconfig. A message is logged when table creation failed and PSP for HW GRO will not work in that case.
Nothing happens on HW without the ability to decapsulate PSP transport.
An upcoming patch will add dynamic reconfiguration of PSP steering based on HW GRO.
--- .../net/ethernet/mellanox/mlx5/core/en/fs.h | 1 + .../mellanox/mlx5/core/en_accel/psp.c | 254 +++++++++++++++++- 2 files changed, 253 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en/fs.h +1 -0
@@ -98,6 +98,7 @@ enum {#if defined(CONFIG_MLX5_EN_PSP )MLX5E_ACCEL_FS_PSP_FT_LEVEL =MLX5E_INNER_TTC_FT_LEVEL +1 ,MLX5E_ACCEL_FS_PSP_ERR_FT_LEVEL , +MLX5E_ACCEL_FS_PSP_DECAP_FT_LEVEL , MLX5E_ACCEL_FS_PSP_RX_FT_LEVEL, #endif };
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +252 -2
338 lines not on this page · raw message
Signature · 1 line
2.44.0
Re: [PATCH net-next 09/13] net/mlx5e: psp: Add an rx_decap steering table
Quoted 18 lines · Tariq Toukan
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +18
-0+static +void accel_psp_fs_rx_decap_ft_destroy (struct mlx5e_psp_fs *fs, +struct mlx5e_psp_rx_decap_table *decap) +{ +int i; + +accel_psp_fs_del_flow_rule (&decap->unsupported_rule); +for (i =0 ; i <MLX5E_PSP_NUM_SUPPORTED_VERSIONS ; i++) +accel_psp_fs_del_flow_rule (&decap->rule[i]); +if (decap->reformat) { +mlx5_packet_reformat_dealloc (fs->mdev, decap->reformat); + decap->reformat =NULL ; + } +accel_psp_fs_dealloc_modify_hdr (fs->mdev, &decap->modify_hdr); +accel_psp_fs_destroy_flow_group (&decap->drop_group); +accel_psp_fs_destroy_ft (&decap->ft); +} +
[...]
Quoted 12 lines
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +12
-0+static int accel_psp_fs_rx_reconfigure (struct mlx5e_psp_fs *fs, +bool decap_wanted) +{ +bool decap_supported = +MLX5_CAP_FLOWTABLE (fs->mdev, + flow_table_properties_nic_receive.reformat_del_psp_transport); +bool decap_enable = decap_wanted && decap_supported; +struct mlx5_flow_destination dest = {}; +int err; + +/* Create the decap table if needed. */ +if (decap_enable && !fs->decap.ft) {
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c accel_psp_fs_rx_decrypt_ft_create(struct mlx5e_psp_fs *fs, · line 731
Quoted 10 lines
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +10
-0+ err =accel_psp_fs_rx_decap_ft_create (fs, &fs->decap); +if (err) { +mlx5_core_warn (fs->mdev, +"Failed to create PSP decapsulation rules (err %d), HW GRO for PSP unavailable" , + err); + decap_enable = false; + } + } +if (decap_enable == fs->decap_enabled) +return 0 ;
An llm is complaining that fs->decap_enabled is not set to false when the decap table is destroyed with accel_psp_fs_rx_decap_ft_destroy(), which would cause problems after doing a psp_dev_ops::set_config() down and up.
I'm not sure if this is legit, but figured I would flag it as it seems plausible and would be easily reachable by a user.
Re: [PATCH net-next 09/13] net/mlx5e: psp: Add an rx_decap steering table
Quoted 58 lines · Daniel Zahka
Quoted 21 lines · Tariq Toukan
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +3
-0+static +void accel_psp_fs_rx_decap_ft_destroy(struct mlx5e_psp_fs *fs, + struct
mlx5e_psp_rx_decap_table*decap)drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +8
-0+{ +int i; + +accel_psp_fs_del_flow_rule (&decap->unsupported_rule); +for (i =0 ; i <MLX5E_PSP_NUM_SUPPORTED_VERSIONS ; i++) +accel_psp_fs_del_flow_rule (&decap->rule[i]); +if (decap->reformat) { + mlx5_packet_reformat_dealloc(fs->mdev, decap-Quoted 1 line
reformat);
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +3
-0+ decap->reformat =NULL ; + } + accel_psp_fs_dealloc_modify_hdr(fs->mdev, &decap-Quoted 1 line
modify_hdr);drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +4
-0+accel_psp_fs_destroy_flow_group (&decap->drop_group); +accel_psp_fs_destroy_ft (&decap->ft); +} +[...]
Quoted 19 lines
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +6
-0+static int accel_psp_fs_rx_reconfigure (struct mlx5e_psp_fs *fs, + bool decap_wanted) +{ +bool decap_supported = + MLX5_CAP_FLOWTABLE(fs->mdev, +flow_table_properties_nic_receive.reformat_del_psp_transport);
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +7
-0+bool decap_enable = decap_wanted && decap_supported; +struct mlx5_flow_destination dest = {}; +int err; + +/* Create the decap table if needed. */ +if (decap_enable && !fs->decap.ft) { + err = accel_psp_fs_rx_decap_ft_create(fs, &fs-Quoted 1 line
decap);
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +3
-0+if (err) { + mlx5_core_warn(fs->mdev, + "Failed to create PSPdecapsulation rules (err %d), HW GRO for PSP unavailable",
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c accel_psp_fs_rx_decrypt_ft_create(struct mlx5e_psp_fs *fs, · line 731
Quoted 6 lines
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +6
-0+ err); + decap_enable = false; + } + } +if (decap_enable == fs->decap_enabled) +return 0 ;An llm is complaining that
fs->decap_enabledis not set to false when the decap table is destroyed withaccel_psp_fs_rx_decap_ft_destroy(), which would cause problems after doing apsp_dev_ops::set_config()down
and up.
Quoted 3 lines
I'm not sure if this is legit, but figured I would flag it as it seems
plausible and would be easily reachable by a user.
This is a legit issue, one that me and our llms missed. Thanks for spotting and reporting it.
Will be fixed in the next version.
Cosmin.
Tariq Toukan
maintainerPATCH 10/13 net/mlx5e: shampo: Flush session on PSP mismatch
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
Flush SHAMPO session on PSP status change (no PSP -> PSP or vice-versa) or on SPI/version mismatch. HW aggregates by 5-tuple but is unaware of PSP fields, so sessions must be terminated when security parameters change to avoid mixing packets from different PSP associations.
The session is also forcefully flushed if non-decapsulated packets are received. This could happen if, for example, the decap table could not be created so HW GRO is active but there's no decapsulation.
--- .../net/ethernet/mellanox/mlx5/core/en_rx.c | 32 +++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +30 -2
@@ -2221,6 +2221,33 @@ static bool mlx5e_hw_gro_skb_has_enough_space(struct sk_buff *skb,return page_size * nr_frags + data_bcnt <=GRO_LEGACY_MAX_SIZE ; } +static bool mlx5e_hw_gro_psp_match (struct sk_buff *skb,struct mlx5_cqe64 *cqe) +{ +#ifdef CONFIG_MLX5_EN_PSP +struct psp_skb_ext *pse =skb_ext_find (skb,SKB_EXT_PSP ); +bool is_psp =mlx5e_psp_is_rx_flow (cqe); + +if (likely (!is_psp && !pse)) +return true; + +/* No match on PSP status change (no crypto -> crypto or vice-versa). */ +if (unlikely (is_psp != !!pse)) +return false; + +/* SPI and version are only available in CQE metadata for decap flows. +* Non-decap PSP cannot be matched here, force a flush. +*/ +if (unlikely (!mlx5e_psp_is_decap (cqe))) +return false; + +/* No match on security parameters change. */ +return pse->spi ==mlx5e_psp_get_spi (cqe) && + pse->version ==mlx5e_psp_get_version (cqe); +#else +return true; +#endif +} +static void mlx5e_handle_rx_cqe_mpwrq_shampo (struct mlx5e_rq *rq,struct mlx5_cqe64 *cqe) {u16 data_bcnt =mpwrq_get_cqe_byte_cnt (cqe) - cqe->shampo.header_size;
@@ -2258,8 +2285,9 @@ static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cq data_offset = wqe_offset & (page_size -1 ); page_idx = wqe_offset >> rq->mpwqe.page_shift;if (*skb && - !(match &&mlx5e_hw_gro_skb_has_enough_space (*skb, data_bcnt, - page_size))) { + !(match && +mlx5e_hw_gro_skb_has_enough_space (*skb, data_bcnt, page_size) && +mlx5e_hw_gro_psp_match (*skb, cqe))) { match = false;mlx5e_shampo_flush_skb (rq, cqe, match); }
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 11/13 net/mlx5e: psp: Dynamically reconfigure based on SHAMPO mode
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
Add an mlx5e_update_nic_rx() -> mlx5e_psp_update_rx() hook. This allows PSP steering to reconfigure when netdev features change.
When this signal is received, toggle PSP between standard and decap modes when SHAMPO mode changes, optionally creating the decap steering table.
The toggling of traffic is atomic (a single steering rule update), so no packets should get eaten/lost. It is probable though that the rxhash will change, and decapsulated packets will be received by a different core than before. Established PSP connections receiving traffic might experience reordering/retransmits as a result.
Also enable decap when configuring the device if the mode indicates it.
--- .../mellanox/mlx5/core/en_accel/en_accel.h | 9 ++++++++ .../mellanox/mlx5/core/en_accel/psp.c | 23 +++++++++++++++++-- .../mellanox/mlx5/core/en_accel/psp.h | 2 ++ .../net/ethernet/mellanox/mlx5/core/en_main.c | 8 ++++++- 4 files changed, 39 insertions(+), 3 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h +9 -0
@@ -251,6 +251,15 @@ static inline void mlx5e_accel_cleanup_rx(struct mlx5e_priv *priv)mlx5_accel_psp_fs_cleanup_rx_tables (priv); } +static inline int mlx5e_accel_update_rx (struct mlx5e_priv *priv) +{ +#ifdef CONFIG_MLX5_EN_PSP +return mlx5e_psp_update_rx (priv); +#else +return 0 ; +#endif +} +static inline int mlx5e_accel_init_tx (struct mlx5e_priv *priv) {return mlx5e_ktls_init_tx (priv);
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c +21 -2
@@ -92,6 +92,12 @@ struct mlx5e_psp_fs {struct mlx5e_psp_rx_table rx; }; +static bool shampo_enabled (struct mlx5e_priv *priv) +{ +return priv->channels.params.packet_merge.type == +MLX5E_PACKET_MERGE_SHAMPO ; +} +/* PSP RX flow steering */ static enum mlx5_traffic_types fs_psp2tt (enum accel_fs_psp_type i) {
@@ -766,6 +772,7 @@ static void accel_psp_fs_rx_destroy(struct mlx5e_psp_fs *fs) }static int accel_psp_fs_rx_create (struct mlx5e_psp_fs *fs, +bool decap_enable,struct netlink_ext_ack *extack) {struct mlx5_ttc_table *ttc =mlx5e_fs_get_ttc (fs->fs, false);
@@ -807,7 +814,7 @@ static int accel_psp_fs_rx_create(struct mlx5e_psp_fs *fs,mlx5_ttc_fwd_dest (ttc,fs_psp2tt (i), &dest); } - err =accel_psp_fs_rx_reconfigure (fs, false); + err =accel_psp_fs_rx_reconfigure (fs, decap_enable);if (err) {NL_SET_ERR_MSG (extack,"Failed RX steering config for HW GRO" ); goto err_decrypt_ft;
@@ -1081,7 +1088,8 @@ static int accel_psp_fs_create(struct mlx5e_priv *priv, {int err; - err =accel_psp_fs_rx_create (priv->psp->fs, extack); + err =accel_psp_fs_rx_create (priv->psp->fs,shampo_enabled (priv), + extack);if (err)return err;
@@ -1358,3 +1366,14 @@ void mlx5e_psp_cleanup(struct mlx5e_priv *priv) priv->psp =NULL ;kfree (psp); } + +int mlx5e_psp_update_rx (struct mlx5e_priv *priv) +{ +struct mlx5e_psp *psp = priv->psp; + +netdev_assert_locked (priv->netdev); +if (!psp || !psp->fs->check.ft) +return 0 ; + +return accel_psp_fs_rx_reconfigure (psp->fs,shampo_enabled (priv)); +}
drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.h +2 -0
@@ -49,6 +49,7 @@ int mlx5e_psp_register(struct mlx5e_priv *priv);void mlx5e_psp_unregister (struct mlx5e_priv *priv);int mlx5e_psp_init (struct mlx5e_priv *priv);void mlx5e_psp_cleanup (struct mlx5e_priv *priv); +int mlx5e_psp_update_rx (struct mlx5e_priv *priv);#else static inline void mlx5_accel_psp_fs_cleanup_rx_tables (struct mlx5e_priv *priv) { }static inline void mlx5_accel_psp_fs_cleanup_tx_tables (struct mlx5e_priv *priv) { }
@@ -61,5 +62,6 @@ static inline int mlx5e_psp_register(struct mlx5e_priv *priv) { return 0; }static inline void mlx5e_psp_unregister (struct mlx5e_priv *priv) { }static inline int mlx5e_psp_init (struct mlx5e_priv *priv) {return 0 ; }static inline void mlx5e_psp_cleanup (struct mlx5e_priv *priv) { } +static inline int mlx5e_psp_update_rx (struct mlx5e_priv *priv) {return 0 ; }#endif /* CONFIG_MLX5_EN_PSP */ #endif /* __MLX5E_ACCEL_PSP_H__ */
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +7 -1
@@ -6286,7 +6286,13 @@ static void mlx5e_nic_disable(struct mlx5e_priv *priv)static int mlx5e_update_nic_rx (struct mlx5e_priv *priv) { -return mlx5e_refresh_tirs (priv->mdev, false, false); +int err; + + err =mlx5e_refresh_tirs (priv->mdev, false, false); +if (err) +return err; + +return mlx5e_accel_update_rx (priv); }static const struct mlx5e_profile mlx5e_nic_profile = {
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 12/13 selftests: drv-net: psp: Fix responder parsing
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
The psp_responder accumulates received data in a buffer and parses received messages from it, but a message that's split in two across command and argument (e.g. "psp conn" and "1") will permanently choke the parser, because the stand-alone arg is never parsed after discarding the command.
This is mostly a theoretical issue since a sent TCP segment from psp.py of the form "psp conn 1" will arrive in one piece to psp_responder, but the AI tools complain about the possibility that it might get split, so fix it now before another command with an argument is added in the next patch.
--- .../selftests/drivers/net/psp_responder.c | 38 +++++++++---------- 1 file changed, 17 insertions(+), 21 deletions(-)
tools/testing/selftests/drivers/net/psp_responder.c +17 -21
@@ -185,22 +185,22 @@ run_session(struct ynl_sock *ys, struct opts *opts, } off += n; - n = off;#define __consume (sz) \ ({ \ - if (n == (sz)) { \ - off = 0; \ - } else { \ - off -= (sz); \ - memmove(buf, &buf[(sz)], off); \ - } \ + off -= (sz); \ + memmove(buf, &buf[(sz)], off); \ }) -#define cmd (_name) \ +/* Only match once the command and its _extra_sz byte payload are both +* buffered, otherwise a split read would consume the name and strand +* the payload, desynchronizing the parser for good. +*/ +#define cmd (_name, _extra_sz) \ ({ \ ssize_t sz = sizeof(_name); \ - bool match = n >= sz && !memcmp(buf, _name, sz); \ + bool match = off >= sz + (_extra_sz) && \ + !memcmp(buf, _name, sz); \ \ if (match) { \ dbg("command: " _name "\n"); \
@@ -213,10 +213,10 @@ run_session(struct ynl_sock *ys, struct opts *opts, do { consumed = false; - if (cmd("read len")) + if (cmd("read len", 0)) send_str(comm_sock, data_read); - if (cmd("data echo")) { + if (cmd("data echo", 0)) { if (data_sock >= 0) send(data_sock, "echo", 5, MSG_WAITALL);
@@ -224,7 +224,7 @@ run_session(struct ynl_sock *ys, struct opts *opts,fprintf (stderr,"WARN: echo but no data sock\n" );send_ack (comm_sock); } -if (cmd ("data close" )) { +if (cmd ("data close" ,0 )) {if (data_sock >=0 ) {close (data_sock); data_sock =-1 ;
@@ -233,26 +233,22 @@ run_session(struct ynl_sock *ys, struct opts *opts, race_close = true; } } -if (cmd ("conn psp" )) { +if (cmd ("conn psp" ,2 )) {if (accept_cfg !=ACCEPT_CFG_NONE )fprintf (stderr,"WARN: old conn config still set!\n" ); accept_cfg =ACCEPT_CFG_PSP ;send_ack (comm_sock);/* next two bytes are versions */ -if (off >=2 ) { -memcpy (&psp_vers, buf,2 ); -__consume (2 ); - }else { -fprintf (stderr,"WARN: short conn psp command!\n" ); - } +memcpy (&psp_vers, buf,2 ); +__consume (2 ); } -if (cmd ("conn clr" )) { +if (cmd ("conn clr" ,0 )) {if (accept_cfg !=ACCEPT_CFG_NONE )fprintf (stderr,"WARN: old conn config still set!\n" ); accept_cfg =ACCEPT_CFG_CLEAR ;send_ack (comm_sock); } -if (cmd ("exit" )) +if (cmd ("exit" ,0 )) should_quit = true;#undef cmd
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH 12/12 net: psp: Add a self test for PSP with HW-GRO
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
Add a test case which attempts to send several 64K chunks over PSP, which is more than MTU, and thus trigger GSO on the TX side and hopefully HW GRO on the RX side. Verify HW GRO counters increase. This required the addition of a new command in the psp responder, so it sends the data in order to get it reassembled on the local (DUT) side. This is required in order to read the qstats.
Some light refactoring of the PSP responder was also done, because the number of indentation levels started making checkpack unhappy.
--- tools/testing/selftests/drivers/net/psp.py | 129 ++++++++++++++++-- .../selftests/drivers/net/psp_responder.c | 51 +++++++ 2 files changed, 172 insertions(+), 8 deletions(-)
tools/testing/selftests/drivers/net/psp.py +121 -8
@@ -18,9 +18,10 @@ from lib.py import ksft_not_none from lib.py import ksft_variants, KsftNamedVariant from lib.py import KsftSkipEx, KsftFailEx from lib.py import NetDrvEpEnv, NetDrvContEnv -from lib.py import Netlink, NlError, PSPFamily, RtnlFamily +from lib.py import Netlink, NlError, NetdevFamily, PSPFamily, RtnlFamily from lib.py import NetNSEnter from lib.py import bkg, rand_port, wait_port_listen +from lib.py import bkg, ethtool, rand_port, wait_port_listen, CmdExitFailure from lib.py import ip
@@ -30,15 +31,25 @@ def _get_outq(s): return struct.unpack("I", outq)[0] +def _recv_exact(sock, n): + buf = b'' + while len(buf) < n: + chunk = sock.recv(n - len(buf)) + if not chunk: + break + buf += chunk + return buf + + def _send_with_ack(cfg, msg): - cfg.comm_sock.send(msg) - response = cfg.comm_sock.recv(4) + cfg.comm_sock.sendall(msg) + response = _recv_exact(cfg.comm_sock, 4) if response != b'ack\0': raise RuntimeError("Unexpected server response", response) def _remote_read_len(cfg): - cfg.comm_sock.send(b'read len\0') + cfg.comm_sock.sendall(b'read len\0') return int(cfg.comm_sock.recv(1024)[:-1].decode('utf-8'))
@@ -97,7 +108,7 @@ def _send_careful(cfg, s, rounds): def _check_data_rx(cfg, exp_len): read_len = -1 for _ in range(30): - cfg.comm_sock.send(b'read len\0') + cfg.comm_sock.sendall(b'read len\0') read_len = int(cfg.comm_sock.recv(1024)[:-1].decode('utf-8')) if read_len == exp_len: break
@@ -583,6 +594,108 @@ def _get_psp_ver_ip_variants(): for ipv in ("4", "6"): yield KsftNamedVariant(f"v{ver}_ip{ipv}", ver, ipv) +def _enable_local_hw_gro(cfg): + cfg.require_cmd("ethtool") + feat = ethtool(f"-k {cfg.ifname}", json=True)[0] + gro = feat.get("rx-gro-hw") + if not gro or "active" not in gro or "fixed" not in gro: + raise KsftSkipEx("HW GRO feature not reported by ethtool") + if gro["fixed"] and not gro["active"]: + raise KsftSkipEx("HW GRO not supported by device") + if not gro["active"]: + try: + ethtool(f"-K {cfg.ifname} rx-gro-hw on") + except CmdExitFailure as e: + raise KsftSkipEx("Cannot enable HW GRO via ethtool") from e + defer(ethtool, f"-K {cfg.ifname} rx-gro-hw off") + feat = ethtool(f"-k {cfg.ifname}", json=True)[0] + gro = feat.get("rx-gro-hw", {}) + if not gro.get("active"): + raise KsftSkipEx("HW GRO failed to activate") + + +def _remote_send(cfg, size): + cfg.comm_sock.sendall(b'data send\0' + struct.pack('I', size)) + response = _recv_exact(cfg.comm_sock, 4) + if response != b'ack\0': + raise RuntimeError("Unexpected server response to data send", response) + + +def _recv_all(s, expected, timeout=20): + s.settimeout(timeout) + total = 0 + try: + while total < expected: + data = s.recv(min(65536, expected - total)) + if not data: + break + total += len(data) + except socket.timeout: + raise RuntimeError( + f"Timed out receiving data: got {total}/{expected} bytes " + f"(responder may have failed to send)") + if total < expected: + raise RuntimeError( + f"Short read: got {total}/{expected} bytes " + f"(responder connection closed early)") + return total + + +def _data_hw_gro(cfg, version, ipver): + """ Test PSP data transfer with HW GRO enabled """ + _init_psp_dev(cfg) + # Version 0 is required by spec, don't let it skip + if version: + name = cfg.pspnl.consts["version"].entries_by_val[version].name + if name not in cfg.psp_info['psp-versions-cap']: + raise KsftSkipEx("PSP version not supported", name) + + _enable_local_hw_gro(cfg) + + s = _make_psp_conn(cfg, version, ipver) + try: + rx_assoc = cfg.pspnl.rx_assoc({"version": version, + "dev-id": cfg.psp_dev_id, + "sock-fd": s.fileno()}) + rx = rx_assoc['rx-key'] + tx = _spi_xchg(s, rx) + cfg.pspnl.tx_assoc({"dev-id": cfg.psp_dev_id, + "version": version, + "tx-key": tx, + "sock-fd": s.fileno()}) + + try: + before = cfg.netnl.qstats_get({"ifindex": cfg.ifindex}, dump=True)[0] + except NlError as e: + if e.error == errno.EOPNOTSUPP: + raise KsftSkipEx("qstats not supported by the device") from e + raise + if ('rx-hw-gro-packets' not in before or + 'rx-hw-gro-wire-packets' not in before): + raise KsftSkipEx("rx-hw-gro-packets counter not available") + + # Remote sends data in 8KB chunks; GSO on remote TX segments them, + # HW GRO reassembles on local RX + data_len = 10 * 65536 + _remote_send(cfg, data_len) + recv_len = _recv_all(s, data_len) + ksft_eq(recv_len, data_len) + + cfg.wait_hw_stats_settle() + after = cfg.netnl.qstats_get({"ifindex": cfg.ifindex}, dump=True)[0] + ksft_gt(after['rx-hw-gro-packets'], + before['rx-hw-gro-packets']) + ksft_gt(after['rx-hw-gro-wire-packets'], + before['rx-hw-gro-wire-packets']) + finally: + _close_psp_conn(cfg, s) + +@ksft_variants(_get_psp_ver_ip_variants()) +def data_hw_gro(cfg, version, ipver): + """ Test PSP data transfer with HW GRO enabled """ + cfg.require_ipver(ipver) + _data_hw_gro(cfg, version, ipver) + def _get_ip_variants(): for ipv in ("4", "6"):
@@ -937,7 +1050,6 @@ def _setup_psp_attributes(cfg): cfg.psp_dev_peer_nsid = _get_nsid(cfg.netns.name) - def main() -> None: """ Ksft boiler plate main """
@@ -954,6 +1066,7 @@ def main() -> None: with env as cfg: cfg.pspnl = PSPFamily() + cfg.netnl = NetdevFamily() if has_cont: _setup_psp_attributes(cfg)
@@ -973,7 +1086,7 @@ def main() -> None: cfg.comm_port), timeout=1) - cases = [data_basic_send, data_mss_adjust] + cases = [data_basic_send, data_hw_gro, data_mss_adjust] if has_cont: cases += [
@@ -990,7 +1103,7 @@ def main() -> None: case_pfx={"dev_", "data_", "assoc_", "removal_"}, args=(cfg, )) - cfg.comm_sock.send(b"exit\0") + cfg.comm_sock.sendall(b"exit\0") cfg.comm_sock.close() finally: if srv and (srv.stdout or srv.stderr):
tools/testing/selftests/drivers/net/psp_responder.c +51 -0
68 lines not on this page · raw message
Signature · 1 line
2.44.0
Tariq Toukan
maintainerRe: [PATCH net-next 12/12] net: psp: Add a self test for PSP with HW-GRO
Quoted 9 lines · Tariq Toukan
From: Cosmin Ratiu <[email protected]>
Add a test case which attempts to send several 64K chunks over PSP, which is more than MTU, and thus trigger GSO on the TX side and hopefully HW GRO on the RX side. Verify HW GRO counters increase. This required the addition of a new command in the psp responder, so it sends the data in order to get it reassembled on the local (DUT) side. This is required in order to read the qstats.
Quoted 7 lines
Some light refactoring of the PSP responder was also done, because the number of indentation levels started making checkpack unhappy.
Signed-off-by Cosmin Ratiu <[email protected]>Reviewed-by Dragos Tatulea <[email protected]>Signed-off-by Tariq Toukan <[email protected]>---
Oh, please ignore this one. Sent by mistake.
It's a leftover I had locally.
Jakub Kicinski
maintainerRe: [PATCH net-next 12/12] net: psp: Add a self test for PSP with HW-GRO
Quoted 2 lines · On Thu, 30 Jul 2026 12:58:07 +0300 Tariq Toukan wrote:
Oh, please ignore this one. Sent by mistake.
It's a leftover I had locally.
Ack, looks like patchwork did the right thing so all good
Tariq Toukan
maintainerPATCH 13/13 selftests: drv-net: psp: Add a test for PSP with HW-GRO
In reply to Tariq ToukanFrom: Cosmin Ratiu <[email protected]>
Add a test case which attempts to send several 64K chunks over PSP, which is more than MTU, and thus trigger GSO on the TX side and HW GRO on the RX side. Verify HW GRO counters increase.
This required the addition of a new command in the psp responder, so it sends the data in order to get it reassembled on the local (DUT) side.
--- tools/testing/selftests/drivers/net/psp.py | 129 ++++++++++++++++-- .../selftests/drivers/net/psp_responder.c | 49 +++++++ 2 files changed, 170 insertions(+), 8 deletions(-)
tools/testing/selftests/drivers/net/psp.py +121 -8
@@ -18,9 +18,10 @@ from lib.py import ksft_not_none from lib.py import ksft_variants, KsftNamedVariant from lib.py import KsftSkipEx, KsftFailEx from lib.py import NetDrvEpEnv, NetDrvContEnv -from lib.py import Netlink, NlError, PSPFamily, RtnlFamily +from lib.py import Netlink, NlError, NetdevFamily, PSPFamily, RtnlFamily from lib.py import NetNSEnter from lib.py import bkg, rand_port, wait_port_listen +from lib.py import bkg, ethtool, rand_port, wait_port_listen, CmdExitFailure from lib.py import ip
@@ -30,15 +31,25 @@ def _get_outq(s): return struct.unpack("I", outq)[0] +def _recv_exact(sock, n): + buf = b'' + while len(buf) < n: + chunk = sock.recv(n - len(buf)) + if not chunk: + break + buf += chunk + return buf + + def _send_with_ack(cfg, msg): - cfg.comm_sock.send(msg) - response = cfg.comm_sock.recv(4) + cfg.comm_sock.sendall(msg) + response = _recv_exact(cfg.comm_sock, 4) if response != b'ack\0': raise RuntimeError("Unexpected server response", response) def _remote_read_len(cfg): - cfg.comm_sock.send(b'read len\0') + cfg.comm_sock.sendall(b'read len\0') return int(cfg.comm_sock.recv(1024)[:-1].decode('utf-8'))
@@ -97,7 +108,7 @@ def _send_careful(cfg, s, rounds): def _check_data_rx(cfg, exp_len): read_len = -1 for _ in range(30): - cfg.comm_sock.send(b'read len\0') + cfg.comm_sock.sendall(b'read len\0') read_len = int(cfg.comm_sock.recv(1024)[:-1].decode('utf-8')) if read_len == exp_len: break
@@ -583,6 +594,108 @@ def _get_psp_ver_ip_variants(): for ipv in ("4", "6"): yield KsftNamedVariant(f"v{ver}_ip{ipv}", ver, ipv) +def _enable_local_hw_gro(cfg): + cfg.require_cmd("ethtool") + feat = ethtool(f"-k {cfg.ifname}", json=True)[0] + gro = feat.get("rx-gro-hw") + if not gro or "active" not in gro or "fixed" not in gro: + raise KsftSkipEx("HW GRO feature not reported by ethtool") + if gro["fixed"] and not gro["active"]: + raise KsftSkipEx("HW GRO not supported by device") + if not gro["active"]: + try: + ethtool(f"-K {cfg.ifname} rx-gro-hw on") + except CmdExitFailure as e: + raise KsftSkipEx("Cannot enable HW GRO via ethtool") from e + defer(ethtool, f"-K {cfg.ifname} rx-gro-hw off") + feat = ethtool(f"-k {cfg.ifname}", json=True)[0] + gro = feat.get("rx-gro-hw", {}) + if not gro.get("active"): + raise KsftSkipEx("HW GRO failed to activate") + + +def _remote_send(cfg, size): + cfg.comm_sock.sendall(b'data send\0' + struct.pack('!I', size)) + response = _recv_exact(cfg.comm_sock, 4) + if response != b'ack\0': + raise RuntimeError("Unexpected server response to data send", response) + + +def _recv_all(s, expected, timeout=20): + s.settimeout(timeout) + total = 0 + try: + while total < expected: + data = s.recv(min(65536, expected - total)) + if not data: + break + total += len(data) + except socket.timeout: + raise RuntimeError( + f"Timed out receiving data: got {total}/{expected} bytes " + f"(responder may have failed to send)") + if total < expected: + raise RuntimeError( + f"Short read: got {total}/{expected} bytes " + f"(responder connection closed early)") + return total + + +def _data_hw_gro(cfg, version, ipver): + """ Test PSP data transfer with HW GRO enabled """ + _init_psp_dev(cfg) + # Version 0 is required by spec, don't let it skip + if version: + name = cfg.pspnl.consts["version"].entries_by_val[version].name + if name not in cfg.psp_info['psp-versions-cap']: + raise KsftSkipEx("PSP version not supported", name) + + _enable_local_hw_gro(cfg) + + s = _make_psp_conn(cfg, version, ipver) + try: + rx_assoc = cfg.pspnl.rx_assoc({"version": version, + "dev-id": cfg.psp_dev_id, + "sock-fd": s.fileno()}) + rx = rx_assoc['rx-key'] + tx = _spi_xchg(s, rx) + cfg.pspnl.tx_assoc({"dev-id": cfg.psp_dev_id, + "version": version, + "tx-key": tx, + "sock-fd": s.fileno()}) + + try: + before = cfg.netnl.qstats_get({"ifindex": cfg.ifindex}, dump=True)[0] + except NlError as e: + if e.error == errno.EOPNOTSUPP: + raise KsftSkipEx("qstats not supported by the device") from e + raise + if ('rx-hw-gro-packets' not in before or + 'rx-hw-gro-wire-packets' not in before): + raise KsftSkipEx("rx-hw-gro-packets counter not available") + + # Remote sends data in 8KB chunks; GSO on remote TX segments them, + # HW GRO reassembles on local RX + data_len = 10 * 65536 + _remote_send(cfg, data_len) + recv_len = _recv_all(s, data_len) + ksft_eq(recv_len, data_len) + + cfg.wait_hw_stats_settle() + after = cfg.netnl.qstats_get({"ifindex": cfg.ifindex}, dump=True)[0] + ksft_gt(after['rx-hw-gro-packets'], + before['rx-hw-gro-packets']) + ksft_gt(after['rx-hw-gro-wire-packets'], + before['rx-hw-gro-wire-packets']) + finally: + _close_psp_conn(cfg, s) + +@ksft_variants(_get_psp_ver_ip_variants()) +def data_hw_gro(cfg, version, ipver): + """ Test PSP data transfer with HW GRO enabled """ + cfg.require_ipver(ipver) + _data_hw_gro(cfg, version, ipver) + def _get_ip_variants(): for ipv in ("4", "6"):
@@ -937,7 +1050,6 @@ def _setup_psp_attributes(cfg): cfg.psp_dev_peer_nsid = _get_nsid(cfg.netns.name) - def main() -> None: """ Ksft boiler plate main """
@@ -954,6 +1066,7 @@ def main() -> None: with env as cfg: cfg.pspnl = PSPFamily() + cfg.netnl = NetdevFamily() if has_cont: _setup_psp_attributes(cfg)
@@ -973,7 +1086,7 @@ def main() -> None: cfg.comm_port), timeout=1) - cases = [data_basic_send, data_mss_adjust] + cases = [data_basic_send, data_hw_gro, data_mss_adjust] if has_cont: cases += [
@@ -990,7 +1103,7 @@ def main() -> None: case_pfx={"dev_", "data_", "assoc_", "removal_"}, args=(cfg, )) - cfg.comm_sock.send(b"exit\0") + cfg.comm_sock.sendall(b"exit\0") cfg.comm_sock.close() finally: if srv and (srv.stdout or srv.stderr):
tools/testing/selftests/drivers/net/psp_responder.c +49 -0
66 lines not on this page · raw message
Signature · 1 line
2.44.0
Jakub Kicinski
maintainerRe: [PATCH net-next 13/13] selftests: drv-net: psp: Add a test for PSP with HW-GRO
tools/testing/selftests/drivers/net/psp.py def _get_psp_ver_ip_variants(): · line 604
Quoted 2 lines · On Thu, 30 Jul 2026 12:17:56 +0300 Tariq Toukan wrote:
tools/testing/selftests/drivers/net/psp.py +2
-0+ if gro["fixed"] and not gro["active"]: + raise KsftSkipEx("HW GRO not supported by device")
XFail, this runs over netdevsim, SW tests must not skip.