Skip to content

mlx5e use TLS TX pool to improve connection rate

Tariq Toukan netdev 2 versions – Applied

Status from this message: This series was applied to netdev/net-next.git (master)

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest6 patches 2 repliesApplied 6/6
  2. v26 patches 1 replySuperseded

Changes in v3 from the cover letter

Rebased on top of relevant fixes in TLS module.

Patches in v3 All 6 applied

#Subject and filesSizeRepliesApplied
1/6net/tls: Perform immediate device ctx cleanup when possiblenet/tls/tls_device.c+18 -80 repliesApplied
2/6net/tls: Multi-threaded calls to TX tls_dev_delinclude/net/tls.h net/tls/tls_device.c+33 -322 repliesApplied
3/6net/mlx5e: kTLS, Introduce TLS-specific create TISdrivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c+9 -50 repliesApplied
4/6net/mlx5e: kTLS, Take stats out of OOO handlerdrivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c+11 -160 repliesApplied
5/6net/mlx5e: kTLS, Recycle objects of device-offloaded TLS TX connectionsdrivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_stats.c +2 more+199 -470 repliesApplied
6/6net/mlx5e: kTLS, Dynamically re-size TX recycling pooldrivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c+289 -260 repliesApplied

Reviews trailers per patch

5 of 6 patches have a Reviewed-by or Acked-by.

PatchMaxim MikityanskiyGal Pressman
1/6R-b
2/6R-b
3/6R-b
4/6R-b
5/6R-b
6/6

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [PATCH net-next V3 0/6] mlx5e use TLS TX pool to improve connection rate

To offload encryption operations, the mlx5 device maintains state and keeps track of every kTLS device-offloaded connection. Two HW objects are used per TX context of a kTLS offloaded connection: a. Transport interface send (TIS) object, to reach the HW context. b. Data Encryption Key (DEK) to perform the crypto operations.

These two objects are created and destroyed per TLS TX context, via FW commands. In total, 4 FW commands are issued per TLS TX context, which seriously limits the connection rate.

In this series, we aim to save creation and destroy of TIS objects by recycling them. Upon recycling of a TIS, the HW still needs to be notified for the re-mapping between a TIS and a context. This is done by posting WQEs via an SQ, significantly faster API than the FW command interface.

A pool is used for recycling. The pool dynamically interacts to the load and connection rate, growing and shrinking accordingly.

Saving the TIS FW commands per context increases connection rate by ~42%, from 11.6K to 16.5K connections per sec.

Connection rate is still limited by FW bottleneck due to the remaining per context FW commands (DEK create/destroy). This will soon be addressed in a followup series. By combining the two series, the FW bottleneck will be released, and a significantly higher (about 100K connections per sec) kTLS TX device-offloaded connection rate is reached.

Regards,
Tariq

v3:
Rebased on top of relevant fixes in TLS module.

Tariq Toukan (6):
net/tls: Perform immediate device ctx cleanup when possible net/tls: Multi-threaded calls to TX tls_dev_del net/mlx5e: kTLS, Introduce TLS-specific create TIS net/mlx5e: kTLS, Take stats out of OOO handler net/mlx5e: kTLS, Recycle objects of device-offloaded TLS TX
connections
net/mlx5e: kTLS, Dynamically re-size TX recycling pool

 .../mellanox/mlx5/core/en_accel/en_accel.h    |  10 +
 .../mellanox/mlx5/core/en_accel/ktls.h        |  14 +
 .../mellanox/mlx5/core/en_accel/ktls_stats.c  |   2 +
 .../mellanox/mlx5/core/en_accel/ktls_tx.c     | 513 +++++++++++++++---
 .../net/ethernet/mellanox/mlx5/core/en_main.c |   9 +
 include/net/tls.h                             |   2 +
 net/tls/tls_device.c                          |  79 +--
 7 files changed, 527 insertions(+), 102 deletions(-)
Signature · 1 line
2.21.0