mlx5e use TLS TX pool to improve connection rate
Status from this message: This series was applied to netdev/net-next.git (master)
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v3 Latest6 patches 2 repliesApplied 6/6
- v26 patches 1 replySuperseded
Changes in v3 from the cover letter
Rebased on top of relevant fixes in TLS module.
Patches in v3 All 6 applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/6 | net/tls: Perform immediate device ctx cleanup when possiblenet/ | +18 | 0 replies | Applied |
| 2/6 | net/tls: Multi-threaded calls to TX tls_dev_delinclude/ net/ | +33 | 2 replies | Applied |
| 3/6 | net/mlx5e: kTLS, Introduce TLS-specific create TISdrivers/ | +9 | 0 replies | Applied |
| 4/6 | net/mlx5e: kTLS, Take stats out of OOO handlerdrivers/ | +11 | 0 replies | Applied |
| 5/6 | net/mlx5e: kTLS, Recycle objects of device-offloaded TLS TX connectionsdrivers/ drivers/ drivers/ +2 more | +199 | 0 replies | Applied |
| 6/6 | net/mlx5e: kTLS, Dynamically re-size TX recycling pooldrivers/ | +289 | 0 replies | Applied |
Reviews trailers per patch
5 of 6 patches have a Reviewed-by or Acked-by.
| Patch | Maxim Mikityanskiy | Gal Pressman |
|---|---|---|
| 1/6 | R-b | |
| 2/6 | R-b | |
| 3/6 | R-b | |
| 4/6 | R-b | |
| 5/6 | R-b | |
| 6/6 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH net-next V3 0/6] mlx5e use TLS TX pool to improve connection rate
To offload encryption operations, the mlx5 device maintains state and keeps track of every kTLS device-offloaded connection. Two HW objects are used per TX context of a kTLS offloaded connection: a. Transport interface send (TIS) object, to reach the HW context. b. Data Encryption Key (DEK) to perform the crypto operations.
These two objects are created and destroyed per TLS TX context, via FW commands. In total, 4 FW commands are issued per TLS TX context, which seriously limits the connection rate.
In this series, we aim to save creation and destroy of TIS objects by recycling them. Upon recycling of a TIS, the HW still needs to be notified for the re-mapping between a TIS and a context. This is done by posting WQEs via an SQ, significantly faster API than the FW command interface.
A pool is used for recycling. The pool dynamically interacts to the load and connection rate, growing and shrinking accordingly.
Saving the TIS FW commands per context increases connection rate by ~42%, from 11.6K to 16.5K connections per sec.
Connection rate is still limited by FW bottleneck due to the remaining per context FW commands (DEK create/destroy). This will soon be addressed in a followup series. By combining the two series, the FW bottleneck will be released, and a significantly higher (about 100K connections per sec) kTLS TX device-offloaded connection rate is reached.
Regards,
Tariq
v3:
Rebased on top of relevant fixes in TLS module.
Tariq Toukan (6):
net/tls: Perform immediate device ctx cleanup when possible net/tls: Multi-threaded calls to TX tls_dev_del net/mlx5e: kTLS, Introduce TLS-specific create TIS net/mlx5e: kTLS, Take stats out of OOO handler net/mlx5e: kTLS, Recycle objects of device-offloaded TLS TX
connections
net/mlx5e: kTLS, Dynamically re-size TX recycling pool
.../mellanox/mlx5/core/en_accel/en_accel.h | 10 + .../mellanox/mlx5/core/en_accel/ktls.h | 14 + .../mellanox/mlx5/core/en_accel/ktls_stats.c | 2 + .../mellanox/mlx5/core/en_accel/ktls_tx.c | 513 +++++++++++++++--- .../net/ethernet/mellanox/mlx5/core/en_main.c | 9 + include/net/tls.h | 2 + net/tls/tls_device.c | 79 +-- 7 files changed, 527 insertions(+), 102 deletions(-)
Signature · 1 line
2.21.0