net/mlx5: support ESP SPI match and RSS hash
Subsystems (2)
PATCH 1/1 net/mlx5: support ESP SPI match and RSS hash
In packets with ESP header, The Inner IP will be encrypted, and its fields cannot be used for RSS hashing. So, ESP packets can be hashed only by the outer IP layer.
So, when using RSS on ESP packets, hashing may not be efficient, because the fields used by the hash functions are only the Outer IPs, causing all traffic belonging to all tunnels between a given pair of GWs to land on one core.
Adding the SPI hash field can extend the spreading of IPSec packets.
support matching on ESP SPI.
Support RSS on ESP header by hashing SPI field.
--- drivers/common/mlx5/linux/meson.build | 2 + drivers/common/mlx5/mlx5_prm.h | 5 +- drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_devx.c | 4 +- drivers/net/mlx5/mlx5_flow.c | 84 ++++++++++++++++++++++++- drivers/net/mlx5/mlx5_flow.h | 20 ++++++ drivers/net/mlx5/mlx5_flow_dv.c | 88 ++++++++++++++++++++++++++- 7 files changed, 200 insertions(+), 5 deletions(-)
drivers/common/mlx5/linux/meson.build +2 -0
@@ -72,6 +72,8 @@ has_member_args = [ # [ "MACRO to define if found", "header for the search", # "symbol to search" ] has_sym_args = [ + [ 'HAVE_IBV_RX_HASH_IPSEC_SPI', 'infiniband/verbs.h', + 'IBV_RX_HASH_IPSEC_SPI' ], [ 'HAVE_IBV_RELAXED_ORDERING', 'infiniband/verbs.h', 'IBV_ACCESS_RELAXED_ORDERING ' ], [ 'HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT', 'infiniband/mlx5dv.h',
drivers/common/mlx5/mlx5_prm.h +4 -1
@@ -899,7 +899,10 @@ struct mlx5_ifc_fte_match_set_misc_bits {u8 reserved_at_120[0xa ];u8 geneve_opt_len[0x6 ];u8 geneve_protocol_type[0x10 ]; -u8 reserved_at_140[0xc0 ]; +u8 reserved_at_140[0x20 ]; +u8 inner_esp_spi[0x20 ]; +u8 outer_esp_spi[0x20 ]; +u8 reserved_at_1a0[0x60 ]; };struct mlx5_ifc_ipv4_layout_bits {
drivers/net/mlx5/mlx5_defs.h +1 -1
@@ -90,7 +90,7 @@/* Supported RSS */ #define MLX5_RSS_HF_MASK (~(RTE_ETH_RSS_IP | RTE_ETH_RSS_UDP | RTE_ETH_RSS_TCP | \ - MLX5_RSS_SRC_DST_ONLY)) + MLX5_RSS_SRC_DST_ONLY | RTE_ETH_RSS_ESP))/* Timeout in seconds to get a valid link status. */ #define MLX5_LINK_STATUS_TIMEOUT 10
drivers/net/mlx5/mlx5_devx.c +3 -1
@@ -765,7 +765,9 @@ mlx5_devx_tir_attr_set(struct rte_eth_dev *dev, const uint8_t *rss_key, (!!(hash_fields &MLX5_L4_SRC_IBV_RX_HASH )) <<MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_SPORT | (!!(hash_fields &MLX5_L4_DST_IBV_RX_HASH )) << -MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT ; +MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT | + (!!(hash_fields &IBV_RX_HASH_IPSEC_SPI )) << +MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_IPSEC_SPI ; }if (is_hairpin) tir_attr->transport_domain = priv->sh->td->id;
drivers/net/mlx5/mlx5_flow.c +83 -1
163 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow.h +20 -0
@@ -186,6 +186,9 @@ enum mlx5_feature_name {#define MLX5_FLOW_ITEM_INNER_FLEX (UINT64_C(1) << 38)#define MLX5_FLOW_ITEM_FLEX_TUNNEL (UINT64_C(1) << 39) +/* ESP item */ +#define MLX5_FLOW_ITEM_ESP (UINT64_C(1) << 40) +/* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ (MLX5_FLOW_LAYER_OUTER_L3_IPV4 | MLX5_FLOW_LAYER_OUTER_L3_IPV6)
@@ -1185,6 +1188,16 @@ struct rte_flow_template_table { (MLX5_RSS_HASH_IPV6 | IBV_RX_HASH_SRC_PORT_TCP)#define MLX5_RSS_HASH_IPV6_TCP_DST_ONLY \ (MLX5_RSS_HASH_IPV6 | IBV_RX_HASH_DST_PORT_TCP) + +#ifndef HAVE_IBV_RX_HASH_IPSEC_SPI +#define IBV_RX_HASH_IPSEC_SPI (1U << 8) +#endif + +#define MLX5_RSS_HASH_ESP_SPI IBV_RX_HASH_IPSEC_SPI +#define MLX5_RSS_HASH_IPV4_ESP (MLX5_RSS_HASH_IPV4 | \ + MLX5_RSS_HASH_ESP_SPI) +#define MLX5_RSS_HASH_IPV6_ESP (MLX5_RSS_HASH_IPV6 | \ + MLX5_RSS_HASH_ESP_SPI)#define MLX5_RSS_HASH_NONE 0ULL
@@ -1200,9 +1213,12 @@ static const uint64_t mlx5_rss_hash_fields[] = {MLX5_RSS_HASH_IPV4 ,MLX5_RSS_HASH_IPV4_TCP ,MLX5_RSS_HASH_IPV4_UDP , +MLX5_RSS_HASH_IPV4_ESP ,MLX5_RSS_HASH_IPV6 ,MLX5_RSS_HASH_IPV6_TCP ,MLX5_RSS_HASH_IPV6_UDP , +MLX5_RSS_HASH_IPV6_ESP , +MLX5_RSS_HASH_ESP_SPI , MLX5_RSS_HASH_NONE, };
@@ -1812,6 +1828,10 @@ int mlx5_flow_validate_item_tcp(const struct rte_flow_item *item,uint8_t target_protocol, const struct rte_flow_item_tcp *flow_mask, struct rte_flow_error *error); +int mlx5_flow_validate_item_esp (const struct rte_flow_item *item, +uint64_t item_flags, +uint8_t target_protocol, +struct rte_flow_error *error); int mlx5_flow_validate_item_udp(const struct rte_flow_item *item, uint64_t item_flags, uint8_t target_protocol,
drivers/net/mlx5/mlx5_flow_dv.c +87 -1
129 lines not on this page · raw message
Signature · 1 line
2.21.0
Hi Raja,
Quoted 21 lines
-----Original Message-----
From: Raja Zidane <[email protected]>
Sent: Monday, April 4, 2022 4:04 PM
To: [email protected]
Cc: Matan Azrad <[email protected]>
Subject: [PATCH] net/mlx5: support ESP SPI match and RSS hashIn packets with ESP header, The Inner IP will be encrypted, and its fields cannot be used for RSS hashing. So, ESP packets can be hashed only by the outer IP layer.
So, when using RSS on ESP packets, hashing may not be efficient, because the fields used by the hash functions are only the Outer IPs, causing all traffic belonging to all tunnels between a given pair of GWs to land on one core.
Adding the SPI hash field can extend the spreading of IPSec packets.support matching on ESP SPI.
Support RSS on ESP header by hashing SPI field.Signed-off-by Raja Zidane <[email protected]>Acked-by Matan Azrad <[email protected]>
Quoted 10 lines
--- drivers/common/mlx5/linux/meson.build | 2 + drivers/common/mlx5/mlx5_prm.h | 5 +- drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_devx.c | 4 +- drivers/net/mlx5/mlx5_flow.c | 84 ++++++++++++++++++++++++- drivers/net/mlx5/mlx5_flow.h | 20 ++++++ drivers/net/mlx5/mlx5_flow_dv.c | 88 ++++++++++++++++++++++++++- 7 files changed, 200 insertions(+), 5 deletions(-)
It seems that you missed updating two parts, 1- release notes as it's a new feature being added. 2- features list in mlx5.ini
devtools/check-doc-vs-code.shrte_flow doc out of sync for mlx5
item esp
Kindest regards,
Raslan Darawsheh
PATCH v2 1/1 net/mlx5: support ESP SPI match and RSS hash
In reply to Raja ZidaneIn packets with ESP header, The Inner IP will be encrypted, and its fields cannot be used for RSS hashing. So, ESP packets can be hashed only by the outer IP layer.
So, when using RSS on ESP packets, hashing may not be efficient, because the fields used by the hash functions are only the Outer IPs, causing all traffic belonging to all tunnels between a given pair of GWs to land on one core.
Adding the SPI hash field can extend the spreading of IPSec packets.
support matching on ESP SPI.
Support RSS on ESP header by hashing SPI field.
---
V2: added doc.
doc/guides/nics/features/mlx5.ini | 1 + doc/guides/rel_notes/release_22_07.rst | 2 + drivers/common/mlx5/linux/meson.build | 2 + drivers/common/mlx5/mlx5_prm.h | 5 +- drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_devx.c | 4 +- drivers/net/mlx5/mlx5_flow.c | 84 +++++++++++++++++++++++- drivers/net/mlx5/mlx5_flow.h | 20 ++++++ drivers/net/mlx5/mlx5_flow_dv.c | 88 +++++++++++++++++++++++++- 9 files changed, 203 insertions(+), 5 deletions(-)
doc/guides/nics/features/mlx5.ini +1 -0
@@ -56,6 +56,7 @@ Usage doc = Y [rte_flow items] conntrack = Y ecpri = Y +esp = Y eth = Y flex = Y geneve = Y
doc/guides/rel_notes/release_22_07.rst +2 -0
@@ -59,6 +59,8 @@ New Features * Added support for set promiscuous mode in Windows. * Added support for set and get MTU in Windows. + * Added new item (ESP). + * Added RSS for ESP. Removed Items -------------
drivers/common/mlx5/linux/meson.build +2 -0
@@ -72,6 +72,8 @@ has_member_args = [ # [ "MACRO to define if found", "header for the search", # "symbol to search" ] has_sym_args = [ + [ 'HAVE_IBV_RX_HASH_IPSEC_SPI', 'infiniband/verbs.h', + 'IBV_RX_HASH_IPSEC_SPI' ], [ 'HAVE_IBV_RELAXED_ORDERING', 'infiniband/verbs.h', 'IBV_ACCESS_RELAXED_ORDERING ' ], [ 'HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT', 'infiniband/mlx5dv.h',
drivers/common/mlx5/mlx5_prm.h +4 -1
@@ -899,7 +899,10 @@ struct mlx5_ifc_fte_match_set_misc_bits {u8 reserved_at_120[0xa ];u8 geneve_opt_len[0x6 ];u8 geneve_protocol_type[0x10 ]; -u8 reserved_at_140[0xc0 ]; +u8 reserved_at_140[0x20 ]; +u8 inner_esp_spi[0x20 ]; +u8 outer_esp_spi[0x20 ]; +u8 reserved_at_1a0[0x60 ]; };struct mlx5_ifc_ipv4_layout_bits {
drivers/net/mlx5/mlx5_defs.h +1 -1
@@ -90,7 +90,7 @@/* Supported RSS */ #define MLX5_RSS_HF_MASK (~(RTE_ETH_RSS_IP | RTE_ETH_RSS_UDP | RTE_ETH_RSS_TCP | \ - MLX5_RSS_SRC_DST_ONLY)) + MLX5_RSS_SRC_DST_ONLY | RTE_ETH_RSS_ESP))/* Timeout in seconds to get a valid link status. */ #define MLX5_LINK_STATUS_TIMEOUT 10
drivers/net/mlx5/mlx5_devx.c +3 -1
@@ -765,7 +765,9 @@ mlx5_devx_tir_attr_set(struct rte_eth_dev *dev, const uint8_t *rss_key, (!!(hash_fields &MLX5_L4_SRC_IBV_RX_HASH )) <<MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_SPORT | (!!(hash_fields &MLX5_L4_DST_IBV_RX_HASH )) << -MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT ; +MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT | + (!!(hash_fields &IBV_RX_HASH_IPSEC_SPI )) << +MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_IPSEC_SPI ; }if (is_hairpin) tir_attr->transport_domain = priv->sh->td->id;
drivers/net/mlx5/mlx5_flow.c +83 -1
163 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow.h +20 -0
@@ -186,6 +186,9 @@ enum mlx5_feature_name {#define MLX5_FLOW_ITEM_INNER_FLEX (UINT64_C(1) << 38)#define MLX5_FLOW_ITEM_FLEX_TUNNEL (UINT64_C(1) << 39) +/* ESP item */ +#define MLX5_FLOW_ITEM_ESP (UINT64_C(1) << 40) +/* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ (MLX5_FLOW_LAYER_OUTER_L3_IPV4 | MLX5_FLOW_LAYER_OUTER_L3_IPV6)
@@ -1185,6 +1188,16 @@ struct rte_flow_template_table { (MLX5_RSS_HASH_IPV6 | IBV_RX_HASH_SRC_PORT_TCP)#define MLX5_RSS_HASH_IPV6_TCP_DST_ONLY \ (MLX5_RSS_HASH_IPV6 | IBV_RX_HASH_DST_PORT_TCP) + +#ifndef HAVE_IBV_RX_HASH_IPSEC_SPI +#define IBV_RX_HASH_IPSEC_SPI (1U << 8) +#endif + +#define MLX5_RSS_HASH_ESP_SPI IBV_RX_HASH_IPSEC_SPI +#define MLX5_RSS_HASH_IPV4_ESP (MLX5_RSS_HASH_IPV4 | \ + MLX5_RSS_HASH_ESP_SPI) +#define MLX5_RSS_HASH_IPV6_ESP (MLX5_RSS_HASH_IPV6 | \ + MLX5_RSS_HASH_ESP_SPI)#define MLX5_RSS_HASH_NONE 0ULL
@@ -1200,9 +1213,12 @@ static const uint64_t mlx5_rss_hash_fields[] = {MLX5_RSS_HASH_IPV4 ,MLX5_RSS_HASH_IPV4_TCP ,MLX5_RSS_HASH_IPV4_UDP , +MLX5_RSS_HASH_IPV4_ESP ,MLX5_RSS_HASH_IPV6 ,MLX5_RSS_HASH_IPV6_TCP ,MLX5_RSS_HASH_IPV6_UDP , +MLX5_RSS_HASH_IPV6_ESP , +MLX5_RSS_HASH_ESP_SPI , MLX5_RSS_HASH_NONE, };
@@ -1812,6 +1828,10 @@ int mlx5_flow_validate_item_tcp(const struct rte_flow_item *item,uint8_t target_protocol, const struct rte_flow_item_tcp *flow_mask, struct rte_flow_error *error); +int mlx5_flow_validate_item_esp (const struct rte_flow_item *item, +uint64_t item_flags, +uint8_t target_protocol, +struct rte_flow_error *error); int mlx5_flow_validate_item_udp(const struct rte_flow_item *item, uint64_t item_flags, uint8_t target_protocol,
drivers/net/mlx5/mlx5_flow_dv.c +87 -1
129 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH v3 1/1 net/mlx5: support ESP SPI match and RSS hash
In packets with ESP header, The Inner IP will be encrypted, and its fields cannot be used for RSS hashing. So, ESP packets can be hashed only by the outer IP layer.
So, when using RSS on ESP packets, hashing may not be efficient, because the fields used by the hash functions are only the Outer IPs, causing all traffic belonging to all tunnels between a given pair of GWs to land on one core.
Adding the SPI hash field can extend the spreading of IPSec packets.
support matching on ESP SPI.
Support RSS on ESP header by hashing SPI field.
---
V2: added doc.
V3: fixed checkpatch.
doc/guides/nics/features/mlx5.ini | 1 + doc/guides/rel_notes/release_22_07.rst | 2 + drivers/common/mlx5/linux/meson.build | 2 + drivers/common/mlx5/mlx5_prm.h | 5 +- drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_devx.c | 4 +- drivers/net/mlx5/mlx5_flow.c | 84 +++++++++++++++++++++++- drivers/net/mlx5/mlx5_flow.h | 20 ++++++ drivers/net/mlx5/mlx5_flow_dv.c | 88 +++++++++++++++++++++++++- 9 files changed, 203 insertions(+), 5 deletions(-)
doc/guides/nics/features/mlx5.ini +1 -0
@@ -56,6 +56,7 @@ Usage doc = Y [rte_flow items] conntrack = Y ecpri = Y +esp = Y eth = Y flex = Y geneve = Y
doc/guides/rel_notes/release_22_07.rst +2 -0
@@ -59,6 +59,8 @@ New Features * Added support for set promiscuous mode in Windows. * Added support for set and get MTU in Windows. + * Added new item (ESP). + * Added RSS for ESP. Removed Items -------------
drivers/common/mlx5/linux/meson.build +2 -0
@@ -72,6 +72,8 @@ has_member_args = [ # [ "MACRO to define if found", "header for the search", # "symbol to search" ] has_sym_args = [ + [ 'HAVE_IBV_RX_HASH_IPSEC_SPI', 'infiniband/verbs.h', + 'IBV_RX_HASH_IPSEC_SPI' ], [ 'HAVE_IBV_RELAXED_ORDERING', 'infiniband/verbs.h', 'IBV_ACCESS_RELAXED_ORDERING ' ], [ 'HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT', 'infiniband/mlx5dv.h',
drivers/common/mlx5/mlx5_prm.h +4 -1
@@ -899,7 +899,10 @@ struct mlx5_ifc_fte_match_set_misc_bits {u8 reserved_at_120[0xa ];u8 geneve_opt_len[0x6 ];u8 geneve_protocol_type[0x10 ]; -u8 reserved_at_140[0xc0 ]; +u8 reserved_at_140[0x20 ]; +u8 inner_esp_spi[0x20 ]; +u8 outer_esp_spi[0x20 ]; +u8 reserved_at_1a0[0x60 ]; };struct mlx5_ifc_ipv4_layout_bits {
drivers/net/mlx5/mlx5_defs.h +1 -1
@@ -90,7 +90,7 @@/* Supported RSS */ #define MLX5_RSS_HF_MASK (~(RTE_ETH_RSS_IP | RTE_ETH_RSS_UDP | RTE_ETH_RSS_TCP | \ - MLX5_RSS_SRC_DST_ONLY)) + MLX5_RSS_SRC_DST_ONLY | RTE_ETH_RSS_ESP))/* Timeout in seconds to get a valid link status. */ #define MLX5_LINK_STATUS_TIMEOUT 10
drivers/net/mlx5/mlx5_devx.c +3 -1
@@ -765,7 +765,9 @@ mlx5_devx_tir_attr_set(struct rte_eth_dev *dev, const uint8_t *rss_key, (!!(hash_fields &MLX5_L4_SRC_IBV_RX_HASH )) <<MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_SPORT | (!!(hash_fields &MLX5_L4_DST_IBV_RX_HASH )) << -MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT ; +MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT | + (!!(hash_fields &IBV_RX_HASH_IPSEC_SPI )) << +MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_IPSEC_SPI ; }if (is_hairpin) tir_attr->transport_domain = priv->sh->td->id;
drivers/net/mlx5/mlx5_flow.c +83 -1
163 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow.h +20 -0
@@ -186,6 +186,9 @@ enum mlx5_feature_name {#define MLX5_FLOW_ITEM_INNER_FLEX (UINT64_C(1) << 38)#define MLX5_FLOW_ITEM_FLEX_TUNNEL (UINT64_C(1) << 39) +/* ESP item */ +#define MLX5_FLOW_ITEM_ESP (UINT64_C(1) << 40) +/* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ (MLX5_FLOW_LAYER_OUTER_L3_IPV4 | MLX5_FLOW_LAYER_OUTER_L3_IPV6)
@@ -1185,6 +1188,16 @@ struct rte_flow_template_table { (MLX5_RSS_HASH_IPV6 | IBV_RX_HASH_SRC_PORT_TCP)#define MLX5_RSS_HASH_IPV6_TCP_DST_ONLY \ (MLX5_RSS_HASH_IPV6 | IBV_RX_HASH_DST_PORT_TCP) + +#ifndef HAVE_IBV_RX_HASH_IPSEC_SPI +#define IBV_RX_HASH_IPSEC_SPI (1U << 8) +#endif + +#define MLX5_RSS_HASH_ESP_SPI IBV_RX_HASH_IPSEC_SPI +#define MLX5_RSS_HASH_IPV4_ESP (MLX5_RSS_HASH_IPV4 | \ + MLX5_RSS_HASH_ESP_SPI) +#define MLX5_RSS_HASH_IPV6_ESP (MLX5_RSS_HASH_IPV6 | \ + MLX5_RSS_HASH_ESP_SPI)#define MLX5_RSS_HASH_NONE 0ULL
@@ -1200,9 +1213,12 @@ static const uint64_t mlx5_rss_hash_fields[] = {MLX5_RSS_HASH_IPV4 ,MLX5_RSS_HASH_IPV4_TCP ,MLX5_RSS_HASH_IPV4_UDP , +MLX5_RSS_HASH_IPV4_ESP ,MLX5_RSS_HASH_IPV6 ,MLX5_RSS_HASH_IPV6_TCP ,MLX5_RSS_HASH_IPV6_UDP , +MLX5_RSS_HASH_IPV6_ESP , +MLX5_RSS_HASH_ESP_SPI , MLX5_RSS_HASH_NONE, };
@@ -1812,6 +1828,10 @@ int mlx5_flow_validate_item_tcp(const struct rte_flow_item *item,uint8_t target_protocol, const struct rte_flow_item_tcp *flow_mask, struct rte_flow_error *error); +int mlx5_flow_validate_item_esp (const struct rte_flow_item *item, +uint64_t item_flags, +uint8_t target_protocol, +struct rte_flow_error *error); int mlx5_flow_validate_item_udp(const struct rte_flow_item *item, uint64_t item_flags, uint8_t target_protocol,
drivers/net/mlx5/mlx5_flow_dv.c +87 -1
129 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH v4 1/1 net/mlx5: support ESP SPI match and RSS hash
In packets with ESP header, The Inner IP will be encrypted, and its fields cannot be used for RSS hashing. So, ESP packets can be hashed only by the outer IP layer.
So, when using RSS on ESP packets, hashing may not be efficient, because the fields used by the hash functions are only the Outer IPs, causing all traffic belonging to all tunnels between a given pair of GWs to land on one core.
Adding the SPI hash field can extend the spreading of IPSec packets.
support matching on ESP SPI.
Support RSS on ESP header by hashing SPI field.
---
V2: added doc.
V3: fixed checkpatch.
V4: rebase
doc/guides/nics/features/mlx5.ini | 1 + doc/guides/rel_notes/release_22_07.rst | 2 + drivers/common/mlx5/linux/meson.build | 2 + drivers/common/mlx5/mlx5_prm.h | 5 +- drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_devx.c | 4 +- drivers/net/mlx5/mlx5_flow.c | 84 +++++++++++++++++++++++- drivers/net/mlx5/mlx5_flow.h | 20 ++++++ drivers/net/mlx5/mlx5_flow_dv.c | 88 +++++++++++++++++++++++++- 9 files changed, 203 insertions(+), 5 deletions(-)
doc/guides/nics/features/mlx5.ini +1 -0
@@ -56,6 +56,7 @@ Usage doc = Y [rte_flow items] conntrack = Y ecpri = Y +esp = Y eth = Y flex = Y geneve = Y
doc/guides/rel_notes/release_22_07.rst +2 -0
@@ -64,6 +64,8 @@ New Features * Added support for promiscuous mode on Windows. * Added support for MTU on Windows. + * Added new item (ESP). + * Added RSS for ESP. Removed Items
drivers/common/mlx5/linux/meson.build +2 -0
@@ -72,6 +72,8 @@ has_member_args = [ # [ "MACRO to define if found", "header for the search", # "symbol to search" ] has_sym_args = [ + [ 'HAVE_IBV_RX_HASH_IPSEC_SPI', 'infiniband/verbs.h', + 'IBV_RX_HASH_IPSEC_SPI' ], [ 'HAVE_IBV_RELAXED_ORDERING', 'infiniband/verbs.h', 'IBV_ACCESS_RELAXED_ORDERING ' ], [ 'HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT', 'infiniband/mlx5dv.h',
drivers/common/mlx5/mlx5_prm.h +4 -1
@@ -899,7 +899,10 @@ struct mlx5_ifc_fte_match_set_misc_bits {u8 reserved_at_120[0xa ];u8 geneve_opt_len[0x6 ];u8 geneve_protocol_type[0x10 ]; -u8 reserved_at_140[0xc0 ]; +u8 reserved_at_140[0x20 ]; +u8 inner_esp_spi[0x20 ]; +u8 outer_esp_spi[0x20 ]; +u8 reserved_at_1a0[0x60 ]; };struct mlx5_ifc_ipv4_layout_bits {
drivers/net/mlx5/mlx5_defs.h +1 -1
@@ -90,7 +90,7 @@/* Supported RSS */ #define MLX5_RSS_HF_MASK (~(RTE_ETH_RSS_IP | RTE_ETH_RSS_UDP | RTE_ETH_RSS_TCP | \ - MLX5_RSS_SRC_DST_ONLY)) + MLX5_RSS_SRC_DST_ONLY | RTE_ETH_RSS_ESP))/* Timeout in seconds to get a valid link status. */ #define MLX5_LINK_STATUS_TIMEOUT 10
drivers/net/mlx5/mlx5_devx.c +3 -1
@@ -765,7 +765,9 @@ mlx5_devx_tir_attr_set(struct rte_eth_dev *dev, const uint8_t *rss_key, (!!(hash_fields &MLX5_L4_SRC_IBV_RX_HASH )) <<MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_SPORT | (!!(hash_fields &MLX5_L4_DST_IBV_RX_HASH )) << -MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT ; +MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT | + (!!(hash_fields &IBV_RX_HASH_IPSEC_SPI )) << +MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_IPSEC_SPI ; }if (is_hairpin) tir_attr->transport_domain = priv->sh->td->id;
drivers/net/mlx5/mlx5_flow.c +83 -1
163 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow.h +20 -0
@@ -186,6 +186,9 @@ enum mlx5_feature_name {#define MLX5_FLOW_ITEM_INNER_FLEX (UINT64_C(1) << 38)#define MLX5_FLOW_ITEM_FLEX_TUNNEL (UINT64_C(1) << 39) +/* ESP item */ +#define MLX5_FLOW_ITEM_ESP (UINT64_C(1) << 40) +/* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ (MLX5_FLOW_LAYER_OUTER_L3_IPV4 | MLX5_FLOW_LAYER_OUTER_L3_IPV6)
@@ -1185,6 +1188,16 @@ struct rte_flow_template_table { (MLX5_RSS_HASH_IPV6 | IBV_RX_HASH_SRC_PORT_TCP)#define MLX5_RSS_HASH_IPV6_TCP_DST_ONLY \ (MLX5_RSS_HASH_IPV6 | IBV_RX_HASH_DST_PORT_TCP) + +#ifndef HAVE_IBV_RX_HASH_IPSEC_SPI +#define IBV_RX_HASH_IPSEC_SPI (1U << 8) +#endif + +#define MLX5_RSS_HASH_ESP_SPI IBV_RX_HASH_IPSEC_SPI +#define MLX5_RSS_HASH_IPV4_ESP (MLX5_RSS_HASH_IPV4 | \ + MLX5_RSS_HASH_ESP_SPI) +#define MLX5_RSS_HASH_IPV6_ESP (MLX5_RSS_HASH_IPV6 | \ + MLX5_RSS_HASH_ESP_SPI)#define MLX5_RSS_HASH_NONE 0ULL
@@ -1200,9 +1213,12 @@ static const uint64_t mlx5_rss_hash_fields[] = {MLX5_RSS_HASH_IPV4 ,MLX5_RSS_HASH_IPV4_TCP ,MLX5_RSS_HASH_IPV4_UDP , +MLX5_RSS_HASH_IPV4_ESP ,MLX5_RSS_HASH_IPV6 ,MLX5_RSS_HASH_IPV6_TCP ,MLX5_RSS_HASH_IPV6_UDP , +MLX5_RSS_HASH_IPV6_ESP , +MLX5_RSS_HASH_ESP_SPI , MLX5_RSS_HASH_NONE, };
@@ -1812,6 +1828,10 @@ int mlx5_flow_validate_item_tcp(const struct rte_flow_item *item,uint8_t target_protocol, const struct rte_flow_item_tcp *flow_mask, struct rte_flow_error *error); +int mlx5_flow_validate_item_esp (const struct rte_flow_item *item, +uint64_t item_flags, +uint8_t target_protocol, +struct rte_flow_error *error); int mlx5_flow_validate_item_udp(const struct rte_flow_item *item, uint64_t item_flags, uint8_t target_protocol,
drivers/net/mlx5/mlx5_flow_dv.c +87 -1
129 lines not on this page · raw message
Signature · 1 line
2.21.0
From: Raja Zidane
Quoted 8 lines
In packets with ESP header, The Inner IP will be encrypted, and its fields cannot be used for RSS hashing. So, ESP packets can be hashed only by the outer IP layer.
So, when using RSS on ESP packets, hashing may not be efficient, because the fields used by the hash functions are only the Outer IPs, causing all traffic belonging to all tunnels between a given pair of GWs to land on one core. Adding the SPI hash field can extend the spreading of IPSec packets.
Quoted 4 lines
support matching on ESP SPI.
Support RSS on ESP header by hashing SPI field.Signed-off-by Raja Zidane <[email protected]>
PATCH v5 1/1 net/mlx5: support ESP SPI match and RSS hash
In reply to Raja ZidaneIn packets with ESP header, The Inner IP will be encrypted, and its fields cannot be used for RSS hashing. So, ESP packets can be hashed only by the outer IP layer.
So, when using RSS on ESP packets, hashing may not be efficient, because the fields used by the hash functions are only the Outer IPs, causing all traffic belonging to all tunnels between a given pair of GWs to land on one core.
Adding the SPI hash field can extend the spreading of IPSec packets.
support matching on ESP SPI.
Support RSS on ESP header by hashing SPI field.
---
V2: added doc.
V3: fixed checkpatch.
V4: rebase
V5: rebase and doc.
doc/guides/nics/features/mlx5.ini | 1 + doc/guides/nics/mlx5.rst | 1 + doc/guides/rel_notes/release_22_07.rst | 2 + drivers/common/mlx5/linux/meson.build | 2 + drivers/common/mlx5/mlx5_prm.h | 5 +- drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_devx.c | 4 +- drivers/net/mlx5/mlx5_flow.c | 84 +++++++++++++++++++++++- drivers/net/mlx5/mlx5_flow.h | 20 ++++++ drivers/net/mlx5/mlx5_flow_dv.c | 88 +++++++++++++++++++++++++- 10 files changed, 204 insertions(+), 5 deletions(-)
doc/guides/nics/features/mlx5.ini +1 -0
@@ -56,6 +56,7 @@ Usage doc = Y [rte_flow items] conntrack = Y ecpri = Y +esp = Y eth = Y flex = Y geneve = Y
doc/guides/nics/mlx5.rst +1 -0
@@ -81,6 +81,7 @@ Features - Matching on IPv4 Internet Header Length (IHL). - Matching on GTP extension header with raw encap/decap action. - Matching on Geneve TLV option header with raw encap/decap action. +- Matching on ESP header SPI field. - RSS support in sample action. - E-Switch mirroring and jump. - E-Switch mirroring and modify.
doc/guides/rel_notes/release_22_07.rst +2 -0
@@ -69,6 +69,8 @@ New Features * Added support for promiscuous mode on Windows. * Added support for MTU on Windows. + * Added new item (ESP). + * Added RSS for ESP. Removed Items
drivers/common/mlx5/linux/meson.build +2 -0
@@ -72,6 +72,8 @@ has_member_args = [ # [ "MACRO to define if found", "header for the search", # "symbol to search" ] has_sym_args = [ + [ 'HAVE_IBV_RX_HASH_IPSEC_SPI', 'infiniband/verbs.h', + 'IBV_RX_HASH_IPSEC_SPI' ], [ 'HAVE_IBV_RELAXED_ORDERING', 'infiniband/verbs.h', 'IBV_ACCESS_RELAXED_ORDERING ' ], [ 'HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT', 'infiniband/mlx5dv.h',
drivers/common/mlx5/mlx5_prm.h +4 -1
@@ -899,7 +899,10 @@ struct mlx5_ifc_fte_match_set_misc_bits {u8 reserved_at_120[0xa ];u8 geneve_opt_len[0x6 ];u8 geneve_protocol_type[0x10 ]; -u8 reserved_at_140[0xc0 ]; +u8 reserved_at_140[0x20 ]; +u8 inner_esp_spi[0x20 ]; +u8 outer_esp_spi[0x20 ]; +u8 reserved_at_1a0[0x60 ]; };struct mlx5_ifc_ipv4_layout_bits {
drivers/net/mlx5/mlx5_defs.h +1 -1
@@ -90,7 +90,7 @@/* Supported RSS */ #define MLX5_RSS_HF_MASK (~(RTE_ETH_RSS_IP | RTE_ETH_RSS_UDP | RTE_ETH_RSS_TCP | \ - MLX5_RSS_SRC_DST_ONLY)) + MLX5_RSS_SRC_DST_ONLY | RTE_ETH_RSS_ESP))/* Timeout in seconds to get a valid link status. */ #define MLX5_LINK_STATUS_TIMEOUT 10
drivers/net/mlx5/mlx5_devx.c +3 -1
@@ -765,7 +765,9 @@ mlx5_devx_tir_attr_set(struct rte_eth_dev *dev, const uint8_t *rss_key, (!!(hash_fields &MLX5_L4_SRC_IBV_RX_HASH )) <<MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_SPORT | (!!(hash_fields &MLX5_L4_DST_IBV_RX_HASH )) << -MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT ; +MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_L4_DPORT | + (!!(hash_fields &IBV_RX_HASH_IPSEC_SPI )) << +MLX5_RX_HASH_FIELD_SELECT_SELECTED_FIELDS_IPSEC_SPI ; }if (is_hairpin) tir_attr->transport_domain = priv->sh->td->id;
drivers/net/mlx5/mlx5_flow.c +83 -1
163 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow.h +20 -0
@@ -186,6 +186,9 @@ enum mlx5_feature_name {#define MLX5_FLOW_ITEM_INNER_FLEX (UINT64_C(1) << 38)#define MLX5_FLOW_ITEM_FLEX_TUNNEL (UINT64_C(1) << 39) +/* ESP item */ +#define MLX5_FLOW_ITEM_ESP (UINT64_C(1) << 40) +/* Outer Masks. */ #define MLX5_FLOW_LAYER_OUTER_L3 \ (MLX5_FLOW_LAYER_OUTER_L3_IPV4 | MLX5_FLOW_LAYER_OUTER_L3_IPV6)
@@ -1185,6 +1188,16 @@ struct rte_flow_template_table { (MLX5_RSS_HASH_IPV6 | IBV_RX_HASH_SRC_PORT_TCP)#define MLX5_RSS_HASH_IPV6_TCP_DST_ONLY \ (MLX5_RSS_HASH_IPV6 | IBV_RX_HASH_DST_PORT_TCP) + +#ifndef HAVE_IBV_RX_HASH_IPSEC_SPI +#define IBV_RX_HASH_IPSEC_SPI (1U << 8) +#endif + +#define MLX5_RSS_HASH_ESP_SPI IBV_RX_HASH_IPSEC_SPI +#define MLX5_RSS_HASH_IPV4_ESP (MLX5_RSS_HASH_IPV4 | \ + MLX5_RSS_HASH_ESP_SPI) +#define MLX5_RSS_HASH_IPV6_ESP (MLX5_RSS_HASH_IPV6 | \ + MLX5_RSS_HASH_ESP_SPI)#define MLX5_RSS_HASH_NONE 0ULL
@@ -1200,9 +1213,12 @@ static const uint64_t mlx5_rss_hash_fields[] = {MLX5_RSS_HASH_IPV4 ,MLX5_RSS_HASH_IPV4_TCP ,MLX5_RSS_HASH_IPV4_UDP , +MLX5_RSS_HASH_IPV4_ESP ,MLX5_RSS_HASH_IPV6 ,MLX5_RSS_HASH_IPV6_TCP ,MLX5_RSS_HASH_IPV6_UDP , +MLX5_RSS_HASH_IPV6_ESP , +MLX5_RSS_HASH_ESP_SPI , MLX5_RSS_HASH_NONE, };
@@ -1812,6 +1828,10 @@ int mlx5_flow_validate_item_tcp(const struct rte_flow_item *item,uint8_t target_protocol, const struct rte_flow_item_tcp *flow_mask, struct rte_flow_error *error); +int mlx5_flow_validate_item_esp (const struct rte_flow_item *item, +uint64_t item_flags, +uint8_t target_protocol, +struct rte_flow_error *error); int mlx5_flow_validate_item_udp(const struct rte_flow_item *item, uint64_t item_flags, uint8_t target_protocol,
drivers/net/mlx5/mlx5_flow_dv.c +87 -1
129 lines not on this page · raw message
Signature · 1 line
2.21.0
From: Raja Zidane
Quoted 8 lines
In packets with ESP header, The Inner IP will be encrypted, and its fields cannot be used for RSS hashing. So, ESP packets can be hashed only by the outer IP layer.
So, when using RSS on ESP packets, hashing may not be efficient, because the fields used by the hash functions are only the Outer IPs, causing all traffic belonging to all tunnels between a given pair of GWs to land on one core. Adding the SPI hash field can extend the spreading of IPSec packets.
Quoted 4 lines
support matching on ESP SPI.
Support RSS on ESP header by hashing SPI field.Signed-off-by Raja Zidane <[email protected]>
Hi,
Quoted 24 lines
-----Original Message-----
From: Raja Zidane <[email protected]>
Sent: Thursday, May 12, 2022 12:17 PM
To: [email protected]
Cc: Matan Azrad <[email protected]>
Subject: [PATCH V5] net/mlx5: support ESP SPI match and RSS hashIn packets with ESP header, The Inner IP will be encrypted, and its fields cannot be used for RSS hashing. So, ESP packets can be hashed only by the outer IP layer.
So, when using RSS on ESP packets, hashing may not be efficient, because the fields used by the hash functions are only the Outer IPs, causing all traffic belonging to all tunnels between a given pair of GWs to land on one core. Adding the SPI hash field can extend the spreading of IPSec packets.support matching on ESP SPI.
Support RSS on ESP header by hashing SPI field.Signed-off-by Raja Zidane <[email protected]>---
V2: added doc.
V3: fixed checkpatch.
V4: rebase
V5: rebase and doc.
Quoted 12 lines
doc/guides/nics/features/mlx5.ini | 1 + doc/guides/nics/mlx5.rst | 1 + doc/guides/rel_notes/release_22_07.rst | 2 + drivers/common/mlx5/linux/meson.build | 2 + drivers/common/mlx5/mlx5_prm.h | 5 +- drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_devx.c | 4 +- drivers/net/mlx5/mlx5_flow.c | 84 +++++++++++++++++++++++- drivers/net/mlx5/mlx5_flow.h | 20 ++++++ drivers/net/mlx5/mlx5_flow_dv.c | 88 +++++++++++++++++++++++++- 10 files changed, 204 insertions(+), 5 deletions(-)
Patch applied to next-net-mlx,
Kindest regards,
Raslan Darawsheh