Skip to contentSkip to conversation

net/mlx5/hws: support range and partial hash matching

PATCH Superseded Alex Vesker 3 participants 36 messages Jan 31, 2023 – Feb 13, 2023 Network drivers

lore
Share & tools
lore
#0 lore raw

cover net/mlx5/hws: support range and partial hash matching

ConnectX and BlueField devices which support HWS are capable of performing advanced match, range-match and hashing operations on packet headers. This patch-set introduces support for (1) range matching - allows combining range and exact match over provided item fields using spec, last and mask. A useful example for Range and exact match is matching exact IP and port range. This series also introduces (2) partial hash, which allows using the same matcher and performing partial hashing in case of multiple match templates are provided. The matcher create code finds the union and intersection fields to split the hash and match in a way the HW can support. This increases the match templates combinations possible although it may limit the maximum PPS. Since (1) and (2) are not fully support on current HW the implementation is done on top of a HW like FW API using WQEs.

Alex Vesker (16):
net/mlx5/hws: support synchronous drain net/mlx5/hws: matcher remove AT and MT limitation net/mlx5/hws: support GTA WQE write using FW command net/mlx5/hws: add capability query for gen wqe command net/mlx5/hws: align RTC create command with PRM format net/mlx5/hws: add send FW match STE using gen WQE net/mlx5/hws: add send FW range STE WQE net/mlx5/hws: move matcher size check to function net/mlx5/hws: support range match net/mlx5/hws: redesign definer create net/mlx5/hws: support partial hash net/mlx5/hws: add range definer creation support net/mlx5/hws: add FW WQE rule creation logic net/mlx5/hws: add debug dump support for range and hash net/mlx5/hws: rename pattern cache object net/mlx5/hws: cache definer for reuse

 drivers/common/mlx5/mlx5_prm.h         |  53 +-
 drivers/net/mlx5/hws/mlx5dr.h          |   6 +-
 drivers/net/mlx5/hws/mlx5dr_cmd.c      |  72 ++-
 drivers/net/mlx5/hws/mlx5dr_cmd.h      |  27 +-
 drivers/net/mlx5/hws/mlx5dr_context.c  |  12 +-
 drivers/net/mlx5/hws/mlx5dr_context.h  |   1 +
 drivers/net/mlx5/hws/mlx5dr_debug.c    |  41 +-
 drivers/net/mlx5/hws/mlx5dr_debug.h    |   4 +-
 drivers/net/mlx5/hws/mlx5dr_definer.c  | 750 +++++++++++++++++++++----
 drivers/net/mlx5/hws/mlx5dr_definer.h  |  44 +-
 drivers/net/mlx5/hws/mlx5dr_internal.h |   2 +-
 drivers/net/mlx5/hws/mlx5dr_matcher.c  | 279 ++++++---
 drivers/net/mlx5/hws/mlx5dr_matcher.h  |  43 +-
 drivers/net/mlx5/hws/mlx5dr_pat_arg.c  |  49 +-
 drivers/net/mlx5/hws/mlx5dr_pat_arg.h  |   6 +-
 drivers/net/mlx5/hws/mlx5dr_rule.c     | 193 ++++++-
 drivers/net/mlx5/hws/mlx5dr_rule.h     |   2 +
 drivers/net/mlx5/hws/mlx5dr_send.c     | 169 +++++-
 drivers/net/mlx5/hws/mlx5dr_send.h     |  24 +-
 drivers/net/mlx5/mlx5_flow_hw.c        |   2 +-
 20 files changed, 1475 insertions(+), 304 deletions(-)
Signature · 1 line
2.18.1
#1 lore raw

PATCH 1/16 net/mlx5/hws: support synchronous drain

Until now we supported asynchronous drain, triggering the queue to start the drain, now we added support for synchronous which assures all the work was processed on the queue.

This is useful when working over a FW command and HW queue in parallel sending arguments over the HW queue and match over the FW command which requires synchronization.

This also fixes an issue with shared arguments send that require more than one WQE.

---
 drivers/net/mlx5/hws/mlx5dr.h         |  6 ++++--
 drivers/net/mlx5/hws/mlx5dr_pat_arg.c | 27 ++++-----------------------
 drivers/net/mlx5/hws/mlx5dr_send.c    | 16 ++++++++++++++--
 drivers/net/mlx5/hws/mlx5dr_send.h    |  5 +++++
 drivers/net/mlx5/mlx5_flow_hw.c       |  2 +-
 5 files changed, 28 insertions(+), 28 deletions(-)
drivers/net/mlx5/hws/mlx5dr.h +4 -2
index b3b2bf34f2..2b02884dc3 100644
@@ -86,8 +86,10 @@ enum mlx5dr_match_template_flags {
 };
 
 enum mlx5dr_send_queue_actions {
-	/* Start executing all pending queued rules and write to HW */
-	MLX5DR_SEND_QUEUE_ACTION_DRAIN = 1 << 0,
+	/* Start executing all pending queued rules */
+	MLX5DR_SEND_QUEUE_ACTION_DRAIN_ASYNC = 1 << 0,
+	/* Start executing all pending queued rules wait till completion */
+	MLX5DR_SEND_QUEUE_ACTION_DRAIN_SYNC = 1 << 1,
 };
 
 struct mlx5dr_context_attr {
drivers/net/mlx5/hws/mlx5dr_pat_arg.c +4 -23
index df451f1ae0..152025d302 100644
@@ -306,27 +306,6 @@ void mlx5dr_arg_decapl3_write(struct mlx5dr_send_engine *queue,
 	mlx5dr_send_engine_post_end(&ctrl, &send_attr);
 }
 
-static int
-mlx5dr_arg_poll_for_comp(struct mlx5dr_context *ctx, uint16_t queue_id)
-{
-	struct rte_flow_op_result comp[1];
-	int ret;
-
-	while (true) {
-		ret = mlx5dr_send_queue_poll(ctx, queue_id, comp, 1);
-		if (ret) {
-			if (ret < 0) {
-				DR_LOG(ERR, "Failed mlx5dr_send_queue_poll");
-			} else if (comp[0].status == RTE_FLOW_OP_ERROR) {
-				DR_LOG(ERR, "Got comp with error");
-				rte_errno = ENOENT;
-			}
-			break;
-		}
-	}
-	return (ret == 1 ? 0 : ret);
-}
-
 void mlx5dr_arg_write(struct mlx5dr_send_engine *queue,
 		      void *comp_data,
 		      uint32_t arg_idx,
@@ -388,9 +367,11 @@ int mlx5dr_arg_write_inline_arg_data(struct mlx5dr_context *ctx,
 	mlx5dr_send_engine_flush_queue(queue);
 
 	/* Poll for completion */
-	ret = mlx5dr_arg_poll_for_comp(ctx, ctx->queues - 1);
+	ret = mlx5dr_send_queue_action(ctx, ctx->queues - 1,
+				       MLX5DR_SEND_QUEUE_ACTION_DRAIN_SYNC);
+
 	if (ret)
-		DR_LOG(ERR, "Failed to get completions for shared action");
+		DR_LOG(ERR, "Failed to drain arg queue");
 
 	pthread_spin_unlock(&ctx->ctrl_lock);
 
drivers/net/mlx5/hws/mlx5dr_send.c +14 -2
index 5c8bbe6fc6..a507e5f626 100644
@@ -830,18 +830,30 @@ int mlx5dr_send_queue_action(struct mlx5dr_context *ctx,
 {
 	struct mlx5dr_send_ring_sq *send_sq;
 	struct mlx5dr_send_engine *queue;
+	bool wait_comp = false;
+	int64_t polled = 0;
 
 	queue = &ctx->send_queue[queue_id];
 	send_sq = &queue->send_ring->send_sq;
 
-	if (actions == MLX5DR_SEND_QUEUE_ACTION_DRAIN) {
+	switch (actions) {
+	case MLX5DR_SEND_QUEUE_ACTION_DRAIN_SYNC:
+		wait_comp = true;
+		/* FALLTHROUGH */
+	case MLX5DR_SEND_QUEUE_ACTION_DRAIN_ASYNC:
 		if (send_sq->head_dep_idx != send_sq->tail_dep_idx)
 			/* Send dependent WQEs to drain the queue */
 			mlx5dr_send_all_dep_wqe(queue);
 		else
 			/* Signal on the last posted WQE */
 			mlx5dr_send_engine_flush_queue(queue);
-	} else {
+
+		/* Poll queue until empty */
+		while (wait_comp && !mlx5dr_send_engine_empty(queue))
+			mlx5dr_send_engine_poll_cqs(queue, NULL, &polled, 0);
+
+		break;
+	default:
 		rte_errno = -EINVAL;
 		return rte_errno;
 	}
drivers/net/mlx5/hws/mlx5dr_send.h +5 -0
index 8d4769495d..fcddcc6366 100644
@@ -240,6 +240,11 @@ void mlx5dr_send_ste(struct mlx5dr_send_engine *queue,
 
 void mlx5dr_send_engine_flush_queue(struct mlx5dr_send_engine *queue);
 
+static inline bool mlx5dr_send_engine_empty(struct mlx5dr_send_engine *queue)
+{
+	return (queue->send_ring->send_sq.cur_post == queue->send_ring->send_cq.poll_wqe);
+}
+
 static inline bool mlx5dr_send_engine_full(struct mlx5dr_send_engine *queue)
 {
 	return queue->used_entries >= queue->th_entries;
drivers/net/mlx5/mlx5_flow_hw.c +1 -1
index 20c71ff7f0..7e87d589cb 100644
@@ -2851,7 +2851,7 @@ flow_hw_push(struct rte_eth_dev *dev,
 
 	__flow_hw_push_action(dev, queue);
 	ret = mlx5dr_send_queue_action(priv->dr_ctx, queue,
-				       MLX5DR_SEND_QUEUE_ACTION_DRAIN);
+				       MLX5DR_SEND_QUEUE_ACTION_DRAIN_ASYNC);
 	if (ret) {
 		rte_flow_error_set(error, rte_errno,
 				   RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
Signature · 1 line
2.18.1
#2 lore raw

PATCH 2/16 net/mlx5/hws: matcher remove AT and MT limitation

In reply to Alex Vesker

The action and match templates were stored on the matcher in a fixed size array to reduce cache misses and reuse template calculations. This approuch introduced two issues:
-limitation of fixed array
-definer is bindind to match template and cannot be used with
union definer since the layout is fixed

---
 drivers/net/mlx5/hws/mlx5dr_debug.c   |   4 +-
 drivers/net/mlx5/hws/mlx5dr_matcher.c | 109 ++++++++++++++++----------
 drivers/net/mlx5/hws/mlx5dr_matcher.h |   8 +-
 drivers/net/mlx5/hws/mlx5dr_rule.c    |  15 ++--
 4 files changed, 78 insertions(+), 58 deletions(-)
drivers/net/mlx5/hws/mlx5dr_debug.c +2 -2
index 0815327b18..9199ec16e0 100644
@@ -92,7 +92,7 @@ mlx5dr_debug_dump_matcher_match_template(FILE *f, struct mlx5dr_matcher *matcher
 	int i, ret;
 
 	for (i = 0; i < matcher->num_of_mt; i++) {
-		struct mlx5dr_match_template *mt = matcher->mt[i];
+		struct mlx5dr_match_template *mt = &matcher->mt[i];
 
 		ret = fprintf(f, "%d,0x%" PRIx64 ",0x%" PRIx64 ",%d,%d\n",
 			      MLX5DR_DEBUG_RES_TYPE_MATCHER_MATCH_TEMPLATE,
@@ -123,7 +123,7 @@ mlx5dr_debug_dump_matcher_action_template(FILE *f, struct mlx5dr_matcher *matche
 	int i, j, ret;
 
 	for (i = 0; i < matcher->num_of_at; i++) {
-		struct mlx5dr_action_template *at = matcher->at[i];
+		struct mlx5dr_action_template *at = &matcher->at[i];
 
 		ret = fprintf(f, "%d,0x%" PRIx64 ",0x%" PRIx64 ",%d,%d,%d",
 			      MLX5DR_DEBUG_RES_TYPE_MATCHER_ACTION_TEMPLATE,
drivers/net/mlx5/hws/mlx5dr_matcher.c +66 -43

230 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_matcher.h +2 -6
index 2bebc4bcce..b957f5ea4b 100644
@@ -6,12 +6,8 @@
 #define MLX5DR_MATCHER_H_
 
 /* Max supported match template */
-#define MLX5DR_MATCHER_MAX_MT 2
 #define MLX5DR_MATCHER_MAX_MT_ROOT 1
 
-/* Max supported action template */
-#define MLX5DR_MATCHER_MAX_AT 4
-
 /* We calculated that concatenating a collision table to the main table with
  * 3% of the main table rows will be enough resources for high insertion
  * success probability.
@@ -59,9 +55,9 @@ struct mlx5dr_matcher {
 	struct mlx5dr_table *tbl;
 	struct mlx5dr_matcher_attr attr;
 	struct mlx5dv_flow_matcher *dv_matcher;
-	struct mlx5dr_match_template *mt[MLX5DR_MATCHER_MAX_MT];
+	struct mlx5dr_match_template *mt;
 	uint8_t num_of_mt;
-	struct mlx5dr_action_template *at[MLX5DR_MATCHER_MAX_AT];
+	struct mlx5dr_action_template *at;
 	uint8_t num_of_at;
 	struct mlx5dr_devx_obj *end_ft;
 	struct mlx5dr_matcher *col_matcher;
drivers/net/mlx5/hws/mlx5dr_rule.c +8 -7
index 60a82c022f..f5a0c46315 100644
@@ -5,10 +5,10 @@
 #include "mlx5dr_internal.h"
 
 static void mlx5dr_rule_skip(struct mlx5dr_matcher *matcher,
+			     struct mlx5dr_match_template *mt,
 			     const struct rte_flow_item *items,
 			     bool *skip_rx, bool *skip_tx)
 {
-	struct mlx5dr_match_template *mt = matcher->mt[0];
 	const struct flow_hw_port_info *vport;
 	const struct rte_flow_item_ethdev *v;
 
@@ -43,6 +43,7 @@ static void mlx5dr_rule_skip(struct mlx5dr_matcher *matcher,
 static void mlx5dr_rule_init_dep_wqe(struct mlx5dr_send_ring_dep_wqe *dep_wqe,
 				     struct mlx5dr_rule *rule,
 				     const struct rte_flow_item *items,
+				     struct mlx5dr_match_template *mt,
 				     void *user_data)
 {
 	struct mlx5dr_matcher *matcher = rule->matcher;
@@ -63,7 +64,7 @@ static void mlx5dr_rule_init_dep_wqe(struct mlx5dr_send_ring_dep_wqe *dep_wqe,
 		break;
 
 	case MLX5DR_TABLE_TYPE_FDB:
-		mlx5dr_rule_skip(matcher, items, &skip_rx, &skip_tx);
+		mlx5dr_rule_skip(matcher, mt, items, &skip_rx, &skip_tx);
 
 		if (!skip_rx) {
 			dep_wqe->rtc_0 = matcher->match_ste.rtc_0->id;
@@ -186,8 +187,8 @@ static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 				  uint8_t at_idx,
 				  struct mlx5dr_rule_action rule_actions[])
 {
-	struct mlx5dr_action_template *at = rule->matcher->at[at_idx];
-	struct mlx5dr_match_template *mt = rule->matcher->mt[mt_idx];
+	struct mlx5dr_action_template *at = &rule->matcher->at[at_idx];
+	struct mlx5dr_match_template *mt = &rule->matcher->mt[mt_idx];
 	bool is_jumbo = mlx5dr_definer_is_jumbo(mt->definer);
 	struct mlx5dr_matcher *matcher = rule->matcher;
 	struct mlx5dr_context *ctx = matcher->tbl->ctx;
@@ -212,7 +213,7 @@ static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 	 * dep_wqe buffers (ctrl, data) are also reused for all STE writes.
 	 */
 	dep_wqe = mlx5dr_send_add_new_dep_wqe(queue);
-	mlx5dr_rule_init_dep_wqe(dep_wqe, rule, items, attr->user_data);
+	mlx5dr_rule_init_dep_wqe(dep_wqe, rule, items, mt, attr->user_data);
 
 	ste_attr.wqe_ctrl = &dep_wqe->wqe_ctrl;
 	ste_attr.wqe_data = &dep_wqe->wqe_data;
@@ -371,7 +372,7 @@ static int mlx5dr_rule_destroy_hws(struct mlx5dr_rule *rule,
 	ste_attr.used_id_rtc_1 = &rule->rtc_1;
 	ste_attr.wqe_ctrl = &wqe_ctrl;
 	ste_attr.wqe_tag = &rule->tag;
-	ste_attr.wqe_tag_is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt[0]->definer);
+	ste_attr.wqe_tag_is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt->definer);
 	ste_attr.gta_opcode = MLX5DR_WQE_GTA_OP_DEACTIVATE;
 	if (unlikely(mlx5dr_matcher_is_insert_by_idx(matcher)))
 		ste_attr.direct_index = attr->rule_idx;
@@ -388,7 +389,7 @@ static int mlx5dr_rule_create_root(struct mlx5dr_rule *rule,
 				   struct mlx5dr_rule_action rule_actions[])
 {
 	struct mlx5dv_flow_matcher *dv_matcher = rule->matcher->dv_matcher;
-	uint8_t num_actions = rule->matcher->at[at_idx]->num_actions;
+	uint8_t num_actions = rule->matcher->at[at_idx].num_actions;
 	struct mlx5dr_context *ctx = rule->matcher->tbl->ctx;
 	struct mlx5dv_flow_match_parameters *value;
 	struct mlx5_flow_attr flow_attr = {0};
Signature · 1 line
2.18.1
#3 lore raw

PATCH 3/16 net/mlx5/hws: support GTA WQE write using FW command

In reply to Alex Vesker

The generate WQE command is used as an interface to writing GTA WQEs with fields that are not supported in current HW, for example extended match definer.

---
 drivers/common/mlx5/mlx5_prm.h    | 27 +++++++++++++++++-
 drivers/net/mlx5/hws/mlx5dr_cmd.c | 47 +++++++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_cmd.h | 13 +++++++++
 3 files changed, 86 insertions(+), 1 deletion(-)
drivers/common/mlx5/mlx5_prm.h +26 -1
index 9294f65e24..d4d8ddcd2a 100644
@@ -1141,6 +1141,7 @@ enum {
 	MLX5_CMD_QUERY_REGEX_REGISTERS = 0xb07,
 	MLX5_CMD_OP_ACCESS_REGISTER_USER = 0xb0c,
 	MLX5_CMD_OP_ALLOW_OTHER_VHCA_ACCESS = 0xb16,
+	MLX5_CMD_OP_GENERATE_WQE = 0xb17,
 };
 
 enum {
@@ -2159,7 +2160,8 @@ struct mlx5_ifc_cmd_hca_cap_2_bits {
 	u8 format_select_dw_gtpu_dw_1[0x8];
 	u8 format_select_dw_gtpu_dw_2[0x8];
 	u8 format_select_dw_gtpu_first_ext_dw_0[0x8];
-	u8 reserved_at_2a0[0x560];
+	u8 generate_wqe_type[0x20];
+	u8 reserved_at_2c0[0x540];
 };
 
 struct mlx5_ifc_esw_cap_bits {
@@ -3529,6 +3531,29 @@ struct mlx5_ifc_create_alias_obj_in_bits {
 	struct mlx5_ifc_alias_context_bits alias_ctx;
 };
 
+struct mlx5_ifc_generate_wqe_in_bits {
+	u8 opcode[0x10];
+	u8 uid[0x10];
+	u8 reserved_at_20[0x10];
+	u8 op_mode[0x10];
+	u8 reserved_at_40[0x40];
+	u8 reserved_at_80[0x8];
+	u8 pdn[0x18];
+	u8 reserved_at_a0[0x160];
+	u8 wqe_ctrl[0x80];
+	u8 wqe_gta_ctrl[0x180];
+	u8 wqe_gta_data_0[0x200];
+	u8 wqe_gta_data_1[0x200];
+};
+
+struct mlx5_ifc_generate_wqe_out_bits {
+	u8 status[0x8];
+	u8 reserved_at_8[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_at_40[0x1c0];
+	u8 cqe_data[0x200];
+};
+
 enum {
 	MLX5_CRYPTO_KEY_SIZE_128b = 0x0,
 	MLX5_CRYPTO_KEY_SIZE_256b = 0x1,
drivers/net/mlx5/hws/mlx5dr_cmd.c +47 -0

53 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_cmd.h +13 -0
index 468557ba16..3689d09897 100644
@@ -153,6 +153,14 @@ struct mlx5dr_cmd_query_vport_caps {
 	uint32_t metadata_c_mask;
 };
 
+struct mlx5dr_cmd_generate_wqe_attr {
+	uint8_t *wqe_ctrl;
+	uint8_t *gta_ctrl;
+	uint8_t *gta_data_0;
+	uint8_t *gta_data_1;
+	uint32_t pdn;
+};
+
 struct mlx5dr_cmd_query_caps {
 	uint32_t wire_regc;
 	uint32_t wire_regc_mask;
@@ -212,6 +220,11 @@ int
 mlx5dr_cmd_stc_modify(struct mlx5dr_devx_obj *devx_obj,
 		      struct mlx5dr_cmd_stc_modify_attr *stc_attr);
 
+int
+mlx5dr_cmd_generate_wqe(struct ibv_context *ctx,
+			struct mlx5dr_cmd_generate_wqe_attr *attr,
+			struct mlx5_cqe64 *ret_cqe);
+
 struct mlx5dr_devx_obj *
 mlx5dr_cmd_ste_create(struct ibv_context *ctx,
 		      struct mlx5dr_cmd_ste_create_attr *ste_attr);
Signature · 1 line
2.18.1
#4 lore raw

PATCH 4/16 net/mlx5/hws: add capability query for gen wqe command

In reply to Alex Vesker

Read the capabilities required to determine support for GENERATE_WQE.

---
 drivers/common/mlx5/mlx5_prm.h    |  6 ++++--
 drivers/net/mlx5/hws/mlx5dr_cmd.c | 12 ++++++++++++
 drivers/net/mlx5/hws/mlx5dr_cmd.h |  3 +++
 3 files changed, 19 insertions(+), 2 deletions(-)
drivers/common/mlx5/mlx5_prm.h +4 -2
index d4d8ddcd2a..6d0b5e640c 100644
@@ -2205,10 +2205,12 @@ struct mlx5_ifc_wqe_based_flow_table_cap_bits {
 	u8 header_insert_type[0x10];
 	u8 header_remove_type[0x10];
 	u8 trivial_match_definer[0x20];
-	u8 reserved_at_140[0x20];
+	u8 reserved_at_140[0x1b];
+	u8 rtc_max_num_hash_definer_gen_wqe[0x5];
 	u8 reserved_at_160[0x18];
 	u8 access_index_mode[0x8];
-	u8 reserved_at_180[0x20];
+	u8 reserved_at_180[0x10];
+	u8 ste_fromat_gen_wqe[0x10];
 	u8 linear_match_definer_reg_c3[0x20];
 };
 
drivers/net/mlx5/hws/mlx5dr_cmd.c +12 -0
index c648eacd03..e311be780b 100644
@@ -928,6 +928,10 @@ int mlx5dr_cmd_query_caps(struct ibv_context *ctx,
 						     capability.cmd_hca_cap_2.
 						     format_select_dw_gtpu_first_ext_dw_0);
 
+	caps->supp_type_gen_wqe = MLX5_GET(query_hca_cap_out, out,
+					   capability.cmd_hca_cap_2.
+					   generate_wqe_type);
+
 	/* check cross-VHCA support in cap2 */
 	res =
 	MLX5_GET(query_hca_cap_out, out,
@@ -1033,6 +1037,14 @@ int mlx5dr_cmd_query_caps(struct ibv_context *ctx,
 		caps->linear_match_definer = MLX5_GET(query_hca_cap_out, out,
 						      capability.wqe_based_flow_table_cap.
 						      linear_match_definer_reg_c3);
+
+		caps->rtc_max_hash_def_gen_wqe = MLX5_GET(query_hca_cap_out, out,
+							  capability.wqe_based_flow_table_cap.
+							  rtc_max_num_hash_definer_gen_wqe);
+
+		caps->supp_ste_fromat_gen_wqe = MLX5_GET(query_hca_cap_out, out,
+							 capability.wqe_based_flow_table_cap.
+							 ste_fromat_gen_wqe);
 	}
 
 	if (caps->eswitch_manager) {
drivers/net/mlx5/hws/mlx5dr_cmd.h +3 -0
index 3689d09897..a42218ba74 100644
@@ -183,6 +183,9 @@ struct mlx5dr_cmd_query_caps {
 	bool full_dw_jumbo_support;
 	bool rtc_hash_split_table;
 	bool rtc_linear_lookup_table;
+	uint32_t supp_type_gen_wqe;
+	uint8_t rtc_max_hash_def_gen_wqe;
+	uint16_t supp_ste_fromat_gen_wqe;
 	struct mlx5dr_cmd_query_ft_caps nic_ft;
 	struct mlx5dr_cmd_query_ft_caps fdb_ft;
 	bool eswitch_manager;
Signature · 1 line
2.18.1
#5 lore raw

PATCH 5/16 net/mlx5/hws: align RTC create command with PRM format

In reply to Alex Vesker

Rename rtc params create for new format.

---
 drivers/common/mlx5/mlx5_prm.h        | 16 ++++++++++------
 drivers/net/mlx5/hws/mlx5dr_cmd.c     | 13 +++++++++++--
 drivers/net/mlx5/hws/mlx5dr_cmd.h     | 11 +++++++----
 drivers/net/mlx5/hws/mlx5dr_matcher.c | 19 ++++++++++++-------
 4 files changed, 40 insertions(+), 19 deletions(-)
drivers/common/mlx5/mlx5_prm.h +10 -6
index 6d0b5e640c..cf46296afb 100644
@@ -3237,6 +3237,7 @@ enum mlx5_ifc_rtc_access_mode {
 enum mlx5_ifc_rtc_ste_format {
 	MLX5_IFC_RTC_STE_FORMAT_8DW = 0x4,
 	MLX5_IFC_RTC_STE_FORMAT_11DW = 0x5,
+	MLX5_IFC_RTC_STE_FORMAT_RANGE = 0x7,
 };
 
 enum mlx5_ifc_rtc_reparse_mode {
@@ -3251,24 +3252,27 @@ struct mlx5_ifc_rtc_bits {
 	u8 reserved_at_40[0x40];
 	u8 update_index_mode[0x2];
 	u8 reparse_mode[0x2];
-	u8 reserved_at_84[0x4];
+	u8 num_match_ste[0x4];
 	u8 pd[0x18];
 	u8 reserved_at_a0[0x9];
 	u8 access_index_mode[0x3];
 	u8 num_hash_definer[0x4];
-	u8 reserved_at_b0[0x3];
+	u8 update_method[0x1];
+	u8 reserved_at_b1[0x2];
 	u8 log_depth[0x5];
 	u8 log_hash_size[0x8];
-	u8 ste_format[0x8];
+	u8 ste_format_0[0x8];
 	u8 table_type[0x8];
-	u8 reserved_at_d0[0x10];
-	u8 match_definer_id[0x20];
+	u8 ste_format_1[0x8];
+	u8 reserved_at_d8[0x8];
+	u8 match_definer_0[0x20];
 	u8 stc_id[0x20];
 	u8 ste_table_base_id[0x20];
 	u8 ste_table_offset[0x20];
 	u8 reserved_at_160[0x8];
 	u8 miss_flow_table_id[0x18];
-	u8 reserved_at_180[0x280];
+	u8 match_definer_1[0x20];
+	u8 reserved_at_1a0[0x260];
 };
 
 struct mlx5_ifc_alias_context_bits {
drivers/net/mlx5/hws/mlx5dr_cmd.c +11 -2
index e311be780b..a8d1cf0322 100644
@@ -259,17 +259,26 @@ mlx5dr_cmd_rtc_create(struct ibv_context *ctx,
 		 attr, obj_type, MLX5_GENERAL_OBJ_TYPE_RTC);
 
 	attr = MLX5_ADDR_OF(create_rtc_in, in, rtc);
-	MLX5_SET(rtc, attr, ste_format, rtc_attr->is_jumbo ?
+	MLX5_SET(rtc, attr, ste_format_0, rtc_attr->is_frst_jumbo ?
 		MLX5_IFC_RTC_STE_FORMAT_11DW :
 		MLX5_IFC_RTC_STE_FORMAT_8DW);
+
+	if (rtc_attr->is_scnd_range) {
+		MLX5_SET(rtc, attr, ste_format_1, MLX5_IFC_RTC_STE_FORMAT_RANGE);
+		MLX5_SET(rtc, attr, num_match_ste, 2);
+	}
+
 	MLX5_SET(rtc, attr, pd, rtc_attr->pd);
+	MLX5_SET(rtc, attr, update_method, rtc_attr->fw_gen_wqe);
 	MLX5_SET(rtc, attr, update_index_mode, rtc_attr->update_index_mode);
 	MLX5_SET(rtc, attr, access_index_mode, rtc_attr->access_index_mode);
 	MLX5_SET(rtc, attr, num_hash_definer, rtc_attr->num_hash_definer);
 	MLX5_SET(rtc, attr, log_depth, rtc_attr->log_depth);
 	MLX5_SET(rtc, attr, log_hash_size, rtc_attr->log_size);
 	MLX5_SET(rtc, attr, table_type, rtc_attr->table_type);
-	MLX5_SET(rtc, attr, match_definer_id, rtc_attr->definer_id);
+	MLX5_SET(rtc, attr, num_hash_definer, rtc_attr->num_hash_definer);
+	MLX5_SET(rtc, attr, match_definer_0, rtc_attr->match_definer_0);
+	MLX5_SET(rtc, attr, match_definer_1, rtc_attr->match_definer_1);
 	MLX5_SET(rtc, attr, stc_id, rtc_attr->stc_base);
 	MLX5_SET(rtc, attr, ste_table_base_id, rtc_attr->ste_base);
 	MLX5_SET(rtc, attr, ste_table_offset, rtc_attr->ste_offset);
drivers/net/mlx5/hws/mlx5dr_cmd.h +7 -4
index a42218ba74..e062cb8171 100644
@@ -23,8 +23,8 @@ struct mlx5dr_cmd_ft_modify_attr {
 };
 
 struct mlx5dr_cmd_fg_attr {
-	uint32_t	table_id;
-	uint32_t	table_type;
+	uint32_t table_id;
+	uint32_t table_type;
 };
 
 struct mlx5dr_cmd_forward_tbl {
@@ -40,14 +40,17 @@ struct mlx5dr_cmd_rtc_create_attr {
 	uint32_t ste_base;
 	uint32_t ste_offset;
 	uint32_t miss_ft_id;
+	bool fw_gen_wqe;
 	uint8_t update_index_mode;
 	uint8_t access_index_mode;
 	uint8_t num_hash_definer;
 	uint8_t log_depth;
 	uint8_t log_size;
 	uint8_t table_type;
-	uint8_t definer_id;
-	bool is_jumbo;
+	uint8_t match_definer_0;
+	uint8_t match_definer_1;
+	bool is_frst_jumbo;
+	bool is_scnd_range;
 };
 
 struct mlx5dr_cmd_alias_obj_create_attr {
drivers/net/mlx5/hws/mlx5dr_matcher.c +12 -7
index 913bb9d447..101a12d361 100644
@@ -413,6 +413,8 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 	struct mlx5dr_pool *ste_pool, *stc_pool;
 	struct mlx5dr_devx_obj *devx_obj;
 	struct mlx5dr_pool_chunk *ste;
+	uint8_t first_definer_id;
+	bool is_jumbo;
 	int ret;
 
 	switch (rtc_type) {
@@ -426,12 +428,15 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 		rtc_attr.log_depth = attr->table.sz_col_log;
 		rtc_attr.miss_ft_id = matcher->end_ft->id;
 
+		is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt->definer);
+		first_definer_id = mlx5dr_definer_get_id(matcher->mt->definer);
+
 		if (attr->insert_mode == MLX5DR_MATCHER_INSERT_BY_HASH) {
 			/* The usual Hash Table */
 			rtc_attr.update_index_mode = MLX5_IFC_RTC_STE_UPDATE_MODE_BY_HASH;
 			/* The first match template is used since all share the same definer */
-			rtc_attr.definer_id = mlx5dr_definer_get_id(matcher->mt->definer);
-			rtc_attr.is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt->definer);
+			rtc_attr.match_definer_0 = first_definer_id;
+			rtc_attr.is_frst_jumbo = is_jumbo;
 		} else if (attr->insert_mode == MLX5DR_MATCHER_INSERT_BY_INDEX) {
 			rtc_attr.update_index_mode = MLX5_IFC_RTC_STE_UPDATE_MODE_BY_OFFSET;
 			rtc_attr.num_hash_definer = 1;
@@ -439,12 +444,12 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 			if (attr->distribute_mode == MLX5DR_MATCHER_DISTRIBUTE_BY_HASH) {
 				/* Hash Split Table */
 				rtc_attr.access_index_mode = MLX5_IFC_RTC_STE_ACCESS_MODE_BY_HASH;
-				rtc_attr.definer_id = mlx5dr_definer_get_id(matcher->mt->definer);
-				rtc_attr.is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt->definer);
+				rtc_attr.match_definer_0 = first_definer_id;
+				rtc_attr.is_frst_jumbo = is_jumbo;
 			} else if (attr->distribute_mode == MLX5DR_MATCHER_DISTRIBUTE_BY_LINEAR) {
 				/* Linear Lookup Table */
 				rtc_attr.access_index_mode = MLX5_IFC_RTC_STE_ACCESS_MODE_LINEAR;
-				rtc_attr.definer_id = ctx->caps->linear_match_definer;
+				rtc_attr.match_definer_0 = ctx->caps->linear_match_definer;
 			}
 		}
 
@@ -468,8 +473,8 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 		rtc_attr.log_depth = 0;
 		rtc_attr.update_index_mode = MLX5_IFC_RTC_STE_UPDATE_MODE_BY_OFFSET;
 		/* The action STEs use the default always hit definer */
-		rtc_attr.definer_id = ctx->caps->trivial_match_definer;
-		rtc_attr.is_jumbo = false;
+		rtc_attr.match_definer_0 = ctx->caps->trivial_match_definer;
+		rtc_attr.is_frst_jumbo = false;
 		rtc_attr.miss_ft_id = 0;
 		break;
 
Signature · 1 line
2.18.1
#6 lore raw

PATCH 6/16 net/mlx5/hws: add send FW match STE using gen WQE

In reply to Alex Vesker

Send STE WQE function wraps the send WQE command to support WQE build and FDB abstraction. Sending using FW is different from sending from HW since FW returns the completion immediately which requires us to retry on failure and prepare the completion as part of the send process.

---
 drivers/net/mlx5/hws/mlx5dr_send.c | 134 +++++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_send.h |   7 +-
 2 files changed, 140 insertions(+), 1 deletion(-)
drivers/net/mlx5/hws/mlx5dr_send.c +134 -0
index a507e5f626..a9958df4f2 100644
@@ -235,6 +235,140 @@ void mlx5dr_send_ste(struct mlx5dr_send_engine *queue,
 	send_attr->fence = fence;
 }
 
+static
+int mlx5dr_send_wqe_fw(struct ibv_context *ibv_ctx,
+		       uint32_t pd_num,
+		       struct mlx5dr_send_engine_post_attr *send_attr,
+		       struct mlx5dr_wqe_gta_ctrl_seg *send_wqe_ctrl,
+		       void *send_wqe_match_data,
+		       void *send_wqe_match_tag,
+		       bool is_jumbo,
+		       uint8_t gta_opcode)
+{
+	bool has_match = send_wqe_match_data || send_wqe_match_tag;
+	struct mlx5dr_wqe_gta_data_seg_ste gta_wqe_data0 = {0};
+	struct mlx5dr_wqe_gta_ctrl_seg gta_wqe_ctrl = {0};
+	struct mlx5dr_cmd_generate_wqe_attr attr = {0};
+	struct mlx5dr_wqe_ctrl_seg wqe_ctrl = {0};
+	struct mlx5_cqe64 cqe;
+	uint32_t flags = 0;
+	int ret;
+
+	/* Set WQE control */
+	wqe_ctrl.opmod_idx_opcode =
+		rte_cpu_to_be_32((send_attr->opmod << 24) | send_attr->opcode);
+	wqe_ctrl.qpn_ds =
+		rte_cpu_to_be_32((send_attr->len + sizeof(struct mlx5dr_wqe_ctrl_seg)) / 16);
+	flags |= send_attr->notify_hw ? MLX5_WQE_CTRL_CQ_UPDATE : 0;
+	wqe_ctrl.flags = rte_cpu_to_be_32(flags);
+	wqe_ctrl.imm = rte_cpu_to_be_32(send_attr->id);
+
+	/* Set GTA WQE CTRL */
+	memcpy(gta_wqe_ctrl.stc_ix, send_wqe_ctrl->stc_ix, sizeof(send_wqe_ctrl->stc_ix));
+	gta_wqe_ctrl.op_dirix = htobe32(gta_opcode << 28);
+
+	/* Set GTA match WQE DATA */
+	if (has_match) {
+		if (send_wqe_match_data)
+			memcpy(&gta_wqe_data0, send_wqe_match_data, sizeof(gta_wqe_data0));
+		else
+			mlx5dr_send_wqe_set_tag(&gta_wqe_data0, send_wqe_match_tag, is_jumbo);
+
+		gta_wqe_data0.rsvd1_definer = htobe32(send_attr->match_definer_id << 8);
+		attr.gta_data_0 = (uint8_t *)&gta_wqe_data0;
+	}
+
+	attr.pdn = pd_num;
+	attr.wqe_ctrl = (uint8_t *)&wqe_ctrl;
+	attr.gta_ctrl = (uint8_t *)&gta_wqe_ctrl;
+
+send_wqe:
+	ret = mlx5dr_cmd_generate_wqe(ibv_ctx, &attr, &cqe);
+	if (ret) {
+		DR_LOG(ERR, "Failed to write WQE using command");
+		return ret;
+	}
+
+	if ((mlx5dv_get_cqe_opcode(&cqe) == MLX5_CQE_REQ) &&
+	    (rte_be_to_cpu_32(cqe.byte_cnt) >> 31 == 0)) {
+		*send_attr->used_id = send_attr->id;
+		return 0;
+	}
+
+	/* Retry if rule failed */
+	if (send_attr->retry_id) {
+		wqe_ctrl.imm = rte_cpu_to_be_32(send_attr->retry_id);
+		send_attr->id = send_attr->retry_id;
+		send_attr->retry_id = 0;
+		goto send_wqe;
+	}
+
+	return -1;
+}
+
+void mlx5dr_send_stes_fw(struct mlx5dr_send_engine *queue,
+			 struct mlx5dr_send_ste_attr *ste_attr)
+{
+	struct mlx5dr_send_engine_post_attr *send_attr = &ste_attr->send_attr;
+	struct mlx5dr_rule *rule = send_attr->rule;
+	struct ibv_context *ibv_ctx;
+	struct mlx5dr_context *ctx;
+	uint16_t queue_id;
+	uint32_t pdn;
+	int ret;
+
+	ctx = rule->matcher->tbl->ctx;
+	queue_id = queue - ctx->send_queue;
+	ibv_ctx = ctx->ibv_ctx;
+	pdn = ctx->pd_num;
+
+	/* Writing through FW can't HW fence, therefore we drain the queue */
+	if (send_attr->fence)
+		mlx5dr_send_queue_action(ctx,
+					 queue_id,
+					 MLX5DR_SEND_QUEUE_ACTION_DRAIN_SYNC);
+
+	if (ste_attr->rtc_1) {
+		send_attr->id = ste_attr->rtc_1;
+		send_attr->used_id = ste_attr->used_id_rtc_1;
+		send_attr->retry_id = ste_attr->retry_rtc_1;
+		ret = mlx5dr_send_wqe_fw(ibv_ctx, pdn, send_attr,
+					 ste_attr->wqe_ctrl,
+					 ste_attr->wqe_data,
+					 ste_attr->wqe_tag,
+					 ste_attr->wqe_tag_is_jumbo,
+					 ste_attr->gta_opcode);
+		if (ret)
+			goto fail_rule;
+	}
+
+	if (ste_attr->rtc_0) {
+		send_attr->id = ste_attr->rtc_0;
+		send_attr->used_id = ste_attr->used_id_rtc_0;
+		send_attr->retry_id = ste_attr->retry_rtc_0;
+		ret = mlx5dr_send_wqe_fw(ibv_ctx, pdn, send_attr,
+					 ste_attr->wqe_ctrl,
+					 ste_attr->wqe_data,
+					 ste_attr->wqe_tag,
+					 ste_attr->wqe_tag_is_jumbo,
+					 ste_attr->gta_opcode);
+		if (ret)
+			goto fail_rule;
+	}
+
+	/* Increase the status, this only works on good flow as the enum
+	 * is arrange it away creating -> created -> deleting -> deleted
+	 */
+	rule->status++;
+	mlx5dr_send_engine_gen_comp(queue, send_attr->user_data, RTE_FLOW_OP_SUCCESS);
+	return;
+
+fail_rule:
+	rule->status = !rule->rtc_0 && !rule->rtc_1 ?
+		MLX5DR_RULE_STATUS_FAILED : MLX5DR_RULE_STATUS_FAILING;
+	mlx5dr_send_engine_gen_comp(queue, send_attr->user_data, RTE_FLOW_OP_ERROR);
+}
+
 static void mlx5dr_send_engine_retry_post_send(struct mlx5dr_send_engine *queue,
 					       struct mlx5dr_send_ring_priv *priv,
 					       uint16_t wqe_cnt)
drivers/net/mlx5/hws/mlx5dr_send.h +6 -1
index fcddcc6366..1e845b1c7a 100644
@@ -52,7 +52,8 @@ struct mlx5dr_wqe_gta_ctrl_seg {
 
 struct mlx5dr_wqe_gta_data_seg_ste {
 	__be32 rsvd0_ctr_id;
-	__be32 rsvd1[4];
+	__be32 rsvd1_definer;
+	__be32 rsvd2[3];
 	__be32 action[3];
 	__be32 tag[8];
 };
@@ -159,6 +160,7 @@ struct mlx5dr_send_engine_post_attr {
 	uint8_t opmod;
 	uint8_t notify_hw;
 	uint8_t fence;
+	uint8_t match_definer_id;
 	size_t len;
 	struct mlx5dr_rule *rule;
 	uint32_t id;
@@ -238,6 +240,9 @@ void mlx5dr_send_engine_post_end(struct mlx5dr_send_engine_post_ctrl *ctrl,
 void mlx5dr_send_ste(struct mlx5dr_send_engine *queue,
 		     struct mlx5dr_send_ste_attr *ste_attr);
 
+void mlx5dr_send_stes_fw(struct mlx5dr_send_engine *queue,
+			 struct mlx5dr_send_ste_attr *ste_attr);
+
 void mlx5dr_send_engine_flush_queue(struct mlx5dr_send_engine *queue);
 
 static inline bool mlx5dr_send_engine_empty(struct mlx5dr_send_engine *queue)
Signature · 1 line
2.18.1
#7 lore raw

PATCH 7/16 net/mlx5/hws: add send FW range STE WQE

In reply to Alex Vesker

FW WQE supports complex rules, constructed from 2 STEs, for example:
Hash(DefinerA)
SteMatch(DefinerB)
SteRange(DefinerC)
DefinerA is a subset of DefinerB

This complex rule is written using a single FW command which has a single WQE control, STE match data0 and STE range data1. FW manages STEs/ICM and coherency between deletion and creation. It is possible to also pass the definer value as part of the STE, this is not supported with current HW.

---
 drivers/net/mlx5/hws/mlx5dr_send.c | 19 +++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_send.h |  3 +++
 2 files changed, 22 insertions(+)
drivers/net/mlx5/hws/mlx5dr_send.c +19 -0
index a9958df4f2..51aaf5c8e2 100644
@@ -242,11 +242,15 @@ int mlx5dr_send_wqe_fw(struct ibv_context *ibv_ctx,
 		       struct mlx5dr_wqe_gta_ctrl_seg *send_wqe_ctrl,
 		       void *send_wqe_match_data,
 		       void *send_wqe_match_tag,
+		       void *send_wqe_range_data,
+		       void *send_wqe_range_tag,
 		       bool is_jumbo,
 		       uint8_t gta_opcode)
 {
+	bool has_range = send_wqe_range_data || send_wqe_range_tag;
 	bool has_match = send_wqe_match_data || send_wqe_match_tag;
 	struct mlx5dr_wqe_gta_data_seg_ste gta_wqe_data0 = {0};
+	struct mlx5dr_wqe_gta_data_seg_ste gta_wqe_data1 = {0};
 	struct mlx5dr_wqe_gta_ctrl_seg gta_wqe_ctrl = {0};
 	struct mlx5dr_cmd_generate_wqe_attr attr = {0};
 	struct mlx5dr_wqe_ctrl_seg wqe_ctrl = {0};
@@ -278,6 +282,17 @@ int mlx5dr_send_wqe_fw(struct ibv_context *ibv_ctx,
 		attr.gta_data_0 = (uint8_t *)&gta_wqe_data0;
 	}
 
+	/* Set GTA range WQE DATA */
+	if (has_range) {
+		if (send_wqe_range_data)
+			memcpy(&gta_wqe_data1, send_wqe_range_data, sizeof(gta_wqe_data1));
+		else
+			mlx5dr_send_wqe_set_tag(&gta_wqe_data1, send_wqe_range_tag, false);
+
+		gta_wqe_data1.rsvd1_definer = htobe32(send_attr->range_definer_id << 8);
+		attr.gta_data_1 = (uint8_t *)&gta_wqe_data1;
+	}
+
 	attr.pdn = pd_num;
 	attr.wqe_ctrl = (uint8_t *)&wqe_ctrl;
 	attr.gta_ctrl = (uint8_t *)&gta_wqe_ctrl;
@@ -336,6 +351,8 @@ void mlx5dr_send_stes_fw(struct mlx5dr_send_engine *queue,
 					 ste_attr->wqe_ctrl,
 					 ste_attr->wqe_data,
 					 ste_attr->wqe_tag,
+					 ste_attr->range_wqe_data,
+					 ste_attr->range_wqe_tag,
 					 ste_attr->wqe_tag_is_jumbo,
 					 ste_attr->gta_opcode);
 		if (ret)
@@ -350,6 +367,8 @@ void mlx5dr_send_stes_fw(struct mlx5dr_send_engine *queue,
 					 ste_attr->wqe_ctrl,
 					 ste_attr->wqe_data,
 					 ste_attr->wqe_tag,
+					 ste_attr->range_wqe_data,
+					 ste_attr->range_wqe_tag,
 					 ste_attr->wqe_tag_is_jumbo,
 					 ste_attr->gta_opcode);
 		if (ret)
drivers/net/mlx5/hws/mlx5dr_send.h +3 -0
index 1e845b1c7a..47bb66b3c7 100644
@@ -161,6 +161,7 @@ struct mlx5dr_send_engine_post_attr {
 	uint8_t notify_hw;
 	uint8_t fence;
 	uint8_t match_definer_id;
+	uint8_t range_definer_id;
 	size_t len;
 	struct mlx5dr_rule *rule;
 	uint32_t id;
@@ -182,8 +183,10 @@ struct mlx5dr_send_ste_attr {
 	uint32_t direct_index;
 	struct mlx5dr_send_engine_post_attr send_attr;
 	struct mlx5dr_rule_match_tag *wqe_tag;
+	struct mlx5dr_rule_match_tag *range_wqe_tag;
 	struct mlx5dr_wqe_gta_ctrl_seg *wqe_ctrl;
 	struct mlx5dr_wqe_gta_data_seg_ste *wqe_data;
+	struct mlx5dr_wqe_gta_data_seg_ste *range_wqe_data;
 };
 
 /**
Signature · 1 line
2.18.1
#8 lore raw

PATCH 8/16 net/mlx5/hws: move matcher size check to function

In reply to Alex Vesker

This check can be later on reused for other places, it will look better in a function

---
 drivers/net/mlx5/hws/mlx5dr_matcher.c | 43 +++++++++++++++++----------
 1 file changed, 27 insertions(+), 16 deletions(-)
drivers/net/mlx5/hws/mlx5dr_matcher.c +27 -16
index 101a12d361..b8db0a27ae 100644
@@ -565,6 +565,32 @@ static void mlx5dr_matcher_destroy_rtc(struct mlx5dr_matcher *matcher,
 		mlx5dr_pool_chunk_free(ste_pool, ste);
 }
 
+static int
+mlx5dr_matcher_check_attr_sz(struct mlx5dr_cmd_query_caps *caps,
+			     struct mlx5dr_matcher_attr *attr)
+{
+	if (attr->table.sz_col_log > caps->rtc_log_depth_max) {
+		DR_LOG(ERR, "Matcher depth exceeds limit %d", caps->rtc_log_depth_max);
+		goto not_supported;
+	}
+
+	if (attr->table.sz_col_log + attr->table.sz_row_log > caps->ste_alloc_log_max) {
+		DR_LOG(ERR, "Total matcher size exceeds limit %d", caps->ste_alloc_log_max);
+		goto not_supported;
+	}
+
+	if (attr->table.sz_col_log + attr->table.sz_row_log < caps->ste_alloc_log_gran) {
+		DR_LOG(ERR, "Total matcher size below limit %d", caps->ste_alloc_log_gran);
+		goto not_supported;
+	}
+
+	return 0;
+
+not_supported:
+	rte_errno = EOPNOTSUPP;
+	return rte_errno;
+}
+
 static void mlx5dr_matcher_set_pool_attr(struct mlx5dr_pool_attr *attr,
 					 struct mlx5dr_matcher *matcher)
 {
@@ -840,22 +866,7 @@ mlx5dr_matcher_process_attr(struct mlx5dr_cmd_query_caps *caps,
 	    attr->insert_mode == MLX5DR_MATCHER_INSERT_BY_HASH)
 		attr->table.sz_col_log = mlx5dr_matcher_rules_to_tbl_depth(attr->rule.num_log);
 
-	if (attr->table.sz_col_log > caps->rtc_log_depth_max) {
-		DR_LOG(ERR, "Matcher depth exceeds limit %d", caps->rtc_log_depth_max);
-		goto not_supported;
-	}
-
-	if (attr->table.sz_col_log + attr->table.sz_row_log > caps->ste_alloc_log_max) {
-		DR_LOG(ERR, "Total matcher size exceeds limit %d", caps->ste_alloc_log_max);
-		goto not_supported;
-	}
-
-	if (attr->table.sz_col_log + attr->table.sz_row_log < caps->ste_alloc_log_gran) {
-		DR_LOG(ERR, "Total matcher size below limit %d", caps->ste_alloc_log_gran);
-		goto not_supported;
-	}
-
-	return 0;
+	return mlx5dr_matcher_check_attr_sz(caps, attr);
 
 not_supported:
 	rte_errno = EOPNOTSUPP;
Signature · 1 line
2.18.1
#9 lore raw

PATCH 9/16 net/mlx5/hws: support range match

In reply to Alex Vesker

Support range matching over selected items and range is not supported over all the items. The range match is done using:
item->last.field - maximum value item->mask.field - bitmask
item->spec.field - minimum value

When items are processed if item last and mask fields are non zero range matching will be done over these fields. There are two field setter, field copy (fc) and field copy range (fcr).

---
 drivers/net/mlx5/hws/mlx5dr_definer.c | 73 +++++++++++++++++++++++++--
 drivers/net/mlx5/hws/mlx5dr_definer.h |  5 +-
 2 files changed, 72 insertions(+), 6 deletions(-)
drivers/net/mlx5/hws/mlx5dr_definer.c +69 -4
index 6b98eb8c96..c268f94ad3 100644
@@ -123,6 +123,7 @@ struct mlx5dr_definer_conv_data {
 	X(SET,		ipv4_next_proto,	v->next_proto_id,	rte_ipv4_hdr) \
 	X(SET,		ipv4_version,		STE_IPV4,		rte_ipv4_hdr) \
 	X(SET_BE16,	ipv4_frag,		v->fragment_offset,	rte_ipv4_hdr) \
+	X(SET_BE16,	ipv4_len,		v->total_length,	rte_ipv4_hdr) \
 	X(SET_BE16,	ipv6_payload_len,	v->hdr.payload_len,	rte_flow_item_ipv6) \
 	X(SET,		ipv6_proto,		v->hdr.proto,		rte_flow_item_ipv6) \
 	X(SET,		ipv6_hop_limits,	v->hdr.hop_limits,	rte_flow_item_ipv6) \
@@ -516,6 +517,7 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 			      int item_idx)
 {
 	const struct rte_ipv4_hdr *m = item->mask;
+	const struct rte_ipv4_hdr *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	bool inner = cd->tunnel;
 
@@ -533,8 +535,8 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 	if (!m)
 		return 0;
 
-	if (m->total_length || m->packet_id ||
-	    m->hdr_checksum) {
+	if (m->packet_id || m->hdr_checksum ||
+	    (l && (l->next_proto_id || l->type_of_service))) {
 		rte_errno = ENOTSUP;
 		return rte_errno;
 	}
@@ -553,9 +555,18 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 		DR_CALC_SET(fc, eth_l3, protocol_next_header, inner);
 	}
 
+	if (m->total_length) {
+		fc = &cd->fc[DR_CALC_FNAME(IP_LEN, inner)];
+		fc->item_idx = item_idx;
+		fc->is_range = l && l->total_length;
+		fc->tag_set = &mlx5dr_definer_ipv4_len_set;
+		DR_CALC_SET(fc, eth_l3, ipv4_total_length, inner);
+	}
+
 	if (m->dst_addr) {
 		fc = &cd->fc[DR_CALC_FNAME(IPV4_DST, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->dst_addr;
 		fc->tag_set = &mlx5dr_definer_ipv4_dst_addr_set;
 		DR_CALC_SET(fc, ipv4_src_dest, destination_address, inner);
 	}
@@ -563,6 +574,7 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 	if (m->src_addr) {
 		fc = &cd->fc[DR_CALC_FNAME(IPV4_SRC, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->src_addr;
 		fc->tag_set = &mlx5dr_definer_ipv4_src_addr_set;
 		DR_CALC_SET(fc, ipv4_src_dest, source_address, inner);
 	}
@@ -570,6 +582,7 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 	if (m->ihl) {
 		fc = &cd->fc[DR_CALC_FNAME(IPV4_IHL, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->ihl;
 		fc->tag_set = &mlx5dr_definer_ipv4_ihl_set;
 		DR_CALC_SET(fc, eth_l3, ihl, inner);
 	}
@@ -577,6 +590,7 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 	if (m->time_to_live) {
 		fc = &cd->fc[DR_CALC_FNAME(IP_TTL, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->time_to_live;
 		fc->tag_set = &mlx5dr_definer_ipv4_time_to_live_set;
 		DR_CALC_SET(fc, eth_l3, time_to_live_hop_limit, inner);
 	}
@@ -597,6 +611,7 @@ mlx5dr_definer_conv_item_ipv6(struct mlx5dr_definer_conv_data *cd,
 			      int item_idx)
 {
 	const struct rte_flow_item_ipv6 *m = item->mask;
+	const struct rte_flow_item_ipv6 *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	bool inner = cd->tunnel;
 
@@ -616,7 +631,10 @@ mlx5dr_definer_conv_item_ipv6(struct mlx5dr_definer_conv_data *cd,
 
 	if (m->has_hop_ext || m->has_route_ext || m->has_auth_ext ||
 	    m->has_esp_ext || m->has_dest_ext || m->has_mobil_ext ||
-	    m->has_hip_ext || m->has_shim6_ext) {
+	    m->has_hip_ext || m->has_shim6_ext ||
+	    (l && (l->has_frag_ext || l->hdr.vtc_flow || l->hdr.proto ||
+		   !is_mem_zero(l->hdr.src_addr, 16) ||
+		   !is_mem_zero(l->hdr.dst_addr, 16)))) {
 		rte_errno = ENOTSUP;
 		return rte_errno;
 	}
@@ -643,8 +661,9 @@ mlx5dr_definer_conv_item_ipv6(struct mlx5dr_definer_conv_data *cd,
 	}
 
 	if (m->hdr.payload_len) {
-		fc = &cd->fc[DR_CALC_FNAME(IPV6_PAYLOAD_LEN, inner)];
+		fc = &cd->fc[DR_CALC_FNAME(IP_LEN, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.payload_len;
 		fc->tag_set = &mlx5dr_definer_ipv6_payload_len_set;
 		DR_CALC_SET(fc, eth_l3, ipv6_payload_length, inner);
 	}
@@ -659,6 +678,7 @@ mlx5dr_definer_conv_item_ipv6(struct mlx5dr_definer_conv_data *cd,
 	if (m->hdr.hop_limits) {
 		fc = &cd->fc[DR_CALC_FNAME(IP_TTL, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.hop_limits;
 		fc->tag_set = &mlx5dr_definer_ipv6_hop_limits_set;
 		DR_CALC_SET(fc, eth_l3, time_to_live_hop_limit, inner);
 	}
@@ -728,6 +748,7 @@ mlx5dr_definer_conv_item_udp(struct mlx5dr_definer_conv_data *cd,
 			     int item_idx)
 {
 	const struct rte_flow_item_udp *m = item->mask;
+	const struct rte_flow_item_udp *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	bool inner = cd->tunnel;
 
@@ -751,6 +772,7 @@ mlx5dr_definer_conv_item_udp(struct mlx5dr_definer_conv_data *cd,
 	if (m->hdr.src_port) {
 		fc = &cd->fc[DR_CALC_FNAME(L4_SPORT, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.src_port;
 		fc->tag_set = &mlx5dr_definer_udp_src_port_set;
 		DR_CALC_SET(fc, eth_l4, source_port, inner);
 	}
@@ -758,6 +780,7 @@ mlx5dr_definer_conv_item_udp(struct mlx5dr_definer_conv_data *cd,
 	if (m->hdr.dst_port) {
 		fc = &cd->fc[DR_CALC_FNAME(L4_DPORT, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.dst_port;
 		fc->tag_set = &mlx5dr_definer_udp_dst_port_set;
 		DR_CALC_SET(fc, eth_l4, destination_port, inner);
 	}
@@ -771,6 +794,7 @@ mlx5dr_definer_conv_item_tcp(struct mlx5dr_definer_conv_data *cd,
 			     int item_idx)
 {
 	const struct rte_flow_item_tcp *m = item->mask;
+	const struct rte_flow_item_tcp *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	bool inner = cd->tunnel;
 
@@ -786,9 +810,16 @@ mlx5dr_definer_conv_item_tcp(struct mlx5dr_definer_conv_data *cd,
 	if (!m)
 		return 0;
 
+	if (m->hdr.sent_seq || m->hdr.recv_ack || m->hdr.data_off ||
+	    m->hdr.rx_win || m->hdr.cksum || m->hdr.tcp_urp) {
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+
 	if (m->hdr.tcp_flags) {
 		fc = &cd->fc[DR_CALC_FNAME(TCP_FLAGS, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.tcp_flags;
 		fc->tag_set = &mlx5dr_definer_tcp_flags_set;
 		DR_CALC_SET(fc, eth_l4, tcp_flags, inner);
 	}
@@ -796,6 +827,7 @@ mlx5dr_definer_conv_item_tcp(struct mlx5dr_definer_conv_data *cd,
 	if (m->hdr.src_port) {
 		fc = &cd->fc[DR_CALC_FNAME(L4_SPORT, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.src_port;
 		fc->tag_set = &mlx5dr_definer_tcp_src_port_set;
 		DR_CALC_SET(fc, eth_l4, source_port, inner);
 	}
@@ -803,6 +835,7 @@ mlx5dr_definer_conv_item_tcp(struct mlx5dr_definer_conv_data *cd,
 	if (m->hdr.dst_port) {
 		fc = &cd->fc[DR_CALC_FNAME(L4_DPORT, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.dst_port;
 		fc->tag_set = &mlx5dr_definer_tcp_dst_port_set;
 		DR_CALC_SET(fc, eth_l4, destination_port, inner);
 	}
@@ -1108,6 +1141,7 @@ mlx5dr_definer_conv_item_tag(struct mlx5dr_definer_conv_data *cd,
 {
 	const struct rte_flow_item_tag *m = item->mask;
 	const struct rte_flow_item_tag *v = item->spec;
+	const struct rte_flow_item_tag *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	int reg;
 
@@ -1130,7 +1164,9 @@ mlx5dr_definer_conv_item_tag(struct mlx5dr_definer_conv_data *cd,
 		return rte_errno;
 
 	fc->item_idx = item_idx;
+	fc->is_range = l && l->index;
 	fc->tag_set = &mlx5dr_definer_tag_set;
+
 	return 0;
 }
 
@@ -1140,6 +1176,7 @@ mlx5dr_definer_conv_item_metadata(struct mlx5dr_definer_conv_data *cd,
 				  int item_idx)
 {
 	const struct rte_flow_item_meta *m = item->mask;
+	const struct rte_flow_item_meta *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	int reg;
 
@@ -1158,7 +1195,9 @@ mlx5dr_definer_conv_item_metadata(struct mlx5dr_definer_conv_data *cd,
 		return rte_errno;
 
 	fc->item_idx = item_idx;
+	fc->is_range = l && l->data;
 	fc->tag_set = &mlx5dr_definer_metadata_set;
+
 	return 0;
 }
 
@@ -1465,6 +1504,28 @@ mlx5dr_definer_conv_item_meter_color(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static int
+mlx5dr_definer_check_item_range_supp(struct rte_flow_item *item)
+{
+	if (!item->last)
+		return 0;
+
+	switch ((int)item->type) {
+	case RTE_FLOW_ITEM_TYPE_IPV4:
+	case RTE_FLOW_ITEM_TYPE_IPV6:
+	case RTE_FLOW_ITEM_TYPE_UDP:
+	case RTE_FLOW_ITEM_TYPE_TCP:
+	case RTE_FLOW_ITEM_TYPE_TAG:
+	case RTE_FLOW_ITEM_TYPE_META:
+	case MLX5_RTE_FLOW_ITEM_TYPE_TAG:
+		return 0;
+	default:
+		DR_LOG(ERR, "Range not supported over item type %d", item->type);
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+}
+
 static int
 mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 				struct mlx5dr_match_template *mt,
@@ -1487,6 +1548,10 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 	for (i = 0; items->type != RTE_FLOW_ITEM_TYPE_END; i++, items++) {
 		cd.tunnel = !!(item_flags & MLX5_FLOW_LAYER_TUNNEL);
 
+		ret = mlx5dr_definer_check_item_range_supp(items);
+		if (ret)
+			return ret;
+
 		switch ((int)items->type) {
 		case RTE_FLOW_ITEM_TYPE_ETH:
 			ret = mlx5dr_definer_conv_item_eth(&cd, items, i);
drivers/net/mlx5/hws/mlx5dr_definer.h +3 -2
index d52c6b0627..bab4baae4a 100644
@@ -38,8 +38,8 @@ enum mlx5dr_definer_fname {
 	MLX5DR_DEFINER_FNAME_IP_VERSION_I,
 	MLX5DR_DEFINER_FNAME_IP_FRAG_O,
 	MLX5DR_DEFINER_FNAME_IP_FRAG_I,
-	MLX5DR_DEFINER_FNAME_IPV6_PAYLOAD_LEN_O,
-	MLX5DR_DEFINER_FNAME_IPV6_PAYLOAD_LEN_I,
+	MLX5DR_DEFINER_FNAME_IP_LEN_O,
+	MLX5DR_DEFINER_FNAME_IP_LEN_I,
 	MLX5DR_DEFINER_FNAME_IP_TOS_O,
 	MLX5DR_DEFINER_FNAME_IP_TOS_I,
 	MLX5DR_DEFINER_FNAME_IPV6_FLOW_LABEL_O,
@@ -116,6 +116,7 @@ enum mlx5dr_definer_type {
 
 struct mlx5dr_definer_fc {
 	uint8_t item_idx;
+	uint8_t is_range;
 	uint32_t byte_off;
 	int bit_off;
 	uint32_t bit_mask;
Signature · 1 line
2.18.1
#10 lore raw

PATCH 10/16 net/mlx5/hws: redesign definer create

In reply to Alex Vesker

Until now definer creation and deletion get and put functions were used, the get function would calculate the definer field copy (fc), header layout (hl) and definer layout internally without taking into account other match templates used over the same matcher.
This logic had to be split to allow sharing the hl over multiple definers. First calculate the shared hl than create definers based on the definer shared layout. Once all definers use the same layout it is possible to hash over the shared fields since the location is the same across all of the definers.

---
 drivers/net/mlx5/hws/mlx5dr_definer.c  | 301 ++++++++++++++++---------
 drivers/net/mlx5/hws/mlx5dr_definer.h  |  11 +-
 drivers/net/mlx5/hws/mlx5dr_internal.h |   2 +-
 drivers/net/mlx5/hws/mlx5dr_matcher.c  |  61 ++---
 drivers/net/mlx5/hws/mlx5dr_matcher.h  |  16 +-
 5 files changed, 230 insertions(+), 161 deletions(-)
drivers/net/mlx5/hws/mlx5dr_definer.c +192 -109
index c268f94ad3..9560f8a0af 100644
@@ -104,7 +104,6 @@ struct mlx5dr_definer_conv_data {
 	struct mlx5dr_definer_fc *fc;
 	uint8_t relaxed;
 	uint8_t tunnel;
-	uint8_t *hl;
 };
 
 /* Xmacro used to create generic item setter from items */
@@ -1504,6 +1503,36 @@ mlx5dr_definer_conv_item_meter_color(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static int
+mlx5dr_definer_mt_set_fc(struct mlx5dr_match_template *mt,
+			 struct mlx5dr_definer_fc *fc,
+			 uint8_t *hl)
+{
+	uint32_t fc_sz = 0;
+	int i;
+
+	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++)
+		if (fc[i].tag_set)
+			fc_sz++;
+
+	mt->fc = simple_calloc(fc_sz, sizeof(*mt->fc));
+	if (!mt->fc) {
+		rte_errno = ENOMEM;
+		return rte_errno;
+	}
+
+	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++) {
+		if (!fc[i].tag_set)
+			continue;
+
+		fc[i].fname = i;
+		memcpy(&mt->fc[mt->fc_sz++], &fc[i], sizeof(*mt->fc));
+		DR_SET(hl, -1, fc[i].byte_off, fc[i].bit_off, fc[i].bit_mask);
+	}
+
+	return 0;
+}
+
 static int
 mlx5dr_definer_check_item_range_supp(struct rte_flow_item *item)
 {
@@ -1535,12 +1564,9 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 	struct mlx5dr_definer_conv_data cd = {0};
 	struct rte_flow_item *items = mt->items;
 	uint64_t item_flags = 0;
-	uint32_t total = 0;
-	int i, j;
-	int ret;
+	int i, ret;
 
 	cd.fc = fc;
-	cd.hl = hl;
 	cd.caps = ctx->caps;
 	cd.relaxed = mt->flags & MLX5DR_MATCH_TEMPLATE_FLAG_RELAXED_MATCH;
 
@@ -1660,29 +1686,11 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 
 	mt->item_flags = item_flags;
 
-	/* Fill in headers layout and calculate total number of fields  */
-	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++) {
-		if (fc[i].tag_set) {
-			total++;
-			DR_SET(hl, -1, fc[i].byte_off, fc[i].bit_off, fc[i].bit_mask);
-		}
-	}
-
-	mt->fc_sz = total;
-	mt->fc = simple_calloc(total, sizeof(*mt->fc));
-	if (!mt->fc) {
-		DR_LOG(ERR, "Failed to allocate field copy array");
-		rte_errno = ENOMEM;
-		return rte_errno;
-	}
-
-	j = 0;
-	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++) {
-		if (fc[i].tag_set) {
-			memcpy(&mt->fc[j], &fc[i], sizeof(*mt->fc));
-			mt->fc[j].fname = i;
-			j++;
-		}
+	/* Fill in headers layout and allocate fc array on mt */
+	ret = mlx5dr_definer_mt_set_fc(mt, fc, hl);
+	if (ret) {
+		DR_LOG(ERR, "Failed to set field copy to match template");
+		return ret;
 	}
 
 	return 0;
@@ -1837,8 +1845,8 @@ mlx5dr_definer_best_hl_fit_recu(struct mlx5dr_definer_sel_ctrl *ctrl,
 }
 
 static void
-mlx5dr_definer_apply_sel_ctrl(struct mlx5dr_definer_sel_ctrl *ctrl,
-			      struct mlx5dr_definer *definer)
+mlx5dr_definer_copy_sel_ctrl(struct mlx5dr_definer_sel_ctrl *ctrl,
+			     struct mlx5dr_definer *definer)
 {
 	memcpy(definer->byte_selector, ctrl->byte_selector, ctrl->allowed_bytes);
 	memcpy(definer->dw_selector, ctrl->full_dw_selector, ctrl->allowed_full_dw);
@@ -1848,7 +1856,7 @@ mlx5dr_definer_apply_sel_ctrl(struct mlx5dr_definer_sel_ctrl *ctrl,
 
 static int
 mlx5dr_definer_find_best_hl_fit(struct mlx5dr_context *ctx,
-				struct mlx5dr_match_template *mt,
+				struct mlx5dr_definer *definer,
 				uint8_t *hl)
 {
 	struct mlx5dr_definer_sel_ctrl ctrl = {0};
@@ -1861,8 +1869,8 @@ mlx5dr_definer_find_best_hl_fit(struct mlx5dr_context *ctx,
 
 	found = mlx5dr_definer_best_hl_fit_recu(&ctrl, 0, (uint32_t *)hl);
 	if (found) {
-		mlx5dr_definer_apply_sel_ctrl(&ctrl, mt->definer);
-		mt->definer->type = MLX5DR_DEFINER_TYPE_MATCH;
+		mlx5dr_definer_copy_sel_ctrl(&ctrl, definer);
+		definer->type = MLX5DR_DEFINER_TYPE_MATCH;
 		return 0;
 	}
 
@@ -1875,8 +1883,8 @@ mlx5dr_definer_find_best_hl_fit(struct mlx5dr_context *ctx,
 
 	found = mlx5dr_definer_best_hl_fit_recu(&ctrl, 0, (uint32_t *)hl);
 	if (found) {
-		mlx5dr_definer_apply_sel_ctrl(&ctrl, mt->definer);
-		mt->definer->type = MLX5DR_DEFINER_TYPE_JUMBO;
+		mlx5dr_definer_copy_sel_ctrl(&ctrl, definer);
+		definer->type = MLX5DR_DEFINER_TYPE_JUMBO;
 		return 0;
 	}
 
@@ -1920,114 +1928,189 @@ int mlx5dr_definer_get_id(struct mlx5dr_definer *definer)
 	return definer->obj->id;
 }
 
-int mlx5dr_definer_compare(struct mlx5dr_definer *definer_a,
-			   struct mlx5dr_definer *definer_b)
+static int
+mlx5dr_definer_calc_layout(struct mlx5dr_matcher *matcher,
+			   struct mlx5dr_definer *match_definer)
 {
-	int i;
+	struct mlx5dr_context *ctx = matcher->tbl->ctx;
+	struct mlx5dr_match_template *mt = matcher->mt;
+	uint8_t *match_hl, *hl_buff;
+	int i, ret;
 
-	if (definer_a->type != definer_b->type)
-		return 1;
+	/* Union header-layout (hl) is used for creating a single definer
+	 * field layout used with different bitmasks for hash and match.
+	 */
+	hl_buff = simple_calloc(1, MLX5_ST_SZ_BYTES(definer_hl));
+	if (!hl_buff) {
+		DR_LOG(ERR, "Failed to allocate memory for header layout");
+		rte_errno = ENOMEM;
+		return rte_errno;
+	}
 
-	for (i = 0; i < BYTE_SELECTORS; i++)
-		if (definer_a->byte_selector[i] != definer_b->byte_selector[i])
-			return 1;
+	match_hl = hl_buff;
 
-	for (i = 0; i < DW_SELECTORS; i++)
-		if (definer_a->dw_selector[i] != definer_b->dw_selector[i])
-			return 1;
+	/* Convert all mt items to header layout (hl)
+	 * and allocate the match field copy array (fc).
+	 */
+	for (i = 0; i < matcher->num_of_mt; i++) {
+		ret = mlx5dr_definer_conv_items_to_hl(ctx, &mt[i], match_hl);
+		if (ret) {
+			DR_LOG(ERR, "Failed to convert items to header layout");
+			goto free_fc;
+		}
+	}
 
-	for (i = 0; i < MLX5DR_JUMBO_TAG_SZ; i++)
-		if (definer_a->mask.jumbo[i] != definer_b->mask.jumbo[i])
-			return 1;
+	/* Find the match definer layout for header layout match union */
+	ret = mlx5dr_definer_find_best_hl_fit(ctx, match_definer, match_hl);
+	if (ret) {
+		DR_LOG(ERR, "Failed to create match definer from header layout");
+		goto free_fc;
+	}
 
+	simple_free(hl_buff);
 	return 0;
+
+free_fc:
+	for (i = 0; i < matcher->num_of_mt; i++)
+		if (mt[i].fc)
+			simple_free(mt[i].fc);
+
+	simple_free(hl_buff);
+	return rte_errno;
 }
 
-int mlx5dr_definer_get(struct mlx5dr_context *ctx,
-		       struct mlx5dr_match_template *mt)
+static struct mlx5dr_definer *
+mlx5dr_definer_alloc(struct ibv_context *ibv_ctx,
+		     struct mlx5dr_definer_fc *fc,
+		     int fc_sz,
+		     struct rte_flow_item *items,
+		     struct mlx5dr_definer *layout)
 {
 	struct mlx5dr_cmd_definer_create_attr def_attr = {0};
-	struct ibv_context *ibv_ctx = ctx->ibv_ctx;
-	uint8_t *hl;
+	struct mlx5dr_definer *definer;
 	int ret;
 
-	if (mt->refcount++)
-		return 0;
-
-	mt->definer = simple_calloc(1, sizeof(*mt->definer));
-	if (!mt->definer) {
+	definer = simple_calloc(1, sizeof(*definer));
+	if (!definer) {
 		DR_LOG(ERR, "Failed to allocate memory for definer");
 		rte_errno = ENOMEM;
-		goto dec_refcount;
-	}
-
-	/* Header layout (hl) holds full bit mask per field */
-	hl = simple_calloc(1, MLX5_ST_SZ_BYTES(definer_hl));
-	if (!hl) {
-		DR_LOG(ERR, "Failed to allocate memory for header layout");
-		rte_errno = ENOMEM;
-		goto free_definer;
+		return NULL;
 	}
 
-	/* Convert items to hl and allocate the field copy array (fc) */
-	ret = mlx5dr_definer_conv_items_to_hl(ctx, mt, hl);
-	if (ret) {
-		DR_LOG(ERR, "Failed to convert items to hl");
-		goto free_hl;
-	}
+	memcpy(definer, layout, sizeof(*definer));
 
-	/* Find the definer for given header layout */
-	ret = mlx5dr_definer_find_best_hl_fit(ctx, mt, hl);
-	if (ret) {
-		DR_LOG(ERR, "Failed to create definer from header layout");
-		goto free_field_copy;
-	}
-
-	/* Align field copy array based on the new definer */
-	ret = mlx5dr_definer_fc_bind(mt->definer,
-				     mt->fc,
-				     mt->fc_sz);
+	/* Align field copy array based on given layout */
+	ret = mlx5dr_definer_fc_bind(definer, fc, fc_sz);
 	if (ret) {
 		DR_LOG(ERR, "Failed to bind field copy to definer");
-		goto free_field_copy;
+		goto free_definer;
 	}
 
 	/* Create the tag mask used for definer creation */
-	mlx5dr_definer_create_tag_mask(mt->items,
-				       mt->fc,
-				       mt->fc_sz,
-				       mt->definer->mask.jumbo);
+	mlx5dr_definer_create_tag_mask(items, fc, fc_sz, definer->mask.jumbo);
 
 	/* Create definer based on the bitmask tag */
-	def_attr.match_mask = mt->definer->mask.jumbo;
-	def_attr.dw_selector = mt->definer->dw_selector;
-	def_attr.byte_selector = mt->definer->byte_selector;
-	mt->definer->obj = mlx5dr_cmd_definer_create(ibv_ctx, &def_attr);
-	if (!mt->definer->obj)
-		goto free_field_copy;
+	def_attr.match_mask = definer->mask.jumbo;
+	def_attr.dw_selector = layout->dw_selector;
+	def_attr.byte_selector = layout->byte_selector;
 
-	simple_free(hl);
+	definer->obj = mlx5dr_cmd_definer_create(ibv_ctx, &def_attr);
+	if (!definer->obj)
+		goto free_definer;
 
-	return 0;
+	return definer;
 
-free_field_copy:
-	simple_free(mt->fc);
-free_hl:
-	simple_free(hl);
 free_definer:
-	simple_free(mt->definer);
-dec_refcount:
-	mt->refcount--;
+	simple_free(definer);
+	return NULL;
+}
+
+static void
+mlx5dr_definer_free(struct mlx5dr_definer *definer)
+{
+	mlx5dr_cmd_destroy_obj(definer->obj);
+	simple_free(definer);
+}
+
+static int
+mlx5dr_definer_matcher_match_init(struct mlx5dr_context *ctx,
+				  struct mlx5dr_matcher *matcher,
+				  struct mlx5dr_definer *match_layout)
+{
+	struct mlx5dr_match_template *mt = matcher->mt;
+	int i;
+
+	/* Create mendatory match definer */
+	for (i = 0; i < matcher->num_of_mt; i++) {
+		mt[i].definer = mlx5dr_definer_alloc(ctx->ibv_ctx,
+						     mt[i].fc,
+						     mt[i].fc_sz,
+						     mt[i].items,
+						     match_layout);
+		if (!mt[i].definer) {
+			DR_LOG(ERR, "Failed to create match definer");
+			goto free_definers;
+		}
+	}
+	return 0;
+
+free_definers:
+	while (i--)
+		mlx5dr_definer_free(mt[i].definer);
 
 	return rte_errno;
 }
 
-void mlx5dr_definer_put(struct mlx5dr_match_template *mt)
+static void
+mlx5dr_definer_matcher_match_uninit(struct mlx5dr_matcher *matcher)
 {
-	if (--mt->refcount)
+	int i;
+
+	for (i = 0; i < matcher->num_of_mt; i++)
+		mlx5dr_definer_free(matcher->mt[i].definer);
+}
+
+int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
+				struct mlx5dr_matcher *matcher)
+{
+	struct mlx5dr_definer match_layout = {0};
+	int ret, i;
+
+	if (matcher->flags & MLX5DR_MATCHER_FLAGS_COLISION)
+		return 0;
+
+	/* Calculate header layout based on matcher items */
+	ret = mlx5dr_definer_calc_layout(matcher, &match_layout);
+	if (ret) {
+		DR_LOG(ERR, "Failed to calculate matcher definer layout");
+		return ret;
+	}
+
+	/* Calculate definers needed for exact match */
+	ret = mlx5dr_definer_matcher_match_init(ctx, matcher, &match_layout);
+	if (ret) {
+		DR_LOG(ERR, "Failed to init match definers");
+		goto free_fc;
+	}
+
+	return 0;
+
+free_fc:
+	for (i = 0; i < matcher->num_of_mt; i++)
+		simple_free(matcher->mt[i].fc);
+
+	return ret;
+}
+
+void mlx5dr_definer_matcher_uninit(struct mlx5dr_matcher *matcher)
+{
+	int i;
+
+	if (matcher->flags & MLX5DR_MATCHER_FLAGS_COLISION)
 		return;
 
-	simple_free(mt->fc);
-	mlx5dr_cmd_destroy_obj(mt->definer->obj);
-	simple_free(mt->definer);
+	mlx5dr_definer_matcher_match_uninit(matcher);
+
+	for (i = 0; i < matcher->num_of_mt; i++)
+		simple_free(matcher->mt[i].fc);
 }
drivers/net/mlx5/hws/mlx5dr_definer.h +4 -7
index bab4baae4a..a14a08838a 100644
@@ -573,14 +573,11 @@ void mlx5dr_definer_create_tag(const struct rte_flow_item *items,
 			       uint32_t fc_sz,
 			       uint8_t *tag);
 
-int mlx5dr_definer_compare(struct mlx5dr_definer *definer_a,
-			   struct mlx5dr_definer *definer_b);
-
 int mlx5dr_definer_get_id(struct mlx5dr_definer *definer);
 
-int mlx5dr_definer_get(struct mlx5dr_context *ctx,
-		       struct mlx5dr_match_template *mt);
+int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
+				struct mlx5dr_matcher *matcher);
 
-void mlx5dr_definer_put(struct mlx5dr_match_template *mt);
+void mlx5dr_definer_matcher_uninit(struct mlx5dr_matcher *matcher);
 
-#endif /* MLX5DR_DEFINER_H_ */
+#endif
drivers/net/mlx5/hws/mlx5dr_internal.h +1 -1
index faad2bbd0f..c3c077667d 100644
@@ -30,12 +30,12 @@
 #include "mlx5dr_pool.h"
 #include "mlx5dr_context.h"
 #include "mlx5dr_table.h"
-#include "mlx5dr_matcher.h"
 #include "mlx5dr_send.h"
 #include "mlx5dr_rule.h"
 #include "mlx5dr_cmd.h"
 #include "mlx5dr_action.h"
 #include "mlx5dr_definer.h"
+#include "mlx5dr_matcher.h"
 #include "mlx5dr_debug.h"
 #include "mlx5dr_pat_arg.h"
 
drivers/net/mlx5/hws/mlx5dr_matcher.c +19 -42

133 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_matcher.h +14 -2
index b957f5ea4b..4bdb33b11f 100644
@@ -22,15 +22,19 @@
 /* Required depth of the main large table */
 #define MLX5DR_MATCHER_ASSURED_MAIN_TBL_DEPTH 2
 
+enum mlx5dr_matcher_flags {
+	MLX5DR_MATCHER_FLAGS_COLISION		= 1 << 0,
+};
+
 struct mlx5dr_match_template {
 	struct rte_flow_item *items;
 	struct mlx5dr_definer *definer;
+	struct mlx5dr_definer *range_definer;
 	struct mlx5dr_definer_fc *fc;
-	uint32_t fc_sz;
+	uint16_t fc_sz;
 	uint64_t item_flags;
 	uint8_t vport_item_id;
 	enum mlx5dr_match_template_flags flags;
-	uint32_t refcount;
 };
 
 struct mlx5dr_matcher_match_ste {
@@ -59,6 +63,8 @@ struct mlx5dr_matcher {
 	uint8_t num_of_mt;
 	struct mlx5dr_action_template *at;
 	uint8_t num_of_at;
+	/* enum mlx5dr_matcher_flags */
+	uint8_t flags;
 	struct mlx5dr_devx_obj *end_ft;
 	struct mlx5dr_matcher *col_matcher;
 	struct mlx5dr_matcher_match_ste match_ste;
@@ -66,6 +72,12 @@ struct mlx5dr_matcher {
 	LIST_ENTRY(mlx5dr_matcher) next;
 };
 
+static inline bool
+mlx5dr_matcher_mt_is_jumbo(struct mlx5dr_match_template *mt)
+{
+	return mlx5dr_definer_is_jumbo(mt->definer);
+}
+
 int mlx5dr_matcher_conv_items_to_prm(uint64_t *match_buf,
 				     struct rte_flow_item *items,
 				     uint8_t *match_criteria,
Signature · 1 line
2.18.1
#11 lore raw

PATCH 11/16 net/mlx5/hws: support partial hash

In reply to Alex Vesker

Hash definers allow performing hashing over a subset of the fields which are used for matching. This allows combining match templates which were considered invalid until now. During matcher creation mlx5dr code will process the match templates and check if such hash definer is needed based on the definers bitmasks intersection.
Since current HW GTA implementation doesn't allow specifying match and hash definers rule insertion is done using the FW GTA WQE command.

---
 drivers/common/mlx5/mlx5_prm.h        |   4 +
 drivers/net/mlx5/hws/mlx5dr_definer.c | 105 ++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_matcher.c |  66 +++++++++++++++-
 drivers/net/mlx5/hws/mlx5dr_matcher.h |  10 ++-
 4 files changed, 181 insertions(+), 4 deletions(-)
drivers/common/mlx5/mlx5_prm.h +4 -0
index cf46296afb..cca2fb6af7 100644
@@ -2112,6 +2112,10 @@ enum mlx5_ifc_cross_vhca_allowed_objects_types {
 	MLX5_CROSS_VHCA_ALLOWED_OBJS_RTC = 1 << 0xa,
 };
 
+enum {
+	MLX5_GENERATE_WQE_TYPE_FLOW_UPDATE = 1 << 1,
+};
+
 /*
  *  HCA Capabilities 2
  */
drivers/net/mlx5/hws/mlx5dr_definer.c +105 -0

131 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_matcher.c +63 -3

108 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_matcher.h +9 -1
index 4bdb33b11f..c012c0c193 100644
@@ -23,7 +23,8 @@
 #define MLX5DR_MATCHER_ASSURED_MAIN_TBL_DEPTH 2
 
 enum mlx5dr_matcher_flags {
-	MLX5DR_MATCHER_FLAGS_COLISION		= 1 << 0,
+	MLX5DR_MATCHER_FLAGS_HASH_DEFINER	= 1 << 0,
+	MLX5DR_MATCHER_FLAGS_COLISION		= 1 << 1,
 };
 
 struct mlx5dr_match_template {
@@ -69,6 +70,7 @@ struct mlx5dr_matcher {
 	struct mlx5dr_matcher *col_matcher;
 	struct mlx5dr_matcher_match_ste match_ste;
 	struct mlx5dr_matcher_action_ste action_ste;
+	struct mlx5dr_definer *hash_definer;
 	LIST_ENTRY(mlx5dr_matcher) next;
 };
 
@@ -78,6 +80,12 @@ mlx5dr_matcher_mt_is_jumbo(struct mlx5dr_match_template *mt)
 	return mlx5dr_definer_is_jumbo(mt->definer);
 }
 
+static inline bool mlx5dr_matcher_req_fw_wqe(struct mlx5dr_matcher *matcher)
+{
+	/* Currently HWS doesn't support hash different from match or range */
+	return unlikely(matcher->flags & MLX5DR_MATCHER_FLAGS_HASH_DEFINER);
+}
+
 int mlx5dr_matcher_conv_items_to_prm(uint64_t *match_buf,
 				     struct rte_flow_item *items,
 				     uint8_t *match_criteria,
Signature · 1 line
2.18.1
#12 lore raw

PATCH 12/16 net/mlx5/hws: add range definer creation support

In reply to Alex Vesker

Calculate and create an additional definer used for range check during matcher creation. In such case two definers will be created one for specific matching and a range definer. Since range HW GTA WQE doesn't support the needed range support rule insertion rule insertion is done using the FW GTA WQE command.

---
 drivers/net/mlx5/hws/mlx5dr_definer.c | 255 +++++++++++++++++++++++---
 drivers/net/mlx5/hws/mlx5dr_definer.h |  16 +-
 drivers/net/mlx5/hws/mlx5dr_matcher.c |  27 ++-
 drivers/net/mlx5/hws/mlx5dr_matcher.h |  17 +-
 4 files changed, 281 insertions(+), 34 deletions(-)
drivers/net/mlx5/hws/mlx5dr_definer.c +227 -28
index 260e6c5d1d..cf84fbea71 100644
@@ -1508,26 +1508,33 @@ mlx5dr_definer_mt_set_fc(struct mlx5dr_match_template *mt,
 			 struct mlx5dr_definer_fc *fc,
 			 uint8_t *hl)
 {
-	uint32_t fc_sz = 0;
+	uint32_t fc_sz = 0, fcr_sz = 0;
 	int i;
 
 	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++)
 		if (fc[i].tag_set)
-			fc_sz++;
+			fc[i].is_range ? fcr_sz++ : fc_sz++;
 
-	mt->fc = simple_calloc(fc_sz, sizeof(*mt->fc));
+	mt->fc = simple_calloc(fc_sz + fcr_sz, sizeof(*mt->fc));
 	if (!mt->fc) {
 		rte_errno = ENOMEM;
 		return rte_errno;
 	}
 
+	mt->fcr = mt->fc + fc_sz;
+
 	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++) {
 		if (!fc[i].tag_set)
 			continue;
 
 		fc[i].fname = i;
-		memcpy(&mt->fc[mt->fc_sz++], &fc[i], sizeof(*mt->fc));
-		DR_SET(hl, -1, fc[i].byte_off, fc[i].bit_off, fc[i].bit_mask);
+
+		if (fc[i].is_range) {
+			memcpy(&mt->fcr[mt->fcr_sz++], &fc[i], sizeof(*mt->fcr));
+		} else {
+			memcpy(&mt->fc[mt->fc_sz++], &fc[i], sizeof(*mt->fc));
+			DR_SET(hl, -1, fc[i].byte_off, fc[i].bit_off, fc[i].bit_mask);
+		}
 	}
 
 	return 0;
@@ -1686,7 +1693,7 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 
 	mt->item_flags = item_flags;
 
-	/* Fill in headers layout and allocate fc array on mt */
+	/* Fill in headers layout and allocate fc & fcr array on mt */
 	ret = mlx5dr_definer_mt_set_fc(mt, fc, hl);
 	if (ret) {
 		DR_LOG(ERR, "Failed to set field copy to match template");
@@ -1855,9 +1862,92 @@ mlx5dr_definer_copy_sel_ctrl(struct mlx5dr_definer_sel_ctrl *ctrl,
 }
 
 static int
-mlx5dr_definer_find_best_hl_fit(struct mlx5dr_context *ctx,
-				struct mlx5dr_definer *definer,
-				uint8_t *hl)
+mlx5dr_definer_find_best_range_fit(struct mlx5dr_definer *definer,
+				   struct mlx5dr_matcher *matcher)
+{
+	uint8_t tag_byte_offset[MLX5DR_DEFINER_FNAME_MAX] = {0};
+	uint8_t field_select[MLX5DR_DEFINER_FNAME_MAX] = {0};
+	struct mlx5dr_definer_sel_ctrl ctrl = {0};
+	uint32_t byte_offset, algn_byte_off;
+	struct mlx5dr_definer_fc *fcr;
+	bool require_dw;
+	int idx, i, j;
+
+	/* Try to create a range definer */
+	ctrl.allowed_full_dw = DW_SELECTORS_RANGE;
+	ctrl.allowed_bytes = BYTE_SELECTORS_RANGE;
+
+	/* Multiple fields cannot share the same DW for range match.
+	 * The HW doesn't recognize each field but compares the full dw.
+	 * For example definer DW consists of FieldA_FieldB
+	 * FieldA: Mask 0xFFFF range 0x1 to 0x2
+	 * FieldB: Mask 0xFFFF range 0x3 to 0x4
+	 * STE DW range will be 0x00010003 - 0x00020004
+	 * This will cause invalid match for FieldB if FieldA=1 and FieldB=8
+	 * Since 0x10003 < 0x10008 < 0x20004
+	 */
+	for (i = 0; i < matcher->num_of_mt; i++) {
+		for (j = 0; j < matcher->mt[i].fcr_sz; j++) {
+			fcr = &matcher->mt[i].fcr[j];
+
+			/* Found - Reuse previous mt binding */
+			if (field_select[fcr->fname]) {
+				fcr->byte_off = tag_byte_offset[fcr->fname];
+				continue;
+			}
+
+			/* Not found */
+			require_dw = fcr->byte_off >= (64 * DW_SIZE);
+			if (require_dw || ctrl.used_bytes == ctrl.allowed_bytes) {
+				/* Try to cover using DW selector */
+				if (ctrl.used_full_dw == ctrl.allowed_full_dw)
+					goto not_supported;
+
+				ctrl.full_dw_selector[ctrl.used_full_dw++] =
+					fcr->byte_off / DW_SIZE;
+
+				/* Bind DW */
+				idx = ctrl.used_full_dw - 1;
+				byte_offset = fcr->byte_off % DW_SIZE;
+				byte_offset += DW_SIZE * (DW_SELECTORS - idx - 1);
+			} else {
+				/* Try to cover using Bytes selectors */
+				if (ctrl.used_bytes == ctrl.allowed_bytes)
+					goto not_supported;
+
+				algn_byte_off = DW_SIZE * (fcr->byte_off / DW_SIZE);
+				ctrl.byte_selector[ctrl.used_bytes++] = algn_byte_off + 3;
+				ctrl.byte_selector[ctrl.used_bytes++] = algn_byte_off + 2;
+				ctrl.byte_selector[ctrl.used_bytes++] = algn_byte_off + 1;
+				ctrl.byte_selector[ctrl.used_bytes++] = algn_byte_off;
+
+				/* Bind BYTE */
+				byte_offset = DW_SIZE * DW_SELECTORS;
+				byte_offset += BYTE_SELECTORS - ctrl.used_bytes;
+				byte_offset += fcr->byte_off % DW_SIZE;
+			}
+
+			fcr->byte_off = byte_offset;
+			tag_byte_offset[fcr->fname] = byte_offset;
+			field_select[fcr->fname] = 1;
+		}
+	}
+
+	mlx5dr_definer_copy_sel_ctrl(&ctrl, definer);
+	definer->type = MLX5DR_DEFINER_TYPE_RANGE;
+
+	return 0;
+
+not_supported:
+	DR_LOG(ERR, "Unable to find supporting range definer combination");
+	rte_errno = ENOTSUP;
+	return rte_errno;
+}
+
+static int
+mlx5dr_definer_find_best_match_fit(struct mlx5dr_context *ctx,
+				   struct mlx5dr_definer *definer,
+				   uint8_t *hl)
 {
 	struct mlx5dr_definer_sel_ctrl ctrl = {0};
 	bool found;
@@ -1923,6 +2013,43 @@ void mlx5dr_definer_create_tag(const struct rte_flow_item *items,
 	}
 }
 
+static uint32_t mlx5dr_definer_get_range_byte_off(uint32_t match_byte_off)
+{
+	uint8_t curr_dw_idx = match_byte_off / DW_SIZE;
+	uint8_t new_dw_idx;
+
+	/* Range DW can have the following values 7,8,9,10
+	 * -DW7 is mapped to DW9
+	 * -DW8 is mapped to DW7
+	 * -DW9 is mapped to DW5
+	 * -DW10 is mapped to DW3
+	 * To reduce calculation the following formula is used:
+	 */
+	new_dw_idx = curr_dw_idx * (-2) + 23;
+
+	return new_dw_idx * DW_SIZE + match_byte_off % DW_SIZE;
+}
+
+void mlx5dr_definer_create_tag_range(const struct rte_flow_item *items,
+				     struct mlx5dr_definer_fc *fc,
+				     uint32_t fc_sz,
+				     uint8_t *tag)
+{
+	struct mlx5dr_definer_fc tmp_fc;
+	uint32_t i;
+
+	for (i = 0; i < fc_sz; i++) {
+		tmp_fc = *fc;
+		/* Set MAX value */
+		tmp_fc.byte_off = mlx5dr_definer_get_range_byte_off(fc->byte_off);
+		tmp_fc.tag_set(&tmp_fc, items[fc->item_idx].last, tag);
+		/* Set MIN value */
+		tmp_fc.byte_off += DW_SIZE;
+		tmp_fc.tag_set(&tmp_fc, items[fc->item_idx].spec, tag);
+		fc++;
+	}
+}
+
 int mlx5dr_definer_get_id(struct mlx5dr_definer *definer)
 {
 	return definer->obj->id;
@@ -1951,27 +2078,26 @@ mlx5dr_definer_compare(struct mlx5dr_definer *definer_a,
 
 static int
 mlx5dr_definer_calc_layout(struct mlx5dr_matcher *matcher,
-			   struct mlx5dr_definer *match_definer)
+			   struct mlx5dr_definer *match_definer,
+			   struct mlx5dr_definer *range_definer)
 {
 	struct mlx5dr_context *ctx = matcher->tbl->ctx;
 	struct mlx5dr_match_template *mt = matcher->mt;
-	uint8_t *match_hl, *hl_buff;
+	uint8_t *match_hl;
 	int i, ret;
 
 	/* Union header-layout (hl) is used for creating a single definer
 	 * field layout used with different bitmasks for hash and match.
 	 */
-	hl_buff = simple_calloc(1, MLX5_ST_SZ_BYTES(definer_hl));
-	if (!hl_buff) {
+	match_hl = simple_calloc(1, MLX5_ST_SZ_BYTES(definer_hl));
+	if (!match_hl) {
 		DR_LOG(ERR, "Failed to allocate memory for header layout");
 		rte_errno = ENOMEM;
 		return rte_errno;
 	}
 
-	match_hl = hl_buff;
-
 	/* Convert all mt items to header layout (hl)
-	 * and allocate the match field copy array (fc).
+	 * and allocate the match and range field copy array (fc & fcr).
 	 */
 	for (i = 0; i < matcher->num_of_mt; i++) {
 		ret = mlx5dr_definer_conv_items_to_hl(ctx, &mt[i], match_hl);
@@ -1982,13 +2108,20 @@ mlx5dr_definer_calc_layout(struct mlx5dr_matcher *matcher,
 	}
 
 	/* Find the match definer layout for header layout match union */
-	ret = mlx5dr_definer_find_best_hl_fit(ctx, match_definer, match_hl);
+	ret = mlx5dr_definer_find_best_match_fit(ctx, match_definer, match_hl);
 	if (ret) {
 		DR_LOG(ERR, "Failed to create match definer from header layout");
 		goto free_fc;
 	}
 
-	simple_free(hl_buff);
+	/* Find the range definer layout for match templates fcrs */
+	ret = mlx5dr_definer_find_best_range_fit(range_definer, matcher);
+	if (ret) {
+		DR_LOG(ERR, "Failed to create range definer from header layout");
+		goto free_fc;
+	}
+
+	simple_free(match_hl);
 	return 0;
 
 free_fc:
@@ -1996,7 +2129,7 @@ mlx5dr_definer_calc_layout(struct mlx5dr_matcher *matcher,
 		if (mt[i].fc)
 			simple_free(mt[i].fc);
 
-	simple_free(hl_buff);
+	simple_free(match_hl);
 	return rte_errno;
 }
 
@@ -2005,7 +2138,8 @@ mlx5dr_definer_alloc(struct ibv_context *ibv_ctx,
 		     struct mlx5dr_definer_fc *fc,
 		     int fc_sz,
 		     struct rte_flow_item *items,
-		     struct mlx5dr_definer *layout)
+		     struct mlx5dr_definer *layout,
+		     bool bind_fc)
 {
 	struct mlx5dr_cmd_definer_create_attr def_attr = {0};
 	struct mlx5dr_definer *definer;
@@ -2021,10 +2155,12 @@ mlx5dr_definer_alloc(struct ibv_context *ibv_ctx,
 	memcpy(definer, layout, sizeof(*definer));
 
 	/* Align field copy array based on given layout */
-	ret = mlx5dr_definer_fc_bind(definer, fc, fc_sz);
-	if (ret) {
-		DR_LOG(ERR, "Failed to bind field copy to definer");
-		goto free_definer;
+	if (bind_fc) {
+		ret = mlx5dr_definer_fc_bind(definer, fc, fc_sz);
+		if (ret) {
+			DR_LOG(ERR, "Failed to bind field copy to definer");
+			goto free_definer;
+		}
 	}
 
 	/* Create the tag mask used for definer creation */
@@ -2067,7 +2203,8 @@ mlx5dr_definer_matcher_match_init(struct mlx5dr_context *ctx,
 						     mt[i].fc,
 						     mt[i].fc_sz,
 						     mt[i].items,
-						     match_layout);
+						     match_layout,
+						     true);
 		if (!mt[i].definer) {
 			DR_LOG(ERR, "Failed to create match definer");
 			goto free_definers;
@@ -2091,6 +2228,58 @@ mlx5dr_definer_matcher_match_uninit(struct mlx5dr_matcher *matcher)
 		mlx5dr_definer_free(matcher->mt[i].definer);
 }
 
+static int
+mlx5dr_definer_matcher_range_init(struct mlx5dr_context *ctx,
+				  struct mlx5dr_matcher *matcher,
+				  struct mlx5dr_definer *range_layout)
+{
+	struct mlx5dr_match_template *mt = matcher->mt;
+	int i;
+
+	/* Create optional range definers */
+	for (i = 0; i < matcher->num_of_mt; i++) {
+		if (!mt[i].fcr_sz)
+			continue;
+
+		/* All must use range if requested */
+		if (i && !mt[i - 1].range_definer) {
+			DR_LOG(ERR, "Using range and non range templates is not allowed");
+			goto free_definers;
+		}
+
+		matcher->flags |= MLX5DR_MATCHER_FLAGS_RANGE_DEFINER;
+		/* Create definer without fcr binding, already binded */
+		mt[i].range_definer = mlx5dr_definer_alloc(ctx->ibv_ctx,
+							   mt[i].fcr,
+							   mt[i].fcr_sz,
+							   mt[i].items,
+							   range_layout,
+							   false);
+		if (!mt[i].range_definer) {
+			DR_LOG(ERR, "Failed to create match definer");
+			goto free_definers;
+		}
+	}
+	return 0;
+
+free_definers:
+	while (i--)
+		if (mt[i].range_definer)
+			mlx5dr_definer_free(mt[i].range_definer);
+
+	return rte_errno;
+}
+
+static void
+mlx5dr_definer_matcher_range_uninit(struct mlx5dr_matcher *matcher)
+{
+	int i;
+
+	for (i = 0; i < matcher->num_of_mt; i++)
+		if (matcher->mt[i].range_definer)
+			mlx5dr_definer_free(matcher->mt[i].range_definer);
+}
+
 static int
 mlx5dr_definer_matcher_hash_init(struct mlx5dr_context *ctx,
 				 struct mlx5dr_matcher *matcher)
@@ -2169,13 +2358,13 @@ int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
 				struct mlx5dr_matcher *matcher)
 {
 	struct mlx5dr_definer match_layout = {0};
+	struct mlx5dr_definer range_layout = {0};
 	int ret, i;
 
 	if (matcher->flags & MLX5DR_MATCHER_FLAGS_COLISION)
 		return 0;
 
-	/* Calculate header layout based on matcher items */
-	ret = mlx5dr_definer_calc_layout(matcher, &match_layout);
+	ret = mlx5dr_definer_calc_layout(matcher, &match_layout, &range_layout);
 	if (ret) {
 		DR_LOG(ERR, "Failed to calculate matcher definer layout");
 		return ret;
@@ -2188,15 +2377,24 @@ int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
 		goto free_fc;
 	}
 
+	/* Calculate definers needed for range */
+	ret = mlx5dr_definer_matcher_range_init(ctx, matcher, &range_layout);
+	if (ret) {
+		DR_LOG(ERR, "Failed to init range definers");
+		goto uninit_match_definer;
+	}
+
 	/* Calculate partial hash definer */
 	ret = mlx5dr_definer_matcher_hash_init(ctx, matcher);
 	if (ret) {
 		DR_LOG(ERR, "Failed to init hash definer");
-		goto uninit_match_definer;
+		goto uninit_range_definer;
 	}
 
 	return 0;
 
+uninit_range_definer:
+	mlx5dr_definer_matcher_range_uninit(matcher);
 uninit_match_definer:
 	mlx5dr_definer_matcher_match_uninit(matcher);
 free_fc:
@@ -2214,6 +2412,7 @@ void mlx5dr_definer_matcher_uninit(struct mlx5dr_matcher *matcher)
 		return;
 
 	mlx5dr_definer_matcher_hash_uninit(matcher);
+	mlx5dr_definer_matcher_range_uninit(matcher);
 	mlx5dr_definer_matcher_match_uninit(matcher);
 
 	for (i = 0; i < matcher->num_of_mt; i++)
drivers/net/mlx5/hws/mlx5dr_definer.h +14 -2
index a14a08838a..dd9a297007 100644
@@ -5,11 +5,17 @@
 #ifndef MLX5DR_DEFINER_H_
 #define MLX5DR_DEFINER_H_
 
+/* Max available selecotrs */
+#define DW_SELECTORS 9
+#define BYTE_SELECTORS 8
+
 /* Selectors based on match TAG */
 #define DW_SELECTORS_MATCH 6
 #define DW_SELECTORS_LIMITED 3
-#define DW_SELECTORS 9
-#define BYTE_SELECTORS 8
+
+/* Selectors based on range TAG */
+#define DW_SELECTORS_RANGE 2
+#define BYTE_SELECTORS_RANGE 8
 
 enum mlx5dr_definer_fname {
 	MLX5DR_DEFINER_FNAME_ETH_SMAC_48_16_O,
@@ -112,6 +118,7 @@ enum mlx5dr_definer_fname {
 enum mlx5dr_definer_type {
 	MLX5DR_DEFINER_TYPE_MATCH,
 	MLX5DR_DEFINER_TYPE_JUMBO,
+	MLX5DR_DEFINER_TYPE_RANGE,
 };
 
 struct mlx5dr_definer_fc {
@@ -573,6 +580,11 @@ void mlx5dr_definer_create_tag(const struct rte_flow_item *items,
 			       uint32_t fc_sz,
 			       uint8_t *tag);
 
+void mlx5dr_definer_create_tag_range(const struct rte_flow_item *items,
+				     struct mlx5dr_definer_fc *fc,
+				     uint32_t fc_sz,
+				     uint8_t *tag);
+
 int mlx5dr_definer_get_id(struct mlx5dr_definer *definer);
 
 int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
drivers/net/mlx5/hws/mlx5dr_matcher.c +26 -1
index e860c274cf..de688f6873 100644
@@ -360,6 +360,12 @@ static bool mlx5dr_matcher_supp_fw_wqe(struct mlx5dr_matcher *matcher)
 		return false;
 	}
 
+	if ((matcher->flags & MLX5DR_MATCHER_FLAGS_RANGE_DEFINER) &&
+	    !IS_BIT_SET(caps->supp_ste_fromat_gen_wqe, MLX5_IFC_RTC_STE_FORMAT_RANGE)) {
+		DR_LOG(INFO, "Extended match gen wqe RANGE format not supported");
+		return false;
+	}
+
 	if (!(caps->supp_type_gen_wqe & MLX5_GENERATE_WQE_TYPE_FLOW_UPDATE)) {
 		DR_LOG(ERR, "Gen WQE command not supporting GTA");
 		return false;
@@ -460,14 +466,20 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 		ste = &matcher->match_ste.ste;
 		ste->order = attr->table.sz_col_log + attr->table.sz_row_log;
 
+		/* Add additional rows due to additional range STE */
+		if (mlx5dr_matcher_mt_is_range(mt))
+			ste->order++;
+
 		rtc_attr.log_size = attr->table.sz_row_log;
 		rtc_attr.log_depth = attr->table.sz_col_log;
 		rtc_attr.is_frst_jumbo = mlx5dr_matcher_mt_is_jumbo(mt);
+		rtc_attr.is_scnd_range = mlx5dr_matcher_mt_is_range(mt);
 		rtc_attr.miss_ft_id = matcher->end_ft->id;
 
 		if (attr->insert_mode == MLX5DR_MATCHER_INSERT_BY_HASH) {
 			/* The usual Hash Table */
 			rtc_attr.update_index_mode = MLX5_IFC_RTC_STE_UPDATE_MODE_BY_HASH;
+
 			if (matcher->hash_definer) {
 				/* Specify definer_id_0 is used for hashing */
 				rtc_attr.fw_gen_wqe = true;
@@ -477,6 +489,16 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 			} else {
 				/* The first mt is used since all share the same definer */
 				rtc_attr.match_definer_0 = mlx5dr_definer_get_id(mt->definer);
+
+				/* This is tricky, instead of passing two definers for
+				 * match and range, we specify that this RTC uses a hash
+				 * definer, this will allow us to use any range definer
+				 * since only first STE is used for hashing anyways.
+				 */
+				if (matcher->flags & MLX5DR_MATCHER_FLAGS_RANGE_DEFINER) {
+					rtc_attr.fw_gen_wqe = true;
+					rtc_attr.num_hash_definer = 1;
+				}
 			}
 		} else if (attr->insert_mode == MLX5DR_MATCHER_INSERT_BY_INDEX) {
 			rtc_attr.update_index_mode = MLX5_IFC_RTC_STE_UPDATE_MODE_BY_OFFSET;
@@ -751,7 +773,7 @@ static int mlx5dr_matcher_bind_mt(struct mlx5dr_matcher *matcher)
 	struct mlx5dr_pool_attr pool_attr = {0};
 	int ret;
 
-	/* Calculate match and hash definers */
+	/* Calculate match, range and hash definers */
 	ret = mlx5dr_definer_matcher_init(ctx, matcher);
 	if (ret) {
 		DR_LOG(ERR, "Failed to set matcher templates with match definers");
@@ -772,6 +794,9 @@ static int mlx5dr_matcher_bind_mt(struct mlx5dr_matcher *matcher)
 	pool_attr.flags = MLX5DR_POOL_FLAGS_FOR_MATCHER_STE_POOL;
 	pool_attr.alloc_log_sz = matcher->attr.table.sz_col_log +
 				 matcher->attr.table.sz_row_log;
+	/* Add additional rows due to additional range STE */
+	if (matcher->flags & MLX5DR_MATCHER_FLAGS_RANGE_DEFINER)
+		pool_attr.alloc_log_sz++;
 	mlx5dr_matcher_set_pool_attr(&pool_attr, matcher);
 
 	matcher->match_ste.pool = mlx5dr_pool_create(ctx, &pool_attr);
drivers/net/mlx5/hws/mlx5dr_matcher.h +14 -3
index c012c0c193..a95cfdec6f 100644
@@ -23,8 +23,9 @@
 #define MLX5DR_MATCHER_ASSURED_MAIN_TBL_DEPTH 2
 
 enum mlx5dr_matcher_flags {
-	MLX5DR_MATCHER_FLAGS_HASH_DEFINER	= 1 << 0,
-	MLX5DR_MATCHER_FLAGS_COLISION		= 1 << 1,
+	MLX5DR_MATCHER_FLAGS_RANGE_DEFINER	= 1 << 0,
+	MLX5DR_MATCHER_FLAGS_HASH_DEFINER	= 1 << 1,
+	MLX5DR_MATCHER_FLAGS_COLISION		= 1 << 2,
 };
 
 struct mlx5dr_match_template {
@@ -32,7 +33,9 @@ struct mlx5dr_match_template {
 	struct mlx5dr_definer *definer;
 	struct mlx5dr_definer *range_definer;
 	struct mlx5dr_definer_fc *fc;
+	struct mlx5dr_definer_fc *fcr;
 	uint16_t fc_sz;
+	uint16_t fcr_sz;
 	uint64_t item_flags;
 	uint8_t vport_item_id;
 	enum mlx5dr_match_template_flags flags;
@@ -80,10 +83,18 @@ mlx5dr_matcher_mt_is_jumbo(struct mlx5dr_match_template *mt)
 	return mlx5dr_definer_is_jumbo(mt->definer);
 }
 
+static inline bool
+mlx5dr_matcher_mt_is_range(struct mlx5dr_match_template *mt)
+{
+	return (!!mt->range_definer);
+}
+
 static inline bool mlx5dr_matcher_req_fw_wqe(struct mlx5dr_matcher *matcher)
 {
 	/* Currently HWS doesn't support hash different from match or range */
-	return unlikely(matcher->flags & MLX5DR_MATCHER_FLAGS_HASH_DEFINER);
+	return unlikely(matcher->flags &
+			(MLX5DR_MATCHER_FLAGS_HASH_DEFINER |
+			 MLX5DR_MATCHER_FLAGS_RANGE_DEFINER));
 }
 
 int mlx5dr_matcher_conv_items_to_prm(uint64_t *match_buf,
Signature · 1 line
2.18.1
#13 lore raw

PATCH 13/16 net/mlx5/hws: add FW WQE rule creation logic

In reply to Alex Vesker

FW WQE and HW WQE are done in a similar way but not to jeopardize the performance rule creation is done over the new FW rule creation function. The deletion function is shared between both flows.

---
 drivers/net/mlx5/hws/mlx5dr_rule.c | 180 +++++++++++++++++++++++++++--
 drivers/net/mlx5/hws/mlx5dr_rule.h |   2 +
 drivers/net/mlx5/hws/mlx5dr_send.h |   9 +-
 3 files changed, 180 insertions(+), 11 deletions(-)
drivers/net/mlx5/hws/mlx5dr_rule.c +171 -9
index f5a0c46315..9d5e5b11a5 100644
@@ -112,6 +112,62 @@ static void mlx5dr_rule_gen_comp(struct mlx5dr_send_engine *queue,
 	mlx5dr_send_engine_gen_comp(queue, user_data, comp_status);
 }
 
+static void
+mlx5dr_rule_save_delete_info(struct mlx5dr_rule *rule,
+			     struct mlx5dr_send_ste_attr *ste_attr)
+{
+	if (unlikely(mlx5dr_matcher_req_fw_wqe(rule->matcher))) {
+		uint8_t *src_tag;
+
+		/* Save match definer id and tag for delete */
+		rule->tag_ptr = simple_calloc(2, sizeof(*rule->tag_ptr));
+		assert(rule->tag_ptr);
+
+		src_tag = (uint8_t *)ste_attr->wqe_data->tag;
+		memcpy(rule->tag_ptr[0].match, src_tag, MLX5DR_MATCH_TAG_SZ);
+		rule->tag_ptr[1].reserved[0] = ste_attr->send_attr.match_definer_id;
+
+		/* Save range definer id and tag for delete */
+		if (ste_attr->range_wqe_data) {
+			src_tag = (uint8_t *)ste_attr->range_wqe_data->tag;
+			memcpy(rule->tag_ptr[1].match, src_tag, MLX5DR_MATCH_TAG_SZ);
+			rule->tag_ptr[1].reserved[1] = ste_attr->send_attr.range_definer_id;
+		}
+		return;
+	}
+
+	if (ste_attr->wqe_tag_is_jumbo)
+		memcpy(rule->tag.jumbo, ste_attr->wqe_data->jumbo, MLX5DR_JUMBO_TAG_SZ);
+	else
+		memcpy(rule->tag.match, ste_attr->wqe_data->tag, MLX5DR_MATCH_TAG_SZ);
+}
+
+static void
+mlx5dr_rule_clear_delete_info(struct mlx5dr_rule *rule)
+{
+	if (unlikely(mlx5dr_matcher_req_fw_wqe(rule->matcher)))
+		simple_free(rule->tag_ptr);
+}
+
+static void
+mlx5dr_rule_load_delete_info(struct mlx5dr_rule *rule,
+			     struct mlx5dr_send_ste_attr *ste_attr)
+{
+	if (unlikely(mlx5dr_matcher_req_fw_wqe(rule->matcher))) {
+		/* Load match definer id and tag for delete */
+		ste_attr->wqe_tag = &rule->tag_ptr[0];
+		ste_attr->send_attr.match_definer_id = rule->tag_ptr[1].reserved[0];
+
+		/* Load range definer id and tag for delete */
+		if (rule->matcher->flags & MLX5DR_MATCHER_FLAGS_RANGE_DEFINER) {
+			ste_attr->range_wqe_tag = &rule->tag_ptr[1];
+			ste_attr->send_attr.range_definer_id = rule->tag_ptr[1].reserved[1];
+		}
+	} else {
+		ste_attr->wqe_tag = &rule->tag;
+	}
+}
+
 static int mlx5dr_rule_alloc_action_ste(struct mlx5dr_rule *rule,
 					struct mlx5dr_rule_attr *attr)
 {
@@ -180,6 +236,97 @@ static void mlx5dr_rule_create_init(struct mlx5dr_rule *rule,
 	apply->require_dep = 0;
 }
 
+static int mlx5dr_rule_create_hws_fw_wqe(struct mlx5dr_rule *rule,
+					 struct mlx5dr_rule_attr *attr,
+					 uint8_t mt_idx,
+					 const struct rte_flow_item items[],
+					 uint8_t at_idx,
+					 struct mlx5dr_rule_action rule_actions[])
+{
+	struct mlx5dr_action_template *at = &rule->matcher->at[at_idx];
+	struct mlx5dr_match_template *mt = &rule->matcher->mt[mt_idx];
+	struct mlx5dr_send_ring_dep_wqe range_wqe = {{0}};
+	struct mlx5dr_send_ring_dep_wqe match_wqe = {{0}};
+	bool is_range = mlx5dr_matcher_mt_is_range(mt);
+	bool is_jumbo = mlx5dr_matcher_mt_is_jumbo(mt);
+	struct mlx5dr_matcher *matcher = rule->matcher;
+	struct mlx5dr_context *ctx = matcher->tbl->ctx;
+	struct mlx5dr_send_ste_attr ste_attr = {0};
+	struct mlx5dr_actions_apply_data apply;
+	struct mlx5dr_send_engine *queue;
+
+	queue = &ctx->send_queue[attr->queue_id];
+	if (unlikely(mlx5dr_send_engine_err(queue))) {
+		rte_errno = EIO;
+		return rte_errno;
+	}
+
+	mlx5dr_rule_create_init(rule, &ste_attr, &apply);
+	mlx5dr_rule_init_dep_wqe(&match_wqe, rule, items, mt, attr->user_data);
+	mlx5dr_rule_init_dep_wqe(&range_wqe, rule, items, mt, attr->user_data);
+
+	ste_attr.direct_index = 0;
+	ste_attr.rtc_0 = match_wqe.rtc_0;
+	ste_attr.rtc_1 = match_wqe.rtc_1;
+	ste_attr.used_id_rtc_0 = &rule->rtc_0;
+	ste_attr.used_id_rtc_1 = &rule->rtc_1;
+	ste_attr.retry_rtc_0 = match_wqe.retry_rtc_0;
+	ste_attr.retry_rtc_1 = match_wqe.retry_rtc_1;
+	ste_attr.send_attr.rule = match_wqe.rule;
+	ste_attr.send_attr.user_data = match_wqe.user_data;
+
+	ste_attr.send_attr.fence = 1;
+	ste_attr.send_attr.notify_hw = 1;
+	ste_attr.wqe_tag_is_jumbo = is_jumbo;
+
+	/* Prepare match STE TAG */
+	ste_attr.wqe_ctrl = &match_wqe.wqe_ctrl;
+	ste_attr.wqe_data = &match_wqe.wqe_data;
+	ste_attr.send_attr.match_definer_id = mlx5dr_definer_get_id(mt->definer);
+
+	mlx5dr_definer_create_tag(items,
+				  mt->fc,
+				  mt->fc_sz,
+				  (uint8_t *)match_wqe.wqe_data.action);
+
+	/* Prepare range STE TAG */
+	if (is_range) {
+		ste_attr.range_wqe_data = &range_wqe.wqe_data;
+		ste_attr.send_attr.len += MLX5DR_WQE_SZ_GTA_DATA;
+		ste_attr.send_attr.range_definer_id = mlx5dr_definer_get_id(mt->range_definer);
+
+		mlx5dr_definer_create_tag_range(items,
+						mt->fcr,
+						mt->fcr_sz,
+						(uint8_t *)range_wqe.wqe_data.action);
+	}
+
+	/* Apply the actions on the last STE */
+	apply.queue = queue;
+	apply.next_direct_idx = 0;
+	apply.rule_action = rule_actions;
+	apply.wqe_ctrl = &match_wqe.wqe_ctrl;
+	apply.wqe_data = (uint32_t *)(is_range ?
+				      &range_wqe.wqe_data :
+				      &match_wqe.wqe_data);
+
+	/* Skip setters[0] used for jumbo STE since not support with FW WQE */
+	mlx5dr_action_apply_setter(&apply, &at->setters[1], 0);
+
+	/* Send WQEs to FW */
+	mlx5dr_send_stes_fw(queue, &ste_attr);
+
+	/* Backup TAG on the rule for deletion */
+	mlx5dr_rule_save_delete_info(rule, &ste_attr);
+	mlx5dr_send_engine_inc_rule(queue);
+
+	/* Send dependent WQEs */
+	if (!attr->burst)
+		mlx5dr_send_all_dep_wqe(queue);
+
+	return 0;
+}
+
 static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 				  struct mlx5dr_rule_attr *attr,
 				  uint8_t mt_idx,
@@ -189,7 +336,7 @@ static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 {
 	struct mlx5dr_action_template *at = &rule->matcher->at[at_idx];
 	struct mlx5dr_match_template *mt = &rule->matcher->mt[mt_idx];
-	bool is_jumbo = mlx5dr_definer_is_jumbo(mt->definer);
+	bool is_jumbo = mlx5dr_matcher_mt_is_jumbo(mt);
 	struct mlx5dr_matcher *matcher = rule->matcher;
 	struct mlx5dr_context *ctx = matcher->tbl->ctx;
 	struct mlx5dr_send_ste_attr ste_attr = {0};
@@ -200,6 +347,11 @@ static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 	uint8_t total_stes, action_stes;
 	int i, ret;
 
+	/* Insert rule using FW WQE if cannot use GTA WQE */
+	if (unlikely(mlx5dr_matcher_req_fw_wqe(matcher)))
+		return mlx5dr_rule_create_hws_fw_wqe(rule, attr, mt_idx, items,
+						     at_idx, rule_actions);
+
 	queue = &ctx->send_queue[attr->queue_id];
 	if (unlikely(mlx5dr_send_engine_err(queue))) {
 		rte_errno = EIO;
@@ -283,11 +435,7 @@ static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 	}
 
 	/* Backup TAG on the rule for deletion */
-	if (is_jumbo)
-		memcpy(rule->tag.jumbo, dep_wqe->wqe_data.action, MLX5DR_JUMBO_TAG_SZ);
-	else
-		memcpy(rule->tag.match, dep_wqe->wqe_data.tag, MLX5DR_MATCH_TAG_SZ);
-
+	mlx5dr_rule_save_delete_info(rule, &ste_attr);
 	mlx5dr_send_engine_inc_rule(queue);
 
 	/* Send dependent WQEs */
@@ -311,6 +459,9 @@ static void mlx5dr_rule_destroy_failed_hws(struct mlx5dr_rule *rule,
 	/* Rule failed now we can safely release action STEs */
 	mlx5dr_rule_free_action_ste_idx(rule);
 
+	/* Clear complex tag */
+	mlx5dr_rule_clear_delete_info(rule);
+
 	/* If a rule that was indicated as burst (need to trigger HW) has failed
 	 * insertion we won't ring the HW as nothing is being written to the WQ.
 	 * In such case update the last WQE and ring the HW with that work
@@ -327,6 +478,9 @@ static int mlx5dr_rule_destroy_hws(struct mlx5dr_rule *rule,
 {
 	struct mlx5dr_context *ctx = rule->matcher->tbl->ctx;
 	struct mlx5dr_matcher *matcher = rule->matcher;
+	bool fw_wqe = mlx5dr_matcher_req_fw_wqe(matcher);
+	bool is_range = mlx5dr_matcher_mt_is_range(matcher->mt);
+	bool is_jumbo = mlx5dr_matcher_mt_is_jumbo(matcher->mt);
 	struct mlx5dr_wqe_gta_ctrl_seg wqe_ctrl = {0};
 	struct mlx5dr_send_ste_attr ste_attr = {0};
 	struct mlx5dr_send_engine *queue;
@@ -361,6 +515,8 @@ static int mlx5dr_rule_destroy_hws(struct mlx5dr_rule *rule,
 	ste_attr.send_attr.opmod = MLX5DR_WQE_GTA_OPMOD_STE;
 	ste_attr.send_attr.opcode = MLX5DR_WQE_OPCODE_TBL_ACCESS;
 	ste_attr.send_attr.len = MLX5DR_WQE_SZ_GTA_CTRL + MLX5DR_WQE_SZ_GTA_DATA;
+	if (unlikely(is_range))
+		ste_attr.send_attr.len += MLX5DR_WQE_SZ_GTA_DATA;
 
 	ste_attr.send_attr.rule = rule;
 	ste_attr.send_attr.notify_hw = !attr->burst;
@@ -371,13 +527,19 @@ static int mlx5dr_rule_destroy_hws(struct mlx5dr_rule *rule,
 	ste_attr.used_id_rtc_0 = &rule->rtc_0;
 	ste_attr.used_id_rtc_1 = &rule->rtc_1;
 	ste_attr.wqe_ctrl = &wqe_ctrl;
-	ste_attr.wqe_tag = &rule->tag;
-	ste_attr.wqe_tag_is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt->definer);
+	ste_attr.wqe_tag_is_jumbo = is_jumbo;
 	ste_attr.gta_opcode = MLX5DR_WQE_GTA_OP_DEACTIVATE;
 	if (unlikely(mlx5dr_matcher_is_insert_by_idx(matcher)))
 		ste_attr.direct_index = attr->rule_idx;
 
-	mlx5dr_send_ste(queue, &ste_attr);
+	mlx5dr_rule_load_delete_info(rule, &ste_attr);
+
+	if (unlikely(fw_wqe)) {
+		mlx5dr_send_stes_fw(queue, &ste_attr);
+		mlx5dr_rule_clear_delete_info(rule);
+	} else {
+		mlx5dr_send_ste(queue, &ste_attr);
+	}
 
 	return 0;
 }
drivers/net/mlx5/hws/mlx5dr_rule.h +2 -0
index f2fe418159..886cf77992 100644
@@ -36,6 +36,8 @@ struct mlx5dr_rule {
 	struct mlx5dr_matcher *matcher;
 	union {
 		struct mlx5dr_rule_match_tag tag;
+		/* Pointer to tag to store more than one tag */
+		struct mlx5dr_rule_match_tag *tag_ptr;
 		struct ibv_flow *flow;
 	};
 	uint32_t rtc_0; /* The RTC into which the STE was inserted */
drivers/net/mlx5/hws/mlx5dr_send.h +7 -2
index 47bb66b3c7..d0977ec851 100644
@@ -54,8 +54,13 @@ struct mlx5dr_wqe_gta_data_seg_ste {
 	__be32 rsvd0_ctr_id;
 	__be32 rsvd1_definer;
 	__be32 rsvd2[3];
-	__be32 action[3];
-	__be32 tag[8];
+	union {
+		struct {
+			__be32 action[3];
+			__be32 tag[8];
+		};
+		__be32 jumbo[11];
+	};
 };
 
 struct mlx5dr_wqe_gta_data_seg_arg {
Signature · 1 line
2.18.1
#14 lore raw

PATCH 14/16 net/mlx5/hws: add debug dump support for range and hash

In reply to Alex Vesker

Add support for dumping range and hash definers objects. Hash definer is a per matcher object describing the fields used for hashing. Range definer is per match template object describing the fields used for range matching. Both are optional based on the given match templates.

---
 drivers/net/mlx5/hws/mlx5dr_debug.c | 37 ++++++++++++++++++++---------
 drivers/net/mlx5/hws/mlx5dr_debug.h |  4 +++-
 2 files changed, 29 insertions(+), 12 deletions(-)
drivers/net/mlx5/hws/mlx5dr_debug.c +26 -11
index 9199ec16e0..b1d271eebe 100644
@@ -34,15 +34,19 @@ const char *mlx5dr_debug_action_type_to_str(enum mlx5dr_action_type action_type)
 
 static int
 mlx5dr_debug_dump_matcher_template_definer(FILE *f,
-					   struct mlx5dr_match_template *mt)
+					   void *parent_obj,
+					   struct mlx5dr_definer *definer,
+					   enum mlx5dr_debug_res_type type)
 {
-	struct mlx5dr_definer *definer = mt->definer;
 	int i, ret;
 
+	if (!definer)
+		return 0;
+
 	ret = fprintf(f, "%d,0x%" PRIx64 ",0x%" PRIx64 ",%d,%d,",
-		      MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_DEFINER,
+		      type,
 		      (uint64_t)(uintptr_t)definer,
-		      (uint64_t)(uintptr_t)mt,
+		      (uint64_t)(uintptr_t)parent_obj,
 		      definer->obj->id,
 		      definer->type);
 	if (ret < 0) {
@@ -89,29 +93,40 @@ static int
 mlx5dr_debug_dump_matcher_match_template(FILE *f, struct mlx5dr_matcher *matcher)
 {
 	bool is_root = matcher->tbl->level == MLX5DR_ROOT_LEVEL;
+	enum mlx5dr_debug_res_type res_type;
 	int i, ret;
 
 	for (i = 0; i < matcher->num_of_mt; i++) {
 		struct mlx5dr_match_template *mt = &matcher->mt[i];
 
-		ret = fprintf(f, "%d,0x%" PRIx64 ",0x%" PRIx64 ",%d,%d\n",
+		ret = fprintf(f, "%d,0x%" PRIx64 ",0x%" PRIx64 ",%d,%d,%d\n",
 			      MLX5DR_DEBUG_RES_TYPE_MATCHER_MATCH_TEMPLATE,
 			      (uint64_t)(uintptr_t)mt,
 			      (uint64_t)(uintptr_t)matcher,
 			      is_root ? 0 : mt->fc_sz,
-			      mt->flags);
+			      mt->flags,
+			      is_root ? 0 : mt->fcr_sz);
 		if (ret < 0) {
 			rte_errno = EINVAL;
 			return rte_errno;
 		}
 
-		if (!is_root) {
-			ret = mlx5dr_debug_dump_matcher_template_definer(f, mt);
-			if (ret)
-				return ret;
-		}
+		res_type = MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_MATCH_DEFINER;
+		ret = mlx5dr_debug_dump_matcher_template_definer(f, mt, mt->definer, res_type);
+		if (ret)
+			return ret;
+
+		res_type = MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_RANGE_DEFINER;
+		ret = mlx5dr_debug_dump_matcher_template_definer(f, mt, mt->range_definer, res_type);
+		if (ret)
+			return ret;
 	}
 
+	res_type = MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_HASH_DEFINER;
+	ret = mlx5dr_debug_dump_matcher_template_definer(f, matcher, matcher->hash_definer, res_type);
+	if (ret)
+		return ret;
+
 	return 0;
 }
 
drivers/net/mlx5/hws/mlx5dr_debug.h +3 -1
index cf00170f7d..2c29ca295c 100644
@@ -19,8 +19,10 @@ enum mlx5dr_debug_res_type {
 	MLX5DR_DEBUG_RES_TYPE_MATCHER = 4200,
 	MLX5DR_DEBUG_RES_TYPE_MATCHER_ATTR = 4201,
 	MLX5DR_DEBUG_RES_TYPE_MATCHER_MATCH_TEMPLATE = 4202,
+	MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_MATCH_DEFINER = 4203,
 	MLX5DR_DEBUG_RES_TYPE_MATCHER_ACTION_TEMPLATE = 4204,
-	MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_DEFINER = 4203,
+	MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_HASH_DEFINER = 4205,
+	MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_RANGE_DEFINER = 4206,
 };
 
 const char *mlx5dr_debug_action_type_to_str(enum mlx5dr_action_type action_type);
Signature · 1 line
2.18.1
#15 lore raw

PATCH 15/16 net/mlx5/hws: rename pattern cache object

In reply to Alex Vesker

To have the same name convention for future caches, use cache and cache item naming.

---
 drivers/net/mlx5/hws/mlx5dr_pat_arg.c | 22 +++++++++++-----------
 drivers/net/mlx5/hws/mlx5dr_pat_arg.h |  6 +++---
 2 files changed, 14 insertions(+), 14 deletions(-)
drivers/net/mlx5/hws/mlx5dr_pat_arg.c +11 -11
index 152025d302..6ed04dac6d 100644
@@ -94,13 +94,13 @@ static bool mlx5dr_pat_compare_pattern(enum mlx5dr_action_type cur_type,
 	return true;
 }
 
-static struct mlx5dr_pat_cached_pattern *
+static struct mlx5dr_pattern_cache_item *
 mlx5dr_pat_find_cached_pattern(struct mlx5dr_pattern_cache *cache,
 			       struct mlx5dr_action *action,
 			       uint16_t num_of_actions,
 			       __be64 *actions)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pat;
+	struct mlx5dr_pattern_cache_item *cached_pat;
 
 	LIST_FOREACH(cached_pat, &cache->head, next) {
 		if (mlx5dr_pat_compare_pattern(cached_pat->type,
@@ -115,13 +115,13 @@ mlx5dr_pat_find_cached_pattern(struct mlx5dr_pattern_cache *cache,
 	return NULL;
 }
 
-static struct mlx5dr_pat_cached_pattern *
+static struct mlx5dr_pattern_cache_item *
 mlx5dr_pat_get_existing_cached_pattern(struct mlx5dr_pattern_cache *cache,
 				       struct mlx5dr_action *action,
 				       uint16_t num_of_actions,
 				       __be64 *actions)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pattern;
+	struct mlx5dr_pattern_cache_item *cached_pattern;
 
 	cached_pattern = mlx5dr_pat_find_cached_pattern(cache, action, num_of_actions, actions);
 	if (cached_pattern) {
@@ -134,11 +134,11 @@ mlx5dr_pat_get_existing_cached_pattern(struct mlx5dr_pattern_cache *cache,
 	return cached_pattern;
 }
 
-static struct mlx5dr_pat_cached_pattern *
+static struct mlx5dr_pattern_cache_item *
 mlx5dr_pat_get_cached_pattern_by_action(struct mlx5dr_pattern_cache *cache,
 					struct mlx5dr_action *action)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pattern;
+	struct mlx5dr_pattern_cache_item *cached_pattern;
 
 	LIST_FOREACH(cached_pattern, &cache->head, next) {
 		if (cached_pattern->mh_data.pattern_obj->id == action->modify_header.pattern_obj->id)
@@ -148,14 +148,14 @@ mlx5dr_pat_get_cached_pattern_by_action(struct mlx5dr_pattern_cache *cache,
 	return NULL;
 }
 
-static struct mlx5dr_pat_cached_pattern *
+static struct mlx5dr_pattern_cache_item *
 mlx5dr_pat_add_pattern_to_cache(struct mlx5dr_pattern_cache *cache,
 				struct mlx5dr_devx_obj *pattern_obj,
 				enum mlx5dr_action_type type,
 				uint16_t num_of_actions,
 				__be64 *actions)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pattern;
+	struct mlx5dr_pattern_cache_item *cached_pattern;
 
 	cached_pattern = simple_calloc(1, sizeof(*cached_pattern));
 	if (!cached_pattern) {
@@ -189,7 +189,7 @@ mlx5dr_pat_add_pattern_to_cache(struct mlx5dr_pattern_cache *cache,
 }
 
 static void
-mlx5dr_pat_remove_pattern(struct mlx5dr_pat_cached_pattern *cached_pattern)
+mlx5dr_pat_remove_pattern(struct mlx5dr_pattern_cache_item *cached_pattern)
 {
 	LIST_REMOVE(cached_pattern, next);
 	simple_free(cached_pattern->mh_data.data);
@@ -200,7 +200,7 @@ static void
 mlx5dr_pat_put_pattern(struct mlx5dr_pattern_cache *cache,
 		       struct mlx5dr_action *action)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pattern;
+	struct mlx5dr_pattern_cache_item *cached_pattern;
 
 	pthread_spin_lock(&cache->lock);
 	cached_pattern = mlx5dr_pat_get_cached_pattern_by_action(cache, action);
@@ -225,7 +225,7 @@ static int mlx5dr_pat_get_pattern(struct mlx5dr_context *ctx,
 				  size_t pattern_sz,
 				  __be64 *pattern)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pattern;
+	struct mlx5dr_pattern_cache_item *cached_pattern;
 	int ret = 0;
 
 	pthread_spin_lock(&ctx->pattern_cache->lock);
drivers/net/mlx5/hws/mlx5dr_pat_arg.h +3 -3
index d9353e9a3e..92db6d6aee 100644
@@ -24,10 +24,10 @@ enum {
 struct mlx5dr_pattern_cache {
 	/* Protect pattern list */
 	pthread_spinlock_t lock;
-	LIST_HEAD(pattern_head, mlx5dr_pat_cached_pattern) head;
+	LIST_HEAD(pattern_head, mlx5dr_pattern_cache_item) head;
 };
 
-struct mlx5dr_pat_cached_pattern {
+struct mlx5dr_pattern_cache_item {
 	enum mlx5dr_action_type type;
 	struct {
 		struct mlx5dr_devx_obj *pattern_obj;
@@ -36,7 +36,7 @@ struct mlx5dr_pat_cached_pattern {
 		uint16_t num_of_actions;
 	} mh_data;
 	uint32_t refcount;
-	LIST_ENTRY(mlx5dr_pat_cached_pattern) next;
+	LIST_ENTRY(mlx5dr_pattern_cache_item) next;
 };
 
 enum mlx5dr_arg_chunk_size
Signature · 1 line
2.18.1
#16 lore raw

PATCH 16/16 net/mlx5/hws: cache definer for reuse

In reply to Alex Vesker

Definers are a limited resource in the system per GVMI, to avoid failure we try to improve bt checking if it is possible to reuse the definers in some cases. Added a cache on the context for this purpose.

---
 drivers/net/mlx5/hws/mlx5dr_context.c |  12 ++-
 drivers/net/mlx5/hws/mlx5dr_context.h |   1 +
 drivers/net/mlx5/hws/mlx5dr_definer.c | 122 ++++++++++++++++++++++----
 drivers/net/mlx5/hws/mlx5dr_definer.h |  14 +++
 4 files changed, 130 insertions(+), 19 deletions(-)
drivers/net/mlx5/hws/mlx5dr_context.c +9 -3
index 6627337d9e..08a5ee92a5 100644
@@ -13,6 +13,9 @@ static int mlx5dr_context_pools_init(struct mlx5dr_context *ctx)
 	if (mlx5dr_pat_init_pattern_cache(&ctx->pattern_cache))
 		return rte_errno;
 
+	if (mlx5dr_definer_init_cache(&ctx->definer_cache))
+		goto uninit_pat_cache;
+
 	/* Create an STC pool per FT type */
 	pool_attr.pool_type = MLX5DR_POOL_TYPE_STC;
 	pool_attr.flags = MLX5DR_POOL_FLAGS_FOR_STC_POOL;
@@ -35,8 +38,10 @@ static int mlx5dr_context_pools_init(struct mlx5dr_context *ctx)
 		if (ctx->stc_pool[i])
 			mlx5dr_pool_destroy(ctx->stc_pool[i]);
 
-	mlx5dr_pat_uninit_pattern_cache(ctx->pattern_cache);
+	mlx5dr_definer_uninit_cache(ctx->definer_cache);
 
+uninit_pat_cache:
+	mlx5dr_pat_uninit_pattern_cache(ctx->pattern_cache);
 	return rte_errno;
 }
 
@@ -44,12 +49,13 @@ static void mlx5dr_context_pools_uninit(struct mlx5dr_context *ctx)
 {
 	int i;
 
-	mlx5dr_pat_uninit_pattern_cache(ctx->pattern_cache);
-
 	for (i = 0; i < MLX5DR_TABLE_TYPE_MAX; i++) {
 		if (ctx->stc_pool[i])
 			mlx5dr_pool_destroy(ctx->stc_pool[i]);
 	}
+
+	mlx5dr_definer_uninit_cache(ctx->definer_cache);
+	mlx5dr_pat_uninit_pattern_cache(ctx->pattern_cache);
 }
 
 static int mlx5dr_context_init_pd(struct mlx5dr_context *ctx,
drivers/net/mlx5/hws/mlx5dr_context.h +1 -0
index a38d9484b3..0ba8d0c92e 100644
@@ -39,6 +39,7 @@ struct mlx5dr_context {
 	struct mlx5dr_context_common_res common_res[MLX5DR_TABLE_TYPE_MAX];
 	struct mlx5dr_context_shared_gvmi_res gvmi_res[MLX5DR_TABLE_TYPE_MAX];
 	struct mlx5dr_pattern_cache *pattern_cache;
+	struct mlx5dr_definer_cache *definer_cache;
 	pthread_spinlock_t ctrl_lock;
 	enum mlx5dr_context_flags flags;
 	struct mlx5dr_send_engine *send_queue;
drivers/net/mlx5/hws/mlx5dr_definer.c +106 -16

197 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_definer.h +14 -0
index dd9a297007..464872acd6 100644
@@ -569,6 +569,16 @@ struct mlx5dr_definer {
 	struct mlx5dr_devx_obj *obj;
 };
 
+struct mlx5dr_definer_cache {
+	LIST_HEAD(definer_head, mlx5dr_definer_cache_item) head;
+};
+
+struct mlx5dr_definer_cache_item {
+	struct mlx5dr_definer definer;
+	uint32_t refcount;
+	LIST_ENTRY(mlx5dr_definer_cache_item) next;
+};
+
 static inline bool
 mlx5dr_definer_is_jumbo(struct mlx5dr_definer *definer)
 {
@@ -592,4 +602,8 @@ int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
 
 void mlx5dr_definer_matcher_uninit(struct mlx5dr_matcher *matcher);
 
+int mlx5dr_definer_init_cache(struct mlx5dr_definer_cache **cache);
+
+void mlx5dr_definer_uninit_cache(struct mlx5dr_definer_cache *cache);
+
 #endif
Signature · 1 line
2.18.1
#17 lore raw

v2 cover net/mlx5/hws: support range and partial hash matching

In reply to Alex Vesker

ConnectX and BlueField devices which support HWS are capable of performing advanced match, range-match and hashing operations on packet headers. This patch-set introduces support for (1) range matching - allows combining range and exact match over provided item fields using spec, last and mask. A useful example for Range and exact match is matching exact IP and port range. This series also introduces (2) partial hash, which allows using the same matcher and performing partial hashing in case of multiple match templates are provided. The matcher create code finds the union and intersection fields to split the hash and match in a way the HW can support. This increases the match templates combinations possible although it may limit the maximum PPS. Since (1) and (2) are not fully support on current HW the implementation is done on top of a HW like FW API using WQEs.

v2:
-Fix missed checkpatch comments

Alex Vesker (16):
net/mlx5/hws: support synchronous drain net/mlx5/hws: matcher remove AT and MT limitation net/mlx5/hws: support GTA WQE write using FW command net/mlx5/hws: add capability query for gen wqe command net/mlx5/hws: align RTC create command with PRM format net/mlx5/hws: add send FW match STE using gen WQE net/mlx5/hws: add send FW range STE WQE net/mlx5/hws: move matcher size check to function net/mlx5/hws: support range match net/mlx5/hws: redesign definer create net/mlx5/hws: support partial hash net/mlx5/hws: add range definer creation support net/mlx5/hws: add FW WQE rule creation logic net/mlx5/hws: add debug dump support for range and hash net/mlx5/hws: rename pattern cache object net/mlx5/hws: cache definer for reuse

 drivers/common/mlx5/mlx5_prm.h         |  53 +-
 drivers/net/mlx5/hws/mlx5dr.h          |   6 +-
 drivers/net/mlx5/hws/mlx5dr_cmd.c      |  72 ++-
 drivers/net/mlx5/hws/mlx5dr_cmd.h      |  27 +-
 drivers/net/mlx5/hws/mlx5dr_context.c  |  12 +-
 drivers/net/mlx5/hws/mlx5dr_context.h  |   1 +
 drivers/net/mlx5/hws/mlx5dr_debug.c    |  41 +-
 drivers/net/mlx5/hws/mlx5dr_debug.h    |   4 +-
 drivers/net/mlx5/hws/mlx5dr_definer.c  | 750 +++++++++++++++++++++----
 drivers/net/mlx5/hws/mlx5dr_definer.h  |  44 +-
 drivers/net/mlx5/hws/mlx5dr_internal.h |   2 +-
 drivers/net/mlx5/hws/mlx5dr_matcher.c  | 279 ++++++---
 drivers/net/mlx5/hws/mlx5dr_matcher.h  |  43 +-
 drivers/net/mlx5/hws/mlx5dr_pat_arg.c  |  49 +-
 drivers/net/mlx5/hws/mlx5dr_pat_arg.h  |   6 +-
 drivers/net/mlx5/hws/mlx5dr_rule.c     | 193 ++++++-
 drivers/net/mlx5/hws/mlx5dr_rule.h     |   2 +
 drivers/net/mlx5/hws/mlx5dr_send.c     | 169 +++++-
 drivers/net/mlx5/hws/mlx5dr_send.h     |  24 +-
 drivers/net/mlx5/mlx5_flow_hw.c        |   2 +-
 20 files changed, 1475 insertions(+), 304 deletions(-)
Signature · 1 line
2.18.1
#18 lore raw

PATCH v2 1/16 net/mlx5/hws: support synchronous drain

Until now we supported asynchronous drain, triggering the queue to start the drain, now we added support for synchronous which assures all the work was processed on the queue.

This is useful when working over a FW command and HW queue in parallel sending arguments over the HW queue and match over the FW command which requires synchronization.

This also fixes an issue with shared arguments send that require more than one WQE.

---
 drivers/net/mlx5/hws/mlx5dr.h         |  6 ++++--
 drivers/net/mlx5/hws/mlx5dr_pat_arg.c | 27 ++++-----------------------
 drivers/net/mlx5/hws/mlx5dr_send.c    | 16 ++++++++++++++--
 drivers/net/mlx5/hws/mlx5dr_send.h    |  5 +++++
 drivers/net/mlx5/mlx5_flow_hw.c       |  2 +-
 5 files changed, 28 insertions(+), 28 deletions(-)
drivers/net/mlx5/hws/mlx5dr.h +4 -2
index b3b2bf34f2..2b02884dc3 100644
@@ -86,8 +86,10 @@ enum mlx5dr_match_template_flags {
 };
 
 enum mlx5dr_send_queue_actions {
-	/* Start executing all pending queued rules and write to HW */
-	MLX5DR_SEND_QUEUE_ACTION_DRAIN = 1 << 0,
+	/* Start executing all pending queued rules */
+	MLX5DR_SEND_QUEUE_ACTION_DRAIN_ASYNC = 1 << 0,
+	/* Start executing all pending queued rules wait till completion */
+	MLX5DR_SEND_QUEUE_ACTION_DRAIN_SYNC = 1 << 1,
 };
 
 struct mlx5dr_context_attr {
drivers/net/mlx5/hws/mlx5dr_pat_arg.c +4 -23
index df451f1ae0..152025d302 100644
@@ -306,27 +306,6 @@ void mlx5dr_arg_decapl3_write(struct mlx5dr_send_engine *queue,
 	mlx5dr_send_engine_post_end(&ctrl, &send_attr);
 }
 
-static int
-mlx5dr_arg_poll_for_comp(struct mlx5dr_context *ctx, uint16_t queue_id)
-{
-	struct rte_flow_op_result comp[1];
-	int ret;
-
-	while (true) {
-		ret = mlx5dr_send_queue_poll(ctx, queue_id, comp, 1);
-		if (ret) {
-			if (ret < 0) {
-				DR_LOG(ERR, "Failed mlx5dr_send_queue_poll");
-			} else if (comp[0].status == RTE_FLOW_OP_ERROR) {
-				DR_LOG(ERR, "Got comp with error");
-				rte_errno = ENOENT;
-			}
-			break;
-		}
-	}
-	return (ret == 1 ? 0 : ret);
-}
-
 void mlx5dr_arg_write(struct mlx5dr_send_engine *queue,
 		      void *comp_data,
 		      uint32_t arg_idx,
@@ -388,9 +367,11 @@ int mlx5dr_arg_write_inline_arg_data(struct mlx5dr_context *ctx,
 	mlx5dr_send_engine_flush_queue(queue);
 
 	/* Poll for completion */
-	ret = mlx5dr_arg_poll_for_comp(ctx, ctx->queues - 1);
+	ret = mlx5dr_send_queue_action(ctx, ctx->queues - 1,
+				       MLX5DR_SEND_QUEUE_ACTION_DRAIN_SYNC);
+
 	if (ret)
-		DR_LOG(ERR, "Failed to get completions for shared action");
+		DR_LOG(ERR, "Failed to drain arg queue");
 
 	pthread_spin_unlock(&ctx->ctrl_lock);
 
drivers/net/mlx5/hws/mlx5dr_send.c +14 -2
index 5c8bbe6fc6..a507e5f626 100644
@@ -830,18 +830,30 @@ int mlx5dr_send_queue_action(struct mlx5dr_context *ctx,
 {
 	struct mlx5dr_send_ring_sq *send_sq;
 	struct mlx5dr_send_engine *queue;
+	bool wait_comp = false;
+	int64_t polled = 0;
 
 	queue = &ctx->send_queue[queue_id];
 	send_sq = &queue->send_ring->send_sq;
 
-	if (actions == MLX5DR_SEND_QUEUE_ACTION_DRAIN) {
+	switch (actions) {
+	case MLX5DR_SEND_QUEUE_ACTION_DRAIN_SYNC:
+		wait_comp = true;
+		/* FALLTHROUGH */
+	case MLX5DR_SEND_QUEUE_ACTION_DRAIN_ASYNC:
 		if (send_sq->head_dep_idx != send_sq->tail_dep_idx)
 			/* Send dependent WQEs to drain the queue */
 			mlx5dr_send_all_dep_wqe(queue);
 		else
 			/* Signal on the last posted WQE */
 			mlx5dr_send_engine_flush_queue(queue);
-	} else {
+
+		/* Poll queue until empty */
+		while (wait_comp && !mlx5dr_send_engine_empty(queue))
+			mlx5dr_send_engine_poll_cqs(queue, NULL, &polled, 0);
+
+		break;
+	default:
 		rte_errno = -EINVAL;
 		return rte_errno;
 	}
drivers/net/mlx5/hws/mlx5dr_send.h +5 -0
index 8d4769495d..fcddcc6366 100644
@@ -240,6 +240,11 @@ void mlx5dr_send_ste(struct mlx5dr_send_engine *queue,
 
 void mlx5dr_send_engine_flush_queue(struct mlx5dr_send_engine *queue);
 
+static inline bool mlx5dr_send_engine_empty(struct mlx5dr_send_engine *queue)
+{
+	return (queue->send_ring->send_sq.cur_post == queue->send_ring->send_cq.poll_wqe);
+}
+
 static inline bool mlx5dr_send_engine_full(struct mlx5dr_send_engine *queue)
 {
 	return queue->used_entries >= queue->th_entries;
drivers/net/mlx5/mlx5_flow_hw.c +1 -1
index 20c71ff7f0..7e87d589cb 100644
@@ -2851,7 +2851,7 @@ flow_hw_push(struct rte_eth_dev *dev,
 
 	__flow_hw_push_action(dev, queue);
 	ret = mlx5dr_send_queue_action(priv->dr_ctx, queue,
-				       MLX5DR_SEND_QUEUE_ACTION_DRAIN);
+				       MLX5DR_SEND_QUEUE_ACTION_DRAIN_ASYNC);
 	if (ret) {
 		rte_flow_error_set(error, rte_errno,
 				   RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
Signature · 1 line
2.18.1
#19 lore raw

PATCH v2 2/16 net/mlx5/hws: matcher remove AT and MT limitation

In reply to Alex Vesker

The action and match templates were stored on the matcher in a fixed size array to reduce cache misses and reuse template calculations. This approuch introduced two issues:
-limitation of fixed array
-definer is bindind to match template and cannot be used with
union definer since the layout is fixed

---
 drivers/net/mlx5/hws/mlx5dr_debug.c   |   4 +-
 drivers/net/mlx5/hws/mlx5dr_matcher.c | 109 ++++++++++++++++----------
 drivers/net/mlx5/hws/mlx5dr_matcher.h |   8 +-
 drivers/net/mlx5/hws/mlx5dr_rule.c    |  15 ++--
 4 files changed, 78 insertions(+), 58 deletions(-)
drivers/net/mlx5/hws/mlx5dr_debug.c +2 -2
index 0815327b18..9199ec16e0 100644
@@ -92,7 +92,7 @@ mlx5dr_debug_dump_matcher_match_template(FILE *f, struct mlx5dr_matcher *matcher
 	int i, ret;
 
 	for (i = 0; i < matcher->num_of_mt; i++) {
-		struct mlx5dr_match_template *mt = matcher->mt[i];
+		struct mlx5dr_match_template *mt = &matcher->mt[i];
 
 		ret = fprintf(f, "%d,0x%" PRIx64 ",0x%" PRIx64 ",%d,%d\n",
 			      MLX5DR_DEBUG_RES_TYPE_MATCHER_MATCH_TEMPLATE,
@@ -123,7 +123,7 @@ mlx5dr_debug_dump_matcher_action_template(FILE *f, struct mlx5dr_matcher *matche
 	int i, j, ret;
 
 	for (i = 0; i < matcher->num_of_at; i++) {
-		struct mlx5dr_action_template *at = matcher->at[i];
+		struct mlx5dr_action_template *at = &matcher->at[i];
 
 		ret = fprintf(f, "%d,0x%" PRIx64 ",0x%" PRIx64 ",%d,%d,%d",
 			      MLX5DR_DEBUG_RES_TYPE_MATCHER_ACTION_TEMPLATE,
drivers/net/mlx5/hws/mlx5dr_matcher.c +66 -43

230 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_matcher.h +2 -6
index 2bebc4bcce..b957f5ea4b 100644
@@ -6,12 +6,8 @@
 #define MLX5DR_MATCHER_H_
 
 /* Max supported match template */
-#define MLX5DR_MATCHER_MAX_MT 2
 #define MLX5DR_MATCHER_MAX_MT_ROOT 1
 
-/* Max supported action template */
-#define MLX5DR_MATCHER_MAX_AT 4
-
 /* We calculated that concatenating a collision table to the main table with
  * 3% of the main table rows will be enough resources for high insertion
  * success probability.
@@ -59,9 +55,9 @@ struct mlx5dr_matcher {
 	struct mlx5dr_table *tbl;
 	struct mlx5dr_matcher_attr attr;
 	struct mlx5dv_flow_matcher *dv_matcher;
-	struct mlx5dr_match_template *mt[MLX5DR_MATCHER_MAX_MT];
+	struct mlx5dr_match_template *mt;
 	uint8_t num_of_mt;
-	struct mlx5dr_action_template *at[MLX5DR_MATCHER_MAX_AT];
+	struct mlx5dr_action_template *at;
 	uint8_t num_of_at;
 	struct mlx5dr_devx_obj *end_ft;
 	struct mlx5dr_matcher *col_matcher;
drivers/net/mlx5/hws/mlx5dr_rule.c +8 -7
index 60a82c022f..f5a0c46315 100644
@@ -5,10 +5,10 @@
 #include "mlx5dr_internal.h"
 
 static void mlx5dr_rule_skip(struct mlx5dr_matcher *matcher,
+			     struct mlx5dr_match_template *mt,
 			     const struct rte_flow_item *items,
 			     bool *skip_rx, bool *skip_tx)
 {
-	struct mlx5dr_match_template *mt = matcher->mt[0];
 	const struct flow_hw_port_info *vport;
 	const struct rte_flow_item_ethdev *v;
 
@@ -43,6 +43,7 @@ static void mlx5dr_rule_skip(struct mlx5dr_matcher *matcher,
 static void mlx5dr_rule_init_dep_wqe(struct mlx5dr_send_ring_dep_wqe *dep_wqe,
 				     struct mlx5dr_rule *rule,
 				     const struct rte_flow_item *items,
+				     struct mlx5dr_match_template *mt,
 				     void *user_data)
 {
 	struct mlx5dr_matcher *matcher = rule->matcher;
@@ -63,7 +64,7 @@ static void mlx5dr_rule_init_dep_wqe(struct mlx5dr_send_ring_dep_wqe *dep_wqe,
 		break;
 
 	case MLX5DR_TABLE_TYPE_FDB:
-		mlx5dr_rule_skip(matcher, items, &skip_rx, &skip_tx);
+		mlx5dr_rule_skip(matcher, mt, items, &skip_rx, &skip_tx);
 
 		if (!skip_rx) {
 			dep_wqe->rtc_0 = matcher->match_ste.rtc_0->id;
@@ -186,8 +187,8 @@ static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 				  uint8_t at_idx,
 				  struct mlx5dr_rule_action rule_actions[])
 {
-	struct mlx5dr_action_template *at = rule->matcher->at[at_idx];
-	struct mlx5dr_match_template *mt = rule->matcher->mt[mt_idx];
+	struct mlx5dr_action_template *at = &rule->matcher->at[at_idx];
+	struct mlx5dr_match_template *mt = &rule->matcher->mt[mt_idx];
 	bool is_jumbo = mlx5dr_definer_is_jumbo(mt->definer);
 	struct mlx5dr_matcher *matcher = rule->matcher;
 	struct mlx5dr_context *ctx = matcher->tbl->ctx;
@@ -212,7 +213,7 @@ static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 	 * dep_wqe buffers (ctrl, data) are also reused for all STE writes.
 	 */
 	dep_wqe = mlx5dr_send_add_new_dep_wqe(queue);
-	mlx5dr_rule_init_dep_wqe(dep_wqe, rule, items, attr->user_data);
+	mlx5dr_rule_init_dep_wqe(dep_wqe, rule, items, mt, attr->user_data);
 
 	ste_attr.wqe_ctrl = &dep_wqe->wqe_ctrl;
 	ste_attr.wqe_data = &dep_wqe->wqe_data;
@@ -371,7 +372,7 @@ static int mlx5dr_rule_destroy_hws(struct mlx5dr_rule *rule,
 	ste_attr.used_id_rtc_1 = &rule->rtc_1;
 	ste_attr.wqe_ctrl = &wqe_ctrl;
 	ste_attr.wqe_tag = &rule->tag;
-	ste_attr.wqe_tag_is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt[0]->definer);
+	ste_attr.wqe_tag_is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt->definer);
 	ste_attr.gta_opcode = MLX5DR_WQE_GTA_OP_DEACTIVATE;
 	if (unlikely(mlx5dr_matcher_is_insert_by_idx(matcher)))
 		ste_attr.direct_index = attr->rule_idx;
@@ -388,7 +389,7 @@ static int mlx5dr_rule_create_root(struct mlx5dr_rule *rule,
 				   struct mlx5dr_rule_action rule_actions[])
 {
 	struct mlx5dv_flow_matcher *dv_matcher = rule->matcher->dv_matcher;
-	uint8_t num_actions = rule->matcher->at[at_idx]->num_actions;
+	uint8_t num_actions = rule->matcher->at[at_idx].num_actions;
 	struct mlx5dr_context *ctx = rule->matcher->tbl->ctx;
 	struct mlx5dv_flow_match_parameters *value;
 	struct mlx5_flow_attr flow_attr = {0};
Signature · 1 line
2.18.1
#20 lore raw

PATCH v2 3/16 net/mlx5/hws: support GTA WQE write using FW command

In reply to Alex Vesker

The generate WQE command is used as an interface to writing GTA WQEs with fields that are not supported in current HW, for example extended match definer.

---
 drivers/common/mlx5/mlx5_prm.h    | 27 +++++++++++++++++-
 drivers/net/mlx5/hws/mlx5dr_cmd.c | 47 +++++++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_cmd.h | 13 +++++++++
 3 files changed, 86 insertions(+), 1 deletion(-)
drivers/common/mlx5/mlx5_prm.h +26 -1
index 9294f65e24..d4d8ddcd2a 100644
@@ -1141,6 +1141,7 @@ enum {
 	MLX5_CMD_QUERY_REGEX_REGISTERS = 0xb07,
 	MLX5_CMD_OP_ACCESS_REGISTER_USER = 0xb0c,
 	MLX5_CMD_OP_ALLOW_OTHER_VHCA_ACCESS = 0xb16,
+	MLX5_CMD_OP_GENERATE_WQE = 0xb17,
 };
 
 enum {
@@ -2159,7 +2160,8 @@ struct mlx5_ifc_cmd_hca_cap_2_bits {
 	u8 format_select_dw_gtpu_dw_1[0x8];
 	u8 format_select_dw_gtpu_dw_2[0x8];
 	u8 format_select_dw_gtpu_first_ext_dw_0[0x8];
-	u8 reserved_at_2a0[0x560];
+	u8 generate_wqe_type[0x20];
+	u8 reserved_at_2c0[0x540];
 };
 
 struct mlx5_ifc_esw_cap_bits {
@@ -3529,6 +3531,29 @@ struct mlx5_ifc_create_alias_obj_in_bits {
 	struct mlx5_ifc_alias_context_bits alias_ctx;
 };
 
+struct mlx5_ifc_generate_wqe_in_bits {
+	u8 opcode[0x10];
+	u8 uid[0x10];
+	u8 reserved_at_20[0x10];
+	u8 op_mode[0x10];
+	u8 reserved_at_40[0x40];
+	u8 reserved_at_80[0x8];
+	u8 pdn[0x18];
+	u8 reserved_at_a0[0x160];
+	u8 wqe_ctrl[0x80];
+	u8 wqe_gta_ctrl[0x180];
+	u8 wqe_gta_data_0[0x200];
+	u8 wqe_gta_data_1[0x200];
+};
+
+struct mlx5_ifc_generate_wqe_out_bits {
+	u8 status[0x8];
+	u8 reserved_at_8[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_at_40[0x1c0];
+	u8 cqe_data[0x200];
+};
+
 enum {
 	MLX5_CRYPTO_KEY_SIZE_128b = 0x0,
 	MLX5_CRYPTO_KEY_SIZE_256b = 0x1,
drivers/net/mlx5/hws/mlx5dr_cmd.c +47 -0

53 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_cmd.h +13 -0
index 468557ba16..3689d09897 100644
@@ -153,6 +153,14 @@ struct mlx5dr_cmd_query_vport_caps {
 	uint32_t metadata_c_mask;
 };
 
+struct mlx5dr_cmd_generate_wqe_attr {
+	uint8_t *wqe_ctrl;
+	uint8_t *gta_ctrl;
+	uint8_t *gta_data_0;
+	uint8_t *gta_data_1;
+	uint32_t pdn;
+};
+
 struct mlx5dr_cmd_query_caps {
 	uint32_t wire_regc;
 	uint32_t wire_regc_mask;
@@ -212,6 +220,11 @@ int
 mlx5dr_cmd_stc_modify(struct mlx5dr_devx_obj *devx_obj,
 		      struct mlx5dr_cmd_stc_modify_attr *stc_attr);
 
+int
+mlx5dr_cmd_generate_wqe(struct ibv_context *ctx,
+			struct mlx5dr_cmd_generate_wqe_attr *attr,
+			struct mlx5_cqe64 *ret_cqe);
+
 struct mlx5dr_devx_obj *
 mlx5dr_cmd_ste_create(struct ibv_context *ctx,
 		      struct mlx5dr_cmd_ste_create_attr *ste_attr);
Signature · 1 line
2.18.1
#21 lore raw

PATCH v2 4/16 net/mlx5/hws: add capability query for gen wqe command

In reply to Alex Vesker

Read the capabilities required to determine support for GENERATE_WQE.

---
 drivers/common/mlx5/mlx5_prm.h    |  6 ++++--
 drivers/net/mlx5/hws/mlx5dr_cmd.c | 12 ++++++++++++
 drivers/net/mlx5/hws/mlx5dr_cmd.h |  3 +++
 3 files changed, 19 insertions(+), 2 deletions(-)
drivers/common/mlx5/mlx5_prm.h +4 -2
index d4d8ddcd2a..140534ea46 100644
@@ -2205,10 +2205,12 @@ struct mlx5_ifc_wqe_based_flow_table_cap_bits {
 	u8 header_insert_type[0x10];
 	u8 header_remove_type[0x10];
 	u8 trivial_match_definer[0x20];
-	u8 reserved_at_140[0x20];
+	u8 reserved_at_140[0x1b];
+	u8 rtc_max_num_hash_definer_gen_wqe[0x5];
 	u8 reserved_at_160[0x18];
 	u8 access_index_mode[0x8];
-	u8 reserved_at_180[0x20];
+	u8 reserved_at_180[0x10];
+	u8 ste_format_gen_wqe[0x10];
 	u8 linear_match_definer_reg_c3[0x20];
 };
 
drivers/net/mlx5/hws/mlx5dr_cmd.c +12 -0
index c648eacd03..b0cef5e944 100644
@@ -928,6 +928,10 @@ int mlx5dr_cmd_query_caps(struct ibv_context *ctx,
 						     capability.cmd_hca_cap_2.
 						     format_select_dw_gtpu_first_ext_dw_0);
 
+	caps->supp_type_gen_wqe = MLX5_GET(query_hca_cap_out, out,
+					   capability.cmd_hca_cap_2.
+					   generate_wqe_type);
+
 	/* check cross-VHCA support in cap2 */
 	res =
 	MLX5_GET(query_hca_cap_out, out,
@@ -1033,6 +1037,14 @@ int mlx5dr_cmd_query_caps(struct ibv_context *ctx,
 		caps->linear_match_definer = MLX5_GET(query_hca_cap_out, out,
 						      capability.wqe_based_flow_table_cap.
 						      linear_match_definer_reg_c3);
+
+		caps->rtc_max_hash_def_gen_wqe = MLX5_GET(query_hca_cap_out, out,
+							  capability.wqe_based_flow_table_cap.
+							  rtc_max_num_hash_definer_gen_wqe);
+
+		caps->supp_ste_format_gen_wqe = MLX5_GET(query_hca_cap_out, out,
+							 capability.wqe_based_flow_table_cap.
+							 ste_format_gen_wqe);
 	}
 
 	if (caps->eswitch_manager) {
drivers/net/mlx5/hws/mlx5dr_cmd.h +3 -0
index 3689d09897..36295e0c76 100644
@@ -183,6 +183,9 @@ struct mlx5dr_cmd_query_caps {
 	bool full_dw_jumbo_support;
 	bool rtc_hash_split_table;
 	bool rtc_linear_lookup_table;
+	uint32_t supp_type_gen_wqe;
+	uint8_t rtc_max_hash_def_gen_wqe;
+	uint16_t supp_ste_format_gen_wqe;
 	struct mlx5dr_cmd_query_ft_caps nic_ft;
 	struct mlx5dr_cmd_query_ft_caps fdb_ft;
 	bool eswitch_manager;
Signature · 1 line
2.18.1
#22 lore raw

PATCH v2 5/16 net/mlx5/hws: align RTC create command with PRM format

In reply to Alex Vesker

Rename rtc params create for new format.

---
 drivers/common/mlx5/mlx5_prm.h        | 16 ++++++++++------
 drivers/net/mlx5/hws/mlx5dr_cmd.c     | 13 +++++++++++--
 drivers/net/mlx5/hws/mlx5dr_cmd.h     | 11 +++++++----
 drivers/net/mlx5/hws/mlx5dr_matcher.c | 19 ++++++++++++-------
 4 files changed, 40 insertions(+), 19 deletions(-)
drivers/common/mlx5/mlx5_prm.h +10 -6
index 140534ea46..298cc48b06 100644
@@ -3237,6 +3237,7 @@ enum mlx5_ifc_rtc_access_mode {
 enum mlx5_ifc_rtc_ste_format {
 	MLX5_IFC_RTC_STE_FORMAT_8DW = 0x4,
 	MLX5_IFC_RTC_STE_FORMAT_11DW = 0x5,
+	MLX5_IFC_RTC_STE_FORMAT_RANGE = 0x7,
 };
 
 enum mlx5_ifc_rtc_reparse_mode {
@@ -3251,24 +3252,27 @@ struct mlx5_ifc_rtc_bits {
 	u8 reserved_at_40[0x40];
 	u8 update_index_mode[0x2];
 	u8 reparse_mode[0x2];
-	u8 reserved_at_84[0x4];
+	u8 num_match_ste[0x4];
 	u8 pd[0x18];
 	u8 reserved_at_a0[0x9];
 	u8 access_index_mode[0x3];
 	u8 num_hash_definer[0x4];
-	u8 reserved_at_b0[0x3];
+	u8 update_method[0x1];
+	u8 reserved_at_b1[0x2];
 	u8 log_depth[0x5];
 	u8 log_hash_size[0x8];
-	u8 ste_format[0x8];
+	u8 ste_format_0[0x8];
 	u8 table_type[0x8];
-	u8 reserved_at_d0[0x10];
-	u8 match_definer_id[0x20];
+	u8 ste_format_1[0x8];
+	u8 reserved_at_d8[0x8];
+	u8 match_definer_0[0x20];
 	u8 stc_id[0x20];
 	u8 ste_table_base_id[0x20];
 	u8 ste_table_offset[0x20];
 	u8 reserved_at_160[0x8];
 	u8 miss_flow_table_id[0x18];
-	u8 reserved_at_180[0x280];
+	u8 match_definer_1[0x20];
+	u8 reserved_at_1a0[0x260];
 };
 
 struct mlx5_ifc_alias_context_bits {
drivers/net/mlx5/hws/mlx5dr_cmd.c +11 -2
index b0cef5e944..3578fe4b69 100644
@@ -259,17 +259,26 @@ mlx5dr_cmd_rtc_create(struct ibv_context *ctx,
 		 attr, obj_type, MLX5_GENERAL_OBJ_TYPE_RTC);
 
 	attr = MLX5_ADDR_OF(create_rtc_in, in, rtc);
-	MLX5_SET(rtc, attr, ste_format, rtc_attr->is_jumbo ?
+	MLX5_SET(rtc, attr, ste_format_0, rtc_attr->is_frst_jumbo ?
 		MLX5_IFC_RTC_STE_FORMAT_11DW :
 		MLX5_IFC_RTC_STE_FORMAT_8DW);
+
+	if (rtc_attr->is_scnd_range) {
+		MLX5_SET(rtc, attr, ste_format_1, MLX5_IFC_RTC_STE_FORMAT_RANGE);
+		MLX5_SET(rtc, attr, num_match_ste, 2);
+	}
+
 	MLX5_SET(rtc, attr, pd, rtc_attr->pd);
+	MLX5_SET(rtc, attr, update_method, rtc_attr->fw_gen_wqe);
 	MLX5_SET(rtc, attr, update_index_mode, rtc_attr->update_index_mode);
 	MLX5_SET(rtc, attr, access_index_mode, rtc_attr->access_index_mode);
 	MLX5_SET(rtc, attr, num_hash_definer, rtc_attr->num_hash_definer);
 	MLX5_SET(rtc, attr, log_depth, rtc_attr->log_depth);
 	MLX5_SET(rtc, attr, log_hash_size, rtc_attr->log_size);
 	MLX5_SET(rtc, attr, table_type, rtc_attr->table_type);
-	MLX5_SET(rtc, attr, match_definer_id, rtc_attr->definer_id);
+	MLX5_SET(rtc, attr, num_hash_definer, rtc_attr->num_hash_definer);
+	MLX5_SET(rtc, attr, match_definer_0, rtc_attr->match_definer_0);
+	MLX5_SET(rtc, attr, match_definer_1, rtc_attr->match_definer_1);
 	MLX5_SET(rtc, attr, stc_id, rtc_attr->stc_base);
 	MLX5_SET(rtc, attr, ste_table_base_id, rtc_attr->ste_base);
 	MLX5_SET(rtc, attr, ste_table_offset, rtc_attr->ste_offset);
drivers/net/mlx5/hws/mlx5dr_cmd.h +7 -4
index 36295e0c76..0fc65aad49 100644
@@ -23,8 +23,8 @@ struct mlx5dr_cmd_ft_modify_attr {
 };
 
 struct mlx5dr_cmd_fg_attr {
-	uint32_t	table_id;
-	uint32_t	table_type;
+	uint32_t table_id;
+	uint32_t table_type;
 };
 
 struct mlx5dr_cmd_forward_tbl {
@@ -40,14 +40,17 @@ struct mlx5dr_cmd_rtc_create_attr {
 	uint32_t ste_base;
 	uint32_t ste_offset;
 	uint32_t miss_ft_id;
+	bool fw_gen_wqe;
 	uint8_t update_index_mode;
 	uint8_t access_index_mode;
 	uint8_t num_hash_definer;
 	uint8_t log_depth;
 	uint8_t log_size;
 	uint8_t table_type;
-	uint8_t definer_id;
-	bool is_jumbo;
+	uint8_t match_definer_0;
+	uint8_t match_definer_1;
+	bool is_frst_jumbo;
+	bool is_scnd_range;
 };
 
 struct mlx5dr_cmd_alias_obj_create_attr {
drivers/net/mlx5/hws/mlx5dr_matcher.c +12 -7
index 913bb9d447..101a12d361 100644
@@ -413,6 +413,8 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 	struct mlx5dr_pool *ste_pool, *stc_pool;
 	struct mlx5dr_devx_obj *devx_obj;
 	struct mlx5dr_pool_chunk *ste;
+	uint8_t first_definer_id;
+	bool is_jumbo;
 	int ret;
 
 	switch (rtc_type) {
@@ -426,12 +428,15 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 		rtc_attr.log_depth = attr->table.sz_col_log;
 		rtc_attr.miss_ft_id = matcher->end_ft->id;
 
+		is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt->definer);
+		first_definer_id = mlx5dr_definer_get_id(matcher->mt->definer);
+
 		if (attr->insert_mode == MLX5DR_MATCHER_INSERT_BY_HASH) {
 			/* The usual Hash Table */
 			rtc_attr.update_index_mode = MLX5_IFC_RTC_STE_UPDATE_MODE_BY_HASH;
 			/* The first match template is used since all share the same definer */
-			rtc_attr.definer_id = mlx5dr_definer_get_id(matcher->mt->definer);
-			rtc_attr.is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt->definer);
+			rtc_attr.match_definer_0 = first_definer_id;
+			rtc_attr.is_frst_jumbo = is_jumbo;
 		} else if (attr->insert_mode == MLX5DR_MATCHER_INSERT_BY_INDEX) {
 			rtc_attr.update_index_mode = MLX5_IFC_RTC_STE_UPDATE_MODE_BY_OFFSET;
 			rtc_attr.num_hash_definer = 1;
@@ -439,12 +444,12 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 			if (attr->distribute_mode == MLX5DR_MATCHER_DISTRIBUTE_BY_HASH) {
 				/* Hash Split Table */
 				rtc_attr.access_index_mode = MLX5_IFC_RTC_STE_ACCESS_MODE_BY_HASH;
-				rtc_attr.definer_id = mlx5dr_definer_get_id(matcher->mt->definer);
-				rtc_attr.is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt->definer);
+				rtc_attr.match_definer_0 = first_definer_id;
+				rtc_attr.is_frst_jumbo = is_jumbo;
 			} else if (attr->distribute_mode == MLX5DR_MATCHER_DISTRIBUTE_BY_LINEAR) {
 				/* Linear Lookup Table */
 				rtc_attr.access_index_mode = MLX5_IFC_RTC_STE_ACCESS_MODE_LINEAR;
-				rtc_attr.definer_id = ctx->caps->linear_match_definer;
+				rtc_attr.match_definer_0 = ctx->caps->linear_match_definer;
 			}
 		}
 
@@ -468,8 +473,8 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 		rtc_attr.log_depth = 0;
 		rtc_attr.update_index_mode = MLX5_IFC_RTC_STE_UPDATE_MODE_BY_OFFSET;
 		/* The action STEs use the default always hit definer */
-		rtc_attr.definer_id = ctx->caps->trivial_match_definer;
-		rtc_attr.is_jumbo = false;
+		rtc_attr.match_definer_0 = ctx->caps->trivial_match_definer;
+		rtc_attr.is_frst_jumbo = false;
 		rtc_attr.miss_ft_id = 0;
 		break;
 
Signature · 1 line
2.18.1
#23 lore raw

PATCH v2 6/16 net/mlx5/hws: add send FW match STE using gen WQE

In reply to Alex Vesker

Send STE WQE function wraps the send WQE command to support WQE build and FDB abstraction. Sending using FW is different from sending from HW since FW returns the completion immediately which requires us to retry on failure and prepare the completion as part of the send process.

---
 drivers/net/mlx5/hws/mlx5dr_send.c | 134 +++++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_send.h |   7 +-
 2 files changed, 140 insertions(+), 1 deletion(-)
drivers/net/mlx5/hws/mlx5dr_send.c +134 -0
index a507e5f626..a9958df4f2 100644
@@ -235,6 +235,140 @@ void mlx5dr_send_ste(struct mlx5dr_send_engine *queue,
 	send_attr->fence = fence;
 }
 
+static
+int mlx5dr_send_wqe_fw(struct ibv_context *ibv_ctx,
+		       uint32_t pd_num,
+		       struct mlx5dr_send_engine_post_attr *send_attr,
+		       struct mlx5dr_wqe_gta_ctrl_seg *send_wqe_ctrl,
+		       void *send_wqe_match_data,
+		       void *send_wqe_match_tag,
+		       bool is_jumbo,
+		       uint8_t gta_opcode)
+{
+	bool has_match = send_wqe_match_data || send_wqe_match_tag;
+	struct mlx5dr_wqe_gta_data_seg_ste gta_wqe_data0 = {0};
+	struct mlx5dr_wqe_gta_ctrl_seg gta_wqe_ctrl = {0};
+	struct mlx5dr_cmd_generate_wqe_attr attr = {0};
+	struct mlx5dr_wqe_ctrl_seg wqe_ctrl = {0};
+	struct mlx5_cqe64 cqe;
+	uint32_t flags = 0;
+	int ret;
+
+	/* Set WQE control */
+	wqe_ctrl.opmod_idx_opcode =
+		rte_cpu_to_be_32((send_attr->opmod << 24) | send_attr->opcode);
+	wqe_ctrl.qpn_ds =
+		rte_cpu_to_be_32((send_attr->len + sizeof(struct mlx5dr_wqe_ctrl_seg)) / 16);
+	flags |= send_attr->notify_hw ? MLX5_WQE_CTRL_CQ_UPDATE : 0;
+	wqe_ctrl.flags = rte_cpu_to_be_32(flags);
+	wqe_ctrl.imm = rte_cpu_to_be_32(send_attr->id);
+
+	/* Set GTA WQE CTRL */
+	memcpy(gta_wqe_ctrl.stc_ix, send_wqe_ctrl->stc_ix, sizeof(send_wqe_ctrl->stc_ix));
+	gta_wqe_ctrl.op_dirix = htobe32(gta_opcode << 28);
+
+	/* Set GTA match WQE DATA */
+	if (has_match) {
+		if (send_wqe_match_data)
+			memcpy(&gta_wqe_data0, send_wqe_match_data, sizeof(gta_wqe_data0));
+		else
+			mlx5dr_send_wqe_set_tag(&gta_wqe_data0, send_wqe_match_tag, is_jumbo);
+
+		gta_wqe_data0.rsvd1_definer = htobe32(send_attr->match_definer_id << 8);
+		attr.gta_data_0 = (uint8_t *)&gta_wqe_data0;
+	}
+
+	attr.pdn = pd_num;
+	attr.wqe_ctrl = (uint8_t *)&wqe_ctrl;
+	attr.gta_ctrl = (uint8_t *)&gta_wqe_ctrl;
+
+send_wqe:
+	ret = mlx5dr_cmd_generate_wqe(ibv_ctx, &attr, &cqe);
+	if (ret) {
+		DR_LOG(ERR, "Failed to write WQE using command");
+		return ret;
+	}
+
+	if ((mlx5dv_get_cqe_opcode(&cqe) == MLX5_CQE_REQ) &&
+	    (rte_be_to_cpu_32(cqe.byte_cnt) >> 31 == 0)) {
+		*send_attr->used_id = send_attr->id;
+		return 0;
+	}
+
+	/* Retry if rule failed */
+	if (send_attr->retry_id) {
+		wqe_ctrl.imm = rte_cpu_to_be_32(send_attr->retry_id);
+		send_attr->id = send_attr->retry_id;
+		send_attr->retry_id = 0;
+		goto send_wqe;
+	}
+
+	return -1;
+}
+
+void mlx5dr_send_stes_fw(struct mlx5dr_send_engine *queue,
+			 struct mlx5dr_send_ste_attr *ste_attr)
+{
+	struct mlx5dr_send_engine_post_attr *send_attr = &ste_attr->send_attr;
+	struct mlx5dr_rule *rule = send_attr->rule;
+	struct ibv_context *ibv_ctx;
+	struct mlx5dr_context *ctx;
+	uint16_t queue_id;
+	uint32_t pdn;
+	int ret;
+
+	ctx = rule->matcher->tbl->ctx;
+	queue_id = queue - ctx->send_queue;
+	ibv_ctx = ctx->ibv_ctx;
+	pdn = ctx->pd_num;
+
+	/* Writing through FW can't HW fence, therefore we drain the queue */
+	if (send_attr->fence)
+		mlx5dr_send_queue_action(ctx,
+					 queue_id,
+					 MLX5DR_SEND_QUEUE_ACTION_DRAIN_SYNC);
+
+	if (ste_attr->rtc_1) {
+		send_attr->id = ste_attr->rtc_1;
+		send_attr->used_id = ste_attr->used_id_rtc_1;
+		send_attr->retry_id = ste_attr->retry_rtc_1;
+		ret = mlx5dr_send_wqe_fw(ibv_ctx, pdn, send_attr,
+					 ste_attr->wqe_ctrl,
+					 ste_attr->wqe_data,
+					 ste_attr->wqe_tag,
+					 ste_attr->wqe_tag_is_jumbo,
+					 ste_attr->gta_opcode);
+		if (ret)
+			goto fail_rule;
+	}
+
+	if (ste_attr->rtc_0) {
+		send_attr->id = ste_attr->rtc_0;
+		send_attr->used_id = ste_attr->used_id_rtc_0;
+		send_attr->retry_id = ste_attr->retry_rtc_0;
+		ret = mlx5dr_send_wqe_fw(ibv_ctx, pdn, send_attr,
+					 ste_attr->wqe_ctrl,
+					 ste_attr->wqe_data,
+					 ste_attr->wqe_tag,
+					 ste_attr->wqe_tag_is_jumbo,
+					 ste_attr->gta_opcode);
+		if (ret)
+			goto fail_rule;
+	}
+
+	/* Increase the status, this only works on good flow as the enum
+	 * is arrange it away creating -> created -> deleting -> deleted
+	 */
+	rule->status++;
+	mlx5dr_send_engine_gen_comp(queue, send_attr->user_data, RTE_FLOW_OP_SUCCESS);
+	return;
+
+fail_rule:
+	rule->status = !rule->rtc_0 && !rule->rtc_1 ?
+		MLX5DR_RULE_STATUS_FAILED : MLX5DR_RULE_STATUS_FAILING;
+	mlx5dr_send_engine_gen_comp(queue, send_attr->user_data, RTE_FLOW_OP_ERROR);
+}
+
 static void mlx5dr_send_engine_retry_post_send(struct mlx5dr_send_engine *queue,
 					       struct mlx5dr_send_ring_priv *priv,
 					       uint16_t wqe_cnt)
drivers/net/mlx5/hws/mlx5dr_send.h +6 -1
index fcddcc6366..1e845b1c7a 100644
@@ -52,7 +52,8 @@ struct mlx5dr_wqe_gta_ctrl_seg {
 
 struct mlx5dr_wqe_gta_data_seg_ste {
 	__be32 rsvd0_ctr_id;
-	__be32 rsvd1[4];
+	__be32 rsvd1_definer;
+	__be32 rsvd2[3];
 	__be32 action[3];
 	__be32 tag[8];
 };
@@ -159,6 +160,7 @@ struct mlx5dr_send_engine_post_attr {
 	uint8_t opmod;
 	uint8_t notify_hw;
 	uint8_t fence;
+	uint8_t match_definer_id;
 	size_t len;
 	struct mlx5dr_rule *rule;
 	uint32_t id;
@@ -238,6 +240,9 @@ void mlx5dr_send_engine_post_end(struct mlx5dr_send_engine_post_ctrl *ctrl,
 void mlx5dr_send_ste(struct mlx5dr_send_engine *queue,
 		     struct mlx5dr_send_ste_attr *ste_attr);
 
+void mlx5dr_send_stes_fw(struct mlx5dr_send_engine *queue,
+			 struct mlx5dr_send_ste_attr *ste_attr);
+
 void mlx5dr_send_engine_flush_queue(struct mlx5dr_send_engine *queue);
 
 static inline bool mlx5dr_send_engine_empty(struct mlx5dr_send_engine *queue)
Signature · 1 line
2.18.1
#24 lore raw

PATCH v2 7/16 net/mlx5/hws: add send FW range STE WQE

In reply to Alex Vesker

FW WQE supports complex rules, constructed from 2 STEs, for example:
Hash(DefinerA)
SteMatch(DefinerB)
SteRange(DefinerC)
DefinerA is a subset of DefinerB

This complex rule is written using a single FW command which has a single WQE control, STE match data0 and STE range data1. FW manages STEs/ICM and coherency between deletion and creation. It is possible to also pass the definer value as part of the STE, this is not supported with current HW.

---
 drivers/net/mlx5/hws/mlx5dr_send.c | 19 +++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_send.h |  3 +++
 2 files changed, 22 insertions(+)
drivers/net/mlx5/hws/mlx5dr_send.c +19 -0
index a9958df4f2..51aaf5c8e2 100644
@@ -242,11 +242,15 @@ int mlx5dr_send_wqe_fw(struct ibv_context *ibv_ctx,
 		       struct mlx5dr_wqe_gta_ctrl_seg *send_wqe_ctrl,
 		       void *send_wqe_match_data,
 		       void *send_wqe_match_tag,
+		       void *send_wqe_range_data,
+		       void *send_wqe_range_tag,
 		       bool is_jumbo,
 		       uint8_t gta_opcode)
 {
+	bool has_range = send_wqe_range_data || send_wqe_range_tag;
 	bool has_match = send_wqe_match_data || send_wqe_match_tag;
 	struct mlx5dr_wqe_gta_data_seg_ste gta_wqe_data0 = {0};
+	struct mlx5dr_wqe_gta_data_seg_ste gta_wqe_data1 = {0};
 	struct mlx5dr_wqe_gta_ctrl_seg gta_wqe_ctrl = {0};
 	struct mlx5dr_cmd_generate_wqe_attr attr = {0};
 	struct mlx5dr_wqe_ctrl_seg wqe_ctrl = {0};
@@ -278,6 +282,17 @@ int mlx5dr_send_wqe_fw(struct ibv_context *ibv_ctx,
 		attr.gta_data_0 = (uint8_t *)&gta_wqe_data0;
 	}
 
+	/* Set GTA range WQE DATA */
+	if (has_range) {
+		if (send_wqe_range_data)
+			memcpy(&gta_wqe_data1, send_wqe_range_data, sizeof(gta_wqe_data1));
+		else
+			mlx5dr_send_wqe_set_tag(&gta_wqe_data1, send_wqe_range_tag, false);
+
+		gta_wqe_data1.rsvd1_definer = htobe32(send_attr->range_definer_id << 8);
+		attr.gta_data_1 = (uint8_t *)&gta_wqe_data1;
+	}
+
 	attr.pdn = pd_num;
 	attr.wqe_ctrl = (uint8_t *)&wqe_ctrl;
 	attr.gta_ctrl = (uint8_t *)&gta_wqe_ctrl;
@@ -336,6 +351,8 @@ void mlx5dr_send_stes_fw(struct mlx5dr_send_engine *queue,
 					 ste_attr->wqe_ctrl,
 					 ste_attr->wqe_data,
 					 ste_attr->wqe_tag,
+					 ste_attr->range_wqe_data,
+					 ste_attr->range_wqe_tag,
 					 ste_attr->wqe_tag_is_jumbo,
 					 ste_attr->gta_opcode);
 		if (ret)
@@ -350,6 +367,8 @@ void mlx5dr_send_stes_fw(struct mlx5dr_send_engine *queue,
 					 ste_attr->wqe_ctrl,
 					 ste_attr->wqe_data,
 					 ste_attr->wqe_tag,
+					 ste_attr->range_wqe_data,
+					 ste_attr->range_wqe_tag,
 					 ste_attr->wqe_tag_is_jumbo,
 					 ste_attr->gta_opcode);
 		if (ret)
drivers/net/mlx5/hws/mlx5dr_send.h +3 -0
index 1e845b1c7a..47bb66b3c7 100644
@@ -161,6 +161,7 @@ struct mlx5dr_send_engine_post_attr {
 	uint8_t notify_hw;
 	uint8_t fence;
 	uint8_t match_definer_id;
+	uint8_t range_definer_id;
 	size_t len;
 	struct mlx5dr_rule *rule;
 	uint32_t id;
@@ -182,8 +183,10 @@ struct mlx5dr_send_ste_attr {
 	uint32_t direct_index;
 	struct mlx5dr_send_engine_post_attr send_attr;
 	struct mlx5dr_rule_match_tag *wqe_tag;
+	struct mlx5dr_rule_match_tag *range_wqe_tag;
 	struct mlx5dr_wqe_gta_ctrl_seg *wqe_ctrl;
 	struct mlx5dr_wqe_gta_data_seg_ste *wqe_data;
+	struct mlx5dr_wqe_gta_data_seg_ste *range_wqe_data;
 };
 
 /**
Signature · 1 line
2.18.1
#25 lore raw

PATCH v2 8/16 net/mlx5/hws: move matcher size check to function

In reply to Alex Vesker

This check can be later on reused for other places, it will look better in a function

---
 drivers/net/mlx5/hws/mlx5dr_matcher.c | 43 +++++++++++++++++----------
 1 file changed, 27 insertions(+), 16 deletions(-)
drivers/net/mlx5/hws/mlx5dr_matcher.c +27 -16
index 101a12d361..b8db0a27ae 100644
@@ -565,6 +565,32 @@ static void mlx5dr_matcher_destroy_rtc(struct mlx5dr_matcher *matcher,
 		mlx5dr_pool_chunk_free(ste_pool, ste);
 }
 
+static int
+mlx5dr_matcher_check_attr_sz(struct mlx5dr_cmd_query_caps *caps,
+			     struct mlx5dr_matcher_attr *attr)
+{
+	if (attr->table.sz_col_log > caps->rtc_log_depth_max) {
+		DR_LOG(ERR, "Matcher depth exceeds limit %d", caps->rtc_log_depth_max);
+		goto not_supported;
+	}
+
+	if (attr->table.sz_col_log + attr->table.sz_row_log > caps->ste_alloc_log_max) {
+		DR_LOG(ERR, "Total matcher size exceeds limit %d", caps->ste_alloc_log_max);
+		goto not_supported;
+	}
+
+	if (attr->table.sz_col_log + attr->table.sz_row_log < caps->ste_alloc_log_gran) {
+		DR_LOG(ERR, "Total matcher size below limit %d", caps->ste_alloc_log_gran);
+		goto not_supported;
+	}
+
+	return 0;
+
+not_supported:
+	rte_errno = EOPNOTSUPP;
+	return rte_errno;
+}
+
 static void mlx5dr_matcher_set_pool_attr(struct mlx5dr_pool_attr *attr,
 					 struct mlx5dr_matcher *matcher)
 {
@@ -840,22 +866,7 @@ mlx5dr_matcher_process_attr(struct mlx5dr_cmd_query_caps *caps,
 	    attr->insert_mode == MLX5DR_MATCHER_INSERT_BY_HASH)
 		attr->table.sz_col_log = mlx5dr_matcher_rules_to_tbl_depth(attr->rule.num_log);
 
-	if (attr->table.sz_col_log > caps->rtc_log_depth_max) {
-		DR_LOG(ERR, "Matcher depth exceeds limit %d", caps->rtc_log_depth_max);
-		goto not_supported;
-	}
-
-	if (attr->table.sz_col_log + attr->table.sz_row_log > caps->ste_alloc_log_max) {
-		DR_LOG(ERR, "Total matcher size exceeds limit %d", caps->ste_alloc_log_max);
-		goto not_supported;
-	}
-
-	if (attr->table.sz_col_log + attr->table.sz_row_log < caps->ste_alloc_log_gran) {
-		DR_LOG(ERR, "Total matcher size below limit %d", caps->ste_alloc_log_gran);
-		goto not_supported;
-	}
-
-	return 0;
+	return mlx5dr_matcher_check_attr_sz(caps, attr);
 
 not_supported:
 	rte_errno = EOPNOTSUPP;
Signature · 1 line
2.18.1
#26 lore raw

PATCH v2 9/16 net/mlx5/hws: support range match

In reply to Alex Vesker

Support range matching over selected items and range is not supported over all the items. The range match is done using:
item->last.field - maximum value item->mask.field - bitmask
item->spec.field - minimum value

When items are processed if item last and mask fields are non zero range matching will be done over these fields. There are two field setter, field copy (fc) and field copy range (fcr).

---
 drivers/net/mlx5/hws/mlx5dr_definer.c | 73 +++++++++++++++++++++++++--
 drivers/net/mlx5/hws/mlx5dr_definer.h |  5 +-
 2 files changed, 72 insertions(+), 6 deletions(-)
drivers/net/mlx5/hws/mlx5dr_definer.c +69 -4
index 6b98eb8c96..c268f94ad3 100644
@@ -123,6 +123,7 @@ struct mlx5dr_definer_conv_data {
 	X(SET,		ipv4_next_proto,	v->next_proto_id,	rte_ipv4_hdr) \
 	X(SET,		ipv4_version,		STE_IPV4,		rte_ipv4_hdr) \
 	X(SET_BE16,	ipv4_frag,		v->fragment_offset,	rte_ipv4_hdr) \
+	X(SET_BE16,	ipv4_len,		v->total_length,	rte_ipv4_hdr) \
 	X(SET_BE16,	ipv6_payload_len,	v->hdr.payload_len,	rte_flow_item_ipv6) \
 	X(SET,		ipv6_proto,		v->hdr.proto,		rte_flow_item_ipv6) \
 	X(SET,		ipv6_hop_limits,	v->hdr.hop_limits,	rte_flow_item_ipv6) \
@@ -516,6 +517,7 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 			      int item_idx)
 {
 	const struct rte_ipv4_hdr *m = item->mask;
+	const struct rte_ipv4_hdr *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	bool inner = cd->tunnel;
 
@@ -533,8 +535,8 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 	if (!m)
 		return 0;
 
-	if (m->total_length || m->packet_id ||
-	    m->hdr_checksum) {
+	if (m->packet_id || m->hdr_checksum ||
+	    (l && (l->next_proto_id || l->type_of_service))) {
 		rte_errno = ENOTSUP;
 		return rte_errno;
 	}
@@ -553,9 +555,18 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 		DR_CALC_SET(fc, eth_l3, protocol_next_header, inner);
 	}
 
+	if (m->total_length) {
+		fc = &cd->fc[DR_CALC_FNAME(IP_LEN, inner)];
+		fc->item_idx = item_idx;
+		fc->is_range = l && l->total_length;
+		fc->tag_set = &mlx5dr_definer_ipv4_len_set;
+		DR_CALC_SET(fc, eth_l3, ipv4_total_length, inner);
+	}
+
 	if (m->dst_addr) {
 		fc = &cd->fc[DR_CALC_FNAME(IPV4_DST, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->dst_addr;
 		fc->tag_set = &mlx5dr_definer_ipv4_dst_addr_set;
 		DR_CALC_SET(fc, ipv4_src_dest, destination_address, inner);
 	}
@@ -563,6 +574,7 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 	if (m->src_addr) {
 		fc = &cd->fc[DR_CALC_FNAME(IPV4_SRC, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->src_addr;
 		fc->tag_set = &mlx5dr_definer_ipv4_src_addr_set;
 		DR_CALC_SET(fc, ipv4_src_dest, source_address, inner);
 	}
@@ -570,6 +582,7 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 	if (m->ihl) {
 		fc = &cd->fc[DR_CALC_FNAME(IPV4_IHL, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->ihl;
 		fc->tag_set = &mlx5dr_definer_ipv4_ihl_set;
 		DR_CALC_SET(fc, eth_l3, ihl, inner);
 	}
@@ -577,6 +590,7 @@ mlx5dr_definer_conv_item_ipv4(struct mlx5dr_definer_conv_data *cd,
 	if (m->time_to_live) {
 		fc = &cd->fc[DR_CALC_FNAME(IP_TTL, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->time_to_live;
 		fc->tag_set = &mlx5dr_definer_ipv4_time_to_live_set;
 		DR_CALC_SET(fc, eth_l3, time_to_live_hop_limit, inner);
 	}
@@ -597,6 +611,7 @@ mlx5dr_definer_conv_item_ipv6(struct mlx5dr_definer_conv_data *cd,
 			      int item_idx)
 {
 	const struct rte_flow_item_ipv6 *m = item->mask;
+	const struct rte_flow_item_ipv6 *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	bool inner = cd->tunnel;
 
@@ -616,7 +631,10 @@ mlx5dr_definer_conv_item_ipv6(struct mlx5dr_definer_conv_data *cd,
 
 	if (m->has_hop_ext || m->has_route_ext || m->has_auth_ext ||
 	    m->has_esp_ext || m->has_dest_ext || m->has_mobil_ext ||
-	    m->has_hip_ext || m->has_shim6_ext) {
+	    m->has_hip_ext || m->has_shim6_ext ||
+	    (l && (l->has_frag_ext || l->hdr.vtc_flow || l->hdr.proto ||
+		   !is_mem_zero(l->hdr.src_addr, 16) ||
+		   !is_mem_zero(l->hdr.dst_addr, 16)))) {
 		rte_errno = ENOTSUP;
 		return rte_errno;
 	}
@@ -643,8 +661,9 @@ mlx5dr_definer_conv_item_ipv6(struct mlx5dr_definer_conv_data *cd,
 	}
 
 	if (m->hdr.payload_len) {
-		fc = &cd->fc[DR_CALC_FNAME(IPV6_PAYLOAD_LEN, inner)];
+		fc = &cd->fc[DR_CALC_FNAME(IP_LEN, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.payload_len;
 		fc->tag_set = &mlx5dr_definer_ipv6_payload_len_set;
 		DR_CALC_SET(fc, eth_l3, ipv6_payload_length, inner);
 	}
@@ -659,6 +678,7 @@ mlx5dr_definer_conv_item_ipv6(struct mlx5dr_definer_conv_data *cd,
 	if (m->hdr.hop_limits) {
 		fc = &cd->fc[DR_CALC_FNAME(IP_TTL, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.hop_limits;
 		fc->tag_set = &mlx5dr_definer_ipv6_hop_limits_set;
 		DR_CALC_SET(fc, eth_l3, time_to_live_hop_limit, inner);
 	}
@@ -728,6 +748,7 @@ mlx5dr_definer_conv_item_udp(struct mlx5dr_definer_conv_data *cd,
 			     int item_idx)
 {
 	const struct rte_flow_item_udp *m = item->mask;
+	const struct rte_flow_item_udp *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	bool inner = cd->tunnel;
 
@@ -751,6 +772,7 @@ mlx5dr_definer_conv_item_udp(struct mlx5dr_definer_conv_data *cd,
 	if (m->hdr.src_port) {
 		fc = &cd->fc[DR_CALC_FNAME(L4_SPORT, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.src_port;
 		fc->tag_set = &mlx5dr_definer_udp_src_port_set;
 		DR_CALC_SET(fc, eth_l4, source_port, inner);
 	}
@@ -758,6 +780,7 @@ mlx5dr_definer_conv_item_udp(struct mlx5dr_definer_conv_data *cd,
 	if (m->hdr.dst_port) {
 		fc = &cd->fc[DR_CALC_FNAME(L4_DPORT, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.dst_port;
 		fc->tag_set = &mlx5dr_definer_udp_dst_port_set;
 		DR_CALC_SET(fc, eth_l4, destination_port, inner);
 	}
@@ -771,6 +794,7 @@ mlx5dr_definer_conv_item_tcp(struct mlx5dr_definer_conv_data *cd,
 			     int item_idx)
 {
 	const struct rte_flow_item_tcp *m = item->mask;
+	const struct rte_flow_item_tcp *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	bool inner = cd->tunnel;
 
@@ -786,9 +810,16 @@ mlx5dr_definer_conv_item_tcp(struct mlx5dr_definer_conv_data *cd,
 	if (!m)
 		return 0;
 
+	if (m->hdr.sent_seq || m->hdr.recv_ack || m->hdr.data_off ||
+	    m->hdr.rx_win || m->hdr.cksum || m->hdr.tcp_urp) {
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+
 	if (m->hdr.tcp_flags) {
 		fc = &cd->fc[DR_CALC_FNAME(TCP_FLAGS, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.tcp_flags;
 		fc->tag_set = &mlx5dr_definer_tcp_flags_set;
 		DR_CALC_SET(fc, eth_l4, tcp_flags, inner);
 	}
@@ -796,6 +827,7 @@ mlx5dr_definer_conv_item_tcp(struct mlx5dr_definer_conv_data *cd,
 	if (m->hdr.src_port) {
 		fc = &cd->fc[DR_CALC_FNAME(L4_SPORT, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.src_port;
 		fc->tag_set = &mlx5dr_definer_tcp_src_port_set;
 		DR_CALC_SET(fc, eth_l4, source_port, inner);
 	}
@@ -803,6 +835,7 @@ mlx5dr_definer_conv_item_tcp(struct mlx5dr_definer_conv_data *cd,
 	if (m->hdr.dst_port) {
 		fc = &cd->fc[DR_CALC_FNAME(L4_DPORT, inner)];
 		fc->item_idx = item_idx;
+		fc->is_range = l && l->hdr.dst_port;
 		fc->tag_set = &mlx5dr_definer_tcp_dst_port_set;
 		DR_CALC_SET(fc, eth_l4, destination_port, inner);
 	}
@@ -1108,6 +1141,7 @@ mlx5dr_definer_conv_item_tag(struct mlx5dr_definer_conv_data *cd,
 {
 	const struct rte_flow_item_tag *m = item->mask;
 	const struct rte_flow_item_tag *v = item->spec;
+	const struct rte_flow_item_tag *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	int reg;
 
@@ -1130,7 +1164,9 @@ mlx5dr_definer_conv_item_tag(struct mlx5dr_definer_conv_data *cd,
 		return rte_errno;
 
 	fc->item_idx = item_idx;
+	fc->is_range = l && l->index;
 	fc->tag_set = &mlx5dr_definer_tag_set;
+
 	return 0;
 }
 
@@ -1140,6 +1176,7 @@ mlx5dr_definer_conv_item_metadata(struct mlx5dr_definer_conv_data *cd,
 				  int item_idx)
 {
 	const struct rte_flow_item_meta *m = item->mask;
+	const struct rte_flow_item_meta *l = item->last;
 	struct mlx5dr_definer_fc *fc;
 	int reg;
 
@@ -1158,7 +1195,9 @@ mlx5dr_definer_conv_item_metadata(struct mlx5dr_definer_conv_data *cd,
 		return rte_errno;
 
 	fc->item_idx = item_idx;
+	fc->is_range = l && l->data;
 	fc->tag_set = &mlx5dr_definer_metadata_set;
+
 	return 0;
 }
 
@@ -1465,6 +1504,28 @@ mlx5dr_definer_conv_item_meter_color(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static int
+mlx5dr_definer_check_item_range_supp(struct rte_flow_item *item)
+{
+	if (!item->last)
+		return 0;
+
+	switch ((int)item->type) {
+	case RTE_FLOW_ITEM_TYPE_IPV4:
+	case RTE_FLOW_ITEM_TYPE_IPV6:
+	case RTE_FLOW_ITEM_TYPE_UDP:
+	case RTE_FLOW_ITEM_TYPE_TCP:
+	case RTE_FLOW_ITEM_TYPE_TAG:
+	case RTE_FLOW_ITEM_TYPE_META:
+	case MLX5_RTE_FLOW_ITEM_TYPE_TAG:
+		return 0;
+	default:
+		DR_LOG(ERR, "Range not supported over item type %d", item->type);
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+}
+
 static int
 mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 				struct mlx5dr_match_template *mt,
@@ -1487,6 +1548,10 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 	for (i = 0; items->type != RTE_FLOW_ITEM_TYPE_END; i++, items++) {
 		cd.tunnel = !!(item_flags & MLX5_FLOW_LAYER_TUNNEL);
 
+		ret = mlx5dr_definer_check_item_range_supp(items);
+		if (ret)
+			return ret;
+
 		switch ((int)items->type) {
 		case RTE_FLOW_ITEM_TYPE_ETH:
 			ret = mlx5dr_definer_conv_item_eth(&cd, items, i);
drivers/net/mlx5/hws/mlx5dr_definer.h +3 -2
index d52c6b0627..bab4baae4a 100644
@@ -38,8 +38,8 @@ enum mlx5dr_definer_fname {
 	MLX5DR_DEFINER_FNAME_IP_VERSION_I,
 	MLX5DR_DEFINER_FNAME_IP_FRAG_O,
 	MLX5DR_DEFINER_FNAME_IP_FRAG_I,
-	MLX5DR_DEFINER_FNAME_IPV6_PAYLOAD_LEN_O,
-	MLX5DR_DEFINER_FNAME_IPV6_PAYLOAD_LEN_I,
+	MLX5DR_DEFINER_FNAME_IP_LEN_O,
+	MLX5DR_DEFINER_FNAME_IP_LEN_I,
 	MLX5DR_DEFINER_FNAME_IP_TOS_O,
 	MLX5DR_DEFINER_FNAME_IP_TOS_I,
 	MLX5DR_DEFINER_FNAME_IPV6_FLOW_LABEL_O,
@@ -116,6 +116,7 @@ enum mlx5dr_definer_type {
 
 struct mlx5dr_definer_fc {
 	uint8_t item_idx;
+	uint8_t is_range;
 	uint32_t byte_off;
 	int bit_off;
 	uint32_t bit_mask;
Signature · 1 line
2.18.1
#27 lore raw

PATCH v2 10/16 net/mlx5/hws: redesign definer create

In reply to Alex Vesker

Until now definer creation and deletion get and put functions were used, the get function would calculate the definer field copy (fc), header layout (hl) and definer layout internally without taking into account other match templates used over the same matcher.
This logic had to be split to allow sharing the hl over multiple definers. First calculate the shared hl than create definers based on the definer shared layout. Once all definers use the same layout it is possible to hash over the shared fields since the location is the same across all of the definers.

---
 drivers/net/mlx5/hws/mlx5dr_definer.c  | 301 ++++++++++++++++---------
 drivers/net/mlx5/hws/mlx5dr_definer.h  |  11 +-
 drivers/net/mlx5/hws/mlx5dr_internal.h |   2 +-
 drivers/net/mlx5/hws/mlx5dr_matcher.c  |  61 ++---
 drivers/net/mlx5/hws/mlx5dr_matcher.h  |  16 +-
 5 files changed, 230 insertions(+), 161 deletions(-)
drivers/net/mlx5/hws/mlx5dr_definer.c +192 -109
index c268f94ad3..75e997ba28 100644
@@ -104,7 +104,6 @@ struct mlx5dr_definer_conv_data {
 	struct mlx5dr_definer_fc *fc;
 	uint8_t relaxed;
 	uint8_t tunnel;
-	uint8_t *hl;
 };
 
 /* Xmacro used to create generic item setter from items */
@@ -1504,6 +1503,36 @@ mlx5dr_definer_conv_item_meter_color(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static int
+mlx5dr_definer_mt_set_fc(struct mlx5dr_match_template *mt,
+			 struct mlx5dr_definer_fc *fc,
+			 uint8_t *hl)
+{
+	uint32_t fc_sz = 0;
+	int i;
+
+	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++)
+		if (fc[i].tag_set)
+			fc_sz++;
+
+	mt->fc = simple_calloc(fc_sz, sizeof(*mt->fc));
+	if (!mt->fc) {
+		rte_errno = ENOMEM;
+		return rte_errno;
+	}
+
+	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++) {
+		if (!fc[i].tag_set)
+			continue;
+
+		fc[i].fname = i;
+		memcpy(&mt->fc[mt->fc_sz++], &fc[i], sizeof(*mt->fc));
+		DR_SET(hl, -1, fc[i].byte_off, fc[i].bit_off, fc[i].bit_mask);
+	}
+
+	return 0;
+}
+
 static int
 mlx5dr_definer_check_item_range_supp(struct rte_flow_item *item)
 {
@@ -1535,12 +1564,9 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 	struct mlx5dr_definer_conv_data cd = {0};
 	struct rte_flow_item *items = mt->items;
 	uint64_t item_flags = 0;
-	uint32_t total = 0;
-	int i, j;
-	int ret;
+	int i, ret;
 
 	cd.fc = fc;
-	cd.hl = hl;
 	cd.caps = ctx->caps;
 	cd.relaxed = mt->flags & MLX5DR_MATCH_TEMPLATE_FLAG_RELAXED_MATCH;
 
@@ -1660,29 +1686,11 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 
 	mt->item_flags = item_flags;
 
-	/* Fill in headers layout and calculate total number of fields  */
-	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++) {
-		if (fc[i].tag_set) {
-			total++;
-			DR_SET(hl, -1, fc[i].byte_off, fc[i].bit_off, fc[i].bit_mask);
-		}
-	}
-
-	mt->fc_sz = total;
-	mt->fc = simple_calloc(total, sizeof(*mt->fc));
-	if (!mt->fc) {
-		DR_LOG(ERR, "Failed to allocate field copy array");
-		rte_errno = ENOMEM;
-		return rte_errno;
-	}
-
-	j = 0;
-	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++) {
-		if (fc[i].tag_set) {
-			memcpy(&mt->fc[j], &fc[i], sizeof(*mt->fc));
-			mt->fc[j].fname = i;
-			j++;
-		}
+	/* Fill in headers layout and allocate fc array on mt */
+	ret = mlx5dr_definer_mt_set_fc(mt, fc, hl);
+	if (ret) {
+		DR_LOG(ERR, "Failed to set field copy to match template");
+		return ret;
 	}
 
 	return 0;
@@ -1837,8 +1845,8 @@ mlx5dr_definer_best_hl_fit_recu(struct mlx5dr_definer_sel_ctrl *ctrl,
 }
 
 static void
-mlx5dr_definer_apply_sel_ctrl(struct mlx5dr_definer_sel_ctrl *ctrl,
-			      struct mlx5dr_definer *definer)
+mlx5dr_definer_copy_sel_ctrl(struct mlx5dr_definer_sel_ctrl *ctrl,
+			     struct mlx5dr_definer *definer)
 {
 	memcpy(definer->byte_selector, ctrl->byte_selector, ctrl->allowed_bytes);
 	memcpy(definer->dw_selector, ctrl->full_dw_selector, ctrl->allowed_full_dw);
@@ -1848,7 +1856,7 @@ mlx5dr_definer_apply_sel_ctrl(struct mlx5dr_definer_sel_ctrl *ctrl,
 
 static int
 mlx5dr_definer_find_best_hl_fit(struct mlx5dr_context *ctx,
-				struct mlx5dr_match_template *mt,
+				struct mlx5dr_definer *definer,
 				uint8_t *hl)
 {
 	struct mlx5dr_definer_sel_ctrl ctrl = {0};
@@ -1861,8 +1869,8 @@ mlx5dr_definer_find_best_hl_fit(struct mlx5dr_context *ctx,
 
 	found = mlx5dr_definer_best_hl_fit_recu(&ctrl, 0, (uint32_t *)hl);
 	if (found) {
-		mlx5dr_definer_apply_sel_ctrl(&ctrl, mt->definer);
-		mt->definer->type = MLX5DR_DEFINER_TYPE_MATCH;
+		mlx5dr_definer_copy_sel_ctrl(&ctrl, definer);
+		definer->type = MLX5DR_DEFINER_TYPE_MATCH;
 		return 0;
 	}
 
@@ -1875,8 +1883,8 @@ mlx5dr_definer_find_best_hl_fit(struct mlx5dr_context *ctx,
 
 	found = mlx5dr_definer_best_hl_fit_recu(&ctrl, 0, (uint32_t *)hl);
 	if (found) {
-		mlx5dr_definer_apply_sel_ctrl(&ctrl, mt->definer);
-		mt->definer->type = MLX5DR_DEFINER_TYPE_JUMBO;
+		mlx5dr_definer_copy_sel_ctrl(&ctrl, definer);
+		definer->type = MLX5DR_DEFINER_TYPE_JUMBO;
 		return 0;
 	}
 
@@ -1920,114 +1928,189 @@ int mlx5dr_definer_get_id(struct mlx5dr_definer *definer)
 	return definer->obj->id;
 }
 
-int mlx5dr_definer_compare(struct mlx5dr_definer *definer_a,
-			   struct mlx5dr_definer *definer_b)
+static int
+mlx5dr_definer_calc_layout(struct mlx5dr_matcher *matcher,
+			   struct mlx5dr_definer *match_definer)
 {
-	int i;
+	struct mlx5dr_context *ctx = matcher->tbl->ctx;
+	struct mlx5dr_match_template *mt = matcher->mt;
+	uint8_t *match_hl, *hl_buff;
+	int i, ret;
 
-	if (definer_a->type != definer_b->type)
-		return 1;
+	/* Union header-layout (hl) is used for creating a single definer
+	 * field layout used with different bitmasks for hash and match.
+	 */
+	hl_buff = simple_calloc(1, MLX5_ST_SZ_BYTES(definer_hl));
+	if (!hl_buff) {
+		DR_LOG(ERR, "Failed to allocate memory for header layout");
+		rte_errno = ENOMEM;
+		return rte_errno;
+	}
 
-	for (i = 0; i < BYTE_SELECTORS; i++)
-		if (definer_a->byte_selector[i] != definer_b->byte_selector[i])
-			return 1;
+	match_hl = hl_buff;
 
-	for (i = 0; i < DW_SELECTORS; i++)
-		if (definer_a->dw_selector[i] != definer_b->dw_selector[i])
-			return 1;
+	/* Convert all mt items to header layout (hl)
+	 * and allocate the match field copy array (fc).
+	 */
+	for (i = 0; i < matcher->num_of_mt; i++) {
+		ret = mlx5dr_definer_conv_items_to_hl(ctx, &mt[i], match_hl);
+		if (ret) {
+			DR_LOG(ERR, "Failed to convert items to header layout");
+			goto free_fc;
+		}
+	}
 
-	for (i = 0; i < MLX5DR_JUMBO_TAG_SZ; i++)
-		if (definer_a->mask.jumbo[i] != definer_b->mask.jumbo[i])
-			return 1;
+	/* Find the match definer layout for header layout match union */
+	ret = mlx5dr_definer_find_best_hl_fit(ctx, match_definer, match_hl);
+	if (ret) {
+		DR_LOG(ERR, "Failed to create match definer from header layout");
+		goto free_fc;
+	}
 
+	simple_free(hl_buff);
 	return 0;
+
+free_fc:
+	for (i = 0; i < matcher->num_of_mt; i++)
+		if (mt[i].fc)
+			simple_free(mt[i].fc);
+
+	simple_free(hl_buff);
+	return rte_errno;
 }
 
-int mlx5dr_definer_get(struct mlx5dr_context *ctx,
-		       struct mlx5dr_match_template *mt)
+static struct mlx5dr_definer *
+mlx5dr_definer_alloc(struct ibv_context *ibv_ctx,
+		     struct mlx5dr_definer_fc *fc,
+		     int fc_sz,
+		     struct rte_flow_item *items,
+		     struct mlx5dr_definer *layout)
 {
 	struct mlx5dr_cmd_definer_create_attr def_attr = {0};
-	struct ibv_context *ibv_ctx = ctx->ibv_ctx;
-	uint8_t *hl;
+	struct mlx5dr_definer *definer;
 	int ret;
 
-	if (mt->refcount++)
-		return 0;
-
-	mt->definer = simple_calloc(1, sizeof(*mt->definer));
-	if (!mt->definer) {
+	definer = simple_calloc(1, sizeof(*definer));
+	if (!definer) {
 		DR_LOG(ERR, "Failed to allocate memory for definer");
 		rte_errno = ENOMEM;
-		goto dec_refcount;
-	}
-
-	/* Header layout (hl) holds full bit mask per field */
-	hl = simple_calloc(1, MLX5_ST_SZ_BYTES(definer_hl));
-	if (!hl) {
-		DR_LOG(ERR, "Failed to allocate memory for header layout");
-		rte_errno = ENOMEM;
-		goto free_definer;
+		return NULL;
 	}
 
-	/* Convert items to hl and allocate the field copy array (fc) */
-	ret = mlx5dr_definer_conv_items_to_hl(ctx, mt, hl);
-	if (ret) {
-		DR_LOG(ERR, "Failed to convert items to hl");
-		goto free_hl;
-	}
+	memcpy(definer, layout, sizeof(*definer));
 
-	/* Find the definer for given header layout */
-	ret = mlx5dr_definer_find_best_hl_fit(ctx, mt, hl);
-	if (ret) {
-		DR_LOG(ERR, "Failed to create definer from header layout");
-		goto free_field_copy;
-	}
-
-	/* Align field copy array based on the new definer */
-	ret = mlx5dr_definer_fc_bind(mt->definer,
-				     mt->fc,
-				     mt->fc_sz);
+	/* Align field copy array based on given layout */
+	ret = mlx5dr_definer_fc_bind(definer, fc, fc_sz);
 	if (ret) {
 		DR_LOG(ERR, "Failed to bind field copy to definer");
-		goto free_field_copy;
+		goto free_definer;
 	}
 
 	/* Create the tag mask used for definer creation */
-	mlx5dr_definer_create_tag_mask(mt->items,
-				       mt->fc,
-				       mt->fc_sz,
-				       mt->definer->mask.jumbo);
+	mlx5dr_definer_create_tag_mask(items, fc, fc_sz, definer->mask.jumbo);
 
 	/* Create definer based on the bitmask tag */
-	def_attr.match_mask = mt->definer->mask.jumbo;
-	def_attr.dw_selector = mt->definer->dw_selector;
-	def_attr.byte_selector = mt->definer->byte_selector;
-	mt->definer->obj = mlx5dr_cmd_definer_create(ibv_ctx, &def_attr);
-	if (!mt->definer->obj)
-		goto free_field_copy;
+	def_attr.match_mask = definer->mask.jumbo;
+	def_attr.dw_selector = layout->dw_selector;
+	def_attr.byte_selector = layout->byte_selector;
 
-	simple_free(hl);
+	definer->obj = mlx5dr_cmd_definer_create(ibv_ctx, &def_attr);
+	if (!definer->obj)
+		goto free_definer;
 
-	return 0;
+	return definer;
 
-free_field_copy:
-	simple_free(mt->fc);
-free_hl:
-	simple_free(hl);
 free_definer:
-	simple_free(mt->definer);
-dec_refcount:
-	mt->refcount--;
+	simple_free(definer);
+	return NULL;
+}
+
+static void
+mlx5dr_definer_free(struct mlx5dr_definer *definer)
+{
+	mlx5dr_cmd_destroy_obj(definer->obj);
+	simple_free(definer);
+}
+
+static int
+mlx5dr_definer_matcher_match_init(struct mlx5dr_context *ctx,
+				  struct mlx5dr_matcher *matcher,
+				  struct mlx5dr_definer *match_layout)
+{
+	struct mlx5dr_match_template *mt = matcher->mt;
+	int i;
+
+	/* Create mendatory match definer */
+	for (i = 0; i < matcher->num_of_mt; i++) {
+		mt[i].definer = mlx5dr_definer_alloc(ctx->ibv_ctx,
+						     mt[i].fc,
+						     mt[i].fc_sz,
+						     mt[i].items,
+						     match_layout);
+		if (!mt[i].definer) {
+			DR_LOG(ERR, "Failed to create match definer");
+			goto free_definers;
+		}
+	}
+	return 0;
+
+free_definers:
+	while (i--)
+		mlx5dr_definer_free(mt[i].definer);
 
 	return rte_errno;
 }
 
-void mlx5dr_definer_put(struct mlx5dr_match_template *mt)
+static void
+mlx5dr_definer_matcher_match_uninit(struct mlx5dr_matcher *matcher)
 {
-	if (--mt->refcount)
+	int i;
+
+	for (i = 0; i < matcher->num_of_mt; i++)
+		mlx5dr_definer_free(matcher->mt[i].definer);
+}
+
+int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
+				struct mlx5dr_matcher *matcher)
+{
+	struct mlx5dr_definer match_layout = {0};
+	int ret, i;
+
+	if (matcher->flags & MLX5DR_MATCHER_FLAGS_COLLISION)
+		return 0;
+
+	/* Calculate header layout based on matcher items */
+	ret = mlx5dr_definer_calc_layout(matcher, &match_layout);
+	if (ret) {
+		DR_LOG(ERR, "Failed to calculate matcher definer layout");
+		return ret;
+	}
+
+	/* Calculate definers needed for exact match */
+	ret = mlx5dr_definer_matcher_match_init(ctx, matcher, &match_layout);
+	if (ret) {
+		DR_LOG(ERR, "Failed to init match definers");
+		goto free_fc;
+	}
+
+	return 0;
+
+free_fc:
+	for (i = 0; i < matcher->num_of_mt; i++)
+		simple_free(matcher->mt[i].fc);
+
+	return ret;
+}
+
+void mlx5dr_definer_matcher_uninit(struct mlx5dr_matcher *matcher)
+{
+	int i;
+
+	if (matcher->flags & MLX5DR_MATCHER_FLAGS_COLLISION)
 		return;
 
-	simple_free(mt->fc);
-	mlx5dr_cmd_destroy_obj(mt->definer->obj);
-	simple_free(mt->definer);
+	mlx5dr_definer_matcher_match_uninit(matcher);
+
+	for (i = 0; i < matcher->num_of_mt; i++)
+		simple_free(matcher->mt[i].fc);
 }
drivers/net/mlx5/hws/mlx5dr_definer.h +4 -7
index bab4baae4a..a14a08838a 100644
@@ -573,14 +573,11 @@ void mlx5dr_definer_create_tag(const struct rte_flow_item *items,
 			       uint32_t fc_sz,
 			       uint8_t *tag);
 
-int mlx5dr_definer_compare(struct mlx5dr_definer *definer_a,
-			   struct mlx5dr_definer *definer_b);
-
 int mlx5dr_definer_get_id(struct mlx5dr_definer *definer);
 
-int mlx5dr_definer_get(struct mlx5dr_context *ctx,
-		       struct mlx5dr_match_template *mt);
+int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
+				struct mlx5dr_matcher *matcher);
 
-void mlx5dr_definer_put(struct mlx5dr_match_template *mt);
+void mlx5dr_definer_matcher_uninit(struct mlx5dr_matcher *matcher);
 
-#endif /* MLX5DR_DEFINER_H_ */
+#endif
drivers/net/mlx5/hws/mlx5dr_internal.h +1 -1
index faad2bbd0f..c3c077667d 100644
@@ -30,12 +30,12 @@
 #include "mlx5dr_pool.h"
 #include "mlx5dr_context.h"
 #include "mlx5dr_table.h"
-#include "mlx5dr_matcher.h"
 #include "mlx5dr_send.h"
 #include "mlx5dr_rule.h"
 #include "mlx5dr_cmd.h"
 #include "mlx5dr_action.h"
 #include "mlx5dr_definer.h"
+#include "mlx5dr_matcher.h"
 #include "mlx5dr_debug.h"
 #include "mlx5dr_pat_arg.h"
 
drivers/net/mlx5/hws/mlx5dr_matcher.c +19 -42

133 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_matcher.h +14 -2
index b957f5ea4b..fabf50a343 100644
@@ -22,15 +22,19 @@
 /* Required depth of the main large table */
 #define MLX5DR_MATCHER_ASSURED_MAIN_TBL_DEPTH 2
 
+enum mlx5dr_matcher_flags {
+	MLX5DR_MATCHER_FLAGS_COLLISION		= 1 << 0,
+};
+
 struct mlx5dr_match_template {
 	struct rte_flow_item *items;
 	struct mlx5dr_definer *definer;
+	struct mlx5dr_definer *range_definer;
 	struct mlx5dr_definer_fc *fc;
-	uint32_t fc_sz;
+	uint16_t fc_sz;
 	uint64_t item_flags;
 	uint8_t vport_item_id;
 	enum mlx5dr_match_template_flags flags;
-	uint32_t refcount;
 };
 
 struct mlx5dr_matcher_match_ste {
@@ -59,6 +63,8 @@ struct mlx5dr_matcher {
 	uint8_t num_of_mt;
 	struct mlx5dr_action_template *at;
 	uint8_t num_of_at;
+	/* enum mlx5dr_matcher_flags */
+	uint8_t flags;
 	struct mlx5dr_devx_obj *end_ft;
 	struct mlx5dr_matcher *col_matcher;
 	struct mlx5dr_matcher_match_ste match_ste;
@@ -66,6 +72,12 @@ struct mlx5dr_matcher {
 	LIST_ENTRY(mlx5dr_matcher) next;
 };
 
+static inline bool
+mlx5dr_matcher_mt_is_jumbo(struct mlx5dr_match_template *mt)
+{
+	return mlx5dr_definer_is_jumbo(mt->definer);
+}
+
 int mlx5dr_matcher_conv_items_to_prm(uint64_t *match_buf,
 				     struct rte_flow_item *items,
 				     uint8_t *match_criteria,
Signature · 1 line
2.18.1
#28 lore raw

PATCH v2 11/16 net/mlx5/hws: support partial hash

In reply to Alex Vesker

Hash definers allow performing hashing over a subset of the fields which are used for matching. This allows combining match templates which were considered invalid until now. During matcher creation mlx5dr code will process the match templates and check if such hash definer is needed based on the definers bitmasks intersection.
Since current HW GTA implementation doesn't allow specifying match and hash definers rule insertion is done using the FW GTA WQE command.

---
 drivers/common/mlx5/mlx5_prm.h        |   4 +
 drivers/net/mlx5/hws/mlx5dr_definer.c | 105 ++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_matcher.c |  66 +++++++++++++++-
 drivers/net/mlx5/hws/mlx5dr_matcher.h |  10 ++-
 4 files changed, 181 insertions(+), 4 deletions(-)
drivers/common/mlx5/mlx5_prm.h +4 -0
index 298cc48b06..243952bf85 100644
@@ -2112,6 +2112,10 @@ enum mlx5_ifc_cross_vhca_allowed_objects_types {
 	MLX5_CROSS_VHCA_ALLOWED_OBJS_RTC = 1 << 0xa,
 };
 
+enum {
+	MLX5_GENERATE_WQE_TYPE_FLOW_UPDATE = 1 << 1,
+};
+
 /*
  *  HCA Capabilities 2
  */
drivers/net/mlx5/hws/mlx5dr_definer.c +105 -0

131 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_matcher.c +63 -3

108 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_matcher.h +9 -1
index fabf50a343..8dadc0ee66 100644
@@ -23,7 +23,8 @@
 #define MLX5DR_MATCHER_ASSURED_MAIN_TBL_DEPTH 2
 
 enum mlx5dr_matcher_flags {
-	MLX5DR_MATCHER_FLAGS_COLLISION		= 1 << 0,
+	MLX5DR_MATCHER_FLAGS_HASH_DEFINER	= 1 << 0,
+	MLX5DR_MATCHER_FLAGS_COLLISION		= 1 << 1,
 };
 
 struct mlx5dr_match_template {
@@ -69,6 +70,7 @@ struct mlx5dr_matcher {
 	struct mlx5dr_matcher *col_matcher;
 	struct mlx5dr_matcher_match_ste match_ste;
 	struct mlx5dr_matcher_action_ste action_ste;
+	struct mlx5dr_definer *hash_definer;
 	LIST_ENTRY(mlx5dr_matcher) next;
 };
 
@@ -78,6 +80,12 @@ mlx5dr_matcher_mt_is_jumbo(struct mlx5dr_match_template *mt)
 	return mlx5dr_definer_is_jumbo(mt->definer);
 }
 
+static inline bool mlx5dr_matcher_req_fw_wqe(struct mlx5dr_matcher *matcher)
+{
+	/* Currently HWS doesn't support hash different from match or range */
+	return unlikely(matcher->flags & MLX5DR_MATCHER_FLAGS_HASH_DEFINER);
+}
+
 int mlx5dr_matcher_conv_items_to_prm(uint64_t *match_buf,
 				     struct rte_flow_item *items,
 				     uint8_t *match_criteria,
Signature · 1 line
2.18.1
#29 lore raw

PATCH v2 12/16 net/mlx5/hws: add range definer creation support

In reply to Alex Vesker

Calculate and create an additional definer used for range check during matcher creation. In such case two definers will be created one for specific matching and a range definer. Since range HW GTA WQE doesn't support the needed range support rule insertion rule insertion is done using the FW GTA WQE command.

---
 drivers/net/mlx5/hws/mlx5dr_definer.c | 255 +++++++++++++++++++++++---
 drivers/net/mlx5/hws/mlx5dr_definer.h |  16 +-
 drivers/net/mlx5/hws/mlx5dr_matcher.c |  27 ++-
 drivers/net/mlx5/hws/mlx5dr_matcher.h |  17 +-
 4 files changed, 281 insertions(+), 34 deletions(-)
drivers/net/mlx5/hws/mlx5dr_definer.c +227 -28
index 0da38a818c..ed91a0998c 100644
@@ -1508,26 +1508,33 @@ mlx5dr_definer_mt_set_fc(struct mlx5dr_match_template *mt,
 			 struct mlx5dr_definer_fc *fc,
 			 uint8_t *hl)
 {
-	uint32_t fc_sz = 0;
+	uint32_t fc_sz = 0, fcr_sz = 0;
 	int i;
 
 	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++)
 		if (fc[i].tag_set)
-			fc_sz++;
+			fc[i].is_range ? fcr_sz++ : fc_sz++;
 
-	mt->fc = simple_calloc(fc_sz, sizeof(*mt->fc));
+	mt->fc = simple_calloc(fc_sz + fcr_sz, sizeof(*mt->fc));
 	if (!mt->fc) {
 		rte_errno = ENOMEM;
 		return rte_errno;
 	}
 
+	mt->fcr = mt->fc + fc_sz;
+
 	for (i = 0; i < MLX5DR_DEFINER_FNAME_MAX; i++) {
 		if (!fc[i].tag_set)
 			continue;
 
 		fc[i].fname = i;
-		memcpy(&mt->fc[mt->fc_sz++], &fc[i], sizeof(*mt->fc));
-		DR_SET(hl, -1, fc[i].byte_off, fc[i].bit_off, fc[i].bit_mask);
+
+		if (fc[i].is_range) {
+			memcpy(&mt->fcr[mt->fcr_sz++], &fc[i], sizeof(*mt->fcr));
+		} else {
+			memcpy(&mt->fc[mt->fc_sz++], &fc[i], sizeof(*mt->fc));
+			DR_SET(hl, -1, fc[i].byte_off, fc[i].bit_off, fc[i].bit_mask);
+		}
 	}
 
 	return 0;
@@ -1686,7 +1693,7 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 
 	mt->item_flags = item_flags;
 
-	/* Fill in headers layout and allocate fc array on mt */
+	/* Fill in headers layout and allocate fc & fcr array on mt */
 	ret = mlx5dr_definer_mt_set_fc(mt, fc, hl);
 	if (ret) {
 		DR_LOG(ERR, "Failed to set field copy to match template");
@@ -1855,9 +1862,92 @@ mlx5dr_definer_copy_sel_ctrl(struct mlx5dr_definer_sel_ctrl *ctrl,
 }
 
 static int
-mlx5dr_definer_find_best_hl_fit(struct mlx5dr_context *ctx,
-				struct mlx5dr_definer *definer,
-				uint8_t *hl)
+mlx5dr_definer_find_best_range_fit(struct mlx5dr_definer *definer,
+				   struct mlx5dr_matcher *matcher)
+{
+	uint8_t tag_byte_offset[MLX5DR_DEFINER_FNAME_MAX] = {0};
+	uint8_t field_select[MLX5DR_DEFINER_FNAME_MAX] = {0};
+	struct mlx5dr_definer_sel_ctrl ctrl = {0};
+	uint32_t byte_offset, algn_byte_off;
+	struct mlx5dr_definer_fc *fcr;
+	bool require_dw;
+	int idx, i, j;
+
+	/* Try to create a range definer */
+	ctrl.allowed_full_dw = DW_SELECTORS_RANGE;
+	ctrl.allowed_bytes = BYTE_SELECTORS_RANGE;
+
+	/* Multiple fields cannot share the same DW for range match.
+	 * The HW doesn't recognize each field but compares the full dw.
+	 * For example definer DW consists of FieldA_FieldB
+	 * FieldA: Mask 0xFFFF range 0x1 to 0x2
+	 * FieldB: Mask 0xFFFF range 0x3 to 0x4
+	 * STE DW range will be 0x00010003 - 0x00020004
+	 * This will cause invalid match for FieldB if FieldA=1 and FieldB=8
+	 * Since 0x10003 < 0x10008 < 0x20004
+	 */
+	for (i = 0; i < matcher->num_of_mt; i++) {
+		for (j = 0; j < matcher->mt[i].fcr_sz; j++) {
+			fcr = &matcher->mt[i].fcr[j];
+
+			/* Found - Reuse previous mt binding */
+			if (field_select[fcr->fname]) {
+				fcr->byte_off = tag_byte_offset[fcr->fname];
+				continue;
+			}
+
+			/* Not found */
+			require_dw = fcr->byte_off >= (64 * DW_SIZE);
+			if (require_dw || ctrl.used_bytes == ctrl.allowed_bytes) {
+				/* Try to cover using DW selector */
+				if (ctrl.used_full_dw == ctrl.allowed_full_dw)
+					goto not_supported;
+
+				ctrl.full_dw_selector[ctrl.used_full_dw++] =
+					fcr->byte_off / DW_SIZE;
+
+				/* Bind DW */
+				idx = ctrl.used_full_dw - 1;
+				byte_offset = fcr->byte_off % DW_SIZE;
+				byte_offset += DW_SIZE * (DW_SELECTORS - idx - 1);
+			} else {
+				/* Try to cover using Bytes selectors */
+				if (ctrl.used_bytes == ctrl.allowed_bytes)
+					goto not_supported;
+
+				algn_byte_off = DW_SIZE * (fcr->byte_off / DW_SIZE);
+				ctrl.byte_selector[ctrl.used_bytes++] = algn_byte_off + 3;
+				ctrl.byte_selector[ctrl.used_bytes++] = algn_byte_off + 2;
+				ctrl.byte_selector[ctrl.used_bytes++] = algn_byte_off + 1;
+				ctrl.byte_selector[ctrl.used_bytes++] = algn_byte_off;
+
+				/* Bind BYTE */
+				byte_offset = DW_SIZE * DW_SELECTORS;
+				byte_offset += BYTE_SELECTORS - ctrl.used_bytes;
+				byte_offset += fcr->byte_off % DW_SIZE;
+			}
+
+			fcr->byte_off = byte_offset;
+			tag_byte_offset[fcr->fname] = byte_offset;
+			field_select[fcr->fname] = 1;
+		}
+	}
+
+	mlx5dr_definer_copy_sel_ctrl(&ctrl, definer);
+	definer->type = MLX5DR_DEFINER_TYPE_RANGE;
+
+	return 0;
+
+not_supported:
+	DR_LOG(ERR, "Unable to find supporting range definer combination");
+	rte_errno = ENOTSUP;
+	return rte_errno;
+}
+
+static int
+mlx5dr_definer_find_best_match_fit(struct mlx5dr_context *ctx,
+				   struct mlx5dr_definer *definer,
+				   uint8_t *hl)
 {
 	struct mlx5dr_definer_sel_ctrl ctrl = {0};
 	bool found;
@@ -1923,6 +2013,43 @@ void mlx5dr_definer_create_tag(const struct rte_flow_item *items,
 	}
 }
 
+static uint32_t mlx5dr_definer_get_range_byte_off(uint32_t match_byte_off)
+{
+	uint8_t curr_dw_idx = match_byte_off / DW_SIZE;
+	uint8_t new_dw_idx;
+
+	/* Range DW can have the following values 7,8,9,10
+	 * -DW7 is mapped to DW9
+	 * -DW8 is mapped to DW7
+	 * -DW9 is mapped to DW5
+	 * -DW10 is mapped to DW3
+	 * To reduce calculation the following formula is used:
+	 */
+	new_dw_idx = curr_dw_idx * (-2) + 23;
+
+	return new_dw_idx * DW_SIZE + match_byte_off % DW_SIZE;
+}
+
+void mlx5dr_definer_create_tag_range(const struct rte_flow_item *items,
+				     struct mlx5dr_definer_fc *fc,
+				     uint32_t fc_sz,
+				     uint8_t *tag)
+{
+	struct mlx5dr_definer_fc tmp_fc;
+	uint32_t i;
+
+	for (i = 0; i < fc_sz; i++) {
+		tmp_fc = *fc;
+		/* Set MAX value */
+		tmp_fc.byte_off = mlx5dr_definer_get_range_byte_off(fc->byte_off);
+		tmp_fc.tag_set(&tmp_fc, items[fc->item_idx].last, tag);
+		/* Set MIN value */
+		tmp_fc.byte_off += DW_SIZE;
+		tmp_fc.tag_set(&tmp_fc, items[fc->item_idx].spec, tag);
+		fc++;
+	}
+}
+
 int mlx5dr_definer_get_id(struct mlx5dr_definer *definer)
 {
 	return definer->obj->id;
@@ -1951,27 +2078,26 @@ mlx5dr_definer_compare(struct mlx5dr_definer *definer_a,
 
 static int
 mlx5dr_definer_calc_layout(struct mlx5dr_matcher *matcher,
-			   struct mlx5dr_definer *match_definer)
+			   struct mlx5dr_definer *match_definer,
+			   struct mlx5dr_definer *range_definer)
 {
 	struct mlx5dr_context *ctx = matcher->tbl->ctx;
 	struct mlx5dr_match_template *mt = matcher->mt;
-	uint8_t *match_hl, *hl_buff;
+	uint8_t *match_hl;
 	int i, ret;
 
 	/* Union header-layout (hl) is used for creating a single definer
 	 * field layout used with different bitmasks for hash and match.
 	 */
-	hl_buff = simple_calloc(1, MLX5_ST_SZ_BYTES(definer_hl));
-	if (!hl_buff) {
+	match_hl = simple_calloc(1, MLX5_ST_SZ_BYTES(definer_hl));
+	if (!match_hl) {
 		DR_LOG(ERR, "Failed to allocate memory for header layout");
 		rte_errno = ENOMEM;
 		return rte_errno;
 	}
 
-	match_hl = hl_buff;
-
 	/* Convert all mt items to header layout (hl)
-	 * and allocate the match field copy array (fc).
+	 * and allocate the match and range field copy array (fc & fcr).
 	 */
 	for (i = 0; i < matcher->num_of_mt; i++) {
 		ret = mlx5dr_definer_conv_items_to_hl(ctx, &mt[i], match_hl);
@@ -1982,13 +2108,20 @@ mlx5dr_definer_calc_layout(struct mlx5dr_matcher *matcher,
 	}
 
 	/* Find the match definer layout for header layout match union */
-	ret = mlx5dr_definer_find_best_hl_fit(ctx, match_definer, match_hl);
+	ret = mlx5dr_definer_find_best_match_fit(ctx, match_definer, match_hl);
 	if (ret) {
 		DR_LOG(ERR, "Failed to create match definer from header layout");
 		goto free_fc;
 	}
 
-	simple_free(hl_buff);
+	/* Find the range definer layout for match templates fcrs */
+	ret = mlx5dr_definer_find_best_range_fit(range_definer, matcher);
+	if (ret) {
+		DR_LOG(ERR, "Failed to create range definer from header layout");
+		goto free_fc;
+	}
+
+	simple_free(match_hl);
 	return 0;
 
 free_fc:
@@ -1996,7 +2129,7 @@ mlx5dr_definer_calc_layout(struct mlx5dr_matcher *matcher,
 		if (mt[i].fc)
 			simple_free(mt[i].fc);
 
-	simple_free(hl_buff);
+	simple_free(match_hl);
 	return rte_errno;
 }
 
@@ -2005,7 +2138,8 @@ mlx5dr_definer_alloc(struct ibv_context *ibv_ctx,
 		     struct mlx5dr_definer_fc *fc,
 		     int fc_sz,
 		     struct rte_flow_item *items,
-		     struct mlx5dr_definer *layout)
+		     struct mlx5dr_definer *layout,
+		     bool bind_fc)
 {
 	struct mlx5dr_cmd_definer_create_attr def_attr = {0};
 	struct mlx5dr_definer *definer;
@@ -2021,10 +2155,12 @@ mlx5dr_definer_alloc(struct ibv_context *ibv_ctx,
 	memcpy(definer, layout, sizeof(*definer));
 
 	/* Align field copy array based on given layout */
-	ret = mlx5dr_definer_fc_bind(definer, fc, fc_sz);
-	if (ret) {
-		DR_LOG(ERR, "Failed to bind field copy to definer");
-		goto free_definer;
+	if (bind_fc) {
+		ret = mlx5dr_definer_fc_bind(definer, fc, fc_sz);
+		if (ret) {
+			DR_LOG(ERR, "Failed to bind field copy to definer");
+			goto free_definer;
+		}
 	}
 
 	/* Create the tag mask used for definer creation */
@@ -2067,7 +2203,8 @@ mlx5dr_definer_matcher_match_init(struct mlx5dr_context *ctx,
 						     mt[i].fc,
 						     mt[i].fc_sz,
 						     mt[i].items,
-						     match_layout);
+						     match_layout,
+						     true);
 		if (!mt[i].definer) {
 			DR_LOG(ERR, "Failed to create match definer");
 			goto free_definers;
@@ -2091,6 +2228,58 @@ mlx5dr_definer_matcher_match_uninit(struct mlx5dr_matcher *matcher)
 		mlx5dr_definer_free(matcher->mt[i].definer);
 }
 
+static int
+mlx5dr_definer_matcher_range_init(struct mlx5dr_context *ctx,
+				  struct mlx5dr_matcher *matcher,
+				  struct mlx5dr_definer *range_layout)
+{
+	struct mlx5dr_match_template *mt = matcher->mt;
+	int i;
+
+	/* Create optional range definers */
+	for (i = 0; i < matcher->num_of_mt; i++) {
+		if (!mt[i].fcr_sz)
+			continue;
+
+		/* All must use range if requested */
+		if (i && !mt[i - 1].range_definer) {
+			DR_LOG(ERR, "Using range and non range templates is not allowed");
+			goto free_definers;
+		}
+
+		matcher->flags |= MLX5DR_MATCHER_FLAGS_RANGE_DEFINER;
+		/* Create definer without fcr binding, already binded */
+		mt[i].range_definer = mlx5dr_definer_alloc(ctx->ibv_ctx,
+							   mt[i].fcr,
+							   mt[i].fcr_sz,
+							   mt[i].items,
+							   range_layout,
+							   false);
+		if (!mt[i].range_definer) {
+			DR_LOG(ERR, "Failed to create match definer");
+			goto free_definers;
+		}
+	}
+	return 0;
+
+free_definers:
+	while (i--)
+		if (mt[i].range_definer)
+			mlx5dr_definer_free(mt[i].range_definer);
+
+	return rte_errno;
+}
+
+static void
+mlx5dr_definer_matcher_range_uninit(struct mlx5dr_matcher *matcher)
+{
+	int i;
+
+	for (i = 0; i < matcher->num_of_mt; i++)
+		if (matcher->mt[i].range_definer)
+			mlx5dr_definer_free(matcher->mt[i].range_definer);
+}
+
 static int
 mlx5dr_definer_matcher_hash_init(struct mlx5dr_context *ctx,
 				 struct mlx5dr_matcher *matcher)
@@ -2169,13 +2358,13 @@ int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
 				struct mlx5dr_matcher *matcher)
 {
 	struct mlx5dr_definer match_layout = {0};
+	struct mlx5dr_definer range_layout = {0};
 	int ret, i;
 
 	if (matcher->flags & MLX5DR_MATCHER_FLAGS_COLLISION)
 		return 0;
 
-	/* Calculate header layout based on matcher items */
-	ret = mlx5dr_definer_calc_layout(matcher, &match_layout);
+	ret = mlx5dr_definer_calc_layout(matcher, &match_layout, &range_layout);
 	if (ret) {
 		DR_LOG(ERR, "Failed to calculate matcher definer layout");
 		return ret;
@@ -2188,15 +2377,24 @@ int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
 		goto free_fc;
 	}
 
+	/* Calculate definers needed for range */
+	ret = mlx5dr_definer_matcher_range_init(ctx, matcher, &range_layout);
+	if (ret) {
+		DR_LOG(ERR, "Failed to init range definers");
+		goto uninit_match_definer;
+	}
+
 	/* Calculate partial hash definer */
 	ret = mlx5dr_definer_matcher_hash_init(ctx, matcher);
 	if (ret) {
 		DR_LOG(ERR, "Failed to init hash definer");
-		goto uninit_match_definer;
+		goto uninit_range_definer;
 	}
 
 	return 0;
 
+uninit_range_definer:
+	mlx5dr_definer_matcher_range_uninit(matcher);
 uninit_match_definer:
 	mlx5dr_definer_matcher_match_uninit(matcher);
 free_fc:
@@ -2214,6 +2412,7 @@ void mlx5dr_definer_matcher_uninit(struct mlx5dr_matcher *matcher)
 		return;
 
 	mlx5dr_definer_matcher_hash_uninit(matcher);
+	mlx5dr_definer_matcher_range_uninit(matcher);
 	mlx5dr_definer_matcher_match_uninit(matcher);
 
 	for (i = 0; i < matcher->num_of_mt; i++)
drivers/net/mlx5/hws/mlx5dr_definer.h +14 -2
index a14a08838a..dd9a297007 100644
@@ -5,11 +5,17 @@
 #ifndef MLX5DR_DEFINER_H_
 #define MLX5DR_DEFINER_H_
 
+/* Max available selecotrs */
+#define DW_SELECTORS 9
+#define BYTE_SELECTORS 8
+
 /* Selectors based on match TAG */
 #define DW_SELECTORS_MATCH 6
 #define DW_SELECTORS_LIMITED 3
-#define DW_SELECTORS 9
-#define BYTE_SELECTORS 8
+
+/* Selectors based on range TAG */
+#define DW_SELECTORS_RANGE 2
+#define BYTE_SELECTORS_RANGE 8
 
 enum mlx5dr_definer_fname {
 	MLX5DR_DEFINER_FNAME_ETH_SMAC_48_16_O,
@@ -112,6 +118,7 @@ enum mlx5dr_definer_fname {
 enum mlx5dr_definer_type {
 	MLX5DR_DEFINER_TYPE_MATCH,
 	MLX5DR_DEFINER_TYPE_JUMBO,
+	MLX5DR_DEFINER_TYPE_RANGE,
 };
 
 struct mlx5dr_definer_fc {
@@ -573,6 +580,11 @@ void mlx5dr_definer_create_tag(const struct rte_flow_item *items,
 			       uint32_t fc_sz,
 			       uint8_t *tag);
 
+void mlx5dr_definer_create_tag_range(const struct rte_flow_item *items,
+				     struct mlx5dr_definer_fc *fc,
+				     uint32_t fc_sz,
+				     uint8_t *tag);
+
 int mlx5dr_definer_get_id(struct mlx5dr_definer *definer);
 
 int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
drivers/net/mlx5/hws/mlx5dr_matcher.c +26 -1
index 0158a60ac3..5508cfe230 100644
@@ -360,6 +360,12 @@ static bool mlx5dr_matcher_supp_fw_wqe(struct mlx5dr_matcher *matcher)
 		return false;
 	}
 
+	if ((matcher->flags & MLX5DR_MATCHER_FLAGS_RANGE_DEFINER) &&
+	    !IS_BIT_SET(caps->supp_ste_format_gen_wqe, MLX5_IFC_RTC_STE_FORMAT_RANGE)) {
+		DR_LOG(INFO, "Extended match gen wqe RANGE format not supported");
+		return false;
+	}
+
 	if (!(caps->supp_type_gen_wqe & MLX5_GENERATE_WQE_TYPE_FLOW_UPDATE)) {
 		DR_LOG(ERR, "Gen WQE command not supporting GTA");
 		return false;
@@ -460,14 +466,20 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 		ste = &matcher->match_ste.ste;
 		ste->order = attr->table.sz_col_log + attr->table.sz_row_log;
 
+		/* Add additional rows due to additional range STE */
+		if (mlx5dr_matcher_mt_is_range(mt))
+			ste->order++;
+
 		rtc_attr.log_size = attr->table.sz_row_log;
 		rtc_attr.log_depth = attr->table.sz_col_log;
 		rtc_attr.is_frst_jumbo = mlx5dr_matcher_mt_is_jumbo(mt);
+		rtc_attr.is_scnd_range = mlx5dr_matcher_mt_is_range(mt);
 		rtc_attr.miss_ft_id = matcher->end_ft->id;
 
 		if (attr->insert_mode == MLX5DR_MATCHER_INSERT_BY_HASH) {
 			/* The usual Hash Table */
 			rtc_attr.update_index_mode = MLX5_IFC_RTC_STE_UPDATE_MODE_BY_HASH;
+
 			if (matcher->hash_definer) {
 				/* Specify definer_id_0 is used for hashing */
 				rtc_attr.fw_gen_wqe = true;
@@ -477,6 +489,16 @@ static int mlx5dr_matcher_create_rtc(struct mlx5dr_matcher *matcher,
 			} else {
 				/* The first mt is used since all share the same definer */
 				rtc_attr.match_definer_0 = mlx5dr_definer_get_id(mt->definer);
+
+				/* This is tricky, instead of passing two definers for
+				 * match and range, we specify that this RTC uses a hash
+				 * definer, this will allow us to use any range definer
+				 * since only first STE is used for hashing anyways.
+				 */
+				if (matcher->flags & MLX5DR_MATCHER_FLAGS_RANGE_DEFINER) {
+					rtc_attr.fw_gen_wqe = true;
+					rtc_attr.num_hash_definer = 1;
+				}
 			}
 		} else if (attr->insert_mode == MLX5DR_MATCHER_INSERT_BY_INDEX) {
 			rtc_attr.update_index_mode = MLX5_IFC_RTC_STE_UPDATE_MODE_BY_OFFSET;
@@ -751,7 +773,7 @@ static int mlx5dr_matcher_bind_mt(struct mlx5dr_matcher *matcher)
 	struct mlx5dr_pool_attr pool_attr = {0};
 	int ret;
 
-	/* Calculate match and hash definers */
+	/* Calculate match, range and hash definers */
 	ret = mlx5dr_definer_matcher_init(ctx, matcher);
 	if (ret) {
 		DR_LOG(ERR, "Failed to set matcher templates with match definers");
@@ -772,6 +794,9 @@ static int mlx5dr_matcher_bind_mt(struct mlx5dr_matcher *matcher)
 	pool_attr.flags = MLX5DR_POOL_FLAGS_FOR_MATCHER_STE_POOL;
 	pool_attr.alloc_log_sz = matcher->attr.table.sz_col_log +
 				 matcher->attr.table.sz_row_log;
+	/* Add additional rows due to additional range STE */
+	if (matcher->flags & MLX5DR_MATCHER_FLAGS_RANGE_DEFINER)
+		pool_attr.alloc_log_sz++;
 	mlx5dr_matcher_set_pool_attr(&pool_attr, matcher);
 
 	matcher->match_ste.pool = mlx5dr_pool_create(ctx, &pool_attr);
drivers/net/mlx5/hws/mlx5dr_matcher.h +14 -3
index 8dadc0ee66..4759068ab4 100644
@@ -23,8 +23,9 @@
 #define MLX5DR_MATCHER_ASSURED_MAIN_TBL_DEPTH 2
 
 enum mlx5dr_matcher_flags {
-	MLX5DR_MATCHER_FLAGS_HASH_DEFINER	= 1 << 0,
-	MLX5DR_MATCHER_FLAGS_COLLISION		= 1 << 1,
+	MLX5DR_MATCHER_FLAGS_RANGE_DEFINER	= 1 << 0,
+	MLX5DR_MATCHER_FLAGS_HASH_DEFINER	= 1 << 1,
+	MLX5DR_MATCHER_FLAGS_COLLISION		= 1 << 2,
 };
 
 struct mlx5dr_match_template {
@@ -32,7 +33,9 @@ struct mlx5dr_match_template {
 	struct mlx5dr_definer *definer;
 	struct mlx5dr_definer *range_definer;
 	struct mlx5dr_definer_fc *fc;
+	struct mlx5dr_definer_fc *fcr;
 	uint16_t fc_sz;
+	uint16_t fcr_sz;
 	uint64_t item_flags;
 	uint8_t vport_item_id;
 	enum mlx5dr_match_template_flags flags;
@@ -80,10 +83,18 @@ mlx5dr_matcher_mt_is_jumbo(struct mlx5dr_match_template *mt)
 	return mlx5dr_definer_is_jumbo(mt->definer);
 }
 
+static inline bool
+mlx5dr_matcher_mt_is_range(struct mlx5dr_match_template *mt)
+{
+	return (!!mt->range_definer);
+}
+
 static inline bool mlx5dr_matcher_req_fw_wqe(struct mlx5dr_matcher *matcher)
 {
 	/* Currently HWS doesn't support hash different from match or range */
-	return unlikely(matcher->flags & MLX5DR_MATCHER_FLAGS_HASH_DEFINER);
+	return unlikely(matcher->flags &
+			(MLX5DR_MATCHER_FLAGS_HASH_DEFINER |
+			 MLX5DR_MATCHER_FLAGS_RANGE_DEFINER));
 }
 
 int mlx5dr_matcher_conv_items_to_prm(uint64_t *match_buf,
Signature · 1 line
2.18.1
#30 lore raw

PATCH v2 13/16 net/mlx5/hws: add FW WQE rule creation logic

In reply to Alex Vesker

FW WQE and HW WQE are done in a similar way but not to jeopardize the performance rule creation is done over the new FW rule creation function. The deletion function is shared between both flows.

---
 drivers/net/mlx5/hws/mlx5dr_rule.c | 180 +++++++++++++++++++++++++++--
 drivers/net/mlx5/hws/mlx5dr_rule.h |   2 +
 drivers/net/mlx5/hws/mlx5dr_send.h |   9 +-
 3 files changed, 180 insertions(+), 11 deletions(-)
drivers/net/mlx5/hws/mlx5dr_rule.c +171 -9
index f5a0c46315..9d5e5b11a5 100644
@@ -112,6 +112,62 @@ static void mlx5dr_rule_gen_comp(struct mlx5dr_send_engine *queue,
 	mlx5dr_send_engine_gen_comp(queue, user_data, comp_status);
 }
 
+static void
+mlx5dr_rule_save_delete_info(struct mlx5dr_rule *rule,
+			     struct mlx5dr_send_ste_attr *ste_attr)
+{
+	if (unlikely(mlx5dr_matcher_req_fw_wqe(rule->matcher))) {
+		uint8_t *src_tag;
+
+		/* Save match definer id and tag for delete */
+		rule->tag_ptr = simple_calloc(2, sizeof(*rule->tag_ptr));
+		assert(rule->tag_ptr);
+
+		src_tag = (uint8_t *)ste_attr->wqe_data->tag;
+		memcpy(rule->tag_ptr[0].match, src_tag, MLX5DR_MATCH_TAG_SZ);
+		rule->tag_ptr[1].reserved[0] = ste_attr->send_attr.match_definer_id;
+
+		/* Save range definer id and tag for delete */
+		if (ste_attr->range_wqe_data) {
+			src_tag = (uint8_t *)ste_attr->range_wqe_data->tag;
+			memcpy(rule->tag_ptr[1].match, src_tag, MLX5DR_MATCH_TAG_SZ);
+			rule->tag_ptr[1].reserved[1] = ste_attr->send_attr.range_definer_id;
+		}
+		return;
+	}
+
+	if (ste_attr->wqe_tag_is_jumbo)
+		memcpy(rule->tag.jumbo, ste_attr->wqe_data->jumbo, MLX5DR_JUMBO_TAG_SZ);
+	else
+		memcpy(rule->tag.match, ste_attr->wqe_data->tag, MLX5DR_MATCH_TAG_SZ);
+}
+
+static void
+mlx5dr_rule_clear_delete_info(struct mlx5dr_rule *rule)
+{
+	if (unlikely(mlx5dr_matcher_req_fw_wqe(rule->matcher)))
+		simple_free(rule->tag_ptr);
+}
+
+static void
+mlx5dr_rule_load_delete_info(struct mlx5dr_rule *rule,
+			     struct mlx5dr_send_ste_attr *ste_attr)
+{
+	if (unlikely(mlx5dr_matcher_req_fw_wqe(rule->matcher))) {
+		/* Load match definer id and tag for delete */
+		ste_attr->wqe_tag = &rule->tag_ptr[0];
+		ste_attr->send_attr.match_definer_id = rule->tag_ptr[1].reserved[0];
+
+		/* Load range definer id and tag for delete */
+		if (rule->matcher->flags & MLX5DR_MATCHER_FLAGS_RANGE_DEFINER) {
+			ste_attr->range_wqe_tag = &rule->tag_ptr[1];
+			ste_attr->send_attr.range_definer_id = rule->tag_ptr[1].reserved[1];
+		}
+	} else {
+		ste_attr->wqe_tag = &rule->tag;
+	}
+}
+
 static int mlx5dr_rule_alloc_action_ste(struct mlx5dr_rule *rule,
 					struct mlx5dr_rule_attr *attr)
 {
@@ -180,6 +236,97 @@ static void mlx5dr_rule_create_init(struct mlx5dr_rule *rule,
 	apply->require_dep = 0;
 }
 
+static int mlx5dr_rule_create_hws_fw_wqe(struct mlx5dr_rule *rule,
+					 struct mlx5dr_rule_attr *attr,
+					 uint8_t mt_idx,
+					 const struct rte_flow_item items[],
+					 uint8_t at_idx,
+					 struct mlx5dr_rule_action rule_actions[])
+{
+	struct mlx5dr_action_template *at = &rule->matcher->at[at_idx];
+	struct mlx5dr_match_template *mt = &rule->matcher->mt[mt_idx];
+	struct mlx5dr_send_ring_dep_wqe range_wqe = {{0}};
+	struct mlx5dr_send_ring_dep_wqe match_wqe = {{0}};
+	bool is_range = mlx5dr_matcher_mt_is_range(mt);
+	bool is_jumbo = mlx5dr_matcher_mt_is_jumbo(mt);
+	struct mlx5dr_matcher *matcher = rule->matcher;
+	struct mlx5dr_context *ctx = matcher->tbl->ctx;
+	struct mlx5dr_send_ste_attr ste_attr = {0};
+	struct mlx5dr_actions_apply_data apply;
+	struct mlx5dr_send_engine *queue;
+
+	queue = &ctx->send_queue[attr->queue_id];
+	if (unlikely(mlx5dr_send_engine_err(queue))) {
+		rte_errno = EIO;
+		return rte_errno;
+	}
+
+	mlx5dr_rule_create_init(rule, &ste_attr, &apply);
+	mlx5dr_rule_init_dep_wqe(&match_wqe, rule, items, mt, attr->user_data);
+	mlx5dr_rule_init_dep_wqe(&range_wqe, rule, items, mt, attr->user_data);
+
+	ste_attr.direct_index = 0;
+	ste_attr.rtc_0 = match_wqe.rtc_0;
+	ste_attr.rtc_1 = match_wqe.rtc_1;
+	ste_attr.used_id_rtc_0 = &rule->rtc_0;
+	ste_attr.used_id_rtc_1 = &rule->rtc_1;
+	ste_attr.retry_rtc_0 = match_wqe.retry_rtc_0;
+	ste_attr.retry_rtc_1 = match_wqe.retry_rtc_1;
+	ste_attr.send_attr.rule = match_wqe.rule;
+	ste_attr.send_attr.user_data = match_wqe.user_data;
+
+	ste_attr.send_attr.fence = 1;
+	ste_attr.send_attr.notify_hw = 1;
+	ste_attr.wqe_tag_is_jumbo = is_jumbo;
+
+	/* Prepare match STE TAG */
+	ste_attr.wqe_ctrl = &match_wqe.wqe_ctrl;
+	ste_attr.wqe_data = &match_wqe.wqe_data;
+	ste_attr.send_attr.match_definer_id = mlx5dr_definer_get_id(mt->definer);
+
+	mlx5dr_definer_create_tag(items,
+				  mt->fc,
+				  mt->fc_sz,
+				  (uint8_t *)match_wqe.wqe_data.action);
+
+	/* Prepare range STE TAG */
+	if (is_range) {
+		ste_attr.range_wqe_data = &range_wqe.wqe_data;
+		ste_attr.send_attr.len += MLX5DR_WQE_SZ_GTA_DATA;
+		ste_attr.send_attr.range_definer_id = mlx5dr_definer_get_id(mt->range_definer);
+
+		mlx5dr_definer_create_tag_range(items,
+						mt->fcr,
+						mt->fcr_sz,
+						(uint8_t *)range_wqe.wqe_data.action);
+	}
+
+	/* Apply the actions on the last STE */
+	apply.queue = queue;
+	apply.next_direct_idx = 0;
+	apply.rule_action = rule_actions;
+	apply.wqe_ctrl = &match_wqe.wqe_ctrl;
+	apply.wqe_data = (uint32_t *)(is_range ?
+				      &range_wqe.wqe_data :
+				      &match_wqe.wqe_data);
+
+	/* Skip setters[0] used for jumbo STE since not support with FW WQE */
+	mlx5dr_action_apply_setter(&apply, &at->setters[1], 0);
+
+	/* Send WQEs to FW */
+	mlx5dr_send_stes_fw(queue, &ste_attr);
+
+	/* Backup TAG on the rule for deletion */
+	mlx5dr_rule_save_delete_info(rule, &ste_attr);
+	mlx5dr_send_engine_inc_rule(queue);
+
+	/* Send dependent WQEs */
+	if (!attr->burst)
+		mlx5dr_send_all_dep_wqe(queue);
+
+	return 0;
+}
+
 static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 				  struct mlx5dr_rule_attr *attr,
 				  uint8_t mt_idx,
@@ -189,7 +336,7 @@ static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 {
 	struct mlx5dr_action_template *at = &rule->matcher->at[at_idx];
 	struct mlx5dr_match_template *mt = &rule->matcher->mt[mt_idx];
-	bool is_jumbo = mlx5dr_definer_is_jumbo(mt->definer);
+	bool is_jumbo = mlx5dr_matcher_mt_is_jumbo(mt);
 	struct mlx5dr_matcher *matcher = rule->matcher;
 	struct mlx5dr_context *ctx = matcher->tbl->ctx;
 	struct mlx5dr_send_ste_attr ste_attr = {0};
@@ -200,6 +347,11 @@ static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 	uint8_t total_stes, action_stes;
 	int i, ret;
 
+	/* Insert rule using FW WQE if cannot use GTA WQE */
+	if (unlikely(mlx5dr_matcher_req_fw_wqe(matcher)))
+		return mlx5dr_rule_create_hws_fw_wqe(rule, attr, mt_idx, items,
+						     at_idx, rule_actions);
+
 	queue = &ctx->send_queue[attr->queue_id];
 	if (unlikely(mlx5dr_send_engine_err(queue))) {
 		rte_errno = EIO;
@@ -283,11 +435,7 @@ static int mlx5dr_rule_create_hws(struct mlx5dr_rule *rule,
 	}
 
 	/* Backup TAG on the rule for deletion */
-	if (is_jumbo)
-		memcpy(rule->tag.jumbo, dep_wqe->wqe_data.action, MLX5DR_JUMBO_TAG_SZ);
-	else
-		memcpy(rule->tag.match, dep_wqe->wqe_data.tag, MLX5DR_MATCH_TAG_SZ);
-
+	mlx5dr_rule_save_delete_info(rule, &ste_attr);
 	mlx5dr_send_engine_inc_rule(queue);
 
 	/* Send dependent WQEs */
@@ -311,6 +459,9 @@ static void mlx5dr_rule_destroy_failed_hws(struct mlx5dr_rule *rule,
 	/* Rule failed now we can safely release action STEs */
 	mlx5dr_rule_free_action_ste_idx(rule);
 
+	/* Clear complex tag */
+	mlx5dr_rule_clear_delete_info(rule);
+
 	/* If a rule that was indicated as burst (need to trigger HW) has failed
 	 * insertion we won't ring the HW as nothing is being written to the WQ.
 	 * In such case update the last WQE and ring the HW with that work
@@ -327,6 +478,9 @@ static int mlx5dr_rule_destroy_hws(struct mlx5dr_rule *rule,
 {
 	struct mlx5dr_context *ctx = rule->matcher->tbl->ctx;
 	struct mlx5dr_matcher *matcher = rule->matcher;
+	bool fw_wqe = mlx5dr_matcher_req_fw_wqe(matcher);
+	bool is_range = mlx5dr_matcher_mt_is_range(matcher->mt);
+	bool is_jumbo = mlx5dr_matcher_mt_is_jumbo(matcher->mt);
 	struct mlx5dr_wqe_gta_ctrl_seg wqe_ctrl = {0};
 	struct mlx5dr_send_ste_attr ste_attr = {0};
 	struct mlx5dr_send_engine *queue;
@@ -361,6 +515,8 @@ static int mlx5dr_rule_destroy_hws(struct mlx5dr_rule *rule,
 	ste_attr.send_attr.opmod = MLX5DR_WQE_GTA_OPMOD_STE;
 	ste_attr.send_attr.opcode = MLX5DR_WQE_OPCODE_TBL_ACCESS;
 	ste_attr.send_attr.len = MLX5DR_WQE_SZ_GTA_CTRL + MLX5DR_WQE_SZ_GTA_DATA;
+	if (unlikely(is_range))
+		ste_attr.send_attr.len += MLX5DR_WQE_SZ_GTA_DATA;
 
 	ste_attr.send_attr.rule = rule;
 	ste_attr.send_attr.notify_hw = !attr->burst;
@@ -371,13 +527,19 @@ static int mlx5dr_rule_destroy_hws(struct mlx5dr_rule *rule,
 	ste_attr.used_id_rtc_0 = &rule->rtc_0;
 	ste_attr.used_id_rtc_1 = &rule->rtc_1;
 	ste_attr.wqe_ctrl = &wqe_ctrl;
-	ste_attr.wqe_tag = &rule->tag;
-	ste_attr.wqe_tag_is_jumbo = mlx5dr_definer_is_jumbo(matcher->mt->definer);
+	ste_attr.wqe_tag_is_jumbo = is_jumbo;
 	ste_attr.gta_opcode = MLX5DR_WQE_GTA_OP_DEACTIVATE;
 	if (unlikely(mlx5dr_matcher_is_insert_by_idx(matcher)))
 		ste_attr.direct_index = attr->rule_idx;
 
-	mlx5dr_send_ste(queue, &ste_attr);
+	mlx5dr_rule_load_delete_info(rule, &ste_attr);
+
+	if (unlikely(fw_wqe)) {
+		mlx5dr_send_stes_fw(queue, &ste_attr);
+		mlx5dr_rule_clear_delete_info(rule);
+	} else {
+		mlx5dr_send_ste(queue, &ste_attr);
+	}
 
 	return 0;
 }
drivers/net/mlx5/hws/mlx5dr_rule.h +2 -0
index f2fe418159..886cf77992 100644
@@ -36,6 +36,8 @@ struct mlx5dr_rule {
 	struct mlx5dr_matcher *matcher;
 	union {
 		struct mlx5dr_rule_match_tag tag;
+		/* Pointer to tag to store more than one tag */
+		struct mlx5dr_rule_match_tag *tag_ptr;
 		struct ibv_flow *flow;
 	};
 	uint32_t rtc_0; /* The RTC into which the STE was inserted */
drivers/net/mlx5/hws/mlx5dr_send.h +7 -2
index 47bb66b3c7..d0977ec851 100644
@@ -54,8 +54,13 @@ struct mlx5dr_wqe_gta_data_seg_ste {
 	__be32 rsvd0_ctr_id;
 	__be32 rsvd1_definer;
 	__be32 rsvd2[3];
-	__be32 action[3];
-	__be32 tag[8];
+	union {
+		struct {
+			__be32 action[3];
+			__be32 tag[8];
+		};
+		__be32 jumbo[11];
+	};
 };
 
 struct mlx5dr_wqe_gta_data_seg_arg {
Signature · 1 line
2.18.1
#31 lore raw

PATCH v2 14/16 net/mlx5/hws: add debug dump support for range and hash

In reply to Alex Vesker

Add support for dumping range and hash definers objects. Hash definer is a per matcher object describing the fields used for hashing. Range definer is per match template object describing the fields used for range matching. Both are optional based on the given match templates.

---
 drivers/net/mlx5/hws/mlx5dr_debug.c | 37 ++++++++++++++++++++---------
 drivers/net/mlx5/hws/mlx5dr_debug.h |  4 +++-
 2 files changed, 29 insertions(+), 12 deletions(-)
drivers/net/mlx5/hws/mlx5dr_debug.c +26 -11
index 9199ec16e0..dfef785d4d 100644
@@ -34,15 +34,19 @@ const char *mlx5dr_debug_action_type_to_str(enum mlx5dr_action_type action_type)
 
 static int
 mlx5dr_debug_dump_matcher_template_definer(FILE *f,
-					   struct mlx5dr_match_template *mt)
+					   void *parent_obj,
+					   struct mlx5dr_definer *definer,
+					   enum mlx5dr_debug_res_type type)
 {
-	struct mlx5dr_definer *definer = mt->definer;
 	int i, ret;
 
+	if (!definer)
+		return 0;
+
 	ret = fprintf(f, "%d,0x%" PRIx64 ",0x%" PRIx64 ",%d,%d,",
-		      MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_DEFINER,
+		      type,
 		      (uint64_t)(uintptr_t)definer,
-		      (uint64_t)(uintptr_t)mt,
+		      (uint64_t)(uintptr_t)parent_obj,
 		      definer->obj->id,
 		      definer->type);
 	if (ret < 0) {
@@ -89,29 +93,40 @@ static int
 mlx5dr_debug_dump_matcher_match_template(FILE *f, struct mlx5dr_matcher *matcher)
 {
 	bool is_root = matcher->tbl->level == MLX5DR_ROOT_LEVEL;
+	enum mlx5dr_debug_res_type type;
 	int i, ret;
 
 	for (i = 0; i < matcher->num_of_mt; i++) {
 		struct mlx5dr_match_template *mt = &matcher->mt[i];
 
-		ret = fprintf(f, "%d,0x%" PRIx64 ",0x%" PRIx64 ",%d,%d\n",
+		ret = fprintf(f, "%d,0x%" PRIx64 ",0x%" PRIx64 ",%d,%d,%d\n",
 			      MLX5DR_DEBUG_RES_TYPE_MATCHER_MATCH_TEMPLATE,
 			      (uint64_t)(uintptr_t)mt,
 			      (uint64_t)(uintptr_t)matcher,
 			      is_root ? 0 : mt->fc_sz,
-			      mt->flags);
+			      mt->flags,
+			      is_root ? 0 : mt->fcr_sz);
 		if (ret < 0) {
 			rte_errno = EINVAL;
 			return rte_errno;
 		}
 
-		if (!is_root) {
-			ret = mlx5dr_debug_dump_matcher_template_definer(f, mt);
-			if (ret)
-				return ret;
-		}
+		type = MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_MATCH_DEFINER;
+		ret = mlx5dr_debug_dump_matcher_template_definer(f, mt, mt->definer, type);
+		if (ret)
+			return ret;
+
+		type = MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_RANGE_DEFINER;
+		ret = mlx5dr_debug_dump_matcher_template_definer(f, mt, mt->range_definer, type);
+		if (ret)
+			return ret;
 	}
 
+	type = MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_HASH_DEFINER;
+	ret = mlx5dr_debug_dump_matcher_template_definer(f, matcher, matcher->hash_definer, type);
+	if (ret)
+		return ret;
+
 	return 0;
 }
 
drivers/net/mlx5/hws/mlx5dr_debug.h +3 -1
index cf00170f7d..2c29ca295c 100644
@@ -19,8 +19,10 @@ enum mlx5dr_debug_res_type {
 	MLX5DR_DEBUG_RES_TYPE_MATCHER = 4200,
 	MLX5DR_DEBUG_RES_TYPE_MATCHER_ATTR = 4201,
 	MLX5DR_DEBUG_RES_TYPE_MATCHER_MATCH_TEMPLATE = 4202,
+	MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_MATCH_DEFINER = 4203,
 	MLX5DR_DEBUG_RES_TYPE_MATCHER_ACTION_TEMPLATE = 4204,
-	MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_DEFINER = 4203,
+	MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_HASH_DEFINER = 4205,
+	MLX5DR_DEBUG_RES_TYPE_MATCHER_TEMPLATE_RANGE_DEFINER = 4206,
 };
 
 const char *mlx5dr_debug_action_type_to_str(enum mlx5dr_action_type action_type);
Signature · 1 line
2.18.1
#32 lore raw

PATCH v2 15/16 net/mlx5/hws: rename pattern cache object

In reply to Alex Vesker

To have the same name convention for future caches, use cache and cache item naming.

---
 drivers/net/mlx5/hws/mlx5dr_pat_arg.c | 22 +++++++++++-----------
 drivers/net/mlx5/hws/mlx5dr_pat_arg.h |  6 +++---
 2 files changed, 14 insertions(+), 14 deletions(-)
drivers/net/mlx5/hws/mlx5dr_pat_arg.c +11 -11
index 152025d302..6ed04dac6d 100644
@@ -94,13 +94,13 @@ static bool mlx5dr_pat_compare_pattern(enum mlx5dr_action_type cur_type,
 	return true;
 }
 
-static struct mlx5dr_pat_cached_pattern *
+static struct mlx5dr_pattern_cache_item *
 mlx5dr_pat_find_cached_pattern(struct mlx5dr_pattern_cache *cache,
 			       struct mlx5dr_action *action,
 			       uint16_t num_of_actions,
 			       __be64 *actions)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pat;
+	struct mlx5dr_pattern_cache_item *cached_pat;
 
 	LIST_FOREACH(cached_pat, &cache->head, next) {
 		if (mlx5dr_pat_compare_pattern(cached_pat->type,
@@ -115,13 +115,13 @@ mlx5dr_pat_find_cached_pattern(struct mlx5dr_pattern_cache *cache,
 	return NULL;
 }
 
-static struct mlx5dr_pat_cached_pattern *
+static struct mlx5dr_pattern_cache_item *
 mlx5dr_pat_get_existing_cached_pattern(struct mlx5dr_pattern_cache *cache,
 				       struct mlx5dr_action *action,
 				       uint16_t num_of_actions,
 				       __be64 *actions)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pattern;
+	struct mlx5dr_pattern_cache_item *cached_pattern;
 
 	cached_pattern = mlx5dr_pat_find_cached_pattern(cache, action, num_of_actions, actions);
 	if (cached_pattern) {
@@ -134,11 +134,11 @@ mlx5dr_pat_get_existing_cached_pattern(struct mlx5dr_pattern_cache *cache,
 	return cached_pattern;
 }
 
-static struct mlx5dr_pat_cached_pattern *
+static struct mlx5dr_pattern_cache_item *
 mlx5dr_pat_get_cached_pattern_by_action(struct mlx5dr_pattern_cache *cache,
 					struct mlx5dr_action *action)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pattern;
+	struct mlx5dr_pattern_cache_item *cached_pattern;
 
 	LIST_FOREACH(cached_pattern, &cache->head, next) {
 		if (cached_pattern->mh_data.pattern_obj->id == action->modify_header.pattern_obj->id)
@@ -148,14 +148,14 @@ mlx5dr_pat_get_cached_pattern_by_action(struct mlx5dr_pattern_cache *cache,
 	return NULL;
 }
 
-static struct mlx5dr_pat_cached_pattern *
+static struct mlx5dr_pattern_cache_item *
 mlx5dr_pat_add_pattern_to_cache(struct mlx5dr_pattern_cache *cache,
 				struct mlx5dr_devx_obj *pattern_obj,
 				enum mlx5dr_action_type type,
 				uint16_t num_of_actions,
 				__be64 *actions)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pattern;
+	struct mlx5dr_pattern_cache_item *cached_pattern;
 
 	cached_pattern = simple_calloc(1, sizeof(*cached_pattern));
 	if (!cached_pattern) {
@@ -189,7 +189,7 @@ mlx5dr_pat_add_pattern_to_cache(struct mlx5dr_pattern_cache *cache,
 }
 
 static void
-mlx5dr_pat_remove_pattern(struct mlx5dr_pat_cached_pattern *cached_pattern)
+mlx5dr_pat_remove_pattern(struct mlx5dr_pattern_cache_item *cached_pattern)
 {
 	LIST_REMOVE(cached_pattern, next);
 	simple_free(cached_pattern->mh_data.data);
@@ -200,7 +200,7 @@ static void
 mlx5dr_pat_put_pattern(struct mlx5dr_pattern_cache *cache,
 		       struct mlx5dr_action *action)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pattern;
+	struct mlx5dr_pattern_cache_item *cached_pattern;
 
 	pthread_spin_lock(&cache->lock);
 	cached_pattern = mlx5dr_pat_get_cached_pattern_by_action(cache, action);
@@ -225,7 +225,7 @@ static int mlx5dr_pat_get_pattern(struct mlx5dr_context *ctx,
 				  size_t pattern_sz,
 				  __be64 *pattern)
 {
-	struct mlx5dr_pat_cached_pattern *cached_pattern;
+	struct mlx5dr_pattern_cache_item *cached_pattern;
 	int ret = 0;
 
 	pthread_spin_lock(&ctx->pattern_cache->lock);
drivers/net/mlx5/hws/mlx5dr_pat_arg.h +3 -3
index d9353e9a3e..92db6d6aee 100644
@@ -24,10 +24,10 @@ enum {
 struct mlx5dr_pattern_cache {
 	/* Protect pattern list */
 	pthread_spinlock_t lock;
-	LIST_HEAD(pattern_head, mlx5dr_pat_cached_pattern) head;
+	LIST_HEAD(pattern_head, mlx5dr_pattern_cache_item) head;
 };
 
-struct mlx5dr_pat_cached_pattern {
+struct mlx5dr_pattern_cache_item {
 	enum mlx5dr_action_type type;
 	struct {
 		struct mlx5dr_devx_obj *pattern_obj;
@@ -36,7 +36,7 @@ struct mlx5dr_pat_cached_pattern {
 		uint16_t num_of_actions;
 	} mh_data;
 	uint32_t refcount;
-	LIST_ENTRY(mlx5dr_pat_cached_pattern) next;
+	LIST_ENTRY(mlx5dr_pattern_cache_item) next;
 };
 
 enum mlx5dr_arg_chunk_size
Signature · 1 line
2.18.1
#33 lore raw

PATCH v2 16/16 net/mlx5/hws: cache definer for reuse

In reply to Alex Vesker

Definers are a limited resource in the system per GVMI, to avoid failure we try to improve bt checking if it is possible to reuse the definers in some cases. Added a cache on the context for this purpose.

---
 drivers/net/mlx5/hws/mlx5dr_context.c |  12 ++-
 drivers/net/mlx5/hws/mlx5dr_context.h |   1 +
 drivers/net/mlx5/hws/mlx5dr_definer.c | 122 ++++++++++++++++++++++----
 drivers/net/mlx5/hws/mlx5dr_definer.h |  14 +++
 4 files changed, 130 insertions(+), 19 deletions(-)
drivers/net/mlx5/hws/mlx5dr_context.c +9 -3
index 6627337d9e..08a5ee92a5 100644
@@ -13,6 +13,9 @@ static int mlx5dr_context_pools_init(struct mlx5dr_context *ctx)
 	if (mlx5dr_pat_init_pattern_cache(&ctx->pattern_cache))
 		return rte_errno;
 
+	if (mlx5dr_definer_init_cache(&ctx->definer_cache))
+		goto uninit_pat_cache;
+
 	/* Create an STC pool per FT type */
 	pool_attr.pool_type = MLX5DR_POOL_TYPE_STC;
 	pool_attr.flags = MLX5DR_POOL_FLAGS_FOR_STC_POOL;
@@ -35,8 +38,10 @@ static int mlx5dr_context_pools_init(struct mlx5dr_context *ctx)
 		if (ctx->stc_pool[i])
 			mlx5dr_pool_destroy(ctx->stc_pool[i]);
 
-	mlx5dr_pat_uninit_pattern_cache(ctx->pattern_cache);
+	mlx5dr_definer_uninit_cache(ctx->definer_cache);
 
+uninit_pat_cache:
+	mlx5dr_pat_uninit_pattern_cache(ctx->pattern_cache);
 	return rte_errno;
 }
 
@@ -44,12 +49,13 @@ static void mlx5dr_context_pools_uninit(struct mlx5dr_context *ctx)
 {
 	int i;
 
-	mlx5dr_pat_uninit_pattern_cache(ctx->pattern_cache);
-
 	for (i = 0; i < MLX5DR_TABLE_TYPE_MAX; i++) {
 		if (ctx->stc_pool[i])
 			mlx5dr_pool_destroy(ctx->stc_pool[i]);
 	}
+
+	mlx5dr_definer_uninit_cache(ctx->definer_cache);
+	mlx5dr_pat_uninit_pattern_cache(ctx->pattern_cache);
 }
 
 static int mlx5dr_context_init_pd(struct mlx5dr_context *ctx,
drivers/net/mlx5/hws/mlx5dr_context.h +1 -0
index a38d9484b3..0ba8d0c92e 100644
@@ -39,6 +39,7 @@ struct mlx5dr_context {
 	struct mlx5dr_context_common_res common_res[MLX5DR_TABLE_TYPE_MAX];
 	struct mlx5dr_context_shared_gvmi_res gvmi_res[MLX5DR_TABLE_TYPE_MAX];
 	struct mlx5dr_pattern_cache *pattern_cache;
+	struct mlx5dr_definer_cache *definer_cache;
 	pthread_spinlock_t ctrl_lock;
 	enum mlx5dr_context_flags flags;
 	struct mlx5dr_send_engine *send_queue;
drivers/net/mlx5/hws/mlx5dr_definer.c +106 -16

197 lines not on this page · raw message

drivers/net/mlx5/hws/mlx5dr_definer.h +14 -0
index dd9a297007..464872acd6 100644
@@ -569,6 +569,16 @@ struct mlx5dr_definer {
 	struct mlx5dr_devx_obj *obj;
 };
 
+struct mlx5dr_definer_cache {
+	LIST_HEAD(definer_head, mlx5dr_definer_cache_item) head;
+};
+
+struct mlx5dr_definer_cache_item {
+	struct mlx5dr_definer definer;
+	uint32_t refcount;
+	LIST_ENTRY(mlx5dr_definer_cache_item) next;
+};
+
 static inline bool
 mlx5dr_definer_is_jumbo(struct mlx5dr_definer *definer)
 {
@@ -592,4 +602,8 @@ int mlx5dr_definer_matcher_init(struct mlx5dr_context *ctx,
 
 void mlx5dr_definer_matcher_uninit(struct mlx5dr_matcher *matcher);
 
+int mlx5dr_definer_init_cache(struct mlx5dr_definer_cache **cache);
+
+void mlx5dr_definer_uninit_cache(struct mlx5dr_definer_cache *cache);
+
 #endif
Signature · 1 line
2.18.1
#34 lore raw
In reply to Alex Vesker

From: Alex Vesker

Quoted 26 lines

ConnectX and BlueField devices which support HWS are capable of performing advanced match, range-match and hashing operations on packet headers. This patch-set introduces support for (1) range matching - allows combining range and exact match over provided item fields using spec, last and mask.
A useful example for Range and exact match is matching exact IP and port range. This series also introduces (2) partial hash, which allows using the same matcher and performing partial hashing in case of multiple match templates are provided. The matcher create code finds the union and intersection fields to split the hash and match in a way the HW can support. This increases the match templates combinations possible although it may limit the maximum PPS. Since (1) and (2) are not fully support on current HW the implementation is done on top of a HW like FW API using WQEs.

v2:
-Fix missed checkpatch comments

Alex Vesker (16):
net/mlx5/hws: support synchronous drain net/mlx5/hws: matcher remove AT and MT limitation net/mlx5/hws: support GTA WQE write using FW command net/mlx5/hws: add capability query for gen wqe command net/mlx5/hws: align RTC create command with PRM format net/mlx5/hws: add send FW match STE using gen WQE net/mlx5/hws: add send FW range STE WQE net/mlx5/hws: move matcher size check to function

Quoted 8 lines

net/mlx5/hws: support range match net/mlx5/hws: redesign definer create net/mlx5/hws: support partial hash net/mlx5/hws: add range definer creation support net/mlx5/hws: add FW WQE rule creation logic net/mlx5/hws: add debug dump support for range and hash net/mlx5/hws: rename pattern cache object net/mlx5/hws: cache definer for reuse

Quoted 24 lines
 drivers/common/mlx5/mlx5_prm.h         |  53 +-
 drivers/net/mlx5/hws/mlx5dr.h          |   6 +-
 drivers/net/mlx5/hws/mlx5dr_cmd.c      |  72 ++-
 drivers/net/mlx5/hws/mlx5dr_cmd.h      |  27 +-
 drivers/net/mlx5/hws/mlx5dr_context.c  |  12 +-
 drivers/net/mlx5/hws/mlx5dr_context.h  |   1 +
 drivers/net/mlx5/hws/mlx5dr_debug.c    |  41 +-
 drivers/net/mlx5/hws/mlx5dr_debug.h    |   4 +-
 drivers/net/mlx5/hws/mlx5dr_definer.c  | 750 +++++++++++++++++++++--
--  drivers/net/mlx5/hws/mlx5dr_definer.h  |  44 +-
 drivers/net/mlx5/hws/mlx5dr_internal.h |   2 +-
 drivers/net/mlx5/hws/mlx5dr_matcher.c  | 279 ++++++---
drivers/net/mlx5/hws/mlx5dr_matcher.h  |  43 +-
drivers/net/mlx5/hws/mlx5dr_pat_arg.c  |  49 +-
 drivers/net/mlx5/hws/mlx5dr_pat_arg.h  |   6 +-
 drivers/net/mlx5/hws/mlx5dr_rule.c     | 193 ++++++-
 drivers/net/mlx5/hws/mlx5dr_rule.h     |   2 +
 drivers/net/mlx5/hws/mlx5dr_send.c     | 169 +++++-
 drivers/net/mlx5/hws/mlx5dr_send.h     |  24 +-
 drivers/net/mlx5/mlx5_flow_hw.c        |   2 +-
 20 files changed, 1475 insertions(+), 304 deletions(-)

--
2.18.1

#35 lore raw
In reply to Alex Vesker

Hi,

Quoted 67 lines

-----Original Message-----
From: Alex Vesker <[email protected]>
Sent: Wednesday, February 1, 2023 9:28 AM
To: Alex Vesker <[email protected]>; Slava Ovsiienko <[email protected]>; NBU-Contact-Thomas Monjalon (EXTERNAL) <[email protected]>
Cc: Matan Azrad <[email protected]>; [email protected]; Ori Kam <[email protected]>
Subject: [v2 00/16] net/mlx5/hws: support range and partial hash matching

ConnectX and BlueField devices which support HWS are capable of performing advanced match, range-match and hashing operations on packet headers. This patch-set introduces support for (1) range matching - allows combining range and exact match over provided item fields using spec, last and mask.
A useful example for Range and exact match is matching exact IP and port range. This series also introduces (2) partial hash, which allows using the same matcher and performing partial hashing in case of multiple match templates are provided. The matcher create code finds the union and intersection fields to split the hash and match in a way the HW can support. This increases the match templates combinations possible although it may limit the maximum PPS. Since (1) and (2) are not fully support on current HW the implementation is done on top of a HW like FW API using WQEs.

v2:
-Fix missed checkpatch comments

Alex Vesker (16):
net/mlx5/hws: support synchronous drain net/mlx5/hws: matcher remove AT and MT limitation net/mlx5/hws: support GTA WQE write using FW command net/mlx5/hws: add capability query for gen wqe command net/mlx5/hws: align RTC create command with PRM format net/mlx5/hws: add send FW match STE using gen WQE net/mlx5/hws: add send FW range STE WQE net/mlx5/hws: move matcher size check to function net/mlx5/hws: support range match net/mlx5/hws: redesign definer create net/mlx5/hws: support partial hash net/mlx5/hws: add range definer creation support net/mlx5/hws: add FW WQE rule creation logic net/mlx5/hws: add debug dump support for range and hash net/mlx5/hws: rename pattern cache object net/mlx5/hws: cache definer for reuse

 drivers/common/mlx5/mlx5_prm.h         |  53 +-
 drivers/net/mlx5/hws/mlx5dr.h          |   6 +-
 drivers/net/mlx5/hws/mlx5dr_cmd.c      |  72 ++-
 drivers/net/mlx5/hws/mlx5dr_cmd.h      |  27 +-
 drivers/net/mlx5/hws/mlx5dr_context.c  |  12 +-
 drivers/net/mlx5/hws/mlx5dr_context.h  |   1 +
 drivers/net/mlx5/hws/mlx5dr_debug.c    |  41 +-
 drivers/net/mlx5/hws/mlx5dr_debug.h    |   4 +-
 drivers/net/mlx5/hws/mlx5dr_definer.c  | 750 +++++++++++++++++++++--
--  drivers/net/mlx5/hws/mlx5dr_definer.h  |  44 +-
 drivers/net/mlx5/hws/mlx5dr_internal.h |   2 +-
 drivers/net/mlx5/hws/mlx5dr_matcher.c  | 279 ++++++---
drivers/net/mlx5/hws/mlx5dr_matcher.h  |  43 +-
drivers/net/mlx5/hws/mlx5dr_pat_arg.c  |  49 +-
 drivers/net/mlx5/hws/mlx5dr_pat_arg.h  |   6 +-
 drivers/net/mlx5/hws/mlx5dr_rule.c     | 193 ++++++-
 drivers/net/mlx5/hws/mlx5dr_rule.h     |   2 +
 drivers/net/mlx5/hws/mlx5dr_send.c     | 169 +++++-
 drivers/net/mlx5/hws/mlx5dr_send.h     |  24 +-
 drivers/net/mlx5/mlx5_flow_hw.c        |   2 +-
 20 files changed, 1475 insertions(+), 304 deletions(-)
Quoted 2 lines

--
2.18.1

Series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh