net/mlx5: support flow hit steering action
Subsystems (2)
cover net/mlx5: support flow hit steering action
A new ASO (Advanced Steering Operation) feature was added in the last mlx5 adapters to support flow hit detection.
Using this new steering action, the driver can detect flow traffic hit and to reset this indication any time.
Add support for flow aging action in rte_flow using this new feature.
The counter aging mode will be taken only when the ASO feature is not supported.
Dekel Peled (6):
common/mlx5: add DevX API to create ASO flow hit object common/mlx5: use general object type for cap index common/mlx5: add read ASO flow hit HCA capability common/mlx5: add glue func create flow hit action common/mlx5: add definitions for ASO flow hit net/mlx5: support flow hit action for aging
Matan Azrad (2):
net/mlx5: optimize shared RSS action memory
net/mlx5: support shared age action
drivers/common/mlx5/linux/meson.build | 4 + drivers/common/mlx5/linux/mlx5_glue.c | 16 + drivers/common/mlx5/linux/mlx5_glue.h | 3 + drivers/common/mlx5/mlx5_devx_cmds.c | 48 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 5 + drivers/common/mlx5/mlx5_prm.h | 98 ++++- drivers/common/mlx5/version.map | 1 + drivers/net/mlx5/linux/mlx5_os.c | 11 + drivers/net/mlx5/meson.build | 1 + drivers/net/mlx5/mlx5.c | 82 +++++ drivers/net/mlx5/mlx5.h | 103 +++++- drivers/net/mlx5/mlx5_flow.c | 181 +++++---- drivers/net/mlx5/mlx5_flow.h | 36 +- drivers/net/mlx5/mlx5_flow_age.c | 675 ++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_flow_dv.c | 592 +++++++++++++++++++++++------ 15 files changed, 1651 insertions(+), 205 deletions(-) create mode 100644 drivers/net/mlx5/mlx5_flow_age.c
Signature · 1 line
1.8.3.1
PATCH 1/8 common/mlx5: add DevX API to create ASO flow hit object
From: Dekel Peled <[email protected]>
Add DevX API to create ASO flow hit object.
--- drivers/common/mlx5/mlx5_devx_cmds.c | 45 ++++++++++++++++++++++++++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 4 ++++ drivers/common/mlx5/mlx5_prm.h | 14 +++++++++++ drivers/common/mlx5/version.map | 1 + 4 files changed, 64 insertions(+)
drivers/common/mlx5/mlx5_devx_cmds.c +45 -0
@@ -1991,3 +1991,48 @@ struct mlx5_devx_obj * invalid_buffer);return ret; } + +/** +* Create general object of type FLOW_HIT_ASO using DevX API. +* +* @param[in] ctx +* Context returned from mlx5 open_device() glue function. +* @param [in] pd +* PD value to associate the FLOW_HIT_ASO object with. +* +* @return +* The DevX object created, NULL otherwise and rte_errno is set. +*/ +struct mlx5_devx_obj * +mlx5_devx_cmd_create_flow_hit_aso_obj (void *ctx,uint32_t pd) +{ +uint32_t in[MLX5_ST_SZ_DW (create_flow_hit_aso_in)] = {0 }; +uint32_t out[MLX5_ST_SZ_DW (general_obj_out_cmd_hdr)] = {0 }; +struct mlx5_devx_obj *flow_hit_aso_obj =NULL ; +void *ptr =NULL ; + + flow_hit_aso_obj =mlx5_malloc (MLX5_MEM_ZERO ,sizeof (*flow_hit_aso_obj), +0 ,SOCKET_ID_ANY ); +if (!flow_hit_aso_obj) { +DRV_LOG (ERR ,"Failed to allocate FLOW_HIT_ASO object data" ); + rte_errno =ENOMEM ; +return NULL ; + } + ptr =MLX5_ADDR_OF (create_flow_hit_aso_in, in, hdr); +MLX5_SET (general_obj_in_cmd_hdr, ptr, opcode, +MLX5_CMD_OP_CREATE_GENERAL_OBJECT ); +MLX5_SET (general_obj_in_cmd_hdr, ptr, obj_type, +MLX5_GENERAL_OBJ_TYPE_FLOW_HIT_ASO ); + ptr =MLX5_ADDR_OF (create_flow_hit_aso_in, in, flow_hit_aso); +MLX5_SET (flow_hit_aso, ptr, access_pd, pd); + flow_hit_aso_obj->obj = mlx5_glue->devx_obj_create (ctx, in,sizeof (in), + out,sizeof (out)); +if (!flow_hit_aso_obj->obj) { + rte_errno = errno; +DRV_LOG (ERR ,"Failed to create FLOW_HIT_ASO obj using DevX." ); +mlx5_free (flow_hit_aso_obj); +return NULL ; + } + flow_hit_aso_obj->id =MLX5_GET (general_obj_out_cmd_hdr, out, obj_id); +return flow_hit_aso_obj; +}
drivers/common/mlx5/mlx5_devx_cmds.h +4 -0
@@ -483,4 +483,8 @@ int mlx5_devx_cmd_register_read(void *ctx, uint16_t reg_id,int mlx5_devx_cmd_query_virtio_q_counters (struct mlx5_devx_obj *couners_obj,struct mlx5_devx_virtio_q_couners_attr *attr); +__rte_internal +struct mlx5_devx_obj *mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx, +uint32_t pd); +#endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */
drivers/common/mlx5/mlx5_prm.h +14 -0
@@ -2211,6 +2211,7 @@ enum {MLX5_GENERAL_OBJ_TYPE_VIRTQ =0x000d ,MLX5_GENERAL_OBJ_TYPE_VIRTIO_Q_COUNTERS =0x001c ,MLX5_GENERAL_OBJ_TYPE_FLEX_PARSE_GRAPH =0x0022 , + MLX5_GENERAL_OBJ_TYPE_FLOW_HIT_ASO = 0x0025, }; struct mlx5_ifc_general_obj_in_cmd_hdr_bits {
@@ -2329,6 +2330,19 @@ struct mlx5_ifc_query_virtq_out_bits {struct mlx5_ifc_virtio_net_q_bits virtq; }; +struct mlx5_ifc_flow_hit_aso_bits { +u8 modify_field_select[0x40 ]; +u8 reserved_at_40[0x48 ]; +u8 access_pd[0x18 ]; +u8 reserved_at_a0[0x160 ]; +u8 flag[0x200 ]; +}; + +struct mlx5_ifc_create_flow_hit_aso_in_bits { +struct mlx5_ifc_general_obj_in_cmd_hdr_bits hdr; +struct mlx5_ifc_flow_hit_aso_bits flow_hit_aso; +}; +enum {MLX5_QP_ST_RC =0x0 , };
drivers/common/mlx5/version.map +1 -0
@@ -21,6 +21,7 @@ INTERNAL { mlx5_devx_cmd_create_tis; mlx5_devx_cmd_create_virtio_q_counters; mlx5_devx_cmd_create_virtq; + mlx5_devx_cmd_create_flow_hit_aso_obj; mlx5_devx_cmd_destroy; mlx5_devx_cmd_flow_counter_alloc; mlx5_devx_cmd_flow_counter_query;
Signature · 1 line
1.8.3.1
PATCH 2/8 common/mlx5: use general object type for cap index
In reply to Matan AzradFrom: Dekel Peled <[email protected]>
PRM defines the general object types using positive numbers. The same values are used as index for the relevant bit in HCA capabilities general_obj_types bit mask.
--- drivers/common/mlx5/mlx5_prm.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
drivers/common/mlx5/mlx5_prm.h +6 -3
@@ -1041,9 +1041,12 @@ enum { MLX5_GET_HCA_CAP_OP_MOD_VDPA_EMULATION = 0x13 << 1, }; -#define MLX5_GENERAL_OBJ_TYPES_CAP_VIRTQ_NET_Q (1ULL << 0xd) -#define MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_Q_COUNTERS (1ULL << 0x1c) -#define MLX5_GENERAL_OBJ_TYPES_CAP_PARSE_GRAPH_FLEX_NODE (1ULL << 0x22) +#define MLX5_GENERAL_OBJ_TYPES_CAP_VIRTQ_NET_Q \ + (1ULL << MLX5_GENERAL_OBJ_TYPE_VIRTQ) +#define MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_Q_COUNTERS \ + (1ULL << MLX5_GENERAL_OBJ_TYPE_VIRTIO_Q_COUNTERS) +#define MLX5_GENERAL_OBJ_TYPES_CAP_PARSE_GRAPH_FLEX_NODE \ + (1ULL << MLX5_GENERAL_OBJ_TYPE_FLEX_PARSE_GRAPH)enum {MLX5_HCA_CAP_OPMOD_GET_MAX =0 ,
Signature · 1 line
1.8.3.1
PATCH 3/8 common/mlx5: add read ASO flow hit HCA capability
In reply to Matan AzradFrom: Dekel Peled <[email protected]>
Read and store the device capability of FLOW_HIT_ASO general object, using the DevX API.
--- drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 1 + drivers/common/mlx5/mlx5_prm.h | 2 ++ 3 files changed, 6 insertions(+)
drivers/common/mlx5/mlx5_devx_cmds.c +3 -0
@@ -718,6 +718,9 @@ struct mlx5_devx_obj * attr->regex =MLX5_GET (cmd_hca_cap, hcattr, regexp); attr->regexp_num_of_engines =MLX5_GET (cmd_hca_cap, hcattr, regexp_num_of_engines); + attr->flow_hit_aso = !!(MLX5_GET64 (cmd_hca_cap, hcattr, + general_obj_types) & +MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO );if (attr->qos.sup) { MLX5_SET(query_hca_cap_in, in, op_mod, MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP |
drivers/common/mlx5/mlx5_devx_cmds.h +1 -0
@@ -101,6 +101,7 @@ struct mlx5_hca_attr { uint32_t log_max_static_sq_wq:5; /* Static WQE size SQ. */uint32_t dev_freq_khz;/* Timestamp counter frequency, kHz. */ uint32_t scatter_fcs_w_decap_disable:1; + uint32_t flow_hit_aso:1; /* General obj type FLOW_HIT_ASO supported. */ uint32_t regex:1;uint32_t regexp_num_of_engines; uint32_t log_max_ft_sampler_num:8;
drivers/common/mlx5/mlx5_prm.h +2 -0
@@ -1047,6 +1047,8 @@ enum { (1ULL << MLX5_GENERAL_OBJ_TYPE_VIRTIO_Q_COUNTERS)#define MLX5_GENERAL_OBJ_TYPES_CAP_PARSE_GRAPH_FLEX_NODE \ (1ULL << MLX5_GENERAL_OBJ_TYPE_FLEX_PARSE_GRAPH) +#define MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO \ + (1ULL << MLX5_GENERAL_OBJ_TYPE_FLOW_HIT_ASO)enum {MLX5_HCA_CAP_OPMOD_GET_MAX =0 ,
Signature · 1 line
1.8.3.1
PATCH 4/8 common/mlx5: add glue func create flow hit action
In reply to Matan AzradFrom: Dekel Peled <[email protected]>
Add glue function to create the flow hit action using DV API, if rdma-core support exists.
--- drivers/common/mlx5/linux/meson.build | 2 ++ drivers/common/mlx5/linux/mlx5_glue.c | 16 ++++++++++++++++ drivers/common/mlx5/linux/mlx5_glue.h | 3 +++ 3 files changed, 21 insertions(+)
drivers/common/mlx5/linux/meson.build +2 -0
@@ -179,6 +179,8 @@ has_sym_args = [ [ 'HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY', 'infiniband/mlx5dv.h', 'mlx5dv_dr_action_create_dest_array'], [ 'HAVE_DEVLINK', 'linux/devlink.h', 'DEVLINK_GENL_NAME' ], + [ 'HAVE_MLX5DV_DR_ACTION_FLOW_HIT', 'infiniband/mlx5dv.h', + 'mlx5dv_dr_action_create_flow_hit'], ] config = configuration_data() foreach arg:has_sym_args
drivers/common/mlx5/linux/mlx5_glue.c +16 -0
@@ -1283,6 +1283,21 @@#endif } +static void * +mlx5_glue_dr_action_create_flow_hit (struct mlx5dv_devx_obj *devx_obj, +uint32_t offset,uint8_t reg_c_index) +{ +#ifdef HAVE_MLX5DV_DR_ACTION_FLOW_HIT +return mlx5dv_dr_action_create_flow_hit (devx_obj, offset, reg_c_index); +#else + (void )(devx_obj); + (void )(offset); + (void )(reg_c_index); + errno =ENOTSUP ; +return NULL ; +#endif +} +__rte_cache_aligned const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) { .version =MLX5_GLUE_VERSION ,
@@ -1402,4 +1417,5 @@ .dv_free_var = mlx5_glue_dv_free_var, .dv_alloc_pp = mlx5_glue_dv_alloc_pp, .dv_free_pp = mlx5_glue_dv_free_pp, + .dr_action_create_flow_hit = mlx5_glue_dr_action_create_flow_hit, };
drivers/common/mlx5/linux/mlx5_glue.h +3 -0
@@ -345,6 +345,9 @@ struct mlx5_glue { (void *domain, size_t num_dest, struct mlx5dv_dr_action_dest_attr *dests[]); +void *(*dr_action_create_flow_hit)(struct mlx5dv_devx_obj *devx_obj, +uint32_t offset, +uint8_t reg_c_index); };extern const struct mlx5_glue *mlx5_glue;
Signature · 1 line
1.8.3.1
PATCH 5/8 common/mlx5: add definitions for ASO flow hit
In reply to Matan AzradFrom: Dekel Peled <[email protected]>
This patch adds different PRM definitions, related to ASO flow hit feature, in MLX5 PMD code.
--- drivers/common/mlx5/linux/meson.build | 2 ++ drivers/common/mlx5/mlx5_prm.h | 66 ++++++++++++++++++++++++++++++++++- 2 files changed, 67 insertions(+), 1 deletion(-)
drivers/common/mlx5/linux/meson.build +2 -0
@@ -126,6 +126,8 @@ has_sym_args = [ 'MLX5_OPCODE_SEND_EN' ], [ 'HAVE_MLX5_OPCODE_WAIT', 'infiniband/mlx5dv.h', 'MLX5_OPCODE_WAIT' ], + [ 'HAVE_MLX5_OPCODE_ACCESS_ASO', 'infiniband/mlx5dv.h', + 'MLX5_OPCODE_ACCESS_ASO' ], [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h', 'SUPPORTED_40000baseKR4_Full' ], [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
drivers/common/mlx5/mlx5_prm.h +65 -1
84 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 6/8 net/mlx5: support flow hit action for aging
In reply to Matan AzradFrom: Dekel Peled <[email protected]>
A new ASO (Advanced Steering Operation) feature was added in the last mlx5 adapters to support flow hit detection.
Using this new steering action, the driver can detect flow traffic hit and to reset this indication any time.
Add support for flow aging action in rte_flow using this new feature.
The counter aging mode will be taken only when the ASO feature is not supported.
--- drivers/common/mlx5/mlx5_prm.h | 9 +- drivers/net/mlx5/linux/mlx5_os.c | 11 + drivers/net/mlx5/meson.build | 1 + drivers/net/mlx5/mlx5.c | 70 ++++ drivers/net/mlx5/mlx5.h | 98 +++++- drivers/net/mlx5/mlx5_flow.h | 1 + drivers/net/mlx5/mlx5_flow_age.c | 675 +++++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_flow_dv.c | 338 ++++++++++++++++++-- 8 files changed, 1171 insertions(+), 32 deletions(-) create mode 100644 drivers/net/mlx5/mlx5_flow_age.c
drivers/common/mlx5/mlx5_prm.h +8 -1
@@ -2360,12 +2360,17 @@ enum mlx5_access_aso_op_mod { ASO_OP_MOD_FLOW_HIT = 0x4, }; +#define ASO_CSEG_DATA_MASK_MODE_OFFSET 30 +enum mlx5_aso_data_mask_mode {BITWISE_64BIT =0x0 ,BYTEWISE_64BYTE =0x1 ,CALCULATED_64BYTE =0x2 , }; +#define ASO_CSEG_COND_0_OPER_OFFSET 20 +#define ASO_CSEG_COND_1_OPER_OFFSET 16 +enum mlx5_aso_pre_cond_op {ASO_OP_ALWAYS_FALSE =0x0 ,ASO_OP_ALWAYS_TRUE =0x1 ,
@@ -2379,6 +2384,8 @@ enum mlx5_aso_pre_cond_op { ASO_OP_CYCLIC_LESSER = 0x9, }; +#define ASO_CSEG_COND_OPER_OFFSET 6 +enum mlx5_aso_op {ASO_OPER_LOGICAL_AND =0x0 ,ASO_OPER_LOGICAL_OR =0x1 ,
@@ -2387,7 +2394,7 @@ enum mlx5_aso_op {/* ASO WQE CTRL segment. */ struct mlx5_aso_cseg {uint32_t va_h; -uint32_t va_l_ro; +uint32_t va_l_r;uint32_t lkey;uint32_t operand_masks;uint32_t condition_0_data;
drivers/net/mlx5/linux/mlx5_os.c +11 -0
@@ -1129,6 +1129,17 @@ err = -err; goto error; } +#ifdef HAVE_MLX5DV_DR_ACTION_FLOW_HIT +if (config->hca_attr.flow_hit_aso) { + sh->flow_hit_aso_en =1 ; + err =mlx5_flow_aso_age_mng_init (sh); +if (err) { + err = -err; + goto error; + } +DRV_LOG (DEBUG ,"Flow Hit ASO is supported." ); + } +#endif /* HAVE_MLX5DV_DR_ACTION_FLOW_HIT */ /* Check relax ordering support. */ if (config->hca_attr.relaxed_ordering_write && config->hca_attr.relaxed_ordering_read &&
drivers/net/mlx5/meson.build +1 -0
@@ -16,6 +16,7 @@ sources = files( 'mlx5_flow_meter.c', 'mlx5_flow_dv.c', 'mlx5_flow_verbs.c', + 'mlx5_flow_age.c', 'mlx5_mac.c', 'mlx5_mr.c', 'mlx5_rss.c',
drivers/net/mlx5/mlx5.c +70 -0
82 lines not on this page · raw message
drivers/net/mlx5/mlx5.h +96 -2
138 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow.h +1 -0
@@ -1035,6 +1035,7 @@ struct rte_flow {/**< Index to metadata register copy table resource. */ uint32_t counter;/**< Holds flow counter. */ uint32_t tunnel_id;/**< Tunnel id */ +uint32_t age;/**< Holds ASO age bit index. */ } __rte_packed;/*
drivers/net/mlx5/mlx5_flow_age.c new file +675 -0
675 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow_dv.c +309 -29
395 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 7/8 net/mlx5: optimize shared RSS action memory
In reply to Matan AzradThe RSS shared action was saved in flow memory by a pointer. It means that every flow memory includes 8B only for optional shared RSS case.
Move the RSS objects to be used by indexed pool which reduces the flow handle memory to 4B.
So, now, the shared action handler is also just a 4B index.
--- drivers/net/mlx5/mlx5.c | 12 +++ drivers/net/mlx5/mlx5.h | 8 +- drivers/net/mlx5/mlx5_flow.c | 128 +++++++++++++-------------- drivers/net/mlx5/mlx5_flow.h | 24 +++--- drivers/net/mlx5/mlx5_flow_dv.c | 186 +++++++++++++++++++++++----------------- 5 files changed, 194 insertions(+), 164 deletions(-)
drivers/net/mlx5/mlx5.c +12 -0
@@ -336,6 +336,18 @@ static LIST_HEAD(, mlx5_dev_ctx_shared) mlx5_dev_ctx_list = .need_lock = 1, .type = "mlx5_flow_tnl_tbl_ipool", }, + { + .size = sizeof(struct mlx5_shared_action_rss), + .trunk_size = 64, + .grow_trunk = 3, + .grow_shift = 2, + .need_lock = 1, + .release_mem_en = 1, + .malloc = mlx5_malloc, + .free = mlx5_free, + .type = "mlx5_shared_action_rss", + }, + };
drivers/net/mlx5/mlx5.h +4 -4
@@ -53,6 +53,7 @@ enum mlx5_ipool_index {MLX5_IPOOL_RSS_EXPANTION_FLOW_ID ,/* Pool for Queue/RSS flow ID. */ MLX5_IPOOL_TUNNEL_ID ,/* Pool for flow tunnel ID. */ MLX5_IPOOL_TNL_TBL_ID ,/* Pool for tunnel table ID. */ +MLX5_IPOOL_RSS_SHARED_ACTIONS ,/* Pool for RSS shared actions. */ MLX5_IPOOL_MAX, };
@@ -895,8 +896,6 @@ enum mlx5_txq_modify_type { MLX5_TXQ_MOD_ERR2RDY, /* modify state from error to ready. */ }; - -/* HW objects operations structure. */ struct mlx5_obj_ops {int (*rxq_obj_modify_vlan_strip)(struct mlx5_rxq_obj *rxq_obj,int on);
@@ -922,6 +921,8 @@ struct mlx5_obj_ops {void (*txq_obj_release)(struct mlx5_txq_obj *txq_obj); }; +#define MLX5_RSS_HASH_FIELDS_LEN RTE_DIM(mlx5_rss_hash_fields) +struct mlx5_priv {struct rte_eth_dev_data *dev_data;/* Pointer to device data. */ struct mlx5_dev_ctx_shared *sh;/* Shared device context. */
@@ -998,8 +999,7 @@ struct mlx5_priv {struct mlx5_mp_id mp_id;/* ID of a multi-process process */ LIST_HEAD(fdir, mlx5_fdir_flow) fdir_flows; /* fdir flows. */rte_spinlock_t shared_act_sl;/* Shared actions spinlock. */ - LIST_HEAD(shared_action, rte_flow_shared_action) shared_actions; -/* shared actions */ +uint32_t rss_shared_actions;/* RSS shared actions. */ };#define PORT_ID (priv) ((priv)->dev_data->port_id)
drivers/net/mlx5/mlx5_flow.c +59 -69
211 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow.h +10 -14
@@ -35,10 +35,15 @@ enum mlx5_rte_flow_action_type { MLX5_RTE_FLOW_ACTION_TYPE_MARK,MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG ,MLX5_RTE_FLOW_ACTION_TYPE_DEFAULT_MISS , -MLX5_RTE_FLOW_ACTION_TYPE_SHARED_RSS , MLX5_RTE_FLOW_ACTION_TYPE_TUNNEL_SET, }; +#define MLX5_SHARED_ACTION_TYPE_OFFSET 30 + +enum { +MLX5_SHARED_ACTION_TYPE_RSS , +}; +/* Matches on selected register. */ struct mlx5_rte_flow_item_tag {enum modify_reg id;
@@ -1024,7 +1029,7 @@ struct tunnel_tbl_entry {/* Flow structure. */ struct rte_flow {ILIST_ENTRY (uint32_t )next;/**< Index to the next flow structure. */ -struct mlx5_shared_action_rss *shared_rss;/** < Shred RSS action. */ +uint32_t shared_rss;/** < Shared RSS action ID. */ uint32_t dev_handles;/**< Device flow handles that are part of the flow. */ uint32_t drv_type:2 ;/**< Driver type. */
@@ -1069,10 +1074,10 @@ struct rte_flow { MLX5_RSS_HASH_NONE, }; -#define MLX5_RSS_HASH_FIELDS_LEN RTE_DIM(mlx5_rss_hash_fields) -/* Shared RSS action structure */ struct mlx5_shared_action_rss { +ILIST_ENTRY (uint32_t )next;/**< Index to the next RSS structure. */ +uint32_t refcnt;/**< Atomically accessed refcnt. */ struct rte_flow_action_rss origin;/**< Original rte RSS action. */ uint8_t key[MLX5_RSS_HASH_KEY_LEN ];/**< RSS hash key. */ uint16_t *queue;/**< Queue indices to use. */
@@ -1083,15 +1088,7 @@ struct mlx5_shared_action_rss { };struct rte_flow_shared_action { -LIST_ENTRY (rte_flow_shared_action) next; -/**< Pointer to the next element. */ -uint32_t refcnt;/**< Atomically accessed refcnt. */ -uint64_t type; -/**< Shared action type (see MLX5_FLOW_ACTION_SHARED_*). */ -union { -struct mlx5_shared_action_rss rss; -/**< Shared RSS action. */ - }; +uint32_t id; };/* Thread specific flow workspace intermediate data. */
@@ -1391,7 +1388,6 @@ int mlx5_flow_destroy_policer_rules(struct rte_eth_dev *dev,int mlx5_flow_meter_flush (struct rte_eth_dev *dev,struct rte_mtr_error *error);int mlx5_flow_dv_discover_counter_offset_support (struct rte_eth_dev *dev); -struct rte_flow_shared_action *mlx5_flow_get_shared_rss (struct rte_flow *flow);int mlx5_shared_action_flush (struct rte_eth_dev *dev);void mlx5_release_tunnel_hub (struct mlx5_dev_ctx_shared *sh,uint16_t port_id);int mlx5_alloc_tunnel_hub (struct mlx5_dev_ctx_shared *sh);
drivers/net/mlx5/mlx5_flow_dv.c +109 -77
350 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 8/8 net/mlx5: support shared age action
In reply to Matan AzradAdd support for rte_flow shared action API for age action.
First step here to support validate, create, query and destroy.
The support is only for age ASO mode.
--- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_flow.c | 61 +++++++++++++++-- drivers/net/mlx5/mlx5_flow.h | 11 ++++ drivers/net/mlx5/mlx5_flow_dv.c | 140 ++++++++++++++++++++++++++++------------ 4 files changed, 165 insertions(+), 48 deletions(-)
drivers/net/mlx5/mlx5.h +1 -0
@@ -524,6 +524,7 @@ struct mlx5_aso_sq {struct mlx5_aso_age_action {LIST_ENTRY (mlx5_aso_age_action) next;void *dr_action; +uint32_t refcnt;/* Following fields relevant only when action is active. */ uint16_t offset;/* Offset of ASO Flow Hit flag in DevX object. */ struct mlx5_age_param age_params;
drivers/net/mlx5/mlx5_flow.c +56 -5
85 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow.h +11 -0
@@ -36,12 +36,14 @@ enum mlx5_rte_flow_action_type {MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG ,MLX5_RTE_FLOW_ACTION_TYPE_DEFAULT_MISS ,MLX5_RTE_FLOW_ACTION_TYPE_TUNNEL_SET , + MLX5_RTE_FLOW_ACTION_TYPE_AGE, };#define MLX5_SHARED_ACTION_TYPE_OFFSET 30enum {MLX5_SHARED_ACTION_TYPE_RSS , +MLX5_SHARED_ACTION_TYPE_AGE , };/* Matches on selected register. */
@@ -1173,6 +1175,12 @@ typedef int (*mlx5_flow_action_update_t)struct rte_flow_shared_action *action, const void *action_conf, struct rte_flow_error *error); +typedef int (*mlx5_flow_action_query_t ) + (struct rte_eth_dev *dev, +const struct rte_flow_shared_action *action, +void *data, +struct rte_flow_error *error); +typedef int (*mlx5_flow_sync_domain_t ) (struct rte_eth_dev *dev, uint32_t domains,
@@ -1197,6 +1205,7 @@ struct mlx5_flow_driver_ops {mlx5_flow_action_create_t action_create;mlx5_flow_action_destroy_t action_destroy;mlx5_flow_action_update_t action_update; +mlx5_flow_action_query_t action_query;mlx5_flow_sync_domain_t sync_domain; };
@@ -1465,4 +1474,6 @@ struct mlx5_cache_entry *flow_dv_dest_array_create_cb struct mlx5_cache_entry *entry, void *cb_ctx);void flow_dv_dest_array_remove_cb (struct mlx5_cache_list *list,struct mlx5_cache_entry *entry); +struct mlx5_aso_age_action *flow_aso_age_get_by_idx (struct rte_eth_dev *dev, +uint32_t age_idx);#endif /* RTE_PMD_MLX5_FLOW_H_ */
drivers/net/mlx5/mlx5_flow_dv.c +97 -43
246 lines not on this page · raw message
Signature · 1 line
1.8.3.1
v2 cover net/mlx5: support flow hit steering action
In reply to Matan AzradA new ASO (Advanced Steering Operation) feature was added in the last mlx5 adapters to support flow hit detection.
Using this new steering action, the driver can detect flow traffic hit and to reset this indication any time.
Add support for flow aging action in rte_flow using this new feature.
The counter aging mode will be taken only when the ASO feature is not supported.
v2:
- rebase.
- fix refcnt issue with rss action.
- allow group 0 use counter base age action.
- cosmetics.
Dekel Peled (6):
common/mlx5: add DevX API to create ASO flow hit object common/mlx5: use general object type for cap index common/mlx5: add read ASO flow hit HCA capability common/mlx5: add glue func create flow hit action common/mlx5: add definitions for ASO flow hit net/mlx5: support flow hit action for aging
Matan Azrad (3):
net/mlx5: optimize shared RSS action memory
net/mlx5: support shared age action
net/mlx5: allow age modes combination
drivers/common/mlx5/linux/meson.build | 4 + drivers/common/mlx5/linux/mlx5_glue.c | 16 + drivers/common/mlx5/linux/mlx5_glue.h | 3 + drivers/common/mlx5/mlx5_devx_cmds.c | 48 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 5 + drivers/common/mlx5/mlx5_prm.h | 98 ++++- drivers/common/mlx5/version.map | 1 + drivers/net/mlx5/linux/mlx5_os.c | 11 + drivers/net/mlx5/meson.build | 1 + drivers/net/mlx5/mlx5.c | 102 ++++++ drivers/net/mlx5/mlx5.h | 102 +++++- drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_flow.c | 192 ++++++---- drivers/net/mlx5/mlx5_flow.h | 36 +- drivers/net/mlx5/mlx5_flow_age.c | 667 ++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_flow_dv.c | 592 ++++++++++++++++++++++++------ 16 files changed, 1672 insertions(+), 208 deletions(-) create mode 100644 drivers/net/mlx5/mlx5_flow_age.c
Signature · 1 line
1.8.3.1
PATCH v2 1/9 common/mlx5: add DevX API to create ASO flow hit object
From: Dekel Peled <[email protected]>
Add DevX API to create ASO flow hit object.
--- drivers/common/mlx5/mlx5_devx_cmds.c | 45 ++++++++++++++++++++++++++++++++++++ drivers/common/mlx5/mlx5_devx_cmds.h | 4 ++++ drivers/common/mlx5/mlx5_prm.h | 14 +++++++++++ drivers/common/mlx5/version.map | 1 + 4 files changed, 64 insertions(+)
drivers/common/mlx5/mlx5_devx_cmds.c +45 -0
@@ -1995,3 +1995,48 @@ struct mlx5_devx_obj * invalid_buffer);return ret; } + +/** +* Create general object of type FLOW_HIT_ASO using DevX API. +* +* @param[in] ctx +* Context returned from mlx5 open_device() glue function. +* @param [in] pd +* PD value to associate the FLOW_HIT_ASO object with. +* +* @return +* The DevX object created, NULL otherwise and rte_errno is set. +*/ +struct mlx5_devx_obj * +mlx5_devx_cmd_create_flow_hit_aso_obj (void *ctx,uint32_t pd) +{ +uint32_t in[MLX5_ST_SZ_DW (create_flow_hit_aso_in)] = {0 }; +uint32_t out[MLX5_ST_SZ_DW (general_obj_out_cmd_hdr)] = {0 }; +struct mlx5_devx_obj *flow_hit_aso_obj =NULL ; +void *ptr =NULL ; + + flow_hit_aso_obj =mlx5_malloc (MLX5_MEM_ZERO ,sizeof (*flow_hit_aso_obj), +0 ,SOCKET_ID_ANY ); +if (!flow_hit_aso_obj) { +DRV_LOG (ERR ,"Failed to allocate FLOW_HIT_ASO object data" ); + rte_errno =ENOMEM ; +return NULL ; + } + ptr =MLX5_ADDR_OF (create_flow_hit_aso_in, in, hdr); +MLX5_SET (general_obj_in_cmd_hdr, ptr, opcode, +MLX5_CMD_OP_CREATE_GENERAL_OBJECT ); +MLX5_SET (general_obj_in_cmd_hdr, ptr, obj_type, +MLX5_GENERAL_OBJ_TYPE_FLOW_HIT_ASO ); + ptr =MLX5_ADDR_OF (create_flow_hit_aso_in, in, flow_hit_aso); +MLX5_SET (flow_hit_aso, ptr, access_pd, pd); + flow_hit_aso_obj->obj = mlx5_glue->devx_obj_create (ctx, in,sizeof (in), + out,sizeof (out)); +if (!flow_hit_aso_obj->obj) { + rte_errno = errno; +DRV_LOG (ERR ,"Failed to create FLOW_HIT_ASO obj using DevX." ); +mlx5_free (flow_hit_aso_obj); +return NULL ; + } + flow_hit_aso_obj->id =MLX5_GET (general_obj_out_cmd_hdr, out, obj_id); +return flow_hit_aso_obj; +}
drivers/common/mlx5/mlx5_devx_cmds.h +4 -0
@@ -485,4 +485,8 @@ int mlx5_devx_cmd_register_read(void *ctx, uint16_t reg_id,int mlx5_devx_cmd_query_virtio_q_counters (struct mlx5_devx_obj *couners_obj,struct mlx5_devx_virtio_q_couners_attr *attr); +__rte_internal +struct mlx5_devx_obj *mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx, +uint32_t pd); +#endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */
drivers/common/mlx5/mlx5_prm.h +14 -0
@@ -2211,6 +2211,7 @@ enum {MLX5_GENERAL_OBJ_TYPE_VIRTQ =0x000d ,MLX5_GENERAL_OBJ_TYPE_VIRTIO_Q_COUNTERS =0x001c ,MLX5_GENERAL_OBJ_TYPE_FLEX_PARSE_GRAPH =0x0022 , + MLX5_GENERAL_OBJ_TYPE_FLOW_HIT_ASO = 0x0025, }; struct mlx5_ifc_general_obj_in_cmd_hdr_bits {
@@ -2330,6 +2331,19 @@ struct mlx5_ifc_query_virtq_out_bits {struct mlx5_ifc_virtio_net_q_bits virtq; }; +struct mlx5_ifc_flow_hit_aso_bits { +u8 modify_field_select[0x40 ]; +u8 reserved_at_40[0x48 ]; +u8 access_pd[0x18 ]; +u8 reserved_at_a0[0x160 ]; +u8 flag[0x200 ]; +}; + +struct mlx5_ifc_create_flow_hit_aso_in_bits { +struct mlx5_ifc_general_obj_in_cmd_hdr_bits hdr; +struct mlx5_ifc_flow_hit_aso_bits flow_hit_aso; +}; +enum {MLX5_EVENT_TYPE_OBJECT_CHANGE =0x27 , };
drivers/common/mlx5/version.map +1 -0
@@ -21,6 +21,7 @@ INTERNAL { mlx5_devx_cmd_create_tis; mlx5_devx_cmd_create_virtio_q_counters; mlx5_devx_cmd_create_virtq; + mlx5_devx_cmd_create_flow_hit_aso_obj; mlx5_devx_cmd_destroy; mlx5_devx_cmd_flow_counter_alloc; mlx5_devx_cmd_flow_counter_query;
Signature · 1 line
1.8.3.1
PATCH v2 2/9 common/mlx5: use general object type for cap index
In reply to Matan AzradFrom: Dekel Peled <[email protected]>
PRM defines the general object types using positive numbers. The same values are used as index for the relevant bit in HCA capabilities general_obj_types bit mask.
--- drivers/common/mlx5/mlx5_prm.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
drivers/common/mlx5/mlx5_prm.h +6 -3
@@ -1041,9 +1041,12 @@ enum { MLX5_GET_HCA_CAP_OP_MOD_VDPA_EMULATION = 0x13 << 1, }; -#define MLX5_GENERAL_OBJ_TYPES_CAP_VIRTQ_NET_Q (1ULL << 0xd) -#define MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_Q_COUNTERS (1ULL << 0x1c) -#define MLX5_GENERAL_OBJ_TYPES_CAP_PARSE_GRAPH_FLEX_NODE (1ULL << 0x22) +#define MLX5_GENERAL_OBJ_TYPES_CAP_VIRTQ_NET_Q \ + (1ULL << MLX5_GENERAL_OBJ_TYPE_VIRTQ) +#define MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_Q_COUNTERS \ + (1ULL << MLX5_GENERAL_OBJ_TYPE_VIRTIO_Q_COUNTERS) +#define MLX5_GENERAL_OBJ_TYPES_CAP_PARSE_GRAPH_FLEX_NODE \ + (1ULL << MLX5_GENERAL_OBJ_TYPE_FLEX_PARSE_GRAPH)enum {MLX5_HCA_CAP_OPMOD_GET_MAX =0 ,
Signature · 1 line
1.8.3.1
PATCH v2 3/9 common/mlx5: add read ASO flow hit HCA capability
In reply to Matan AzradFrom: Dekel Peled <[email protected]>
Read and store the device capability of FLOW_HIT_ASO general object, using the DevX API.
--- drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 1 + drivers/common/mlx5/mlx5_prm.h | 2 ++ 3 files changed, 6 insertions(+)
drivers/common/mlx5/mlx5_devx_cmds.c +3 -0
@@ -719,6 +719,9 @@ struct mlx5_devx_obj * attr->regex =MLX5_GET (cmd_hca_cap, hcattr, regexp); attr->regexp_num_of_engines =MLX5_GET (cmd_hca_cap, hcattr, regexp_num_of_engines); + attr->flow_hit_aso = !!(MLX5_GET64 (cmd_hca_cap, hcattr, + general_obj_types) & +MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO );if (attr->qos.sup) { MLX5_SET(query_hca_cap_in, in, op_mod, MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP |
drivers/common/mlx5/mlx5_devx_cmds.h +1 -0
@@ -102,6 +102,7 @@ struct mlx5_hca_attr { uint32_t num_lag_ports:4; /* Number of ports can be bonded. */uint32_t dev_freq_khz;/* Timestamp counter frequency, kHz. */ uint32_t scatter_fcs_w_decap_disable:1; + uint32_t flow_hit_aso:1; /* General obj type FLOW_HIT_ASO supported. */ uint32_t regex:1;uint32_t regexp_num_of_engines; uint32_t log_max_ft_sampler_num:8;
drivers/common/mlx5/mlx5_prm.h +2 -0
@@ -1047,6 +1047,8 @@ enum { (1ULL << MLX5_GENERAL_OBJ_TYPE_VIRTIO_Q_COUNTERS)#define MLX5_GENERAL_OBJ_TYPES_CAP_PARSE_GRAPH_FLEX_NODE \ (1ULL << MLX5_GENERAL_OBJ_TYPE_FLEX_PARSE_GRAPH) +#define MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO \ + (1ULL << MLX5_GENERAL_OBJ_TYPE_FLOW_HIT_ASO)enum {MLX5_HCA_CAP_OPMOD_GET_MAX =0 ,
Signature · 1 line
1.8.3.1
PATCH v2 4/9 common/mlx5: add glue func create flow hit action
In reply to Matan AzradFrom: Dekel Peled <[email protected]>
Add glue function to create the flow hit action using DV API, if rdma-core support exists.
--- drivers/common/mlx5/linux/meson.build | 2 ++ drivers/common/mlx5/linux/mlx5_glue.c | 16 ++++++++++++++++ drivers/common/mlx5/linux/mlx5_glue.h | 3 +++ 3 files changed, 21 insertions(+)
drivers/common/mlx5/linux/meson.build +2 -0
@@ -179,6 +179,8 @@ has_sym_args = [ [ 'HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY', 'infiniband/mlx5dv.h', 'mlx5dv_dr_action_create_dest_array'], [ 'HAVE_DEVLINK', 'linux/devlink.h', 'DEVLINK_GENL_NAME' ], + [ 'HAVE_MLX5DV_DR_ACTION_FLOW_HIT', 'infiniband/mlx5dv.h', + 'mlx5dv_dr_action_create_flow_hit'], ] config = configuration_data() foreach arg:has_sym_args
drivers/common/mlx5/linux/mlx5_glue.c +16 -0
@@ -1281,6 +1281,21 @@#endif } +static void * +mlx5_glue_dr_action_create_flow_hit (struct mlx5dv_devx_obj *devx_obj, +uint32_t offset,uint8_t reg_c_index) +{ +#ifdef HAVE_MLX5DV_DR_ACTION_FLOW_HIT +return mlx5dv_dr_action_create_flow_hit (devx_obj, offset, reg_c_index); +#else + (void )(devx_obj); + (void )(offset); + (void )(reg_c_index); + errno =ENOTSUP ; +return NULL ; +#endif +} +__rte_cache_aligned const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) { .version =MLX5_GLUE_VERSION ,
@@ -1400,4 +1415,5 @@ .dv_free_var = mlx5_glue_dv_free_var, .dv_alloc_pp = mlx5_glue_dv_alloc_pp, .dv_free_pp = mlx5_glue_dv_free_pp, + .dr_action_create_flow_hit = mlx5_glue_dr_action_create_flow_hit, };
drivers/common/mlx5/linux/mlx5_glue.h +3 -0
@@ -344,6 +344,9 @@ struct mlx5_glue { (void *domain, size_t num_dest, struct mlx5dv_dr_action_dest_attr *dests[]); +void *(*dr_action_create_flow_hit)(struct mlx5dv_devx_obj *devx_obj, +uint32_t offset, +uint8_t reg_c_index); };extern const struct mlx5_glue *mlx5_glue;
Signature · 1 line
1.8.3.1
PATCH v2 5/9 common/mlx5: add definitions for ASO flow hit
In reply to Matan AzradFrom: Dekel Peled <[email protected]>
This patch adds different PRM definitions, related to ASO flow hit feature, in MLX5 PMD code.
--- drivers/common/mlx5/linux/meson.build | 2 ++ drivers/common/mlx5/mlx5_prm.h | 66 ++++++++++++++++++++++++++++++++++- 2 files changed, 67 insertions(+), 1 deletion(-)
drivers/common/mlx5/linux/meson.build +2 -0
@@ -126,6 +126,8 @@ has_sym_args = [ 'MLX5_OPCODE_SEND_EN' ], [ 'HAVE_MLX5_OPCODE_WAIT', 'infiniband/mlx5dv.h', 'MLX5_OPCODE_WAIT' ], + [ 'HAVE_MLX5_OPCODE_ACCESS_ASO', 'infiniband/mlx5dv.h', + 'MLX5_OPCODE_ACCESS_ASO' ], [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h', 'SUPPORTED_40000baseKR4_Full' ], [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h',
drivers/common/mlx5/mlx5_prm.h +65 -1
84 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v2 6/9 net/mlx5: support flow hit action for aging
In reply to Matan AzradFrom: Dekel Peled <[email protected]>
A new ASO (Advanced Steering Operation) feature was added in the last mlx5 adapters to support flow hit detection.
Using this new steering action, the driver can detect flow traffic hit and to reset this indication any time.
The ASO age action cannot support flows in table 0.
Add support for flow aging action in rte_flow using this new feature.
The counter aging mode will be taken only when the ASO feature is not supported for the user flow groups.
--- drivers/common/mlx5/mlx5_prm.h | 9 +- drivers/net/mlx5/linux/mlx5_os.c | 11 + drivers/net/mlx5/meson.build | 1 + drivers/net/mlx5/mlx5.c | 89 ++++++ drivers/net/mlx5/mlx5.h | 97 +++++- drivers/net/mlx5/mlx5_flow.c | 11 +- drivers/net/mlx5/mlx5_flow.h | 1 + drivers/net/mlx5/mlx5_flow_age.c | 667 +++++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_flow_dv.c | 336 ++++++++++++++++++-- 9 files changed, 1180 insertions(+), 42 deletions(-) create mode 100644 drivers/net/mlx5/mlx5_flow_age.c
drivers/common/mlx5/mlx5_prm.h +8 -1
@@ -2361,12 +2361,17 @@ enum mlx5_access_aso_op_mod { ASO_OP_MOD_FLOW_HIT = 0x4, }; +#define ASO_CSEG_DATA_MASK_MODE_OFFSET 30 +enum mlx5_aso_data_mask_mode {BITWISE_64BIT =0x0 ,BYTEWISE_64BYTE =0x1 ,CALCULATED_64BYTE =0x2 , }; +#define ASO_CSEG_COND_0_OPER_OFFSET 20 +#define ASO_CSEG_COND_1_OPER_OFFSET 16 +enum mlx5_aso_pre_cond_op {ASO_OP_ALWAYS_FALSE =0x0 ,ASO_OP_ALWAYS_TRUE =0x1 ,
@@ -2380,6 +2385,8 @@ enum mlx5_aso_pre_cond_op { ASO_OP_CYCLIC_LESSER = 0x9, }; +#define ASO_CSEG_COND_OPER_OFFSET 6 +enum mlx5_aso_op {ASO_OPER_LOGICAL_AND =0x0 ,ASO_OPER_LOGICAL_OR =0x1 ,
@@ -2388,7 +2395,7 @@ enum mlx5_aso_op {/* ASO WQE CTRL segment. */ struct mlx5_aso_cseg {uint32_t va_h; -uint32_t va_l_ro; +uint32_t va_l_r;uint32_t lkey;uint32_t operand_masks;uint32_t condition_0_data;
drivers/net/mlx5/linux/mlx5_os.c +11 -0
@@ -1129,6 +1129,17 @@ err = -err; goto error; } +#ifdef HAVE_MLX5DV_DR_ACTION_FLOW_HIT +if (config->hca_attr.flow_hit_aso) { + sh->flow_hit_aso_en =1 ; + err =mlx5_flow_aso_age_mng_init (sh); +if (err) { + err = -err; + goto error; + } +DRV_LOG (DEBUG ,"Flow Hit ASO is supported." ); + } +#endif /* HAVE_MLX5DV_DR_ACTION_FLOW_HIT */ /* Check relax ordering support. */ if (config->hca_attr.relaxed_ordering_write && config->hca_attr.relaxed_ordering_read &&
drivers/net/mlx5/meson.build +1 -0
@@ -16,6 +16,7 @@ sources = files( 'mlx5_flow_meter.c', 'mlx5_flow_dv.c', 'mlx5_flow_verbs.c', + 'mlx5_flow_age.c', 'mlx5_mac.c', 'mlx5_mr.c', 'mlx5_rss.c',
drivers/net/mlx5/mlx5.c +89 -0
107 lines not on this page · raw message
drivers/net/mlx5/mlx5.h +95 -2
137 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow.c +1 -10
@@ -6816,16 +6816,7 @@ struct mlx5_meter_domains_infos * }rte_spinlock_unlock (&age_info->aged_sl); } -for (i =0 ; i < sh->max_port; i++) { - age_info = &sh->port[i].age_info; -if (!MLX5_AGE_GET (age_info,MLX5_AGE_EVENT_NEW )) -continue ; -if (MLX5_AGE_GET (age_info,MLX5_AGE_TRIGGER )) -rte_eth_dev_callback_process - (&rte_eth_devices[sh->port[i].devx_ih_port_id], -RTE_ETH_EVENT_FLOW_AGED ,NULL ); - age_info->flags =0 ; - } +mlx5_age_event_prepare (sh); }/**
drivers/net/mlx5/mlx5_flow.h +1 -0
@@ -1027,6 +1027,7 @@ struct rte_flow {/**< Index to metadata register copy table resource. */ uint32_t counter;/**< Holds flow counter. */ uint32_t tunnel_id;/**< Tunnel id */ +uint32_t age;/**< Holds ASO age bit index. */ } __rte_packed;/*
drivers/net/mlx5/mlx5_flow_age.c new file +667 -0
667 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow_dv.c +307 -29
393 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v2 7/9 net/mlx5: optimize shared RSS action memory
In reply to Matan AzradThe RSS shared action was saved in flow memory by a pointer. It means that every flow memory includes 8B only for optional shared RSS case.
Move the RSS objects to be used by indexed pool which reduces the flow handle memory to 4B.
So, now, the shared action handler is also just a 4B index.
--- drivers/net/mlx5/mlx5.c | 12 +++ drivers/net/mlx5/mlx5.h | 6 +- drivers/net/mlx5/mlx5_flow.c | 128 +++++++++++++-------------- drivers/net/mlx5/mlx5_flow.h | 24 +++--- drivers/net/mlx5/mlx5_flow_dv.c | 186 +++++++++++++++++++++++----------------- 5 files changed, 194 insertions(+), 162 deletions(-)
drivers/net/mlx5/mlx5.c +12 -0
@@ -336,6 +336,18 @@ static LIST_HEAD(, mlx5_dev_ctx_shared) mlx5_dev_ctx_list = .need_lock = 1, .type = "mlx5_flow_tnl_tbl_ipool", }, + { + .size = sizeof(struct mlx5_shared_action_rss), + .trunk_size = 64, + .grow_trunk = 3, + .grow_shift = 2, + .need_lock = 1, + .release_mem_en = 1, + .malloc = mlx5_malloc, + .free = mlx5_free, + .type = "mlx5_shared_action_rss", + }, + };
drivers/net/mlx5/mlx5.h +4 -2
@@ -53,6 +53,7 @@ enum mlx5_ipool_index {MLX5_IPOOL_RSS_EXPANTION_FLOW_ID ,/* Pool for Queue/RSS flow ID. */ MLX5_IPOOL_TUNNEL_ID ,/* Pool for flow tunnel ID. */ MLX5_IPOOL_TNL_TBL_ID ,/* Pool for tunnel table ID. */ +MLX5_IPOOL_RSS_SHARED_ACTIONS ,/* Pool for RSS shared actions. */ MLX5_IPOOL_MAX, };
@@ -920,6 +921,8 @@ struct mlx5_obj_ops {void (*txq_obj_release)(struct mlx5_txq_obj *txq_obj); }; +#define MLX5_RSS_HASH_FIELDS_LEN RTE_DIM(mlx5_rss_hash_fields) +struct mlx5_priv {struct rte_eth_dev_data *dev_data;/* Pointer to device data. */ struct mlx5_dev_ctx_shared *sh;/* Shared device context. */
@@ -996,8 +999,7 @@ struct mlx5_priv {struct mlx5_mp_id mp_id;/* ID of a multi-process process */ LIST_HEAD(fdir, mlx5_fdir_flow) fdir_flows; /* fdir flows. */rte_spinlock_t shared_act_sl;/* Shared actions spinlock. */ - LIST_HEAD(shared_action, rte_flow_shared_action) shared_actions; -/* shared actions */ +uint32_t rss_shared_actions;/* RSS shared actions. */ };#define PORT_ID (priv) ((priv)->dev_data->port_id)
drivers/net/mlx5/mlx5_flow.c +59 -69
211 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow.h +10 -14
@@ -35,10 +35,15 @@ enum mlx5_rte_flow_action_type { MLX5_RTE_FLOW_ACTION_TYPE_MARK,MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG ,MLX5_RTE_FLOW_ACTION_TYPE_DEFAULT_MISS , -MLX5_RTE_FLOW_ACTION_TYPE_SHARED_RSS , MLX5_RTE_FLOW_ACTION_TYPE_TUNNEL_SET, }; +#define MLX5_SHARED_ACTION_TYPE_OFFSET 30 + +enum { +MLX5_SHARED_ACTION_TYPE_RSS , +}; +/* Matches on selected register. */ struct mlx5_rte_flow_item_tag {enum modify_reg id;
@@ -1017,7 +1022,7 @@ struct tunnel_tbl_entry {/* Flow structure. */ struct rte_flow {ILIST_ENTRY (uint32_t )next;/**< Index to the next flow structure. */ -struct mlx5_shared_action_rss *shared_rss;/** < Shred RSS action. */ +uint32_t shared_rss;/** < Shared RSS action ID. */ uint32_t dev_handles;/**< Device flow handles that are part of the flow. */ uint32_t drv_type:2 ;/**< Driver type. */
@@ -1061,10 +1066,10 @@ struct rte_flow { MLX5_RSS_HASH_NONE, }; -#define MLX5_RSS_HASH_FIELDS_LEN RTE_DIM(mlx5_rss_hash_fields) -/* Shared RSS action structure */ struct mlx5_shared_action_rss { +ILIST_ENTRY (uint32_t )next;/**< Index to the next RSS structure. */ +uint32_t refcnt;/**< Atomically accessed refcnt. */ struct rte_flow_action_rss origin;/**< Original rte RSS action. */ uint8_t key[MLX5_RSS_HASH_KEY_LEN ];/**< RSS hash key. */ uint16_t *queue;/**< Queue indices to use. */
@@ -1075,15 +1080,7 @@ struct mlx5_shared_action_rss { };struct rte_flow_shared_action { -LIST_ENTRY (rte_flow_shared_action) next; -/**< Pointer to the next element. */ -uint32_t refcnt;/**< Atomically accessed refcnt. */ -uint64_t type; -/**< Shared action type (see MLX5_FLOW_ACTION_SHARED_*). */ -union { -struct mlx5_shared_action_rss rss; -/**< Shared RSS action. */ - }; +uint32_t id; };/* Thread specific flow workspace intermediate data. */
@@ -1383,7 +1380,6 @@ int mlx5_flow_destroy_policer_rules(struct rte_eth_dev *dev,int mlx5_flow_meter_flush (struct rte_eth_dev *dev,struct rte_mtr_error *error);int mlx5_flow_dv_discover_counter_offset_support (struct rte_eth_dev *dev); -struct rte_flow_shared_action *mlx5_flow_get_shared_rss (struct rte_flow *flow);int mlx5_shared_action_flush (struct rte_eth_dev *dev);void mlx5_release_tunnel_hub (struct mlx5_dev_ctx_shared *sh,uint16_t port_id);int mlx5_alloc_tunnel_hub (struct mlx5_dev_ctx_shared *sh);
drivers/net/mlx5/mlx5_flow_dv.c +109 -77
350 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v2 8/9 net/mlx5: support shared age action
In reply to Matan AzradAdd support for rte_flow shared action API for ASO age action.
First step here to support validate, create, query and destroy.
The support is only for age ASO mode.
--- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_flow.c | 61 ++++++++++++++-- drivers/net/mlx5/mlx5_flow.h | 11 +++ drivers/net/mlx5/mlx5_flow_dv.c | 155 +++++++++++++++++++++++++++++----------- 5 files changed, 181 insertions(+), 49 deletions(-)
drivers/net/mlx5/mlx5.h +1 -0
@@ -524,6 +524,7 @@ struct mlx5_aso_sq {struct mlx5_aso_age_action {LIST_ENTRY (mlx5_aso_age_action) next;void *dr_action; +uint32_t refcnt;/* Following fields relevant only when action is active. */ uint16_t offset;/* Offset of ASO Flow Hit flag in DevX object. */ struct mlx5_age_param age_params;
drivers/net/mlx5/mlx5_defs.h +1 -1
@@ -197,7 +197,7 @@#define MLX5_HAIRPIN_JUMBO_LOG_SIZE (14 + 2)/* Maximum number of shared actions supported by rte_flow */ -#define MLX5_MAX_SHARED_ACTIONS 1 +#define MLX5_MAX_SHARED_ACTIONS 2/* Definition of static_assert found in /usr/include/assert.h */ #ifndef HAVE_STATIC_ASSERT
drivers/net/mlx5/mlx5_flow.c +56 -5
85 lines not on this page · raw message
drivers/net/mlx5/mlx5_flow.h +11 -0
@@ -36,12 +36,14 @@ enum mlx5_rte_flow_action_type {MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG ,MLX5_RTE_FLOW_ACTION_TYPE_DEFAULT_MISS ,MLX5_RTE_FLOW_ACTION_TYPE_TUNNEL_SET , + MLX5_RTE_FLOW_ACTION_TYPE_AGE, };#define MLX5_SHARED_ACTION_TYPE_OFFSET 30enum {MLX5_SHARED_ACTION_TYPE_RSS , +MLX5_SHARED_ACTION_TYPE_AGE , };/* Matches on selected register. */
@@ -1165,10 +1167,16 @@ typedef int (*mlx5_flow_action_update_t)struct rte_flow_shared_action *action, const void *action_conf, struct rte_flow_error *error); +typedef int (*mlx5_flow_action_query_t ) + (struct rte_eth_dev *dev, +const struct rte_flow_shared_action *action, +void *data, +struct rte_flow_error *error);typedef int (*mlx5_flow_sync_domain_t ) (struct rte_eth_dev *dev,uint32_t domains,uint32_t flags); +struct mlx5_flow_driver_ops {mlx5_flow_validate_t validate;mlx5_flow_prepare_t prepare;
@@ -1189,6 +1197,7 @@ struct mlx5_flow_driver_ops {mlx5_flow_action_create_t action_create;mlx5_flow_action_destroy_t action_destroy;mlx5_flow_action_update_t action_update; +mlx5_flow_action_query_t action_query;mlx5_flow_sync_domain_t sync_domain; };
@@ -1457,4 +1466,6 @@ struct mlx5_cache_entry *flow_dv_dest_array_create_cb struct mlx5_cache_entry *entry, void *cb_ctx);void flow_dv_dest_array_remove_cb (struct mlx5_cache_list *list,struct mlx5_cache_entry *entry); +struct mlx5_aso_age_action *flow_aso_age_get_by_idx (struct rte_eth_dev *dev, +uint32_t age_idx);#endif /* RTE_PMD_MLX5_FLOW_H_ */
drivers/net/mlx5/mlx5_flow_dv.c +112 -43
261 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [dpdk-dev] [PATCH v2 8/9] net/mlx5: support shared age action
Hi Self note
From: Matan Azrad:
Quoted 74 lines
Add support for
rte_flowshared action API for ASO age action.First step here to support validate, create, query and destroy.
The support is only for age ASO mode.
Signed-off-by Matan Azrad <[email protected]>Acked-by Dekel Peled <[email protected]>--- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_flow.c | 61 ++++++++++++++-- drivers/net/mlx5/mlx5_flow.h | 11 +++ drivers/net/mlx5/mlx5_flow_dv.c | 155 +++++++++++++++++++++++++++++---- ------- 5 files changed, 181 insertions(+), 49 deletions(-)drivers/net/mlx5/mlx5.h index renamed from drivers/net/mlx5/mlx5.h +0
-02bc47d8..a156e5c 100644
drivers/net/mlx5/mlx5.h +1
-0@@ -524,6 +524,7 @@ struct mlx5_aso_sq { struct mlx5_aso_age_action {LIST_ENTRY (mlx5_aso_age_action) next;void *dr_action; +uint32_t refcnt;/* Following fields relevant only when action is active. */ uint16_t offset;/* Offset of ASO Flow Hit flag in DevX object. */ struct mlx5_age_param age_params;drivers/net/mlx5/mlx5_defs.h index renamed from drivers/net/mlx5/mlx5_defs.h +0
-02657081..4980352 100644
drivers/net/mlx5/mlx5_defs.h +0
-0@@ -197,7 +197,7 @@#define MLX5_HAIRPIN_JUMBO_LOG_SIZE (14 + 2) /* Maximum number of shared actions supported by rte_flow */ -#define
MLX5_MAX_SHARED_ACTIONS1
+#defineMLX5_MAX_SHARED_ACTIONS2/* Definition of
static_assertfound in/usr/include/assert.h*/ #ifndefHAVE_STATIC_ASSERTdiff --gita/drivers/net/mlx5/mlx5_flow.cb/drivers/net/mlx5/mlx5_flow.cindex b08ee30..c0a2a04 100644drivers/net/mlx5/mlx5_flow.c +22
-0@@ -3266,6 +3266,29 @@ struct mlx5_flow_tunnel_info {return NULL ; } +/** +* Get ASO age action by index. +* +* @param[in] dev +* Pointer to the Ethernet device structure. +* @param[in] age_idx +* Index to the ASO age action. +* +* @return +* The specified ASO age action. +*/ +struct mlx5_aso_age_action * +flow_aso_age_get_by_idx (struct rte_eth_dev *dev,uint32_t age_idx) { +uint16_t pool_idx = age_idx &UINT16_MAX ; +uint16_t offset = (age_idx >>16 ) &UINT16_MAX ; +struct mlx5_priv *priv = dev->data->dev_private; +struct mlx5_aso_age_mng *mng = priv->sh->aso_age_mng; +struct mlx5_aso_age_pool *pool = mng->pools[pool_idx]; + +return &pool->actions[offset -1 ]; +} +/* maps shared action to translated non shared in some actions array */ struct mlx5_translated_shared_action { struct rte_flow_shared_action *action; /**< Shared action */ @@ -3353,6 +3376,15 @@ struct
mlx5_translated_shared_action{
drivers/net/mlx5/mlx5_flow.c struct mlx5_translated_shared_action { · line 3381
Quoted 6 lines
drivers/net/mlx5/mlx5_flow.c +3
-0translated[shared->index].conf = &shared_rss->origin;break ; +case MLX5_SHARED_ACTION_TYPE_AGE : +if (priv->sh->flow_hit_aso_en) { + translated[shared->index].type =
In order not get a warning in compilation for some gcc versions need to add this conversion to the enum below:
(enum rte_flow_action_type)
Raslan, let me know if you need more version for all the series for this, or you can do it in integration time....
Quoted 396 lines
+
MLX5_RTE_FLOW_ACTION_TYPE_AGE;drivers/net/mlx5/mlx5_flow.c +27
-5+ translated[shared->index].conf = + (void *)(uintptr_t )idx; +break ; + } +/* Fall-through */ default :mlx5_free (translated);return rte_flow_error_set@@ -7273,6 +7305,25 @@ struct mlx5_meter_domains_infos *return fops->action_update (dev, action, action_conf, error); } +/* Wrapper for driver action_destroy op callback */ static int +flow_drv_action_query (struct rte_eth_dev *dev, +const struct rte_flow_shared_action *action, +void *data, +const struct mlx5_flow_driver_ops *fops, +struct rte_flow_error *error) +{ +static const char err_msg[] ="shared action query unsupported" ; + +if (!fops->action_query) { +DRV_LOG (ERR ,"port %u %s." , dev->data->port_id, err_msg); +rte_flow_error_set (error,ENOTSUP ,RTE_FLOW_ERROR_TYPE_ACTION , +NULL , err_msg); +return -rte_errno; + } +return fops->action_query (dev, action, data, error); } +/** * Create shared action for reuse in multiple flow rules. * @@ -7375,11 +7426,11 @@ struct mlx5_meter_domains_infos *void *data, struct rte_flow_error *error) { - (void)dev; - (void)action; - (void)data; - return rte_flow_error_set(error, ENOTSUP,RTE_FLOW_ERROR_TYPE_ACTION , - NULL, "action type query not supported"); + struct rte_flow_attr attr = { .transfer = 0 }; +const struct mlx5_flow_driver_ops *fops = +flow_get_drv_ops (flow_get_drv_type (dev, &attr)); + +return flow_drv_action_query (dev, action, data, fops, error); }/**
drivers/net/mlx5/mlx5_flow.h +10
-0@@ -36,12 +36,14 @@ enum mlx5_rte_flow_action_type {MLX5_RTE_FLOW_ACTION_TYPE_COPY_MREG ,MLX5_RTE_FLOW_ACTION_TYPE_DEFAULT_MISS ,MLX5_RTE_FLOW_ACTION_TYPE_TUNNEL_SET , + MLX5_RTE_FLOW_ACTION_TYPE_AGE, };#define MLX5_SHARED_ACTION_TYPE_OFFSET 30enum {MLX5_SHARED_ACTION_TYPE_RSS , +MLX5_SHARED_ACTION_TYPE_AGE , };/* Matches on selected register. */ @@ -1165,10 +1167,16 @@ typedef int (*mlx5_flow_action_update_t)struct rte_flow_shared_action *action, const void *action_conf, struct rte_flow_error *error); +typedef int (*mlx5_flow_action_query_t ) + (struct rte_eth_dev *dev, +const struct rte_flow_shared_action *action, +void *data, +struct rte_flow_error *error);typedef int (*mlx5_flow_sync_domain_t ) (struct rte_eth_dev *dev,uint32_t domains,uint32_t flags); +struct mlx5_flow_driver_ops {mlx5_flow_validate_t validate;mlx5_flow_prepare_t prepare;@@ -1189,6 +1197,7 @@ struct mlx5_flow_driver_ops {mlx5_flow_action_create_t action_create;mlx5_flow_action_destroy_t action_destroy;mlx5_flow_action_update_t action_update; +mlx5_flow_action_query_t action_query;mlx5_flow_sync_domain_t sync_domain; };@@ -1457,4 +1466,6 @@ struct mlx5_cache_entry *flow_dv_dest_array_create_cb struct mlx5_cache_entry *entry, void *cb_ctx); void flow_dv_dest_array_remove_cb(struct mlx5_cache_list *list,struct mlx5_cache_entry *entry); +struct mlx5_aso_age_action *flow_aso_age_get_by_idx(struct rte_eth_dev*dev,
drivers/net/mlx5/mlx5_flow.h +1
-0+ uint32_t age_idx);#endif /* RTE_PMD_MLX5_FLOW_H_ */ (quoted diff) +0
-0
b/drivers/net/mlx5/mlx5_flow_dv.cindex be93ba9..d60626c 100644drivers/net/mlx5/mlx5_flow_dv.c +28
-29@@ -5926,6 +5926,10 @@ struct mlx5_hlist_entry */* Meter action will add one more TAG action. */ rw_act_num +=MLX5_ACT_NUM_SET_TAG ;break ; +case MLX5_RTE_FLOW_ACTION_TYPE_AGE : + action_flags |=MLX5_FLOW_ACTION_AGE ; + ++actions_n; +break ;case RTE_FLOW_ACTION_TYPE_AGE : ret = flow_dv_validate_action_age(action_flags, actions, dev,@@ -9241,29 +9245,6 @@ struct mlx5_cache_entry * }/** -* Get ASO age action by index. -* -* @param[in] dev -* Pointer to the Ethernet device structure. -* @param[in] age_idx -* Index to the ASO age action. -* -* @return -* The specified ASO age action. -*/ -static struct mlx5_aso_age_action * -flow_dv_aso_age_get_by_idx(struct rte_eth_dev *dev, uint32_t age_idx) -{ -uint16_t pool_idx = age_idx &UINT16_MAX ; -uint16_t offset = (age_idx >>16 ) &UINT16_MAX ; -struct mlx5_priv *priv = dev->data->dev_private; -struct mlx5_aso_age_mng *mng = priv->sh->aso_age_mng; -struct mlx5_aso_age_pool *pool = mng->pools[pool_idx]; - -return &pool->actions[offset -1 ]; -} - -/** * Remove an ASO age action from age actions list. * * @param[in] dev @@ -9295,18 +9276,35 @@ struct mlx5_cache_entry * } } -static void +/** +* Release an ASO age action. +* +* @param[in] dev +* Pointer to the Ethernet device structure. +* @param[in] age_idx +* Index of ASO age action to release. +* @param[in] flow +* True if the release operation is during flow destroy operation. +* False if the release operation is during action destroy operation. +* +* @return +* 0 when age action was removed, otherwise the number of references. +*/ +static int flow_dv_aso_age_release (struct rte_eth_dev *dev,uint32_t age_idx) {struct mlx5_priv *priv = dev->data->dev_private;struct mlx5_aso_age_mng *mng = priv->sh->aso_age_mng; -struct mlx5_aso_age_action *age =flow_dv_aso_age_get_by_idx (dev, - age_idx); +struct mlx5_aso_age_action *age =flow_aso_age_get_by_idx (dev, age_idx); +uint32_t ret =__atomic_sub_fetch (&age->refcnt,1 , __ATOMIC_RELAXED); -flow_dv_aso_age_remove_from_age (dev, age); -rte_spinlock_lock (&mng->free_sl); -LIST_INSERT_HEAD (&mng->free, age, next); -rte_spinlock_unlock (&mng->free_sl); +if (!ret) { +flow_dv_aso_age_remove_from_age (dev, age); +rte_spinlock_lock (&mng->free_sl); +LIST_INSERT_HEAD (&mng->free, age, next); +rte_spinlock_unlock (&mng->free_sl); + } +return ret; }/**
drivers/net/mlx5/mlx5_flow_dv.c +5
-2@@ -9450,6 +9448,7 @@ struct mlx5_cache_entry *return 0 ;/* 0 is an error.*/ } } +__atomic_store_n (&age_free->refcnt,1 , __ATOMIC_RELAXED);return pool->index | ((age_free->offset +1 ) <<16 ); }@@ -9469,12 +9468,12 @@ struct mlx5_cache_entry * const struct rte_flow_action_age *age) {uint32_t age_idx =0 ; -struct mlx5_aso_age_action *aso_age =NULL ; +struct mlx5_aso_age_action *aso_age; age_idx =flow_dv_aso_age_alloc (dev);if (!age_idx)return 0 ; - aso_age =flow_dv_aso_age_get_by_idx (dev, age_idx); + aso_age =flow_aso_age_get_by_idx (dev, age_idx); aso_age->age_params.context = age->context; aso_age->age_params.timeout = age->timeout; aso_age->age_params.port_id = dev->data->port_id; @@ -9638,6 +9637,7 @@ struct mlx5_cache_entry * const uint8_t *rss_key;const struct rte_flow_action_meter *mtr;struct mlx5_flow_tbl_resource *tbl; +struct mlx5_aso_age_action *age_act;uint32_t port_id =0 ;struct mlx5_flow_dv_port_id_action_resource
port_id_resource;drivers/net/mlx5/mlx5_flow_dv.c +8
-0int action_type = actions->type;@@ -9774,6 +9774,14 @@ struct mlx5_cache_entry * action_flags |=MLX5_FLOW_ACTION_RSS ; dev_flow->handle->fate_action =MLX5_FLOW_FATE_QUEUE ;break ; +case MLX5_RTE_FLOW_ACTION_TYPE_AGE : + flow->age = (uint32_t )(uintptr_t )(action->conf); + age_act =flow_aso_age_get_by_idx (dev, flow->age); +__atomic_fetch_add (&age_act->refcnt,1 , + __ATOMIC_RELAXED); + dev_flow->dv.actions[actions_n++] = age_act- >dr_action; + action_flags |=MLX5_FLOW_ACTION_AGE ; +break ;case RTE_FLOW_ACTION_TYPE_AGE :drivers/net/mlx5/mlx5_flow_dv.c +0
-0if (priv->sh->flow_hit_aso_en) { flow->age =
flow_dv_translate_create_aso_age@@ -9785,7 +9793,7 @@ structmlx5_cache_entry*drivers/net/mlx5/mlx5_flow_dv.c +14
-1NULL , "can't create age action"); dev_flow->dv.actions[actions_n++] = - (flow_dv_aso_age_get_by_idx + (flow_aso_age_get_by_idx (dev, flow->age))->dr_action; action_flags |=MLX5_FLOW_ACTION_AGE ;break ;@@ -11441,6 +11449,19 @@ struct mlx5_cache_entry * idx = (MLX5_SHARED_ACTION_TYPE_RSS <<MLX5_SHARED_ACTION_TYPE_OFFSET ) | ret;break ; +case RTE_FLOW_ACTION_TYPE_AGE : + ret =flow_dv_translate_create_aso_age (dev, action->conf); + idx = (MLX5_SHARED_ACTION_TYPE_AGE << +MLX5_SHARED_ACTION_TYPE_OFFSET ) | ret; +if (ret) { +struct mlx5_aso_age_action *aso_age = +flow_aso_age_get_by_idx (dev, ret); + +if (!aso_age->age_params.context) + aso_age->age_params.context = + (void *)(uintptr_t )idx; + } +break ;default : rte_flow_error_set(err, ENOTSUP,
RTE_FLOW_ERROR_TYPE_ACTION,
NULL, "action type not supported"); @@ -
11478,17 +11499,23 @@ structmlx5_cache_entry*drivers/net/mlx5/mlx5_flow_dv.c +9
-2switch (type) { case MLX5_SHARED_ACTION_TYPE_RSS: - ret =__flow_dv_action_rss_release (dev, idx, error); -break ; + return __flow_dv_action_rss_release(dev, idx, error); + case MLX5_SHARED_ACTION_TYPE_AGE: + ret = flow_dv_aso_age_release(dev, idx); + if (ret) + /* + * In this case, the last flow has a reference will + * actually release the age action. + */ + DRV_LOG(DEBUG, "Shared age action %" PRIu32 "was"
drivers/net/mlx5/mlx5_flow_dv.c +23
-4+ " released with references %d.", idx, ret); +return 0 ;default :return rte_flow_error_set (error,ENOTSUP ,RTE_FLOW_ERROR_TYPE_ACTION ,NULL ,"action type not supported" ); } -if (ret) -return ret; -return 0 ; }/** @@ -11609,9 +11636,41 @@ struct mlx5_cache_entry *return rte_flow_error_set (err,ENOTSUP ,RTE_FLOW_ERROR_TYPE_ACTION ,NULL , -"action type not supported" ); +"action type update not supported" ); + } +} + +static int +flow_dv_action_query (struct rte_eth_dev *dev, +const struct rte_flow_shared_action *action,void *data, +struct rte_flow_error *error) +{ +struct mlx5_age_param *age_param; +struct rte_flow_query_age *resp; +uint32_t act_idx = (uint32_t )(uintptr_t )action; +uint32_t type = act_idx >>MLX5_SHARED_ACTION_TYPE_OFFSET ; +uint32_t idx = act_idx & ((1u <<MLX5_SHARED_ACTION_TYPE_OFFSET ) -1 ); + +switch (type) { +case MLX5_SHARED_ACTION_TYPE_AGE : + age_param = &flow_aso_age_get_by_idx (dev, idx)- >age_params; + resp = data; + resp->aged =__atomic_load_n (&age_param->state, + __ATOMIC_RELAXED) ==
AGE_TMOUT?drivers/net/mlx5/mlx5_flow_dv.c +5
-0+ 1 : 0; + resp->sec_since_last_hit_valid = !resp->aged; +if (resp->sec_since_last_hit_valid) + resp->sec_since_last_hit = __atomic_load_n + (&age_param->sec_since_last_hit,
__ATOMIC_RELAXED);drivers/net/mlx5/mlx5_flow_dv.c +8
-1+return 0 ; +default : +return rte_flow_error_set (error,ENOTSUP , +RTE_FLOW_ERROR_TYPE_ACTION , +NULL , +"action type query not supported" ); } } +/** * Query a dv flow rule for its statistics via devx. * @@ -11692,7 +11751,7 @@ struct mlx5_cache_entry *if (flow->age) {struct mlx5_aso_age_action *act = -flow_dv_aso_age_get_by_idx (dev, flow- >age); +flow_aso_age_get_by_idx (dev, flow->age); age_param = &act->age_params;drivers/net/mlx5/mlx5_flow_dv.c +12
-3}else if (flow->counter) {@@ -12402,14 +12461,23 @@ struct mlx5_cache_entry * flow_dv_action_validate(struct rte_eth_dev *dev, const struct rte_flow_shared_action_conf *conf, const struct rte_flow_action *action, - struct rte_flow_error *error) + struct rte_flow_error *err) { + struct mlx5_priv *priv = dev->data->dev_private; +RTE_SET_USED (conf);switch (action->type) {case RTE_FLOW_ACTION_TYPE_RSS : -return mlx5_validate_action_rss (dev, action, error); +return mlx5_validate_action_rss (dev, action, err); +case RTE_FLOW_ACTION_TYPE_AGE : +if (!priv->sh->aso_age_mng) +return rte_flow_error_set (err,ENOTSUP , +RTE_FLOW_ERROR_TYPE_UNSPECIFIED , +NULL , +"shared age action not supported" ); +return flow_dv_validate_action_age (0 , action, dev, err);default : - return rte_flow_error_set(error, ENOTSUP, + return rte_flow_error_set(err, ENOTSUP, RTE_FLOW_ERROR_TYPE_ACTION, NULL,drivers/net/mlx5/mlx5_flow_dv.c +1
-1"action type not supported");@@ -12461,6 +12529,7 @@ struct mlx5_cache_entry * .action_create = flow_dv_action_create, .action_destroy = flow_dv_action_destroy, .action_update = flow_dv_action_update, + .action_query = flow_dv_action_query, .sync_domain = flow_dv_sync_domain, }; --1.8.3.1
Re: [dpdk-dev] [PATCH v2 8/9] net/mlx5: support shared age action
Hi,
Quoted 26 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Matan Azrad
Sent: Monday, November 2, 2020 8:17 AM
To: Matan Azrad <[email protected]>; Slava Ovsiienko <[email protected]>
Cc: [email protected]
Subject: Re: [dpdk-dev] [PATCH v2 8/9] net/mlx5: support shared age actionHi Self note
From: Matan Azrad:
Quoted 6 lines
Add support for
rte_flowshared action API for ASO age action.First step here to support validate, create, query and destroy.
The support is only for age ASO mode.
Quoted 8 lines
Signed-off-by Matan Azrad <[email protected]>Acked-by Dekel Peled <[email protected]>--- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_flow.c | 61 ++++++++++++++-- drivers/net/mlx5/mlx5_flow.h | 11 +++ drivers/net/mlx5/mlx5_flow_dv.c | 155+++++++++++++++++++++++++++++----
[...]
Quoted 11 lines
drivers/net/mlx5/mlx5_flow.c struct mlx5_translated_shared_action { · line 3381
Quoted 5 lines
drivers/net/mlx5/mlx5_flow.c +3
-0&shared_rss->origin;break ; +case MLX5_SHARED_ACTION_TYPE_AGE : +if (priv->sh->flow_hit_aso_en) { + translated[shared->index].type =In order not get a warning in compilation for some gcc versions need to add this conversion to the enum below:
(enumrte_flow_action_type)
Quoted 3 lines
Raslan, let me know if you need more version for all the series for this, or you can do it in integration time....
It's OK I'll update it during integration,
[...]
Kindest regards
Raslan Darawsheh
PATCH v2 9/9 net/mlx5: allow age modes combination
In reply to Matan AzradASO age action mode is not supported in group 0 while counter base age action mode supports group 0.
Allow using the 2 modes of age action in parallel, so group 0 flows will use counter base age actions and group > 0 flows will use ASO age actions.
Currently, counter base age action doesn't support shared action API so group 0 flows cannot share age actions.
--- drivers/net/mlx5/mlx5.c | 1 + drivers/net/mlx5/mlx5.h | 6 ++---- drivers/net/mlx5/mlx5_flow_dv.c | 43 ++++++++++++++++++++++------------------- 3 files changed, 26 insertions(+), 24 deletions(-)
drivers/net/mlx5/mlx5.c +1 -0
@@ -438,6 +438,7 @@ static LIST_HEAD(, mlx5_dev_ctx_shared) mlx5_dev_ctx_list = age_info = &sh->port[i].age_info; age_info->flags =0 ;TAILQ_INIT (&age_info->aged_counters); +LIST_INIT (&age_info->aged_aso);rte_spinlock_init (&age_info->aged_sl);MLX5_AGE_SET (age_info,MLX5_AGE_TRIGGER ); }
drivers/net/mlx5/mlx5.h +2 -4
@@ -565,10 +565,8 @@ struct mlx5_aso_age_mng {/* Aging information for per port. */ struct mlx5_age_info {uint8_t flags;/* Indicate if is new event or need to be triggered. */ -union { -struct mlx5_counters aged_counters;/* Aged counter list. */ -struct aso_age_list aged_aso;/* Aged ASO actions list. */ - }; +struct mlx5_counters aged_counters;/* Aged counter list. */ +struct aso_age_list aged_aso;/* Aged ASO actions list. */ rte_spinlock_t aged_sl;/* Aged flow list lock. */ };
drivers/net/mlx5/mlx5_flow_dv.c +23 -20
69 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Hi,
Quoted 59 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Matan Azrad
Sent: Sunday, November 1, 2020 7:58 PM
To: Slava Ovsiienko <[email protected]>
Cc: [email protected]
Subject: [dpdk-dev] [PATCH v2 0/9] net/mlx5: support flow hit steering actionA new ASO (Advanced Steering Operation) feature was added in the last mlx5 adapters to support flow hit detection.
Using this new steering action, the driver can detect flow traffic hit and to reset this indication any time.
Add support for flow aging action in
rte_flowusing this new feature.The counter aging mode will be taken only when the ASO feature is not supported.
v2:
- rebase.
- fix refcnt issue with rss action.
- allow group 0 use counter base age action.
- cosmetics.Dekel Peled (6):
common/mlx5: add DevX API to create ASO flow hit object common/mlx5: use general object type for cap index common/mlx5: add read ASO flow hit HCA capability common/mlx5: add glue func create flow hit action common/mlx5: add definitions for ASO flow hit net/mlx5: support flow hit action for agingMatan Azrad (3):
net/mlx5: optimize shared RSS action memory
net/mlx5: support shared age action
net/mlx5: allow age modes combinationdrivers/common/mlx5/linux/meson.build | 4 + drivers/common/mlx5/linux/mlx5_glue.c | 16 + drivers/common/mlx5/linux/mlx5_glue.h | 3 + drivers/common/mlx5/mlx5_devx_cmds.c | 48 +++ drivers/common/mlx5/mlx5_devx_cmds.h | 5 + drivers/common/mlx5/mlx5_prm.h | 98 ++++- drivers/common/mlx5/version.map | 1 + drivers/net/mlx5/linux/mlx5_os.c | 11 + drivers/net/mlx5/meson.build | 1 + drivers/net/mlx5/mlx5.c | 102 ++++++ drivers/net/mlx5/mlx5.h | 102 +++++- drivers/net/mlx5/mlx5_defs.h | 2 +- drivers/net/mlx5/mlx5_flow.c | 192 ++++++---- drivers/net/mlx5/mlx5_flow.h | 36 +- drivers/net/mlx5/mlx5_flow_age.c | 667 ++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_flow_dv.c | 592 ++++++++++++++++++++++++-- ---- 16 files changed, 1672 insertions(+), 208 deletions(-) create mode 100644 drivers/net/mlx5/mlx5_flow_age.c
Quoted 2 lines
--
1.8.3.1
Series applied to next-net-mlx,
Kindest regards,
Raslan Darawsheh