net/mlx5: HWS misc enhancements
Subsystems (4)
Tariq Toukan
maintainerv2 cover net/mlx5: HWS misc enhancements
Hi,
This series by Yevgeny brings several misc improvements to the mlx5 HWS driver.
Regards,
Tariq
V2:
- Rebased.
V1:
https://lore.kernel.org/all/[email protected]/
Yevgeny Kliteynik (5):
net/mlx5: HWS, Print more details for bad completion net/mlx5: HWS, Log syndrome on STC modify failure net/mlx5: HWS, Set the num of queues only when alloc succeeded net/mlx5: HWS, Remove redundant MLX5_SET in RTC creation net/mlx5: HWS, Remove redundant FW command when reading caps
.../mellanox/mlx5/core/steering/hws/cmd.c | 15 ++-- .../mellanox/mlx5/core/steering/hws/send.c | 72 +++++++++++++------ 2 files changed, 56 insertions(+), 31 deletions(-)
base-commit: 001b5d347d8ba39b2dccaefcc57967b18caec8fe
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH v2 1/5 net/mlx5: HWS, Print more details for bad completion
From: Yevgeny Kliteynik <[email protected]>
When polling for completion returned completion with error, parse some more details: QP number and WQE count. Also, extract all the long value-to-string if conditions to a short value-to-string functions: do it for rule resize state, rule status, and syndrome.
--- .../mellanox/mlx5/core/steering/hws/send.c | 68 +++++++++++++------ 1 file changed, 49 insertions(+), 19 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c +49 -19
@@ -344,6 +344,36 @@ hws_send_engine_update_rule_resize(struct mlx5hws_send_engine *queue, } } +static const char *hws_rule_status_to_string (enum mlx5hws_rule_status status) +{ +switch (status) { +case MLX5HWS_RULE_STATUS_CREATING :return "CREATING" ; +case MLX5HWS_RULE_STATUS_UPDATING :return "UPDATING" ; +case MLX5HWS_RULE_STATUS_DELETING :return "DELETING" ; +case MLX5HWS_RULE_STATUS_FAILING :return "FAILING" ; +default :return "NA" ; + } +} + +static const char *hws_rule_resize_state_to_string (u8 state) +{ +switch (state) { +case MLX5HWS_RULE_RESIZE_STATE_IDLE :return "IDLE" ; +case MLX5HWS_RULE_RESIZE_STATE_WRITING :return "WRITING" ; +case MLX5HWS_RULE_RESIZE_STATE_DELETING :return "DELETING" ; +default :return "UNKNOWN" ; + } +} + +static const char *hws_gta_syndrome_to_string (u8 syndrome) +{ +switch (syndrome) { +case 1 :return "SET_FLOW_FAIL" ; +case 2 :return "DISABLE_FLOW_FAIL" ; +default :return "UNKNOWN" ; + } +} +static void hws_send_engine_dump_error_cqe (struct mlx5hws_send_engine *queue,struct mlx5hws_send_ring_priv *priv,struct mlx5_cqe64 *cqe)
@@ -352,6 +382,7 @@ static void hws_send_engine_dump_error_cqe(struct mlx5hws_send_engine *queue,struct mlx5hws_context *ctx = priv->rule->matcher->tbl->ctx;u32 opcode = cqe ?get_cqe_opcode (cqe) :0 ;struct mlx5hws_rule *rule = priv->rule; +u8 syndrome;/* If something bad happens and lots of rules are failing, we don't * want to pollute dmesg. Print only the first bad cqe per engine,
@@ -364,26 +395,17 @@ static void hws_send_engine_dump_error_cqe(struct mlx5hws_send_engine *queue,if (mlx5hws_rule_move_in_progress (rule))mlx5hws_err (ctx, -"--- rule 0x%08llx: error completion moving rule: phase %s, wqes left %d\n" , +"--- rule 0x%08llx: error completion moving rule: phase %s (%d), wqes left %d\n" ,HWS_PTR_TO_ID (rule), - rule->resize_info->state == -MLX5HWS_RULE_RESIZE_STATE_WRITING ?"WRITING" : - rule->resize_info->state == -MLX5HWS_RULE_RESIZE_STATE_DELETING ?"DELETING" : -"UNKNOWN" , +hws_rule_resize_state_to_string + (rule->resize_info->state), + rule->resize_info->state, rule->pending_wqes);else mlx5hws_err (ctx,"--- rule 0x%08llx: error completion %s (%d), wqes left %d\n" ,HWS_PTR_TO_ID (rule), - rule->status == -MLX5HWS_RULE_STATUS_CREATING ?"CREATING" : - rule->status == -MLX5HWS_RULE_STATUS_DELETING ?"DELETING" : - rule->status == -MLX5HWS_RULE_STATUS_FAILING ?"FAILING" : - rule->status == -MLX5HWS_RULE_STATUS_UPDATING ?"UPDATING" :"NA" , +hws_rule_status_to_string (rule->status), rule->status, rule->pending_wqes);
@@ -423,6 +445,15 @@ static void hws_send_engine_dump_error_cqe(struct mlx5hws_send_engine *queue," rule 0x%08llx: |--- syndrome = 0x%x\n" ,HWS_PTR_TO_ID (rule), err_cqe->syndrome); +mlx5hws_err (ctx, +" rule 0x%08llx: |--- QPN = 0x%x\n" , +HWS_PTR_TO_ID (rule), +be32_to_cpu (err_cqe->s_wqe_opcode_qpn) & +0xffffff ); +mlx5hws_err (ctx, +" rule 0x%08llx: |--- WQE_CNT = 0x%04x\n" , +HWS_PTR_TO_ID (rule), + (u32 )be16_to_cpu (err_cqe->wqe_counter)); } mlx5hws_err(ctx,
@@ -433,13 +464,12 @@ static void hws_send_engine_dump_error_cqe(struct mlx5hws_send_engine *queue,HWS_PTR_TO_ID (rule), (be32_to_cpu (cqe->byte_cnt) &0x80000000 ) ? "FAILURE" : "SUCCESS"); +/* syndrome is in the lower 2 bits of byte_cnt */ + syndrome =be32_to_cpu (cqe->byte_cnt) &3 ;mlx5hws_err (ctx, -" rule 0x%08llx: |------- SYNDROME = %s\n" , +" rule 0x%08llx: |------- SYNDROME = %s (%u)\n" ,HWS_PTR_TO_ID (rule), - ((be32_to_cpu (cqe->byte_cnt) &0x00000003 ) ==1 ) ? -"SET_FLOW_FAIL" : - ((be32_to_cpu (cqe->byte_cnt) &0x00000003 ) ==2 ) ? -"DISABLE_FLOW_FAIL" :"UNKNOWN" ); +hws_gta_syndrome_to_string (syndrome), syndrome);mlx5hws_err (ctx," rule 0x%08llx: cqe->sop_drop_qpn = 0x%08x\n" ,HWS_PTR_TO_ID (rule),be32_to_cpu (cqe->sop_drop_qpn));
Signature · 1 line
2.44.0
Simon Horman
reviewerRe: [PATCH net-next V2 1/5] net/mlx5: HWS, Print more details for bad completion
Quoted 11 lines · Tariq Toukan
From: Yevgeny Kliteynik <[email protected]>
When polling for completion returned completion with error, parse some more details: QP number and WQE count. Also, extract all the long value-to-string if conditions to a short value-to-string functions: do it for rule resize state, rule status, and syndrome.
Signed-off-by Yevgeny Kliteynik <[email protected]>Reviewed-by Erez Shitrit <[email protected]>Signed-off-by Tariq Toukan <[email protected]>
Overall this series looks good to me.
But the AI-generated review of this patch, pasted below, does seem relevant.
...
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c static void hws_send_engine_dump_error_cqe(struct mlx5hws_send_engine *queue, · line 452
Quoted 9 lines
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c +5
-0@@ -423,6 +445,15 @@ static void hws_send_engine_dump_error_cqe(struct mlx5hws_send_engine *queue," rule 0x%08llx: |--- syndrome = 0x%x\n" ,HWS_PTR_TO_ID (rule), err_cqe->syndrome); +mlx5hws_err (ctx, +" rule 0x%08llx: |--- QPN = 0x%x\n" , +HWS_PTR_TO_ID (rule), +be32_to_cpu (err_cqe->s_wqe_opcode_qpn) & +0xffffff );
AI-generated review text:
Is this new QPN line printing something that the same function already dumps a few lines later?
At the tail of hws_send_engine_dump_error_cqe() there is already:
mlx5hws_err(ctx, " rule 0x%08llx: |------------ qpn = 0x%06x\n", HWS_PTR_TO_ID(rule), be32_to_cpu(cqe->sop_drop_qpn) & 0xffffff);
err_cqe is just a cast of the same cqe pointer, and both fields land on the same four bytes of the 64 byte CQE (include/linux/mlx5/device.h):
struct mlx5_err_cqe {
u8 rsvd0[32]; /* 0..31 */
__be32 srqn; /* 32..35 */
u8 rsvd1[18]; /* 36..53 */
u8 vendor_err_synd;/* 54 */
u8 syndrome; /* 55 */
__be32 s_wqe_opcode_qpn;/* 56..59 */
...
};
struct mlx5_cqe64 {
...
__be32 byte_cnt; /* 44..47 */
__be32 timestamp_h; /* 48..51 */
__be32 timestamp_l; /* 52..55 */
__be32 sop_drop_qpn; /* 56..59 */
...
}; Same offset, same 0xffffff mask, so a single dump now reports the QP number twice, once as "QPN = 0x%x" and once as "qpn = 0x%06x", with different labels and widths.
Would it make sense to drop the new QPN line, or at least align its label and width with the existing one, given the one-shot queue->error_cqe_printed guard is there to keep this dump compact?
Source: https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260810092630.3137666-1-tariqt%40nvidia.com
Quoted 7 lines
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c +4
-0+mlx5hws_err (ctx, +" rule 0x%08llx: |--- WQE_CNT = 0x%04x\n" , +HWS_PTR_TO_ID (rule), + (u32 )be16_to_cpu (err_cqe->wqe_counter)); } mlx5hws_err(ctx,
...
Tariq Toukan
maintainerRe: [PATCH net-next V2 1/5] net/mlx5: HWS, Print more details for bad completion
Quoted 9 lines · Simon Horman
Quoted 8 lines · Tariq Toukan
From: Yevgeny Kliteynik <[email protected]>
When polling for completion returned completion with error, parse some more details: QP number and WQE count. Also, extract all the long value-to-string if conditions to a short value-to-string functions: do it for rule resize state, rule status, and syndrome.
Quoted 7 lines
Quoted 3 lines
Signed-off-by Yevgeny Kliteynik <[email protected]>Reviewed-by Erez Shitrit <[email protected]>Signed-off-by Tariq Toukan <[email protected]>Overall this series looks good to me.
But the AI-generated review of this patch, pasted below, does seem relevant.
Thanks.
Yeah it seems relevant indeed.
We'll handle.
Quoted 65 lines
...
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c static void hws_send_engine_dump_error_cqe(struct mlx5hws_send_engine *queue, · line 452
Quoted 9 lines
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c +5
-0@@ -423,6 +445,15 @@ static void hws_send_engine_dump_error_cqe(struct mlx5hws_send_engine *queue," rule 0x%08llx: |--- syndrome = 0x%x\n" ,HWS_PTR_TO_ID (rule), err_cqe->syndrome); +mlx5hws_err (ctx, +" rule 0x%08llx: |--- QPN = 0x%x\n" , +HWS_PTR_TO_ID (rule), +be32_to_cpu (err_cqe->s_wqe_opcode_qpn) & +0xffffff );AI-generated review text:
Is this new QPN line printing something that the same function already dumps a few lines later? At the tail of hws_send_engine_dump_error_cqe() there is already: mlx5hws_err(ctx, " rule 0x%08llx: |------------ qpn = 0x%06x\n", HWS_PTR_TO_ID(rule), be32_to_cpu(cqe->sop_drop_qpn) & 0xffffff); err_cqe is just a cast of the same cqe pointer, and both fields land on the same four bytes of the 64 byte CQE (include/linux/mlx5/device.h): struct mlx5_err_cqe { u8 rsvd0[32]; /* 0..31 */ __be32 srqn; /* 32..35 */ u8 rsvd1[18]; /* 36..53 */ u8 vendor_err_synd;/* 54 */ u8 syndrome; /* 55 */ __be32 s_wqe_opcode_qpn;/* 56..59 */ ... }; struct mlx5_cqe64 { ... __be32 byte_cnt; /* 44..47 */ __be32 timestamp_h; /* 48..51 */ __be32 timestamp_l; /* 52..55 */ __be32 sop_drop_qpn; /* 56..59 */ ... }; Same offset, same 0xffffff mask, so a single dump now reports the QP number twice, once as "QPN = 0x%x" and once as "qpn = 0x%06x", with different labels and widths. Would it make sense to drop the new QPN line, or at least align its label and width with the existing one, given the one-shot queue->error_cqe_printed guard is there to keep this dump compact?Source: https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260810092630.3137666-1-tariqt%40nvidia.com
Quoted 7 lines
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c +4
-0+mlx5hws_err (ctx, +" rule 0x%08llx: |--- WQE_CNT = 0x%04x\n" , +HWS_PTR_TO_ID (rule), + (u32 )be16_to_cpu (err_cqe->wqe_counter)); } mlx5hws_err(ctx,...
Tariq Toukan
maintainerPATCH v2 2/5 net/mlx5: HWS, Log syndrome on STC modify failure
In reply to Tariq ToukanFrom: Yevgeny Kliteynik <[email protected]>
When mlx5_cmd_exec fails for STC modify, include the command syndrome from the output buffer in the error message to aid debugging.
--- .../net/ethernet/mellanox/mlx5/core/steering/hws/cmd.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.c +5 -3
@@ -611,9 +611,11 @@ int mlx5hws_cmd_stc_modify(struct mlx5_core_dev *mdev, ret =mlx5_cmd_exec (mdev, in,sizeof (in), out,sizeof (out));if (ret) -mlx5_core_err (mdev,"Failed to modify STC FW action_type %d\n" , - stc_attr->action_type); - +mlx5_core_err (mdev, +"Failed to modify STC action_type %d, err %d, syndrome 0x%x\n" , + stc_attr->action_type, ret, +MLX5_GET (general_obj_out_cmd_hdr, + out, syndrome));return ret; }
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH v2 3/5 net/mlx5: HWS, Set the num of queues only when alloc succeeded
In reply to Tariq ToukanFrom: Yevgeny Kliteynik <[email protected]>
When initializing send queues, set the number of queues only when allocations are over.
--- drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c +2 -2
@@ -1121,8 +1121,6 @@ static int hws_bwc_send_queues_init(struct mlx5hws_context *ctx)if (!mlx5hws_context_bwc_supported (ctx))return 0 ; - ctx->queues += bwc_queues; - ctx->bwc_send_queue_locks =kzalloc_objs (*ctx->bwc_send_queue_locks, bwc_queues);
@@ -1134,6 +1132,8 @@ static int hws_bwc_send_queues_init(struct mlx5hws_context *ctx)if (!ctx->bwc_lock_class_keys) goto err_lock_class_keys; + ctx->queues += bwc_queues; +for (i =0 ; i < bwc_queues; i++) {mutex_init (&ctx->bwc_send_queue_locks[i]);lockdep_register_key (ctx->bwc_lock_class_keys + i);
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH v2 4/5 net/mlx5: HWS, Remove redundant MLX5_SET in RTC creation
In reply to Tariq ToukanFrom: Yevgeny Kliteynik <[email protected]>
Remove duplicated setting of field in mlx5hws_cmd_rtc_create().
--- drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.c | 1 - 1 file changed, 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.c +0 -1
@@ -398,7 +398,6 @@ int mlx5hws_cmd_rtc_create(struct mlx5_core_dev *mdev,MLX5_SET (rtc, attr, update_method, rtc_attr->fw_gen_wqe);MLX5_SET (rtc, attr, update_index_mode, rtc_attr->update_index_mode);MLX5_SET (rtc, attr, access_index_mode, rtc_attr->access_index_mode); -MLX5_SET (rtc, attr, num_hash_definer, rtc_attr->num_hash_definer);MLX5_SET (rtc, attr, log_depth, rtc_attr->log_depth);MLX5_SET (rtc, attr, log_hash_size, rtc_attr->log_size);MLX5_SET (rtc, attr, table_type, rtc_attr->table_type);
Signature · 1 line
2.44.0
Tariq Toukan
maintainerPATCH v2 5/5 net/mlx5: HWS, Remove redundant FW command when reading caps
In reply to Tariq ToukanFrom: Yevgeny Kliteynik <[email protected]>
Remove redundant FW query that isn't really in use.
--- drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.c | 6 ------ 1 file changed, 6 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.c +0 -6
@@ -1182,12 +1182,6 @@ int mlx5hws_cmd_query_caps(struct mlx5_core_dev *mdev, capability.e_switch_cap.merged_eswitch); } - ret =mlx5_cmd_exec (mdev, in,sizeof (in), out, out_size); -if (ret) { -mlx5_core_err (mdev,"Failed to query device attributes\n" ); - goto out; - } -snprintf (caps->fw_ver,sizeof (caps->fw_ver),"%d.%d.%d" ,fw_rev_maj (mdev),fw_rev_min (mdev),fw_rev_sub (mdev));
Signature · 1 line
2.44.0