mlx5 HWS fixes 2025-08-17
Subsystems (4)
Mark Bloch
maintainercover mlx5 HWS fixes 2025-08-17
Hi,
The following patch set focuses on hardware steering fixes found by the team.
Alex Vesker (1):
net/mlx5: HWS, Fix table creation UID
Vlad Dogaru (1):
net/mlx5: CT: Use the correct counter offset
Yevgeny Kliteynik (5):
net/mlx5: HWS, fix bad parameter in CQ creation net/mlx5: HWS, fix simple rules rehash error flow net/mlx5: HWS, fix complex rules rehash error flow net/mlx5: HWS, prevent rehash from filling up the queues net/mlx5: HWS, don't rehash on every kind of insertion failure
.../mellanox/mlx5/core/en/tc/ct_fs_hmfs.c | 2 + .../mellanox/mlx5/core/steering/hws/bwc.c | 81 ++++++++++++++----- .../mlx5/core/steering/hws/bwc_complex.c | 41 +++++++--- .../mellanox/mlx5/core/steering/hws/cmd.c | 1 + .../mellanox/mlx5/core/steering/hws/cmd.h | 1 + .../mellanox/mlx5/core/steering/hws/fs_hws.c | 1 + .../mellanox/mlx5/core/steering/hws/matcher.c | 5 +- .../mellanox/mlx5/core/steering/hws/mlx5hws.h | 1 + .../mellanox/mlx5/core/steering/hws/send.c | 1 - .../mellanox/mlx5/core/steering/hws/table.c | 13 ++- .../mellanox/mlx5/core/steering/hws/table.h | 3 +- 11 files changed, 112 insertions(+), 38 deletions(-)
base-commit: 715c7a36d59f54162a26fac1d1ed8dc087a24cf1
Signature · 1 line
2.34.1
Mark Bloch
maintainerPATCH 1/7 net/mlx5: HWS, fix bad parameter in CQ creation
From: Yevgeny Kliteynik <[email protected]>
'cqe_sz' valid value should be 0 for 64-byte CQE.
--- drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c | 1 - 1 file changed, 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/send.c +0 -1
@@ -964,7 +964,6 @@ static int hws_send_ring_open_cq(struct mlx5_core_dev *mdev,return -ENOMEM ;MLX5_SET (cqc, cqc_data, uar_page, mdev->priv.uar->index); -MLX5_SET (cqc, cqc_data, cqe_sz, queue->num_entries);MLX5_SET (cqc, cqc_data, log_cq_size,ilog2 (queue->num_entries)); err =hws_send_ring_alloc_cq (mdev, numa_node, queue, cqc_data, cq);
Signature · 1 line
2.34.1
Mark Bloch
maintainerPATCH 2/7 net/mlx5: HWS, fix simple rules rehash error flow
In reply to Mark BlochFrom: Yevgeny Kliteynik <[email protected]>
Moving rules from matcher to matcher should not fail. However, if it does fail due to various reasons, the error flow should allow the kernel to continue functioning (albeit with broken steering rules) instead of going into series of soft lock-ups or some other problematic behaviour.
This patch fixes the error flow for moving simple rules:
- If new rule creation fails before it was even enqeued, do not poll for completion
- If TIMEOUT happened while moving the rule, no point trying to poll for completions for other rules. Something is broken, completion won't come, just abort the rehash sequence.
- If some other completion with error received, don't give up. Continue handling rest of the rules to minimize the damage.
- Make sure that the first error code that was received will be actually returned to the caller instead of replacing it with the generic error code.
All the aforementioned issues stem from the same bad error flow, so no point fixing them one by one and leaving partially broken code - fixing them in one patch.
--- .../mellanox/mlx5/core/steering/hws/bwc.c | 61 +++++++++++++------ 1 file changed, 43 insertions(+), 18 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c +43 -18
@@ -74,9 +74,9 @@ static void hws_bwc_matcher_init_attr(struct mlx5hws_bwc_matcher *bwc_matcher,static int hws_bwc_matcher_move_all_simple (struct mlx5hws_bwc_matcher *bwc_matcher) { -bool move_error = false, poll_error = false, drain_error = false;struct mlx5hws_context *ctx = bwc_matcher->matcher->tbl->ctx;struct mlx5hws_matcher *matcher = bwc_matcher->matcher; +int drain_error =0 , move_error =0 , poll_error =0 ;u16 bwc_queues =mlx5hws_bwc_queues (ctx);struct mlx5hws_rule_attr rule_attr;struct mlx5hws_bwc_rule *bwc_rule;
@@ -99,11 +99,15 @@ hws_bwc_matcher_move_all_simple(struct mlx5hws_bwc_matcher *bwc_matcher) ret =mlx5hws_matcher_resize_rule_move (matcher, bwc_rule->rule, &rule_attr); -if (unlikely (ret && !move_error)) { -mlx5hws_err (ctx, -"Moving BWC rule: move failed (%d), attempting to move rest of the rules\n" , - ret); - move_error = true; +if (unlikely (ret)) { +if (!move_error) { +mlx5hws_err (ctx, +"Moving BWC rule: move failed (%d), attempting to move rest of the rules\n" , + ret); + move_error = ret; + } +/* Rule wasn't queued, no need to poll */ +continue ; } pending_rules++;
@@ -111,11 +115,19 @@ hws_bwc_matcher_move_all_simple(struct mlx5hws_bwc_matcher *bwc_matcher) rule_attr.queue_id, &pending_rules, false); -if (unlikely (ret && !poll_error)) { -mlx5hws_err (ctx, -"Moving BWC rule: poll failed (%d), attempting to move rest of the rules\n" , - ret); - poll_error = true; +if (unlikely (ret)) { +if (ret == -ETIMEDOUT ) { +mlx5hws_err (ctx, +"Moving BWC rule: timeout polling for completions (%d), aborting rehash\n" , + ret); +return ret; + } +if (!poll_error) { +mlx5hws_err (ctx, +"Moving BWC rule: polling for completions failed (%d), attempting to move rest of the rules\n" , + ret); + poll_error = ret; + } } }
@@ -126,17 +138,30 @@ hws_bwc_matcher_move_all_simple(struct mlx5hws_bwc_matcher *bwc_matcher) rule_attr.queue_id, &pending_rules, true); -if (unlikely (ret && !drain_error)) { -mlx5hws_err (ctx, -"Moving BWC rule: drain failed (%d), attempting to move rest of the rules\n" , - ret); - drain_error = true; +if (unlikely (ret)) { +if (ret == -ETIMEDOUT ) { +mlx5hws_err (ctx, +"Moving bwc rule: timeout draining completions (%d), aborting rehash\n" , + ret); +return ret; + } +if (!drain_error) { +mlx5hws_err (ctx, +"Moving bwc rule: drain failed (%d), attempting to move rest of the rules\n" , + ret); + drain_error = ret; + } } } } -if (move_error || poll_error || drain_error) - ret = -EINVAL ; +/* Return the first error that happened */ +if (unlikely (move_error)) +return move_error; +if (unlikely (poll_error)) +return poll_error; +if (unlikely (drain_error)) +return drain_error;return ret; }
Signature · 1 line
2.34.1
Mark Bloch
maintainerPATCH 3/7 net/mlx5: HWS, fix complex rules rehash error flow
In reply to Mark BlochFrom: Yevgeny Kliteynik <[email protected]>
Moving rules from matcher to matcher should not fail. However, if it does fail due to various reasons, the error flow should allow the kernel to continue functioning (albeit with broken steering rules) instead of going into series of soft lock-ups or some other problematic behaviour.
Similar to the simple rules, complex rules rehash logic suffers from the same problems. This patch fixes the error flow for moving complex rules:
- If new rule creation fails before it was even enqeued, do not poll for completion
- If TIMEOUT happened while moving the rule, no point trying to poll for completions for other rules. Something is broken, completion won't come, just abort the rehash sequence.
- If some other completion with error received, don't give up. Continue handling rest of the rules to minimize the damage.
- Make sure that the first error code that was received will be actually returned to the caller instead of replacing it with the generic error code.
All the aforementioned issues stem from the same bad error flow, so no point fixing them one by one and leaving partially broken code - fixing them in one patch.
--- .../mlx5/core/steering/hws/bwc_complex.c | 41 +++++++++++++------ 1 file changed, 28 insertions(+), 13 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc_complex.c +28 -13
@@ -1328,11 +1328,11 @@ mlx5hws_bwc_matcher_move_all_complex(struct mlx5hws_bwc_matcher *bwc_matcher) {struct mlx5hws_context *ctx = bwc_matcher->matcher->tbl->ctx;struct mlx5hws_matcher *matcher = bwc_matcher->matcher; -bool move_error = false, poll_error = false;u16 bwc_queues =mlx5hws_bwc_queues (ctx);struct mlx5hws_bwc_rule *tmp_bwc_rule;struct mlx5hws_rule_attr rule_attr;struct mlx5hws_table *isolated_tbl; +int move_error =0 , poll_error =0 ;struct mlx5hws_rule *tmp_rule;struct list_head *rules_list;u32 expected_completions =1 ;
@@ -1391,11 +1391,15 @@ mlx5hws_bwc_matcher_move_all_complex(struct mlx5hws_bwc_matcher *bwc_matcher) ret =mlx5hws_matcher_resize_rule_move (matcher, tmp_rule, &rule_attr); -if (unlikely (ret && !move_error)) { -mlx5hws_err (ctx, -"Moving complex BWC rule failed (%d), attempting to move rest of the rules\n" , - ret); - move_error = true; +if (unlikely (ret)) { +if (!move_error) { +mlx5hws_err (ctx, +"Moving complex BWC rule: move failed (%d), attempting to move rest of the rules\n" , + ret); + move_error = ret; + } +/* Rule wasn't queued, no need to poll */ +continue ; } expected_completions =1 ;
@@ -1403,11 +1407,19 @@ mlx5hws_bwc_matcher_move_all_complex(struct mlx5hws_bwc_matcher *bwc_matcher) rule_attr.queue_id, &expected_completions, true); -if (unlikely (ret && !poll_error)) { -mlx5hws_err (ctx, -"Moving complex BWC rule: poll failed (%d), attempting to move rest of the rules\n" , - ret); - poll_error = true; +if (unlikely (ret)) { +if (ret == -ETIMEDOUT ) { +mlx5hws_err (ctx, +"Moving complex BWC rule: timeout polling for completions (%d), aborting rehash\n" , + ret); +return ret; + } +if (!poll_error) { +mlx5hws_err (ctx, +"Moving complex BWC rule: polling for completions failed (%d), attempting to move rest of the rules\n" , + ret); + poll_error = ret; + } }/* Done moving the rule to the new matcher,
@@ -1422,8 +1434,11 @@ mlx5hws_bwc_matcher_move_all_complex(struct mlx5hws_bwc_matcher *bwc_matcher) } } -if (move_error || poll_error) - ret = -EINVAL ; +/* Return the first error that happened */ +if (unlikely (move_error)) +return move_error; +if (unlikely (poll_error)) +return poll_error;return ret; }
Signature · 1 line
2.34.1
Mark Bloch
maintainerPATCH 4/7 net/mlx5: HWS, prevent rehash from filling up the queues
In reply to Mark BlochFrom: Yevgeny Kliteynik <[email protected]>
While moving the rules during rehash, CQ is not drained. The flush and drain happens only when all the rules of a certain queue have been moved. This behaviour can lead to accumulating large quantity of rules that haven't got their completion yet, and eventually will fill up the queue and will cause the rehash to fail.
Fix this problem by requiring drain once the number of outstanding completions reaches a certain threshold.
--- drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c +4 -1
@@ -84,6 +84,7 @@ hws_bwc_matcher_move_all_simple(struct mlx5hws_bwc_matcher *bwc_matcher)struct list_head *rules_list;u32 pending_rules;int i, ret =0 ; +bool drain;mlx5hws_bwc_rule_fill_attr (bwc_matcher,0 ,0 , &rule_attr);
@@ -111,10 +112,12 @@ hws_bwc_matcher_move_all_simple(struct mlx5hws_bwc_matcher *bwc_matcher) } pending_rules++; + drain = pending_rules >= +hws_bwc_get_burst_th (ctx, rule_attr.queue_id); ret =mlx5hws_bwc_queue_poll (ctx, rule_attr.queue_id, &pending_rules, - false); + drain);if (unlikely (ret)) {if (ret == -ETIMEDOUT ) { mlx5hws_err(ctx,
Signature · 1 line
2.34.1
Mark Bloch
maintainerPATCH 5/7 net/mlx5: HWS, don't rehash on every kind of insertion failure
In reply to Mark BlochFrom: Yevgeny Kliteynik <[email protected]>
If rule creation failed due to a full queue, due to timeout in polling for completion, or due to matcher being in resize, don't try to initiate rehash sequence - rehash would have failed anyway.
--- .../mellanox/mlx5/core/steering/hws/bwc.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/bwc.c +15 -0
@@ -1063,6 +1063,21 @@ int mlx5hws_bwc_rule_create_simple(struct mlx5hws_bwc_rule *bwc_rule,return 0 ;/* rule inserted successfully */ } +/* Rule insertion could fail due to queue being full, timeout, or +* matcher in resize. In such cases, no point in trying to rehash. +*/ +if (ret == -EBUSY || ret == -ETIMEDOUT || ret == -EAGAIN ) { +mutex_unlock (queue_lock); +mlx5hws_err (ctx, +"BWC rule insertion failed - %s (%d)\n" , + ret == -EBUSY ?"queue is full" : + ret == -ETIMEDOUT ?"timeout" : + ret == -EAGAIN ?"matcher in resize" :"N/A" , + ret); +hws_bwc_rule_cnt_dec (bwc_rule); +return ret; + } +/* At this point the rule wasn't added. * It could be because there was collision, or some other problem. * Try rehash by size and insert rule again - last chance.
Signature · 1 line
2.34.1
Mark Bloch
maintainerPATCH 6/7 net/mlx5: HWS, Fix table creation UID
In reply to Mark BlochFrom: Alex Vesker <[email protected]>
During table creation, caller passes a UID using ft_attr. The UID value was ignored, which leads to problems when the caller sets the UID to a non-zero value, such as SHARED_RESOURCE_UID (0xffff) - the internal FT objects will be created with UID=0.
--- .../ethernet/mellanox/mlx5/core/steering/hws/cmd.c | 1 + .../ethernet/mellanox/mlx5/core/steering/hws/cmd.h | 1 + .../mellanox/mlx5/core/steering/hws/fs_hws.c | 1 + .../mellanox/mlx5/core/steering/hws/matcher.c | 5 ++++- .../mellanox/mlx5/core/steering/hws/mlx5hws.h | 1 + .../mellanox/mlx5/core/steering/hws/table.c | 13 ++++++++++--- .../mellanox/mlx5/core/steering/hws/table.h | 3 ++- 7 files changed, 20 insertions(+), 5 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.c +1 -0
@@ -55,6 +55,7 @@ int mlx5hws_cmd_flow_table_create(struct mlx5_core_dev *mdev,MLX5_SET (create_flow_table_in, in, opcode,MLX5_CMD_OP_CREATE_FLOW_TABLE );MLX5_SET (create_flow_table_in, in, table_type, ft_attr->type); +MLX5_SET (create_flow_table_in, in, uid, ft_attr->uid); ft_ctx =MLX5_ADDR_OF (create_flow_table_in, in, flow_table_context);MLX5_SET (flow_table_context, ft_ctx, level, ft_attr->level);
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/cmd.h +1 -0
@@ -36,6 +36,7 @@ struct mlx5hws_cmd_set_fte_attr {struct mlx5hws_cmd_ft_create_attr {u8 type;u8 level; +u16 uid;bool rtc_valid;bool decap_en;bool reformat_en;
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/fs_hws.c +1 -0
@@ -267,6 +267,7 @@ static int mlx5_cmd_hws_create_flow_table(struct mlx5_flow_root_namespace *ns, tbl_attr.type =MLX5HWS_TABLE_TYPE_FDB ; tbl_attr.level = ft_attr->level; + tbl_attr.uid = ft_attr->uid; tbl =mlx5hws_table_create (ctx, &tbl_attr);if (!tbl) {mlx5_core_err (ns->dev,"Failed creating hws flow_table\n" );
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/matcher.c +4 -1
@@ -85,6 +85,7 @@ static int hws_matcher_create_end_ft_isolated(struct mlx5hws_matcher *matcher) ret =mlx5hws_table_create_default_ft (tbl->ctx->mdev, tbl, +0 , &matcher->end_ft_id);if (ret) {mlx5hws_err (tbl->ctx,"Isolated matcher: failed to create end flow table\n" );
@@ -112,7 +113,9 @@ static int hws_matcher_create_end_ft(struct mlx5hws_matcher *matcher)if (mlx5hws_matcher_is_isolated (matcher)) ret =hws_matcher_create_end_ft_isolated (matcher);else - ret =mlx5hws_table_create_default_ft (tbl->ctx->mdev, tbl, + ret =mlx5hws_table_create_default_ft (tbl->ctx->mdev, + tbl, +0 , &matcher->end_ft_id);if (ret) {
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/mlx5hws.h +1 -0
@@ -75,6 +75,7 @@ struct mlx5hws_context_attr {struct mlx5hws_table_attr {enum mlx5hws_table_type type;u32 level; +u16 uid; };enum mlx5hws_matcher_flow_src {
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/table.c +10 -3
@@ -9,6 +9,7 @@ u32 mlx5hws_table_get_id(struct mlx5hws_table *tbl) }static void hws_table_init_next_ft_attr (struct mlx5hws_table *tbl, +u16 uid,struct mlx5hws_cmd_ft_create_attr *ft_attr) { ft_attr->type = tbl->fw_ft_type;
@@ -16,7 +17,9 @@ static void hws_table_init_next_ft_attr(struct mlx5hws_table *tbl, ft_attr->level = tbl->ctx->caps->fdb_ft.max_level -1 ; else ft_attr->level = tbl->ctx->caps->nic_ft.max_level - 1; + ft_attr->rtc_valid = true; + ft_attr->uid = uid; } static void hws_table_set_cap_attr(struct mlx5hws_table *tbl,
@@ -119,12 +122,12 @@ static int hws_table_connect_to_default_miss_tbl(struct mlx5hws_table *tbl, u32int mlx5hws_table_create_default_ft (struct mlx5_core_dev *mdev,struct mlx5hws_table *tbl, -u32 *ft_id) +u16 uid,u32 *ft_id) {struct mlx5hws_cmd_ft_create_attr ft_attr = {0 };int ret; -hws_table_init_next_ft_attr (tbl, &ft_attr); +hws_table_init_next_ft_attr (tbl, uid, &ft_attr);hws_table_set_cap_attr (tbl, &ft_attr); ret =mlx5hws_cmd_flow_table_create (mdev, &ft_attr, ft_id);
@@ -189,7 +192,10 @@ static int hws_table_init(struct mlx5hws_table *tbl) }mutex_lock (&ctx->ctrl_lock); - ret =mlx5hws_table_create_default_ft (tbl->ctx->mdev, tbl, &tbl->ft_id); + ret =mlx5hws_table_create_default_ft (tbl->ctx->mdev, + tbl, + tbl->uid, + &tbl->ft_id);if (ret) {mlx5hws_err (tbl->ctx,"Failed to create flow table object\n" );mutex_unlock (&ctx->ctrl_lock);
@@ -239,6 +245,7 @@ struct mlx5hws_table *mlx5hws_table_create(struct mlx5hws_context *ctx, tbl->ctx = ctx; tbl->type = attr->type; tbl->level = attr->level; + tbl->uid = attr->uid; ret =hws_table_init (tbl);if (ret) {
drivers/net/ethernet/mellanox/mlx5/core/steering/hws/table.h +2 -1
@@ -18,6 +18,7 @@ struct mlx5hws_table {enum mlx5hws_table_type type;u32 fw_ft_type;u32 level; +u16 uid;struct list_head matchers_list;struct list_head tbl_list_node;struct mlx5hws_default_miss default_miss;
@@ -47,7 +48,7 @@ u32 mlx5hws_table_get_res_fw_ft_type(enum mlx5hws_table_type tbl_type,int mlx5hws_table_create_default_ft (struct mlx5_core_dev *mdev,struct mlx5hws_table *tbl, -u32 *ft_id); +u16 uid,u32 *ft_id);void mlx5hws_table_destroy_default_ft (struct mlx5hws_table *tbl,u32 ft_id);
Signature · 1 line
2.34.1
Mark Bloch
maintainerPATCH 7/7 net/mlx5: CT: Use the correct counter offset
In reply to Mark BlochFrom: Vlad Dogaru <[email protected]>
Specifying the counter action is not enough, as it is used by multiple counters that were allocated in a bulk. By omitting the offset, rules will be associated with a different counter from the same bulk. Subsequently, the CT subsystem checks the correct counter, assumes that no traffic has triggered the rule, and ages out the rule. The end result is intermittent offloading of long lived connections, as rules are aged out then promptly re-added.
Fix this by specifying the correct offset along with the counter rule.
--- drivers/net/ethernet/mellanox/mlx5/core/en/tc/ct_fs_hmfs.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/en/tc/ct_fs_hmfs.c +2 -0
@@ -173,6 +173,8 @@ static void mlx5_ct_fs_hmfs_fill_rule_actions(struct mlx5_ct_fs_hmfs *fs_hmfs,memset (rule_actions,0 ,NUM_CT_HMFS_RULES *sizeof (*rule_actions)); rule_actions[0 ].action =mlx5_fc_get_hws_action (fs_hmfs->ctx, attr->counter); + rule_actions[0 ].counter.offset = + attr->counter->id - attr->counter->bulk->base_id;/* Modify header is special, it may require extra arguments outside the action itself. */ if (mh_action->mh_data) { rule_actions[1 ].modify_header.offset = mh_action->mh_data->offset;
Signature · 1 line
2.34.1
This series was applied to netdev/net.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This series was applied to netdev/net.git (main)
by Jakub Kicinski <[email protected]>:
Quoted 9 lines · On Sun, 17 Aug 2025 23:23:16 +0300 you wrote:
Hi,
The following patch set focuses on hardware steering fixes found by the team.
Alex Vesker (1):
net/mlx5: HWS, Fix table creation UID[...]
Here is the summary with links:
- [net,1/7] net/mlx5: HWS, fix bad parameter in CQ creation
https://git.kernel.org/netdev/net/c/2462c1b92172 - [net,2/7] net/mlx5: HWS, fix simple rules rehash error flow https://git.kernel.org/netdev/net/c/615b690612b7 - [net,3/7] net/mlx5: HWS, fix complex rules rehash error flow https://git.kernel.org/netdev/net/c/4a842b1bf18a - [net,4/7] net/mlx5: HWS, prevent rehash from filling up the queues https://git.kernel.org/netdev/net/c/1a72298d27ce - [net,5/7] net/mlx5: HWS, don't rehash on every kind of insertion failure https://git.kernel.org/netdev/net/c/7c60952f8358 - [net,6/7] net/mlx5: HWS, Fix table creation UID https://git.kernel.org/netdev/net/c/8a51507320eb - [net,7/7] net/mlx5: CT: Use the correct counter offset https://git.kernel.org/netdev/net/c/d2d6f950cb43
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html