mlx5 fixes 2022-12-28
Subsystems (5)
From: Saeed Mahameed <[email protected]>
This series provides bug fixes to mlx5 driver. Please pull and let me know if there is any problem.
Thanks,
Saeed.
The following changes since commit 40cab44b9089a41f71bbd0eff753eb91d5dafd68:
net/sched: fix retpoline wrapper compilation on configs without tc filters (2022-12-28 12:11:32 +0000)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/mlx5-fixes-2022-12-28
for you to fetch changes up to 4d1c1379d71777ddeda3e54f8fc26e9ecbfd1009:
net/mlx5: Lag, fix failure to cancel delayed bond work (2022-12-28 11:38:51 -0800)
----------------------------------------------------------------
mlx5-fixes-2022-12-28
----------------------------------------------------------------
Adham Faris (1):
net/mlx5e: Fix hw mtu initializing at XDP SQ allocation
Chris Mi (2):
net/mlx5e: CT: Fix ct debugfs folder name net/mlx5e: Always clear dest encap in neigh-update-del
Dragos Tatulea (1):
net/mlx5e: IPoIB, Don't allow CQE compression to be turned on by default
Eli Cohen (1):
net/mlx5: Lag, fix failure to cancel delayed bond work
Jiri Pirko (1):
net/mlx5: Add forgotten cleanup calls into mlx5_init_once() error path
Maor Dickman (1):
net/mlx5e: Set geneve_tlv_option_0_exist when matching on geneve option
Moshe Shemesh (1):
net/mlx5: E-Switch, properly handle ingress tagged packets on VST
Shay Drory (3):
net/mlx5: Fix io_eq_size and event_eq_size params validation net/mlx5: Avoid recovery in probe flows
net/mlx5: Fix RoCE setting at HCA level
Tariq Toukan (1):
net/mlx5e: Fix RX reporter for XSK RQs
drivers/net/ethernet/mellanox/mlx5/core/devlink.c | 4 +-- .../ethernet/mellanox/mlx5/core/en/reporter_rx.c | 6 +++- drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c | 7 +---- .../ethernet/mellanox/mlx5/core/en/tc_tun_encap.c | 9 +++++- .../ethernet/mellanox/mlx5/core/en/tc_tun_geneve.c | 5 ++++ drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- .../mellanox/mlx5/core/esw/acl/egress_lgcy.c | 7 ++++- .../mellanox/mlx5/core/esw/acl/ingress_lgcy.c | 33 ++++++++++++++++++---- drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 30 ++++++++++++++------ drivers/net/ethernet/mellanox/mlx5/core/eswitch.h | 6 ++++ drivers/net/ethernet/mellanox/mlx5/core/health.c | 6 ++++ .../net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c | 4 +++ drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c | 1 + drivers/net/ethernet/mellanox/mlx5/core/main.c | 4 ++- include/linux/mlx5/device.h | 5 ++++ include/linux/mlx5/mlx5_ifc.h | 3 +- 16 files changed, 104 insertions(+), 28 deletions(-)
PATCH 1/12 net/mlx5: E-Switch, properly handle ingress tagged packets on VST
From: Moshe Shemesh <[email protected]>
Fix SRIOV VST mode behavior to insert cvlan when a guest tag is already present in the frame. Previous VST mode behavior was to drop packets or override existing tag, depending on the device version.
In this patch we fix this behavior by correctly building the HW steering rule with a push vlan action, or for older devices we ask the FW to stack the vlan when a vlan is already present.
--- .../mellanox/mlx5/core/esw/acl/egress_lgcy.c | 7 +++- .../mellanox/mlx5/core/esw/acl/ingress_lgcy.c | 33 ++++++++++++++++--- .../net/ethernet/mellanox/mlx5/core/eswitch.c | 30 ++++++++++++----- .../net/ethernet/mellanox/mlx5/core/eswitch.h | 6 ++++ include/linux/mlx5/device.h | 5 +++ include/linux/mlx5/mlx5_ifc.h | 3 +- 6 files changed, 68 insertions(+), 16 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_lgcy.c +6 -1
@@ -67,6 +67,7 @@ static void esw_acl_egress_lgcy_groups_destroy(struct mlx5_vport *vport)int esw_acl_egress_lgcy_setup (struct mlx5_eswitch *esw,struct mlx5_vport *vport) { +bool vst_mode_steering =esw_vst_mode_is_steering (esw);struct mlx5_flow_destination drop_ctr_dst = {};struct mlx5_flow_destination *dst =NULL ;struct mlx5_fc *drop_counter =NULL ;
@@ -77,6 +78,7 @@ int esw_acl_egress_lgcy_setup(struct mlx5_eswitch *esw,*/ int table_size =2 ;int dest_num =0 ; +int actions_flag;int err =0 ;if (vport->egress.legacy.drop_counter) {
@@ -119,8 +121,11 @@ int esw_acl_egress_lgcy_setup(struct mlx5_eswitch *esw, vport->vport, vport->info.vlan, vport->info.qos);/* Allowed vlan rule */ + actions_flag =MLX5_FLOW_CONTEXT_ACTION_ALLOW ; +if (vst_mode_steering) + actions_flag |=MLX5_FLOW_CONTEXT_ACTION_VLAN_POP ; err =esw_egress_acl_vlan_create (esw, vport,NULL , vport->info.vlan, -MLX5_FLOW_CONTEXT_ACTION_ALLOW ); + actions_flag);if (err) goto out;
drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_lgcy.c +28 -5
@@ -139,11 +139,14 @@ static void esw_acl_ingress_lgcy_groups_destroy(struct mlx5_vport *vport)int esw_acl_ingress_lgcy_setup (struct mlx5_eswitch *esw,struct mlx5_vport *vport) { +bool vst_mode_steering =esw_vst_mode_is_steering (esw);struct mlx5_flow_destination drop_ctr_dst = {};struct mlx5_flow_destination *dst =NULL ;struct mlx5_flow_act flow_act = {};struct mlx5_flow_spec *spec =NULL ;struct mlx5_fc *counter =NULL ; +bool vst_check_cvlan = false; +bool vst_push_cvlan = false;/* The ingress acl table contains 4 groups * (2 active rules at the same time - * 1 allow rule from one of the first 3 groups.
@@ -203,7 +206,26 @@ int esw_acl_ingress_lgcy_setup(struct mlx5_eswitch *esw, goto out; } -if (vport->info.vlan || vport->info.qos) +if ((vport->info.vlan || vport->info.qos)) { +if (vst_mode_steering) + vst_push_cvlan = true; +else if (!MLX5_CAP_ESW (esw->dev, vport_cvlan_insert_always)) + vst_check_cvlan = true; + } + +if (vst_check_cvlan || vport->info.spoofchk) + spec->match_criteria_enable =MLX5_MATCH_OUTER_HEADERS ; + +/* Create ingress allow rule */ + flow_act.action =MLX5_FLOW_CONTEXT_ACTION_ALLOW ; +if (vst_push_cvlan) { + flow_act.action |=MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH ; + flow_act.vlan[0 ].prio = vport->info.qos; + flow_act.vlan[0 ].vid = vport->info.vlan; + flow_act.vlan[0 ].ethtype =ETH_P_8021Q ; + } + +if (vst_check_cvlan)MLX5_SET_TO_ONES (fte_match_param, spec->match_criteria, outer_headers.cvlan_tag);
@@ -218,9 +240,6 @@ int esw_acl_ingress_lgcy_setup(struct mlx5_eswitch *esw,ether_addr_copy (smac_v, vport->info.mac); } -/* Create ingress allow rule */ - spec->match_criteria_enable =MLX5_MATCH_OUTER_HEADERS ; - flow_act.action =MLX5_FLOW_CONTEXT_ACTION_ALLOW ; vport->ingress.allow_rule =mlx5_add_flow_rules (vport->ingress.acl, spec, &flow_act,NULL ,0 );if (IS_ERR (vport->ingress.allow_rule)) {
@@ -232,6 +251,9 @@ int esw_acl_ingress_lgcy_setup(struct mlx5_eswitch *esw, goto out; } +if (!vst_check_cvlan && !vport->info.spoofchk) + goto out; +memset (&flow_act,0 ,sizeof (flow_act)); flow_act.action =MLX5_FLOW_CONTEXT_ACTION_DROP ;/* Attach drop flow counter */
@@ -257,7 +279,8 @@ int esw_acl_ingress_lgcy_setup(struct mlx5_eswitch *esw,return 0 ; out: -esw_acl_ingress_lgcy_cleanup (esw, vport); +if (err) +esw_acl_ingress_lgcy_cleanup (esw, vport);kvfree (spec);return err; }
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +21 -9
@@ -161,10 +161,17 @@ static int modify_esw_vport_cvlan(struct mlx5_core_dev *dev, u16 vport, esw_vport_context.vport_cvlan_strip, 1);if (set_flags &SET_VLAN_INSERT ) { -/* insert only if no vlan in packet */ -MLX5_SET (modify_esw_vport_context_in, in, - esw_vport_context.vport_cvlan_insert,1 ); - +if (MLX5_CAP_ESW (dev, vport_cvlan_insert_always)) { +/* insert either if vlan exist in packet or not */ +MLX5_SET (modify_esw_vport_context_in, in, + esw_vport_context.vport_cvlan_insert, +MLX5_VPORT_CVLAN_INSERT_ALWAYS ); + }else { +/* insert only if no vlan in packet */ +MLX5_SET (modify_esw_vport_context_in, in, + esw_vport_context.vport_cvlan_insert, +MLX5_VPORT_CVLAN_INSERT_WHEN_NO_CVLAN ); + }MLX5_SET (modify_esw_vport_context_in, in, esw_vport_context.cvlan_pcp, qos); MLX5_SET(modify_esw_vport_context_in, in,
@@ -809,6 +816,7 @@ static int mlx5_esw_vport_caps_get(struct mlx5_eswitch *esw, struct mlx5_vport *static int esw_vport_setup (struct mlx5_eswitch *esw,struct mlx5_vport *vport) { +bool vst_mode_steering =esw_vst_mode_is_steering (esw);u16 vport_num = vport->vport;int flags;int err;
@@ -839,8 +847,9 @@ static int esw_vport_setup(struct mlx5_eswitch *esw, struct mlx5_vport *vport) flags = (vport->info.vlan || vport->info.qos) ?SET_VLAN_STRIP |SET_VLAN_INSERT :0 ; -modify_esw_vport_cvlan (esw->dev, vport_num, vport->info.vlan, - vport->info.qos, flags); +if (esw->mode ==MLX5_ESWITCH_OFFLOADS || !vst_mode_steering) +modify_esw_vport_cvlan (esw->dev, vport_num, vport->info.vlan, + vport->info.qos, flags);return 0 ;
@@ -1848,6 +1857,7 @@ int __mlx5_eswitch_set_vport_vlan(struct mlx5_eswitch *esw, u16 vport, u16 vlan, u8 qos, u8 set_flags) { struct mlx5_vport *evport = mlx5_eswitch_get_vport(esw, vport); +bool vst_mode_steering =esw_vst_mode_is_steering (esw);int err =0 ;if (IS_ERR (evport))
@@ -1855,9 +1865,11 @@ int __mlx5_eswitch_set_vport_vlan(struct mlx5_eswitch *esw,if (vlan >4095 || qos >7 )return -EINVAL ; - err =modify_esw_vport_cvlan (esw->dev, vport, vlan, qos, set_flags); -if (err) -return err; +if (esw->mode ==MLX5_ESWITCH_OFFLOADS || !vst_mode_steering) { + err =modify_esw_vport_cvlan (esw->dev, vport, vlan, qos, set_flags); +if (err) +return err; + } evport->info.vlan = vlan; evport->info.qos = qos;
drivers/net/ethernet/mellanox/mlx5/core/eswitch.h +6 -0
@@ -527,6 +527,12 @@ int mlx5_eswitch_del_vlan_action(struct mlx5_eswitch *esw,int __mlx5_eswitch_set_vport_vlan (struct mlx5_eswitch *esw,u16 vport,u16 vlan,u8 qos,u8 set_flags); +static inline bool esw_vst_mode_is_steering (struct mlx5_eswitch *esw) +{ +return (MLX5_CAP_ESW_EGRESS_ACL (esw->dev, pop_vlan) && +MLX5_CAP_ESW_INGRESS_ACL (esw->dev, push_vlan)); +} +static inline bool mlx5_eswitch_vlan_actions_supported (struct mlx5_core_dev *dev,u8 vlan_depth) {
include/linux/mlx5/device.h +5 -0
@@ -1090,6 +1090,11 @@ enum { MLX5_VPORT_ADMIN_STATE_AUTO = 0x2, }; +enum { +MLX5_VPORT_CVLAN_INSERT_WHEN_NO_CVLAN =0x1 , +MLX5_VPORT_CVLAN_INSERT_ALWAYS =0x3 , +}; +enum {MLX5_L3_PROT_TYPE_IPV4 =0 ,MLX5_L3_PROT_TYPE_IPV6 =1 ,
include/linux/mlx5/mlx5_ifc.h +2 -1
@@ -913,7 +913,8 @@ struct mlx5_ifc_e_switch_cap_bits {u8 vport_svlan_insert[0x1 ];u8 vport_cvlan_insert_if_not_exist[0x1 ];u8 vport_cvlan_insert_overwrite[0x1 ]; -u8 reserved_at_5[0x2 ]; +u8 reserved_at_5[0x1 ]; +u8 vport_cvlan_insert_always[0x1 ];u8 esw_shared_ingress_acl[0x1 ];u8 esw_uplink_ingress_acl[0x1 ];u8 root_ft_on_other_esw[0x1 ];
Signature · 1 line
2.38.1
Re: [net 01/12] net/mlx5: E-Switch, properly handle ingress tagged packets on VST
This series was applied to netdev/net.git (master) by Saeed Mahameed <[email protected]>:
Hello:
This series was applied to netdev/net.git (master) by Saeed Mahameed <[email protected]>:
Quoted 11 lines · On Wed, 28 Dec 2022 11:43:20 -0800 you wrote:
From: Moshe Shemesh <[email protected]>
Fix SRIOV VST mode behavior to insert cvlan when a guest tag is already present in the frame. Previous VST mode behavior was to drop packets or override existing tag, depending on the device version.
In this patch we fix this behavior by correctly building the HW steering rule with a push vlan action, or for older devices we ask the FW to stack the vlan when a vlan is already present.
[...]
Here is the summary with links:
- [net,01/12] net/mlx5: E-Switch, properly handle ingress tagged packets on VST
https://git.kernel.org/netdev/net/c/1f0ae22ab470 - [net,02/12] net/mlx5: Add forgotten cleanup calls into mlx5_init_once() error path https://git.kernel.org/netdev/net/c/2a35b2c2e6a2 - [net,03/12] net/mlx5: Fix io_eq_size and event_eq_size params validation https://git.kernel.org/netdev/net/c/44aee8ea15ac - [net,04/12] net/mlx5: Avoid recovery in probe flows https://git.kernel.org/netdev/net/c/9078e843efec - [net,05/12] net/mlx5: Fix RoCE setting at HCA level https://git.kernel.org/netdev/net/c/c4ad5f2bdad5 - [net,06/12] net/mlx5e: IPoIB, Don't allow CQE compression to be turned on by default https://git.kernel.org/netdev/net/c/b12d581e83e3 - [net,07/12] net/mlx5e: Fix RX reporter for XSK RQs https://git.kernel.org/netdev/net/c/f8c18a5749cf - [net,08/12] net/mlx5e: CT: Fix ct debugfs folder name https://git.kernel.org/netdev/net/c/849190e3e4cc - [net,09/12] net/mlx5e: Always clear dest encap in neigh-update-del https://git.kernel.org/netdev/net/c/2951b2e142ec - [net,10/12] net/mlx5e: Fix hw mtu initializing at XDP SQ allocation https://git.kernel.org/netdev/net/c/1e267ab88dc4 - [net,11/12] net/mlx5e: Set geneve_tlv_option_0_exist when matching on geneve option https://git.kernel.org/netdev/net/c/e54638a8380b - [net,12/12] net/mlx5: Lag, fix failure to cancel delayed bond work https://git.kernel.org/netdev/net/c/4d1c1379d717
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
PATCH 2/12 net/mlx5: Add forgotten cleanup calls into mlx5_init_once() error path
In reply to Saeed MahameedFrom: Jiri Pirko <[email protected]>
There are two cleanup calls missing in mlx5_init_once() error path. Add them making the error path flow to be the same as mlx5_cleanup_once().
--- drivers/net/ethernet/mellanox/mlx5/core/main.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/main.c +2 -0
@@ -1050,6 +1050,8 @@ static int mlx5_init_once(struct mlx5_core_dev *dev) err_tables_cleanup:mlx5_geneve_destroy (dev->geneve);mlx5_vxlan_destroy (dev->vxlan); +mlx5_cleanup_clock (dev); +mlx5_cleanup_reserved_gids (dev);mlx5_cq_debugfs_cleanup (dev);mlx5_fw_reset_cleanup (dev); err_events_cleanup:
Signature · 1 line
2.38.1
PATCH 3/12 net/mlx5: Fix io_eq_size and event_eq_size params validation
In reply to Saeed MahameedFrom: Shay Drory <[email protected]>
io_eq_size and event_eq_size params are of param type DEVLINK_PARAM_TYPE_U32. But, the validation callback is addressing them as DEVLINK_PARAM_TYPE_U16.
This cause mismatch in validation in big-endian systems, in which values in range were rejected while 268500991 was accepted. Fix it by checking the U32 value in the validation callback.
--- drivers/net/ethernet/mellanox/mlx5/core/devlink.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/devlink.c +1 -1
@@ -563,7 +563,7 @@ static int mlx5_devlink_eq_depth_validate(struct devlink *devlink, u32 id, union devlink_param_value val, struct netlink_ext_ack *extack) { -return (val.vu16 >=64 && val.vu16 <=4096 ) ?0 : -EINVAL ; +return (val.vu32 >=64 && val.vu32 <=4096 ) ?0 : -EINVAL ; }static const struct devlink_param mlx5_devlink_params[] = {
Signature · 1 line
2.38.1
PATCH 4/12 net/mlx5: Avoid recovery in probe flows
In reply to Saeed MahameedFrom: Shay Drory <[email protected]>
Currently, recovery is done without considering whether the device is still in probe flow.
This may lead to recovery before device have finished probed successfully. e.g.: while mlx5_init_one() is running. Recovery flow is using functionality that is loaded only by mlx5_init_one(), and there is no point in running recovery without mlx5_init_one() finished successfully.
Fix it by waiting for probe flow to finish and checking whether the device is probed before trying to perform recovery.
--- drivers/net/ethernet/mellanox/mlx5/core/health.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/health.c +6 -0
@@ -674,6 +674,12 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work) dev = container_of(priv, struct mlx5_core_dev, priv); devlink =priv_to_devlink (dev); +mutex_lock (&dev->intf_state_mutex); +if (test_bit (MLX5_DROP_NEW_HEALTH_WORK , &health->flags)) { +mlx5_core_err (dev,"health works are not permitted at this stage\n" ); +return ; + } +mutex_unlock (&dev->intf_state_mutex);enter_error_state (dev, false);if (IS_ERR_OR_NULL (health->fw_fatal_reporter)) {devl_lock (devlink);
Signature · 1 line
2.38.1
Re: [net 04/12] net/mlx5: Avoid recovery in probe flows
Quoted 26 lines · Saeed Mahameed
From: Shay Drory <[email protected]>
Currently, recovery is done without considering whether the device is still in probe flow.
This may lead to recovery before device have finished probed successfully. e.g.: whilemlx5_init_one()is running. Recovery flow is using functionality that is loaded only bymlx5_init_one(), and there is no point in running recovery withoutmlx5_init_one()finished successfully.Fix it by waiting for probe flow to finish and checking whether the device is probed before trying to perform recovery.
Fixes 51d138c2610a ("net/mlx5: Fix health error state handling")Signed-off-by Shay Drory <[email protected]>Reviewed-by Moshe Shemesh <[email protected]>Signed-off-by Saeed Mahameed <[email protected]>--- drivers/net/ethernet/mellanox/mlx5/core/health.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/net/ethernet/mellanox/mlx5/core/health.c +0
-0
drivers/net/ethernet/mellanox/mlx5/core/health.c static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work) · line 681
Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/health.c +5
-0@@ -674,6 +674,12 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work) dev = container_of(priv, struct mlx5_core_dev, priv); devlink =priv_to_devlink (dev); +mutex_lock (&dev->intf_state_mutex); +if (test_bit (MLX5_DROP_NEW_HEALTH_WORK , &health->flags)) { +mlx5_core_err (dev,"health works are not permitted at this stage\n" ); +return ; + }
This bit is already checked when health recovery is queued in mlx5_trigger_health_work().
764 void mlx5_trigger_health_work(struct mlx5_core_dev *dev)
765 {
766 struct mlx5_core_health *health = &dev->priv.health;
767 unsigned long flags;
768
769 spin_lock_irqsave(&health->wq_lock, flags);
770 if (!test_bit(MLX5_DROP_NEW_HEALTH_WORK, &health->flags))
771 queue_work(health->wq, &health->fatal_report_work);
772 else
773 mlx5_core_err(dev, "new health works are not permitted at this stage\n");
774 spin_unlock_irqrestore(&health->wq_lock, flags);
775 }You probably need to elevate this check to poll_health() routine and change intf_state_mutex to be spinlock.
Or another solution is to start health polling only when init complete.
Thanks
Quoted 7 lines
drivers/net/ethernet/mellanox/mlx5/core/health.c +1
-0+mutex_unlock (&dev->intf_state_mutex);enter_error_state (dev, false);if (IS_ERR_OR_NULL (health->fw_fatal_reporter)) {devl_lock (devlink);Signature · 1 line
2.38.1
Re: [net 04/12] net/mlx5: Avoid recovery in probe flows
Quoted 51 lines · Leon Romanovsky
Quoted 26 lines · Saeed Mahameed
From: Shay Drory <[email protected]>
Currently, recovery is done without considering whether the device is still in probe flow.
This may lead to recovery before device have finished probed successfully. e.g.: whilemlx5_init_one()is running. Recovery flow is using functionality that is loaded only bymlx5_init_one(), and there is no point in running recovery withoutmlx5_init_one()finished successfully.Fix it by waiting for probe flow to finish and checking whether the device is probed before trying to perform recovery.
Fixes 51d138c2610a ("net/mlx5: Fix health error state handling")Signed-off-by Shay Drory <[email protected]>Reviewed-by Moshe Shemesh <[email protected]>Signed-off-by Saeed Mahameed <[email protected]>--- drivers/net/ethernet/mellanox/mlx5/core/health.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/net/ethernet/mellanox/mlx5/core/health.c +0
-0drivers/net/ethernet/mellanox/mlx5/core/health.c static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work) · line 681
Quoted 8 lines
drivers/net/ethernet/mellanox/mlx5/core/health.c +5
-0@@ -674,6 +674,12 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work) dev = container_of(priv, struct mlx5_core_dev, priv); devlink =priv_to_devlink (dev); +mutex_lock (&dev->intf_state_mutex); +if (test_bit (MLX5_DROP_NEW_HEALTH_WORK , &health->flags)) { +mlx5_core_err (dev,"health works are not permitted at this stage\n" ); +return ; + }This bit is already checked when health recovery is queued in
mlx5_trigger_health_work().764 void
mlx5_trigger_health_work(struct mlx5_core_dev *dev)765 {
766 structmlx5_core_health*health =&dev->priv.health;
767 unsigned long flags;
768
769spin_lock_irqsave(&health->wq_lock, flags);
770 if (!test_bit(MLX5_DROP_NEW_HEALTH_WORK,&health->flags))
771queue_work(health->wq, &health->fatal_report_work);
772 else
773mlx5_core_err(dev, "new health works are not permitted at this stage\n");
774spin_unlock_irqrestore(&health->wq_lock, flags);
775 }
Quoted 2 lines
You probably need to elevate this check to
poll_health()routine and changeintf_state_mutexto be spinlock.
not possible, big design change to the driver..
Quoted 3 lines
Or another solution is to start health polling only when init complete.
Also very complex and very risky to do in rc.
Health poll should be running on dynamic driver reloads,
for example devlink reload, but not on first probe.. if we are going to start after probe then we will have to stop (sync) any health work before .remove, which is a locking nightmare.. we've been there
before.
Re: [net 04/12] net/mlx5: Avoid recovery in probe flows
Quoted 37 lines · Saeed Mahameed
Quoted 36 lines · Leon Romanovsky
Quoted 34 lines · Saeed Mahameed
From: Shay Drory <[email protected]>
Currently, recovery is done without considering whether the device is still in probe flow.
This may lead to recovery before device have finished probed successfully. e.g.: whilemlx5_init_one()is running. Recovery flow is using functionality that is loaded only bymlx5_init_one(), and there is no point in running recovery withoutmlx5_init_one()finished successfully.Fix it by waiting for probe flow to finish and checking whether the device is probed before trying to perform recovery.
Fixes 51d138c2610a ("net/mlx5: Fix health error state handling")Signed-off-by Shay Drory <[email protected]>Reviewed-by Moshe Shemesh <[email protected]>Signed-off-by Saeed Mahameed <[email protected]>--- drivers/net/ethernet/mellanox/mlx5/core/health.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/net/ethernet/mellanox/mlx5/core/health.c +5
-0@@ -674,6 +674,12 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work) dev = container_of(priv, struct mlx5_core_dev, priv); devlink =priv_to_devlink (dev); +mutex_lock (&dev->intf_state_mutex); +if (test_bit (MLX5_DROP_NEW_HEALTH_WORK , &health->flags)) { +mlx5_core_err (dev,"health works are not permitted at this stage\n" ); +return ; + }
<...>
Quoted 9 lines
Quoted 2 lines
Or another solution is to start health polling only when init complete.
Also very complex and very risky to do in rc.
Health poll should be running on dynamic driver reloads,
for example devlink reload, but not on first probe.. if we are going to start after probe then we will have to stop (sync) any
health work before .remove, which is a locking nightmare.. we've been there before.
I afraid that my proposed solution distracted you. The real issue is that this patch can't be correct.
Let's focus on MLX5_DROP_NEW_HEALTH_WORK bit. It is checked while holding different locks, so one of the locks is wrong and not needed.
If MLX5_DROP_NEW_HEALTH_WORK bit can't be changed after/during queuing the work, the newly added check in mlx5_fw_fatal_reporter_err_work will be redundant.
If MLX5_DROP_NEW_HEALTH_WORK bit can be changed after queuing the work. the check is racy and can have different results immediately after releasing intf_state_mutex.
Thanks
PATCH 5/12 net/mlx5: Fix RoCE setting at HCA level
In reply to Saeed MahameedFrom: Shay Drory <[email protected]>
mlx5 PF can disable RoCE for its VFs and SFs. In such case RoCE is marked as unsupported on those VFs/SFs.
The cited patch added an option for disable (and enable) RoCE at HCA level. However, that commit didn't check whether RoCE is supported on the HCA and enabled user to try and set RoCE to on.
Fix it by checking whether the HCA supports RoCE.
--- drivers/net/ethernet/mellanox/mlx5/core/devlink.c | 2 +- drivers/net/ethernet/mellanox/mlx5/core/main.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/devlink.c +1 -1
@@ -468,7 +468,7 @@ static int mlx5_devlink_enable_roce_validate(struct devlink *devlink, u32 id,bool new_state = val.vbool;if (new_state && !MLX5_CAP_GEN (dev, roce) && - !MLX5_CAP_GEN (dev, roce_rw_supported)) { + !(MLX5_CAP_GEN (dev, roce_rw_supported) &&MLX5_CAP_GEN_MAX (dev, roce))) {NL_SET_ERR_MSG_MOD (extack,"Device doesn't support RoCE" );return -EOPNOTSUPP ; }
drivers/net/ethernet/mellanox/mlx5/core/main.c +1 -1
@@ -613,7 +613,7 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)MLX5_SET (cmd_hca_cap, set_hca_cap, num_total_dynamic_vf_msix,MLX5_CAP_GEN_MAX (dev, num_total_dynamic_vf_msix)); -if (MLX5_CAP_GEN (dev, roce_rw_supported)) +if (MLX5_CAP_GEN (dev, roce_rw_supported) &&MLX5_CAP_GEN_MAX (dev, roce))MLX5_SET (cmd_hca_cap, set_hca_cap, roce,mlx5_is_roce_on (dev));
Signature · 1 line
2.38.1
PATCH 6/12 net/mlx5e: IPoIB, Don't allow CQE compression to be turned on by default
In reply to Saeed MahameedFrom: Dragos Tatulea <[email protected]>
mlx5e_build_nic_params will turn CQE compression on if the hardware capability is enabled and the slow_pci_heuristic condition is detected. As IPoIB doesn't support CQE compression, make sure to disable the feature in the IPoIB profile init.
Please note that the feature is not exposed to the user for IPoIB interfaces, so it can't be subsequently turned on.
--- drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c +4 -0
@@ -71,6 +71,10 @@ static void mlx5i_build_nic_params(struct mlx5_core_dev *mdev, params->packet_merge.type =MLX5E_PACKET_MERGE_NONE ; params->hard_mtu =MLX5_IB_GRH_BYTES +MLX5_IPOIB_HARD_LEN ; params->tunneled_offload_en = false; + +/* CQE compression is not supported for IPoIB */ + params->rx_cqe_compress_def = false; +MLX5E_SET_PFLAG (params,MLX5E_PFLAG_RX_CQE_COMPRESS , params->rx_cqe_compress_def); }/* Called directly after IPoIB netdevice was created to initialize SW structs */
Signature · 1 line
2.38.1
PATCH 7/12 net/mlx5e: Fix RX reporter for XSK RQs
In reply to Saeed MahameedFrom: Tariq Toukan <[email protected]>
RX reporter mistakenly reads from the regular (inactive) RQ when XSK RQ is active. Fix it here.
--- drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c +5 -1
@@ -459,7 +459,11 @@ static int mlx5e_rx_reporter_diagnose(struct devlink_health_reporter *reporter, goto unlock;for (i =0 ; i < priv->channels.num; i++) { -struct mlx5e_rq *rq = &priv->channels.c[i]->rq; +struct mlx5e_channel *c = priv->channels.c[i]; +struct mlx5e_rq *rq; + + rq =test_bit (MLX5E_CHANNEL_STATE_XSK , c->state) ? + &c->xskrq : &c->rq; err =mlx5e_rx_reporter_build_diagnose_output (rq, fmsg);if (err)
Signature · 1 line
2.38.1
PATCH 8/12 net/mlx5e: CT: Fix ct debugfs folder name
In reply to Saeed MahameedFrom: Chris Mi <[email protected]>
Need to use sprintf to build a string instead of sscanf. Otherwise dirname is null and both "ct_nic" and "ct_fdb" won't be created. But its redundant anyway as driver could be in switchdev mode but still add nic rules. So use "ct" as folder name.
--- drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c +1 -6
@@ -2103,14 +2103,9 @@ mlx5_tc_ct_init_check_support(struct mlx5e_priv *priv,static void mlx5_ct_tc_create_dbgfs (struct mlx5_tc_ct_priv *ct_priv) { -bool is_fdb = ct_priv->ns_type ==MLX5_FLOW_NAMESPACE_FDB ;struct mlx5_tc_ct_debugfs *ct_dbgfs = &ct_priv->debugfs; -char dirname[16 ] = {}; -if (sscanf (dirname,"ct_%s" , is_fdb ?"fdb" :"nic" ) <0 ) -return ; - - ct_dbgfs->root =debugfs_create_dir (dirname,mlx5_debugfs_get_dev_root (ct_priv->dev)); + ct_dbgfs->root =debugfs_create_dir ("ct" ,mlx5_debugfs_get_dev_root (ct_priv->dev));debugfs_create_atomic_t ("offloaded" ,0400 , ct_dbgfs->root, &ct_dbgfs->stats.offloaded); debugfs_create_atomic_t("rx_dropped", 0400, ct_dbgfs->root,
Signature · 1 line
2.38.1
PATCH 9/12 net/mlx5e: Always clear dest encap in neigh-update-del
In reply to Saeed MahameedFrom: Chris Mi <[email protected]>
The cited commit introduced a bug for multiple encapsulations flow. If one dest encap becomes invalid, the flow is set slow path flag. But when other dests encap become invalid, they are not cleared due to slow path flag of the flow. When neigh-update-add is running, it will use invalid encap.
Fix it by checking slow path flag after clearing dest encap.
--- .../net/ethernet/mellanox/mlx5/core/en/tc_tun_encap.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_encap.c +8 -1
@@ -222,7 +222,7 @@ void mlx5e_tc_encap_flows_del(struct mlx5e_priv *priv,int err;list_for_each_entry (flow, flow_list, tmp_list) { -if (!mlx5e_is_offloaded_flow (flow) ||flow_flag_test (flow,SLOW )) +if (!mlx5e_is_offloaded_flow (flow))continue ; attr =mlx5e_tc_get_encap_attr (flow);
@@ -231,6 +231,13 @@ void mlx5e_tc_encap_flows_del(struct mlx5e_priv *priv, esw_attr->dests[flow->tmp_entry_index].flags &= ~MLX5_ESW_DEST_ENCAP_VALID ; esw_attr->dests[flow->tmp_entry_index].pkt_reformat =NULL ; +/* Clear pkt_reformat before checking slow path flag. Because +* in next iteration, the same flow is already set slow path +* flag, but still need to clear the pkt_reformat. +*/ +if (flow_flag_test (flow,SLOW )) +continue ; +/* update from encap rule to slow path rule */ spec = &flow->attr->parse_attr->spec; rule =mlx5e_tc_offload_to_slow_path (esw, flow, spec);
Signature · 1 line
2.38.1
PATCH 10/12 net/mlx5e: Fix hw mtu initializing at XDP SQ allocation
In reply to Saeed MahameedFrom: Adham Faris <[email protected]>
Current xdp xmit functions logic (mlx5e_xmit_xdp_frame_mpwqe or mlx5e_xmit_xdp_frame), validates xdp packet length by comparing it to hw mtu (configured at xdp sq allocation) before xmiting it. This check does not account for ethernet fcs length (calculated and filled by the nic). Hence, when we try sending packets with length > (hw-mtu - ethernet-fcs-size), the device port drops it and tx_errors_phy is incremented. Desired behavior is to catch these packets and drop them by the driver.
Fix this behavior in XDP SQ allocation function (mlx5e_alloc_xdpsq) by subtracting ethernet FCS header size (4 Bytes) from current hw mtu value, since ethernet FCS is calculated and written to ethernet frames by the nic.
--- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/ethernet/mellanox/mlx5/core/en_main.c +1 -1
@@ -1305,7 +1305,7 @@ static int mlx5e_alloc_xdpsq(struct mlx5e_channel *c, sq->channel = c; sq->uar_map = mdev->mlx5e_res.hw_objs.bfreg.map; sq->min_inline_mode = params->tx_min_inline_mode; - sq->hw_mtu =MLX5E_SW2HW_MTU (params, params->sw_mtu); + sq->hw_mtu =MLX5E_SW2HW_MTU (params, params->sw_mtu) -ETH_FCS_LEN ; sq->xsk_pool = xsk_pool; sq->stats = sq->xsk_pool ?
Signature · 1 line
2.38.1
PATCH 11/12 net/mlx5e: Set geneve_tlv_option_0_exist when matching on geneve option
In reply to Saeed MahameedFrom: Maor Dickman <[email protected]>
The cited patch added support of matching on geneve option by setting geneve_tlv_option_0_data mask and key but didn't set geneve_tlv_option_0_exist bit which is required on some HWs when matching geneve_tlv_option_0_data parameter, this may cause in some cases for packets to wrongly match on rules with different geneve option.
Example of such case is packet with geneve_tlv_object class=789 and data=456 will wrongly match on rule with match geneve_tlv_object class=123 and data=456.
Fix it by setting geneve_tlv_option_0_exist bit when supported by the HW when matching on geneve_tlv_option_0_data parameter.
--- drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_geneve.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_geneve.c +5 -0
@@ -273,6 +273,11 @@ static int mlx5e_tc_tun_parse_geneve_options(struct mlx5e_priv *priv, geneve_tlv_option_0_data, be32_to_cpu(opt_data_key));MLX5_SET (fte_match_set_misc3, misc_3_c, geneve_tlv_option_0_data,be32_to_cpu (opt_data_mask)); +if (MLX5_CAP_ESW_FLOWTABLE_FDB (priv->mdev, + ft_field_support.geneve_tlv_option_0_exist)) { +MLX5_SET_TO_ONES (fte_match_set_misc, misc_c, geneve_tlv_option_0_exist); +MLX5_SET_TO_ONES (fte_match_set_misc, misc_v, geneve_tlv_option_0_exist); + } spec->match_criteria_enable |=MLX5_MATCH_MISC_PARAMETERS_3 ;
Signature · 1 line
2.38.1
PATCH 12/12 net/mlx5: Lag, fix failure to cancel delayed bond work
In reply to Saeed MahameedFrom: Eli Cohen <[email protected]>
Commit 0d4e8ed139d8 ("net/mlx5: Lag, avoid lockdep warnings") accidentally removed a call to cancel delayed bond work thus it may cause queued delay to expire and fall on an already destroyed work queue.
Fix by restoring the call cancel_delayed_work_sync() before destroying the workqueue.
This prevents call trace such as this:
[ 329.230417] BUG: kernel NULL pointer dereference, address: 0000000000000000 [ 329.231444] #PF: supervisor write access in kernel mode [ 329.232233] #PF: error_code(0x0002) - not-present page [ 329.233007] PGD 0 P4D 0 [ 329.233476] Oops: 0002 [#1] SMP [ 329.234012] CPU: 5 PID: 145 Comm: kworker/u20:4 Tainted: G OE 6.0.0-rc5_mlnx #1 [ 329.235282] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014 [ 329.236868] Workqueue: mlx5_cmd_0000:08:00.1 cmd_work_handler [mlx5_core] [ 329.237886] RIP: 0010:_raw_spin_lock+0xc/0x20 [ 329.238585] Code: f0 0f b1 17 75 02 f3 c3 89 c6 e9 6f 3c 5f ff 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 0f 1f 44 00 00 31 c0 ba 01 00 00 00 <f0> 0f b1 17 75 02 f3 c3 89 c6 e9 45 3c 5f ff 0f 1f 44 00 00 0f 1f [ 329.241156] RSP: 0018:ffffc900001b0e98 EFLAGS: 00010046 [ 329.241940] RAX: 0000000000000000 RBX: ffffffff82374ae0 RCX: 0000000000000000 [ 329.242954] RDX: 0000000000000001 RSI: 0000000000000014 RDI: 0000000000000000 [ 329.243974] RBP: ffff888106ccf000 R08: ffff8881004000c8 R09: ffff888100400000 [ 329.244990] R10: 0000000000000000 R11: ffffffff826669f8 R12: 0000000000002000 [ 329.246009] R13: 0000000000000005 R14: ffff888100aa7ce0 R15: ffff88852ca80000 [ 329.247030] FS: 0000000000000000(0000) GS:ffff88852ca80000(0000) knlGS:0000000000000000 [ 329.248260] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 329.249111] CR2: 0000000000000000 CR3: 000000016d675001 CR4: 0000000000770ee0 [ 329.250133] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 329.251152] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 329.252176] PKRU: 55555554
--- drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c +1 -0
@@ -228,6 +228,7 @@ static void mlx5_ldev_free(struct kref *ref)if (ldev->nb.notifier_call)unregister_netdevice_notifier_net (&init_net, &ldev->nb);mlx5_lag_mp_cleanup (ldev); +cancel_delayed_work_sync (&ldev->bond_work);destroy_workqueue (ldev->wq);mlx5_lag_mpesw_cleanup (ldev);mutex_destroy (&ldev->lock);
Signature · 1 line
2.38.1