ice: Reliably reset VFs
Subsystems (2)
Tony Nguyen
maintainerPATCH v2 1/9 ice: Reliably reset VFs
From: Mitch Williams <[email protected]>
When a PFR (or bigger reset) occurs, the device clears the VF_MBX_ARQLEN register for all VFs. But if a VFR is triggered by a VF, the device does NOT clear this register, and the VF driver will never see the reset.
When this happens, the VF driver will eventually timeout and attempt recovery, and usually it will be successful. But this makes resets take a long time and there are occasional failures.
We cannot just blithely clear this register on every reset; this has been shown to cause synchronization problems when a PFR is triggered with a large number of VFs.
Fix this by clearing VF_MBX_ARQLEN when the reset source is not PFR. GlobR will trigger PFR, so this test catches that occurrence as well.
--- drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c +10 -6
@@ -353,12 +353,13 @@ void ice_free_vfs(struct ice_pf *pf)* ice_trigger_vf_reset - Reset a VF on HW * @vf: pointer to the VF structure * @is_vflr: true if VFLR was issued, false if not +* @is_pfr: true if the reset was triggered due to a previous PFR * * Trigger hardware to start a reset for a particular VF. Expects the caller * to wait the proper amount of time to allow hardware to reset the VF before * it cleans up and restores VF functionality. */ -static void ice_trigger_vf_reset (struct ice_vf *vf,bool is_vflr) +static void ice_trigger_vf_reset (struct ice_vf *vf,bool is_vflr,bool is_pfr) {struct ice_pf *pf = vf->pf;u32 reg, reg_idx, bit_idx;
@@ -379,10 +380,13 @@ static void ice_trigger_vf_reset(struct ice_vf *vf, bool is_vflr)*/ clear_bit (ICE_VF_STATE_INIT , vf->vf_states); -/* Clear the VF's ARQLEN register. This is how the VF detects reset, -* since the VFGEN_RSTAT register doesn't stick at 0 after reset. +/* VF_MBX_ARQLEN is cleared by PFR, so the driver needs to clear it +* in the case of VFR. If this is done for PFR, it can mess up VF +* resets because the VF driver may already have started cleanup +* by the time we get here. */ -wr32 (hw,VF_MBX_ARQLEN (vf_abs_id),0 ); +if (!is_pfr) +wr32 (hw,VF_MBX_ARQLEN (vf_abs_id),0 );/* In the case of a VFLR, the HW has already reset the VF and we * just need to clean up, so don't hit the VFRTRIG register.
@@ -1072,7 +1076,7 @@ bool ice_reset_all_vfs(struct ice_pf *pf, bool is_vflr)/* Begin reset on all VFs at once */ for (v =0 ; v < pf->num_alloc_vfs; v++) -ice_trigger_vf_reset (&pf->vf[v], is_vflr); +ice_trigger_vf_reset (&pf->vf[v], is_vflr, true);for (v =0 ; v < pf->num_alloc_vfs; v++) {struct ice_vsi *vsi;
@@ -1172,7 +1176,7 @@ static bool ice_reset_vf(struct ice_vf *vf, bool is_vflr)if (test_and_set_bit (ICE_VF_STATE_DIS , vf->vf_states))return false; -ice_trigger_vf_reset (vf, is_vflr); +ice_trigger_vf_reset (vf, is_vflr, false); vsi = pf->vsi[vf->lan_vsi_idx];
Signature · 1 line
2.20.1
Tony Nguyen
maintainerPATCH v2 2/9 ice: report link down for VF when PF's queues are not enabled
From: Lukasz Czapnik <[email protected]>
This is port of a fix from i40e commit 2ad1274fa35a ("i40e: don't report link up for a VF who hasn't enabled queues")
Older VF drivers do not respond well to receiving a link
up notification before queues are enabled. This can cause their state machine to think that it is safe to send traffic. This results in a Tx hang on the VF.
Record whether the PF has actually enabled queues for the VF. When reporting link status, always report link down if the queues aren't enabled. In this way, the VF driver will never receive a link up notification until after its queues are enabled.
--- drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c +4 -1
@@ -129,7 +129,10 @@ static void ice_vc_notify_vf_link_state(struct ice_vf *vf) pfe.event =VIRTCHNL_EVENT_LINK_CHANGE ; pfe.severity =PF_EVENT_SEVERITY_INFO ; -if (vf->link_forced) +/* Always report link is down if the VF queues aren't enabled */ +if (!vf->num_qs_ena) +ice_set_pfe_link (vf, &pfe,ICE_AQ_LINK_SPEED_UNKNOWN , false); +else if (vf->link_forced)ice_set_pfe_link_forced (vf, &pfe, vf->link_up);else ice_set_pfe_link(vf, &pfe, ls->link_speed, ls->link_info &
Signature · 1 line
2.20.1
Bowers
PATCH v2 2/9 ice: report link down for VF when PF's queues are not enabled
Quoted 22 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf Of Tony Nguyen
Sent: Tuesday, September 3, 2019 1:31 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH S28 v2 2/9] ice: report link down for VF when PF's queues are not enabledFrom: Lukasz Czapnik <[email protected]>
This is port of a fix from i40e commit
2ad1274fa35a("i40e: don't report link up for a VF who hasn't enabled queues")Older VF drivers do not respond well to receiving a link up notification before queues are enabled. This can cause their state machine to think that it is safe to send traffic. This results in a Tx hang on the VF.
Record whether the PF has actually enabled queues for the VF. When reporting link status, always report link down if the queues aren't enabled. In this way, the VF driver will never receive a link up notification until after its queues are enabled.
Quoted 5 lines
Signed-off-by Lukasz Czapnik <[email protected]>Signed-off-by Tony Nguyen <[email protected]>--- drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
Tony Nguyen
maintainerPATCH v2 3/9 ice: Check for DCB capability before initializing DCB
In reply to Tony NguyenFrom: Anirudh Venkataramanan <[email protected]>
Check the ICE_FLAG_DCB_CAPABLE before calling ice_init_pf_dcb.
--- drivers/net/ethernet/intel/ice/ice_dcb_lib.c | 3 --- drivers/net/ethernet/intel/ice/ice_main.c | 15 ++++++++------- 2 files changed, 8 insertions(+), 10 deletions(-)
drivers/net/ethernet/intel/ice/ice_dcb_lib.c +0 -3
@@ -474,7 +474,6 @@ int ice_init_pf_dcb(struct ice_pf *pf, bool locked) } pf->dcbx_cap =DCB_CAP_DCBX_HOST |DCB_CAP_DCBX_VER_IEEE ; -set_bit (ICE_FLAG_DCB_CAPABLE , pf->flags);return 0 ; }
@@ -483,8 +482,6 @@ int ice_init_pf_dcb(struct ice_pf *pf, bool locked)/* DCBX in FW and LLDP enabled in FW */ pf->dcbx_cap =DCB_CAP_DCBX_LLD_MANAGED |DCB_CAP_DCBX_VER_IEEE ; -set_bit (ICE_FLAG_DCB_CAPABLE , pf->flags); - err =ice_dcb_init_cfg (pf, locked);if (err) goto dcb_init_err;
drivers/net/ethernet/intel/ice/ice_main.c +8 -7
@@ -2252,6 +2252,8 @@ static void ice_deinit_pf(struct ice_pf *pf)static int ice_init_pf (struct ice_pf *pf) {bitmap_zero (pf->flags,ICE_PF_FLAGS_NBITS ); +if (pf->hw.func_caps.common_cap.dcb) +set_bit (ICE_FLAG_DCB_CAPABLE , pf->flags);#ifdef CONFIG_PCI_IOV if (pf->hw.func_caps.common_cap.sr_iov_1_1) {struct ice_hw *hw = &pf->hw;
@@ -2529,13 +2531,12 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) goto err_init_pf_unroll; } - err =ice_init_pf_dcb (pf, false); -if (err) { -clear_bit (ICE_FLAG_DCB_CAPABLE , pf->flags); -clear_bit (ICE_FLAG_DCB_ENA , pf->flags); - -/* do not fail overall init if DCB init fails */ - err =0 ; +if (test_bit (ICE_FLAG_DCB_CAPABLE , pf->flags)) { +/* Note: DCB init failure is non-fatal to load */ +if (ice_init_pf_dcb (pf, false)) { +clear_bit (ICE_FLAG_DCB_CAPABLE , pf->flags); +clear_bit (ICE_FLAG_DCB_ENA , pf->flags); + } }ice_determine_q_usage (pf);
Signature · 1 line
2.20.1
Bowers
PATCH v2 3/9 ice: Check for DCB capability before initializing DCB
Quoted 10 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf Of Tony Nguyen
Sent: Tuesday, September 3, 2019 1:31 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH S28 v2 3/9] ice: Check for DCB capability before initializing DCBFrom: Anirudh Venkataramanan <[email protected]>
Quoted 8 lines
Check the
ICE_FLAG_DCB_CAPABLEbefore callingice_init_pf_dcb.Signed-off-by: Anirudh Venkataramanan
<[email protected]>--- drivers/net/ethernet/intel/ice/ice_dcb_lib.c | 3 --- drivers/net/ethernet/intel/ice/ice_main.c | 15 ++++++++------- 2 files changed, 8 insertions(+), 10 deletions(-)
Tony Nguyen
maintainerPATCH v2 4/9 ice: Report VF link status with opcode to get resources
In reply to Tony NguyenFrom: Akeem G Abodunrin <[email protected]>
This patch changes how and when the driver report link status, instead of waiting till the call to enable queues for VF, we should report link status earlier with opcode to get VF resources - So as to avoid reporting erroneous information, especially when queues have not been configured. In addition, we can also make a call to get and report link status change after when queue is enabled, at least to report netdev or PHY link status. This is in accordance to how link speed is being reported for PF...
--- drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c +1 -0
@@ -2934,6 +2934,7 @@ void ice_vc_process_vf_msg(struct ice_pf *pf, struct ice_rq_event_info *event)break ;case VIRTCHNL_OP_GET_VF_RESOURCES : err =ice_vc_get_vf_res_msg (vf, msg); +ice_vc_notify_vf_link_state (vf);break ;case VIRTCHNL_OP_RESET_VF :ice_vc_reset_vf_msg (vf);
Signature · 1 line
2.20.1
Bowers
PATCH v2 4/9 ice: Report VF link status with opcode to get resources
Quoted 18 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf Of Tony Nguyen
Sent: Tuesday, September 3, 2019 1:31 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH S28 v2 4/9] ice: Report VF link status with opcode to get resourcesFrom: Akeem G Abodunrin <[email protected]>
This patch changes how and when the driver report link status, instead of waiting till the call to enable queues for VF, we should report link status earlier with opcode to get VF resources - So as to avoid reporting erroneous information, especially when queues have not been configured.
In addition, we can also make a call to get and report link status change after when queue is enabled, at least to report netdev or PHY link status. This is in accordance to how link speed is being reported for PF...
Quoted 4 lines
Signed-off-by Akeem G Abodunrin <[email protected]>--- drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c | 1 + 1 file changed, 1 insertion(+)
Tony Nguyen
maintainerPATCH v2 5/9 ice: update Tx context struct
In reply to Tony NguyenFrom: Ashish Shah <[email protected]>
Add internal usage flag, bit 91 as described in spec. Update width of internal queue state to 122 also as described in spec.
--- drivers/net/ethernet/intel/ice/ice_common.c | 3 ++- drivers/net/ethernet/intel/ice/ice_lan_tx_rx.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ice/ice_common.c +2 -1
@@ -1132,6 +1132,7 @@ const struct ice_ctx_ele ice_tlan_ctx_info[] = {ICE_CTX_STORE (ice_tlan_ctx, vmvf_type,2 ,78 ),ICE_CTX_STORE (ice_tlan_ctx, src_vsi,10 ,80 ),ICE_CTX_STORE (ice_tlan_ctx, tsyn_ena,1 ,90 ), +ICE_CTX_STORE (ice_tlan_ctx, internal_usage_flag,1 ,91 ),ICE_CTX_STORE (ice_tlan_ctx, alt_vlan,1 ,92 ),ICE_CTX_STORE (ice_tlan_ctx, cpuid,8 ,93 ), ICE_CTX_STORE(ice_tlan_ctx, wb_mode, 1, 101),
@@ -1150,7 +1151,7 @@ const struct ice_ctx_ele ice_tlan_ctx_info[] = {ICE_CTX_STORE (ice_tlan_ctx, drop_ena,1 ,165 ),ICE_CTX_STORE (ice_tlan_ctx, cache_prof_idx,2 ,166 ),ICE_CTX_STORE (ice_tlan_ctx, pkt_shaper_prof_idx,3 ,168 ), -ICE_CTX_STORE (ice_tlan_ctx, int_q_state,110 ,171 ), +ICE_CTX_STORE (ice_tlan_ctx, int_q_state,122 ,171 ), { 0 } };
drivers/net/ethernet/intel/ice/ice_lan_tx_rx.h +1 -0
@@ -428,6 +428,7 @@ struct ice_tlan_ctx {#define ICE_TLAN_CTX_VMVF_TYPE_PF 2u16 src_vsi;u8 tsyn_ena; +u8 internal_usage_flag;u8 alt_vlan;u16 cpuid;/* bigger than needed, see above for reason */ u8 wb_mode;
Signature · 1 line
2.20.1
Bowers
PATCH v2 5/9 ice: update Tx context struct
Quoted 9 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf Of Tony Nguyen
Sent: Tuesday, September 3, 2019 1:31 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH S28 v2 5/9] ice: update Tx context structFrom: Ashish Shah <[email protected]>
Quoted 8 lines
Add internal usage flag, bit 91 as described in spec. Update width of internal queue state to 122 also as described in spec.
Signed-off-by Ashish Shah <[email protected]>--- drivers/net/ethernet/intel/ice/ice_common.c | 3 ++- drivers/net/ethernet/intel/ice/ice_lan_tx_rx.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)
Tony Nguyen
maintainerPATCH v2 6/9 ice: Allow for delayed LLDP MIB change registration
In reply to Tony NguyenFrom: Dave Ertman <[email protected]>
Add an additional boolean parameter to the ice_init_dcb function. This boolean controls if the LLDP MIB change events are registered for. Also, add a new function defined ice_cfg_lldp_mib_change. The additional function is necessary to be able to register for LLDP MIB change events after calling ice_init_dcb. The net effect of these two changes is to allow a delayed registration for MIB change events so that the driver is not accepting events before it is ready for them.
--- drivers/net/ethernet/intel/ice/ice_dcb.c | 39 ++++++++++++++++++-- drivers/net/ethernet/intel/ice/ice_dcb.h | 11 ++---- drivers/net/ethernet/intel/ice/ice_dcb_lib.c | 4 +- drivers/net/ethernet/intel/ice/ice_ethtool.c | 10 ++++- drivers/net/ethernet/intel/ice/ice_main.c | 2 + 5 files changed, 51 insertions(+), 15 deletions(-)
drivers/net/ethernet/intel/ice/ice_dcb.c +35 -4
@@ -60,7 +60,7 @@ ice_aq_get_lldp_mib(struct ice_hw *hw, u8 bridge_type, u8 mib_type, void *buf,* Enable or Disable posting of an event on ARQ when LLDP MIB * associated with the interface changes (0x0A01) */ -enum ice_status +static enum ice_status ice_aq_cfg_lldp_mib_change (struct ice_hw *hw,bool ena_update,struct ice_sq_cd *cd) {
@@ -943,10 +943,11 @@ enum ice_status ice_get_dcb_cfg(struct ice_port_info *pi)/** * ice_init_dcb * @hw: pointer to the HW struct +* @enable_mib_change: enable MIB change event * * Update DCB configuration from the Firmware */ -enum ice_status ice_init_dcb (struct ice_hw *hw) +enum ice_status ice_init_dcb (struct ice_hw *hw,bool enable_mib_change) {struct ice_port_info *pi = hw->port_info;enum ice_status ret =0 ;
@@ -972,9 +973,39 @@ enum ice_status ice_init_dcb(struct ice_hw *hw) }/* Configure the LLDP MIB change event */ - ret =ice_aq_cfg_lldp_mib_change (hw, true,NULL ); +if (enable_mib_change) { + ret =ice_aq_cfg_lldp_mib_change (hw, true,NULL ); +if (!ret) + pi->is_sw_lldp = false; + } + +return ret; +} + +/** +* ice_cfg_lldp_mib_change +* @hw: pointer to the HW struct +* @ena_mib: enable/disable MIB change event +* +* Configure (disable/enable) MIB +*/ +enum ice_status ice_cfg_lldp_mib_change (struct ice_hw *hw,bool ena_mib) +{ +struct ice_port_info *pi = hw->port_info; +enum ice_status ret; + +if (!hw->func_caps.common_cap.dcb) +return ICE_ERR_NOT_SUPPORTED ; + +/* Get DCBX status */ + pi->dcbx_status =ice_get_dcbx_status (hw); + +if (pi->dcbx_status ==ICE_DCBX_STATUS_DIS ) +return ICE_ERR_NOT_READY ; + + ret =ice_aq_cfg_lldp_mib_change (hw, ena_mib,NULL );if (!ret) - pi->is_sw_lldp = false; + pi->is_sw_lldp = !ena_mib;return ret; }
drivers/net/ethernet/intel/ice/ice_dcb.h +4 -7
@@ -125,7 +125,7 @@ ice_aq_get_dcb_cfg(struct ice_hw *hw, u8 mib_type, u8 bridgetype, struct ice_dcbx_cfg *dcbcfg);enum ice_status ice_get_dcb_cfg (struct ice_port_info *pi);enum ice_status ice_set_dcb_cfg (struct ice_port_info *pi); -enum ice_status ice_init_dcb (struct ice_hw *hw); +enum ice_status ice_init_dcb (struct ice_hw *hw,bool enable_mib_change); enum ice_status ice_query_port_ets(struct ice_port_info *pi, struct ice_aqc_port_ets_elem *buf, u16 buf_size,
@@ -139,9 +139,7 @@ ice_aq_start_lldp(struct ice_hw *hw, bool persist, struct ice_sq_cd *cd);enum ice_status ice_aq_start_stop_dcbx (struct ice_hw *hw,bool start_dcbx_agent,bool *dcbx_agent_status,struct ice_sq_cd *cd); -enum ice_status -ice_aq_cfg_lldp_mib_change (struct ice_hw *hw,bool ena_update, -struct ice_sq_cd *cd); +enum ice_status ice_cfg_lldp_mib_change (struct ice_hw *hw,bool ena_mib);#else /* CONFIG_DCB */ static inline enum ice_status ice_aq_stop_lldp(struct ice_hw __always_unused *hw,
@@ -172,9 +170,8 @@ ice_aq_start_stop_dcbx(struct ice_hw __always_unused *hw, }static inline enum ice_status -ice_aq_cfg_lldp_mib_change (struct ice_hw __always_unused *hw, -bool __always_unused ena_update, -struct ice_sq_cd __always_unused *cd) +ice_cfg_lldp_mib_change (struct ice_hw __always_unused *hw, +bool __always_unused ena_mib) {return 0 ; }
drivers/net/ethernet/intel/ice/ice_dcb_lib.c +2 -2
@@ -318,7 +318,7 @@ void ice_dcb_rebuild(struct ice_pf *pf) goto dcb_error; } -ice_init_dcb (&pf->hw); +ice_init_dcb (&pf->hw, true);if (pf->hw.port_info->dcbx_status ==ICE_DCBX_STATUS_DIS ) pf->hw.port_info->is_sw_lldp = true;else
@@ -451,7 +451,7 @@ int ice_init_pf_dcb(struct ice_pf *pf, bool locked) port_info = hw->port_info; - err =ice_init_dcb (hw); + err =ice_init_dcb (hw, false);if (err && !port_info->is_sw_lldp) {dev_err (&pf->pdev->dev,"Error initializing DCB %d\n" , err); goto dcb_init_err;
drivers/net/ethernet/intel/ice/ice_ethtool.c +8 -2
@@ -1206,8 +1206,8 @@ static int ice_set_priv_flags(struct net_device *netdev, u32 flags)enum ice_status status;/* Disable FW LLDP engine */ - status =ice_aq_cfg_lldp_mib_change (&pf->hw, false, -NULL ); + status =ice_cfg_lldp_mib_change (&pf->hw, false); +/* If unregistering for LLDP events fails, this is * not an error state, as there shouldn't be any * events to respond to.
@@ -1273,6 +1273,12 @@ static int ice_set_priv_flags(struct net_device *netdev, u32 flags)* The FW LLDP engine will now be consuming them. */ ice_cfg_sw_lldp (vsi, false, false); + +/* Register for MIB change events */ + status =ice_cfg_lldp_mib_change (&pf->hw, true); +if (status) +dev_dbg (&pf->pdev->dev, +"Fail to enable MIB change events\n" ); } }clear_bit (ICE_FLAG_ETHTOOL_CTXT , pf->flags);
drivers/net/ethernet/intel/ice/ice_main.c +2 -0
@@ -2536,6 +2536,8 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)if (ice_init_pf_dcb (pf, false)) {clear_bit (ICE_FLAG_DCB_CAPABLE , pf->flags);clear_bit (ICE_FLAG_DCB_ENA , pf->flags); + }else { +ice_cfg_lldp_mib_change (&pf->hw, true); } }
Signature · 1 line
2.20.1
Bowers
PATCH v2 6/9 ice: Allow for delayed LLDP MIB change registration
Quoted 20 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf Of Tony Nguyen
Sent: Tuesday, September 3, 2019 1:31 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH S28 v2 6/9] ice: Allow for delayed LLDP MIB change registrationFrom: Dave Ertman <[email protected]>
Add an additional boolean parameter to the
ice_init_dcbfunction. This boolean controls if the LLDP MIB change events are registered for. Also, add a new function definedice_cfg_lldp_mib_change. The additional function is necessary to be able to register for LLDP MIB change events after callingice_init_dcb. The net effect of these two changes is to allow a delayed registration for MIB change events so that the driver is not accepting events before it is ready for them.Signed-off-by Dave Ertman <[email protected]>Signed-off-by Tony Nguyen <[email protected]>
Quoted 7 lines
--- drivers/net/ethernet/intel/ice/ice_dcb.c | 39 ++++++++++++++++++-- drivers/net/ethernet/intel/ice/ice_dcb.h | 11 ++---- drivers/net/ethernet/intel/ice/ice_dcb_lib.c | 4 +- drivers/net/ethernet/intel/ice/ice_ethtool.c | 10 ++++- drivers/net/ethernet/intel/ice/ice_main.c | 2 + 5 files changed, 51 insertions(+), 15 deletions(-)
Tony Nguyen
maintainerPATCH v2 7/9 ice: Minor refactor in queue management
In reply to Tony NguyenFrom: Anirudh Venkataramanan <[email protected]>
Remove q_left_tx and q_left_rx from the PF struct as these can be obtained by calling ice_get_avail_txq_count and ice_get_avail_rxq_count respectively.
The function ice_determine_q_usage is only setting num_lan_tx and num_lan_rx in the PF structure, and these are later assigned to vsi->alloc_txq and vsi->alloc_rxq respectively. This is an unnecessary indirection, so remove ice_determine_q_usage and just assign values for vsi->alloc_txq and vsi->alloc_rxq in ice_vsi_set_num_qs and use these to set num_lan_tx and num_lan_rx respectively.
---
v2:
Move ice_get_avail_txq_count() and ice_get_avail_rxq_count() to ice_main.c to avoid static namespace issues
--- drivers/net/ethernet/intel/ice/ice.h | 4 +- drivers/net/ethernet/intel/ice/ice_lib.c | 25 ++++++---- drivers/net/ethernet/intel/ice/ice_main.c | 50 +++++++++++-------- .../net/ethernet/intel/ice/ice_virtchnl_pf.c | 14 +++--- 4 files changed, 54 insertions(+), 39 deletions(-)
drivers/net/ethernet/intel/ice/ice.h +2 -2
@@ -368,8 +368,6 @@ struct ice_pf {u32 num_lan_msix;/* Total MSIX vectors for base driver */ u16 num_lan_tx;/* num LAN Tx queues setup */ u16 num_lan_rx;/* num LAN Rx queues setup */ -u16 q_left_tx;/* remaining num Tx queues left unclaimed */ -u16 q_left_rx;/* remaining num Rx queues left unclaimed */ u16 next_vsi;/* Next free slot in pf->vsi[] - 0-based! */ u16 num_alloc_vsi;u16 corer_count;/* Core reset count */
@@ -438,6 +436,8 @@ static inline struct ice_vsi *ice_get_main_vsi(struct ice_pf *pf)int ice_vsi_setup_tx_rings (struct ice_vsi *vsi);int ice_vsi_setup_rx_rings (struct ice_vsi *vsi);void ice_set_ethtool_ops (struct net_device *netdev); +u16 ice_get_avail_txq_count (struct ice_pf *pf); +u16 ice_get_avail_rxq_count (struct ice_pf *pf);void ice_update_vsi_stats (struct ice_vsi *vsi);void ice_update_pf_stats (struct ice_pf *pf);int ice_up (struct ice_vsi *vsi);
drivers/net/ethernet/intel/ice/ice_lib.c +14 -11
@@ -343,8 +343,20 @@ static void ice_vsi_set_num_qs(struct ice_vsi *vsi, u16 vf_id)switch (vsi->type) {case ICE_VSI_PF : - vsi->alloc_txq = pf->num_lan_tx; - vsi->alloc_rxq = pf->num_lan_rx; + vsi->alloc_txq =min_t (int,ice_get_avail_txq_count (pf), +num_online_cpus ()); + + pf->num_lan_tx = vsi->alloc_txq; + +/* only 1 Rx queue unless RSS is enabled */ +if (!test_bit (ICE_FLAG_RSS_ENA , pf->flags)) + vsi->alloc_rxq =1 ; +else + vsi->alloc_rxq =min_t (int,ice_get_avail_rxq_count (pf), +num_online_cpus ()); + + pf->num_lan_rx = vsi->alloc_rxq; + vsi->num_q_vectors =max_t (int, vsi->alloc_rxq, vsi->alloc_txq);break ;case ICE_VSI_VF :
@@ -2577,9 +2589,6 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi,if (ret) goto unroll_vector_base; - pf->q_left_tx -= vsi->alloc_txq; - pf->q_left_rx -= vsi->alloc_rxq; -/* Do not exit if configuring RSS had an issue, at least * receive traffic on first queue. Hence no need to capture * return value
@@ -2643,8 +2652,6 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi,ice_vsi_delete (vsi); unroll_get_qs:ice_vsi_put_qs (vsi); - pf->q_left_tx += vsi->alloc_txq; - pf->q_left_rx += vsi->alloc_rxq;ice_vsi_clear (vsi);return NULL ;
@@ -2992,8 +2999,6 @@ int ice_vsi_release(struct ice_vsi *vsi)ice_vsi_clear_rings (vsi);ice_vsi_put_qs (vsi); - pf->q_left_tx += vsi->alloc_txq; - pf->q_left_rx += vsi->alloc_rxq;/* retain SW VSI data structure since it is needed to unregister and * free VSI netdev when PF is not in reset recovery pending state,\
@@ -3102,8 +3107,6 @@ int ice_vsi_rebuild(struct ice_vsi *vsi)if (ret) goto err_vectors; - pf->q_left_tx -= vsi->alloc_txq; - pf->q_left_rx -= vsi->alloc_rxq;break ;default :break ;
drivers/net/ethernet/intel/ice/ice_main.c +30 -20
74 lines not on this page · raw message
drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c +8 -6
@@ -595,7 +595,8 @@ static int ice_alloc_vf_res(struct ice_vf *vf)/* Update number of VF queues, in case VF had requested for queue * changes */ - tx_rx_queue_left =min_t (int, pf->q_left_tx, pf->q_left_rx); + tx_rx_queue_left =min_t (int,ice_get_avail_txq_count (pf), +ice_get_avail_rxq_count (pf)); tx_rx_queue_left +=ICE_DFLT_QS_PER_VF ;if (vf->num_req_qs && vf->num_req_qs <= tx_rx_queue_left && vf->num_req_qs != vf->num_vf_qs)
@@ -898,11 +899,11 @@ static int ice_check_avail_res(struct ice_pf *pf)* at runtime through Virtchnl, that is the reason we start by reserving * few queues. */ - num_txq =ice_determine_res (pf, pf->q_left_tx,ICE_DFLT_QS_PER_VF , -ICE_MIN_QS_PER_VF ); + num_txq =ice_determine_res (pf,ice_get_avail_txq_count (pf), +ICE_DFLT_QS_PER_VF ,ICE_MIN_QS_PER_VF ); - num_rxq =ice_determine_res (pf, pf->q_left_rx,ICE_DFLT_QS_PER_VF , -ICE_MIN_QS_PER_VF ); + num_rxq =ice_determine_res (pf,ice_get_avail_rxq_count (pf), +ICE_DFLT_QS_PER_VF ,ICE_MIN_QS_PER_VF );if (!num_txq || !num_rxq)return -EIO ;
@@ -2511,7 +2512,8 @@ static int ice_vc_request_qs_msg(struct ice_vf *vf, u8 *msg) } cur_queues = vf->num_vf_qs; - tx_rx_queue_left =min_t (u16, pf->q_left_tx, pf->q_left_rx); + tx_rx_queue_left =min_t (u16,ice_get_avail_txq_count (pf), +ice_get_avail_rxq_count (pf)); max_allowed_vf_queues = tx_rx_queue_left + cur_queues;if (!req_queues) { dev_err(&pf->pdev->dev,
Signature · 1 line
2.20.1
Bowers
PATCH v2 7/9 ice: Minor refactor in queue management
Quoted 27 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf Of Tony Nguyen
Sent: Tuesday, September 3, 2019 1:31 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH S28 v2 7/9] ice: Minor refactor in queue managementFrom: Anirudh Venkataramanan <[email protected]>
Remove
q_left_txandq_left_rxfrom the PF struct as these can be obtained by callingice_get_avail_txq_countandice_get_avail_rxq_countrespectively.The function
ice_determine_q_usageis only settingnum_lan_txandnum_lan_rxin the PF structure, and these are later assigned tovsi->alloc_txqandvsi->alloc_rxqrespectively. This is an unnecessary indirection, so removeice_determine_q_usageand just assign values for vsi-Quoted 1 line
alloc_txqandvsi->alloc_rxqinice_vsi_set_num_qsand use these to set
num_lan_txandnum_lan_rxrespectively.Signed-off-by: Anirudh Venkataramanan
<[email protected]>Signed-off-by Tony Nguyen <[email protected]>---
v2:
Moveice_get_avail_txq_count()andice_get_avail_rxq_count()toice_main.cto avoid static namespace issues
Quoted 6 lines
--- drivers/net/ethernet/intel/ice/ice.h | 4 +- drivers/net/ethernet/intel/ice/ice_lib.c | 25 ++++++---- drivers/net/ethernet/intel/ice/ice_main.c | 50 +++++++++++-------- .../net/ethernet/intel/ice/ice_virtchnl_pf.c | 14 +++--- 4 files changed, 54 insertions(+), 39 deletions(-)
Tony Nguyen
maintainerPATCH v2 8/9 ice: change default number of receive descriptors
In reply to Tony NguyenFrom: Jesse Brandeburg <[email protected]>
The driver should start out with a reasonable number of descriptors that can prevent drops due to a CPU being in a power management state. Change the default number of descriptors to 2048.
The user can always change the value at runtime. Transmit descriptor counts are not modified because they don't need to change due to the speed of the interface, or for power managed CPUs, but the code is simplified to a fixed value for the transmit default.
--- drivers/net/ethernet/intel/ice/ice.h | 19 ++----------------- 1 file changed, 2 insertions(+), 17 deletions(-)
drivers/net/ethernet/intel/ice/ice.h +2 -17
@@ -47,23 +47,8 @@ extern const char ice_drv_ver[];#define ICE_MIN_NUM_DESC 64#define ICE_MAX_NUM_DESC 8160#define ICE_DFLT_MIN_RX_DESC 512 -/* if the default number of Rx descriptors between ICE_MAX_NUM_DESC and the -* number of descriptors to fill up an entire page is greater than or equal to -* ICE_DFLT_MIN_RX_DESC set it based on page size, otherwise set it to -* ICE_DFLT_MIN_RX_DESC -*/ -#define ICE_DFLT_NUM_RX_DESC \ - min_t(u16, ICE_MAX_NUM_DESC, \ - max_t(u16, ALIGN(PAGE_SIZE / sizeof(union ice_32byte_rx_desc), \ - ICE_REQ_DESC_MULTIPLE), \ - ICE_DFLT_MIN_RX_DESC)) -/* set default number of Tx descriptors to the minimum between ICE_MAX_NUM_DESC -* and the number of descriptors to fill up an entire page -*/ -#define ICE_DFLT_NUM_TX_DESC min_t(u16, ICE_MAX_NUM_DESC, \ - ALIGN(PAGE_SIZE / \ - sizeof(struct ice_tx_desc), \ - ICE_REQ_DESC_MULTIPLE)) +#define ICE_DFLT_NUM_TX_DESC 256 +#define ICE_DFLT_NUM_RX_DESC 2048#define ICE_DFLT_TRAFFIC_CLASS BIT(0)#define ICE_INT_NAME_STR_LEN (IFNAMSIZ + 16)
Signature · 1 line
2.20.1
Bowers
PATCH v2 8/9 ice: change default number of receive descriptors
Quoted 18 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf Of Tony Nguyen
Sent: Tuesday, September 3, 2019 1:31 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH S28 v2 8/9] ice: change default number of receive descriptorsFrom: Jesse Brandeburg <[email protected]>
The driver should start out with a reasonable number of descriptors that can prevent drops due to a CPU being in a power management state. Change the default number of descriptors to 2048.
The user can always change the value at runtime. Transmit descriptor counts are not modified because they don't need to change due to the speed of the interface, or for power managed CPUs, but the code is simplified to a fixed value for the transmit default.
Quoted 4 lines
Signed-off-by Jesse Brandeburg <[email protected]>--- drivers/net/ethernet/intel/ice/ice.h | 19 ++----------------- 1 file changed, 2 insertions(+), 17 deletions(-)
Tony Nguyen
maintainerPATCH v2 9/9 ice: Rework around device/function capabilities
In reply to Tony NguyenFrom: Anirudh Venkataramanan <[email protected]>
ice_parse_caps is printing capabilities in a different way when compared to the variable names. This makes it difficult to search for the right strings in the debug logs. So this patch updates the print strings to be exactly the same as the fields' name in the structure.
--- drivers/net/ethernet/intel/ice/ice_common.c | 40 ++++++++++----------- 1 file changed, 20 insertions(+), 20 deletions(-)
drivers/net/ethernet/intel/ice/ice_common.c +20 -20
@@ -1551,29 +1551,29 @@ ice_parse_caps(struct ice_hw *hw, void *buf, u32 cap_count,case ICE_AQC_CAPS_VALID_FUNCTIONS : caps->valid_functions = number;ice_debug (hw,ICE_DBG_INIT , -"%s: valid functions = %d\n" , prefix, +"%s: valid_functions (bitmap) = %d\n" , prefix, caps->valid_functions);break ;case ICE_AQC_CAPS_SRIOV : caps->sr_iov_1_1 = (number ==1 );ice_debug (hw,ICE_DBG_INIT , -"%s: SR-IOV = %d\n" , prefix, +"%s: sr_iov_1_1 = %d\n" , prefix, caps->sr_iov_1_1);break ;case ICE_AQC_CAPS_VF :if (dev_p) { dev_p->num_vfs_exposed = number;ice_debug (hw,ICE_DBG_INIT , -"%s: VFs exposed = %d\n" , prefix, +"%s: num_vfs_exposed = %d\n" , prefix, dev_p->num_vfs_exposed); }else if (func_p) { func_p->num_allocd_vfs = number; func_p->vf_base_id = logical_id;ice_debug (hw,ICE_DBG_INIT , -"%s: VFs allocated = %d\n" , prefix, +"%s: num_allocd_vfs = %d\n" , prefix, func_p->num_allocd_vfs);ice_debug (hw,ICE_DBG_INIT , -"%s: VF base_id = %d\n" , prefix, +"%s: vf_base_id = %d\n" , prefix, func_p->vf_base_id); }break ;
@@ -1581,17 +1581,17 @@ ice_parse_caps(struct ice_hw *hw, void *buf, u32 cap_count,if (dev_p) { dev_p->num_vsi_allocd_to_host = number;ice_debug (hw,ICE_DBG_INIT , -"%s: num VSI alloc to host = %d\n" , +"%s: num_vsi_allocd_to_host = %d\n" , prefix, dev_p->num_vsi_allocd_to_host); }else if (func_p) { func_p->guar_num_vsi =ice_get_num_per_func (hw,ICE_MAX_VSI );ice_debug (hw,ICE_DBG_INIT , -"%s: num guaranteed VSI (fw) = %d\n" , +"%s: guar_num_vsi (fw) = %d\n" , prefix, number);ice_debug (hw,ICE_DBG_INIT , -"%s: num guaranteed VSI = %d\n" , +"%s: guar_num_vsi = %d\n" , prefix, func_p->guar_num_vsi); }break ;
@@ -1600,56 +1600,56 @@ ice_parse_caps(struct ice_hw *hw, void *buf, u32 cap_count, caps->active_tc_bitmap = logical_id; caps->maxtc = phys_id;ice_debug (hw,ICE_DBG_INIT , -"%s: DCB = %d\n" , prefix, caps->dcb); +"%s: dcb = %d\n" , prefix, caps->dcb);ice_debug (hw,ICE_DBG_INIT , -"%s: active TC bitmap = %d\n" , prefix, +"%s: active_tc_bitmap = %d\n" , prefix, caps->active_tc_bitmap);ice_debug (hw,ICE_DBG_INIT , -"%s: TC max = %d\n" , prefix, caps->maxtc); +"%s: maxtc = %d\n" , prefix, caps->maxtc);break ;case ICE_AQC_CAPS_RSS : caps->rss_table_size = number; caps->rss_table_entry_width = logical_id;ice_debug (hw,ICE_DBG_INIT , -"%s: RSS table size = %d\n" , prefix, +"%s: rss_table_size = %d\n" , prefix, caps->rss_table_size);ice_debug (hw,ICE_DBG_INIT , -"%s: RSS table width = %d\n" , prefix, +"%s: rss_table_entry_width = %d\n" , prefix, caps->rss_table_entry_width);break ;case ICE_AQC_CAPS_RXQS : caps->num_rxq = number; caps->rxq_first_id = phys_id;ice_debug (hw,ICE_DBG_INIT , -"%s: num Rx queues = %d\n" , prefix, +"%s: num_rxq = %d\n" , prefix, caps->num_rxq);ice_debug (hw,ICE_DBG_INIT , -"%s: Rx first queue ID = %d\n" , prefix, +"%s: rxq_first_id = %d\n" , prefix, caps->rxq_first_id);break ;case ICE_AQC_CAPS_TXQS : caps->num_txq = number; caps->txq_first_id = phys_id;ice_debug (hw,ICE_DBG_INIT , -"%s: num Tx queues = %d\n" , prefix, +"%s: num_txq = %d\n" , prefix, caps->num_txq);ice_debug (hw,ICE_DBG_INIT , -"%s: Tx first queue ID = %d\n" , prefix, +"%s: txq_first_id = %d\n" , prefix, caps->txq_first_id);break ;case ICE_AQC_CAPS_MSIX : caps->num_msix_vectors = number; caps->msix_vector_first_id = phys_id;ice_debug (hw,ICE_DBG_INIT , -"%s: MSIX vector count = %d\n" , prefix, +"%s: num_msix_vectors = %d\n" , prefix, caps->num_msix_vectors);ice_debug (hw,ICE_DBG_INIT , -"%s: MSIX first vector index = %d\n" , prefix, +"%s: msix_vector_first_id = %d\n" , prefix, caps->msix_vector_first_id);break ;case ICE_AQC_CAPS_MAX_MTU : caps->max_mtu = number; -ice_debug (hw,ICE_DBG_INIT ,"%s: max MTU = %d\n" , +ice_debug (hw,ICE_DBG_INIT ,"%s: max_mtu = %d\n" , prefix, caps->max_mtu);break ;default :
Signature · 1 line
2.20.1
Bowers
PATCH v2 9/9 ice: Rework around device/function capabilities
Quoted 15 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf Of Tony Nguyen
Sent: Tuesday, September 3, 2019 1:31 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH S28 v2 9/9] ice: Rework around device/function capabilitiesFrom: Anirudh Venkataramanan <[email protected]>
ice_parse_capsis printing capabilities in a different way when compared to the variable names. This makes it difficult to search for the right strings in the debug logs. So this patch updates the print strings to be exactly the same as the fields' name in the structure.
Quoted 6 lines
Signed-off-by: Anirudh Venkataramanan
<[email protected]>Signed-off-by Tony Nguyen <[email protected]>--- drivers/net/ethernet/intel/ice/ice_common.c | 40 ++++++++++----------- 1 file changed, 20 insertions(+), 20 deletions(-)
Bowers
PATCH v2 1/9 ice: Reliably reset VFs
In reply to Tony NguyenQuoted 21 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf Of Tony Nguyen
Sent: Tuesday, September 3, 2019 1:31 AM
To: intel-wired-lan at lists.osuosl.org
Subject: [Intel-wired-lan] [PATCH S28 v2 1/9] ice: Reliably reset VFsFrom: Mitch Williams <[email protected]>
When a PFR (or bigger reset) occurs, the device clears the
VF_MBX_ARQLENregister for all VFs. But if a VFR is triggered by a VF, the device does NOT clear this register, and the VF driver will never see the reset.When this happens, the VF driver will eventually timeout and attempt recovery, and usually it will be successful. But this makes resets take a long time and there are occasional failures.
We cannot just blithely clear this register on every reset; this has been shown to cause synchronization problems when a PFR is triggered with a large number of VFs.
Quoted 7 lines
Fix this by clearing
VF_MBX_ARQLENwhen the reset source is not PFR. GlobR will trigger PFR, so this test catches that occurrence as well.Signed-off-by Mitch Williams <[email protected]>--- drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)