i40e: Fix for failed to init adminq while VF reset
Subsystems (2)
PATCH 1/1 i40e: Fix for failed to init adminq while VF reset
Fix for failed to init adminq: -53 while VF is resetting via MAC address changing procedure.
Added sync module to avoid reading deadbeef value in reinit adminq during software reset.
Without this patch it is possible to trigger VF reset procedure during reinit adminq. This resulted in an incorrect reading of value from the AQP registers and generated the -53 error.
--- .../ethernet/intel/i40e/i40e_virtchnl_pf.c | 46 ++++++++++++++++++- .../ethernet/intel/i40e/i40e_virtchnl_pf.h | 2 + 2 files changed, 46 insertions(+), 2 deletions(-)
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +44 -2
@@ -1297,6 +1297,32 @@ static i40e_status i40e_config_vf_promiscuous_mode(struct i40e_vf *vf,return aq_ret; } +/** +* i40e_sync_vfr_reset +* @hw: pointer to hw struct +* @vf_id: VF identifier +* +* Before trigger hardware reset, we need to know if no other process has +* reserved the hardware for any reset operations. This check is done by +* examining the status of the ADMINQ bit in VF interrupt register. +**/ +static int i40e_sync_vfr_reset (struct i40e_hw *hw,int vf_id) +{ +u32 reg; +int i; + +for (i =0 ; i <I40E_VFR_WAIT_COUNT ; i++) { + reg =rd32 (hw,I40E_VFINT_ICR0_ENA (vf_id)) & +I40E_VFINT_ICR0_ADMINQ_MASK ; +if (reg) +return 0 ; + +usleep_range (10 ,200 ); + } + +return -EAGAIN ; +} +/** * i40e_trigger_vf_reset * @vf: pointer to the VF structure
@@ -1311,9 +1337,11 @@ static void i40e_trigger_vf_reset(struct i40e_vf *vf, bool flr)struct i40e_pf *pf = vf->pf;struct i40e_hw *hw = &pf->hw;u32 reg, reg_idx, bit_idx; +bool vf_active; +u32 radq;/* warn the VF */ -clear_bit (I40E_VF_STATE_ACTIVE , &vf->vf_states); + vf_active =test_and_clear_bit (I40E_VF_STATE_ACTIVE , &vf->vf_states);/* Disable VF's configuration API during reset. The flag is re-enabled * in i40e_alloc_vf_res(), when it's safe again to access VF's VSI.
@@ -1327,7 +1355,19 @@ static void i40e_trigger_vf_reset(struct i40e_vf *vf, bool flr)* just need to clean up, so don't hit the VFRTRIG register. */ if (!flr) { -/* reset VF using VPGEN_VFRTRIG reg */ +/* Sync VFR reset before trigger next one */ + radq =rd32 (hw,I40E_VFINT_ICR0_ENA (vf->vf_id)) & +I40E_VFINT_ICR0_ADMINQ_MASK ; +if (vf_active && !radq) +/* waiting for finish reset by virtual driver */ +if (i40e_sync_vfr_reset (hw, vf->vf_id)) +dev_info (&pf->pdev->dev, +"Reset VF %d never finished\n" , + vf->vf_id); + +/* Reset VF using VPGEN_VFRTRIG reg. It is also setting +* in progress state in rstat1 register. +*/ reg =rd32 (hw,I40E_VPGEN_VFRTRIG (vf->vf_id)); reg |=I40E_VPGEN_VFRTRIG_VFSWR_MASK ;wr32 (hw,I40E_VPGEN_VFRTRIG (vf->vf_id), reg);
@@ -1457,6 +1497,7 @@ bool i40e_reset_vf(struct i40e_vf *vf, bool flr)i40e_cleanup_reset_vf (vf);i40e_flush (hw); +usleep_range (20000 ,40000 );clear_bit (__I40E_VF_DISABLE, pf->state);return true;
@@ -1561,6 +1602,7 @@ bool i40e_reset_all_vfs(struct i40e_pf *pf, bool flr)i40e_cleanup_reset_vf (&pf->vf[v]);i40e_flush (hw); +usleep_range (20000 ,40000 );clear_bit (__I40E_VF_DISABLE, pf->state);return true;
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h +2 -0
@@ -18,6 +18,8 @@#define I40E_MAX_VF_PROMISC_FLAGS 3 +#define I40E_VFR_WAIT_COUNT 100 +/* Various queue ctrls */ enum i40e_queue_ctrl {I40E_QUEUE_CTRL_UNKNOWN =0 ,
Signature · 7 lines
2.17.1 --------------------------------------------------------------------- Intel Technology Poland sp. z o.o. ul. Sowackiego 173 | 80-298 Gdask | Sd Rejonowy Gdask Pnoc | VII Wydzia Gospodarczy Krajowego Rejestru Sdowego - KRS 101882 | NIP 957-07-52-316 | Kapita zakadowy 200.000 PLN. Ta wiadomo wraz z zacznikami jest przeznaczona dla okrelonego adresata i moe zawiera informacje poufne. W razie przypadkowego otrzymania tej wiadomoci, prosimy o powiadomienie nadawcy oraz trwae jej usunicie; jakiekolwiek przegldanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.
PATCH 1/1 i40e: Fix for failed to init adminq while VF reset
Dear Mateusz,
Am 07.12.20 um 12:42 schrieb Mateusz Palczewski:
Quoted 7 lines
Fix for failed to init adminq: -53 while VF is resetting via MAC address changing procedure.
Added sync module to avoid reading deadbeef value in reinit adminq during software reset.
Without this patch it is possible to trigger VF reset procedure during reinit adminq. This resulted in an incorrect reading of value from the AQP registers and generated the -53 error.
Thank you for fixing this issue. (Blank lines between paragraphs would make them better readable.)
How can the problem be reproduced? Do you have a recipe do cause this?
I have some nits below, but maybe you also can use the summary below for the git commit message summary, so it?s a statement with a verb in imperative mood.
Quoted 1 line
Fix init adminq failure while VF reset
You are adding several delays below. What is the effect on that on overall timings of the driver?
Quoted 19 lines
Fixes 9dc2e417383815"(i40e: split some code in i40e_reset_vf into helpers")Signed-off-by Grzegorz Szczurek <[email protected]>Signed-off-by Mateusz Palczewski <[email protected]>Reviewed-by Jaroslaw Gawin <[email protected]>Reviewed-by Slawomir Laba <[email protected]>Reviewed-by Karen Sornek <[email protected]>Reviewed-by Grzegorz Siwik <[email protected]>Reviewed-by Arkadiusz Kubalewski <[email protected]>Reviewed-by Aleksandr Loktionov <[email protected]>--- .../ethernet/intel/i40e/i40e_virtchnl_pf.c | 46 ++++++++++++++++++- .../ethernet/intel/i40e/i40e_virtchnl_pf.h | 2 + 2 files changed, 46 insertions(+), 2 deletions(-)drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +0
-0
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c static i40e_status i40e_config_vf_promiscuous_mode(struct i40e_vf *vf, · line 1305
Quoted 9 lines
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +6
-0@@ -1297,6 +1297,32 @@ static i40e_status i40e_config_vf_promiscuous_mode(struct i40e_vf *vf,return aq_ret; } +/** +* i40e_sync_vfr_reset +* @hw: pointer to hw struct +* @vf_id: VF identifier +* +* Before trigger hardware reset, we need to know if no other process has
Small typo:
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c static i40e_status i40e_config_vf_promiscuous_mode(struct i40e_vf *vf, · line 1307
Quoted 4 lines
Before trigger*ing a* hardware reset, ?
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +2
-0+* reserved the hardware for any reset operations. This check is done by +* examining the status of the ADMINQ bit in VF interrupt register.
Please add how much time this delays.
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c static i40e_status i40e_config_vf_promiscuous_mode(struct i40e_vf *vf, · line 1318
Quoted 11 lines
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +11
-0+**/ +static int i40e_sync_vfr_reset (struct i40e_hw *hw,int vf_id) +{ +u32 reg; +int i; + +for (i =0 ; i <I40E_VFR_WAIT_COUNT ; i++) { + reg =rd32 (hw,I40E_VFINT_ICR0_ENA (vf_id)) & +I40E_VFINT_ICR0_ADMINQ_MASK ; +if (reg) +return 0 ;
The variable reg does not seem to be needed.
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c static i40e_status i40e_config_vf_promiscuous_mode(struct i40e_vf *vf, · line 1321
Quoted 3 lines
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +3
-0+ +usleep_range (10 ,200 ); + }
Should an error be logged in case of failure?
Quoted 21 lines
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +7
-1+ +return -EAGAIN ; +} +/** * i40e_trigger_vf_reset * @vf: pointer to the VF structure @@ -1311,9 +1337,11 @@ static void i40e_trigger_vf_reset(struct i40e_vf *vf, bool flr)struct i40e_pf *pf = vf->pf;struct i40e_hw *hw = &pf->hw;u32 reg, reg_idx, bit_idx; +bool vf_active; +u32 radq;/* warn the VF */ -clear_bit (I40E_VF_STATE_ACTIVE , &vf->vf_states); + vf_active =test_and_clear_bit (I40E_VF_STATE_ACTIVE , &vf->vf_states);/* Disable VF's configuration API during reset. The flag is re-enabled * in i40e_alloc_vf_res(), when it's safe again to access VF's VSI.
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c static void i40e_trigger_vf_reset(struct i40e_vf *vf, bool flr) · line 1358
Quoted 5 lines
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +1
-1@@ -1327,7 +1355,19 @@ static void i40e_trigger_vf_reset(struct i40e_vf *vf, bool flr)* just need to clean up, so don't hit the VFRTRIG register. */ if (!flr) { -/* reset VF using VPGEN_VFRTRIG reg */ +/* Sync VFR reset before trigger next one */
triggering
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c static void i40e_trigger_vf_reset(struct i40e_vf *vf, bool flr) · line 1362
Quoted 4 lines
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +4
-0+ radq =rd32 (hw,I40E_VFINT_ICR0_ENA (vf->vf_id)) & +I40E_VFINT_ICR0_ADMINQ_MASK ; +if (vf_active && !radq) +/* waiting for finish reset by virtual driver */
waiting for virtual driver to finish reset
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c static void i40e_trigger_vf_reset(struct i40e_vf *vf, bool flr) · line 1365
Quoted 3 lines
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +3
-0+if (i40e_sync_vfr_reset (hw, vf->vf_id)) + dev_info(&pf->pdev->dev, + "Reset VF %d never finished\n",
? after X ms.
Also, please add the consequences of that failure. What does it mean for the user, that the reset was unsuccessful?
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c bool i40e_reset_vf(struct i40e_vf *vf, bool flr) · line 1500
Quoted 13 lines
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +6
-0+ vf->vf_id); + +/* Reset VF using VPGEN_VFRTRIG reg. It is also setting +* in progress state in rstat1 register. +*/ reg =rd32 (hw,I40E_VPGEN_VFRTRIG (vf->vf_id)); reg |=I40E_VPGEN_VFRTRIG_VFSWR_MASK ;wr32 (hw,I40E_VPGEN_VFRTRIG (vf->vf_id), reg);@@ -1457,6 +1497,7 @@ bool i40e_reset_vf(struct i40e_vf *vf, bool flr)i40e_cleanup_reset_vf (vf);i40e_flush (hw); +usleep_range (20000 ,40000 );
This seems unrelated to the commit?
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c bool i40e_reset_all_vfs(struct i40e_pf *pf, bool flr) · line 1605
Quoted 8 lines
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +1
-0clear_bit (__I40E_VF_DISABLE, pf->state);return true;@@ -1561,6 +1602,7 @@ bool i40e_reset_all_vfs(struct i40e_pf *pf, bool flr)i40e_cleanup_reset_vf (&pf->vf[v]);i40e_flush (hw); +usleep_range (20000 ,40000 );
Ditto.
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h line 21
Quoted 12 lines
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c +0
-0clear_bit (__I40E_VF_DISABLE, pf->state);return true;drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h +1
-0@@ -18,6 +18,8 @@#define I40E_MAX_VF_PROMISC_FLAGS 3 +#define I40E_VFR_WAIT_COUNT 100
Why is 100 chosen? The commit message does not say.
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h line 25
Quoted 5 lines
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.h +1
-0+/* Various queue ctrls */ enum i40e_queue_ctrl {I40E_QUEUE_CTRL_UNKNOWN =0 ,
Kind regards,
Paul
kernel test robot
build botThank you for the patch! Yet something to improve:
Hi Mateusz,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on net/master]
url: https://github.com/0day-ci/linux/commits/Mateusz-Palczewski/i40e-Fix-for-failed-to-init-adminq-while-VF-reset/20201207-194429 base: https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git 10c678bd0a035ac2c64a9b26b222f20556227a53 config: alpha-randconfig-r034-20201207 (attached as .config) compiler: alpha-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/3473ae8b20a6b744f90aa0479ad4d66567f8d832
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mateusz-Palczewski/i40e-Fix-for-failed-to-init-adminq-while-VF-reset/20201207-194429 git checkout 3473ae8b20a6b744f90aa0479ad4d66567f8d832
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=alpha
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
In file included from drivers/net/ethernet/intel/i40e/i40e_type.h:8,
from drivers/net/ethernet/intel/i40e/i40e.h:40,
from drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c:4:
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c: In function 'i40e_sync_vfr_reset':
>> drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c:1315:18: error: implicit declaration of function 'I40E_VFINT_ICR0_ENA'; did you mean 'I40E_PFINT_ICR0_ENA'? [-Werror=implicit-function-declaration]
1315 | reg = rd32(hw, I40E_VFINT_ICR0_ENA(vf_id)) &
| ^~~~~~~~~~~~~~~~~~~
drivers/net/ethernet/intel/i40e/i40e_osdep.h:27:45: note: in definition of macro 'rd32'
27 | #define rd32(a, reg) readl((a)->hw_addr + (reg))
| ^~~
>> drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c:1316:7: error: 'I40E_VFINT_ICR0_ADMINQ_MASK' undeclared (first use in this function); did you mean 'I40E_PFINT_ICR0_ADMINQ_MASK'?
1316 | I40E_VFINT_ICR0_ADMINQ_MASK;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
| I40E_PFINT_ICR0_ADMINQ_MASK
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c:1316:7: note: each undeclared identifier is reported only once for each function it appears in
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c: In function 'i40e_trigger_vf_reset':
drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c:1360:8: error: 'I40E_VFINT_ICR0_ADMINQ_MASK' undeclared (first use in this function); did you mean 'I40E_PFINT_ICR0_ADMINQ_MASK'?
1360 | I40E_VFINT_ICR0_ADMINQ_MASK;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
| I40E_PFINT_ICR0_ADMINQ_MASKcc1: some warnings being treated as errors
vim +1315 drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c
1299
1300 /**
1301 * i40e_sync_vfr_reset
1302 * @hw: pointer to hw struct
1303 * @vf_id: VF identifier
1304 *
1305 * Before trigger hardware reset, we need to know if no other process has
1306 * reserved the hardware for any reset operations. This check is done by
1307 * examining the status of the ADMINQ bit in VF interrupt register.
1308 **/
1309 static int i40e_sync_vfr_reset(struct i40e_hw *hw, int vf_id)
1310 {
1311 u32 reg;
1312 int i;
1313
1314 for (i = 0; i < I40E_VFR_WAIT_COUNT; i++) {
> 1315 reg = rd32(hw, I40E_VFINT_ICR0_ENA(vf_id)) & > 1316 I40E_VFINT_ICR0_ADMINQ_MASK;
1317 if (reg)
1318 return 0;
1319
1320 usleep_range(10, 200);
1321 }
1322
1323 return -EAGAIN;
1324 }
1325
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all at lists.01.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: .config.gz
Type: application/gzip
Size: 28591 bytes
Desc: not available
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20201208/5cd5ec94/attachment-0001.bin>