idpf: Acquire the lock before accessing the xn->salt
Subsystems (3)
PATCH 1/1 idpf: Acquire the lock before accessing the xn->salt
The transaction salt was being accessed
before acquiring the idpf_vc_xn_lock when idpf has to forward the virtchnl reply
--- drivers/net/ethernet/intel/idpf/idpf_virtchnl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +2 -1
@@ -612,14 +612,15 @@ idpf_vc_xn_forward_reply(struct idpf_adapter *adapter,return -EINVAL ; } xn = &adapter->vcxn_mngr->ring[xn_idx]; +idpf_vc_xn_lock (xn); salt =FIELD_GET (IDPF_VC_XN_SALT_M , msg_info);if (xn->salt != salt) {dev_err_ratelimited (&adapter->pdev->dev,"Transaction salt does not match (%02x != %02x)\n" , xn->salt, salt); +idpf_vc_xn_unlock (xn);return -EINVAL ; } -idpf_vc_xn_lock (xn);switch (xn->state) {case IDPF_VC_XN_WAITING :/* success */
Signature · 1 line
2.46.0.rc2.264.g509ed76dc8-goog
Przemek Kitszel
maintainerQuoted 3 lines · Manoj Vishwanathan
The transaction salt was being accessed
before acquiring theidpf_vc_xn_lockwhen idpf has to forward the virtchnl reply
nits: This text would fit into two lines (the column limit is 75). Missing dot at the very end.
Quoted 2 lines
Signed-off-by Manoj Vishwanathan <[email protected]>
thank you!, code looks fine
it would be great if you could add a Fixes: tag, and add iwl-net as the target tree
Quoted 25 lines
--- drivers/net/ethernet/intel/idpf/idpf_virtchnl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +2
-1@@ -612,14 +612,15 @@ idpf_vc_xn_forward_reply(struct idpf_adapter *adapter,return -EINVAL ; } xn = &adapter->vcxn_mngr->ring[xn_idx]; +idpf_vc_xn_lock (xn); salt =FIELD_GET (IDPF_VC_XN_SALT_M , msg_info);if (xn->salt != salt) {dev_err_ratelimited (&adapter->pdev->dev,"Transaction salt does not match (%02x != %02x)\n" , xn->salt, salt); +idpf_vc_xn_unlock (xn);return -EINVAL ; } -idpf_vc_xn_lock (xn);switch (xn->state) {case IDPF_VC_XN_WAITING :/* success */
PATCH 1/1 idpf: Acquire the lock before accessing the xn->salt
In reply to Manoj VishwanathanThe transaction salt was being accessed before acquiring the idpf_vc_xn_lock when idpf has to forward the virtchnl reply.
--- drivers/net/ethernet/intel/idpf/idpf_virtchnl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +2 -1
@@ -612,14 +612,15 @@ idpf_vc_xn_forward_reply(struct idpf_adapter *adapter,return -EINVAL ; } xn = &adapter->vcxn_mngr->ring[xn_idx]; +idpf_vc_xn_lock (xn); salt =FIELD_GET (IDPF_VC_XN_SALT_M , msg_info);if (xn->salt != salt) {dev_err_ratelimited (&adapter->pdev->dev,"Transaction salt does not match (%02x != %02x)\n" , xn->salt, salt); +idpf_vc_xn_unlock (xn);return -EINVAL ; } -idpf_vc_xn_lock (xn);switch (xn->state) {case IDPF_VC_XN_WAITING :/* success */
Signature · 1 line
2.46.0.rc2.264.g509ed76dc8-goog
Quoted 5 lines · Manoj Vishwanathan
The transaction salt was being accessed before acquiring the
idpf_vc_xn_lockwhen idpf has to forward the virtchnl reply.Fixes 34c21fa894a1a (“idpf: implement virtchnl transaction manager”)Signed-off-by Manoj Vishwanathan <[email protected]>
Thank you!
(next time please add "v2" (and so on) to subsequent versions of the patch/series; please also send as a standalone thread, instead of in-reply-to v1; no need to repost this one for any of those though)
Quoted 25 lines
--- drivers/net/ethernet/intel/idpf/idpf_virtchnl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +2
-1@@ -612,14 +612,15 @@ idpf_vc_xn_forward_reply(struct idpf_adapter *adapter,return -EINVAL ; } xn = &adapter->vcxn_mngr->ring[xn_idx]; +idpf_vc_xn_lock (xn); salt =FIELD_GET (IDPF_VC_XN_SALT_M , msg_info);if (xn->salt != salt) {dev_err_ratelimited (&adapter->pdev->dev,"Transaction salt does not match (%02x != %02x)\n" , xn->salt, salt); +idpf_vc_xn_unlock (xn);return -EINVAL ; } -idpf_vc_xn_lock (xn);switch (xn->state) {case IDPF_VC_XN_WAITING :/* success */
From: Manoj Vishwanathan <[email protected]>
Date: Mon, 5 Aug 2024 18:21:59 +0000
Quoted 2 lines
The transaction salt was being accessed before acquiring the
idpf_vc_xn_lockwhen idpf has to forward the virtchnl reply.
You need to explain in details here what issue you have faced due to that, how to reproduce it and how this fix does help. Otherwise, it's impossible to suggest what is happening and how to test whether the fix is correct.
Quoted 12 lines
Fixes 34c21fa894a1a (“idpf: implement virtchnl transaction manager”)Signed-off-by Manoj Vishwanathan <[email protected]>--- drivers/net/ethernet/intel/idpf/idpf_virtchnl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +0
-0
drivers/net/ethernet/intel/idpf/idpf_virtchnl.c idpf_vc_xn_forward_reply(struct idpf_adapter *adapter, · line 616
Quoted 5 lines
drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +1
-0@@ -612,14 +612,15 @@ idpf_vc_xn_forward_reply(struct idpf_adapter *adapter,return -EINVAL ; } xn = &adapter->vcxn_mngr->ring[xn_idx]; +idpf_vc_xn_lock (xn); salt =FIELD_GET (IDPF_VC_XN_SALT_M , msg_info);
The lock can be taken here after the FIELD_GET(), not before, to reduce the critical/locked section execution time.
drivers/net/ethernet/intel/idpf/idpf_virtchnl.c idpf_vc_xn_forward_reply(struct idpf_adapter *adapter, · line 626
Quoted 11 lines
drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +1
-1if (xn->salt != salt) {dev_err_ratelimited (&adapter->pdev->dev,"Transaction salt does not match (%02x != %02x)\n" , xn->salt, salt); +idpf_vc_xn_unlock (xn);return -EINVAL ; } -idpf_vc_xn_lock (xn);switch (xn->state) {case IDPF_VC_XN_WAITING :/* success */
Thanks,
Olek
Thanks Przemek & Olek for your quick feedback and responses. Hi Olek,
I can add more details about the issue we faced in the commit message. The bug we had here was a virtchnl delay leading to the xn->salt mismatch. This could be due to several factors including default CPU bounded kworker workqueue for virtchnl message processing being starved by aggressive userspace load causing the virtchnl to be delayed. While debugging this issue, this locking order appeared like a potential issue, hence the change was made.
But, this change is more a clean up we felt based on concurrent access to the virtchnl transaction struct and does not fix the issue. This is more of the patch to do the right thing before we access the "xn". I wanted to start with a first patch to the community for acceptance followed by a series of other patches that are general clean up or improvements to IDPF in general. Will follow with with [PATCH v3]
Thanks,
Manoj
Quoted 47 lines · Alexander Lobakin
From: Manoj Vishwanathan <[email protected]>
Date: Mon, 5 Aug 2024 18:21:59 +0000Quoted 2 lines
The transaction salt was being accessed before acquiring the
idpf_vc_xn_lockwhen idpf has to forward the virtchnl reply.You need to explain in details here what issue you have faced due to that, how to reproduce it and how this fix does help. Otherwise, it's impossible to suggest what is happening and how to test whether the fix is correct.
Quoted 12 lines
Fixes 34c21fa894a1a (“idpf: implement virtchnl transaction manager”)Signed-off-by Manoj Vishwanathan <[email protected]>--- drivers/net/ethernet/intel/idpf/idpf_virtchnl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +0
-0drivers/net/ethernet/intel/idpf/idpf_virtchnl.c idpf_vc_xn_forward_reply(struct idpf_adapter *adapter, · line 616
Quoted 5 lines
drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +1
-0@@ -612,14 +612,15 @@ idpf_vc_xn_forward_reply(struct idpf_adapter *adapter,return -EINVAL ; } xn = &adapter->vcxn_mngr->ring[xn_idx]; +idpf_vc_xn_lock (xn); salt =FIELD_GET (IDPF_VC_XN_SALT_M , msg_info);The lock can be taken here after the
FIELD_GET(), not before, to reduce the critical/locked section execution time.drivers/net/ethernet/intel/idpf/idpf_virtchnl.c idpf_vc_xn_forward_reply(struct idpf_adapter *adapter, · line 626
Quoted 11 lines
drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +1
-1if (xn->salt != salt) {dev_err_ratelimited (&adapter->pdev->dev,"Transaction salt does not match (%02x != %02x)\n" , xn->salt, salt); +idpf_vc_xn_unlock (xn);return -EINVAL ; } -idpf_vc_xn_lock (xn);switch (xn->state) {case IDPF_VC_XN_WAITING :/* success */ Thanks,
Olek
From: Manoj Vishwanathan <[email protected]>
Date: Wed, 7 Aug 2024 06:58:59 -0700
Quoted 6 lines
Thanks Przemek & Olek for your quick feedback and responses. Hi Olek,
I can add more details about the issue we faced in the commit message. The bug we had here was a virtchnl delay leading to thexn->saltmismatch. This could be due to several factors including default CPU bounded kworker workqueue for virtchnl message processing being
That's why I always tell people to stop creating more and more private workqueues and just use the system ones, there's a whole collection for every need...
Quoted 9 lines
starved by aggressive userspace load causing the virtchnl to be delayed. While debugging this issue, this locking order appeared like a potential issue, hence the change was made.
But, this change is more a clean up we felt based on concurrent access to the virtchnl transaction struct and does not fix the issue. This is more of the patch to do the right thing before we access the "xn". I wanted to start with a first patch to the community for acceptance followed by a series of other patches that are general clean up or improvements to IDPF in general. Will follow with with [PATCH v3]
Maybe you'd prepare a full series then right away?
I hope it won't conflict much with my tree (but you always can double-check[0]) (Chapter II is already posted here on IWL and netdev@)
Quoted 53 lines
Thanks,
ManojQuoted 47 lines · Alexander Lobakin
From: Manoj Vishwanathan <[email protected]>
Date: Mon, 5 Aug 2024 18:21:59 +0000Quoted 2 lines
The transaction salt was being accessed before acquiring the
idpf_vc_xn_lockwhen idpf has to forward the virtchnl reply.You need to explain in details here what issue you have faced due to that, how to reproduce it and how this fix does help. Otherwise, it's impossible to suggest what is happening and how to test whether the fix is correct.
Quoted 12 lines
Fixes 34c21fa894a1a (“idpf: implement virtchnl transaction manager”)Signed-off-by Manoj Vishwanathan <[email protected]>--- drivers/net/ethernet/intel/idpf/idpf_virtchnl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +0
-0drivers/net/ethernet/intel/idpf/idpf_virtchnl.c idpf_vc_xn_forward_reply(struct idpf_adapter *adapter, · line 616
Quoted 5 lines
drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +1
-0@@ -612,14 +612,15 @@ idpf_vc_xn_forward_reply(struct idpf_adapter *adapter,return -EINVAL ; } xn = &adapter->vcxn_mngr->ring[xn_idx]; +idpf_vc_xn_lock (xn); salt =FIELD_GET (IDPF_VC_XN_SALT_M , msg_info);The lock can be taken here after the
FIELD_GET(), not before, to reduce the critical/locked section execution time.drivers/net/ethernet/intel/idpf/idpf_virtchnl.c idpf_vc_xn_forward_reply(struct idpf_adapter *adapter, · line 626
Quoted 11 lines
drivers/net/ethernet/intel/idpf/idpf_virtchnl.c +1
-1if (xn->salt != salt) {dev_err_ratelimited (&adapter->pdev->dev,"Transaction salt does not match (%02x != %02x)\n" , xn->salt, salt); +idpf_vc_xn_unlock (xn);return -EINVAL ; } -idpf_vc_xn_lock (xn);switch (xn->state) {case IDPF_VC_XN_WAITING :/* success */ Thanks,
Olek
[0] https://github.com/alobakin/linux/commits/idpf-libie-new
Thanks,
Olek
Simon Horman
reviewerQuoted 7 lines · Manoj Vishwanathan
Thanks Przemek & Olek for your quick feedback and responses. Hi Olek,
I can add more details about the issue we faced in the commit message. The bug we had here was a virtchnl delay leading to thexn->saltmismatch. This could be due to several factors including default CPU bounded kworker workqueue for virtchnl message processing being starved by aggressive userspace load causing the virtchnl to be
Quoted 8 lines
delayed. While debugging this issue, this locking order appeared like a potential issue, hence the change was made.
But, this change is more a clean up we felt based on concurrent access to the virtchnl transaction struct and does not fix the issue. This is more of the patch to do the right thing before we access the "xn". I wanted to start with a first patch to the community for acceptance followed by a series of other patches that are general clean up or improvements to IDPF in general. Will follow with with [PATCH v3]
Still, I am a little confused about the protection offered to xn->salt.
My analysis is as follows, where guarded is used loosely to mean the lock is held.
* In idpf_vc_xn_pop_free() it is guarded by vcxn_mngr->xn_bm_lock.
* In idpf_vc_xn_exec() it is guarded by:
1. vcxn_mngr->xn_bm_lock when idpf_vc_xn_pop_free is called
2. idpf_vc_xn_lock, otherwise
* And with this patch, in idpf_vc_xn_forward_reply it is guarded by idpf_vc_xn_lock().
This doesn't seem entirely consistent.
Also, please don't top-post on Kernel mailing lists.
...