wireless: ath10k: Return early in ath10k_qmi_event_server_exit() to avoid hard crash on reboot
PATCH 1/1 wireless: ath10k: Return early in ath10k_qmi_event_server_exit() to avoid hard crash on reboot
Ever since 5.7-rc1, if we call ath10k_qmi_remove_msa_permission(), the db845c hard crashes on reboot, resulting in the device getting stuck in the usb crash debug mode and not coming back up wihthout a hard power off.
This hack avoids the issue by returning early inath10k_qmi_event_server_exit().
A better solution is very much desired!
Feedback and suggestions welcome!
--- drivers/net/wireless/ath/ath10k/qmi.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/net/wireless/ath/ath10k/qmi.c +5 -0
@@ -854,6 +854,11 @@ static void ath10k_qmi_event_server_exit(struct ath10k_qmi *qmi)struct ath10k *ar = qmi->ar;struct ath10k_snoc *ar_snoc =ath10k_snoc_priv (ar); +/* +* HACK: Calling ath10k_qmi_remove_msa_permission causes +* hardware to hard crash on reboot +*/ +return ;ath10k_qmi_remove_msa_permission (qmi);ath10k_core_free_board_files (ar);if (!test_bit (ATH10K_SNOC_FLAG_UNREGISTERING , &ar_snoc->flags))
Signature · 1 line
2.17.1
+ Sibi
Quoted 10 lines · John Stultz
Ever since 5.7-rc1, if we call
ath10k_qmi_remove_msa_permission(), the db845c hard crashes on reboot, resulting in the device getting stuck in the usb crash debug mode and not coming back up wihthout a hard power off.This hack avoids the issue by returning early in
ath10k_qmi_event_server_exit().A better solution is very much desired!
Any chance you can bisect what caused this? There are a lot of non-ath10k pieces involved in this stuff.
Brian
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 10 lines · Brian Norris
+ Sibi
Quoted 6 lines · John Stultz
Ever since 5.7-rc1, if we call
ath10k_qmi_remove_msa_permission(), the db845c hard crashes on reboot, resulting in the device getting stuck in the usb crash debug mode and not coming back up wihthout a hard power off.
Quoted 7 lines
Quoted 4 lines
This hack avoids the issue by returning early in
ath10k_qmi_event_server_exit().A better solution is very much desired!
Any chance you can bisect what caused this? There are a lot of non-ath10k pieces involved in this stuff.
Amit had spent some work on chasing it down to the in kernel qrtr-ns work, and reported it here:
https://lists.infradead.org/pipermail/ath10k/2020-April/014970.html
But that discussion seemingly stalled out, so I came up with this hack to workaround it for us.
thanks
-john
Quoted 16 lines · John Stultz
Quoted 7 lines · Brian Norris
Quoted 6 lines · John Stultz
Ever since 5.7-rc1, if we call
ath10k_qmi_remove_msa_permission(), the db845c hard crashes on reboot, resulting in the device getting stuck in the usb crash debug mode and not coming back up wihthout a hard power off.Quoted 7 lines
Quoted 4 lines
This hack avoids the issue by returning early in
ath10k_qmi_event_server_exit().A better solution is very much desired!
Any chance you can bisect what caused this? There are a lot of non-ath10k pieces involved in this stuff.
Quoted 6 lines
Amit had spent some work on chasing it down to the in kernel qrtr-ns work, and reported it here:
https://lists.infradead.org/pipermail/ath10k/2020-April/014970.htmlBut that discussion seemingly stalled out, so I came up with this hack to workaround it for us.
If I'm reading it right, then that means we should revert this stuff from v5.7-rc1:
0c2204a4ad71 net: qrtr: Migrate nameservice to kernel from userspace
At least, until people can resolve the tail end of that thread. New features (ath11k, etc.) are not a reason to break existing features (ath10k/wcn3990).
Brian
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 24 lines · Brian Norris
Quoted 16 lines · John Stultz
Quoted 7 lines · Brian Norris
Quoted 6 lines · John Stultz
Ever since 5.7-rc1, if we call
ath10k_qmi_remove_msa_permission(), the db845c hard crashes on reboot, resulting in the device getting stuck in the usb crash debug mode and not coming back up wihthout a hard power off.Quoted 7 lines
Quoted 4 lines
This hack avoids the issue by returning early in
ath10k_qmi_event_server_exit().A better solution is very much desired!
Any chance you can bisect what caused this? There are a lot of non-ath10k pieces involved in this stuff.
Quoted 6 lines
Amit had spent some work on chasing it down to the in kernel qrtr-ns work, and reported it here:
https://lists.infradead.org/pipermail/ath10k/2020-April/014970.htmlBut that discussion seemingly stalled out, so I came up with this hack to workaround it for us.
Quoted 8 lines
If I'm reading it right, then that means we should revert this stuff from v5.7-rc1:
0c2204a4ad71net: qrtr: Migrate nameservice to kernel from userspaceAt least, until people can resolve the tail end of that thread. New features (ath11k, etc.) are not a reason to break existing features (ath10k/wcn3990).
I don't agree with this. If you read through the replies to the bug report, it is clear that NS migration uncovered a corner case or even a bug. So we should try to fix that indeed.
Govind: Did you get chance to work on fixing this issue?
Thanks,
Mani
Quoted 2 lines
Brian
Hi Mani,
Quoted 34 lines · Manivannan Sadhasivam
Quoted 24 lines · Brian Norris
Quoted 16 lines · John Stultz
Quoted 7 lines · Brian Norris
Quoted 6 lines · John Stultz
Ever since 5.7-rc1, if we call
ath10k_qmi_remove_msa_permission(), the db845c hard crashes on reboot, resulting in the device getting stuck in the usb crash debug mode and not coming back up wihthout a hard power off.Quoted 7 lines
Quoted 4 lines
This hack avoids the issue by returning early in
ath10k_qmi_event_server_exit().A better solution is very much desired!
Any chance you can bisect what caused this? There are a lot of non-ath10k pieces involved in this stuff.
Quoted 6 lines
Amit had spent some work on chasing it down to the in kernel qrtr-ns work, and reported it here:
https://lists.infradead.org/pipermail/ath10k/2020-April/014970.htmlBut that discussion seemingly stalled out, so I came up with this hack to workaround it for us.
Quoted 8 lines
If I'm reading it right, then that means we should revert this stuff from v5.7-rc1:
0c2204a4ad71net: qrtr: Migrate nameservice to kernel from userspaceAt least, until people can resolve the tail end of that thread. New features (ath11k, etc.) are not a reason to break existing features (ath10k/wcn3990).
Quoted 6 lines
I don't agree with this. If you read through the replies to the bug report, it is clear that NS migration uncovered a corner case or even a bug. So we should try to fix that indeed.
Govind: Did you get chance to work on fixing this issue?
I have done basic testing by moving msa map/unmap from qmi service callbacks to init/de-init path. I will send patch for review.
Reason for del_server needs to investigated from rproc side.
Quoted 5 lines
Thanks,
ManiQuoted 2 lines
Brian
Thanks,
Govind
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 41 lines · On 2020-06-03 15:37, [email protected] wrote:
Hi Mani,
Quoted 38 lines · Manivannan Sadhasivam
Quoted 24 lines · Brian Norris
Quoted 16 lines · John Stultz
Quoted 7 lines · Brian Norris
Quoted 6 lines · John Stultz
Ever since 5.7-rc1, if we call
ath10k_qmi_remove_msa_permission(), the db845c hard crashes on reboot, resulting in the device getting stuck in the usb crash debug mode and not coming back up wihthout a hard power off.Quoted 7 lines
Quoted 4 lines
This hack avoids the issue by returning early in
ath10k_qmi_event_server_exit().A better solution is very much desired!
Any chance you can bisect what caused this? There are a lot of non-ath10k pieces involved in this stuff.
Quoted 6 lines
Amit had spent some work on chasing it down to the in kernel qrtr-ns work, and reported it here:
https://lists.infradead.org/pipermail/ath10k/2020-April/014970.htmlBut that discussion seemingly stalled out, so I came up with this hack to workaround it for us.
Quoted 8 lines
If I'm reading it right, then that means we should revert this stuff from v5.7-rc1:
0c2204a4ad71net: qrtr: Migrate nameservice to kernel from userspaceAt least, until people can resolve the tail end of that thread. New features (ath11k, etc.) are not a reason to break existing features (ath10k/wcn3990).
I don't agree with this. If you read through the replies to the bug report, it is clear that NS migration uncovered a corner case or even a bug. So we should try to fix that indeed.
Quoted 7 lines
Quoted 2 lines
Govind: Did you get chance to work on fixing this issue?
I have done basic testing by moving msa map/unmap from qmi service callbacks to
init/de-initpath.
I will send patch for review.
Reason fordel_serverneeds to investigated from rproc side.
Govind,
On receiving SIGTERM, rmtfs would try
to perform a graceful shutdown of the
modem, that should be the source of
the del_server.
Quoted 9 lines
Quoted 6 lines
Thanks,
ManiQuoted 2 lines
Brian
Thanks,
Govind
Signature · 7 lines
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project. _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 30 lines · Manivannan Sadhasivam
Quoted 29 lines · Brian Norris
Quoted 16 lines · John Stultz
Quoted 7 lines · Brian Norris
Quoted 6 lines · John Stultz
Ever since 5.7-rc1, if we call
ath10k_qmi_remove_msa_permission(), the db845c hard crashes on reboot, resulting in the device getting stuck in the usb crash debug mode and not coming back up wihthout a hard power off.Quoted 7 lines
Quoted 4 lines
This hack avoids the issue by returning early in
ath10k_qmi_event_server_exit().A better solution is very much desired!
Any chance you can bisect what caused this? There are a lot of non-ath10k pieces involved in this stuff.
Quoted 6 lines
Amit had spent some work on chasing it down to the in kernel qrtr-ns work, and reported it here:
https://lists.infradead.org/pipermail/ath10k/2020-April/014970.htmlBut that discussion seemingly stalled out, so I came up with this hack to workaround it for us.
If I'm reading it right, then that means we should revert this stuff from v5.7-rc1:
0c2204a4ad71net: qrtr: Migrate nameservice to kernel from userspace
Quoted 7 lines
Quoted 3 lines
At least, until people can resolve the tail end of that thread. New features (ath11k, etc.) are not a reason to break existing features (ath10k/wcn3990).
I don't agree with this. If you read through the replies to the bug report, it is clear that NS migration uncovered a corner case or even a bug. So we should try to fix that indeed.
I'm with Mani, we should try to fix ath10k instead. Hopefully we can find a fix soon.
Forcing QCA6390 users to use the userspace qrtr-ns would be bad user experience, I really would want to avoid that.
Signature · 6 lines
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 6 lines · Kalle Valo
Quoted 3 lines
I don't agree with this. If you read through the replies to the bug report, it is clear that NS migration uncovered a corner case or even a bug. So we should try to fix that indeed.
I'm with Mani, we should try to fix ath10k instead. Hopefully we can find a fix soon.
Hi Team,
Any updates on this? I can reproduce this hard crash on v5.9-rc1 as well.
It is not a blocker for us because we switched to a userspace workaround, where we do not wait for modem to shutdown gracefully and SIGKILL it instead, during the shutdown/reboot process. But I'm happy to take a swing at any intermediate/in-progress solution available.
Regards,
Amit Pundir
Quoted 6 lines
Forcing QCA6390 users to use the userspace qrtr-ns would be bad user experience, I really would want to avoid that.
--
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches
Quoted 8 lines · Amit Pundir
Quoted 6 lines · Kalle Valo
Quoted 3 lines
I don't agree with this. If you read through the replies to the bug report, it is clear that NS migration uncovered a corner case or even a bug. So we should try to fix that indeed.
I'm with Mani, we should try to fix ath10k instead. Hopefully we can find a fix soon.
Quoted 8 lines
Hi Team,
Any updates on this? I can reproduce this hard crash on v5.9-rc1 as well.
It is not a blocker for us because we switched to a userspace workaround, where we do not wait for modem to shutdown gracefully and SIGKILL it instead, during the shutdown/reboot process. But I'm happy to take a swing at any intermediate/in-progress solution available.
Govind submitted this patch and later he asked to drop it, but I think it would be a good idea to test it anyway:
ath10k: Move msa region map/unmap to init/deinit path
https://lkml.kernel.org/r/[email protected]
(patchwork is down so I cannot give a patchwork link)
Signature · 6 lines
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Hi Kalle,
Quoted 19 lines · Kalle Valo
Quoted 8 lines · Amit Pundir
Quoted 6 lines · Kalle Valo
Quoted 3 lines
I don't agree with this. If you read through the replies to the bug report, it is clear that NS migration uncovered a corner case or even a bug. So we should try to fix that indeed.
I'm with Mani, we should try to fix ath10k instead. Hopefully we can find a fix soon.
Quoted 8 lines
Hi Team,
Any updates on this? I can reproduce this hard crash on v5.9-rc1 as well.
It is not a blocker for us because we switched to a userspace workaround, where we do not wait for modem to shutdown gracefully and SIGKILL it instead, during the shutdown/reboot process. But I'm happy to take a swing at any intermediate/in-progress solution available.
Quoted 8 lines
Govind submitted this patch and later he asked to drop it, but I think it would be a good idea to test it anyway:
ath10k: Move msa region map/unmap to init/deinit path
https://lkml.kernel.org/r/[email protected]
(patchwork is down so I cannot give a patchwork link)
This patchwork is not fixing the issue and changing MSA mapping sequence is major design change.
This issue is only seen with DB845 which uses SCM call, newer targets QCS404/SC7180/SM8150 will not have this issue as MSA mapping is hard-coded in TZ. Probably changes in qmi layer to give different indication for this scenario and changes in FW is required to mitigate this issue gracefully.
BR,
Govind
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 28 lines · Govind Singh
Quoted 27 lines · Kalle Valo
Quoted 10 lines · Amit Pundir
Quoted 6 lines · Kalle Valo
Quoted 3 lines
I don't agree with this. If you read through the replies to the bug report, it is clear that NS migration uncovered a corner case or even a bug. So we should try to fix that indeed.
I'm with Mani, we should try to fix ath10k instead. Hopefully we can find a fix soon.
Hi Team,
Quoted 7 lines
Any updates on this? I can reproduce this hard crash on v5.9-rc1 as well.
It is not a blocker for us because we switched to a userspace workaround, where we do not wait for modem to shutdown gracefully and SIGKILL it instead, during the shutdown/reboot process. But I'm happy to take a swing at any intermediate/in-progress solution available.
Govind submitted this patch and later he asked to drop it, but I think it would be a good idea to test it anyway:
ath10k: Move msa region map/unmap to init/deinit path
Quoted 8 lines
Quoted 1 line
(patchwork is down so I cannot give a patchwork link)
This patchwork is not fixing the issue and changing MSA mapping sequence is major design change. This issue is only seen with DB845 which uses SCM call, newer targets QCS404/SC7180/SM8150 will not have this issue as MSA mapping is hard-coded in TZ. Probably changes in qmi layer to give different indication for this scenario and changes in FW is required to mitigate this issue gracefully.
Oh, bad news :/ Can anyone look at that in detail? Even a quick hack patch would get this forward.
Signature · 6 lines
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 28 lines · Govind Singh
Quoted 27 lines · Kalle Valo
Quoted 10 lines · Amit Pundir
Quoted 6 lines · Kalle Valo
Quoted 3 lines
I don't agree with this. If you read through the replies to the bug report, it is clear that NS migration uncovered a corner case or even a bug. So we should try to fix that indeed.
I'm with Mani, we should try to fix ath10k instead. Hopefully we can find a fix soon.
Hi Team,
Quoted 7 lines
Any updates on this? I can reproduce this hard crash on v5.9-rc1 as well.
It is not a blocker for us because we switched to a userspace workaround, where we do not wait for modem to shutdown gracefully and SIGKILL it instead, during the shutdown/reboot process. But I'm happy to take a swing at any intermediate/in-progress solution available.
Govind submitted this patch and later he asked to drop it, but I think it would be a good idea to test it anyway:
ath10k: Move msa region map/unmap to init/deinit path
Quoted 8 lines
Quoted 1 line
(patchwork is down so I cannot give a patchwork link)
This patchwork is not fixing the issue and changing MSA mapping sequence is major design change. This issue is only seen with DB845 which uses SCM call, newer targets QCS404/SC7180/SM8150 will not have this issue as MSA mapping is hard-coded in TZ. Probably changes in qmi layer to give different indication for this scenario and changes in FW is required to mitigate this issue gracefully.
Oh, bad news :/ Can anyone look at that in detail? Even a quick hack patch would get this forward.
Signature · 1 line
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches
Quoted 12 lines · John Stultz
Ever since 5.7-rc1, if we call
ath10k_qmi_remove_msa_permission(), the db845c hard crashes on reboot, resulting in the device getting stuck in the usb crash debug mode and not coming back up wihthout a hard power off.This hack avoids the issue by returning early in
ath10k_qmi_event_server_exit().A better solution is very much desired!
Feedback and suggestions welcome!
Quoted 11 lines
Cc Rakesh Pillai <[email protected]>Cc Govind Singh <[email protected]>Cc Bjorn Andersson <[email protected]>Cc Niklas Cassel <[email protected]>Cc Manivannan Sadhasivam <[email protected]>Cc Amit Pundir <[email protected]>Cc Brian Norris <[email protected]>Cc Kalle Valo <[email protected]>Reported-by Amit Pundir <[email protected]>Signed-off-by John Stultz <[email protected]>
Just so you know: as you didn't CC linux-wireless it's not on patchwork and hence not on my radar. But hopefully we find a better solution to fix this.
Signature · 1 line
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches