Frequent ath10k crashes with QCA6174 on kernels 4.16 and 4.17
Hi,
I'm experiencing a lot of ath10k driver crashes on both 4.16 (as shipped in Fedora) and 4.17 (tried rc5 and rc6) using a Dell XPS 13 9370 with a QCA6174.
With 4.16 the driver crash always results in the whole system hanging. With 4.17 it does not always hang the system completely, but it still happends. I also tried to upgrade the firmware which also reduces the frequency of the crashes but does not eliminate them completely.
The common pattern I see in dmesg is this:
[30372.900832] wlp2s0: AP 64:7c:34:3f:c3:b0 changed bandwidth, new config is 2437 MHz, width 2 (2447/0 MHz)
Searching a bit in kernel sources, this message is printed by this line:
https://github.com/torvalds/linux/blob/master/net/mac80211/mlme.c#L364
And width 2 seems to correspond to NL80211_CHAN_WIDTH_40 here:
https://github.com/torvalds/linux/blob/master/include/uapi/linux/nl80211.h#L3880
So it seems that either the driver or the firmware can't cope with it when the AP changes the bandwidth. Here is the Fedora bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1577106
Is there anything that can be done to improve the stability of ath10k on this system? I see there is a lot going on in the upstream repo, but not sure which branch to try: ath-next or qca?
Best regards,
Tim
ps. I'm not on the mailing list, please add me to CC when you reply. Thanks.
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
+ ryan
Quoted 18 lines · Timur Kristóf
I'm experiencing a lot of ath10k driver crashes on both 4.16 (as shipped in Fedora) and 4.17 (tried rc5 and rc6) using a Dell XPS 13 9370 with a QCA6174.
With 4.16 the driver crash always results in the whole system hanging. With 4.17 it does not always hang the system completely, but it still happends. I also tried to upgrade the firmware which also reduces the frequency of the crashes but does not eliminate them completely.
The common pattern I see in dmesg is this:
[30372.900832] wlp2s0: AP 64:7c:34:3f:c3:b0 changed bandwidth, newconfig is 2437 MHz, width 2 (2447/0 MHz)
Searching a bit in kernel sources, this message is printed by this line:
https://github.com/torvalds/linux/blob/master/net/mac80211/mlme.c#L364
And width 2 seems to correspond toNL80211_CHAN_WIDTH_40here:
https://github.com/torvalds/linux/blob/master/include/uapi/linux/nl80211.h#L3880
Quoted 8 lines
So it seems that either the driver or the firmware can't cope with it when the AP changes the bandwidth.
Here is the Fedora bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1577106Is there anything that can be done to improve the stability of ath10k on this system? I see there is a lot going on in the upstream repo, but not sure which branch to try: ath-next or qca?
I think this patch should fix the firmware crash:
ath10k: Update the phymode along with bandwidth change request
https://patchwork.kernel.org/patch/10183453/
Unfortunately it didn't apply to my tree so I had to drop it. Few days ago I was talking with Ryan and he said he is planning to send v2, but I'm sure he don't mind if someone else can send the v2 either. The faster we can apply the patch, and get it to stable releases, the better as we seems to be a very common issue.
But there also seems to be another bug related to firmware restart, it shouldn't crash the kernel! Does the kernel crash also when crashing the kernel using simulate_fw_crash debugfs file?
Signature · 6 lines
Kalle Valo _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 18 lines · Kalle Valo
Quoted 11 lines
So it seems that either the driver or the firmware can't cope with it
when the AP changes the bandwidth.
Here is the Fedora bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1577106Is there anything that can be done to improve the stability of ath10k on this system?
I see there is a lot going on in the upstream repo, but not sure which
branch to try: ath-next or qca?I think this patch should fix the firmware crash:
ath10k: Update the phymode along with bandwidth change request
Quoted 8 lines
Unfortunately it didn't apply to my tree so I had to drop it. Few days
ago I was talking with Ryan and he said he is planning to send v2, but
I'm sure he don't mind if someone else can send the v2 either. The faster we can apply the patch, and get it to stable releases, the better
as we seems to be a very common issue.
I don't have much experience with this, but will take a look and let you know how that goes.
Quoted 5 lines
But there also seems to be another bug related to firmware restart, it
shouldn't crash the kernel! Does the kernel crash also when crashing the
kernel usingsimulate_fw_crashdebugfs file?
Yes, tried it just now. The driver can survive 2 simulated hard crashes. After the 3rd hard crash, it reproduces the same issue that I described in the bugreport: Wifi stops working. I can turn it off. When I turn it on again, the whole system hangs completely.
You would expect that a single driver crash should not affect the whole system like this, but it does.
Is there a chance this is already fixed upstream? In general, which of your branches should I try to use with this device? ath-next or ath-qca?
Thanks & best regards,
Timur
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 19 lines · Timur Kristóf
Quoted 18 lines · Kalle Valo
Quoted 11 lines
So it seems that either the driver or the firmware can't cope with it
when the AP changes the bandwidth.
Here is the Fedora bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1577106Is there anything that can be done to improve the stability of ath10k on this system?
I see there is a lot going on in the upstream repo, but not sure which
branch to try: ath-next or qca?I think this patch should fix the firmware crash:
ath10k: Update the phymode along with bandwidth change request
Quoted 8 lines
Quoted 5 lines
Unfortunately it didn't apply to my tree so I had to drop it. Few days ago I was talking with Ryan and he said he is planning to send v2, but I'm sure he don't mind if someone else can send the v2 either. The faster we can apply the patch, and get it to stable releases, the better as we seems to be a very common issue.
I don't have much experience with this, but will take a look and let you know how that goes.
Ok, let me know how it goes. Some info and pointers here:
https://wireless.wiki.kernel.org/en/users/drivers/ath10k/submittingpatches
Quoted 6 lines
Quoted 5 lines
But there also seems to be another bug related to firmware restart, it
shouldn't crash the kernel! Does the kernel crash also when crashing the
kernel usingsimulate_fw_crashdebugfs file?
Quoted 7 lines
Yes, tried it just now. The driver can survive 2 simulated hard crashes. After the 3rd hard crash, it reproduces the same issue that I described in the bugreport: Wifi stops working. I can turn it off. When I turn it on again, the whole system hangs completely.
You would expect that a single driver crash should not affect the whole system like this, but it does.
Yeah, it definitely should not crash like that. I wonder what has broken it.
Quoted 3 lines
Is there a chance this is already fixed upstream? In general, which of your branches should I try to use with this device? ath-next or ath-qca?
I doubt neither of the bugs (the firmware crash and the kernel hang) are fixed, at least I don't recall seeing any fixes. But if you have time to test, use the master branch of my ath.git tree for any testing.
https://wireless.wiki.kernel.org/en/users/drivers/ath10k/sources
Signature · 6 lines
Kalle Valo _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 6 lines · Kalle Valo
Quoted 2 lines · Timur Kristóf
I don't have much experience with this, but will take a look and let you know how that goes.
Ok, let me know how it goes. Some info and pointers here:
https://wireless.wiki.kernel.org/en/users/drivers/ath10k/submittingpatches
Thanks!
It does not apply to 4.17-rc6 as-is, either, but it seems fixable at the first glance.
Quoted 4 lines
Quoted 2 lines
You would expect that a single driver crash should not affect the whole system like this, but it does.
Yeah, it definitely should not crash like that. I wonder what has broken it.
Do you mean that this used to work correctly at some point? If that is the case, can you tell me which version it worked with?
Quoted 3 lines
I doubt neither of the bugs (the firmware crash and the kernel hang) are fixed, at least I don't recall seeing any fixes. But if you have time to test, use the master branch of my ath.git tree for any testing.
Allright, I will take a look at master as well. Were you able to reproduce these issues on master?
Thanks & best regards,
Timur
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 6 lines · Timur Kristóf
Quoted 5 lines · Kalle Valo
Quoted 3 lines · Timur Kristóf
I don't have much experience with this, but will take a look and let
you know how that goes.
Quoted 8 lines
Quoted 3 lines
Ok, let me know how it goes. Some info and pointers here:
https://wireless.wiki.kernel.org/en/users/drivers/ath10k/submitting
patchesThanks!
It does not apply to 4.17-rc6 as-is, either, but it seems fixable at the first glance.
Basically, the author forgot to define WMI_PEER_PHYMODE but some searching found another QCA driver where it is 0xd so I decided to go with that. The other error was trivial.
I fixed the patch and applied locally on top of 4.17-rc7. Currently running it to test if it really mitigates the issue. Will report back when I have results, and submit the patch if I'm satisfied.
Cheers,
Tim
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 6 lines · Timur Kristóf
Quoted 5 lines · Timur Kristóf
Thanks!
It does not apply to 4.17-rc6 as-is, either, but it seems fixable at the first glance.
Quoted 8 lines
Basically, the author forgot to define
WMI_PEER_PHYMODEbut some searching found another QCA driver where it is0xdso I decided to go with that. The other error was trivial.I fixed the patch and applied locally on top of 4.17-rc7. Currently running it to test if it really mitigates the issue. Will report back when I have results, and submit the patch if I'm satisfied.
Sorry, I've been dragged by something else and didn't update this promptly. I just sent the v2 including the missing WMI_PEER_PHYMODE.
Can you pick up the v2 and see if that help your issue?
https://patchwork.kernel.org/patch/10454967/
Signature · 7 lines
Ryan Hsu _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 21 lines · Ryan Hsu
Quoted 13 lines · Timur Kristóf
Quoted 6 lines · Timur Kristóf
Thanks!
It does not apply to 4.17-rc6 as-is, either, but it seems fixable at
the first glance.Basically, the author forgot to define
WMI_PEER_PHYMODEbut some searching found another QCA driver where it is0xdso I decided to go
with that. The other error was trivial.Quoted 5 lines
I fixed the patch and applied locally on top of 4.17-rc7. Currently running it to test if it really mitigates the issue. Will report back
when I have results, and submit the patch if I'm satisfied.
Quoted 8 lines
Sorry, I've been dragged by something else and didn't update this promptly.
I just sent the v2 including the missingWMI_PEER_PHYMODE.Can you pick up the v2 and see if that help your issue?
Hi Ryan,
I already posted a patch v2 here:
http://lists.infradead.org/pipermail/ath10k/2018-May/011545.html
(Basically made the same changes as you.)
Yes, it fixes the problem.
Best regards,
Tim
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 6 lines · On 06/08/2018 12:27 PM, [email protected] wrote:
Hi Ryan,
I already posted a patch v2 here:
http://lists.infradead.org/pipermail/ath10k/2018-May/011545.html
(Basically made the same changes as you.)
Yes, it fixes the problem.
Cool, thanks a lot!
Signature · 7 lines
Ryan Hsu _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 9 lines · Ryan Hsu
Quoted 6 lines · On 06/08/2018 12:27 PM, [email protected] wrote:
Hi Ryan,
I already posted a patch v2 here:
http://lists.infradead.org/pipermail/ath10k/2018-May/011545.html
(Basically made the same changes as you.)
Yes, it fixes the problem.Cool, thanks a lot!
Sure thing. I'm happy to see this issue go away!
During my testing I just ran the system for some time while connected to an AP which I set into 20MHz/40MHz mode.
Without the patch: the firmware crashes, which after a while takes the whole system down with it.
With the patch: I observed several successful bandwidth changes in the logs, without any crash at all.
I believe the patch is ready to get merged. Doesn't matter if yours or mine, as they are basically the same.
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k