Firmware debug messages stopped sometime after 3.18
While trying to reproduce some bugs on ath10k in 3.19.0-rc2, I could not get any debuglog messages from the firmware. For instance, in stock kernel it should print out something like this:
wmi event debug mesg len 28
I got other wmi debug messages, so the debug level appeared correct.
The message shows up as expected in stock 3.18 kernel.
Was this done on purpose?
I've started a bisect...should track it down directly.
Thanks,
Ben
Signature · 8 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Re: Firmware debug messages stopped sometime after 3.18 (bisected)
Quoted 6 lines · Ben Greear
While trying to reproduce some bugs on ath10k in 3.19.0-rc2, I could not get any debuglog messages from the firmware. For instance, in stock kernel it should print out something like this:
wmi event debug mesg len 28
Quoted 7 lines
I got other wmi debug messages, so the debug level appeared correct.
The message shows up as expected in stock 3.18 kernel.
Was this done on purpose?
I've started a bisect...should track it down directly.
Took a while, but bisect points to this. If you are doing bisect yourself, or playing with changes near here, you must manually apply the patch below to allow scan to work. I'll try to figure out why the patch below disables firmware debuglog messages next. I assume that was not the intention?
90174455ae058a1dcf1f3d344a9752385803c1d8 is the first bad commit commit 90174455ae058a1dcf1f3d344a9752385803c1d8
Author: Rajkumar Manoharan <[email protected]> Date: Fri Oct 3 08:02:33 2014 +0300
ath10k: add support to configure pktlog filter
Add support to configure packet log filters (tx, rx, rate control) via debugfs. To disable htt pktlog events set the filters to 0.
ex:
To enable pktlog for all filters
echo 0x1f > /sys/kernel/debug/ieee80211/phy*/ath10k/pktlog_filter
To disable pktlog
echo 0 > /sys/kernel/debug/ieee80211/phy*/ath10k/pktlog_filter
:040000 040000 1ff3b609db06d718fb22a7d8444a70e38a60ebcd 6504c0828edb532f19826167eab9ba0634b3f044 M drivers
Patch to allow scan to work (this patch made it upstream a bit later):
[greearb@ben-dt2 linux-bisect]$ git diff
drivers/net/wireless/ath/ath10k/wmi.c +2 -2
@@ -3506,10 +3506,10 @@ int ath10k_wmi_start_scan(struct ath10k *ar,return ret;if (test_bit (ATH10K_FW_FEATURE_WMI_10X , ar->fw_features)) - len =sizeof (struct wmi_start_scan_cmd ) + + len =sizeof (struct wmi_10x_start_scan_cmd ) +ath10k_wmi_start_scan_tlvs_len (arg);else - len =sizeof (struct wmi_10x_start_scan_cmd ) + + len =sizeof (struct wmi_start_scan_cmd ) +ath10k_wmi_start_scan_tlvs_len (arg); skb =ath10k_wmi_alloc_skb (ar, len);
[greearb@ben-dt2 linux-bisect]$
Thanks,
Ben
Quoted 4 lines
Thanks,
Ben
Signature · 8 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Re: Firmware debug messages stopped sometime after 3.18 (bisected)
I think I found another problem with this patch.
When user enables ATH10K_PKTLOG_DBG_PRINT pktlog filter, then the firmware can send pkts to the CE1 pipe, which has only 512 byte message length. firmware debuglog messages can be larger than that, and so (10.1.467, at least) firmware will crash.
If anyone with QCA firmware source rights wants the nasty details, contact me offline.
I will try to hack around this in my firmware, but I think this patch needs some more thought and/or fixes.
Thanks,
Ben
Quoted 79 lines · Ben Greear
Quoted 6 lines · Ben Greear
While trying to reproduce some bugs on ath10k in 3.19.0-rc2, I could not get any debuglog messages from the firmware. For instance, in stock kernel it should print out something like this:
wmi event debug mesg len 28
Quoted 7 lines
I got other wmi debug messages, so the debug level appeared correct.
The message shows up as expected in stock 3.18 kernel.
Was this done on purpose?
I've started a bisect...should track it down directly.
Took a while, but bisect points to this. If you are doing bisect yourself, or playing with changes near here, you must manually apply the patch below to allow scan to work. I'll try to figure out why the patch below disables firmware debuglog messages next. I assume that was not the intention?
90174455ae058a1dcf1f3d344a9752385803c1d8is the first bad commit commit90174455ae058a1dcf1f3d344a9752385803c1d8Author: Rajkumar Manoharan <[email protected]> Date: Fri Oct 3 08:02:33 2014 +0300ath10k: add support to configure pktlog filter
Add support to configure packet log filters (tx, rx, rate control) via debugfs. To disable htt pktlog events set the filters to 0.
ex:
To enable pktlog for all filters
echo
0x1f>/sys/kernel/debug/ieee80211/phy*/ath10k/pktlog_filterTo disable pktlog
echo 0 >
/sys/kernel/debug/ieee80211/phy*/ath10k/pktlog_filterSigned-off-by Rajkumar Manoharan <[email protected]>Signed-off-by Kalle Valo <[email protected]>:040000 040000
1ff3b609db06d718fb22a7d8444a70e38a60ebcd6504c0828edb532f19826167eab9ba0634b3f044M driversPatch to allow scan to work (this patch made it upstream a bit later):
[greearb@ben-dt2 linux-bisect]$ git diff
drivers/net/wireless/ath/ath10k/wmi.c +2
-2@@ -3506,10 +3506,10 @@ int ath10k_wmi_start_scan(struct ath10k *ar,return ret;if (test_bit (ATH10K_FW_FEATURE_WMI_10X , ar->fw_features)) - len =sizeof (struct wmi_start_scan_cmd ) + + len =sizeof (struct wmi_10x_start_scan_cmd ) +ath10k_wmi_start_scan_tlvs_len (arg);else - len =sizeof (struct wmi_10x_start_scan_cmd ) + + len =sizeof (struct wmi_start_scan_cmd ) +ath10k_wmi_start_scan_tlvs_len (arg); skb =ath10k_wmi_alloc_skb (ar, len);[greearb@ben-dt2 linux-bisect]$
Thanks,
BenQuoted 4 lines
Thanks,
Ben
Signature · 8 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Re: Firmware debug messages stopped sometime after 3.18 (bisected)
Quoted 11 lines · Ben Greear
I think I found another problem with this patch.
When user enables
ATH10K_PKTLOG_DBG_PRINTpktlog filter, then the firmware can send pkts to the CE1 pipe, which has only 512 byte message length. firmware debuglog messages can be larger than that, and so (10.1.467, at least) firmware will crash.If anyone with QCA firmware source rights wants the nasty details, contact me offline.
I will try to hack around this in my firmware, but I think this patch needs some more thought and/or fixes.
So, root cause of not getting dbglog over wmi after this patch is bug in 10.1.467 (and maybe other) firmware in it's dbglog code.
I have fixed that in CT firmware, and disabled pktlog ability to send dbglog messages to work around that crash scenario, and now things are working better again.
Upstream firmware is still going to have issues with this patch, however.
Thanks,
Ben
Quoted 86 lines
Thanks,
BenQuoted 79 lines · Ben Greear
Quoted 6 lines · Ben Greear
While trying to reproduce some bugs on ath10k in 3.19.0-rc2, I could not get any debuglog messages from the firmware. For instance, in stock kernel it should print out something like this:
wmi event debug mesg len 28
Quoted 7 lines
I got other wmi debug messages, so the debug level appeared correct.
The message shows up as expected in stock 3.18 kernel.
Was this done on purpose?
I've started a bisect...should track it down directly.
Took a while, but bisect points to this. If you are doing bisect yourself, or playing with changes near here, you must manually apply the patch below to allow scan to work. I'll try to figure out why the patch below disables firmware debuglog messages next. I assume that was not the intention?
90174455ae058a1dcf1f3d344a9752385803c1d8is the first bad commit commit90174455ae058a1dcf1f3d344a9752385803c1d8Author: Rajkumar Manoharan <[email protected]> Date: Fri Oct 3 08:02:33 2014 +0300ath10k: add support to configure pktlog filter
Add support to configure packet log filters (tx, rx, rate control) via debugfs. To disable htt pktlog events set the filters to 0.
ex:
To enable pktlog for all filters
echo
0x1f>/sys/kernel/debug/ieee80211/phy*/ath10k/pktlog_filterTo disable pktlog
echo 0 >
/sys/kernel/debug/ieee80211/phy*/ath10k/pktlog_filterSigned-off-by Rajkumar Manoharan <[email protected]>Signed-off-by Kalle Valo <[email protected]>:040000 040000
1ff3b609db06d718fb22a7d8444a70e38a60ebcd6504c0828edb532f19826167eab9ba0634b3f044M driversPatch to allow scan to work (this patch made it upstream a bit later):
[greearb@ben-dt2 linux-bisect]$ git diff
drivers/net/wireless/ath/ath10k/wmi.c +2
-2@@ -3506,10 +3506,10 @@ int ath10k_wmi_start_scan(struct ath10k *ar,return ret;if (test_bit (ATH10K_FW_FEATURE_WMI_10X , ar->fw_features)) - len =sizeof (struct wmi_start_scan_cmd ) + + len =sizeof (struct wmi_10x_start_scan_cmd ) +ath10k_wmi_start_scan_tlvs_len (arg);else - len =sizeof (struct wmi_10x_start_scan_cmd ) + + len =sizeof (struct wmi_start_scan_cmd ) +ath10k_wmi_start_scan_tlvs_len (arg); skb =ath10k_wmi_alloc_skb (ar, len);[greearb@ben-dt2 linux-bisect]$
Thanks,
BenQuoted 4 lines
Thanks,
Ben
Signature · 8 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k