Firmware crash
Hi List,
i've seen a strange firmware crash:
Athos/Xt BSP version Oct 19 2012-16:00:59
Baud rate host set is 115200.
Mac address is : 00:03:07:12:34:56
alloc rem: 197524
**WMI Service Ready **
NUM_DEV=16TGT_VER: 4100016C
RegDomain: 0, CfgCtl: 224 - 224ar_wal_peer_attach: num_peer_entries 145, num_tid_entries 443 allocated 64 tids at 0x439a08 (each 96 bytes) allocated 379 tids in IRAM at 0x9baeb4 (each 96 bytes) allocated 64 stateless tids at 0x43b230 (each 40 bytes) ### turn on Tx completion index tracking ###
**WMI Ready **TGT_VER: 4100016C
RegDomain: 3A, CfgCtl: 16 - 16
peer create command for ****0:3:7:12:34:56**** _wlan_vdev_set_param: unimplemented command 0x2 _wlan_vdev_set_param: unimplemented command 0x7 _wlan_vdev_set_param: unimplemented command 0x8
TODO: Add DBGLOG for WMM Params
TODO: Add DBGLOG for WMM Params
TODO: Add DBGLOG for WMM Params
TODO: Add DBGLOG for WMM Params
TSF id: 1, free_tsf_id_map: 1
assertion failed? pc=0x9a0e15, line=0, dump area=0x401930 Target ID: 0x4100016c (1090519404)
Debug Info:0x4100016c 0x00000000 0x009a0e15 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x009a0e15 0x00000000 0x00401930 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
This happened on two APs at the same time, which were running before without problems. As both APs are in short distance (about 10m) i guess it was caused by some packet on the Air.
I see a 'assertion failed' above, can anyone tell what that assertion is?
Firmware Version is:
[ 27.120000] ath10k: hardware name qca988x hw2.0 version 0x4100016c [ 27.130000] ath10k: firmware version: 10.1.467.2-1
Thanks
Sven
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 39 lines · Sven Schnelle
Hi List,
i've seen a strange firmware crash:
Athos/Xt BSP version Oct 19 2012-16:00:59
Baud rate host set is 115200.
Mac address is : 00:03:07:12:34:56
alloc rem: 197524
**WMI Service Ready **
NUM_DEV=16TGT_VER: 4100016C
RegDomain: 0, CfgCtl: 224 - 224ar_wal_peer_attach:num_peer_entries145,num_tid_entries443 allocated 64 tids at0x439a08(each 96 bytes) allocated 379 tids in IRAM at0x9baeb4(each 96 bytes) allocated 64 stateless tids at0x43b230(each 40 bytes) ### turn on Tx completion index tracking ###
**WMI Ready **TGT_VER: 4100016C
RegDomain: 3A, CfgCtl: 16 - 16
peer create command for ****0:3:7:12:34:56****_wlan_vdev_set_param: unimplemented command0x2_wlan_vdev_set_param: unimplemented command0x7_wlan_vdev_set_param: unimplemented command0x8
TODO: Add DBGLOG for WMM Params
TODO: Add DBGLOG for WMM Params
TODO: Add DBGLOG for WMM Params
TODO: Add DBGLOG for WMM Params
TSF id: 1,free_tsf_id_map: 1
assertion failed? pc=0x9a0e15, line=0, dump area=0x401930 Target ID:0x4100016c(1090519404)
Debug Info:0x4100016c0x000000000x009a0e150x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x009a0e150x000000000x004019300x000000000x000000000x000000000x000000000x000000000x00000000
Quoted 8 lines
0x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x00000000
Apparently firmware crashes while processing a beacon frame with a NULL dereference.
Quoted 11 lines
This happened on two APs at the same time, which were running before without problems. As both APs are in short distance (about 10m) i guess it was caused
by some packet on the Air.I see a 'assertion failed' above, can anyone tell what that assertion is?
Firmware Version is:
[ 27.120000] ath10k: hardware name qca988x hw2.0 version 0x4100016c [ 27.130000] ath10k: firmware version: 10.1.467.2-1
Did you happen to have the ath10k interface in a bridge? If so monitor vdev is most likely the culprit here. A very similar issue was reported for 4addr station bridging and also involves monitor vdev.
Michał
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 20 lines · Michal Kazior
Quoted 11 lines
TSF id: 1,
free_tsf_id_map: 1
assertion failed? pc=0x9a0e15, line=0, dump area=0x401930 Target ID:0x4100016c(1090519404)
Debug Info:0x4100016c0x000000000x009a0e150x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x009a0e150x000000000x004019300x000000000x000000000x000000000x000000000x000000000x00000000Quoted 8 lines
0x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x00000000
Quoted 2 lines
Apparently firmware crashes while processing a beacon frame with a NULL dereference.
I fixed some issues in this area in my CT firmware, so you could try it if you want. If you can crash my firmware, please send me the similar crash dump and I can probably fix it.
Note this is not official QCA firmware, so any problems specific to this firmware should be directed at me, and not QCA, and when reporting general bugs (say, with driver), if you are using CT firmware be sure to note that.
http://www.candelatech.com/ath10k.php
Thanks,
Ben
Signature · 7 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 4 lines · Emanuel Taube
Hi Ben,
I am able to reproduce it with your firmware-2-ct-no-commercial-7.bin here is the crash dump:
Thanks. I am going to have to update my decode program to parse this...it doesn't look like 'normal' dmesg output on my systems. Should have it done shortly...
Ben
Quoted 84 lines
assertion failed? pc=0x9a1a49, line=0, dump area=0x401930 Target ID:
0x4100016c(1090519404)
Debug Info:0x4100016c0x000000000x009a1a490x000000000x000000000x000603240x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x009b58cb0x0094085d0x000000000x009a1a490x809430b80x00401a400x000000010x000000020x809409750x00401a600x0000001f0x00403bec0x409406b90x00401a800x0000001f0x004191540x000000000x00401aa00x000500240x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x00000000to reproduce it I run 2 ath10k APs, one with the
WIPHY_FLAG_IBSS_RSNenabled and one without. Whichever AP starts first keeps running while the other one crashes when trying to start hostapd.Thanks,
Emanuel2014-06-26 15:06 GMT+02:00 Ben Greear <[email protected] <mailto:[email protected]>>:
On 06/26/2014 04:47 AM, Michal Kazior wrote: TSF id: 1, free_tsf_id_map: 1 assertion failed? pc=0x9a0e15, line=0, dump area=0x401930 Target ID: 0x4100016c (1090519404) Debug Info: 0x4100016c 0x00000000 0x009a0e15 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x009a0e15 0x00000000 0x00401930 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 Apparently firmware crashes while processing a beacon frame with a NULL dereference. I fixed some issues in this area in my CT firmware, so you could try it if you want. If you can crash my firmware, please send me the similar crash dump and I can probably fix it. Note this is not official QCA firmware, so any problems specific to this firmware should be directed at me, and not QCA, and when reporting general bugs (say, with driver), if you are using CT firmware be sure to note that. http://www.candelatech.com/__ath10k.php <http://www.candelatech.com/ath10k.php> Thanks, Ben -- Ben Greear <[email protected] <mailto:[email protected]>> Candela Technologies Inc http://www.candelatech.com _________________________________________________ ath10k mailing list [email protected] <mailto:[email protected]> http://lists.infradead.org/__mailman/listinfo/ath10k <http://lists.infradead.org/mailman/listinfo/ath10k>
Signature · 8 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 4 lines · Emanuel Taube
Hi Ben,
I am able to reproduce it with your firmware-2-ct-no-commercial-7.bin here is the crash dump:
Can you reproduce with this firmware? I have been making local changes, and I will get a better decode easier if you can re-run the test with this binary...
http://www.candelatech.com/downloads/firmware-2-community.bin
I fixed my tool to decode the bare dump, but dmesg output will probably be more convenient for the future (looks like you missed a line of the dump in the first email??)
Thanks,
Ben
Quoted 84 lines
assertion failed? pc=0x9a1a49, line=0, dump area=0x401930 Target ID:
0x4100016c(1090519404)
Debug Info:0x4100016c0x000000000x009a1a490x000000000x000000000x000603240x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x009b58cb0x0094085d0x000000000x009a1a490x809430b80x00401a400x000000010x000000020x809409750x00401a600x0000001f0x00403bec0x409406b90x00401a800x0000001f0x004191540x000000000x00401aa00x000500240x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x000000000x00000000to reproduce it I run 2 ath10k APs, one with the
WIPHY_FLAG_IBSS_RSNenabled and one without. Whichever AP starts first keeps running while the other one crashes when trying to start hostapd.Thanks,
Emanuel2014-06-26 15:06 GMT+02:00 Ben Greear <[email protected] <mailto:[email protected]>>:
On 06/26/2014 04:47 AM, Michal Kazior wrote: TSF id: 1, free_tsf_id_map: 1 assertion failed? pc=0x9a0e15, line=0, dump area=0x401930 Target ID: 0x4100016c (1090519404) Debug Info: 0x4100016c 0x00000000 0x009a0e15 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x009a0e15 0x00000000 0x00401930 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 Apparently firmware crashes while processing a beacon frame with a NULL dereference. I fixed some issues in this area in my CT firmware, so you could try it if you want. If you can crash my firmware, please send me the similar crash dump and I can probably fix it. Note this is not official QCA firmware, so any problems specific to this firmware should be directed at me, and not QCA, and when reporting general bugs (say, with driver), if you are using CT firmware be sure to note that. http://www.candelatech.com/__ath10k.php <http://www.candelatech.com/ath10k.php> Thanks, Ben -- Ben Greear <[email protected] <mailto:[email protected]>> Candela Technologies Inc http://www.candelatech.com _________________________________________________ ath10k mailing list [email protected] <mailto:[email protected]> http://lists.infradead.org/__mailman/listinfo/ath10k <http://lists.infradead.org/mailman/listinfo/ath10k>
Signature · 8 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 3 lines · Emanuel Taube
Hi Ben,
here is the dmesg output with your new community firmware:
Looks like null-dereference (or possibly some other bad memory dereference), not actually an assert (at least in my code).
Can you re-download the firmware (same location) and see if that fixes the problem?
Thanks,
Ben
Signature · 8 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 3 lines · Emanuel Taube
Hi Ben,
your latest firmware fixes the crashes I was able to reproduce before.
Excellent, thanks for confirming. Looks like it was simple null-pointer deref. I'll roll this into my firmware, so next 'official' release will have the fix as well.
I'll send QCA a note off-list on what fixes it in case they want to put the fix upstream some day.
Thanks,
Ben
Quoted 25 lines
Thanks and regards,
Emanuel2014-06-26 17:44 GMT+02:00 Ben Greear <[email protected] <mailto:[email protected]>>:
On 06/26/2014 08:39 AM, Emanuel Taube wrote:
> Hi Ben,
>
> here is the dmesg output with your new community firmware:Looks like null-dereference (or possibly some other bad memory dereference), not actually an assert (at least in my code).
Can you re-download the firmware (same location) and see if that fixes the problem?
Thanks,
Ben--
Ben Greear <[email protected] <mailto:[email protected]>> Candela Technologies Inc http://www.candelatech.com
Signature · 8 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k