ath10k: add fw coredump for sdio when firmware assert
PATCH v2 1/1 ath10k: add fw coredump for sdio when firmware assert
When firmware assert, it need coredump to analyze, this patch will collect the register and memory info for sdio chip.
The coredump configuration is different between PCIE and SDIO for the same reversion, so this patch add bus type to distinguish PCIE and SDIO chip for coredump.
Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.
---
v2: change code style
drivers/net/wireless/ath/ath10k/bmi.c | 1 + drivers/net/wireless/ath/ath10k/core.c | 6 + drivers/net/wireless/ath/ath10k/core.h | 2 + drivers/net/wireless/ath/ath10k/coredump.c | 338 +++++++++++++++++++++++++++- drivers/net/wireless/ath/ath10k/coredump.h | 1 + drivers/net/wireless/ath/ath10k/hw.h | 1 + drivers/net/wireless/ath/ath10k/sdio.c | 335 ++++++++++++++++++++++++++- drivers/net/wireless/ath/ath10k/targaddrs.h | 10 + 8 files changed, 691 insertions(+), 3 deletions(-)
drivers/net/wireless/ath/ath10k/bmi.c +1 -0
@@ -197,6 +197,7 @@ int ath10k_bmi_read_memory(struct ath10k *ar,return 0 ; } +EXPORT_SYMBOL (ath10k_bmi_read_memory);int ath10k_bmi_write_soc_reg (struct ath10k *ar,u32 address,u32 reg_val) {
drivers/net/wireless/ath/ath10k/core.c +6 -0
@@ -708,6 +708,10 @@ static void ath10k_init_sdio(struct ath10k *ar, enum ath10k_firmware_mode mode)ath10k_bmi_read32 (ar, hi_option_flag, ¶m); param |=HI_OPTION_DISABLE_DBGLOG ;ath10k_bmi_write32 (ar, hi_option_flag, param); + +ath10k_bmi_read32 (ar, hi_option_flag2, ¶m); + param |=HI_OPTION_SDIO_CRASH_DUMP_ENHANCEMENT_HOST ; +ath10k_bmi_write32 (ar, hi_option_flag2, param); }static int ath10k_init_configure_target (struct ath10k *ar)
@@ -1953,6 +1957,8 @@ static void ath10k_core_get_fw_name(struct ath10k *ar, char *fw_name,scnprintf (fw_name, fw_name_len,"%s-%d.bin" ,ATH10K_FW_FILE_BASE , fw_api);break ; +default : +break ; } }
drivers/net/wireless/ath/ath10k/core.h +2 -0
@@ -97,6 +97,8 @@ static inline const char *ath10k_bus_str(enum ath10k_bus bus)return "usb" ;case ATH10K_BUS_SNOC :return "snoc" ; +default : +return "unknown" ; }return "unknown" ;
drivers/net/wireless/ath/ath10k/coredump.c +337 -1
368 lines not on this page · raw message
drivers/net/wireless/ath/ath10k/coredump.h +1 -0
@@ -155,6 +155,7 @@ struct ath10k_mem_region {struct ath10k_hw_mem_layout {u32 hw_id;u32 hw_rev; +enum ath10k_bus bus;struct {const struct ath10k_mem_region *regions;
drivers/net/wireless/ath/ath10k/hw.h +1 -0
@@ -11,6 +11,7 @@#include "targaddrs.h" enum ath10k_bus { +ATH10K_BUS_ANY ,ATH10K_BUS_PCI ,ATH10K_BUS_AHB ,ATH10K_BUS_SDIO ,
drivers/net/wireless/ath/ath10k/sdio.c +333 -2
353 lines not on this page · raw message
drivers/net/wireless/ath/ath10k/targaddrs.h +10 -0
@@ -334,6 +334,16 @@ struct host_interest {#define HI_ACS_FLAGS_SDIO_REDUCE_TX_COMPL_FW_ACK (1 << 17)/* +* If both SDIO_CRASH_DUMP_ENHANCEMENT_HOST and SDIO_CRASH_DUMP_ENHANCEMENT_FW +* flags are set, then crashdump upload will be done using the BMI host/target +* communication channel. +*/ +/* HOST to support using BMI dump FW memory when hit assert */ +#define HI_OPTION_SDIO_CRASH_DUMP_ENHANCEMENT_HOST 0x400 +/* FW to support using BMI dump FW memory when hit assert */ +#define HI_OPTION_SDIO_CRASH_DUMP_ENHANCEMENT_FW 0x800 + +/* * CONSOLE FLAGS * * Bit Range Meaning
Signature · 1 line
1.9.1
Quoted 11 lines · Wen Gong
When firmware assert, it need coredump to analyze, this patch will collect the register and memory info for sdio chip.
The coredump configuration is different between PCIE and SDIO for the same reversion, so this patch add bus type to distinguish PCIE and SDIO chip for coredump.
Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.Signed-off-by Wen Gong <[email protected]>
I think this patch should be split into two, the first patch preparing coredump layouts to support different bus types and the second patch adding coredump support for SDIO.
drivers/net/wireless/ath/ath10k/hw.h line 17
Quoted 10 lines
drivers/net/wireless/ath/ath10k/hw.h +1
-0@@ -11,6 +11,7 @@#include "targaddrs.h" enum ath10k_bus { +ATH10K_BUS_ANY ,ATH10K_BUS_PCI ,ATH10K_BUS_AHB ,ATH10K_BUS_SDIO ,
I don't think we need add this type ANY. From a quick look QCA4019 is AHB and all the rest are PCI, right? So in the first patch you only need to add correct bus type for each entry and everything should work as before.
Signature · 6 lines
Kalle Valo _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 11 lines · Wen Gong
When firmware assert, it need coredump to analyze, this patch will collect the register and memory info for sdio chip.
The coredump configuration is different between PCIE and SDIO for the same reversion, so this patch add bus type to distinguish PCIE and SDIO chip for coredump.
Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.Signed-off-by Wen Gong <[email protected]>
[...]
drivers/net/wireless/ath/ath10k/sdio.c static SIMPLE_DEV_PM_OPS(ath10k_sdio_pm_ops, ath10k_sdio_pm_suspend, · line 2016
Quoted 9 lines
drivers/net/wireless/ath/ath10k/sdio.c +9
-0+void ath10k_sdio_check_fw_reg (struct ath10k *ar,u32 *fast_dump) +{ +int ret =0 ; +u32 param; + + ret =ath10k_sdio_read_host_interest_value (ar,HI_ITEM (hi_option_flag2), ¶m); + + *fast_dump = ((param &HI_OPTION_SDIO_CRASH_DUMP_ENHANCEMENT_FW ) + ==HI_OPTION_SDIO_CRASH_DUMP_ENHANCEMENT_FW );
The commit log tells nothing about fast, it should always document the design decisions. Why this fast dump, what's the benefit? Why not always use the fast dump and forget the slow dump (or vice versa)? There needs to be really good reasons to have all this complexity to support both slow and fast dumps.
Review
Quoted 2 lines
(quoted diff) +2
-0+ ath10k_err(ar, "check fw reg : %x\n", param); +}
This is a debug message, not an error. And debug messages should use format "sdio hi_option_flag2 %x\n".
Signature · 6 lines
Kalle Valo _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 6 lines · Kalle Valo
The commit log tells nothing about fast, it should always document the design decisions. Why this fast dump, what's the benefit? Why not always use the fast dump and forget the slow dump (or vice versa)? There needs to be really good reasons to have all this complexity to support both slow and fast dumps.
fastdump is new added in fimrware of sdio, so if it is use old firmeare, then it will not support fastdump, then it need to use slow dump type.
it is to compatibility for old/new version firmware.
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ ath10k_err(ar, "check fw reg : %x\n", param); +}This is a debug message, not an error. And debug messages should use format "sdio
hi_option_flag2%x\n".
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
patch v3 sent.
https://patchwork.kernel.org/patch/11156663/
https://patchwork.kernel.org/patch/11156659/
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 11 lines · Wen Gong
When firmware assert, it need coredump to analyze, this patch will collect the register and memory info for sdio chip.
The coredump configuration is different between PCIE and SDIO for the same reversion, so this patch add bus type to distinguish PCIE and SDIO chip for coredump.
Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.Signed-off-by Wen Gong <[email protected]>
[...]
Quoted 7 lines
drivers/net/wireless/ath/ath10k/sdio.c +7
-0+static int ath10k_sdio_read_mem (struct ath10k *ar,u32 address,void *buf, +u32 buf_len) +{ +u32 val; +int i, ret; + +for (i =0 ; i < buf_len; i +=4 ) {
drivers/net/wireless/ath/ath10k/sdio.c static SIMPLE_DEV_PM_OPS(ath10k_sdio_pm_ops, ath10k_sdio_pm_suspend, · line 2006
Quoted 10 lines
drivers/net/wireless/ath/ath10k/sdio.c +10
-0+ ret =ath10k_sdio_hif_diag_read32 (ar, address + i, &val); +if (ret) { +ath10k_warn (ar,"unable to read mem %d value\n" , address + i); +break ; + } +memcpy (buf + i, &val,4 ); + } + +return ret; +}
What's wrong with ath10k_sdio_hif_diag_read()? AFAICS this whole function duplicates just what it does.
Signature · 6 lines
Kalle Valo _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 3 lines · Kalle Valo
What's wrong with
ath10k_sdio_hif_diag_read()? AFAICS this whole function duplicates just what it does.
ath10k_sdio_hif_diag_read's buffer size is limit,
and the dump memory/register's buffer size is larger than the diag window's limit, if use it directly will trigger crash like this for every time.
[ 149.947624] ath10k_sdio mmc1:0001:1: ath10k_sdio_hif_diag_read buf_len :4 [ 149.954741] ath10k_sdio mmc1:0001:1: ath10k_sdio_hif_diag_read buf_len :240 [ 151.005143] Unable to handle kernel paging request at virtual address ffffffc0080ab980 [ 151.013077] Mem abort info: [ 151.015866] ESR = 0x96000045 [ 151.018918] Exception class = DABT (current EL), IL = 32 bits [ 151.024830] SET = 0, FnV = 0 [ 151.027880] EA = 0, S1PTW = 0 [ 151.031016] Data abort info: [ 151.033892] ISV = 0, ISS = 0x00000045 [ 151.037723] CM = 0, WnR = 1 [ 151.040691] swapper pgtable: 4k pages, 39-bit VAs, pgdp = 0000000073b23692 [ 151.047560] [ffffffc0080ab980] pgd=0000000000000000, pud=0000000000000000 [ 151.054354] Internal error: Oops: 96000045 [#1] PREEMPT SMP [ 151.059925] Modules linked in: ath10k_sdio ath10k_core rfcomm uinput cros_ec_rpmsg mtk_cam_isp ath mac80211 mtk_fd hci_uart btqca bluetooth mtk_scp mtk_rpmsg rpmsg_core ecdh_generic mtk_scp_ipi bridge stp llc nf_nat_tftp nf_conntrack_tftp nf_nat_ftp nf_conntrack_ftp esp6 ah6 xfrm6_mode_tunnel xfrm6_mode_transport xfrm4_mode_tunnel xfrm4_mode_transport ip6t_REJECT ip6t_ipv6header ipt_MASQUERADE fuse cfg80211 iio_trig_sysfs cros_ec_sensors_sync cros_ec_sensors cros_ec_sensors_ring industrialio_triggered_buffer kfifo_buf cros_ec_sensors_core lzo_rle lzo_compress zram asix usbnet mii joydev [last unloaded: ath10k_core] [ 151.114537] Process swapper/0 (pid: 0, stack limit = 0x00000000e30dc665) [ 151.121238] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.19.72 #11 [ 151.127327] Hardware name: MediaTek kukui rev1 board (DT) [ 151.132724] pstate: 20000085 (nzCv daIf -PAN -UAO) [ 151.137525] pc : __memcpy+0x110/0x180 [ 151.141193] lr : swiotlb_tbl_unmap_single+0x84/0x150 [ 151.146151] sp : ffffff8008003c60 [ 151.149462] x29: ffffff8008003c90 x28: ffffffa2a9611f38 [ 151.154774] x27: ffffffa2a92cc018 x26: 0000000000000000 [ 151.160087] x25: ffffffa2a90b8000 x24: 0000000000000001 [ 151.165399] x23: ffffffa2a975e000 x22: 0000000000001400 [ 151.170710] x21: 0000000000000000 x20: 00000000fc7ff000 [ 151.176021] x19: 00000000000000f0 x18: 0000000000000020 [ 151.181332] x17: 0000000000000000 x16: 0000000000000000 [ 151.186643] x15: 00000000ffffffff x14: 0000000000000000 [ 151.191955] x13: 0000000000000000 x12: 0000000000000000 [ 151.197266] x11: 0000000000000000 x10: 0000000000000000 [ 151.202578] x9 : 0000000000000000 x8 : 0000000000000000 [ 151.207890] x7 : 0000000000000000 x6 : ffffffc0080ab980 [ 151.213202] x5 : ffffff8016ffbdc8 x4 : 0000000000000000 [ 151.218514] x3 : 0000000000000002 x2 : 0000000000000070 [ 151.223825] x1 : fffffff37c7ff040 x0 : ffffffc0080ab980 [ 151.229138] Call trace: [ 151.231585] __memcpy+0x110/0x180 [ 151.234899] unmap_single+0x6c/0x84 [ 151.238386] swiotlb_unmap_sg_attrs+0x54/0x80 [ 151.242744] __swiotlb_unmap_sg_attrs+0xa0/0xb8 [ 151.247277] msdc_unprepare_data+0x6c/0x84 [ 151.251372] msdc_request_done+0x58/0x84 [ 151.255292] msdc_data_xfer_done+0x1a0/0x1c8 [ 151.259559] msdc_irq+0x12c/0x17c [ 151.262876] __handle_irq_event_percpu+0xe4/0x250 [ 151.267578] handle_irq_event_percpu+0x28/0x68 [ 151.272018] handle_irq_event+0x48/0x78 [ 151.275853] handle_fasteoi_irq+0xd0/0x1a0 [ 151.279948] __handle_domain_irq+0x84/0xc4 [ 151.284042] gic_handle_irq+0x124/0x1a4 [ 151.287875] el1_irq+0xb0/0x128 [ 151.291016] cpuidle_enter_state+0x298/0x328 [ 151.295282] cpuidle_enter+0x30/0x40 [ 151.298856] do_idle+0x190/0x270 [ 151.302081] cpu_startup_entry+0x24/0x28 [ 151.306005] rest_init+0xd4/0xe0 [ 151.309233] start_kernel+0x30c/0x38c [ 151.312896] Code: a8c12027 a8c12829 a8c1302b a8c1382d (a88120c7) [ 151.318987] ---[ end trace 4d779fa7753769c5 ]--- [ 151.342768] Kernel panic - not syncing: Fatal exception in interrupt [ 151.349134] SMP: stopping secondary CPUs [ 151.353074] Kernel Offset: 0x22a0400000 from 0xffffff8008000000 [ 151.358991] CPU features: 0x0,2188200c [ 151.362735] Memory Limit: none
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 9 lines · Wen Gong
Quoted 3 lines · Kalle Valo
What's wrong with
ath10k_sdio_hif_diag_read()? AFAICS this whole function duplicates just what it does.ath10k_sdio_hif_diag_read's buffer size is limit,
and the dump memory/register's buffer size is larger than the diag window's limit,
if use it directly will trigger crash like this for every time.
You shouldn't blindly add extra code to ath10k workaround issues. And if you really need to use a workaround, then it needs to properly explained in the commit and as well as commented in the code. But before that the issue needs to be thoroughly investigated and understood where the problem is coming from. Because it might be even completely unrelated to ath10k.
Quoted 53 lines
[ 149.947624] ath10k_sdio mmc1:0001:1: ath10k_sdio_hif_diag_read buf_len :4 [ 149.954741] ath10k_sdio mmc1:0001:1: ath10k_sdio_hif_diag_read buf_len :240 [ 151.005143] Unable to handle kernel paging request at virtual address ffffffc0080ab980 [ 151.013077] Mem abort info: [ 151.015866] ESR = 0x96000045 [ 151.018918] Exception class = DABT (current EL), IL = 32 bits [ 151.024830] SET = 0, FnV = 0 [ 151.027880] EA = 0, S1PTW = 0 [ 151.031016] Data abort info: [ 151.033892] ISV = 0, ISS = 0x00000045 [ 151.037723] CM = 0, WnR = 1 [ 151.040691] swapper pgtable: 4k pages, 39-bit VAs, pgdp = 0000000073b23692 [ 151.047560] [ffffffc0080ab980] pgd=0000000000000000, pud=0000000000000000 [ 151.054354] Internal error: Oops: 96000045 [#1] PREEMPT SMP [ 151.059925] Modules linked in: ath10k_sdio ath10k_core rfcommuinput
cros_ec_rpmsgmtk_cam_ispath mac80211mtk_fdhci_uartbtqca bluetoothmtk_scpmtk_rpmsgrpmsg_coreecdh_genericmtk_scp_ipibridge stp llcnf_nat_tftpnf_conntrack_tftpnf_nat_ftpnf_conntrack_ftpesp6 ah6xfrm6_mode_tunnelxfrm6_mode_transportxfrm4_mode_tunnelxfrm4_mode_transportip6t_REJECTip6t_ipv6headeript_MASQUERADEfuse cfg80211iio_trig_sysfscros_ec_sensors_synccros_ec_sensorscros_ec_sensors_ringindustrialio_triggered_bufferkfifo_bufcros_ec_sensors_corelzo_rlelzo_compresszram asix usbnet mii joydev [last unloaded:ath10k_core][ 151.114537] Process swapper/0 (pid: 0, stack limit = 0x00000000e30dc665) [ 151.121238] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.19.72 #11 [ 151.127327] Hardware name: MediaTek kukui rev1 board (DT) [ 151.132724] pstate: 20000085 (nzCv daIf -PAN -UAO) [ 151.137525] pc : __memcpy+0x110/0x180 [ 151.141193] lr : swiotlb_tbl_unmap_single+0x84/0x150 [ 151.146151] sp : ffffff8008003c60 [ 151.149462] x29: ffffff8008003c90 x28: ffffffa2a9611f38 [ 151.154774] x27: ffffffa2a92cc018 x26: 0000000000000000 [ 151.160087] x25: ffffffa2a90b8000 x24: 0000000000000001 [ 151.165399] x23: ffffffa2a975e000 x22: 0000000000001400 [ 151.170710] x21: 0000000000000000 x20: 00000000fc7ff000 [ 151.176021] x19: 00000000000000f0 x18: 0000000000000020 [ 151.181332] x17: 0000000000000000 x16: 0000000000000000 [ 151.186643] x15: 00000000ffffffff x14: 0000000000000000 [ 151.191955] x13: 0000000000000000 x12: 0000000000000000 [ 151.197266] x11: 0000000000000000 x10: 0000000000000000 [ 151.202578] x9 : 0000000000000000 x8 : 0000000000000000 [ 151.207890] x7 : 0000000000000000 x6 : ffffffc0080ab980 [ 151.213202] x5 : ffffff8016ffbdc8 x4 : 0000000000000000 [ 151.218514] x3 : 0000000000000002 x2 : 0000000000000070 [ 151.223825] x1 : fffffff37c7ff040 x0 : ffffffc0080ab980 [ 151.229138] Call trace:
Quoted 8 lines
[ 151.231585] __memcpy+0x110/0x180 [ 151.234899] unmap_single+0x6c/0x84 [ 151.238386] swiotlb_unmap_sg_attrs+0x54/0x80 [ 151.242744] __swiotlb_unmap_sg_attrs+0xa0/0xb8 [ 151.247277] msdc_unprepare_data+0x6c/0x84 [ 151.251372] msdc_request_done+0x58/0x84 [ 151.255292] msdc_data_xfer_done+0x1a0/0x1c8 [ 151.259559] msdc_irq+0x12c/0x17c
Did you investigate this? Is the buffer you reading to DMA accessible? What about the alignment? Is there a certain length which is the limit for crashes? And so on...
Signature · 6 lines
Kalle Valo _______________________________________________ ath10k mailing list [email protected] http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 12 lines · Kalle Valo
Quoted 9 lines · Wen Gong
Quoted 3 lines · Kalle Valo
What's wrong with
ath10k_sdio_hif_diag_read()? AFAICS this whole function duplicates just what it does.ath10k_sdio_hif_diag_read's buffer size is limit,
and the dump memory/register's buffer size is larger than the diag window's limit,
if use it directly will trigger crash like this for every time.
Quoted 16 lines
You shouldn't blindly add extra code to ath10k workaround issues. And if you really need to use a workaround, then it needs to properly explained in the commit and as well as commented in the code. But before that the issue needs to be thoroughly investigated and understood where the problem is coming from. Because it might be even completely unrelated to ath10k.
Quoted 8 lines
[ 149.947624] ath10k_sdio mmc1:0001:1: ath10k_sdio_hif_diag_read buf_len :4 [ 149.954741] ath10k_sdio mmc1:0001:1: ath10k_sdio_hif_diag_read buf_len :240 [ 151.005143] Unable to handle kernel paging request at virtual[ 151.114537] Process swapper/0 (pid: 0, stack limit =
0x00000000e30dc665)
、
Quoted 4 lines
Did you investigate this? Is the buffer you reading to DMA accessible? What about the alignment? Is there a certain length which is the limit for crashes? And so on...
the limit is 4 byte.
for 5 bytes, it will crash.
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k
Quoted 11 lines · Wen Gong
When firmware assert, it need coredump to analyze, this patch will collect the register and memory info for sdio chip.
The coredump configuration is different between PCIE and SDIO for the same reversion, so this patch add bus type to distinguish PCIE and SDIO chip for coredump.
Tested with QCA6174 SDIO with firmware
WLAN.RMH.4.4.1-00007-QCARMSWP-1.Signed-off-by Wen Gong <[email protected]>
[...]
drivers/net/wireless/ath/ath10k/sdio.c static SIMPLE_DEV_PM_OPS(ath10k_sdio_pm_ops, ath10k_sdio_pm_suspend, · line 2278
Quoted 12 lines
drivers/net/wireless/ath/ath10k/sdio.c +12
-0+void ath10k_sdio_fw_crashed_dump (struct ath10k *ar) +{ +struct ath10k_fw_crash_data *crash_data; +char guid[UUID_STRING_LEN +1 ]; +u32 fast_dump =0 ; + +ath10k_err (ar,"begin fw dump\n" , guid); + +ath10k_sdio_check_fw_reg (ar, &fast_dump); + +if (fast_dump) + ar->bmi.done_sent = false;
After looking more closely, the ar->bmi.done_set checks in bmi.c does not look necessary to me, I have never seen that warning. I would remove those and the done_sent field altogether from ath10k to make the code cleaner and I avoid hacks like above. This should be done in a separate patch, of course.
Signature · 1 line
Kalle Valo
Quoted 5 lines · Kalle Valo
After looking more closely, the
ar->bmi.done_setchecks inbmi.cdoes not look necessary to me, I have never seen that warning. I would remove those and thedone_sentfield altogether from ath10k to make the code cleaner and I avoid hacks like above. This should be done in a separate patch, of course.
so I will use it this patch :)
_______________________________________________
ath10k mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/ath10k