Oops while booting 5.15.2 on Nokia N900
Subsystems (2)
Hi,
Moving from Linux 5.1 to 5.15 (not very helpful, I know), I'm now occasionally seeing this problem [1] during boot. Whether or not it happens seems to mostly be random.
The kernel is patched with the experimental and out of tree SGX patches [2], but I am pretty sure they are not relevant here, since the modules aren't even loaded at this point.
Any ideas what could cause this? Please let me know if I should provide additional debug info.
Regards,
Merlijn
[1]
Quoted 55 lines
[ 10.256835] Internal error: Oops - undefined instruction: 0 [#1] PREEMPT SMP ARM [ 10.264343] Modules linked in: [ 10.267456] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.2-218694-g19b9514dd867-dirty #1 [ 10.275787] Hardware name: Nokia RX-51 board [ 10.280090] PC is at omap3_l3_app_irq+0x100/0x120 [ 10.284851] LR is at __handle_irq_event_percpu+0x48/0x250 [ 10.290313] pc : [<c05b0ac8>] lr : [<c019c2a4>] psr: 20030193 [ 10.296600] sp : c0f01e10 ip : 3ccf0000 fp : c1834200 [ 10.301879] r10: c0ff8760 r9 : c0ff8780 r8 : 0000001a [ 10.307128] r7 : 00000000 r6 : 00080000 r5 : 00000000 r4 : f8000000 [ 10.313720] r3 : 00080000 r2 : a3b464b6 r1 : 00000000 r0 : 00080000 [ 10.320281] Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none [ 10.327575] Control: 10c5387d Table: 83398019 DAC: 00000051 [ 10.333343] Register r0 information: non-paged memory [ 10.338439] Register r1 information: NULL pointer [ 10.343200] Register r2 information: non-paged memory [ 10.348297] Register r3 information: non-paged memory [ 10.353393] Register r4 information: 0-page vmalloc region starting at 0xf8000000 allocated at iotable_init+0x0/0xf4 [ 10.364013] Register r5 information: NULL pointer [ 10.368774] Register r6 information: non-paged memory [ 10.373870] Register r7 information: NULL pointer [ 10.378631] Register r8 information: non-paged memory [ 10.383728] Register r9 information: non-slab/vmalloc memory [ 10.389434] Register r10 information: non-slab/vmalloc memory [ 10.395233] Register r11 information: slab kmalloc-256 start c1834200 pointer offset 0 size 256 [ 10.404022] Register r12 information: non-paged memory [ 10.409210] Process swapper/0 (pid: 0, stack limit = 0x(ptrval)) [ 10.415283] Stack: (0xc0f01e10 to 0xc0f02000) [ 10.419677] 1e00: c0627a10 c1fc5d80 c0f04dc8 c1a47500 [ 10.427917] 1e20: c0f04dc8 00000000 c0f01e68 0000001a c0ff8780 c019c2a4 00000000 a3b464b6 [ 10.436157] 1e40: 625a0000 c1834200 c0f04dc8 00000000 c1805a00 0000000a c0f01ec0 63597fc0 [ 10.444396] 1e60: 00000002 c019c4d4 00000000 a3b464b6 c1834200 c183426c 00000000 c019c55c [ 10.452636] 1e80: c1834200 c183426c 00000000 c01a0d60 c0e6d5b0 00000000 00000000 c019bba8 [ 10.460876] 1ea0: c07af038 20030013 ffffffff c0f01ef4 cfb63a78 c0f00000 63597fc0 c0100b3c [ 10.469116] 1ec0: 00000000 c0f0b798 ffffe000 0ecf6000 00000004 c0f0b618 00000000 00000004 [ 10.477325] 1ee0: cfb63a78 c0ff94c0 63597fc0 00000002 00000001 c0f01f10 c07aef64 c07af038 [ 10.485565] 1f00: 20030013 ffffffff 00000051 c07aef3c cfb63d70 cfb63a78 633d8f31 00000000 [ 10.493804] 1f20: c0f01f73 00000002 61c16980 00000002 7aca1a80 cfb63a78 c0f0b618 00000004 [ 10.502044] 1f40: c0f04e64 cfb63a78 c0f04dc8 c0f0b618 c0e6da70 c07af2fc ffffe000 c0f04e10 [ 10.510284] 1f60: 00000001 c016e754 00000004 c0f04dc0 01000000 a3b464b6 c0f084ac 000000e4 [ 10.518524] 1f80: c0e43a60 cfdff0d4 c0f04dc0 00000000 cfdff0c0 c0f084ac c0ccdbc8 c016eae8 [ 10.526763] 1fa0: c1000068 c0e00f0c ffffffff ffffffff 00000000 c0e00598 00000000 c0f04dc8 [ 10.535003] 1fc0: 00000000 c0e43a60 a3b16bb4 00000000 00000000 c0e00330 00000051 10c0387d [ 10.543243] 1fe0: 000007a3 8249b600 411fc083 10c5387d 00000000 00000000 00000000 00000000 [ 10.551483] [<c05b0ac8>] (omap3_l3_app_irq) from [<c019c2a4>] (__handle_irq_event_percpu+0x48/0x250) [ 10.560699] [<c019c2a4>] (__handle_irq_event_percpu) from [<c019c4d4>] (handle_irq_event_percpu+0x28/0x78) [ 10.570465] [<c019c4d4>] (handle_irq_event_percpu) from [<c019c55c>] (handle_irq_event+0x38/0x5c) [ 10.579406] [<c019c55c>] (handle_irq_event) from [<c01a0d60>] (handle_level_irq+0xc0/0x14c) [ 10.587829] [<c01a0d60>] (handle_level_irq) from [<c019bba8>] (handle_domain_irq+0x70/0xac) [ 10.596282] [<c019bba8>] (handle_domain_irq) from [<c0100b3c>] (__irq_svc+0x5c/0x90) [ 10.604095] Exception stack(0xc0f01ec0 to 0xc0f01f08) [ 10.609191] 1ec0: 00000000 c0f0b798 ffffe000 0ecf6000 00000004 c0f0b618 00000000 00000004 [ 10.617431] 1ee0: cfb63a78 c0ff94c0 63597fc0 00000002 00000001 c0f01f10 c07aef64 c07af038 [ 10.625671] 1f00: 20030013 ffffffff [ 10.629180] [<c0100b3c>] (__irq_svc) from [<c07af038>] (cpuidle_enter_state+0x258/0x4b8)
Quoted 8 lines
[ 10.637359] [<c07af038>] (cpuidle_enter_state) from [<c07af2fc>] (cpuidle_enter+0x50/0x54) [ 10.645690] [<c07af2fc>] (cpuidle_enter) from [<c016e754>] (do_idle+0x224/0x284) [ 10.653167] [<c016e754>] (do_idle) from [<c016eae8>] (cpu_startup_entry+0x18/0x20) [ 10.660827] [<c016eae8>] (cpu_startup_entry) from [<c0e00f0c>] (start_kernel+0x5c8/0x6c8) [ 10.669067] Code: e3c0003c e007100c e1903001 0affffc9 (e7f001f2) [ 10.675231] ---[ end trace 737d680415879d80 ]--- [ 10.679901] Kernel panic - not syncing: Fatal exception in interrupt [ 10.686309] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
[2]
https://github.com/tmlind/linux_openpvrsgx/commits/droid4-pending-pvr-omapdrm-v5.15
Hi,
Quoted 3 lines · Merlijn B.W. Wajer
Any ideas what could cause this? Please let me know if I should provide additional debug info.
Just a follow up, this also seems to happen after booting, depending on the modules that are loaded, it looks like. I blacklisted a few more modules and now my system consistently doesn't stay up for more than a few minutes [1]. (I am trying to make it idle properly, but of course wasn't counting on that causing crashes more often. :-))
In my case, commenting these in my modprobe blacklist made crashes go away (but that could simply be because my system was finally idling with these modules blocked and unblocking them makes it not idle...):
Quoted 6 lines
# Blocks idle (maybe audio does not complete?) #blacklist
omap3_isp
# Untested, just adding to prevent more usb loading #blacklist libcomposite
#blacklistusb_f_ecm
#blacklistusb_f_mass_storage
Not sure if it helps...
Cheers,
Merlijn
[1]
Quoted 89 lines
[ 132.892974] Internal error: Oops - undefined instruction: 0 [#1] PREEMPT SMP ARM [ 132.900573] Modules linked in: bluetooth ecdh_generic ecc libaes zram zsmalloc cmt_speech nokia_modem
ssi_protocolphonet evdevwl1251_spiwl1251 mac80211 libarc4omap3_rom_rngrng_coreir_rx51rc_coresnd_soc_rx51leds_gpioisp1704_chargerpvrsrvkm_omap3_sgx530_121sha256_genericlibsha256sha256_armudc_coregpio_keyspwm_omap_dmtimerdisplay_connectorcpufreq_dtsnd_soc_omap_mcbspsnd_soc_ti_sdmasnd_soc_tpa6130a2cfg80211snd_soc_coresnd_pcm_dmaenginesnd_pcmomap_shamcrypto_enginesnd_timersnd bq27xxx_battery_i2csoundcorebq27xxx_batterybq2415x_chargerleds_lp5523leds_lp55xx_commonled_classpanel_sony_acx565akmomap_mailboxrtc_twlohci_platformohci_hcdehci_hcdtwl4030_pwrbuttonpwm_twl_ledst_accel_i2cst_sensors_i2ctwl4030_keypadst_accelpwm_twltwl4030_madcmatrix_keymapst_sensorstwl4030_chargerindustrialio_triggered_bufferkfifo_bufusbcore industrialiousb_commonomap_ssiomapdrm hsi drm_kms_helpercfbfillrect syscopyarea cfbimgblt sysfillrect sysimgblt[ 132.902496] fb_sys_fops cfbcopyarea drm drm_panel_orientation_quirks cec [ 132.996398] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.2-218697-gd1b31b78bf22-dirty #1 [ 133.004760] Hardware name: Nokia RX-51 board [ 133.009094] PC is at omap3_l3_app_irq+0x100/0x120 [ 133.013885] LR is at __handle_irq_event_percpu+0x48/0x250 [ 133.019409] pc : [<c05b0ac8>] lr : [<c019c2a4>] psr: 20010193 [ 133.025756] sp : c0f01d58 ip : 3ccf0000 fp : c1834200 [ 133.031036] r10: c0ff8760 r9 : c0ff8780 r8 : 0000001a [ 133.036315] r7 : 00000000 r6 : 00400000 r5 : 00000000 r4 : f8000000 [ 133.042907] r3 : 00400000 r2 : 49b67397 r1 : 00000000 r0 : 00400000 [ 133.049530] Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none [ 133.056823] Control: 10c5387d Table: 850d8019 DAC: 00000051 [ 133.062652] Register r0 information: non-paged memory [ 133.067779] Register r1 information: NULL pointer [ 133.072570] Register r2 information: non-paged memory [ 133.077728] Register r3 information: non-paged memory [ 133.082855] Register r4 information: 0-page vmalloc region starting at 0xf8000000 allocated at iotab le_init+0x0/0xf4 [ 133.093536] Register r5 information: NULL pointer [ 133.098327] Register r6 information: non-paged memory [ 133.103485] Register r7 information: NULL pointer [ 133.108245] Register r8 information: non-paged memory [ 133.113403] Register r9 information: non-slab/vmalloc memory [ 133.119140] Register r10 information: non-slab/vmalloc memory [ 133.124999] Register r11 information: slab kmalloc-256 start c1834200 pointer offset 0 size 256 [ 133.133880] Register r12 information: non-paged memory [ 133.139129] Process swapper/0 (pid: 0, stack limit = 0xcece1d46) [ 133.145202] Stack: (0xc0f01d58 to 0xc0f02000) [ 133.149627] 1d40: c0627a10 c1fc4d80 [ 133.157928] 1d60: c0f04dc8 c1a47500 c0f04dc8 00000000 c0f01db0 0000001a c0ff8780 c019c2a4 [ 133.166198] 1d80: 00000000 49b67397 c0f04dc8 c1834200 c0f04dc8 c0f01ec0 c1805a00 0000000a [ 133.174468] 1da0: c0f01e08 f1069c8e c0f03080 c019c4d4 00000000 49b67397 c1834200 c183426c [ 133.182739] 1dc0: c0f01ec0 c019c55c c1834200 c183426c c0f01ec0 c01a0d60 c0e6d5b0 00000000 [ 133.191009] 1de0: c0f01ec0 c019bba8 c0101268 20010113 ffffffff c0f01e3c 0000001f c0f00000 [ 133.199310] 1e00: f1069c8e c0100b3c 00000000 00000000 0ecf6000 c0e6e480 c0e6e480 ffffffc0 [ 133.207580] 1e20: 00000000 00000040 0000001f c0f01ec0 f1069c8e c0f03080 00000101 c0f01e58 [ 133.215850] 1e40: c0101250 c0101268 20010113 ffffffff 00000051 c0101250 c1805c50 80000000 [ 133.224121] 1e60: 0000000a ffffbeb9 04200002 c01a3124 c1835700 c0e6e480 00000000 00000000 [ 133.232391] 1e80: c1805a00 0000001f c0f01ec0 f1069c8e 0000001e c013d7d8 c0e6d5b0 c019bbac [ 133.240661] 1ea0: c07af038 20010013 ffffffff c0f01ef4 cfb63a78 c0f00000 f1069c8e c0100b3c [ 133.248931] 1ec0: 00000000 c0f0b798 ffffe000 0ecf6000 00000004 c0f0b618 00000000 00000004 [ 133.257232] 1ee0: cfb63a78 c0ff94c0 f1069c8e 0000001e 00000001 c0f01f10 c07aef64 c07af038 [ 133.265502] 1f00: 20010013 ffffffff 00000051 c07aef3c 0000000a cfb63a78 f0e7e0be 00000000 [ 133.273773] 1f20: c0f01f73 0000001e f043e400 0000001e 007689ec cfb63a78 c0f0b618 00000004 [ 133.282043] 1f40: c0f04e64 cfb63a78 c0f04dc8 c0f0b618 c0e6da70 c07af2fc ffffe000 c0f04e10 [ 133.290313] 1f60: 00000001 c016e754 00000004 c0f04dc0 01000000 49b67397 c0f084ac 000000e4 [ 133.298583] 1f80: c0e43a60 cfdff0d4 c0f04dc0 00000000 cfdff0c0 c0f084ac c0ccdbc8 c016eae8 [ 133.306884] 1fa0: c1000068 c0e00f0c ffffffff ffffffff 00000000 c0e00598 00000000 c0f04dc8 [ 133.315155] 1fc0: 00000000 c0e43a60 49b37c95 00000000 00000000 c0e00330 00000051 10c0387d [ 133.323425] 1fe0: 000007a3 8249c028 411fc083 10c5387d 00000000 00000000 00000000 00000000 [ 133.331695] [<c05b0ac8>] (omap3_l3_app_irq) from [<c019c2a4>] (__handle_irq_event_percpu+0x48/0x250) [ 133.340972] [<c019c2a4>] (__handle_irq_event_percpu) from [<c019c4d4>] (handle_irq_event_percpu+0x28 /0x78) [ 133.350769] [<c019c4d4>] (handle_irq_event_percpu) from [<c019c55c>] (handle_irq_event+0x38/0x5c) [ 133.359802] [<c019c55c>] (handle_irq_event) from [<c01a0d60>] (handle_level_irq+0xc0/0x14c) [ 133.368286] [<c01a0d60>] (handle_level_irq) from [<c019bba8>] (handle_domain_irq+0x70/0xac) [ 133.376770] [<c019bba8>] (handle_domain_irq) from [<c0100b3c>] (__irq_svc+0x5c/0x90) [ 133.384643] Exception stack(0xc0f01e08 to 0xc0f01e50) [ 133.389770] 1e00: 00000000 00000000 0ecf6000 c0e6e480 c0e6e480 ffffffc0 [ 133.398040] 1e20: 00000000 00000040 0000001f c0f01ec0 f1069c8e c0f03080 00000101 c0f01e58 [ 133.406311] 1e40: c0101250 c0101268 20010113 ffffffff [ 133.411437] [<c0100b3c>] (__irq_svc) from [<c0101268>] (__do_softirq+0x80/0x418) [ 133.418945] [<c0101268>] (__do_softirq) from [<c013d7d8>] (irq_exit+0xc4/0x104) [ 133.426391] [<c013d7d8>] (irq_exit) from [<c019bbac>] (handle_domain_irq+0x74/0xac) [ 133.434173] [<c019bbac>] (handle_domain_irq) from [<c0100b3c>] (__irq_svc+0x5c/0x90) [ 133.442047] Exception stack(0xc0f01ec0 to 0xc0f01f08) [ 133.447174] 1ec0: 00000000 c0f0b798 ffffe000 0ecf6000 00000004 c0f0b618 00000000 00000004 [ 133.455474] 1ee0: cfb63a78 c0ff94c0 f1069c8e 0000001e 00000001 c0f01f10 c07aef64 c07af038 [ 133.463714] 1f00: 20010013 ffffffff [ 133.467285] [<c0100b3c>] (__irq_svc) from [<c07af038>] (cpuidle_enter_state+0x258/0x4b8) [ 133.475524] [<c07af038>] (cpuidle_enter_state) from [<c07af2fc>] (cpuidle_enter+0x50/0x54) [ 133.483917] [<c07af2fc>] (cpuidle_enter) from [<c016e754>] (do_idle+0x224/0x284) [ 133.491424] [<c016e754>] (do_idle) from [<c016eae8>] (cpu_startup_entry+0x18/0x20) [ 133.499114] [<c016eae8>] (cpu_startup_entry) from [<c0e00f0c>] (start_kernel+0x5c8/0x6c8) [ 133.507446] Code: e3c0003c e007100c e1903001 0affffc9 (e7f001f2) [ 133.513641] ---[ end trace acca9e0df4827c8a ]--- [ 133.518341] Kernel panic - not syncing: Fatal exception in interrupt [ 133.524780] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
Hello again,
I am back with a bit more news, but no solution.
I've been trying to figure out when this problem might have been introduced. It looks like v5.9 did not have the problem yet, then v5.10 has a (different or similar?) problem which prevents booting [1], which looks like it was addressed in Andreas' upstream commit ec76c2eea903947202098090bbe07a739b5246e9 ("ARM: OMAP2+: omap_device: fix idling of devices during probe"). No kernel after v5.10 is free of the issue, but I am not sure how to continue the search.
It looks like the problem was likely introduced between v5.9 and v5.10.
If I take linux-5.10.y (5.10.84), I can boot some of the times, but the problem occurs still (like with every kernel thereafter).
How would you suggest I bisect this issue now? v5.10 boot some of times (like any anything after it) with commit 21b2cec61c04bd175f0860d9411a472d5a0e7ba1 ("mmc: Set PROBE_PREFER_ASYNCHRONOUS for drivers that existed in v4.4") reverted.
That would suggest the problem is not introduced in any of the stable patches to v5.10. Should I attempt to bisect (between v5.9 and v5.10) this problem and revert 21b2cec61c04bd175f0860d9411a472d5a0e7ba1 where appropriate (if it exists), or perhaps cherry pick commit ec76c2eea903947202098090bbe07a739b5246e9 (when appropriate) ?
Regards,
Merlijn
[1]
Quoted 294 lines
[ 4.602050] omap_hsmmc 4809c000.mmc: omap_device_late_idle: enabled but no driver. Idling [ 4.610778] omap_hsmmc 480b4000.mmc: omap_device_late_idle: enabled but no driver. Idling [ 4.622741] In-band Error seen by MPU at address 0 [ 4.627685] ------------[ cut here ]------------ [ 4.632354] WARNING: CPU: 0 PID: 7 at drivers/bus/omap_l3_smx.c:152 omap3_l3_app_irq+0x10c/0x130 [ 4.641204] Modules linked in: [ 4.644287] CPU: 0 PID: 7 Comm: kworker/u2:0 Not tainted 5.10.0 #9 [ 4.650512] Hardware name: Nokia RX-51 board [ 4.654846] Workqueue: events_unbound async_run_entry_fn [ 4.660247] [<c0110924>] (unwind_backtrace) from [<c010b1f4>] (show_stack+0x10/0x14) [ 4.668060] [<c010b1f4>] (show_stack) from [<c09c8478>] (dump_stack+0xc0/0xdc) [ 4.675354] [<c09c8478>] (dump_stack) from [<c0138564>] (__warn+0xd4/0x140) [ 4.682373] [<c0138564>] (__warn) from [<c09c3f3c>] (warn_slowpath_fmt+0x60/0xbc) [ 4.689910] [<c09c3f3c>] (warn_slowpath_fmt) from [<c05a444c>] (omap3_l3_app_irq+0x10c/0x130) [ 4.698486] [<c05a444c>] (omap3_l3_app_irq) from [<c019d3f8>] (__handle_irq_event_percpu+0x58/0x208) [ 4.707702] [<c019d3f8>] (__handle_irq_event_percpu) from [<c019d694>] (handle_irq_event+0x68/0xd4) [ 4.716827] [<c019d694>] (handle_irq_event) from [<c01a1c4c>] (handle_level_irq+0xc4/0x1c8) [ 4.725219] [<c01a1c4c>] (handle_level_irq) from [<c019cc90>] (__handle_domain_irq+0x84/0xfc) [ 4.733825] [<c019cc90>] (__handle_domain_irq) from [<c0100b6c>] (__irq_svc+0x6c/0x90) [ 4.741790] Exception stack(0xc10d3df8 to 0xc10d3e40) [ 4.746856] 3de0: c17ef3b0 20000013 [ 4.755096] 3e00: 00000000 00000002 c17ef000 00000000 c17ef3b0 00000000 c124e810 00000000 [ 4.763336] 3e20: c1248f40 00000066 00000000 c10d3e48 c07c3fd8 c09d5f20 60000013 ffffffff [ 4.771575] [<c0100b6c>] (__irq_svc) from [<c09d5f20>] (_raw_spin_unlock_irqrestore+0x20/0x24) [ 4.780273] [<c09d5f20>] (_raw_spin_unlock_irqrestore) from [<c07c3fd8>] (omap_hsmmc_probe+0x4ec/0x9cc) [ 4.789733] [<c07c3fd8>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 4.798248] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 4.806579] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 4.814819] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 4.823394] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 4.832763] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 4.842407] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 4.851074] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 4.859222] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 4.866699] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 4.873962] Exception stack(0xc10d3fb0 to 0xc10d3ff8) [ 4.879058] 3fa0: 00000000 00000000 00000000 00000000 [ 4.887268] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 4.895507] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 4.902160] ---[ end trace c76b7828d6f2f576 ]--- [ 4.908721] In-band Error seen by MPU at address 0 [ 4.913635] ------------[ cut here ]------------ [ 4.918304] WARNING: CPU: 0 PID: 50 at drivers/bus/omap_l3_smx.c:152 omap3_l3_app_irq+0x10c/0x130 [ 4.927246] Modules linked in: [ 4.930328] CPU: 0 PID: 50 Comm: kworker/u2:1 Tainted: G W 5.10.0 #9 [ 4.938049] Hardware name: Nokia RX-51 board [ 4.942352] Workqueue: events_unbound async_run_entry_fn [ 4.947753] [<c0110924>] (unwind_backtrace) from [<c010b1f4>] (show_stack+0x10/0x14) [ 4.955566] [<c010b1f4>] (show_stack) from [<c09c8478>] (dump_stack+0xc0/0xdc) [ 4.962829] [<c09c8478>] (dump_stack) from [<c0138564>] (__warn+0xd4/0x140) [ 4.969848] [<c0138564>] (__warn) from [<c09c3f3c>] (warn_slowpath_fmt+0x60/0xbc) [ 4.977386] [<c09c3f3c>] (warn_slowpath_fmt) from [<c05a444c>] (omap3_l3_app_irq+0x10c/0x130) [ 4.985992] [<c05a444c>] (omap3_l3_app_irq) from [<c019d3f8>] (__handle_irq_event_percpu+0x58/0x208) [ 4.995178] [<c019d3f8>] (__handle_irq_event_percpu) from [<c019d694>] (handle_irq_event+0x68/0xd4) [ 5.004302] [<c019d694>] (handle_irq_event) from [<c01a1c4c>] (handle_level_irq+0xc4/0x1c8) [ 5.012725] [<c01a1c4c>] (handle_level_irq) from [<c019cc90>] (__handle_domain_irq+0x84/0xfc) [ 5.021301] [<c019cc90>] (__handle_domain_irq) from [<c0100b6c>] (__irq_svc+0x6c/0x90) [ 5.029266] Exception stack(0xc160ddf8 to 0xc160de40) [ 5.034362] dde0: c28a7bb0 20000013 [ 5.042572] de00: 00000000 00000002 c28a7800 00000000 c28a7bb0 00000000 c124c410 00000000 [ 5.050811] de20: c1248400 00000063 00000000 c160de48 c07c3fd8 c09d5f20 60000013 ffffffff [ 5.059051] [<c0100b6c>] (__irq_svc) from [<c09d5f20>] (_raw_spin_unlock_irqrestore+0x20/0x24) [ 5.067749] [<c09d5f20>] (_raw_spin_unlock_irqrestore) from [<c07c3fd8>] (omap_hsmmc_probe+0x4ec/0x9cc) [ 5.077209] [<c07c3fd8>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 5.085723] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 5.094024] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 5.102264] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 5.110870] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 5.120239] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 5.129882] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 5.138549] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 5.146697] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 5.154144] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 5.161407] Exception stack(0xc160dfb0 to 0xc160dff8) [ 5.166503] dfa0: 00000000 00000000 00000000 00000000 [ 5.174743] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 5.182952] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 5.189605] ---[ end trace c76b7828d6f2f577 ]--- [ 5.198516] 8<--- cut here --- [ 5.201599] Unhandled fault: external abort on non-linefetch (0x1028) at 0xfa0b402c [ 5.209320] pgd = (ptrval) [ 5.212036] [fa0b402c] *pgd=48011452(bad) [ 5.216125] Internal error: : 1028 [#1] SMP ARM [ 5.220672] Modules linked in: [ 5.223785] CPU: 0 PID: 7 Comm: kworker/u2:0 Tainted: G W 5.10.0 #9 [ 5.231384] Hardware name: Nokia RX-51 board [ 5.235717] Workqueue: events_unbound async_run_entry_fn [ 5.241088] PC is at omap_hsmmc_set_bus_width+0x8/0x78 [ 5.246276] LR is at omap_hsmmc_set_ios+0x44/0x23c [ 5.251098] pc : [<c07c23f0>] lr : [<c07c377c>] psr: 20000013 [ 5.257385] sp : c10d3e00 ip : c17ef000 fp : 00000066 [ 5.262664] r10: c1248f40 r9 : 00000000 r8 : c124e810 [ 5.267913] r7 : 00000000 r6 : c17ef380 r5 : c17ef258 r4 : c17ef000 [ 5.274475] r3 : 00000001 r2 : 54192eb7 r1 : fa0b4000 r0 : c17ef380 [ 5.281036] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 5.288238] Control: 10c5387d Table: 80004019 DAC: 00000051 [ 5.294006] Process kworker/u2:0 (pid: 7, stack limit = 0x(ptrval)) [ 5.300323] Stack: (0xc10d3e00 to 0xc10d4000) [ 5.304718] 3e00: c17ef000 00000012 00000001 00000000 c124e810 c07a4ccc c17ef000 00000000 [ 5.312957] 3e20: c17ef3b0 00000000 c124e810 c07a5f9c c17ef000 c07a7138 c17ef000 00000000 [ 5.321197] 3e40: c17ef3b0 c07c4034 00000000 c2857a00 c17ef380 54192eb7 00000000 00000000 [ 5.329406] 3e60: c124e810 c0edb05c c0f38c2c 00000000 c0edb05c 00000020 c1009400 c0685ba4 [ 5.337646] 3e80: c124e810 c0f38c24 00000000 c0f38c2c 00000000 c06839b4 00000000 c124e810 [ 5.345886] 3ea0: c0edb05c c10d2000 c068409c 00000000 00000000 c0efa620 c1009400 c0683f40 [ 5.354125] 3ec0: 00000000 c10d3efc c10d2000 c06819c8 c2857100 c1039e6c c1604e38 54192eb7 [ 5.362365] 3ee0: c1009400 c124e810 c10d2000 c124e854 c2857100 c0683198 c1009430 c124e810 [ 5.370605] 3f00: 00010101 54192eb7 c2857110 c0f02018 c1009400 c0160458 c2857110 c10bdf80 [ 5.378845] 3f20: c1009400 c100f000 00000000 c0155660 c10d2000 c1009400 c1009418 c10bdf80 [ 5.387054] 3f40: c1009400 c10bdf94 c1009418 c0e03d00 00000088 c10d2000 c1009400 c01559f8 [ 5.395294] 3f60: 00000000 c10b3c40 c10b3dc0 00000000 c10d2000 c01559b0 c10bdf80 c10c5e9c [ 5.403533] 3f80: c10b3c64 c015b738 00000001 c10b3dc0 c015b608 00000000 00000000 00000000 [ 5.411773] 3fa0: 00000000 00000000 00000000 c0100168 00000000 00000000 00000000 00000000 [ 5.420013] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 5.428253] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000 [ 5.436492] [<c07c23f0>] (omap_hsmmc_set_bus_width) from [<c07c377c>] (omap_hsmmc_set_ios+0x44/0x23c) [ 5.445770] [<c07c377c>] (omap_hsmmc_set_ios) from [<c07a4ccc>] (mmc_power_up.part.0+0x3c/0x154) [ 5.454620] [<c07a4ccc>] (mmc_power_up.part.0) from [<c07a5f9c>] (mmc_start_host+0x88/0x9c) [ 5.463043] [<c07a5f9c>] (mmc_start_host) from [<c07a7138>] (mmc_add_host+0x58/0x84) [ 5.470855] [<c07a7138>] (mmc_add_host) from [<c07c4034>] (omap_hsmmc_probe+0x548/0x9cc) [ 5.479003] [<c07c4034>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 5.487518] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 5.495819] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 5.504058] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 5.512664] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 5.522033] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 5.531677] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 5.540344] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 5.548492] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 5.555938] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 5.563201] Exception stack(0xc10d3fb0 to 0xc10d3ff8) [ 5.568298] 3fa0: 00000000 00000000 00000000 00000000 [ 5.576538] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 5.584777] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 5.591430] Code: e7f001f2 c0ba858c e590c004 e5901024 (e591302c) [ 5.597595] ---[ end trace c76b7828d6f2f578 ]--- [ 5.602264] In-band Error seen by MPU at address 0 [ 5.607177] ------------[ cut here ]------------ [ 5.611846] WARNING: CPU: 0 PID: 7 at drivers/bus/omap_l3_smx.c:152 omap3_l3_app_irq+0x10c/0x130 [ 5.620666] Modules linked in: [ 5.623748] CPU: 0 PID: 7 Comm: kworker/u2:0 Tainted: G D W 5.10.0 #9 [ 5.631378] Hardware name: Nokia RX-51 board [ 5.635681] Workqueue: events_unbound async_run_entry_fn [ 5.641052] [<c0110924>] (unwind_backtrace) from [<c010b1f4>] (show_stack+0x10/0x14) [ 5.648864] [<c010b1f4>] (show_stack) from [<c09c8478>] (dump_stack+0xc0/0xdc) [ 5.656158] [<c09c8478>] (dump_stack) from [<c0138564>] (__warn+0xd4/0x140) [ 5.663177] [<c0138564>] (__warn) from [<c09c3f3c>] (warn_slowpath_fmt+0x60/0xbc) [ 5.670715] [<c09c3f3c>] (warn_slowpath_fmt) from [<c05a444c>] (omap3_l3_app_irq+0x10c/0x130) [ 5.679321] [<c05a444c>] (omap3_l3_app_irq) from [<c019d3f8>] (__handle_irq_event_percpu+0x58/0x208) [ 5.688507] [<c019d3f8>] (__handle_irq_event_percpu) from [<c019d694>] (handle_irq_event+0x68/0xd4) [ 5.697631] [<c019d694>] (handle_irq_event) from [<c01a1c4c>] (handle_level_irq+0xc4/0x1c8) [ 5.706024] [<c01a1c4c>] (handle_level_irq) from [<c019cc90>] (__handle_domain_irq+0x84/0xfc) [ 5.714630] [<c019cc90>] (__handle_domain_irq) from [<c0100b6c>] (__irq_svc+0x6c/0x90) [ 5.722564] Exception stack(0xc10d3cd8 to 0xc10d3d20) [ 5.727661] 3cc0: 00000000 00000000 [ 5.735900] 3ce0: c10bfa80 00000002 c10c6b40 00000000 0000000b ffffe000 c0e09fec 00000000 [ 5.744140] 3d00: bf000000 c10d3d58 08f0d180 c10d3d28 00000000 c01e342c 00000113 ffffffff [ 5.752380] [<c0100b6c>] (__irq_svc) from [<c01e342c>] (acct_collect+0x1b0/0x1cc) [ 5.759918] [<c01e342c>] (acct_collect) from [<c013d79c>] (do_exit+0x174/0xa40) [ 5.767303] [<c013d79c>] (do_exit) from [<c010b510>] (die+0x318/0x354) [ 5.773864] [<c010b510>] (die) from [<c01154e4>] (do_DataAbort+0xa4/0xbc) [ 5.780700] [<c01154e4>] (do_DataAbort) from [<c0100ac0>] (__dabt_svc+0x60/0xa0) [ 5.788146] Exception stack(0xc10d3db0 to 0xc10d3df8) [ 5.793243] 3da0: c17ef380 fa0b4000 54192eb7 00000001 [ 5.801483] 3dc0: c17ef000 c17ef258 c17ef380 00000000 c124e810 00000000 c1248f40 00000066 [ 5.809692] 3de0: c17ef000 c10d3e00 c07c377c c07c23f0 20000013 ffffffff [ 5.816375] [<c0100ac0>] (__dabt_svc) from [<c07c23f0>] (omap_hsmmc_set_bus_width+0x8/0x78) [ 5.824798] [<c07c23f0>] (omap_hsmmc_set_bus_width) from [<c07c377c>] (omap_hsmmc_set_ios+0x44/0x23c) [ 5.834075] [<c07c377c>] (omap_hsmmc_set_ios) from [<c07a4ccc>] (mmc_power_up.part.0+0x3c/0x154) [ 5.842926] [<c07a4ccc>] (mmc_power_up.part.0) from [<c07a5f9c>] (mmc_start_host+0x88/0x9c) [ 5.851348] [<c07a5f9c>] (mmc_start_host) from [<c07a7138>] (mmc_add_host+0x58/0x84) [ 5.859130] [<c07a7138>] (mmc_add_host) from [<c07c4034>] (omap_hsmmc_probe+0x548/0x9cc) [ 5.867279] [<c07c4034>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 5.875793] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 5.884124] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 5.892364] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 5.900939] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 5.910308] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 5.919952] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 5.928619] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 5.936767] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 5.944213] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 5.951477] Exception stack(0xc10d3fb0 to 0xc10d3ff8) [ 5.956573] 3fa0: 00000000 00000000 00000000 00000000 [ 5.964813] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 5.973052] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 5.979705] ---[ end trace c76b7828d6f2f579 ]--- [ 5.990539] 8<--- cut here --- [ 5.993682] Unhandled fault: external abort on non-linefetch (0x1028) at 0xfa09c02c [ 6.001373] pgd = (ptrval) [ 6.004119] [fa09c02c] *pgd=48011452(bad) [ 6.008178] Internal error: : 1028 [#2] SMP ARM [ 6.012756] Modules linked in: [ 6.015838] CPU: 0 PID: 50 Comm: kworker/u2:1 Tainted: G D W 5.10.0 #9 [ 6.023559] Hardware name: Nokia RX-51 board [ 6.027893] Workqueue: events_unbound async_run_entry_fn [ 6.033264] PC is at omap_hsmmc_set_bus_width+0x8/0x78 [ 6.038452] LR is at omap_hsmmc_set_ios+0x44/0x23c [ 6.043273] pc : [<c07c23f0>] lr : [<c07c377c>] psr: 60000113 [ 6.049560] sp : c160de00 ip : c28a7800 fp : 00000063 [ 6.054840] r10: c1248400 r9 : 00000000 r8 : c124c410 [ 6.060089] r7 : 00000000 r6 : c28a7b80 r5 : c28a7a58 r4 : c28a7800 [ 6.066650] r3 : 00000001 r2 : 24f1df74 r1 : fa09c000 r0 : c28a7b80 [ 6.073211] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 6.080413] Control: 10c5387d Table: 80004019 DAC: 00000051 [ 6.086181] Process kworker/u2:1 (pid: 50, stack limit = 0x(ptrval)) [ 6.092590] Stack: (0xc160de00 to 0xc160e000) [ 6.096984] de00: c28a7800 00000013 00000001 00000000 c124c410 c07a4ccc c28a7800 00000000 [ 6.105224] de20: c28a7bb0 00000000 c124c410 c07a5f9c c28a7800 c07a7138 c28a7800 00000000 [ 6.113464] de40: c28a7bb0 c07c4034 00000000 c28a1980 c28a7b80 24f1df74 00000000 00000000 [ 6.121673] de60: c124c410 c0edb05c c0f38c2c 00000000 c0edb05c 00000020 c1009400 c0685ba4 [ 6.129913] de80: c124c410 c0f38c24 00000000 c0f38c2c 00000000 c06839b4 00000000 c124c410 [ 6.138153] dea0: c0edb05c c160c000 c068409c 00000000 00000000 c0efa620 c1009400 c0683f40 [ 6.146392] dec0: 00000000 c160defc c160c000 c06819c8 c2857040 c1039e6c c1604e38 24f1df74 [ 6.154632] dee0: c1009400 c124c410 c160c000 c124c454 c2857040 c0683198 c1009400 c124c410 [ 6.162872] df00: 00010101 24f1df74 c2857050 c0f02018 c1009400 c0160458 c2857050 c1604d00 [ 6.171112] df20: c1009400 c100f000 00000000 c0155660 c160c000 c1009400 c1009418 c1604d00 [ 6.179321] df40: c1009400 c1604d14 c1009418 c0e03d00 00000088 c160c000 c1009400 c01559f8 [ 6.187561] df60: 00000000 c16029c0 c1602a00 00000000 c160c000 c01559b0 c1604d00 c10d3ebc [ 6.195800] df80: c16029e4 c015b738 00000001 c1602a00 c015b608 00000000 00000000 00000000 [ 6.204040] dfa0: 00000000 00000000 00000000 c0100168 00000000 00000000 00000000 00000000 [ 6.212280] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 6.220520] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000 [ 6.228759] [<c07c23f0>] (omap_hsmmc_set_bus_width) from [<c07c377c>] (omap_hsmmc_set_ios+0x44/0x23c) [ 6.238067] [<c07c377c>] (omap_hsmmc_set_ios) from [<c07a4ccc>] (mmc_power_up.part.0+0x3c/0x154) [ 6.246887] [<c07a4ccc>] (mmc_power_up.part.0) from [<c07a5f9c>] (mmc_start_host+0x88/0x9c) [ 6.255310] [<c07a5f9c>] (mmc_start_host) from [<c07a7138>] (mmc_add_host+0x58/0x84) [ 6.263122] [<c07a7138>] (mmc_add_host) from [<c07c4034>] (omap_hsmmc_probe+0x548/0x9cc) [ 6.271270] [<c07c4034>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 6.279785] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 6.288116] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 6.296356] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 6.304931] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 6.314300] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 6.323944] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 6.332641] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 6.340789] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 6.348236] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 6.355499] Exception stack(0xc160dfb0 to 0xc160dff8) [ 6.360595] dfa0: 00000000 00000000 00000000 00000000 [ 6.368835] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 6.377044] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 6.383728] Code: e7f001f2 c0ba858c e590c004 e5901024 (e591302c) [ 6.389862] ---[ end trace c76b7828d6f2f57a ]--- [ 6.394561] In-band Error seen by MPU at address 0 [ 6.399475] ------------[ cut here ]------------ [ 6.404113] WARNING: CPU: 0 PID: 50 at drivers/bus/omap_l3_smx.c:152 omap3_l3_app_irq+0x10c/0x130 [ 6.413055] Modules linked in: [ 6.416137] CPU: 0 PID: 50 Comm: kworker/u2:1 Tainted: G D W 5.10.0 #9 [ 6.423828] Hardware name: Nokia RX-51 board [ 6.428161] Workqueue: events_unbound async_run_entry_fn [ 6.433532] [<c0110924>] (unwind_backtrace) from [<c010b1f4>] (show_stack+0x10/0x14) [ 6.441345] [<c010b1f4>] (show_stack) from [<c09c8478>] (dump_stack+0xc0/0xdc) [ 6.448638] [<c09c8478>] (dump_stack) from [<c0138564>] (__warn+0xd4/0x140) [ 6.455657] [<c0138564>] (__warn) from [<c09c3f3c>] (warn_slowpath_fmt+0x60/0xbc) [ 6.463195] [<c09c3f3c>] (warn_slowpath_fmt) from [<c05a444c>] (omap3_l3_app_irq+0x10c/0x130) [ 6.471801] [<c05a444c>] (omap3_l3_app_irq) from [<c019d3f8>] (__handle_irq_event_percpu+0x58/0x208) [ 6.480987] [<c019d3f8>] (__handle_irq_event_percpu) from [<c019d694>] (handle_irq_event+0x68/0xd4) [ 6.490112] [<c019d694>] (handle_irq_event) from [<c01a1c4c>] (handle_level_irq+0xc4/0x1c8) [ 6.498504] [<c01a1c4c>] (handle_level_irq) from [<c019cc90>] (__handle_domain_irq+0x84/0xfc) [ 6.507110] [<c019cc90>] (__handle_domain_irq) from [<c0100b6c>] (__irq_svc+0x6c/0x90) [ 6.515075] Exception stack(0xc160dcd8 to 0xc160dd20) [ 6.520141] dcc0: 00000000 00000000 [ 6.528381] dce0: c15bb000 00000002 c12ad6c0 00000000 0000000b ffffe000 c0e09fec 00000000 [ 6.536621] dd00: bf000000 c160dd58 09896800 c160dd28 00000000 c01e342c 00000113 ffffffff [ 6.544860] [<c0100b6c>] (__irq_svc) from [<c01e342c>] (acct_collect+0x1b0/0x1cc) [ 6.552398] [<c01e342c>] (acct_collect) from [<c013d79c>] (do_exit+0x174/0xa40) [ 6.559783] [<c013d79c>] (do_exit) from [<c010b510>] (die+0x318/0x354) [ 6.566345] [<c010b510>] (die) from [<c01154e4>] (do_DataAbort+0xa4/0xbc) [ 6.573181] [<c01154e4>] (do_DataAbort) from [<c0100ac0>] (__dabt_svc+0x60/0xa0) [ 6.580627] Exception stack(0xc160ddb0 to 0xc160ddf8) [ 6.585723] dda0: c28a7b80 fa09c000 24f1df74 00000001 [ 6.593963] ddc0: c28a7800 c28a7a58 c28a7b80 00000000 c124c410 00000000 c1248400 00000063 [ 6.602203] dde0: c28a7800 c160de00 c07c377c c07c23f0 60000113 ffffffff [ 6.608856] [<c0100ac0>] (__dabt_svc) from [<c07c23f0>] (omap_hsmmc_set_bus_width+0x8/0x78) [ 6.617279] [<c07c23f0>] (omap_hsmmc_set_bus_width) from [<c07c377c>] (omap_hsmmc_set_ios+0x44/0x23c) [ 6.626556] [<c07c377c>] (omap_hsmmc_set_ios) from [<c07a4ccc>] (mmc_power_up.part.0+0x3c/0x154) [ 6.635406] [<c07a4ccc>] (mmc_power_up.part.0) from [<c07a5f9c>] (mmc_start_host+0x88/0x9c) [ 6.643829] [<c07a5f9c>] (mmc_start_host) from [<c07a7138>] (mmc_add_host+0x58/0x84) [ 6.651611] [<c07a7138>] (mmc_add_host) from [<c07c4034>] (omap_hsmmc_probe+0x548/0x9cc) [ 6.659759] [<c07c4034>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 6.668273] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 6.676605] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 6.684844] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 6.693420] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 6.702789] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 6.712432] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 6.721099] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 6.729248] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 6.736694] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 6.743957] Exception stack(0xc160dfb0 to 0xc160dff8) [ 6.749053] dfa0: 00000000 00000000 00000000 00000000 [ 6.757293] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 6.765533] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 6.772186] ---[ end trace c76b7828d6f2f57b ]---
Hi Tony, Andreas,
Quoted 3 lines · Merlijn B.W. Wajer
Hello again,
I am back with a bit more news, but no solution.
It looks like commit fb2c599f056640d289b2147fbe6d9eaee689f1b2 ("ARM:
omap3: enable off mode automatically") is causing these failures on the Nokia N900. Reverting this commit on top of v5.10 (with 21b2cec61c04bd175f0860d9411a472d5a0e7ba1 also reverted), makes the problem go away (I tried 10 times).
The same holds for 5.15.y, just reverting fb2c599f056640d289b2147fbe6d9eaee689f1b2 makes the system "stable" (at least the problem reported in the first email is gone).
Perhaps the "pwrst->next_state" change is causing these problems, since I can enter OFF mode just fine with init=/bin/sh and a custom script?
Cheers,
Merlijn
PS: Nikolaus - might be worth getting that N900 going some time soon. ;-)
PPS: Looks like there is a regression between v5.7 and v5.9 where OFF mode is no longer hit normally (I hacked around that for my 5.15.y branch, but this is *not* what is causing the problems here). I'll try to bisect that regression later. I think I was able to hit 7mA at 3.8V with v5.7, so that's worth bisecting I think.
Quoted 324 lines
I've been trying to figure out when this problem might have been introduced. It looks like v5.9 did not have the problem yet, then v5.10 has a (different or similar?) problem which prevents booting [1], which looks like it was addressed in Andreas' upstream commit
ec76c2eea903947202098090bbe07a739b5246e9("ARM: OMAP2+:omap_device: fix idling of devices during probe"). No kernel after v5.10 is free of the issue, but I am not sure how to continue the search.It looks like the problem was likely introduced between v5.9 and v5.10.
If I take linux-5.10.y (5.10.84), I can boot some of the times, but the problem occurs still (like with every kernel thereafter).
How would you suggest I bisect this issue now? v5.10 boot some of times (like any anything after it) with commit
21b2cec61c04bd175f0860d9411a472d5a0e7ba1("mmc: SetPROBE_PREFER_ASYNCHRONOUSfor drivers that existed in v4.4") reverted.That would suggest the problem is not introduced in any of the stable patches to v5.10. Should I attempt to bisect (between v5.9 and v5.10) this problem and revert
21b2cec61c04bd175f0860d9411a472d5a0e7ba1where appropriate (if it exists), or perhaps cherry pick commitec76c2eea903947202098090bbe07a739b5246e9(when appropriate) ?Regards,
Merlijn[1]
Quoted 294 lines
[ 4.602050] omap_hsmmc 4809c000.mmc: omap_device_late_idle: enabled but no driver. Idling [ 4.610778] omap_hsmmc 480b4000.mmc: omap_device_late_idle: enabled but no driver. Idling [ 4.622741] In-band Error seen by MPU at address 0 [ 4.627685] ------------[ cut here ]------------ [ 4.632354] WARNING: CPU: 0 PID: 7 at drivers/bus/omap_l3_smx.c:152 omap3_l3_app_irq+0x10c/0x130 [ 4.641204] Modules linked in: [ 4.644287] CPU: 0 PID: 7 Comm: kworker/u2:0 Not tainted 5.10.0 #9 [ 4.650512] Hardware name: Nokia RX-51 board [ 4.654846] Workqueue: events_unbound async_run_entry_fn [ 4.660247] [<c0110924>] (unwind_backtrace) from [<c010b1f4>] (show_stack+0x10/0x14) [ 4.668060] [<c010b1f4>] (show_stack) from [<c09c8478>] (dump_stack+0xc0/0xdc) [ 4.675354] [<c09c8478>] (dump_stack) from [<c0138564>] (__warn+0xd4/0x140) [ 4.682373] [<c0138564>] (__warn) from [<c09c3f3c>] (warn_slowpath_fmt+0x60/0xbc) [ 4.689910] [<c09c3f3c>] (warn_slowpath_fmt) from [<c05a444c>] (omap3_l3_app_irq+0x10c/0x130) [ 4.698486] [<c05a444c>] (omap3_l3_app_irq) from [<c019d3f8>] (__handle_irq_event_percpu+0x58/0x208) [ 4.707702] [<c019d3f8>] (__handle_irq_event_percpu) from [<c019d694>] (handle_irq_event+0x68/0xd4) [ 4.716827] [<c019d694>] (handle_irq_event) from [<c01a1c4c>] (handle_level_irq+0xc4/0x1c8) [ 4.725219] [<c01a1c4c>] (handle_level_irq) from [<c019cc90>] (__handle_domain_irq+0x84/0xfc) [ 4.733825] [<c019cc90>] (__handle_domain_irq) from [<c0100b6c>] (__irq_svc+0x6c/0x90) [ 4.741790] Exception stack(0xc10d3df8 to 0xc10d3e40) [ 4.746856] 3de0: c17ef3b0 20000013 [ 4.755096] 3e00: 00000000 00000002 c17ef000 00000000 c17ef3b0 00000000 c124e810 00000000 [ 4.763336] 3e20: c1248f40 00000066 00000000 c10d3e48 c07c3fd8 c09d5f20 60000013 ffffffff [ 4.771575] [<c0100b6c>] (__irq_svc) from [<c09d5f20>] (_raw_spin_unlock_irqrestore+0x20/0x24) [ 4.780273] [<c09d5f20>] (_raw_spin_unlock_irqrestore) from [<c07c3fd8>] (omap_hsmmc_probe+0x4ec/0x9cc) [ 4.789733] [<c07c3fd8>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 4.798248] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 4.806579] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 4.814819] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 4.823394] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 4.832763] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 4.842407] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 4.851074] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 4.859222] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 4.866699] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 4.873962] Exception stack(0xc10d3fb0 to 0xc10d3ff8) [ 4.879058] 3fa0: 00000000 00000000 00000000 00000000 [ 4.887268] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 4.895507] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 4.902160] ---[ end trace c76b7828d6f2f576 ]--- [ 4.908721] In-band Error seen by MPU at address 0 [ 4.913635] ------------[ cut here ]------------ [ 4.918304] WARNING: CPU: 0 PID: 50 at drivers/bus/omap_l3_smx.c:152 omap3_l3_app_irq+0x10c/0x130 [ 4.927246] Modules linked in: [ 4.930328] CPU: 0 PID: 50 Comm: kworker/u2:1 Tainted: G W 5.10.0 #9 [ 4.938049] Hardware name: Nokia RX-51 board [ 4.942352] Workqueue: events_unbound async_run_entry_fn [ 4.947753] [<c0110924>] (unwind_backtrace) from [<c010b1f4>] (show_stack+0x10/0x14) [ 4.955566] [<c010b1f4>] (show_stack) from [<c09c8478>] (dump_stack+0xc0/0xdc) [ 4.962829] [<c09c8478>] (dump_stack) from [<c0138564>] (__warn+0xd4/0x140) [ 4.969848] [<c0138564>] (__warn) from [<c09c3f3c>] (warn_slowpath_fmt+0x60/0xbc) [ 4.977386] [<c09c3f3c>] (warn_slowpath_fmt) from [<c05a444c>] (omap3_l3_app_irq+0x10c/0x130) [ 4.985992] [<c05a444c>] (omap3_l3_app_irq) from [<c019d3f8>] (__handle_irq_event_percpu+0x58/0x208) [ 4.995178] [<c019d3f8>] (__handle_irq_event_percpu) from [<c019d694>] (handle_irq_event+0x68/0xd4) [ 5.004302] [<c019d694>] (handle_irq_event) from [<c01a1c4c>] (handle_level_irq+0xc4/0x1c8) [ 5.012725] [<c01a1c4c>] (handle_level_irq) from [<c019cc90>] (__handle_domain_irq+0x84/0xfc) [ 5.021301] [<c019cc90>] (__handle_domain_irq) from [<c0100b6c>] (__irq_svc+0x6c/0x90) [ 5.029266] Exception stack(0xc160ddf8 to 0xc160de40) [ 5.034362] dde0: c28a7bb0 20000013 [ 5.042572] de00: 00000000 00000002 c28a7800 00000000 c28a7bb0 00000000 c124c410 00000000 [ 5.050811] de20: c1248400 00000063 00000000 c160de48 c07c3fd8 c09d5f20 60000013 ffffffff [ 5.059051] [<c0100b6c>] (__irq_svc) from [<c09d5f20>] (_raw_spin_unlock_irqrestore+0x20/0x24) [ 5.067749] [<c09d5f20>] (_raw_spin_unlock_irqrestore) from [<c07c3fd8>] (omap_hsmmc_probe+0x4ec/0x9cc) [ 5.077209] [<c07c3fd8>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 5.085723] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 5.094024] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 5.102264] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 5.110870] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 5.120239] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 5.129882] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 5.138549] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 5.146697] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 5.154144] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 5.161407] Exception stack(0xc160dfb0 to 0xc160dff8) [ 5.166503] dfa0: 00000000 00000000 00000000 00000000 [ 5.174743] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 5.182952] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 5.189605] ---[ end trace c76b7828d6f2f577 ]--- [ 5.198516] 8<--- cut here --- [ 5.201599] Unhandled fault: external abort on non-linefetch (0x1028) at 0xfa0b402c [ 5.209320] pgd = (ptrval) [ 5.212036] [fa0b402c] *pgd=48011452(bad) [ 5.216125] Internal error: : 1028 [#1] SMP ARM [ 5.220672] Modules linked in: [ 5.223785] CPU: 0 PID: 7 Comm: kworker/u2:0 Tainted: G W 5.10.0 #9 [ 5.231384] Hardware name: Nokia RX-51 board [ 5.235717] Workqueue: events_unbound async_run_entry_fn [ 5.241088] PC is at omap_hsmmc_set_bus_width+0x8/0x78 [ 5.246276] LR is at omap_hsmmc_set_ios+0x44/0x23c [ 5.251098] pc : [<c07c23f0>] lr : [<c07c377c>] psr: 20000013 [ 5.257385] sp : c10d3e00 ip : c17ef000 fp : 00000066 [ 5.262664] r10: c1248f40 r9 : 00000000 r8 : c124e810 [ 5.267913] r7 : 00000000 r6 : c17ef380 r5 : c17ef258 r4 : c17ef000 [ 5.274475] r3 : 00000001 r2 : 54192eb7 r1 : fa0b4000 r0 : c17ef380 [ 5.281036] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 5.288238] Control: 10c5387d Table: 80004019 DAC: 00000051 [ 5.294006] Process kworker/u2:0 (pid: 7, stack limit = 0x(ptrval)) [ 5.300323] Stack: (0xc10d3e00 to 0xc10d4000) [ 5.304718] 3e00: c17ef000 00000012 00000001 00000000 c124e810 c07a4ccc c17ef000 00000000 [ 5.312957] 3e20: c17ef3b0 00000000 c124e810 c07a5f9c c17ef000 c07a7138 c17ef000 00000000 [ 5.321197] 3e40: c17ef3b0 c07c4034 00000000 c2857a00 c17ef380 54192eb7 00000000 00000000 [ 5.329406] 3e60: c124e810 c0edb05c c0f38c2c 00000000 c0edb05c 00000020 c1009400 c0685ba4 [ 5.337646] 3e80: c124e810 c0f38c24 00000000 c0f38c2c 00000000 c06839b4 00000000 c124e810 [ 5.345886] 3ea0: c0edb05c c10d2000 c068409c 00000000 00000000 c0efa620 c1009400 c0683f40 [ 5.354125] 3ec0: 00000000 c10d3efc c10d2000 c06819c8 c2857100 c1039e6c c1604e38 54192eb7 [ 5.362365] 3ee0: c1009400 c124e810 c10d2000 c124e854 c2857100 c0683198 c1009430 c124e810 [ 5.370605] 3f00: 00010101 54192eb7 c2857110 c0f02018 c1009400 c0160458 c2857110 c10bdf80 [ 5.378845] 3f20: c1009400 c100f000 00000000 c0155660 c10d2000 c1009400 c1009418 c10bdf80 [ 5.387054] 3f40: c1009400 c10bdf94 c1009418 c0e03d00 00000088 c10d2000 c1009400 c01559f8 [ 5.395294] 3f60: 00000000 c10b3c40 c10b3dc0 00000000 c10d2000 c01559b0 c10bdf80 c10c5e9c [ 5.403533] 3f80: c10b3c64 c015b738 00000001 c10b3dc0 c015b608 00000000 00000000 00000000 [ 5.411773] 3fa0: 00000000 00000000 00000000 c0100168 00000000 00000000 00000000 00000000 [ 5.420013] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 5.428253] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000 [ 5.436492] [<c07c23f0>] (omap_hsmmc_set_bus_width) from [<c07c377c>] (omap_hsmmc_set_ios+0x44/0x23c) [ 5.445770] [<c07c377c>] (omap_hsmmc_set_ios) from [<c07a4ccc>] (mmc_power_up.part.0+0x3c/0x154) [ 5.454620] [<c07a4ccc>] (mmc_power_up.part.0) from [<c07a5f9c>] (mmc_start_host+0x88/0x9c) [ 5.463043] [<c07a5f9c>] (mmc_start_host) from [<c07a7138>] (mmc_add_host+0x58/0x84) [ 5.470855] [<c07a7138>] (mmc_add_host) from [<c07c4034>] (omap_hsmmc_probe+0x548/0x9cc) [ 5.479003] [<c07c4034>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 5.487518] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 5.495819] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 5.504058] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 5.512664] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 5.522033] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 5.531677] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 5.540344] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 5.548492] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 5.555938] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 5.563201] Exception stack(0xc10d3fb0 to 0xc10d3ff8) [ 5.568298] 3fa0: 00000000 00000000 00000000 00000000 [ 5.576538] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 5.584777] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 5.591430] Code: e7f001f2 c0ba858c e590c004 e5901024 (e591302c) [ 5.597595] ---[ end trace c76b7828d6f2f578 ]--- [ 5.602264] In-band Error seen by MPU at address 0 [ 5.607177] ------------[ cut here ]------------ [ 5.611846] WARNING: CPU: 0 PID: 7 at drivers/bus/omap_l3_smx.c:152 omap3_l3_app_irq+0x10c/0x130 [ 5.620666] Modules linked in: [ 5.623748] CPU: 0 PID: 7 Comm: kworker/u2:0 Tainted: G D W 5.10.0 #9 [ 5.631378] Hardware name: Nokia RX-51 board [ 5.635681] Workqueue: events_unbound async_run_entry_fn [ 5.641052] [<c0110924>] (unwind_backtrace) from [<c010b1f4>] (show_stack+0x10/0x14) [ 5.648864] [<c010b1f4>] (show_stack) from [<c09c8478>] (dump_stack+0xc0/0xdc) [ 5.656158] [<c09c8478>] (dump_stack) from [<c0138564>] (__warn+0xd4/0x140) [ 5.663177] [<c0138564>] (__warn) from [<c09c3f3c>] (warn_slowpath_fmt+0x60/0xbc) [ 5.670715] [<c09c3f3c>] (warn_slowpath_fmt) from [<c05a444c>] (omap3_l3_app_irq+0x10c/0x130) [ 5.679321] [<c05a444c>] (omap3_l3_app_irq) from [<c019d3f8>] (__handle_irq_event_percpu+0x58/0x208) [ 5.688507] [<c019d3f8>] (__handle_irq_event_percpu) from [<c019d694>] (handle_irq_event+0x68/0xd4) [ 5.697631] [<c019d694>] (handle_irq_event) from [<c01a1c4c>] (handle_level_irq+0xc4/0x1c8) [ 5.706024] [<c01a1c4c>] (handle_level_irq) from [<c019cc90>] (__handle_domain_irq+0x84/0xfc) [ 5.714630] [<c019cc90>] (__handle_domain_irq) from [<c0100b6c>] (__irq_svc+0x6c/0x90) [ 5.722564] Exception stack(0xc10d3cd8 to 0xc10d3d20) [ 5.727661] 3cc0: 00000000 00000000 [ 5.735900] 3ce0: c10bfa80 00000002 c10c6b40 00000000 0000000b ffffe000 c0e09fec 00000000 [ 5.744140] 3d00: bf000000 c10d3d58 08f0d180 c10d3d28 00000000 c01e342c 00000113 ffffffff [ 5.752380] [<c0100b6c>] (__irq_svc) from [<c01e342c>] (acct_collect+0x1b0/0x1cc) [ 5.759918] [<c01e342c>] (acct_collect) from [<c013d79c>] (do_exit+0x174/0xa40) [ 5.767303] [<c013d79c>] (do_exit) from [<c010b510>] (die+0x318/0x354) [ 5.773864] [<c010b510>] (die) from [<c01154e4>] (do_DataAbort+0xa4/0xbc) [ 5.780700] [<c01154e4>] (do_DataAbort) from [<c0100ac0>] (__dabt_svc+0x60/0xa0) [ 5.788146] Exception stack(0xc10d3db0 to 0xc10d3df8) [ 5.793243] 3da0: c17ef380 fa0b4000 54192eb7 00000001 [ 5.801483] 3dc0: c17ef000 c17ef258 c17ef380 00000000 c124e810 00000000 c1248f40 00000066 [ 5.809692] 3de0: c17ef000 c10d3e00 c07c377c c07c23f0 20000013 ffffffff [ 5.816375] [<c0100ac0>] (__dabt_svc) from [<c07c23f0>] (omap_hsmmc_set_bus_width+0x8/0x78) [ 5.824798] [<c07c23f0>] (omap_hsmmc_set_bus_width) from [<c07c377c>] (omap_hsmmc_set_ios+0x44/0x23c) [ 5.834075] [<c07c377c>] (omap_hsmmc_set_ios) from [<c07a4ccc>] (mmc_power_up.part.0+0x3c/0x154) [ 5.842926] [<c07a4ccc>] (mmc_power_up.part.0) from [<c07a5f9c>] (mmc_start_host+0x88/0x9c) [ 5.851348] [<c07a5f9c>] (mmc_start_host) from [<c07a7138>] (mmc_add_host+0x58/0x84) [ 5.859130] [<c07a7138>] (mmc_add_host) from [<c07c4034>] (omap_hsmmc_probe+0x548/0x9cc) [ 5.867279] [<c07c4034>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 5.875793] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 5.884124] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 5.892364] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 5.900939] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 5.910308] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 5.919952] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 5.928619] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 5.936767] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 5.944213] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 5.951477] Exception stack(0xc10d3fb0 to 0xc10d3ff8) [ 5.956573] 3fa0: 00000000 00000000 00000000 00000000 [ 5.964813] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 5.973052] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 5.979705] ---[ end trace c76b7828d6f2f579 ]--- [ 5.990539] 8<--- cut here --- [ 5.993682] Unhandled fault: external abort on non-linefetch (0x1028) at 0xfa09c02c [ 6.001373] pgd = (ptrval) [ 6.004119] [fa09c02c] *pgd=48011452(bad) [ 6.008178] Internal error: : 1028 [#2] SMP ARM [ 6.012756] Modules linked in: [ 6.015838] CPU: 0 PID: 50 Comm: kworker/u2:1 Tainted: G D W 5.10.0 #9 [ 6.023559] Hardware name: Nokia RX-51 board [ 6.027893] Workqueue: events_unbound async_run_entry_fn [ 6.033264] PC is at omap_hsmmc_set_bus_width+0x8/0x78 [ 6.038452] LR is at omap_hsmmc_set_ios+0x44/0x23c [ 6.043273] pc : [<c07c23f0>] lr : [<c07c377c>] psr: 60000113 [ 6.049560] sp : c160de00 ip : c28a7800 fp : 00000063 [ 6.054840] r10: c1248400 r9 : 00000000 r8 : c124c410 [ 6.060089] r7 : 00000000 r6 : c28a7b80 r5 : c28a7a58 r4 : c28a7800 [ 6.066650] r3 : 00000001 r2 : 24f1df74 r1 : fa09c000 r0 : c28a7b80 [ 6.073211] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 6.080413] Control: 10c5387d Table: 80004019 DAC: 00000051 [ 6.086181] Process kworker/u2:1 (pid: 50, stack limit = 0x(ptrval)) [ 6.092590] Stack: (0xc160de00 to 0xc160e000) [ 6.096984] de00: c28a7800 00000013 00000001 00000000 c124c410 c07a4ccc c28a7800 00000000 [ 6.105224] de20: c28a7bb0 00000000 c124c410 c07a5f9c c28a7800 c07a7138 c28a7800 00000000 [ 6.113464] de40: c28a7bb0 c07c4034 00000000 c28a1980 c28a7b80 24f1df74 00000000 00000000 [ 6.121673] de60: c124c410 c0edb05c c0f38c2c 00000000 c0edb05c 00000020 c1009400 c0685ba4 [ 6.129913] de80: c124c410 c0f38c24 00000000 c0f38c2c 00000000 c06839b4 00000000 c124c410 [ 6.138153] dea0: c0edb05c c160c000 c068409c 00000000 00000000 c0efa620 c1009400 c0683f40 [ 6.146392] dec0: 00000000 c160defc c160c000 c06819c8 c2857040 c1039e6c c1604e38 24f1df74 [ 6.154632] dee0: c1009400 c124c410 c160c000 c124c454 c2857040 c0683198 c1009400 c124c410 [ 6.162872] df00: 00010101 24f1df74 c2857050 c0f02018 c1009400 c0160458 c2857050 c1604d00 [ 6.171112] df20: c1009400 c100f000 00000000 c0155660 c160c000 c1009400 c1009418 c1604d00 [ 6.179321] df40: c1009400 c1604d14 c1009418 c0e03d00 00000088 c160c000 c1009400 c01559f8 [ 6.187561] df60: 00000000 c16029c0 c1602a00 00000000 c160c000 c01559b0 c1604d00 c10d3ebc [ 6.195800] df80: c16029e4 c015b738 00000001 c1602a00 c015b608 00000000 00000000 00000000 [ 6.204040] dfa0: 00000000 00000000 00000000 c0100168 00000000 00000000 00000000 00000000 [ 6.212280] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 6.220520] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000 [ 6.228759] [<c07c23f0>] (omap_hsmmc_set_bus_width) from [<c07c377c>] (omap_hsmmc_set_ios+0x44/0x23c) [ 6.238067] [<c07c377c>] (omap_hsmmc_set_ios) from [<c07a4ccc>] (mmc_power_up.part.0+0x3c/0x154) [ 6.246887] [<c07a4ccc>] (mmc_power_up.part.0) from [<c07a5f9c>] (mmc_start_host+0x88/0x9c) [ 6.255310] [<c07a5f9c>] (mmc_start_host) from [<c07a7138>] (mmc_add_host+0x58/0x84) [ 6.263122] [<c07a7138>] (mmc_add_host) from [<c07c4034>] (omap_hsmmc_probe+0x548/0x9cc) [ 6.271270] [<c07c4034>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 6.279785] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 6.288116] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 6.296356] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 6.304931] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 6.314300] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 6.323944] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 6.332641] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 6.340789] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 6.348236] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 6.355499] Exception stack(0xc160dfb0 to 0xc160dff8) [ 6.360595] dfa0: 00000000 00000000 00000000 00000000 [ 6.368835] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 6.377044] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 6.383728] Code: e7f001f2 c0ba858c e590c004 e5901024 (e591302c) [ 6.389862] ---[ end trace c76b7828d6f2f57a ]--- [ 6.394561] In-band Error seen by MPU at address 0 [ 6.399475] ------------[ cut here ]------------ [ 6.404113] WARNING: CPU: 0 PID: 50 at drivers/bus/omap_l3_smx.c:152 omap3_l3_app_irq+0x10c/0x130 [ 6.413055] Modules linked in: [ 6.416137] CPU: 0 PID: 50 Comm: kworker/u2:1 Tainted: G D W 5.10.0 #9 [ 6.423828] Hardware name: Nokia RX-51 board [ 6.428161] Workqueue: events_unbound async_run_entry_fn [ 6.433532] [<c0110924>] (unwind_backtrace) from [<c010b1f4>] (show_stack+0x10/0x14) [ 6.441345] [<c010b1f4>] (show_stack) from [<c09c8478>] (dump_stack+0xc0/0xdc) [ 6.448638] [<c09c8478>] (dump_stack) from [<c0138564>] (__warn+0xd4/0x140) [ 6.455657] [<c0138564>] (__warn) from [<c09c3f3c>] (warn_slowpath_fmt+0x60/0xbc) [ 6.463195] [<c09c3f3c>] (warn_slowpath_fmt) from [<c05a444c>] (omap3_l3_app_irq+0x10c/0x130) [ 6.471801] [<c05a444c>] (omap3_l3_app_irq) from [<c019d3f8>] (__handle_irq_event_percpu+0x58/0x208) [ 6.480987] [<c019d3f8>] (__handle_irq_event_percpu) from [<c019d694>] (handle_irq_event+0x68/0xd4) [ 6.490112] [<c019d694>] (handle_irq_event) from [<c01a1c4c>] (handle_level_irq+0xc4/0x1c8) [ 6.498504] [<c01a1c4c>] (handle_level_irq) from [<c019cc90>] (__handle_domain_irq+0x84/0xfc) [ 6.507110] [<c019cc90>] (__handle_domain_irq) from [<c0100b6c>] (__irq_svc+0x6c/0x90) [ 6.515075] Exception stack(0xc160dcd8 to 0xc160dd20) [ 6.520141] dcc0: 00000000 00000000 [ 6.528381] dce0: c15bb000 00000002 c12ad6c0 00000000 0000000b ffffe000 c0e09fec 00000000 [ 6.536621] dd00: bf000000 c160dd58 09896800 c160dd28 00000000 c01e342c 00000113 ffffffff [ 6.544860] [<c0100b6c>] (__irq_svc) from [<c01e342c>] (acct_collect+0x1b0/0x1cc) [ 6.552398] [<c01e342c>] (acct_collect) from [<c013d79c>] (do_exit+0x174/0xa40) [ 6.559783] [<c013d79c>] (do_exit) from [<c010b510>] (die+0x318/0x354) [ 6.566345] [<c010b510>] (die) from [<c01154e4>] (do_DataAbort+0xa4/0xbc) [ 6.573181] [<c01154e4>] (do_DataAbort) from [<c0100ac0>] (__dabt_svc+0x60/0xa0) [ 6.580627] Exception stack(0xc160ddb0 to 0xc160ddf8) [ 6.585723] dda0: c28a7b80 fa09c000 24f1df74 00000001 [ 6.593963] ddc0: c28a7800 c28a7a58 c28a7b80 00000000 c124c410 00000000 c1248400 00000063 [ 6.602203] dde0: c28a7800 c160de00 c07c377c c07c23f0 60000113 ffffffff [ 6.608856] [<c0100ac0>] (__dabt_svc) from [<c07c23f0>] (omap_hsmmc_set_bus_width+0x8/0x78) [ 6.617279] [<c07c23f0>] (omap_hsmmc_set_bus_width) from [<c07c377c>] (omap_hsmmc_set_ios+0x44/0x23c) [ 6.626556] [<c07c377c>] (omap_hsmmc_set_ios) from [<c07a4ccc>] (mmc_power_up.part.0+0x3c/0x154) [ 6.635406] [<c07a4ccc>] (mmc_power_up.part.0) from [<c07a5f9c>] (mmc_start_host+0x88/0x9c) [ 6.643829] [<c07a5f9c>] (mmc_start_host) from [<c07a7138>] (mmc_add_host+0x58/0x84) [ 6.651611] [<c07a7138>] (mmc_add_host) from [<c07c4034>] (omap_hsmmc_probe+0x548/0x9cc) [ 6.659759] [<c07c4034>] (omap_hsmmc_probe) from [<c0685ba4>] (platform_drv_probe+0x48/0x98) [ 6.668273] [<c0685ba4>] (platform_drv_probe) from [<c06839b4>] (really_probe+0x208/0x4a0) [ 6.676605] [<c06839b4>] (really_probe) from [<c0683f40>] (driver_probe_device+0x58/0xb4) [ 6.684844] [<c0683f40>] (driver_probe_device) from [<c06819c8>] (bus_for_each_drv+0x84/0xc8) [ 6.693420] [<c06819c8>] (bus_for_each_drv) from [<c0683198>] (__device_attach_async_helper+0xa0/0xd0) [ 6.702789] [<c0683198>] (__device_attach_async_helper) from [<c0160458>] (async_run_entry_fn+0x3c/0x100) [ 6.712432] [<c0160458>] (async_run_entry_fn) from [<c0155660>] (process_one_work+0x1ec/0x53c) [ 6.721099] [<c0155660>] (process_one_work) from [<c01559f8>] (worker_thread+0x48/0x594) [ 6.729248] [<c01559f8>] (worker_thread) from [<c015b738>] (kthread+0x130/0x144) [ 6.736694] [<c015b738>] (kthread) from [<c0100168>] (ret_from_fork+0x14/0x2c) [ 6.743957] Exception stack(0xc160dfb0 to 0xc160dff8) [ 6.749053] dfa0: 00000000 00000000 00000000 00000000 [ 6.757293] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 6.765533] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 6.772186] ---[ end trace c76b7828d6f2f57b ]---
Hi!
You may want to cc: [email protected] and [email protected], since this is kernel issue.
Quoted 8 lines
Quoted 1 line
I am back with a bit more news, but no solution.
It looks like commit
fb2c599f056640d289b2147fbe6d9eaee689f1b2("ARM:
omap3: enable off mode automatically") is causing these failures on the Nokia N900. Reverting this commit on top of v5.10 (with21b2cec61c04bd175f0860d9411a472d5a0e7ba1also reverted), makes the problem go away (I tried 10 times).
Quoted 7 lines
The same holds for 5.15.y, just reverting
fb2c599f056640d289b2147fbe6d9eaee689f1b2makes the system "stable" (at least the problem reported in the first email is gone).Perhaps the "
pwrst->next_state" change is causing these problems, since I can enter OFF mode just fine with init=/bin/sh and a custom script?
fb2c599f056640d28 has some hints in it. Perhaps Andreas can mention what boards he tested it on?
And rather than reverting it, we may want to delete ti,twl4030-idle* hints from n900 device tree for now?
Best regards,
Pavel
Signature · 1 line
http://www.livejournal.com/~pavelmachek
Hi,
Quoted 17 lines · On Wed, 8 Dec 2021 21:57:00 +0100 Pavel Machek <[email protected]> wrote:
Hi!
You may want to cc: [email protected] and [email protected], since this is kernel issue.
Quoted 12 lines
Quoted 1 line
I am back with a bit more news, but no solution.
It looks like commit
fb2c599f056640d289b2147fbe6d9eaee689f1b2("ARM:
omap3: enable off mode automatically") is causing these failures on the Nokia N900. Reverting this commit on top of v5.10 (with21b2cec61c04bd175f0860d9411a472d5a0e7ba1also reverted), makes the problem go away (I tried 10 times).The same holds for 5.15.y, just reverting
fb2c599f056640d289b2147fbe6d9eaee689f1b2makes the system "stable" (at least the problem reported in the first email is gone).
Quoted 7 lines
Quoted 3 lines
Perhaps the "
pwrst->next_state" change is causing these problems, since I can enter OFF mode just fine with init=/bin/sh and a custom script?
fb2c599f056640d28has some hints in it. Perhaps Andreas can mention what boards he tested it on?
well, I tested it on gta04a4 and gta04a5 (both dm3730). There is one 1Ghz issue at least on the a5, so I disabled that opp.
Quoted 3 lines
And rather than reverting it, we may want to delete ti,twl4030-idle* hints from n900 device tree for now?
well, it has side effects on power consumption, power consumption should be worse without ti,twl4030-idle-*. So the real solution would be to understand what is causing the trouble here.
Regards,
Andreas
Hi,
Quoted 32 lines · Andreas Kemnade
Hi,
Quoted 17 lines · On Wed, 8 Dec 2021 21:57:00 +0100 Pavel Machek <[email protected]> wrote:
Hi!
You may want to cc: [email protected] and [email protected], since this is kernel issue.
Quoted 12 lines
Quoted 1 line
I am back with a bit more news, but no solution.
It looks like commit
fb2c599f056640d289b2147fbe6d9eaee689f1b2("ARM:
omap3: enable off mode automatically") is causing these failures on the Nokia N900. Reverting this commit on top of v5.10 (with21b2cec61c04bd175f0860d9411a472d5a0e7ba1also reverted), makes the problem go away (I tried 10 times).The same holds for 5.15.y, just reverting
fb2c599f056640d289b2147fbe6d9eaee689f1b2makes the system "stable" (at least the problem reported in the first email is gone).Quoted 7 lines
Quoted 3 lines
Perhaps the "
pwrst->next_state" change is causing these problems, since I can enter OFF mode just fine with init=/bin/sh and a custom script?
fb2c599f056640d28has some hints in it. Perhaps Andreas can mention what boards he tested it on?well, I tested it on gta04a4 and gta04a5 (both dm3730). There is one 1Ghz issue at least on the a5, so I disabled that opp.
Quoted 6 lines
Quoted 3 lines
And rather than reverting it, we may want to delete ti,twl4030-idle* hints from n900 device tree for now?
well, it has side effects on power consumption, power consumption should be worse without ti,twl4030-idle-*. So the real solution would be to understand what is causing the trouble here.
What I have seen is that if off mode is enabled from userspace (debugfs), it does not cause a problem (or I don't hit the problem at least). That said, my off mode tests are pretty minimal with init=/bin/sh, and I haven't gotten a fully booted (with lots of modules loaded, gui and daemons) system to enter off mode yet. But as noted before, sometimes this problem would occur not on boot, but far into the boot process (~4 minutes after power up), and that problem is also gone now with the revert, so there's something going on...
I can't really comment on what would be causing the trouble specifically (I'm not really familiar with omap pm), just noticed the regression for the N900, where off mode does appear to work, but this commit/method doesn't at least. I'm also not sure if removing the hint from the device tree is the right way to go - I'm happy to see off mode getting some more testing by turning it on by default.
Regards,
Merlijn
Hi,
Quoted 5 lines · Merlijn B.W. Wajer
What I have seen is that if off mode is enabled from userspace (debugfs), it does not cause a problem (or I don't hit the problem at least). That said, my off mode tests are pretty minimal with init=/bin/sh, and I haven't gotten a fully booted (with lots of modules loaded, gui and daemons) system to enter off mode yet.
I also started seeing crashes with fb2c599f0566 on N900. It's been several months since I last tested, but I remember I was able to trigger the crashes reliably with MMC access from the minimal shell enviroment. I see the MMC is also visible in your crash logs. My test case was something like "sleep 30 ; blkid ; sleep 30".
A.
Hi,
Quoted 9 lines · Aaro Koskinen
Hi,
Quoted 5 lines · Merlijn B.W. Wajer
What I have seen is that if off mode is enabled from userspace (debugfs), it does not cause a problem (or I don't hit the problem at least). That said, my off mode tests are pretty minimal with init=/bin/sh, and I haven't gotten a fully booted (with lots of modules loaded, gui and daemons) system to enter off mode yet.
Quoted 5 lines
I also started seeing crashes with
fb2c599f0566on N900. It's been several months since I last tested, but I remember I was able to trigger the crashes reliably with MMC access from the minimal shell enviroment. I see the MMC is also visible in your crash logs. My test case was something like "sleep 30 ; blkid ; sleep 30".
I can confirm now that enable off mode through userspace later on (when the system actually can idle) can indeed still cause trouble. That means that likely fb2c599f0566 is not at fault (since I have it reverted currently), and we have other problems related to idle.
This is the output of scripts/decode_stacktrace.sh:
Quoted 74 lines
# [ 194.349151] Internal error: Oops - undefined instruction: 0 [#1] PREEMPT SMP ARM
[ 194.356750] Modules linked in: cbc aes_arm_bs crypto_simd cryptd aes_arm aes_generic ccm bluetooth ecdh_generic ecc libaes zram zsmalloc evdev wl1251_spi wl1251 mac80211 libarc4
omap3_rom_rngisp1704_chargerrng_coreir_rx51rc_coresnd_soc_rx51udc_coreleds_gpiosha256_genericlibsha256sha256_armpvrsrvkm_omap3_sgx530_121gpio_keyspwm_omap_dmtimerdisplay_connectorcpufreq_dtsnd_soc_omap_mcbspsnd_soc_ti_sdmacfg80211snd_soc_tpa6130a2snd_soc_coreomap_shamsnd_pcm_dmaenginecrypto_enginesnd_pcmsnd_timersndbq2415x_chargerpanel_sony_acx565akmleds_lp5523soundcorebq27xxx_battery_i2cleds_lp55xx_commonbq27xxx_batteryled_classomap_mailboxrtc_twlohci_platformohci_hcdehci_hcdtwl4030_pwrbuttonpwm_twl_ledst_accel_i2ctwl4030_keypadst_sensors_i2cpwm_twlst_accelmatrix_keymapusbcoretwl4030_madctwl4030_chargerst_sensorsindustrialio_triggered_bufferkfifo_bufomapdrm industrialiousb_commonomap_ssihsidrm_kms_helpercfbfillrect syscopyarea cfbimgblt sysfillrect[ 194.358612] sysimgblt fb_sys_fops cfbcopyarea drm drm_panel_orientation_quirks cec [ 194.453826] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.2-00597-g68be8fac7cbd #48 [ 194.461669] Hardware name: Nokia RX-51 board [ 194.466003] PC is at omap3_l3_app_irq (drivers/bus/omap_l3_smx.c:177 (discriminator 1)) [ 194.470733] LR is at __handle_irq_event_percpu (./include/linux/atomic/atomic-instrumented.h:28) [ 194.476226] pc : lr : psr: 20070193 [ 194.482574] sp : c0f01e10 ip : ffffe000 fp : c0f00000 [ 194.487854] r10: c0ff8160 r9 : c0ff8180 r8 : 0000001a [ 194.493164] r7 : 00000000 r6 : 00400000 r5 : 00000000 r4 : f8000000 [ 194.499755] r3 : 00400000 r2 : 00010001 r1 : 00400000 r0 : 00000000 [ 194.506347] Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none [ 194.513671] Control: 10c5387d Table: 86150019 DAC: 00000051 [ 194.519470] Register r0 information: NULL pointer [ 194.524261] Register r1 information: non-paged memory [ 194.529418] Register r2 information: non-paged memory [ 194.534545] Register r3 information: non-paged memory [ 194.539672] Register r4 information: 0-page vmalloc region starting at 0xf8000000 allocated at iotable_init (arch/arm/mm/mmu.c:982) [ 194.550354] Register r5 information: NULL pointer [ 194.555145] Register r6 information: non-paged memory [ 194.560272] Register r7 information: NULL pointer [ 194.565063] Register r8 information: non-paged memory [ 194.570190] Register r9 information: non-slab/vmalloc memory [ 194.575958] Register r10 information: non-slab/vmalloc memory [ 194.581787] Register r11 information: non-slab/vmalloc memory [ 194.587615] Register r12 information: non-paged memory [ 194.592834] Process swapper/0 (pid: 0, stack limit = 0x4aa785a2) [ 194.598937] Stack: (0xc0f01e10 to 0xc0f02000) [ 194.603363] 1e00: c0644a40 c1f9fa00 c0f00000 c1a45500 [ 194.611633] 1e20: c0f00000 00000000 c0f01e70 0000001a c0ff8180 c01a203c c0f00000 c0697fd4 [ 194.619903] 1e40: c183db40 c1832200 c0f00000 c1832200 c0f00000 c0f00000 c183226c 00000000 [ 194.628173] 1e60: c1805a00 0000000a c0f01ed8 c01a22ec 00000000 3d6d10de c0f01ed8 c1832200 [ 194.636474] 1e80: c183226c c0f00000 c0f05638 c01a6b00 c0e6d570 c0e6d570 c0f00000 c0f05638 [ 194.644744] 1ea0: 00000000 c01a18b0 00000000 c0f00000 c1f75440 c07d3478 20070013 ffffffff [ 194.653015] 1ec0: c0f01f0c cfb63a38 c0f00000 00000000 0000002d c0100b3c 00000000 c0f0ba30 [ 194.661285] 1ee0: 00000001 0ecf6000 00000004 c0f0b8b0 00000004 401a616e cfb63a38 0000002d [ 194.669555] 1f00: 00000000 0000002d ffffe000 c0f01f28 c07d33b0 c07d3478 20070013 ffffffff [ 194.677825] 1f20: 00000051 c07d3388 3ef87a80 4016308c 03f44f77 cfb63a38 c0f0b8b0 00000004 [ 194.686096] 1f40: c0f05260 cfb63a38 c0f0b8b0 c0f00000 c0e6da30 c07d3710 c0f00000 00000000 [ 194.694366] 1f60: c0f0520c c01740ec 00000004 c0f00000 01f00000 3d6d10de cfdff0c0 000000e4 [ 194.702636] 1f80: c0f00000 c0e43a60 cfdff0c0 c0f051c0 00000000 c0f0872c c0cc1404 c01744e4 [ 194.710906] 1fa0: c1000068 c0e00ef8 ffffffff ffffffff 00000000 c0e00588 00000000 c0f00000 [ 194.719177] 1fc0: 00000000 c0e43a60 3d681fdc 00000000 00000000 c0e00330 00000051 10c0387d [ 194.727447] 1fe0: 000007a3 8248ab30 411fc083 10c5387d 00000000 00000000 00000000 00000000 [ 194.735717] (omap3_l3_app_irq) from __handle_irq_event_percpu (./include/linux/atomic/atomic-instrumented.h:28) [ 194.745025] (__handle_irq_event_percpu) from handle_irq_event (kernel/irq/handle.c:198) [ 194.754211] (handle_irq_event) from handle_level_irq (./include/linux/irq.h:346) [ 194.762695] (handle_level_irq) from handle_domain_irq (kernel/irq/irqdesc.c:694) [ 194.771179] (handle_domain_irq) from __irq_svc (arch/arm/kernel/entry-armv.S:208) [ 194.779022] Exception stack(0xc0f01ed8 to 0xc0f01f20) [ 194.784149] 1ec0: 00000000 c0f0ba30 [ 194.792419] 1ee0: 00000001 0ecf6000 00000004 c0f0b8b0 00000004 401a616e cfb63a38 0000002d [ 194.800720] 1f00: 00000000 0000002d ffffe000 c0f01f28 c07d33b0 c07d3478 20070013 ffffffff [ 194.808959] (__irq_svc) from cpuidle_enter_state (drivers/cpuidle/cpuidle.c:259) [ 194.817199] (cpuidle_enter_state) from cpuidle_enter (drivers/cpuidle/cpuidle.c:351) [ 194.825592] (cpuidle_enter) from do_idle (kernel/sched/idle.c:158) [ 194.833129] (do_idle) from cpu_startup_entry (kernel/sched/idle.c:402 (discriminator 1)) [ 194.840820] (cpu_startup_entry) from start_kernel (init/main.c:1144) [ 194.849151] Code: e0000007 e3c1103c e1913000 0affffe1 (e7f001f2) All code ======== 0: 07 (bad) 1: 00 00 add %al,(%rax) 3: e0 3c loopne 0x41 5: 10 c1 adc %al,%cl 7: e3 00 jrcxz 0x9 9: 30 91 e1 e1 ff ff xor %dl,-0x1e1f(%rcx) f: 0a f2 or %dl,%dh
Quoted 8 lines
11:* 01 f0 add %esi,%eax <-- trapping instruction
13: e7 .byte0xe7Code starting with the faulting instruction
===========================================
0: f2 01 f0 repnz add %esi,%eax
3: e7 .byte0xe7[ 194.855346] ---[ end trace f0dbb8dd5ceba01a ]---
Regards,
Merlijn
Hi Aaro,
Quoted 9 lines · Aaro Koskinen
Hi,
Quoted 5 lines · Merlijn B.W. Wajer
What I have seen is that if off mode is enabled from userspace (debugfs), it does not cause a problem (or I don't hit the problem at least). That said, my off mode tests are pretty minimal with init=/bin/sh, and I haven't gotten a fully booted (with lots of modules loaded, gui and daemons) system to enter off mode yet.
Quoted 5 lines
I also started seeing crashes with
fb2c599f0566on N900. It's been several months since I last tested, but I remember I was able to trigger the crashes reliably with MMC access from the minimal shell enviroment. I see the MMC is also visible in your crash logs. My test case was something like "sleep 30 ; blkid ; sleep 30".
Thanks for this info -- I can confirm, I see the same when I do this:
1. boot to init=/bin/bash
2. run:
Quoted 9 lines
modprobe panel-sony-acx565akm
mount -t proc none /proc
mount -t sysfs none /sys
mount -t debugfs none/sys/kernel/debug
mount -o rw,remount /echo 1 >
/sys/kernel/debug/pm_debug/enable_off_modeecho 0 >/sys/class/backlight/acx565akm/brightness
3. run:
Quoted 1 line
sleep 30 ; blkid
So we have a pretty reproducible test case here I think.
Regards,
Merlijn
Quoted 31 lines · Merlijn Wajer
Hi Aaro,
Quoted 9 lines · Aaro Koskinen
Hi,
Quoted 5 lines · Merlijn B.W. Wajer
What I have seen is that if off mode is enabled from userspace (debugfs), it does not cause a problem (or I don't hit the problem at least). That said, my off mode tests are pretty minimal with init=/bin/sh, and I haven't gotten a fully booted (with lots of modules loaded, gui and daemons) system to enter off mode yet.
Quoted 5 lines
I also started seeing crashes with
fb2c599f0566on N900. It's been several months since I last tested, but I remember I was able to trigger the crashes reliably with MMC access from the minimal shell enviroment. I see the MMC is also visible in your crash logs. My test case was something like "sleep 30 ; blkid ; sleep 30".Thanks for this info -- I can confirm, I see the same when I do this:
1. boot to init=/bin/bash
2. run:
Quoted 7 lines
modprobe panel-sony-acx565akm
mount -t proc none /proc
mount -t sysfs none /sys
mount -t debugfs none/sys/kernel/debug
mount -o rw,remount /
Quoted 8 lines
Quoted 2 lines
echo 1 >
/sys/kernel/debug/pm_debug/enable_off_modeecho 0 >/sys/class/backlight/acx565akm/brightness3. run:
Quoted 1 line
sleep 30 ; blkid
So we have a pretty reproducible test case here I think.
Just to see if this relates to the omap_hsmmc.c driver, maybe try this with v5.16-rc1 also with a patch that changes omap3.dtsi ti,omap3-hsmmc compatibles with ti,omap3-sdhci compatibles.
Regards,
Tony
Hi Tony,
Quoted 10 lines · Tony Lindgren
Quoted 6 lines
3. run:
Quoted 1 line
sleep 30 ; blkid
So we have a pretty reproducible test case here I think.
Just to see if this relates to the
omap_hsmmc.cdriver, maybe try this with v5.16-rc1 also with a patch that changesomap3.dtsiti,omap3-hsmmc compatibles with ti,omap3-sdhci compatibles.
I tried this just now on v5.16-rc4 with the following change [1] (and a few reverts for off mode) and I don't think it helped get rid of the error, I see the following. [2] Maybe it does help narrow the specific error down if it happens with both hsmmc and sdhci?
Regards,
Merlijn
[1]
Quoted 23 lines
arch/arm/boot/dts/omap3.dtsi +2
-2@@ -510,7 +510,7 @@ hdqw1w: 1w@480b2000 { }; mmc1: mmc@4809c000 { - compatible = "ti,omap3-hsmmc"; + compatible = "ti,omap3-sdhci"; reg = <0x4809c000 0x200>; interrupts = <83>; ti,hwmods = "mmc1";@@ -521,7 +521,7 @@ mmc1: mmc@4809c000 { }; mmc2: mmc@480b4000 { - compatible = "ti,omap3-hsmmc"; + compatible = "ti,omap3-sdhci"; reg = <0x480b4000 0x200>; interrupts = <86>; ti,hwmods = "mmc2";
arch/arm/boot/dts/omap3.dtsi mmc2: mmc@480b4000 { · line 536
Quoted 8 lines
arch/arm/boot/dts/omap3.dtsi +1
-1@@ -530,7 +530,7 @@ mmc2: mmc@480b4000 { }; mmc3: mmc@480ad000 { - compatible = "ti,omap3-hsmmc"; + compatible = "ti,omap3-sdhci"; reg = <0x480ad000 0x200>; interrupts = <94>; ti,hwmods = "mmc3";
[2]
Quoted 77 lines
[ 72.589294] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.16.0-rc4-00003-g67523c186596-dirty #4 [ 72.597869] Hardware name: Nokia RX-51 board INSTALL /home/merlijn/maemo-leste/droid4-linux/modout/lib/modules/5.16.0-rc4-00003-g67523c186596-dirty/kernel/sound/soc/codecs/snd-soc-cpcap.ko [ 72.602172] PC is at omap3_l3_app_irq (drivers/bus/omap_l3_smx.c:177 (discriminator 1)) [ 72.606842] LR is at __handle_irq_event_percpu (./include/linux/atomic/atomic-instrumented.h:28) [ 72.612304] pc : lr : psr: 20010193 [ 72.618621] sp : c0f01e38 ip : c0f05638 fp : c1833200 [ 72.623901] r10: c0ff9e20 r9 : c0ff9e40 r8 : 0000001a [ 72.629150] r7 : 00000000 r6 : 00400000 r5 : 00000000 r4 : f8000000 [ 72.635711] r3 : 00400000 r2 : 00000008 r1 : 00400000 r0 : 00000000 [ 72.642303] Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment none [ 72.649566] Control: 10c5387d Table: 81b5c019 DAC: 00000051 [ 72.655334] Register r0 information: NULL pointer [ 72.660095] Register r1 information: non-paged memory [ 72.665191] Register r2 information: non-paged memory [ 72.670288] Register r3 information: non-paged memory [ 72.675384] Register r4 information: 0-page vmalloc region starting at 0xf8000000 allocated at iotable_init (arch/arm/mm/mmu.c:982) [ 72.686004] Register r5 information: NULL pointer [ 72.690765] Register r6 information: non-paged memory [ 72.695861] Register r7 information: NULL pointer [ 72.700592] Register r8 information: non-paged memory [ 72.705688] Register r9 information: non-slab/vmalloc memory [ 72.711395] Register r10 information: non-slab/vmalloc memory [ 72.717193] Register r11 information: slab kmalloc-256 start c1833200 pointer offset 0 size 256 [ 72.726013] Register r12 information: non-slab/vmalloc memory [ 72.731781] Process swapper/0 (pid: 0, stack limit = 0x(ptrval)) [ 72.737854] Stack: (0xc0f01e38 to 0xc0f02000) [ 72.742248] 1e20: c065bcf4 c1fbd980 [ 72.750488] 1e40: 0000000d c1a42040 0000000a 00000000 c0f01e98 0000001a c0ff9e40 c01a2274 [ 72.758728] 1e60: c388d000 00000010 00006000 c0f00000 ff38801d c1833200 0000000a c183326c [ 72.766967] 1e80: c0f05638 00000000 c0f00000 00000000 00000010 c01a24dc 00000000 1f933612 [ 72.775177] 1ea0: c1833200 0000000a c0f01ee0 c01a6d84 c1805a00 c01a1b3c c0e75430 ffffe000 [ 72.783416] 1ec0: c0f01ee0 c0a37afc c07eb424 20010013 ffffffff c0f01f14 cfb6b8f8 c0100b34 [ 72.791656] 1ee0: 00000000 c0f0ba68 0ecf6000 cfb6c740 00000004 c0f0b8e8 00000004 e5c57d3e [ 72.799896] 1f00: cfb6b8f8 00000010 00000000 00000010 00055522 c0f01f30 c07eb35c c07eb424 [ 72.808135] 1f20: 20010013 ffffffff 00000051 c07eb318 c1838900 e5c8487f e4e9bc00 cfb6b8f8 [ 72.816375] 1f40: c0f0b8e8 00000004 c0f05264 c0e758f0 c0f05638 cfb6b8f8 c0f0b8e8 c07eb644 [ 72.824615] 1f60: c0f00000 00000000 c0f05210 c0173fa8 00000004 c0f00000 01e4ba60 1f933612 [ 72.832855] 1f80: ffffffff 000000e4 c0e4ba60 cfdffb40 ffffffff c0f051c0 00000000 10c5387d [ 72.841094] 1fa0: 00000000 c0174384 c1002068 c0e00ef8 ffffffff ffffffff 00000000 c0e00588 [ 72.849334] 1fc0: 00000000 c0e4ba60 1f962612 00000000 00000000 c0e00330 00000051 10c0387d [ 72.857574] 1fe0: 000007a3 8248c360 411fc083 10c5387d 00000000 00000000 00000000 00000000 [ 72.865814] (omap3_l3_app_irq) from __handle_irq_event_percpu (./include/linux/atomic/atomic-instrumented.h:28) [ 72.875030] (__handle_irq_event_percpu) from handle_irq_event (kernel/irq/handle.c:200) [ 72.884155] (handle_irq_event) from handle_level_irq (./include/linux/irq.h:346) [ 72.892578] (handle_level_irq) from generic_handle_domain_irq (kernel/irq/irqdesc.c:647) [ 72.901702] (generic_handle_domain_irq) from generic_handle_arch_irq (./arch/arm/include/asm/thread_info.h:104) [ 72.911468] (generic_handle_arch_irq) from __irq_svc (arch/arm/kernel/entry-armv.S:213) [ 72.919799] Exception stack(0xc0f01ee0 to 0xc0f01f28) [ 72.924896] 1ee0: 00000000 c0f0ba68 0ecf6000 cfb6c740 00000004 c0f0b8e8 00000004 e5c57d3e [ 72.933135] 1f00: cfb6b8f8 00000010 00000000 00000010 00055522 c0f01f30 c07eb35c c07eb424 [ 72.941375] 1f20: 20010013 ffffffff [ 72.944885] (__irq_svc) from cpuidle_enter_state (drivers/cpuidle/cpuidle.c:259) [ 72.953063] (cpuidle_enter_state) from cpuidle_enter (drivers/cpuidle/cpuidle.c:351) [ 72.961395] (cpuidle_enter) from do_idle (kernel/sched/idle.c:158) [ 72.968872] (do_idle) from cpu_startup_entry (kernel/sched/idle.c:402 (discriminator 1)) [ 72.976501] (cpu_startup_entry) from start_kernel (init/main.c:1137) [ 72.984802] Code: e0000007 e3c1103c e1913000 0affffe1 (e7f001f2) All code ======== 0: 07 (bad) 1: 00 00 add %al,(%rax) 3: e0 3c loopne 0x41 5: 10 c1 adc %al,%cl 7: e3 00 jrcxz 0x9 9: 30 91 e1 e1 ff ff xor %dl,-0x1e1f(%rcx) f: 0a f2 or %dl,%dh11:* 01 f0 add %esi,%eax <-- trapping instruction
13: e7 .byte0xe7Code starting with the faulting instruction
===========================================
0: f2 01 f0 repnz add %esi,%eax
3: e7 .byte0xe7[ 72.990936] ---[ end trace 231dd61ec2cdee97 ]--- [ 72.995605] Kernel panic - not syncing: Fatal exception in interrupt [ 73.002014] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
Quoted 9 lines · Merlijn Wajer
Quoted 3 lines · Tony Lindgren
Just to see if this relates to the
omap_hsmmc.cdriver, maybe try this with v5.16-rc1 also with a patch that changesomap3.dtsiti,omap3-hsmmc compatibles with ti,omap3-sdhci compatibles.I tried this just now on v5.16-rc4 with the following change [1] (and a few reverts for off mode) and I don't think it helped get rid of the error, I see the following. [2] Maybe it does help narrow the specific error down if it happens with both hsmmc and sdhci?
OK so it's not omap_hsmmc vs sdhci-omap issue. So does setting the thermal driver to status = "disabled" make the issue disappear?
Regards,
Tony
Hi Tony,
Quoted 6 lines · Tony Lindgren
Quoted 5 lines · Merlijn Wajer
Quoted 3 lines · Tony Lindgren
Just to see if this relates to the
omap_hsmmc.cdriver, maybe try this with v5.16-rc1 also with a patch that changesomap3.dtsiti,omap3-hsmmc compatibles with ti,omap3-sdhci compatibles.
Quoted 7 lines
Quoted 4 lines
I tried this just now on v5.16-rc4 with the following change [1] (and a few reverts for off mode) and I don't think it helped get rid of the error, I see the following. [2] Maybe it does help narrow the specific error down if it happens with both hsmmc and sdhci?
OK so it's not
omap_hsmmcvs sdhci-omap issue. So does setting the thermal driver to status = "disabled" make the issue disappear?
In my kernel I have CONFIG_OMAP3_THERMAL disabled currently (per my other email), but my understanding is that you're suggesting that disabling the node all together in the dts can help. Would this be the bandgap node, thermal_sensors or cpu_thermal node(s)?
Thanks,
Regards,
Merlijn
Hi Tony,
Quoted 8 lines · Merlijn Wajer
Quoted 3 lines
OK so it's not
omap_hsmmcvs sdhci-omap issue. So does setting the thermal driver to status = "disabled" make the issue disappear?In my kernel I have
CONFIG_OMAP3_THERMALdisabled currently (per my other email), but my understanding is that you're suggesting that disabling the node all together in the dts can help. Would this be the bandgap node,thermal_sensorsorcpu_thermalnode(s)?
I made this change (I just disabled all three of them, not knowing which one was the right one, will try to narrow that down further if useful):
Quoted 10 lines
arch/arm/boot/dts/omap3-n900.dts +2
-0@@ -28,6 +28,19 @@ &aes2_target { status = "disabled"; }; +&cpu_thermal { + status = "disabled";
arch/arm/boot/dts/omap3-n900.dts &aes2_target { · line 42
Quoted 10 lines
arch/arm/boot/dts/omap3-n900.dts +10
-0+}; + +&bandgap { + status = "disabled"; +}; + +&thermal_zones { + status = "disabled"; +}; +
and I am no longer seeing the oopses or resets when running "sleep 30; blkid" from minimal userspace.
Thanks,
Regards,
Merlijn
Quoted 24 lines · Merlijn Wajer
Hi Tony,
Quoted 8 lines · Merlijn Wajer
Quoted 3 lines
OK so it's not
omap_hsmmcvs sdhci-omap issue. So does setting the thermal driver to status = "disabled" make the issue disappear?In my kernel I have
CONFIG_OMAP3_THERMALdisabled currently (per my other email), but my understanding is that you're suggesting that disabling the node all together in the dts can help. Would this be the bandgap node,thermal_sensorsorcpu_thermalnode(s)?I made this change (I just disabled all three of them, not knowing which one was the right one, will try to narrow that down further if useful):
Quoted 8 lines
arch/arm/boot/dts/omap3-n900.dts +0
-0@@ -28,6 +28,19 @@ &aes2_target { status = "disabled"; };
Quoted 15 lines
arch/arm/boot/dts/omap3-n900.dts &aes2_target { · line 42
Quoted 12 lines
arch/arm/boot/dts/omap3-n900.dts +12
-0+&cpu_thermal { + status = "disabled"; +}; + +&bandgap { + status = "disabled"; +}; + +&thermal_zones { + status = "disabled"; +}; +and I am no longer seeing the oopses or resets when running "sleep 30; blkid" from minimal userspace.
If disabling the cpu_thermal makes things behave, chances are we have wrong thermal values in the dts for n900 and possible all 34xx devices. The 36xx values are behaving AFAIK.
Regards,
Tony
Hi,
Quoted 8 lines · Tony Lindgren
Quoted 2 lines · Merlijn Wajer
and I am no longer seeing the oopses or resets when running "sleep 30; blkid" from minimal userspace.
If disabling the
cpu_thermalmakes things behave, chances are we have wrong thermal values in the dts for n900 and possible all 34xx devices. The 36xx values are behaving AFAIK.
Just disabling the bandgap doesn't seem to be enough. I also tried disabling just the bandgap and thermal_zones, but that is also isn't enough. However, *just* disabling cpu_thermal also isn't enough.
Disabling both cpu_thermal and bandgap is enough to stop the oopses [1]. So it sounds like there might potentially be some problems in cpu_thermal as well then?
Regards,
Merlijn
[1]
Quoted 24 lines
arch/arm/boot/dts/omap3-n900.dts +13
-0@@ -28,6 +28,19 @@ &aes2_target { status = "disabled"; }; +&cpu_thermal { + status = "disabled"; +}; + +&bandgap { + status = "disabled"; +}; +// +//&thermal_zones { +// status = "disabled"; +//}; + + / { model = "Nokia N900"; compatible = "nokia,omap3-n900", "ti,omap3430", "ti,omap3";
Quoted 12 lines · Merlijn Wajer
Hi,
Quoted 8 lines · Tony Lindgren
Quoted 2 lines · Merlijn Wajer
and I am no longer seeing the oopses or resets when running "sleep 30; blkid" from minimal userspace.
If disabling the
cpu_thermalmakes things behave, chances are we have wrong thermal values in the dts for n900 and possible all 34xx devices. The 36xx values are behaving AFAIK.
Quoted 7 lines
Just disabling the bandgap doesn't seem to be enough. I also tried disabling just the bandgap and
thermal_zones, but that is also isn't enough. However, *just* disablingcpu_thermalalso isn't enough.Disabling both
cpu_thermaland bandgap is enough to stop the oopses [1]. So it sounds like there might potentially be some problems incpu_thermalas well then?
Not sure what's wrong..
But meanwhile, looks like the patch below produces thermal values, so maybe check if it allows idling with thermal enabled on n900. It might just leave out all the extra polling.
Regards,
Tony
drivers/thermal/ti-soc-thermal/omap3-thermal-data.c +2 -1
@@ -66,7 +66,8 @@ omap34xx_adc_to_temp[128] = {/* OMAP34XX data */ const struct ti_bandgap_data omap34xx_data = { - .features =TI_BANDGAP_FEATURE_CLK_CTRL |TI_BANDGAP_FEATURE_UNRELIABLE , + .features =TI_BANDGAP_FEATURE_CLK_CTRL |TI_BANDGAP_FEATURE_UNRELIABLE | +TI_BANDGAP_FEATURE_CONT_MODE_ONLY , .fclock_name ="ts_fck" , .div_ck_name ="ts_fck" , .conv_table = omap34xx_adc_to_temp,