New: PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
https://bugzilla.kernel.org/show_bug.cgi?id=207129
Bug ID: 207129
Summary: PowerMac G4 DP (5.6.2 debug kernel + inline KASAN)
freezes shortly after booting with "do_IRQ: stack
overflow: 1760"
Product: Platform Specific/Hardware
Version: 2.5
Kernel Version: 5.6.2
Hardware: PPC-32
OS: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: PPC-32
Assignee: [email protected]
Reporter: [email protected]
CC: [email protected]
Regression: NoCreated attachment 288221
--> https://bugzilla.kernel.org/attachment.cgi?id=288221&action=edit
kernel .config (5.6.2, INLINE KASAN, PowerMac G4 DP)
Was trying to do some testing with the PowerMac G4 DP again, running a 5.6.2 debug kernel w. KASAN INLINE. The G4 boots fine, but crashes shortly afterwards when using it, leaving no stack trace, but only this message on the screen:
do_IRQ: stack overflow: 1760
CPU: 0 PID: 209 Comm: rsync Tained: G W 5.6.2-PowerMacG4+ #3 Call Trace:
120 seconds panic timer does not kick in. I have to manually switch off/switch on the G4.
Signature · 2 lines
You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
https://bugzilla.kernel.org/show_bug.cgi?id=207129
--- Comment #1 from Christophe Leroy ([email protected]) --- So it hands in show_stack().
Does it also hang without CONFIG_DEBUG_STACKOVERFLOW ? If not, it means we have a problem with check_stack_overflow()
Regardless of the result above, can you try increasing CONFIG_THREAD_SHIFT ?
Can you maybe also do a test without CONFIG_VMAP_STACK ?
Signature · 2 lines
You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=207129
--- Comment #2 from Erhard F. ([email protected]) --- Created attachment 288229
--> https://bugzilla.kernel.org/attachment.cgi?id=288229&action=edit
screenshot01.jpg
Without CONFIG_DEBUG_STACKOVERFLOW things are better. The rsync completes, the G4 was building stuff for 2 hours or so until I got these errors and a hard freeze:
[...]
Oops: kernel stack overflow, sig: 11 [#1] BE PAGE_SIZE=4K MMU=Hash SMP NR_CPUS=2 PowerMac Modules linked in: ... CPU: 1 PID: 17105 Comm: kworker/u4:5 Tainted: G W 5.6.2-PowerMacG4+ #5 ------------[ cut here ]------------ kernel BUG at mm/usercopy.c:99! Oops: Exception in kernel mode, sig: 5 [#2] BE PAGE_SIZE=4K MMU=Hash SMP NR_CPUS=2 PowerMac Modules linked in: ... CPU: 1 PID: 17185 Comm: kworker/u4:5 Tainted: G W
5.6.2-PowerMacG4+ #5
usercopy: Kernel memory overwrite attempt detected to kernel text (offset 6336, size 4)!
------------[ cut here ]------------
kernel BUG at mm/usercopy.c:99!
Oops: Exception in kernel mode, sig: 5 [#3] BE PAGE_SIZE=4K MMU=Hash SMP NR_CPUS=2 PowerMac Modules linked in: ... CPU: 1 PID: 17185 Comm: kworker/u4:5 Tainted: G W
5.6.2-PowerMacG4+ #5
usercopy: Kernel memory overwrite attempt detected to kernel text (offset 5336, size 4)!
------------[ cut here ]------------
kernel BUG at mm/usercopy.c:99!
Oops: Exception in kernel mode, sig: 5 [#4] BE PAGE_SIZE=4K MMU=Hash SMP NR_CPUS=2 PowerMac Modules linked in: ... CPU: 1 PID: 17185 Comm: kworker/u4:5 Tainted: G W
5.6.2-PowerMacG4+ #5
usercopy: Kernel memory overwrite attempt detected to kernel text (offset 4336, size 4)!
------------[ cut here ]------------
kernel BUG at mm/usercopy.c:99!
Oops: Exception in kernel mode, sig: 5 [#5] BE PAGE_SIZE=4K MMU=Hash SMP NR_CPUS=2 PowerMac Modules linked in: ... Unrecoverable FP Unavailable Exception 801 at 9b8 CPU: 1 PID: 17185 Comm: kworker/u4:5 Tainted: G W
5.6.2-PowerMacG4+ #5
usercopy: Kernel memory overwrite attempt detected to kernel text (offset 3336, size 4)!
------------[ cut here ]------------
Now running with CONFIG_THREAD_SHIFT=14 which runs fine so far... Did not try without CONFIG_VMAP_STACK yet.
Signature · 2 lines
You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=207129
--- Comment #3 from Erhard F. ([email protected]) --- Created attachment 288231
--> https://bugzilla.kernel.org/attachment.cgi?id=288231&action=edit
screenshot02.jpg
Signature · 2 lines
You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=207129
--- Comment #4 from Erhard F. ([email protected]) ---
Without CONFIG_VMAP_STACK I had one crash after 2-3 hours of building but the panic timer kicked in and rebooted the machine. Now it has been building packages for hours again without any anomalies.
Signature · 2 lines
You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=207129
--- Comment #5 from Christophe Leroy ([email protected]) --- Ok, so as a summary:
- With CONFIG_THREAD_SHIFT = 13 and CONFIG_DEBUG_STACKOVERFLOW, the system gets stuck
- With CONFIG_THREAD_SHIFT = 13 and without CONFIG_DEBUG_STACKOVERFLOW, stack overflow is not really detected until it gets into kernel text !!!
- With CONFIG_THREAD_SHIFT = 14 it runs fine
- With CONFIG_VMAP_STACK, the automatic restart doesn't work
- Without CONFIG_VMAP_STACK, the automatic restart works
So I'll send a patch to set CONFIG_THREAD_SHIFT to 14 when CONFIG_KASAN is selected. x86 and arm64 already do that.
And I'll try to investigate the other points when I have time.
Signature · 2 lines
You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=207129
--- Comment #6 from Erhard F. ([email protected]) --- Yes, precisely summarized! Thanks for your efforts!
CONFIG_KASAN though only is x86_64 not x86 AFAIK.
Signature · 2 lines
You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=207129
Michael Ellerman ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
CC| |[email protected]
Resolution|--- |CODE_FIX--- Comment #7 from Michael Ellerman ([email protected]) --- I think this was resolved by increasing the stack size for KASAN builds.
ie.edbadaf06710 ("powerpc/kasan: Fix stack overflow by increasing THREAD_SHIFT") and later3e8635fb2e07 ("powerpc/kasan: Force thread size increase with KASAN")
If not feel free to reopen.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=207129
Michael Ellerman ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |CLOSED
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=207129
Erhard F. ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|CLOSED |REOPENED
Resolution|CODE_FIX |------ Comment #8 from Erhard F. ([email protected]) --- Currently (as of v6.8-rc6) and since quite a while my G4 does not boot at all with CONFIG_KASAN_INLINE=y.
When I try booting an KASAN_INLINE enabled kernel it fails with an invalid memory access and I get dropped out to the OpenFirmaware console.
A small difference is when I boot the 56M vmlinux-6.8.0-rc6-PMacG4:
Please wait, loading kernel...
Elf32 kernel loaded...
Invalid memory access at %SRR0: 00000013 %SRR1: 00001300
Apple PowerMac3,6 4.6.0f1 BootROM built on 02/20/03 at 13:52:27 [...]
vs. booting the 16M arch/powerpc/boot/zImage:
Please wait, loading kernel...
Elf32 kernel loaded...
zImage starting: loaded at 0x00400000 (sp: 0x012eefb0) OF version = 'OpenFirmware 3'
Allocating 0x2c337e0 bytes for kernel...
Trying to claim from 0x400000 to 0x12ef5d8 (0xeef5d8) got ffffffff Decompressing (0x01414000 <- 0x00410000:0x12ea9c9)... Done! Decompressed 0x2bcc80c bytes
Linux/PowerPC load: ro root=/dev/sda5 slub_debug=FZP page_poison=1 [email protected]/eth0,[email protected]/A8:A1:59:16:4F:EA debug
Finalizing device tree... using OF tree (promptr=ff847240)
Invalid memory access at %SRR0: 40000000 %SRR1: 00000000
Apple PowerMac3,6 4.6.0f1 BootROM built on 02/20/03 at 13:52:27 [...]
Same kernel with CONFIG_KASAN_OUTLINE=y instead of KASAN_INLINE boots and runs ok.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=207129
--- Comment #9 from Erhard F. ([email protected]) --- Created attachment 305910
--> https://bugzilla.kernel.org/attachment.cgi?id=305910&action=edit
kernel .config (6.8-rc6, KASAN_INLINE=y, PowerMac G4 DP)
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=207129
--- Comment #10 from Christophe Leroy ([email protected]) --- I built a kernel with your .config, the problem is a size problem.
PPC32 kernels are not designed to be that big.
Extract from generated System.map:
c2394000 D _etext
c2800000 T _sinittext
c2bf5000 B _end
You need to keep the size of the kernel below 32Mbytes, or a deep work is required to enable the kernel to perform far jumps before the kernel is relocated.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 207129] PowerMac G4 DP (5.6.2 debug kernel + inline KASAN) freezes shortly after booting with "do_IRQ: stack overflow: 1760"
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=207129
Erhard F. ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution|--- |CODE_FIX--- Comment #11 from Erhard F. ([email protected]) --- You were correct! I forgot about that...
I shrunk the size by using -Os and disabling some debugging stuff and changing some statically built-in stuff to 'M' without sacrificing debugging capabilities too much until it fit < 32 MiB:
KASAN_OUTLINE vs.
# size vmlinux-6.8.0-rc6-PMacG4 text data bss dec hex filename 12367737 6652440 426336 19446513 128baf1 vmlinux-6.8.0-rc6-PMacG4
KASAN_INLINE
# size vmlinux-6.8.0-rc6-PMacG4 text data bss dec hex filename 24660169 6652440 426336 31738945 1e44c41 vmlinux-6.8.0-rc6-PMacG4
Apart from that I can confirm inline KASAN runs fine now and I really no longer get this stack overflow when using it.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.