s390: kernel panic on specification exception: 0006
Ihor Solodrai
reviewerHi everyone.
Stumbled on the following while testing unrelated changes to BPF CI scripts:
#353 select_reuseport:OK
specification exception: 0006 ilc:2 [#1]SMP
Modules linked in: bpf_testmod(OE) [last unloaded: bpf_test_modorder_x(OE)]
CPU: 0 UID: 0 PID: 108 Comm: new_name Tainted: G OE 6.15.0-rc4-g169491540638-dirty #1 NONE
Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Hardware name: IBM 8561 LT1 400 (KVM/Linux)
Krnl PSW : 0404e00180000000 000001e43ac595e4 (hrtimer_interrupt+0x4/0x2a0)
R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3
Krnl GPRS: 7fffffffffffffff 000001e43ac595e0 00000000fff8f200 000001e43c52a0c0
00000164c50740d8 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 000001643afabe00 0000000000000000
000003ff9fbadf98 000001e43b747550 000001e43ab12774 00000000fffdfe68
Krnl Code:#000001e43ac595e0: 67756573 mxd %f7,1395(%r5,%r6)
>000001e43ac595e4: 743d696e unknown
000001e43ac595e8: 73303031 unknown
000001e43ac595ec: 61652c64 unknown
000001e43ac595f0: 65627567 unknown
000001e43ac595f4: 2d74 ddr %f7,%f4
000001e43ac595f6: 68726561 ld %f7,1377(%r2,%r6)
000001e43ac595fa: 64733d6f unknown
Call Trace:
[<000001e43ac595e4>] hrtimer_interrupt+0x4/0x2a0
[<000001e43ab128ca>] do_irq_async+0x5a/0x78
[<000001e43b65f694>] do_ext_irq+0xac/0x168
[<000001e43b66ae90>] ext_int_handler+0xc8/0xf8
Last Breaking-Event-Address:
[<000001e43ab08a9e>] clock_comparator_work+0x2e/0x30
Kernel panic - not syncing: Fatal exception in interruptThis is on the current tip of bpf-next (9325d53fe9ad).
Job: https://github.com/kernel-patches/vmtest/actions/runs/15051985809/job/42309244372
You can download full logs from there.
It only happened once so far, didn't repeat on restart.
Heiko Carstens
maintainer[full quote, and adding Ilya]
Quoted 33 lines · Ihor Solodrai
Hi everyone.
Stumbled on the following while testing unrelated changes to BPF CI scripts:
#353 select_reuseport:OK specification exception: 0006 ilc:2 [#1]SMP Modules linked in: bpf_testmod(OE) [last unloaded: bpf_test_modorder_x(OE)] CPU: 0 UID: 0 PID: 108 Comm: new_name Tainted: G OE 6.15.0-rc4-g169491540638-dirty #1 NONE Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: IBM 8561 LT1 400 (KVM/Linux) Krnl PSW : 0404e00180000000 000001e43ac595e4 (hrtimer_interrupt+0x4/0x2a0) R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3 Krnl GPRS: 7fffffffffffffff 000001e43ac595e0 00000000fff8f200 000001e43c52a0c0 00000164c50740d8 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 000001643afabe00 0000000000000000 000003ff9fbadf98 000001e43b747550 000001e43ab12774 00000000fffdfe68 Krnl Code:#000001e43ac595e0: 67756573 mxd %f7,1395(%r5,%r6) >000001e43ac595e4: 743d696e unknown 000001e43ac595e8: 73303031 unknown 000001e43ac595ec: 61652c64 unknown 000001e43ac595f0: 65627567 unknown 000001e43ac595f4: 2d74 ddr %f7,%f4 000001e43ac595f6: 68726561 ld %f7,1377(%r2,%r6) 000001e43ac595fa: 64733d6f unknown Call Trace: [<000001e43ac595e4>] hrtimer_interrupt+0x4/0x2a0 [<000001e43ab128ca>] do_irq_async+0x5a/0x78 [<000001e43b65f694>] do_ext_irq+0xac/0x168 [<000001e43b66ae90>] ext_int_handler+0xc8/0xf8 Last Breaking-Event-Address: [<000001e43ab08a9e>] clock_comparator_work+0x2e/0x30 Kernel panic - not syncing: Fatal exception in interrupt
Quoted 6 lines
This is on the current tip of bpf-next (
9325d53fe9ad).Job: https://github.com/kernel-patches/vmtest/actions/runs/15051985809/job/42309244372
You can download full logs from there.It only happened once so far, didn't repeat on restart.
Looks like the text section, which contains the hrtimer_interrupt function, got corrupted, and an ASCII string was written there. The hexdump above with the corrupted code translates to the following ASCII text: "guest=ins001ae,debug-threads=o".
Does that ring a bell?
Quoted 49 lines · Heiko Carstens
[full quote, and adding Ilya]
Quoted 46 lines · Ihor Solodrai
Hi everyone.
Stumbled on the following while testing unrelated changes to BPF CI scripts:
#353 select_reuseport:OK
specification exception: 0006 ilc:2 [#1]SMP Modules linked in:bpf_testmod(OE)[last unloaded:bpf_test_modorder_x(OE)]CPU: 0 UID: 0 PID: 108 Comm: new_name Tainted: G OE 6.15.0-rc4-g169491540638-dirty #1 NONE Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: IBM 8561 LT1 400 (KVM/Linux) Krnl PSW : 0404e00180000000 000001e43ac595e4 (hrtimer_interrupt+0x4/0x2a0) R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3 Krnl GPRS: 7fffffffffffffff 000001e43ac595e0 00000000fff8f200 000001e43c52a0c0 00000164c50740d8 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 000001643afabe00 0000000000000000 000003ff9fbadf98 000001e43b747550 000001e43ab12774 00000000fffdfe68 Krnl Code:#000001e43ac595e0: 67756573 mxd %f7,1395(%r5,%r6) >000001e43ac595e4: 743d696e unknown 000001e43ac595e8: 73303031 unknown 000001e43ac595ec: 61652c64 unknown 000001e43ac595f0: 65627567 unknown 000001e43ac595f4: 2d74 ddr %f7,%f4 000001e43ac595f6: 68726561 ld %f7,1377(%r2,%r6) 000001e43ac595fa: 64733d6f unknown Call Trace: [<000001e43ac595e4>] hrtimer_interrupt+0x4/0x2a0 [<000001e43ab128ca>] do_irq_async+0x5a/0x78 [<000001e43b65f694>] do_ext_irq+0xac/0x168 [<000001e43b66ae90>] ext_int_handler+0xc8/0xf8 Last Breaking-Event-Address: [<000001e43ab08a9e>] clock_comparator_work+0x2e/0x30 Kernel panic - not syncing: Fatal exception in interruptThis is on the current tip of bpf-next (
9325d53fe9ad).Job: https://github.com/kernel-patches/vmtest/actions/runs/15051985809/job/42309244372
You can download full logs from there.
Quoted 8 lines
Quoted 1 line
It only happened once so far, didn't repeat on restart.
Looks like the text section, which contains the
hrtimer_interruptfunction, got corrupted, and an ASCII string was written there. The hexdump above with the corrupted code translates to the following ASCII text: "guest=ins001ae,debug-threads=o".Does that ring a bell?
This looks like a part of the following QEMU config:
static QemuOptsList qemu_name_opts = {
.name = "name",
.implied_opt_name = "guest",
.merge_lists = true,
.head = QTAILQ_HEAD_INITIALIZER(qemu_name_opts.head), .desc = {
{
.name = "guest",
.type = QEMU_OPT_STRING,
.help = "Sets the name of the guest.\n"
"This name will be displayed in the SDL window
caption.\n"
"The name will also be used for the VNC server",
}, {
.name = "process",
.type = QEMU_OPT_STRING,
.help = "Sets the name of the QEMU process, as shown in top
etc",
}, {
.name = "debug-threads",
.type = QEMU_OPT_BOOL,
.help = "When enabled, name the individual threads; defaults
off.\n"
"NOTE: The thread names are for debugging and not
a\n"
"stable API.",
},
{ /* End of list */ }
},
};Can it be that the hypervisor has corrupted guest memory?
But I'm not sure what exactly in the CI software stack can specify these values. I checked vmtest, and it doesn't seem to use debug-threads.