2.6.26-rc8 pv_ops causes Unhandled invalid opcode fault/trap
Xen: 3.2.1-rc5 64bit
Dom0: 2.6.18.8 (at cs 524) 32-pae
DomU: 2.6.26-rc8 32-pae
root@newark13:~# xm create -f /linodes/xencaker/xen.conf -c
Using config file "/linodes/xencaker/xen.conf".
Started domain xencaker
root@newark13:~#root@newark13:~# xm dmesg ... (XEN) traps.c:413:d332 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000] (XEN) domain_crash_sync called from entry.S (XEN) Domain 332 (vcpu#0) crashed on cpu#0: (XEN) ----[ Xen-3.2.1-rc5 x86_64 debug=y Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e019:[<00000000c0103b55>] (XEN) RFLAGS: 0000000000000282 CONTEXT: guest (XEN) rax: 00000000ffffffea rbx: 00000000deadbeef rcx: 00000000deadbeef (XEN) rdx: 00000000deadbeef rsi: 00000000deadbeef rdi: 00000000c06c1000 (XEN) rbp: 00000000c06c0000 rsp: 00000000c0667f08 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026b0 (XEN) cr3: 00000002d351b000 cr2: 0000000000000000 (XEN) ds: e021 es: e021 fs: e021 gs: e021 ss: e021 cs: e019 (XEN) Guest stack trace from esp=c0667f08: (XEN) c0103b55 0001e019 00010082 00000001 006c0000 c0756d50 c06c1000 c065c580 (XEN) c067e4dc c0667f44 c0678ac9 c0667f38 00100000 c0756d50 00000000 00000000 (XEN) 00040800 00010676 c0667fb8 c06a3b00 c065c680 c065c580 c0676452 00000000 (XEN) 00000000 00000000 c065c580 00000000 c0667fb0 c0667fa4 c0667fac 1f898975 (XEN) 40000000 c0667fdc f5800000 00000007 40000000 c0667fdc f5800000 00000007 (XEN) c066e771 c0537060 f5800000 00000007 c012500f 40000000 c06753a2 00000000 (XEN) c069c5c0 c069c5dc c069c698 c069c6b0 00000000 1f898975 00080201 00040800 (XEN) 00010676 f5800000 00000000 c0752000 00000000 00000000
-Chris
Quoted 7 lines · Christopher S. Aker
Xen: 3.2.1-rc5 64bit
Dom0: 2.6.18.8 (at cs 524) 32-pae
DomU: 2.6.26-rc8 32-paeroot@newark13:~# xm create -f /linodes/xencaker/xen.conf -c Using config file "/linodes/xencaker/xen.conf". Started domain xencaker
Is this consistent or sporadic? Does it depend on how much guest memory you give it? What was the last kernel version which worked?
Quoted 4 lines
root@newark13:~#root@newark13:~# xm dmesg ...
Were there any other errors/warnings printed here?
Quoted 6 lines
(XEN) traps.c:413:d332 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000] (XEN)
domain_crash_synccalled fromentry.S
(XEN) Domain 332 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-3.2.1-rc5x86_64debug=y Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e019:[<00000000c0103b55>]
Could you decode this to see which function this corresponds to?
Quoted 19 lines
(XEN) RFLAGS: 0000000000000282 CONTEXT: guest
(XEN) rax: 00000000ffffffea rbx: 00000000deadbeef rcx: 00000000deadbeef (XEN) rdx: 00000000deadbeef rsi: 00000000deadbeef rdi: 00000000c06c1000 (XEN) rbp: 00000000c06c0000 rsp: 00000000c0667f08 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026b0 (XEN) cr3: 00000002d351b000 cr2: 0000000000000000 (XEN) ds: e021 es: e021 fs: e021 gs: e021 ss: e021 cs: e019 (XEN) Guest stack trace from esp=c0667f08: (XEN) c0103b55 0001e019 00010082 00000001 006c0000 c0756d50 c06c1000 c065c580 (XEN) c067e4dc c0667f44 c0678ac9 c0667f38 00100000 c0756d50 00000000 00000000 (XEN) 00040800 00010676 c0667fb8 c06a3b00 c065c680 c065c580 c0676452 00000000 (XEN) 00000000 00000000 c065c580 00000000 c0667fb0 c0667fa4 c0667fac 1f898975 (XEN) 40000000 c0667fdc f5800000 00000007 40000000 c0667fdc f5800000 00000007 (XEN) c066e771 c0537060 f5800000 00000007 c012500f 40000000 c06753a2 00000000 (XEN) c069c5c0 c069c5dc c069c698 c069c6b0 00000000 1f898975 00080201 00040800 (XEN) 00010676 f5800000 00000000 c0752000 00000000 00000000
Quoted 1 line
-Chris
Thanks,
J
Quoted 7 lines · Christopher S. Aker
Xen: 3.2.1-rc5 64bit
Dom0: 2.6.18.8 (at cs 524) 32-pae
DomU: 2.6.26-rc8 32-paeroot@newark13:~# xm create -f /linodes/xencaker/xen.conf -c Using config file "/linodes/xencaker/xen.conf". Started domain xencaker
Is this consistent or sporadic? Does it depend on how much guest memory you give it? What was the last kernel version which worked?
Quoted 4 lines
root@newark13:~#root@newark13:~# xm dmesg ...
Were there any other errors/warnings printed here?
Quoted 7 lines
(XEN) traps.c:413:d332 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000]
(XEN)domain_crash_synccalled fromentry.S
(XEN) Domain 332 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-3.2.1-rc5x86_64debug=y Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e019:[<00000000c0103b55>]
Could you decode this to see which function this corresponds to?
Quoted 32 lines
(XEN) RFLAGS: 0000000000000282 CONTEXT: guest
(XEN) rax:00000000ffffffearbx:00000000deadbeefrcx:00000000deadbeef
(XEN) rdx:00000000deadbeefrsi:00000000deadbeefrdi:00000000c06c1000
(XEN) rbp:00000000c06c0000rsp:00000000c0667f08r8:
0000000000000000
(XEN) r9: 0000000000000000 r10: 0000000000000000 r11:
0000000000000000
(XEN) r12: 0000000000000000 r13: 0000000000000000 r14:
0000000000000000
(XEN) r15: 0000000000000000 cr0:000000008005003bcr4:00000000000026b0
(XEN) cr3:00000002d351b000cr2: 0000000000000000
(XEN) ds: e021 es: e021 fs: e021 gs: e021 ss: e021 cs: e019
(XEN) Guest stack trace from esp=c0667f08:
(XEN)c0103b550001e01900010082 00000001006c0000c0756d50c06c1000c065c580
(XEN)c067e4dcc0667f44c0678ac9c0667f3800100000c0756d5000000000
00000000
(XEN) 00040800 00010676c0667fb8c06a3b00c065c680c065c580c0676452
00000000
(XEN) 00000000 00000000c065c58000000000c0667fb0c0667fa4c0667fac1f898975
(XEN) 40000000c0667fdcf580000000000007 40000000c0667fdcf5800000
00000007
(XEN)c066e771c0537060f580000000000007c012500f40000000c06753a2
00000000
(XEN)c069c5c0c069c5dcc069c698c069c6b0000000001f89897500080201
00040800
(XEN) 00010676f580000000000000c075200000000000 00000000
Quoted 1 line
-Chris
Thanks,
J
Quoted 1 line · Jeremy Fitzhardinge
Is this consistent or sporadic?
Consistently fails to boot, across a few test boxes, slightly different Xen versions -- all around Xen 3.2.1, same config.
Quoted 1 line
Does it depend on how much guest memory you give it?
No. Tried with 128, 360, 5000. They all fail in the same way.
Quoted 1 line
What was the last kernel version which worked?
I built a 2.6.25.10 pv_ops this morning and it boots fine.
Quoted 1 line
Were there any other errors/warnings printed here?
Nope. Just some whitespace.
Quoted 8 lines
Quoted 6 lines
(XEN) traps.c:413:d332 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000] (XEN)
domain_crash_synccalled fromentry.S
(XEN) Domain 332 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-3.2.1-rc5x86_64debug=y Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e019:[<00000000c0103b55>]Could you decode this to see which function this corresponds to?
I'd love to know how for future reference, but for now I've placed the files here:
http://www.theshore.net/~caker/xen/pvops-bug1/
Thanks,
-Chris
Quoted 1 line · Jeremy Fitzhardinge
Is this consistent or sporadic?
Consistently fails to boot, across a few test boxes, slightly different Xen versions -- all around Xen 3.2.1, same config.
Quoted 1 line
Does it depend on how much guest memory you give it?
No. Tried with 128, 360, 5000. They all fail in the same way.
Quoted 1 line
What was the last kernel version which worked?
I built a 2.6.25.10 pv_ops this morning and it boots fine.
Quoted 1 line
Were there any other errors/warnings printed here?
Nope. Just some whitespace.
Quoted 9 lines
Quoted 7 lines
(XEN) traps.c:413:d332 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000]
(XEN)domain_crash_synccalled fromentry.S
(XEN) Domain 332 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-3.2.1-rc5x86_64debug=y Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e019:[<00000000c0103b55>]Could you decode this to see which function this corresponds to?
I'd love to know how for future reference, but for now I've placed the files here:
http://www.theshore.net/~caker/xen/pvops-bug1/
Thanks,
-Chris
Quoted 9 lines · Christopher S. Aker
Quoted 1 line · Jeremy Fitzhardinge
Is this consistent or sporadic?
Consistently fails to boot, across a few test boxes, slightly different Xen versions -- all around Xen 3.2.1, same config.
Quoted 1 line
Does it depend on how much guest memory you give it?
No. Tried with 128, 360, 5000. They all fail in the same way.
Quoted 7 lines
Quoted 1 line
What was the last kernel version which worked?
I built a 2.6.25.10
pv_opsthis morning and it boots fine.Quoted 1 line
Were there any other errors/warnings printed here?
Nope. Just some whitespace.
Are you sure? Could you send a full xm dmesg log output? The crash is from a BUG() caused by a failing hypercall. Since you have compiled Xen with debug enabled, it should have printed something about why it was failing the hypercall.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
(XEN) traps.c:413:d332 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000] (XEN)
domain_crash_synccalled fromentry.S
(XEN) Domain 332 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-3.2.1-rc5x86_64debug=y Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e019:[<00000000c0103b55>]
Quoted 5 lines
Quoted 1 line
Could you decode this to see which function this corresponds to?
I'd love to know how for future reference, but for now I've placed the files here:
Thanks, that's very interesting. I had a very similar crash in a current -rc kernel when running it under the Citrix XenServer product version of Xen, and I wasn't sure where to point fingers.
For reference:
$ gdb 2.6.26-rc8-linode11 GNU gdb Fedora (6.8-10.fc9) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... (no debugging symbols found) (gdb) x/i 0x00000000c0103b55 <- note: need to add 0x to start 0xc0103b55 <make_lowmem_page_readonly+85>: ud2a (gdb) disassemble make_lowmem_page_readonly Dump of assembler code for function make_lowmem_page_readonly: 0xc0103b00 <make_lowmem_page_readonly+0>: push %ebp 0xc0103b01 <make_lowmem_page_readonly+1>: mov %eax,%ebp 0xc0103b03 <make_lowmem_page_readonly+3>: push %edi ...
Thanks,
J
Quoted 10 lines · Christopher S. Aker
Quoted 1 line · Jeremy Fitzhardinge
Is this consistent or sporadic?
Consistently fails to boot, across a few test boxes, slightly different Xen versions -- all around Xen 3.2.1, same config.
Quoted 1 line
Does it depend on how much guest memory you give it?
No. Tried with 128, 360, 5000. They all fail in the same way.
Quoted 7 lines
Quoted 1 line
What was the last kernel version which worked?
I built a 2.6.25.10
pv_opsthis morning and it boots fine.Quoted 1 line
Were there any other errors/warnings printed here?
Nope. Just some whitespace.
Are you sure? Could you send a full xm dmesg log output? The crash is from a BUG() caused by a failing hypercall. Since you have compiled Xen with debug enabled, it should have printed something about why it was failing the hypercall.
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
(XEN) traps.c:413:d332 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000]
(XEN)domain_crash_synccalled fromentry.S
(XEN) Domain 332 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-3.2.1-rc5x86_64debug=y Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e019:[<00000000c0103b55>]
Quoted 6 lines
Quoted 1 line
Could you decode this to see which function this corresponds to?
I'd love to know how for future reference, but for now I've placed the files here:
Thanks, that's very interesting. I had a very similar crash in a current -rc kernel when running it under the Citrix XenServer product version of Xen, and I wasn't sure where to point fingers.
For reference:
$ gdb 2.6.26-rc8-linode11 GNU gdb Fedora (6.8-10.fc9) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... (no debugging symbols found) (gdb) x/i 0x00000000c0103b55 <- note: need to add 0x to start 0xc0103b55 <make_lowmem_page_readonly+85>: ud2a (gdb) disassemble make_lowmem_page_readonly Dump of assembler code for function make_lowmem_page_readonly: 0xc0103b00 <make_lowmem_page_readonly+0>: push %ebp 0xc0103b01 <make_lowmem_page_readonly+1>: mov %eax,%ebp 0xc0103b03 <make_lowmem_page_readonly+3>: push %edi ...
Thanks,
J
Quoted 5 lines · Jeremy Fitzhardinge
Quoted 3 lines
Quoted 1 line
Were there any other errors/warnings printed here?
Nope. Just some whitespace.
Are you sure? Could you send a full xm dmesg log output? The crash is from a
BUG()caused by a failing hypercall. Since you have compiled Xen with debug enabled, it should have printed something about why it was failing the hypercall.
Unfortunately, I included the entire dmesg output from that xm create run. Here's more of the transcript in case it's useful:
http://www.theshore.net/~caker/xen/pvops-bug1/log.txt
Although, looking at it again, I did miss this line:
(XEN) mm.c:645:d341 Non-privileged (341) attempt to map I/O space 000d2e50
Quoted 16 lines
For reference:
$ gdb 2.6.26-rc8-linode11 GNU gdb Fedora (6.8-10.fc9) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... (no debugging symbols found) (gdb) x/i 0x00000000c0103b55 <- note: need to add 0x to start 0xc0103b55 <make_lowmem_page_readonly+85>: ud2a (gdb) disassemble make_lowmem_page_readonly Dump of assembler code for function make_lowmem_page_readonly: 0xc0103b00 <make_lowmem_page_readonly+0>: push %ebp 0xc0103b01 <make_lowmem_page_readonly+1>: mov %eax,%ebp 0xc0103b03 <make_lowmem_page_readonly+3>: push %edi
Cool, thanks :)
-Chris
Quoted 8 lines · Jeremy Fitzhardinge
Quoted 3 lines
Quoted 1 line
Were there any other errors/warnings printed here?
Nope. Just some whitespace.
Are you sure? Could you send a full xm dmesg log output? The crash is from a
BUG()caused by a failing hypercall. Since you have compiled Xen with debug enabled, it should have printed something about why it was failing the hypercall.
Unfortunately, I included the entire dmesg output from that xm create run. Here's more of the transcript in case it's useful:
http://www.theshore.net/~caker/xen/pvops-bug1/log.txt
Although, looking at it again, I did miss this line:
(XEN) mm.c:645:d341 Non-privileged (341) attempt to map I/O space 000d2e50
Quoted 11 lines
For reference:
$ gdb 2.6.26-rc8-linode11 GNU gdb Fedora (6.8-10.fc9) Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... (no debugging symbols found)
Quoted 8 lines
(gdb) x/i 0x00000000c0103b55 <- note: need to add 0x to start 0xc0103b55 <make_lowmem_page_readonly+85>: ud2a (gdb) disassemble make_lowmem_page_readonly Dump of assembler code for function make_lowmem_page_readonly: 0xc0103b00 <make_lowmem_page_readonly+0>: push %ebp 0xc0103b01 <make_lowmem_page_readonly+1>: mov %eax,%ebp 0xc0103b03 <make_lowmem_page_readonly+3>: push %edi
Cool, thanks :)
-Chris
Quoted 7 lines · Christopher S. Aker
Quoted 5 lines · Jeremy Fitzhardinge
Quoted 3 lines
Quoted 1 line
Were there any other errors/warnings printed here?
Nope. Just some whitespace.
Are you sure? Could you send a full xm dmesg log output? The crash is from a
BUG()caused by a failing hypercall. Since you have compiled Xen with debug enabled, it should have printed something about why it was failing the hypercall.
Quoted 7 lines
Unfortunately, I included the entire dmesg output from that xm create run. Here's more of the transcript in case it's useful:
http://www.theshore.net/~caker/xen/pvops-bug1/log.txt
Although, looking at it again, I did miss this line:
(XEN) mm.c:645:d341 Non-privileged (341) attempt to map I/O space
000d2e50
Good, that's what I was hoping to see. (Well, actually not that *particular* message, which is strange, but some explanatory message at least.)
J
Quoted 13 lines · Christopher S. Aker
Quoted 8 lines · Jeremy Fitzhardinge
Quoted 3 lines
Quoted 1 line
Were there any other errors/warnings printed here?
Nope. Just some whitespace.
Are you sure? Could you send a full xm dmesg log output? The crash is from a
BUG()caused by a failing hypercall. Since you have compiled Xen with debug enabled, it should have printed something about why it was failing the hypercall.Unfortunately, I included the entire dmesg output from that xm create run. Here's more of the transcript in case it's useful:
Quoted 6 lines
http://www.theshore.net/~caker/xen/pvops-bug1/log.txt
Although, looking at it again, I did miss this line:
(XEN) mm.c:645:d341 Non-privileged (341) attempt to map I/O space
000d2e50
Good, that's what I was hoping to see. (Well, actually not that *particular* message, which is strange, but some explanatory message at least.)
J
Quoted 5 lines · Jeremy Fitzhardinge
Quoted 3 lines
Although, looking at it again, I did miss this line:
(XEN) mm.c:645:d341 Non-privileged (341) attempt to map I/O space
000d2e50Good, that's what I was hoping to see. (Well, actually not that *particular* message, which is strange, but some explanatory message at least.)
FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it down further if that'd be useful to you.
-Chris
Quoted 8 lines · Jeremy Fitzhardinge
Quoted 4 lines
Although, looking at it again, I did miss this line:
(XEN) mm.c:645:d341 Non-privileged (341) attempt to map I/O space
000d2e50Good, that's what I was hoping to see. (Well, actually not that *particular* message, which is strange, but some explanatory message at least.)
FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it down further if that'd be useful to you.
-Chris
Quoted 7 lines · Christopher S. Aker
Quoted 5 lines · Jeremy Fitzhardinge
Quoted 5 lines
Although, looking at it again, I did miss this line:
(XEN) mm.c:645:d341 Non-privileged (341) attempt to map I/O space
000d2e50
Quoted 8 lines
Quoted 4 lines
Good, that's what I was hoping to see. (Well, actually not that *particular* message, which is strange, but some explanatory message at least.)
FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it down further if that'd be useful to you.
Yes, please.
J
Quoted 11 lines · Christopher S. Aker
Quoted 6 lines · Jeremy Fitzhardinge
Quoted 4 lines
Although, looking at it again, I did miss this line:
(XEN) mm.c:645:d341 Non-privileged (341) attempt to map I/O space
000d2e50Good, that's what I was hoping to see. (Well, actually not that *particular* message, which is strange, but some explanatory message at least.)
FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it down further if that'd be useful to you.
Yes, please.
J
Quoted 4 lines · Jeremy Fitzhardinge
Quoted 2 lines
FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it down further if that'd be useful to you.
Yes, please.
Was there any final conclusion there ? Because I'm currently trying a 2.6.27.5 with the same kind of issue:
(XEN) mm.c:1482:d31 Bad L1 flags 800000
(XEN) traps.c:437:d31 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000]
(XEN) domain_crash_sync called from entry.S
(XEN) Domain 31 (vcpu#0) crashed on cpu#2:
(XEN) ----[ Xen-3.3.0 x86_64 debug=n Not tainted ]----
(XEN) CPU: 2
(XEN) RIP: e033:[<ffffffff8020b3ed>]
(XEN) RFLAGS: 0000000000000282 EM: 1 CONTEXT: pv guest
(XEN) rax: 00000000ffffffea rbx: 000000000000077f rcx: 0000000000000054 (XEN) rdx: 0000000000000000 rsi: 8000000331df5061 rdi: ffffffff8077f000 (XEN) rbp: ffffffff806e1f88 rsp: ffffffff806e1f28 r8: 0000000000000010 (XEN) r9: 0000000000000010 r10: ffffffff806e1e88 r11: ffffffff80827f60 (XEN) r12: ffffffff8077f000 r13: 8000000000000161 r14: 8000000331df5061 (XEN) r15: 0000000000000800 cr0: 000000008005003b cr4: 00000000000026b0 (XEN) cr3: 000000033c35c000 cr2: 0000000000000000 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033 (XEN) Guest stack trace from rsp=ffffffff806e1f28: (XEN) 0000000000000054 ffffffff80827f60 ffffffff8020b3ed 000000010000e030 (XEN) 0000000000010082 ffffffff806e1f68 000000000000e02b ffffffff8020b3e9 (XEN) 0000000000000200 000000000000000c 000000000000000c 0000000000001800 (XEN) ffffffff806e1ff8 ffffffff806f99cf 0000000000000000 0000000000001000 (XEN) 0000000000001800 ffffffff80e48000 0000000000020000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffffffff80d45000 (XEN) ffffffff80d46000 ffffffff80d47000 ffffffff80d48000 ffffffff80d49000 (XEN) ffffffff80d4a000 ffffffff80d4b000 ffffffff80d4c000 ffffffff80d4d000 (XEN) ffffffff80d4e000 ffffffff80d4f000 ffffffff80d50000 ffffffff80d51000 (XEN) ffffffff80d52000 ffffffff80d53000 ffffffff80d54000 ffffffff80d55000 (XEN) ffffffff80d56000 ffffffff80d57000 ffffffff80d58000 ffffffff80d59000 (XEN) ffffffff80d5a000 ffffffff80d5b000 ffffffff80d5c000 ffffffff80d5d000 (XEN) ffffffff80d5e000 ffffffff80d5f000 ffffffff80d60000 ffffffff80d61000 (XEN) ffffffff80d62000 ffffffff80d63000 ffffffff80d64000 ffffffff80d65000 (XEN) ffffffff80d66000 ffffffff80d67000 ffffffff80d68000 ffffffff80d69000 (XEN) ffffffff80d6a000 ffffffff80d6b000 ffffffff80d6c000 ffffffff80d6d000 (XEN) ffffffff80d6e000 ffffffff80d6f000 ffffffff80d70000 ffffffff80d71000 (XEN) ffffffff80d72000 ffffffff80d73000 ffffffff80d74000 ffffffff80d75000 (XEN) ffffffff80d76000 ffffffff80d77000 ffffffff80d78000 ffffffff80d79000
Signature · 4 lines
Guillaume Rousse Service des Moyens Informatiques INRIA Saclay - Ile de France Tel: 01 69 35 69 62
Quoted 6 lines · Guillaume Rousse
Quoted 4 lines · Jeremy Fitzhardinge
Quoted 2 lines
FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it down further if that'd be useful to you.
Yes, please.
Was there any final conclusion there ? Because I'm currently trying a 2.6.27.5 with the same kind of issue:
I don't know of any outstanding bugs which cause this kind of symptom. What's happening on the Linux side when the domain crashes this way?
J
Quoted 37 lines
(XEN) mm.c:1482:d31 Bad L1 flags 800000
(XEN) traps.c:437:d31 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000] (XEN)domain_crash_synccalled fromentry.S
(XEN) Domain 31 (vcpu#0) crashed on cpu#2:
(XEN) ----[ Xen-3.3.0x86_64debug=n Not tainted ]----
(XEN) CPU: 2
(XEN) RIP: e033:[<ffffffff8020b3ed>]
(XEN) RFLAGS: 0000000000000282 EM: 1 CONTEXT: pv guest(XEN) rax: 00000000ffffffea rbx: 000000000000077f rcx: 0000000000000054 (XEN) rdx: 0000000000000000 rsi: 8000000331df5061 rdi: ffffffff8077f000 (XEN) rbp: ffffffff806e1f88 rsp: ffffffff806e1f28 r8: 0000000000000010 (XEN) r9: 0000000000000010 r10: ffffffff806e1e88 r11: ffffffff80827f60 (XEN) r12: ffffffff8077f000 r13: 8000000000000161 r14: 8000000331df5061 (XEN) r15: 0000000000000800 cr0: 000000008005003b cr4: 00000000000026b0 (XEN) cr3: 000000033c35c000 cr2: 0000000000000000 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033 (XEN) Guest stack trace from rsp=ffffffff806e1f28: (XEN) 0000000000000054 ffffffff80827f60 ffffffff8020b3ed 000000010000e030 (XEN) 0000000000010082 ffffffff806e1f68 000000000000e02b ffffffff8020b3e9 (XEN) 0000000000000200 000000000000000c 000000000000000c 0000000000001800 (XEN) ffffffff806e1ff8 ffffffff806f99cf 0000000000000000 0000000000001000 (XEN) 0000000000001800 ffffffff80e48000 0000000000020000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 0000000000000000 ffffffff80d45000 (XEN) ffffffff80d46000 ffffffff80d47000 ffffffff80d48000 ffffffff80d49000 (XEN) ffffffff80d4a000 ffffffff80d4b000 ffffffff80d4c000 ffffffff80d4d000 (XEN) ffffffff80d4e000 ffffffff80d4f000 ffffffff80d50000 ffffffff80d51000 (XEN) ffffffff80d52000 ffffffff80d53000 ffffffff80d54000 ffffffff80d55000 (XEN) ffffffff80d56000 ffffffff80d57000 ffffffff80d58000 ffffffff80d59000 (XEN) ffffffff80d5a000 ffffffff80d5b000 ffffffff80d5c000 ffffffff80d5d000 (XEN) ffffffff80d5e000 ffffffff80d5f000 ffffffff80d60000 ffffffff80d61000 (XEN) ffffffff80d62000 ffffffff80d63000 ffffffff80d64000 ffffffff80d65000 (XEN) ffffffff80d66000 ffffffff80d67000 ffffffff80d68000 ffffffff80d69000 (XEN) ffffffff80d6a000 ffffffff80d6b000 ffffffff80d6c000 ffffffff80d6d000 (XEN) ffffffff80d6e000 ffffffff80d6f000 ffffffff80d70000 ffffffff80d71000 (XEN) ffffffff80d72000 ffffffff80d73000 ffffffff80d74000 ffffffff80d75000 (XEN) ffffffff80d76000 ffffffff80d77000 ffffffff80d78000 ffffffff80d79000
Quoted 9 lines · Jeremy Fitzhardinge
Quoted 6 lines · Guillaume Rousse
Quoted 4 lines · Jeremy Fitzhardinge
Quoted 2 lines
FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it down further if that'd be useful to you.
Yes, please.
Was there any final conclusion there ? Because I'm currently trying a 2.6.27.5 with the same kind of issue:
I don't know of any outstanding bugs which cause this kind of symptom. What's happening on the Linux side when the domain crashes this way?
Immediate crash, without further information.
I've opened a but report on mandriva bugzilla:
https://qa.mandriva.com/show_bug.cgi?id=46470
And I've yet to try with vanilla linus kernel.
Signature · 3 lines
Guillaume Rousse Moyens Informatiques - INRIA Futurs Tel: 01 69 35 69 62
Quoted 6 lines · Guillaume Rousse
Quoted 5 lines · Jeremy Fitzhardinge
Quoted 4 lines · Guillaume Rousse
Quoted 3 lines · Jeremy Fitzhardinge
Quoted 2 lines
FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it down further if that'd be useful to you.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Quoted 1 line
Yes, please.
Was there any final conclusion there ? Because I'm currently trying a 2.6.27.5 with the same kind of issue:
I don't know of any outstanding bugs which cause this kind of symptom. What's happening on the Linux side when the domain crashes this way?
Immediate crash, without further information.
I've opened a but report on mandriva bugzilla:
https://qa.mandriva.com/show_bug.cgi?id=46470
Does earlyprintk=xen reveal anything more? What does the rip (ffffffff8020b3ed) map to in the kernel?
Quoted 1 line
And I've yet to try with vanilla linus kernel.
Please do. If possible, build a linus kernel with the mandriva config.
J
Quoted 10 lines · Jeremy Fitzhardinge
Quoted 9 lines · Guillaume Rousse
Quoted 8 lines · Jeremy Fitzhardinge
Quoted 6 lines · Guillaume Rousse
Quoted 4 lines · Jeremy Fitzhardinge
Quoted 2 lines
FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it down further if that'd be useful to you.
Yes, please.
Was there any final conclusion there ? Because I'm currently trying a 2.6.27.5 with the same kind of issue:
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
I don't know of any outstanding bugs which cause this kind of symptom. What's happening on the Linux side when the domain crashes this way?
Immediate crash, without further information.
I've opened a but report on mandriva bugzilla:
https://qa.mandriva.com/show_bug.cgi?id=46470Does earlyprintk=xen reveal anything more? What does the rip (
ffffffff8020b3ed) map to in the kernel?
I guess earlyprintk=xen is a dom0 option, because passing it to domU didn't change anything. The given adress map to set_page_prot, according to the System.map file.
BTW, it only happens with ISCSI-stored hosts, whereas it works OK with LVM-stored hosts (tough on slightly different hardware). On other hosts, the guest kernel boots OK, and then fails to mount underlying FS (but that's a different issue). I updated upstream report with details.
Signature · 4 lines
Guillaume Rousse Service des Moyens Informatiques INRIA Saclay - Ile de France Tel: 01 69 35 69 62
Quoted 11 lines · Guillaume Rousse
Quoted 10 lines · Jeremy Fitzhardinge
Quoted 9 lines · Guillaume Rousse
Quoted 8 lines · Jeremy Fitzhardinge
Quoted 6 lines · Guillaume Rousse
Quoted 4 lines · Jeremy Fitzhardinge
Quoted 2 lines
FWIW, I just tried 2.6.26-rc1 and it booted fine. I can narrow it down further if that'd be useful to you.
Yes, please.
Was there any final conclusion there ? Because I'm currently trying a 2.6.27.5 with the same kind of issue:
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
I don't know of any outstanding bugs which cause this kind of symptom. What's happening on the Linux side when the domain crashes this way?
Immediate crash, without further information.
I've opened a but report on mandriva bugzilla:
https://qa.mandriva.com/show_bug.cgi?id=46470Does earlyprintk=xen reveal anything more? What does the rip (
ffffffff8020b3ed) map to in the kernel?I guess earlyprintk=xen is a dom0 option, because passing it to domU didn't change anything.
It will be most helpful if you're running a debug=y build of Xen, so the domU's early console output appears on the Xen console.
Quoted 1 line
The given adress map to
set_page_prot, according to the System.map file.
That's interesting.
Quoted 1 line
BTW, it only happens with ISCSI-stored hosts, whereas it works OK with LVM-stored hosts (tough on slightly different hardware).
Uh, that's very mysterious. The underlying storage shouldn't have any effect on whether the kernel can boot - it's either loaded or not.
Quoted 1 line
On other hosts, the guest kernel boots OK, and then fails to mount underlying FS (but that's a different issue). I updated upstream report with details.
I'm on vacation at the moment, but I'll give this a closer look when I return.
J