Skip to contentSkip to conversation

2.6.29-rc3 oops with kvmclock

DISCUSSION Mark McLoughlin 3 participants 5 messages Feb 3, 2009 – Feb 9, 2009 KVM

lore
Share & tools
lore
#0 lore raw

Hi,

Same guest, this time a kvmclock related oops.

Cheers,
Mark.

kernel BUG at include/linux/mm.h:302!
invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
last sysfs file: /sys/devices/pci0000:00/0000:00:01.1/host1/target1:0:0/1:0:0:0/ type
CPU 1

Modules linked in: e1000e(+) virtio_pci virtio_ring virtio iscsi_ibft iscsi_tcp 
libiscsi_tcp libiscsi scsi_transport_iscsi ext2 ext4 jbd2 crc16 squashfs pcspkr 
floppy nfs lockd nfs_acl auth_rpcgss sunrpc vfat fat cramfs
Pid: 679, comm: udevd Not tainted 2.6.29-0.53.rc2.git1.fc11.x86_64 #1
RIP: 0010:[<ffffffff810bb332>]  [<ffffffff810bb332>] do_wp_page+0x324/0x6d8
RSP: 0000:ffff88003826da88  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff880038264d60 RCX: ffff880038264d60
RDX: ffffe200016d9230 RSI: 00000000019ac7b0 RDI: 8000000000000065
RBP: ffff88003826daf8 R08: ffff88003825d060 R09: ffffe200016cf8b0
R10: ffff8800381f1648 R11: ffffffff81385e94 R12: ffffe200016d9230
R13: 80000000383de065 R14: ffff8800381f1648 R15: ffffe200016cf8b0
FS:  00007fcc71dde790(0000) GS:ffff88003b44c3e8(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000019ac7b0 CR3: 000000003825b000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process udevd (pid: 679, threadinfo ffff88003826c000, task ffff88003877c720)
Stack:
 0000000000000246 ffff88003825d060 ffff880038264d60 00000000019ac7b0
 ffff8800381f1648 ffff880038228540 ffff8800381f1648 00000000019ac7b0
 ffff88003826daf8 ffff880038264d60 ffffe200016cf8b0 80000000383de065
Call Trace:
 [<ffffffff810bd1e5>] handle_mm_fault+0x7d7/0x88b
 [<ffffffff81385e94>] ? do_page_fault+0x58a/0xa35
 [<ffffffff81385f40>] do_page_fault+0x636/0xa35
 [<ffffffff812da338>] ? sock_recvmsg+0xca/0xe3
 [<ffffffff8105f090>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff810ba0c3>] ? might_fault+0xaf/0xb1
 [<ffffffff810ba071>] ? might_fault+0x5d/0xb1
 [<ffffffff810ebad0>] ? set_fd_set+0x39/0x43
 [<ffffffff812e28e6>] ? verify_iovec+0x4f/0x91
 [<ffffffff812db319>] ? sys_recvmsg+0x142/0x217
 [<ffffffff8102a24b>] ? kvm_clock_read+0x1c/0x1e
 [<ffffffff81016ea3>] ? sched_clock+0x9/0xc
 [<ffffffff81018b28>] ? restore_i387_xstate+0x6e/0x17a
 [<ffffffff81383002>] ? _spin_unlock_irq+0x2b/0x37
 [<ffffffff8106d719>] ? trace_hardirqs_on_caller+0x12f/0x153
 [<ffffffff8102ad8c>] ? pvclock_clocksource_read+0x42/0x7e
 [<ffffffff81383be0>] ? error_sti+0x5/0x6
 [<ffffffff81382c33>] ? trace_hardirqs_off_thunk+0x3a/0x3c
 [<ffffffff813839a5>] page_fault+0x25/0x30
Code: 48 89 c1 41 bd 08 00 00 00 e8 81 95 f7 ff e9 43 03 00 00 49 8b 04 24 4c 89
 e2 f6 c4 40 74 05 49 8b 54 24 10 8b 42 08 85 c0 75 04 <0f> 0b eb fe 48 8d 42 08
 f0 ff 42 08 4c 89 ff e8 1f 7d 2c 00 48 
RIP  [<ffffffff810bb332>] do_wp_page+0x324/0x6d8
 RSP <ffff88003826da88>
#1 lore raw
Quoted 3 lines · Mark McLoughlin

Hi,

Same guest, this time a kvmclock related oops.

How do you know that? There is kvmclock related functions in the trace as could be
any clock related function. The problem smells like bad mmu to me at a first glance.

Quoted 61 lines

Cheers,
Mark.

kernel BUG at include/linux/mm.h:302!
invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
last sysfs file: /sys/devices/pci0000:00/0000:00:01.1/host1/target1:0:0/1:0:0:0/ type
CPU 1

Modules linked in: e1000e(+) virtio_pci virtio_ring virtio iscsi_ibft iscsi_tcp
libiscsi_tcp libiscsi scsi_transport_iscsi ext2 ext4 jbd2 crc16 squashfs pcspkr
floppy nfs lockd nfs_acl auth_rpcgss sunrpc vfat fat cramfs
Pid: 679, comm: udevd Not tainted 2.6.29-0.53.rc2.git1.fc11.x86_64 #1
RIP: 0010:[<ffffffff810bb332>]  [<ffffffff810bb332>] do_wp_page+0x324/0x6d8
RSP: 0000:ffff88003826da88  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff880038264d60 RCX: ffff880038264d60
RDX: ffffe200016d9230 RSI: 00000000019ac7b0 RDI: 8000000000000065
RBP: ffff88003826daf8 R08: ffff88003825d060 R09: ffffe200016cf8b0
R10: ffff8800381f1648 R11: ffffffff81385e94 R12: ffffe200016d9230
R13: 80000000383de065 R14: ffff8800381f1648 R15: ffffe200016cf8b0
FS:  00007fcc71dde790(0000) GS:ffff88003b44c3e8(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000019ac7b0 CR3: 000000003825b000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process udevd (pid: 679, threadinfo ffff88003826c000, task ffff88003877c720)
Stack:
 0000000000000246 ffff88003825d060 ffff880038264d60 00000000019ac7b0
 ffff8800381f1648 ffff880038228540 ffff8800381f1648 00000000019ac7b0
 ffff88003826daf8 ffff880038264d60 ffffe200016cf8b0 80000000383de065
Call Trace:
 [<ffffffff810bd1e5>] handle_mm_fault+0x7d7/0x88b
 [<ffffffff81385e94>] ? do_page_fault+0x58a/0xa35
 [<ffffffff81385f40>] do_page_fault+0x636/0xa35
 [<ffffffff812da338>] ? sock_recvmsg+0xca/0xe3
 [<ffffffff8105f090>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff810ba0c3>] ? might_fault+0xaf/0xb1
 [<ffffffff810ba071>] ? might_fault+0x5d/0xb1
 [<ffffffff810ebad0>] ? set_fd_set+0x39/0x43
 [<ffffffff812e28e6>] ? verify_iovec+0x4f/0x91
 [<ffffffff812db319>] ? sys_recvmsg+0x142/0x217
 [<ffffffff8102a24b>] ? kvm_clock_read+0x1c/0x1e
 [<ffffffff81016ea3>] ? sched_clock+0x9/0xc
 [<ffffffff81018b28>] ? restore_i387_xstate+0x6e/0x17a
 [<ffffffff81383002>] ? _spin_unlock_irq+0x2b/0x37
 [<ffffffff8106d719>] ? trace_hardirqs_on_caller+0x12f/0x153
 [<ffffffff8102ad8c>] ? pvclock_clocksource_read+0x42/0x7e
 [<ffffffff81383be0>] ? error_sti+0x5/0x6
 [<ffffffff81382c33>] ? trace_hardirqs_off_thunk+0x3a/0x3c
 [<ffffffff813839a5>] page_fault+0x25/0x30
Code: 48 89 c1 41 bd 08 00 00 00 e8 81 95 f7 ff e9 43 03 00 00 49 8b 04 24 4c 89
 e2 f6 c4 40 74 05 49 8b 54 24 10 8b 42 08 85 c0 75 04 <0f> 0b eb fe 48 8d 42 08
 f0 ff 42 08 4c 89 ff e8 1f 7d 2c 00 48
RIP  [<ffffffff810bb332>] do_wp_page+0x324/0x6d8
 RSP <ffff88003826da88>

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 5 lines
Glauber  Costa.
"Free as in Freedom"
http://glommer.net

"The less confident you are, the more serious you have to act."
#2 lore raw
Quoted 8 lines · Glauber Costa
Quoted 3 lines · Mark McLoughlin

Hi,

Same guest, this time a kvmclock related oops.

How do you know that? There is kvmclock related functions in the trace as could be
any clock related function.

Yep, that's all I meant - "kvmclock is in the trace".

Quoted 1 line

The problem smells like bad mmu to me at a first glance.

Sure.

Cheers,
Mark.

#3 lore raw
In reply to Mark McLoughlin

Hi Mark,

Quoted 8 lines · Mark McLoughlin

Hi,

Same guest, this time a kvmclock related oops.

Cheers,
Mark.

kernel BUG at include/linux/mm.h:302!

static inline void get_page(struct page *page)

{
        page = compound_head(page);
        VM_BUG_ON(atomic_read(&page->_count) == 0);
        atomic_inc(&page->_count);
}
Quoted 3 lines

RIP [<ffffffff810bb332>] do_wp_page+0x324/0x6d8

From do_wp_page (COW pagefault handler). So the page was freed (or

thats how do_wp_page perceives it) while expected not to be (probably because the page is pinned by the read-only pte reference, by the time do_wp_page starts).

Quoted 1 line

invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC

The FC11 kernel has pvmmu on. It _could_ be a weird side effect of the pvmmu slab bug (which only happens with DEBUG_PAGEALLOC). What were you doing when this happened? Reproducible?

Or some other corruption that causes the crazy symptom.

#4 lore raw
Quoted 5 lines · Marcelo Tosatti
Quoted 1 line

invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC

The FC11 kernel has pvmmu on. It _could_ be a weird side effect of the pvmmu slab bug (which only happens with DEBUG_PAGEALLOC). What were you doing when this happened? Reproducible?

I was just running a guest with an assigned PCI device. I haven't seen it again since with similar testing.

Cheers,
Mark.