Skip to contentSkip to conversation

New: Oops IP: [<ffffffffa0047081>] rmap_get_first+0x21/0x60

Bug 135471 [email protected] 1 participant 10 messages Jul 18, 2016 KVM

lore
Share & tools
lore
#0 lore raw

https://bugzilla.kernel.org/show_bug.cgi?id=135471

            Bug ID: 135471
           Summary: Oops IP: [<ffffffffa0047081>] rmap_get_first+0x21/0x60
                    [kvm]
           Product: Virtualization
           Version: unspecified
    Kernel Version: 4.6.3
          Hardware: i386
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: kvm
          Assignee: [email protected]
          Reporter: [email protected]
        Regression: No

Hello Devs.

Every now and then my machine crashes hard with the following message:

Jul 18 01:46:30 ovh kernel: BUG: unable to handle kernel NULL pointer
dereference at 0000000000000010
Jul 18 01:46:30 ovh kernel: IP: [<ffffffffa0047081>] rmap_get_first+0x21/0x60
[kvm]
Jul 18 01:46:30 ovh kernel: PGD 0 
Jul 18 01:46:30 ovh kernel: Oops: 0000 [#1] SMP 
Jul 18 01:46:30 ovh kernel: Modules linked in: vhost_net vhost xt_physdev
xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4
nf_nat tun xt_multiport coretemp kvm_intel kvm irqbypass i2c_i801
Jul 18 01:46:30 ovh kernel: CPU: 3 PID: 692 Comm: kswapd0 Not tainted 4.6.3 #1
Jul 18 01:46:30 ovh kernel: Hardware name: Supermicro X8STi/X8STi, BIOS 2.0    
   09/17/10  
Jul 18 01:46:30 ovh kernel: task: ffff88040ceb0000 ti: ffff88040c8c4000
task.ti: ffff88040c8c4000
Jul 18 01:46:30 ovh kernel: RIP: 0010:[<ffffffffa0047081>] 
[<ffffffffa0047081>] rmap_get_first+0x21/0x60 [kvm]
Jul 18 01:46:30 ovh kernel: RSP: 0018:ffff88040c8c7920  EFLAGS: 00010286
Jul 18 01:46:31 ovh kernel: RAX: 0000000000000010 RBX: 0000000000000000 RCX:
0000000000108e83
Jul 18 01:46:31 ovh kernel: RDX: ffffc90003872008 RSI: ffff88040c8c7930 RDI:
ffffc9000394b418
Jul 18 01:46:31 ovh kernel: RBP: ffff88040c8c7920 R08: 0000000000000001 R09:
0000000000000000
Jul 18 01:46:31 ovh kernel: R10: 0000000000000003 R11: 0000000000000002 R12:
ffffc9000394b418
Jul 18 01:46:31 ovh kernel: R13: ffff88040bb78000 R14: 0000000000000000 R15:
ffffc90003872008
Jul 18 01:46:31 ovh kernel: FS:  0000000000000000(0000)
GS:ffff88041fc60000(0000) knlGS:0000000000000000
Jul 18 01:46:31 ovh kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jul 18 01:46:31 ovh kernel: CR2: 0000000000000010 CR3: 0000000001c07000 CR4:
00000000000026e0
Jul 18 01:46:31 ovh kernel: Stack:
Jul 18 01:46:31 ovh kernel:  ffff88040c8c7958 ffffffffa004ae73 ffff88040bb5b9c0
00007fe400000000
Jul 18 01:46:31 ovh kernel:  0000000000000000 ffff88040bb78000 ffffffffa004ae90
ffff88040c8c7968
Jul 18 01:46:31 ovh kernel:  ffffffffa004ae99 ffff88040c8c7a08 ffffffffa0046b93
ffff88040bb78048
Jul 18 01:46:31 ovh kernel: Call Trace:
Jul 18 01:46:31 ovh kernel:  [<ffffffffa004ae73>] kvm_zap_rmapp+0x33/0x50 [kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffffa004ae90>] ? kvm_zap_rmapp+0x50/0x50
[kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffffa004ae99>] kvm_unmap_rmapp+0x9/0x10
[kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffffa0046b93>]
kvm_handle_hva_range+0x113/0x170 [kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffffa00516d4>] kvm_age_hva+0x34/0x40 [kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffffa002cbc8>]
kvm_mmu_notifier_clear_flush_young+0x48/0x80 [kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffff8115f866>]
__mmu_notifier_clear_flush_young+0x56/0x80
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114ab54>] page_referenced_one+0xe4/0xf0
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114a301>] rmap_walk_anon+0xc1/0x1d0
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114bde7>] rmap_walk+0x27/0x30
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114becd>] page_referenced+0xdd/0x1a0
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114aa70>] ?
__page_check_address+0x180/0x180
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114b570>] ?
page_get_anon_vma+0x120/0x120
Jul 18 01:46:31 ovh kernel:  [<ffffffff8112a669>]
shrink_active_list+0x1c9/0x390
Jul 18 01:46:31 ovh kernel:  [<ffffffff8112ae0b>] shrink_zone_memcg+0x5db/0x7d0
Jul 18 01:46:31 ovh kernel:  [<ffffffff8112b0c6>] shrink_zone+0xc6/0x1f0
Jul 18 01:46:31 ovh kernel:  [<ffffffff8112bf5a>] kswapd+0x43a/0x840
Jul 18 01:46:31 ovh kernel:  [<ffffffff8112bb20>] ?
mem_cgroup_shrink_node_zone+0x150/0x150
Jul 18 01:46:31 ovh kernel:  [<ffffffff81072fb4>] kthread+0xc4/0xe0
Jul 18 01:46:31 ovh kernel:  [<ffffffff81740292>] ret_from_fork+0x22/0x40
Jul 18 01:46:31 ovh kernel:  [<ffffffff81072ef0>] ? kthread_park+0x50/0x50
Jul 18 01:46:31 ovh kernel: Code: 00 00 eb c4 66 0f 1f 44 00 00 48 8b 07 55 48
89 e5 48 85 c0 74 42 a8 01 74 2a 48 83 e0 fe c7 46 08 00 00 00 00 48 89 06 48
8b 00 <48> 8b 10 f6 c2 01 74 23 48 8b 0d b0 dc 03 00 48 21 ca 48 39 d1 
Jul 18 01:46:31 ovh kernel: RIP  [<ffffffffa0047081>] rmap_get_first+0x21/0x60
[kvm]
Jul 18 01:46:31 ovh kernel:  RSP <ffff88040c8c7920>
Jul 18 01:46:31 ovh kernel: CR2: 0000000000000010
Jul 18 01:46:31 ovh kernel: ---[ end trace f30608d1658423c5 ]---
Jul 18 01:46:31 ovh kernel: ------------[ cut here ]------------
Jul 18 01:46:31 ovh kernel: WARNING: CPU: 3 PID: 692 at kernel/exit.c:661
do_exit+0x4e/0xbf0
Jul 18 01:46:31 ovh kernel: Modules linked in: vhost_net vhost xt_physdev
xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4
nf_nat tun xt_multiport coretemp kvm_intel kvm irqbypass i2c_i801
Jul 18 01:46:31 ovh kernel: CPU: 3 PID: 692 Comm: kswapd0 Tainted: G      D    
    4.6.3 #1
Jul 18 01:46:31 ovh kernel: Hardware name: Supermicro X8STi/X8STi, BIOS 2.0    
   09/17/10  
Jul 18 01:46:31 ovh kernel:  0000000000000000 ffff88040c8c7630 ffffffff813191c8
0000000000000000
Jul 18 01:46:31 ovh kernel:  0000000000000000 ffff88040c8c7670 ffffffff81056cfc
0000029500000009
Jul 18 01:46:31 ovh kernel:  ffff88040ceb0000 0000000000000009 0000000000000046
ffff88040ceb0000
Jul 18 01:46:31 ovh kernel: Call Trace:
Jul 18 01:46:31 ovh kernel:  [<ffffffff813191c8>] dump_stack+0x4d/0x65
Jul 18 01:46:31 ovh kernel:  [<ffffffff81056cfc>] __warn+0xcc/0xf0
Jul 18 01:46:31 ovh kernel:  [<ffffffff81056dd8>] warn_slowpath_null+0x18/0x20
Jul 18 01:46:31 ovh kernel:  [<ffffffff81059fce>] do_exit+0x4e/0xbf0
Jul 18 01:46:31 ovh kernel:  [<ffffffff8101e6e4>] oops_end+0x84/0xc0
Jul 18 01:46:31 ovh kernel:  [<ffffffff81049b49>] no_context+0x109/0x350
Jul 18 01:46:31 ovh kernel:  [<ffffffff81049e01>]
__bad_area_nosemaphore+0x71/0x1f0
Jul 18 01:46:31 ovh kernel:  [<ffffffff81049f8f>] bad_area_nosemaphore+0xf/0x20
Jul 18 01:46:31 ovh kernel:  [<ffffffff8104a2a4>] __do_page_fault+0x84/0x490
Jul 18 01:46:31 ovh kernel:  [<ffffffffa00462ee>] ?
mmu_spte_clear_track_bits+0xee/0x100 [kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffff8104a6d3>] do_page_fault+0x23/0x80
Jul 18 01:46:31 ovh kernel:  [<ffffffff81741ce2>] page_fault+0x22/0x30
Jul 18 01:46:31 ovh kernel:  [<ffffffffa0047081>] ? rmap_get_first+0x21/0x60
[kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffffa004ae73>] kvm_zap_rmapp+0x33/0x50 [kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffffa004ae90>] ? kvm_zap_rmapp+0x50/0x50
[kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffffa004ae99>] kvm_unmap_rmapp+0x9/0x10
[kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffffa0046b93>]
kvm_handle_hva_range+0x113/0x170 [kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffffa00516d4>] kvm_age_hva+0x34/0x40 [kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffffa002cbc8>]
kvm_mmu_notifier_clear_flush_young+0x48/0x80 [kvm]
Jul 18 01:46:31 ovh kernel:  [<ffffffff8115f866>]
__mmu_notifier_clear_flush_young+0x56/0x80
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114ab54>] page_referenced_one+0xe4/0xf0
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114a301>] rmap_walk_anon+0xc1/0x1d0
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114bde7>] rmap_walk+0x27/0x30
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114becd>] page_referenced+0xdd/0x1a0
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114aa70>] ?
__page_check_address+0x180/0x180
Jul 18 01:46:31 ovh kernel:  [<ffffffff8114b570>] ?
page_get_anon_vma+0x120/0x120
Jul 18 01:46:31 ovh kernel:  [<ffffffff8112a669>]
shrink_active_list+0x1c9/0x390
Jul 18 01:46:31 ovh kernel:  [<ffffffff8112ae0b>] shrink_zone_memcg+0x5db/0x7d0
Jul 18 01:46:31 ovh kernel:  [<ffffffff8112b0c6>] shrink_zone+0xc6/0x1f0
Jul 18 01:46:31 ovh kernel:  [<ffffffff8112bf5a>] kswapd+0x43a/0x840
Jul 18 01:46:31 ovh kernel:  [<ffffffff8112bb20>] ?
mem_cgroup_shrink_node_zone+0x150/0x150
Jul 18 01:46:31 ovh kernel:  [<ffffffff81072fb4>] kthread+0xc4/0xe0
Jul 18 01:46:31 ovh kernel:  [<ffffffff81740292>] ret_from_fork+0x22/0x40
Jul 18 01:46:31 ovh kernel:  [<ffffffff81072ef0>] ? kthread_park+0x50/0x50
Jul 18 01:46:31 ovh kernel: ---[ end trace f30608d1658423c6 ]---

Is there a way to tell what the reason of that could be?

thanks and cheers
t.

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
#1 lore raw

[Bug 135471] Oops IP: [<ffffffffa0047081>] rmap_get_first+0x21/0x60

https://bugzilla.kernel.org/show_bug.cgi?id=135471

Wanpeng Li <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #1 from Wanpeng Li <[email protected]> --- How to reproduce it?

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
#3 lore raw

[Bug 135471] Oops IP: [<ffffffffa0047081>] rmap_get_first+0x21/0x60

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=135471

Paolo Bonzini <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #3 from Paolo Bonzini <[email protected]> ---
Have you seen it with kernels before 4.6.3? Also please include the output of

grep -r . /sys/module/kvm*/parameters

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
#5 lore raw

[Bug 135471] Oops IP: [<ffffffffa0047081>] rmap_get_first+0x21/0x60

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=135471

--- Comment #4 from Thomas Stein <[email protected]> --- Hello.

I started running this machine with 4.6.1. The problem occurs since the beginning.

ovh ~ # grep -r . /sys/module/kvm*/parameters /sys/module/kvm_intel/parameters/ept:Y /sys/module/kvm_intel/parameters/pml:N /sys/module/kvm_intel/parameters/vpid:Y /sys/module/kvm_intel/parameters/enable_shadow_vmcs:N /sys/module/kvm_intel/parameters/eptad:N /sys/module/kvm_intel/parameters/flexpriority:Y /sys/module/kvm_intel/parameters/ple_window_grow:2 /sys/module/kvm_intel/parameters/ple_window_shrink:0 /sys/module/kvm_intel/parameters/ple_window_max:1073741823 /sys/module/kvm_intel/parameters/fasteoi:Y /sys/module/kvm_intel/parameters/enable_apicv:N /sys/module/kvm_intel/parameters/unrestricted_guest:N /sys/module/kvm_intel/parameters/emulate_invalid_guest_state:Y /sys/module/kvm_intel/parameters/nested:Y /sys/module/kvm_intel/parameters/vmm_exclusive:Y /sys/module/kvm_intel/parameters/ple_window:4096 /sys/module/kvm_intel/parameters/ple_gap:0 /sys/module/kvm/parameters/tsc_tolerance_ppm:250 /sys/module/kvm/parameters/halt_poll_ns:400000 /sys/module/kvm/parameters/min_timer_period_us:500 /sys/module/kvm/parameters/halt_poll_ns_shrink:0 /sys/module/kvm/parameters/vector_hashing:Y /sys/module/kvm/parameters/mmu_audit:N /sys/module/kvm/parameters/kvmclock_periodic_sync:Y /sys/module/kvm/parameters/ignore_msrs:N /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts:N /sys/module/kvm/parameters/halt_poll_ns_grow:2 /sys/module/kvm/parameters/lapic_timer_advance_ns:0 ovh ~ #

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
#6 lore raw

[Bug 135471] Oops IP: [<ffffffffa0047081>] rmap_get_first+0x21/0x60

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=135471

--- Comment #5 from Paolo Bonzini <[email protected]> ---
Is there any chance you can try running a 4.5 or 4.4 kernel for a while? Also, please include the configuration of the virtual machine (e.g. qemu command line from "ps").

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
#7 lore raw

[Bug 135471] Oops IP: [<ffffffffa0047081>] rmap_get_first+0x21/0x60

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=135471

--- Comment #6 from Thomas Stein <[email protected]> --- I can try to downgrade to 4.4, yes. Here the two running vm's.

qemu 2467 29.4 22.3 7014104 3674892 ? Sl 11:10 56:23
/usr/bin/qemu-system-x86_64 -name domain1,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-domain1/master-key.aes -machine pc-i440fx-2.6,accel=kvm,usb=off -cpu Nehalem,+rdtscp,+dca,+pdcm,+xtpr,+tm2,+est,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid fe47b4f7-bde6-4d8e-80a3-34c184e38fdf -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-1-domain1/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
-boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive file=/var/lib/libvirt/images/domain1.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=writeback,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=20,id=hostnet0,vhost=on,vhostfd=22 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:7d:28:15,bus=pci.0,addr=0x3
-chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0
-chardev
socket,id=charchannel0,path=/var/lib/libvirt/qemu/f16x86_64.agent,server,nowait
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -vnc 127.0.0.1:0 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -msg timestamp=on

qemu 2703 12.8 13.3 6316328 2194004 ? Sl 11:10 24:38
/usr/bin/qemu-system-x86_64 -name domain2,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-domain2/master-key.aes -machine pc-q35-2.4,accel=kvm,usb=off -cpu Nehalem,+rdtscp,+dca,+pdcm,+xtpr,+tm2,+est,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid c71f66b3-2fd1-42f0-b8be-42999857eb95 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-2-domain2/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
-boot strict=on -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e -device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x1 -device ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,addr=0x1d
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2
-device lsi,id=scsi0,bus=pci.2,addr=0x3 -device virtio-serial-pci,id=virtio-serial0,bus=pci.2,addr=0x2 -drive file=/var/lib/libvirt/images/domain2.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=writethrough,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.2,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=21,id=hostnet0,vhost=on,vhostfd=23 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:18:75:ef,bus=pci.2,addr=0x1
-chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0
-chardev
socket,id=charchannel0,path=/var/lib/libvirt/qemu/f16x86_64.agent,server,nowait
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -vnc 127.0.0.1:1 -device cirrus-vga,id=video0,bus=pcie.0,addr=0x1 -device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x5 -msg timestamp=on

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.