Skip to contentSkip to conversation

qemu-kvm loops after kernel udpate

PATCH New Jiri Slaby 6 participants 16 messages Sep 11, 2012 – Sep 19, 2012

Subsystems (3)
lore
Share & tools
lore

Jiri Slaby

author
#0 lore raw

Hi,

it looks like an update from next-20120824 to next-20120910 makes kvm defunct. When I try to run qemu, it loops forever without printing anything on the monitor.

-no-kvm makes it indeed work.

Cmdline I use:
qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m 1000M -net user -net nic,model=e1000 -usb -serial pty

Before I start investigating further, like biscection, is this a known issue already and fix available somewhere?

thanks,

Signature · 2 lines
js
suse labs
#1 lore raw
Quoted 8 lines · Jiri Slaby

Hi,

it looks like an update from next-20120824 to next-20120910 makes kvm defunct. When I try to run qemu, it loops forever without printing anything on the monitor.

-no-kvm makes it indeed work.

Quoted 8 lines

Cmdline I use:
qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m 1000M -net user -net nic,model=e1000 -usb -serial pty

Before I start investigating further, like biscection, is this a known issue already and fix available somewhere?

thanks,

Its not a known issue. 'info registers' (for both vcpus) and 'x /20i $eip' might help track it down.

author
#2 lore raw
Quoted 13 lines · Marcelo Tosatti
Quoted 12 lines · Jiri Slaby

Hi,

it looks like an update from next-20120824 to next-20120910 makes kvm defunct. When I try to run qemu, it loops forever without printing anything on the monitor.

-no-kvm makes it indeed work.

Cmdline I use:
qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m 1000M -net user -net nic,model=e1000 -usb -serial pty

Quoted 5 lines
Quoted 2 lines

Before I start investigating further, like biscection, is this a known issue already and fix available somewhere?

Its not a known issue. 'info registers' (for both vcpus) and 'x /20i $eip' might help track it down.

I don't think it's in a state with something loaded.

(qemu) cpu 0
(qemu) info registers
EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623 ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 EIP=0000fff0 EFL=00010002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =0000 00000000 0000ffff 00009300
CS =f000 000f0000 0000ffff 0000f300
SS =0000 00000000 0000ffff 0000f300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT= 00000000 0000ffff
IDT= 00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80 FPR0=0000000000000000 0000 FPR1=0000000000000000 0000 FPR2=0000000000000000 0000 FPR3=0000000000000000 0000 FPR4=0000000000000000 0000 FPR5=0000000000000000 0000 FPR6=0000000000000000 0000 FPR7=0000000000000000 0000 XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000 XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000 XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000 XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
(qemu) cpu 1
(qemu) info registers
EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623 ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 EIP=0000fff0 EFL=00000002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =0000 00000000 0000ffff 00009300
CS =f000 000f0000 0000ffff 0000f300
SS =0000 00000000 0000ffff 0000f300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT= 00000000 0000ffff
IDT= 00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80 FPR0=0000000000000000 0000 FPR1=0000000000000000 0000 FPR2=0000000000000000 0000 FPR3=0000000000000000 0000 FPR4=0000000000000000 0000 FPR5=0000000000000000 0000 FPR6=0000000000000000 0000 FPR7=0000000000000000 0000 XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000 XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000 XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000 XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000

(qemu) x /20i $eip

0x000000000000fff0:  add    %al,(%bx,%si)
0x000000000000fff2:  add    %al,(%bx,%si)
0x000000000000fff4:  add    %al,(%bx,%si)
0x000000000000fff6:  add    %al,(%bx,%si)
0x000000000000fff8:  add    %al,(%bx,%si)
0x000000000000fffa:  add    %al,(%bx,%si)
0x000000000000fffc:  add    %al,(%bx,%si)
0x000000000000fffe:  add    %al,(%bx,%si)
0x0000000000010000:  add    %al,(%bx,%si)
0x0000000000010002:  add    %al,(%bx,%si)
0x0000000000010004:  add    %al,(%bx,%si)
0x0000000000010006:  add    %al,(%bx,%si)
0x0000000000010008:  add    %al,(%bx,%si)
0x000000000001000a:  add    %al,(%bx,%si)
0x000000000001000c:  add    %al,(%bx,%si)
0x000000000001000e:  add    %al,(%bx,%si)
0x0000000000010010:  add    %al,(%bx,%si)
0x0000000000010012:  add    %al,(%bx,%si)
0x0000000000010014:  add    %al,(%bx,%si)
0x0000000000010016:  add    %al,(%bx,%si)

thanks,

Signature · 2 lines
js
suse labs
#3 lore raw
Quoted 22 lines · Jiri Slaby
Quoted 13 lines · Marcelo Tosatti
Quoted 12 lines · Jiri Slaby

Hi,

it looks like an update from next-20120824 to next-20120910 makes kvm defunct. When I try to run qemu, it loops forever without printing anything on the monitor.

-no-kvm makes it indeed work.

Cmdline I use:
qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m 1000M -net user -net nic,model=e1000 -usb -serial pty

Quoted 5 lines
Quoted 2 lines

Before I start investigating further, like biscection, is this a known issue already and fix available somewhere?

Its not a known issue. 'info registers' (for both vcpus) and 'x /20i $eip' might help track it down.

I don't think it's in a state with something loaded.

Quoted 7 lines

(qemu) cpu 0
(qemu) info registers
EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623 ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 EIP=0000fff0 EFL=00010002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =0000 00000000 0000ffff 00009300
CS =f000 000f0000 0000ffff 0000f300

This is before the first instruction is executed.

You're on an Intel host, yes? Please provide the output of vmxcap
(http://goo.gl/c5lUO), and a snapshot of kvm_stat while the guest is hung.

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#4 lore raw
Quoted 31 lines · Avi Kivity
Quoted 22 lines · Jiri Slaby
Quoted 13 lines · Marcelo Tosatti
Quoted 12 lines · Jiri Slaby

Hi,

it looks like an update from next-20120824 to next-20120910 makes kvm defunct. When I try to run qemu, it loops forever without printing anything on the monitor.

-no-kvm makes it indeed work.

Cmdline I use:
qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m 1000M -net user -net nic,model=e1000 -usb -serial pty

Quoted 5 lines
Quoted 2 lines

Before I start investigating further, like biscection, is this a known issue already and fix available somewhere?

Its not a known issue. 'info registers' (for both vcpus) and 'x /20i $eip' might help track it down.

I don't think it's in a state with something loaded.

Quoted 7 lines

(qemu) cpu 0
(qemu) info registers
EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623 ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 EIP=0000fff0 EFL=00010002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =0000 00000000 0000ffff 00009300
CS =f000 000f0000 0000ffff 0000f300

Quoted 3 lines

This is before the first instruction is executed.

You're on an Intel host, yes?

Yes.

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 Duo CPU E6850 @ 3.00GHz
stepping : 11
microcode : 0xba
...
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts nopl aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm lahf_lm dtherm tpr_shadow vnmi flexpriority

Quoted 2 lines

Please provide the output of vmxcap
(http://goo.gl/c5lUO),

pin-based controls

  External interrupt exiting               yes
  NMI exiting                              yes
  Virtual NMIs                             yes
  Activate VMX-preemption timer            no
primary processor-based controls
  Interrupt window exiting                 yes
  Use TSC offsetting                       yes
  HLT exiting                              yes
  INVLPG exiting                           yes
  MWAIT exiting                            yes
  RDPMC exiting                            yes
  RDTSC exiting                            yes
  CR3-load exiting                         forced
  CR3-store exiting                        forced
  CR8-load exiting                         yes
  CR8-store exiting                        yes
  Use TPR shadow                           yes
  NMI-window exiting                       yes
  MOV-DR exiting                           yes
  Unconditional I/O exiting                yes
  Use I/O bitmaps                          yes
  Monitor trap flag                        no
  Use MSR bitmaps                          yes
  MONITOR exiting                          yes
  PAUSE exiting                            yes
  Activate secondary control               yes
secondary processor-based controls
  Virtualize APIC accesses                 yes
  Enable EPT                               no
  Descriptor-table exiting                 no
  Virtualize x2APIC mode                   no
  Enable VPID                              no
  WBINVD exiting                           no
  Unrestricted guest                       no
  PAUSE-loop exiting                       no
  RDRAND exiting                           no
  Enable INVPCID                           no
  Enable VM functions                      no
VM-Exit controls
  Save debug controls                      forced
  Host address-space size                  yes
  Load IA32_PERF_GLOBAL_CTRL               no
  Acknowledge interrupt on exit            yes
  Save IA32_PAT                            no
  Load IA32_PAT                            no
  Save IA32_EFER                           no
  Load IA32_EFER                           no
  Save VMX-preemption timer value          no
VM-Entry controls
  Load debug controls                      forced
  IA-64 mode guest                         yes
  Entry to SMM                             yes
  Deactivate dual-monitor treatment        yes
  Load IA32_PERF_GLOBAL_CTRL               no
  Load IA32_PAT                            no
  Load IA32_EFER                           no
Miscellaneous data
  VMX-preemption timer scale (log2)        0
  Store EFER.LMA into IA-32e mode guest control no
  HLT activity state                       yes
  Shutdown activity state                  yes
  Wait-for-SIPI activity state             yes
  Number of CR3-target values              4
  MSR-load/store count recommenation       0
  IA32_SMM_MONITOR_CTL[2] can be set to 1  no
  MSEG revision identifier                 0
VPID and EPT capabilities
  Execute-only EPT translations            no
  Page-walk length 4                       no
  Paging-structure memory type UC          no
  Paging-structure memory type WB          no
  2MB EPT pages                            no
  1GB EPT pages                            no
  INVEPT supported                         no
  EPT accessed and dirty flags             no
  Single-context INVEPT                    no
  All-context INVEPT                       no
  INVVPID supported                        no
  Individual-address INVVPID               no
  Single-context INVVPID                   no
  All-context INVVPID                      no

Single-context-retaining-globals INVVPID no
VM Functions
EPTP Switching no

Quoted 1 line

and a snapshot of kvm_stat while the guest is hung.

kvm statistics

exits 6778198 615942

 host_state_reload                             1988     187
 irq_exits                                     1523     138
 mmu_cache_miss                                   4       0
 fpu_reload                                       1       0

thanks,

Signature · 2 lines
js
suse labs
#5 lore raw
Quoted 5 lines · Jiri Slaby
Quoted 3 lines

Please provide the output of vmxcap
(http://goo.gl/c5lUO),

Unrestricted guest no

The big real mode fixes.

Quoted 10 lines
Quoted 2 lines

and a snapshot of kvm_stat while the guest is hung.

kvm statistics

exits 6778198 615942

 host_state_reload                             1988     187
 irq_exits                                     1523     138
 mmu_cache_miss                                   4       0
 fpu_reload                                       1       0

Please run this as root so we get the tracepoint based output; and press 'x' when it's running so we get more detailed output.

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#6 lore raw
Quoted 11 lines · Avi Kivity
Quoted 7 lines · Jiri Slaby

kvm statistics

exits 6778198 615942

 host_state_reload                             1988     187
 irq_exits                                     1523     138
 mmu_cache_miss                                   4       0
 fpu_reload                                       1       0

Please run this as root so we get the tracepoint based output; and press 'x' when it's running so we get more detailed output.

I need to build a kernel with trace points enabled first... I will return later today.

thanks,

Signature · 2 lines
js
suse labs
author
#7 lore raw
In reply to Avi Kivity
Quoted 15 lines · Avi Kivity
Quoted 5 lines · Jiri Slaby
Quoted 3 lines

Please provide the output of vmxcap
(http://goo.gl/c5lUO),

Unrestricted guest no

The big real mode fixes.

Quoted 5 lines
Quoted 2 lines

and a snapshot of kvm_stat while the guest is hung.

kvm statistics

Quoted 8 lines
Quoted 5 lines

exits 6778198 615942

 host_state_reload                             1988     187
 irq_exits                                     1523     138
 mmu_cache_miss                                   4       0
 fpu_reload                                       1       0

Please run this as root so we get the tracepoint based output; and press 'x' when it's running so we get more detailed output.

kvm statistics

 kvm_exit                                  13798699  330708
 kvm_entry                                 13799110  330708
 kvm_page_fault                            13793650  330604
 kvm_exit(EXCEPTION_NMI)                    6188458  330604
 kvm_exit(EXTERNAL_INTERRUPT)                  2169     105
 kvm_exit(TPR_BELOW_THRESHOLD)                   82       0
 kvm_exit(IO_INSTRUCTION)                         6       0
 kvm_exit(PAUSE_INSTRUCTION)                      5       0
 kvm_exit(MCE_DURING_VMENTRY)                     5       0
 kvm_exit(DR_ACCESS)                              5       0
 kvm_exit(VMPTRLD)                                5       0
 kvm_exit(VMLAUNCH)                               5       0
 kvm_exit(INVLPG)                                 5       0
 kvm_exit(VMCALL)                                 5       0
 kvm_exit(WBINVD)                                 5       0
 kvm_exit(MONITOR_INSTRUCTION)                    5       0
 kvm_exit(MWAIT_INSTRUCTION)                      4       0
 kvm_exit(NMI_WINDOW)                             4       0
 kvm_exit(VMREAD)                                 4       0
 kvm_exit(XSETBV)                                 4       0
 kvm_exit(RDPMC)                                  4       0
 kvm_exit(CPUID)                                  4       0
 kvm_exit(VMCLEAR)                                4       0
 kvm_exit(PENDING_INTERRUPT)                      4       0
 kvm_exit(VMRESUME)                               4       0
 kvm_exit(APIC_ACCESS)                            4       0
 kvm_exit(INVALID_STATE)                          4       0
 kvm_exit(MSR_READ)                               4       0
 kvm_exit(VMOFF)                                  4       0
 kvm_exit(RDTSC)                                  4       0
 kvm_exit(MSR_WRITE)                              4       0
 kvm_exit(VMPTRST)                                4       0
 kvm_exit(VMWRITE)                                4       0
 kvm_exit(HLT)                                    3       0
 kvm_exit(EPT_VIOLATION)                          3       0
 kvm_exit(TASK_SWITCH)                            3       0
 kvm_exit(CR_ACCESS)                              3       0
 kvm_exit(TRIPLE_FAULT)                           3       0
 kvm_exit(VMON)                                   3       0
 kvm_exit(EPT_MISCONFIG)                          3       0

thanks,

Signature · 2 lines
js
suse labs
#8 lore raw
Quoted 27 lines · Jiri Slaby
Quoted 15 lines · Avi Kivity
Quoted 5 lines · Jiri Slaby
Quoted 3 lines

Please provide the output of vmxcap
(http://goo.gl/c5lUO),

Unrestricted guest no

The big real mode fixes.

Quoted 5 lines
Quoted 2 lines

and a snapshot of kvm_stat while the guest is hung.

kvm statistics

Quoted 8 lines
Quoted 5 lines

exits 6778198 615942

 host_state_reload                             1988     187
 irq_exits                                     1523     138
 mmu_cache_miss                                   4       0
 fpu_reload                                       1       0

Please run this as root so we get the tracepoint based output; and press 'x' when it's running so we get more detailed output.

kvm statistics

Quoted 7 lines
 kvm_exit                                  13798699  330708
 kvm_entry                                 13799110  330708
 kvm_page_fault                            13793650  330604
 kvm_exit(EXCEPTION_NMI)                    6188458  330604

kvm_exit(EXTERNAL_INTERRUPT) 2169 105
kvm_exit(TPR_BELOW_THRESHOLD) 82 0
kvm_exit(IO_INSTRUCTION) 6 0

Strange, it's unable to fault in the very first page.

Please provide a trace as per http://www.linux-kvm.org/page/Tracing (but append -e kvmmmu to the command line).

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#9 lore raw
Quoted 36 lines · Avi Kivity
Quoted 27 lines · Jiri Slaby
Quoted 15 lines · Avi Kivity
Quoted 5 lines · Jiri Slaby
Quoted 3 lines

Please provide the output of vmxcap
(http://goo.gl/c5lUO),

Unrestricted guest no

The big real mode fixes.

Quoted 5 lines
Quoted 2 lines

and a snapshot of kvm_stat while the guest is hung.

kvm statistics

Quoted 8 lines
Quoted 5 lines

exits 6778198 615942

 host_state_reload                             1988     187
 irq_exits                                     1523     138
 mmu_cache_miss                                   4       0
 fpu_reload                                       1       0

Please run this as root so we get the tracepoint based output; and press 'x' when it's running so we get more detailed output.

kvm statistics

Quoted 7 lines
 kvm_exit                                  13798699  330708
 kvm_entry                                 13799110  330708
 kvm_page_fault                            13793650  330604
 kvm_exit(EXCEPTION_NMI)                    6188458  330604

kvm_exit(EXTERNAL_INTERRUPT) 2169 105
kvm_exit(TPR_BELOW_THRESHOLD) 82 0
kvm_exit(IO_INSTRUCTION) 6 0

Quoted 4 lines

Strange, it's unable to fault in the very first page.

Please provide a trace as per http://www.linux-kvm.org/page/Tracing (but append -e kvmmmu to the command line).

Attached. Does it make sense? It wrote things like:
failed to read event print fmt for kvm_mmu_unsync_page
to the stderr.

thanks,

Signature · 2 lines
js
suse labs

version = 6
CPU 0 is empty
cpus=2

        qemu-kvm-6170  [001]   457.811896: kvm_mmu_get_page:     [FAILED TO PARSE] gfn=0 role=122882 root_count=0 unsync=0 created=1
        qemu-kvm-6170  [001]   457.811899: kvm_mmu_get_page:     [FAILED TO PARSE] gfn=262144 role=122882 root_count=0 unsync=0 created=1
        qemu-kvm-6170  [001]   457.811900: kvm_mmu_get_page:     [FAILED TO PARSE] gfn=524288 role=122882 root_count=0 unsync=0 created=1
        qemu-kvm-6170  [001]   457.811902: kvm_mmu_get_page:     [FAILED TO PARSE] gfn=786432 role=122882 root_count=0 unsync=0 created=1
        qemu-kvm-6171  [001]   462.416705: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=786432 role=122882 root_count=1 unsync=0
        qemu-kvm-6171  [001]   462.416712: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=524288 role=122882 root_count=1 unsync=0
        qemu-kvm-6171  [001]   462.416715: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=262144 role=122882 root_count=1 unsync=0
        qemu-kvm-6171  [001]   462.416717: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=0 role=122882 root_count=1 unsync=0
        qemu-kvm-6171  [001]   462.485197: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=0 role=253954 root_count=0 unsync=0
        qemu-kvm-6171  [001]   462.485202: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=262144 role=253954 root_count=0 unsync=0
        qemu-kvm-6171  [001]   462.485205: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=524288 role=253954 root_count=0 unsync=0
        qemu-kvm-6171  [001]   462.485209: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=786432 role=253954 root_count=0 unsync=0
author
#10 lore raw
In reply to Avi Kivity
Quoted 36 lines · Avi Kivity
Quoted 27 lines · Jiri Slaby
Quoted 15 lines · Avi Kivity
Quoted 5 lines · Jiri Slaby
Quoted 3 lines

Please provide the output of vmxcap
(http://goo.gl/c5lUO),

Unrestricted guest no

The big real mode fixes.

Quoted 5 lines
Quoted 2 lines

and a snapshot of kvm_stat while the guest is hung.

kvm statistics

Quoted 8 lines
Quoted 5 lines

exits 6778198 615942

  host_state_reload                             1988     187
  irq_exits                                     1523     138
  mmu_cache_miss                                   4       0
  fpu_reload                                       1       0

Please run this as root so we get the tracepoint based output; and press 'x' when it's running so we get more detailed output.

kvm statistics

Quoted 7 lines
  kvm_exit                                  13798699  330708
  kvm_entry                                 13799110  330708
  kvm_page_fault                            13793650  330604
  kvm_exit(EXCEPTION_NMI)                    6188458  330604

kvm_exit(EXTERNAL_INTERRUPT) 2169 105
kvm_exit(TPR_BELOW_THRESHOLD) 82 0
kvm_exit(IO_INSTRUCTION) 6 0

Quoted 1 line

Strange, it's unable to fault in the very first page.

I bisected that. Note the bisection log. I have never seen something like that :D:
git bisect start
git bisect bad 3de9d1a1500472bc80478bd75e33fa9c1eba1422
git bisect good fea7a08acb13524b47711625eebea40a0ede69a0
git bisect good 95a2fe4baa1ad444df5f94bfc9416fc6b4b34cef
git bisect good f42c0d57a5a60da03c705bdea9fbba381112dd60
git bisect good 31a2e241a9e37a133278959044960c229acc5714
git bisect good f15fb01c5593fa1b58cc7a8a9c59913e2625bf2e
git bisect good 16d21ff46f5d50e311d07406c31f96916e5e8e1a
git bisect good 0b84592f458b4e8567aa7d803aff382c1d3b64fd
git bisect bad b955428e7f14cd29fe9d8059efa3ea4be679c83d
git bisect bad 20c4da4f68fcade05eda9c9b7dbad0a78cc5efe8
git bisect bad 31b90ed2a90f80fb528ac55ee357a815e1dedc36
git bisect bad b273fe14ee5b38cecc7bce94f7777f35a0bf9ee4
git bisect bad de426dbe9a60706b91b40397f69f819a39a06b6b
git bisect bad 6b998094ec50248e72b9f251d0607b58b18dba38
git bisect bad cf9b81d47a89f5d404a0cd8013b461617751e520

=== 8< ===

Reverting cf9b81d47a89 (mm: wrap calls to set_pte_at_notify with invalidate_range_start and invalidate_range_end) on the top of today's -next fixes the issue.

thanks,

Signature · 8 lines
js
suse labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
maintainer
#11 lore raw
Quoted 58 lines · On Tue, 18 Sep 2012 21:40:31 +0200 Jiri Slaby <[email protected]> wrote:
Quoted 36 lines · Avi Kivity
Quoted 27 lines · Jiri Slaby
Quoted 15 lines · Avi Kivity
Quoted 5 lines · Jiri Slaby
Quoted 3 lines

Please provide the output of vmxcap
(http://goo.gl/c5lUO),

Unrestricted guest no

The big real mode fixes.

Quoted 5 lines
Quoted 2 lines

and a snapshot of kvm_stat while the guest is hung.

kvm statistics

Quoted 8 lines
Quoted 5 lines

exits 6778198 615942

  host_state_reload                             1988     187
  irq_exits                                     1523     138
  mmu_cache_miss                                   4       0
  fpu_reload                                       1       0

Please run this as root so we get the tracepoint based output; and press 'x' when it's running so we get more detailed output.

kvm statistics

Quoted 7 lines
  kvm_exit                                  13798699  330708
  kvm_entry                                 13799110  330708
  kvm_page_fault                            13793650  330604
  kvm_exit(EXCEPTION_NMI)                    6188458  330604

kvm_exit(EXTERNAL_INTERRUPT) 2169 105
kvm_exit(TPR_BELOW_THRESHOLD) 82 0
kvm_exit(IO_INSTRUCTION) 6 0

Quoted 1 line

Strange, it's unable to fault in the very first page.

I bisected that. Note the bisection log. I have never seen something like that :D:
git bisect start
git bisect bad 3de9d1a1500472bc80478bd75e33fa9c1eba1422 git bisect good fea7a08acb13524b47711625eebea40a0ede69a0 git bisect good 95a2fe4baa1ad444df5f94bfc9416fc6b4b34cef git bisect good f42c0d57a5a60da03c705bdea9fbba381112dd60 git bisect good 31a2e241a9e37a133278959044960c229acc5714 git bisect good f15fb01c5593fa1b58cc7a8a9c59913e2625bf2e git bisect good 16d21ff46f5d50e311d07406c31f96916e5e8e1a git bisect good 0b84592f458b4e8567aa7d803aff382c1d3b64fd git bisect bad b955428e7f14cd29fe9d8059efa3ea4be679c83d git bisect bad 20c4da4f68fcade05eda9c9b7dbad0a78cc5efe8 git bisect bad 31b90ed2a90f80fb528ac55ee357a815e1dedc36 git bisect bad b273fe14ee5b38cecc7bce94f7777f35a0bf9ee4 git bisect bad de426dbe9a60706b91b40397f69f819a39a06b6b git bisect bad 6b998094ec50248e72b9f251d0607b58b18dba38 git bisect bad cf9b81d47a89f5d404a0cd8013b461617751e520

Quoted 5 lines

=== 8< ===

Reverting cf9b81d47a89 (mm: wrap calls to set_pte_at_notify with invalidate_range_start and invalidate_range_end) on the top of today's -next fixes the issue.

hm, thanks. This will probably take some time to resolve so I think I'll drop

mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#12 lore raw

Hi Andrew,

Quoted 8 lines · On Tue, 18 Sep 2012 12:46:46 -0700 Andrew Morton <[email protected]>…

hm, thanks. This will probably take some time to resolve so I think I'll drop

mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch

Should I attempt to remove these from the akpm tree in linux-next today? Or should I just wait for a new mmotm?

Signature · 2 lines
Cheers,
Stephen Rothwell                    [email protected]
maintainer
#13 lore raw
Quoted 7 lines · On Wed, 19 Sep 2012 10:00:34 +1000 Stephen Rothwell <[email protected]>…

Hi Andrew,

Quoted 4 lines · On Tue, 18 Sep 2012 12:46:46 -0700 Andrew Morton <[email protected]>…

hm, thanks. This will probably take some time to resolve so I think I'll drop

Quoted 6 lines
Quoted 4 lines

mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch

Should I attempt to remove these from the akpm tree in linux-next today?

That would be best - there's no point in having people test (and debug) dead stuff.

Quoted 1 line

Or should I just wait for a new mmotm?

You could be brave and test http://ozlabs.org/~akpm/mmots/ for me :)

#14 lore raw

Hi Andrew,

Quoted 13 lines · On Tue, 18 Sep 2012 17:20:29 -0700 Andrew Morton <[email protected]>…
Quoted 10 lines · On Wed, 19 Sep 2012 10:00:34 +1000 Stephen Rothwell <[email protected]>…
Quoted 8 lines · On Tue, 18 Sep 2012 12:46:46 -0700 Andrew Morton <[email protected]>…

hm, thanks. This will probably take some time to resolve so I think I'll drop

mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch

Quoted 4 lines
Quoted 1 line

Should I attempt to remove these from the akpm tree in linux-next today?

That would be best - there's no point in having people test (and debug) dead stuff.

OK, I removed them.

Quoted 3 lines
Quoted 1 line

Or should I just wait for a new mmotm?

You could be brave and test http://ozlabs.org/~akpm/mmots/ for me :)

Brave? maybe. Stupid? no :-)

Signature · 2 lines
Cheers,
Stephen Rothwell                    [email protected]
#15 lore raw

PATCH 1/1 mm: call invalidate_range_end in do_wp_page even for zero pages

In reply to Jiri Slaby

The previous patch "mm: wrap calls to set_pte_at_notify with invalidate_range_start and invalidate_range_end" only called the invalidate_range_end mmu notifier function in do_wp_page when the new_page variable wasn't NULL. This was done in order to only call invalidate_range_end after invalidate_range_start was called. Unfortunately, there are situations where new_page is NULL and invalidate_range_start is called. This caused invalidate_range_start to be called without a matching invalidate_range_end, causing kvm to loop indefinitely on the first page fault.

This patch adds a flag variable to do_wp_page that marks whether the invalidate_range_start notifier was called. invalidate_range_end is then called if the flag is true.

---
I tested this patch against yesterday's linux-next (next-20120918), and it seems to solve the problem with kvm. I used the same command line you reported:

  qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 \
  -m 1000M -net user -net nic,model=e1000 -usb -serial pty

I was hoping you could also test it yourself, and see that it also works for you, if you don't mind.

 mm/memory.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
mm/memory.c +5 -4
index 1a92d87..76ec199 100644
@@ -2529,6 +2529,7 @@ static int do_wp_page(struct mm_struct *mm, struct vm_area_struct *vma,
 	struct page *dirty_page = NULL;
 	unsigned long mmun_start;	/* For mmu_notifiers */
 	unsigned long mmun_end;		/* For mmu_notifiers */
+	bool mmun_called = false;	/* For mmu_notifiers */
 
 	old_page = vm_normal_page(vma, address, orig_pte);
 	if (!old_page) {
@@ -2706,8 +2707,9 @@ gotten:
 	if (mem_cgroup_newpage_charge(new_page, mm, GFP_KERNEL))
 		goto oom_free_new;
 
-	mmun_start = address & PAGE_MASK;
-	mmun_end   = (address & PAGE_MASK) + PAGE_SIZE;
+	mmun_start  = address & PAGE_MASK;
+	mmun_end    = (address & PAGE_MASK) + PAGE_SIZE;
+	mmun_called = true;
 	mmu_notifier_invalidate_range_start(mm, mmun_start, mmun_end);
 
 	/*
@@ -2776,8 +2778,7 @@ gotten:
 		page_cache_release(new_page);
 unlock:
 	pte_unmap_unlock(page_table, ptl);
-	if (new_page)
-		/* Only call the end notifier if the begin was called. */
+	if (mmun_called)
 		mmu_notifier_invalidate_range_end(mm, mmun_start, mmun_end);
 	if (old_page) {
 		/*
Signature · 7 lines
1.7.11.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>