kvm: emulation failure

I see this problem with a x86 sles10 guest running on x86_64 intel host. If the guest is reset abruptly and rebooted, some where before grub sequence it hangs and the following message is seen in the logs

emulation failed (pagetable) rip 7ed5 66 60 ac 20.

I located this instruction sequence in isolinux.bin on the iso ;if that is relevant.

I did some analysis and find that there is an ept violation, which is handled and then the next instruction '66 60' is attempted to decode and emulate. But decode fails. kvm continues loops in the kernel in __vcpu_run().

the code path is

kvm_run() -> __vcpu_run() -> vcpu_enter_guest() -> kvm_handle_exit() -> handle_ept_violation() -> kvm_mmu_page_fault() -> emulate_instruction() -> x86_decode_insn()

Any insights here on how to fix the problem is appreciated. And if a fix already exists even better :)

thanks,
RP

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 16 lines · On Saturday 20 June 2009 03:23:40 Ram Pai wrote:

I see this problem with a x86 sles10 guest running on x86_64 intel host. If the guest is reset abruptly and rebooted, some where before grub sequence it hangs and the following message is seen in the logs

emulation failed (pagetable) rip 7ed5 66 60 ac 20.

I located this instruction sequence in isolinux.bin on the iso ;if that is relevant.

I did some analysis and find that there is an ept violation, which is handled and then the next instruction '66 60' is attempted to decode and emulate. But decode fails. kvm continues loops in the kernel in __vcpu_run().

Quoted 6 lines

the code path is

kvm_run() -> __vcpu_run() -> vcpu_enter_guest() -> kvm_handle_exit() -> handle_ept_violation() -> kvm_mmu_page_fault() -> emulate_instruction() -> x86_decode_insn()

Hi Ram

Seems KVM failed to emulate a unknown instruction.

00000000  6660              pushad
00000002  AC                lodsb

And PUSHAD have not implemented in x86_emulate.c.

But I am a little curious about why this code path was only triggered when reset. Maybe other issue exists.

Quoted 23 lines · Sheng Yang

Quoted 16 lines · On Saturday 20 June 2009 03:23:40 Ram Pai wrote:

I see this problem with a x86 sles10 guest running on x86_64 intel host. If the guest is reset abruptly and rebooted, some where before grub sequence it hangs and the following message is seen in the logs

emulation failed (pagetable) rip 7ed5 66 60 ac 20.

I located this instruction sequence in isolinux.bin on the iso ;if that is relevant.

I did some analysis and find that there is an ept violation, which is handled and then the next instruction '66 60' is attempted to decode and emulate. But decode fails. kvm continues loops in the kernel in __vcpu_run().

Quoted 5 lines

the code path is

kvm_run() -> __vcpu_run() -> vcpu_enter_guest() -> kvm_handle_exit() -> handle_ept_violation() -> kvm_mmu_page_fault() -> emulate_instruction() -> x86_decode_insn()

Quoted 8 lines

Hi Ram

Seems KVM failed to emulate a unknown instruction.

00000000  6660              pushad
00000002  AC                lodsb

And PUSHAD have not implemented in x86_emulate.c.

Thanks Sheng for your response,

Good. that was the conclusion i had reached reading the code. However was not sure whether the (a) the code path should have never reached there or (b) the code must have learnt to emulate pushad.

Sounds like (b) is the case.

Quoted 2 lines

But I am a little curious about why this code path was only triggered when reset. Maybe other issue exists.

What do you want me to check? I have seen ept violation code getting triggered a few number of times at various stages. But the one reported above is the only case where the instruction being emulated is 66 60.

one more observation:
seen only if the /boot partition is reiserfs. I have been unable to reproduce this with /boot being ext3.

thanks and let me know,
RP

Quoted 25 lines · Sheng Yang

Quoted 16 lines · On Saturday 20 June 2009 03:23:40 Ram Pai wrote:

I see this problem with a x86 sles10 guest running on x86_64 intel host. If the guest is reset abruptly and rebooted, some where before grub sequence it hangs and the following message is seen in the logs

emulation failed (pagetable) rip 7ed5 66 60 ac 20.

I located this instruction sequence in isolinux.bin on the iso ;if that is relevant.

I did some analysis and find that there is an ept violation, which is handled and then the next instruction '66 60' is attempted to decode and emulate. But decode fails. kvm continues loops in the kernel in __vcpu_run().

Quoted 7 lines

the code path is

kvm_run() -> __vcpu_run() -> vcpu_enter_guest() -> kvm_handle_exit() -> handle_ept_violation() -> kvm_mmu_page_fault() -> emulate_instruction() -> x86_decode_insn()

Quoted 9 lines

Hi Ram

Seems KVM failed to emulate a unknown instruction.

00000000  6660              pushad
00000002  AC                lodsb

And PUSHAD have not implemented in x86_emulate.c.

Thanks Sheng for your response,

Good. that was the conclusion i had reached reading the code. However was not sure whether the (a) the code path should have never reached there or (b) the code must have learnt to emulate pushad.

Sounds like (b) is the case.

Quoted 3 lines

But I am a little curious about why this code path was only triggered when reset. Maybe other issue exists.

What do you want me to check? I have seen ept violation code getting triggered a few number of times at various stages. But the one reported above is the only case where the instruction being emulated is 66 60.

one more observation:
seen only if the /boot partition is reiserfs. I have been unable to reproduce this with /boot being ext3.

Quoted 37 lines · Ram Pai

Quoted 25 lines · Sheng Yang

Quoted 16 lines · On Saturday 20 June 2009 03:23:40 Ram Pai wrote:

I see this problem with a x86 sles10 guest running on x86_64 intel host. If the guest is reset abruptly and rebooted, some where before grub sequence it hangs and the following message is seen in the logs

emulation failed (pagetable) rip 7ed5 66 60 ac 20.

I located this instruction sequence in isolinux.bin on the iso ;if that is relevant.

I did some analysis and find that there is an ept violation, which is handled and then the next instruction '66 60' is attempted to decode and emulate. But decode fails. kvm continues loops in the kernel in __vcpu_run().

Quoted 7 lines

the code path is

kvm_run() -> __vcpu_run() -> vcpu_enter_guest() -> kvm_handle_exit() -> handle_ept_violation() -> kvm_mmu_page_fault() -> emulate_instruction() -> x86_decode_insn()

Quoted 9 lines

Hi Ram

Seems KVM failed to emulate a unknown instruction.

00000000  6660              pushad
00000002  AC                lodsb

And PUSHAD have not implemented in x86_emulate.c.

Quoted 9 lines

Thanks Sheng for your response,

Good. that was the conclusion i had reached reading the code. However was not sure whether the (a) the code path should have never reached there or (b) the code must have learnt to emulate pushad.

Sounds like (b) is the case.

With ept, the only reason to emulate is mmio. It's very unlikely that the guest is using the pusha instruction for mmio, so the guest is probably confused here.

Current kvm.git will return an error here, and current qemu-kvm.git will stop the guest on error so we can debug. But the real problem likely started much earlier, I'm not sure we'll get much useful information.

Is this problem reproducible?

Quoted 9 lines

Quoted 4 lines

But I am a little curious about why this code path was only triggered when reset. Maybe other issue exists.

What do you want me to check? I have seen ept violation code getting triggered a few number of times at various stages. But the one reported above is the only case where the instruction being emulated is 66 60.

Quoted 4 lines

one more observation:
seen only if the /boot partition is reiserfs. I have been unable to reproduce this with /boot being ext3.

Please try it with current sources and post the output of 'info registers' in the monitor.