nSVM: Booting L2 results in L1 hang and a "skip_emulated_instruction"

DISCUSSION Kashyap Chamarthy 2 participants 3 messages Feb 11, 2015 – Feb 17, 2015 KVM

lore

Share & tools

lore

Kashyap Chamarthy

author

2015-02-11 18:12 UTC #0 lore raw

Hi,

This was tested with kernel-3.19.0-1.fc22) and QEMU (qemu-2.2.0-5.fc22) on L0 & L1.

Description
-----------

Inside L1, boot a nested KVM guest (L2) . Instead of a full blown guest, let's use qemu-sanity-check with KVM:

    $ qemu-sanity-check --accel=kvm

Wwich gives you this CLI (run from a different shell), that confirms that the L2 guest is indeed running on KVM (and not TCG):

  $ ps -ef | grep -i qemu
  root       763   762 35 11:49 ttyS0    00:00:00 qemu-system-x86_64 -nographic -nodefconfig -nodefaults -machine accel=kvm -no-reboot -serial file:/tmp/tmp.rl3naPaCkZ.out -kernel /boot/vmlinuz-3.19.0-1.fc21.x86_64 -initrd /usr/lib64/qemu-sanity-check/initrd -append console=ttyS0 oops=panic panic=-1

Which results in:

(a) L1 (guest hypervisor) completely hangs and is unresponsive. But when I query libvirt, (virsh list) the guest is still reported as 'running'

(b) On L0, I notice a ton of these messages:

        skip_emulated_instruction: ip 0xffec next 0xffffffff8105e964

I can get dmesg, dmidecode , x86info -a on L0 and L1 if it helps in narrowing down the issue.

Related bug and reproducer details
----------------------------------

https://bugzilla.redhat.com/show_bug.cgi?id=1191665 -- Nested KVM with
AMD: L2 (nested guest) fails with "divide error: 0000 [#1] SMP"

Signature · 1 line

/kashyap

Jan Kiszka

2015-02-12 06:12 UTC #1 lore raw

Quoted 36 lines · Kashyap Chamarthy

Hi,
This was tested with kernel-3.19.0-1.fc22) and QEMU (qemu-2.2.0-5.fc22) on L0 & L1.
Description
-----------
Inside L1, boot a nested KVM guest (L2) . Instead of a full blown guest, let's use qemu-sanity-check with KVM:
    $ qemu-sanity-check --accel=kvm
Wwich gives you this CLI (run from a different shell), that confirms that the L2 guest is indeed running on KVM (and not TCG):
  $ ps -ef | grep -i qemu
  root       763   762 35 11:49 ttyS0    00:00:00 qemu-system-x86_64 -nographic -nodefconfig -nodefaults -machine accel=kvm -no-reboot -serial file:/tmp/tmp.rl3naPaCkZ.out -kernel /boot/vmlinuz-3.19.0-1.fc21.x86_64 -initrd /usr/lib64/qemu-sanity-check/initrd -append console=ttyS0 oops=panic panic=-1
Which results in:
(a) L1 (guest hypervisor) completely hangs and is unresponsive. But when I query libvirt, (virsh list) the guest is still reported as 'running'
(b) On L0, I notice a ton of these messages:
        skip_emulated_instruction: ip 0xffec next 0xffffffff8105e964
I can get dmesg, dmidecode , x86info -a on L0 and L1 if it helps in narrowing down the issue.

Quoted 8 lines

Related bug and reproducer details
----------------------------------
https://bugzilla.redhat.com/show_bug.cgi?id=1191665 -- Nested KVM with
AMD: L2 (nested guest) fails with "divide error: 0000 [#1] SMP"

Is this a regression (of the kernel)? If so, can you bisect to the commit that introduced it?

Jan

Signature · 2 lines

Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

Kashyap Chamarthy

author

2015-02-17 11:32 UTC #2 lore raw

Quoted 41 lines · Jan Kiszka

Quoted 40 lines · Kashyap Chamarthy
Hi,
This was tested with kernel-3.19.0-1.fc22) and QEMU (qemu-2.2.0-5.fc22) on L0 & L1.
Description
-----------
Inside L1, boot a nested KVM guest (L2) . Instead of a full blown guest, let's use qemu-sanity-check with KVM:
    $ qemu-sanity-check --accel=kvm
Wwich gives you this CLI (run from a different shell), that confirms that the L2 guest is indeed running on KVM (and not TCG):
  $ ps -ef | grep -i qemu
  root       763   762 35 11:49 ttyS0    00:00:00 qemu-system-x86_64 -nographic -nodefconfig -nodefaults -machine accel=kvm -no-reboot -serial file:/tmp/tmp.rl3naPaCkZ.out -kernel /boot/vmlinuz-3.19.0-1.fc21.x86_64 -initrd /usr/lib64/qemu-sanity-check/initrd -append console=ttyS0 oops=panic panic=-1
Which results in:
(a) L1 (guest hypervisor) completely hangs and is unresponsive. But when I query libvirt, (virsh list) the guest is still reported as 'running'
(b) On L0, I notice a ton of these messages:
        skip_emulated_instruction: ip 0xffec next 0xffffffff8105e964
I can get dmesg, dmidecode , x86info -a on L0 and L1 if it helps in narrowing down the issue.
Related bug and reproducer details
----------------------------------

Quoted 7 lines

Quoted 4 lines
https://bugzilla.redhat.com/show_bug.cgi?id=1191665 -- Nested KVM with
AMD: L2 (nested guest) fails with "divide error: 0000 [#1] SMP"
Is this a regression (of the kernel)? If so, can you bisect to the commit that introduced it?

[Sorry, I missed this reply and just noticed it.]

I can't certainly say that it's a regression of Kernel. I also heard from Dan Berrange (CCed), that when an booting L2 guest it caused L1 to panic and reboot on AMD. I don't have the AMD physical machine that I tested this on, will try to find one this week and see if I can bisect.

Thanks.

Signature · 1 line

/kashyap