Skip to contentSkip to conversation

New: [Nested]L2 guest boot up fail(kvm on kvm).

Bug 67061 [email protected] 1 participant 5 messages Dec 16, 2013 – Mar 11, 2014 KVM

lore
Share & tools
lore

[email protected]

author
#0 lore raw

https://bugzilla.kernel.org/show_bug.cgi?id=67061

            Bug ID: 67061
           Summary: [Nested]L2 guest boot up fail(kvm on kvm).
           Product: Virtualization
           Version: unspecified
    Kernel Version: 3.13.0-rc1
          Hardware: All
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: kvm
          Assignee: [email protected]
          Reporter: [email protected]
        Regression: No

Environment:
------------
Host OS (ia32/ia32e/IA64):ia32e
Guest OS (ia32/ia32e/IA64):ia32e
Guest OS Type (Linux/Windows):Linux
kvm.git Commit:2ecd1aba597726c9053ba1a112e3d8fdc5ca08ff qemu-kvm Commit:5c5432e7d630592ddcc1876ac8a1505f8f14ef15 Host Kernel Version:3.12.0
Hardware:Romley_EP

Bug detailed description:
--------------------------
when create L2 guest, the L2 guest boot up fail.
note:
1. L1 guest is kvm on kvm.
2. This should be a kernel bug
kvm + qemu = result
f0804804 + 5c5432e7 = bad
ede58222 + 5c5432e7 = good

Reproduce steps:
----------------
1.create L1 guest:
qemu-system-x86_64 -enable-kvm -m 6G -smp 4 -net nic,macaddr=00:12:52:13:46:67 -net tap,script=/etc/kvm/qemu-ifup ia32e_nested_kvm.img -cpu host 2.create L2 guest:
qemu-system-x86_64 -enable-kvm -m 1G -smp 2 -net none ia32e_rhel6u4.img

Current result:
----------------
L2 guest boot up fail
Expected result:
----------------
after create L2 guest, L2 guest works fine

Basic root-causing log:
----------------------

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
author
#1 lore raw

[Bug 67061] [Nested]L2 guest boot up fail(kvm on kvm).

https://bugzilla.kernel.org/show_bug.cgi?id=67061

--- Comment #1 from Zhou, Chao <[email protected]> --- the first bad commit is:
commit 8a29111c7ca68d928dfab58636f3f6acf0ac04f7

Author: Eric Dumazet <[email protected]>
Date:   Tue Oct 8 09:02:23 2013 -0700

net: gro: allow to build full sized skb

skb_gro_receive() is currently limited to 16 or 17 MSS per GRO skb, typically 24616 bytes, because it fills up to MAX_SKB_FRAGS frags.

It's relatively easy to extend the skb using frag_list to allow more frags to be appended into the last sk_buff.

This still builds very efficient skbs, and allows reaching 45 MSS per skb.

(45 MSS GRO packet uses one skb plus a frag_list containing 2 additional sk_buff)

High speed TCP flows benefit from this extension by lowering TCP stack cpu usage (less packets stored in receive queue, less ACK packets processed)

Forwarding setups could be hurt, as such skbs will need to be linearized, although its not a new problem, as GRO could already provide skbs with a frag_list.

We could make the 65536 bytes threshold a tunable to mitigate this.

(First time we need to linearize skb in skb_needs_linearize(), we could lower the tunable to ~16*1460 so that following skb_gro_receive() calls build smaller skbs)

Signed-off-by Eric Dumazet <[email protected]>
Signed-off-by David S. Miller <[email protected]>
Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
author
#2 lore raw

[Bug 67061] [BISECTED][Nested]L2 guest boot up fail(kvm on kvm).

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=67061

Alan <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

                 CC|                            |[email protected]
            Summary|[Nested]L2 guest boot up    |[BISECTED][Nested]L2 guest
                   |fail(kvm on kvm).           |boot up fail(kvm on kvm).

--- Comment #2 from Alan <[email protected]> --- Thats a very odd bisection point it must be said

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
author
#3 lore raw

[Bug 67061] [BISECTED][Nested]L2 guest boot up fail(kvm on kvm).

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=67061

Thilo Fromm <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #3 from Thilo Fromm <[email protected]> --- This bug may have been fixed in 3.13.6.

I was affected by this bug, too - I use nested virtualization a lot, mainly for system testing purposes. I was able to reproduce the issue on all 3.13 kernels so far; my hardware is a Intel Core i7-2640M.

Kernel release 3.13.6 fixed the issue for me. Looking at the changelog at
https://www.kernel.org/pub/linux/kernel/v3.x/ChangeLog-3.13.6, I guess the fix was in commit 92cb6f3c5533832aa70a7dc9326ef21270169583 " kvm, vmx: Really fix lazy FPU on nested guest".

The issue is discussed on the kvm-devel mailing list:
http://thread.gmane.org/gmane.comp.emulators.kvm.devel/119328 leading to the
fix: http://permalink.gmane.org/gmane.linux.kernel.stable/82043.

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
author
#4 lore raw

[Bug 67061] [BISECTED][Nested]L2 guest boot up fail(kvm on kvm).

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=67061

Paolo Bonzini <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|NEW                         |RESOLVED
                 CC|                            |[email protected]
         Resolution|---                         |CODE_FIX

--- Comment #4 from Paolo Bonzini <[email protected]> --- I agree with the explanation in comment 3.

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.