New: kernel crash in kvm_intel module on Xeon Silver 4314s
https://bugzilla.kernel.org/show_bug.cgi?id=220631
URL: https://bugzilla.proxmox.com/show_bug.cgi?id=6767 Bug ID: 220631 Summary: kernel crash in kvm_intel module on Xeon Silver 4314s Product: Virtualization Version: unspecified Hardware: Intel OS: Linux Status: NEW Severity: normal Priority: P3 Component: kvm Assignee: [email protected] Reporter: [email protected] Regression: No
Created attachment 308754
--> https://bugzilla.kernel.org/attachment.cgi?id=308754&action=edit
Qemu commandline
one of our users reports a crash in kvm_intel with Windows guests and live migration. the source and target hosts are identical hardware, kernel version and bios version wise:
- Supermicro SYS-120C-TN10R servers with Xeon Silver 4314s
- microcode : 0xd000404
- 6.14.8-2-pve (based on Ubuntu 6.14.0-26.26 and upstream stable 6.14.
- guest OS: Windows 11 24H2
the Qemu commandline and dmesg output is attached, the user is CCed to this bug. unfortunately, we don't have the hardware to reproduce and bisect ourselves.
this is likely a regression, since the user reports running into the issue since upgrading to PVE 9, which means switching from kernel 6.8.x to 6.14.x
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 220631] kernel crash in kvm_intel module on Xeon Silver 4314s
https://bugzilla.kernel.org/show_bug.cgi?id=220631
--- Comment #1 from Fabian Grünbichler ([email protected]) --- Created attachment 308755
--> https://bugzilla.kernel.org/attachment.cgi?id=308755&action=edit
dmesg with kvm_intel.dump_invalid_vmcs=1
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 220631] kernel crash in kvm_intel module on Xeon Silver 4314s
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=220631
Artem S. Tashkinov ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #2 from Artem S. Tashkinov ([email protected]) ---
Vendor kernels are not supported here and neither is 6.14 because it's not an LTS kernel.
Is this reproducible under 6.17.1?
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 220631] kernel crash in kvm_intel module on Xeon Silver 4314s
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=220631
--- Comment #3 from Fabian Grünbichler ([email protected]) --- I asked the original reporter to register here, hopefully they can attempt to reproduce using a newer kernel, e.g. using
https://kernel.ubuntu.com/mainline/v6.17.1/ once that one has built, or
https://kernel.ubuntu.com/mainline/v6.17/amd64/
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 220631] kernel crash in kvm_intel module on Xeon Silver 4314s
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=220631
Sean Christopherson ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]
--- Comment #4 from Sean Christopherson ([email protected]) --- This is due to a known a CPU/ucode bug where some Intel CPUs generate spurious #VEs.
[ +0.000151] kvm: #VE 65dbc7f8
[ +0.000154] kvm: #VE 65dbc7f8, spte[4] = 0x8010000418c52807, spte[3] = 0x801000045b676807, spte[2] = 0x86100023d0000bf7
The "fix" is to disable CONFIG_KVM_INTEL_PROVE_VE. KVM_INTEL_PROVE_VE isn't intended for production use (it's akin to PROVE_LOCKING), and the help text even calls out that some CPUs generate spurious #VEs.
config KVM_INTEL_PROVE_VE
bool "Check that guests do not receive #VE exceptions" depends on KVM_INTEL && EXPERT
help
Checks that KVM's page table management code will not incorrectly let guests receive a virtualization exception. Virtualization exceptions will be trapped by the hypervisor rather than injected in the guest.
Note: some CPUs appear to generate spurious EPT Violations #VEs that trigger KVM's WARN, in particular with eptad=0 and/or nested virtualization.
If unsure, say N.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 220631] kernel crash in kvm_intel module on Xeon Silver 4314s
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=220631
Kevin Boyd ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]
--- Comment #5 from Kevin Boyd ([email protected]) ---
Original poster here. Happy to try a different kernel but it seems like the final comment suggests that the CONFIG_KVM_INTEL_PROVE_VE KFLAG should be set to “N” to quiet the microcode bug.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 220631] kernel crash in kvm_intel module on Xeon Silver 4314s
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=220631
--- Comment #6 from Fabian Grünbichler ([email protected]) --- thanks Sean!
it seems like disabling the Kconfig (at least in non-debug builds) seems like a good way forward, I'll forward this to the Ubuntu folks as well (we inherited the Y from their kernel config).
@Kevin: I'll ping you once a kernel with that disabled is built, so you can confirm that this makes your issues no longer reproduce and we can close this!
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.