kvm crash on 5.7-rc1 and later
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 30 lines · Woody Suwalski
I am observing a 100% reproducible kvm crash on kernels starting with 5.7-rc1, always with the same opcode 0000.
It happens during wake up from the host suspended state. Worked OK on 5.6 and older.
The host is based on Debian testing, Thinkpad T440, i5 cpu.[ 61.576664] kernel BUG at
arch/x86/kvm/x86.c:387!
[ 61.576672] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 61.576678] CPU: 0 PID: 3851 Comm: qemu-system-x86 Not tainted 5.7-pingu #0
[ 61.576680] Hardware name: LENOVO 20B6005JUS/20B6005JUS, BIOS GJETA4WW (2.54 ) 03/27/2020
[ 61.576700] RIP: 0010:kvm_spurious_fault+0xa/0x10 [kvm]Crash results in a dead kvm and occasionally a very unstable system.
Bisecting the problem between v5.6 and v5.7-rc1 points to
commit
6650cdd9a8ccf00555dbbe743d58541ad8feb6a7
Author: Peter Zijlstra (Intel) <[email protected]> Date: Sun Jan 26 12:05:35 2020 -0800x86/split_lock: Enable split lock detection by kernel
Reversing that patch seems to actually "cure" the issue.
The problem is present in all kernels past 5.7-rc1, however the patch is not reversing directly in later source trees, so can not retest the logic on recent kernels.
Quoted 5 lines
Peter, would you have idea how to debug that (or even better - would you happen to know the fix)?
I have attached dmesg logs from a "good" 5.6.9 kernel, and then "bad" 5.7.0 and 5.8-rc3
I have no clue about kvm. Nor do I actually have hardware with SLD on. I've Cc'ed a bunch of folks who might have more ideas.
Quoted 31 lines · Peter Zijlstra
Quoted 30 lines · Woody Suwalski
I am observing a 100% reproducible kvm crash on kernels starting with 5.7-rc1, always with the same opcode 0000.
It happens during wake up from the host suspended state. Worked OK on 5.6 and older.
The host is based on Debian testing, Thinkpad T440, i5 cpu.[ 61.576664] kernel BUG at
arch/x86/kvm/x86.c:387!
[ 61.576672] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 61.576678] CPU: 0 PID: 3851 Comm: qemu-system-x86 Not tainted 5.7-pingu #0
[ 61.576680] Hardware name: LENOVO 20B6005JUS/20B6005JUS, BIOS GJETA4WW (2.54 ) 03/27/2020
[ 61.576700] RIP: 0010:kvm_spurious_fault+0xa/0x10 [kvm]Crash results in a dead kvm and occasionally a very unstable system.
Bisecting the problem between v5.6 and v5.7-rc1 points to
commit
6650cdd9a8ccf00555dbbe743d58541ad8feb6a7
Author: Peter Zijlstra (Intel) <[email protected]> Date: Sun Jan 26 12:05:35 2020 -0800x86/split_lock: Enable split lock detection by kernel
Reversing that patch seems to actually "cure" the issue.
The problem is present in all kernels past 5.7-rc1, however the patch is not reversing directly in later source trees, so can not retest the logic on recent kernels.
Quoted 9 lines
Quoted 5 lines
Peter, would you have idea how to debug that (or even better - would you happen to know the fix)?
I have attached dmesg logs from a "good" 5.6.9 kernel, and then "bad" 5.7.0 and 5.8-rc3
I have no clue about kvm. Nor do I actually have hardware with SLD on. I've Cc'ed a bunch of folks who might have more ideas.
I think this bug is the same as the one found by Sean, and is already fixed in 5.8-rc4.
Quoted 38 lines · Xiaoyao Li
Quoted 37 lines · Peter Zijlstra
Quoted 35 lines · Woody Suwalski
I am observing a 100% reproducible kvm crash on kernels starting with 5.7-rc1, always with the same opcode 0000.
It happens during wake up from the host suspended state. Worked OK on 5.6 and older.
The host is based on Debian testing, Thinkpad T440, i5 cpu.[ 61.576664] kernel BUG at
arch/x86/kvm/x86.c:387!
[ 61.576672] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 61.576678] CPU: 0 PID: 3851 Comm: qemu-system-x86 Not tainted 5.7-pingu #0
[ 61.576680] Hardware name: LENOVO 20B6005JUS/20B6005JUS, BIOS GJETA4WW (2.54 ) 03/27/2020
[ 61.576700] RIP: 0010:kvm_spurious_fault+0xa/0x10 [kvm]Crash results in a dead kvm and occasionally a very unstable system.
Bisecting the problem between v5.6 and v5.7-rc1 points to
commit
6650cdd9a8ccf00555dbbe743d58541ad8feb6a7
Author: Peter Zijlstra (Intel) <[email protected]> Date: Sun Jan 26 12:05:35 2020 -0800x86/split_lock: Enable split lock detection by kernel
Reversing that patch seems to actually "cure" the issue.
The problem is present in all kernels past 5.7-rc1, however the patch is not reversing directly in later source trees, so can not retest the logic on recent kernels.
Peter, would you have idea how to debug that (or even better - would you happen to know the fix)?
I have attached dmesg logs from a "good" 5.6.9 kernel, and then "bad" 5.7.0 and 5.8-rc3
Quoted 8 lines
Quoted 3 lines
I have no clue about kvm. Nor do I actually have hardware with SLD on. I've Cc'ed a bunch of folks who might have more ideas.
I think this bug is the same as the one found by Sean, and is already fixed in 5.8-rc4.
You are right, kvm works OK on 5.8-rc4.
The fix will need to be backported to 5.7.
Thanks, Woody
Quoted 45 lines · Woody Suwalski
Quoted 44 lines · Xiaoyao Li
Quoted 31 lines · Peter Zijlstra
Quoted 30 lines · Woody Suwalski
I am observing a 100% reproducible kvm crash on kernels starting with 5.7-rc1, always with the same opcode 0000.
It happens during wake up from the host suspended state. Worked OK on 5.6 and older.
The host is based on Debian testing, Thinkpad T440, i5 cpu.[ 61.576664] kernel BUG at
arch/x86/kvm/x86.c:387!
[ 61.576672] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
[ 61.576678] CPU: 0 PID: 3851 Comm: qemu-system-x86 Not tainted 5.7-pingu #0
[ 61.576680] Hardware name: LENOVO 20B6005JUS/20B6005JUS, BIOS GJETA4WW (2.54 ) 03/27/2020
[ 61.576700] RIP: 0010:kvm_spurious_fault+0xa/0x10 [kvm]Crash results in a dead kvm and occasionally a very unstable system.
Bisecting the problem between v5.6 and v5.7-rc1 points to
commit
6650cdd9a8ccf00555dbbe743d58541ad8feb6a7
Author: Peter Zijlstra (Intel) <[email protected]> Date: Sun Jan 26 12:05:35 2020 -0800x86/split_lock: Enable split lock detection by kernel
Reversing that patch seems to actually "cure" the issue.
The problem is present in all kernels past 5.7-rc1, however the patch is not reversing directly in later source trees, so can not retest the logic on recent kernels.
Quoted 9 lines
Quoted 5 lines
Peter, would you have idea how to debug that (or even better - would you happen to know the fix)?
I have attached dmesg logs from a "good" 5.6.9 kernel, and then "bad" 5.7.0 and 5.8-rc3
I have no clue about kvm. Nor do I actually have hardware with SLD on. I've Cc'ed a bunch of folks who might have more ideas.
I think this bug is the same as the one found by Sean, and is already fixed in 5.8-rc4.
Quoted 8 lines
Quoted 2 lines
You are right, kvm works OK on 5.8-rc4.
The fix will need to be backported to 5.7.Thanks, Woody
I see it is already in 5.7.8. Great :-)
Thanks, Woody