linux-4.4: qemu starting VM emits vmwrite error

DISCUSSION Zdenek Kaspar 3 participants 4 messages Jan 11, 2016 – Jan 12, 2016

Subsystems (3)

x86 KVM KVM x86

lore

Share & tools

lore

Zdenek Kaspar

author

2016-01-11 18:19 UTC #0 lore raw

Hi, today I compiled linux-4.4 kernel on older machine and noticed qemu complains:

Jan 11 14:28:58 HOST kernel: vmwrite error: reg 401e value 10 (err 12)
Jan 11 14:28:58 HOST kernel: CPU: 0 PID: 6004 Comm: qemu-build Not
tainted 4.4.0-0-core2 #1
Jan 11 14:28:58 HOST kernel: Hardware name:                  /DG35EC,
BIOS ECG3510M.86A.0118.2010.0113.1426 01/13/2010
Jan 11 14:28:58 HOST kernel: ffff88007be68000 ffffffff8116e2b9
0000000000000000 ffffffffa020b88d
Jan 11 14:28:58 HOST kernel: ffff88007be68000 ffff880061173d80
fffffffffffffff2 ffffffffa01d8fdc
Jan 11 14:28:58 HOST kernel: 0000000000000000 00007f7ff40008c0
ffffffffa01b8363 ffffffff81608c98
Jan 11 14:28:58 HOST kernel: Call Trace:
Jan 11 14:28:58 HOST kernel: [<ffffffff8116e2b9>] ? dump_stack+0x40/0x57
Jan 11 14:28:58 HOST kernel: [<ffffffffa020b88d>] ?
vmx_cpuid_update+0x5d/0x150 [kvm_intel]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01d8fdc>] ?
kvm_vcpu_ioctl_set_cpuid2+0x4c/0x70 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01b8363>] ?
kvm_arch_vcpu_ioctl+0x903/0xfa0 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffff810d1401>] ?
vunmap_page_range+0x211/0x330
Jan 11 14:28:58 HOST kernel: [<ffffffffa02088ab>] ?
vmx_vcpu_load+0x14b/0x1b0 [kvm_intel]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01b78a5>] ?
kvm_arch_vcpu_load+0x55/0x210 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01ad29b>] ?
kvm_vcpu_ioctl+0xbb/0x510 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01d9260>] ?
kvm_dev_ioctl_get_cpuid+0x210/0x2b0 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffff810f3a53>] ? do_vfs_ioctl+0x293/0x470
Jan 11 14:28:58 HOST kernel: [<ffffffff81034556>] ?
__do_page_fault+0x146/0x380
Jan 11 14:28:58 HOST kernel: [<ffffffff810f3c66>] ? SyS_ioctl+0x36/0x70
Jan 11 14:28:58 HOST kernel: [<ffffffff812d4eee>] ?
entry_SYSCALL_64_fastpath+0x12/0x71
Jan 11 14:28:58 HOST kernel: vmwrite error: reg 401e value 10 (err 12)
Jan 11 14:28:58 HOST kernel: CPU: 0 PID: 6005 Comm: qemu-build Not
tainted 4.4.0-0-core2 #1
Jan 11 14:28:58 HOST kernel: Hardware name:                  /DG35EC,
BIOS ECG3510M.86A.0118.2010.0113.1426 01/13/2010
Jan 11 14:28:58 HOST kernel: ffff88004af40000 ffffffff8116e2b9
0000000000000000 ffffffffa020b88d
Jan 11 14:28:58 HOST kernel: ffff88004af40000 ffff88007bc83d80
fffffffffffffff2 ffffffffa01d8fdc
Jan 11 14:28:58 HOST kernel: 0000000000000000 00007f7fec0008c0
ffffffffa01b8363 ffffffff81608c98
Jan 11 14:28:58 HOST kernel: Call Trace:
Jan 11 14:28:58 HOST kernel: [<ffffffff8116e2b9>] ? dump_stack+0x40/0x57
Jan 11 14:28:58 HOST kernel: [<ffffffffa020b88d>] ?
vmx_cpuid_update+0x5d/0x150 [kvm_intel]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01d8fdc>] ?
kvm_vcpu_ioctl_set_cpuid2+0x4c/0x70 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01b8363>] ?
kvm_arch_vcpu_ioctl+0x903/0xfa0 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffff810d1401>] ?
vunmap_page_range+0x211/0x330
Jan 11 14:28:58 HOST kernel: [<ffffffffa02088ab>] ?
vmx_vcpu_load+0x14b/0x1b0 [kvm_intel]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01b78a5>] ?
kvm_arch_vcpu_load+0x55/0x210 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01ad29b>] ?
kvm_vcpu_ioctl+0xbb/0x510 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01d9260>] ?
kvm_dev_ioctl_get_cpuid+0x210/0x2b0 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffff810f3a53>] ? do_vfs_ioctl+0x293/0x470
Jan 11 14:28:58 HOST kernel: [<ffffffff81034556>] ?
__do_page_fault+0x146/0x380
Jan 11 14:28:58 HOST kernel: [<ffffffff810f3c66>] ? SyS_ioctl+0x36/0x70
Jan 11 14:28:58 HOST kernel: [<ffffffff812d4eee>] ?

entry_SYSCALL_64_fastpath+0x12/0x71

# uname -r; cat /proc/cpuinfo | grep -E -m 2 "(flags|name)"
4.4.0-0-core2
model name      : Intel(R) Core(TM)2 CPU          6600  @ 2.40GHz
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl
aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
lahf_lm dtherm tpr_shadow

# uname -r; cat /proc/cpuinfo | grep -E -m 2 "(flags|name)"
4.4.0-0-core2
model name      : Intel(R) Core(TM)2 CPU          6600  @ 2.40GHz
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx lm
constant_tsc arch_perfmon nopl pni ssse3 cx16 x2apic tsc_deadline_timer
hypervisor lahf_lm tsc_adjust

# rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.479.el6_7.3.x86_64

Not sure if it's something new, HOST with 4.1 kernel didn't complain... TIA Z.

Han, Huaitong

2016-01-12 08:30 UTC #1 lore raw

I don't have a old machine to test it, please patch it and give me feedback. Thanks a lot.

From f06a85c76db210ce7145be6c704c44a9cef70ee3 Mon Sep 17 00:00:00 2001
From: Huaitong Han <[email protected]>
Date: Tue, 12 Jan 2016 16:04:20 +0800
Subject: [PATCH] kvm: x86: Fix vmwrite to SECONDARY_VM_EXEC_CONTROL

SECONDARY_VM_EXEC_CONTROL vmcs will be updated with vmx_cpuid_update, but it
will vmwrite error on the older CPU because the code is written without CPU_BASED_ACTIVATE_SECONDARY_CONTROLS check.

This will get rid of the following trace:

vmwrite error: reg 401e value 10 (err 12)

Call Trace:
[<ffffffff8116e2b9>] ? dump_stack+0x40/0x57
[<ffffffffa020b88d>] ?vmx_cpuid_update+0x5d/0x150 [kvm_intel]
[<ffffffffa01d8fdc>] ?kvm_vcpu_ioctl_set_cpuid2+0x4c/0x70 [kvm]
[<ffffffffa01b8363>] ?kvm_arch_vcpu_ioctl+0x903/0xfa0 [kvm]

Signen-off-by Huaitong Han <[email protected]>

---
 arch/x86/kvm/vmx.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

arch/x86/kvm/vmx.c +0 -0

index 44976a5..80f21d3 100644
@@ -8904,7 +8904,7 @@ static void vmx_cpuid_update(struct kvm_vcpu

*vcpu)
{
struct kvm_cpuid_entry2 *best;
struct vcpu_vmx *vmx = to_vmx(vcpu);
- u32 secondary_exec_ctl = vmx_secondary_exec_control(vmx);
+ u32 secondary_exec_ctl = -1;

if (vmx_rdtscp_supported()) {
bool rdtscp_enabled = guest_cpuid_has_rdtscp(vcpu);
@@ -8932,7 +8932,10 @@ static void vmx_cpuid_update(struct kvm_vcpu *vcpu)
best->ebx &= ~bit(X86_FEATURE_INVPCID);
}

- vmcs_set_secondary_exec_control(secondary_exec_ctl);
+ if (cpu_has_secondary_exec_ctrls()) {
+ secondary_exec_ctl &= vmx_secondary_exec_control(vmx);
+ vmcs_set_secondary_exec_control(secondary_exec_ctl);
+ }

if (static_cpu_has(X86_FEATURE_PCOMMIT) && nested) {
if (guest_cpuid_has_pcommit(vcpu))

Signature · 5 lines

2.4.3


Thanks
Huaitong

Quoted 113 lines · Zdenek Kaspar

Hi, today I compiled linux-4.4 kernel on older machine and noticed qemu
complains:

Jan 11 14:28:58 HOST kernel: vmwrite error: reg 401e value 10 (err
12)
Jan 11 14:28:58 HOST kernel: CPU: 0 PID: 6004 Comm: qemu-build Not
tainted 4.4.0-0-core2 #1
Jan 11 14:28:58 HOST kernel: Hardware name:                  /DG35EC,
BIOS ECG3510M.86A.0118.2010.0113.1426 01/13/2010
Jan 11 14:28:58 HOST kernel: ffff88007be68000 ffffffff8116e2b9
0000000000000000 ffffffffa020b88d
Jan 11 14:28:58 HOST kernel: ffff88007be68000 ffff880061173d80
fffffffffffffff2 ffffffffa01d8fdc
Jan 11 14:28:58 HOST kernel: 0000000000000000 00007f7ff40008c0
ffffffffa01b8363 ffffffff81608c98
Jan 11 14:28:58 HOST kernel: Call Trace:
Jan 11 14:28:58 HOST kernel: [<ffffffff8116e2b9>] ?
dump_stack+0x40/0x57
Jan 11 14:28:58 HOST kernel: [<ffffffffa020b88d>] ?
vmx_cpuid_update+0x5d/0x150 [kvm_intel]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01d8fdc>] ?
kvm_vcpu_ioctl_set_cpuid2+0x4c/0x70 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01b8363>] ?
kvm_arch_vcpu_ioctl+0x903/0xfa0 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffff810d1401>] ?
vunmap_page_range+0x211/0x330
Jan 11 14:28:58 HOST kernel: [<ffffffffa02088ab>] ?
vmx_vcpu_load+0x14b/0x1b0 [kvm_intel]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01b78a5>] ?
kvm_arch_vcpu_load+0x55/0x210 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01ad29b>] ?
kvm_vcpu_ioctl+0xbb/0x510 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01d9260>] ?
kvm_dev_ioctl_get_cpuid+0x210/0x2b0 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffff810f3a53>] ?
do_vfs_ioctl+0x293/0x470
Jan 11 14:28:58 HOST kernel: [<ffffffff81034556>] ?
__do_page_fault+0x146/0x380
Jan 11 14:28:58 HOST kernel: [<ffffffff810f3c66>] ?
SyS_ioctl+0x36/0x70
Jan 11 14:28:58 HOST kernel: [<ffffffff812d4eee>] ?
entry_SYSCALL_64_fastpath+0x12/0x71
Jan 11 14:28:58 HOST kernel: vmwrite error: reg 401e value 10 (err
12)
Jan 11 14:28:58 HOST kernel: CPU: 0 PID: 6005 Comm: qemu-build Not
tainted 4.4.0-0-core2 #1
Jan 11 14:28:58 HOST kernel: Hardware name:                  /DG35EC,
BIOS ECG3510M.86A.0118.2010.0113.1426 01/13/2010
Jan 11 14:28:58 HOST kernel: ffff88004af40000 ffffffff8116e2b9
0000000000000000 ffffffffa020b88d
Jan 11 14:28:58 HOST kernel: ffff88004af40000 ffff88007bc83d80
fffffffffffffff2 ffffffffa01d8fdc
Jan 11 14:28:58 HOST kernel: 0000000000000000 00007f7fec0008c0
ffffffffa01b8363 ffffffff81608c98
Jan 11 14:28:58 HOST kernel: Call Trace:
Jan 11 14:28:58 HOST kernel: [<ffffffff8116e2b9>] ?
dump_stack+0x40/0x57
Jan 11 14:28:58 HOST kernel: [<ffffffffa020b88d>] ?
vmx_cpuid_update+0x5d/0x150 [kvm_intel]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01d8fdc>] ?
kvm_vcpu_ioctl_set_cpuid2+0x4c/0x70 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01b8363>] ?
kvm_arch_vcpu_ioctl+0x903/0xfa0 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffff810d1401>] ?
vunmap_page_range+0x211/0x330
Jan 11 14:28:58 HOST kernel: [<ffffffffa02088ab>] ?
vmx_vcpu_load+0x14b/0x1b0 [kvm_intel]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01b78a5>] ?
kvm_arch_vcpu_load+0x55/0x210 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01ad29b>] ?
kvm_vcpu_ioctl+0xbb/0x510 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffffa01d9260>] ?
kvm_dev_ioctl_get_cpuid+0x210/0x2b0 [kvm]
Jan 11 14:28:58 HOST kernel: [<ffffffff810f3a53>] ?
do_vfs_ioctl+0x293/0x470
Jan 11 14:28:58 HOST kernel: [<ffffffff81034556>] ?
__do_page_fault+0x146/0x380
Jan 11 14:28:58 HOST kernel: [<ffffffff810f3c66>] ?
SyS_ioctl+0x36/0x70
Jan 11 14:28:58 HOST kernel: [<ffffffff812d4eee>] ?

entry_SYSCALL_64_fastpath+0x12/0x71

# uname -r; cat /proc/cpuinfo | grep -E -m 2 "(flags|name)"
4.4.0-0-core2
model name      : Intel(R) Core(TM)2 CPU          6600  @ 2.40GHz
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
pge
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl
aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
lahf_lm dtherm tpr_shadow

# uname -r; cat /proc/cpuinfo | grep -E -m 2 "(flags|name)"
4.4.0-0-core2
model name      : Intel(R) Core(TM)2 CPU          6600  @ 2.40GHz
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx lm
constant_tsc arch_perfmon nopl pni ssse3 cx16 x2apic
tsc_deadline_timer
hypervisor lahf_lm tsc_adjust

# rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.479.el6_7.3.x86_64

Not sure if it's something new, HOST with 4.1 kernel didn't complain...
TIA Z.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Paolo Bonzini

maintainer

2016-01-12 10:42 UTC #2 lore raw

Quoted 53 lines · Huaitong

I don't have a old machine to test it, please patch it and give me feedback. Thanks a lot.
From f06a85c76db210ce7145be6c704c44a9cef70ee3 Mon Sep 17 00:00:00 2001
From: Huaitong Han <[email protected]>
Date: Tue, 12 Jan 2016 16:04:20 +0800
Subject: [PATCH] kvm: x86: Fix vmwrite to SECONDARY_VM_EXEC_CONTROL
SECONDARY_VM_EXEC_CONTROL vmcs will be updated with vmx_cpuid_update, but it
will vmwrite error on the older CPU because the code is written without CPU_BASED_ACTIVATE_SECONDARY_CONTROLS check.
This will get rid of the following trace:
vmwrite error: reg 401e value 10 (err 12)
Call Trace:
[<ffffffff8116e2b9>] ? dump_stack+0x40/0x57
[<ffffffffa020b88d>] ?vmx_cpuid_update+0x5d/0x150 [kvm_intel]
[<ffffffffa01d8fdc>] ?kvm_vcpu_ioctl_set_cpuid2+0x4c/0x70 [kvm]
[<ffffffffa01b8363>] ?kvm_arch_vcpu_ioctl+0x903/0xfa0 [kvm]
Signen-off-by Huaitong Han <[email protected]>
---
 arch/x86/kvm/vmx.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx.c +1 -1
index 44976a5..80f21d3 100644
@@ -8904,7 +8904,7 @@ static void vmx_cpuid_update(struct kvm_vcpu
 *vcpu)
 {
 	struct kvm_cpuid_entry2 *best;
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
-	u32 secondary_exec_ctl = vmx_secondary_exec_control(vmx);
+	u32 secondary_exec_ctl = -1;
 
 	if (vmx_rdtscp_supported()) {
arch/x86/kvm/vmx.c +4 -1
 		bool rdtscp_enabled = guest_cpuid_has_rdtscp(vcpu);
@@ -8932,7 +8932,10 @@ static void vmx_cpuid_update(struct kvm_vcpu
 *vcpu)
 			best->ebx &= ~bit(X86_FEATURE_INVPCID);
 	}
 
-	vmcs_set_secondary_exec_control(secondary_exec_ctl);
+	if (cpu_has_secondary_exec_ctrls()) {
+		secondary_exec_ctl &= vmx_secondary_exec_control(vmx);
+		vmcs_set_secondary_exec_control(secondary_exec_ctl);
+	}
 
 	if (static_cpu_has(X86_FEATURE_PCOMMIT) && nested) {

Quoted 2 lines

if (guest_cpuid_has_pcommit(vcpu))

Looks good, thanks!

Paolo

Zdenek Kaspar

author

2016-01-12 17:58 UTC #3 lore raw

Quoted 55 lines · Paolo Bonzini

Quoted 52 lines · Huaitong
I don't have a old machine to test it, please patch it and give me feedback. Thanks a lot.
From f06a85c76db210ce7145be6c704c44a9cef70ee3 Mon Sep 17 00:00:00 2001
From: Huaitong Han <[email protected]>
Date: Tue, 12 Jan 2016 16:04:20 +0800
Subject: [PATCH] kvm: x86: Fix vmwrite to SECONDARY_VM_EXEC_CONTROL
SECONDARY_VM_EXEC_CONTROL vmcs will be updated with vmx_cpuid_update, but it
will vmwrite error on the older CPU because the code is written without CPU_BASED_ACTIVATE_SECONDARY_CONTROLS check.
This will get rid of the following trace:
vmwrite error: reg 401e value 10 (err 12)
Call Trace:
[<ffffffff8116e2b9>] ? dump_stack+0x40/0x57
[<ffffffffa020b88d>] ?vmx_cpuid_update+0x5d/0x150 [kvm_intel]
[<ffffffffa01d8fdc>] ?kvm_vcpu_ioctl_set_cpuid2+0x4c/0x70 [kvm]
[<ffffffffa01b8363>] ?kvm_arch_vcpu_ioctl+0x903/0xfa0 [kvm]
Signen-off-by Huaitong Han <[email protected]>
---
 arch/x86/kvm/vmx.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx.c +1 -1
index 44976a5..80f21d3 100644
@@ -8904,7 +8904,7 @@ static void vmx_cpuid_update(struct kvm_vcpu
 *vcpu)
 {
 	struct kvm_cpuid_entry2 *best;
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
-	u32 secondary_exec_ctl = vmx_secondary_exec_control(vmx);
+	u32 secondary_exec_ctl = -1;
 
 	if (vmx_rdtscp_supported()) {
arch/x86/kvm/vmx.c +4 -1
 		bool rdtscp_enabled = guest_cpuid_has_rdtscp(vcpu);
@@ -8932,7 +8932,10 @@ static void vmx_cpuid_update(struct kvm_vcpu
 *vcpu)
 			best->ebx &= ~bit(X86_FEATURE_INVPCID);
 	}
 
-	vmcs_set_secondary_exec_control(secondary_exec_ctl);
+	if (cpu_has_secondary_exec_ctrls()) {
+		secondary_exec_ctl &= vmx_secondary_exec_control(vmx);
+		vmcs_set_secondary_exec_control(secondary_exec_ctl);
+	}
 

Quoted 9 lines

Quoted 3 lines
arch/x86/kvm/vmx.c +0 -0
 	if (static_cpu_has(X86_FEATURE_PCOMMIT) && nested) {
if (guest_cpuid_has_pcommit(vcpu))
Looks good, thanks!
Paolo

Patch tested and works as expected. I see it's queued for v4.5 and with stable tag, so thank you both.