i386: Don't try to set MSR_KVM_ASYNC_PF_EN if kernel-irqchip=off
PATCH 1/1 i386: Don't try to set MSR_KVM_ASYNC_PF_EN if kernel-irqchip=off
This addresses the following crash when running Linux v5.8 with kernel-irqchip=off:
qemu-system-x86_64: error: failed to set MSR 0x4b564d02 to 0x0
qemu-system-x86_64: ../target/i386/kvm.c:2714: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
There is a kernel-side fix for the issue too (kernel commit d831de177217 "KVM: x86: always allow writing '0' to MSR_KVM_ASYNC_PF_EN"), but it's nice to simply not trigger the bug if running an older kernel.
--- target/i386/kvm.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
target/i386/kvm.c +6 -1
@@ -2818,7 +2818,12 @@ static int kvm_put_msrs(X86CPU *cpu, int level)kvm_msr_entry_add (cpu,MSR_IA32_TSC , env->tsc);kvm_msr_entry_add (cpu,MSR_KVM_SYSTEM_TIME , env->system_time_msr);kvm_msr_entry_add (cpu,MSR_KVM_WALL_CLOCK , env->wall_clock_msr); -if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_ASYNC_PF )) { +/* +* Some kernel versions (v5.8) won't let MSR_KVM_ASYNC_PF_EN to be set +* at all if kernel-irqchip=off, so don't try to set it in that case. +*/ +if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_ASYNC_PF ) && +kvm_irqchip_in_kernel ()) {kvm_msr_entry_add (cpu,MSR_KVM_ASYNC_PF_EN , env->async_pf_en_msr); }if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_PV_EOI )) {
Signature · 1 line
2.26.2
Quoted 25 lines · Eduardo Habkost
This addresses the following crash when running Linux v5.8 with kernel-irqchip=off:
qemu-system-x86_64: error: failed to set MSR
0x4b564d02to0x0
qemu-system-x86_64:../target/i386/kvm.c:2714:kvm_buf_set_msrs: Assertion `ret ==cpu->kvm_msr_buf->nmsrs' failed.There is a kernel-side fix for the issue too (kernel commit
d831de177217"KVM: x86: always allow writing '0' toMSR_KVM_ASYNC_PF_EN"), but it's nice to simply not trigger the bug if running an older kernel.Signed-off-by Eduardo Habkost <[email protected]>--- target/i386/kvm.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)target/i386/kvm.c +0
-0@@ -2818,7 +2818,12 @@ static int kvm_put_msrs(X86CPU *cpu, int level)kvm_msr_entry_add (cpu,MSR_IA32_TSC , env->tsc);kvm_msr_entry_add (cpu,MSR_KVM_SYSTEM_TIME , env->system_time_msr);kvm_msr_entry_add (cpu,MSR_KVM_WALL_CLOCK , env->wall_clock_msr);
target/i386/kvm.c static int kvm_put_msrs(X86CPU *cpu, int level) · line 2829
Quoted 10 lines
target/i386/kvm.c +6
-1-if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_ASYNC_PF )) { +/* +* Some kernel versions (v5.8) won't let MSR_KVM_ASYNC_PF_EN to be set +* at all if kernel-irqchip=off, so don't try to set it in that case. +*/ +if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_ASYNC_PF ) && +kvm_irqchip_in_kernel ()) {kvm_msr_entry_add (cpu,MSR_KVM_ASYNC_PF_EN , env->async_pf_en_msr); }if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_PV_EOI )) {
I'm not sure kvm_irqchip_in_kernel() was required before we switched to interrupt-based APF (as we were always injecting #PF) but with kernel-5.8+ this should work. We'll need to merge this with
https://lists.nongnu.org/archive/html/qemu-devel/2020-09/msg02963.html
(queued by Paolo) and
https://lists.nongnu.org/archive/html/qemu-devel/2020-09/msg06196.html
which fixes a bug in it.
as kvm_irqchip_in_kernel() should go around both KVM_FEATURE_ASYNC_PF and KVM_FEATURE_ASYNC_PF_INT I believe.
Signature · 1 line
Vitaly
Quoted 33 lines · Vitaly Kuznetsov
Quoted 31 lines · Eduardo Habkost
This addresses the following crash when running Linux v5.8 with kernel-irqchip=off:
qemu-system-x86_64: error: failed to set MSR
0x4b564d02to0x0
qemu-system-x86_64:../target/i386/kvm.c:2714:kvm_buf_set_msrs: Assertion `ret ==cpu->kvm_msr_buf->nmsrs' failed.There is a kernel-side fix for the issue too (kernel commit
d831de177217"KVM: x86: always allow writing '0' toMSR_KVM_ASYNC_PF_EN"), but it's nice to simply not trigger the bug if running an older kernel.Signed-off-by Eduardo Habkost <[email protected]>--- target/i386/kvm.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)target/i386/kvm.c +5
-1@@ -2818,7 +2818,12 @@ static int kvm_put_msrs(X86CPU *cpu, int level)kvm_msr_entry_add (cpu,MSR_IA32_TSC , env->tsc);kvm_msr_entry_add (cpu,MSR_KVM_SYSTEM_TIME , env->system_time_msr);kvm_msr_entry_add (cpu,MSR_KVM_WALL_CLOCK , env->wall_clock_msr); -if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_ASYNC_PF )) { +/* +* Some kernel versions (v5.8) won't let MSR_KVM_ASYNC_PF_EN to be set +* at all if kernel-irqchip=off, so don't try to set it in that case. +*/ + if (env->features[FEAT_KVM] & (1 << KVM_FEATURE_ASYNC_PF) &&
Quoted 8 lines
target/i386/kvm.c static int kvm_put_msrs(X86CPU *cpu, int level) · line 2829
Quoted 4 lines
target/i386/kvm.c +1
-0+ kvm_irqchip_in_kernel()) {kvm_msr_entry_add (cpu,MSR_KVM_ASYNC_PF_EN , env->async_pf_en_msr); }if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_PV_EOI )) {I'm not sure
kvm_irqchip_in_kernel()was required before we switched to interrupt-based APF (as we were always injecting #PF) but with kernel-5.8+ this should work. [...]
Were guests able to set MSR_KVM_ASYNC_PF_EN to non-zero with kernel-irqchip=off on hosts running Linux <= 5.7? I am assuming kvm-asyncpf never worked with kernel-irqchip=off (and enabling it by default with kernel-irqchip=off was a mistake).
Quoted 9 lines
[...] We'll need to merge this with
https://lists.nongnu.org/archive/html/qemu-devel/2020-09/msg02963.html
(queued by Paolo) and
https://lists.nongnu.org/archive/html/qemu-devel/2020-09/msg06196.html
which fixes a bug in it.as
kvm_irqchip_in_kernel()should go around bothKVM_FEATURE_ASYNC_PFandKVM_FEATURE_ASYNC_PF_INTI believe.
Shouldn't we just disallow kvm-asyncpf-int=on if kernel-irqchip=off?
Signature · 1 line
Eduardo
Quoted 43 lines · Eduardo Habkost
Quoted 33 lines · Vitaly Kuznetsov
Quoted 31 lines · Eduardo Habkost
This addresses the following crash when running Linux v5.8 with kernel-irqchip=off:
qemu-system-x86_64: error: failed to set MSR
0x4b564d02to0x0
qemu-system-x86_64:../target/i386/kvm.c:2714:kvm_buf_set_msrs: Assertion `ret ==cpu->kvm_msr_buf->nmsrs' failed.There is a kernel-side fix for the issue too (kernel commit
d831de177217"KVM: x86: always allow writing '0' toMSR_KVM_ASYNC_PF_EN"), but it's nice to simply not trigger the bug if running an older kernel.Signed-off-by Eduardo Habkost <[email protected]>--- target/i386/kvm.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)target/i386/kvm.c +5
-1@@ -2818,7 +2818,12 @@ static int kvm_put_msrs(X86CPU *cpu, int level)kvm_msr_entry_add (cpu,MSR_IA32_TSC , env->tsc);kvm_msr_entry_add (cpu,MSR_KVM_SYSTEM_TIME , env->system_time_msr);kvm_msr_entry_add (cpu,MSR_KVM_WALL_CLOCK , env->wall_clock_msr); -if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_ASYNC_PF )) { +/* +* Some kernel versions (v5.8) won't let MSR_KVM_ASYNC_PF_EN to be set +* at all if kernel-irqchip=off, so don't try to set it in that case. +*/ + if (env->features[FEAT_KVM] & (1 << KVM_FEATURE_ASYNC_PF) &&Quoted 8 lines
target/i386/kvm.c static int kvm_put_msrs(X86CPU *cpu, int level) · line 2829
Quoted 4 lines
target/i386/kvm.c +1
-0+ kvm_irqchip_in_kernel()) {kvm_msr_entry_add (cpu,MSR_KVM_ASYNC_PF_EN , env->async_pf_en_msr); }if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_PV_EOI )) {I'm not sure
kvm_irqchip_in_kernel()was required before we switched to interrupt-based APF (as we were always injecting #PF) but with kernel-5.8+ this should work. [...]
Quoted 2 lines
Were guests able to set
MSR_KVM_ASYNC_PF_ENto non-zero with kernel-irqchip=off on hosts running Linux <= 5.7?
lapic_in_kernel() check appeared in kernel with the following commit:
commit 9d3c447c72fb2337ca39f245c6ae89f2369de216
Author: Wanpeng Li <[email protected]> Date: Mon Jun 29 18:26:31 2020 +0800
KVM: X86: Fix async pf caused null-ptr-deref
which was post-interrupt-based-APF. I *think* it was OK to enable APF with !lapic_in_kernel() before (at least I don't see what would not allow that).
Quoted 12 lines
I am assuming
kvm-asyncpf never worked with kernel-irqchip=off (and enabling it by default with kernel-irqchip=off was a mistake).Quoted 7 lines
[...] We'll need to merge this with
https://lists.nongnu.org/archive/html/qemu-devel/2020-09/msg02963.html
(queued by Paolo) and
https://lists.nongnu.org/archive/html/qemu-devel/2020-09/msg06196.html
which fixes a bug in it.
Quoted 4 lines
Quoted 2 lines
as
kvm_irqchip_in_kernel()should go around bothKVM_FEATURE_ASYNC_PFandKVM_FEATURE_ASYNC_PF_INTI believe.Shouldn't we just disallow kvm-asyncpf-int=on if kernel-irqchip=off?
(Sarcasm: if disallowing 'kernel-irqchip=off' is not an option, then) yes, we probably can, but kvm-asyncpf-int=on is the default we have so we can't just implicitly change it underneath or migration will break...
Signature · 1 line
Vitaly
Quoted 51 lines · Vitaly Kuznetsov
Quoted 43 lines · Eduardo Habkost
Quoted 33 lines · Vitaly Kuznetsov
Quoted 31 lines · Eduardo Habkost
This addresses the following crash when running Linux v5.8 with kernel-irqchip=off:
qemu-system-x86_64: error: failed to set MSR
0x4b564d02to0x0
qemu-system-x86_64:../target/i386/kvm.c:2714:kvm_buf_set_msrs: Assertion `ret ==cpu->kvm_msr_buf->nmsrs' failed.There is a kernel-side fix for the issue too (kernel commit
d831de177217"KVM: x86: always allow writing '0' toMSR_KVM_ASYNC_PF_EN"), but it's nice to simply not trigger the bug if running an older kernel.Signed-off-by Eduardo Habkost <[email protected]>--- target/i386/kvm.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)target/i386/kvm.c +5
-1@@ -2818,7 +2818,12 @@ static int kvm_put_msrs(X86CPU *cpu, int level)kvm_msr_entry_add (cpu,MSR_IA32_TSC , env->tsc);kvm_msr_entry_add (cpu,MSR_KVM_SYSTEM_TIME , env->system_time_msr);kvm_msr_entry_add (cpu,MSR_KVM_WALL_CLOCK , env->wall_clock_msr); -if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_ASYNC_PF )) { +/* +* Some kernel versions (v5.8) won't let MSR_KVM_ASYNC_PF_EN to be set +* at all if kernel-irqchip=off, so don't try to set it in that case. +*/ + if (env->features[FEAT_KVM] & (1 << KVM_FEATURE_ASYNC_PF) &&Quoted 8 lines
target/i386/kvm.c static int kvm_put_msrs(X86CPU *cpu, int level) · line 2829
Quoted 4 lines
target/i386/kvm.c +1
-0+ kvm_irqchip_in_kernel()) {kvm_msr_entry_add (cpu,MSR_KVM_ASYNC_PF_EN , env->async_pf_en_msr); }if (env->features[FEAT_KVM ] & (1 <<KVM_FEATURE_PV_EOI )) {I'm not sure
kvm_irqchip_in_kernel()was required before we switched to interrupt-based APF (as we were always injecting #PF) but with kernel-5.8+ this should work. [...]Quoted 2 lines
Were guests able to set
MSR_KVM_ASYNC_PF_ENto non-zero with kernel-irqchip=off on hosts running Linux <= 5.7?
lapic_in_kernel()check appeared in kernel with the following commit:commit
9d3c447c72fb2337ca39f245c6ae89f2369de216
Quoted 8 lines
Author: Wanpeng Li <[email protected]> Date: Mon Jun 29 18:26:31 2020 +0800KVM: X86: Fix async pf caused null-ptr-deref
which was post-interrupt-based-APF. I *think* it was OK to enable APF with !lapic_in_kernel() before (at least I don't see what would not allow that).
If it was possible, did KVM break live migration of kernel-irqchip=off guests that enabled APF? This would mean my patch is replacing a crash with a silent migration bug. Not nice either way.
Quoted 13 lines
Quoted 13 lines
I am assuming
kvm-asyncpf never worked with kernel-irqchip=off (and enabling it by default with kernel-irqchip=off was a mistake).Quoted 7 lines
[...] We'll need to merge this with
https://lists.nongnu.org/archive/html/qemu-devel/2020-09/msg02963.html
(queued by Paolo) and
https://lists.nongnu.org/archive/html/qemu-devel/2020-09/msg06196.html
which fixes a bug in it.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
as
kvm_irqchip_in_kernel()should go around bothKVM_FEATURE_ASYNC_PFandKVM_FEATURE_ASYNC_PF_INTI believe.Shouldn't we just disallow kvm-asyncpf-int=on if kernel-irqchip=off?
(Sarcasm: if disallowing 'kernel-irqchip=off' is not an option, then)
I'm working on it. :-)
Quoted 2 lines
yes, we probably can, but kvm-asyncpf-int=on is the default we have so we can't just implicitly change it underneath or migration will break...
kvm-asyncpf-int wasn't merged yet, was it? This means we don't have compatibility issues to care about.
Signature · 1 line
Eduardo
Paolo Bonzini
maintainerQuoted 4 lines · Eduardo Habkost
If it was possible, did KVM break live migration of kernel-irqchip=off guests that enabled APF? This would mean my patch is replacing a crash with a silent migration bug. Not nice either way.
Let's drop kernel-irqchip=off completely so migration is not broken. :)
I'm actually serious, I don't think we need a deprecation period even.
Paolo
Quoted 9 lines · Paolo Bonzini
Quoted 4 lines · Eduardo Habkost
If it was possible, did KVM break live migration of kernel-irqchip=off guests that enabled APF? This would mean my patch is replacing a crash with a silent migration bug. Not nice either way.
Let's drop kernel-irqchip=off completely so migration is not broken. :)
I'm actually serious, I don't think we need a deprecation period even.
I wasn't sure about this, but then I've noticed the man page says "disabling the in-kernel irqchip completely is not recommended except for debugging purposes."
Does this note apply to all architectures?
Signature · 1 line
Eduardo
Paolo Bonzini
maintainerIl mar 22 set 2020, 21:39 Eduardo Habkost <[email protected]> ha scritto:
Quoted 9 lines · Paolo Bonzini
Quoted 8 lines · Eduardo Habkost
Quoted 4 lines
If it was possible, did KVM break live migration of kernel-irqchip=off guests that enabled APF? This would mean my patch is replacing a crash with a silent migration bug. Not nice either way.
Let's drop kernel-irqchip=off completely so migration is not broken. :)
Quoted 8 lines
Quoted 1 line
I'm actually serious, I don't think we need a deprecation period even.
I wasn't sure about this, but then I've noticed the man page says "disabling the in-kernel irqchip completely is not recommended except for debugging purposes."
Does this note apply to all architectures?
At least on ARM, Raspberry Pi 4 cannot use kernel-irqchip=on at all. But remember I am only talking about x86.
Paolo
Quoted 4 lines
--
Eduardo