x86/kvm: fix interrupts based APF mechanism
Subsystems (3)
Vitaly Kuznetsov
reviewercover x86/kvm: fix interrupts based APF mechanism
Linux-5.9 switched KVM guests to interrupt based APF mechanism for 'page ready' events (instead of the previously used '#PF' exception) but a collision with the newly introduced IDTENTRY magic happened and it wasn't properly resolved. QEMU doesn't currently enable KVM_FEATURE_ASYNC_PF_INT bit so the breakage is invisible but all KVM guests will hang as soon as the bit will get exposed.
Vitaly Kuznetsov (2):
x86/kvm: properly use DEFINE_IDTENTRY_SYSVEC() macro x86/kvm: don't forget to ACK async PF IRQ
arch/x86/kernel/kvm.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
Signature · 1 line
2.25.4
Vitaly Kuznetsov
reviewerPATCH 1/2 x86/kvm: properly use DEFINE_IDTENTRY_SYSVEC() macro
DEFINE_IDTENTRY_SYSVEC() already contains irqentry_enter()/ irqentry_exit().
--- arch/x86/kernel/kvm.c | 4 ---- 1 file changed, 4 deletions(-)
arch/x86/kernel/kvm.c +0 -4
@@ -270,9 +270,6 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_kvm_asyncpf_interrupt) {struct pt_regs *old_regs =set_irq_regs (regs);u32 token; -irqentry_state_t state; - - state =irqentry_enter (regs);inc_irq_stat (irq_hv_callback_count);
@@ -283,7 +280,6 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_kvm_asyncpf_interrupt)wrmsrl (MSR_KVM_ASYNC_PF_ACK ,1 ); } -irqentry_exit (regs, state);set_irq_regs (old_regs); }
Signature · 1 line
2.25.4
Vitaly Kuznetsov
reviewerPATCH 2/2 x86/kvm: don't forget to ACK async PF IRQ
In reply to Vitaly KuznetsovMerge commit 26d05b368a5c0 ("Merge branch 'kvm-async-pf-int' into HEAD") tried to adapt the new interrupt based async PF mechanism to the newly introduced IDTENTRY magic but unfortunately it missed the fact that DEFINE_IDTENTRY_SYSVEC() doesn't call ack_APIC_irq() on its own and all DEFINE_IDTENTRY_SYSVEC() users have to call it manually.
As the result all multi-CPU KVM guest hang on boot when KVM_FEATURE_ASYNC_PF_INT is present. The breakage went unnoticed because no KVM userspace (e.g. QEMU) currently set it (and thus async PF mechanism is currently disabled) but we're about to change that.
--- arch/x86/kernel/kvm.c | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/kernel/kvm.c +2 -0
@@ -271,6 +271,8 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_kvm_asyncpf_interrupt)struct pt_regs *old_regs =set_irq_regs (regs);u32 token; +ack_APIC_irq (); +inc_irq_stat (irq_hv_callback_count);if (__this_cpu_read (apf_reason.enabled)) {
Signature · 1 line
2.25.4
Re: [PATCH 2/2] x86/kvm: don't forget to ACK async PF IRQ
Quoted 6 lines · Vitaly Kuznetsov
Merge commit
26d05b368a5c0("Merge branch 'kvm-async-pf-int' into HEAD") tried to adapt the new interrupt based async PF mechanism to the newly introduced IDTENTRY magic but unfortunately it missed the fact thatDEFINE_IDTENTRY_SYSVEC()doesn't callack_APIC_irq()on its own and allDEFINE_IDTENTRY_SYSVEC()users have to call it manually.
Quoted 7 lines
As the result all multi-CPU KVM guest hang on boot when
KVM_FEATURE_ASYNC_PF_INTis present. The breakage went unnoticed because no KVM userspace (e.g. QEMU) currently set it (and thus async PF mechanism is currently disabled) but we're about to change that.Fixes 26d05b368a5c0 ("Merge branch 'kvm-async-pf-int' into HEAD")Signed-off-by Vitaly Kuznetsov <[email protected]>
This also fixes a kvmtool regression, but interestingly it does not set KVM_FEATURE_ASYNC_PF_INT either AFAICS:
kepler:~/kvmtool.git> git grep KVM_FEATURE_ASYNC_PF_INT kepler:~/kvmtool.git>
kepler:~/kvmtool.git> grep url .git/config
url = https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git
So either I missed the flag-setting in the kvmtools.git source, or maybe there's some other way to trigger this bug?
Anyway, please handle this as a v5.9 regression:
Thanks,
Ingo
Vitaly Kuznetsov
reviewerRe: [PATCH 2/2] x86/kvm: don't forget to ACK async PF IRQ
Quoted 13 lines · Ingo Molnar
Quoted 11 lines · Vitaly Kuznetsov
Merge commit
26d05b368a5c0("Merge branch 'kvm-async-pf-int' into HEAD") tried to adapt the new interrupt based async PF mechanism to the newly introduced IDTENTRY magic but unfortunately it missed the fact thatDEFINE_IDTENTRY_SYSVEC()doesn't callack_APIC_irq()on its own and allDEFINE_IDTENTRY_SYSVEC()users have to call it manually.As the result all multi-CPU KVM guest hang on boot when
KVM_FEATURE_ASYNC_PF_INTis present. The breakage went unnoticed because no KVM userspace (e.g. QEMU) currently set it (and thus async PF mechanism is currently disabled) but we're about to change that.
Quoted 8 lines
Quoted 2 lines
Fixes 26d05b368a5c0 ("Merge branch 'kvm-async-pf-int' into HEAD")Signed-off-by Vitaly Kuznetsov <[email protected]>This also fixes a kvmtool regression, but interestingly it does not set
KVM_FEATURE_ASYNC_PF_INTeither AFAICS:kepler:~/kvmtool.git> git grep
KVM_FEATURE_ASYNC_PF_INTkepler:~/kvmtool.git>
My wild guess would be that kvmtool doesn't manually set any of the KVM PV features:
[vitty@vitty kvmtool]$ git grep KVM_FEATURE_
[vitty@vitty kvmtool]$
it just blindly passes whatever it gets from KVM via KVM_GET_SUPPORTED_CPUID to KVM_SET_CPUID2 and KVM_FEATURE_ASYNC_PF_INT among other PV features is set there by default.
Quoted 10 lines
kepler:~/kvmtool.git> grep url .git/config
url = https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.gitSo either I missed the flag-setting in the kvmtools.git source, or maybe there's some other way to trigger this bug?
Anyway, please handle this as a v5.9 regression:
Tested-by Ingo Molnar <[email protected]>
Thanks!
Signature · 1 line
Vitaly
Re: [PATCH 2/2] x86/kvm: don't forget to ACK async PF IRQ
Quoted 38 lines · Vitaly Kuznetsov
Quoted 13 lines · Ingo Molnar
Quoted 11 lines · Vitaly Kuznetsov
Merge commit
26d05b368a5c0("Merge branch 'kvm-async-pf-int' into HEAD") tried to adapt the new interrupt based async PF mechanism to the newly introduced IDTENTRY magic but unfortunately it missed the fact thatDEFINE_IDTENTRY_SYSVEC()doesn't callack_APIC_irq()on its own and allDEFINE_IDTENTRY_SYSVEC()users have to call it manually.As the result all multi-CPU KVM guest hang on boot when
KVM_FEATURE_ASYNC_PF_INTis present. The breakage went unnoticed because no KVM userspace (e.g. QEMU) currently set it (and thus async PF mechanism is currently disabled) but we're about to change that.Quoted 8 lines
Quoted 2 lines
Fixes 26d05b368a5c0 ("Merge branch 'kvm-async-pf-int' into HEAD")Signed-off-by Vitaly Kuznetsov <[email protected]>This also fixes a kvmtool regression, but interestingly it does not set
KVM_FEATURE_ASYNC_PF_INTeither AFAICS:kepler:~/kvmtool.git> git grep
KVM_FEATURE_ASYNC_PF_INTkepler:~/kvmtool.git>My wild guess would be that kvmtool doesn't manually set any of the KVM PV features:
[vitty@vitty kvmtool]$ git grep
KVM_FEATURE_
[vitty@vitty kvmtool]$it just blindly passes whatever it gets from KVM via
KVM_GET_SUPPORTED_CPUIDtoKVM_SET_CPUID2andKVM_FEATURE_ASYNC_PF_INTamong other PV features is set there by default.Quoted 4 lines
kepler:~/kvmtool.git> grep url .git/config
url = https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git
Quoted 9 lines
Quoted 6 lines
So either I missed the flag-setting in the kvmtools.git source, or maybe there's some other way to trigger this bug?
Anyway, please handle this as a v5.9 regression:
Tested-by Ingo Molnar <[email protected]>Thanks!
Queued, thanks.
Paolo