KVM/arm updates for 5.3-rc4
Subsystems (4)
Marc Zyngier
maintainerPaolo, Radim,
Here's a set of update for -rc4. Yet another reset fix, and two subtle VGIC fixes for issues that can be observed in interesting corner cases.
Note that this is on top of kvmarm-fixes-for-5.3[1], which hasn't been pulled yet. Hopefully you can pull both at the same time!
Thanks,
M.
[1] https://lore.kernel.org/kvmarm/[email protected]
The following changes since commit cdb2d3ee0436d74fa9092f2df46aaa6f9e03c969:
arm64: KVM: hyp: debug-sr: Mark expected switch fall-through (2019-07-29 11:01:37 +0100)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm.git tags/kvmarm-fixes-for-5.3-2
for you to fetch changes up to 16e604a437c89751dc626c9e90cf88ba93c5be64:
KVM: arm/arm64: vgic: Reevaluate level sensitive interrupts on enable (2019-08-09 08:07:26 +0100)
----------------------------------------------------------------
KVM/arm fixes for 5.3, take #2
- Fix our system register reset so that we stop writing non-sensical values to them, and track which registers get reset instead.
- Sync VMCR back from the GIC on WFI so that KVM has an exact vue of PMR.
- Reevaluate state of HW-mapped, level triggered interrupts on enable.
----------------------------------------------------------------
Alexandru Elisei (1):
KVM: arm/arm64: vgic: Reevaluate level sensitive interrupts on enable
Marc Zyngier (3):
KVM: arm/arm64: Sync ICH_VMCR_EL2 back when about to block
KVM: arm64: Don't write junk to sysregs on reset
KVM: arm: Don't write junk to CP15 registers on reset
arch/arm/kvm/coproc.c | 23 +++++++++++++++-------- arch/arm64/kvm/sys_regs.c | 32 ++++++++++++++++++-------------- include/kvm/arm_vgic.h | 1 + virt/kvm/arm/arm.c | 11 +++++++++++ virt/kvm/arm/vgic/vgic-mmio.c | 16 ++++++++++++++++ virt/kvm/arm/vgic/vgic-v2.c | 9 ++++++++- virt/kvm/arm/vgic/vgic-v3.c | 7 ++++++- virt/kvm/arm/vgic/vgic.c | 11 +++++++++++ virt/kvm/arm/vgic/vgic.h | 2 ++ 9 files changed, 88 insertions(+), 24 deletions(-)
Marc Zyngier
maintainerPATCH 1/4 KVM: arm/arm64: Sync ICH_VMCR_EL2 back when about to block
Since commit commit 328e56647944 ("KVM: arm/arm64: vgic: Defer touching GICH_VMCR to vcpu_load/put"), we leave ICH_VMCR_EL2 (or its GICv2 equivalent) loaded as long as we can, only syncing it back when we're scheduled out.
There is a small snag with that though: kvm_vgic_vcpu_pending_irq(), which is indirectly called from kvm_vcpu_check_block(), needs to evaluate the guest's view of ICC_PMR_EL1. At the point were we call kvm_vcpu_check_block(), the vcpu is still loaded, and whatever changes to PMR is not visible in memory until we do a vcpu_put().
Things go really south if the guest does the following:
mov x0, #0 // or any small value masking interrupts
msr ICC_PMR_EL1, x0
[vcpu preempted, then rescheduled, VMCR sampled]
mov x0, #ff // allow all interrupts
msr ICC_PMR_EL1, x0
wfi // traps to EL2, so samping of VMCR
[interrupt arrives just after WFI]
Here, the hypervisor's view of PMR is zero, while the guest has enabled its interrupts. kvm_vgic_vcpu_pending_irq() will then say that no interrupts are pending (despite an interrupt being received) and we'll block for no reason. If the guest doesn't have a periodic interrupt firing once it has blocked, it will stay there forever.
To avoid this unfortuante situation, let's resync VMCR from kvm_arch_vcpu_blocking(), ensuring that a following kvm_vcpu_check_block() will observe the latest value of PMR.
This has been found by booting an arm64 Linux guest with the pseudo NMI feature, and thus using interrupt priorities to mask interrupts instead of the usual PSTATE masking.
--- include/kvm/arm_vgic.h | 1 + virt/kvm/arm/arm.c | 11 +++++++++++ virt/kvm/arm/vgic/vgic-v2.c | 9 ++++++++- virt/kvm/arm/vgic/vgic-v3.c | 7 ++++++- virt/kvm/arm/vgic/vgic.c | 11 +++++++++++ virt/kvm/arm/vgic/vgic.h | 2 ++ 6 files changed, 39 insertions(+), 2 deletions(-)
include/kvm/arm_vgic.h +1 -0
@@ -350,6 +350,7 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);void kvm_vgic_load (struct kvm_vcpu *vcpu);void kvm_vgic_put (struct kvm_vcpu *vcpu); +void kvm_vgic_vmcr_sync (struct kvm_vcpu *vcpu);#define irqchip_in_kernel (k) (!!((k)->arch.vgic.in_kernel))#define vgic_initialized (k) ((k)->arch.vgic.initialized)
virt/kvm/arm/arm.c +11 -0
@@ -323,6 +323,17 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)void kvm_arch_vcpu_blocking (struct kvm_vcpu *vcpu) { +/* +* If we're about to block (most likely because we've just hit a +* WFI), we need to sync back the state of the GIC CPU interface +* so that we have the lastest PMR and group enables. This ensures +* that kvm_arch_vcpu_runnable has up-to-date data to decide +* whether we have pending interrupts. +*/ +preempt_disable (); +kvm_vgic_vmcr_sync (vcpu); +preempt_enable (); +kvm_vgic_v4_enable_doorbell (vcpu); }
virt/kvm/arm/vgic/vgic-v2.c +8 -1
@@ -484,10 +484,17 @@ void vgic_v2_load(struct kvm_vcpu *vcpu) kvm_vgic_global_state.vctrl_base + GICH_APR); } -void vgic_v2_put (struct kvm_vcpu *vcpu) +void vgic_v2_vmcr_sync (struct kvm_vcpu *vcpu) {struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2; cpu_if->vgic_vmcr =readl_relaxed (kvm_vgic_global_state.vctrl_base +GICH_VMCR ); +} + +void vgic_v2_put (struct kvm_vcpu *vcpu) +{ +struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2; + +vgic_v2_vmcr_sync (vcpu); cpu_if->vgic_apr =readl_relaxed (kvm_vgic_global_state.vctrl_base +GICH_APR ); }
virt/kvm/arm/vgic/vgic-v3.c +6 -1
@@ -662,12 +662,17 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)__vgic_v3_activate_traps (vcpu); } -void vgic_v3_put (struct kvm_vcpu *vcpu) +void vgic_v3_vmcr_sync (struct kvm_vcpu *vcpu) {struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;if (likely (cpu_if->vgic_sre)) cpu_if->vgic_vmcr =kvm_call_hyp_ret (__vgic_v3_read_vmcr); +} + +void vgic_v3_put (struct kvm_vcpu *vcpu) +{ +vgic_v3_vmcr_sync (vcpu);kvm_call_hyp (__vgic_v3_save_aprs, vcpu);
virt/kvm/arm/vgic/vgic.c +11 -0
@@ -919,6 +919,17 @@ void kvm_vgic_put(struct kvm_vcpu *vcpu)vgic_v3_put (vcpu); } +void kvm_vgic_vmcr_sync (struct kvm_vcpu *vcpu) +{ +if (unlikely (!irqchip_in_kernel (vcpu->kvm))) +return ; + +if (kvm_vgic_global_state.type ==VGIC_V2 ) +vgic_v2_vmcr_sync (vcpu); +else +vgic_v3_vmcr_sync (vcpu); +} +int kvm_vgic_vcpu_pending_irq (struct kvm_vcpu *vcpu) {struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
virt/kvm/arm/vgic/vgic.h +2 -0
@@ -193,6 +193,7 @@ int vgic_register_dist_iodev(struct kvm *kvm, gpa_t dist_base_address,void vgic_v2_init_lrs (void );void vgic_v2_load (struct kvm_vcpu *vcpu);void vgic_v2_put (struct kvm_vcpu *vcpu); +void vgic_v2_vmcr_sync (struct kvm_vcpu *vcpu);void vgic_v2_save_state (struct kvm_vcpu *vcpu);void vgic_v2_restore_state (struct kvm_vcpu *vcpu);
@@ -223,6 +224,7 @@ bool vgic_v3_check_base(struct kvm *kvm);void vgic_v3_load (struct kvm_vcpu *vcpu);void vgic_v3_put (struct kvm_vcpu *vcpu); +void vgic_v3_vmcr_sync (struct kvm_vcpu *vcpu);bool vgic_has_its (struct kvm *kvm);int kvm_vgic_register_its_device (void );
Signature · 1 line
2.20.1
Marc Zyngier
maintainerPATCH 2/4 KVM: arm64: Don't write junk to sysregs on reset
In reply to Marc ZyngierAt the moment, the way we reset system registers is mildly insane:
We write junk to them, call the reset functions, and then check that we have something else in them.
The "fun" thing is that this can happen while the guest is running (PSCI, for example). If anything in KVM has to evaluate the state of a system register while junk is in there, bad thing may happen.
Let's stop doing that. Instead, we track that we have called a reset function for that register, and assume that the reset function has done something. This requires fixing a couple of sysreg refinition in the trap table.
In the end, the very need of this reset check is pretty dubious, as it doesn't check everything (a lot of the sysregs leave outside of the sys_regs[] array). It may well be axed in the near future.
--- arch/arm64/kvm/sys_regs.c | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-)
arch/arm64/kvm/sys_regs.c +18 -14
@@ -632,7 +632,7 @@ static void reset_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)*/ val = ((pmcr & ~ARMV8_PMU_PMCR_MASK ) | (ARMV8_PMU_PMCR_MASK &0xdecafbad )) & (~ARMV8_PMU_PMCR_E ); -__vcpu_sys_reg (vcpu,PMCR_EL0 ) = val; +__vcpu_sys_reg (vcpu, r->reg) = val; }static bool check_pmu_access_disabled (struct kvm_vcpu *vcpu,u64 flags)
@@ -981,13 +981,13 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,/* Silly macro to expand the DBG{BCR,BVR,WVR,WCR}n_EL1 registers in one go */ #define DBG_BCR_BVR_WCR_WVR_EL1 (n) \ { SYS_DESC(SYS_DBGBVRn_EL1(n)), \ - trap_bvr, reset_bvr, n, 0, get_bvr, set_bvr }, \ + trap_bvr, reset_bvr, 0, 0, get_bvr, set_bvr }, \ { SYS_DESC(SYS_DBGBCRn_EL1(n)), \ - trap_bcr, reset_bcr, n, 0, get_bcr, set_bcr }, \ + trap_bcr, reset_bcr, 0, 0, get_bcr, set_bcr }, \ { SYS_DESC(SYS_DBGWVRn_EL1(n)), \ - trap_wvr, reset_wvr, n, 0, get_wvr, set_wvr }, \ + trap_wvr, reset_wvr, 0, 0, get_wvr, set_wvr }, \ { SYS_DESC(SYS_DBGWCRn_EL1(n)), \ - trap_wcr, reset_wcr, n, 0, get_wcr, set_wcr } + trap_wcr, reset_wcr, 0, 0, get_wcr, set_wcr }/* Macro to expand the PMEVCNTRn_EL0 register */ #define PMU_PMEVCNTR_EL0 (n) \
@@ -1540,7 +1540,7 @@ static const struct sys_reg_desc sys_reg_descs[] = { { SYS_DESC(SYS_CSSELR_EL1), access_csselr, reset_unknown, CSSELR_EL1 }, { SYS_DESC(SYS_CTR_EL0), access_ctr }, - { SYS_DESC(SYS_PMCR_EL0), access_pmcr, reset_pmcr, }, + { SYS_DESC(SYS_PMCR_EL0), access_pmcr, reset_pmcr, PMCR_EL0 }, { SYS_DESC(SYS_PMCNTENSET_EL0), access_pmcnten, reset_unknown, PMCNTENSET_EL0 }, { SYS_DESC(SYS_PMCNTENCLR_EL0), access_pmcnten, NULL, PMCNTENSET_EL0 }, { SYS_DESC(SYS_PMOVSCLR_EL0), access_pmovs, NULL, PMOVSSET_EL0 },
@@ -2254,13 +2254,19 @@ static int emulate_sys_reg(struct kvm_vcpu *vcpu, }static void reset_sys_reg_descs (struct kvm_vcpu *vcpu, -const struct sys_reg_desc *table,size_t num) +const struct sys_reg_desc *table,size_t num, +unsigned long *bmap) {unsigned long i;for (i =0 ; i < num; i++) -if (table[i].reset) +if (table[i].reset) { +int reg = table[i].reg; + table[i].reset (vcpu, &table[i]); +if (reg >0 && reg <NR_SYS_REGS ) +set_bit (reg, bmap); + } }/**
@@ -2774,18 +2780,16 @@ void kvm_reset_sys_regs(struct kvm_vcpu *vcpu) {size_t num;const struct sys_reg_desc *table; - -/* Catch someone adding a register without putting in reset entry. */ -memset (&vcpu->arch.ctxt.sys_regs,0x42 ,sizeof (vcpu->arch.ctxt.sys_regs)); + DECLARE_BITMAP(bmap, NR_SYS_REGS) = { 0, };/* Generic chip reset first (so target could override). */ -reset_sys_reg_descs (vcpu, sys_reg_descs,ARRAY_SIZE (sys_reg_descs)); +reset_sys_reg_descs (vcpu, sys_reg_descs,ARRAY_SIZE (sys_reg_descs), bmap); table =get_target_table (vcpu->arch.target, true, &num); -reset_sys_reg_descs (vcpu, table, num); +reset_sys_reg_descs (vcpu, table, num, bmap);for (num =1 ; num <NR_SYS_REGS ; num++) { -if (WARN (__vcpu_sys_reg (vcpu, num) ==0x4242424242424242 , +if (WARN (!test_bit (num, bmap),"Didn't reset __vcpu_sys_reg(%zi)\n" , num))break ; }
Signature · 1 line
2.20.1
Marc Zyngier
maintainerPATCH 3/4 KVM: arm: Don't write junk to CP15 registers on reset
In reply to Marc ZyngierAt the moment, the way we reset CP15 registers is mildly insane:
We write junk to them, call the reset functions, and then check that we have something else in them.
The "fun" thing is that this can happen while the guest is running (PSCI, for example). If anything in KVM has to evaluate the state of a CP15 register while junk is in there, bad thing may happen.
Let's stop doing that. Instead, we track that we have called a reset function for that register, and assume that the reset function has done something.
In the end, the very need of this reset check is pretty dubious, as it doesn't check everything (a lot of the CP15 reg leave outside of the cp15_regs[] array). It may well be axed in the near future.
--- arch/arm/kvm/coproc.c | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-)
arch/arm/kvm/coproc.c +15 -8
@@ -651,13 +651,22 @@ int kvm_handle_cp14_64(struct kvm_vcpu *vcpu, struct kvm_run *run) }static void reset_coproc_regs (struct kvm_vcpu *vcpu, -const struct coproc_reg *table,size_t num) +const struct coproc_reg *table,size_t num, +unsigned long *bmap) {unsigned long i;for (i =0 ; i < num; i++) -if (table[i].reset) +if (table[i].reset) { +int reg = table[i].reg; + table[i].reset (vcpu, &table[i]); +if (reg >0 && reg <NR_CP15_REGS ) { +set_bit (reg, bmap); +if (table[i].is_64bit) +set_bit (reg +1 , bmap); + } + } }static struct coproc_params decode_32bit_hsr (struct kvm_vcpu *vcpu)
@@ -1432,17 +1441,15 @@ void kvm_reset_coprocs(struct kvm_vcpu *vcpu) {size_t num;const struct coproc_reg *table; - -/* Catch someone adding a register without putting in reset entry. */ -memset (vcpu->arch.ctxt.cp15,0x42 ,sizeof (vcpu->arch.ctxt.cp15)); + DECLARE_BITMAP(bmap, NR_CP15_REGS) = { 0, };/* Generic chip reset first (so target could override). */ -reset_coproc_regs (vcpu, cp15_regs,ARRAY_SIZE (cp15_regs)); +reset_coproc_regs (vcpu, cp15_regs,ARRAY_SIZE (cp15_regs), bmap); table =get_target_table (vcpu->arch.target, &num); -reset_coproc_regs (vcpu, table, num); +reset_coproc_regs (vcpu, table, num, bmap);for (num =1 ; num <NR_CP15_REGS ; num++) -WARN (vcpu_cp15 (vcpu, num) ==0x42424242 , +WARN (!test_bit (num, bmap),"Didn't reset vcpu_cp15(vcpu, %zi)" , num); }
Signature · 1 line
2.20.1
Marc Zyngier
maintainerPATCH 4/4 KVM: arm/arm64: vgic: Reevaluate level sensitive interrupts on enable
In reply to Marc ZyngierFrom: Alexandru Elisei <[email protected]>
A HW mapped level sensitive interrupt asserted by a device will not be put into the ap_list if it is disabled at the VGIC level. When it is enabled again, it will be inserted into the ap_list and written to a list register on guest entry regardless of the state of the device.
We could argue that this can also happen on real hardware, when the command to enable the interrupt reached the GIC before the device had the chance to de-assert the interrupt signal; however, we emulate the distributor and redistributors in software and we can do better than that.
--- virt/kvm/arm/vgic/vgic-mmio.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
virt/kvm/arm/vgic/vgic-mmio.c +16 -0
@@ -113,6 +113,22 @@ void vgic_mmio_write_senable(struct kvm_vcpu *vcpu,struct vgic_irq *irq =vgic_get_irq (vcpu->kvm, vcpu, intid + i);raw_spin_lock_irqsave (&irq->irq_lock, flags); +if (vgic_irq_is_mapped_level (irq)) { +bool was_high = irq->line_level; + +/* +* We need to update the state of the interrupt because +* the guest might have changed the state of the device +* while the interrupt was disabled at the VGIC level. +*/ + irq->line_level =vgic_get_phys_line_level (irq); +/* +* Deactivate the physical interrupt so the GIC will let +* us know when it is asserted again. +*/ +if (!irq->active && was_high && !irq->line_level) +vgic_irq_set_phys_active (irq, false); + } irq->enabled = true;vgic_queue_irq_unlock (vcpu->kvm, irq, flags);
Signature · 1 line
2.20.1
Paolo Bonzini
maintainerQuoted 2 lines · Marc Zyngier
git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm.gittags/kvmarm-fixes-for-5.3
git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm.gittags/kvmarm-fixes-for-5.3-2
Pulled both. Thank you for including the lore link to the 5.3-rc3 pull request.
Paolo