KVM: arm64: ARMv8.3/8.4 Nested Virtualization support

This is the 4th drop of NV support on arm64 for this year.

For the previous episodes, see [1].

What's changed:

- New framework to track system register traps that are reinjected in guest EL2. It is expected to replace the discrete handling we have enjoyed so far, which didn't scale at all. This has already fixed a number of bugs that were hidden (a bunch of traps were never forwarded...). Still a work in progress, but this is going in the right direction.

- Allow the L1 hypervisor to have a S2 that has an input larger than the L0 IPA space. This fixes a number of subtle issues, depending on how the initial guest was created.

- Consequently, the patch series has gone longer again. Boo. But hopefully some of it is easier to review...

[1] https://lore.kernel.org/r/[email protected]

Andre Przywara (1):
KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ

Christoffer Dall (5):
KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
KVM: arm64: nv: Implement nested Stage-2 page table walk logic
KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
KVM: arm64: nv: vgic: Emulate the HW bit in software
KVM: arm64: nv: Sync nested timer state with FEAT_NV2

Jintack Lim (7):
KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
KVM: arm64: nv: Respect virtual HCR_EL2.{NV,TSC) settings
KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
KVM: arm64: nv: Nested GICv3 Support

Marc Zyngier (46):
KVM: arm64: Move VTCR_EL2 into struct s2_mmu
arm64: Add missing Set/Way CMO encodings
arm64: Add missing VA CMO encodings
arm64: Add missing ERXMISCx_EL1 encodings
arm64: Add missing DC ZVA/GVA/GZVA encodings
arm64: Add TLBI operation encodings
arm64: Add AT operation encodings
KVM: arm64: Add missing HCR_EL2 trap bits
KVM: arm64: nv: Add trap forwarding infrastructure
KVM: arm64: nv: Add trap forwarding for HCR_EL2
KVM: arm64: nv: Expose FEAT_EVT to nested guests
KVM: arm64: nv: Add trap forwarding for MDCR_EL2
KVM: arm64: nv: Add trap forwarding for CNTHCTL_EL2
KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
KVM: arm64: nv: Handle virtual EL2 registers in
vcpu_read/write_sys_reg()
KVM: arm64: nv: Handle SPSR_EL2 specially
KVM: arm64: nv: Handle HCR_EL2.E2H specially
KVM: arm64: nv: Save/Restore vEL2 sysregs
KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
KVM: arm64: nv: Handle shadow stage 2 page faults
KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
KVM: arm64: nv: Set a handler for the system instruction traps
KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
KVM: arm64: nv: Hide RAS from nested guests
KVM: arm64: nv: Add handling of EL2-specific timer registers
KVM: arm64: nv: Load timer before the GIC
KVM: arm64: nv: Don't load the GICv4 context on entering a nested
guest
KVM: arm64: nv: Implement maintenance interrupt forwarding
KVM: arm64: nv: Deal with broken VGIC on maintenance interrupt
delivery
KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
KVM: arm64: nv: Add handling of FEAT_TTL TLB invalidation
KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
information
KVM: arm64: nv: Tag shadow S2 entries with nested level
KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
KVM: arm64: nv: Map VNCR-capable registers to a separate page
KVM: arm64: nv: Move nested vgic state into the sysreg file
KVM: arm64: Add FEAT_NV2 cpu feature
KVM: arm64: nv: Fold GICv3 host trapping requirements into guest setup
KVM: arm64: nv: Publish emulated timer interrupt state in the
in-memory state
KVM: arm64: nv: Allocate VNCR page when required
KVM: arm64: nv: Enable ARMv8.4-NV support
KVM: arm64: nv: Fast-track 'InHost' exception returns
KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests
KVM: arm64: nv: Use FEAT_ECV to trap access to EL0 timers
KVM: arm64: nv: Accelerate EL0 timer read accesses when FEAT_ECV is on

 .../virt/kvm/devices/arm-vgic-v3.rst          |  12 +-
 arch/arm64/include/asm/esr.h                  |   1 +
 arch/arm64/include/asm/kvm_arm.h              |  14 +
 arch/arm64/include/asm/kvm_asm.h              |   4 +
 arch/arm64/include/asm/kvm_emulate.h          |  93 +-
 arch/arm64/include/asm/kvm_host.h             | 181 +++-
 arch/arm64/include/asm/kvm_hyp.h              |   2 +
 arch/arm64/include/asm/kvm_mmu.h              |  20 +-
 arch/arm64/include/asm/kvm_nested.h           | 133 +++
 arch/arm64/include/asm/stage2_pgtable.h       |   4 +-
 arch/arm64/include/asm/sysreg.h               | 196 ++++
 arch/arm64/include/asm/vncr_mapping.h         |  74 ++
 arch/arm64/include/uapi/asm/kvm.h             |   1 +
 arch/arm64/kernel/cpufeature.c                |  11 +
 arch/arm64/kvm/Makefile                       |   4 +-
 arch/arm64/kvm/arch_timer.c                   |  98 +-
 arch/arm64/kvm/arm.c                          |  33 +-
 arch/arm64/kvm/at.c                           | 219 ++++
 arch/arm64/kvm/emulate-nested.c               | 934 ++++++++++++++++-
 arch/arm64/kvm/handle_exit.c                  |  29 +-
 arch/arm64/kvm/hyp/include/hyp/switch.h       |   8 +-
 arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h    |   5 +-
 arch/arm64/kvm/hyp/nvhe/mem_protect.c         |   8 +-
 arch/arm64/kvm/hyp/nvhe/pkvm.c                |   4 +-
 arch/arm64/kvm/hyp/nvhe/switch.c              |   2 +-
 arch/arm64/kvm/hyp/nvhe/sysreg-sr.c           |   2 +-
 arch/arm64/kvm/hyp/pgtable.c                  |   2 +-
 arch/arm64/kvm/hyp/vgic-v3-sr.c               |   6 +-
 arch/arm64/kvm/hyp/vhe/switch.c               | 206 +++-
 arch/arm64/kvm/hyp/vhe/sysreg-sr.c            | 124 ++-
 arch/arm64/kvm/hyp/vhe/tlb.c                  |  83 ++
 arch/arm64/kvm/mmu.c                          | 255 ++++-
 arch/arm64/kvm/nested.c                       | 799 ++++++++++++++-
 arch/arm64/kvm/pkvm.c                         |   2 +-
 arch/arm64/kvm/reset.c                        |   7 +
 arch/arm64/kvm/sys_regs.c                     | 958 +++++++++++++++++-
 arch/arm64/kvm/trace_arm.h                    |  19 +
 arch/arm64/kvm/vgic/vgic-init.c               |  33 +
 arch/arm64/kvm/vgic/vgic-kvm-device.c         |  32 +-
 arch/arm64/kvm/vgic/vgic-v3-nested.c          | 248 +++++
 arch/arm64/kvm/vgic/vgic-v3.c                 |  43 +-
 arch/arm64/kvm/vgic/vgic.c                    |  29 +
 arch/arm64/kvm/vgic/vgic.h                    |  10 +
 arch/arm64/tools/cpucaps                      |   1 +
 include/clocksource/arm_arch_timer.h          |   4 +
 include/kvm/arm_arch_timer.h                  |   1 +
 include/kvm/arm_vgic.h                        |  17 +
 include/uapi/linux/kvm.h                      |   1 +
 tools/arch/arm/include/uapi/asm/kvm.h         |   1 +
 49 files changed, 4790 insertions(+), 183 deletions(-)
 create mode 100644 arch/arm64/include/asm/vncr_mapping.h
 create mode 100644 arch/arm64/kvm/at.c
 create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c

2.34.1