KVM: arm64: Preliminary NV patches
Marc Zyngier
maintainerv3 cover KVM: arm64: Preliminary NV patches
Hi all,
In order not to repeat the 90+ patch series that resulted in a deafening silence last time, I've extracted a smaller set of patches that form the required dependencies that allow the rest of the 65 NV patches to be added on top. Yes, it is that bad.
The one real feature here is support for the ARMv8.4-TTL extension at Stage-2 only. The reason to support it is that it helps the hypervisor a lot when it comes to finding out how much to invalidate. It is thus always "supported" with NV.
The rest doesn't contain any functionality change. Most of it reworks existing data structures and adds new accessors for the things that get moved around. The reason for this is that:
- With NV, we end-up with multiple Stage-2 MMU contexts per VM instead of a single one. This requires we divorce struct kvm from the S2 MMU configuration. Of course, we stick with a single MMU context for now.
- With ARMv8.4-NV, a number of system register accesses are turned into memory accesses into the so-called VNCR page. It is thus convenient to make this VNCR page part of the vcpu context and avoid copying data back and forth. For this to work, we need to make sure that all the VNCR-aware sysregs are moved into our per-vcpu sys_regs array instead of leaving in other data structures (the timers, for example). The VNCR page itself isn't introduced with these patches.
- As some of these data structures change, we need a way to isolate the userspace ABI from such change.
There is also a number of cleanups that were in the full fat series that I decided to move early to get them out of the way.
The whole this is a bit of a mix of vaguely unrelated "stuff", but it all comes together if you look at the final series. This applies on top of David Brazdil's series splitting the VHE and nVHE objects.
I plan on taking this early into v5.9, and I really mean it this time!
Catalin: How do you want to proceed for patches 2, 3, and 4? I could make a stable branch that gets you pull into the arm64 tree, or the other way around. Just let me know.
Thanks,
M.
* From v2:
- Rebased on top of David's el2-obj series
- Fixed the terrible __kvm_tlb_flush_local_vmid bug
- Renamed TLBI_TTL_PS_* to TLBI_TTL_TG_* (Alex)
- Fixed a misleading comment in mmu.c (Alex)
- Fixed the debug patch commit log
- Collected Alex's RBs, with thanks.
* From v1:
- A bunch of patches have been merged. These are the current leftovers.
- Rebased on top of v5.8-rc1, and it wasn't fun.
Christoffer Dall (1):
KVM: arm64: Factor out stage 2 page table data from struct kvm
Marc Zyngier (16):
arm64: Detect the ARMv8.4 TTL feature
arm64: Document SW reserved PTE/PMD bits in Stage-2 descriptors
arm64: Add level-hinted TLB invalidation helper
KVM: arm64: Use TTL hint in when invalidating stage-2 translations
KVM: arm64: Introduce accessor for ctxt->sys_reg
KVM: arm64: hyp: Use ctxt_sys_reg/__vcpu_sys_reg instead of raw
sys_regs access
KVM: arm64: sve: Use __vcpu_sys_reg() instead of raw sys_regs access
KVM: arm64: pauth: Use ctxt_sys_reg() instead of raw sys_regs access
KVM: arm64: debug: Drop useless vpcu parameter
KVM: arm64: Make struct kvm_regs userspace-only
KVM: arm64: Move ELR_EL1 to the system register array
KVM: arm64: Move SP_EL1 to the system register array
KVM: arm64: Disintegrate SPSR array
KVM: arm64: Move SPSR_EL1 to the system register array
KVM: arm64: timers: Rename kvm_timer_sync_hwstate to
kvm_timer_sync_user
KVM: arm64: timers: Move timer registers to the sys_regs file
arch/arm64/include/asm/cpucaps.h | 3 +- arch/arm64/include/asm/kvm_asm.h | 8 +- arch/arm64/include/asm/kvm_emulate.h | 37 +-- arch/arm64/include/asm/kvm_host.h | 71 ++++-- arch/arm64/include/asm/kvm_mmu.h | 16 +- arch/arm64/include/asm/pgtable-hwdef.h | 2 + arch/arm64/include/asm/stage2_pgtable.h | 9 + arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/asm/tlbflush.h | 45 ++++ arch/arm64/kernel/asm-offsets.c | 3 +- arch/arm64/kernel/cpufeature.c | 11 + arch/arm64/kvm/arch_timer.c | 157 +++++++++--- arch/arm64/kvm/arm.c | 40 +-- arch/arm64/kvm/fpsimd.c | 6 +- arch/arm64/kvm/guest.c | 79 +++++- arch/arm64/kvm/hyp/entry.S | 3 +- arch/arm64/kvm/hyp/include/hyp/debug-sr.h | 22 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 38 +-- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 152 ++++++----- arch/arm64/kvm/hyp/nvhe/switch.c | 6 +- arch/arm64/kvm/hyp/nvhe/tlb.c | 36 +-- arch/arm64/kvm/hyp/vhe/switch.c | 2 +- arch/arm64/kvm/hyp/vhe/tlb.c | 29 ++- arch/arm64/kvm/inject_fault.c | 2 +- arch/arm64/kvm/mmu.c | 281 ++++++++++++--------- arch/arm64/kvm/regmap.c | 37 ++- arch/arm64/kvm/reset.c | 2 +- arch/arm64/kvm/sys_regs.c | 2 + arch/arm64/kvm/trace_arm.h | 8 +- include/kvm/arm_arch_timer.h | 13 +- 30 files changed, 696 insertions(+), 425 deletions(-)
Signature · 1 line
2.27.0
Marc Zyngier
maintainerPATCH v3 1/17 KVM: arm64: Factor out stage 2 page table data from struct kvm
From: Christoffer Dall <[email protected]>
As we are about to reuse our stage 2 page table manipulation code for shadow stage 2 page tables in the context of nested virtualization, we are going to manage multiple stage 2 page tables for a single VM.
This requires some pretty invasive changes to our data structures, which moves the vmid and pgd pointers into a separate structure and change pretty much all of our mmu code to operate on this structure instead.
The new structure is called struct kvm_s2_mmu.
There is no intended functional change by this patch alone.
--- arch/arm64/include/asm/kvm_asm.h | 7 +- arch/arm64/include/asm/kvm_host.h | 32 ++- arch/arm64/include/asm/kvm_mmu.h | 16 +- arch/arm64/kvm/arm.c | 36 +-- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 +- arch/arm64/kvm/hyp/nvhe/switch.c | 2 +- arch/arm64/kvm/hyp/nvhe/tlb.c | 33 ++- arch/arm64/kvm/hyp/vhe/switch.c | 2 +- arch/arm64/kvm/hyp/vhe/tlb.c | 26 +-- arch/arm64/kvm/mmu.c | 280 +++++++++++++----------- 10 files changed, 240 insertions(+), 198 deletions(-)
arch/arm64/include/asm/kvm_asm.h +4 -3
@@ -77,6 +77,7 @@struct kvm ;struct kvm_vcpu ; +struct kvm_s2_mmu ;DECLARE_KVM_NVHE_SYM (__kvm_hyp_init);DECLARE_KVM_HYP_SYM (__kvm_hyp_vector);
@@ -90,9 +91,9 @@ DECLARE_KVM_HYP_SYM(__bp_harden_hyp_vecs);#endif extern void __kvm_flush_vm_context (void ); -extern void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa); -extern void __kvm_tlb_flush_vmid (struct kvm *kvm); -extern void __kvm_tlb_flush_local_vmid (struct kvm_vcpu *vcpu); +extern void __kvm_tlb_flush_vmid_ipa (struct kvm_s2_mmu *mmu,phys_addr_t ipa); +extern void __kvm_tlb_flush_vmid (struct kvm_s2_mmu *mmu); +extern void __kvm_tlb_flush_local_vmid (struct kvm_s2_mmu *mmu);extern void __kvm_timer_set_cntvoff (u64 cntvoff);
arch/arm64/include/asm/kvm_host.h +25 -7
@@ -66,19 +66,34 @@ struct kvm_vmid {u32 vmid; }; -struct kvm_arch { +struct kvm_s2_mmu {struct kvm_vmid vmid; -/* stage2 entry level table */ -pgd_t *pgd; -phys_addr_t pgd_phys; - -/* VTCR_EL2 value for this VM */ -u64 vtcr; +/* +* stage2 entry level table +* +* Two kvm_s2_mmu structures in the same VM can point to the same +* pgd here. This happens when running a guest using a +* translation regime that isn't affected by its own stage-2 +* translation, such as a non-VHE hypervisor running at vEL2, or +* for vEL1/EL0 with vHCR_EL2.VM == 0. In that case, we use the +* canonical stage-2 page tables. +*/ +pgd_t *pgd; +phys_addr_t pgd_phys;/* The last vcpu id that ran on each physical CPU */ int __percpu *last_vcpu_ran; +struct kvm *kvm; +}; + +struct kvm_arch { +struct kvm_s2_mmu mmu; + +/* VTCR_EL2 value for this VM */ +u64 vtcr; +/* The maximum number of vCPUs depends on the used GIC model */ int max_vcpus;
@@ -254,6 +269,9 @@ struct kvm_vcpu_arch {void *sve_state;unsigned int sve_max_vl; +/* Stage 2 paging state used by the hardware on next switch */ +struct kvm_s2_mmu *hw_mmu; +/* HYP configuration */ u64 hcr_el2;u32 mdcr_el2;
arch/arm64/include/asm/kvm_mmu.h +8 -8
@@ -134,8 +134,8 @@ int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,void free_hyp_pgds (void );void stage2_unmap_vm (struct kvm *kvm); -int kvm_alloc_stage2_pgd (struct kvm *kvm); -void kvm_free_stage2_pgd (struct kvm *kvm); +int kvm_init_stage2_mmu (struct kvm *kvm,struct kvm_s2_mmu *mmu); +void kvm_free_stage2_pgd (struct kvm_s2_mmu *mmu);int kvm_phys_addr_ioremap (struct kvm *kvm,phys_addr_t guest_ipa,phys_addr_t pa,unsigned long size,bool writable);
@@ -577,13 +577,13 @@ static inline u64 kvm_vttbr_baddr_mask(struct kvm *kvm)return vttbr_baddr_mask (kvm_phys_shift (kvm),kvm_stage2_levels (kvm)); } -static __always_inline u64 kvm_get_vttbr(struct kvm *kvm) +static __always_inline u64 kvm_get_vttbr(struct kvm_s2_mmu *mmu) { -struct kvm_vmid *vmid = &kvm->arch.vmid; +struct kvm_vmid *vmid = &mmu->vmid;u64 vmid_field, baddr;u64 cnp =system_supports_cnp () ?VTTBR_CNP_BIT :0 ; - baddr = kvm->arch.pgd_phys; + baddr = mmu->pgd_phys; vmid_field = (u64 )vmid->vmid <<VTTBR_VMID_SHIFT ;return kvm_phys_to_vttbr (baddr) | vmid_field | cnp; }
@@ -592,10 +592,10 @@ static __always_inline u64 kvm_get_vttbr(struct kvm *kvm)* Must be called from hyp code running at EL2 with an updated VTTBR * and interrupts disabled. */ -static __always_inline void __load_guest_stage2(struct kvm *kvm) +static __always_inline void __load_guest_stage2(struct kvm_s2_mmu *mmu) { -write_sysreg (kvm->arch.vtcr, vtcr_el2); -write_sysreg (kvm_get_vttbr (kvm), vttbr_el2); +write_sysreg (kern_hyp_va (mmu->kvm)->arch.vtcr, vtcr_el2); +write_sysreg (kvm_get_vttbr (mmu), vttbr_el2);/* * ARM errata 1165522 and 1530923 require the actual execution of the
arch/arm64/kvm/arm.c +12 -24
@@ -106,22 +106,15 @@ static int kvm_arm_default_max_vcpus(void)*/ int kvm_arch_init_vm (struct kvm *kvm,unsigned long type) { -int ret, cpu; +int ret; ret =kvm_arm_setup_stage2 (kvm, type);if (ret)return ret; - kvm->arch.last_vcpu_ran =alloc_percpu (typeof (*kvm->arch.last_vcpu_ran)); -if (!kvm->arch.last_vcpu_ran) -return -ENOMEM ; - -for_each_possible_cpu (cpu) - *per_cpu_ptr (kvm->arch.last_vcpu_ran, cpu) =-1 ; - - ret =kvm_alloc_stage2_pgd (kvm); + ret =kvm_init_stage2_mmu (kvm, &kvm->arch.mmu);if (ret) - goto out_fail_alloc; +return ret; ret =create_hyp_mappings (kvm, kvm +1 ,PAGE_HYP );if (ret)
@@ -129,18 +122,12 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)kvm_vgic_early_init (kvm); -/* Mark the initial VMID generation invalid */ - kvm->arch.vmid.vmid_gen =0 ; -/* The maximum number of VCPUs is limited by the host's GIC model */ kvm->arch.max_vcpus =kvm_arm_default_max_vcpus ();return ret; out_free_stage2_pgd: -kvm_free_stage2_pgd (kvm); -out_fail_alloc: -free_percpu (kvm->arch.last_vcpu_ran); - kvm->arch.last_vcpu_ran =NULL ; +kvm_free_stage2_pgd (&kvm->arch.mmu);return ret; }
@@ -160,9 +147,6 @@ void kvm_arch_destroy_vm(struct kvm *kvm)kvm_vgic_destroy (kvm); -free_percpu (kvm->arch.last_vcpu_ran); - kvm->arch.last_vcpu_ran =NULL ; -for (i =0 ; i <KVM_MAX_VCPUS ; ++i) {if (kvm->vcpus[i]) {kvm_vcpu_destroy (kvm->vcpus[i]);
@@ -279,6 +263,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)kvm_arm_pvtime_vcpu_init (&vcpu->arch); + vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu; + err =kvm_vgic_vcpu_init (vcpu);if (err)return err;
@@ -334,16 +320,18 @@ void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu)void kvm_arch_vcpu_load (struct kvm_vcpu *vcpu,int cpu) { +struct kvm_s2_mmu *mmu;int *last_ran; - last_ran =this_cpu_ptr (vcpu->kvm->arch.last_vcpu_ran); + mmu = vcpu->arch.hw_mmu; + last_ran =this_cpu_ptr (mmu->last_vcpu_ran);/* * We might get preempted before the vCPU actually runs, but * over-invalidation doesn't affect correctness. */ if (*last_ran != vcpu->vcpu_id) { -kvm_call_hyp (__kvm_tlb_flush_local_vmid, vcpu); +kvm_call_hyp (__kvm_tlb_flush_local_vmid, mmu); *last_ran = vcpu->vcpu_id; }
@@ -680,7 +668,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)*/ cond_resched (); -update_vmid (&vcpu->kvm->arch.vmid); +update_vmid (&vcpu->arch.hw_mmu->vmid);check_vcpu_requests (vcpu);
@@ -729,7 +717,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)*/ smp_store_mb (vcpu->mode,IN_GUEST_MODE ); -if (ret <=0 ||need_new_vmid_gen (&vcpu->kvm->arch.vmid) || +if (ret <=0 ||need_new_vmid_gen (&vcpu->arch.hw_mmu->vmid) ||kvm_request_pending (vcpu)) { vcpu->mode =OUTSIDE_GUEST_MODE ;isb ();/* Ensure work in x_flush_hwstate is committed */
arch/arm64/kvm/hyp/include/hyp/switch.h +2 -2
@@ -122,9 +122,9 @@ static inline void ___deactivate_traps(struct kvm_vcpu *vcpu) } } -static inline void __activate_vm (struct kvm *kvm) +static inline void __activate_vm (struct kvm_s2_mmu *mmu) { -__load_guest_stage2 (kvm); +__load_guest_stage2 (mmu); }static inline bool __translate_far_to_hpfar (u64 far,u64 *hpfar)
arch/arm64/kvm/hyp/nvhe/switch.c +1 -1
@@ -194,7 +194,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)__sysreg32_restore_state (vcpu);__sysreg_restore_state_nvhe (guest_ctxt); -__activate_vm (kern_hyp_va (vcpu->kvm)); +__activate_vm (kern_hyp_va (vcpu->arch.hw_mmu));__activate_traps (vcpu);__hyp_vgic_restore_state (vcpu);
arch/arm64/kvm/hyp/nvhe/tlb.c +16 -17
@@ -12,7 +12,8 @@ struct tlb_inv_context {u64 tcr; }; -static void __tlb_switch_to_guest (struct kvm *kvm,struct tlb_inv_context *cxt) +static void __tlb_switch_to_guest (struct kvm_s2_mmu *mmu, +struct tlb_inv_context *cxt) {if (cpus_have_final_cap (ARM64_WORKAROUND_SPECULATIVE_AT )) {u64 val;
@@ -30,12 +31,10 @@ static void __tlb_switch_to_guest(struct kvm *kvm, struct tlb_inv_context *cxt)isb (); } -/* __load_guest_stage2() includes an ISB for the workaround. */ -__load_guest_stage2 (kvm); - asm(ALTERNATIVE("isb", "nop", ARM64_WORKAROUND_SPECULATIVE_AT)); +__load_guest_stage2 (mmu); } -static void __tlb_switch_to_host (struct kvm *kvm,struct tlb_inv_context *cxt) +static void __tlb_switch_to_host (struct tlb_inv_context *cxt) {write_sysreg (0 , vttbr_el2);
@@ -47,15 +46,15 @@ static void __tlb_switch_to_host(struct kvm *kvm, struct tlb_inv_context *cxt) } } -void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa) +void __kvm_tlb_flush_vmid_ipa (struct kvm_s2_mmu *mmu,phys_addr_t ipa) {struct tlb_inv_context cxt;dsb (ishst);/* Switch to requested VMID */ - kvm =kern_hyp_va (kvm); -__tlb_switch_to_guest (kvm, &cxt); + mmu =kern_hyp_va (mmu); +__tlb_switch_to_guest (mmu, &cxt);/* * We could do so much better if we had the VA as well.
@@ -98,39 +97,39 @@ void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)if (icache_is_vpipt ())__flush_icache_all (); -__tlb_switch_to_host (kvm, &cxt); +__tlb_switch_to_host (&cxt); } -void __kvm_tlb_flush_vmid (struct kvm *kvm) +void __kvm_tlb_flush_vmid (struct kvm_s2_mmu *mmu) {struct tlb_inv_context cxt;dsb (ishst);/* Switch to requested VMID */ - kvm =kern_hyp_va (kvm); -__tlb_switch_to_guest (kvm, &cxt); + mmu =kern_hyp_va (mmu); +__tlb_switch_to_guest (mmu, &cxt);__tlbi (vmalls12e1is);dsb (ish);isb (); -__tlb_switch_to_host (kvm, &cxt); +__tlb_switch_to_host (&cxt); } -void __kvm_tlb_flush_local_vmid (struct kvm_vcpu *vcpu) +void __kvm_tlb_flush_local_vmid (struct kvm_s2_mmu *mmu) { -struct kvm *kvm =kern_hyp_va (kern_hyp_va (vcpu)->kvm);struct tlb_inv_context cxt;/* Switch to requested VMID */ -__tlb_switch_to_guest (kvm, &cxt); + mmu =kern_hyp_va (mmu); +__tlb_switch_to_guest (mmu, &cxt);__tlbi (vmalle1);dsb (nsh);isb (); -__tlb_switch_to_host (kvm, &cxt); +__tlb_switch_to_host (&cxt); }void __kvm_flush_vm_context (void )
arch/arm64/kvm/hyp/vhe/switch.c +1 -1
@@ -125,7 +125,7 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu)* stage 2 translation, and __activate_traps clear HCR_EL2.TGE * (among other things). */ -__activate_vm (vcpu->kvm); +__activate_vm (vcpu->arch.hw_mmu);__activate_traps (vcpu);sysreg_restore_guest_state_vhe (guest_ctxt);
arch/arm64/kvm/hyp/vhe/tlb.c +13 -13
@@ -16,7 +16,8 @@ struct tlb_inv_context {u64 sctlr; }; -static void __tlb_switch_to_guest (struct kvm *kvm,struct tlb_inv_context *cxt) +static void __tlb_switch_to_guest (struct kvm_s2_mmu *mmu, +struct tlb_inv_context *cxt) {u64 val;
@@ -52,14 +53,14 @@ static void __tlb_switch_to_guest(struct kvm *kvm, struct tlb_inv_context *cxt)* place before clearing TGE. __load_guest_stage2() already * has an ISB in order to deal with this. */ -__load_guest_stage2 (kvm); +__load_guest_stage2 (mmu); val =read_sysreg (hcr_el2); val &= ~HCR_TGE ;write_sysreg (val, hcr_el2);isb (); } -static void __tlb_switch_to_host (struct kvm *kvm,struct tlb_inv_context *cxt) +static void __tlb_switch_to_host (struct tlb_inv_context *cxt) {/* * We're done with the TLB operation, let's restore the host's
@@ -78,14 +79,14 @@ static void __tlb_switch_to_host(struct kvm *kvm, struct tlb_inv_context *cxt)local_irq_restore (cxt->flags); } -void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa) +void __kvm_tlb_flush_vmid_ipa (struct kvm_s2_mmu *mmu,phys_addr_t ipa) {struct tlb_inv_context cxt;dsb (ishst);/* Switch to requested VMID */ -__tlb_switch_to_guest (kvm, &cxt); +__tlb_switch_to_guest (mmu, &cxt);/* * We could do so much better if we had the VA as well.
@@ -106,38 +107,37 @@ void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)dsb (ish);isb (); -__tlb_switch_to_host (kvm, &cxt); +__tlb_switch_to_host (&cxt); } -void __kvm_tlb_flush_vmid (struct kvm *kvm) +void __kvm_tlb_flush_vmid (struct kvm_s2_mmu *mmu) {struct tlb_inv_context cxt;dsb (ishst);/* Switch to requested VMID */ -__tlb_switch_to_guest (kvm, &cxt); +__tlb_switch_to_guest (mmu, &cxt);__tlbi (vmalls12e1is);dsb (ish);isb (); -__tlb_switch_to_host (kvm, &cxt); +__tlb_switch_to_host (&cxt); } -void __kvm_tlb_flush_local_vmid (struct kvm_vcpu *vcpu) +void __kvm_tlb_flush_local_vmid (struct kvm_s2_mmu *mmu) { -struct kvm *kvm = vcpu->kvm;struct tlb_inv_context cxt;/* Switch to requested VMID */ -__tlb_switch_to_guest (kvm, &cxt); +__tlb_switch_to_guest (mmu, &cxt);__tlbi (vmalle1);dsb (nsh);isb (); -__tlb_switch_to_host (kvm, &cxt); +__tlb_switch_to_host (&cxt); }void __kvm_flush_vm_context (void )
arch/arm64/kvm/mmu.c +158 -122
854 lines not on this page · raw message
Signature · 1 line
2.27.0
Marc Zyngier
maintainerPATCH v3 2/17 arm64: Detect the ARMv8.4 TTL feature
In reply to Marc ZyngierIn order to reduce the cost of TLB invalidation, the ARMv8.4 TTL feature allows TLBs to be issued with a level allowing for quicker invalidation.
Let's detect the feature for now. Further patches will implement its actual usage.
--- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/kernel/cpufeature.c | 11 +++++++++++ 3 files changed, 14 insertions(+), 1 deletion(-)
arch/arm64/include/asm/cpucaps.h +2 -1
@@ -62,7 +62,8 @@#define ARM64_HAS_GENERIC_AUTH 52#define ARM64_HAS_32BIT_EL1 53#define ARM64_BTI 54 +#define ARM64_HAS_ARMv8_4_TTL 55 -#define ARM64_NCAPS 55 +#define ARM64_NCAPS 56#endif /* __ASM_CPUCAPS_H */
arch/arm64/include/asm/sysreg.h +1 -0
@@ -746,6 +746,7 @@/* id_aa64mmfr2 */ #define ID_AA64MMFR2_E0PD_SHIFT 60 +#define ID_AA64MMFR2_TTL_SHIFT 48#define ID_AA64MMFR2_FWB_SHIFT 40#define ID_AA64MMFR2_AT_SHIFT 32#define ID_AA64MMFR2_LVA_SHIFT 16
arch/arm64/kernel/cpufeature.c +11 -0
@@ -323,6 +323,7 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = {static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {ARM64_FTR_BITS (FTR_HIDDEN ,FTR_NONSTRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR2_E0PD_SHIFT ,4 ,0 ), +ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR2_TTL_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR2_FWB_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_VISIBLE ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR2_AT_SHIFT ,4 ,0 ),ARM64_FTR_BITS (FTR_HIDDEN ,FTR_STRICT ,FTR_LOWER_SAFE ,ID_AA64MMFR2_LVA_SHIFT ,4 ,0 ),
@@ -1882,6 +1883,16 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_cpuid_feature, .cpu_enable = cpu_has_fwb, }, + { + .desc = "ARMv8.4 Translation Table Level", + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .capability = ARM64_HAS_ARMv8_4_TTL, + .sys_reg = SYS_ID_AA64MMFR2_EL1, + .sign = FTR_UNSIGNED, + .field_pos = ID_AA64MMFR2_TTL_SHIFT, + .min_field_value = 1, + .matches = has_cpuid_feature, + }, #ifdef CONFIG_ARM64_HW_AFDBM { /*
Signature · 1 line
2.27.0
Re: [PATCH v3 02/17] arm64: Detect the ARMv8.4 TTL feature
Quoted 9 lines · Marc Zyngier
In order to reduce the cost of TLB invalidation, the ARMv8.4 TTL feature allows TLBs to be issued with a level allowing for quicker invalidation.
Let's detect the feature for now. Further patches will implement its actual usage.
Reviewed-by Suzuki K Polose <[email protected]>Signed-off-by Marc Zyngier <[email protected]>
It looks like I had a reviewed-by in the other series [1]. Feel free to add it here as well:
[1] https://lore.kernel.org/linux-arm-kernel/[email protected]/
Marc Zyngier
maintainerPATCH v3 3/17 arm64: Document SW reserved PTE/PMD bits in Stage-2 descriptors
In reply to Marc ZyngierAdvertise bits [58:55] as reserved for SW in the S2 descriptors.
--- arch/arm64/include/asm/pgtable-hwdef.h | 2 ++ 1 file changed, 2 insertions(+)
arch/arm64/include/asm/pgtable-hwdef.h +2 -0
@@ -178,10 +178,12 @@#define PTE_S2_RDONLY (_AT(pteval_t, 1) << 6)/* HAP[2:1] */ #define PTE_S2_RDWR (_AT(pteval_t, 3) << 6)/* HAP[2:1] */ #define PTE_S2_XN (_AT(pteval_t, 2) << 53)/* XN[1:0] */ +#define PTE_S2_SW_RESVD (_AT(pteval_t, 15) << 55)/* Reserved for SW */ #define PMD_S2_RDONLY (_AT(pmdval_t, 1) << 6)/* HAP[2:1] */ #define PMD_S2_RDWR (_AT(pmdval_t, 3) << 6)/* HAP[2:1] */ #define PMD_S2_XN (_AT(pmdval_t, 2) << 53)/* XN[1:0] */ +#define PMD_S2_SW_RESVD (_AT(pmdval_t, 15) << 55)/* Reserved for SW */ #define PUD_S2_RDONLY (_AT(pudval_t, 1) << 6)/* HAP[2:1] */ #define PUD_S2_RDWR (_AT(pudval_t, 3) << 6)/* HAP[2:1] */
Signature · 1 line
2.27.0
Re: [PATCH v3 03/17] arm64: Document SW reserved PTE/PMD bits in Stage-2 descriptors
Quoted 4 lines · Marc Zyngier
Advertise bits [58:55] as reserved for SW in the S2 descriptors.
Reviewed-by Andrew Scull <[email protected]>Signed-off-by Marc Zyngier <[email protected]>
Marc Zyngier
maintainerPATCH v3 4/17 arm64: Add level-hinted TLB invalidation helper
In reply to Marc ZyngierAdd a level-hinted TLB invalidation helper that only gets used if ARMv8.4-TTL gets detected.
--- arch/arm64/include/asm/stage2_pgtable.h | 9 +++++ arch/arm64/include/asm/tlbflush.h | 45 +++++++++++++++++++++++++ 2 files changed, 54 insertions(+)
arch/arm64/include/asm/stage2_pgtable.h +9 -0
@@ -256,4 +256,13 @@ stage2_pgd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end) return (boundary - 1 < end - 1) ? boundary : end; } +/* +* Level values for the ARMv8.4-TTL extension, mapping PUD/PMD/PTE and +* the architectural page-table level. +*/ +#define S2_NO_LEVEL_HINT 0 +#define S2_PUD_LEVEL 1 +#define S2_PMD_LEVEL 2 +#define S2_PTE_LEVEL 3 +#endif /* __ARM64_S2_PGTABLE_H_ */
arch/arm64/include/asm/tlbflush.h +45 -0
57 lines not on this page · raw message
Signature · 1 line
2.27.0
Re: [PATCH v3 04/17] arm64: Add level-hinted TLB invalidation helper
Quoted 5 lines · Marc Zyngier
Add a level-hinted TLB invalidation helper that only gets used if ARMv8.4-TTL gets detected.
Reviewed-by Alexandru Elisei <[email protected]>Signed-off-by Marc Zyngier <[email protected]>
Same here:
Marc Zyngier
maintainerPATCH v3 5/17 KVM: arm64: Use TTL hint in when invalidating stage-2 translations
In reply to Marc ZyngierSince we often have a precise idea of the level we're dealing with when invalidating TLBs, we can provide it to as a hint to our invalidation helper.
--- arch/arm64/include/asm/kvm_asm.h | 3 ++- arch/arm64/kvm/hyp/nvhe/tlb.c | 5 +++-- arch/arm64/kvm/hyp/vhe/tlb.c | 5 +++-- arch/arm64/kvm/mmu.c | 29 +++++++++++++++-------------- 4 files changed, 23 insertions(+), 19 deletions(-)
arch/arm64/include/asm/kvm_asm.h +2 -1
@@ -91,7 +91,8 @@ DECLARE_KVM_HYP_SYM(__bp_harden_hyp_vecs);#endif extern void __kvm_flush_vm_context (void ); -extern void __kvm_tlb_flush_vmid_ipa (struct kvm_s2_mmu *mmu,phys_addr_t ipa); +extern void __kvm_tlb_flush_vmid_ipa (struct kvm_s2_mmu *mmu,phys_addr_t ipa, +int level);extern void __kvm_tlb_flush_vmid (struct kvm_s2_mmu *mmu);extern void __kvm_tlb_flush_local_vmid (struct kvm_s2_mmu *mmu);
arch/arm64/kvm/hyp/nvhe/tlb.c +3 -2
@@ -46,7 +46,8 @@ static void __tlb_switch_to_host(struct tlb_inv_context *cxt) } } -void __kvm_tlb_flush_vmid_ipa (struct kvm_s2_mmu *mmu,phys_addr_t ipa) +void __kvm_tlb_flush_vmid_ipa (struct kvm_s2_mmu *mmu, +phys_addr_t ipa,int level) {struct tlb_inv_context cxt;
@@ -62,7 +63,7 @@ void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, phys_addr_t ipa)* whole of Stage-1. Weep... */ ipa >>=12 ; -__tlbi (ipas2e1is, ipa); +__tlbi_level (ipas2e1is, ipa, level);/* * We have to ensure completion of the invalidation at Stage-2,
arch/arm64/kvm/hyp/vhe/tlb.c +3 -2
@@ -79,7 +79,8 @@ static void __tlb_switch_to_host(struct tlb_inv_context *cxt)local_irq_restore (cxt->flags); } -void __kvm_tlb_flush_vmid_ipa (struct kvm_s2_mmu *mmu,phys_addr_t ipa) +void __kvm_tlb_flush_vmid_ipa (struct kvm_s2_mmu *mmu, +phys_addr_t ipa,int level) {struct tlb_inv_context cxt;
@@ -94,7 +95,7 @@ void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, phys_addr_t ipa)* whole of Stage-1. Weep... */ ipa >>=12 ; -__tlbi (ipas2e1is, ipa); +__tlbi_level (ipas2e1is, ipa, level);/* * We have to ensure completion of the invalidation at Stage-2,
arch/arm64/kvm/mmu.c +15 -14
@@ -58,9 +58,10 @@ void kvm_flush_remote_tlbs(struct kvm *kvm)kvm_call_hyp (__kvm_tlb_flush_vmid, &kvm->arch.mmu); } -static void kvm_tlb_flush_vmid_ipa (struct kvm_s2_mmu *mmu,phys_addr_t ipa) +static void kvm_tlb_flush_vmid_ipa (struct kvm_s2_mmu *mmu,phys_addr_t ipa, +int level) { -kvm_call_hyp (__kvm_tlb_flush_vmid_ipa, mmu, ipa); +kvm_call_hyp (__kvm_tlb_flush_vmid_ipa, mmu, ipa, level); }/*
@@ -102,7 +103,7 @@ static void stage2_dissolve_pmd(struct kvm_s2_mmu *mmu, phys_addr_t addr, pmd_treturn ;pmd_clear (pmd); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_PMD_LEVEL );put_page (virt_to_page (pmd)); }
@@ -122,7 +123,7 @@ static void stage2_dissolve_pud(struct kvm_s2_mmu *mmu, phys_addr_t addr, pud_treturn ;stage2_pud_clear (kvm, pudp); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_PUD_LEVEL );put_page (virt_to_page (pudp)); }
@@ -163,7 +164,7 @@ static void clear_stage2_pgd_entry(struct kvm_s2_mmu *mmu, pgd_t *pgd, phys_addrstruct kvm *kvm = mmu->kvm;p4d_t *p4d_table__maybe_unused =stage2_p4d_offset (kvm, pgd,0UL );stage2_pgd_clear (kvm, pgd); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_NO_LEVEL_HINT );stage2_p4d_free (kvm, p4d_table);put_page (virt_to_page (pgd)); }
@@ -173,7 +174,7 @@ static void clear_stage2_p4d_entry(struct kvm_s2_mmu *mmu, p4d_t *p4d, phys_addrstruct kvm *kvm = mmu->kvm;pud_t *pud_table__maybe_unused =stage2_pud_offset (kvm, p4d,0 );stage2_p4d_clear (kvm, p4d); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_NO_LEVEL_HINT );stage2_pud_free (kvm, pud_table);put_page (virt_to_page (p4d)); }
@@ -185,7 +186,7 @@ static void clear_stage2_pud_entry(struct kvm_s2_mmu *mmu, pud_t *pud, phys_addrVM_BUG_ON (stage2_pud_huge (kvm, *pud));stage2_pud_clear (kvm, pud); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_NO_LEVEL_HINT );stage2_pmd_free (kvm, pmd_table);put_page (virt_to_page (pud)); }
@@ -195,7 +196,7 @@ static void clear_stage2_pmd_entry(struct kvm_s2_mmu *mmu, pmd_t *pmd, phys_addrpte_t *pte_table =pte_offset_kernel (pmd,0 );VM_BUG_ON (pmd_thp_or_huge (*pmd));pmd_clear (pmd); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_NO_LEVEL_HINT );free_page ((unsigned long )pte_table);put_page (virt_to_page (pmd)); }
@@ -273,7 +274,7 @@ static void unmap_stage2_ptes(struct kvm_s2_mmu *mmu, pmd_t *pmd,pte_t old_pte = *pte;kvm_set_pte (pte,__pte (0 )); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_PTE_LEVEL );/* No need to invalidate the cache for device mappings */ if (!kvm_is_device_pfn (pte_pfn (old_pte)))
@@ -302,7 +303,7 @@ static void unmap_stage2_pmds(struct kvm_s2_mmu *mmu, pud_t *pud,pmd_t old_pmd = *pmd;pmd_clear (pmd); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_PMD_LEVEL );kvm_flush_dcache_pmd (old_pmd);
@@ -332,7 +333,7 @@ static void unmap_stage2_puds(struct kvm_s2_mmu *mmu, p4d_t *p4d,pud_t old_pud = *pud;stage2_pud_clear (kvm, pud); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_PUD_LEVEL );kvm_flush_dcache_pud (old_pud);put_page (virt_to_page (pud)); }else {
@@ -1260,7 +1261,7 @@ static int stage2_set_pmd_huge(struct kvm_s2_mmu *mmu,*/ WARN_ON_ONCE (pmd_pfn (old_pmd) !=pmd_pfn (*new_pmd));pmd_clear (pmd); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_PMD_LEVEL ); }else {get_page (virt_to_page (pmd)); }
@@ -1302,7 +1303,7 @@ static int stage2_set_pud_huge(struct kvm_s2_mmu *mmu,WARN_ON_ONCE (kvm_pud_pfn (old_pud) !=kvm_pud_pfn (*new_pudp));stage2_pud_clear (kvm, pudp); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_PUD_LEVEL ); }else {get_page (virt_to_page (pudp)); }
@@ -1451,7 +1452,7 @@ static int stage2_set_pte(struct kvm_s2_mmu *mmu,return 0 ;kvm_set_pte (pte,__pte (0 )); -kvm_tlb_flush_vmid_ipa (mmu, addr); +kvm_tlb_flush_vmid_ipa (mmu, addr,S2_PTE_LEVEL ); }else {get_page (virt_to_page (pte)); }
Signature · 1 line
2.27.0
Marc Zyngier
maintainerPATCH v3 6/17 KVM: arm64: Introduce accessor for ctxt->sys_reg
In reply to Marc ZyngierIn order to allow the disintegration of the per-vcpu sysreg array, let's introduce a new helper (ctxt_sys_reg()) that returns the in-memory copy of a system register, picked from a given context.
__vcpu_sys_reg() is rewritten to use this helper.
--- arch/arm64/include/asm/kvm_host.h | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
arch/arm64/include/asm/kvm_host.h +10 -5
@@ -405,12 +405,17 @@ struct kvm_vcpu_arch {#define vcpu_gp_regs (v) (&(v)->arch.ctxt.gp_regs)/* -* Only use __vcpu_sys_reg if you know you want the memory backed version of a -* register, and not the one most recently accessed by a running VCPU. For -* example, for userspace access or for system registers that are never context -* switched, but only emulated. +* Only use __vcpu_sys_reg/ctxt_sys_reg if you know you want the +* memory backed version of a register, and not the one most recently +* accessed by a running VCPU. For example, for userspace access or +* for system registers that are never context switched, but only +* emulated. */ -#define __vcpu_sys_reg (v,r) ((v)->arch.ctxt.sys_regs[(r)]) +#define __ctxt_sys_reg (c,r) (&(c)->sys_regs[(r)]) + +#define ctxt_sys_reg (c,r) (*__ctxt_sys_reg(c,r)) + +#define __vcpu_sys_reg (v,r) (ctxt_sys_reg(&(v)->arch.ctxt, (r)))u64 vcpu_read_sys_reg (const struct kvm_vcpu *vcpu,int reg);void vcpu_write_sys_reg (struct kvm_vcpu *vcpu,u64 val,int reg);
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerPATCH v3 7/17 KVM: arm64: hyp: Use ctxt_sys_reg/__vcpu_sys_reg instead of raw sys_regs access
In reply to Marc ZyngierSwitch the hypervisor code to using ctxt_sys_reg/__vcpu_sys_reg instead of raw sys_regs accesses. No intended functionnal change.
--- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/kvm/hyp/include/hyp/debug-sr.h | 4 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 7 +- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 110 ++++++++++----------- arch/arm64/kvm/hyp/nvhe/switch.c | 4 +- 5 files changed, 62 insertions(+), 65 deletions(-)
arch/arm64/include/asm/kvm_host.h +1 -1
@@ -561,7 +561,7 @@ DECLARE_PER_CPU(kvm_host_data_t, kvm_host_data);static inline void kvm_init_host_cpu_context (struct kvm_cpu_context *cpu_ctxt) {/* The host's MPIDR is immutable, so let's set it up at boot time */ - cpu_ctxt->sys_regs[MPIDR_EL1 ] =read_cpuid_mpidr (); +ctxt_sys_reg (cpu_ctxt,MPIDR_EL1 ) =read_cpuid_mpidr (); }static inline bool kvm_arch_requires_vhe (void )
arch/arm64/kvm/hyp/include/hyp/debug-sr.h +2 -2
@@ -104,7 +104,7 @@ static inline void __debug_save_state(struct kvm_vcpu *vcpu,save_debug (dbg->dbg_wcr, dbgwcr, wrps);save_debug (dbg->dbg_wvr, dbgwvr, wrps); - ctxt->sys_regs[MDCCINT_EL1 ] =read_sysreg (mdccint_el1); +ctxt_sys_reg (ctxt,MDCCINT_EL1 ) =read_sysreg (mdccint_el1); } static inline void __debug_restore_state(struct kvm_vcpu *vcpu,
@@ -124,7 +124,7 @@ static inline void __debug_restore_state(struct kvm_vcpu *vcpu,restore_debug (dbg->dbg_wcr, dbgwcr, wrps);restore_debug (dbg->dbg_wvr, dbgwvr, wrps); -write_sysreg (ctxt->sys_regs[MDCCINT_EL1 ], mdccint_el1); +write_sysreg (ctxt_sys_reg (ctxt,MDCCINT_EL1 ), mdccint_el1); }static inline void __debug_switch_to_guest_common (struct kvm_vcpu *vcpu)
arch/arm64/kvm/hyp/include/hyp/switch.h +3 -4
@@ -53,7 +53,7 @@ static inline void __fpsimd_save_fpexc32(struct kvm_vcpu *vcpu)if (!vcpu_el1_is_32bit (vcpu))return ; - vcpu->arch.ctxt.sys_regs[FPEXC32_EL2 ] =read_sysreg (fpexc32_el2); +__vcpu_sys_reg (vcpu,FPEXC32_EL2 ) =read_sysreg (fpexc32_el2); }static inline void __activate_traps_fpsimd32 (struct kvm_vcpu *vcpu)
@@ -268,15 +268,14 @@ static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)sve_load_state (vcpu_sve_pffr (vcpu), &vcpu->arch.ctxt.gp_regs.fp_regs.fpsr,sve_vq_from_vl (vcpu->arch.sve_max_vl) -1 ); -write_sysreg_s (vcpu->arch.ctxt.sys_regs[ZCR_EL1 ],SYS_ZCR_EL12 ); +write_sysreg_s (__vcpu_sys_reg (vcpu,ZCR_EL1 ),SYS_ZCR_EL12 ); }else {__fpsimd_restore_state (&vcpu->arch.ctxt.gp_regs.fp_regs); }/* Skip restoring fpexc32 for AArch64 guests */ if (!(read_sysreg (hcr_el2) &HCR_RW )) -write_sysreg (vcpu->arch.ctxt.sys_regs[FPEXC32_EL2 ], - fpexc32_el2); +write_sysreg (__vcpu_sys_reg (vcpu,FPEXC32_EL2 ), fpexc32_el2); vcpu->arch.flags |=KVM_ARM64_FP_ENABLED ;
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +54 -56
203 lines not on this page · raw message
arch/arm64/kvm/hyp/nvhe/switch.c +2 -2
@@ -52,9 +52,9 @@ static void __activate_traps(struct kvm_vcpu *vcpu)* configured and enabled. We can now restore the guest's S1 * configuration: SCTLR, and only then TCR. */ -write_sysreg_el1 (ctxt->sys_regs[SCTLR_EL1 ],SYS_SCTLR ); +write_sysreg_el1 (ctxt_sys_reg (ctxt,SCTLR_EL1 ),SYS_SCTLR );isb (); -write_sysreg_el1 (ctxt->sys_regs[TCR_EL1 ],SYS_TCR ); +write_sysreg_el1 (ctxt_sys_reg (ctxt,TCR_EL1 ),SYS_TCR ); } }
Signature · 1 line
2.27.0
Re: [PATCH v3 07/17] KVM: arm64: hyp: Use ctxt_sys_reg/__vcpu_sys_reg instead of raw sys_regs access
Hi,
Quoted 11 lines · Marc Zyngier
Switch the hypervisor code to using ctxt_sys_reg/__vcpu_sys_reg instead of raw
sys_regsaccesses. No intended functionnal change.Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/kvm/hyp/include/hyp/debug-sr.h | 4 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 7 +- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 110 ++++++++++----------- arch/arm64/kvm/hyp/nvhe/switch.c | 4 +- 5 files changed, 62 insertions(+), 65 deletions(-)
Looks fine to me, checked that the correct function is used (either __vcpu_sys_reg, or ctxt_sys_reg), and that the registers names match. Also did a compile test to make extra sure I didn't miss anything:
Thanks,
Alex
Marc Zyngier
maintainerPATCH v3 8/17 KVM: arm64: sve: Use __vcpu_sys_reg() instead of raw sys_regs access
In reply to Marc ZyngierNow that we have a wrapper for the sysreg accesses, let's use that consistently.
--- arch/arm64/kvm/fpsimd.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
arch/arm64/kvm/fpsimd.c +1 -3
@@ -109,12 +109,10 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)local_irq_save (flags);if (vcpu->arch.flags &KVM_ARM64_FP_ENABLED ) { -u64 *guest_zcr = &vcpu->arch.ctxt.sys_regs[ZCR_EL1 ]; -fpsimd_save_and_flush_cpu_state ();if (guest_has_sve) - *guest_zcr =read_sysreg_s (SYS_ZCR_EL12 ); +__vcpu_sys_reg (vcpu,ZCR_EL1 ) =read_sysreg_s (SYS_ZCR_EL12 ); }else if (host_has_sve) {/* * The FPSIMD/SVE state in the CPU has not been touched, and we
Signature · 1 line
2.27.0
Re: [PATCH v3 08/17] KVM: arm64: sve: Use __vcpu_sys_reg() instead of raw sys_regs access
Hi,
Quoted 16 lines · Marc Zyngier
Now that we have a wrapper for the sysreg accesses, let's use that consistently.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/fpsimd.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)arch/arm64/kvm/fpsimd.c +0
-0@@ -109,12 +109,10 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)local_irq_save (flags);if (vcpu->arch.flags &KVM_ARM64_FP_ENABLED ) {
arch/arm64/kvm/fpsimd.c void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu) · line 118
Quoted 10 lines
arch/arm64/kvm/fpsimd.c +1
-3-u64 *guest_zcr = &vcpu->arch.ctxt.sys_regs[ZCR_EL1 ]; -fpsimd_save_and_flush_cpu_state ();if (guest_has_sve) - *guest_zcr =read_sysreg_s (SYS_ZCR_EL12 ); +__vcpu_sys_reg (vcpu,ZCR_EL1 ) =read_sysreg_s (SYS_ZCR_EL12 ); }else if (host_has_sve) {/* * The FPSIMD/SVE state in the CPU has not been touched, and we
Looks fine to me, it's a good change:
Thanks,
Alex
Marc Zyngier
maintainerPATCH v3 9/17 KVM: arm64: pauth: Use ctxt_sys_reg() instead of raw sys_regs access
In reply to Marc ZyngierNow that we have a wrapper for the sysreg accesses, let's use that consistently.
--- arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-)
arch/arm64/kvm/hyp/include/hyp/switch.h +13 -10
@@ -364,11 +364,14 @@ static inline bool esr_is_ptrauth_trap(u32 esr)return false; } -#define __ptrauth_save_key (regs, key) \ -({ \ - regs[key ## KEYLO_EL1] = read_sysreg_s(SYS_ ## key ## KEYLO_EL1); \ - regs[key ## KEYHI_EL1] = read_sysreg_s(SYS_ ## key ## KEYHI_EL1); \ -}) +#define __ptrauth_save_key (ctxt, key) \ + do { \ + u64 __val; \ + __val = read_sysreg_s(SYS_ ## key ## KEYLO_EL1); \ + ctxt_sys_reg(ctxt, key ## KEYLO_EL1) = __val; \ + __val = read_sysreg_s(SYS_ ## key ## KEYHI_EL1); \ + ctxt_sys_reg(ctxt, key ## KEYHI_EL1) = __val; \ +} while(0)static inline bool __hyp_handle_ptrauth (struct kvm_vcpu *vcpu) {
@@ -380,11 +383,11 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu)return false; ctxt = &__hyp_this_cpu_ptr (kvm_host_data)->host_ctxt; -__ptrauth_save_key (ctxt->sys_regs,APIA ); -__ptrauth_save_key (ctxt->sys_regs,APIB ); -__ptrauth_save_key (ctxt->sys_regs,APDA ); -__ptrauth_save_key (ctxt->sys_regs,APDB ); -__ptrauth_save_key (ctxt->sys_regs,APGA ); +__ptrauth_save_key (ctxt,APIA ); +__ptrauth_save_key (ctxt,APIB ); +__ptrauth_save_key (ctxt,APDA ); +__ptrauth_save_key (ctxt,APDB ); +__ptrauth_save_key (ctxt,APGA );vcpu_ptrauth_enable (vcpu);
Signature · 1 line
2.27.0
Re: [PATCH v3 09/17] KVM: arm64: pauth: Use ctxt_sys_reg() instead of raw sys_regs access
Hi,
Quoted 13 lines · Marc Zyngier
Now that we have a wrapper for the sysreg accesses, let's use that consistently.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-)arch/arm64/kvm/hyp/include/hyp/switch.h +0
-0
arch/arm64/kvm/hyp/include/hyp/switch.h static inline bool esr_is_ptrauth_trap(u32 esr) · line 368
Quoted 10 lines
arch/arm64/kvm/hyp/include/hyp/switch.h +2
-5@@ -364,11 +364,14 @@ static inline bool esr_is_ptrauth_trap(u32 esr)return false; } -#define __ptrauth_save_key (regs, key) \ -({ \ - regs[key ## KEYLO_EL1] = read_sysreg_s(SYS_ ## key ## KEYLO_EL1); \ - regs[key ## KEYHI_EL1] = read_sysreg_s(SYS_ ## key ## KEYHI_EL1); \ -}) +#define __ptrauth_save_key (ctxt, key) \ + do { \
Nitpick: the indentation for the do instruction doesn't match the indentation for while(0).
Quoted 16 lines
arch/arm64/kvm/hyp/include/hyp/switch.h +6
-3+u64 __val; \ + __val = read_sysreg_s(SYS_ ## key ## KEYLO_EL1); \ + ctxt_sys_reg(ctxt, key ## KEYLO_EL1) = __val; \ + __val = read_sysreg_s(SYS_ ## key ## KEYHI_EL1); \ + ctxt_sys_reg(ctxt, key ## KEYHI_EL1) = __val; \ +} while(0)static inline bool __hyp_handle_ptrauth (struct kvm_vcpu *vcpu) {@@ -380,11 +383,11 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu)return false; ctxt = &__hyp_this_cpu_ptr (kvm_host_data)->host_ctxt; -__ptrauth_save_key (ctxt->sys_regs,APIA ); -__ptrauth_save_key (ctxt->sys_regs,APIB ); -__ptrauth_save_key (ctxt->sys_regs,APDA );
arch/arm64/kvm/hyp/include/hyp/switch.h static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu) · line 393
Quoted 10 lines
arch/arm64/kvm/hyp/include/hyp/switch.h +5
-2-__ptrauth_save_key (ctxt->sys_regs,APDB ); -__ptrauth_save_key (ctxt->sys_regs,APGA ); +__ptrauth_save_key (ctxt,APIA ); +__ptrauth_save_key (ctxt,APIB ); +__ptrauth_save_key (ctxt,APDA ); +__ptrauth_save_key (ctxt,APDB ); +__ptrauth_save_key (ctxt,APGA );vcpu_ptrauth_enable (vcpu);
Looks good to me. I also grep'ed for sys_regs at the top level of the sources and I didn't find any instances that we could replace with the accessors:
Thanks,
Alex
Marc Zyngier
maintainerPATCH v3 10/17 KVM: arm64: debug: Drop useless vpcu parameter
In reply to Marc ZyngierAs part of the ongoing spring cleanup, remove the now useless vcpu parameter that is passed around (host and guest contexts give us everything we need).
--- arch/arm64/kvm/hyp/include/hyp/debug-sr.h | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)
arch/arm64/kvm/hyp/include/hyp/debug-sr.h +8 -10
@@ -88,9 +88,8 @@default :write_debug (ptr[0 ], reg,0 ); \ } -static inline void __debug_save_state (struct kvm_vcpu *vcpu, -struct kvm_guest_debug_arch *dbg, -struct kvm_cpu_context *ctxt) +static void __debug_save_state (struct kvm_guest_debug_arch *dbg, +struct kvm_cpu_context *ctxt) {u64 aa64dfr0;int brps, wrps;
@@ -107,9 +106,8 @@ static inline void __debug_save_state(struct kvm_vcpu *vcpu,ctxt_sys_reg (ctxt,MDCCINT_EL1 ) =read_sysreg (mdccint_el1); } -static inline void __debug_restore_state (struct kvm_vcpu *vcpu, -struct kvm_guest_debug_arch *dbg, -struct kvm_cpu_context *ctxt) +static void __debug_restore_state (struct kvm_guest_debug_arch *dbg, +struct kvm_cpu_context *ctxt) {u64 aa64dfr0;int brps, wrps;
@@ -142,8 +140,8 @@ static inline void __debug_switch_to_guest_common(struct kvm_vcpu *vcpu) host_dbg = &vcpu->arch.host_debug_state.regs; guest_dbg =kern_hyp_va (vcpu->arch.debug_ptr); -__debug_save_state (vcpu, host_dbg, host_ctxt); -__debug_restore_state (vcpu, guest_dbg, guest_ctxt); +__debug_save_state (host_dbg, host_ctxt); +__debug_restore_state (guest_dbg, guest_ctxt); }static inline void __debug_switch_to_host_common (struct kvm_vcpu *vcpu)
@@ -161,8 +159,8 @@ static inline void __debug_switch_to_host_common(struct kvm_vcpu *vcpu) host_dbg = &vcpu->arch.host_debug_state.regs; guest_dbg =kern_hyp_va (vcpu->arch.debug_ptr); -__debug_save_state (vcpu, guest_dbg, guest_ctxt); -__debug_restore_state (vcpu, host_dbg, host_ctxt); +__debug_save_state (guest_dbg, guest_ctxt); +__debug_restore_state (host_dbg, host_ctxt); vcpu->arch.flags &= ~KVM_ARM64_DEBUG_DIRTY ; }
Signature · 1 line
2.27.0
Re: [PATCH v3 10/17] KVM: arm64: debug: Drop useless vpcu parameter
Hi Marc,
Quoted 49 lines · Marc Zyngier
As part of the ongoing spring cleanup, remove the now useless vcpu parameter that is passed around (host and guest contexts give us everything we need).
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/hyp/include/hyp/debug-sr.h | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)arch/arm64/kvm/hyp/include/hyp/debug-sr.h +6
-8@@ -88,9 +88,8 @@default :write_debug (ptr[0 ], reg,0 ); \ } -static inline void __debug_save_state (struct kvm_vcpu *vcpu, -struct kvm_guest_debug_arch *dbg, -struct kvm_cpu_context *ctxt) +static void __debug_save_state (struct kvm_guest_debug_arch *dbg, +struct kvm_cpu_context *ctxt) {u64 aa64dfr0;int brps, wrps;@@ -107,9 +106,8 @@ static inline void __debug_save_state(struct kvm_vcpu *vcpu,ctxt_sys_reg (ctxt,MDCCINT_EL1 ) =read_sysreg (mdccint_el1); } -static inline void __debug_restore_state (struct kvm_vcpu *vcpu, -struct kvm_guest_debug_arch *dbg, -struct kvm_cpu_context *ctxt) +static void __debug_restore_state (struct kvm_guest_debug_arch *dbg, +struct kvm_cpu_context *ctxt) {u64 aa64dfr0;int brps, wrps;@@ -142,8 +140,8 @@ static inline void __debug_switch_to_guest_common(struct kvm_vcpu *vcpu) host_dbg = &vcpu->arch.host_debug_state.regs; guest_dbg =kern_hyp_va (vcpu->arch.debug_ptr); -__debug_save_state (vcpu, host_dbg, host_ctxt); -__debug_restore_state (vcpu, guest_dbg, guest_ctxt); +__debug_save_state (host_dbg, host_ctxt); +__debug_restore_state (guest_dbg, guest_ctxt); }static inline void __debug_switch_to_host_common (struct kvm_vcpu *vcpu)
arch/arm64/kvm/hyp/include/hyp/debug-sr.h static inline void __debug_switch_to_host_common(struct kvm_vcpu *vcpu) · line 166
Quoted 10 lines
arch/arm64/kvm/hyp/include/hyp/debug-sr.h +2
-2@@ -161,8 +159,8 @@ static inline void __debug_switch_to_host_common(struct kvm_vcpu *vcpu) host_dbg = &vcpu->arch.host_debug_state.regs; guest_dbg =kern_hyp_va (vcpu->arch.debug_ptr); -__debug_save_state (vcpu, guest_dbg, guest_ctxt); -__debug_restore_state (vcpu, host_dbg, host_ctxt); +__debug_save_state (guest_dbg, guest_ctxt); +__debug_restore_state (host_dbg, host_ctxt); vcpu->arch.flags &= ~KVM_ARM64_DEBUG_DIRTY ; }
Looks fine to me. I also had a look at the other files where we started using the __vcpu_sys_reg/ctxt_sys_reg accessors, and I didn't find any functions where we could eliminate the vcpu parameter, like we did here:
Thanks,
Alex
Marc Zyngier
maintainerPATCH v3 11/17 KVM: arm64: Make struct kvm_regs userspace-only
In reply to Marc Zyngierstruct kvm_regs is used by userspace to indicate which register gets accessed by the {GET,SET}_ONE_REG API. But as we're about to refactor the layout of the in-kernel register structures, we need the kernel to move away from it.
Let's make kvm_regs userspace only, and let the kernel map it to its own internal representation.
--- arch/arm64/include/asm/kvm_emulate.h | 18 +++--- arch/arm64/include/asm/kvm_host.h | 12 +++- arch/arm64/kernel/asm-offsets.c | 3 +- arch/arm64/kvm/fpsimd.c | 2 +- arch/arm64/kvm/guest.c | 70 +++++++++++++++++----- arch/arm64/kvm/hyp/entry.S | 3 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 +- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 24 ++++---- arch/arm64/kvm/regmap.c | 6 +- arch/arm64/kvm/reset.c | 2 +- 10 files changed, 96 insertions(+), 48 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +9 -9
@@ -124,12 +124,12 @@ static inline void vcpu_set_vsesr(struct kvm_vcpu *vcpu, u64 vsesr)static __always_inline unsigned long *vcpu_pc (const struct kvm_vcpu *vcpu) { -return (unsigned long *)&vcpu_gp_regs (vcpu)->regs.pc; +return (unsigned long *)&vcpu_gp_regs (vcpu)->pc; }static inline unsigned long *__vcpu_elr_el1 (const struct kvm_vcpu *vcpu) { -return (unsigned long *)&vcpu_gp_regs (vcpu)->elr_el1; +return (unsigned long *)&vcpu->arch.ctxt.elr_el1; }static inline unsigned long vcpu_read_elr_el1 (const struct kvm_vcpu *vcpu)
@@ -150,7 +150,7 @@ static inline void vcpu_write_elr_el1(const struct kvm_vcpu *vcpu, unsigned longstatic __always_inline unsigned long *vcpu_cpsr (const struct kvm_vcpu *vcpu) { -return (unsigned long *)&vcpu_gp_regs (vcpu)->regs.pstate; +return (unsigned long *)&vcpu_gp_regs (vcpu)->pstate; } static __always_inline bool vcpu_mode_is_32bit(const struct kvm_vcpu *vcpu)
@@ -179,14 +179,14 @@ static inline void vcpu_set_thumb(struct kvm_vcpu *vcpu)static __always_inline unsigned long vcpu_get_reg (const struct kvm_vcpu *vcpu,u8 reg_num) { -return (reg_num ==31 ) ?0 :vcpu_gp_regs (vcpu)->regs.regs[reg_num]; +return (reg_num ==31 ) ?0 :vcpu_gp_regs (vcpu)->regs[reg_num]; } static __always_inline void vcpu_set_reg(struct kvm_vcpu *vcpu, u8 reg_num,unsigned long val) {if (reg_num !=31 ) -vcpu_gp_regs (vcpu)->regs.regs[reg_num] = val; +vcpu_gp_regs (vcpu)->regs[reg_num] = val; }static inline unsigned long vcpu_read_spsr (const struct kvm_vcpu *vcpu)
@@ -197,7 +197,7 @@ static inline unsigned long vcpu_read_spsr(const struct kvm_vcpu *vcpu)if (vcpu->arch.sysregs_loaded_on_cpu)return read_sysreg_el1 (SYS_SPSR );else -return vcpu_gp_regs (vcpu)->spsr[KVM_SPSR_EL1 ]; +return vcpu->arch.ctxt.spsr[KVM_SPSR_EL1 ]; }static inline void vcpu_write_spsr (struct kvm_vcpu *vcpu,unsigned long v)
@@ -210,7 +210,7 @@ static inline void vcpu_write_spsr(struct kvm_vcpu *vcpu, unsigned long v)if (vcpu->arch.sysregs_loaded_on_cpu)write_sysreg_el1 (v,SYS_SPSR );else -vcpu_gp_regs (vcpu)->spsr[KVM_SPSR_EL1 ] = v; + vcpu->arch.ctxt.spsr[KVM_SPSR_EL1 ] = v; }/*
@@ -519,11 +519,11 @@ static __always_inline void kvm_skip_instr(struct kvm_vcpu *vcpu, bool is_wide_i static __always_inline void __kvm_skip_instr(struct kvm_vcpu *vcpu) { *vcpu_pc (vcpu) =read_sysreg_el2 (SYS_ELR ); - vcpu->arch.ctxt.gp_regs.regs.pstate =read_sysreg_el2 (SYS_SPSR ); +vcpu_gp_regs (vcpu)->pstate =read_sysreg_el2 (SYS_SPSR );kvm_skip_instr (vcpu,kvm_vcpu_trap_il_is32bit (vcpu)); -write_sysreg_el2 (vcpu->arch.ctxt.gp_regs.regs.pstate,SYS_SPSR ); +write_sysreg_el2 (vcpu_gp_regs (vcpu)->pstate,SYS_SPSR );write_sysreg_el2 (*vcpu_pc (vcpu),SYS_ELR ); }
arch/arm64/include/asm/kvm_host.h +10 -2
@@ -236,7 +236,15 @@ enum vcpu_sysreg {#define NR_COPRO_REGS (NR_SYS_REGS * 2)struct kvm_cpu_context { -struct kvm_regs gp_regs; +struct user_pt_regs regs;/* sp = sp_el0 */ + +u64 sp_el1; +u64 elr_el1; + +u64 spsr[KVM_NR_SPSR ]; + +struct user_fpsimd_state fp_regs; +union {u64 sys_regs[NR_SYS_REGS ];u32 copro[NR_COPRO_REGS ];
@@ -402,7 +410,7 @@ struct kvm_vcpu_arch { system_supports_generic_auth()) && \ ((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_PTRAUTH)) -#define vcpu_gp_regs (v) (&(v)->arch.ctxt.gp_regs) +#define vcpu_gp_regs (v) (&(v)->arch.ctxt.regs)/* * Only use __vcpu_sys_reg/ctxt_sys_reg if you know you want the
arch/arm64/kernel/asm-offsets.c +1 -2
@@ -102,13 +102,12 @@ int main(void) DEFINE(VCPU_FAULT_DISR, offsetof(struct kvm_vcpu, arch.fault.disr_el1)); DEFINE(VCPU_WORKAROUND_FLAGS, offsetof(struct kvm_vcpu, arch.workaround_flags)); DEFINE(VCPU_HCR_EL2, offsetof(struct kvm_vcpu, arch.hcr_el2)); -DEFINE (CPU_GP_REGS , offsetof(struct kvm_cpu_context , gp_regs)); +DEFINE (CPU_USER_PT_REGS , offsetof(struct kvm_cpu_context , regs)); DEFINE(CPU_APIAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APIAKEYLO_EL1])); DEFINE(CPU_APIBKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APIBKEYLO_EL1])); DEFINE(CPU_APDAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APDAKEYLO_EL1])); DEFINE(CPU_APDBKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APDBKEYLO_EL1])); DEFINE(CPU_APGAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APGAKEYLO_EL1])); -DEFINE (CPU_USER_PT_REGS , offsetof(struct kvm_regs , regs));DEFINE (HOST_CONTEXT_VCPU , offsetof(struct kvm_cpu_context , __hyp_running_vcpu));DEFINE (HOST_DATA_CONTEXT , offsetof(struct kvm_host_data , host_ctxt));#endif
arch/arm64/kvm/fpsimd.c +1 -1
@@ -85,7 +85,7 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu)WARN_ON_ONCE (!irqs_disabled ());if (vcpu->arch.flags &KVM_ARM64_FP_ENABLED ) { -fpsimd_bind_state_to_cpu (&vcpu->arch.ctxt.gp_regs.fp_regs, +fpsimd_bind_state_to_cpu (&vcpu->arch.ctxt.fp_regs, vcpu->arch.sve_state, vcpu->arch.sve_max_vl);
arch/arm64/kvm/guest.c +56 -14
117 lines not on this page · raw message
arch/arm64/kvm/hyp/entry.S +1 -2
@@ -16,8 +16,7 @@ #include <asm/kvm_mmu.h> #include <asm/kvm_ptrauth.h> -#define CPU_GP_REG_OFFSET(x) (CPU_GP_REGS + x) -#define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x) +#define CPU_XREG_OFFSET(x) (CPU_USER_PT_REGS + 8*x) #define CPU_SP_EL0_OFFSET (CPU_XREG_OFFSET(30) + 8) .text
arch/arm64/kvm/hyp/include/hyp/switch.h +2 -2
@@ -266,11 +266,11 @@ static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)if (sve_guest) {sve_load_state (vcpu_sve_pffr (vcpu), - &vcpu->arch.ctxt.gp_regs.fp_regs.fpsr, + &vcpu->arch.ctxt.fp_regs.fpsr,sve_vq_from_vl (vcpu->arch.sve_max_vl) -1 );write_sysreg_s (__vcpu_sys_reg (vcpu,ZCR_EL1 ),SYS_ZCR_EL12 ); }else { -__fpsimd_restore_state (&vcpu->arch.ctxt.gp_regs.fp_regs); +__fpsimd_restore_state (&vcpu->arch.ctxt.fp_regs); }/* Skip restoring fpexc32 for AArch64 guests */
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +12 -12
@@ -46,15 +46,15 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)ctxt_sys_reg (ctxt,PAR_EL1 ) =read_sysreg (par_el1);ctxt_sys_reg (ctxt,TPIDR_EL1 ) =read_sysreg (tpidr_el1); - ctxt->gp_regs.sp_el1 =read_sysreg (sp_el1); - ctxt->gp_regs.elr_el1 =read_sysreg_el1 (SYS_ELR ); - ctxt->gp_regs.spsr[KVM_SPSR_EL1 ]=read_sysreg_el1 (SYS_SPSR ); + ctxt->sp_el1 =read_sysreg (sp_el1); + ctxt->elr_el1 =read_sysreg_el1 (SYS_ELR ); + ctxt->spsr[KVM_SPSR_EL1 ] =read_sysreg_el1 (SYS_SPSR ); }static inline void __sysreg_save_el2_return_state (struct kvm_cpu_context *ctxt) { - ctxt->gp_regs.regs.pc =read_sysreg_el2 (SYS_ELR ); - ctxt->gp_regs.regs.pstate =read_sysreg_el2 (SYS_SPSR ); + ctxt->regs.pc =read_sysreg_el2 (SYS_ELR ); + ctxt->regs.pstate =read_sysreg_el2 (SYS_SPSR );if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN ))ctxt_sys_reg (ctxt,DISR_EL1 ) =read_sysreg_s (SYS_VDISR_EL2 );
@@ -125,14 +125,14 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)write_sysreg_el1 (ctxt_sys_reg (ctxt,TCR_EL1 ),SYS_TCR ); } -write_sysreg (ctxt->gp_regs.sp_el1, sp_el1); -write_sysreg_el1 (ctxt->gp_regs.elr_el1,SYS_ELR ); -write_sysreg_el1 (ctxt->gp_regs.spsr[KVM_SPSR_EL1 ],SYS_SPSR ); +write_sysreg (ctxt->sp_el1, sp_el1); +write_sysreg_el1 (ctxt->elr_el1,SYS_ELR ); +write_sysreg_el1 (ctxt->spsr[KVM_SPSR_EL1 ],SYS_SPSR ); }static inline void __sysreg_restore_el2_return_state (struct kvm_cpu_context *ctxt) { -u64 pstate = ctxt->gp_regs.regs.pstate; +u64 pstate = ctxt->regs.pstate;u64 mode = pstate &PSR_AA32_MODE_MASK ;/*
@@ -149,7 +149,7 @@ static inline void __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxif (!(mode &PSR_MODE32_BIT ) && mode >= PSR_MODE_EL2t) pstate = PSR_MODE_EL2h |PSR_IL_BIT ; -write_sysreg_el2 (ctxt->gp_regs.regs.pc,SYS_ELR ); +write_sysreg_el2 (ctxt->regs.pc,SYS_ELR );write_sysreg_el2 (pstate,SYS_SPSR );if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN ))
@@ -163,7 +163,7 @@ static inline void __sysreg32_save_state(struct kvm_vcpu *vcpu)if (!vcpu_el1_is_32bit (vcpu))return ; - spsr = vcpu->arch.ctxt.gp_regs.spsr; + spsr = vcpu->arch.ctxt.spsr; spsr[KVM_SPSR_ABT ] =read_sysreg (spsr_abt); spsr[KVM_SPSR_UND ] =read_sysreg (spsr_und);
@@ -184,7 +184,7 @@ static inline void __sysreg32_restore_state(struct kvm_vcpu *vcpu)if (!vcpu_el1_is_32bit (vcpu))return ; - spsr = vcpu->arch.ctxt.gp_regs.spsr; + spsr = vcpu->arch.ctxt.spsr;write_sysreg (spsr[KVM_SPSR_ABT ], spsr_abt);write_sysreg (spsr[KVM_SPSR_UND ], spsr_und);
arch/arm64/kvm/regmap.c +3 -3
@@ -100,7 +100,7 @@ static const unsigned long vcpu_reg_offsets[VCPU_NR_MODES][16] = {*/ unsigned long *vcpu_reg32 (const struct kvm_vcpu *vcpu,u8 reg_num) { -unsigned long *reg_array = (unsigned long *)&vcpu->arch.ctxt.gp_regs.regs; +unsigned long *reg_array = (unsigned long *)&vcpu->arch.ctxt.regs;unsigned long mode = *vcpu_cpsr (vcpu) &PSR_AA32_MODE_MASK ;switch (mode) {
@@ -148,7 +148,7 @@ unsigned long vcpu_read_spsr32(const struct kvm_vcpu *vcpu)int spsr_idx =vcpu_spsr32_mode (vcpu);if (!vcpu->arch.sysregs_loaded_on_cpu) -return vcpu_gp_regs (vcpu)->spsr[spsr_idx]; +return vcpu->arch.ctxt.spsr[spsr_idx];switch (spsr_idx) {case KVM_SPSR_SVC :
@@ -171,7 +171,7 @@ void vcpu_write_spsr32(struct kvm_vcpu *vcpu, unsigned long v)int spsr_idx =vcpu_spsr32_mode (vcpu);if (!vcpu->arch.sysregs_loaded_on_cpu) { -vcpu_gp_regs (vcpu)->spsr[spsr_idx] = v; + vcpu->arch.ctxt.spsr[spsr_idx] = v;return ; }
arch/arm64/kvm/reset.c +1 -1
@@ -288,7 +288,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)/* Reset core registers */ memset (vcpu_gp_regs (vcpu),0 ,sizeof (*vcpu_gp_regs (vcpu))); -vcpu_gp_regs (vcpu)->regs.pstate = pstate; +vcpu_gp_regs (vcpu)->pstate = pstate;/* Reset system registers */ kvm_reset_sys_regs (vcpu);
Signature · 1 line
2.27.0
Marc Zyngier
maintainerPATCH v3 12/17 KVM: arm64: Move ELR_EL1 to the system register array
In reply to Marc ZyngierAs ELR-EL1 is a VNCR-capable register with ARMv8.4-NV, let's move it to the sys_regs array and repaint the accessors. While we're at it, let's kill the now useless accessors used only on the fault injection path.
--- arch/arm64/include/asm/kvm_emulate.h | 21 --------------------- arch/arm64/include/asm/kvm_host.h | 3 ++- arch/arm64/kvm/guest.c | 2 +- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 4 ++-- arch/arm64/kvm/inject_fault.c | 2 +- arch/arm64/kvm/sys_regs.c | 2 ++ 6 files changed, 8 insertions(+), 26 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +0 -21
@@ -127,27 +127,6 @@ static __always_inline unsigned long *vcpu_pc(const struct kvm_vcpu *vcpu)return (unsigned long *)&vcpu_gp_regs (vcpu)->pc; } -static inline unsigned long *__vcpu_elr_el1 (const struct kvm_vcpu *vcpu) -{ -return (unsigned long *)&vcpu->arch.ctxt.elr_el1; -} - -static inline unsigned long vcpu_read_elr_el1 (const struct kvm_vcpu *vcpu) -{ -if (vcpu->arch.sysregs_loaded_on_cpu) -return read_sysreg_el1 (SYS_ELR ); -else -return *__vcpu_elr_el1 (vcpu); -} - -static inline void vcpu_write_elr_el1 (const struct kvm_vcpu *vcpu,unsigned long v) -{ -if (vcpu->arch.sysregs_loaded_on_cpu) -write_sysreg_el1 (v,SYS_ELR ); -else - *__vcpu_elr_el1 (vcpu) = v; -} -static __always_inline unsigned long *vcpu_cpsr (const struct kvm_vcpu *vcpu) {return (unsigned long *)&vcpu_gp_regs (vcpu)->pstate;
arch/arm64/include/asm/kvm_host.h +2 -1
@@ -185,6 +185,8 @@ enum vcpu_sysreg {APGAKEYLO_EL1 ,APGAKEYHI_EL1 , +ELR_EL1 , +/* 32bit specific registers. Keep them at the end of the range */ DACR32_EL2 ,/* Domain Access Control Register */ IFSR32_EL2, /* Instruction Fault Status Register */
@@ -239,7 +241,6 @@ struct kvm_cpu_context {struct user_pt_regs regs;/* sp = sp_el0 */ u64 sp_el1; -u64 elr_el1;u64 spsr[KVM_NR_SPSR ];
arch/arm64/kvm/guest.c +1 -1
@@ -132,7 +132,7 @@ static void *core_reg_addr(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)return &vcpu->arch.ctxt.sp_el1;case KVM_REG_ARM_CORE_REG (elr_el1): -return &vcpu->arch.ctxt.elr_el1; +return __ctxt_sys_reg (&vcpu->arch.ctxt,ELR_EL1 ); case KVM_REG_ARM_CORE_REG(spsr[0]) ...KVM_REG_ARM_CORE_REG (spsr[KVM_NR_SPSR -1 ]):
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +2 -2
@@ -47,7 +47,7 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)ctxt_sys_reg (ctxt,TPIDR_EL1 ) =read_sysreg (tpidr_el1); ctxt->sp_el1 =read_sysreg (sp_el1); - ctxt->elr_el1 =read_sysreg_el1 (SYS_ELR ); +ctxt_sys_reg (ctxt,ELR_EL1 ) =read_sysreg_el1 (SYS_ELR ); ctxt->spsr[KVM_SPSR_EL1 ] =read_sysreg_el1 (SYS_SPSR ); }
@@ -126,7 +126,7 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt) }write_sysreg (ctxt->sp_el1, sp_el1); -write_sysreg_el1 (ctxt->elr_el1,SYS_ELR ); +write_sysreg_el1 (ctxt_sys_reg (ctxt,ELR_EL1 ),SYS_ELR );write_sysreg_el1 (ctxt->spsr[KVM_SPSR_EL1 ],SYS_SPSR ); }
arch/arm64/kvm/inject_fault.c +1 -1
@@ -64,7 +64,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,case PSR_MODE_EL1h: vbar =vcpu_read_sys_reg (vcpu,VBAR_EL1 ); sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL1 ); -vcpu_write_elr_el1 (vcpu, *vcpu_pc (vcpu)); +vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL1 );break ;default :/* Don't do that */
arch/arm64/kvm/sys_regs.c +2 -0
@@ -94,6 +94,7 @@ static bool __vcpu_read_sys_reg_from_cpu(int reg, u64 *val)case TPIDR_EL1 : *val =read_sysreg_s (SYS_TPIDR_EL1 );break ;case AMAIR_EL1 : *val =read_sysreg_s (SYS_AMAIR_EL12 );break ;case CNTKCTL_EL1 : *val =read_sysreg_s (SYS_CNTKCTL_EL12 );break ; +case ELR_EL1 : *val =read_sysreg_s (SYS_ELR_EL12 );break ;case PAR_EL1 : *val =read_sysreg_s (SYS_PAR_EL1 );break ;case DACR32_EL2 : *val =read_sysreg_s (SYS_DACR32_EL2 );break ;case IFSR32_EL2 : *val =read_sysreg_s (SYS_IFSR32_EL2 );break ;
@@ -133,6 +134,7 @@ static bool __vcpu_write_sys_reg_to_cpu(u64 val, int reg)case TPIDR_EL1 :write_sysreg_s (val,SYS_TPIDR_EL1 );break ;case AMAIR_EL1 :write_sysreg_s (val,SYS_AMAIR_EL12 );break ;case CNTKCTL_EL1 :write_sysreg_s (val,SYS_CNTKCTL_EL12 );break ; +case ELR_EL1 :write_sysreg_s (val,SYS_ELR_EL12 );break ;case PAR_EL1 :write_sysreg_s (val,SYS_PAR_EL1 );break ;case DACR32_EL2 :write_sysreg_s (val,SYS_DACR32_EL2 );break ;case IFSR32_EL2 :write_sysreg_s (val,SYS_IFSR32_EL2 );break ;
Signature · 1 line
2.27.0
Marc Zyngier
maintainerPATCH v3 13/17 KVM: arm64: Move SP_EL1 to the system register array
In reply to Marc ZyngierSP_EL1 being a VNCR-capable register with ARMv8.4-NV, move it to the system register array and update the accessors.
--- arch/arm64/include/asm/kvm_host.h | 3 +-- arch/arm64/kvm/guest.c | 2 +- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 4 ++-- 3 files changed, 4 insertions(+), 5 deletions(-)
arch/arm64/include/asm/kvm_host.h +1 -2
@@ -186,6 +186,7 @@ enum vcpu_sysreg {APGAKEYHI_EL1 ,ELR_EL1 , +SP_EL1 ,/* 32bit specific registers. Keep them at the end of the range */ DACR32_EL2, /* Domain Access Control Register */
@@ -240,8 +241,6 @@ enum vcpu_sysreg {struct kvm_cpu_context {struct user_pt_regs regs;/* sp = sp_el0 */ -u64 sp_el1; -u64 spsr[KVM_NR_SPSR ];struct user_fpsimd_state fp_regs;
arch/arm64/kvm/guest.c +1 -1
@@ -129,7 +129,7 @@ static void *core_reg_addr(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)return &vcpu->arch.ctxt.regs.pstate;case KVM_REG_ARM_CORE_REG (sp_el1): -return &vcpu->arch.ctxt.sp_el1; +return __ctxt_sys_reg (&vcpu->arch.ctxt,SP_EL1 );case KVM_REG_ARM_CORE_REG (elr_el1):return __ctxt_sys_reg (&vcpu->arch.ctxt,ELR_EL1 );
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +2 -2
@@ -46,7 +46,7 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)ctxt_sys_reg (ctxt,PAR_EL1 ) =read_sysreg (par_el1);ctxt_sys_reg (ctxt,TPIDR_EL1 ) =read_sysreg (tpidr_el1); - ctxt->sp_el1 =read_sysreg (sp_el1); +ctxt_sys_reg (ctxt,SP_EL1 ) =read_sysreg (sp_el1);ctxt_sys_reg (ctxt,ELR_EL1 ) =read_sysreg_el1 (SYS_ELR ); ctxt->spsr[KVM_SPSR_EL1 ] =read_sysreg_el1 (SYS_SPSR ); }
@@ -125,7 +125,7 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)write_sysreg_el1 (ctxt_sys_reg (ctxt,TCR_EL1 ),SYS_TCR ); } -write_sysreg (ctxt->sp_el1, sp_el1); +write_sysreg (ctxt_sys_reg (ctxt,SP_EL1 ), sp_el1);write_sysreg_el1 (ctxt_sys_reg (ctxt,ELR_EL1 ),SYS_ELR );write_sysreg_el1 (ctxt->spsr[KVM_SPSR_EL1 ],SYS_SPSR ); }
Signature · 1 line
2.27.0
Marc Zyngier
maintainerPATCH v3 14/17 KVM: arm64: Disintegrate SPSR array
In reply to Marc ZyngierAs we're about to move SPSR_EL1 into the VNCR page, we need to disassociate it from the rest of the 32bit cruft. Let's break the array into individual fields.
--- arch/arm64/include/asm/kvm_emulate.h | 4 +-- arch/arm64/include/asm/kvm_host.h | 6 +++- arch/arm64/kvm/guest.c | 19 ++++++++---- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 28 +++++++---------- arch/arm64/kvm/regmap.c | 35 ++++++++++++++++++++-- 5 files changed, 63 insertions(+), 29 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +2 -2
@@ -176,7 +176,7 @@ static inline unsigned long vcpu_read_spsr(const struct kvm_vcpu *vcpu)if (vcpu->arch.sysregs_loaded_on_cpu)return read_sysreg_el1 (SYS_SPSR );else -return vcpu->arch.ctxt.spsr[KVM_SPSR_EL1 ]; +return vcpu->arch.ctxt.spsr_el1; }static inline void vcpu_write_spsr (struct kvm_vcpu *vcpu,unsigned long v)
@@ -189,7 +189,7 @@ static inline void vcpu_write_spsr(struct kvm_vcpu *vcpu, unsigned long v)if (vcpu->arch.sysregs_loaded_on_cpu)write_sysreg_el1 (v,SYS_SPSR );else - vcpu->arch.ctxt.spsr[KVM_SPSR_EL1 ] = v; + vcpu->arch.ctxt.spsr_el1 = v; }/*
arch/arm64/include/asm/kvm_host.h +5 -1
@@ -241,7 +241,11 @@ enum vcpu_sysreg {struct kvm_cpu_context {struct user_pt_regs regs;/* sp = sp_el0 */ -u64 spsr[KVM_NR_SPSR ]; +u64 spsr_el1;/* aka spsr_svc */ +u64 spsr_abt; +u64 spsr_und; +u64 spsr_irq; +u64 spsr_fiq;struct user_fpsimd_state fp_regs;
arch/arm64/kvm/guest.c +14 -5
@@ -134,11 +134,20 @@ static void *core_reg_addr(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)case KVM_REG_ARM_CORE_REG (elr_el1):return __ctxt_sys_reg (&vcpu->arch.ctxt,ELR_EL1 ); - case KVM_REG_ARM_CORE_REG(spsr[0]) ... -KVM_REG_ARM_CORE_REG (spsr[KVM_NR_SPSR -1 ]): - off -=KVM_REG_ARM_CORE_REG (spsr[0 ]); - off /=2 ; -return &vcpu->arch.ctxt.spsr[off]; +case KVM_REG_ARM_CORE_REG (spsr[KVM_SPSR_EL1 ]): +return &vcpu->arch.ctxt.spsr_el1; + +case KVM_REG_ARM_CORE_REG (spsr[KVM_SPSR_ABT ]): +return &vcpu->arch.ctxt.spsr_abt; + +case KVM_REG_ARM_CORE_REG (spsr[KVM_SPSR_UND ]): +return &vcpu->arch.ctxt.spsr_und; + +case KVM_REG_ARM_CORE_REG (spsr[KVM_SPSR_IRQ ]): +return &vcpu->arch.ctxt.spsr_irq; + +case KVM_REG_ARM_CORE_REG (spsr[KVM_SPSR_FIQ ]): +return &vcpu->arch.ctxt.spsr_fiq; case KVM_REG_ARM_CORE_REG(fp_regs.vregs[0]) ...KVM_REG_ARM_CORE_REG (fp_regs.vregs[31 ]):
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +10 -18
@@ -48,7 +48,7 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)ctxt_sys_reg (ctxt,SP_EL1 ) =read_sysreg (sp_el1);ctxt_sys_reg (ctxt,ELR_EL1 ) =read_sysreg_el1 (SYS_ELR ); - ctxt->spsr[KVM_SPSR_EL1 ] =read_sysreg_el1 (SYS_SPSR ); + ctxt->spsr_el1 =read_sysreg_el1 (SYS_SPSR ); }static inline void __sysreg_save_el2_return_state (struct kvm_cpu_context *ctxt)
@@ -127,7 +127,7 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)write_sysreg (ctxt_sys_reg (ctxt,SP_EL1 ), sp_el1);write_sysreg_el1 (ctxt_sys_reg (ctxt,ELR_EL1 ),SYS_ELR ); -write_sysreg_el1 (ctxt->spsr[KVM_SPSR_EL1 ],SYS_SPSR ); +write_sysreg_el1 (ctxt->spsr_el1,SYS_SPSR ); }static inline void __sysreg_restore_el2_return_state (struct kvm_cpu_context *ctxt)
@@ -158,17 +158,13 @@ static inline void __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxstatic inline void __sysreg32_save_state (struct kvm_vcpu *vcpu) { -u64 *spsr; -if (!vcpu_el1_is_32bit (vcpu))return ; - spsr = vcpu->arch.ctxt.spsr; - - spsr[KVM_SPSR_ABT ] =read_sysreg (spsr_abt); - spsr[KVM_SPSR_UND ] =read_sysreg (spsr_und); - spsr[KVM_SPSR_IRQ ] =read_sysreg (spsr_irq); - spsr[KVM_SPSR_FIQ ] =read_sysreg (spsr_fiq); + vcpu->arch.ctxt.spsr_abt =read_sysreg (spsr_abt); + vcpu->arch.ctxt.spsr_und =read_sysreg (spsr_und); + vcpu->arch.ctxt.spsr_irq =read_sysreg (spsr_irq); + vcpu->arch.ctxt.spsr_fiq =read_sysreg (spsr_fiq);__vcpu_sys_reg (vcpu,DACR32_EL2 ) =read_sysreg (dacr32_el2);__vcpu_sys_reg (vcpu,IFSR32_EL2 ) =read_sysreg (ifsr32_el2);
@@ -179,17 +175,13 @@ static inline void __sysreg32_save_state(struct kvm_vcpu *vcpu)static inline void __sysreg32_restore_state (struct kvm_vcpu *vcpu) { -u64 *spsr; -if (!vcpu_el1_is_32bit (vcpu))return ; - spsr = vcpu->arch.ctxt.spsr; - -write_sysreg (spsr[KVM_SPSR_ABT ], spsr_abt); -write_sysreg (spsr[KVM_SPSR_UND ], spsr_und); -write_sysreg (spsr[KVM_SPSR_IRQ ], spsr_irq); -write_sysreg (spsr[KVM_SPSR_FIQ ], spsr_fiq); +write_sysreg (vcpu->arch.ctxt.spsr_abt, spsr_abt); +write_sysreg (vcpu->arch.ctxt.spsr_und, spsr_und); +write_sysreg (vcpu->arch.ctxt.spsr_irq, spsr_irq); +write_sysreg (vcpu->arch.ctxt.spsr_fiq, spsr_fiq);write_sysreg (__vcpu_sys_reg (vcpu,DACR32_EL2 ), dacr32_el2);write_sysreg (__vcpu_sys_reg (vcpu,IFSR32_EL2 ), ifsr32_el2);
arch/arm64/kvm/regmap.c +32 -3
@@ -147,8 +147,20 @@ unsigned long vcpu_read_spsr32(const struct kvm_vcpu *vcpu) {int spsr_idx =vcpu_spsr32_mode (vcpu); -if (!vcpu->arch.sysregs_loaded_on_cpu) -return vcpu->arch.ctxt.spsr[spsr_idx]; +if (!vcpu->arch.sysregs_loaded_on_cpu) { +switch (spsr_idx) { +case KVM_SPSR_SVC : +return vcpu->arch.ctxt.spsr_el1; +case KVM_SPSR_ABT : +return vcpu->arch.ctxt.spsr_abt; +case KVM_SPSR_UND : +return vcpu->arch.ctxt.spsr_und; +case KVM_SPSR_IRQ : +return vcpu->arch.ctxt.spsr_irq; +case KVM_SPSR_FIQ : +return vcpu->arch.ctxt.spsr_fiq; + } + }switch (spsr_idx) {case KVM_SPSR_SVC :
@@ -171,7 +183,24 @@ void vcpu_write_spsr32(struct kvm_vcpu *vcpu, unsigned long v)int spsr_idx =vcpu_spsr32_mode (vcpu);if (!vcpu->arch.sysregs_loaded_on_cpu) { - vcpu->arch.ctxt.spsr[spsr_idx] = v; +switch (spsr_idx) { +case KVM_SPSR_SVC : + vcpu->arch.ctxt.spsr_el1 = v; +break ; +case KVM_SPSR_ABT : + vcpu->arch.ctxt.spsr_abt = v; +break ; +case KVM_SPSR_UND : + vcpu->arch.ctxt.spsr_und = v; +break ; +case KVM_SPSR_IRQ : + vcpu->arch.ctxt.spsr_irq = v; +break ; +case KVM_SPSR_FIQ : + vcpu->arch.ctxt.spsr_fiq = v; +break ; + } +return ; }
Signature · 1 line
2.27.0
Marc Zyngier
maintainerPATCH v3 15/17 KVM: arm64: Move SPSR_EL1 to the system register array
In reply to Marc ZyngierSPSR_EL1 being a VNCR-capable register with ARMv8.4-NV, move it to the sysregs array and update the accessors.
--- arch/arm64/include/asm/kvm_emulate.h | 4 ++-- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/kvm/guest.c | 2 +- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 4 ++-- arch/arm64/kvm/regmap.c | 4 ++-- 5 files changed, 8 insertions(+), 8 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +2 -2
@@ -176,7 +176,7 @@ static inline unsigned long vcpu_read_spsr(const struct kvm_vcpu *vcpu)if (vcpu->arch.sysregs_loaded_on_cpu)return read_sysreg_el1 (SYS_SPSR );else -return vcpu->arch.ctxt.spsr_el1; +return __vcpu_sys_reg (vcpu,SPSR_EL1 ); }static inline void vcpu_write_spsr (struct kvm_vcpu *vcpu,unsigned long v)
@@ -189,7 +189,7 @@ static inline void vcpu_write_spsr(struct kvm_vcpu *vcpu, unsigned long v)if (vcpu->arch.sysregs_loaded_on_cpu)write_sysreg_el1 (v,SYS_SPSR );else - vcpu->arch.ctxt.spsr_el1 = v; +__vcpu_sys_reg (vcpu,SPSR_EL1 ) = v; }/*
arch/arm64/include/asm/kvm_host.h +1 -1
@@ -187,6 +187,7 @@ enum vcpu_sysreg {ELR_EL1 ,SP_EL1 , +SPSR_EL1 ,/* 32bit specific registers. Keep them at the end of the range */ DACR32_EL2, /* Domain Access Control Register */
@@ -241,7 +242,6 @@ enum vcpu_sysreg {struct kvm_cpu_context {struct user_pt_regs regs;/* sp = sp_el0 */ -u64 spsr_el1;/* aka spsr_svc */ u64 spsr_abt;u64 spsr_und;u64 spsr_irq;
arch/arm64/kvm/guest.c +1 -1
@@ -135,7 +135,7 @@ static void *core_reg_addr(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)return __ctxt_sys_reg (&vcpu->arch.ctxt,ELR_EL1 );case KVM_REG_ARM_CORE_REG (spsr[KVM_SPSR_EL1 ]): -return &vcpu->arch.ctxt.spsr_el1; +return __ctxt_sys_reg (&vcpu->arch.ctxt,SPSR_EL1 );case KVM_REG_ARM_CORE_REG (spsr[KVM_SPSR_ABT ]):return &vcpu->arch.ctxt.spsr_abt;
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +2 -2
@@ -48,7 +48,7 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)ctxt_sys_reg (ctxt,SP_EL1 ) =read_sysreg (sp_el1);ctxt_sys_reg (ctxt,ELR_EL1 ) =read_sysreg_el1 (SYS_ELR ); - ctxt->spsr_el1 =read_sysreg_el1 (SYS_SPSR ); +ctxt_sys_reg (ctxt,SPSR_EL1 ) =read_sysreg_el1 (SYS_SPSR ); }static inline void __sysreg_save_el2_return_state (struct kvm_cpu_context *ctxt)
@@ -127,7 +127,7 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)write_sysreg (ctxt_sys_reg (ctxt,SP_EL1 ), sp_el1);write_sysreg_el1 (ctxt_sys_reg (ctxt,ELR_EL1 ),SYS_ELR ); -write_sysreg_el1 (ctxt->spsr_el1,SYS_SPSR ); +write_sysreg_el1 (ctxt_sys_reg (ctxt,SPSR_EL1 ),SYS_SPSR ); }static inline void __sysreg_restore_el2_return_state (struct kvm_cpu_context *ctxt)
arch/arm64/kvm/regmap.c +2 -2
@@ -150,7 +150,7 @@ unsigned long vcpu_read_spsr32(const struct kvm_vcpu *vcpu)if (!vcpu->arch.sysregs_loaded_on_cpu) {switch (spsr_idx) {case KVM_SPSR_SVC : -return vcpu->arch.ctxt.spsr_el1; +return __vcpu_sys_reg (vcpu,SPSR_EL1 );case KVM_SPSR_ABT :return vcpu->arch.ctxt.spsr_abt;case KVM_SPSR_UND :
@@ -185,7 +185,7 @@ void vcpu_write_spsr32(struct kvm_vcpu *vcpu, unsigned long v)if (!vcpu->arch.sysregs_loaded_on_cpu) {switch (spsr_idx) {case KVM_SPSR_SVC : - vcpu->arch.ctxt.spsr_el1 = v; +__vcpu_sys_reg (vcpu,SPSR_EL1 ) = v;break ;case KVM_SPSR_ABT : vcpu->arch.ctxt.spsr_abt = v;
Signature · 1 line
2.27.0
Marc Zyngier
maintainerPATCH v3 16/17 KVM: arm64: timers: Rename kvm_timer_sync_hwstate to kvm_timer_sync_user
In reply to Marc Zyngierkvm_timer_sync_hwstate() has nothing to do with the timer HW state, but more to do with the state of a userspace interrupt controller. Change the suffix from _hwstate to_user, in keeping with the rest of the code.
--- arch/arm64/kvm/arch_timer.c | 2 +- arch/arm64/kvm/arm.c | 4 ++-- include/kvm/arm_arch_timer.h | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)
arch/arm64/kvm/arch_timer.c +1 -1
@@ -615,7 +615,7 @@ static void unmask_vtimer_irq_user(struct kvm_vcpu *vcpu) } } -void kvm_timer_sync_hwstate (struct kvm_vcpu *vcpu) +void kvm_timer_sync_user (struct kvm_vcpu *vcpu) {struct arch_timer_cpu *timer =vcpu_timer (vcpu);
arch/arm64/kvm/arm.c +2 -2
@@ -723,7 +723,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)isb ();/* Ensure work in x_flush_hwstate is committed */ kvm_pmu_sync_hwstate (vcpu);if (static_branch_unlikely (&userspace_irqchip_in_use)) -kvm_timer_sync_hwstate (vcpu); +kvm_timer_sync_user (vcpu);kvm_vgic_sync_hwstate (vcpu);local_irq_enable ();preempt_enable ();
@@ -768,7 +768,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)* timer virtual interrupt state. */ if (static_branch_unlikely (&userspace_irqchip_in_use)) -kvm_timer_sync_hwstate (vcpu); +kvm_timer_sync_user (vcpu);kvm_arch_vcpu_ctxsync_fp (vcpu);
include/kvm/arm_arch_timer.h +1 -1
@@ -71,7 +71,7 @@ int kvm_timer_hyp_init(bool);int kvm_timer_enable (struct kvm_vcpu *vcpu);int kvm_timer_vcpu_reset (struct kvm_vcpu *vcpu);void kvm_timer_vcpu_init (struct kvm_vcpu *vcpu); -void kvm_timer_sync_hwstate (struct kvm_vcpu *vcpu); +void kvm_timer_sync_user (struct kvm_vcpu *vcpu);bool kvm_timer_should_notify_user (struct kvm_vcpu *vcpu);void kvm_timer_update_run (struct kvm_vcpu *vcpu);void kvm_timer_vcpu_terminate (struct kvm_vcpu *vcpu);
Signature · 1 line
2.27.0
Re: [PATCH v3 16/17] KVM: arm64: timers: Rename kvm_timer_sync_hwstate to kvm_timer_sync_user
Hi Marc,
Quoted 52 lines · Marc Zyngier
kvm_timer_sync_hwstate()has nothing to do with the timer HW state, but more to do with the state of a userspace interrupt controller. Change the suffix from_hwstateto_user, in keeping with the rest of the code.Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/arch_timer.c | 2 +- arch/arm64/kvm/arm.c | 4 ++-- include/kvm/arm_arch_timer.h | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)arch/arm64/kvm/arch_timer.c +1
-1@@ -615,7 +615,7 @@ static void unmask_vtimer_irq_user(struct kvm_vcpu *vcpu) } } -void kvm_timer_sync_hwstate (struct kvm_vcpu *vcpu) +void kvm_timer_sync_user (struct kvm_vcpu *vcpu) {struct arch_timer_cpu *timer =vcpu_timer (vcpu);arch/arm64/kvm/arm.c +2
-2@@ -723,7 +723,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)isb ();/* Ensure work in x_flush_hwstate is committed */ kvm_pmu_sync_hwstate (vcpu);if (static_branch_unlikely (&userspace_irqchip_in_use)) -kvm_timer_sync_hwstate (vcpu); +kvm_timer_sync_user (vcpu);kvm_vgic_sync_hwstate (vcpu);local_irq_enable ();preempt_enable ();@@ -768,7 +768,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)* timer virtual interrupt state. */ if (static_branch_unlikely (&userspace_irqchip_in_use)) -kvm_timer_sync_hwstate (vcpu); +kvm_timer_sync_user (vcpu);kvm_arch_vcpu_ctxsync_fp (vcpu);include/kvm/arm_arch_timer.h +0
-0
include/kvm/arm_arch_timer.h int kvm_timer_hyp_init(bool); · line 77
Quoted 8 lines
include/kvm/arm_arch_timer.h +1
-1@@ -71,7 +71,7 @@ int kvm_timer_hyp_init(bool);int kvm_timer_enable (struct kvm_vcpu *vcpu);int kvm_timer_vcpu_reset (struct kvm_vcpu *vcpu);void kvm_timer_vcpu_init (struct kvm_vcpu *vcpu); -void kvm_timer_sync_hwstate (struct kvm_vcpu *vcpu); +void kvm_timer_sync_user (struct kvm_vcpu *vcpu);bool kvm_timer_should_notify_user (struct kvm_vcpu *vcpu);void kvm_timer_update_run (struct kvm_vcpu *vcpu);void kvm_timer_vcpu_terminate (struct kvm_vcpu *vcpu);
This patch makes sense to me. All throughout arm64 KVM we use hwstate for hardware registers, for both the vgic and the pmu. On top of that, the function calls unmask_vtimer_irq_*user*() (emphasis added):
Thanks,
Alex
Marc Zyngier
maintainerPATCH v3 17/17 KVM: arm64: timers: Move timer registers to the sys_regs file
In reply to Marc ZyngierMove the timer gsisters to the sysreg file. This will further help when they are directly changed by a nesting hypervisor in the VNCR page.
This requires moving the initialisation of the timer struct so that some of the helpers (such as arch_timer_ctx_index) can work correctly at an early stage.
--- arch/arm64/include/asm/kvm_host.h | 6 ++ arch/arm64/kvm/arch_timer.c | 155 +++++++++++++++++++++++------- arch/arm64/kvm/trace_arm.h | 8 +- include/kvm/arm_arch_timer.h | 11 +-- 4 files changed, 136 insertions(+), 44 deletions(-)
arch/arm64/include/asm/kvm_host.h +6 -0
@@ -189,6 +189,12 @@ enum vcpu_sysreg {SP_EL1 ,SPSR_EL1 , +CNTVOFF_EL2 , +CNTV_CVAL_EL0 , +CNTV_CTL_EL0 , +CNTP_CVAL_EL0 , +CNTP_CTL_EL0 , +/* 32bit specific registers. Keep them at the end of the range */ DACR32_EL2 ,/* Domain Access Control Register */ IFSR32_EL2, /* Instruction Fault Status Register */
arch/arm64/kvm/arch_timer.c +122 -33
272 lines not on this page · raw message
arch/arm64/kvm/trace_arm.h +4 -4
@@ -301,8 +301,8 @@ TRACE_EVENT(kvm_timer_save_state, ), TP_fast_assign( - __entry->ctl = ctx->cnt_ctl; - __entry->cval = ctx->cnt_cval; + __entry->ctl = timer_get_ctl(ctx); + __entry->cval =timer_get_cval (ctx); __entry->timer_idx =arch_timer_ctx_index (ctx); ),
@@ -323,8 +323,8 @@ TRACE_EVENT(kvm_timer_restore_state, ), TP_fast_assign( - __entry->ctl = ctx->cnt_ctl; - __entry->cval = ctx->cnt_cval; + __entry->ctl = timer_get_ctl(ctx); + __entry->cval =timer_get_cval (ctx); __entry->timer_idx =arch_timer_ctx_index (ctx); ),
include/kvm/arm_arch_timer.h +4 -7
@@ -26,16 +26,9 @@ enum kvm_arch_timer_regs {struct arch_timer_context {struct kvm_vcpu *vcpu; -/* Registers: control register, timer value */ -u32 cnt_ctl; -u64 cnt_cval; -/* Timer IRQ */ struct kvm_irq_level irq; -/* Virtual offset */ -u64 cntvoff; -/* Emulated Timer (may be unused) */ struct hrtimer hrtimer;
@@ -109,4 +102,8 @@ void kvm_arm_timer_write_sysreg(struct kvm_vcpu *vcpu,enum kvm_arch_timer_regs treg, u64 val); +/* Needed for tracing */ +u32 timer_get_ctl (struct arch_timer_context *ctxt); +u64 timer_get_cval (struct arch_timer_context *ctxt); +#endif
Signature · 1 line
2.27.0
Re: [PATCH v3 17/17] KVM: arm64: timers: Move timer registers to the sys_regs file
Hi Marc,
At first I was a bit apprehensive about adding yet another layer of indirection to the arch timer. But after spending some time trying to figure out if there's a better way to do it, I came to the conclusion that the patch is actually an improvement because it makes it obvious what timers we are emulating today, and the registers associated with each of them.
Quoted 1 line · Marc Zyngier
Move the timer gsisters to the sysreg file. This will further help when
s/gsisters/registers
Quoted 68 lines
they are directly changed by a nesting hypervisor in the VNCR page.
This requires moving the initialisation of the timer struct so that some of the helpers (such as
arch_timer_ctx_index) can work correctly at an early stage.Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/kvm_host.h | 6 ++ arch/arm64/kvm/arch_timer.c | 155 +++++++++++++++++++++++------- arch/arm64/kvm/trace_arm.h | 8 +- include/kvm/arm_arch_timer.h | 11 +-- 4 files changed, 136 insertions(+), 44 deletions(-)arch/arm64/include/asm/kvm_host.h +6
-0@@ -189,6 +189,12 @@ enum vcpu_sysreg {SP_EL1 ,SPSR_EL1 , +CNTVOFF_EL2 , +CNTV_CVAL_EL0 , +CNTV_CTL_EL0 , +CNTP_CVAL_EL0 , +CNTP_CTL_EL0 , +/* 32bit specific registers. Keep them at the end of the range */ DACR32_EL2 ,/* Domain Access Control Register */ IFSR32_EL2, /* Instruction Fault Status Register */arch/arm64/kvm/arch_timer.c +29
-0@@ -51,6 +51,93 @@ static u64 kvm_arm_timer_read(struct kvm_vcpu *vcpu,struct arch_timer_context *timer, enum kvm_arch_timer_regs treg); +u32 timer_get_ctl (struct arch_timer_context *ctxt) +{ +struct kvm_vcpu *vcpu = ctxt->vcpu; + +switch (arch_timer_ctx_index (ctxt)) { +case TIMER_VTIMER : +return __vcpu_sys_reg (vcpu,CNTV_CTL_EL0 ); +case TIMER_PTIMER : +return __vcpu_sys_reg (vcpu,CNTP_CTL_EL0 ); +default : +WARN_ON (1 ); +return 0 ; + } +} + +u64 timer_get_cval (struct arch_timer_context *ctxt) +{ +struct kvm_vcpu *vcpu = ctxt->vcpu; + +switch (arch_timer_ctx_index (ctxt)) { +case TIMER_VTIMER : +return __vcpu_sys_reg (vcpu,CNTV_CVAL_EL0 ); +case TIMER_PTIMER : +return __vcpu_sys_reg (vcpu,CNTP_CVAL_EL0 ); +default : +WARN_ON (1 ); +return 0 ; + } +}
arch/arm64/kvm/arch_timer.c static u64 kvm_arm_timer_read(struct kvm_vcpu *vcpu, · line 92
Quoted 10 lines
arch/arm64/kvm/arch_timer.c +10
-0+ +static u64 timer_get_offset (struct arch_timer_context *ctxt) +{ +struct kvm_vcpu *vcpu = ctxt->vcpu; + +switch (arch_timer_ctx_index (ctxt)) { +case TIMER_VTIMER : +return __vcpu_sys_reg (vcpu,CNTVOFF_EL2 ); +default : +return 0 ;
How about adding a TIMER_PTIMER case that returns 0, and adding a warning to the default case?
Quoted 35 lines
arch/arm64/kvm/arch_timer.c +35
-0+ } +} + +static void timer_set_ctl (struct arch_timer_context *ctxt,u32 ctl) +{ +struct kvm_vcpu *vcpu = ctxt->vcpu; + +switch (arch_timer_ctx_index (ctxt)) { +case TIMER_VTIMER : +__vcpu_sys_reg (vcpu,CNTV_CTL_EL0 ) = ctl; +break ; +case TIMER_PTIMER : +__vcpu_sys_reg (vcpu,CNTP_CTL_EL0 ) = ctl; +break ; +default : +WARN_ON (1 ); + } +} + +static void timer_set_cval (struct arch_timer_context *ctxt,u64 cval) +{ +struct kvm_vcpu *vcpu = ctxt->vcpu; + +switch (arch_timer_ctx_index (ctxt)) { +case TIMER_VTIMER : +__vcpu_sys_reg (vcpu,CNTV_CVAL_EL0 ) = cval; +break ; +case TIMER_PTIMER : +__vcpu_sys_reg (vcpu,CNTP_CVAL_EL0 ) = cval; +break ; +default : +WARN_ON (1 ); + } +} +
arch/arm64/kvm/arch_timer.c static u64 kvm_arm_timer_read(struct kvm_vcpu *vcpu, · line 137
Quoted 10 lines
arch/arm64/kvm/arch_timer.c +10
-0+static void timer_set_offset (struct arch_timer_context *ctxt,u64 offset) +{ +struct kvm_vcpu *vcpu = ctxt->vcpu; + +switch (arch_timer_ctx_index (ctxt)) { +case TIMER_VTIMER : +__vcpu_sys_reg (vcpu,CNTVOFF_EL2 ) = offset; +break ; +default : +WARN (offset,"timer %ld\n" ,arch_timer_ctx_index (ctxt));
Hmm.. looks to me like the warning isn't triggered when the offset is TIMER_PTIMER, which is 0.
A more general remark about the accessors. They only access the copy in memory, perhaps it's worth adding a WARN_ON(ctxt->loaded) to catch future errors.
I've checked the rest of the patch, and the changes look semantically equivalent to me.
Thanks,
Alex
Quoted 256 lines
arch/arm64/kvm/arch_timer.c +38
-33+ } +} +u64 kvm_phys_timer_read (void ) {return timecounter->cc->read (timecounter->cc);@@ -124,8 +211,8 @@ static u64 kvm_timer_compute_delta(struct arch_timer_context *timer_ctx) {u64 cval, now; - cval = timer_ctx->cnt_cval; - now =kvm_phys_timer_read () - timer_ctx->cntvoff; + cval =timer_get_cval (timer_ctx); + now =kvm_phys_timer_read () -timer_get_offset (timer_ctx);if (now < cval) {u64 ns;@@ -144,8 +231,8 @@ static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx) {WARN_ON (timer_ctx && timer_ctx->loaded);return timer_ctx && - !(timer_ctx->cnt_ctl &ARCH_TIMER_CTRL_IT_MASK ) && - (timer_ctx->cnt_ctl &ARCH_TIMER_CTRL_ENABLE ); + ((timer_get_ctl (timer_ctx) & + (ARCH_TIMER_CTRL_IT_MASK |ARCH_TIMER_CTRL_ENABLE )) ==ARCH_TIMER_CTRL_ENABLE ); }/* @@ -256,8 +343,8 @@ static bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)if (!kvm_timer_irq_can_fire (timer_ctx))return false; - cval = timer_ctx->cnt_cval; - now =kvm_phys_timer_read () - timer_ctx->cntvoff; + cval =timer_get_cval (timer_ctx); + now =kvm_phys_timer_read () -timer_get_offset (timer_ctx);return cval <= now; }@@ -350,8 +437,8 @@ static void timer_save_state(struct arch_timer_context *ctx)switch (index) {case TIMER_VTIMER : - ctx->cnt_ctl =read_sysreg_el0 (SYS_CNTV_CTL ); - ctx->cnt_cval =read_sysreg_el0 (SYS_CNTV_CVAL ); +timer_set_ctl (ctx,read_sysreg_el0 (SYS_CNTV_CTL )); +timer_set_cval (ctx,read_sysreg_el0 (SYS_CNTV_CVAL ));/* Disable the timer */ write_sysreg_el0 (0 ,SYS_CNTV_CTL );@@ -359,8 +446,8 @@ static void timer_save_state(struct arch_timer_context *ctx)break ;case TIMER_PTIMER : - ctx->cnt_ctl =read_sysreg_el0 (SYS_CNTP_CTL ); - ctx->cnt_cval =read_sysreg_el0 (SYS_CNTP_CVAL ); +timer_set_ctl (ctx,read_sysreg_el0 (SYS_CNTP_CTL )); +timer_set_cval (ctx,read_sysreg_el0 (SYS_CNTP_CVAL ));/* Disable the timer */ write_sysreg_el0 (0 ,SYS_CNTP_CTL );@@ -429,14 +516,14 @@ static void timer_restore_state(struct arch_timer_context *ctx)switch (index) {case TIMER_VTIMER : -write_sysreg_el0 (ctx->cnt_cval,SYS_CNTV_CVAL ); +write_sysreg_el0 (timer_get_cval (ctx),SYS_CNTV_CVAL );isb (); -write_sysreg_el0 (ctx->cnt_ctl,SYS_CNTV_CTL ); +write_sysreg_el0 (timer_get_ctl (ctx),SYS_CNTV_CTL );break ;case TIMER_PTIMER : -write_sysreg_el0 (ctx->cnt_cval,SYS_CNTP_CVAL ); +write_sysreg_el0 (timer_get_cval (ctx),SYS_CNTP_CVAL );isb (); -write_sysreg_el0 (ctx->cnt_ctl,SYS_CNTP_CTL ); +write_sysreg_el0 (timer_get_ctl (ctx),SYS_CNTP_CTL );break ;case NR_KVM_TIMERS :BUG ();@@ -528,7 +615,7 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)kvm_timer_vcpu_load_nogic (vcpu); } -set_cntvoff (map.direct_vtimer->cntvoff); +set_cntvoff (timer_get_offset (map.direct_vtimer));kvm_timer_unblocking (vcpu);@@ -639,8 +726,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)* resets the timer to be disabled and unmasked and is compliant with * the ARMv7 architecture. */ -vcpu_vtimer (vcpu)->cnt_ctl =0 ; -vcpu_ptimer (vcpu)->cnt_ctl =0 ; +timer_set_ctl (vcpu_vtimer (vcpu),0 ); +timer_set_ctl (vcpu_ptimer (vcpu),0 );if (timer->enabled) {kvm_timer_update_irq (vcpu, false,vcpu_vtimer (vcpu));@@ -668,13 +755,13 @@ static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)mutex_lock (&kvm->lock);kvm_for_each_vcpu (i, tmp, kvm) -vcpu_vtimer (tmp)->cntvoff = cntvoff; +timer_set_offset (vcpu_vtimer (tmp), cntvoff);/* * When called from the vcpu create path, the CPU being created is not * included in the loop above, so we just set it here as well. */ -vcpu_vtimer (vcpu)->cntvoff = cntvoff; +timer_set_offset (vcpu_vtimer (vcpu), cntvoff);mutex_unlock (&kvm->lock); }@@ -684,9 +771,12 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)struct arch_timer_context *vtimer =vcpu_vtimer (vcpu);struct arch_timer_context *ptimer =vcpu_ptimer (vcpu); + vtimer->vcpu = vcpu; + ptimer->vcpu = vcpu; +/* Synchronize cntvoff across all vtimers of a VM. */ update_vtimer_cntvoff (vcpu,kvm_phys_timer_read ()); - ptimer->cntvoff =0 ; +timer_set_offset (ptimer,0 );hrtimer_init (&timer->bg_timer,CLOCK_MONOTONIC ,HRTIMER_MODE_ABS_HARD ); timer->bg_timer.function = kvm_bg_timer_expire;@@ -704,9 +794,6 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu) vtimer->host_timer_irq_flags = host_vtimer_irq_flags; ptimer->host_timer_irq_flags = host_ptimer_irq_flags; - - vtimer->vcpu = vcpu; - ptimer->vcpu = vcpu; }static void kvm_timer_init_interrupt (void *info)@@ -756,10 +843,12 @@ static u64 read_timer_ctl(struct arch_timer_context *timer)* UNKNOWN when ENABLE bit is 0, so we chose to set ISTATUS bit * regardless of ENABLE bit for our implementation convenience. */ +u32 ctl =timer_get_ctl (timer); +if (!kvm_timer_compute_delta (timer)) -return timer->cnt_ctl |ARCH_TIMER_CTRL_IT_STAT ; -else -return timer->cnt_ctl; + ctl |=ARCH_TIMER_CTRL_IT_STAT ; + +return ctl; }u64 kvm_arm_timer_get_reg (struct kvm_vcpu *vcpu,u64 regid)@@ -795,8 +884,8 @@ static u64 kvm_arm_timer_read(struct kvm_vcpu *vcpu,switch (treg) {case TIMER_REG_TVAL : - val = timer->cnt_cval -kvm_phys_timer_read () + timer->cntvoff; - val &=lower_32_bits (val); + val =timer_get_cval (timer) -kvm_phys_timer_read () +timer_get_offset (timer); + val =lower_32_bits (val);break ;case TIMER_REG_CTL :@@ -804,11 +893,11 @@ static u64 kvm_arm_timer_read(struct kvm_vcpu *vcpu,break ;case TIMER_REG_CVAL : - val = timer->cnt_cval; + val =timer_get_cval (timer);break ;case TIMER_REG_CNT : - val =kvm_phys_timer_read () - timer->cntvoff; + val =kvm_phys_timer_read () -timer_get_offset (timer);break ;default :@@ -842,15 +931,15 @@ static void kvm_arm_timer_write(struct kvm_vcpu *vcpu, {switch (treg) {case TIMER_REG_TVAL : - timer->cnt_cval =kvm_phys_timer_read () - timer->cntvoff + (s32 )val; +timer_set_cval (timer,kvm_phys_timer_read () -timer_get_offset (timer) + (s32 )val);break ;case TIMER_REG_CTL : - timer->cnt_ctl = val & ~ARCH_TIMER_CTRL_IT_STAT ; +timer_set_ctl (timer, val & ~ARCH_TIMER_CTRL_IT_STAT );break ;case TIMER_REG_CVAL : - timer->cnt_cval = val; +timer_set_cval (timer, val);break ;default :arch/arm64/kvm/trace_arm.h +4
-4@@ -301,8 +301,8 @@ TRACE_EVENT(kvm_timer_save_state, ), TP_fast_assign( - __entry->ctl = ctx->cnt_ctl; - __entry->cval = ctx->cnt_cval; + __entry->ctl = timer_get_ctl(ctx); + __entry->cval =timer_get_cval (ctx); __entry->timer_idx =arch_timer_ctx_index (ctx); ),@@ -323,8 +323,8 @@ TRACE_EVENT(kvm_timer_restore_state, ), TP_fast_assign( - __entry->ctl = ctx->cnt_ctl; - __entry->cval = ctx->cnt_cval; + __entry->ctl = timer_get_ctl(ctx); + __entry->cval =timer_get_cval (ctx); __entry->timer_idx =arch_timer_ctx_index (ctx); ),include/kvm/arm_arch_timer.h +4
-7@@ -26,16 +26,9 @@ enum kvm_arch_timer_regs {struct arch_timer_context {struct kvm_vcpu *vcpu; -/* Registers: control register, timer value */ -u32 cnt_ctl; -u64 cnt_cval; -/* Timer IRQ */ struct kvm_irq_level irq; -/* Virtual offset */ -u64 cntvoff; -/* Emulated Timer (may be unused) */ struct hrtimer hrtimer;@@ -109,4 +102,8 @@ void kvm_arm_timer_write_sysreg(struct kvm_vcpu *vcpu,enum kvm_arch_timer_regs treg, u64 val); +/* Needed for tracing */ +u32 timer_get_ctl (struct arch_timer_context *ctxt); +u64 timer_get_cval (struct arch_timer_context *ctxt); +#endif
Catalin Marinas
maintainerQuoted 3 lines · Marc Zyngier
Catalin: How do you want to proceed for patches 2, 3, and 4? I could make a stable branch that gets you pull into the arm64 tree, or the other way around. Just let me know.
Please create a separate branch for the S2 TTL patches (ideally based on no later than -rc3). I plan to queue the rest of Zhenyu's patches on top.
https://lore.kernel.org/linux-arm-kernel/[email protected]/
Thanks.
Signature · 1 line
Catalin
Marc Zyngier
maintainerHi Catalin,
Quoted 11 lines · Catalin Marinas
Quoted 3 lines · Marc Zyngier
Catalin: How do you want to proceed for patches 2, 3, and 4? I could make a stable branch that gets you pull into the arm64 tree, or the other way around. Just let me know.
Please create a separate branch for the S2 TTL patches (ideally based on no later than -rc3). I plan to queue the rest of Zhenyu's patches on top.
I've now pushed out this branch[1], containing the three patches in isolation. Unless you tell me otherwise, I will push this into -next today, together with the rest of the KVM/arm64 queue.
Thanks,
M.
[1] git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git kvm-arm64/ttl-for-arm64
Signature · 1 line
Without deviation from the norm, progress is not possible.
Hi,
Quoted 44 lines · Marc Zyngier
Hi all,
In order not to repeat the 90+ patch series that resulted in a deafening silence last time, I've extracted a smaller set of patches that form the required dependencies that allow the rest of the 65 NV patches to be added on top. Yes, it is that bad.
The one real feature here is support for the ARMv8.4-TTL extension at Stage-2 only. The reason to support it is that it helps the hypervisor a lot when it comes to finding out how much to invalidate. It is thus always "supported" with NV.
The rest doesn't contain any functionality change. Most of it reworks existing data structures and adds new accessors for the things that get moved around. The reason for this is that:
- With NV, we end-up with multiple Stage-2 MMU contexts per VM instead of a single one. This requires we divorce struct kvm from the S2 MMU configuration. Of course, we stick with a single MMU context for now.
- With ARMv8.4-NV, a number of system register accesses are turned into memory accesses into the so-called VNCR page. It is thus convenient to make this VNCR page part of the vcpu context and avoid copying data back and forth. For this to work, we need to make sure that all the VNCR-aware sysregs are moved into our per-vcpu
sys_regsarray instead of leaving in other data structures (the timers, for example). The VNCR page itself isn't introduced with these patches.- As some of these data structures change, we need a way to isolate the userspace ABI from such change.
There is also a number of cleanups that were in the full fat series that I decided to move early to get them out of the way.
The whole this is a bit of a mix of vaguely unrelated "stuff", but it all comes together if you look at the final series. This applies on top of David Brazdil's series splitting the VHE and nVHE objects.
I plan on taking this early into v5.9, and I really mean it this time!
Catalin: How do you want to proceed for patches 2, 3, and 4? I could make a stable branch that gets you pull into the arm64 tree, or the other way around. Just let me know.
Quoted 6 lines
Thanks,
M.* From v2:
- Rebased on top of David's el2-obj series
I tried to apply the patches on top of v5.8-rc1, but I get a conflict on the very first patch. I guess it's because I don't have the el2-obj series. Is that v4 of "Split off nVHE hyp code" patches from David Brazil?
Thanks,
Alex
Hi,
Quoted 49 lines · Alexandru Elisei
Hi,
Quoted 46 lines · Marc Zyngier
Hi all,
In order not to repeat the 90+ patch series that resulted in a deafening silence last time, I've extracted a smaller set of patches that form the required dependencies that allow the rest of the 65 NV patches to be added on top. Yes, it is that bad.
The one real feature here is support for the ARMv8.4-TTL extension at Stage-2 only. The reason to support it is that it helps the hypervisor a lot when it comes to finding out how much to invalidate. It is thus always "supported" with NV.
The rest doesn't contain any functionality change. Most of it reworks existing data structures and adds new accessors for the things that get moved around. The reason for this is that:
- With NV, we end-up with multiple Stage-2 MMU contexts per VM instead of a single one. This requires we divorce struct kvm from the S2 MMU configuration. Of course, we stick with a single MMU context for now.
- With ARMv8.4-NV, a number of system register accesses are turned into memory accesses into the so-called VNCR page. It is thus convenient to make this VNCR page part of the vcpu context and avoid copying data back and forth. For this to work, we need to make sure that all the VNCR-aware sysregs are moved into our per-vcpu
sys_regsarray instead of leaving in other data structures (the timers, for example). The VNCR page itself isn't introduced with these patches.- As some of these data structures change, we need a way to isolate the userspace ABI from such change.
There is also a number of cleanups that were in the full fat series that I decided to move early to get them out of the way.
The whole this is a bit of a mix of vaguely unrelated "stuff", but it all comes together if you look at the final series. This applies on top of David Brazdil's series splitting the VHE and nVHE objects.
I plan on taking this early into v5.9, and I really mean it this time!
Catalin: How do you want to proceed for patches 2, 3, and 4? I could make a stable branch that gets you pull into the arm64 tree, or the other way around. Just let me know.
Thanks,
Quoted 7 lines
Quoted 4 lines
M.* From v2:
- Rebased on top of David's el2-obj seriesI tried to apply the patches on top of v5.8-rc1, but I get a conflict on the very first patch. I guess it's because I don't have the el2-obj series. Is that v4 of "Split off nVHE hyp code" patches from David Brazil?
Nevermind, figured it out and used your kvm-arm64/el2-obj-4.1 branch as a base.
Thanks,
Alex
Marc Zyngier
maintainerQuoted 52 lines · Alexandru Elisei
Hi,
Quoted 49 lines · Marc Zyngier
Hi all,
In order not to repeat the 90+ patch series that resulted in a deafening silence last time, I've extracted a smaller set of patches that form the required dependencies that allow the rest of the 65 NV patches to be added on top. Yes, it is that bad.
The one real feature here is support for the ARMv8.4-TTL extension at Stage-2 only. The reason to support it is that it helps the hypervisor a lot when it comes to finding out how much to invalidate. It is thus always "supported" with NV.
The rest doesn't contain any functionality change. Most of it reworks existing data structures and adds new accessors for the things that get moved around. The reason for this is that:
- With NV, we end-up with multiple Stage-2 MMU contexts per VM instead of a single one. This requires we divorce struct kvm from the S2 MMU configuration. Of course, we stick with a single MMU context for
now.- With ARMv8.4-NV, a number of system register accesses are turned into memory accesses into the so-called VNCR page. It is thus convenient to make this VNCR page part of the vcpu context and avoid copying data back and forth. For this to work, we need to make sure that all the VNCR-aware sysregs are moved into our per-vcpu
sys_regsarray instead of leaving in other data structures (the timers, for example). The VNCR page itself isn't introduced with these patches.- As some of these data structures change, we need a way to isolate the userspace ABI from such change.
There is also a number of cleanups that were in the full fat series that I decided to move early to get them out of the way.
The whole this is a bit of a mix of vaguely unrelated "stuff", but it all comes together if you look at the final series. This applies on top of David Brazdil's series splitting the VHE and nVHE objects.
I plan on taking this early into v5.9, and I really mean it this time!
Catalin: How do you want to proceed for patches 2, 3, and 4? I could make a stable branch that gets you pull into the arm64 tree, or the other way around. Just let me know.
Thanks,
M.
Quoted 7 lines
Quoted 2 lines
* From v2:
- Rebased on top of David's el2-obj seriesI tried to apply the patches on top of v5.8-rc1, but I get a conflict on the very
first patch. I guess it's because I don't have the el2-obj series. Is that v4 of "Split off nVHE hyp code" patches from David Brazil?
You need the slightly amended version (kvm-arm6/el2-obj-v4.1 from my tree). Otherwise, just pick kvmarm/next, which has everything put together in one scary lot.
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...