KVM: arm64: nv: Shadow stage-2 page table handling
Marc Zyngier
maintainerv3 cover KVM: arm64: nv: Shadow stage-2 page table handling
Here's the thurd version of the shadow stage-2 handling for NV support on arm64.
* From v2 [2]
- Simplified the S2 walker by dropping a bunch of redundant fields from the walker info structure
- Added some more lockdep assertions (Oliver)
- Added more precise comments for the TTL-like annotations in the shadow S2 (Oliver)
* From v1 [1]
- Reworked the allocation of shadow S2 structures at init time to be slightly clearer
- Lots of small cleanups
- Rebased on v6.10-rc1
[1] https://lore.kernel.org/r/[email protected]
Christoffer Dall (2):
KVM: arm64: nv: Implement nested Stage-2 page table walk logic
KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
Marc Zyngier (14):
KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
KVM: arm64: nv: Handle shadow stage 2 page faults
KVM: arm64: nv: Add Stage-1 EL2 invalidation primitives
KVM: arm64: nv: Handle EL2 Stage-1 TLB invalidation
KVM: arm64: nv: Handle TLB invalidation targeting L2 stage-1
KVM: arm64: nv: Handle TLBI VMALLS12E1{,IS} operations
KVM: arm64: nv: Handle TLBI ALLE1{,IS} operations
KVM: arm64: nv: Handle TLBI IPAS2E1{,IS} operations
KVM: arm64: nv: Handle FEAT_TTL hinted TLB operations
KVM: arm64: nv: Tag shadow S2 entries with guest's leaf S2 level
KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
information
KVM: arm64: nv: Add handling of outer-shareable TLBI operations
KVM: arm64: nv: Add handling of range-based TLBI operations
KVM: arm64: nv: Add handling of NXS-flavoured TLBI operations
arch/arm64/include/asm/esr.h | 1 + arch/arm64/include/asm/kvm_asm.h | 2 + arch/arm64/include/asm/kvm_host.h | 36 ++ arch/arm64/include/asm/kvm_mmu.h | 26 + arch/arm64/include/asm/kvm_nested.h | 127 +++++ arch/arm64/include/asm/sysreg.h | 17 + arch/arm64/kvm/arm.c | 11 + arch/arm64/kvm/hyp/vhe/switch.c | 51 +- arch/arm64/kvm/hyp/vhe/tlb.c | 147 ++++++ arch/arm64/kvm/mmu.c | 213 ++++++-- arch/arm64/kvm/nested.c | 781 +++++++++++++++++++++++++++- arch/arm64/kvm/reset.c | 6 + arch/arm64/kvm/sys_regs.c | 398 ++++++++++++++ 13 files changed, 1775 insertions(+), 41 deletions(-)
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 1/16 KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
Add Stage-2 mmu data structures for virtual EL2 and for nested guests. We don't yet populate shadow Stage-2 page tables, but we now have a framework for getting to a shadow Stage-2 pgd.
We allocate twice the number of vcpus as Stage-2 mmu structures because that's sufficient for each vcpu running two translation regimes without having to flush the Stage-2 page tables.
--- arch/arm64/include/asm/kvm_host.h | 36 +++++ arch/arm64/include/asm/kvm_mmu.h | 24 +++ arch/arm64/include/asm/kvm_nested.h | 6 + arch/arm64/kvm/arm.c | 11 ++ arch/arm64/kvm/mmu.c | 69 ++++++--- arch/arm64/kvm/nested.c | 218 ++++++++++++++++++++++++++++ arch/arm64/kvm/reset.c | 6 + 7 files changed, 349 insertions(+), 21 deletions(-)
arch/arm64/include/asm/kvm_host.h +36 -0
@@ -188,6 +188,33 @@ struct kvm_s2_mmu {uint64_t split_page_chunk_size;struct kvm_arch *arch; + +/* +* For a shadow stage-2 MMU, the virtual vttbr used by the +* host to parse the guest S2. +* This either contains: +* - the virtual VTTBR programmed by the guest hypervisor with +* CnP cleared +* - The value 1 (VMID=0, BADDR=0, CnP=1) if invalid +* +* We also cache the full VTCR which gets used for TLB invalidation, +* taking the ARM ARM's "Any of the bits in VTCR_EL2 are permitted +* to be cached in a TLB" to the letter. +*/ +u64 tlb_vttbr; +u64 tlb_vtcr; + +/* +* true when this represents a nested context where virtual +* HCR_EL2.VM == 1 +*/ +bool nested_stage2_enabled; + +/* +* 0: Nobody is currently using this, check vttbr for validity +* >0: Somebody is actively using this. +*/ +atomic_t refcnt; };struct kvm_arch_memory_slot {
@@ -255,6 +282,14 @@ struct kvm_arch {*/ u64 fgu[__NR_FGT_GROUP_IDS__]; +/* +* Stage 2 paging state for VMs with nested S2 using a virtual +* VMID. +*/ +struct kvm_s2_mmu *nested_mmus; +size_t nested_mmus_size; +int nested_mmus_next; +/* Interrupt controller */ struct vgic_dist vgic;
@@ -1283,6 +1318,7 @@ void kvm_vcpu_load_vhe(struct kvm_vcpu *vcpu);void kvm_vcpu_put_vhe (struct kvm_vcpu *vcpu);int __init kvm_set_ipa_limit (void ); +u32 kvm_get_pa_bits (struct kvm *kvm);#define __KVM_HAVE_ARCH_VM_ALLOCstruct kvm *kvm_arch_alloc_vm (void );
arch/arm64/include/asm/kvm_mmu.h +24 -0
@@ -98,6 +98,7 @@ alternative_cb_end#include <asm/mmu_context.h> #include <asm/kvm_emulate.h> #include <asm/kvm_host.h> +#include <asm/kvm_nested.h> void kvm_update_va_mask (struct alt_instr *alt,__le32 *origptr,__le32 *updptr,int nr_inst);
@@ -165,6 +166,8 @@ int create_hyp_exec_mappings(phys_addr_t phys_addr, size_t size,int create_hyp_stack (phys_addr_t phys_addr,unsigned long *haddr);void __init free_hyp_pgds (void ); +void kvm_stage2_unmap_range (struct kvm_s2_mmu *mmu,phys_addr_t start,u64 size); +void stage2_unmap_vm (struct kvm *kvm);int kvm_init_stage2_mmu (struct kvm *kvm,struct kvm_s2_mmu *mmu,unsigned long type);void kvm_uninit_stage2_mmu (struct kvm *kvm);
@@ -326,5 +329,26 @@ static inline struct kvm *kvm_s2_mmu_to_kvm(struct kvm_s2_mmu *mmu) { return container_of(mmu->arch, struct kvm, arch); } + +static inline u64 get_vmid (u64 vttbr) +{ +return (vttbr &VTTBR_VMID_MASK (kvm_get_vmid_bits ())) >> +VTTBR_VMID_SHIFT ; +} + +static inline bool kvm_s2_mmu_valid (struct kvm_s2_mmu *mmu) +{ +return !(mmu->tlb_vttbr &VTTBR_CNP_BIT ); +} + +static inline bool kvm_is_nested_s2_mmu (struct kvm *kvm,struct kvm_s2_mmu *mmu) +{ +/* +* Be careful, mmu may not be fully initialised so do look at +* *any* of its fields. +*/ +return &kvm->arch.mmu != mmu; +} +#endif /* __ASSEMBLY__ */ #endif /* __ARM64_KVM_MMU_H__ */
arch/arm64/include/asm/kvm_nested.h +6 -0
@@ -61,6 +61,12 @@ static inline u64 translate_ttbr0_el2_to_ttbr0_el1(u64 ttbr0) }extern bool forward_smc_trap (struct kvm_vcpu *vcpu); +extern void kvm_init_nested (struct kvm *kvm); +extern int kvm_vcpu_init_nested (struct kvm_vcpu *vcpu); +extern void kvm_init_nested_s2_mmu (struct kvm_s2_mmu *mmu); +extern struct kvm_s2_mmu *lookup_s2_mmu (struct kvm_vcpu *vcpu); +extern void kvm_vcpu_load_hw_mmu (struct kvm_vcpu *vcpu); +extern void kvm_vcpu_put_hw_mmu (struct kvm_vcpu *vcpu);int kvm_init_nv_sysregs (struct kvm *kvm);
arch/arm64/kvm/arm.c +11 -0
@@ -170,6 +170,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)mutex_unlock (&kvm->lock);#endif +kvm_init_nested (kvm); + ret =kvm_share_hyp (kvm, kvm +1 );if (ret)return ret;
@@ -551,6 +553,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)struct kvm_s2_mmu *mmu;int *last_ran; +if (vcpu_has_nv (vcpu)) +kvm_vcpu_load_hw_mmu (vcpu); + mmu = vcpu->arch.hw_mmu; last_ran =this_cpu_ptr (mmu->last_vcpu_ran);
@@ -601,6 +606,8 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)kvm_timer_vcpu_put (vcpu);kvm_vgic_put (vcpu);kvm_vcpu_pmu_restore_host (vcpu); +if (vcpu_has_nv (vcpu)) +kvm_vcpu_put_hw_mmu (vcpu);kvm_arm_vmid_clear_active ();vcpu_clear_on_unsupported_cpu (vcpu);
@@ -1459,6 +1466,10 @@ static int kvm_setup_vcpu(struct kvm_vcpu *vcpu)if (kvm_vcpu_has_pmu (vcpu) && !kvm->arch.arm_pmu) ret =kvm_arm_set_default_pmu (kvm); +/* Prepare for nested if required */ +if (!ret &&vcpu_has_nv (vcpu)) + ret =kvm_vcpu_init_nested (vcpu); +return ret; }
arch/arm64/kvm/mmu.c +48 -21
118 lines not on this page · raw message
arch/arm64/kvm/nested.c +218 -0
231 lines not on this page · raw message
arch/arm64/kvm/reset.c +6 -0
@@ -265,6 +265,12 @@ void kvm_reset_vcpu(struct kvm_vcpu *vcpu)preempt_enable (); } +u32 kvm_get_pa_bits (struct kvm *kvm) +{ +/* Fixed limit until we can configure ID_AA64MMFR0.PARange */ +return kvm_ipa_limit; +} +u32 get_kvm_ipa_limit (void ) {return kvm_ipa_limit;
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 2/16 KVM: arm64: nv: Implement nested Stage-2 page table walk logic
In reply to Marc ZyngierFrom: Christoffer Dall <[email protected]>
Based on the pseudo-code in the ARM ARM, implement a stage 2 software page table walker.
--- arch/arm64/include/asm/esr.h | 1 + arch/arm64/include/asm/kvm_nested.h | 13 ++ arch/arm64/kvm/nested.c | 264 ++++++++++++++++++++++++++++ 3 files changed, 278 insertions(+)
arch/arm64/include/asm/esr.h +1 -0
@@ -152,6 +152,7 @@#define ESR_ELx_Xs_MASK (GENMASK_ULL(4, 0))/* ISS field definitions for exceptions taken in to Hyp */ +#define ESR_ELx_FSC_ADDRSZ (0x00)#define ESR_ELx_CV (UL(1) << 24)#define ESR_ELx_COND_SHIFT (20)#define ESR_ELx_COND_MASK (UL(0xF) << ESR_ELx_COND_SHIFT)
arch/arm64/include/asm/kvm_nested.h +13 -0
@@ -68,6 +68,19 @@ extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm_vcpu *vcpu);extern void kvm_vcpu_load_hw_mmu (struct kvm_vcpu *vcpu);extern void kvm_vcpu_put_hw_mmu (struct kvm_vcpu *vcpu); +struct kvm_s2_trans { +phys_addr_t output; +unsigned long block_size; +bool writable; +bool readable; +int level; +u32 esr; +u64 upper_attr; +}; + +extern int kvm_walk_nested_s2 (struct kvm_vcpu *vcpu,phys_addr_t gipa, +struct kvm_s2_trans *result); +int kvm_init_nv_sysregs (struct kvm *kvm);#ifdef CONFIG_ARM64_PTR_AUTH
arch/arm64/kvm/nested.c +264 -0
270 lines not on this page · raw message
Signature · 1 line
2.39.2
Re: [PATCH v3 02/16] KVM: arm64: nv: Implement nested Stage-2 page table walk logic
Hi Marc,
[ chewing through the NV code.. ;-) ]
Quoted 4 lines · Marc Zyngier
From: Christoffer Dall <[email protected]>
Based on the pseudo-code in the ARM ARM, implement a stage 2 software page table walker.
[...]
Quoted 11 lines
arch/arm64/kvm/nested.c +11
-0+static u32 compute_fsc (int level,u32 fsc) +{ +return fsc | (level &0x3 ); +} + +static int get_ia_size (struct s2_walk_info *wi) +{ +return 64 - wi->t0sz; +} + +static int check_base_s2_limits(struct s2_walk_info *wi,
arch/arm64/kvm/nested.c int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu) · line 139
Quoted 10 lines
arch/arm64/kvm/nested.c +10
-0+ int level, int input_size, int stride) +{ + int start_size, ia_size; + + ia_size =get_ia_size (wi); + +/* Check translation limits */ +switch (BIT (wi->pgshift)) { +case SZ_64K : +if (level ==0 || (level ==1 && ia_size <=42 ))
It looks broken as the pseudocode checks the limits based on *implemented PA size*, rather than on ia_size, which is essentially the input address size (64 - T0SZ) programmed by L1 hypervisor. They're different.
We can probably get the implemented PA size by:
AArch64.PAMax()
{
parange = get_idreg_field_enum(kvm, ID_AA64MMFR0_EL1, PARANGE);
return id_aa64mmfr0_parange_to_phys_shift(parange);
}Not sure if I've read the spec correctly.
Quoted 4 lines
arch/arm64/kvm/nested.c +4
-0+return -EFAULT ; +break ; +case SZ_16K : +if (level ==0 || (level ==1 && ia_size <=40 ))
arch/arm64/kvm/nested.c int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu) · line 153
Quoted 10 lines
arch/arm64/kvm/nested.c +10
-0+return -EFAULT ; +break ; +case SZ_4K : +if (level <0 || (level ==0 && ia_size <=42 )) +return -EFAULT ; +break ; + } + +/* Check input size limits */ +if (input_size > ia_size)
This is always false for the current code. ;-)
Quoted 56 lines
arch/arm64/kvm/nested.c +56
-0+return -EFAULT ; + +/* Check number of entries in starting level table */ + start_size = input_size - ((3 - level) * stride + wi->pgshift); +if (start_size <1 || start_size > stride +4 ) +return -EFAULT ; + +return 0 ; +} + +/* Check if output is within boundaries */ +static int check_output_size (struct s2_walk_info *wi,phys_addr_t output) +{ +unsigned int output_size = wi->max_oa_bits; + +if (output_size !=48 && (output &GENMASK_ULL (47 , output_size))) +return -1 ; + +return 0 ; +} + +/* +* This is essentially a C-version of the pseudo code from the ARM ARM +* AArch64.TranslationTableWalk function. I strongly recommend looking at +* that pseudocode in trying to understand this. +* +* Must be called with the kvm->srcu read lock held +*/ +static int walk_nested_s2_pgd (phys_addr_t ipa, +struct s2_walk_info *wi,struct kvm_s2_trans *out) +{ +int first_block_level, level, stride, input_size, base_lower_bound; +phys_addr_t base_addr; +unsigned int addr_top, addr_bottom; +u64 desc;/* page table entry */ +int ret; +phys_addr_t paddr; + +switch (BIT (wi->pgshift)) { +default : +case SZ_64K : +case SZ_16K : + level =3 - wi->sl; + first_block_level =2 ; +break ; +case SZ_4K : + level =2 - wi->sl; + first_block_level =1 ; +break ; + } + + stride = wi->pgshift -3 ; + input_size =get_ia_size (wi); +if (input_size >48 || input_size <25 ) +return -EFAULT ; +
arch/arm64/kvm/nested.c int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu) · line 219
Quoted 10 lines
arch/arm64/kvm/nested.c +10
-0+ ret =check_base_s2_limits (wi, level, input_size, stride); +if (WARN_ON (ret)) +return ret; + + base_lower_bound =3 + input_size - ((3 - level) * stride + + wi->pgshift); + base_addr = wi->baddr &GENMASK_ULL (47 , base_lower_bound); + +if (check_output_size (wi, base_addr)) { + out->esr =compute_fsc (level, ESR_ELx_FSC_ADDRSZ);
With a wrongly programmed base address, we should report the ADDRSZ fault at level 0 (as per R_BFHQH and the pseudocode). It's easy to fix.
Quoted 7 lines
arch/arm64/kvm/nested.c +7
-0+static void vtcr_to_walk_info (u64 vtcr,struct s2_walk_info *wi) +{ + wi->t0sz = vtcr &TCR_EL2_T0SZ_MASK ; + +switch (vtcr &VTCR_EL2_TG0_MASK ) { +case VTCR_EL2_TG0_4K : + wi->pgshift =12 ;break ;
arch/arm64/kvm/nested.c int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu) · line 327
Quoted 10 lines
arch/arm64/kvm/nested.c +10
-0+case VTCR_EL2_TG0_16K : + wi->pgshift =14 ;break ; +case VTCR_EL2_TG0_64K : +default :/* IMPDEF: treat any other value as 64k */ + wi->pgshift =16 ;break ; + } + + wi->sl =FIELD_GET (VTCR_EL2_SL0_MASK , vtcr); +/* Global limit for now, should eventually be per-VM */ + wi->max_oa_bits = min(get_kvm_ipa_limit(),
^^^
Should we use AArch64.PAMax() instead? As the output address size is never larger than the implemented PA size.
Now I'm wondering if we can let kvm_get_pa_bits() just return PAMax for (based on the exposed (to-L1) AA64MFR0.PARange value) and use it when possible.
Thanks,
Zenghui
Marc Zyngier
maintainerRe: [PATCH v3 02/16] KVM: arm64: nv: Implement nested Stage-2 page table walk logic
Quoted 4 lines · Zenghui Yu
Hi Marc,
[ chewing through the NV code.. ;-) ]
You fool! :)
Quoted 38 lines · Marc Zyngier
Quoted 4 lines
From: Christoffer Dall <[email protected]>
Based on the pseudo-code in the ARM ARM, implement a stage 2 software page table walker.
[...]
Quoted 11 lines
arch/arm64/kvm/nested.c +11
-0+static u32 compute_fsc (int level,u32 fsc) +{ +return fsc | (level &0x3 ); +} + +static int get_ia_size (struct s2_walk_info *wi) +{ +return 64 - wi->t0sz; +} + +static int check_base_s2_limits(struct s2_walk_info *wi,arch/arm64/kvm/nested.c int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu) · line 139
Quoted 10 lines
arch/arm64/kvm/nested.c +10
-0+ int level, int input_size, int stride) +{ + int start_size, ia_size; + + ia_size =get_ia_size (wi); + +/* Check translation limits */ +switch (BIT (wi->pgshift)) { +case SZ_64K : +if (level ==0 || (level ==1 && ia_size <=42 ))It looks broken as the pseudocode checks the limits based on *implemented PA size*, rather than on
ia_size, which is essentially the input address size (64 - T0SZ) programmed by L1 hypervisor. They're different.We can probably get the implemented PA size by:
Quoted 7 lines
AArch64.PAMax(){ parange = get_idreg_field_enum(kvm, ID_AA64MMFR0_EL1, PARANGE); return id_aa64mmfr0_parange_to_phys_shift(parange); }Not sure if I've read the spec correctly.
I think that's also the way I read AArch64_S2InvalidSL(), which more or less mirrors the above.
The question is what should we limit it to? Is it PARange, as you suggest? Or the IPA range defined by userspace at VM creation (the type argument, which ends up in kvm->arch.mmu.pgt->ia_bits)?
I think this is the former, but we probably also need to handle the later on actual access (when reading the descriptor). Failure to read the descriptor (because it is outside of a memslot) should result in a SEA being injected in the guest.
Quoted 17 lines
Quoted 5 lines
arch/arm64/kvm/nested.c +4
-0+return -EFAULT ; +break ; +case SZ_16K : +if (level ==0 || (level ==1 && ia_size <=40 ))arch/arm64/kvm/nested.c int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu) · line 153
Quoted 10 lines
arch/arm64/kvm/nested.c +10
-0+return -EFAULT ; +break ; +case SZ_4K : +if (level <0 || (level ==0 && ia_size <=42 )) +return -EFAULT ; +break ; + } + +/* Check input size limits */ +if (input_size > ia_size)This is always false for the current code. ;-)
Yup. At least that doesn't introduce any extra bug! :)
Quoted 62 lines
Quoted 62 lines
arch/arm64/kvm/nested.c +61
-0+return -EFAULT ; + +/* Check number of entries in starting level table */ + start_size = input_size - ((3 - level) * stride + wi->pgshift); +if (start_size <1 || start_size > stride +4 ) +return -EFAULT ; + +return 0 ; +} + +/* Check if output is within boundaries */ +static int check_output_size (struct s2_walk_info *wi,phys_addr_t output) +{ +unsigned int output_size = wi->max_oa_bits; + +if (output_size !=48 && (output &GENMASK_ULL (47 , output_size))) +return -1 ; + +return 0 ; +} + +/* +* This is essentially a C-version of the pseudo code from the ARM ARM +* AArch64.TranslationTableWalk function. I strongly recommend looking at +* that pseudocode in trying to understand this. +* +* Must be called with the kvm->srcu read lock held +*/ +static int walk_nested_s2_pgd (phys_addr_t ipa, +struct s2_walk_info *wi,struct kvm_s2_trans *out) +{ +int first_block_level, level, stride, input_size, base_lower_bound; +phys_addr_t base_addr; +unsigned int addr_top, addr_bottom; +u64 desc;/* page table entry */ +int ret; +phys_addr_t paddr; + +switch (BIT (wi->pgshift)) { +default : +case SZ_64K : +case SZ_16K : + level =3 - wi->sl; + first_block_level =2 ; +break ; +case SZ_4K : + level =2 - wi->sl; + first_block_level =1 ; +break ; + } + + stride = wi->pgshift -3 ; + input_size =get_ia_size (wi); +if (input_size >48 || input_size <25 ) +return -EFAULT ; + + ret =check_base_s2_limits (wi, level, input_size, stride); +if (WARN_ON (ret)) +return ret; + + base_lower_bound = 3 + input_size - ((3 - level) * stride +
Quoted 9 lines
arch/arm64/kvm/nested.c int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu) · line 219
Quoted 5 lines
arch/arm64/kvm/nested.c +5
-0+ wi->pgshift); + base_addr = wi->baddr &GENMASK_ULL (47 , base_lower_bound); + +if (check_output_size (wi, base_addr)) { + out->esr =compute_fsc (level, ESR_ELx_FSC_ADDRSZ);With a wrongly programmed base address, we should report the ADDRSZ fault at level 0 (as per
R_BFHQHand the pseudocode). It's easy to fix.
Yup. Although this rule describe S1 rather than S2 (we don't seem to have anything saying the same thing for S2), but I expect the behaviour to be exactly the same.
Quoted 19 lines
Quoted 7 lines
arch/arm64/kvm/nested.c +7
-0+static void vtcr_to_walk_info (u64 vtcr,struct s2_walk_info *wi) +{ + wi->t0sz = vtcr &TCR_EL2_T0SZ_MASK ; + +switch (vtcr &VTCR_EL2_TG0_MASK ) { +case VTCR_EL2_TG0_4K : + wi->pgshift =12 ;break ;arch/arm64/kvm/nested.c int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu) · line 327
Quoted 10 lines
arch/arm64/kvm/nested.c +10
-0+case VTCR_EL2_TG0_16K : + wi->pgshift =14 ;break ; +case VTCR_EL2_TG0_64K : +default :/* IMPDEF: treat any other value as 64k */ + wi->pgshift =16 ;break ; + } + + wi->sl =FIELD_GET (VTCR_EL2_SL0_MASK , vtcr); +/* Global limit for now, should eventually be per-VM */ + wi->max_oa_bits = min(get_kvm_ipa_limit(),^^^
Quoted 6 lines
Should we use
AArch64.PAMax()instead? As the output address size is never larger than the implemented PA size.Now I'm wondering if we can let
kvm_get_pa_bits()just return PAMax for (based on the exposed (to-L1) AA64MFR0.PARange value) and use it when possible.
Yes, that was the plan all along, but I got sidetracked.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v3 02/16] KVM: arm64: nv: Implement nested Stage-2 page table walk logic
Quoted 58 lines · Marc Zyngier
Quoted 4 lines · Zenghui Yu
Hi Marc,
[ chewing through the NV code.. ;-) ]
You fool! :)
Quoted 38 lines · Marc Zyngier
Quoted 4 lines
From: Christoffer Dall <[email protected]>
Based on the pseudo-code in the ARM ARM, implement a stage 2 software page table walker.
[...]
Quoted 11 lines
(quoted diff) +11
-0+static u32 compute_fsc(int level, u32 fsc) +{ + return fsc | (level & 0x3); +} + +static int get_ia_size(struct s2_walk_info *wi) +{ + return 64 - wi->t0sz; +} + +static int check_base_s2_limits(struct s2_walk_info *wi,Review
Quoted 10 lines
(quoted diff) +10
-0+ int level, int input_size, int stride) +{ + int start_size, ia_size; + + ia_size = get_ia_size(wi); + + /* Check translation limits */ + switch (BIT(wi->pgshift)) { + case SZ_64K: + if (level == 0 || (level == 1 && ia_size <= 42))It looks broken as the pseudocode checks the limits based on *implemented PA size*, rather than on
ia_size, which is essentially the input address size (64 - T0SZ) programmed by L1 hypervisor. They're different.We can probably get the implemented PA size by:
Quoted 7 lines
AArch64.PAMax(){ parange = get_idreg_field_enum(kvm, ID_AA64MMFR0_EL1, PARANGE); return id_aa64mmfr0_parange_to_phys_shift(parange); }Not sure if I've read the spec correctly.
I think that's also the way I read
AArch64_S2InvalidSL(), which more or less mirrors the above.
Quoted 8 lines
The question is what should we limit it to? Is it PARange, as you suggest? Or the IPA range defined by userspace at VM creation (the type argument, which ends up in
kvm->arch.mmu.pgt->ia_bits)?I think this is the former, but we probably also need to handle the later on actual access (when reading the descriptor). Failure to read the descriptor (because it is outside of a memslot) should result in a SEA being injected in the guest.
Yup. Do you suggest something like
arch/arm64/kvm/nested.c +0 -0
@@ -293,8 +293,10 @@ static int walk_nested_s2_pgd(struct kvm_vcpu
*vcpu, phys_addr_t ipa,
paddr = base_addr | index;
ret = read_guest_s2_desc(vcpu, paddr, &desc, wi);
- if (ret < 0)
+ if (ret < 0) {
+ out->esr = ESR_ELx_FSC_SEA_TTW(level);
return ret;
+ }
new_desc = desc;
The current code doesn't specify the FSC when we fail to read the descriptor.
Quoted 92 lines
Quoted 17 lines
Quoted 5 lines
(quoted diff) +4
-0+ return -EFAULT; + break; + case SZ_16K: + if (level == 0 || (level == 1 && ia_size <= 40))Review
Quoted 10 lines
(quoted diff) +10
-0+ return -EFAULT; + break; + case SZ_4K: + if (level < 0 || (level == 0 && ia_size <= 42)) + return -EFAULT; + break; + } + + /* Check input size limits */ + if (input_size > ia_size)This is always false for the current code. ;-)
Yup. At least that doesn't introduce any extra bug! :)
Quoted 62 lines
Quoted 62 lines
(quoted diff) +61
-0+ return -EFAULT; + + /* Check number of entries in starting level table */ + start_size = input_size - ((3 - level) * stride + wi->pgshift); + if (start_size < 1 || start_size > stride + 4) + return -EFAULT; + + return 0; +} + +/* Check if output is within boundaries */ +static int check_output_size(struct s2_walk_info *wi, phys_addr_t output) +{ + unsigned int output_size = wi->max_oa_bits; + + if (output_size != 48 && (output & GENMASK_ULL(47, output_size))) + return -1; + + return 0; +} + +/* + * This is essentially a C-version of the pseudo code from the ARM ARM + * AArch64.TranslationTableWalk function. I strongly recommend looking at + * that pseudocode in trying to understand this. + * + * Must be called with the kvm->srcu read lock held + */ +static int walk_nested_s2_pgd(phys_addr_t ipa, + struct s2_walk_info *wi, struct kvm_s2_trans *out) +{ + int first_block_level, level, stride, input_size, base_lower_bound; + phys_addr_t base_addr; + unsigned int addr_top, addr_bottom; + u64 desc; /* page table entry */ + int ret; + phys_addr_t paddr; + + switch (BIT(wi->pgshift)) { + default: + case SZ_64K: + case SZ_16K: + level = 3 - wi->sl; + first_block_level = 2; + break; + case SZ_4K: + level = 2 - wi->sl; + first_block_level = 1; + break; + } + + stride = wi->pgshift - 3; + input_size = get_ia_size(wi); + if (input_size > 48 || input_size < 25) + return -EFAULT; + + ret = check_base_s2_limits(wi, level, input_size, stride); + if (WARN_ON(ret)) + return ret; + + base_lower_bound = 3 + input_size - ((3 - level) * stride +Quoted 9 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ wi->pgshift); + base_addr = wi->baddr & GENMASK_ULL(47, base_lower_bound); + + if (check_output_size(wi, base_addr)) { + out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ);With a wrongly programmed base address, we should report the ADDRSZ fault at level 0 (as per
R_BFHQHand the pseudocode). It's easy to fix.
Quoted 3 lines
Yup. Although this rule describe S1 rather than S2 (we don't seem to have anything saying the same thing for S2), but I expect the behaviour to be exactly the same.
The rule does describe s2. :)
R_BFHQH:
" When an Address size fault is generated, the reported fault code
indicates one of the following:
If the fault was generated due to the TTBR_ELx used in the translation having nonzero address bits above the OA size, then a fault at level
0. "
where TTBR_ELx is the general name which also applies to VTTBR_EL2.
AArch64.S2Walk also clearly describes this behaviour (I read DDI0478G.b J1-8122).
Quoted 19 lines
Quoted 19 lines
Quoted 7 lines
(quoted diff) +7
-0+static void vtcr_to_walk_info(u64 vtcr, struct s2_walk_info *wi) +{ + wi->t0sz = vtcr & TCR_EL2_T0SZ_MASK; + + switch (vtcr & VTCR_EL2_TG0_MASK) { + case VTCR_EL2_TG0_4K: + wi->pgshift = 12; break;Review
Quoted 10 lines
(quoted diff) +10
-0+ case VTCR_EL2_TG0_16K: + wi->pgshift = 14; break; + case VTCR_EL2_TG0_64K: + default: /* IMPDEF: treat any other value as 64k */ + wi->pgshift = 16; break; + } + + wi->sl = FIELD_GET(VTCR_EL2_SL0_MASK, vtcr); + /* Global limit for now, should eventually be per-VM */ + wi->max_oa_bits = min(get_kvm_ipa_limit(),^^^
Quoted 8 lines
Quoted 6 lines
Should we use
AArch64.PAMax()instead? As the output address size is never larger than the implemented PA size.Now I'm wondering if we can let
kvm_get_pa_bits()just return PAMax for (based on the exposed (to-L1) AA64MFR0.PARange value) and use it when possible.Yes, that was the plan all along, but I got sidetracked.
OK. The concern is that PARange is writable from userspace, init_nested_s2_mmu() can not get a fixed PAMax vaule (kvm_get_pa_bits()) until AA64MMFR0_EL1 has became immutable (i.e., VM has started).
I need to think it over.
Thanks,
Zenghui
Marc Zyngier
maintainerPATCH v3 3/16 KVM: arm64: nv: Handle shadow stage 2 page faults
In reply to Marc ZyngierIf we are faulting on a shadow stage 2 translation, we first walk the guest hypervisor's stage 2 page table to see if it has a mapping. If not, we inject a stage 2 page fault to the virtual EL2. Otherwise, we create a mapping in the shadow stage 2 page table.
Note that we have to deal with two IPAs when we got a shadow stage 2 page fault. One is the address we faulted on, and is in the L2 guest phys space. The other is from the guest stage-2 page table walk, and is in the L1 guest phys space. To differentiate them, we rename variables so that fault_ipa is used for the former and ipa is used for the latter.
When mapping a page in a shadow stage-2, special care must be taken not to be more permissive than the guest is.
--- arch/arm64/include/asm/kvm_nested.h | 33 ++++++++++ arch/arm64/kvm/mmu.c | 97 ++++++++++++++++++++++++++--- arch/arm64/kvm/nested.c | 45 +++++++++++++ 3 files changed, 166 insertions(+), 9 deletions(-)
arch/arm64/include/asm/kvm_nested.h +33 -0
@@ -78,8 +78,41 @@ struct kvm_s2_trans {u64 upper_attr; }; +static inline phys_addr_t kvm_s2_trans_output (struct kvm_s2_trans *trans) +{ +return trans->output; +} + +static inline unsigned long kvm_s2_trans_size (struct kvm_s2_trans *trans) +{ +return trans->block_size; +} + +static inline u32 kvm_s2_trans_esr (struct kvm_s2_trans *trans) +{ +return trans->esr; +} + +static inline bool kvm_s2_trans_readable (struct kvm_s2_trans *trans) +{ +return trans->readable; +} + +static inline bool kvm_s2_trans_writable (struct kvm_s2_trans *trans) +{ +return trans->writable; +} + +static inline bool kvm_s2_trans_executable (struct kvm_s2_trans *trans) +{ +return !(trans->upper_attr &BIT (54 )); +} +extern int kvm_walk_nested_s2 (struct kvm_vcpu *vcpu,phys_addr_t gipa,struct kvm_s2_trans *result); +extern int kvm_s2_handle_perm_fault (struct kvm_vcpu *vcpu, +struct kvm_s2_trans *trans); +extern int kvm_inject_s2_fault (struct kvm_vcpu *vcpu,u64 esr_el2);int kvm_init_nv_sysregs (struct kvm *kvm);
arch/arm64/kvm/mmu.c +88 -9
165 lines not on this page · raw message
arch/arm64/kvm/nested.c +45 -0
57 lines not on this page · raw message
Signature · 1 line
2.39.2
Zenghui Yu
reviewerRe: [PATCH v3 03/16] KVM: arm64: nv: Handle shadow stage 2 page faults
Quoted 47 lines · Marc Zyngier
arch/arm64/kvm/mmu.c +25
-0@@ -1407,6 +1407,7 @@ static bool kvm_vma_mte_allowed(struct vm_area_struct *vma) }static int user_mem_abort (struct kvm_vcpu *vcpu,phys_addr_t fault_ipa, +struct kvm_s2_trans *nested,struct kvm_memory_slot *memslot,unsigned long hva,bool fault_is_perm) {@@ -1415,6 +1416,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,bool exec_fault, mte_allowed;bool device = false, vfio_allow_any_uc = false;unsigned long mmu_seq; +phys_addr_t ipa = fault_ipa;struct kvm *kvm = vcpu->kvm;struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache;struct vm_area_struct *vma;@@ -1498,10 +1500,38 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, } vma_pagesize =1UL << vma_shift; + +if (nested) { +unsigned long max_map_size; + + max_map_size = force_pte ?PAGE_SIZE :PUD_SIZE ; + + ipa =kvm_s2_trans_output (nested); + +/* +* If we're about to create a shadow stage 2 entry, then we +* can only create a block mapping if the guest stage 2 page +* table uses at least as big a mapping. +*/ + max_map_size =min (kvm_s2_trans_size (nested), max_map_size); + +/* +* Be careful that if the mapping size falls between +* two host sizes, take the smallest of the two. +*/ +if (max_map_size >=PMD_SIZE && max_map_size <PUD_SIZE ) + max_map_size =PMD_SIZE ; +else if (max_map_size >=PAGE_SIZE && max_map_size <PMD_SIZE ) + max_map_size =PAGE_SIZE ;
arch/arm64/kvm/mmu.c static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, · line 1534
Quoted 10 lines
arch/arm64/kvm/mmu.c +6
-1+ + force_pte = (max_map_size ==PAGE_SIZE ); + vma_pagesize =min (vma_pagesize, (long )max_map_size); + } +if (vma_pagesize ==PMD_SIZE || vma_pagesize ==PUD_SIZE ) fault_ipa &= ~(vma_pagesize -1 ); - gfn = fault_ipa >>PAGE_SHIFT ; + gfn = ipa >>PAGE_SHIFT ;
I had seen a non-nested guest boot failure (with vma_pagesize == PUD_SIZE) and bisection led me here.
Is it intentional to ignore the fault_ipa adjustment when calculating gfn if the guest memory is backed by hugetlbfs? This looks broken for the non-nested case.
But since I haven't looked at user_mem_abort() for a long time, I'm not sure if I'd missed something...
Thanks,
Zenghui
Re: [PATCH v3 03/16] KVM: arm64: nv: Handle shadow stage 2 page faults
Quoted 37 lines · Zenghui Yu
Quoted 23 lines
(quoted diff) +23
-0+ + if (nested) { + unsigned long max_map_size; + + max_map_size = force_pte ? PAGE_SIZE : PUD_SIZE; + + ipa = kvm_s2_trans_output(nested); + + /* + * If we're about to create a shadow stage 2 entry, then we + * can only create a block mapping if the guest stage 2 page + * table uses at least as big a mapping. + */ + max_map_size = min(kvm_s2_trans_size(nested), max_map_size); + + /* + * Be careful that if the mapping size falls between + * two host sizes, take the smallest of the two. + */ + if (max_map_size >= PMD_SIZE && max_map_size < PUD_SIZE) + max_map_size = PMD_SIZE; + else if (max_map_size >= PAGE_SIZE && max_map_size < PMD_SIZE) + max_map_size = PAGE_SIZE;Review
Quoted 10 lines
(quoted diff) +6
-1+ + force_pte = (max_map_size == PAGE_SIZE); + vma_pagesize = min(vma_pagesize, (long)max_map_size); + } + if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE) fault_ipa &= ~(vma_pagesize - 1); - gfn = fault_ipa >> PAGE_SHIFT; + gfn = ipa >> PAGE_SHIFT;I had seen a non-nested guest boot failure (with
vma_pagesize==PUD_SIZE) and bisection led me here.
Quoted 6 lines
Is it intentional to ignore the
fault_ipaadjustment when calculating gfn if the guest memory is backed by hugetlbfs? This looks broken for the non-nested case.But since I haven't looked at
user_mem_abort()for a long time, I'm not sure if I'd missed something...
Nope, you're spot on as usual.
Seems like we'd want to make sure both the canonical IPA and fault IPA are hugepage-aligned to get the right PFN and map it at the right place.
I repro'ed the boot failure, the following diff gets me back in business. I was _just_ about to send the second batch of fixes, but this is a rather smelly one.
Unless someone screams, this is getting stuffed on top.
arch/arm64/kvm/mmu.c +8 -1
@@ -1540,8 +1540,15 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, vma_pagesize =min (vma_pagesize, (long )max_map_size); } -if (vma_pagesize ==PMD_SIZE || vma_pagesize ==PUD_SIZE ) +/* +* Both the canonical IPA and fault IPA must be hugepage-aligned to +* ensure we find the right PFN and lay down the mapping in the right +* place. +*/ +if (vma_pagesize ==PMD_SIZE || vma_pagesize ==PUD_SIZE ) { fault_ipa &= ~(vma_pagesize -1 ); + ipa &= ~(vma_pagesize -1 ); + } gfn = ipa >>PAGE_SHIFT ; mte_allowed =kvm_vma_mte_allowed (vma);
Signature · 2 lines
Thanks, Oliver
Marc Zyngier
maintainerPATCH v3 4/16 KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
In reply to Marc ZyngierFrom: Christoffer Dall <[email protected]>
Unmap/flush shadow stage 2 page tables for the nested VMs as well as the stage 2 page table for the guest hypervisor.
Note: A bunch of the code in mmu.c relating to MMU notifiers is currently dealt with in an extremely abrupt way, for example by clearing out an entire shadow stage-2 table. This will be handled in a more efficient way using the reverse mapping feature in a later version of the patch series.
--- arch/arm64/include/asm/kvm_mmu.h | 2 ++ arch/arm64/include/asm/kvm_nested.h | 3 +++ arch/arm64/kvm/mmu.c | 28 +++++++++++++++---- arch/arm64/kvm/nested.c | 42 +++++++++++++++++++++++++++++ 4 files changed, 70 insertions(+), 5 deletions(-)
arch/arm64/include/asm/kvm_mmu.h +2 -0
@@ -167,6 +167,8 @@ int create_hyp_stack(phys_addr_t phys_addr, unsigned long *haddr);void __init free_hyp_pgds (void );void kvm_stage2_unmap_range (struct kvm_s2_mmu *mmu,phys_addr_t start,u64 size); +void kvm_stage2_flush_range (struct kvm_s2_mmu *mmu,phys_addr_t addr,phys_addr_t end); +void kvm_stage2_wp_range (struct kvm_s2_mmu *mmu,phys_addr_t addr,phys_addr_t end);void stage2_unmap_vm (struct kvm *kvm);int kvm_init_stage2_mmu (struct kvm *kvm,struct kvm_s2_mmu *mmu,unsigned long type);
arch/arm64/include/asm/kvm_nested.h +3 -0
@@ -113,6 +113,9 @@ extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,extern int kvm_s2_handle_perm_fault (struct kvm_vcpu *vcpu,struct kvm_s2_trans *trans);extern int kvm_inject_s2_fault (struct kvm_vcpu *vcpu,u64 esr_el2); +extern void kvm_nested_s2_wp (struct kvm *kvm); +extern void kvm_nested_s2_unmap (struct kvm *kvm); +extern void kvm_nested_s2_flush (struct kvm *kvm);int kvm_init_nv_sysregs (struct kvm *kvm);
arch/arm64/kvm/mmu.c +23 -5
@@ -333,13 +333,18 @@ void kvm_stage2_unmap_range(struct kvm_s2_mmu *mmu, phys_addr_t start, u64 size)__unmap_stage2_range (mmu, start, size, true); } +void kvm_stage2_flush_range (struct kvm_s2_mmu *mmu,phys_addr_t addr,phys_addr_t end) +{ +stage2_apply_range_resched (mmu, addr, end, kvm_pgtable_stage2_flush); +} +static void stage2_flush_memslot (struct kvm *kvm,struct kvm_memory_slot *memslot) {phys_addr_t addr = memslot->base_gfn <<PAGE_SHIFT ;phys_addr_t end = addr +PAGE_SIZE * memslot->npages; -stage2_apply_range_resched (&kvm->arch.mmu, addr, end, kvm_pgtable_stage2_flush); +kvm_stage2_flush_range (&kvm->arch.mmu, addr, end); }/**
@@ -362,6 +367,8 @@ static void stage2_flush_vm(struct kvm *kvm)kvm_for_each_memslot (memslot, bkt, slots)stage2_flush_memslot (kvm, memslot); +kvm_nested_s2_flush (kvm); +write_unlock (&kvm->mmu_lock);srcu_read_unlock (&kvm->srcu, idx); }
@@ -1035,6 +1042,8 @@ void stage2_unmap_vm(struct kvm *kvm)kvm_for_each_memslot (memslot, bkt, slots)stage2_unmap_memslot (kvm, memslot); +kvm_nested_s2_unmap (kvm); +write_unlock (&kvm->mmu_lock);mmap_read_unlock (current->mm);srcu_read_unlock (&kvm->srcu, idx);
@@ -1134,12 +1143,12 @@ int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa, }/** -* stage2_wp_range() - write protect stage2 memory region range +* kvm_stage2_wp_range() - write protect stage2 memory region range * @mmu: The KVM stage-2 MMU pointer * @addr: Start address of range * @end: End address of range */ -static void stage2_wp_range (struct kvm_s2_mmu *mmu,phys_addr_t addr,phys_addr_t end) +void kvm_stage2_wp_range (struct kvm_s2_mmu *mmu,phys_addr_t addr,phys_addr_t end) {stage2_apply_range_resched (mmu, addr, end, kvm_pgtable_stage2_wrprotect); }
@@ -1170,7 +1179,8 @@ static void kvm_mmu_wp_memory_region(struct kvm *kvm, int slot) end = (memslot->base_gfn + memslot->npages) <<PAGE_SHIFT ;write_lock (&kvm->mmu_lock); -stage2_wp_range (&kvm->arch.mmu, start, end); +kvm_stage2_wp_range (&kvm->arch.mmu, start, end); +kvm_nested_s2_wp (kvm);write_unlock (&kvm->mmu_lock);kvm_flush_remote_tlbs_memslot (kvm, memslot); }
@@ -1224,7 +1234,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,lockdep_assert_held_write (&kvm->mmu_lock); -stage2_wp_range (&kvm->arch.mmu, start, end); +kvm_stage2_wp_range (&kvm->arch.mmu, start, end);/* * Eager-splitting is done when manual-protect is set. We
@@ -1236,6 +1246,8 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,*/ if (kvm_dirty_log_manual_protect_and_init_set (kvm))kvm_mmu_split_huge_pages (kvm, start, end); + +kvm_nested_s2_wp (kvm); }static void kvm_send_hwpoison_signal (unsigned long address,short lsb)
@@ -1878,6 +1890,7 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) (range->end - range->start) <<PAGE_SHIFT , range->may_block); +kvm_nested_s2_unmap (kvm);return false; }
@@ -1891,6 +1904,10 @@ bool kvm_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)return kvm_pgtable_stage2_test_clear_young (kvm->arch.mmu.pgt, range->start <<PAGE_SHIFT , size, true); +/* +* TODO: Handle nested_mmu structures here using the reverse mapping in +* a later version of patch series. +*/ }bool kvm_test_age_gfn (struct kvm *kvm,struct kvm_gfn_range *range)
@@ -2141,6 +2158,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,write_lock (&kvm->mmu_lock);kvm_stage2_unmap_range (&kvm->arch.mmu, gpa, size); +kvm_nested_s2_unmap (kvm);write_unlock (&kvm->mmu_lock); }
arch/arm64/kvm/nested.c +42 -0
48 lines not on this page · raw message
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 5/16 KVM: arm64: nv: Add Stage-1 EL2 invalidation primitives
In reply to Marc ZyngierProvide the primitives required to handle TLB invalidation for Stage-1 EL2 TLBs, which by definition do not require messing with the Stage-2 page tables.
--- arch/arm64/include/asm/kvm_asm.h | 2 + arch/arm64/kvm/hyp/vhe/tlb.c | 65 ++++++++++++++++++++++++++++++++ 2 files changed, 67 insertions(+)
arch/arm64/include/asm/kvm_asm.h +2 -0
@@ -232,6 +232,8 @@ extern void __kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu, phys_addr_t start, unsigned long pages);extern void __kvm_tlb_flush_vmid (struct kvm_s2_mmu *mmu); +extern int __kvm_tlbi_s1e2 (struct kvm_s2_mmu *mmu,u64 va,u64 sys_encoding); +extern void __kvm_timer_set_cntvoff (u64 cntvoff);extern int __kvm_vcpu_run (struct kvm_vcpu *vcpu);
arch/arm64/kvm/hyp/vhe/tlb.c +65 -0
68 lines not on this page · raw message
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 6/16 KVM: arm64: nv: Handle EL2 Stage-1 TLB invalidation
In reply to Marc ZyngierDue to the way FEAT_NV2 suppresses traps when accessing EL2 system registers, we can't track when the guest changes its HCR_EL2.TGE setting. This means we always trap EL1 TLBIs, even if they don't affect any L2 guest.
Given that invalidating the EL2 TLBs doesn't require any messing with the shadow stage-2 page-tables, we can simply emulate the instructions early and return directly to the guest.
This is conditioned on the instruction being an EL1 one and the guest's HCR_EL2.{E2H,TGE} being {1,1} (indicating that the instruction targets the EL2 S1 TLBs), or the instruction being one of the EL2 ones (which are not ambiguous).
EL1 TLBIs issued with HCR_EL2.{E2H,TGE}={1,0} are not handled here, and cause a full exit so that they can be handled in the context of a VMID.
--- arch/arm64/include/asm/kvm_nested.h | 55 +++++++++++++++++++++++++++++ arch/arm64/include/asm/sysreg.h | 17 +++++++++ arch/arm64/kvm/hyp/vhe/switch.c | 51 +++++++++++++++++++++++++- 3 files changed, 122 insertions(+), 1 deletion(-)
arch/arm64/include/asm/kvm_nested.h +55 -0
@@ -117,6 +117,61 @@ extern void kvm_nested_s2_wp(struct kvm *kvm);extern void kvm_nested_s2_unmap (struct kvm *kvm);extern void kvm_nested_s2_flush (struct kvm *kvm); +static inline bool kvm_supported_tlbi_s1e1_op (struct kvm_vcpu *vpcu,u32 instr) +{ +struct kvm *kvm = vpcu->kvm; +u8 CRm =sys_reg_CRm (instr); + +if (!(sys_reg_Op0 (instr) == TLBI_Op0 && +sys_reg_Op1 (instr) == TLBI_Op1_EL1)) +return false; + +if (!(sys_reg_CRn (instr) == TLBI_CRn_XS || + (sys_reg_CRn (instr) == TLBI_CRn_nXS && +kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,XS ,IMP )))) +return false; + +if (CRm == TLBI_CRm_nROS && + !kvm_has_feat (kvm,ID_AA64ISAR0_EL1 ,TLB ,OS )) +return false; + +if ((CRm == TLBI_CRm_RIS || CRm == TLBI_CRm_ROS || + CRm == TLBI_CRm_RNS) && + !kvm_has_feat (kvm,ID_AA64ISAR0_EL1 ,TLB ,RANGE )) +return false; + +return true; +} + +static inline bool kvm_supported_tlbi_s1e2_op (struct kvm_vcpu *vpcu,u32 instr) +{ +struct kvm *kvm = vpcu->kvm; +u8 CRm =sys_reg_CRm (instr); + +if (!(sys_reg_Op0 (instr) == TLBI_Op0 && +sys_reg_Op1 (instr) == TLBI_Op1_EL2)) +return false; + +if (!(sys_reg_CRn (instr) == TLBI_CRn_XS || + (sys_reg_CRn (instr) == TLBI_CRn_nXS && +kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,XS ,IMP )))) +return false; + +if (CRm == TLBI_CRm_IPAIS || CRm == TLBI_CRm_IPAONS) +return false; + +if (CRm == TLBI_CRm_nROS && + !kvm_has_feat (kvm,ID_AA64ISAR0_EL1 ,TLB ,OS )) +return false; + +if ((CRm == TLBI_CRm_RIS || CRm == TLBI_CRm_ROS || + CRm == TLBI_CRm_RNS) && + !kvm_has_feat (kvm,ID_AA64ISAR0_EL1 ,TLB ,RANGE )) +return false; + +return true; +} +int kvm_init_nv_sysregs (struct kvm *kvm);#ifdef CONFIG_ARM64_PTR_AUTH
arch/arm64/include/asm/sysreg.h +17 -0
@@ -654,6 +654,23 @@#define OP_AT_S12E0W sys_insn(AT_Op0, 4, AT_CRn, 8, 7)/* TLBI instructions */ +#define TLBI_Op0 1 + +#define TLBI_Op1_EL1 0/* Accessible from EL1 or higher */ +#define TLBI_Op1_EL2 4/* Accessible from EL2 or higher */ + +#define TLBI_CRn_XS 8/* Extra Slow (the common one) */ +#define TLBI_CRn_nXS 9/* not Extra Slow (which nobody uses)*/ + +#define TLBI_CRm_IPAIS 0/* S2 Inner-Shareable */ +#define TLBI_CRm_nROS 1/* non-Range, Outer-Sharable */ +#define TLBI_CRm_RIS 2/* Range, Inner-Sharable */ +#define TLBI_CRm_nRIS 3/* non-Range, Inner-Sharable */ +#define TLBI_CRm_IPAONS 4/* S2 Outer and Non-Shareable */ +#define TLBI_CRm_ROS 5/* Range, Outer-Sharable */ +#define TLBI_CRm_RNS 6/* Range, Non-Sharable */ +#define TLBI_CRm_nRNS 7/* non-Range, Non-Sharable */ +#define OP_TLBI_VMALLE1OS sys_insn(1, 0, 8, 1, 0)#define OP_TLBI_VAE1OS sys_insn(1, 0, 8, 1, 1)#define OP_TLBI_ASIDE1OS sys_insn(1, 0, 8, 1, 2)
arch/arm64/kvm/hyp/vhe/switch.c +50 -1
60 lines not on this page · raw message
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 7/16 KVM: arm64: nv: Handle TLB invalidation targeting L2 stage-1
In reply to Marc ZyngierWhile dealing with TLB invalidation targeting the guest hypervisor's own stage-1 was easy, doing the same thing for its own guests is a bit more involved.
Since such an invalidation is scoped by VMID, it needs to apply to all s2_mmu contexts that have been tagged by that VMID, irrespective of the value of VTTBR_EL2.BADDR.
So for each s2_mmu context matching that VMID, we invalidate the corresponding TLBs, each context having its own "physical" VMID.
--- arch/arm64/include/asm/kvm_nested.h | 7 +++ arch/arm64/kvm/nested.c | 35 +++++++++++++ arch/arm64/kvm/sys_regs.c | 80 +++++++++++++++++++++++++++++ 3 files changed, 122 insertions(+)
arch/arm64/include/asm/kvm_nested.h +7 -0
@@ -65,6 +65,13 @@ extern void kvm_init_nested(struct kvm *kvm);extern int kvm_vcpu_init_nested (struct kvm_vcpu *vcpu);extern void kvm_init_nested_s2_mmu (struct kvm_s2_mmu *mmu);extern struct kvm_s2_mmu *lookup_s2_mmu (struct kvm_vcpu *vcpu); + +union tlbi_info ; + +extern void kvm_s2_mmu_iterate_by_vmid (struct kvm *kvm,u16 vmid, +const union tlbi_info *info, +void (*)(struct kvm_s2_mmu *, +const union tlbi_info *));extern void kvm_vcpu_load_hw_mmu (struct kvm_vcpu *vcpu);extern void kvm_vcpu_put_hw_mmu (struct kvm_vcpu *vcpu);
arch/arm64/kvm/nested.c +35 -0
@@ -364,6 +364,41 @@ int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa,return ret; } +/* +* We can have multiple *different* MMU contexts with the same VMID: +* +* - S2 being enabled or not, hence differing by the HCR_EL2.VM bit +* +* - Multiple vcpus using private S2s (huh huh...), hence differing by the +* VBBTR_EL2.BADDR address +* +* - A combination of the above... +* +* We can always identify which MMU context to pick at run-time. However, +* TLB invalidation involving a VMID must take action on all the TLBs using +* this particular VMID. This translates into applying the same invalidation +* operation to all the contexts that are using this VMID. Moar phun! +*/ +void kvm_s2_mmu_iterate_by_vmid (struct kvm *kvm,u16 vmid, +const union tlbi_info *info, +void (*tlbi_callback)(struct kvm_s2_mmu *, +const union tlbi_info *)) +{ +write_lock (&kvm->mmu_lock); + +for (int i =0 ; i < kvm->arch.nested_mmus_size; i++) { +struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i]; + +if (!kvm_s2_mmu_valid (mmu)) +continue ; + +if (vmid ==get_vmid (mmu->tlb_vttbr)) +tlbi_callback (mmu, info); + } + +write_unlock (&kvm->mmu_lock); +} +struct kvm_s2_mmu *lookup_s2_mmu (struct kvm_vcpu *vcpu) {struct kvm *kvm = vcpu->kvm;
arch/arm64/kvm/sys_regs.c +80 -0
92 lines not on this page · raw message
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 8/16 KVM: arm64: nv: Handle TLBI VMALLS12E1{,IS} operations
In reply to Marc ZyngierEmulating TLBI VMALLS12E1* results in tearing down all the shadow S2 PTs that match the current VMID, since our shadow S2s are just some form of SW-managed TLBs. That teardown itself results in a full TLB invalidation for both S1 and S2.
This can result in over-invalidation if two vcpus use the same VMID to tag private S2 PTs, but this is still correct from an architecture perspective.
--- arch/arm64/kvm/sys_regs.c | 51 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+)
arch/arm64/kvm/sys_regs.c +51 -0
@@ -2741,6 +2741,22 @@ static const struct sys_reg_desc sys_reg_descs[] = { EL2_REG(SP_EL2, NULL, reset_unknown, 0), }; +static bool kvm_supported_tlbi_s12_op (struct kvm_vcpu *vpcu,u32 instr) +{ +struct kvm *kvm = vpcu->kvm; +u8 CRm =sys_reg_CRm (instr); + +if (sys_reg_CRn (instr) == TLBI_CRn_nXS && + !kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,XS ,IMP )) +return false; + +if (CRm == TLBI_CRm_nROS && + !kvm_has_feat (kvm,ID_AA64ISAR0_EL1 ,TLB ,OS )) +return false; + +return true; +} +/* Only defined here as this is an internal "abstraction" */ union tlbi_info { struct {
@@ -2758,6 +2774,38 @@ union tlbi_info { } va; }; +static void s2_mmu_unmap_range (struct kvm_s2_mmu *mmu, +const union tlbi_info *info) +{ +kvm_stage2_unmap_range (mmu, info->range.start, info->range.size); +} + +static bool handle_vmalls12e1is (struct kvm_vcpu *vcpu,struct sys_reg_params *p, +const struct sys_reg_desc *r) +{ +u32 sys_encoding =sys_insn (p->Op0, p->Op1, p->CRn, p->CRm, p->Op2); +u64 limit, vttbr; + +if (!kvm_supported_tlbi_s12_op (vcpu, sys_encoding)) { +kvm_inject_undefined (vcpu); +return false; + } + + vttbr =vcpu_read_sys_reg (vcpu,VTTBR_EL2 ); + limit =BIT_ULL (kvm_get_pa_bits (vcpu->kvm)); + +kvm_s2_mmu_iterate_by_vmid (vcpu->kvm,get_vmid (vttbr), + &(union tlbi_info ) { + .range = { + .start =0 , + .size = limit, + }, + }, + s2_mmu_unmap_range); + +return true; +} +static void s2_mmu_tlbi_s1e1 (struct kvm_s2_mmu *mmu,const union tlbi_info *info) {
@@ -2831,6 +2879,9 @@ static struct sys_reg_desc sys_insn_descs[] = {SYS_INSN (TLBI_VAAE1 , handle_tlbi_el1),SYS_INSN (TLBI_VALE1 , handle_tlbi_el1),SYS_INSN (TLBI_VAALE1 , handle_tlbi_el1), + +SYS_INSN (TLBI_VMALLS12E1IS , handle_vmalls12e1is), + SYS_INSN(TLBI_VMALLS12E1, handle_vmalls12e1is), };static const struct sys_reg_desc *first_idreg;
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 9/16 KVM: arm64: nv: Handle TLBI ALLE1{,IS} operations
In reply to Marc ZyngierTLBI ALLE1* is a pretty big hammer that invalides all S1/S2 TLBs.
This translates into the unmapping of all our shadow S2 PTs, itself resulting in the corresponding TLB invalidations.
--- arch/arm64/kvm/sys_regs.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
arch/arm64/kvm/sys_regs.c +25 -0
@@ -2757,6 +2757,29 @@ static bool kvm_supported_tlbi_s12_op(struct kvm_vcpu *vpcu, u32 instr)return true; } +static bool handle_alle1is (struct kvm_vcpu *vcpu,struct sys_reg_params *p, +const struct sys_reg_desc *r) +{ +u32 sys_encoding =sys_insn (p->Op0, p->Op1, p->CRn, p->CRm, p->Op2); + +if (!kvm_supported_tlbi_s12_op (vcpu, sys_encoding)) { +kvm_inject_undefined (vcpu); +return false; + } + +write_lock (&vcpu->kvm->mmu_lock); + +/* +* Drop all shadow S2s, resulting in S1/S2 TLBIs for each of the +* corresponding VMIDs. +*/ +kvm_nested_s2_unmap (vcpu->kvm); + +write_unlock (&vcpu->kvm->mmu_lock); + +return true; +} +/* Only defined here as this is an internal "abstraction" */ union tlbi_info { struct {
@@ -2880,7 +2903,9 @@ static struct sys_reg_desc sys_insn_descs[] = {SYS_INSN (TLBI_VALE1 , handle_tlbi_el1),SYS_INSN (TLBI_VAALE1 , handle_tlbi_el1), +SYS_INSN (TLBI_ALLE1IS , handle_alle1is),SYS_INSN (TLBI_VMALLS12E1IS , handle_vmalls12e1is), +SYS_INSN (TLBI_ALLE1 , handle_alle1is), SYS_INSN(TLBI_VMALLS12E1, handle_vmalls12e1is), };
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 10/16 KVM: arm64: nv: Handle TLBI IPAS2E1{,IS} operations
In reply to Marc ZyngierTLBI IPAS2E1* are the last class of TLBI instructions we need to handle. For each matching S2 MMU context, we invalidate a range corresponding to the largest possible mapping for that context.
At this stage, we don't handle TTL, which means we are likely over-invalidating. Further patches will aim at making this a bit better.
--- arch/arm64/kvm/sys_regs.c | 96 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+)
arch/arm64/kvm/sys_regs.c +96 -0
@@ -2780,6 +2780,31 @@ static bool handle_alle1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,return true; } +static bool kvm_supported_tlbi_ipas2_op (struct kvm_vcpu *vpcu,u32 instr) +{ +struct kvm *kvm = vpcu->kvm; +u8 CRm =sys_reg_CRm (instr); +u8 Op2 =sys_reg_Op2 (instr); + +if (sys_reg_CRn (instr) == TLBI_CRn_nXS && + !kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,XS ,IMP )) +return false; + +if (CRm == TLBI_CRm_IPAIS && (Op2 ==2 || Op2 ==6 ) && + !kvm_has_feat (kvm,ID_AA64ISAR0_EL1 ,TLB ,RANGE )) +return false; + +if (CRm == TLBI_CRm_IPAONS && (Op2 ==0 || Op2 ==4 ) && + !kvm_has_feat (kvm,ID_AA64ISAR0_EL1 ,TLB ,OS )) +return false; + +if (CRm == TLBI_CRm_IPAONS && (Op2 ==3 || Op2 ==7 ) && + !kvm_has_feat (kvm,ID_AA64ISAR0_EL1 ,TLB ,RANGE )) +return false; + +return true; +} +/* Only defined here as this is an internal "abstraction" */ union tlbi_info { struct {
@@ -2829,6 +2854,72 @@ static bool handle_vmalls12e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,return true; } +static void s2_mmu_unmap_ipa (struct kvm_s2_mmu *mmu, +const union tlbi_info *info) +{ +unsigned long max_size; +u64 base_addr; + +/* +* We drop a number of things from the supplied value: +* +* - NS bit: we're non-secure only. +* +* - TTL field: We already have the granule size from the +* VTCR_EL2.TG0 field, and the level is only relevant to the +* guest's S2PT. +* +* - IPA[51:48]: We don't support 52bit IPA just yet... +* +* And of course, adjust the IPA to be on an actual address. +*/ + base_addr = (info->ipa.addr &GENMASK_ULL (35 ,0 )) <<12 ; + +/* Compute the maximum extent of the invalidation */ +switch (mmu->tlb_vtcr &VTCR_EL2_TG0_MASK ) { +case VTCR_EL2_TG0_4K : + max_size =SZ_1G ; +break ; +case VTCR_EL2_TG0_16K : + max_size =SZ_32M ; +break ; +case VTCR_EL2_TG0_64K : +default :/* IMPDEF: treat any other value as 64k */ +/* +* No, we do not support 52bit IPA in nested yet. Once +* we do, this should be 4TB. +*/ + max_size =SZ_512M ; +break ; + } + + base_addr &= ~(max_size -1 ); + +kvm_stage2_unmap_range (mmu, base_addr, max_size); +} + +static bool handle_ipas2e1is (struct kvm_vcpu *vcpu,struct sys_reg_params *p, +const struct sys_reg_desc *r) +{ +u32 sys_encoding =sys_insn (p->Op0, p->Op1, p->CRn, p->CRm, p->Op2); +u64 vttbr =vcpu_read_sys_reg (vcpu,VTTBR_EL2 ); + +if (!kvm_supported_tlbi_ipas2_op (vcpu, sys_encoding)) { +kvm_inject_undefined (vcpu); +return false; + } + +kvm_s2_mmu_iterate_by_vmid (vcpu->kvm,get_vmid (vttbr), + &(union tlbi_info ) { + .ipa = { + .addr = p->regval, + }, + }, + s2_mmu_unmap_ipa); + +return true; +} +static void s2_mmu_tlbi_s1e1 (struct kvm_s2_mmu *mmu,const union tlbi_info *info) {
@@ -2903,8 +2994,13 @@ static struct sys_reg_desc sys_insn_descs[] = {SYS_INSN (TLBI_VALE1 , handle_tlbi_el1),SYS_INSN (TLBI_VAALE1 , handle_tlbi_el1), +SYS_INSN (TLBI_IPAS2E1IS , handle_ipas2e1is), +SYS_INSN (TLBI_IPAS2LE1IS , handle_ipas2e1is), +SYS_INSN (TLBI_ALLE1IS , handle_alle1is),SYS_INSN (TLBI_VMALLS12E1IS , handle_vmalls12e1is), +SYS_INSN (TLBI_IPAS2E1 , handle_ipas2e1is), +SYS_INSN (TLBI_IPAS2LE1 , handle_ipas2e1is),SYS_INSN (TLBI_ALLE1 , handle_alle1is), SYS_INSN(TLBI_VMALLS12E1, handle_vmalls12e1is), };
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 11/16 KVM: arm64: nv: Handle FEAT_TTL hinted TLB operations
In reply to Marc ZyngierSupport guest-provided information information to size the range of required invalidation. This helps with reducing over-invalidation, provided that the guest actually provides accurate information.
--- arch/arm64/include/asm/kvm_nested.h | 2 + arch/arm64/kvm/nested.c | 89 +++++++++++++++++++++++++++++ arch/arm64/kvm/sys_regs.c | 24 +------- 3 files changed, 92 insertions(+), 23 deletions(-)
arch/arm64/include/asm/kvm_nested.h +2 -0
@@ -124,6 +124,8 @@ extern void kvm_nested_s2_wp(struct kvm *kvm);extern void kvm_nested_s2_unmap (struct kvm *kvm);extern void kvm_nested_s2_flush (struct kvm *kvm); +unsigned long compute_tlb_inval_range (struct kvm_s2_mmu *mmu,u64 val); +static inline bool kvm_supported_tlbi_s1e1_op (struct kvm_vcpu *vpcu,u32 instr) {struct kvm *kvm = vpcu->kvm;
arch/arm64/kvm/nested.c +89 -0
95 lines not on this page · raw message
arch/arm64/kvm/sys_regs.c +1 -23
@@ -2865,34 +2865,12 @@ static void s2_mmu_unmap_ipa(struct kvm_s2_mmu *mmu,* * - NS bit: we're non-secure only. * -* - TTL field: We already have the granule size from the -* VTCR_EL2.TG0 field, and the level is only relevant to the -* guest's S2PT. -* * - IPA[51:48]: We don't support 52bit IPA just yet... * * And of course, adjust the IPA to be on an actual address. */ base_addr = (info->ipa.addr &GENMASK_ULL (35 ,0 )) <<12 ; - -/* Compute the maximum extent of the invalidation */ -switch (mmu->tlb_vtcr &VTCR_EL2_TG0_MASK ) { -case VTCR_EL2_TG0_4K : - max_size =SZ_1G ; -break ; -case VTCR_EL2_TG0_16K : - max_size =SZ_32M ; -break ; -case VTCR_EL2_TG0_64K : -default :/* IMPDEF: treat any other value as 64k */ -/* -* No, we do not support 52bit IPA in nested yet. Once -* we do, this should be 4TB. -*/ - max_size =SZ_512M ; -break ; - } - + max_size =compute_tlb_inval_range (mmu, info->ipa.addr); base_addr &= ~(max_size -1 );kvm_stage2_unmap_range (mmu, base_addr, max_size);
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 12/16 KVM: arm64: nv: Tag shadow S2 entries with guest's leaf S2 level
In reply to Marc ZyngierPopulate bits [56:55] of the leaf entry with the level provided by the guest's S2 translation. This will allow us to better scope the invalidation by remembering the mapping size.
Of course, this assume that the guest will issue an invalidation with an address that falls into the same leaf. If the guest doesn't, we'll over-invalidate.
--- arch/arm64/include/asm/kvm_nested.h | 8 ++++++++ arch/arm64/kvm/mmu.c | 19 +++++++++++++++++-- 2 files changed, 25 insertions(+), 2 deletions(-)
arch/arm64/include/asm/kvm_nested.h +8 -0
@@ -5,6 +5,7 @@#include <linux/bitfield.h> #include <linux/kvm_host.h> #include <asm/kvm_emulate.h> +#include <asm/kvm_pgtable.h> static inline bool vcpu_has_nv (const struct kvm_vcpu *vcpu) {
@@ -195,4 +196,11 @@ static inline bool kvm_auth_eretax(struct kvm_vcpu *vcpu, u64 *elr) }#endif +#define KVM_NV_GUEST_MAP_SZ (KVM_PGTABLE_PROT_SW1 | KVM_PGTABLE_PROT_SW0) + +static inline u64 kvm_encode_nested_level (struct kvm_s2_trans *trans) +{ +return FIELD_PREP (KVM_NV_GUEST_MAP_SZ , trans->level); +} +#endif /* __ARM64_KVM_NESTED_H */
arch/arm64/kvm/mmu.c +17 -2
@@ -1598,11 +1598,19 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,* Potentially reduce shadow S2 permissions to match the guest's own * S2. For exec faults, we'd only reach this point if the guest * actually allowed it (see kvm_s2_handle_perm_fault). +* +* Also encode the level of the original translation in the SW bits +* of the leaf entry as a proxy for the span of that translation. +* This will be retrieved on TLB invalidation from the guest and +* used to limit the invalidation scope if a TTL hint or a range +* isn't provided. */ if (nested) { writable &=kvm_s2_trans_writable (nested);if (!kvm_s2_trans_readable (nested)) prot &= ~KVM_PGTABLE_PROT_R ; + + prot |=kvm_encode_nested_level (nested); }read_lock (&kvm->mmu_lock);
@@ -1661,14 +1669,21 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,* permissions only if vma_pagesize equals fault_granule. Otherwise, * kvm_pgtable_stage2_map() should be called to change block size. */ -if (fault_is_perm && vma_pagesize == fault_granule) +if (fault_is_perm && vma_pagesize == fault_granule) { +/* +* Drop the SW bits in favour of those stored in the +* PTE, which will be preserved. +*/ + prot &= ~KVM_NV_GUEST_MAP_SZ ; ret =kvm_pgtable_stage2_relax_perms (pgt, fault_ipa, prot); -else + }else { ret =kvm_pgtable_stage2_map (pgt, fault_ipa, vma_pagesize,__pfn_to_phys (pfn), prot, memcache,KVM_PGTABLE_WALK_HANDLE_FAULT |KVM_PGTABLE_WALK_SHARED ); + } + out_unlock:read_unlock (&kvm->mmu_lock);
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 13/16 KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like information
In reply to Marc ZyngierIn order to be able to make S2 TLB invalidations more performant on NV, let's use a scheme derived from the FEAT_TTL extension.
If bits [56:55] in the leaf descriptor translating the address in the corresponding shadow S2 are non-zero, they indicate a level which can be used as an invalidation range. This allows further reduction of the systematic over-invalidation that takes place otherwise.
--- arch/arm64/kvm/nested.c | 85 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 84 insertions(+), 1 deletion(-)
arch/arm64/kvm/nested.c +84 -1
@@ -4,6 +4,7 @@* Author: Jintack Lim <[email protected]> */ +#include <linux/bitfield.h> #include <linux/kvm.h> #include <linux/kvm_host.h>
@@ -420,12 +421,94 @@ static unsigned int ttl_to_size(u8 ttl)return max_size; } +/* +* Compute the equivalent of the TTL field by parsing the shadow PT. The +* granule size is extracted from the cached VTCR_EL2.TG0 while the level is +* retrieved from first entry carrying the level as a tag. +*/ +static u8 get_guest_mapping_ttl (struct kvm_s2_mmu *mmu,u64 addr) +{ +u64 tmp, sz =0 , vtcr = mmu->tlb_vtcr; +kvm_pte_t pte; +u8 ttl, level; + +lockdep_assert_held_write (&kvm_s2_mmu_to_kvm (mmu)->mmu_lock); + +switch (vtcr &VTCR_EL2_TG0_MASK ) { +case VTCR_EL2_TG0_4K : + ttl = (TLBI_TTL_TG_4K <<2 ); +break ; +case VTCR_EL2_TG0_16K : + ttl = (TLBI_TTL_TG_16K <<2 ); +break ; +case VTCR_EL2_TG0_64K : +default :/* IMPDEF: treat any other value as 64k */ + ttl = (TLBI_TTL_TG_64K <<2 ); +break ; + } + + tmp = addr; + +again: +/* Iteratively compute the block sizes for a particular granule size */ +switch (vtcr &VTCR_EL2_TG0_MASK ) { +case VTCR_EL2_TG0_4K : +if (sz <SZ_4K ) sz =SZ_4K ; +else if (sz <SZ_2M ) sz =SZ_2M ; +else if (sz <SZ_1G ) sz =SZ_1G ; +else sz =0 ; +break ; +case VTCR_EL2_TG0_16K : +if (sz <SZ_16K ) sz =SZ_16K ; +else if (sz <SZ_32M ) sz =SZ_32M ; +else sz =0 ; +break ; +case VTCR_EL2_TG0_64K : +default :/* IMPDEF: treat any other value as 64k */ +if (sz <SZ_64K ) sz =SZ_64K ; +else if (sz <SZ_512M ) sz =SZ_512M ; +else sz =0 ; +break ; + } + +if (sz ==0 ) +return 0 ; + + tmp &= ~(sz -1 ); +if (kvm_pgtable_get_leaf (mmu->pgt, tmp, &pte,NULL )) + goto again; +if (!(pte &PTE_VALID )) + goto again; + level =FIELD_GET (KVM_NV_GUEST_MAP_SZ , pte); +if (!level) + goto again; + + ttl |= level; + +/* +* We now have found some level information in the shadow S2. Check +* that the resulting range is actually including the original IPA. +*/ + sz =ttl_to_size (ttl); +if (addr < (tmp + sz)) +return ttl; + +return 0 ; +} +unsigned long compute_tlb_inval_range (struct kvm_s2_mmu *mmu,u64 val) { +struct kvm *kvm =kvm_s2_mmu_to_kvm (mmu);unsigned long max_size;u8 ttl; - ttl =FIELD_GET (GENMASK_ULL (47 ,44 ), val); + ttl =FIELD_GET (TLBI_TTL_MASK , val); + +if (!ttl || !kvm_has_feat (kvm,ID_AA64MMFR2_EL1 ,TTL ,IMP )) { +/* No TTL, check the shadow S2 for a hint */ +u64 addr = (val &GENMASK_ULL (35 ,0 )) <<12 ; + ttl =get_guest_mapping_ttl (mmu, addr); + } max_size =ttl_to_size (ttl);
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 14/16 KVM: arm64: nv: Add handling of outer-shareable TLBI operations
In reply to Marc ZyngierOur handling of outer-shareable TLBIs is pretty basic: we just map them to the existing inner-shareable ones, because we really don't have anything else.
The only significant change is that we can now advertise FEAT_TLBIOS support if the host supports it.
--- arch/arm64/kvm/hyp/vhe/tlb.c | 10 ++++++++++ arch/arm64/kvm/nested.c | 5 ++++- arch/arm64/kvm/sys_regs.c | 15 +++++++++++++++ 3 files changed, 29 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/vhe/tlb.c +10 -0
@@ -226,6 +226,7 @@ void __kvm_flush_vm_context(void)* * - a TLBI targeting EL2 S1 is remapped to EL1 S1 * - a non-shareable TLBI is upgraded to being inner-shareable +* - an outer-shareable TLBI is also mapped to inner-shareable */ int __kvm_tlbi_s1e2 (struct kvm_s2_mmu *mmu,u64 va,u64 sys_encoding) {
@@ -245,32 +246,41 @@ int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding)switch (sys_encoding) {case OP_TLBI_ALLE2 :case OP_TLBI_ALLE2IS : +case OP_TLBI_ALLE2OS :case OP_TLBI_VMALLE1 :case OP_TLBI_VMALLE1IS : +case OP_TLBI_VMALLE1OS :__tlbi (vmalle1is);break ;case OP_TLBI_VAE2 :case OP_TLBI_VAE2IS : +case OP_TLBI_VAE2OS :case OP_TLBI_VAE1 :case OP_TLBI_VAE1IS : +case OP_TLBI_VAE1OS :__tlbi (vae1is, va);break ;case OP_TLBI_VALE2 :case OP_TLBI_VALE2IS : +case OP_TLBI_VALE2OS :case OP_TLBI_VALE1 :case OP_TLBI_VALE1IS : +case OP_TLBI_VALE1OS :__tlbi (vale1is, va);break ;case OP_TLBI_ASIDE1 :case OP_TLBI_ASIDE1IS : +case OP_TLBI_ASIDE1OS :__tlbi (aside1is, va);break ;case OP_TLBI_VAAE1 :case OP_TLBI_VAAE1IS : +case OP_TLBI_VAAE1OS :__tlbi (vaae1is, va);break ;case OP_TLBI_VAALE1 :case OP_TLBI_VAALE1IS : +case OP_TLBI_VAALE1OS :__tlbi (vaale1is, va);break ;default :
arch/arm64/kvm/nested.c +4 -1
@@ -805,9 +805,12 @@ static u64 limit_nv_id_reg(u32 id, u64 val)switch (id) {case SYS_ID_AA64ISAR0_EL1 : -/* Support everything but TME, O.S. and Range TLBIs */ +/* Support everything but TME and Range TLBIs */ + tmp =FIELD_GET (NV_FTR (ISAR0 ,TLB ), val); + tmp =min (tmp,ID_AA64ISAR0_EL1_TLB_OS ); val &= ~(NV_FTR (ISAR0 ,TLB ) |NV_FTR (ISAR0 ,TME )); + val |=FIELD_PREP (NV_FTR (ISAR0 ,TLB ), tmp);break ;case SYS_ID_AA64ISAR1_EL1 :
arch/arm64/kvm/sys_regs.c +15 -0
@@ -2959,6 +2959,13 @@ static struct sys_reg_desc sys_insn_descs[] = { { SYS_DESC(SYS_DC_CIGSW), access_dcgsw }, { SYS_DESC(SYS_DC_CIGDSW), access_dcgsw }, +SYS_INSN (TLBI_VMALLE1OS , handle_tlbi_el1), +SYS_INSN (TLBI_VAE1OS , handle_tlbi_el1), +SYS_INSN (TLBI_ASIDE1OS , handle_tlbi_el1), +SYS_INSN (TLBI_VAAE1OS , handle_tlbi_el1), +SYS_INSN (TLBI_VALE1OS , handle_tlbi_el1), +SYS_INSN (TLBI_VAALE1OS , handle_tlbi_el1), +SYS_INSN (TLBI_VMALLE1IS , handle_tlbi_el1),SYS_INSN (TLBI_VAE1IS , handle_tlbi_el1), SYS_INSN(TLBI_ASIDE1IS, handle_tlbi_el1),
@@ -2975,9 +2982,17 @@ static struct sys_reg_desc sys_insn_descs[] = {SYS_INSN (TLBI_IPAS2E1IS , handle_ipas2e1is),SYS_INSN (TLBI_IPAS2LE1IS , handle_ipas2e1is), +SYS_INSN (TLBI_ALLE2OS , trap_undef), +SYS_INSN (TLBI_VAE2OS , trap_undef), +SYS_INSN (TLBI_ALLE1OS , handle_alle1is), +SYS_INSN (TLBI_VALE2OS , trap_undef), +SYS_INSN (TLBI_VMALLS12E1OS , handle_vmalls12e1is), +SYS_INSN (TLBI_ALLE1IS , handle_alle1is),SYS_INSN (TLBI_VMALLS12E1IS , handle_vmalls12e1is), +SYS_INSN (TLBI_IPAS2E1OS , handle_ipas2e1is),SYS_INSN (TLBI_IPAS2E1 , handle_ipas2e1is), +SYS_INSN (TLBI_IPAS2LE1OS , handle_ipas2e1is),SYS_INSN (TLBI_IPAS2LE1 , handle_ipas2e1is),SYS_INSN (TLBI_ALLE1 , handle_alle1is), SYS_INSN(TLBI_VMALLS12E1, handle_vmalls12e1is),
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 15/16 KVM: arm64: nv: Add handling of range-based TLBI operations
In reply to Marc ZyngierWe already support some form of range operation by handling FEAT_TTL, but so far the "arbitrary" range operations are unsupported.
Let's fix that.
For EL2 S1, this is simple enough: we just map both NSH, ISH and OSH instructions onto the ISH version for EL1.
For TLBI instructions affecting EL1 S1, we use the same model as their non-range counterpart to invalidate in the context of the correct VMID.
For TLBI instructions affecting S2, we interpret the data passed by the guest to compute the range and use that to tear-down part of the shadow S2 range and invalidate the TLBs.
Finally, we advertise FEAT_TLBIRANGE if the host supports it.
--- arch/arm64/kvm/hyp/vhe/tlb.c | 26 ++++++++++++ arch/arm64/kvm/nested.c | 8 +--- arch/arm64/kvm/sys_regs.c | 80 ++++++++++++++++++++++++++++++++++++ 3 files changed, 108 insertions(+), 6 deletions(-)
arch/arm64/kvm/hyp/vhe/tlb.c +26 -0
@@ -283,6 +283,32 @@ int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding)case OP_TLBI_VAALE1OS :__tlbi (vaale1is, va);break ; +case OP_TLBI_RVAE2 : +case OP_TLBI_RVAE2IS : +case OP_TLBI_RVAE2OS : +case OP_TLBI_RVAE1 : +case OP_TLBI_RVAE1IS : +case OP_TLBI_RVAE1OS : +__tlbi (rvae1is, va); +break ; +case OP_TLBI_RVALE2 : +case OP_TLBI_RVALE2IS : +case OP_TLBI_RVALE2OS : +case OP_TLBI_RVALE1 : +case OP_TLBI_RVALE1IS : +case OP_TLBI_RVALE1OS : +__tlbi (rvale1is, va); +break ; +case OP_TLBI_RVAAE1 : +case OP_TLBI_RVAAE1IS : +case OP_TLBI_RVAAE1OS : +__tlbi (rvaae1is, va); +break ; +case OP_TLBI_RVAALE1 : +case OP_TLBI_RVAALE1IS : +case OP_TLBI_RVAALE1OS : +__tlbi (rvaale1is, va); +break ;default : ret = -EINVAL ; }
arch/arm64/kvm/nested.c +2 -6
@@ -805,12 +805,8 @@ static u64 limit_nv_id_reg(u32 id, u64 val)switch (id) {case SYS_ID_AA64ISAR0_EL1 : -/* Support everything but TME and Range TLBIs */ - tmp =FIELD_GET (NV_FTR (ISAR0 ,TLB ), val); - tmp =min (tmp,ID_AA64ISAR0_EL1_TLB_OS ); - val &= ~(NV_FTR (ISAR0 ,TLB ) | -NV_FTR (ISAR0 ,TME )); - val |=FIELD_PREP (NV_FTR (ISAR0 ,TLB ), tmp); +/* Support everything but TME */ + val &= ~NV_FTR (ISAR0 ,TME );break ;case SYS_ID_AA64ISAR1_EL1 :
arch/arm64/kvm/sys_regs.c +80 -0
117 lines not on this page · raw message
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH v3 16/16 KVM: arm64: nv: Add handling of NXS-flavoured TLBI operations
In reply to Marc ZyngierLatest kid on the block: NXS (Non-eXtra-Slow) TLBI operations.
Let's add those in bulk (NSH, ISH, OSH, both normal and range) as they directly map to their XS (the standard ones) counterparts.
Not a lot to say about them, they are basically useless.
--- arch/arm64/kvm/hyp/vhe/tlb.c | 46 +++++++++++++++++++++++ arch/arm64/kvm/sys_regs.c | 73 ++++++++++++++++++++++++++++++++++++ 2 files changed, 119 insertions(+)
arch/arm64/kvm/hyp/vhe/tlb.c +46 -0
@@ -227,6 +227,7 @@ void __kvm_flush_vm_context(void)* - a TLBI targeting EL2 S1 is remapped to EL1 S1 * - a non-shareable TLBI is upgraded to being inner-shareable * - an outer-shareable TLBI is also mapped to inner-shareable +* - an nXS TLBI is upgraded to XS */ int __kvm_tlbi_s1e2 (struct kvm_s2_mmu *mmu,u64 va,u64 sys_encoding) {
@@ -250,6 +251,12 @@ int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding)case OP_TLBI_VMALLE1 :case OP_TLBI_VMALLE1IS :case OP_TLBI_VMALLE1OS : +case OP_TLBI_ALLE2NXS : +case OP_TLBI_ALLE2ISNXS : +case OP_TLBI_ALLE2OSNXS : +case OP_TLBI_VMALLE1NXS : +case OP_TLBI_VMALLE1ISNXS : +case OP_TLBI_VMALLE1OSNXS :__tlbi (vmalle1is);break ;case OP_TLBI_VAE2 :
@@ -258,6 +265,12 @@ int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding)case OP_TLBI_VAE1 :case OP_TLBI_VAE1IS :case OP_TLBI_VAE1OS : +case OP_TLBI_VAE2NXS : +case OP_TLBI_VAE2ISNXS : +case OP_TLBI_VAE2OSNXS : +case OP_TLBI_VAE1NXS : +case OP_TLBI_VAE1ISNXS : +case OP_TLBI_VAE1OSNXS :__tlbi (vae1is, va);break ;case OP_TLBI_VALE2 :
@@ -266,21 +279,36 @@ int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding)case OP_TLBI_VALE1 :case OP_TLBI_VALE1IS :case OP_TLBI_VALE1OS : +case OP_TLBI_VALE2NXS : +case OP_TLBI_VALE2ISNXS : +case OP_TLBI_VALE2OSNXS : +case OP_TLBI_VALE1NXS : +case OP_TLBI_VALE1ISNXS : +case OP_TLBI_VALE1OSNXS :__tlbi (vale1is, va);break ;case OP_TLBI_ASIDE1 :case OP_TLBI_ASIDE1IS :case OP_TLBI_ASIDE1OS : +case OP_TLBI_ASIDE1NXS : +case OP_TLBI_ASIDE1ISNXS : +case OP_TLBI_ASIDE1OSNXS :__tlbi (aside1is, va);break ;case OP_TLBI_VAAE1 :case OP_TLBI_VAAE1IS :case OP_TLBI_VAAE1OS : +case OP_TLBI_VAAE1NXS : +case OP_TLBI_VAAE1ISNXS : +case OP_TLBI_VAAE1OSNXS :__tlbi (vaae1is, va);break ;case OP_TLBI_VAALE1 :case OP_TLBI_VAALE1IS :case OP_TLBI_VAALE1OS : +case OP_TLBI_VAALE1NXS : +case OP_TLBI_VAALE1ISNXS : +case OP_TLBI_VAALE1OSNXS :__tlbi (vaale1is, va);break ;case OP_TLBI_RVAE2 :
@@ -289,6 +317,12 @@ int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding)case OP_TLBI_RVAE1 :case OP_TLBI_RVAE1IS :case OP_TLBI_RVAE1OS : +case OP_TLBI_RVAE2NXS : +case OP_TLBI_RVAE2ISNXS : +case OP_TLBI_RVAE2OSNXS : +case OP_TLBI_RVAE1NXS : +case OP_TLBI_RVAE1ISNXS : +case OP_TLBI_RVAE1OSNXS :__tlbi (rvae1is, va);break ;case OP_TLBI_RVALE2 :
@@ -297,16 +331,28 @@ int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding)case OP_TLBI_RVALE1 :case OP_TLBI_RVALE1IS :case OP_TLBI_RVALE1OS : +case OP_TLBI_RVALE2NXS : +case OP_TLBI_RVALE2ISNXS : +case OP_TLBI_RVALE2OSNXS : +case OP_TLBI_RVALE1NXS : +case OP_TLBI_RVALE1ISNXS : +case OP_TLBI_RVALE1OSNXS :__tlbi (rvale1is, va);break ;case OP_TLBI_RVAAE1 :case OP_TLBI_RVAAE1IS :case OP_TLBI_RVAAE1OS : +case OP_TLBI_RVAAE1NXS : +case OP_TLBI_RVAAE1ISNXS : +case OP_TLBI_RVAAE1OSNXS :__tlbi (rvaae1is, va);break ;case OP_TLBI_RVAALE1 :case OP_TLBI_RVAALE1IS :case OP_TLBI_RVAALE1OS : +case OP_TLBI_RVAALE1NXS : +case OP_TLBI_RVAALE1ISNXS : +case OP_TLBI_RVAALE1OSNXS :__tlbi (rvaale1is, va);break ;default :
arch/arm64/kvm/sys_regs.c +73 -0
85 lines not on this page · raw message
Signature · 1 line
2.39.2
Oliver Upton
AppliedQuoted 9 lines · Marc Zyngier
Here's the thurd version of the shadow stage-2 handling for NV support on arm64.
* From v2 [2]
- Simplified the S2 walker by dropping a bunch of redundant fields from the walker info structure
[...]
Applied to kvmarm/next, thanks!
[01/16] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
https://git.kernel.org/kvmarm/kvmarm/c/4f128f8e1aaa [02/16] KVM: arm64: nv: Implement nested Stage-2 page table walk logic https://git.kernel.org/kvmarm/kvmarm/c/61e30b9eef7f [03/16] KVM: arm64: nv: Handle shadow stage 2 page faults https://git.kernel.org/kvmarm/kvmarm/c/fd276e71d1e7 [04/16] KVM: arm64: nv: Unmap/flush shadow stage 2 page tables https://git.kernel.org/kvmarm/kvmarm/c/ec14c272408a [05/16] KVM: arm64: nv: Add Stage-1 EL2 invalidation primitives https://git.kernel.org/kvmarm/kvmarm/c/82e86326ec58 [06/16] KVM: arm64: nv: Handle EL2 Stage-1 TLB invalidation https://git.kernel.org/kvmarm/kvmarm/c/67fda56e76da [07/16] KVM: arm64: nv: Handle TLB invalidation targeting L2 stage-1 https://git.kernel.org/kvmarm/kvmarm/c/8e236efa4cd2 [08/16] KVM: arm64: nv: Handle TLBI VMALLS12E1{,IS} operations https://git.kernel.org/kvmarm/kvmarm/c/e6c9a3015ff2 [09/16] KVM: arm64: nv: Handle TLBI ALLE1{,IS} operations https://git.kernel.org/kvmarm/kvmarm/c/5cfb6cec62f2 [10/16] KVM: arm64: nv: Handle TLBI IPAS2E1{,IS} operations https://git.kernel.org/kvmarm/kvmarm/c/70109bcd701e [11/16] KVM: arm64: nv: Handle FEAT_TTL hinted TLB operations https://git.kernel.org/kvmarm/kvmarm/c/d1de1576dc21 [12/16] KVM: arm64: nv: Tag shadow S2 entries with guest's leaf S2 level https://git.kernel.org/kvmarm/kvmarm/c/b1a3a94812b9 [13/16] KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like information https://git.kernel.org/kvmarm/kvmarm/c/809b2e6013a5 [14/16] KVM: arm64: nv: Add handling of outer-shareable TLBI operations https://git.kernel.org/kvmarm/kvmarm/c/0cb8aae22676 [15/16] KVM: arm64: nv: Add handling of range-based TLBI operations https://git.kernel.org/kvmarm/kvmarm/c/5d476ca57d7d [16/16] KVM: arm64: nv: Add handling of NXS-flavoured TLBI operations https://git.kernel.org/kvmarm/kvmarm/c/0feec7769a63
--
Best,
Oliver
Hi Marc,
Quoted 46 lines · Oliver Upton
Quoted 9 lines · Marc Zyngier
Here's the thurd version of the shadow stage-2 handling for NV support on arm64.
* From v2 [2]
- Simplified the S2 walker by dropping a bunch of redundant fields from the walker info structure
[...]
Applied to kvmarm/next, thanks!
[01/16] KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
https://git.kernel.org/kvmarm/kvmarm/c/4f128f8e1aaa [02/16] KVM: arm64: nv: Implement nested Stage-2 page table walk logic https://git.kernel.org/kvmarm/kvmarm/c/61e30b9eef7f [03/16] KVM: arm64: nv: Handle shadow stage 2 page faults https://git.kernel.org/kvmarm/kvmarm/c/fd276e71d1e7 [04/16] KVM: arm64: nv: Unmap/flush shadow stage 2 page tables https://git.kernel.org/kvmarm/kvmarm/c/ec14c272408a [05/16] KVM: arm64: nv: Add Stage-1 EL2 invalidation primitives https://git.kernel.org/kvmarm/kvmarm/c/82e86326ec58 [06/16] KVM: arm64: nv: Handle EL2 Stage-1 TLB invalidation https://git.kernel.org/kvmarm/kvmarm/c/67fda56e76da [07/16] KVM: arm64: nv: Handle TLB invalidation targeting L2 stage-1 https://git.kernel.org/kvmarm/kvmarm/c/8e236efa4cd2 [08/16] KVM: arm64: nv: Handle TLBI VMALLS12E1{,IS} operations https://git.kernel.org/kvmarm/kvmarm/c/e6c9a3015ff2 [09/16] KVM: arm64: nv: Handle TLBI ALLE1{,IS} operations https://git.kernel.org/kvmarm/kvmarm/c/5cfb6cec62f2 [10/16] KVM: arm64: nv: Handle TLBI IPAS2E1{,IS} operations https://git.kernel.org/kvmarm/kvmarm/c/70109bcd701e [11/16] KVM: arm64: nv: Handle FEAT_TTL hinted TLB operations https://git.kernel.org/kvmarm/kvmarm/c/d1de1576dc21 [12/16] KVM: arm64: nv: Tag shadow S2 entries with guest's leaf S2 level https://git.kernel.org/kvmarm/kvmarm/c/b1a3a94812b9 [13/16] KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like information https://git.kernel.org/kvmarm/kvmarm/c/809b2e6013a5 [14/16] KVM: arm64: nv: Add handling of outer-shareable TLBI operations https://git.kernel.org/kvmarm/kvmarm/c/0cb8aae22676 [15/16] KVM: arm64: nv: Add handling of range-based TLBI operations https://git.kernel.org/kvmarm/kvmarm/c/5d476ca57d7d [16/16] KVM: arm64: nv: Add handling of NXS-flavoured TLBI operations https://git.kernel.org/kvmarm/kvmarm/c/0feec7769a63
Quoted 3 lines
--
Best,
Oliver
IIRC, Most of the patches that are specific to NV have been merged upstream. However I do see that, some of the vGIC and Timer related patches are still in your private NV repository. Can these patches be prioritized to upstream, so that we can have have the first working version of NV on mainline.
Signature · 2 lines
Thanks, Ganapat/GK
Marc Zyngier
maintainerOn Thu, 21 Nov 2024 08:11:00 +0000,
Ganapatrao Kulkarni <[email protected]> wrote:
Hi Ganapatrao,
Quoted 5 lines
IIRC, Most of the patches that are specific to NV have been merged upstream. However I do see that, some of the vGIC and Timer related patches are still in your private NV repository. Can these patches be prioritized to upstream, so that we can have have the first working version of NV on mainline.
Who is *we*?
Things get upstreamed when we (people doing the actual work) decide they are ready. At the moment, they are not.
Also, while I enjoy working on NV, this isn't *my* priority.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 11 lines · Marc Zyngier
On Thu, 21 Nov 2024 08:11:00 +0000,
Ganapatrao Kulkarni <[email protected]> wrote:Hi Ganapatrao,
Quoted 5 lines
IIRC, Most of the patches that are specific to NV have been merged upstream. However I do see that, some of the vGIC and Timer related patches are still in your private NV repository. Can these patches be prioritized to upstream, so that we can have have the first working version of NV on mainline.
Quoted 6 lines
Who is *we*?
Things get upstreamed when we (people doing the actual work) decide they are ready. At the moment, they are not.
Also, while I enjoy working on NV, this isn't *my* priority.
Sure, I understand that it's not your priority right now. I'm happy to spend some time on it, please do let us know in what areas/patches needs attention before the code would be ready to merge?
Signature · 2 lines
Thanks, Ganapat/GK
Marc Zyngier
maintainerQuoted 17 lines · Ganapatrao Kulkarni
Quoted 13 lines · Marc Zyngier
On Thu, 21 Nov 2024 08:11:00 +0000,
Ganapatrao Kulkarni <[email protected]> wrote:Hi Ganapatrao,
Quoted 5 lines
IIRC, Most of the patches that are specific to NV have been merged upstream. However I do see that, some of the vGIC and Timer related patches are still in your private NV repository. Can these patches be prioritized to upstream, so that we can have have the first working version of NV on mainline.
Who is *we*?
Quoted 8 lines
Quoted 4 lines
Things get upstreamed when we (people doing the actual work) decide they are ready. At the moment, they are not.
Also, while I enjoy working on NV, this isn't *my* priority.
Sure, I understand that it's not your priority right now. I'm happy to spend some time on it, please do let us know in what areas/patches needs attention before the code would be ready to merge?
Please understand that NV isn't special, and while there is still a bunch of things that need to be merged, it is the whole of KVM/arm64 that needs attention.
For example, there is the debug series from Oliver, the feature handling from Fuad. They may not have NV written all over them, but they do have an impact on the NV behaviour one way or another.
By paying attention to these series, you would help with the groundwork that is required before we can actually enable NV. This is what matters now, not the next 50 or so NV-specific patches.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerQuoted 33 lines · Marc Zyngier
Quoted 17 lines · Ganapatrao Kulkarni
Quoted 13 lines · Marc Zyngier
On Thu, 21 Nov 2024 08:11:00 +0000,
Ganapatrao Kulkarni <[email protected]> wrote:Hi Ganapatrao,
Quoted 5 lines
IIRC, Most of the patches that are specific to NV have been merged upstream. However I do see that, some of the vGIC and Timer related patches are still in your private NV repository. Can these patches be prioritized to upstream, so that we can have have the first working version of NV on mainline.
Who is *we*?
Quoted 8 lines
Quoted 4 lines
Things get upstreamed when we (people doing the actual work) decide they are ready. At the moment, they are not.
Also, while I enjoy working on NV, this isn't *my* priority.
Sure, I understand that it's not your priority right now. I'm happy to spend some time on it, please do let us know in what areas/patches needs attention before the code would be ready to merge?
Please understand that NV isn't special, and while there is still a bunch of things that need to be merged, it is the whole of KVM/arm64 that needs attention.
Quoted 7 lines
For example, there is the debug series from Oliver, the feature handling from Fuad. They may not have NV written all over them, but they do have an impact on the NV behaviour one way or another.
By paying attention to these series, you would help with the groundwork that is required before we can actually enable NV. This is what matters now, not the next 50 or so NV-specific patches.
And one last thing, while I think of it: NV is very unlikely to get merged without a testing infrastructure. Which means that the current selftests must run at EL2, and that *new* selftests must be created to test the NV implementation (all the trap behaviours, for example).
So if you (and I assume your employer, as you keep using the plural) want to help moving NV support upstream, this is an area where you could help and make a massive difference.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 43 lines · Marc Zyngier
Quoted 33 lines · Marc Zyngier
Quoted 17 lines · Ganapatrao Kulkarni
Quoted 13 lines · Marc Zyngier
On Thu, 21 Nov 2024 08:11:00 +0000,
Ganapatrao Kulkarni <[email protected]> wrote:Hi Ganapatrao,
Quoted 5 lines
IIRC, Most of the patches that are specific to NV have been merged upstream. However I do see that, some of the vGIC and Timer related patches are still in your private NV repository. Can these patches be prioritized to upstream, so that we can have have the first working version of NV on mainline.
Who is *we*?
Quoted 8 lines
Quoted 4 lines
Things get upstreamed when we (people doing the actual work) decide they are ready. At the moment, they are not.
Also, while I enjoy working on NV, this isn't *my* priority.
Sure, I understand that it's not your priority right now. I'm happy to spend some time on it, please do let us know in what areas/patches needs attention before the code would be ready to merge?
Please understand that NV isn't special, and while there is still a bunch of things that need to be merged, it is the whole of KVM/arm64 that needs attention.
Quoted 7 lines
For example, there is the debug series from Oliver, the feature handling from Fuad. They may not have NV written all over them, but they do have an impact on the NV behaviour one way or another.
By paying attention to these series, you would help with the groundwork that is required before we can actually enable NV. This is what matters now, not the next 50 or so NV-specific patches.
Quoted 8 lines
And one last thing, while I think of it: NV is very unlikely to get merged without a testing infrastructure. Which means that the current selftests must run at EL2, and that *new* selftests must be created to test the NV implementation (all the trap behaviours, for example).
So if you (and I assume your employer, as you keep using the plural) want to help moving NV support upstream, this is an area where you could help and make a massive difference.
Appreciate the specific examples Marc, thank you. I'll work with my team at Ampere in this direction - specifically the testing infrastructure work and engaging in the NV-adjacent kvm patche series.
Signature · 2 lines
Darren Hart Ampere Computing / Linux Enabling