KVM: arm64: Allow using VHE in the nVHE hypervisor
Subsystems (6)
Marc Zyngier
maintainerv2 cover KVM: arm64: Allow using VHE in the nVHE hypervisor
KVM (on ARMv8.0) and pKVM (on all revisions of the architecture) use the split hypervisor model that makes the EL2 code more or less standalone. In the later case, we totally ignore the VHE mode and stick with the good old v8.0 EL2 setup.
This is all good, but means that the EL2 code is limited in what it can do with its own address space. This series proposes to remove this limitation and to allow VHE to be used even with the split hypervisor model. This has some potential isolation benefits[1], and eventually allow systems that do not support HCR_EL2.E2H==0 to eventually run pKVM.
We introduce a new "mode" for KVM called hVHE, in reference to the nVHE mode, and indicating that only the hypervisor is using VHE. Note that this is all this series does. No effort is made to improve the VA space management, which will be the subject of another series if this one ever makes it.
This has been tested on a M1 box (bare metal) as well as as a nested guest on M2, both with the standard nVHE and protected modes, with no measurable change in performance.
Note: the last patch of this series is not a merge candidate.
Thanks,
M.
[1] https://www.youtube.com/watch?v=1F_Mf2j9eIo&list=PLbzoR-pLrL6qWL3v2KOcvwZ54-w0z5uXV&index=11
* From v1:
- Fixed CNTHCTL_EL2 setup when switching from E2H=0 to E2H=1 Amusingly, this was found on NV...
- Rebased on 6.4-rc2
Marc Zyngier (17):
KVM: arm64: Drop is_kernel_in_hyp_mode() from
__invalidate_icache_guest_page()
arm64: Prevent the use of is_kernel_in_hyp_mode() in hypervisor code
arm64: Turn kaslr_feature_override into a generic SW feature override
arm64: Add KVM_HVHE capability and has_hvhe() predicate
arm64: Don't enable VHE for the kernel if OVERRIDE_HVHE is set
arm64: Allow EL1 physical timer access when running VHE
arm64: Use CPACR_EL1 format to set CPTR_EL2 when E2H is set
KVM: arm64: Remove alternatives from sysreg accessors in VHE
hypervisor context
KVM: arm64: Key use of VHE instructions in nVHE code off
ARM64_KVM_HVHE
KVM: arm64: Force HCR_EL2.E2H when ARM64_KVM_HVHE is set
KVM: arm64: Disable TTBR1_EL2 when using ARM64_KVM_HVHE
KVM: arm64: Adjust EL2 stage-1 leaf AP bits when ARM64_KVM_HVHE is set
KVM: arm64: Rework CPTR_EL2 programming for HVHE configuration
KVM: arm64: Program the timer traps with VHE layout in hVHE mode
KVM: arm64: Force HCR_E2H in guest context when ARM64_KVM_HVHE is set
arm64: Allow arm64_sw.hvhe on command line
KVM: arm64: Terrible timer hack for M1 with hVHE
arch/arm64/include/asm/arch_timer.h | 8 ++++ arch/arm64/include/asm/cpufeature.h | 5 +++ arch/arm64/include/asm/el2_setup.h | 26 ++++++++++++- arch/arm64/include/asm/kvm_arm.h | 4 +- arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/include/asm/kvm_emulate.h | 33 +++++++++++++++- arch/arm64/include/asm/kvm_hyp.h | 37 +++++++++++++----- arch/arm64/include/asm/kvm_mmu.h | 3 +- arch/arm64/include/asm/virt.h | 15 +++++++- arch/arm64/kernel/cpufeature.c | 17 +++++++++ arch/arm64/kernel/hyp-stub.S | 10 ++++- arch/arm64/kernel/idreg-override.c | 25 ++++++++----- arch/arm64/kernel/image-vars.h | 3 ++ arch/arm64/kernel/kaslr.c | 6 +-- arch/arm64/kvm/arch_timer.c | 5 +++ arch/arm64/kvm/arm.c | 12 +++++- arch/arm64/kvm/fpsimd.c | 4 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-init.S | 9 +++++ arch/arm64/kvm/hyp/nvhe/hyp-main.c | 17 ++++++++- arch/arm64/kvm/hyp/nvhe/pkvm.c | 27 ++++++++++--- arch/arm64/kvm/hyp/nvhe/switch.c | 28 ++++++++------ arch/arm64/kvm/hyp/nvhe/timer-sr.c | 25 +++++++++++-- arch/arm64/kvm/hyp/pgtable.c | 6 ++- arch/arm64/kvm/hyp/vhe/switch.c | 2 +- arch/arm64/kvm/sys_regs.c | 2 +- arch/arm64/tools/cpucaps | 1 + drivers/irqchip/irq-apple-aic.c | 50 ++++++++++++++++++++++++- 28 files changed, 319 insertions(+), 64 deletions(-)
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 1/17 KVM: arm64: Drop is_kernel_in_hyp_mode() from __invalidate_icache_guest_page()
It is pretty obvious that is_kernel_in_hyp_mode() doesn't make much sense in the hypervisor part of KVM, and should be reserved to the kernel side.
However, mem_protect.c::invalidate_icache_guest_page() calls into __invalidate_icache_guest_page(), which uses is_kernel_in_hyp_mode(). Given that this is part of the pKVM side of the hypervisor, this helper can only return true.
Nothing goes really bad, but __invalidate_icache_guest_page() could spell out what the actual check is: we cannot invalidate the cache if the i-cache is VPIPT and we're running at EL1.
Drop the is_kernel_in_hyp_mode() check for an explicit check against CurrentEL being EL1 or not.
--- arch/arm64/include/asm/kvm_mmu.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/arm64/include/asm/kvm_mmu.h +2 -1
@@ -227,7 +227,8 @@ static inline void __invalidate_icache_guest_page(void *va, size_t size)if (icache_is_aliasing ()) {/* any kind of VIPT cache */ icache_inval_all_pou (); - }else if (is_kernel_in_hyp_mode () || !icache_is_vpipt ()) { + }else if (read_sysreg (CurrentEL) != CurrentEL_EL1 || + !icache_is_vpipt ()) {/* PIPT or VPIPT at EL2 (see comment in __kvm_tlb_flush_vmid_ipa) */ icache_inval_pou ((unsigned long )va, (unsigned long )va + size); }
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 2/17 arm64: Prevent the use of is_kernel_in_hyp_mode() in hypervisor code
In reply to Marc ZyngierUsing is_kernel_in_hyp_mode() in hypervisor code is a pretty bad mistake. This helper only checks for CurrentEL being EL2, which is always true.
Make the link fail if using the helper in hypervisor context by referencing a non-existent function. Whilst we're at it, flag the helper as __always_inline, which it really should be.
--- arch/arm64/include/asm/virt.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
arch/arm64/include/asm/virt.h +6 -1
@@ -110,8 +110,13 @@ static inline bool is_hyp_mode_mismatched(void)return __boot_cpu_mode[0 ] != __boot_cpu_mode[1 ]; } -static inline bool is_kernel_in_hyp_mode (void ) +extern void gotcha_is_kernel_in_hyp_mode (void ); + +static __always_inline bool is_kernel_in_hyp_mode(void) { +#if defined(__KVM_NVHE_HYPERVISOR__) || defined(__KVM_VHE_HYPERVISOR__) +gotcha_is_kernel_in_hyp_mode (); +#endif return read_sysreg (CurrentEL) == CurrentEL_EL2; }
Signature · 1 line
2.34.1
Re: [PATCH v2 02/17] arm64: Prevent the use of is_kernel_in_hyp_mode() in hypervisor code
Hi Marc,
Quoted 21 lines · Marc Zyngier
Using
is_kernel_in_hyp_mode()in hypervisor code is a pretty bad mistake. This helper only checks for CurrentEL being EL2, which is always true.Make the link fail if using the helper in hypervisor context by referencing a non-existent function. Whilst we're at it, flag the helper as
__always_inline, which it really should be.Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/virt.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)arch/arm64/include/asm/virt.h +0
-0@@ -110,8 +110,13 @@ static inline bool is_hyp_mode_mismatched(void)return __boot_cpu_mode[0 ] != __boot_cpu_mode[1 ]; }
arch/arm64/include/asm/virt.h static inline bool is_hyp_mode_mismatched(void) · line 121
Quoted 10 lines
arch/arm64/include/asm/virt.h +6
-1-static inline bool is_kernel_in_hyp_mode (void ) +extern void gotcha_is_kernel_in_hyp_mode (void ); + +static __always_inline bool is_kernel_in_hyp_mode(void) { +#if defined(__KVM_NVHE_HYPERVISOR__) || defined(__KVM_VHE_HYPERVISOR__) +gotcha_is_kernel_in_hyp_mode (); +#endif return read_sysreg (CurrentEL) == CurrentEL_EL2; }
Would BUILD_BUG() work in this context, or have I missed something?
Signature · 2 lines
Thanks, Oliver
Marc Zyngier
maintainerRe: [PATCH v2 02/17] arm64: Prevent the use of is_kernel_in_hyp_mode() in hypervisor code
Quoted 25 lines · Oliver Upton
Hi Marc,
Quoted 21 lines · Marc Zyngier
Using
is_kernel_in_hyp_mode()in hypervisor code is a pretty bad mistake. This helper only checks for CurrentEL being EL2, which is always true.Make the link fail if using the helper in hypervisor context by referencing a non-existent function. Whilst we're at it, flag the helper as
__always_inline, which it really should be.Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/virt.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)arch/arm64/include/asm/virt.h +0
-0@@ -110,8 +110,13 @@ static inline bool is_hyp_mode_mismatched(void)return __boot_cpu_mode[0 ] != __boot_cpu_mode[1 ]; }
Quoted 12 lines
arch/arm64/include/asm/virt.h static inline bool is_hyp_mode_mismatched(void) · line 121
Quoted 10 lines
arch/arm64/include/asm/virt.h +6
-1-static inline bool is_kernel_in_hyp_mode (void ) +extern void gotcha_is_kernel_in_hyp_mode (void ); + +static __always_inline bool is_kernel_in_hyp_mode(void) { +#if defined(__KVM_NVHE_HYPERVISOR__) || defined(__KVM_VHE_HYPERVISOR__) +gotcha_is_kernel_in_hyp_mode (); +#endif return read_sysreg (CurrentEL) == CurrentEL_EL2; }Would
BUILD_BUG()work in this context, or have I missed something?
Too obvious? :-) I'll fix that.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerPATCH v2 3/17 arm64: Turn kaslr_feature_override into a generic SW feature override
In reply to Marc ZyngierDisabling KASLR from the command line is implemented as a feature override. Repaint it slightly so that it can further be used as more generic infrastructure for SW override purposes.
--- arch/arm64/include/asm/cpufeature.h | 4 ++++ arch/arm64/kernel/cpufeature.c | 2 ++ arch/arm64/kernel/idreg-override.c | 16 ++++++---------- arch/arm64/kernel/kaslr.c | 6 +++--- 4 files changed, 15 insertions(+), 13 deletions(-)
arch/arm64/include/asm/cpufeature.h +4 -0
@@ -15,6 +15,8 @@#define MAX_CPU_FEATURES 128#define cpu_feature (x) KERNEL_HWCAP_ ## x +#define ARM64_SW_FEATURE_OVERRIDE_NOKASLR 0 +#ifndef __ASSEMBLY__#include <linux/bug.h>
@@ -925,6 +927,8 @@ extern struct arm64_ftr_override id_aa64smfr0_override;extern struct arm64_ftr_override id_aa64isar1_override;extern struct arm64_ftr_override id_aa64isar2_override; +extern struct arm64_ftr_override arm64_sw_feature_override; +u32 get_kvm_ipa_limit (void );void dump_cpu_features (void );
arch/arm64/kernel/cpufeature.c +2 -0
@@ -664,6 +664,8 @@ struct arm64_ftr_override __ro_after_init id_aa64smfr0_override;struct arm64_ftr_override __ro_after_init id_aa64isar1_override;struct arm64_ftr_override __ro_after_init id_aa64isar2_override; +struct arm64_ftr_override arm64_sw_feature_override; + static const struct __ftr_reg_entry { u32 sys_id; struct arm64_ftr_reg *reg;
arch/arm64/kernel/idreg-override.c +6 -10
@@ -138,15 +138,11 @@ static const struct ftr_set_desc smfr0 __initconst = { }, }; -extern struct arm64_ftr_override kaslr_feature_override; - -static const struct ftr_set_desc kaslr__initconst = { - .name ="kaslr" , -#ifdef CONFIG_RANDOMIZE_BASE - .override = &kaslr_feature_override, -#endif +static const struct ftr_set_desc sw_features__initconst = { + .name ="arm64_sw" , + .override = &arm64_sw_feature_override, .fields = { -FIELD ("disabled" ,0 ,NULL ), +FIELD ("nokaslr" ,ARM64_SW_FEATURE_OVERRIDE_NOKASLR ,NULL ), {} }, };
@@ -158,7 +154,7 @@ static const struct ftr_set_desc * const regs[] __initconst = { &isar1, &isar2, &smfr0, - &kaslr, + &sw_features, }; static const struct {
@@ -175,7 +171,7 @@ static const struct { "id_aa64isar1.api=0 id_aa64isar1.apa=0 " "id_aa64isar2.gpa3=0 id_aa64isar2.apa3=0" }, { "arm64.nomte", "id_aa64pfr1.mte=0" }, - { "nokaslr", "kaslr.disabled=1" }, + { "nokaslr", "arm64_sw.nokaslr=1" }, };static int __init parse_nokaslr (char *unused)
arch/arm64/kernel/kaslr.c +3 -3
@@ -23,8 +23,6 @@u64 __ro_after_init module_alloc_base;u16 __initdata memstart_offset_seed; -struct arm64_ftr_override kaslr_feature_override__initdata ; -static int __init kaslr_init (void ) {u64 module_range;
@@ -36,7 +34,9 @@ static int __init kaslr_init(void)*/ module_alloc_base = (u64 )_etext -MODULES_VSIZE ; -if (kaslr_feature_override.val & kaslr_feature_override.mask &0xf ) { +if (cpuid_feature_extract_unsigned_field (arm64_sw_feature_override.val & + arm64_sw_feature_override.mask, +ARM64_SW_FEATURE_OVERRIDE_NOKASLR )) {pr_info ("KASLR disabled on command line\n" );return 0 ; }
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 4/17 arm64: Add KVM_HVHE capability and has_hvhe() predicate
In reply to Marc ZyngierExpose a capability keying the hVHE feature as well as a new predicate testing it. Nothing is so far using it, and nothing is enabling it yet.
--- arch/arm64/include/asm/cpufeature.h | 1 + arch/arm64/include/asm/virt.h | 8 ++++++++ arch/arm64/kernel/cpufeature.c | 15 +++++++++++++++ arch/arm64/tools/cpucaps | 1 + 4 files changed, 25 insertions(+)
arch/arm64/include/asm/cpufeature.h +1 -0
@@ -16,6 +16,7 @@#define cpu_feature (x) KERNEL_HWCAP_ ## x#define ARM64_SW_FEATURE_OVERRIDE_NOKASLR 0 +#define ARM64_SW_FEATURE_OVERRIDE_HVHE 4#ifndef __ASSEMBLY__
arch/arm64/include/asm/virt.h +8 -0
@@ -145,6 +145,14 @@ static __always_inline bool is_protected_kvm_enabled(void)return cpus_have_final_cap (ARM64_KVM_PROTECTED_MODE ); } +static __always_inline bool has_hvhe(void) +{ +if (is_vhe_hyp_code ()) +return false; + +return cpus_have_final_cap (ARM64_KVM_HVHE ); +} +static inline bool is_hyp_nvhe (void ) {return is_hyp_mode_available () && !is_kernel_in_hyp_mode ();
arch/arm64/kernel/cpufeature.c +15 -0
@@ -1998,6 +1998,15 @@ static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap,return true; } +static bool hvhe_possible (const struct arm64_cpu_capabilities *entry, +int __unused) +{ +u64 val; + + val = arm64_sw_feature_override.val & arm64_sw_feature_override.mask; +return cpuid_feature_extract_unsigned_field (val,ARM64_SW_FEATURE_OVERRIDE_HVHE ); +} +#ifdef CONFIG_ARM64_PAN static void cpu_enable_pan (const struct arm64_cpu_capabilities *__unused) {
@@ -2643,6 +2652,12 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .cpu_enable = cpu_enable_dit, ARM64_CPUID_FIELDS(ID_AA64PFR0_EL1, DIT, IMP) }, + { + .desc = "VHE for hypervisor only", + .capability = ARM64_KVM_HVHE, + .type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE, + .matches = hvhe_possible, + }, {}, };
arch/arm64/tools/cpucaps +1 -0
@@ -47,6 +47,7 @@ HAS_TLB_RANGE HAS_VIRT_HOST_EXTN HAS_WFXT HW_DBM +KVM_HVHE KVM_PROTECTED_MODE MISMATCHED_CACHE_TYPE MTE
Signature · 1 line
2.34.1
Re: [PATCH v2 04/17] arm64: Add KVM_HVHE capability and has_hvhe() predicate
Hey Marc,
I'm an idiot and was responding to v1. Here's the same damn comment, but on v2!
Quoted 49 lines · Marc Zyngier
Expose a capability keying the hVHE feature as well as a new predicate testing it. Nothing is so far using it, and nothing is enabling it yet.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/cpufeature.h | 1 + arch/arm64/include/asm/virt.h | 8 ++++++++ arch/arm64/kernel/cpufeature.c | 15 +++++++++++++++ arch/arm64/tools/cpucaps | 1 + 4 files changed, 25 insertions(+)arch/arm64/include/asm/cpufeature.h +1
-0@@ -16,6 +16,7 @@#define cpu_feature (x) KERNEL_HWCAP_ ## x#define ARM64_SW_FEATURE_OVERRIDE_NOKASLR 0 +#define ARM64_SW_FEATURE_OVERRIDE_HVHE 4#ifndef __ASSEMBLY__arch/arm64/include/asm/virt.h +8
-0@@ -145,6 +145,14 @@ static __always_inline bool is_protected_kvm_enabled(void)return cpus_have_final_cap (ARM64_KVM_PROTECTED_MODE ); } +static __always_inline bool has_hvhe(void) +{ +if (is_vhe_hyp_code ()) +return false; + +return cpus_have_final_cap (ARM64_KVM_HVHE ); +} +static inline bool is_hyp_nvhe (void ) {return is_hyp_mode_available () && !is_kernel_in_hyp_mode ();arch/arm64/kernel/cpufeature.c +0
-0@@ -1998,6 +1998,15 @@ static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap,return true;
arch/arm64/kernel/cpufeature.c static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap, · line 2008
Quoted 10 lines
arch/arm64/kernel/cpufeature.c +8
-0} +static bool hvhe_possible (const struct arm64_cpu_capabilities *entry, +int __unused) +{ +u64 val; + + val = arm64_sw_feature_override.val & arm64_sw_feature_override.mask; +return cpuid_feature_extract_unsigned_field (val,ARM64_SW_FEATURE_OVERRIDE_HVHE ); +}
Does this need to test ID_AA64MMFR1_EL1.VH as well? Otherwise I don't see what would stop us from attempting hVHE on a system with asymmetric support for VHE, as the software override was only evaluated on the boot CPU.
Quoted 31 lines
arch/arm64/kernel/cpufeature.c +6
-0#ifdef CONFIG_ARM64_PAN static void cpu_enable_pan (const struct arm64_cpu_capabilities *__unused) {@@ -2643,6 +2652,12 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .cpu_enable = cpu_enable_dit, ARM64_CPUID_FIELDS(ID_AA64PFR0_EL1, DIT, IMP) }, + { + .desc = "VHE for hypervisor only", + .capability = ARM64_KVM_HVHE, + .type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE, + .matches = hvhe_possible, + }, {}, };arch/arm64/tools/cpucaps +1
-0@@ -47,6 +47,7 @@ HAS_TLB_RANGE HAS_VIRT_HOST_EXTN HAS_WFXT HW_DBM +KVM_HVHE KVM_PROTECTED_MODE MISMATCHED_CACHE_TYPE MTESignature · 1 line
2.34.1
Signature · 2 lines
Thanks, Oliver
Marc Zyngier
maintainerRe: [PATCH v2 04/17] arm64: Add KVM_HVHE capability and has_hvhe() predicate
Hi Oliver,
Quoted 5 lines · Oliver Upton
Hey Marc,
I'm an idiot and was responding to v1. Here's the same damn comment, but on v2!
Probably means that I'm even more of an idiot by sending the same buggy code twice! :D
Quoted 53 lines · Marc Zyngier
Quoted 51 lines
Expose a capability keying the hVHE feature as well as a new predicate testing it. Nothing is so far using it, and nothing is enabling it yet.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/cpufeature.h | 1 + arch/arm64/include/asm/virt.h | 8 ++++++++ arch/arm64/kernel/cpufeature.c | 15 +++++++++++++++ arch/arm64/tools/cpucaps | 1 + 4 files changed, 25 insertions(+)arch/arm64/include/asm/cpufeature.h +1
-0@@ -16,6 +16,7 @@#define cpu_feature (x) KERNEL_HWCAP_ ## x#define ARM64_SW_FEATURE_OVERRIDE_NOKASLR 0 +#define ARM64_SW_FEATURE_OVERRIDE_HVHE 4#ifndef __ASSEMBLY__arch/arm64/include/asm/virt.h +8
-0@@ -145,6 +145,14 @@ static __always_inline bool is_protected_kvm_enabled(void)return cpus_have_final_cap (ARM64_KVM_PROTECTED_MODE ); } +static __always_inline bool has_hvhe(void) +{ +if (is_vhe_hyp_code ()) +return false; + +return cpus_have_final_cap (ARM64_KVM_HVHE ); +} +static inline bool is_hyp_nvhe (void ) {return is_hyp_mode_available () && !is_kernel_in_hyp_mode ();arch/arm64/kernel/cpufeature.c +0
-0@@ -1998,6 +1998,15 @@ static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap,return true; }
Quoted 13 lines
arch/arm64/kernel/cpufeature.c static bool has_nested_virt_support(const struct arm64_cpu_capabilities *cap, · line 2008
Quoted 8 lines
arch/arm64/kernel/cpufeature.c +8
-0+static bool hvhe_possible (const struct arm64_cpu_capabilities *entry, +int __unused) +{ +u64 val; + + val = arm64_sw_feature_override.val & arm64_sw_feature_override.mask; +return cpuid_feature_extract_unsigned_field (val,ARM64_SW_FEATURE_OVERRIDE_HVHE ); +}Does this need to test ID_AA64MMFR1_EL1.VH as well? Otherwise I don't see what would stop us from attempting hVHE on a system with asymmetric support for VHE, as the software override was only evaluated on the boot CPU.
Huh. You obviously have a filthy mind. Yeah, we could also test for the sanitised view of MMFR1.VH and change our mind at the last minute. I'll add a check. It also probably means that I need to make this a "ARM64_CPUCAP_SYSTEM_FEATURE" instead of a "ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE" (I think...).
But it has to be said that such a system, even without my hacks, would badly explode if the boot CPU was VHE capable and a secondary wasn't. The boot logic would keep one CPU at EL2 and move the secondary to EL1, and things would seamingly work until you try to do things like TLB invalidation (and you probably wouldn't even get a timer interrupt...).
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerPATCH v2 5/17 arm64: Don't enable VHE for the kernel if OVERRIDE_HVHE is set
In reply to Marc ZyngierIf the OVERRIDE_HVHE SW override is set (as a precursor of the KVM_HVHE capability), do not enable VHE for the kernel and drop to EL1 as if VHE was either disabled or unavailable.
Further changes will enable VHE at EL2 only, with the kernel still running at EL1.
--- arch/arm64/kernel/hyp-stub.S | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
arch/arm64/kernel/hyp-stub.S +9 -1
@@ -82,7 +82,15 @@ SYM_CODE_START_LOCAL(__finalise_el2) tbnz x1, #0, 1f // Needs to be VHE capable, obviously - check_override id_aa64mmfr1 ID_AA64MMFR1_EL1_VH_SHIFT 2f 1f x1 x2 + check_override id_aa64mmfr1 ID_AA64MMFR1_EL1_VH_SHIFT 0f 1f x1 x2 + +0: // Check whether we only want the hypervisor to run VHE, not the kernel + adr_l x1, arm64_sw_feature_override + ldr x2, [x1, FTR_OVR_VAL_OFFSET] + ldr x1, [x1, FTR_OVR_MASK_OFFSET] + and x2, x2, x1 + ubfx x2, x2, #ARM64_SW_FEATURE_OVERRIDE_HVHE, #4 + cbz x2, 2f 1: mov_q x0, HVC_STUB_ERR eret
Signature · 1 line
2.34.1
Re: [PATCH v2 05/17] arm64: Don't enable VHE for the kernel if OVERRIDE_HVHE is set
Quoted 18 lines · Marc Zyngier
If the
OVERRIDE_HVHESW override is set (as a precursor of theKVM_HVHEcapability), do not enable VHE for the kernel and drop to EL1 as if VHE was either disabled or unavailable.Further changes will enable VHE at EL2 only, with the kernel still running at EL1.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kernel/hyp-stub.S | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)arch/arm64/kernel/hyp-stub.S +0
-0@@ -82,7 +82,15 @@ SYM_CODE_START_LOCAL(__finalise_el2) tbnz x1, #0, 1f
arch/arm64/kernel/hyp-stub.S SYM_CODE_START_LOCAL(__finalise_el2) · line 91
Quoted 10 lines
arch/arm64/kernel/hyp-stub.S +7
-1// Needs to be VHE capable, obviously - check_override id_aa64mmfr1 ID_AA64MMFR1_EL1_VH_SHIFT 2f 1f x1 x2 + check_override id_aa64mmfr1 ID_AA64MMFR1_EL1_VH_SHIFT 0f 1f x1 x2 + +0: // Check whether we only want the hypervisor to run VHE, not the kernel + adr_l x1, arm64_sw_feature_override + ldr x2, [x1, FTR_OVR_VAL_OFFSET] + ldr x1, [x1, FTR_OVR_MASK_OFFSET] + and x2, x2, x1
nit: is applying the mask even necessary? I get it in the context of an overlay on top of an ID register, but the software features are more of a synthetic ID register in their own right.
Same nit applies to the kaslr case as well.
Quoted 8 lines
arch/arm64/kernel/hyp-stub.S +2
-0+ ubfx x2, x2, #ARM64_SW_FEATURE_OVERRIDE_HVHE, #4 + cbz x2, 2f 1: mov_q x0, HVC_STUB_ERR eretSignature · 1 line
2.34.1
Signature · 2 lines
Thanks, Oliver
Marc Zyngier
maintainerRe: [PATCH v2 05/17] arm64: Don't enable VHE for the kernel if OVERRIDE_HVHE is set
Quoted 21 lines · Oliver Upton
Quoted 19 lines · Marc Zyngier
If the
OVERRIDE_HVHESW override is set (as a precursor of theKVM_HVHEcapability), do not enable VHE for the kernel and drop to EL1 as if VHE was either disabled or unavailable.Further changes will enable VHE at EL2 only, with the kernel still running at EL1.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kernel/hyp-stub.S | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)arch/arm64/kernel/hyp-stub.S +0
-0@@ -82,7 +82,15 @@ SYM_CODE_START_LOCAL(__finalise_el2) tbnz x1, #0, 1f
Quoted 13 lines
arch/arm64/kernel/hyp-stub.S SYM_CODE_START_LOCAL(__finalise_el2) · line 91
Quoted 9 lines
arch/arm64/kernel/hyp-stub.S +7
-1// Needs to be VHE capable, obviously - check_override id_aa64mmfr1 ID_AA64MMFR1_EL1_VH_SHIFT 2f 1f x1 x2 + check_override id_aa64mmfr1 ID_AA64MMFR1_EL1_VH_SHIFT 0f 1f x1 x2 + +0: // Check whether we only want the hypervisor to run VHE, not the kernel + adr_l x1, arm64_sw_feature_override + ldr x2, [x1, FTR_OVR_VAL_OFFSET] + ldr x1, [x1, FTR_OVR_MASK_OFFSET] + and x2, x2, x1nit: is applying the mask even necessary? I get it in the context of an overlay on top of an ID register, but the software features are more of a synthetic ID register in their own right.
I guess I don't have a good reason just yet, but on the other hand it makes things predictable if the override code refuses the override for some reason other than not being VHE-capable (mask becomes 0 and val becomes 0xf).
Overall, I feel that this code is too hard to follow to do anything different from the "standard" case.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v2 05/17] arm64: Don't enable VHE for the kernel if OVERRIDE_HVHE is set
Quoted 37 lines · Marc Zyngier
Quoted 21 lines · Oliver Upton
Quoted 19 lines · Marc Zyngier
If the
OVERRIDE_HVHESW override is set (as a precursor of theKVM_HVHEcapability), do not enable VHE for the kernel and drop to EL1 as if VHE was either disabled or unavailable.Further changes will enable VHE at EL2 only, with the kernel still running at EL1.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kernel/hyp-stub.S | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)arch/arm64/kernel/hyp-stub.S +0
-0@@ -82,7 +82,15 @@ SYM_CODE_START_LOCAL(__finalise_el2) tbnz x1, #0, 1fQuoted 13 lines
arch/arm64/kernel/hyp-stub.S SYM_CODE_START_LOCAL(__finalise_el2) · line 91
Quoted 9 lines
arch/arm64/kernel/hyp-stub.S +7
-1// Needs to be VHE capable, obviously - check_override id_aa64mmfr1 ID_AA64MMFR1_EL1_VH_SHIFT 2f 1f x1 x2 + check_override id_aa64mmfr1 ID_AA64MMFR1_EL1_VH_SHIFT 0f 1f x1 x2 + +0: // Check whether we only want the hypervisor to run VHE, not the kernel + adr_l x1, arm64_sw_feature_override + ldr x2, [x1, FTR_OVR_VAL_OFFSET] + ldr x1, [x1, FTR_OVR_MASK_OFFSET] + and x2, x2, x1nit: is applying the mask even necessary? I get it in the context of an overlay on top of an ID register, but the software features are more of a synthetic ID register in their own right.
Quoted 7 lines
I guess I don't have a good reason just yet, but on the other hand it makes things predictable if the override code refuses the override for some reason other than not being VHE-capable (mask becomes 0 and val becomes
0xf).Overall, I feel that this code is too hard to follow to do anything different from the "standard" case.
Fair enough. Coming at it from the other angle (i.e. ignoring the ID reg side) it is only slightly weird, but does not really matter. I'm fine with this as-is.
Signature · 2 lines
Thanks, Oliver
Marc Zyngier
maintainerPATCH v2 6/17 arm64: Allow EL1 physical timer access when running VHE
In reply to Marc ZyngierTo initialise the timer access from EL2 when HCR_EL2.E2H is set, we must make use the CNTHCTL_EL2 formap used is appropriate.
This amounts to shifting the timer/counter enable bits by 10 to the left.
--- arch/arm64/include/asm/el2_setup.h | 5 +++++ arch/arm64/kvm/hyp/nvhe/hyp-init.S | 9 +++++++++ 2 files changed, 14 insertions(+)
arch/arm64/include/asm/el2_setup.h +5 -0
@@ -34,6 +34,11 @@*/ .macro __init_el2_timers mov x0, #3 // Enable EL1 physical timers +mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .LnVHE_\@ + lsl x0, x0, #10 +.LnVHE_\@: msr cnthctl_el2, x0 msr cntvoff_el2, xzr // Clear virtual offset .endm
arch/arm64/kvm/hyp/nvhe/hyp-init.S +9 -0
@@ -95,6 +95,15 @@ SYM_CODE_START_LOCAL(___kvm_hyp_init) ldr x1, [x0, #NVHE_INIT_HCR_EL2] msr hcr_el2, x1 + mov x2, #HCR_E2H + and x2, x1, x2 + cbz x2, 1f + + mrs x1, cnthctl_el2 + and x1, x1, #~(BIT(0) | BIT(1)) + orr x1, x1, #(BIT(10) | BIT(11)) + msr cnthctl_el2, x1 +1: ldr x1, [x0, #NVHE_INIT_VTTBR] msr vttbr_el2, x1
Signature · 1 line
2.34.1
Re: [PATCH v2 06/17] arm64: Allow EL1 physical timer access when running VHE
Hey Marc,
Quoted 35 lines · Marc Zyngier
To initialise the timer access from EL2 when HCR_EL2.E2H is set, we must make use the
CNTHCTL_EL2formap used is appropriate.This amounts to shifting the timer/counter enable bits by 10 to the left.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/el2_setup.h | 5 +++++ arch/arm64/kvm/hyp/nvhe/hyp-init.S | 9 +++++++++ 2 files changed, 14 insertions(+)arch/arm64/include/asm/el2_setup.h +5
-0@@ -34,6 +34,11 @@*/ .macro __init_el2_timers mov x0, #3 // Enable EL1 physical timers +mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .LnVHE_\@ + lsl x0, x0, #10 +.LnVHE_\@: msr cnthctl_el2, x0 msr cntvoff_el2, xzr // Clear virtual offset .endmarch/arm64/kvm/hyp/nvhe/hyp-init.S +0
-0@@ -95,6 +95,15 @@ SYM_CODE_START_LOCAL(___kvm_hyp_init) ldr x1, [x0, #NVHE_INIT_HCR_EL2] msr hcr_el2, x1
arch/arm64/kvm/hyp/nvhe/hyp-init.S SYM_CODE_START_LOCAL(___kvm_hyp_init) · line 106
Quoted 10 lines
arch/arm64/kvm/hyp/nvhe/hyp-init.S +9
-0+ mov x2, #HCR_E2H + and x2, x1, x2 + cbz x2, 1f + + mrs x1, cnthctl_el2 + and x1, x1, #~(BIT(0) | BIT(1)) + orr x1, x1, #(BIT(10) | BIT(11)) + msr cnthctl_el2, x1 +1:
Can this be done with an alternative block keyed on ARM64_KVM_HVHE? I get that __init_el2_timers needs to run before the cpu caps have been evaluated, but I don't think the same applies for ___kvm_hyp_init
Signature · 2 lines
Thanks, Oliver
Marc Zyngier
maintainerRe: [PATCH v2 06/17] arm64: Allow EL1 physical timer access when running VHE
Quoted 40 lines · Oliver Upton
Hey Marc,
Quoted 36 lines · Marc Zyngier
To initialise the timer access from EL2 when HCR_EL2.E2H is set, we must make use the
CNTHCTL_EL2formap used is appropriate.This amounts to shifting the timer/counter enable bits by 10 to the left.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/el2_setup.h | 5 +++++ arch/arm64/kvm/hyp/nvhe/hyp-init.S | 9 +++++++++ 2 files changed, 14 insertions(+)arch/arm64/include/asm/el2_setup.h +5
-0@@ -34,6 +34,11 @@*/ .macro __init_el2_timers mov x0, #3 // Enable EL1 physical timers +mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .LnVHE_\@ + lsl x0, x0, #10 +.LnVHE_\@: msr cnthctl_el2, x0 msr cntvoff_el2, xzr // Clear virtual offset .endmarch/arm64/kvm/hyp/nvhe/hyp-init.S +0
-0@@ -95,6 +95,15 @@ SYM_CODE_START_LOCAL(___kvm_hyp_init) ldr x1, [x0, #NVHE_INIT_HCR_EL2] msr hcr_el2, x1
Quoted 13 lines
arch/arm64/kvm/hyp/nvhe/hyp-init.S SYM_CODE_START_LOCAL(___kvm_hyp_init) · line 106
Quoted 9 lines
arch/arm64/kvm/hyp/nvhe/hyp-init.S +9
-0+ mov x2, #HCR_E2H + and x2, x1, x2 + cbz x2, 1f + + mrs x1, cnthctl_el2 + and x1, x1, #~(BIT(0) | BIT(1)) + orr x1, x1, #(BIT(10) | BIT(11)) + msr cnthctl_el2, x1 +1:Can this be done with an alternative block keyed on
ARM64_KVM_HVHE? I get that__init_el2_timersneeds to run before the cpu caps have been evaluated, but I don't think the same applies for___kvm_hyp_init
I don't mind either way. My reasoning was that this code is hardly ever executed, so the performance benefit of alternatives don't really apply here (and the resulting object is smaller by 28 bytes... ;-).
Happy to change it if you think this makes sense.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerPATCH v2 7/17 arm64: Use CPACR_EL1 format to set CPTR_EL2 when E2H is set
In reply to Marc ZyngierWhen HCR_EL2.E2H is set, the CPTR_EL2 register takes the CPACR_EL1 format. Yes, this is good fun.
Hack the bits of startup code that assume E2H=0 while setting up CPTR_EL2 to make them grok the CPTR_EL1 format.
--- arch/arm64/include/asm/el2_setup.h | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-)
arch/arm64/include/asm/el2_setup.h +19 -2
@@ -129,8 +129,15 @@ .endm /* Coprocessor traps */ -.macro __init_el2_nvhe_cptr +.macro __init_el2_cptr + mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .LnVHE_\@ + mov x0, #(CPACR_EL1_FPEN_EL1EN | CPACR_EL1_FPEN_EL0EN) + b .Lset_cptr_\@ +.LnVHE_\@: mov x0, #0x33ff +.Lset_cptr_\@: msr cptr_el2, x0 // Disable copro. traps to EL2 .endm
@@ -196,7 +203,7 @@__init_el2_gicv3 __init_el2_hstr__init_el2_nvhe_idregs - __init_el2_nvhe_cptr + __init_el2_cptr__init_el2_fgt __init_el2_nvhe_prepare_eret .endm
@@ -244,7 +251,17 @@ .Linit_sve_\@: /* SVE register access */mrs x0, cptr_el2// Disable SVE traps + mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .Lcptr_nvhe_\@ + +// VHE case + orr x0, x0, #(CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN) + b .Lset_cptr_\@ + +.Lcptr_nvhe_\@: // nVHE case bic x0, x0, #CPTR_EL2_TZ +.Lset_cptr_\@: msr cptr_el2, x0 isb mov x1, #ZCR_ELx_LEN_MASK // SVE: Enable full vector
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 8/17 KVM: arm64: Remove alternatives from sysreg accessors in VHE hypervisor context
In reply to Marc ZyngierIn the VHE hypervisor code, we should be using the remapped VHE accessors, no ifs, no buts. No need to generate any alternative.
--- arch/arm64/include/asm/kvm_hyp.h | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-)
arch/arm64/include/asm/kvm_hyp.h +19 -6
@@ -16,6 +16,23 @@ DECLARE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); DECLARE_PER_CPU(unsigned long, kvm_hyp_vector); DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); +/* +* Unified accessors for registers that have a different encoding +* between VHE and non-VHE. They must be specified without their "ELx" +* encoding, but with the SYS_ prefix, as defined in asm/sysreg.h. +*/ + +#if defined(__KVM_VHE_HYPERVISOR__) + +#define read_sysreg_el0 (r) read_sysreg_s(r##_EL02) +#define write_sysreg_el0 (v,r) write_sysreg_s(v, r##_EL02) +#define read_sysreg_el1 (r) read_sysreg_s(r##_EL12) +#define write_sysreg_el1 (v,r) write_sysreg_s(v, r##_EL12) +#define read_sysreg_el2 (r) read_sysreg_s(r##_EL1) +#define write_sysreg_el2 (v,r) write_sysreg_s(v, r##_EL1) + +#else // !__KVM_VHE_HYPERVISOR__ +#define read_sysreg_elx (r,nvh,vh) \ ({ \ u64 reg; \
@@ -35,12 +52,6 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); : : "rZ" (__val)); \ } while (0) -/* -* Unified accessors for registers that have a different encoding -* between VHE and non-VHE. They must be specified without their "ELx" -* encoding, but with the SYS_ prefix, as defined in asm/sysreg.h. - */ -#define read_sysreg_el0 (r) read_sysreg_elx(r, _EL0, _EL02)#define write_sysreg_el0 (v,r) write_sysreg_elx(v, r, _EL0, _EL02)#define read_sysreg_el1 (r) read_sysreg_elx(r, _EL1, _EL12)
@@ -48,6 +59,8 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);#define read_sysreg_el2 (r) read_sysreg_elx(r, _EL2, _EL1)#define write_sysreg_el2 (v,r) write_sysreg_elx(v, r, _EL2, _EL1) +#endif // __KVM_VHE_HYPERVISOR__ +/* * Without an __arch_swab32(), we fall back to ___constant_swab32(), but the * static inline can allow the compiler to out-of-line this. KVM always wants
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 9/17 KVM: arm64: Key use of VHE instructions in nVHE code off ARM64_KVM_HVHE
In reply to Marc ZyngierWe can now start with the fun stuff: if we enable VHE *only* for the hypervisor, we need to generate the VHE instructions when accessing the system registers.
For this, reporpose the alternative sequence to be keyed off ARM64_KVM_HVHE in the nVHE hypervisor code, and only there.
--- arch/arm64/include/asm/kvm_hyp.h | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
arch/arm64/include/asm/kvm_hyp.h +9 -3
@@ -33,12 +33,18 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);#else // !__KVM_VHE_HYPERVISOR__ +#if defined(__KVM_NVHE_HYPERVISOR__) +#define VHE_ALT_KEY ARM64_KVM_HVHE +#else +#define VHE_ALT_KEY ARM64_HAS_VIRT_HOST_EXTN +#endif +#define read_sysreg_elx (r,nvh,vh) \ ({ \ u64 reg; \ - asm volatile(ALTERNATIVE(__mrs_s("%0", r##nvh), \ + asm volatile(ALTERNATIVE(__mrs_s("%0", r##nvh), \ __mrs_s("%0", r##vh), \ - ARM64_HAS_VIRT_HOST_EXTN) \ + VHE_ALT_KEY) \ : "=r" (reg)); \ reg; \ })
@@ -48,7 +54,7 @@ DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);u64 __val = (u64)(v); \ asm volatile(ALTERNATIVE(__msr_s(r##nvh, "%x0"), \ __msr_s(r##vh, "%x0"), \ - ARM64_HAS_VIRT_HOST_EXTN) \ + VHE_ALT_KEY) \ : : "rZ" (__val)); \ } while (0)
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 10/17 KVM: arm64: Force HCR_EL2.E2H when ARM64_KVM_HVHE is set
In reply to Marc ZyngierObviously, in order to be able to use VHE whilst at EL2, we need to set HCR_EL2.E2H. Do so when ARM64_KVM_HVHE is set.
--- arch/arm64/kvm/arm.c | 2 ++ 1 file changed, 2 insertions(+)
arch/arm64/kvm/arm.c +2 -0
@@ -1676,6 +1676,8 @@ static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits) params->hcr_el2 =HCR_HOST_NVHE_PROTECTED_FLAGS ; else params->hcr_el2 = HCR_HOST_NVHE_FLAGS; +if (cpus_have_final_cap (ARM64_KVM_HVHE )) + params->hcr_el2 |=HCR_E2H ; params->vttbr = params->vtcr =0 ;/*
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 11/17 KVM: arm64: Disable TTBR1_EL2 when using ARM64_KVM_HVHE
In reply to Marc ZyngierWhen using hVHE, we end-up with two TTBRs at EL2. That's great, but we're not quite ready for this just yet.
Disable TTBR1_EL2 by setting TCR_EL2.EPD1 so that we only translate via TTBR0_EL2.
--- arch/arm64/kvm/arm.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
arch/arm64/kvm/arm.c +7 -1
@@ -1666,7 +1666,13 @@ static void __init cpu_prepare_hyp_mode(int cpu, u32 hyp_va_bits) params->mair_el2 =read_sysreg (mair_el1); - tcr = (read_sysreg (tcr_el1) &TCR_EL2_MASK ) |TCR_EL2_RES1 ; + tcr =read_sysreg (tcr_el1); +if (cpus_have_final_cap (ARM64_KVM_HVHE )) { + tcr |=TCR_EPD1_MASK ; + }else { + tcr &=TCR_EL2_MASK ; + tcr |=TCR_EL2_RES1 ; + } tcr &= ~TCR_T0SZ_MASK ; tcr |=TCR_T0SZ (hyp_va_bits); params->tcr_el2 = tcr;
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 12/17 KVM: arm64: Adjust EL2 stage-1 leaf AP bits when ARM64_KVM_HVHE is set
In reply to Marc ZyngierEl2 stage-1 page-table format is subtly (and annoyingly) different when HCR_EL2.E2H is set.
Take the ARM64_KVM_HVHE configuration into account when setting the AP bits.
--- arch/arm64/kvm/hyp/pgtable.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
arch/arm64/kvm/hyp/pgtable.c +4 -2
@@ -21,8 +21,10 @@#define KVM_PTE_LEAF_ATTR_LO_S1_ATTRIDX GENMASK(4, 2)#define KVM_PTE_LEAF_ATTR_LO_S1_AP GENMASK(7, 6) -#define KVM_PTE_LEAF_ATTR_LO_S1_AP_RO 3 -#define KVM_PTE_LEAF_ATTR_LO_S1_AP_RW 1 +#define KVM_PTE_LEAF_ATTR_LO_S1_AP_RO \ + ({ cpus_have_final_cap(ARM64_KVM_HVHE) ? 2 : 3; }) +#define KVM_PTE_LEAF_ATTR_LO_S1_AP_RW \ + ({ cpus_have_final_cap(ARM64_KVM_HVHE) ? 0 : 1; })#define KVM_PTE_LEAF_ATTR_LO_S1_SH GENMASK(9, 8)#define KVM_PTE_LEAF_ATTR_LO_S1_SH_IS 3#define KVM_PTE_LEAF_ATTR_LO_S1_AF BIT(10)
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 13/17 KVM: arm64: Rework CPTR_EL2 programming for HVHE configuration
In reply to Marc ZyngierJust like we repainted the early arm64 code, we need to update the CPTR_EL2 accesses that are taking place in the nVHE code when hVHE is used, making them look as if they were CPACR_EL1 accesses. Just like the VHE code.
--- arch/arm64/include/asm/kvm_arm.h | 4 +--- arch/arm64/include/asm/kvm_emulate.h | 31 +++++++++++++++++++++++++ arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/fpsimd.c | 4 ++-- arch/arm64/kvm/hyp/include/hyp/switch.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 6 ++++- arch/arm64/kvm/hyp/nvhe/pkvm.c | 24 ++++++++++++++----- arch/arm64/kvm/hyp/nvhe/switch.c | 28 ++++++++++++---------- arch/arm64/kvm/hyp/vhe/switch.c | 2 +- arch/arm64/kvm/sys_regs.c | 2 +- 10 files changed, 77 insertions(+), 28 deletions(-)
arch/arm64/include/asm/kvm_arm.h +1 -3
@@ -285,7 +285,6 @@#define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)#define CPTR_EL2_TZ (1 << 8)#define CPTR_NVHE_EL2_RES1 0x000032ff/* known RES1 bits in CPTR_EL2 (nVHE) */ -#define CPTR_EL2_DEFAULT CPTR_NVHE_EL2_RES1#define CPTR_NVHE_EL2_RES0 (GENMASK(63, 32) | \ GENMASK(29, 21) | \ GENMASK(19, 14) | \
@@ -347,8 +346,7 @@ECN (SOFTSTP_CUR ),ECN (WATCHPT_LOW ),ECN (WATCHPT_CUR ), \ ECN(BKPT32), ECN(VECTOR32), ECN(BRK64), ECN(ERET) -#define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN |\ - CPACR_EL1_ZEN_EL1EN) +#define CPACR_EL1_TTA (1 << 28)#define kvm_mode_names \ { PSR_MODE_EL0t, "EL0t" }, \
arch/arm64/include/asm/kvm_emulate.h +31 -0
@@ -570,4 +570,35 @@ static inline bool vcpu_has_feature(struct kvm_vcpu *vcpu, int feature)return test_bit (feature, vcpu->arch.features); } +static __always_inline u64 kvm_get_reset_cptr_el2(struct kvm_vcpu *vcpu) +{ +u64 val; + +if (has_vhe ()) { + val = (CPACR_EL1_FPEN_EL0EN |CPACR_EL1_FPEN_EL1EN | +CPACR_EL1_ZEN_EL1EN ); + }else if (has_hvhe ()) { + val = (CPACR_EL1_FPEN_EL0EN |CPACR_EL1_FPEN_EL1EN ); + }else { + val =CPTR_NVHE_EL2_RES1 ; + +if (vcpu_has_sve (vcpu) && + (vcpu->arch.fp_state ==FP_STATE_GUEST_OWNED )) + val |=CPTR_EL2_TZ ; +if (cpus_have_final_cap (ARM64_SME )) + val &= ~CPTR_EL2_TSM ; + } + +return val; +} + +static __always_inline void kvm_reset_cptr_el2(struct kvm_vcpu *vcpu) +{ +u64 val =kvm_get_reset_cptr_el2 (vcpu); + +if (has_vhe () ||has_hvhe ()) +write_sysreg (val, cpacr_el1); +else +write_sysreg (val, cptr_el2); +}#endif /* __ARM64_KVM_EMULATE_H__ */
arch/arm64/kvm/arm.c +1 -1
@@ -1240,7 +1240,7 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, }vcpu_reset_hcr (vcpu); - vcpu->arch.cptr_el2 =CPTR_EL2_DEFAULT ; + vcpu->arch.cptr_el2 =kvm_get_reset_cptr_el2 (vcpu);/* * Handle the "start in power-off" case.
arch/arm64/kvm/fpsimd.c +2 -2
@@ -172,7 +172,7 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)/* * If we have VHE then the Hyp code will reset CPACR_EL1 to -* CPACR_EL1_DEFAULT and we need to reenable SME. +* the default value and we need to reenable SME. */ if (has_vhe () &&system_supports_sme ()) {/* Also restore EL0 state seen on entry */
@@ -202,7 +202,7 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)/* * The FPSIMD/SVE state in the CPU has not been touched, and we * have SVE (and VHE): CPACR_EL1 (alias CPTR_EL2) has been -* reset to CPACR_EL1_DEFAULT by the Hyp code, disabling SVE +* reset by kvm_reset_cptr_el2() in the Hyp code, disabling SVE * for EL0. To avoid spurious traps, restore the trap state * seen by kvm_arch_vcpu_load_fp(): */
arch/arm64/kvm/hyp/include/hyp/switch.h +1 -1
@@ -184,7 +184,7 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code)/* Valid trap. Switch the context: */ /* First disable enough traps to allow us to update the registers */ -if (has_vhe ()) { +if (has_vhe () ||has_hvhe ()) { reg =CPACR_EL1_FPEN_EL0EN |CPACR_EL1_FPEN_EL1EN ;if (sve_guest) reg |=CPACR_EL1_ZEN_EL0EN |CPACR_EL1_ZEN_EL1EN ;
arch/arm64/kvm/hyp/nvhe/hyp-main.c +5 -1
@@ -392,7 +392,11 @@ void handle_trap(struct kvm_cpu_context *host_ctxt)handle_host_smc (host_ctxt);break ;case ESR_ELx_EC_SVE: -sysreg_clear_set (cptr_el2,CPTR_EL2_TZ ,0 ); +if (has_hvhe ()) +sysreg_clear_set (cpacr_el1,0 , (CPACR_EL1_ZEN_EL1EN | +CPACR_EL1_ZEN_EL0EN )); +else +sysreg_clear_set (cptr_el2,CPTR_EL2_TZ ,0 );isb ();sve_cond_update_zcr_vq (ZCR_ELx_LEN_MASK,SYS_ZCR_EL2 );break ;
arch/arm64/kvm/hyp/nvhe/pkvm.c +18 -6
@@ -27,6 +27,7 @@ static void pvm_init_traps_aa64pfr0(struct kvm_vcpu *vcpu)u64 hcr_set =HCR_RW ;u64 hcr_clear =0 ;u64 cptr_set =0 ; +u64 cptr_clear =0 ;/* Protected KVM does not support AArch32 guests. */ BUILD_BUG_ON(FIELD_GET(ARM64_FEATURE_MASK(ID_AA64PFR0_EL1_EL0),
@@ -57,12 +58,17 @@ static void pvm_init_traps_aa64pfr0(struct kvm_vcpu *vcpu) }/* Trap SVE */ -if (!FIELD_GET (ARM64_FEATURE_MASK (ID_AA64PFR0_EL1_SVE ), feature_ids)) - cptr_set |=CPTR_EL2_TZ ; +if (!FIELD_GET (ARM64_FEATURE_MASK (ID_AA64PFR0_EL1_SVE ), feature_ids)) { +if (has_hvhe ()) + cptr_clear |=CPACR_EL1_ZEN_EL0EN |CPACR_EL1_ZEN_EL1EN ; +else + cptr_set |=CPTR_EL2_TZ ; + } vcpu->arch.hcr_el2 |= hcr_set; vcpu->arch.hcr_el2 &= ~hcr_clear; vcpu->arch.cptr_el2 |= cptr_set; + vcpu->arch.cptr_el2 &= ~cptr_clear; }/*
@@ -120,8 +126,12 @@ static void pvm_init_traps_aa64dfr0(struct kvm_vcpu *vcpu) mdcr_set |=MDCR_EL2_TTRF ;/* Trap Trace */ -if (!FIELD_GET (ARM64_FEATURE_MASK (ID_AA64DFR0_EL1_TraceVer), feature_ids)) - cptr_set |=CPTR_EL2_TTA ; +if (!FIELD_GET (ARM64_FEATURE_MASK (ID_AA64DFR0_EL1_TraceVer), feature_ids)) { +if (has_hvhe ()) + cptr_set |=CPACR_EL1_TTA ; +else + cptr_set |=CPTR_EL2_TTA ; + } vcpu->arch.mdcr_el2 |= mdcr_set; vcpu->arch.mdcr_el2 &= ~mdcr_clear;
@@ -176,8 +186,10 @@ static void pvm_init_trap_regs(struct kvm_vcpu *vcpu)/* Clear res0 and set res1 bits to trap potential new features. */ vcpu->arch.hcr_el2 &= ~(HCR_RES0 ); vcpu->arch.mdcr_el2 &= ~(MDCR_EL2_RES0 ); - vcpu->arch.cptr_el2 |=CPTR_NVHE_EL2_RES1 ; - vcpu->arch.cptr_el2 &= ~(CPTR_NVHE_EL2_RES0 ); +if (!has_hvhe ()) { + vcpu->arch.cptr_el2 |=CPTR_NVHE_EL2_RES1 ; + vcpu->arch.cptr_el2 &= ~(CPTR_NVHE_EL2_RES0 ); + } }/*
arch/arm64/kvm/hyp/nvhe/switch.c +16 -12
@@ -44,13 +44,24 @@ static void __activate_traps(struct kvm_vcpu *vcpu)__activate_traps_common (vcpu); val = vcpu->arch.cptr_el2; - val |=CPTR_EL2_TTA |CPTR_EL2_TAM ; + val |=CPTR_EL2_TAM ;/* Same bit irrespective of E2H */ + val |=has_hvhe () ?CPACR_EL1_TTA :CPTR_EL2_TTA ; +if (cpus_have_final_cap (ARM64_SME )) { +if (has_hvhe ()) + val &= ~(CPACR_EL1_SMEN_EL1EN |CPACR_EL1_SMEN_EL0EN ); +else + val |=CPTR_EL2_TSM ; + } +if (!guest_owns_fp_regs (vcpu)) { - val |=CPTR_EL2_TFP |CPTR_EL2_TZ ; +if (has_hvhe ()) + val &= ~(CPACR_EL1_FPEN_EL0EN |CPACR_EL1_FPEN_EL1EN | +CPACR_EL1_ZEN_EL0EN |CPACR_EL1_ZEN_EL1EN ); +else + val |=CPTR_EL2_TFP |CPTR_EL2_TZ ; +__activate_traps_fpsimd32 (vcpu); } -if (cpus_have_final_cap (ARM64_SME )) - val |=CPTR_EL2_TSM ;write_sysreg (val, cptr_el2);write_sysreg (__this_cpu_read (kvm_hyp_vector), vbar_el2);
@@ -73,7 +84,6 @@ static void __activate_traps(struct kvm_vcpu *vcpu)static void __deactivate_traps (struct kvm_vcpu *vcpu) {extern char __kvm_hyp_host_vector[]; -u64 cptr;___deactivate_traps (vcpu);
@@ -98,13 +108,7 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu)write_sysreg (this_cpu_ptr (&kvm_init_params)->hcr_el2, hcr_el2); - cptr =CPTR_EL2_DEFAULT ; -if (vcpu_has_sve (vcpu) && (vcpu->arch.fp_state ==FP_STATE_GUEST_OWNED )) - cptr |=CPTR_EL2_TZ ; -if (cpus_have_final_cap (ARM64_SME )) - cptr &= ~CPTR_EL2_TSM ; - -write_sysreg (cptr, cptr_el2); +kvm_reset_cptr_el2 (vcpu);write_sysreg (__kvm_hyp_host_vector, vbar_el2); }
arch/arm64/kvm/hyp/vhe/switch.c +1 -1
@@ -84,7 +84,7 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu)*/ asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT)); -write_sysreg (CPACR_EL1_DEFAULT , cpacr_el1); +kvm_reset_cptr_el2 (vcpu);if (!arm64_kernel_unmapped_at_el0 ()) host_vectors =__this_cpu_read (this_cpu_vector);
arch/arm64/kvm/sys_regs.c +1 -1
@@ -2180,7 +2180,7 @@ static const struct sys_reg_desc sys_reg_descs[] = {EL2_REG (ACTLR_EL2 , access_rw, reset_val,0 ),EL2_REG (HCR_EL2 , access_rw, reset_val,0 ),EL2_REG (MDCR_EL2 , access_rw, reset_val,0 ), -EL2_REG (CPTR_EL2 , access_rw, reset_val,CPTR_EL2_DEFAULT ), +EL2_REG (CPTR_EL2 , access_rw, reset_val,CPTR_NVHE_EL2_RES1 ),EL2_REG (HSTR_EL2 , access_rw, reset_val,0 ), EL2_REG(HACR_EL2, access_rw, reset_val, 0),
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 14/17 KVM: arm64: Program the timer traps with VHE layout in hVHE mode
In reply to Marc ZyngierJust like the rest of the timer code, we need to shift the enable bits around when HCR_EL2.E2H is set, which is the case in hVHE mode.
--- arch/arm64/kvm/hyp/nvhe/timer-sr.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
arch/arm64/kvm/hyp/nvhe/timer-sr.c +12 -4
@@ -17,21 +17,24 @@ void __kvm_timer_set_cntvoff(u64 cntvoff) }/* -* Should only be called on non-VHE systems. +* Should only be called on non-VHE or hVHE setups. * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). */ void __timer_disable_traps (struct kvm_vcpu *vcpu) { -u64 val; +u64 val, shift =0 ; + +if (has_hvhe ()) + shift =10 ;/* Allow physical timer/counter access for the host */ val =read_sysreg (cnthctl_el2); - val |=CNTHCTL_EL1PCTEN |CNTHCTL_EL1PCEN ; + val |= (CNTHCTL_EL1PCTEN |CNTHCTL_EL1PCEN ) << shift;write_sysreg (val, cnthctl_el2); }/* -* Should only be called on non-VHE systems. +* Should only be called on non-VHE or hVHE setups. * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). */ void __timer_enable_traps (struct kvm_vcpu *vcpu)
@@ -50,5 +53,10 @@ void __timer_enable_traps(struct kvm_vcpu *vcpu)else clr |=CNTHCTL_EL1PCTEN ; +if (has_hvhe ()) { + clr <<=10 ; + set <<=10 ; + } +sysreg_clear_set (cnthctl_el2, clr, set); }
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 15/17 KVM: arm64: Force HCR_E2H in guest context when ARM64_KVM_HVHE is set
In reply to Marc ZyngierAlso make sure HCR_EL2.E2H is set when switching HCR_EL2 in guest context.
--- arch/arm64/include/asm/kvm_emulate.h | 2 +- arch/arm64/kvm/hyp/nvhe/pkvm.c | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-)
arch/arm64/include/asm/kvm_emulate.h +1 -1
@@ -74,7 +74,7 @@ static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu)static inline void vcpu_reset_hcr (struct kvm_vcpu *vcpu) { vcpu->arch.hcr_el2 =HCR_GUEST_FLAGS ; -if (is_kernel_in_hyp_mode ()) +if (has_vhe () ||has_hvhe ()) vcpu->arch.hcr_el2 |=HCR_E2H ;if (cpus_have_const_cap (ARM64_HAS_RAS_EXTN )) {/* route synchronous external abort exceptions to EL2 */
arch/arm64/kvm/hyp/nvhe/pkvm.c +3 -0
@@ -44,6 +44,9 @@ static void pvm_init_traps_aa64pfr0(struct kvm_vcpu *vcpu)BUILD_BUG_ON (!FIELD_GET (ARM64_FEATURE_MASK (ID_AA64PFR0_EL1_AdvSIMD),PVM_ID_AA64PFR0_ALLOW )); +if (has_hvhe ()) + hcr_set |=HCR_E2H ; +/* Trap RAS unless all current versions are supported */ if (FIELD_GET (ARM64_FEATURE_MASK (ID_AA64PFR0_EL1_RAS ), feature_ids) <ID_AA64PFR0_EL1_RAS_V1P1 ) {
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 16/17 arm64: Allow arm64_sw.hvhe on command line
In reply to Marc ZyngierAdd the arm64_sw.hvhe=1 option to force the use of the hVHE mode in the hypervisor code only.
This enables the hVHE mode of operation when using KVM on VHE hardware.
--- arch/arm64/kernel/idreg-override.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/arm64/kernel/idreg-override.c +11 -0
@@ -138,11 +138,22 @@ static const struct ftr_set_desc smfr0 __initconst = { }, }; +static bool __init hvhe_filter (u64 val) +{ +u64 mmfr1 =read_sysreg (id_aa64mmfr1_el1); + +return (val ==1 && +lower_32_bits (__boot_status) ==BOOT_CPU_MODE_EL2 && +cpuid_feature_extract_unsigned_field (mmfr1, +ID_AA64MMFR1_EL1_VH_SHIFT )); +} +static const struct ftr_set_desc sw_features__initconst = { .name ="arm64_sw" , .override = &arm64_sw_feature_override, .fields = {FIELD ("nokaslr" ,ARM64_SW_FEATURE_OVERRIDE_NOKASLR ,NULL ), +FIELD ("hvhe" ,ARM64_SW_FEATURE_OVERRIDE_HVHE , hvhe_filter), {} }, };
Signature · 1 line
2.34.1
Marc Zyngier
maintainerPATCH v2 17/17 KVM: arm64: Terrible timer hack for M1 with hVHE
In reply to Marc ZyngierAs our M1 friend doesn't have a GIC, it relies on a special hack to deal with masking the guest timers, in the form of an IMPDEF system register.
Unfortunately, this sysreg is EL2-only, which means that the kernel cannot mask the interrupts itself, but has to kindly ask EL2 to do it. Yes, this is terrible, but we should be used to it by now.
Add a M1-specific hypercall to deal with this. No, I'm not seriously suggesting we merge this crap.
--- arch/arm64/include/asm/arch_timer.h | 8 +++++ arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kernel/image-vars.h | 3 ++ arch/arm64/kvm/arch_timer.c | 5 +++ arch/arm64/kvm/hyp/nvhe/hyp-main.c | 11 +++++++ arch/arm64/kvm/hyp/nvhe/timer-sr.c | 9 ++++++ drivers/irqchip/irq-apple-aic.c | 50 +++++++++++++++++++++++++++-- 7 files changed, 85 insertions(+), 2 deletions(-)
arch/arm64/include/asm/arch_timer.h +8 -0
@@ -232,4 +232,12 @@ static inline bool arch_timer_have_evtstrm_feature(void) {return cpu_have_named_feature (EVTSTRM ); } + +#ifdef CONFIG_APPLE_AIC +#define SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2 sys_reg(3, 5, 15, 1, 3) +DECLARE_STATIC_KEY_FALSE (aic_impdef_timer_control); +#endif + +void __aic_timer_fiq_clear_set (u64 clear,u64 set); +#endif
arch/arm64/include/asm/kvm_asm.h +1 -0
@@ -75,6 +75,7 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___vgic_v3_write_vmcr, __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs, __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_aprs, + __KVM_HOST_SMCCC_FUNC___aic_timer_fiq_clear_set, __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_init_traps, __KVM_HOST_SMCCC_FUNC___pkvm_init_vm, __KVM_HOST_SMCCC_FUNC___pkvm_init_vcpu,
arch/arm64/kernel/image-vars.h +3 -0
@@ -106,6 +106,9 @@ KVM_NVHE_ALIAS(__hyp_rodata_end);/* pKVM static key */ KVM_NVHE_ALIAS (kvm_protected_mode_initialized); +/* Hack for M1 timer control in hVHE mode */ +KVM_NVHE_ALIAS (aic_impdef_timer_control); +#endif /* CONFIG_KVM */ #ifdef CONFIG_EFI_ZBOOT
arch/arm64/kvm/arch_timer.c +5 -0
@@ -1259,6 +1259,11 @@ static int timer_irq_set_vcpu_affinity(struct irq_data *d, void *vcpu)return 0 ; } +void __aic_timer_fiq_clear_set (u64 clear,u64 set) +{ +kvm_call_hyp_nvhe (__aic_timer_fiq_clear_set, clear, set); +} +static int timer_irq_set_irqchip_state (struct irq_data *d,enum irqchip_irq_state which,bool val) {
arch/arm64/kvm/hyp/nvhe/hyp-main.c +11 -0
@@ -192,6 +192,16 @@ static void handle___vgic_v3_restore_aprs(struct kvm_cpu_context *host_ctxt)__vgic_v3_restore_aprs (kern_hyp_va (cpu_if)); } +#define SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2 sys_reg(3, 5, 15, 1, 3) + +static void handle___aic_timer_fiq_clear_set (struct kvm_cpu_context *host_ctxt) +{ +DECLARE_REG (u64, clear, host_ctxt,1 ); +DECLARE_REG (u64, set, host_ctxt,2 ); + +__aic_timer_fiq_clear_set (clear, set); +} +static void handle___pkvm_init (struct kvm_cpu_context *host_ctxt) {DECLARE_REG (phys_addr_t, phys, host_ctxt,1 );
@@ -322,6 +332,7 @@ static const hcall_t host_hcall[] = {HANDLE_FUNC (__vgic_v3_write_vmcr),HANDLE_FUNC (__vgic_v3_save_aprs),HANDLE_FUNC (__vgic_v3_restore_aprs), +HANDLE_FUNC (__aic_timer_fiq_clear_set),HANDLE_FUNC (__pkvm_vcpu_init_traps),HANDLE_FUNC (__pkvm_init_vm), HANDLE_FUNC(__pkvm_init_vcpu),
arch/arm64/kvm/hyp/nvhe/timer-sr.c +9 -0
@@ -60,3 +60,12 @@ void __timer_enable_traps(struct kvm_vcpu *vcpu)sysreg_clear_set (cnthctl_el2, clr, set); } + + +void __aic_timer_fiq_clear_set (u64 clear,u64 set) +{ +#ifdef CONFIG_APPLE_AIC +if (has_hvhe () &&static_branch_likely (&aic_impdef_timer_control)) +sysreg_clear_set_s (SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2 , clear, set); +#endif +}
drivers/irqchip/irq-apple-aic.c +48 -2
92 lines not on this page · raw message
Signature · 1 line
2.34.1