KVM: arm64: Make ICH_VTR_EL2 accesses an inlined literal
Marc Zyngier
maintainerv2 cover KVM: arm64: Make ICH_VTR_EL2 accesses an inlined literal
I'm reposting this early as there were so many issues to the original posting that I decided to deviate from my own rules... Bite me.
This is the second version of this series I initially sent as [1], which aims at removing the large number of traps that ICH_VTR_EL2 generates in NV environments, leading to pointless overheads.
By moving from a sysreg access to a patched constant, running an L3 workload with a a healthy amount of IO results in a 5% reduction of wall-clock time from a baseline without this series.
* From v1 [1]:
- Move the "broken SEIS" detection to an erratum, something it should always have been, so that vgic_broken_seis() doesn't have to use non-noinstr function calls (sashiko)
- Drop WARN_ON() in the patch function (sashiko)
- Switch from this_cpu_has_cap() to cpus_have_cap() (shashiko)
- Reorganise the patches so that some of the code movements are less surprising to our favourite Artificial Idiot
- Simplify __vgic_v3_get_config() to only return the MMIO attribute as a boolean, dropping all form of GICv5 support which made little sense
[1] https://lore.kernel.org/r/[email protected]
Marc Zyngier (6):
KVM: arm64: vgic-v3: Make vtr_to_* helpers use architectural field
symbols
KVM: arm64: Move GICv3 broken SEIS implementation detection to a CPU
errrata
KVM: arm64: Add a helper providing an inlined literal value for
ICH_VTR_EL2
KVM: arm64: Convert most ICH_VTR_EL2 accesses to inlined literal value
KVM: arm64: vgic-v3: Simplify initial GICv3 configuration sampling
KVM: arm64: vgic-v3: Kill kvm_vgic_global_state.ich_vtr_el2
arch/arm64/include/asm/kvm_asm.h | 2 +- arch/arm64/kernel/cpu_errata.c | 17 +++++- arch/arm64/kernel/image-vars.h | 1 + arch/arm64/kvm/hyp/vgic-v3-sr.c | 43 +++++---------- arch/arm64/kvm/nested.c | 3 +- arch/arm64/kvm/vgic-sys-reg-v3.c | 8 +-- arch/arm64/kvm/vgic/vgic-v3.c | 93 +++++++++++++++++++++----------- arch/arm64/kvm/vgic/vgic-v5.c | 5 +- arch/arm64/kvm/vgic/vgic.h | 19 ++++++- arch/arm64/tools/cpucaps | 1 + include/kvm/arm_vgic.h | 2 - 11 files changed, 118 insertions(+), 76 deletions(-)
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v2 1/6 KVM: arm64: vgic-v3: Make vtr_to_* helpers use architectural field symbols
vgic-v3-sr.c still contains some hardcoded constants for some of the ICH)VTR_EL2 fields. Bring them into the modern world by using the named symbols from the sysreg file.
--- arch/arm64/kvm/hyp/vgic-v3-sr.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
arch/arm64/kvm/hyp/vgic-v3-sr.c +3 -3
@@ -16,9 +16,9 @@#include "../../vgic/vgic.h" -#define vtr_to_max_lr_idx (v) ((v) & 0xf) -#define vtr_to_nr_pre_bits (v) ((((u32)(v) >> 26) & 7) + 1) -#define vtr_to_nr_apr_regs (v) (1 << (vtr_to_nr_pre_bits(v) - 5)) +#define vtr_to_max_lr_idx (v) FIELD_GET(ICH_VTR_EL2_ListRegs, (v)) +#define vtr_to_nr_pre_bits (v) (FIELD_GET(ICH_VTR_EL2_PREbits, (v)) + 1) +#define vtr_to_nr_apr_regs (v) BIT(vtr_to_nr_pre_bits(v) - 5)u64 __gic_v3_get_lr (unsigned int lr) {
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v2 2/6 KVM: arm64: Move GICv3 broken SEIS implementation detection to a CPU errrata
In reply to Marc ZyngierUsing is_midr_in_range_list() in a noinstr function isn't a good idea. And yet kvm_compute_ich_hcr_trap_bits() is doing that by calling vgic_v3_broken_seis().
Move all the broken SEIS detection logic to the errata detection framework, and use a new ARM64_WORKAROUND_GICv3_BROKEN_SEIS cap to indicate that we're running on broken CPUs.
This reuses the MIDR list used for IMPDEF PMU detection, which has a 100% overlap with the SEIS stuff...
--- arch/arm64/kernel/cpu_errata.c | 17 +++++++++++++++-- arch/arm64/kvm/vgic/vgic-v3.c | 26 +++----------------------- arch/arm64/tools/cpucaps | 1 + 3 files changed, 19 insertions(+), 25 deletions(-)
arch/arm64/kernel/cpu_errata.c +15 -2
@@ -272,7 +272,7 @@ has_neoverse_n1_erratum_1542419(const struct arm64_cpu_capabilities *entry,return is_midr_in_range (&range) && has_dic; } -static const struct midr_range impdef_pmuv3_cpus[] = { +static const struct midr_range apple_cpus[] = {MIDR_ALL_VERSIONS (MIDR_APPLE_M1_ICESTORM ),MIDR_ALL_VERSIONS (MIDR_APPLE_M1_FIRESTORM ),MIDR_ALL_VERSIONS (MIDR_APPLE_M1_ICESTORM_PRO ),
@@ -301,7 +301,14 @@ static bool has_impdef_pmuv3(const struct arm64_cpu_capabilities *entry, int scoif (pmuver != ID_AA64DFR0_EL1_PMUVer_IMP_DEF)return false; -return is_midr_in_range_list (impdef_pmuv3_cpus); +return is_midr_in_range_list (apple_cpus); +} + +static bool has_broken_gic_v3_seis (const struct arm64_cpu_capabilities *entry,int scope) +{ +return (is_kernel_in_hyp_mode () && +is_midr_in_range_list (apple_cpus) && + (read_sysreg_s (SYS_ICH_VTR_EL2 ) &ICH_VTR_EL2_SEIS )); }static void cpu_enable_impdef_pmuv3_traps (const struct arm64_cpu_capabilities *__unused)
@@ -1009,6 +1016,12 @@ const struct arm64_cpu_capabilities arm64_errata[] = { .matches = has_impdef_pmuv3, .cpu_enable = cpu_enable_impdef_pmuv3_traps, }, + { + .desc = "Known broken GICv3 SEIS implementation", + .capability = ARM64_WORKAROUND_GICv3_BROKEN_SEIS, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_broken_gic_v3_seis, + }, { } };
arch/arm64/kvm/vgic/vgic-v3.c +3 -23
@@ -809,27 +809,9 @@ static int __init early_gicv4_enable(char *buf) }early_param ("kvm-arm.vgic_v4_enable" , early_gicv4_enable); -static const struct midr_range broken_seis[] = { -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_ICESTORM ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_FIRESTORM ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_ICESTORM_PRO ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_FIRESTORM_PRO ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_ICESTORM_MAX ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_FIRESTORM_MAX ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_BLIZZARD ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_AVALANCHE ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_BLIZZARD_PRO ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_AVALANCHE_PRO ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_BLIZZARD_MAX ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_AVALANCHE_MAX ), - {}, -}; - -static bool vgic_v3_broken_seis (void ) +static __always_inline bool vgic_v3_broken_seis(void) { -return (is_kernel_in_hyp_mode () && -is_midr_in_range_list (broken_seis) && - (read_sysreg_s (SYS_ICH_VTR_EL2 ) &ICH_VTR_EL2_SEIS )); +return cpus_have_cap (ARM64_WORKAROUND_GICv3_BROKEN_SEIS); } void noinstr kvm_compute_ich_hcr_trap_bits(struct alt_instr *alt,
@@ -959,10 +941,8 @@ int vgic_v3_probe(const struct gic_kvm_info *info)if (has_v2)static_branch_enable (&vgic_v3_has_v2_compat); -if (vgic_v3_broken_seis ()) { -kvm_info ("GICv3 with broken locally generated SEI\n" ); +if (vgic_v3_broken_seis ()) kvm_vgic_global_state.ich_vtr_el2 &= ~ICH_VTR_EL2_SEIS ; - }vgic_v3_enable_cpuif_traps ();
arch/arm64/tools/cpucaps +1 -0
@@ -121,6 +121,7 @@ WORKAROUND_CAVIUM_TX2_219_TVM WORKAROUND_CLEAN_CACHE WORKAROUND_DEVICE_LOAD_ACQUIRE WORKAROUND_DISABLE_CNP +WORKAROUND_GICv3_BROKEN_SEIS WORKAROUND_PMUV3_IMPDEF_TRAPS WORKAROUND_QCOM_FALKOR_E1003 WORKAROUND_QCOM_ORYON_CNTVOFF
Signature · 1 line
2.47.3
Oliver Upton
maintainerRe: [PATCH v2 2/6] KVM: arm64: Move GICv3 broken SEIS implementation detection to a CPU errrata
Hey,
Quoted 10 lines · Marc Zyngier
Using
is_midr_in_range_list()in a noinstr function isn't a good idea. And yetkvm_compute_ich_hcr_trap_bits()is doing that by callingvgic_v3_broken_seis().Move all the broken SEIS detection logic to the errata detection framework, and use a new
ARM64_WORKAROUND_GICv3_BROKEN_SEIScap to indicate that we're running on broken CPUs.This reuses the MIDR list used for IMPDEF PMU detection, which has a 100% overlap with the SEIS stuff...
Hopefully this remains the case! Oh well, that's the next person's problem to sort out anyway.
Thanks,
Oliver
Quoted 110 lines
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kernel/cpu_errata.c | 17 +++++++++++++++-- arch/arm64/kvm/vgic/vgic-v3.c | 26 +++----------------------- arch/arm64/tools/cpucaps | 1 + 3 files changed, 19 insertions(+), 25 deletions(-)arch/arm64/kernel/cpu_errata.c +15
-2@@ -272,7 +272,7 @@ has_neoverse_n1_erratum_1542419(const struct arm64_cpu_capabilities *entry,return is_midr_in_range (&range) && has_dic; } -static const struct midr_range impdef_pmuv3_cpus[] = { +static const struct midr_range apple_cpus[] = {MIDR_ALL_VERSIONS (MIDR_APPLE_M1_ICESTORM ),MIDR_ALL_VERSIONS (MIDR_APPLE_M1_FIRESTORM ),MIDR_ALL_VERSIONS (MIDR_APPLE_M1_ICESTORM_PRO ),@@ -301,7 +301,14 @@ static bool has_impdef_pmuv3(const struct arm64_cpu_capabilities *entry, int scoif (pmuver != ID_AA64DFR0_EL1_PMUVer_IMP_DEF)return false; -return is_midr_in_range_list (impdef_pmuv3_cpus); +return is_midr_in_range_list (apple_cpus); +} + +static bool has_broken_gic_v3_seis (const struct arm64_cpu_capabilities *entry,int scope) +{ +return (is_kernel_in_hyp_mode () && +is_midr_in_range_list (apple_cpus) && + (read_sysreg_s (SYS_ICH_VTR_EL2 ) &ICH_VTR_EL2_SEIS )); }static void cpu_enable_impdef_pmuv3_traps (const struct arm64_cpu_capabilities *__unused)@@ -1009,6 +1016,12 @@ const struct arm64_cpu_capabilities arm64_errata[] = { .matches = has_impdef_pmuv3, .cpu_enable = cpu_enable_impdef_pmuv3_traps, }, + { + .desc = "Known broken GICv3 SEIS implementation", + .capability = ARM64_WORKAROUND_GICv3_BROKEN_SEIS, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_broken_gic_v3_seis, + }, { } };arch/arm64/kvm/vgic/vgic-v3.c +3
-23@@ -809,27 +809,9 @@ static int __init early_gicv4_enable(char *buf) }early_param ("kvm-arm.vgic_v4_enable" , early_gicv4_enable); -static const struct midr_range broken_seis[] = { -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_ICESTORM ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_FIRESTORM ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_ICESTORM_PRO ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_FIRESTORM_PRO ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_ICESTORM_MAX ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M1_FIRESTORM_MAX ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_BLIZZARD ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_AVALANCHE ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_BLIZZARD_PRO ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_AVALANCHE_PRO ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_BLIZZARD_MAX ), -MIDR_ALL_VERSIONS (MIDR_APPLE_M2_AVALANCHE_MAX ), - {}, -}; - -static bool vgic_v3_broken_seis (void ) +static __always_inline bool vgic_v3_broken_seis(void) { -return (is_kernel_in_hyp_mode () && -is_midr_in_range_list (broken_seis) && - (read_sysreg_s (SYS_ICH_VTR_EL2 ) &ICH_VTR_EL2_SEIS )); +return cpus_have_cap (ARM64_WORKAROUND_GICv3_BROKEN_SEIS); } void noinstr kvm_compute_ich_hcr_trap_bits(struct alt_instr *alt,@@ -959,10 +941,8 @@ int vgic_v3_probe(const struct gic_kvm_info *info)if (has_v2)static_branch_enable (&vgic_v3_has_v2_compat); -if (vgic_v3_broken_seis ()) { -kvm_info ("GICv3 with broken locally generated SEI\n" ); +if (vgic_v3_broken_seis ()) kvm_vgic_global_state.ich_vtr_el2 &= ~ICH_VTR_EL2_SEIS ; - }vgic_v3_enable_cpuif_traps ();arch/arm64/tools/cpucaps +1
-0@@ -121,6 +121,7 @@ WORKAROUND_CAVIUM_TX2_219_TVM WORKAROUND_CLEAN_CACHE WORKAROUND_DEVICE_LOAD_ACQUIRE WORKAROUND_DISABLE_CNP +WORKAROUND_GICv3_BROKEN_SEIS WORKAROUND_PMUV3_IMPDEF_TRAPS WORKAROUND_QCOM_FALKOR_E1003 WORKAROUND_QCOM_ORYON_CNTVOFFSignature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v2 3/6 KVM: arm64: Add a helper providing an inlined literal value for ICH_VTR_EL2
In reply to Marc ZyngierWe already have two ways to deal with ICH_VTR_EL2:
- directly read the system register
- read a cached copy in the vgic global state
Add a third way, in the form of an inlined literal value that will eventually replace all of the above. This literal value is computed at boot time, and patched in the relevant code locations.
--- arch/arm64/kernel/image-vars.h | 1 + arch/arm64/kvm/vgic/vgic-v3.c | 55 ++++++++++++++++++++++++++++++++++ arch/arm64/kvm/vgic/vgic.h | 17 +++++++++++ 3 files changed, 73 insertions(+)
arch/arm64/kernel/image-vars.h +1 -0
@@ -92,6 +92,7 @@ KVM_NVHE_ALIAS(spectre_bhb_patch_wa3);KVM_NVHE_ALIAS (spectre_bhb_patch_clearbhb);KVM_NVHE_ALIAS (alt_cb_patch_nops);KVM_NVHE_ALIAS (kvm_compute_ich_hcr_trap_bits); +KVM_NVHE_ALIAS (kvm_patch_ich_vtr_el2);/* Global kernel state accessed by nVHE hyp code. */ KVM_NVHE_ALIAS (kvm_vgic_global_state);
arch/arm64/kvm/vgic/vgic-v3.c +55 -0
61 lines not on this page · raw message
arch/arm64/kvm/vgic/vgic.h +17 -0
@@ -71,6 +71,23 @@ ICH_VTR_EL2_IDbits)#define KVM_ICH_VTR_EL2_RES1 ICH_VTR_EL2_nV4 +void kvm_patch_ich_vtr_el2 (struct alt_instr *alt, +__le32 *origptr,__le32 *updptr,int nr_inst); + +static inline u64 vgic_ich_vtr (void ) +{ +u64 vtr; + +/* All non-RES0 bits are in the bottom 32bits */ + asm volatile(ALTERNATIVE_CB("movz %0, #0\n" + "movk %0, #0, lsl #16\n", + ARM64_ALWAYS_SYSTEM, + kvm_patch_ich_vtr_el2) + : "=r" (vtr)); + +return vtr; +} +static inline u64 kvm_get_guest_vtr_el2 (void ) {u64 vtr;
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v2 4/6 KVM: arm64: Convert most ICH_VTR_EL2 accesses to inlined literal value
In reply to Marc ZyngierNow that we have a new helper that inlines the value of ICH_VTR_EL2 into the code, use this to replace most of the sysreg accesses to that register.
This avoids a lot of traps when running KVM under NV.
--- arch/arm64/kvm/hyp/vgic-v3-sr.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
arch/arm64/kvm/hyp/vgic-v3-sr.c +9 -9
@@ -367,7 +367,7 @@ void __vgic_v3_save_aprs(struct vgic_v3_cpu_if *cpu_if)u64 val;u32 nr_pre_bits; - val =read_gicreg (ICH_VTR_EL2 ); + val =vgic_ich_vtr (); nr_pre_bits =vtr_to_nr_pre_bits (val);switch (nr_pre_bits) {
@@ -400,7 +400,7 @@ static void __vgic_v3_restore_aprs(struct vgic_v3_cpu_if *cpu_if)u64 val;u32 nr_pre_bits; - val =read_gicreg (ICH_VTR_EL2 ); + val =vgic_ich_vtr (); nr_pre_bits =vtr_to_nr_pre_bits (val);switch (nr_pre_bits) {
@@ -430,7 +430,7 @@ static void __vgic_v3_restore_aprs(struct vgic_v3_cpu_if *cpu_if)void __vgic_v3_init_lrs (void ) { -int max_lr_idx =vtr_to_max_lr_idx (read_gicreg (ICH_VTR_EL2 )); +int max_lr_idx =vtr_to_max_lr_idx (vgic_ich_vtr ());int i;for (i =0 ; i <= max_lr_idx; i++)
@@ -455,7 +455,7 @@ u64 __vgic_v3_get_gic_config(void)* system, so we first check if we have GICv5 support. */ if (cpus_have_final_cap (ARM64_HAS_GICV5_CPUIF )) -return read_gicreg (ICH_VTR_EL2 ); +return vgic_ich_vtr (); sre =read_gicreg (ICC_SRE_EL1 );/*
@@ -498,7 +498,7 @@ u64 __vgic_v3_get_gic_config(void) } val = (val &ICC_SRE_EL1_SRE ) ?0 : (1ULL <<63 ); - val |=read_gicreg (ICH_VTR_EL2 ); + val |=vgic_ich_vtr ();return val; }
@@ -540,7 +540,7 @@ void __vgic_v3_restore_vmcr_aprs(struct vgic_v3_cpu_if *cpu_if)static int __vgic_v3_bpr_min (void ) {/* See Pseudocode for VPriorityGroup */ -return 8 -vtr_to_nr_pre_bits (read_gicreg (ICH_VTR_EL2 )); +return 8 -vtr_to_nr_pre_bits (vgic_ich_vtr ()); }static int __vgic_v3_get_group (struct kvm_vcpu *vcpu)
@@ -614,7 +614,7 @@ static int __vgic_v3_find_active_lr(struct kvm_vcpu *vcpu, int intid,static int __vgic_v3_get_highest_active_priority (void ) { -u8 nr_apr_regs =vtr_to_nr_apr_regs (read_gicreg (ICH_VTR_EL2 )); +u8 nr_apr_regs =vtr_to_nr_apr_regs (vgic_ich_vtr ());u32 hap =0 ;int i;
@@ -707,7 +707,7 @@ static void __vgic_v3_set_active_priority(u8 pri, u32 vmcr, int grp)static int __vgic_v3_clear_highest_active_priority (void ) { -u8 nr_apr_regs =vtr_to_nr_apr_regs (read_gicreg (ICH_VTR_EL2 )); +u8 nr_apr_regs =vtr_to_nr_apr_regs (vgic_ich_vtr ());u32 hap =0 ;int i;
@@ -1039,7 +1039,7 @@ static void __vgic_v3_read_ctlr(struct kvm_vcpu *vcpu, u32 vmcr, int rt) {u32 vtr, val; - vtr =read_gicreg (ICH_VTR_EL2 ); + vtr =vgic_ich_vtr ();/* PRIbits */ val = ((vtr >>29 ) &7 ) <<ICC_CTLR_EL1_PRI_BITS_SHIFT ;/* IDbits */
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v2 5/6 KVM: arm64: vgic-v3: Simplify initial GICv3 configuration sampling
In reply to Marc ZyngierNow that we have our magic inline helper for ICH_VTR_EL2, we can get rid of the hack that was reporting a combination of that register and of the indication of the CPU interface supporting GICv2 compatibility. We now only report the latter.
As a small benefit, GICv5 is not involved in this stuff anymore, since it never has GICv2 compatibility..
--- arch/arm64/include/asm/kvm_asm.h | 2 +- arch/arm64/kvm/hyp/vgic-v3-sr.c | 23 +++-------------------- arch/arm64/kvm/vgic/vgic-v3.c | 9 +++------ arch/arm64/kvm/vgic/vgic-v5.c | 2 +- 4 files changed, 8 insertions(+), 28 deletions(-)
arch/arm64/include/asm/kvm_asm.h +1 -1
@@ -281,7 +281,7 @@ extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);extern void __kvm_adjust_pc (struct kvm_vcpu *vcpu); -extern u64 __vgic_v3_get_gic_config (void ); +extern bool __vgic_v3_get_gic_config (void );extern void __vgic_v3_init_lrs (void );#define __KVM_EXTABLE (from, to) \
arch/arm64/kvm/hyp/vgic-v3-sr.c +3 -20
@@ -437,26 +437,12 @@ void __vgic_v3_init_lrs(void)__gic_v3_set_lr (0 , i); } -/* -* Return the GIC CPU configuration: -* - [31:0] ICH_VTR_EL2 -* - [62:32] RES0 -* - [63] MMIO (GICv2) capable -*/ -u64 __vgic_v3_get_gic_config (void ) +/* Return true if GICv3 is MMIO (GICv2) capable, false otherwise */ +bool __vgic_v3_get_gic_config (void ) {u64 val, sre;unsigned long flags =0 ; -/* -* In compat mode, we cannot access ICC_SRE_EL1 at any EL -* other than EL1 itself; just return the -* ICH_VTR_EL2. ICC_IDR0_EL1 is only implemented on a GICv5 -* system, so we first check if we have GICv5 support. -*/ -if (cpus_have_final_cap (ARM64_HAS_GICV5_CPUIF )) -return vgic_ich_vtr (); - sre =read_gicreg (ICC_SRE_EL1 );/* * To check whether we have a MMIO-based (GICv2 compatible)
@@ -497,10 +483,7 @@ u64 __vgic_v3_get_gic_config(void)isb (); } - val = (val &ICC_SRE_EL1_SRE ) ?0 : (1ULL <<63 ); - val |=vgic_ich_vtr (); - -return val; +return !(val &ICC_SRE_EL1_SRE ); }static void __vgic_v3_compat_mode_enable (void )
arch/arm64/kvm/vgic/vgic-v3.c +3 -6
@@ -936,12 +936,12 @@ void vgic_v3_enable_cpuif_traps(void)*/ int vgic_v3_probe (const struct gic_kvm_info *info) { -u64 ich_vtr_el2 =kvm_call_hyp_ret (__vgic_v3_get_gic_config); +u64 ich_vtr_el2;bool has_v2;int ret; - has_v2 = ich_vtr_el2 >>63 ; - ich_vtr_el2 = (u32 )ich_vtr_el2; + has_v2 =kvm_call_hyp_ret (__vgic_v3_get_gic_config); + ich_vtr_el2 =vgic_ich_vtr ();/* * The ListRegs field is 5 bits, but there is an architectural
@@ -996,9 +996,6 @@ int vgic_v3_probe(const struct gic_kvm_info *info)if (has_v2)static_branch_enable (&vgic_v3_has_v2_compat); -if (vgic_v3_broken_seis ()) - kvm_vgic_global_state.ich_vtr_el2 &= ~ICH_VTR_EL2_SEIS ; -vgic_v3_enable_cpuif_traps (); kvm_vgic_global_state.vctrl_base =NULL ;
arch/arm64/kvm/vgic/vgic-v5.c +1 -1
@@ -83,7 +83,7 @@ int vgic_v5_probe(const struct gic_kvm_info *info) } kvm_vgic_global_state.has_gcie_v3_compat = true; - ich_vtr_el2 =kvm_call_hyp_ret (__vgic_v3_get_gic_config); + ich_vtr_el2 =vgic_ich_vtr (); kvm_vgic_global_state.ich_vtr_el2 = (u32 )ich_vtr_el2;/*
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v2 6/6 KVM: arm64: vgic-v3: Kill kvm_vgic_global_state.ich_vtr_el2
In reply to Marc Zyngierkvm_vgic_global_state.ich_vtr_el2 is the last bit of caching that we can get rid of. Not as bad as a sysreg access, but still worse than a constant.
Move over to the inlined stuff and remove the cached value.
--- arch/arm64/kvm/nested.c | 3 ++- arch/arm64/kvm/vgic-sys-reg-v3.c | 8 ++++---- arch/arm64/kvm/vgic/vgic-v3.c | 5 ++--- arch/arm64/kvm/vgic/vgic-v5.c | 5 +---- arch/arm64/kvm/vgic/vgic.h | 2 +- include/kvm/arm_vgic.h | 2 -- 6 files changed, 10 insertions(+), 15 deletions(-)
arch/arm64/kvm/nested.c +2 -1
@@ -16,6 +16,7 @@#include <asm/sysreg.h> #include "sys_regs.h" +#include "vgic/vgic.h" struct vncr_tlb {/* The guest's VNCR_EL2 */
@@ -1904,7 +1905,7 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu)/* ICH_HCR_EL2 */ resx.res0 =ICH_HCR_EL2_RES0 ; resx.res1 =ICH_HCR_EL2_RES1 ; -if (!(kvm_vgic_global_state.ich_vtr_el2 &ICH_VTR_EL2_TDS )) +if (!(vgic_ich_vtr () &ICH_VTR_EL2_TDS )) resx.res0 |=ICH_HCR_EL2_TDIR ;/* No GICv4 is presented to the guest */ resx.res0 |=ICH_HCR_EL2_DVIM | ICH_HCR_EL2_vSGIEOICount;
arch/arm64/kvm/vgic-sys-reg-v3.c +4 -4
@@ -35,12 +35,12 @@ static int set_gic_ctlr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r, vgic_v3_cpu->num_id_bits = host_id_bits; - host_seis =FIELD_GET (ICH_VTR_EL2_SEIS , kvm_vgic_global_state.ich_vtr_el2); + host_seis =FIELD_GET (ICH_VTR_EL2_SEIS ,vgic_ich_vtr ()); seis =FIELD_GET (ICC_CTLR_EL1_SEIS_MASK , val);if (host_seis != seis)return -EINVAL ; - host_a3v =FIELD_GET (ICH_VTR_EL2_A3V , kvm_vgic_global_state.ich_vtr_el2); + host_a3v =FIELD_GET (ICH_VTR_EL2_A3V ,vgic_ich_vtr ()); a3v =FIELD_GET (ICC_CTLR_EL1_A3V_MASK , val);if (host_a3v != a3v)return -EINVAL ;
@@ -69,9 +69,9 @@ static int get_gic_ctlr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r, val |=FIELD_PREP (ICC_CTLR_EL1_ID_BITS_MASK , vgic_v3_cpu->num_id_bits); val |=FIELD_PREP (ICC_CTLR_EL1_SEIS_MASK ,FIELD_GET (ICH_VTR_EL2_SEIS , - kvm_vgic_global_state.ich_vtr_el2)); +vgic_ich_vtr ())); val |=FIELD_PREP (ICC_CTLR_EL1_A3V_MASK , -FIELD_GET (ICH_VTR_EL2_A3V , kvm_vgic_global_state.ich_vtr_el2)); +FIELD_GET (ICH_VTR_EL2_A3V ,vgic_ich_vtr ()));/* * The VMCR.CTLR value is in ICC_CTLR_EL1 layout. * Extract it directly using ICC_CTLR_EL1 reg definitions.
arch/arm64/kvm/vgic/vgic-v3.c +2 -3
@@ -490,9 +490,9 @@ void vgic_v3_reset(struct kvm_vcpu *vcpu) } vcpu->arch.vgic_cpu.num_id_bits =FIELD_GET (ICH_VTR_EL2_IDbits, - kvm_vgic_global_state.ich_vtr_el2); +vgic_ich_vtr ()); vcpu->arch.vgic_cpu.num_pri_bits =FIELD_GET (ICH_VTR_EL2_PRIbits, - kvm_vgic_global_state.ich_vtr_el2) +1 ; +vgic_ich_vtr ()) +1 ; }void vcpu_set_ich_hcr (struct kvm_vcpu *vcpu)
@@ -949,7 +949,6 @@ int vgic_v3_probe(const struct gic_kvm_info *info)*/ kvm_vgic_global_state.nr_lr = (ich_vtr_el2 &0xf ) +1 ; kvm_vgic_global_state.can_emulate_gicv2 = false; - kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;/* GICv4 support? */ if (info->has_v4) {
arch/arm64/kvm/vgic/vgic-v5.c +1 -4
@@ -40,7 +40,6 @@ static void vgic_v5_get_implemented_ppis(void)int vgic_v5_probe (const struct gic_kvm_info *info) {bool v5_registered = false; -u64 ich_vtr_el2;int ret; kvm_vgic_global_state.type =VGIC_V5 ;
@@ -83,14 +82,12 @@ int vgic_v5_probe(const struct gic_kvm_info *info) } kvm_vgic_global_state.has_gcie_v3_compat = true; - ich_vtr_el2 =vgic_ich_vtr (); - kvm_vgic_global_state.ich_vtr_el2 = (u32 )ich_vtr_el2;/* * The ListRegs field is 5 bits, but there is an architectural * maximum of 16 list registers. Just ignore bit 4... */ - kvm_vgic_global_state.nr_lr = (ich_vtr_el2 &0xf ) +1 ; + kvm_vgic_global_state.nr_lr = (vgic_ich_vtr () &0xf ) +1 ; ret =kvm_register_vgic_device (KVM_DEV_TYPE_ARM_VGIC_V3 );if (ret) {
arch/arm64/kvm/vgic/vgic.h +1 -1
@@ -92,7 +92,7 @@ static inline u64 kvm_get_guest_vtr_el2(void) {u64 vtr; - vtr = kvm_vgic_global_state.ich_vtr_el2; + vtr =vgic_ich_vtr (); vtr &= ~KVM_ICH_VTR_EL2_RES0 ; vtr |=KVM_ICH_VTR_EL2_RES1 ;
include/kvm/arm_vgic.h +0 -2
@@ -176,8 +176,6 @@ struct vgic_global {/* GICv3 compat mode on a GICv5 host */ bool has_gcie_v3_compat; -u32 ich_vtr_el2; -/* GICv5 PPI capabilities */ struct { DECLARE_BITMAP(impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);
Signature · 1 line
2.47.3
Quoted 8 lines · Marc Zyngier
I'm reposting this early as there were so many issues to the original posting that I decided to deviate from my own rules... Bite me.
This is the second version of this series I initially sent as [1], which aims at removing the large number of traps that
ICH_VTR_EL2generates in NV environments, leading to pointless overheads.[...]
Sashiko seems to have a done a good job reviewing v1. This looks to be in decent shape now.
Applied to next, thanks!
[1/6] KVM: arm64: vgic-v3: Make vtr_to_* helpers use architectural field symbols
https://git.kernel.org/kvmarm/kvmarm/c/719e67103c5e [2/6] KVM: arm64: Move GICv3 broken SEIS implementation detection to a CPU errrata https://git.kernel.org/kvmarm/kvmarm/c/27a263e1a0fa [3/6] KVM: arm64: Add a helper providing an inlined literal value for ICH_VTR_EL2 https://git.kernel.org/kvmarm/kvmarm/c/4edfdc8ddebd [4/6] KVM: arm64: Convert most ICH_VTR_EL2 accesses to inlined literal value https://git.kernel.org/kvmarm/kvmarm/c/2bba3d2c6092 [5/6] KVM: arm64: vgic-v3: Simplify initial GICv3 configuration sampling https://git.kernel.org/kvmarm/kvmarm/c/bea608299d28 [6/6] KVM: arm64: vgic-v3: Kill kvm_vgic_global_state.ich_vtr_el2 https://git.kernel.org/kvmarm/kvmarm/c/36d32222fcab
--
Best,
Oliver